Request limits & cost cap

A run can spiral — an expensive model loop, a runaway tool-call chain, a cost you didn't budget for. RunLimits puts a ceiling on cumulative tokens, cumulative USD cost and the number of tool calls, caps how many tool calls execute at the same time, and can detect a tool stuck repeating the same call. The ceilings belong to the request: they count every run it starts, the agent's own and every sub-agent's under it. All fields are optional: leave a field unset and that guard is OFF.

Every field is optional and independently off if unset:

RunLimits lives in @gnldev/durable. It is passed to a run and counted for the whole request:

import { createGnl, RunLimitExceededError, type RunLimits } from '@gnldev/durable';

Pass limits to gnl.run. Once a ceiling is reached, RunLimitExceededError is thrown and the run does not start the next step:

// A new model step starts only if budget is left. A step already running is not cut,
// so that one step can go past the ceiling; no step starts after it.
const limits: RunLimits = { maxTokens: 50_000, maxCostUsd: 0.5, maxToolCalls: 20 };

try {
  await gnl.run('asst', { runId: 'r5', prompt: 'do the task', limits });
} catch (e) {
  if (e instanceof RunLimitExceededError) {
    // e.detail: { kind, value, limit }
    console.log('limit exceeded:', e.detail.kind, e.detail.value, e.detail.limit);
  }
}

Set loopDetection to catch a model stuck repeating the same tool call. reflect gives the model a chance to recover before anything is blocked:

// Same tool + identical argsHash repeated 3 times IN A ROW → 'reflect' delivers a nudge
// instead of an immediate block ('block' throws ToolLoopDetectedError right away).
const limits: RunLimits = { loopDetection: { maxRepeats: 3, onRepeat: 'reflect' } };

The journal stays consistent when the error is thrown — raise the limit and resume the SAME runId to pick up where the run left off:

// The journal stays consistent when RunLimitExceededError is thrown.
// Raise the ceiling and resume the SAME runId to continue past the cut-off point:
await gnl.run('asst', { runId: 'r5', prompt: 'continue', limits: { maxTokens: 200_000 } });

A request is everything one call starts: the agent's run and every sub-agent run under it. maxToolCalls, maxTokens and maxCostUsd count all of them — the parent's calls, every sibling's and the child's. Handing work to a sub-agent is not itself a tool call. A new model step starts only if budget is left: on a 60-token budget a request now stops at 60, run directly or through sub-agents (before 0.10: 75 directly, 90 through sub-agents).

Calls issued in parallel each take a seat on the request's tool-call total before they execute, in one atomic step; a finished call gives its seat back in the step that counts it. Every call the limit admits runs, and no more: with a limit of 5, twelve parallel calls ran before 0.10 and five run now; 200 parallel calls with a limit of 50 run exactly 50 on memory, Sqlite and Postgres. A custom journal whose incrBy returns no values keeps a compare-and-set seat: never past the limit, but a call at the edge of the limit may be refused while another finishes.

// The limits are the request's: the agent's calls and every sub-agent's count
// toward one total. Handing work to a sub-agent is not itself a tool call.
const limits: RunLimits = {
  maxToolCalls: 50,  // 200 parallel calls → exactly 50 run
  maxConcurrency: 4, // at most 4 tool calls of the request execute at the same time
};

await gnl.run('coordinator', { runId: 'r7', prompt: 'process the batch', limits });

A request sent to a GNL server may carry its own limits, but it may only tighten maxCostUsd, maxTokens, maxToolCalls, maxConcurrency and loopDetection.maxRepeats — the smaller of the server's value and the request's applies. Any other name in a request's limits is refused with 400; the server's own limits all reach the run.

In a project the CLI wrote, the deployment's own bound is the limits field of gnl.config.ts (maxCostUsd, maxTokens, maxToolCalls and loop detection). gnl dev and the src/app.ts that gnl add host writes pass it to createRestApi as the server-side bound: it applies as it is when a request names none, and a request may only tighten it. Studio Playground runs are not REST requests and are not capped by it.

export default {
  // the server-side bound: `gnl dev` and the src/app.ts that `gnl add host` writes hand it to createRestApi
  limits: { maxCostUsd: 0.5, maxToolCalls: 20 },
};

A maxCostUsd ceiling also fires on a model whose response does not name itself. Pricing used to read the model id from the response only, so a provider, proxy or middleware that left it out priced every step as unknown, $0, and the ceiling never fired. The model's configured id (what wrapLanguageModel keeps) is now the fallback, written beside each step as configuredModelId; the response's own id still wins, and the warning about an unpriced model names the real id. Records written before this lack the field and price as before.

streamDurable / gnl.stream(name, opts) accepts an onBlocked callback, invoked when a loop, maxToolCalls, duplicate, or tainted BLOCK fires. It receives the RAW structured breach the guard produced — the same information a thrown error would carry. result.text now REJECTS on a block too, mirroring RunLimitExceededError being thrown from gnl.run, so a direct consumer of the stream cannot silently miss it.

// onBlocked fires for a loop / maxToolCalls / duplicate / tainted BLOCK — you get the
// RAW breach and decide what (if anything) the user sees. GNL does not invent a
// user-facing message.
const result = await gnl.stream('asst', {
  runId: 'r6',
  prompt: 'do the task',
  limits,
  onBlocked: (breach) => {
    // breach.kind identifies which guard fired; breach.message is a raw diagnostic
    // string; breach.detail carries that guard's own structured payload.
    console.log('blocked:', breach.kind, breach.message);
  },
});

try {
  const text = await result.text;
} catch (e) {
  // result.text REJECTS on a block — mirrors runDurable throwing, so a direct
  // consumer of the stream cannot silently miss the breach.
}