Docs menu
Per-run limits & cost cap
Stop a runaway run on a per-run token/cost/tool-call ceiling with RunLimits.
What it's for / when to use it#
A run can spiral — an expensive model loop, a runaway tool-call chain, a cost you didn't budget for. RunLimits puts a per-run ceiling on cumulative tokens, cumulative USD cost, and the number of successful tool calls, and can also detect a tool stuck repeating the same call. All fields are optional: leave a field unset and that guard is OFF for the run.
RunLimits fields#
Every field is optional and independently off if unset:
RunLimits.maxTokensStops the run if the cumulative total tokens (input + output + cached) EXCEEDS this value.
RunLimits.maxCostUsdStops the run if the cumulative USD cost EXCEEDS this value.
RunLimits.maxToolCallsBlocks the NEXT tool call once the count of successful tool calls REACHES this value (>=).
RunLimits.loopDetectionOptional sub-config: { maxRepeats?, onRepeat? }. Detects the same tool called with the same argsHash repeated in a row.
loopDetection.maxRepeatsHow many consecutive identical (tool + argsHash) calls trigger the response.
loopDetection.onRepeat'block' throws ToolLoopDetectedError immediately, or 'reflect' delivers a nudge to the model first.
Setup / import#
RunLimits lives in @gnldev/durable and is set per run:
import { createGnl, RunLimitExceededError, type RunLimits } from '@gnldev/durable';Step-by-step usage#
Pass limits to gnl.run. Once a ceiling is breached, RunLimitExceededError is thrown and the run does not proceed to the next step:
// Checked AFTER each model step is journaled — a single step can OVERSHOOT the ceiling,
// but the run does not proceed to the NEXT step once it is breached.
const limits: RunLimits = { maxTokens: 50_000, maxCostUsd: 0.5, maxToolCalls: 20 };
try {
await gnl.run('asst', { runId: 'r5', prompt: 'do the task', limits });
} catch (e) {
if (e instanceof RunLimitExceededError) {
// e.detail: { kind, value, limit }
console.log('limit exceeded:', e.detail.kind, e.detail.value, e.detail.limit);
}
}Set loopDetection to catch a model stuck repeating the same tool call. reflect gives the model a chance to recover before anything is blocked:
// Same tool + identical argsHash repeated 3 times IN A ROW → 'reflect' delivers a nudge
// instead of an immediate block ('block' throws ToolLoopDetectedError right away).
const limits: RunLimits = { loopDetection: { maxRepeats: 3, onRepeat: 'reflect' } };The journal stays consistent when the error is thrown — raise the limit and resume the SAME runId to pick up where the run left off:
// The journal stays consistent when RunLimitExceededError is thrown.
// Raise the ceiling and resume the SAME runId to continue past the cut-off point:
await gnl.run('asst', { runId: 'r5', prompt: 'continue', limits: { maxTokens: 200_000 } });Streaming: catching blocked/limit breaches#
streamDurable / gnl.stream(name, opts) accepts an onBlocked callback, invoked when a loop, maxToolCalls, duplicate, or tainted BLOCK fires. It receives the RAW structured breach the guard produced — the same information a thrown error would carry. result.text now REJECTS on a block too, mirroring RunLimitExceededError being thrown from gnl.run, so a direct consumer of the stream cannot silently miss it.
// onBlocked fires for a loop / maxToolCalls / duplicate / tainted BLOCK — you get the
// RAW breach and decide what (if anything) the user sees. GNL does not invent a
// user-facing message.
const result = await gnl.stream('asst', {
runId: 'r6',
prompt: 'do the task',
limits,
onBlocked: (breach) => {
// breach.kind identifies which guard fired; breach.message is a raw diagnostic
// string; breach.detail carries that guard's own structured payload.
console.log('blocked:', breach.kind, breach.message);
},
});
try {
const text = await result.text;
} catch (e) {
// result.text REJECTS on a block — mirrors runDurable throwing, so a direct
// consumer of the stream cannot silently miss the breach.
}API reference#
RunLimitExceededErrorThrown by enforceStepLimits when a ceiling is breached. detail: { kind, value, limit } where kind is a RunLimitKind.
RunLimitKind'maxCostUsd' | 'maxTokens' | 'maxToolCalls' — identifies which ceiling in RunLimitExceededError.detail was breached.
enforceStepLimitsChecks maxTokens/maxCostUsd AFTER each model step is journaled; early-returns if both maxCostUsd and maxTokens are null.
RunOptions.limitsSet via RunOptions.limits, forwarded to runDurable/streamDurable, and inherited by sub-agents as-is.
onBlockedCallback option on streamDurable / gnl.stream(name, opts): (breach) => void. Invoked when a loop, maxToolCalls, duplicate, or tainted guard BLOCKs. breach: { kind, message, detail } — the raw structured payload, not a user-facing string. result.text also rejects when a block fires.
RunLimits / RunLimitExceededError is entirely per-run — it is DISTINCT from the org-level budget system in @gnldev/server (BudgetExceededError → HTTP 402, documented on the budget-quota page), which tracks spend per organization across runs. Do not conflate the two: a run can be well within its RunLimits and still be rejected by the org budget, or vice versa.enforceStepLimits checks maxTokens/maxCostUsd AFTER each step is journaled, so the cut-off lands at the FIRST step that breaches the ceiling — a single step CAN overshoot it. The guarantee is 'does not proceed to the next step', not '<= the exact limit'. It stops the runaway; it does not clip a step mid-flight.onBlocked hands you the framework's structured breach — kind, message, detail — not a sentence meant for an end user. Deciding what, if anything, the user sees is your app's job: GNL does not invent a user-facing message.budget-quota for the org-level counterpart this is often paired with, cost-observability for how the cost figure RunLimits checks against is computed, and duplicate-guard for the sibling per-run guard against a repeated side-effecting call.