Docs menu
Deterministic Replay & Crash Recovery
Because every model/tool step is written to the journal, an interrupted run continues deterministically with the same runId from where it left off, without resupplying the prompt.
What it's for / when to use it#
In the middle of a long-running agent run, the process crashes, a deploy restarts it, or a tool suspends while waiting for approval (see human-in-loop-approvals). A classic generateText knows nothing past that point — you'd have to regenerate the conversation and every tool result. runDurable writes every model step and every tool call's result to the journal; when called again with the same runId (or via resumeRun), it does NOT re-ask the model for steps that were already produced — it reads them from the journal and continues from where it left off. Result: a side-effecting tool (e.g. a payment charge) doesn't run a second time after a crash, and the model doesn't spend the same tokens twice.
Setup / import#
The package is exported from a single entry point — no subpath needed (only the persistent storage adapters @gnldev/durable/sqlite and @gnldev/durable/postgres are separate exports, see storage-adapters).
import { runDurable, resumeRun, InMemoryJournal } from '@gnldev/durable';Step-by-step usage#
The example below is taken from a real test scenario: a tool call requesting a charge of 5000 units is routed to approval (require-approval) by the guard — the run suspends. The prompt is never rewritten anywhere; resumeRun is called with only the runId and the approval information.
const journal = new InMemoryJournal();
const r1 = await runDurable({
runId: 'o1',
journal,
model: makeModel(),
tools: makeTools(counter),
guard,
prompt: 'charge 5000',
stopWhen: stepCountIs(6),
});
// nothing has been charged yet — the guard suspended it
console.log(counter.charges); // 0
console.log(r1.interrupts.length); // 1const r2 = await resumeRun('o1', {
journal,
model: makeModel(),
tools: makeTools(counter),
guard,
approvals: { 'call-c': true },
stopWhen: stepCountIs(6),
});
console.log(counter.charges); // 1 — exactly once
console.log(r2.text); // 'Done.'Internally, resumeRun reads the input (prompt/messages/system) that was written to the journal on the first call and calls runDurable again with the same runId — that's what makes resume self-contained. If there's no recorded input (an unknown runId), it throws a meaningful error.
API reference#
runDurableReplaces generateText: runs the model + tools with durable wrappers, writing every step to the journal. If called again with the same runId, it does not re-run the recorded steps.
resumeRunReads the input recorded for a runId (prompt/messages/system) from the journal and calls runDurable again — no need to resupply the prompt, just runId + agent config + approvals.
ResumeAgentConfigAgent configuration for resumeRun: model, tools?, guard?, stopWhen?, replay? — the common slice shared by Studio embed and resume calls.
reconstructStateA pure function: materializes the conversation state up to a given step (uptoStep) from journal entries (JournalEntry[]) — the foundation of time-travel and fork (see time-travel-fork).
DivergenceErrorIn replay: 'strict' mode, thrown when a tool's regenerated arguments during replay don't match the recorded argsHash (non-determinism detection). In the default 'lenient' mode, it only logs a warning.
RunBusyErrorThrown when the same runId is already being run by another concurrent process/call (opt-in run-level lock, or a tool execute in-flight).
loadReplayCacheInternal optimization: if the journal provides a readRun (JournalReader) — e.g. SQLite/Postgres — it preloads all model/tool records in a single query on resume. Used automatically by runDurable/resumeRun — you generally don't need to call it directly.
InMemoryJournalAn in-memory journal (Journal + JournalReader) for test/development. For persistence in production, the SQLite/Postgres adapters implement the same interface (see storage-adapters).
replay: 'strict' option passed to runDurable stops the run with a DivergenceError if a tool's input doesn't match its recorded hash during replay (e.g. the model prompt changed, or generation was non-deterministic). In the default 'lenient' mode, drift is only logged as a console warning, and the recorded output is still returned.failed record in the journal — against the risk of double-charging, the user must explicitly approve via approvals. The same behavior applies to every tool call within the same runId — see exactly-once-tools.