Model Routing & Fallback

Depending on a single model provider in production is risky: a rate limit, a transient outage, or a quota overrun can take down an entire run. withModelFallback lets you define a list of candidates (e.g. try gpt-4o first, fall back to gpt-4o-mini); whichever model succeeds on the first attempt is selected at that moment, and the choice is written to the journal.

The critical point: this choice is persistent. Subsequent steps of the same runId, or a resume after a crash — even if the transient failure has since cleared — always stick to the same winning model. This preserves replay determinism (it prevents the run from "branching" because a different model produces a different answer on a later step).

No separate sub-package needed; it's exported from @gnldev/durable's root export:

import { withModelFallback, resolveModel, runDurable } from '@gnldev/durable';
import type { FallbackCandidate } from '@gnldev/durable';

Provider packages (@ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google, @ai-sdk/mistral) are only loaded lazily inside resolveModel via import() — there's no provider lock-in, you just need to install whichever one you use.

1) Define your candidates — each candidate is a pair of spec (a stable label written to the journal) and model (an AI SDK LanguageModelV2). 2) Call withModelFallback with the journal and runId, and pass the returned wrapper to runDurable as the model field:

import { withModelFallback } from '@gnldev/durable';
const model = withModelFallback([ { model: 'openai:gpt-4o' }, { model: 'openai:gpt-4o-mini' } ], journal, runId);
await runDurable({ runId, journal, model, tools, prompt });

On the first call, GNL tries the candidates in order; whichever succeeds wins and is written with CAS (claim) to the <runId>:cfg:model key. Every subsequent step of the same run first reads the journal's record — if a record exists, only that winning candidate is tried, and the others are never touched.

If you keep your candidate model as a single 'provider/model' string, first convert it to a real model with resolveModel and assemble it as a FallbackCandidate:

import { resolveModel, withModelFallback } from '@gnldev/durable';

const candidates = await Promise.all(
  ['openai/gpt-4o', 'anthropic/claude-haiku-4-5'].map(async (spec) => ({
    spec,
    model: await resolveModel(spec),
  })),
);
const model = withModelFallback(candidates, journal, runId);

If you pass just a single candidate, withModelFallback short-circuits and returns that model directly — you can keep the same call shape when switching between production and development.

A separate but adjacent problem related to model selection: if two workers resume the same runId concurrently, both may see the same model step and make the real doGenerate call twice — double token cost. If the opt-in exclusiveModelStep is passed to runDurable/streamDurable, a concurrent worker that sees a FRESH (startedAt more recent than ttlMs ago; default 30 seconds) 'running' model claim gets a RunBusyError:

await runDurable({ runId, journal, model, tools, prompt, exclusiveModelStep: { ttlMs: 30_000 } });
// a second concurrent worker that sees a FRESH claim on the same step gets RunBusyError (a duplicate doGenerate is prevented)

A STALE claim (whose owner has crashed) continues with the existing behavior — the fast crash-resume window isn't broken by this opt-in; only a genuinely concurrent/live competing worker is blocked.