Docs menu
Upgrading
Bring a store an earlier version wrote up to this one with gnl upgrade — in what order, what runs while the old processes still run, and what to change in code for 0.16 and 0.17.
What gnl upgrade does#
A store written by an earlier version can hold data this version does not read as it is. gnl upgrade (in code, storage.upgrade()) brings it up by running the store's steps: the format steps run-status and schema, then the steps packages register. Every step is idempotent — run again, it finds nothing pending.
Packages add their steps to the store with registerUpgradeSteps(storage, factories), once, where the store is built, so storage.upgrade(), gnl upgrade and gnl doctor all see one list. A package step is named package/step. gnl upgrade finds the installed @gnldev/queue and @gnldev/scheduler itself (they are optional peers) and registers the steps they export — no flag. gnl init --features job or schedule writes the registration into a new project.
import { registerUpgradeSteps } from '@gnldev/durable';
import { PostgresStorage } from '@gnldev/durable/postgres';
import { queueUpgrade } from '@gnldev/queue';
import { schedulerUpgrade } from '@gnldev/scheduler';
// once, where the store is built: storage.upgrade(), gnl upgrade and gnl doctor see one list
export const storage = registerUpgradeSteps(
new PostgresStorage({ connectionString: process.env.DATABASE_URL! }),
[queueUpgrade, schedulerUpgrade],
);gnl upgrade --list and --dry-run write nothing and show each step (--dry-run also what the data is at and what the package expects: "data vX → package vY"). With no flag it runs every pending step; --only step runs just those. It prints what ran, failed, was not attempted (after a failure), is pending, waits for older processes, is not registered and is done. The exit code is 1 for a failed step or an unregistered debt, and 0 when only steps that wait for older processes are left.
A step marked afterOlderGone cannot run while a process of the earlier version still runs. A plain gnl upgrade reports it as waiting and does not run it; --only is how you say the earlier processes are gone. The format steps wait for work in hand (gnl pending; --wait waits for it as gnl pending --wait does); a package step that is not afterOlderGone does not. gnl doctor lists the registered package steps, the waiting ones and any unregistered debt.
From 0.16 to 0.17, in this order#
The order is the point: the queue's data changes shape, and the step that moves it is safe to run while the old processes still run, but the step that deletes the old shape is not.
- Update the packages. The floors are
ai7.0.149,@ai-sdk/openai4.0.100,@ai-sdk/anthropic4.0.86 and@ai-sdk/provider4.0.32 — the first versions that report a cut stream asinterrupted.@gnldev/clinow requires@gnldev/durableas a peer: a project without it gets it installed (npm) or a peer warning (pnpm, yarn). - Run
gnl upgradebefore this version's queue workers start. The queue keeps jobs in per-organization lanes now, not in 0.16's oneqjoblog. The step isqueue/lanes: it moves the old log's jobs into their lanes (a finished job stays finished, a pending one runs once) and leaves the old log as it is, so it is safe while 0.16 workers and producers still run. A worker of this version on a store whose jobs are still in that log does not start, and says what to run. - Deploy this version, as a rolling deploy. While a 0.16 process still enqueues, this version's workers move what it adds; a job that a 0.16 worker and a new one both find runs once, because there is one lease.
- Once every 0.16 process is gone, run
gnl upgrade --only queue/lanes-cleanup. It deletes the old log and its cursors. It refuses while a 0.16 worker is still seen polling the store: the error, and--dry-run, say how many, when each was last seen and when its record expires. A worker that was killed rather than stopped holds the cleanup back until itsttlMspasses. - The scheduler's step,
scheduler/misfire-default, the same way: once every 0.16 poller is gone it rewrites the triggers 0.16 stored withmisfire: 'skip'to "the default". Until then this version reads those records as the default itself. In code it isupgradeTriggers(journal, opts)withdryRunto look first.
gnl upgrade --dry-run # what would run: data vX → package vY; writes nothing
gnl upgrade # 1. runs queue/lanes (safe beside 0.16 workers and producers)
# ...deploy 0.17 as a rolling deploy; once every 0.16 process is gone:
gnl upgrade --only queue/lanes-cleanup # 2. deletes the old qjob log (refuses while a 0.16 worker is seen)
gnl upgrade --only scheduler/misfire-default # 3. once every 0.16 poller is goneenqueue and runs no worker — writes no record, so the cleanup does not see it. Make sure every 0.16 producer is gone too before you run it: run earlier, a 0.16 worker would read the old log from its head again, and a 0.16 producer would write jobs no worker of this version looks for any more.A store you wrote that has no upgrade() has nowhere to register the steps; run them yourself once every 0.16 process is gone, with queueUpgrade(storage)'s steps and their run().
What to change in code#
Each of these is in the changelog's Breaking section for 0.17; the page it belongs to has the detail.
- Liveness goes to the storage. Passing
livenesstocreateGnlor tocreateWorkerthrowsLivenessConflictErroron a built-in store. Give it to the store instead:PostgresStoragetakes alivenessoption, and so do the other four (SqliteStorageas its second argument). See Storage adapters. listJobsreturns a page, and withoutorgIdlists only the jobs that have no organization. Read.items;listAllJobsis the operator's view of every organization's jobs. See Background work.misfire: 'skip'is gone, with no alias, and the default is'latest'. After an outage a trigger now fires the last missed occurrence, not the oldest; passmisfire: 'earliest'for the old behaviour. Due triggers no longer wait for each other, andconcurrencyhas no default limit. See Background work.- A custom
WorkStoreneedsread,appendManyand the ready set (appendReady,takeReady,heldReady,deferReady,dropReady);listNamespacesandgetManystay optional. Every bundled store implements them, and the conformance kit checks them. - A handler's right to run is a signal. A queue job's
ctx.signalnow also aborts onlease-lostandtimeout; an event handler getsmeta.signal, so a test comparingmetawithtoEqualneedstoMatchObject. See Background work. - A cut model stream is
run_unfinished, 502 — where an OpenAI-compatible endpoint that ends every stream with nofinish_reasonused to answer 200. A run an executor stops atclose()endsreleased. See the REST API. acquireRunLocktakes nonow: a lock is dated on the store's clock, so the fifth argument is gone.storage.upgrade()throwsUpgradeFailedErrorwhen a step throws, instead of the step's own error: read.reportand.cause, fix the cause, and run it again — the steps that ran are idempotent.UpgradeOptions.onlyis areadonly string[].
import { UpgradeFailedError } from '@gnldev/durable';
try {
await storage.upgrade();
} catch (e) {
if (e instanceof UpgradeFailedError) {
console.error(e.step, e.report, e.cause); // the steps that ran are idempotent: fix the cause, run it again
}
throw e;
}From 0.15 to 0.16#
No store step. What to check in code and in Studio:
replayCounterfactual,ReplaySourceError,DEFAULT_SUITE,runSuite,checkScenarioandnormalizeAnswermoved from@gnldev/optimizerto@gnldev/experiment, with no alias; it is installed with the optimizer.compareOnSuitestays. See the optimizer.- Studio's Regression tab,
POST /runs/:id/regressionand theregressionModeloption are removed; the Lab's capsule (POST /lab/capsule, the Inspector's Try in a capsule) replaces them.GET /runs/:id/regression/:otherId, the read-only diff of two runs, stays. - The Lab has three tabs — Improve, New agent, Scenarios — and the four modes and the
mode=capsuleandmode=threadURLs are gone. A capsule play with something changed now plays the runtrialstimes changed andtrialstimes unchanged, so it spends about twice what it did. BestFormResult.bestis absent when no form passed a single scenario run, and a thread diagnosis that tested nothing saysuntested, notnot-found.:memctxcounts describe what the model was sent, and a run id containing the segment:derived:is refused. See Memory.
@gnldev/lab is experimental; its API will move.
API#
registerUpgradeStepsAdds packages' upgrade steps to a store and returns the store; called once, where the store is built. Refuses a bad registration before anything can fail mid-upgrade.
UpgradeFailedErrorWhat storage.upgrade() throws when a step throws: report (what ran) and step, with the step's own error as cause.
queueUpgradeThe queue's steps for registerUpgradeSteps: queue/lanes, then queue/lanes-cleanup (afterOlderGone).
schedulerUpgradeThe scheduler's step: scheduler/misfire-default (afterOlderGone), the same migration as upgradeTriggers.
upgradeTriggersRewrites stored misfire 'skip' (0.16) to the default; dryRun lists what it would. Idempotent; run it once every 0.16 poller is gone.
markUpgradeDueA package's note that the store holds data its step has not moved; upgrade() with no only reports a note no registered step answers.