GNL 0.17: early and moving fast — APIs can change between releasesJoin us

GNL
Docs menu
Core · Free@gnldev/cli

Upgrading

Bring a store an earlier version wrote up to this one with gnl upgrade — in what order, what runs while the old processes still run, and what to change in code for 0.16 and 0.17.

What gnl upgrade does#

A store written by an earlier version can hold data this version does not read as it is. gnl upgrade (in code, storage.upgrade()) brings it up by running the store's steps: the format steps run-status and schema, then the steps packages register. Every step is idempotent — run again, it finds nothing pending.

Packages add their steps to the store with registerUpgradeSteps(storage, factories), once, where the store is built, so storage.upgrade(), gnl upgrade and gnl doctor all see one list. A package step is named package/step. gnl upgrade finds the installed @gnldev/queue and @gnldev/scheduler itself (they are optional peers) and registers the steps they export — no flag. gnl init --features job or schedule writes the registration into a new project.

register the steps beside the store
import { registerUpgradeSteps } from '@gnldev/durable';
import { PostgresStorage } from '@gnldev/durable/postgres';
import { queueUpgrade } from '@gnldev/queue';
import { schedulerUpgrade } from '@gnldev/scheduler';

// once, where the store is built: storage.upgrade(), gnl upgrade and gnl doctor see one list
export const storage = registerUpgradeSteps(
  new PostgresStorage({ connectionString: process.env.DATABASE_URL! }),
  [queueUpgrade, schedulerUpgrade],
);

gnl upgrade --list and --dry-run write nothing and show each step (--dry-run also what the data is at and what the package expects: "data vX → package vY"). With no flag it runs every pending step; --only step runs just those. It prints what ran, failed, was not attempted (after a failure), is pending, waits for older processes, is not registered and is done. The exit code is 1 for a failed step or an unregistered debt, and 0 when only steps that wait for older processes are left.

A step marked afterOlderGone cannot run while a process of the earlier version still runs. A plain gnl upgrade reports it as waiting and does not run it; --only is how you say the earlier processes are gone. The format steps wait for work in hand (gnl pending; --wait waits for it as gnl pending --wait does); a package step that is not afterOlderGone does not. gnl doctor lists the registered package steps, the waiting ones and any unregistered debt.

From 0.16 to 0.17, in this order#

The order is the point: the queue's data changes shape, and the step that moves it is safe to run while the old processes still run, but the step that deletes the old shape is not.

  1. Update the packages. The floors are ai 7.0.149, @ai-sdk/openai 4.0.100, @ai-sdk/anthropic 4.0.86 and @ai-sdk/provider 4.0.32 — the first versions that report a cut stream as interrupted. @gnldev/cli now requires @gnldev/durable as a peer: a project without it gets it installed (npm) or a peer warning (pnpm, yarn).
  2. Run gnl upgrade before this version's queue workers start. The queue keeps jobs in per-organization lanes now, not in 0.16's one qjob log. The step is queue/lanes: it moves the old log's jobs into their lanes (a finished job stays finished, a pending one runs once) and leaves the old log as it is, so it is safe while 0.16 workers and producers still run. A worker of this version on a store whose jobs are still in that log does not start, and says what to run.
  3. Deploy this version, as a rolling deploy. While a 0.16 process still enqueues, this version's workers move what it adds; a job that a 0.16 worker and a new one both find runs once, because there is one lease.
  4. Once every 0.16 process is gone, run gnl upgrade --only queue/lanes-cleanup. It deletes the old log and its cursors. It refuses while a 0.16 worker is still seen polling the store: the error, and --dry-run, say how many, when each was last seen and when its record expires. A worker that was killed rather than stopped holds the cleanup back until its ttlMs passes.
  5. The scheduler's step, scheduler/misfire-default, the same way: once every 0.16 poller is gone it rewrites the triggers 0.16 stored with misfire: 'skip' to "the default". Until then this version reads those records as the default itself. In code it is upgradeTriggers(journal, opts) with dryRun to look first.
gnl upgrade --dry-run                      # what would run: data vX → package vY; writes nothing
gnl upgrade                                # 1. runs queue/lanes (safe beside 0.16 workers and producers)
# ...deploy 0.17 as a rolling deploy; once every 0.16 process is gone:
gnl upgrade --only queue/lanes-cleanup     # 2. deletes the old qjob log (refuses while a 0.16 worker is seen)
gnl upgrade --only scheduler/misfire-default   # 3. once every 0.16 poller is gone
The cleanup cannot see a producer with no worker
Every polling worker records itself in the store, and the cleanup refuses while a 0.16 one is live. A 0.16 process that only enqueues — a web server that calls enqueue and runs no worker — writes no record, so the cleanup does not see it. Make sure every 0.16 producer is gone too before you run it: run earlier, a 0.16 worker would read the old log from its head again, and a 0.16 producer would write jobs no worker of this version looks for any more.

A store you wrote that has no upgrade() has nowhere to register the steps; run them yourself once every 0.16 process is gone, with queueUpgrade(storage)'s steps and their run().

What to change in code#

Each of these is in the changelog's Breaking section for 0.17; the page it belongs to has the detail.

  • Liveness goes to the storage. Passing liveness to createGnl or to createWorker throws LivenessConflictError on a built-in store. Give it to the store instead: PostgresStorage takes a liveness option, and so do the other four (SqliteStorage as its second argument). See Storage adapters.
  • listJobs returns a page, and without orgId lists only the jobs that have no organization. Read .items; listAllJobs is the operator's view of every organization's jobs. See Background work.
  • misfire: 'skip' is gone, with no alias, and the default is 'latest'. After an outage a trigger now fires the last missed occurrence, not the oldest; pass misfire: 'earliest' for the old behaviour. Due triggers no longer wait for each other, and concurrency has no default limit. See Background work.
  • A custom WorkStore needs read, appendMany and the ready set (appendReady, takeReady, heldReady, deferReady, dropReady); listNamespaces and getMany stay optional. Every bundled store implements them, and the conformance kit checks them.
  • A handler's right to run is a signal. A queue job's ctx.signal now also aborts on lease-lost and timeout; an event handler gets meta.signal, so a test comparing meta with toEqual needs toMatchObject. See Background work.
  • A cut model stream is run_unfinished, 502 — where an OpenAI-compatible endpoint that ends every stream with no finish_reason used to answer 200. A run an executor stops at close() ends released. See the REST API.
  • acquireRunLock takes no now: a lock is dated on the store's clock, so the fifth argument is gone. storage.upgrade() throws UpgradeFailedError when a step throws, instead of the step's own error: read .report and .cause, fix the cause, and run it again — the steps that ran are idempotent. UpgradeOptions.only is a readonly string[].
import { UpgradeFailedError } from '@gnldev/durable';

try {
  await storage.upgrade();
} catch (e) {
  if (e instanceof UpgradeFailedError) {
    console.error(e.step, e.report, e.cause); // the steps that ran are idempotent: fix the cause, run it again
  }
  throw e;
}

From 0.15 to 0.16#

No store step. What to check in code and in Studio:

  • replayCounterfactual, ReplaySourceError, DEFAULT_SUITE, runSuite, checkScenario and normalizeAnswer moved from @gnldev/optimizer to @gnldev/experiment, with no alias; it is installed with the optimizer. compareOnSuite stays. See the optimizer.
  • Studio's Regression tab, POST /runs/:id/regression and the regressionModel option are removed; the Lab's capsule (POST /lab/capsule, the Inspector's Try in a capsule) replaces them. GET /runs/:id/regression/:otherId, the read-only diff of two runs, stays.
  • The Lab has three tabs — Improve, New agent, Scenarios — and the four modes and the mode=capsule and mode=thread URLs are gone. A capsule play with something changed now plays the run trials times changed and trials times unchanged, so it spends about twice what it did.
  • BestFormResult.best is absent when no form passed a single scenario run, and a thread diagnosis that tested nothing says untested, not not-found.
  • :memctx counts describe what the model was sent, and a run id containing the segment :derived: is refused. See Memory.

@gnldev/lab is experimental; its API will move.

API#

fnregisterUpgradeSteps

Adds packages' upgrade steps to a store and returns the store; called once, where the store is built. Refuses a bad registration before anything can fail mid-upgrade.

classUpgradeFailedError

What storage.upgrade() throws when a step throws: report (what ran) and step, with the step's own error as cause.

constqueueUpgrade

The queue's steps for registerUpgradeSteps: queue/lanes, then queue/lanes-cleanup (afterOlderGone).

constschedulerUpgrade

The scheduler's step: scheduler/misfire-default (afterOlderGone), the same migration as upgradeTriggers.

fnupgradeTriggers

Rewrites stored misfire 'skip' (0.16) to the default; dryRun lists what it would. Idempotent; run it once every 0.16 poller is gone.

fnmarkUpgradeDue

A package's note that the store holds data its step has not moved; upgrade() with no only reports a note no registered step answers.

Read the changelog line for your version
This page covers the two releases after 0.15. Every release's summary, with its upgrade notes, is on the changelog; the full entry for each is in CHANGELOG.md.