Docs menu
Storage adapters
Pluggable storage backends for journal + memory + vector + work queue: SqliteStorage, PostgresStorage, RedisStorage, InMemoryStorage; a capability matrix and composite() for mixed backends.
What it's for / when to use it#
GNL's durability core (run journal, exactly-once, replay) always speaks the same Storage contract; which backend you plug in depends on the environment. Use SqliteStorage while developing or setting up a single-file demo, PostgresStorage in prod, and InMemoryStorage for tests — none of them change your agent/tool/workflow code, only the storage field you pass to createGnl changes.
A single storage may not serve every port (runs/memory/vectors/work/cache) at the same quality — so each adapter carries a capability matrix and can be routed per-port to a different storage (e.g. Redis for cache) with composite() when needed.
Setup / import#
Concrete implementations live in sub-exports so they don't bloat the core package; the main @gnldev/durable entry only carries the interfaces and pure helpers like composite/toJournal.
import { SqliteStorage } from '@gnldev/durable/sqlite';
import { PostgresStorage } from '@gnldev/durable/postgres';
import { RedisStorage } from '@gnldev/durable/redis';
import { InMemoryStorage, toJournal, composite } from '@gnldev/durable';Step-by-step usage#
1) Create a storage instance and pass it to the createGnl configuration (taken verbatim from the swapi-free example):
import { SqliteStorage } from '@gnldev/durable/sqlite';
import { createGnl, toJournal } from '@gnldev/durable';
import type { CreateGnlConfig } from '@gnldev/durable';
const storage = new SqliteStorage(process.env.DB_PATH ?? 'swapi-free.db');
const config: CreateGnlConfig = {
storage,
agents: {
starwars: { model: buildModel(), system: SYSTEM, tools: makeSwapiTools(), maxSteps: 4 },
},
};2) When Studio or another non-paged consumer expects Journal & JournalReader, bridge storage.runs with toJournal():
app.route('/studio', createStudioApp({
reader: toJournal(storage.runs),
apiBase: '/studio',
gnl: createStudioRunner(gnl, config, { toJsonSchema: aiToolSchema }),
auth,
}));3) When moving to prod, only one line changes — SqliteStorage becomes PostgresStorage:
import { PostgresStorage } from '@gnldev/durable/postgres';
const storage = new PostgresStorage({ connectionString: process.env.DATABASE_URL });
// or inject an existing pg.Pool / pg-mem:
const storage2 = new PostgresStorage({ pool: myPgPool });4) If you don't want persistence in tests, use InMemoryStorage — it needs no setup/file and every port runs at full capacity (full):
import { InMemoryStorage } from '@gnldev/durable';
const storage = new InMemoryStorage();5) When you need atomic CAS and native TTL under heavy concurrency, use RedisStorage — it provides the runs and work ports at 'full' (atomic putIfAbsent via SET NX / exactly-once ack, mget for bulk reads), and cache at native 'ttl' (Redis PX):
import { RedisStorage } from '@gnldev/durable/redis';
const storage = new RedisStorage({ connectionString: process.env.REDIS_URL });
// or inject your own ioredis client: new RedisStorage({ client: myIoredis })RedisStorage is honest in the capability matrix: it returns memory='none' and vectors='none' — Redis (without the RediSearch/RedisJSON modules) isn't suited for queryable memory/vector recall (no index, brute-force pulls the whole network into the app). That's why RedisStorage is typically used not on its own but inside composite() as the default for runs/work/cache, or as just a cache override.
6) If you only want to hand off a single port (e.g. cache) to a different storage, use composite() — runs and meta always come from default (replay must stay on a single journal):
import { composite } from '@gnldev/durable';
import { SqliteStorage } from '@gnldev/durable/sqlite';
import { RedisStorage } from '@gnldev/durable/redis';
const storage = composite({
default: new SqliteStorage('app.db'),
overrides: { cache: new RedisStorage({ connectionString: process.env.REDIS_URL }) },
});If you need a persistent vector store for RAG, PostgresVectorStore in the @gnldev/rag package (pgvector, HNSW/ivfflat index) provides the exact same VectorStore interface as InMemoryVectorStore — it plugs drop-in into composite()'s vectors port (it's not a separate @gnldev/durable adapter; because it runs durable-wrapped inside createRagTool, the query result is also journaled — resume doesn't re-issue the pg query):
import { PostgresVectorStore } from '@gnldev/rag';
const vectors = new PostgresVectorStore({ connectionString: process.env.DATABASE_URL, index: 'hnsw' });API reference#
SqliteStoragenode:sqlite-based implementation of every port — the dev/single-file default (@gnldev/durable/sqlite).
PostgresStoragepg-based prod storage; accepts a connectionString or an injected pool (including pg-mem) (@gnldev/durable/postgres).
PostgresStorageOptions{ connectionString?, pool? } — PostgresStorage constructor options.
RedisStorageioredis-based storage (@gnldev/durable/redis): runs/work='full' (SET NX atomic CAS, mget), cache='ttl' (native PX); memory/vectors='none' (honest — override with composite).
RedisStorageOptions{ connectionString?, client?, keyPrefix?, replicationWarning?, waitReplicas? } — RedisStorage input. The last two are the durability knobs: 'replicationWarning' checks the server's replication setup once and warns (advisory, fail-open); 'waitReplicas: { replicas, timeoutMs, onTimeout }' makes every successful claim WAIT for that many replicas to acknowledge — a write no replica confirmed is the one a failover loses.
InMemoryStorageTest/example storage providing every port at 'full' capability without persistence.
StorageThe unified interface carrying every store port (runs/memory/vectors/work/cache/meta) plus capabilities.
toJournalWraps a RunJournal in the legacy Journal & JournalReader contract — a bridge for non-paged consumers (e.g. Studio).
compositeProduces a Storage from a default storage plus per-port overrides; recomputes the capability matrix.
CompositeConfig{ default, overrides? } — composite() input.
requireCapabilityRequires a port to be available at at least 'scan' level; throws CapabilityError otherwise.
CapabilityMatrix{ runs, memory, vectors, work, cache } — 'full' | 'scan' | 'ttl' | 'none' for each port.
RunJournalThe PROTECTED append-only journal port — get/put/putIfAbsent (CAS)/listKeys/readRun/listRuns (paged).
MemoryStoreThe thread/message/working-memory/observation port — conversation artifacts (not replay state).
VectorStoreThe RAG corpus port — upsert/query (embedding-based similarity).
WorkStoreThe queue/event/scheduler port — lives in its own namespace, doesn't pollute the RunJournal.
runs and meta ports always come from default in composite() and cannot be overridden — replay/exactly-once must stay tied to a single RunJournal. Vector search is still at 'scan' (brute-force cosine) level on both concrete @gnldev/durable adapters (Sqlite/Postgres); for persistent/scaled RAG, @gnldev/rag's PostgresVectorStore (pgvector) is used. RedisStorage returns 'none' for memory/vectors — it isn't used on its own, it's completed with composite.