Lab

An agent did something you did not expect, and a prompt is a pile of sentences: which one brought the call about? Lab answers by running the agent again with parts left out and counting what changed; a part's share is its Shapley value. Nothing here changes your agent. The result is something you read, and a change you then make in your own code.

Lab sits on @gnldev/experiment (the replay, the masking, the trials and the arithmetic), which @gnldev/optimizer uses too; neither needs the other.

Lab is open source under Apache-2.0, in the public repository and on npm like the other packages — it is not part of the paid layer. It needs @gnldev/durable and ai 7 as peers.

npm i @gnldev/lab @gnldev/durable ai

Each function plays something different, and what it plays decides whether a tool can run:

• What brought this call about? explainRun replays a recorded run with each of its parts left out — prompt sections, earlier messages, tool results. No tool runs. Before any share it says whether the recording is what the model usually does (representative) and whether the outcome depends on the parts at all (dependsOnParts), and every share carries its noise. It takes 2^parts × trials model calls.
• Which words decided it? wordShares with toolChoiceProbability reads, from a provider's logprobs, how much each group of words moved the chance that the next call is a given tool. OpenAI's API gives no probability on a tool call (measured), so sampledToolChoice asks each subset several times instead, and each share then says how far chance can move it. It explains one decision, the next one, not a whole run.
• Can this prompt section go? checkPrune plays new prompts for real, with the section and without it, beside the noise floor and a control section that should matter. The tools run — there is nothing recorded to replay — so give it fakes.
• Which parts does a new agent need? bestForm tries every subset of an agent's components on a scenario set with fake tools, checked by a program and not a judge model, and names the cheapest form that does as well as the best. It picks nothing for you: you write the chosen components into the agent's code.
• What would this run have done, changed? runCapsule copies a recorded run and plays it again with a different system prompt, other tools, or without its history or memory. No tool runs and nothing is written. A call the recording never made is answered from answers you write, or listed as pending. The change is read against the run played as it was, as many times, so a difference is not mistaken for chance.
• Why did this turn of a thread go wrong? threadPlan reads a thread's turns and calls no model; diagnoseTurn needs expect — what a right answer does — and refuses to start without it. With guess, a model first names the one or two parts most likely to blame, and only those are played beside the turn unchanged; the cause is named only by plays, never by the guess.

Give Studio the lab and it shows a Lab tab. The tab lists no runs of its own: you open a completed run in the Inspector and send it with Open in Lab (for a thread, from the thread ledger). It has three tabs — Improve, New agent and Scenarios. Improve puts a run or a thread on one screen against one written expectation: the turns on a strip, the turn's explanation, its diagnosis and its capsule. Without the option the routes answer 501 and the tab is hidden.

import { createStudioApp } from '@gnldev/studio';
import { studioLab } from '@gnldev/lab';
import { toJournal, type CreateGnlConfig } from '@gnldev/durable';
import { SqliteStorage } from '@gnldev/durable/sqlite';
import { openai } from '@ai-sdk/openai';

declare const config: CreateGnlConfig; // the config createGnl is given
const storage = new SqliteStorage('runs.db');

const studio = createStudioApp({
  reader: toJournal(storage),
  apiBase: '/studio',
  lab: studioLab({ config, models: { 'gpt-4.1-mini': openai('gpt-4.1-mini') } }),
});

Nothing is spent before you see the plan. Every Lab call shows the plays, the tokens, the price and the provider first, and says that no tool runs and no record changes; a call is audited by what was asked (counts and names, never the text you typed). A long call shows how far it got, with a Stop button; when the provider answers 429 it slows to one call at a time and says it is waiting. The Lab only offers a run of an agent you gave it, and the agent's own tools never run in it: only the fake tools of the scenarios do.

A thread diagnosis is measured on planted faults, with one model. Forty faults were planted in agents' threads (a stale working memory, a turn fallen out of the window, a wrong tool result, a weak rule) and diagnosed with gpt-6-luna through Studio's own routes: the right cause was named in 39, a wrong one in none, and the 40th stayed undecided. On 15 turns with nothing wrong in their input, each made to fail once, no cause was named in any. Elimination alone — the diagnosis before guess — named 15 of the same 40.

The faults were planted to be found: the misleading fact is in the text, where a reader sees it. A misleading recalled message could not be planted at all, because the model never followed one. It is one model. Zero wrong causes in 55 puts the rate at most about 6.5%. The label comes off with faults a reader does not see, and more than one model.

Which words decided it, on a provider that gives no logprobs, is sampled: with few tries most shares are within chance. And Lab is not a verdict machine — the result is something you read, with its noise beside it.