diff --git a/README.md b/README.md
index 204339b..d9578cb 100644
--- a/README.md
+++ b/README.md
@@ -68,7 +68,7 @@ flowchart TB
end
subgraph LIBRARY["LIBRARY — world model (owns data/graph ledgers)"]
- EXTRACT["extraction pipeline:
extractor → auditor → merger
(the SOLE graph writer)"]
+ EXTRACT["extraction pipeline:
extractor → spine → auditor → merger
(the SOLE graph writer)"]
DERIVE["derivation — graphSnapshot · birth-session clusters
(pure reads)"]
LEDG[("append-only ledgers
data/graph/*.jsonl")]
end
@@ -93,7 +93,7 @@ flowchart TB
HOST -.->|"hosts the server side"| TURN
```
-**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer — tree-blind and post-hoc, triggered by token coverage, not quiz counts. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append (ids, provenance, chain dedupe — the same argument re-asserted merges under one `arg_` id). Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in practice records — feedback-only, never written back. The teacher stays a teacher; everything below the ledgers is derived.
+**Core invariant:** the extraction pipeline (extractor → spine → auditor → merger) is the sole graph writer — tree-blind and post-hoc, triggered by token coverage, not quiz counts. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append (ids, provenance, chain dedupe — the same argument re-asserted merges under one `arg_` id). Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in practice records — feedback-only, never written back. The teacher stays a teacher; everything below the ledgers is derived.
## Layout
@@ -114,7 +114,7 @@ src/
tools/web.ts web_search (Google CSE) + web_fetch (researcher only)
subagents/ researcher · demo-builder · visual-maker
· pdf-reader,
- extraction pipeline (extractor/auditor + merger handoff),
+ extraction pipeline (extractor/spine/auditor + merger handoff),
runner (async protocol), headless Chrome helper,
abort registry
server/chat-handler.ts THE turn composition root (liveness · steers · persist · stream)
@@ -154,7 +154,7 @@ data/ chats/ · graph/ (nodes/chains/sessions/coverage
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the client polls `/api/activity` and `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. Delivery is exactly-once (deterministic steer ids, delivered-flag checks at every claim point); stalled jobs are recovered on read; aborted jobs deliver an abort steer. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (vision-capable model — the `vision` slot in settings).
-**The knowledge graph (two channels, one writer).** The write channel: the extraction pipeline — extractor (LLM: sees only the source — the session's teacher prose or a notes chunk — fully trusted, graph-blind; its deliverable is propositions + reasoning chains: premises that jointly entail a conclusion, submitted as chains and stored as chains — `chains.jsonl` is the truth, edges a derived view; owner redesign + chains-primary, 2026-09-30) → auditor (LLM, an agentic loop that never sees the source: explores the existing graph through `find` / `list_sessions` / `list_propositions` tools and stamps each proposition with an identity verdict — link to an existing node (equivalence bias: paraphrases merge) or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, resolves and dedupes chains — identical arguments merge under one `arg_` id — checkpoints coverage) → layout settle. No quiz outcomes or understanding assessments live in the graph (2026-09-27 cleanup); practice (`/practice`) consumes the graph as feedback-only problem sets and writes nothing back. Triggered when a chat's **uncovered tokens** (message-view fingerprints, `coverage.jsonl`) cross `graph.extractionChunkTokens` (a settings knob, default 4096, floor 500 — read fresh per turn); failure checkpoints nothing, the next trigger re-runs. The same pipeline serves the **notes front door** (`POST /api/graph/import`: any `*.md`/`*.txt` becomes a source; re-imports are idempotent by path hash). The read channel: `GET /api/graph` derives the snapshot from the ledgers — a pure read (`deriveView` rebuilds the lossy image: each chain an implication node `⟹` with premise→implication→conclusion arrows, so two arguments for one conclusion stay two diamonds); the home view is a hand-rolled Canvas 2D renderer (one rAF loop, dirty-flag idle, d3-zoom camera, zoom-gated labels, cluster hulls) over a d3-force layout, with server-persisted render settings and drag-committed positions. Clusters are **birth-session groupings**: a node joins the cluster of the session that birthed it (a chat, or an imported notes file as a file-kind session), derived at read — no embedding pass, no naming pass. The teacher reaches the graph on its own terms through the gated prober tools (`list_sessions` / `list_propositions`, the `agent.graphTools` setting) — there is no automatic context injection.
+**The knowledge graph (two channels, one writer).** The write channel: the extraction pipeline — extractor (LLM: sees only the source — the session's teacher prose or a notes chunk — fully trusted, graph-blind; its deliverable is propositions + reasoning chains: premises that jointly entail a conclusion, submitted as chains and stored as chains — `chains.jsonl` is the truth, edges a derived view; owner redesign + chains-primary, 2026-09-30) → spine (LLM, strong tier: sees the same-source digest + the fresh stretch, never the source; mints only what spans them — cross-stretch chains + connective propositions, span law enforced at the submit seam; owner stage, 2026-09-30) → auditor (LLM, an agentic loop that never sees the source: explores the existing graph through `find` / `list_sessions` / `list_propositions` tools and stamps each proposition with an identity verdict — link to an existing node (equivalence bias: paraphrases merge) or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, resolves and dedupes chains — identical arguments merge under one `arg_` id — checkpoints coverage) → layout settle. No quiz outcomes or understanding assessments live in the graph (2026-09-27 cleanup); practice (`/practice`) consumes the graph as feedback-only problem sets and writes nothing back. Triggered when a chat's **uncovered tokens** (message-view fingerprints, `coverage.jsonl`) cross `graph.extractionChunkTokens` (a settings knob, default 8192 since the spine stage, floor 500 — read fresh per turn); failure checkpoints nothing, the next trigger re-runs. The same pipeline serves the **notes front door** (`POST /api/graph/import`: any `*.md`/`*.txt` becomes a source; re-imports are idempotent by path hash). The read channel: `GET /api/graph` derives the snapshot from the ledgers — a pure read (`deriveView` rebuilds the lossy image: each chain an implication node `⟹` with premise→implication→conclusion arrows, so two arguments for one conclusion stay two diamonds); the home view is a hand-rolled Canvas 2D renderer (one rAF loop, dirty-flag idle, d3-zoom camera, zoom-gated labels, cluster hulls) over a d3-force layout, with server-persisted render settings and drag-committed positions. Clusters are **birth-session groupings**: a node joins the cluster of the session that birthed it (a chat, or an imported notes file as a file-kind session), derived at read — no embedding pass, no naming pass. The teacher reaches the graph on its own terms through the gated prober tools (`list_sessions` / `list_propositions`, the `agent.graphTools` setting) — there is no automatic context injection.
**Practice (problem sets).** A node selection resolves to the STORED reasoning chains it fully contains (a chain rides iff every member is selected; unchained nodes ride as singletons); the `/practice` creation dialog picks a birth-session cluster (the graph view's own groups) and freezes its propositions + chains into a self-contained `ProblemSet`. Three named-level axes steer generation (ladder rung 0–4, cue distance 0–2, Theory|Application — never a number to interpolate; defaults Direct exercise · Adjacent context · Theory), plus an optional free-form note the agents weigh but need not follow. Creation has two entries: the cluster picker on `/practice`, or the graph view's chain table — a full-bleed sortable table of every reasoning chain, grouped by birth cluster under the shared top bar, selecting chips, chains, or whole clusters, whose floating selection bar shows the exact `N nodes → M chains` the agents will receive — handing a free-form node selection to the same dialog — and carries the fenced Delete action, making it the graph's management view (whole-cluster cleanup through the same full-death confirm dialog). A partitioner agent covers every chain exactly once, one generator per partition writes problems against the owner's problem-engineering framework (commit-before-pose: statement + solution + rubric together), and grading is **strengthening, not assessment** — one attempt per problem with its own submit button, instant solution/rubric reveal, then a score 0–10 with brief reasoning landing live; nothing ever written back to the graph. Runs are registry-backed background jobs on the activity dock (no steer, never wakes the teacher); sets live in `data/practice`, crash-swept on boot.
diff --git a/components/graph-view.tsx b/components/graph-view.tsx
index f3a8303..2ef385c 100644
--- a/components/graph-view.tsx
+++ b/components/graph-view.tsx
@@ -309,9 +309,14 @@ export function displayCostTotals(costs: Pick): {
totalDollars: number;
unpricedRuns: number;
} {
+ // The three live stages (spine joined 2026-09-30); empty spines
+ // contribute zero — the fold leaves them at 0 when a run never
+ // called the spine (first-stretch skip / pre-spine records).
return {
- totalDollars: costs.stages.extraction.dollars + costs.stages.audit.dollars,
- unpricedRuns: costs.stages.extraction.unpricedRuns + costs.stages.audit.unpricedRuns,
+ totalDollars:
+ costs.stages.extraction.dollars + costs.stages.spine.dollars + costs.stages.audit.dollars,
+ unpricedRuns:
+ costs.stages.extraction.unpricedRuns + costs.stages.spine.unpricedRuns + costs.stages.audit.unpricedRuns,
};
}
@@ -368,6 +373,7 @@ function GraphCostControl({ costs }: { costs: GraphCostSummary }) {
function GraphCostPanel({ costs }: { costs: GraphCostSummary }) {
const stageRows: [string, GraphCostSummary["stages"]["extraction"]][] = [
["Extraction", costs.stages.extraction],
+ ["Assembly", costs.stages.spine],
["Audit", costs.stages.audit],
];
// The displayed figures come from displayCostTotals (naming excluded —
diff --git a/components/setup-dialog.tsx b/components/setup-dialog.tsx
index 2f7a734..1cad93d 100644
--- a/components/setup-dialog.tsx
+++ b/components/setup-dialog.tsx
@@ -49,7 +49,7 @@ import type { ThinkingLevel } from "@earendil-works/pi-agent-core";
/** The pickers' slot identity — same roster language as the Agent
* section's slot rows. */
const SLOT_META: Record = {
- strong: { label: "Strong", roster: "Prober + Teacher" },
+ strong: { label: "Strong", roster: "Prober, teacher, spine, practice trio" },
cheap: { label: "Cheap", roster: "Researcher, demo builder, extractor, auditor" },
vision: { label: "Vision", roster: "Visual maker + PDF reader" },
};
diff --git a/config.ts b/config.ts
index 12baa83..b99fb70 100644
--- a/config.ts
+++ b/config.ts
@@ -53,6 +53,7 @@ export type ModelRole =
| "visualMaker"
| "pdfReader"
| "extractor"
+ | "spine"
| "auditor"
| "partitioner"
| "problemGenerator"
diff --git a/docs/architecture.md b/docs/architecture.md
index bf54a68..a946cd2 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -239,7 +239,7 @@ flowchart TB
end
subgraph LIBRARY["LIBRARY — world model (owns data/graph ledgers)"]
- EXTRACT["extraction pipeline — extractor → auditor → merger (SOLE graph writer)"]
+ EXTRACT["extraction pipeline — extractor → spine → auditor → merger (SOLE graph writer)"]
DERIVE["derivation — graphSnapshot · birth-session clusters (reads, never write)"]
GRAPHDATA["append-only ledgers — data/graph/*.jsonl"]
end
@@ -339,7 +339,7 @@ Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its
```mermaid
flowchart TB
TRIG["trigger seam (turn-service): coverageOf(record, coverageLedger)
— uncovered tokens ≥ graph.extractionChunkTokens (settings)?"] -->|"yes, single-flight per source"| RUN["extraction.runExtraction(chatId)"]
- IMPORT["POST /api/graph/import (notes front door)
deterministic src id = sha256(path); content-fp guard skips unchanged re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
+ IMPORT["POST /api/graph/import (notes front door)
deterministic src id = sha256(path); content-fp guard skips unchanged re-imports"] -->|"importNotes: shared pipeline, per-door preambles"| RUN
RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"]
EXT["EXTRACTOR (LLM) — sees ONLY the teacher-prose view (tree-blind, graph-blind)
submits propositions + REASONING CHAINS (owner redesign, 2026-09-30)
stored AS-IS — chains.jsonl is the truth (chains-primary, 2026-09-30)"]
EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;
explores the graph via find / list_sessions / list_propositions tools,
judges identity ONLY (link-or-create, equivalence bias) — sees propositions, never mints chains"]
@@ -350,11 +350,13 @@ flowchart TB
**Raw reasoning chains only** (2026-09-27 cleanup; teacher-prose view 2026-09-28): the extractor reads ONLY the session — and since the 2026-09-28 ruling, ONLY the teacher agent's prose: plain paragraphs under the session title, no learner messages, no thinking, no tool calls or results, no quiz blocks. Coverage fingerprints, slot tokens, and batch budgets all derive from that same selection (`messageViewLines` in `core/coverage.ts`), so "covered" is exactly "what the extractor saw" — a quiz answer or learner edit can never open a run, and only a teacher-prose edit re-opens its message. The old call+result inseparability bond is deleted (nothing left to keep together); batching is pure greedy fill. The shared markdown engine (`core/session-markdown.ts`) keeps the full session format for the future user-facing export and is no longer used by extraction. The extractor's deliverable is propositions + REASONING CHAINS (owner redesign, 2026-09-30): a chain is premises that JOINTLY entail a conclusion — independent arguments for the same conclusion are separate chains, conclusions are reusable by id as premises, and a bare proposition rides the propositions list alone. The wire schema carries propositions (id/name/statement/optional aliases) + chains (premises minItems 1/conclusion) — and since the chains-primary ruling (2026-09-30) the STORE carries exactly that wire shape: chains append as-is to `chains.jsonl` (`SessionFindings` IS the store shape now; `chainsToEdges` and `chainsToGraph` are deleted — no translation seam exists). **Supports edges are retired entirely** (owner, same evening): `EdgeType`, the supports draw toggle and thickness slider, and every supports code path are gone — a stored typed edge no longer exists anywhere, and every arrow on the canvas is derived from a chain. No migration, no backcompat: the pre-chains graph data was wiped by ruling. The chain laws are enforced by the practice-proven error-back hook at the submit seam: every premise/conclusion id must resolve to a submitted proposition, premises are non-empty, no self-reference — a violation returns an error string and the model re-submits within its turns. The prompt is GOAL-FIRST, no rule armor (owner philosophy ruling, 2026-09-30): one shared `EXTRACTION_CORE` (the goal, the truth law, the submission mechanics) plus two one-sentence contextualizations — session and notes preambles — assembled at the call sites, so both front doors share the core by construction; the old defensive lists (DROP lists, merge-your-own-output, name walls, when-in-doubt-omit, the faithfulness paragraph) are all cut, and the old opening describing Quiz/Recall-check/Question blocks (stale since the teacher-prose ruling) is simply omitted. A `prepareArguments` heal at the pre-validation seam rescues stringified-payload submits (propositions or chains). Every node is an objective, world-facing fact. No quiz outcomes, no understanding assessments — the events ledger and `UnderstandingLevel` are deleted; the learner's relationship to a proposition is practice territory — which landed 2026-09-29 as **feedback-only strengthening** (§7): grades live on problem sets, and nothing ever writes learner states back to the graph.
+**The spine — cross-stretch assembly** (owner-approved stage, 2026-09-30): extraction walks a source in stretch-blind batches, and the live corpus showed the seam — 0 of 133 chains crossed a stretch boundary, because no stage minted cross-stretch chains (the extractor sees one batch, the auditor links propositions only, the merger is deterministic). The spine closes it: after each batch's extract and INSIDE the same serialized audit section (owner-specified order — extract → spine → audit → merge, so the spine reads the graph exactly as the following audit will), one strong-tier call sees the same-source digest (every merged proposition of this source, ids verbatim, plus the chains already recorded) and the fresh stretch's findings, and submits only what SPANS them: cross-stretch chains and the connective propositions a chain needs that no stretch stated on its own. The span law is mechanical, enforced at the submit seam (error-back like every chain law): every chain must touch ≥1 fresh endpoint and ≥1 existing digest endpoint — all-fresh is the stretch extractor's own work, all-existing already had its chance when the later endpoint arrived. Identity stays the auditor's lane: the spine submits no links; its connective propositions ride into the audit as ordinary findings (propositions only — the auditor still never sees chains), and its chains resolve through the merger's job-local map, which now accepts existing graph ids verbatim as endpoints (the mixed namespace: fresh `p`/`s` local ids OR digest `n_` ids). Session and notes front doors share the stage exactly as they share `EXTRACTION_CORE` — only the preamble differs. The first stretch of a source skips the call entirely (empty digest — nothing to join, no spend). The spine bills as its own usage stage (`spine` in the run record and the cost panel's Assembly row) and runs under its own phase (`assembling`).
+
**Serialized audit, parallel extracts — per-chunk jobs** (owner ruling 2026-09-28): every chunk (notes file) and every batch (chat coverage) is its own JOB — registry row, usage record, failure domain. Extracts fan out in parallel across all jobs and sources, throttled by a process-wide 3-permit extract semaphore on `globalThis` (the provider-facing rate stance; a job waiting for a permit shows `queued`). The audit→merge section remains a process-wide FIFO critical section — a mutex on `globalThis` (the same module-instance idiom as the run registries): before any auditor LLM call or graph read, a job waits (`waiting` phase) until every earlier job's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. Failure isolation: one job failing (bad chunk, degraded model output) is absorbed at the fan-out — its messages stay uncovered and honestly re-runnable while its siblings still merge; the run returns what actually merged. Abort is source-wide: every job of a source's fan-out dies together, and merged jobs keep their checkpoints. The status feed shows one row per job (`file.md · 2/5` suffixes for chunked sources), and each job persists its own usage record (failed jobs bill their partial spend too).
-**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings' PROPOSITIONS ONLY (the chains never reach the auditor: they are the extractor→store deliverable and the merger's business — owner ruling, 2026-09-30) and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never creates relations between propositions — cross-source connection happens solely through identified equivalences, and argument structure comes from the extractor alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, chain resolution through the job-local map, chain dedupe by (sorted premises, conclusion) — a known argument re-asserted by another chat appends under the SAME `arg_` id (link-style: the first line's id and createdAt stand, provenance unions) — coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Per-job in-memory resolution** (owner, 2026-09-29): the mappings ledger is deleted — each job resolves its own proposition ids against the graph in memory and merges independently, so no job can see or collide with another's ids (the persisted table once silently dropped same-id propositions across chunks and welded wrong edges). Cross-job and cross-run identity is the auditor's links alone, plus the coverage ledger (nothing re-extracts) and the notes content-fingerprint guard.
+**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The spine sees the graph digest + the fresh findings — never the source — and mints structure only (spanning chains + connectives; no identity verdicts). The auditor never sees the transcript — it receives the findings' PROPOSITIONS ONLY (the chunk's propositions plus the spine's connectives; the chains never reach the auditor: they are the extractor→spine→store deliverable and the merger's business — owner ruling, 2026-09-30) and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never creates relations between propositions — cross-source connection happens solely through identified equivalences, and argument structure comes from the extractor and the spine alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, chain resolution through the job-local map (which passes existing `n_` ids through verbatim for the spine's mixed-namespace chains), chain dedupe by (sorted premises, conclusion) — a known argument re-asserted by another chat appends under the SAME `arg_` id (link-style: the first line's id and createdAt stand, provenance unions) — coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Per-job in-memory resolution** (owner, 2026-09-29): the mappings ledger is deleted — each job resolves its own proposition ids against the graph in memory and merges independently, so no job can see or collide with another's ids (the persisted table once silently dropped same-id propositions across chunks and welded wrong edges). Cross-job and cross-run identity is the auditor's links alone, plus the coverage ledger (nothing re-extracts) and the notes content-fingerprint guard.
-**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
+**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 8192 since the spine stage doubled it 2026-09-30, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
**Two front doors, one pipeline**: chat transcripts (quiz-triggered) and raw personal notes (`POST /api/graph/import` — each file becomes a `NotesSource` playing the `chatId` role, deterministic id from the path hash; a content-fingerprint stamp makes an unchanged re-import a TRUE no-op — no model calls, no usage — while changed text re-runs the whole file, owner 2026-09-29). Notes chunk like chats (owner, 2026-09-28): large files split at line boundaries under the same token budget, and each chunk/batch fans out as its own job exactly like a chat run — parallel extracts under the semaphore, serialized audits under the gate. No single model call carries a whole file. Both submit tools require their top-level fields (`chains`/`links` present even when empty): a payload missing them fails validation rather than silently merging structure-less nodes.
diff --git a/src/agent/subagents/extraction.ts b/src/agent/subagents/extraction.ts
index cd4beee..e48478f 100644
--- a/src/agent/subagents/extraction.ts
+++ b/src/agent/subagents/extraction.ts
@@ -30,7 +30,17 @@ import { batchRuns, coverageOf, messageViewLines } from "../../core/coverage.ts"
import { teacherProseBlocks } from "../../core/coverage.ts";
import type { LhMessage } from "../../core/native-messages.ts";
import { auditorGraphTools } from "../tools/graph.ts";
-import { runSubmitAgent } from "./submit-agent.ts";
+import { runSubmitAgent, healStringifiedArrays } from "./submit-agent.ts";
+import {
+ SPINE_INSTRUCTIONS,
+ NOTES_SPINE_INSTRUCTIONS,
+ SPINE_EMPTY,
+ buildSpinePrompt,
+ combineFindings,
+ runSpine,
+ spineHasDigest,
+ type SpineSubmit,
+} from "./spine.ts";
/**
* The extraction pipeline (vision §1 construction, 2026-09-02 redesign;
@@ -199,32 +209,6 @@ const auditSchema = Type.Object({
type NotesFindings = SessionFindings;
type NotesAuditResult = AuditVerdicts;
-// ── the weak-model string-heal (owner, 2026-09-28) ──
-//
-// OpenRouter log evidence: weak models under a big submit payload often
-// emit the whole `propositions` array as ONE JSON-STRING instead of a
-// native array ("propositions.0: must be object"). pi-agent-core runs
-// `prepareArguments` BEFORE TypeBox validation — the sanctioned shim
-// point (same doctrine as restoreMathEscapes at the display seam: heal
-// weak-model wire behavior, don't punish it with a retry). Best-effort:
-// heal only a string that parses to an array; anything else passes
-// through untouched and fails validation exactly as before.
-function healStringifiedArrays(raw: unknown): unknown {
- if (typeof raw !== "object" || raw === null) return raw;
- const out: Record = { ...(raw as Record) };
- for (const key of ["propositions", "chains"]) {
- const v = out[key];
- if (typeof v !== "string" || v.trim().length === 0) continue;
- try {
- const parsed: unknown = JSON.parse(v);
- if (Array.isArray(parsed)) out[key] = parsed;
- } catch {
- // not a clean stringify (e.g. truncated) — leave for validation
- }
- }
- return out;
-}
-
// ── the submit seam (owner, 2026-09-28; redesigned 2026-09-30) ──
//
// Every top-level field on each submit tool is REQUIRED — a payload
@@ -314,9 +298,11 @@ teacher, or this reading: not "the learner understands X" but X. A misconception
correction. Each statement is one self-contained reasoning step at full precision.
Extract the load-bearing ideas of the subject — what a reference on it would teach — not the full run of
-the discourse. Every proposition must stand alone for a reader who never saw this source: names and
-statements may not lean on the source's own scaffolding (claim numbering, axiom tags, arc labels); a
-claim that only matters as part of how this document is organized is not knowledge — drop it.
+the discourse. The graph stores propositions as STEPS OF REASONING: a proposition earns its place only
+when the source's arguments run through it — examples, notation glosses, and restatements that no
+argument needs are not steps. Every proposition must stand alone for a reader who never saw this source:
+names and statements may not lean on the source's own scaffolding (claim numbering, axiom tags, arc
+labels); a claim that only matters as part of how this document is organized is not knowledge — drop it.
SUBMIT with one submit_findings call: every proposition with a short name and a precise statement, and
every chain as premises + conclusion. Premises within one chain are used TOGETHER — independent arguments
@@ -460,7 +446,7 @@ function runAuditor(
* audit+merge critical section is queued behind another job's; the
* extractor is done but the auditor may not start until the graph is
* stable. */
-export type ExtractionPhase = "queued" | "extracting" | "waiting" | "auditing" | "merging";
+export type ExtractionPhase = "queued" | "extracting" | "waiting" | "assembling" | "auditing" | "merging";
/** One in-flight pipeline run. Module-level: one process, one registry.
* Exported for the status payload's consumer typing (the dock). */
@@ -487,6 +473,7 @@ export interface RunningExtraction {
* context-window blobs read this through /api/graph/status). */
usage: {
extractor?: ExtractionStageUsage;
+ spine?: ExtractionStageUsage;
auditor?: ExtractionStageUsage;
};
/** True-cancellation handle (the dock's abort button): one controller per
@@ -556,6 +543,7 @@ export interface ExtractionModels {
/** Test seam: a scripted engine per stage — checked FIRST, ahead of the
* engineFor chain (production wires no direct values). */
extractor?: PiEngine;
+ spine?: PiEngine;
auditor?: PiEngine;
/** Settings-resolved engine per role — the same slot treatment the
* job runner got, read FRESH per stage call (a settings PUT between
@@ -634,6 +622,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
function refuseIfStagesUnready(): void {
const stages: readonly [ModelRole, PiEngine | undefined][] = [
["extractor", stageEngine("extractor", deps.extractor)],
+ ["spine", stageEngine("spine", deps.spine)],
["auditor", stageEngine("auditor", deps.auditor)],
];
const missing = new Set();
@@ -682,7 +671,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
}
/** Accumulate one billed step into the run's registry entry (live). */
- function recordStep(jobKey: string, stage: "extractor" | "auditor", usage: CallUsage, model: string): void {
+ function recordStep(jobKey: string, stage: "extractor" | "spine" | "auditor", usage: CallUsage, model: string): void {
const run = runningExtractions.get(jobKey);
if (!run) return; // not running (tests driving stages directly)
run.usage[stage] ??= { model, steps: [] };
@@ -748,6 +737,50 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
return findings;
}
+ /** THE SPINE (owner, 2026-09-30): the assembly stage between extract
+ * and audit, INSIDE the serialized section (it reads the graph-so-far
+ * digest; its output rides the same audit + merge). Skipped without a
+ * call when the source owns nothing yet (first stretch — nothing to
+ * join). Identity stays the auditor's: the spine submits chains +
+ * connective propositions only, and the audit sees the COMBINED
+ * propositions — chunk findings plus connectives — never chains. */
+ async function assemble(
+ jobKey: string,
+ sourceId: string,
+ kind: "chat" | "notes",
+ findings: SessionFindings,
+ ): Promise {
+ const signal = runningExtractions.get(jobKey)?.controller.signal;
+ signal?.throwIfAborted();
+ const graph = getGraph(graphStore);
+ if (!spineHasDigest(graph, sourceId)) {
+ console.info(`[extraction ${jobKey}] spine skipped — empty digest (first stretch)`);
+ return findings;
+ }
+ console.info(
+ `[extraction ${jobKey}] spine starting (${graph.nodes.length} graph nodes, ` +
+ `${findings.propositions.length} fresh propositions)`,
+ );
+ const t0 = performance.now();
+ const spine: SpineSubmit = await runSpine(
+ stageEngine("spine", deps.spine),
+ kind === "chat" ? SPINE_INSTRUCTIONS : NOTES_SPINE_INSTRUCTIONS,
+ buildSpinePrompt(graph, sourceId, findings),
+ new Set(findings.propositions.map((p) => p.id)),
+ new Set(graph.nodes.map((n) => n.id)),
+ signal,
+ (usage, model, step, finishReason, tools) => {
+ recordStep(jobKey, "spine", callUsage(usage), model);
+ console.info(`[extraction] spine step ${step}: ${finishReason}, tools: ${tools}`);
+ },
+ );
+ console.info(
+ `[extraction ${jobKey}] spine done in ${Math.round(performance.now() - t0)}ms — ` +
+ `${spine.chains.length} chain(s), ${spine.propositions.length} connective(s)`,
+ );
+ return spine === SPINE_EMPTY ? findings : combineFindings(findings, spine);
+ }
+
async function audit(jobKey: string, findings: SessionFindings): Promise {
const signal = runningExtractions.get(jobKey)?.controller.signal;
signal?.throwIfAborted(); // an abort during the extract→audit gap ends here
@@ -816,6 +849,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
stats,
stages: {
extractor: run.usage.extractor ?? { model: "unknown", steps: [] },
+ spine: run.usage.spine ?? { model: "unknown", steps: [] },
auditor: run.usage.auditor ?? { model: "unknown", steps: [] },
},
};
@@ -935,10 +969,14 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
// SERIALIZED AUDIT (owner 2026-09-28): audit + merge are the
// graph-mutating critical section — one job at a time,
// process-wide, so the auditor explores the graph as it
- // stands AFTER every earlier job's merge.
+ // stands AFTER every earlier job's merge. The spine rides the
+ // same section (owner order, 2026-09-30): extract → spine →
+ // audit → merge.
return withAuditLock(jobKey, "chat", undefined, async () => {
+ setPhase(jobKey, "assembling", "chat", undefined, part);
+ const combined = await assemble(jobKey, chatId, "chat", findings);
setPhase(jobKey, "auditing", "chat", undefined, part);
- const auditResult = await audit(jobKey, findings);
+ const auditResult = await audit(jobKey, combined);
setPhase(jobKey, "merging", "chat", undefined, part);
// SESSION-DELETION GUARD (wave 6): a chat deleted mid-run must never
// merge. The deletion route aborts this run first and runs its purge
@@ -955,7 +993,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
// merges in), the stored fingerprint won't match its new
// one and it re-opens: honest by construction.
const foldedAt = new Date().toISOString();
- const batchStats = mergeFindings(chatId, findings, auditResult, { chatStore, graphStore }, {
+ const batchStats = mergeFindings(chatId, combined, auditResult, { chatStore, graphStore }, {
foldedAt,
coverage: batch
.filter((slot) => messageViewLines(slot.message, slot.message.id).length > 0)
@@ -1088,14 +1126,17 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
// parallel, but audit + merge one at a time, process-wide —
// the auditor explores the graph as it stands AFTER every
// earlier job's merge. Reacquired per chunk so runs
- // interleave fairly.
+ // interleave fairly. The spine rides the same section
+ // (owner order, 2026-09-30): extract → spine → audit → merge.
return withAuditLock(jobKey, "notes", input.filePath, async () => {
+ setPhase(jobKey, "assembling", "notes", input.filePath, part);
+ const combined = await assemble(jobKey, id, "notes", findings);
setPhase(jobKey, "auditing", "notes", input.filePath, part);
- const auditResult = await audit(jobKey, findings);
+ const auditResult = await audit(jobKey, combined);
setPhase(jobKey, "merging", "notes", input.filePath, part);
const merged = mergeFindings(
id,
- findings,
+ combined,
auditResult,
{ chatStore, graphStore },
{ foldedAt: input.importedAt },
diff --git a/src/agent/subagents/spine.ts b/src/agent/subagents/spine.ts
new file mode 100644
index 0000000..3d999cc
--- /dev/null
+++ b/src/agent/subagents/spine.ts
@@ -0,0 +1,261 @@
+/**
+ * The spine stage (owner-approved design, 2026-09-30): the extraction
+ * pipeline's ASSEMBLY pass. Extraction walks a source in stretches, each
+ * stretch-blind — but the source's reasoning runs over the whole span,
+ * and the data showed it: 0 of 133 chains crossed a stretch boundary.
+ * The spine runs once per batch, INSIDE the serialized audit section,
+ * between the extract and the audit (owner-specified order):
+ *
+ * extract → [lock: spine → audit → merge] → next batch
+ *
+ * It sees the same-source digest (everything merged so far) plus the
+ * fresh stretch's findings, and mints ONLY what spans them: cross-stretch
+ * chains and the connective propositions a chain needs that no stretch
+ * stated on its own. The SPAN LAW is mechanical (validateSpine below):
+ * every chain must touch ≥1 fresh endpoint AND ≥1 existing endpoint —
+ * induction over the walk means no pair ever needs a second chance.
+ *
+ * Identity stays the AUDITOR's lane (owner ruling): the spine submits no
+ * links. Its connective propositions ride into the audit as ordinary
+ * findings — the auditor links them like any other — and its chains
+ * resolve through the merger's job-local map (endpoints may be existing
+ * graph ids verbatim; see mergeFindings' mixed-namespace resolution).
+ *
+ * Session and notes front doors share this stage; only the preamble
+ * differs (the extractor's law, verbatim).
+ */
+
+import type { Usage } from "@earendil-works/pi-ai";
+import { Type, type Static } from "typebox";
+import { runSubmitAgent, healStringifiedArrays } from "./submit-agent.ts";
+import type { PiEngine } from "../models.ts";
+import type { ExtractedChain, SessionFindings } from "../../core/contracts.ts";
+import type { DerivedGraph } from "../../core/graph.ts";
+
+// ─── the prompts ─────────────────────────────────────────────────────────────
+
+export const SESSION_SPINE_PREAMBLE =
+ "You are the spine stage of a learning application's extraction pipeline. One source — a teaching " +
+ "session, walked in stretches of the teacher's prose in reading order — has just advanced by one " +
+ "stretch; you join what the new stretch established into the source's accumulated line of argument.";
+
+export const NOTES_SPINE_PREAMBLE =
+ "You are the spine stage of a learning application's extraction pipeline. One source — the author's " +
+ "personal notes, walked in stretches in reading order — has just advanced by one stretch; you join " +
+ "what the new stretch established into the source's accumulated line of argument.";
+
+export const SPINE_CORE = `THE GOAL: carry the source's line of argument ACROSS stretches. Extraction walks the source in
+stretches; each stretch reports its own propositions and chains — but the source's reasoning runs over
+the whole span, and no stretch sees more than its own part. You see everything extracted from this
+source so far, plus what the latest stretch just established. Your deliverable: the chains that JOIN them.
+
+Every chain you submit must SPAN: its premises and conclusion may not all come from the same stretch. A
+chain inside one stretch is that stretch's own work; a chain the graph already holds exists. Direction is
+whatever the reasoning runs — earlier into later, or later material completing an argument opened earlier.
+
+The extraction doctrines bind you unchanged: premises JOINTLY entail the conclusion; entailment is the
+only link that earns a chain; every proposition stands alone for a reader who never saw the source;
+load-bearing connections only — when in doubt, omit.
+
+Restatements: when the fresh stretch restates a proposition the graph already holds, build your chains on
+the EXISTING id, not the restatement.
+
+SUBMIT with one submit_spine call: the connecting chains, and only the connective propositions a chain
+needs that no stretch stated on its own — a missing middle step, the general claim a stretch argued only
+locally. Pass empty arrays when the stretch stands alone — never omit the fields. The tool's field
+descriptions carry the details.`;
+
+export const SPINE_INSTRUCTIONS = `${SESSION_SPINE_PREAMBLE}\n\n${SPINE_CORE}`;
+export const NOTES_SPINE_INSTRUCTIONS = `${NOTES_SPINE_PREAMBLE}\n\n${SPINE_CORE}`;
+
+// ─── the submit schema ───────────────────────────────────────────────────────
+
+const spinePropositionProps = {
+ id: Type.String({ description: 'Local id: "s1", "s2", … — referenced by chains.' }),
+ name: Type.String({
+ description:
+ "A very short label — established terminology, the term the discipline itself uses. " +
+ "Not a sentence, needn't be unique or self-contained: this becomes the node's global label.",
+ }),
+ statement: Type.String({
+ description:
+ "One true, self-contained, world-facing reasoning step at full precision — the connective " +
+ "tissue a spanning chain needs that no stretch stated on its own.",
+ }),
+ aliases: Type.Optional(
+ Type.Array(Type.String(), {
+ description: "Verbatim phrasings the source used for the same proposition, if any.",
+ }),
+ ),
+};
+
+const endpoint = (role: string) =>
+ Type.String({
+ description:
+ `The proposition id this ${role} — EITHER an existing proposition id from the digest, ` +
+ 'verbatim ("n_…"), OR the local id of a proposition in this submission ("s1", …) or of the ' +
+ 'fresh stretch\'s findings ("p1", …).',
+ });
+
+const spineChainSchema = Type.Object({
+ premises: Type.Array(endpoint("premise uses"), {
+ minItems: 1,
+ description:
+ "Proposition ids used TOGETHER: they jointly entail the conclusion; independent reasons are separate chains.",
+ }),
+ conclusion: endpoint("conclusion is"),
+});
+
+const spineSchema = Type.Object({
+ propositions: Type.Array(Type.Object(spinePropositionProps), {
+ description:
+ "Connective propositions ONLY — steps a spanning chain needs that no stretch stated on " +
+ "its own. Usually empty.",
+ }),
+ chains: Type.Array(spineChainSchema, {
+ description:
+ "The cross-stretch chains — each must join material that arrived at different times " +
+ "(≥1 endpoint from the fresh stretch, ≥1 from before it). Pass [] when the stretch " +
+ "stands alone — never omit the field.",
+ }),
+});
+
+/** What the spine SUBMITS. fillSpine normalizes it; validateSpine checks
+ * the span laws; the pipeline combines it with the stretch's findings
+ * before the audit (the auditor sees chunk + connective propositions,
+ * propositions only — never chains, owner 2026-09-30). */
+export type SpineSubmit = Static;
+
+export const SPINE_EMPTY: SpineSubmit = { propositions: [], chains: [] };
+
+export function fillSpine(raw: SpineSubmit): SpineSubmit {
+ return {
+ propositions: raw.propositions.map((p) => ({ ...p, aliases: p.aliases ?? [] })),
+ chains: raw.chains,
+ };
+}
+
+/** The span laws — the LOUD channel (error-back retry), mirroring
+ * validateFindingsChains. `chunkIds` = the fresh stretch's finding ids
+ * ("p1"…); `existingIds` = every node id already in the graph. A chain
+ * must touch at least one FRESH endpoint (chunk finding or this
+ * submission's connectives) and at least one EXISTING endpoint — the
+ * mechanical form of the owner's all-or-nothing span rule. Identical
+ * re-assertions are not errors: the merger dedupes them silently. */
+export function validateSpine(value: SpineSubmit, chunkIds: ReadonlySet, existingIds: ReadonlySet): string | null {
+ const localIds = new Set(value.propositions.map((p) => p.id));
+ for (const p of value.propositions) {
+ if (chunkIds.has(p.id) || existingIds.has(p.id)) {
+ return `propositions: "${p.id}" collides with an id this submission does not own — connective ids are fresh ("s1", "s2", …).`;
+ }
+ }
+ const resolves = (id: string) => chunkIds.has(id) || localIds.has(id) || existingIds.has(id);
+ const isFresh = (id: string) => chunkIds.has(id) || localIds.has(id);
+ for (const chain of value.chains) {
+ if (chain.premises.length === 0) {
+ return "chains: premises must name at least one proposition.";
+ }
+ const endpoints = [...chain.premises, chain.conclusion];
+ const unknown = endpoints.find((id) => !resolves(id));
+ if (unknown !== undefined) {
+ return `chains: "${unknown}" is not an existing digest id, a fresh-stretch finding id, or one of this call's proposition ids.`;
+ }
+ if (chain.premises.includes(chain.conclusion)) {
+ return "chains: a conclusion cannot be its own premise.";
+ }
+ if (!endpoints.some(isFresh)) {
+ return `chains: [${endpoints.join(", ")}] is entirely existing material — a chain the graph already had its chance to hold. Every chain must join the fresh stretch to what came before.`;
+ }
+ if (!endpoints.some((id) => existingIds.has(id))) {
+ return `chains: [${endpoints.join(", ")}] lives entirely inside the fresh stretch — that is the stretch extractor's work, not yours. Every chain must join the fresh stretch to what came before.`;
+ }
+ }
+ return null;
+}
+
+// ─── the digest ──────────────────────────────────────────────────────────────
+
+/** Render the same-source digest + fresh findings for the spine's prompt.
+ * Pure: a string fold over the derived graph. The digest carries ids
+ * verbatim ("n_…") — that is what the spine references in its chains. */
+export function buildSpinePrompt(graph: DerivedGraph, sourceId: string, findings: SessionFindings): string {
+ const nameOf = new Map(graph.nodes.map((n) => [n.id, n.name]));
+ const ours = graph.nodes.filter((n) => n.provenance.some((pt) => pt.chatId === sourceId));
+ const ourChains = graph.chains.filter((c) => c.provenance.some((pt) => pt.chatId === sourceId));
+ const lines: string[] = [];
+ lines.push(
+ ours.length > 0
+ ? `THE GRAPH SO FAR — this source's accumulated propositions (ids verbatim, referenced by your chains):`
+ : `THE GRAPH SO FAR — this source has no accumulated propositions yet (nothing to join).`,
+ );
+ for (const n of ours) lines.push(` ${n.id} | ${n.name} | ${n.statement.replace(/\s+/g, " ")}`);
+ if (ourChains.length > 0) {
+ lines.push(`ARGUMENTS ALREADY RECORDED (do not re-submit these):`);
+ for (const c of ourChains) {
+ const names = c.premises.map((p) => nameOf.get(p) ?? p).join(" ∧ ");
+ lines.push(` [${c.premises.join(" + ")} ⟹ ${c.conclusion}] ${names} ⟹ ${nameOf.get(c.conclusion) ?? c.conclusion}`);
+ }
+ }
+ lines.push(``);
+ lines.push(`THE FRESH STRETCH — what the extractor just reported (local ids "p1", …):`);
+ for (const p of findings.propositions) lines.push(` ${p.id} | ${p.name} | ${p.statement.replace(/\s+/g, " ")}`);
+ if (findings.chains.length > 0) {
+ lines.push(`ITS OWN CHAINS (this stretch's local work):`);
+ const freshName = new Map(findings.propositions.map((p) => [p.id, p.name]));
+ for (const c of findings.chains) {
+ const names = c.premises.map((p) => freshName.get(p) ?? p).join(" ∧ ");
+ lines.push(` [${c.premises.join(" + ")} ⟹ ${c.conclusion}] ${names} ⟹ ${freshName.get(c.conclusion) ?? c.conclusion}`);
+ }
+ }
+ lines.push(``);
+ lines.push(`Join the source's line of argument across these two moments.`);
+ return lines.join("\n");
+}
+
+/** Whether the spine has anything to see at all — the first stretch of a
+ * source has an empty digest, and the call is skipped (no spine run, no
+ * spend) until a source owns at least one merged proposition. */
+export function spineHasDigest(graph: DerivedGraph, sourceId: string): boolean {
+ return graph.nodes.some((n) => n.provenance.some((pt) => pt.chatId === sourceId));
+}
+
+// ─── the runner ──────────────────────────────────────────────────────────────
+
+/** The spine over one batch's findings — a single submit call, retry-only
+ * (no exploration tools; everything it needs is in the prompt). */
+export function runSpine(
+ engine: PiEngine | undefined,
+ systemPrompt: string,
+ prompt: string,
+ chunkIds: ReadonlySet,
+ existingIds: ReadonlySet,
+ signal: AbortSignal | undefined,
+ onStep: (usage: Usage, model: string, step: number, finishReason: string, tools: string) => void,
+): Promise {
+ return runSubmitAgent({
+ role: "spine",
+ toolName: "submit_spine",
+ toolDescription: "Submit the cross-stretch chains (and any connective propositions they need).",
+ parameters: spineSchema,
+ fill: fillSpine,
+ validate: (value) => validateSpine(value, chunkIds, existingIds),
+ systemPrompt,
+ prompt,
+ engine,
+ signal,
+ onStep,
+ neverSubmitted: "the spine never submitted its chains",
+ prepareArguments: healStringifiedArrays,
+ });
+}
+
+/** The combined findings the AUDIT + MERGE see: the stretch's propositions
+ * plus the spine's connectives (ONE findings set — the auditor links
+ * connectives like anything else), and both chain sets (endpoints may be
+ * existing graph ids — the merger resolves the mixed namespace). */
+export function combineFindings(findings: SessionFindings, spine: SpineSubmit): SessionFindings {
+ return {
+ propositions: [...findings.propositions, ...spine.propositions.map((p) => ({ ...p, aliases: p.aliases ?? [] }))],
+ chains: [...findings.chains, ...(spine.chains as ExtractedChain[])],
+ };
+}
diff --git a/src/agent/subagents/submit-agent.ts b/src/agent/subagents/submit-agent.ts
index 3663e52..32395af 100644
--- a/src/agent/subagents/submit-agent.ts
+++ b/src/agent/subagents/submit-agent.ts
@@ -5,9 +5,6 @@
* schema-invalid call). The submitted value is captured from the execute
* seam, which only fires on a schema-valid call — the ??-fill (the old zod
* defaults) applies at the capture.
- *
- * Extraction passes its own weak-model argument heal (healStringifiedArrays)
- * through `prepareArguments`; the default is identity pass-through.
*/
import type { Usage } from "@earendil-works/pi-ai";
@@ -18,6 +15,34 @@ import type { ModelRole } from "../../../config.ts";
import type { PiEngine } from "../models.ts";
import { runSubagent } from "./pi-run.ts";
+// ── the weak-model string-heal (owner, 2026-09-28; moved here from
+// extraction.ts with the spine stage, 2026-09-30 — the heal is wire-format
+// medicine shared by every submit agent) ──
+//
+// OpenRouter log evidence: weak models under a big submit payload often
+// emit a whole array as ONE JSON-STRING instead of a native array
+// ("propositions.0: must be object"). pi-agent-core runs
+// `prepareArguments` BEFORE TypeBox validation — the sanctioned shim
+// point (same doctrine as restoreMathEscapes at the display seam: heal
+// weak-model wire behavior, don't punish it with a retry). Best-effort:
+// heal only a string that parses to an array; anything else passes
+// through untouched and fails validation exactly as before.
+export function healStringifiedArrays(raw: unknown): unknown {
+ if (typeof raw !== "object" || raw === null) return raw;
+ const out: Record = { ...(raw as Record) };
+ for (const key of ["propositions", "chains"]) {
+ const v = out[key];
+ if (typeof v !== "string" || v.trim().length === 0) continue;
+ try {
+ const parsed: unknown = JSON.parse(v);
+ if (Array.isArray(parsed)) out[key] = parsed;
+ } catch {
+ // not a clean stringify (e.g. truncated) — leave for validation
+ }
+ }
+ return out;
+}
+
export async function runSubmitAgent(opts: {
role: ModelRole;
toolName: string;
diff --git a/src/core/contracts/chat.ts b/src/core/contracts/chat.ts
index dc5a3fb..a8ec15c 100644
--- a/src/core/contracts/chat.ts
+++ b/src/core/contracts/chat.ts
@@ -92,6 +92,11 @@ export interface ExtractionRunUsage {
stages: {
extractor: ExtractionStageUsage;
auditor: ExtractionStageUsage;
+ /** The spine stage (2026-09-30): cross-stretch chain assembly.
+ * OPTIONAL — records written before the stage existed (and runs
+ * whose digest was empty, so the call was skipped) carry no
+ * entry; readers default it to an empty step-set. */
+ spine?: ExtractionStageUsage;
};
}
diff --git a/src/core/cost-summary.ts b/src/core/cost-summary.ts
index 641acc2..1d193ba 100644
--- a/src/core/cost-summary.ts
+++ b/src/core/cost-summary.ts
@@ -61,6 +61,7 @@ export interface GraphCostSummary {
* historical spend on either is simply not displayed.) */
stages: {
extraction: StageCost;
+ spine: StageCost;
audit: StageCost;
};
/** Newest first, capped at `recentCap` (default 10). */
@@ -112,11 +113,16 @@ function mergeTokens(into: CallUsage, steps: readonly CallUsage[]): void {
* same output — no clock, no fs, no store reads. */
export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
const extraction = emptyStage();
+ const spine = emptyStage();
const audit = emptyStage();
const runs: RecentRun[] = [];
const foldExtractionRun = (run: ExtractionRunUsage, title: string) => {
const stages = run.stages ?? { extractor: { model: "unknown", steps: [] }, auditor: { model: "unknown", steps: [] } };
+ // The spine stage is OPTIONAL on the record (2026-09-30): records
+ // written before it existed (and runs whose digest was empty, so
+ // the call was skipped) carry no entry — default to empty steps.
+ const spineStage = stages.spine ?? { model: "unknown", steps: [] };
// Per-stage: tokens always count; dollars only when priced.
extraction.runs++;
mergeTokens(extraction.tokens, stages.extractor.steps);
@@ -124,6 +130,14 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
if (extDollars === null) extraction.unpricedRuns++;
else extraction.dollars += extDollars;
+ if (spineStage.steps.length > 0) {
+ spine.runs++;
+ mergeTokens(spine.tokens, spineStage.steps);
+ const spDollars = stageDollars(spineStage.model, spineStage.steps);
+ if (spDollars === null) spine.unpricedRuns++;
+ else spine.dollars += spDollars;
+ }
+
audit.runs++;
mergeTokens(audit.tokens, stages.auditor.steps);
const audDollars = stageDollars(stages.auditor.model, stages.auditor.steps);
@@ -135,7 +149,7 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
const runDollars = allStages.every((d) => d === null)
? null
: allStages.reduce((a, d) => a + (d ?? 0), 0);
- const allSteps = [...stages.extractor.steps, ...stages.auditor.steps];
+ const allSteps = [...stages.extractor.steps, ...spineStage.steps, ...stages.auditor.steps];
runs.push({
startedAt: run.startedAt,
at: run.at,
@@ -163,7 +177,7 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
return {
totalDollars: extraction.dollars + audit.dollars,
unpricedRuns,
- stages: { extraction, audit },
+ stages: { extraction, spine, audit },
runs: capped,
};
}
diff --git a/src/core/graph.ts b/src/core/graph.ts
index 655d0a5..67be24d 100644
--- a/src/core/graph.ts
+++ b/src/core/graph.ts
@@ -7,15 +7,21 @@
*
* extractor (LLM) → SessionFindings: the raw reasoning chains —
* propositions (lonely relevant ones included) and
- * follows/supports edges,
- * from the source ALONE (whole session as markdown,
- * or raw notes). Fully trusted: no stage re-checks it.
+ * chains, from the stretch alone (one batch of the
+ * source). Fully trusted: no stage re-checks it.
+ * spine (LLM) → the ASSEMBLY stage (2026-09-30): cross-stretch
+ * chains + connective propositions joining the
+ * fresh stretch to the same-source digest (the span
+ * law is mechanical — src/agent/subagents/spine.ts).
+ * No links — identity stays the auditor's.
* auditor (LLM) → AuditVerdicts: identity links (local id → an existing
* node, or create) via graph-exploration tools —
* links ONLY (owner, 2026-09-28 evening: the auditor
* never mints edges; cross-source connection happens
* solely through identified equivalences). Never sees
- * the source; never drops.
+ * the source; never drops; never sees chains — its
+ * input is the chunk propositions + the spine's
+ * connectives, propositions only.
* merger (HERE) → mergeFindings(): deterministic ledger appends
*
* The merger is deliberately the ONLY stage that touches the ledgers, and it
@@ -194,10 +200,11 @@ export function mergeFindings(
mapping.set(p.id, graphNodeId);
}
- // 2. Chains — the extractor's own, the ONLY chain source; the stored
- // truth (chains-primary, owner 2026-09-30). Endpoints are this
- // job's local proposition ids and resolve through the job-local
- // map; chains carry the run's chat-level provenance point and
+ // 2. Chains — the extractor's and the spine's (the two chain sources;
+ // the stored truth, chains-primary, owner 2026-09-30). Endpoints
+ // resolve through the job-local map OR pass through verbatim when
+ // they name an existing node (the spine references digest ids
+ // directly); chains carry the run's chat-level provenance point and
// dedupe by (premises, conclusion) already asserted by THIS chat —
// an assertion by another chat stays untouched (provenance unions
// at read time; a known argument re-stated appends under the SAME
@@ -211,7 +218,13 @@ export function mergeFindings(
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
}
const emitChain = (chain: ExtractedChain): void => {
- const resolve = (endpoint: string): string | undefined => mapping.get(endpoint);
+ // Mixed namespace (spine stage, 2026-09-30): an endpoint resolves
+ // through the job-local map (this submit's propositions — chunk
+ // findings and spine connectives alike) OR passes through verbatim
+ // when it names an existing node (a spine chain referencing digest
+ // ids directly). Anything else is unresolvable — skip.
+ const resolve = (endpoint: string): string | undefined =>
+ mapping.get(endpoint) ?? (existingIds.has(endpoint) ? endpoint : undefined);
const conclusion = resolve(chain.conclusion);
if (conclusion === undefined) return; // unresolvable endpoint — skip
// Duplicate local ids resolving to one node (two phrasings of one
@@ -228,7 +241,7 @@ export function mergeFindings(
assertedByChat.add(key);
stats.chains!++;
};
- for (const chain of findings.chains) emitChain(chain); // the extractor's chains — the ONLY chain source // the extractor's edges — the ONLY edge source
+ for (const chain of findings.chains) emitChain(chain); // the extractor's chains + the spine's spanning chains — the two chain sources
// 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md):
// only now, after every append succeeded, is the batch covered. The
diff --git a/src/core/settings.ts b/src/core/settings.ts
index 8177cac..d033d0e 100644
--- a/src/core/settings.ts
+++ b/src/core/settings.ts
@@ -45,6 +45,7 @@ export const BACKGROUND_MODEL_ROLES = [
"demoBuilder",
"visualMaker",
"extractor",
+ "spine",
"auditor",
"partitioner",
"problemGenerator",
@@ -226,6 +227,10 @@ export const ROLE_SLOT: Readonly> = {
researcher: "cheap",
demoBuilder: "cheap",
extractor: "cheap",
+ // The spine (2026-09-30): cross-stretch assembly — judgment-dense
+ // work over the whole same-source digest, strong tier like the
+ // practice trio.
+ spine: "strong",
auditor: "cheap",
visualMaker: "vision",
pdfReader: "vision",
@@ -384,7 +389,7 @@ export const FACTORY_SETTINGS: FactorySettings = {
sessionNameSerif: true,
mdRendering: { headingWeight: 900, boldColor: null, italicColor: null },
},
- graph: { enabled: true, extractionChunkTokens: 4096 },
+ graph: { enabled: true, extractionChunkTokens: 8192 },
};
/** The chunk budget's validated floor (GraphSettings.extractionChunkTokens
diff --git a/tests/extraction.test.ts b/tests/extraction.test.ts
index ea97042..7883bb3 100644
--- a/tests/extraction.test.ts
+++ b/tests/extraction.test.ts
@@ -52,6 +52,15 @@ import {
EXTRACTOR_INSTRUCTIONS,
NOTES_EXTRACTOR_INSTRUCTIONS,
} from "../src/agent/subagents/extraction.ts";
+import {
+ SPINE_CORE,
+ SPINE_INSTRUCTIONS,
+ NOTES_SPINE_INSTRUCTIONS,
+ validateSpine,
+ buildSpinePrompt,
+ spineHasDigest,
+ type SpineSubmit,
+} from "../src/agent/subagents/spine.ts";
import { ModelsNotConfiguredError, ProviderKeyMissingError, providerKeyMissing, type ModelSlotName } from "../src/core/settings.ts";
import { coverageOf, messageViewLines } from "../src/core/coverage.ts";
import { chatStore } from "../src/store/chats.ts";
@@ -275,6 +284,15 @@ function submitModel(name: string, input: unknown, onCall?: () => void): PiEngin
* the timing control for the serialized-audit tests (owner 2026-09-28):
* the model call has STARTED (onCall fired, the run holds the audit
* gate) but will not resolve until the test opens the gate. */
+
+/** The spine fixture every pipeline construction carries (stage gate is
+ * strict, owner 2026-09-30): the honest common case — the stretch stands
+ * alone, empty submit. One-shot; multi-batch tests that need a REAL
+ * spanning spine build their own submitModel("submit_spine", …). */
+function spineModel(): PiEngine {
+ return submitModel("submit_spine", { propositions: [], chains: [] });
+}
+
function gatedSubmitModel(name: string, input: unknown, gate: Promise, onCall?: () => void): PiEngine {
return fauxEngine([
async () => {
@@ -384,7 +402,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chains: FINDINGS.chains,
});
- const pipeline = createExtractionPipeline({ chatStore, graphStore: ledgers, extractor, auditor: submitModel("submit_review", verdict) });
+ const pipeline = createExtractionPipeline({ chatStore, graphStore: ledgers, extractor, spine: spineModel(), auditor: submitModel("submit_review", verdict) });
const stats = await pipeline.runExtraction(chatId);
assert.deepEqual(stats, { nodes: 2, chains: 1 });
});
@@ -400,6 +418,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor,
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await pipeline.runExtraction(chatId), { nodes: 2, chains: 1 });
@@ -428,6 +447,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", verdict),
});
const stats = await pipeline.runExtraction(chatId);
@@ -476,7 +496,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
const pipeline = createExtractionPipeline({ chatStore, graphStore: ledgers });
await assert.rejects(pipeline.runExtraction(chatId), (err: unknown) => {
assert.ok(err instanceof ModelsNotConfiguredError);
- assert.deepEqual(err.envelope.missing, ["cheap"]); // extractor + auditor are the cheap tier
+ assert.deepEqual(err.envelope.missing, ["cheap", "strong"]); // extractor + auditor cheap, spine strong
assert.equal(err.envelope.code, "models_not_configured");
return true;
});
@@ -494,6 +514,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor,
+ spine: spineModel(),
auditor,
providerKeyMissing: (provider, tier) => {
seen.push([provider, tier]);
@@ -521,14 +542,15 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: fauxEngine([]),
+ spine: spineModel(),
auditor: fauxEngine([]),
- providerKeyMissing: (provider, tier) => (calls++ === 0 ? null : providerKeyMissing(provider, [tier])),
+ providerKeyMissing: (provider, tier) => (calls++ < 2 ? null : providerKeyMissing(provider, [tier])),
});
await assert.rejects(pipeline.runExtraction(chatId), (err: unknown) => {
- assert.ok(err instanceof ProviderKeyMissingError, "the auditor's refusal surfaces after the extractor passed");
+ assert.ok(err instanceof ProviderKeyMissingError, "the auditor's refusal surfaces after extractor + spine passed");
return true;
});
- assert.equal(calls, 2, "both stages were checked, in order");
+ assert.equal(calls, 3, "all three stages were checked, in order");
assert.equal(ledgers.readCoverage().length, 0);
});
@@ -541,6 +563,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", verdict),
});
const stats = await pipeline.runExtraction(chatId);
@@ -566,17 +589,17 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
engineFor: (role) => {
consulted.push(role);
if (role === "extractor") return submitModel("submit_findings", FINDINGS);
+ if (role === "spine") return submitModel("submit_spine", { propositions: [], chains: [] });
if (role === "auditor") return submitModel("submit_review", verdict);
return submitModel("submit_labels", { labels: [] });
},
});
const stats = await pipeline.runExtraction(chatId);
assert.equal(stats.nodes, 2, "the chain-fed pipeline ran end to end — both propositions merged");
- // The stages were resolved through the chain, in pipeline order (the
- // labeler is fire-and-forget garnish — extractor + auditor are the
- // pinned order).
- assert.equal(consulted[0], "extractor");
- assert.equal(consulted[1], "auditor");
+ // The stages were resolved through the chain, in pipeline order
+ // (extractor → spine → auditor; the labeler is fire-and-forget
+ // garnish for any other role).
+ assert.deepEqual(consulted.slice(0, 3), ["extractor", "spine", "auditor"]);
});
it("direct test values still WIN over engineFor (the priority pin)", async () => {
@@ -587,6 +610,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", verdict),
// If the direct values did NOT win, this throw would surface.
engineFor: (role) => {
@@ -605,6 +629,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
+ spine: spineModel(),
auditor: silentModel(), // never submits the review
});
// The old run-level throw is gone: one failed job is absorbed at
@@ -618,6 +643,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: silentModel(),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await failingExtractor.runExtraction(chatId), { nodes: 0, chains: 0 });
@@ -639,8 +665,15 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
const mixed = createExtractionPipeline({
chatStore,
graphStore: ledgers,
+ // 4096 budget: the fixture is sized to split in two (the factory
+ // default doubled to 8192 with the spine stage, 2026-09-30).
+ getChunkTokens: () => 4096,
engineFor: (role) =>
- role === "extractor" ? fauxEngine([countingStep]) : submitModel("submit_review", { links: [] }),
+ role === "extractor"
+ ? fauxEngine([countingStep])
+ : role === "spine"
+ ? submitModel("submit_spine", { propositions: [], chains: [] })
+ : submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await mixed.runExtraction(chunkyId), { nodes: 2, chains: 1 }, "batch 1's merge survives batch 2's failure");
assert.equal(invocations, 2);
@@ -675,6 +708,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
// Each stage's first generate is the observation point: the
// registry must already carry this chat at the right phase.
extractor: submitModel("submit_findings", FINDINGS, () => seen.push(phaseOf())),
+ spine: spineModel(),
auditor: submitModel("submit_review", verdict, () => seen.push(phaseOf())),
});
await pipeline.runExtraction(chatId);
@@ -701,6 +735,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", findingsA),
+ spine: spineModel(),
auditor: gatedSubmitModel("submit_review", { links: [] }, gate, () => {
aAuditing = true;
}),
@@ -718,6 +753,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
// THE pin: when B's auditor is finally invoked, A's merge must
// already be in the ledgers — B explores the post-merge graph,
// which is the entire point of the gate.
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }, () => {
bAuditorCalled = true;
bSawAlphasMerge = ledgers.readNodes().some((n) => n.name === "Alpha claim");
@@ -755,6 +791,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", single("pa", "Alpha claim", "Alpha statement")),
+ spine: spineModel(),
auditor: gatedSubmitModel("submit_review", { links: [] }, gate),
});
const runA = pipelineA.runExtraction(a.chatId);
@@ -765,6 +802,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", single("pb", "Beta claim", "Beta statement")),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }, () => {
bAuditorCalled = true;
}),
@@ -787,6 +825,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", single("pc", "Gamma claim", "Gamma statement")),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await pipelineC.runExtraction(c.chatId), { nodes: 1, chains: 0 });
@@ -814,6 +853,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
// During the auditor's generate, the registry must already carry
// the extractor's fully accumulated live usage (2 steps × 10/5).
extractor: submitModel("submit_findings", FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", verdict, () =>
usageSeen.push(extractionStatuses().find((r) => r.chatId === chatId)?.usage.extractor?.steps.at(-1)?.inputTokens),
),
@@ -852,6 +892,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: hangingModel(), // hangs until aborted — no step finishes
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
const runPromise = pipeline.runExtraction(chatId);
@@ -902,6 +943,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: gatedExtractor,
+ spine: spineModel(),
auditor: silentModel(),
});
@@ -938,6 +980,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: gatedExtractor,
+ spine: spineModel(),
auditor: silentModel(),
});
// A SECOND instance over DIFFERENT ledgers — the observer a separate
@@ -947,6 +990,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: memoryLedgers(),
extractor: silentModel(),
+ spine: spineModel(),
auditor: silentModel(),
});
@@ -973,6 +1017,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore,
graphStore: ledgers,
extractor: silentModel(),
+ spine: spineModel(),
auditor: silentModel(),
});
await assert.rejects(pipeline.runExtraction("chat_does_not_exist"), /Chat not found/);
@@ -1042,6 +1087,7 @@ describe("the extractor redesign (owner, 2026-09-30): chains are the deliverable
chatStore,
graphStore: ledgers,
extractor,
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await pipeline.runExtraction(chatId), { nodes: 3, chains: 1 });
@@ -1077,6 +1123,7 @@ describe("the extractor redesign (owner, 2026-09-30): chains are the deliverable
chatStore,
graphStore: ledgers,
extractor,
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
// Independent arguments for the same conclusion are separate chains;
@@ -1097,6 +1144,7 @@ describe("the extractor redesign (owner, 2026-09-30): chains are the deliverable
chatStore,
graphStore: ledgers,
extractor,
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await pipeline.runExtraction(chatId), { nodes: 2, chains: 1 });
@@ -1141,6 +1189,7 @@ describe("the extractor redesign (owner, 2026-09-30): chains are the deliverable
chatStore,
graphStore: ledgers,
extractor: retryEngine(bad, FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
// The first submit was schema-VALID — only the chain law caught it;
@@ -1159,6 +1208,7 @@ describe("the extractor redesign (owner, 2026-09-30): chains are the deliverable
chatStore,
graphStore: ledgers,
extractor: retryEngine(bad, FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await pipeline.runExtraction(chatId), { nodes: 2, chains: 1 });
@@ -1175,6 +1225,7 @@ describe("the extractor redesign (owner, 2026-09-30): chains are the deliverable
chatStore,
graphStore: ledgers,
extractor: retryEngine(bad, FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }),
});
assert.deepEqual(await pipeline.runExtraction(chatId), { nodes: 2, chains: 1 });
@@ -1223,6 +1274,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
graphStore: ledgers,
extractor: submitModel("submit_findings", NOTES_FINDINGS),
// During the auditor's call the registry must show the notes run.
+ spine: spineModel(),
auditor: submitModel("submit_review", NOTES_VERDICT, () => {
const run = extractionStatuses().find((r) => r.chatId.startsWith("src_"));
seen.push({ kind: run?.kind, label: run?.label });
@@ -1282,7 +1334,9 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
engineFor: (role) =>
role === "extractor"
? extractorStep()
- : submitModel("submit_review", { links: [] }),
+ : role === "spine"
+ ? submitModel("submit_spine", { propositions: [], chains: [] })
+ : submitModel("submit_review", { links: [] }),
getChunkTokens: () => 20, // 80 chars/line-groups → several chunks
});
const stats = await pipeline.importNotes({ filePath: "/vault/big.md", text });
@@ -1335,6 +1389,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", NOTES_FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", NOTES_VERDICT),
});
const first = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT });
@@ -1365,6 +1420,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
chatStore,
graphStore: ledgers,
extractor: hangingModel(),
+ spine: spineModel(),
auditor: submitModel("submit_review", NOTES_VERDICT),
});
const runPromise = pipeline.importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT });
@@ -1445,7 +1501,11 @@ describe("the chunked pipeline (extraction coverage)", () => {
const pipeline = createExtractionPipeline({
chatStore,
graphStore: ledgers,
+ // 4096 budget: the fixture is sized to split in two (the factory
+ // default doubled to 8192 with the spine stage, 2026-09-30).
+ getChunkTokens: () => 4096,
extractor: fauxEngine([findingsStep((v) => views.push(v)), findingsStep((v) => views.push(v))]),
+ spine: spineModel(),
auditor: fauxEngine([passThroughAuditStep(), passThroughAuditStep()]),
});
const stats = await pipeline.runExtraction(chatId);
@@ -1493,6 +1553,9 @@ describe("the chunked pipeline (extraction coverage)", () => {
const pipeline = createExtractionPipeline({
chatStore,
graphStore: ledgers,
+ // 4096 budget: the fixture is sized to split in two (the factory
+ // default doubled to 8192 with the spine stage, 2026-09-30).
+ getChunkTokens: () => 4096,
// Batch 1 submits and merges; batch 2's extractor call HANGS
// until the abort fires — the dock's abort button, mid-run.
extractor: fauxEngine([
@@ -1507,6 +1570,7 @@ describe("the chunked pipeline (extraction coverage)", () => {
sig?.addEventListener("abort", () => resolve(fauxAssistantMessage("late")), { once: true });
}),
]),
+ spine: spineModel(),
auditor: fauxEngine([passThroughAuditStep()]),
});
const runPromise = pipeline.runExtraction(chatId);
@@ -1537,6 +1601,7 @@ describe("the chunked pipeline (extraction coverage)", () => {
chatStore,
graphStore: ledgers,
extractor: fauxEngine([findingsStep((v) => reRunViews.push(v))]),
+ spine: spineModel(),
auditor: fauxEngine([passThroughAuditStep()]),
});
await finisher.runExtraction(chatId);
@@ -1561,6 +1626,7 @@ describe("the chunked pipeline (extraction coverage)", () => {
chatStore,
graphStore: ledgers,
extractor: fauxEngine([findingsStep()]),
+ spine: spineModel(),
auditor: fauxEngine([passThroughAuditStep()]),
});
await pipeline.runExtraction(chatId);
@@ -1596,6 +1662,7 @@ describe("the chunked pipeline (extraction coverage)", () => {
chatStore,
graphStore: ledgers,
extractor: fauxEngine([findingsStep((v) => reRunViews.push(v))]),
+ spine: spineModel(),
auditor: fauxEngine([passThroughAuditStep()]),
});
await reOpener.runExtraction(chatId);
@@ -1697,10 +1764,19 @@ describe("per-chunk parallel jobs (owner, 2026-09-28)", () => {
auditInFlight--;
return reviewMessage;
};
+ const spineMessage = fauxAssistantMessage(
+ [fauxToolCall("submit_spine", { propositions: [], chains: [] } as unknown as ToolCall["arguments"], { id: "spine-call" })],
+ { stopReason: "toolUse" },
+ );
const pipeline = createExtractionPipeline({
chatStore,
graphStore: ledgers,
- engineFor: (role) => (role === "extractor" ? fauxEngine([slowExtract]) : fauxEngine([slowAudit])),
+ engineFor: (role) =>
+ role === "extractor"
+ ? fauxEngine([slowExtract])
+ : role === "spine"
+ ? fauxEngine([() => spineMessage])
+ : fauxEngine([slowAudit]),
getChunkTokens: () => 20,
});
const stats = await pipeline.importNotes({ filePath: "/vault/parallel-2.md", text: wideText() });
@@ -1726,6 +1802,7 @@ describe("the session-deletion guard (chat deleted mid-run)", () => {
chatStore,
graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS),
+ spine: spineModel(),
auditor: submitModel("submit_review", { links: [] }, () => chatStore.deleteChat(chatId)),
});
const stats = await pipeline.runExtraction(chatId);
@@ -1749,3 +1826,249 @@ describe("the session-deletion guard (chat deleted mid-run)", () => {
assert.equal(chatStore.getChat(chatId), null);
});
});
+
+// ─── the spine stage (owner, 2026-09-30): extract → spine → audit → merge ────
+
+describe("the spine stage (cross-stretch assembly)", () => {
+ it("the system prompt is goal-first: span doctrine + doctrines carried over, one shared core", () => {
+ assert.match(SPINE_CORE, /carry the source's line of argument ACROSS stretches/);
+ assert.match(SPINE_CORE, /may not all come from the same stretch/);
+ assert.match(SPINE_CORE, /JOINTLY entail the conclusion/);
+ assert.match(SPINE_CORE, /[Ee]ntailment is the\s+only link that earns a chain/);
+ assert.match(SPINE_CORE, /when in doubt, omit/);
+ assert.match(SPINE_CORE, /build your chains on\s+the EXISTING id, not the restatement/);
+ assert.match(SPINE_CORE, /SUBMIT with one submit_spine call/);
+ // Assembly: each front door = its preamble + the shared core.
+ assert.ok(SPINE_INSTRUCTIONS.includes(SPINE_CORE));
+ assert.ok(NOTES_SPINE_INSTRUCTIONS.includes(SPINE_CORE));
+ assert.match(SPINE_INSTRUCTIONS, /a teaching session/);
+ assert.match(NOTES_SPINE_INSTRUCTIONS, /the author's personal notes/);
+ });
+
+ it("validateSpine enforces the span law: every chain joins fresh material to existing material", () => {
+ const chunkIds = new Set(["p1"]);
+ const existing = new Set(["n_a", "n_b"]);
+ const ok: SpineSubmit = { propositions: [], chains: [{ premises: ["n_a", "p1"], conclusion: "n_b" }] };
+ assert.equal(validateSpine(ok, chunkIds, existing), null);
+
+ // Connectives count as fresh; existing endpoints may sit anywhere.
+ const connective: SpineSubmit = {
+ propositions: [{ id: "s1", name: "C", statement: "c", aliases: [] }],
+ chains: [{ premises: ["s1", "p1"], conclusion: "n_a" }],
+ };
+ assert.equal(validateSpine(connective, chunkIds, existing), null);
+
+ // Empty submit is VALID — the honest common case (stretch stands alone).
+ assert.equal(validateSpine({ propositions: [], chains: [] }, chunkIds, existing), null);
+
+ // All-existing: the graph already had its chance — rejected.
+ assert.match(
+ validateSpine({ propositions: [], chains: [{ premises: ["n_a"], conclusion: "n_b" }] }, chunkIds, existing) ?? "",
+ /entirely existing material/,
+ );
+ // All-fresh: the stretch extractor's own work — rejected.
+ assert.match(
+ validateSpine(
+ { propositions: [{ id: "s1", name: "C", statement: "c", aliases: [] }], chains: [{ premises: ["p1"], conclusion: "s1" }] },
+ chunkIds,
+ existing,
+ ) ?? "",
+ /entirely inside the fresh stretch/,
+ );
+ // Unknown ids never pass silently.
+ assert.match(
+ validateSpine({ propositions: [], chains: [{ premises: ["p1"], conclusion: "zz" }] }, chunkIds, existing) ?? "",
+ /not an existing digest id/,
+ );
+ // Self-reference and id collisions are loud.
+ assert.match(
+ validateSpine({ propositions: [], chains: [{ premises: ["n_a", "p1"], conclusion: "p1" }] }, chunkIds, existing) ?? "",
+ /cannot be its own premise/,
+ );
+ assert.match(
+ validateSpine(
+ { propositions: [{ id: "p1", name: "X", statement: "x", aliases: [] }], chains: [] },
+ chunkIds,
+ existing,
+ ) ?? "",
+ /collides with an id this submission does not own/,
+ );
+ });
+
+ it("buildSpinePrompt renders the digest verbatim-id + the fresh stretch; spineHasDigest is same-source only", () => {
+ const graph = {
+ nodes: [
+ {
+ id: "n_a",
+ name: "Self-ownership",
+ statement: "s",
+ provenance: [{ chatId: "src_1" }],
+ },
+ { id: "n_b", name: "Foreign", statement: "s", provenance: [{ chatId: "src_2" }] },
+ ],
+ chains: [
+ {
+ id: "arg_x",
+ premises: ["n_a"],
+ conclusion: "n_b",
+ provenance: [{ chatId: "src_1" }],
+ },
+ ],
+ } as unknown as Parameters[0];
+ const findings: SessionFindings = {
+ propositions: [{ id: "p1", name: "Fresh", statement: "f", aliases: [] }],
+ chains: [],
+ };
+ const prompt = buildSpinePrompt(graph, "src_1", findings);
+ assert.match(prompt, /THE GRAPH SO FAR/);
+ assert.match(prompt, /n_a \| Self-ownership/);
+ assert.doesNotMatch(prompt, /n_b \| Foreign/); // other sources' nodes stay out
+ assert.match(prompt, /\[n_a ⟹ n_b\]/); // recorded chains shown as ids
+ assert.match(prompt, /THE FRESH STRETCH/);
+ assert.match(prompt, /p1 \| Fresh/);
+ // The chain line names the foreign conclusion via the id-name map —
+ // ids always render, names come from anywhere in the graph.
+ assert.match(prompt, /Self-ownership ⟹ Foreign/);
+
+ assert.equal(spineHasDigest(graph, "src_1"), true);
+ assert.equal(spineHasDigest(graph, "src_9"), false);
+ });
+
+ it("a seeded digest joins a fresh stretch: the spanning chain + connective ride the audit and merge, billed as the spine stage", async () => {
+ const { chatId } = fixtureChat();
+ const ledgers = memoryLedgers();
+ // A PREVIOUS stretch's residue: two nodes already in the graph,
+ // provenance on this chat — the spine's digest.
+ const seed = (id: string, name: string): GraphNode => ({
+ id,
+ name,
+ statement: `statement of ${name}`,
+ aliases: [],
+ createdAt: "2026-09-30T00:00:00.000Z",
+ provenance: [{ chatId, messageId: "m0" }],
+ });
+ ledgers.appendNode(seed("n_a", "Self-ownership"));
+ ledgers.appendNode(seed("n_b", "Scarcity"));
+ const FINDINGS_B: SessionFindings = {
+ propositions: [{ id: "p1", name: "First-use survives", statement: "c", aliases: [] }],
+ chains: [],
+ };
+ const spineViews: string[] = [];
+ let spineCalls = 0;
+ const phases: string[] = [];
+ type LiveUsage = NonNullable[number]["usage"]>;
+ const usages: LiveUsage[] = [];
+ const spineEngine = fauxEngine([
+ (context) => {
+ spineCalls++;
+ spineViews.push(capturedView(context));
+ phases.push(extractionStatuses().find((r) => r.chatId === chatId)?.phase ?? "?");
+ const submit: SpineSubmit = {
+ propositions: [{ id: "s1", name: "Appropriation axiom", statement: "d", aliases: [] }],
+ chains: [{ premises: ["n_a", "p1"], conclusion: "s1" }],
+ };
+ return fauxAssistantMessage([fauxToolCall("submit_spine", submit as unknown as ToolCall["arguments"], { id: "sp1" })], { stopReason: "toolUse" });
+ },
+ ]);
+ const reviewMessage = () =>
+ fauxAssistantMessage([fauxToolCall("submit_review", { links: [] } as unknown as ToolCall["arguments"], { id: "rv" })], { stopReason: "toolUse" });
+ const pipeline = createExtractionPipeline({
+ chatStore,
+ graphStore: ledgers,
+ extractor: submitModel("submit_findings", FINDINGS_B),
+ spine: spineEngine,
+ auditor: fauxEngine([
+ () => {
+ usages.push(extractionStatuses().find((r) => r.chatId === chatId)?.usage ?? {});
+ return reviewMessage();
+ },
+ ]),
+ });
+ const stats = await pipeline.runExtraction(chatId);
+
+ // The spine ran once, under its own phase, seeing the digest and the
+ // fresh stretch rendered together (existing ids verbatim).
+ assert.equal(spineCalls, 1);
+ assert.deepEqual(phases, ["assembling"]);
+ assert.match(spineViews[0]!, /THE GRAPH SO FAR/);
+ assert.match(spineViews[0]!, /n_a \| Self-ownership/);
+ assert.match(spineViews[0]!, /p1 \| First-use survives/);
+
+ // 2 nodes this run (the fresh proposition + the connective), 1 spanning chain.
+ assert.deepEqual(stats, { nodes: 2, chains: 1 });
+ const stored = ledgers.readChains();
+ assert.equal(stored.length, 1);
+ const nodes = ledgers.readNodes();
+ const byName = new Map(nodes.map((n) => [n.name, n.id]));
+ // Mixed namespace resolved: the existing n_a passes through
+ // VERBATIM; fresh p1/s1 resolve through the job-local map.
+ assert.deepEqual(stored[0]!.premises, ["n_a", byName.get("First-use survives")]);
+ assert.equal(stored[0]!.conclusion, byName.get("Appropriation axiom"));
+
+ // Billing: the spine is its own stage — one extractor step, one
+ // spine step, one auditor step, live in the registry.
+ assert.equal(usages[0]!.extractor?.steps.length, 1);
+ assert.equal(usages[0]!.spine?.steps.length, 1);
+ });
+
+ it("a non-spanning spine submit is REFUSED with the problem and retried: only the spanning retry merges", async () => {
+ const { chatId } = fixtureChat();
+ const ledgers = memoryLedgers();
+ const seed = (id: string, name: string): GraphNode => ({
+ id,
+ name,
+ statement: `statement of ${name}`,
+ aliases: [],
+ createdAt: "2026-09-30T00:00:00.000Z",
+ provenance: [{ chatId, messageId: "m0" }],
+ });
+ ledgers.appendNode(seed("n_a", "Self-ownership"));
+ ledgers.appendNode(seed("n_b", "Scarcity"));
+ const FINDINGS_B: SessionFindings = {
+ propositions: [{ id: "p1", name: "First-use survives", statement: "c", aliases: [] }],
+ chains: [],
+ };
+ const spineEngine = fauxEngine([
+ // All-existing chain: the graph already had its chance — refused
+ // with the problem, error-back to the model.
+ () =>
+ fauxAssistantMessage(
+ [
+ fauxToolCall(
+ "submit_spine",
+ { propositions: [], chains: [{ premises: ["n_a"], conclusion: "n_b" }] } as unknown as ToolCall["arguments"],
+ { id: "sp-bad" },
+ ),
+ ],
+ { stopReason: "toolUse" },
+ ),
+ // The retry spans: existing n_a + fresh p1 ⟹ existing n_b.
+ () =>
+ fauxAssistantMessage(
+ [
+ fauxToolCall(
+ "submit_spine",
+ { propositions: [], chains: [{ premises: ["n_a", "p1"], conclusion: "n_b" }] } as unknown as ToolCall["arguments"],
+ { id: "sp-good" },
+ ),
+ ],
+ { stopReason: "toolUse" },
+ ),
+ ]);
+ const pipeline = createExtractionPipeline({
+ chatStore,
+ graphStore: ledgers,
+ extractor: submitModel("submit_findings", FINDINGS_B),
+ spine: spineEngine,
+ auditor: submitModel("submit_review", { links: [] }),
+ });
+ const stats = await pipeline.runExtraction(chatId);
+ // The refused attempt left nothing; the retry's spanning chain merged.
+ assert.deepEqual(stats, { nodes: 1, chains: 1 });
+ const stored = ledgers.readChains();
+ assert.equal(stored.length, 1);
+ const firstUse = ledgers.readNodes().find((n) => n.name === "First-use survives")!;
+ assert.deepEqual(stored[0]!.premises, ["n_a", firstUse.id]);
+ assert.equal(stored[0]!.conclusion, "n_b");
+ });
+});
diff --git a/tests/graph-view-filters.test.ts b/tests/graph-view-filters.test.ts
index f984885..9159454 100644
--- a/tests/graph-view-filters.test.ts
+++ b/tests/graph-view-filters.test.ts
@@ -54,12 +54,13 @@ describe("displayCostTotals (naming-free cost fold)", () => {
const costs = {
stages: {
extraction: stage(1.25, 1),
+ spine: stage(0), // spine channel exists since 2026-09-30 — zero spend here
audit: stage(0.75),
naming: stage(9.99, 2), // dead channel — must never reach the display
},
} as const;
- it("sums extraction + audit only — the dead namer's spend is invisible", () => {
+ it("sums the live stages only — the dead namer's spend is invisible", () => {
assert.equal(displayCostTotals(costs).totalDollars, 2);
});
diff --git a/tests/settings-body.test.ts b/tests/settings-body.test.ts
index 0a8fafb..a0ae808 100644
--- a/tests/settings-body.test.ts
+++ b/tests/settings-body.test.ts
@@ -49,7 +49,7 @@ describe("parseAgentSettingsBody", () => {
});
const bad = parseAgentSettingsBody({ models: { teacher: {} } });
assert.equal(bad.ok, false);
- if (!bad.ok) assert.match(bad.error, /unknown role "teacher" \(supported: researcher, demoBuilder, visualMaker, extractor, auditor, partitioner, problemGenerator, grader\)/);
+ if (!bad.ok) assert.match(bad.error, /unknown role "teacher" \(supported: researcher, demoBuilder, visualMaker, extractor, spine, auditor, partitioner, problemGenerator, grader\)/);
const malformed = parseAgentSettingsBody({ models: { researcher: { model: { provider: 1, id: "x" } } } });
assert.equal(malformed.ok, false);
});
diff --git a/tests/settings-resolution.test.ts b/tests/settings-resolution.test.ts
index 63950cb..111cdb7 100644
--- a/tests/settings-resolution.test.ts
+++ b/tests/settings-resolution.test.ts
@@ -42,8 +42,8 @@ describe("resolveSettings", () => {
assert.equal(eff.appearance.sessionNameSerif, true);
assert.equal(eff.appearance.mdRendering.headingWeight, 900);
// wave 7 collapse: the master switch ONLY + the redesign's chunk
- // budget — absent record resolves the factory 4096 (the old config constant)
- assert.deepEqual(eff.graph, { enabled: true, extractionChunkTokens: 4096 });
+ // budget — absent record resolves the factory 8192 (the 2026-09-30 bump)
+ assert.deepEqual(eff.graph, { enabled: true, extractionChunkTokens: 8192 });
});
it("explicit record values win over every factory default", () => {
@@ -75,8 +75,8 @@ describe("resolveSettings", () => {
assert.equal(eff.appearance.proseFontSize, 20);
assert.equal(eff.appearance.mdRendering.headingWeight, 600);
// wave 7 collapse: the section IS the master switch (absent budget
- // field = factory 4096 — a pre-redesign record resolves cleanly).
- assert.deepEqual(eff.graph, { enabled: false, extractionChunkTokens: 4096 });
+ // field = factory 8192 — a pre-redesign record resolves cleanly).
+ assert.deepEqual(eff.graph, { enabled: false, extractionChunkTokens: 8192 });
// The chunk budget rides the same resolution: stored value wins.
const budgeted = resolveSettings({ version: 1, graph: { extractionChunkTokens: 2000 } });
assert.deepEqual(budgeted.graph, { enabled: true, extractionChunkTokens: 2000 });
diff --git a/tests/settings.test.ts b/tests/settings.test.ts
index c909d14..f8c6d1c 100644
--- a/tests/settings.test.ts
+++ b/tests/settings.test.ts
@@ -22,8 +22,9 @@ describe("FACTORY_SETTINGS (the compiled factory layer)", () => {
assert.equal(CURRENT_SETTINGS_VERSION, 1);
assert.deepEqual(FACTORY_SETTINGS.general, { autoUpdate: true });
// wave 7 collapse: the master switch ONLY + the extraction-UX
- // redesign's chunk budget (2026-09-24) — default = the old config constant
- assert.deepEqual(FACTORY_SETTINGS.graph, { enabled: true, extractionChunkTokens: 4096 });
+ // redesign's chunk budget (2024-09-24); default doubled 8192 with
+ // the spine stage (owner, 2026-09-30)
+ assert.deepEqual(FACTORY_SETTINGS.graph, { enabled: true, extractionChunkTokens: 8192 });
const a = FACTORY_SETTINGS.appearance;
assert.equal(a.mode, "dark");
assert.equal(a.background, null); // the mode's own default, not a factory hex
@@ -202,12 +203,13 @@ describe("sanitizeUserSettings", () => {
assert.deepEqual(out.agent?.tools, { quiz: false, demo: true });
});
- it("BACKGROUND_MODEL_ROLES matches the eight background roles (practice trio added, owner 2026-09-29)", () => {
+ it("BACKGROUND_MODEL_ROLES matches the nine background roles (spine added 2026-09-30; practice trio 2026-09-29)", () => {
assert.deepEqual([...BACKGROUND_MODEL_ROLES], [
"researcher",
"demoBuilder",
"visualMaker",
"extractor",
+ "spine",
"auditor",
"partitioner",
"problemGenerator",
diff --git a/tests/zero-keys.test.tsx b/tests/zero-keys.test.tsx
index 4c9ff6f..83d8e20 100644
--- a/tests/zero-keys.test.tsx
+++ b/tests/zero-keys.test.tsx
@@ -127,6 +127,7 @@ const catalog = (strongProvider: string | null, ...keyedProviders: string[]): Mo
demoBuilder: null,
visualMaker: null,
extractor: null,
+ spine: null,
auditor: null,
partitioner: null,
problemGenerator: null,