sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Auditor submits links ONLY — the edges argument is gone (owner, 2026-09-28 evening)

Browse Source 
Hypothesis under test by the owner: coherent extractions turn to mush
after merging because the auditor minted too many bridging edges of its
own — inference, not identity, and out of the auditor's scope. The graph
now connects across sources solely through identified equivalences;
relations between propositions come from the extractor alone.

- auditSchema drops 'edges' (submit_review = links only); the submit-seam
  required-field doctrine reworded per tool
- AUDITOR_INSTRUCTIONS: the BRIDGING EDGES section dies; deliverable is
  the identity verdicts only
- AuditVerdicts drops edges; ExtractedEdge endpoints are extractor-local
  ids only; mergeFindings stops consuming verdict edges and its resolve()
  drops the raw-node-id branch (dead — the extractor never sees graph ids)
- extractor untouched: submit_findings keeps propositions + follows/
  supports edges, required as before
- tests: 17 submit fixtures + 9 typed verdict literals slimmed to links
- docs: architecture.md division-of-trust + pipeline diagram updated
This commit is contained in:
amosblomqvist
2026-09-29 01:16:32 +03:00
parent f72a3f3ada
commit 3d9d20794c
6 changed files with 65 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+3 -3
View File
@@ -332,8 +332,8 @@ flowchart TB
IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path) — idempotent re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path) — idempotent re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"] RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"]
BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"] BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"]
EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity (link-or-create, equivalence bias) + may draw bridging edges"] EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity ONLY (link-or-create, equivalence bias) — never mints edges"]
AUD --> MERGE["MERGER (deterministic, core/graph.ts) — the ONLY ledger writer:<br/>identity via validated linkTo · bridging-edge resolution · dedupe edges · coverage checkpoint"] AUD --> MERGE["MERGER (deterministic, core/graph.ts) — the ONLY ledger writer:<br/>identity via validated linkTo · dedupe edges · coverage checkpoint"]
MERGE --> SETTLE["settleGraphLayout — resting positions ledger"] MERGE --> SETTLE["settleGraphLayout — resting positions ledger"]
MERGE -->|"failure at ANY stage"| NOTHING["checkpoints nothing — the next trigger re-runs"] MERGE -->|"failure at ANY stage"| NOTHING["checkpoints nothing — the next trigger re-runs"]
``` ```
@@ -342,7 +342,7 @@ flowchart TB
**Serialized audit, parallel extracts — per-chunk jobs** (owner ruling 2026-09-28): every chunk (notes file) and every batch (chat coverage) is its own JOB — registry row, usage record, failure domain. Extracts fan out in parallel across all jobs and sources, throttled by a process-wide 3-permit extract semaphore on `globalThis` (the provider-facing rate stance; a job waiting for a permit shows `queued`). The audit→merge section remains a process-wide FIFO critical section — a mutex on `globalThis` (the same module-instance idiom as the run registries): before any auditor LLM call or graph read, a job waits (`waiting` phase) until every earlier job's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. Failure isolation: one job failing (bad chunk, degraded model output) is absorbed at the fan-out — its messages stay uncovered and honestly re-runnable while its siblings still merge; the run returns what actually merged. Abort is source-wide: every job of a source's fan-out dies together, and merged jobs keep their checkpoints. The status feed shows one row per job (`file.md · 2/5` suffixes for chunked sources), and each job persists its own usage record (failed jobs bill their partial spend too). **Serialized audit, parallel extracts — per-chunk jobs** (owner ruling 2026-09-28): every chunk (notes file) and every batch (chat coverage) is its own JOB — registry row, usage record, failure domain. Extracts fan out in parallel across all jobs and sources, throttled by a process-wide 3-permit extract semaphore on `globalThis` (the provider-facing rate stance; a job waiting for a permit shows `queued`). The audit→merge section remains a process-wide FIFO critical section — a mutex on `globalThis` (the same module-instance idiom as the run registries): before any auditor LLM call or graph read, a job waits (`waiting` phase) until every earlier job's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. Failure isolation: one job failing (bad chunk, degraded model output) is absorbed at the fan-out — its messages stay uncovered and honestly re-runnable while its siblings still merge; the run returns what actually merged. Abort is source-wide: every job of a source's fan-out dies together, and merged jobs keep their checkpoints. The status feed shows one row per job (`file.md · 2/5` suffixes for chunked sources), and each job persists its own usage record (failed jobs bill their partial spend too).
**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links, edges }` — identity links (which findings propositions ARE existing nodes) plus bridging edges (`follows`/`supports` connections between existing graph nodes and the batch's propositions). It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never mints edges — cross-source connection happens solely through identified equivalences, and relations between propositions come from the extractor alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output.
**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort. **Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
src/agent/subagents/extraction.ts
+18 -26
View File
@@ -45,9 +45,10 @@ import { auditorGraphTools } from "../tools/graph.ts";
* proposition, linkTo an existing graph node or create — * proposition, linkTo an existing graph node or create —
* decided by EXPLORING the graph through tools (find / * decided by EXPLORING the graph through tools (find /
* list_sessions / list_propositions), not by an injected * list_sessions / list_propositions), not by an injected
* graph dump (context bloat, owner 2026-09-27). It may * graph dump (context bloat, owner 2026-09-27). Links are
* also draw BRIDGING edges from existing graph nodes to * its ONLY deliverable — it never mints edges (owner
* the batch's propositions. * 2026-09-28: the graph connects across sources solely
* through identified equivalences).
* merger (core) mergeFindings() → deterministic ledger appends * merger (core) mergeFindings() → deterministic ledger appends
* *
* The events ledger is GONE (2026-09-27): quiz outcomes and understanding * The events ledger is GONE (2026-09-27): quiz outcomes and understanding
@@ -159,9 +160,10 @@ const findingsProps = {
const findingsSchema = Type.Object(findingsProps); const findingsSchema = Type.Object(findingsProps);
// VERDICTS-ONLY (owner, 2026-09-28): the auditor never alters the // VERDICTS-ONLY (owner, 2026-09-28): the auditor never alters the
// extractor's findings, so it does not restate them — links + bridging // extractor's findings, so it does not restate them — links are its
// edges are its entire deliverable. An order-of-magnitude smaller submit // ENTIRE deliverable (owner, 2026-09-28 evening: the edges argument is
// than the old full-findings echo. // GONE; the auditor never mints edges. The graph connects across sources
// solely through identified equivalences — a link, never an inference).
const auditSchema = Type.Object({ const auditSchema = Type.Object({
links: Type.Array( links: Type.Array(
Type.Object({ Type.Object({
@@ -173,10 +175,6 @@ const auditSchema = Type.Object({
}), }),
{ description: "Identity verdicts — which findings propositions ARE existing graph nodes. A proposition absent from here = create a new node. Pass [] when none — never omit the field." }, { description: "Identity verdicts — which findings propositions ARE existing graph nodes. A proposition absent from here = create a new node. Pass [] when none — never omit the field." },
), ),
edges: Type.Array(edgeSchema, {
description:
"BRIDGING edges ONLY — new edges welding this batch into the existing graph, with an EXISTING graph node id as one endpoint (or both) and/or a findings proposition id. NEVER restate the extractor's own edges. Pass [] when none — never omit the field.",
}),
}); });
// The notes front door's shapes are IDENTICAL to the session shapes now // The notes front door's shapes are IDENTICAL to the session shapes now
@@ -213,8 +211,9 @@ function healStringifiedArrays(raw: unknown): unknown {
// ── the submit seam (owner, 2026-09-28) ── // ── the submit seam (owner, 2026-09-28) ──
// //
// Both top-level fields on both submit tools are REQUIRED — a payload // Every top-level field on each submit tool is REQUIRED — a payload
// missing `edges`/`links` fails validation instead of silently merging // missing `edges` (findings) or `links` (review) fails validation instead
// of silently merging
// edge-less (the degraded-retry hole: the model's first truncated // edge-less (the degraded-retry hole: the model's first truncated
// attempt failed validation, then the retry passed with propositions // attempt failed validation, then the retry passed with propositions
// only — optional edges ??-filled [] — merging a pile of nodes with no // only — optional edges ??-filled [] — merging a pile of nodes with no
@@ -230,7 +229,6 @@ function fillFindings(raw: Static<typeof findingsSchema>): SessionFindings {
function fillAudit(raw: Static<typeof auditSchema>): AuditVerdicts { function fillAudit(raw: Static<typeof auditSchema>): AuditVerdicts {
return { return {
links: raw.links, links: raw.links,
edges: raw.edges,
}; };
} }
@@ -464,18 +462,12 @@ IDENTITY — links, per proposition:
propositions — different claims about the world — stay apart. You are NOT guarding against wrong merges; you are propositions — different claims about the world — stay apart. You are NOT guarding against wrong merges; you are
hunting for right ones. hunting for right ones.
BRIDGING EDGES — the one place you create edges: Your deliverable is ONLY the identity verdicts: "links". NEVER restate the propositions or the extractor's edges —
- You may add edges whose endpoints are an EXISTING graph node id and one of the batch's propositions (use the the merger already has them. You NEVER create edges — relations between the new propositions come from the
existing node's real id as "from" or "to"). This is how the new material gets welded into the old graph: when the extractor alone, and the batch connects to the existing graph ONLY through the equivalences you identify. Your
existing graph's content makes a proposition follow, or grounds it, draw the edge. entire deliverable is ONE submit_review call.
- Keep edges typed like the extractor's: "follows" = derivable; "supports" = grounds without entailing.
- Never invent propositions. NEVER restate the extractor's own edges — they merge as-is.
Your deliverable is ONLY the verdicts: "links" + "edges" (bridging ONLY). NEVER restate the propositions or the Include the "links" field even when empty — "links": [] when nothing links. NEVER omit it because it is empty.`;
extractor's edges — the merger already has them. Your entire deliverable is ONE submit_review call.
Include EVERY top-level field of the call even when empty — "links": [] when nothing links, "edges": [] when there
are no bridges. NEVER omit a field because it is empty.`;
/** The auditor — one shape for both front doors now (the notes schema was /** The auditor — one shape for both front doors now (the notes schema was
* identical once attribution died). An agentic loop: graph-exploration * identical once attribution died). An agentic loop: graph-exploration
@@ -490,7 +482,7 @@ function runAuditor(
return runSubmitAgent({ return runSubmitAgent({
role: "auditor", role: "auditor",
toolName: "submit_review", toolName: "submit_review",
toolDescription: "Submit the verdicts ONLY: identity links + bridging edges — never the propositions themselves.", toolDescription: "Submit the identity verdicts ONLY (links) — never the propositions themselves.",
parameters: auditSchema, parameters: auditSchema,
fill: fillAudit, fill: fillAudit,
systemPrompt: AUDITOR_INSTRUCTIONS, systemPrompt: AUDITOR_INSTRUCTIONS,
@@ -834,7 +826,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
); );
console.info( console.info(
`[extraction ${jobKey}] audit: ${findings.propositions.length} propositions through, ` + `[extraction ${jobKey}] audit: ${findings.propositions.length} propositions through, ` +
`${auditResult.links.length} linked to existing nodes, ${auditResult.edges.length} bridging edges`, `${auditResult.links.length} linked to existing nodes`,
); );
console.info(`[extraction ${jobKey}] auditor done in ${Math.round(performance.now() - t0)}ms`); console.info(`[extraction ${jobKey}] auditor done in ${Math.round(performance.now() - t0)}ms`);
return auditResult; return auditResult;
src/core/contracts/graph.ts
+10 -15
View File
@@ -36,9 +36,7 @@ export interface EmissionStats {
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and /** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
* edges record every (chat, message) where they came up, deep-linkable to * edges record every (chat, message) where they came up, deep-linkable to
* the exact transcript point. Auditor-drawn bridging edges carry the chat * the exact transcript point. */
* but NO message anchor (nothing in the transcript asserts them — the
* auditor is the asserter); messageId is "" there. */
export interface ProvenancePoint { export interface ProvenancePoint {
/** The chat this point anchors to — or a notes source id ("src_…"): /** The chat this point anchors to — or a notes source id ("src_…"):
* imports carry no message anchors (provenance is the file itself, * imports carry no message anchors (provenance is the file itself,
@@ -211,14 +209,12 @@ export interface ExtractedProposition {
} }
/** One proposed relation. Endpoints are extractor-local proposition ids /** One proposed relation. Endpoints are extractor-local proposition ids
* ("p1", …); the AUDITOR may also emit bridging edges whose from/to is an * ("p1", …) — the extractor is the ONLY stage that ever proposes edges
* EXISTING graph node id (connecting the new content to what the graph * (the auditor's edges argument is gone, owner 2026-09-28 evening). */
* already holds) — the merger resolves both. */
export interface ExtractedEdge { export interface ExtractedEdge {
/** Proposition id (or, for auditor-drawn bridges, a graph node id) /** Proposition id that is followed / supported. */
* that is followed / supported. */
from: string; from: string;
/** Proposition id (or graph node id) that follows / is supported. */ /** Proposition id that follows / is supported. */
to: string; to: string;
type: EdgeType; type: EdgeType;
} }
@@ -244,14 +240,13 @@ export interface IdentityLink {
/** The auditor's output — VERDICTS ONLY (owner, 2026-09-28): the auditor /** The auditor's output — VERDICTS ONLY (owner, 2026-09-28): the auditor
* never alters the extractor's findings, so it does not restate them; * never alters the extractor's findings, so it does not restate them;
* the deterministic merger joins findings + verdicts. `links` = the * the deterministic merger joins findings + verdicts. `links` = the
* identity verdicts (local id ⇄ existing node); `edges` = BRIDGING * identity verdicts (local id ⇄ existing node) — the auditor's ENTIRE
* edges the auditor draws from existing graph nodes to the batch's * deliverable (owner, 2026-09-28 evening: the edges argument is GONE —
* propositions (endpoints: existing node ids and/or local proposition * the auditor never mints edges; the graph connects across sources
* ids). No drop channel — the auditor never rejects; the extractor is * solely through identified equivalences). No drop channel — the
* fully trusted (2026-09-27). */ * auditor never rejects; the extractor is fully trusted (2026-09-27). */
export interface AuditVerdicts { export interface AuditVerdicts {
links: IdentityLink[]; links: IdentityLink[];
edges: ExtractedEdge[];
} }
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */ /** The GET /api/graph payload — everything the home view's PURE renderer needs. */
src/core/graph.ts
+9 -8
View File
@@ -11,10 +11,11 @@
* from the source ALONE (whole session as markdown, * from the source ALONE (whole session as markdown,
* or raw notes). Fully trusted: no stage re-checks it. * or raw notes). Fully trusted: no stage re-checks it.
* auditor (LLM) → AuditVerdicts: identity links (local id → an existing * auditor (LLM) → AuditVerdicts: identity links (local id → an existing
* node, or create) via graph-exploration tools, plus * node, or create) via graph-exploration tools —
* any BRIDGING edges it draws between existing graph * links ONLY (owner, 2026-09-28 evening: the auditor
* nodes and the new propositions. Never sees the * never mints edges; cross-source connection happens
* source; never drops. * solely through identified equivalences). Never sees
* the source; never drops.
* merger (HERE) → mergeFindings(): deterministic ledger appends * merger (HERE) → mergeFindings(): deterministic ledger appends
* *
* The merger is deliberately the ONLY stage that touches the ledgers, and it * The merger is deliberately the ONLY stage that touches the ledgers, and it
@@ -199,8 +200,9 @@ export function mergeFindings(
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key); if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
} }
const emitEdge = (edge: ExtractedEdge): void => { const emitEdge = (edge: ExtractedEdge): void => {
const resolve = (endpoint: string): string | undefined => // Extractor-local ids only — the auditor's bridging edges are gone
mapping.get(endpoint) ?? (existingIds.has(endpoint) ? endpoint : undefined); // (owner, 2026-09-28 evening), so raw node-id endpoints no longer occur.
const resolve = (endpoint: string): string | undefined => mapping.get(endpoint);
const from = resolve(edge.from); const from = resolve(edge.from);
const to = resolve(edge.to); const to = resolve(edge.to);
if (!from || !to || from === to) return; // unresolvable endpoint — skip if (!from || !to || from === to) return; // unresolvable endpoint — skip
@@ -210,8 +212,7 @@ export function mergeFindings(
assertedByChat.add(key); assertedByChat.add(key);
stats.edges++; stats.edges++;
}; };
for (const edge of findings.edges) emitEdge(edge); for (const edge of findings.edges) emitEdge(edge); // the extractor's edges — the ONLY edge source
for (const edge of verdicts.edges) emitEdge(edge);
// 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md): // 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md):
// only now, after every append succeeded, is the batch covered. The // only now, after every append succeeded, is the batch covered. The
tests/extraction.test.ts
+17 -20
View File
@@ -367,7 +367,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("stringified-payload heal (owner, 2026-09-28): propositions arriving as ONE JSON string still validate and merge", async () => { it("stringified-payload heal (owner, 2026-09-28): propositions arriving as ONE JSON string still validate and merge", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const verdict: AuditVerdicts = { links: [], edges: [] }; const verdict: AuditVerdicts = { links: [] };
// The weak-model failure from the OpenRouter log: the whole array // The weak-model failure from the OpenRouter log: the whole array
// emitted as a JSON-STRING ("propositions.0: must be object"). // emitted as a JSON-STRING ("propositions.0: must be object").
@@ -400,7 +400,6 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
}); });
const verdict: AuditVerdicts = { const verdict: AuditVerdicts = {
links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }], links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }],
edges: [],
}; };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
@@ -515,7 +514,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
// Empty verdicts: everything creates (verdicts-only, 2026-09-28). // Empty verdicts: everything creates (verdicts-only, 2026-09-28).
const verdict: AuditVerdicts = { links: [], edges: [] }; const verdict: AuditVerdicts = { links: [] };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
@@ -534,7 +533,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("engineFor fills the stages (model slots wave): the chain replaces config pins", async () => { it("engineFor fills the stages (model slots wave): the chain replaces config pins", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const verdict: AuditVerdicts = { links: [], edges: [] }; const verdict: AuditVerdicts = { links: [] };
const consulted: string[] = []; const consulted: string[] = [];
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
@@ -561,7 +560,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("direct test values still WIN over engineFor (the priority pin)", async () => { it("direct test values still WIN over engineFor (the priority pin)", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const verdict: AuditVerdicts = { links: [], edges: [] }; const verdict: AuditVerdicts = { links: [] };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
@@ -597,7 +596,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: silentModel(), extractor: silentModel(),
auditor: submitModel("submit_review", { links: [], edges: [] }), auditor: submitModel("submit_review", { links: [] }),
}); });
assert.deepEqual(await failingExtractor.runExtraction(chatId), { nodes: 0, edges: 0 }); assert.deepEqual(await failingExtractor.runExtraction(chatId), { nodes: 0, edges: 0 });
assert.equal(ledgers.readCheckpoints().length, 0); assert.equal(ledgers.readCheckpoints().length, 0);
@@ -619,7 +618,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
engineFor: (role) => engineFor: (role) =>
role === "extractor" ? fauxEngine([countingStep]) : submitModel("submit_review", { links: [], edges: [] }), role === "extractor" ? fauxEngine([countingStep]) : submitModel("submit_review", { links: [] }),
}); });
assert.deepEqual(await mixed.runExtraction(chunkyId), { nodes: 2, edges: 1 }, "batch 1's merge survives batch 2's failure"); assert.deepEqual(await mixed.runExtraction(chunkyId), { nodes: 2, edges: 1 }, "batch 1's merge survives batch 2's failure");
assert.equal(invocations, 2); assert.equal(invocations, 2);
@@ -647,7 +646,6 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
const phaseOf = () => extractionStatuses().find((r) => r.chatId === chatId)?.phase; const phaseOf = () => extractionStatuses().find((r) => r.chatId === chatId)?.phase;
const verdict: AuditVerdicts = { const verdict: AuditVerdicts = {
links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }], links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }],
edges: [],
}; };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
@@ -681,7 +679,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: submitModel("submit_findings", findingsA), extractor: submitModel("submit_findings", findingsA),
auditor: gatedSubmitModel("submit_review", { links: [], edges: [] }, gate, () => { auditor: gatedSubmitModel("submit_review", { links: [] }, gate, () => {
aAuditing = true; aAuditing = true;
}), }),
}); });
@@ -698,7 +696,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
// THE pin: when B's auditor is finally invoked, A's merge must // THE pin: when B's auditor is finally invoked, A's merge must
// already be in the ledgers — B explores the post-merge graph, // already be in the ledgers — B explores the post-merge graph,
// which is the entire point of the gate. // which is the entire point of the gate.
auditor: submitModel("submit_review", { links: [], edges: [] }, () => { auditor: submitModel("submit_review", { links: [] }, () => {
bAuditorCalled = true; bAuditorCalled = true;
bSawAlphasMerge = ledgers.readNodes().some((n) => n.name === "Alpha claim"); bSawAlphasMerge = ledgers.readNodes().some((n) => n.name === "Alpha claim");
}), }),
@@ -735,7 +733,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: submitModel("submit_findings", single("pa", "Alpha claim", "Alpha statement")), extractor: submitModel("submit_findings", single("pa", "Alpha claim", "Alpha statement")),
auditor: gatedSubmitModel("submit_review", { links: [], edges: [] }, gate), auditor: gatedSubmitModel("submit_review", { links: [] }, gate),
}); });
const runA = pipelineA.runExtraction(a.chatId); const runA = pipelineA.runExtraction(a.chatId);
await until(() => extractionStatuses().some((r) => r.chatId === a.chatId && r.phase === "auditing")); await until(() => extractionStatuses().some((r) => r.chatId === a.chatId && r.phase === "auditing"));
@@ -745,7 +743,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: submitModel("submit_findings", single("pb", "Beta claim", "Beta statement")), extractor: submitModel("submit_findings", single("pb", "Beta claim", "Beta statement")),
auditor: submitModel("submit_review", { links: [], edges: [] }, () => { auditor: submitModel("submit_review", { links: [] }, () => {
bAuditorCalled = true; bAuditorCalled = true;
}), }),
}); });
@@ -767,7 +765,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: submitModel("submit_findings", single("pc", "Gamma claim", "Gamma statement")), extractor: submitModel("submit_findings", single("pc", "Gamma claim", "Gamma statement")),
auditor: submitModel("submit_review", { links: [], edges: [] }), auditor: submitModel("submit_review", { links: [] }),
}); });
assert.deepEqual(await pipelineC.runExtraction(c.chatId), { nodes: 1, edges: 0 }); assert.deepEqual(await pipelineC.runExtraction(c.chatId), { nodes: 1, edges: 0 });
}); });
@@ -787,7 +785,6 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
const usageSeen: (number | undefined)[] = []; const usageSeen: (number | undefined)[] = [];
const verdict: AuditVerdicts = { const verdict: AuditVerdicts = {
links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }], links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }],
edges: [],
}; };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
@@ -833,7 +830,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: hangingModel(), // hangs until aborted — no step finishes extractor: hangingModel(), // hangs until aborted — no step finishes
auditor: submitModel("submit_review", { links: [], edges: [] }), auditor: submitModel("submit_review", { links: [] }),
}); });
const runPromise = pipeline.runExtraction(chatId); const runPromise = pipeline.runExtraction(chatId);
@@ -1057,7 +1054,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
engineFor: (role) => engineFor: (role) =>
role === "extractor" role === "extractor"
? extractorStep() ? extractorStep()
: submitModel("submit_review", { links: [], edges: [] }), : submitModel("submit_review", { links: [] }),
getChunkTokens: () => 20, // 80 chars/line-groups → several chunks getChunkTokens: () => 20, // 80 chars/line-groups → several chunks
}); });
const stats = await pipeline.importNotes({ filePath: "/vault/big.md", text }); const stats = await pipeline.importNotes({ filePath: "/vault/big.md", text });
@@ -1092,7 +1089,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
engineFor: (role) => (role === "extractor" ? noEdges() : submitModel("submit_review", { links: [], edges: [] })), engineFor: (role) => (role === "extractor" ? noEdges() : submitModel("submit_review", { links: [] })),
}); });
// Isolated job failure (per-chunk jobs, owner 2026-09-28): the run // Isolated job failure (per-chunk jobs, owner 2026-09-28): the run
// resolves zeros — no throw — and still nothing merged. // resolves zeros — no throw — and still nothing merged.
@@ -1398,7 +1395,7 @@ describe("per-chunk parallel jobs (owner, 2026-09-28)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
engineFor: (role) => engineFor: (role) =>
role === "extractor" ? fauxEngine([abortingStep]) : submitModel("submit_review", { links: [], edges: [] }), role === "extractor" ? fauxEngine([abortingStep]) : submitModel("submit_review", { links: [] }),
getChunkTokens: () => 20, getChunkTokens: () => 20,
}); });
const runPromise = pipeline.importNotes({ filePath: "/vault/parallel.md", text: wideText() }); const runPromise = pipeline.importNotes({ filePath: "/vault/parallel.md", text: wideText() });
@@ -1440,7 +1437,7 @@ describe("per-chunk parallel jobs (owner, 2026-09-28)", () => {
{ stopReason: "toolUse" }, { stopReason: "toolUse" },
); );
const reviewMessage = fauxAssistantMessage( const reviewMessage = fauxAssistantMessage(
[fauxToolCall("submit_review", { links: [], edges: [] } as unknown as ToolCall["arguments"], { id: "review-call" })], [fauxToolCall("submit_review", { links: [] } as unknown as ToolCall["arguments"], { id: "review-call" })],
{ stopReason: "toolUse" }, { stopReason: "toolUse" },
); );
const slowExtract: FauxResponseStep = async () => { const slowExtract: FauxResponseStep = async () => {
@@ -1485,7 +1482,7 @@ describe("the session-deletion guard (chat deleted mid-run)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS), extractor: submitModel("submit_findings", FINDINGS),
auditor: submitModel("submit_review", { links: [], edges: [] }, () => chatStore.deleteChat(chatId)), auditor: submitModel("submit_review", { links: [] }, () => chatStore.deleteChat(chatId)),
}); });
const stats = await pipeline.runExtraction(chatId); const stats = await pipeline.runExtraction(chatId);
tests/graph.test.ts
+8 -8
View File
@@ -256,7 +256,7 @@ function fixtureFindings(): SessionFindings {
/** The auditor's verdicts when nothing links: an empty verdict set is a /** The auditor's verdicts when nothing links: an empty verdict set is a
* valid full review (verdicts-only, owner 2026-09-28). */ * valid full review (verdicts-only, owner 2026-09-28). */
function fixtureVerdicts(): AuditVerdicts { function fixtureVerdicts(): AuditVerdicts {
return { links: [], edges: [] }; return { links: [] };
} }
/** The chat every merger test folds (shared, so ordering is intentional). */ /** The chat every merger test folds (shared, so ordering is intentional). */
@@ -359,7 +359,7 @@ describe("the merger (mergeFindings)", () => {
}, },
// VERDICTS-ONLY: the identity call is a link entry, not a field // VERDICTS-ONLY: the identity call is a link entry, not a field
// on the proposition (owner, 2026-09-28). // on the proposition (owner, 2026-09-28).
{ links: [{ id: "q1", linkTo: alphaId }], edges: [] }, { links: [{ id: "q1", linkTo: alphaId }] },
graphDeps, graphDeps,
sessionSource(chat2.id), sessionSource(chat2.id),
); );
@@ -397,7 +397,7 @@ describe("the merger (mergeFindings)", () => {
], ],
edges: [], edges: [],
}, },
{ links: [], edges: [] }, { links: [] },
graphDeps, graphDeps,
{ foldedAt: "2026-09-13T00:00:00.000Z" }, { foldedAt: "2026-09-13T00:00:00.000Z" },
); );
@@ -430,7 +430,7 @@ describe("the merger (mergeFindings)", () => {
{ from: "p1", to: "p3", type: "follows", }, { from: "p1", to: "p3", type: "follows", },
], ],
}, },
{ links: [], edges: [] }, { links: [] },
graphDeps, graphDeps,
{ foldedAt: "2026-09-13T00:00:00.000Z" }, { foldedAt: "2026-09-13T00:00:00.000Z" },
); );
@@ -482,7 +482,7 @@ describe("the merger (mergeFindings)", () => {
edges: [], edges: [],
}, },
// empty verdicts — identity is the auditor's call alone // empty verdicts — identity is the auditor's call alone
{ links: [], edges: [] }, { links: [] },
graphDeps, graphDeps,
sessionSource(chat3.id), sessionSource(chat3.id),
); );
@@ -506,7 +506,7 @@ describe("the merger (mergeFindings)", () => {
], ],
edges: [], edges: [],
}, },
{ links: [], edges: [] }, { links: [] },
graphDeps, graphDeps,
{ foldedAt: "2026-09-13T00:00:00.000Z" }, { foldedAt: "2026-09-13T00:00:00.000Z" },
); );
@@ -610,7 +610,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
], ],
edges: [{ from: "p1", to: "p2", type: "follows", }], edges: [{ from: "p1", to: "p2", type: "follows", }],
}; };
mergeFindings(dA.id, findingsA, { links: [], edges: [] }, graphDeps, sessionSource(dA.id)); mergeFindings(dA.id, findingsA, { links: [] }, graphDeps, sessionSource(dA.id));
// Resolve through THIS session's mappings — the shared ledger holds // Resolve through THIS session's mappings — the shared ledger holds
// lookalike statements from earlier tests, so statement-lookup would // lookalike statements from earlier tests, so statement-lookup would
// link dB to the wrong node. // link dB to the wrong node.
@@ -636,7 +636,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
], ],
edges: [{ from: "q1", to: "q2", type: "follows", }], edges: [{ from: "q1", to: "q2", type: "follows", }],
}; };
mergeFindings(dB.id, findingsB, { links: [{ id: "q1", linkTo: xId }], edges: [] }, graphDeps, sessionSource(dB.id)); mergeFindings(dB.id, findingsB, { links: [{ id: "q1", linkTo: xId }] }, graphDeps, sessionSource(dB.id));
const zId = graphStore.readMappings().find((m) => m.chatId === dB.id && m.sourceId === "q2")!.graphNodeId; const zId = graphStore.readMappings().find((m) => m.chatId === dB.id && m.sourceId === "q2")!.graphNodeId;
// Coverage rows (both chats) + positions garnish — the deletion // Coverage rows (both chats) + positions garnish — the deletion