verdicts-only auditor: submit_review is {links, bridging edges} — the findings echo dies

Owner ruling 2026-09-28: the auditor has no control over the
extractor's output and never alters it, so re-emitting the full
findings per submit was pure payload cost — the retry collapse the
schema slim attacked, halved again. submit_review is now links
(local proposition id -> existing graph node; EQUIVALENCE BIAS
carried in the linkTo description) + bridging edges ONLY; the
merger joins extractor findings + verdicts.

mergeFindings(chatId, findings, verdicts, ...) — propositions come
from findings verbatim, links join by id, edges emit extractor's
then auditor's through one dedup-preserving emitter. The old echo
form hid an accidental drop channel: an auditor that omitted a
proposition from its echo silently dropped it from the merge.
Verdicts-only removes it structurally — extractor output always
fully merges (tests updated to full-merge expectations).

AuditResult/AuditedProposition -> IdentityLink/AuditVerdicts;
audit() logs links/bridges; batch view renders anchorless like
transcriptView; auditor prompt rewritten to the verdicts-only
deliverable; architecture.md division-of-trust updated.
This commit is contained in:
amosblomqvist
2026-09-28 19:22:58 +03:00
parent bd9694b0e4
commit f11b1664dc
6 changed files with 163 additions and 142 deletions
+1 -1
View File
@@ -342,7 +342,7 @@ flowchart TB
**Serialized audit** (owner ruling 2026-09-28): extraction may run in parallel across sources, but the audit→merge section is a process-wide critical section — a FIFO mutex on `globalThis` (the same module-instance idiom as the run registries). Before any auditor LLM call or graph read, a run checks the gate and waits (`waiting` phase, visible in the status feed) until every earlier run's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. The lock is reacquired per batch (interleaved runs each see the latest graph instead of starving) and is abort-aware on both sides of the queue. **Serialized audit** (owner ruling 2026-09-28): extraction may run in parallel across sources, but the audit→merge section is a process-wide critical section — a FIFO mutex on `globalThis` (the same module-instance idiom as the run registries). Before any auditor LLM call or graph read, a run checks the gate and waits (`waiting` phase, visible in the status feed) until every earlier run's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. The lock is reacquired per batch (interleaved runs each see the latest graph instead of starving) and is abort-aware on both sides of the queue.
**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach identity verdicts: `linkTo` an existing node or let the merger create one. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The auditor may also draw bridging edges — typed `follows`/`supports` connections between existing graph nodes and the batch's propositions (the auditor is the asserter: chat-scoped provenance, no message anchor). The deterministic merger validates and appends: linkTo resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links, edges }` — identity links (which findings propositions ARE existing nodes) plus bridging edges (`follows`/`supports` connections between existing graph nodes and the batch's propositions). It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output.
**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort. **Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
+47 -43
View File
@@ -15,7 +15,7 @@ import {
import type { PiEngine } from "../models.ts"; import type { PiEngine } from "../models.ts";
import { runSubagent } from "./pi-run.ts"; import { runSubagent } from "./pi-run.ts";
import type { import type {
AuditResult, AuditVerdicts,
CallUsage, CallUsage,
ChatRecord, ChatRecord,
ExtractionRunUsage, ExtractionRunUsage,
@@ -152,31 +152,36 @@ const findingsProps = {
const findingsSchema = Type.Object(findingsProps); const findingsSchema = Type.Object(findingsProps);
const auditProps = { // VERDICTS-ONLY (owner, 2026-09-28): the auditor never alters the
...findingsProps, // extractor's findings, so it does not restate them — links + bridging
propositions: Type.Array( // edges are its entire deliverable. An order-of-magnitude smaller submit
Type.Object({ // than the old full-findings echo.
...propositionProps, const auditSchema = Type.Object({
linkTo: Type.Optional( links: Type.Optional(
Type.String({ Type.Array(
Type.Object({
id: Type.String({ description: 'The proposition id from the findings ("p1", …).' }),
linkTo: Type.String({
description: description:
"The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. EQUIVALENCE BIAS (2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart.", "The id of the EXISTING graph node this proposition IS — your identity verdict. EQUIVALENCE BIAS (2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart.",
}), }),
), }),
}), { description: "Identity verdicts — which findings propositions ARE existing graph nodes. A proposition absent from here = create a new node." },
{ ),
description: "Every distinct proposition the session touched — true statements, misconception-free.",
},
), ),
}; edges: Type.Optional(
Type.Array(edgeSchema, {
const auditSchema = Type.Object(auditProps); description:
"BRIDGING edges ONLY — new edges welding this batch into the existing graph, with an EXISTING graph node id as one endpoint (or both) and/or a findings proposition id. NEVER restate the extractor's own edges.",
}),
),
});
// The notes front door's shapes are IDENTICAL to the session shapes now // The notes front door's shapes are IDENTICAL to the session shapes now
// (the only difference was quiz attribution, which no longer exists): // (the only difference was quiz attribution, which no longer exists):
// one schema, one findings type, both front doors. // one schema, one findings type, both front doors.
type NotesFindings = SessionFindings; type NotesFindings = SessionFindings;
type NotesAuditResult = AuditResult; type NotesAuditResult = AuditVerdicts;
// ── the weak-model string-heal (owner, 2026-09-28) ── // ── the weak-model string-heal (owner, 2026-09-28) ──
// //
@@ -213,9 +218,9 @@ function fillFindings(raw: Static<typeof findingsSchema>): SessionFindings {
}; };
} }
function fillAudit(raw: Static<typeof auditSchema>): AuditResult { function fillAudit(raw: Static<typeof auditSchema>): AuditVerdicts {
return { return {
propositions: raw.propositions.map((p) => ({ ...p, aliases: p.aliases ?? [] })), links: raw.links ?? [],
edges: raw.edges ?? [], edges: raw.edges ?? [],
}; };
} }
@@ -437,12 +442,12 @@ its work. Your job is IDENTITY: connecting those findings to the CURRENT knowled
You explore the graph through your tools: find (glob over names + statements + aliases), list_sessions, and You explore the graph through your tools: find (glob over names + statements + aliases), list_sessions, and
list_propositions (full statement text, per session). The graph may be large — search, don't assume. list_propositions (full statement text, per session). The graph may be large — search, don't assume.
IDENTITY — link-or-create, per proposition: IDENTITY — links, per proposition:
- For each proposition, search the graph (its key words, its synonyms, its phrasings). Does an EXISTING node already - For each proposition, search the graph (its key words, its synonyms, its phrasings). Does an EXISTING node already
state this same proposition? state this same proposition?
- If yes: set "linkTo" to that node's id. The session's phrasings join its aliases and its provenance; the node - If yes: add { "id": <its proposition id>, "linkTo": <that node's id> } to "links". The session's phrasings join
keeps its existing name (labels stay stable — ignore your name for linked nodes). its aliases and its provenance; the node keeps its existing name.
- If no: omit "linkTo" and a new node is created with the extractor's name as its label. - If no: add nothing — a new node is created with the extractor's name as its label.
- EQUIVALENCE BIAS (owner ruling): paraphrases, rewordings, and compressions of the same claim MERGE. When two - EQUIVALENCE BIAS (owner ruling): paraphrases, rewordings, and compressions of the same claim MERGE. When two
statements are the same claim at different precision or from different angles, LINK them. Only genuinely distinct statements are the same claim at different precision or from different angles, LINK them. Only genuinely distinct
propositions — different claims about the world — stay apart. You are NOT guarding against wrong merges; you are propositions — different claims about the world — stay apart. You are NOT guarding against wrong merges; you are
@@ -453,13 +458,13 @@ BRIDGING EDGES — the one place you create edges:
existing node's real id as "from" or "to"). This is how the new material gets welded into the old graph: when the existing node's real id as "from" or "to"). This is how the new material gets welded into the old graph: when the
existing graph's content makes a proposition follow, or grounds it, draw the edge. existing graph's content makes a proposition follow, or grounds it, draw the edge.
- Keep edges typed like the extractor's: "follows" = derivable; "supports" = grounds without entailing. - Keep edges typed like the extractor's: "follows" = derivable; "supports" = grounds without entailing.
- NEVER drop or rewrite the extractor's own propositions or edges — pass them through unchanged (adding linkTo is - Never invent propositions. NEVER restate the extractor's own edges — they merge as-is.
the only edit). Never invent propositions.
Reproduce every proposition and edge you received verbatim. Your entire deliverable Your deliverable is ONLY the verdicts: "links" + "edges" (bridging ONLY). NEVER restate the propositions or the
is ONE submit_review call. extractor's edges — the merger already has them. Your entire deliverable is ONE submit_review call.
Include EVERY top-level field of the call even when empty — "edges": []. NEVER omit a field because it is empty.`; Include EVERY top-level field of the call even when empty — "links": [] when nothing links, "edges": [] when there
are no bridges. NEVER omit a field because it is empty.`;
/** The auditor — one shape for both front doors now (the notes schema was /** The auditor — one shape for both front doors now (the notes schema was
* identical once attribution died). An agentic loop: graph-exploration * identical once attribution died). An agentic loop: graph-exploration
@@ -470,11 +475,11 @@ function runAuditor(
signal: AbortSignal | undefined, signal: AbortSignal | undefined,
onStep: (usage: Usage, model: string, step: number, finishReason: string, tools: string) => void, onStep: (usage: Usage, model: string, step: number, finishReason: string, tools: string) => void,
tools: Record<string, AgentTool<any>>, tools: Record<string, AgentTool<any>>,
): Promise<AuditResult> { ): Promise<AuditVerdicts> {
return runSubmitAgent({ return runSubmitAgent({
role: "auditor", role: "auditor",
toolName: "submit_review", toolName: "submit_review",
toolDescription: "Submit the findings with identity verdicts (linkTo) and any bridging edges — for the merge.", toolDescription: "Submit the verdicts ONLY: identity links + bridging edges — never the propositions themselves.",
parameters: auditSchema, parameters: auditSchema,
fill: fillAudit, fill: fillAudit,
systemPrompt: AUDITOR_INSTRUCTIONS, systemPrompt: AUDITOR_INSTRUCTIONS,
@@ -763,7 +768,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
return findings; return findings;
} }
async function audit(chatId: string, findings: SessionFindings): Promise<AuditResult> { async function audit(chatId: string, findings: SessionFindings): Promise<AuditVerdicts> {
const signal = runningExtractions.get(chatId)?.controller.signal; const signal = runningExtractions.get(chatId)?.controller.signal;
signal?.throwIfAborted(); // an abort during the extract→audit gap ends here signal?.throwIfAborted(); // an abort during the extract→audit gap ends here
const graph = getGraph(graphStore); const graph = getGraph(graphStore);
@@ -788,11 +793,9 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
}, },
auditorGraphTools({ chatStore, graphStore }), auditorGraphTools({ chatStore, graphStore }),
); );
const bridged = auditResult.edges.filter((e) => !e.from.startsWith("p") || !e.to.startsWith("p")).length;
console.info( console.info(
`[extraction ${chatId}] audit: ${auditResult.propositions.length} propositions through, ` + `[extraction ${chatId}] audit: ${findings.propositions.length} propositions through, ` +
`${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes, ` + `${auditResult.links.length} linked to existing nodes, ${auditResult.edges.length} bridging edges`,
`${auditResult.edges.length} edges (${bridged} bridging)`,
); );
console.info(`[extraction ${chatId}] auditor done in ${Math.round(performance.now() - t0)}ms`); console.info(`[extraction ${chatId}] auditor done in ${Math.round(performance.now() - t0)}ms`);
return auditResult; return auditResult;
@@ -861,13 +864,14 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
); );
for (const batch of batches) { for (const batch of batches) {
// The batch's source view: the ONE shared markdown engine // The batch's source view: the ONE shared markdown engine
// (owner 2026-09-27) over the batch's messages — the same // (owner 2026-09-27) over the batch's messages. Anchors OFF
// rendering the user-facing session export will use, id // (2026-09-28): propositions report no message ids, so the
// anchors preserved. The fingerprint's line format is // [id] anchors are noise — same as transcriptView. The
// separate and unchanged (no mass re-extraction). // fingerprint's line format is separate and unchanged (no
// mass re-extraction).
const view = sessionMarkdown( const view = sessionMarkdown(
batch.map((slot) => slot.message), batch.map((slot) => slot.message),
{ title: record.title }, { title: record.title, anchors: false },
); );
setPhase(chatId, "extracting", "chat"); setPhase(chatId, "extracting", "chat");
const findings = await extract(chatId, view); const findings = await extract(chatId, view);
@@ -896,7 +900,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
// merges in), the stored fingerprint won't match its new // merges in), the stored fingerprint won't match its new
// one and it re-opens: honest by construction. // one and it re-opens: honest by construction.
const foldedAt = new Date().toISOString(); const foldedAt = new Date().toISOString();
const batchStats = mergeFindings(chatId, auditResult, { chatStore, graphStore }, { const batchStats = mergeFindings(chatId, findings, auditResult, { chatStore, graphStore }, {
foldedAt, foldedAt,
coverage: batch coverage: batch
.filter((slot) => messageViewLines(slot.message, slot.message.id).length > 0) .filter((slot) => messageViewLines(slot.message, slot.message.id).length > 0)
@@ -996,7 +1000,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
setPhase(id, "auditing", "notes", input.filePath); setPhase(id, "auditing", "notes", input.filePath);
const auditResult = await audit(id, findings); const auditResult = await audit(id, findings);
setPhase(id, "merging", "notes", input.filePath); setPhase(id, "merging", "notes", input.filePath);
return mergeFindings(id, auditResult, { chatStore, graphStore }, { foldedAt: input.importedAt }); return mergeFindings(id, findings, auditResult, { chatStore, graphStore }, { foldedAt: input.importedAt });
}); });
// Same post-merge settle as the chat path. // Same post-merge settle as the chat path.
settleGraphLayout(graphStore); settleGraphLayout(graphStore);
+20 -16
View File
@@ -229,24 +229,28 @@ export interface SessionFindings {
edges: ExtractedEdge[]; edges: ExtractedEdge[];
} }
/** The auditor's verdict on one proposition: the findings, plus `linkTo` — /** One identity verdict: the extractor's local proposition id → the
* the identity judgment. Absent = create a new node. */ * EXISTING graph node that proposition IS. On link, the session's
export interface AuditedProposition extends ExtractedProposition { * phrasings (statement, aliases) join the node's aliases and the
/** The existing GraphNode id this proposition IS — the auditor's * provenance unions. EQUIVALENCE BIAS (2026-09-27): paraphrases,
* identity verdict. On link, the session's phrasings (statement, * rewordings, and compressions of the same claim MERGE — do not create
* aliases) join the node's aliases and the provenance unions. * a duplicate because the wording differs. Only genuinely distinct
* EQUIVALENCE BIAS (2026-09-27): paraphrases, rewordings, and * propositions stay apart. */
* compressions of the same claim MERGE — do not create a duplicate export interface IdentityLink {
* because the wording differs. Only genuinely distinct propositions id: string;
* stay apart. */ linkTo: string;
linkTo?: string;
} }
/** The auditor's output: the findings with identity verdicts, plus any /** The auditor's output — VERDICTS ONLY (owner, 2026-09-28): the auditor
* bridging edges drawn to existing graph nodes. No drop channel — the * never alters the extractor's findings, so it does not restate them;
* auditor never rejects; the extractor is fully trusted (2026-09-27). */ * the deterministic merger joins findings + verdicts. `links` = the
export interface AuditResult { * identity verdicts (local id ⇄ existing node); `edges` = BRIDGING
propositions: AuditedProposition[]; * edges the auditor draws from existing graph nodes to the batch's
* propositions (endpoints: existing node ids and/or local proposition
* ids). No drop channel — the auditor never rejects; the extractor is
* fully trusted (2026-09-27). */
export interface AuditVerdicts {
links: IdentityLink[];
edges: ExtractedEdge[]; edges: ExtractedEdge[];
} }
+34 -31
View File
@@ -7,10 +7,10 @@
* *
* extractor (LLM) → SessionFindings: the raw reasoning chains — * extractor (LLM) → SessionFindings: the raw reasoning chains —
* propositions (lonely relevant ones included) and * propositions (lonely relevant ones included) and
* follows/supports edges, each with verbatim evidence, * follows/supports edges,
* from the source ALONE (whole session as markdown, * from the source ALONE (whole session as markdown,
* or raw notes). Fully trusted: no stage re-checks it. * or raw notes). Fully trusted: no stage re-checks it.
* auditor (LLM) → AuditResult: identity verdicts (linkTo an existing * auditor (LLM) → AuditVerdicts: identity links (local id → an existing
* node, or create) via graph-exploration tools, plus * node, or create) via graph-exploration tools, plus
* any BRIDGING edges it draws between existing graph * any BRIDGING edges it draws between existing graph
* nodes and the new propositions. Never sees the * nodes and the new propositions. Never sees the
@@ -43,7 +43,9 @@
import { randomBytes } from "node:crypto"; import { randomBytes } from "node:crypto";
import type { CoverageEntry } from "./contracts/graph.ts"; import type { CoverageEntry } from "./contracts/graph.ts";
import type { import type {
AuditResult, AuditVerdicts,
ExtractedEdge,
SessionFindings,
Cluster, Cluster,
EmissionStats, EmissionStats,
GraphEdge, GraphEdge,
@@ -114,7 +116,13 @@ const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
* idempotency, one graph. The `source` parameter carries the checkpoint * idempotency, one graph. The `source` parameter carries the checkpoint
* stamp; the merger itself is chat-store-free. * stamp; the merger itself is chat-store-free.
*/ */
export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps, source: MergeSource): EmissionStats { export function mergeFindings(
chatId: string,
findings: SessionFindings,
verdicts: AuditVerdicts,
deps: GraphDeps,
source: MergeSource,
): EmissionStats {
const { graphStore: ledgers } = deps; const { graphStore: ledgers } = deps;
const stats: EmissionStats = { nodes: 0, edges: 0 }; const stats: EmissionStats = { nodes: 0, edges: 0 };
@@ -132,16 +140,20 @@ export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDep
seenMappingKeys.add(m.sourceId); seenMappingKeys.add(m.sourceId);
} }
const now = new Date().toISOString(); const now = new Date().toISOString();
const linkOf = new Map(verdicts.links.map((l) => [l.id, l.linkTo]));
for (const p of audit.propositions) { // VERDICTS-ONLY (owner, 2026-09-28): the propositions come from the
// extractor's findings verbatim; the auditor's links join here by id.
for (const p of findings.propositions) {
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
const points = pointOf(chatId); const points = pointOf(chatId);
const linkTo = linkOf.get(p.id);
let graphNodeId: string; let graphNodeId: string;
if (p.linkTo !== undefined && existingIds.has(p.linkTo)) { if (linkTo !== undefined && existingIds.has(linkTo)) {
// Identity verdict: this proposition IS that node. Append a link // Identity verdict: this proposition IS that node. Append a link
// line — the session's phrasings (statement + aliases) join the // line — the session's phrasings (statement + aliases) join the
// node's aliases, the provenance unions. Reads merge by id. // node's aliases, the provenance unions. Reads merge by id.
graphNodeId = p.linkTo; graphNodeId = linkTo;
// Link-append: phrasings + provenance join at read time (the // Link-append: phrasings + provenance join at read time (the
// first line's name/statement/createdAt stand). // first line's name/statement/createdAt stand).
ledgers.appendNode({ ledgers.appendNode({
@@ -173,42 +185,33 @@ export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDep
} }
} }
// 2. Edges. Resolve endpoints through the mapping — with BRIDGING // 2. Edges — the extractor's own first, then the auditor's BRIDGING
// (2026-09-27): an endpoint the extractor never proposed but that // edges (endpoints may be local ids and/or EXISTING graph node ids —
// names an EXISTING graph node is the auditor drawing a connection // the auditor welding the batch into the graph it explored). Both
// between the graph it explored and this batch's propositions. Such // resolve through the mapping, both carry the run's chat-level
// edges carry chat-scoped provenance with NO message anchor (nothing // provenance point, and both dedupe by (from,to,type) already
// in the transcript asserts them — the auditor is the asserter); // asserted by THIS chat: an echoed duplicate (a weak auditor
// transcript-articulated edges keep message-granular provenance. // restating an extractor edge) merges to one append; an assertion
// Dedupe by (from,to,type) already asserted by THIS chat (an existing // by another chat stays untouched (provenance unions at read time).
// assertion by another chat stays untouched — provenance unions at
// read time).
const assertedByChat = new Set<string>(); const assertedByChat = new Set<string>();
for (const line of ledgers.readEdges()) { for (const line of ledgers.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`; const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key); if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
} }
for (const edge of audit.edges) { const emitEdge = (edge: ExtractedEdge): void => {
const resolve = (endpoint: string): string | undefined => const resolve = (endpoint: string): string | undefined =>
mapping.get(endpoint) ?? (existingIds.has(endpoint) ? endpoint : undefined); mapping.get(endpoint) ?? (existingIds.has(endpoint) ? endpoint : undefined);
const from = resolve(edge.from); const from = resolve(edge.from);
const to = resolve(edge.to); const to = resolve(edge.to);
if (!from || !to || from === to) continue; // unresolvable endpoint — skip if (!from || !to || from === to) return; // unresolvable endpoint — skip
const bridged = !mapping.has(edge.from) || !mapping.has(edge.to);
const key = `${from}\u0000${to}\u0000${edge.type}`; const key = `${from}\u0000${to}\u0000${edge.type}`;
if (assertedByChat.has(key)) continue; if (assertedByChat.has(key)) return;
ledgers.appendEdge({ ledgers.appendEdge({ from, to, type: edge.type, provenance: pointOf(chatId) });
from,
to,
type: edge.type,
// bridged vs transcript edges both carry the run's chat-level
// point now; the distinction lives in the endpoint resolution
// above (mapping vs existing node id), not in the provenance.
provenance: pointOf(chatId),
});
assertedByChat.add(key); assertedByChat.add(key);
stats.edges++; stats.edges++;
} };
for (const edge of findings.edges) emitEdge(edge);
for (const edge of verdicts.edges) emitEdge(edge);
// 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md): // 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md):
// only now, after every append succeeded, is the batch covered. The // only now, after every append succeeded, is the batch covered. The
+34 -36
View File
@@ -30,7 +30,7 @@ import type { LhMessage } from "../src/core/native-messages.ts";
import { newPlanBlockMessage } from "../src/core/native-messages.ts"; import { newPlanBlockMessage } from "../src/core/native-messages.ts";
import { projectView as projectNative } from "../src/core/view.ts"; import { projectView as projectNative } from "../src/core/view.ts";
import type { import type {
AuditResult, AuditVerdicts,
CoverageEntry, CoverageEntry,
ExtractionRunUsage, ExtractionRunUsage,
GraphEdge, GraphEdge,
@@ -371,7 +371,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("stringified-payload heal (owner, 2026-09-28): propositions arriving as ONE JSON string still validate and merge", async () => { it("stringified-payload heal (owner, 2026-09-28): propositions arriving as ONE JSON string still validate and merge", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const verdict: AuditResult = { propositions: FINDINGS.propositions, edges: FINDINGS.edges }; const verdict: AuditVerdicts = { links: [], edges: [] };
// The weak-model failure from the OpenRouter log: the whole array // The weak-model failure from the OpenRouter log: the whole array
// emitted as a JSON-STRING ("propositions.0: must be object"). // emitted as a JSON-STRING ("propositions.0: must be object").
@@ -402,10 +402,8 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
createdAt: "2026-08-01T00:00:00.000Z", createdAt: "2026-08-01T00:00:00.000Z",
provenance: [], provenance: [],
}); });
const verdict: AuditResult = { const verdict: AuditVerdicts = {
propositions: [ links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }],
{ ...FINDINGS.propositions[0]!, linkTo: existingId },
],
edges: [], edges: [],
}; };
@@ -417,13 +415,20 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
}); });
const stats = await pipeline.runExtraction(chatId); const stats = await pipeline.runExtraction(chatId);
// The linked proposition created NO new node. // VERDICTS-ONLY (2026-09-28): findings merge in FULL — the linked
assert.deepEqual(stats, { nodes: 0, edges: 0 }); // p1 creates no node, p2 creates one, and the extractor's edge
// resolves through them. (The old echo form could silently drop
// content by omitting it; that accidental drop channel is gone.)
assert.deepEqual(stats, { nodes: 1, edges: 1 });
// Link-append: the phrasing joined the existing node's aliases and // Link-append: the phrasing joined the existing node's aliases and
// the provenance unioned; no second node line defines a new node. // the provenance unioned; p2's create line is its own node.
assert.equal(ledgers.readNodes().length, 2); // the pre-seeded line + the link line assert.equal(ledgers.readNodes().length, 3); // pre-seeded + link line + p2's create
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "p1", graphNodeId: existingId }]); const mappings = ledgers.readMappings();
assert.equal(mappings.length, 2);
assert.deepEqual(mappings[0], { chatId, sourceId: "p1", graphNodeId: existingId });
assert.equal(mappings[1]!.sourceId, "p2");
assert.match(mappings[1]!.graphNodeId, /^n_/);
// COVERAGE checkpointed per message (not a whole-chat checkpoint): // COVERAGE checkpointed per message (not a whole-chat checkpoint):
// one entry per transcript message, fingerprinted content each — // one entry per transcript message, fingerprinted content each —
@@ -513,11 +518,8 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("the LLM cluster-namer is DEAD (wave 7): a merge mints no naming runs — labels are session names now", async () => { it("the LLM cluster-namer is DEAD (wave 7): a merge mints no naming runs — labels are session names now", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
// Identity-pass verdict: p1 becomes a new node, p2 unlinked. // Empty verdicts: everything creates (verdicts-only, 2026-09-28).
const verdict: AuditResult = { const verdict: AuditVerdicts = { links: [], edges: [] };
propositions: [{ ...FINDINGS.propositions[0]! }],
edges: [],
};
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
@@ -525,7 +527,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
auditor: submitModel("submit_review", verdict), auditor: submitModel("submit_review", verdict),
}); });
const stats = await pipeline.runExtraction(chatId); const stats = await pipeline.runExtraction(chatId);
assert.deepEqual(stats, { nodes: 1, edges: 0 }); assert.deepEqual(stats, { nodes: 2, edges: 1 }); // full findings merge (verdicts-only)
// Wave 7 (950ddfd): birth-session clusters carry session-name labels, // Wave 7 (950ddfd): birth-session clusters carry session-name labels,
// so the post-merge naming garnish is gone — no model spend, no // so the post-merge naming garnish is gone — no model spend, no
// naming-run records, no label writes. // naming-run records, no label writes.
@@ -536,10 +538,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("engineFor fills the stages (model slots wave): the chain replaces config pins", async () => { it("engineFor fills the stages (model slots wave): the chain replaces config pins", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const verdict: AuditResult = { const verdict: AuditVerdicts = { links: [], edges: [] };
propositions: FINDINGS.propositions,
edges: [],
};
const consulted: string[] = []; const consulted: string[] = [];
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
@@ -566,10 +565,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("direct test values still WIN over engineFor (the priority pin)", async () => { it("direct test values still WIN over engineFor (the priority pin)", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const verdict: AuditResult = { const verdict: AuditVerdicts = { links: [], edges: [] };
propositions: FINDINGS.propositions,
edges: [],
};
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
@@ -622,8 +618,8 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
}); });
const seen: (string | undefined)[] = []; const seen: (string | undefined)[] = [];
const phaseOf = () => extractionStatuses().find((r) => r.chatId === chatId)?.phase; const phaseOf = () => extractionStatuses().find((r) => r.chatId === chatId)?.phase;
const verdict: AuditResult = { const verdict: AuditVerdicts = {
propositions: [{ ...FINDINGS.propositions[0]!, linkTo: existingId }], links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }],
edges: [], edges: [],
}; };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
@@ -762,8 +758,8 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
provenance: [], provenance: [],
}); });
const usageSeen: (number | undefined)[] = []; const usageSeen: (number | undefined)[] = [];
const verdict: AuditResult = { const verdict: AuditVerdicts = {
propositions: [{ ...FINDINGS.propositions[0]!, linkTo: existingId }], links: [{ id: FINDINGS.propositions[0]!.id, linkTo: existingId }],
edges: [], edges: [],
}; };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
@@ -789,7 +785,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
const record = chatStore.getChat(chatId)!; const record = chatStore.getChat(chatId)!;
assert.equal(record.extractions?.length, 1); assert.equal(record.extractions?.length, 1);
const run = record.extractions![0]!; const run = record.extractions![0]!;
assert.equal(run.stats?.nodes, 0); // linked, not created assert.equal(run.stats?.nodes, 1); // p2 created; p1 linked, not created
// the scripted engine's model, provider-qualified (model slots wave) // the scripted engine's model, provider-qualified (model slots wave)
assert.equal(run.stages.extractor.model, "faux:faux-1"); assert.equal(run.stages.extractor.model, "faux:faux-1");
assert.equal(run.stages.extractor.steps.length, 1); // the submit call — the run STOPS after a clean submit assert.equal(run.stages.extractor.steps.length, 1); // the submit call — the run STOPS after a clean submit
@@ -1073,7 +1069,7 @@ function passThroughAuditStep(): FauxResponseStep {
fauxToolCall( fauxToolCall(
"submit_review", "submit_review",
{ {
propositions: [{ ...FINDINGS.propositions[0]! }], links: [],
edges: [], edges: [],
} as unknown as ToolCall["arguments"], } as unknown as ToolCall["arguments"],
{ id: "review-call" }, { id: "review-call" },
@@ -1122,9 +1118,10 @@ describe("the chunked pipeline (extraction coverage)", () => {
// The document header is the session title. // The document header is the session title.
assert.ok(views[0]!.startsWith("# Chunky coverage fixture")); assert.ok(views[0]!.startsWith("# Chunky coverage fixture"));
// p1 was a fresh node in batch 1; batch 2's p1 mapped to the same // Batch 1 merged the full findings (verdicts-only: the extractor's
// node (chat-scoped mapping). // output always fully merges); batch 2's identical ids mapped to
assert.deepEqual(stats, { nodes: 1, edges: 0 }); // the same nodes (chat-scoped mapping) — nothing new.
assert.deepEqual(stats, { nodes: 2, edges: 1 });
// Per-batch coverage: every message checkpointed, nothing legacy. // Per-batch coverage: every message checkpointed, nothing legacy.
const coverage = ledgers.readCoverage(); const coverage = ledgers.readCoverage();
@@ -1181,7 +1178,7 @@ describe("the chunked pipeline (extraction coverage)", () => {
const stats = await runPromise; const stats = await runPromise;
// Batch 1's emissions are kept and its messages stay covered. // Batch 1's emissions are kept and its messages stay covered.
assert.deepEqual(stats, { nodes: 1, edges: 0 }); assert.deepEqual(stats, { nodes: 2, edges: 1 });
assert.equal(ledgers.readCoverage().length, 3, "batch 1 checkpointed its three messages"); assert.equal(ledgers.readCoverage().length, 3, "batch 1 checkpointed its three messages");
const cov = coverageOf(record.messages, ledgers.readCoverage(), chatId); const cov = coverageOf(record.messages, ledgers.readCoverage(), chatId);
assert.equal(cov.uncoveredCount, 2, "the in-flight batch's messages stay uncovered"); assert.equal(cov.uncoveredCount, 2, "the in-flight batch's messages stay uncovered");
@@ -1254,7 +1251,8 @@ describe("the chunked pipeline (extraction coverage)", () => {
}); });
await reOpener.runExtraction(chatId); await reOpener.runExtraction(chatId);
assert.equal(reRunViews.length, 1, "one batch: the single re-opened message"); assert.equal(reRunViews.length, 1, "one batch: the single re-opened message");
assert.ok(reRunViews[0]!.includes("### Learner · s1") && !reRunViews[0]!.includes("### Learner · s2")); assert.ok(reRunViews[0]!.includes("the answer merged in late") && !reRunViews[0]!.includes("### Learner · s2"));
assert.ok(reRunViews[0]!.includes("### Learner"), "the re-opened message renders");
assert.equal(coverageOf(messages(), ledgers.readCoverage(), chatId).uncoveredCount, 0, "re-covered"); assert.equal(coverageOf(messages(), ledgers.readCoverage(), chatId).uncoveredCount, 0, "re-covered");
}); });
}); });
+27 -15
View File
@@ -19,7 +19,7 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import * as fs from "node:fs"; import * as fs from "node:fs";
import { after, describe, it } from "node:test"; import { after, describe, it } from "node:test";
import type { AuditResult } from "../src/core/contracts.ts"; import type { AuditVerdicts, SessionFindings } from "../src/core/contracts.ts";
import type { QuestionAnswerData } from "../src/core/question-core.ts"; import type { QuestionAnswerData } from "../src/core/question-core.ts";
import type { LhMessage } from "../src/core/native-messages.ts"; import type { LhMessage } from "../src/core/native-messages.ts";
import { projectView as projectNative } from "../src/core/view.ts"; import { projectView as projectNative } from "../src/core/view.ts";
@@ -214,9 +214,9 @@ function fixtureMessages(): LhMessage[] {
].flat(); ].flat();
} }
/** The auditor's verdict for the fixture chat: three propositions /** The extractor's findings for the fixture chat: three propositions,
* (no linkTo — the graph starts empty), one follows edge. */ * one follows edge (the graph starts empty — no links). */
function fixtureAudit(): AuditResult { function fixtureFindings(): SessionFindings {
return { return {
propositions: [ propositions: [
{ {
@@ -248,6 +248,12 @@ function fixtureAudit(): AuditResult {
}; };
} }
/** The auditor's verdicts when nothing links: an empty verdict set is a
* valid full review (verdicts-only, owner 2026-09-28). */
function fixtureVerdicts(): AuditVerdicts {
return { links: [], edges: [] };
}
/** The chat every merger test folds (shared, so ordering is intentional). */ /** The chat every merger test folds (shared, so ordering is intentional). */
const chat1 = fixtureChat("Graph fixture", fixtureMessages()); const chat1 = fixtureChat("Graph fixture", fixtureMessages());
@@ -275,7 +281,7 @@ function scopedGraph(chatIds: string[]) {
describe("the merger (mergeFindings)", () => { describe("the merger (mergeFindings)", () => {
it("creates nodes from the verdicts with chat-level provenance", () => { it("creates nodes from the verdicts with chat-level provenance", () => {
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps, sessionSource(chat1.id)); const stats = mergeFindings(chat1.id, fixtureFindings(), fixtureVerdicts(), graphDeps, sessionSource(chat1.id));
assert.deepEqual(stats, { nodes: 3, edges: 1 }); assert.deepEqual(stats, { nodes: 3, edges: 1 });
const graph = scopedGraph([chat1.id]); const graph = scopedGraph([chat1.id]);
@@ -309,7 +315,7 @@ describe("the merger (mergeFindings)", () => {
}); });
it("is idempotent — re-merging the same verdicts appends nothing", () => { it("is idempotent — re-merging the same verdicts appends nothing", () => {
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps, sessionSource(chat1.id)); const stats = mergeFindings(chat1.id, fixtureFindings(), fixtureVerdicts(), graphDeps, sessionSource(chat1.id));
assert.deepEqual(stats, { nodes: 0, edges: 0 }); assert.deepEqual(stats, { nodes: 0, edges: 0 });
const graph = scopedGraph([chat1.id]); const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3); assert.equal(graph.nodes.length, 3);
@@ -342,11 +348,13 @@ describe("the merger (mergeFindings)", () => {
name: "Alpha", name: "Alpha",
statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings
aliases: ["alpha intuition"], aliases: ["alpha intuition"],
linkTo: alphaId,
}, },
], ],
edges: [], edges: [],
}, },
// VERDICTS-ONLY: the identity call is a link entry, not a field
// on the proposition (owner, 2026-09-28).
{ links: [{ id: "q1", linkTo: alphaId }], edges: [] },
graphDeps, graphDeps,
sessionSource(chat2.id), sessionSource(chat2.id),
); );
@@ -384,6 +392,7 @@ describe("the merger (mergeFindings)", () => {
], ],
edges: [], edges: [],
}, },
{ links: [], edges: [] },
graphDeps, graphDeps,
{ foldedAt: "2026-09-13T00:00:00.000Z" }, { foldedAt: "2026-09-13T00:00:00.000Z" },
); );
@@ -408,7 +417,7 @@ describe("the merger (mergeFindings)", () => {
id: `p${i + 1}`, id: `p${i + 1}`,
name: `Cluster node ${i + 1}`, name: `Cluster node ${i + 1}`,
statement, statement,
aliases: [], // birth provenance — the message anchors aliases: [],
})), })),
edges: [ edges: [
{ from: "p1", to: "p2", type: "follows", }, { from: "p1", to: "p2", type: "follows", },
@@ -416,6 +425,7 @@ describe("the merger (mergeFindings)", () => {
{ from: "p1", to: "p3", type: "follows", }, { from: "p1", to: "p3", type: "follows", },
], ],
}, },
{ links: [], edges: [] },
graphDeps, graphDeps,
{ foldedAt: "2026-09-13T00:00:00.000Z" }, { foldedAt: "2026-09-13T00:00:00.000Z" },
); );
@@ -462,11 +472,12 @@ describe("the merger (mergeFindings)", () => {
name: "Alpha", name: "Alpha",
statement: "Alpha intuition", // IDENTICAL to the existing node's statement statement: "Alpha intuition", // IDENTICAL to the existing node's statement
aliases: [], aliases: [],
// no linkTo — identity is the auditor's call alone
}, },
], ],
edges: [], edges: [],
}, },
// empty verdicts — identity is the auditor's call alone
{ links: [], edges: [] },
graphDeps, graphDeps,
sessionSource(chat3.id), sessionSource(chat3.id),
); );
@@ -485,11 +496,12 @@ describe("the merger (mergeFindings)", () => {
id: "p1", id: "p1",
name: "Vim normal mode", name: "Vim normal mode",
statement: "Vim starts in Normal mode, where keys are commands.", statement: "Vim starts in Normal mode, where keys are commands.",
aliases: [], // notes carry no message anchors — the file is the provenance aliases: [],
}, },
], ],
edges: [], edges: [],
}, },
{ links: [], edges: [] },
graphDeps, graphDeps,
{ foldedAt: "2026-09-13T00:00:00.000Z" }, { foldedAt: "2026-09-13T00:00:00.000Z" },
); );
@@ -586,14 +598,14 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
), ),
]), ]),
].flat()); ].flat());
const auditA: AuditResult = { const findingsA: SessionFindings = {
propositions: [ propositions: [
{ id: "p1", name: "Shared core", statement: "Shared core statement", aliases: ["shared feel"], }, { id: "p1", name: "Shared core", statement: "Shared core statement", aliases: ["shared feel"], },
{ id: "p2", name: "Solo fact", statement: "Solo fact statement", aliases: ["solo feel"], }, { id: "p2", name: "Solo fact", statement: "Solo fact statement", aliases: ["solo feel"], },
], ],
edges: [{ from: "p1", to: "p2", type: "follows", }], edges: [{ from: "p1", to: "p2", type: "follows", }],
}; };
mergeFindings(dA.id, auditA, graphDeps, sessionSource(dA.id)); mergeFindings(dA.id, findingsA, { links: [], edges: [] }, graphDeps, sessionSource(dA.id));
// Resolve through THIS session's mappings — the shared ledger holds // Resolve through THIS session's mappings — the shared ledger holds
// lookalike statements from earlier tests, so statement-lookup would // lookalike statements from earlier tests, so statement-lookup would
// link dB to the wrong node. // link dB to the wrong node.
@@ -612,14 +624,14 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
), ),
]), ]),
].flat()); ].flat());
const auditB: AuditResult = { const findingsB: SessionFindings = {
propositions: [ propositions: [
{ id: "q1", name: "Shared again", statement: "Shared core, rephrased", aliases: ["B phrasing"], linkTo: xId }, { id: "q1", name: "Shared again", statement: "Shared core, rephrased", aliases: ["B phrasing"] },
{ id: "q2", name: "B extra", statement: "B extra statement", aliases: ["b extra feel"], }, { id: "q2", name: "B extra", statement: "B extra statement", aliases: ["b extra feel"], },
], ],
edges: [{ from: "q1", to: "q2", type: "follows", }], edges: [{ from: "q1", to: "q2", type: "follows", }],
}; };
mergeFindings(dB.id, auditB, graphDeps, sessionSource(dB.id)); mergeFindings(dB.id, findingsB, { links: [{ id: "q1", linkTo: xId }], edges: [] }, graphDeps, sessionSource(dB.id));
const zId = graphStore.readMappings().find((m) => m.chatId === dB.id && m.sourceId === "q2")!.graphNodeId; const zId = graphStore.readMappings().find((m) => m.chatId === dB.id && m.sourceId === "q2")!.graphNodeId;
// Coverage rows (both chats) + positions garnish — the deletion // Coverage rows (both chats) + positions garnish — the deletion