Graph redesign: true propositions, follows/supports, auditor-owned identity (vision §1 minimal implementation)

The minimal implementation of the 2026-09-02 graph vision (docs/vision.md §1):

- contracts/graph.ts: every node is a TRUE proposition (statement is both
  rendering label and identity key; required description; no kind, no name).
  Edges are follows (derivable) / supports (grounds without entailing) with
  message-granular provenance points {chatId, messageId}. Events reduce to
  correct/miss/dont-know plus observation (understanding notes); no taught
  events, no misconception kind.
- extraction.ts: extractor is TREE-BLIND (transcript only, no plan tree) and
  free-form; auditor sees findings + the current graph, owns faithfulness and
  identity (link-or-create verdict per proposition, recorded as linkTo);
  transcriptView tags every line with its message id (provenance anchors).
- core/graph.ts merger: applies verdicts deterministically — linkTo appends
  a link line (phrasings join aliases, provenance unions; reads merge by id),
  absent creates; NO string matching anywhere. Quiz events rebuilt from tool
  parts with attribution-only nodeId resolution; understanding notes become
  observation events (obs:<sourceId>); mapping lines are the verdict audit
  trail and make re-merges idempotent.
- deriveNodeStates: transitional — latest quiz event wins (known/missed/
  dont-know, else candidate); taught unreachable until the state-model
  ideation lands.
- graph-context.ts + 50-graph-context.md: follows chains, hotspot grouping,
  state vocabulary updated for the new event kinds.
- graph-view.tsx: node label is the statement; detail shows statement +
  description.
- tests: merger verdicts (link, no-string-match, cross-chat assertions),
  tree-blind pipeline end-to-end, follows vocabulary in context tests.
  276/276 green.
This commit is contained in:
amosblomqvist
2026-09-08 22:19:37 +03:00
parent e322d775e4
commit b07c907ffd
13 changed files with 945 additions and 813 deletions
+2 -2
View File
@@ -114,7 +114,7 @@ graph TB
LEDG --> SNAP --> RGRAPH LEDG --> SNAP --> RGRAPH
``` ```
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer. The LLM stages propose and verify (propositions, edges, probe attribution — each with verbatim evidence); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs) from the transcript. The teacher stays a teacher; everything below the ledgers is derived. **Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer — tree-blind and post-hoc. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs, provenance) from the transcript. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger. The teacher stays a teacher; everything below the ledgers is derived.
## Layout ## Layout
@@ -165,7 +165,7 @@ data/ chats/ · graph/ · jobs/ — runtime state, git
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts). **The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts).
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM: propositions, edges, probe attribution, each with verbatim evidence) → auditor (LLM: drops anything unbacked) → merger (`mergeFindings`, deterministic: exact/alias identity resolution, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes. **The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM, tree-blind: propositions as true statements + descriptions, follows/supports edges, quiz attribution, understanding notes) → auditor (LLM: drops anything unbacked, stamps each survivor with an identity verdict — link to an existing node or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
## Testing ## Testing
+3 -3
View File
@@ -363,7 +363,7 @@ export function GraphView() {
// frozen layout stable for a given node set across remounts. // frozen layout stable for a given node set across remounts.
const rand = seededRandom(node.id); const rand = seededRandom(node.id);
graph.addNode(node.id, { graph.addNode(node.id, {
label: node.name, label: node.statement,
size: STATE_TOKENS[state].size, size: STATE_TOKENS[state].size,
color: stateColors.get(state) ?? edgeColor, color: stateColors.get(state) ?? edgeColor,
x: (rand() - 0.5) * 100, x: (rand() - 0.5) * 100,
@@ -552,8 +552,8 @@ export function GraphView() {
style={{ visibility: "hidden", left: 0, top: 0 }} style={{ visibility: "hidden", left: 0, top: 0 }}
className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg" className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg"
> >
<div className="text-sm font-medium leading-snug">{selected.node.name}</div> <div className="text-sm font-medium leading-snug">{selected.node.statement}</div>
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.statement}</div> <div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.description}</div>
<div className="mt-1 text-xs uppercase tracking-wide text-muted-foreground"> <div className="mt-1 text-xs uppercase tracking-wide text-muted-foreground">
{STATE_LABELS[selected.state]} {STATE_LABELS[selected.state]}
</div> </div>
+3 -3
View File
@@ -75,11 +75,11 @@ chat record ──▶ EXTRACTOR (LLM) ──▶ AUDITOR (LLM) ──▶ MERGER (
proposes verifies applies proposes verifies applies
``` ```
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned* (propositions, incl. misconceptions and prose-only `supports` edges), *what depends on what*, and *which proposition an untagged probe tested*. Everything the model could silently mis-copy — event kinds, timestamps, idempotency refs, notes, question texts — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (the faithfulness net) drops anything unbacked and logs every drop. Conservative throughout: duplicates over wrong merges, drop over invent. **Tree-blind and post-hoc** (2026-09-02 redesign, docs/vision.md §1): the extractor reads ONLY the transcript — never the session's plan tree — and proposes free-form propositions (true statements with descriptions, misconception-free: a misconception is stated as its true correction), `follows`/`supports` edges, per-proposition understanding notes, and message-granular provenance. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger.
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — statement/alias exact-match identity resolution (fuzzy/embedding adjudication deferred), one mapping line per (chat, graph node) as the merge audit trail, edge dedupe by `(from, to, type)` with per-chat provenance, event dedupe by chat-scoped ref, and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs. **Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned*, *what follows/supports what*, *which proposition each quiz tested* (retroactively, against the whole transcript), and — the auditor's — *identity*: whether a proposition IS an existing graph node (link-or-create). Everything the model could silently mis-copy — event kinds (`correct` / `miss` / `dont-know`), timestamps, idempotency refs, notes, provenance points — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (which sees the extractor's output PLUS the current graph) drops anything unbacked, logs every drop, and stamps each surviving proposition with its identity verdict (`linkTo`). Conservative throughout: duplicates over wrong merges, drop over invent.
**Probe attribution** (vision open question, resolved structurally): tagged events (quiz `nodeId`, `plan_mark_done`'s tree node) resolve deterministically through the proposition mapping. The extractor's judgment is consulted only for **untagged probe quizzes** — retroactively, against the whole transcript, so a pre-tree probe maps to a proposition the tree only articulates later. Attribution can also rescue a tagged-but-unknown id; genuinely unattributable probes land `nodeId: null` with the question text, exactly as they always have. **Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — identity comes from the auditor's `linkTo` verdict alone (validated against the ledger; never string matching), one mapping line per (chat, sourceId) as the verdict audit trail (which also makes re-merging the same chat idempotent), link-appends: a linked proposition appends a node line whose phrasings join the node's `aliases` and whose provenance unions (reads merge by id), edge dedupe by `(from, to, type)`-already-asserted-by-this-chat with message-granular provenance, event dedupe by chat-scoped ref (quiz toolCallId, `obs:<sourceId>` for understanding observations), and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
**The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes. **The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes.
+4 -5
View File
@@ -1,20 +1,19 @@
# The learner's knowledge graph # The learner's knowledge graph
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes and plan trees, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Requires** means *a* must be understood before *b*. After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
The states, exactly: The states, exactly:
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise. - **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
- **[dont-know]** — the learner declined to guess. Unexplored, not broken. - **[dont-know]** — the learner declined to guess. Unexplored, not broken.
- **[known]** — answered correctly. Foundation you can build on and refer to by name. - **[known]** — answered correctly. Foundation you can build on and refer to by name.
- **[taught]** — a lesson covered it. Check that it stuck before stacking anything heavy on it. - **[candidate]** — present in the graph but never probed. Assume nothing.
- **[candidate]** — asserted into a plan but never probed. Assume nothing.
This block is what makes probing precise instead of generic — use it: This block is what makes probing precise instead of generic — use it:
- **Probe the frontier, not the interior.** [known] and [taught] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes. - **Probe the frontier, not the interior.** [known] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss. - **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
- **Read the Requires chains as scaffolding.** They show what the learner's knowledge rests on. If a prerequisite under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself. - **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes. When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
+152 -114
View File
@@ -1,23 +1,28 @@
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai"; import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
import { z } from "zod"; import { z } from "zod";
import { modelFor } from "../models.ts"; import { modelFor } from "../models.ts";
import type { AuditResult, ChatRecord, SessionFindings } from "../../core/contracts.ts"; import type { AuditResult, ChatRecord, GraphNode, SessionFindings } from "../../core/contracts.ts";
import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts"; import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
/** /**
* The extraction pipeline (vision §1 construction) — the sole graph writer. * The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
* Three stages, one orchestrator: * sole graph writer. Three stages, one orchestrator:
* *
* extractor (LLM) transcript → SessionFindings (propositions, edges, * extractor (LLM) transcript → SessionFindings (free-form propositions,
* event attribution — each with verbatim evidence) * follows/supports edges, quiz attribution, understanding
* auditor (LLM) findings → AuditResult (the verified subset + drops) * notes — each with verbatim evidence). TREE-BLIND: it
* sees only the transcript, never the plan tree.
* auditor (LLM) findings + THE CURRENT GRAPH → AuditResult: the
* verified subset with an identity verdict per
* proposition (linkTo an existing node, or create), plus
* the drops for the log.
* merger (core) mergeFindings() → deterministic ledger appends * merger (core) mergeFindings() → deterministic ledger appends
* *
* The LLM stages supply only the judgment calls; the deterministic rebuild * The LLM stages supply only the judgment calls; the deterministic rebuild
* in the merger owns everything that must never be mis-copied (event kinds, * in the merger owns everything that must never be mis-copied (event kinds,
* timestamps, refs). Both agents are single-tool-call workers: their whole * timestamps, refs, provenance points). Both agents are single-tool-call
* deliverable is one zod-validated submit call — a ToolLoopAgent purely so * workers: their whole deliverable is one zod-validated submit call — a
* the model can retry a schema-invalid call. * ToolLoopAgent purely so the model can retry a schema-invalid call.
* *
* Created via createExtractionPipeline(deps) — the composition root * Created via createExtractionPipeline(deps) — the composition root
* (chat-handler) injects the storage ports + models; this module never * (chat-handler) injects the storage ports + models; this module never
@@ -27,7 +32,7 @@ import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/gr
* the next trigger re-runs the whole thing. * the next trigger re-runs the whole thing.
*/ */
// ─── the transcript view (what the LLM sees) ──────────────────────────────── // ─── the transcript view (what the extractor + auditor see) ─────────────────
/** /**
* A quiz tool part's outcome, one line: the attribution evidence. * A quiz tool part's outcome, one line: the attribution evidence.
@@ -44,46 +49,35 @@ function quizLine(part: { toolCallId: string; input: unknown; output: unknown })
} }
/** /**
* The compact transcript the extractor + auditor both see: user/teacher * The compact transcript both LLM stages see: user/teacher text VERBATIM,
* text verbatim (evidence quotes must match it), the final plan tree, and * each tagged with its message id (provenance anchors — the extractor
* the structured tool events (quiz outcomes, mark_done). Job tool parts * reports them back per proposition/edge), and quiz outcome lines.
* (research/demo/visual) are noise for extraction — their outcomes are * TREE-BLIND by design (2026-09-02): no plan tree, no mark_done —
* summarized as one line each. * extraction is free-form, decoupled from the session's plan. Job tool
* parts (research/demo/visual) are noise for extraction — their outcomes
* are summarized as one line each.
*/ */
export function transcriptView(record: ChatRecord): string { export function transcriptView(record: ChatRecord): string {
const lines: string[] = []; const lines: string[] = [];
lines.push(`SESSION: ${record.title}`); lines.push(`SESSION: ${record.title}`);
if (record.tree && record.tree.nodes.length > 0) { record.messages.forEach((message, index) => {
const nodes = record.tree.nodes.map((n) => `${n.id} = ${n.title}`).join("; ");
lines.push(`FINAL PLAN TREE (node ids): ${nodes}`);
if (record.tree.edges.length > 0) {
const edges = record.tree.edges.map((e) => `${e.from} requires ${e.to}`).join("; ");
lines.push(`TREE EDGES: ${edges}`);
}
}
for (const message of record.messages as UIMessage[]) {
const who = message.role === "user" ? "LEARNER" : "TEACHER"; const who = message.role === "user" ? "LEARNER" : "TEACHER";
for (const part of message.parts) { const mid = (message as UIMessage).id || `m${index}`;
for (const part of (message as UIMessage).parts) {
if (part.type === "text" && part.text.trim()) { if (part.type === "text" && part.text.trim()) {
lines.push(`${who}: ${part.text.trim()}`); lines.push(`${who} [${mid}] ${part.text.trim()}`);
} else if (part.type.startsWith("tool-")) { } else if (part.type.startsWith("tool-")) {
const name = part.type.slice("tool-".length); const name = part.type.slice("tool-".length);
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown }; const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
if (name === "quiz" && p.state === "output-available") { if (name === "quiz" && p.state === "output-available") {
lines.push(quizLine(p)); lines.push(quizLine(p));
} else if (name === "plan_mark_done" && p.state === "output-available") {
const input = p.input as { id?: unknown } | undefined;
lines.push(`TEACHER marked tree node done [${p.toolCallId}]: ${typeof input?.id === "string" ? input.id : "?"}`);
} else if (name.startsWith("plan_")) {
// plan ops are folded into the FINAL PLAN TREE line above
} else if (p.state === "output-available") { } else if (p.state === "output-available") {
lines.push(`(${name} tool result delivered)`); lines.push(`(${name} tool result delivered)`);
} }
} }
} }
} });
return lines.join("\n"); return lines.join("\n");
} }
@@ -94,87 +88,98 @@ const evidence = z
.min(1) .min(1)
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote."); .describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
const findingsSchema = z.object({ const messagesField = z
propositions: z .array(z.string())
.array( .describe("The message ids (from the [id] tags in the transcript) where this came up — provenance.");
z.object({
const propositionSchema = z.object({
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."), id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
name: z.string().describe(
"A short 2-6 word name for the fact — its header, used for rendering. Not a sentence.",
),
statement: z statement: z
.string() .string()
.describe( .describe(
"The proposition as one reasoning step — an OBJECTIVE, world-facing fact. " + "The proposition as one reasoning step — a TRUE, self-contained, world-facing statement. " +
"Never reference the learner, the teacher, or the session: not \"the learner " + "Never reference the learner, the teacher, or the session: not \"the learner understands X\" but " +
'understands X\" but \"X\"; not \"the learner thinks Y\" but the wrong idea itself, ' + '"X". A misconception is stated as the TRUE correction ("Nietzsche did not endorse ' +
'stated as a claim (\"Reading: Nietzsche endorsed might-makes-right social Darwinism\"), ' + 'might-makes-right social Darwinism") — the wrongness lives in the quiz outcomes, never here.',
"with kind \"misconception\" carrying that it is wrong.",
), ),
aliases: z.array(z.string()).describe("Verbatim alternate phrasings from this session, if any."), description: z
kind: z.enum(["truth", "misconception"]).describe("misconception = a wrong idea the learner holds or was warned about."), .string()
treeNodeId: z.string().optional().describe("The plan-tree node id this came from, when it did."), .describe(
"The elaborated version of the same proposition — more words: scope, precision, what exactly it means. Still true, still world-facing.",
),
aliases: z.array(z.string()).describe("Verbatim phrasings this session used for the same proposition, if any."),
messages: messagesField,
evidence, evidence,
}), understanding: z
) .string()
.describe("Every distinct proposition the session touched — from the plan tree AND from prose."), .optional()
.describe(
'One note about THIS session\'s evidence of the learner\'s grasp of this proposition, on a named-word scale ' +
'("solid", "shaky", "accepted", "resisted", "not yet engaged") — no numbers, no predictions.',
),
});
const edgeSchema = z.object({
from: z.string().describe("Proposition id that is followed / supported."),
to: z.string().describe("Proposition id that follows / is supported by `from`."),
type: z
.enum(["follows", "supports"])
.describe('"follows" = to is derivable from from; "supports" = from grounds to without entailing it.'),
messages: messagesField,
evidence,
});
const findingsSchema = z.object({
propositions: z
.array(propositionSchema)
.describe("Every distinct proposition the session touched — true statements, misconception-free."),
edges: z edges: z
.array( .array(edgeSchema)
z.object({ .describe("Relations between propositions: follows (derivable) / supports (grounds without entailing)."),
from: z.string().describe("Proposition id that is the prerequisite / support."),
to: z.string().describe("Proposition id that depends on / is supported by `from`."),
type: z.enum(["requires", "supports"]),
evidence,
}),
)
.describe("Relations between propositions. `requires` = prerequisite; `supports` = helpful but not prerequisite. Include relations the plan tree does NOT show when the transcript articulates them."),
attribution: z attribution: z
.object({ .object({
quiz: z quiz: z
.record(z.string(), z.string().nullable()) .record(z.string(), z.string().nullable())
.describe("toolCallId of every UNTAGGED quiz line → the proposition id it was really testing. Probe questions asked BEFORE the tree existed are attributed RETROACTIVELY — the whole transcript is visible; map them to the proposition they were probing (often articulated only later). null when genuinely unattributable. Tagged quizzes (with a tree nodeId in the transcript view) resolve automatically — do not include them."), .describe(
"toolCallId of EVERY quiz line → the proposition id it was really testing. Attribute RETROACTIVELY — the whole transcript is visible; map each quiz to the proposition it tests (often articulated elsewhere in the session). null when genuinely unattributable.",
),
}) })
.describe("Probe attribution — your only event judgment."), .describe("Quiz attribution — your only event judgment."),
}); });
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
transcript and report what was learned — as structured findings for the learner's global knowledge graph. transcript and report what was learned — as structured findings for the learner's global knowledge graph.
The transcript view shows: the session's final plan tree (dependency map, with node ids), the conversation The transcript view shows: the conversation verbatim (LEARNER / TEACHER lines, each tagged with its message id)
verbatim (LEARNER / TEACHER), and structured events (QUIZ lines with the learner's outcome, mark_done lines). and QUIZ outcome lines.
Report, with verbatim evidence quotes for everything: Report, with verbatim evidence quotes for everything:
NODE THEORY — the graph is a set of objective facts, full stop. A node is what is true (or falsely NODE THEORY — the graph is a set of TRUE propositions about the world, full stop. A node is never anything
believed) about the world, never anything about the learner or the session. The learner's about the learner or the session. The learner's relationship to a proposition — whether they know it, missed
relationship to a fact — whether they know it, missed it, or hold it as a misconception — is it, resisted it — is expressed ENTIRELY by the quiz outcomes and your understanding note; it never leaks into
expressed ENTIRELY by the quiz-outcome events you attribute; it never leaks into the node itself. the node itself. So no "the learner understands X", no "discussed in this session": the node is just X. A
So no "the learner understands X", no "discussed in this session": the node is just X. A misconception is NOT a node of its own wrong claim — state the TRUE correction as the proposition, and let the
misconception node states the wrong idea itself as a claim, with kind "misconception" marking it quiz outcomes carry that the learner believed the wrong version.
wrong — never "the learner believes Y".
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched: 1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
- Each carries a "name": a short 2-6 word header (the rendering handle), and a "statement": the - Each is a self-contained TRUE statement ("statement"), plus its elaboration ("description" — more
full fact as one objective reasoning step. words, same proposition: scope, precision, what exactly it means).
- Plan-tree nodes become propositions (pass their treeNodeId) — statement from the node title, enriched to a - Include corrections of misconceptions the teacher articulated, and true facts explained in prose.
complete proposition if the prose elaborates it.
- Propositions that appeared only in prose (teacher explanations, learner questions) get fresh ids.
- Include MISCONCEPTIONS — wrong ideas the learner showed or the teacher warned against — with kind
"misconception".
- One reasoning step per proposition: never a topic blob, never a sub-step. - One reasoning step per proposition: never a topic blob, never a sub-step.
- aliases: verbatim alternate phrasings this session used for the same thing. - "messages": the [id] tags of every message where the proposition came up — provenance, report it.
- "aliases": verbatim phrasings this session used for the same proposition.
- "understanding": ONE named-word-scale note on the evidence of the learner's grasp, for this session
only ("solid", "shaky", "accepted", "resisted", "not yet engaged"). No numbers, no predictions.
2. EDGES between propositions: 2. EDGES between propositions:
- The tree's dependency edges are the backbone: from = prerequisite, to = dependent, type "requires". - "follows": to is DERIVABLE from from (understanding from makes understanding to achievable).
- Also include relations articulated only in prose. Use "supports" when the transcript presents something as - "supports": from GROUNDS to without entailing it (helpful context, not a prerequisite).
helpful but NOT a prerequisite. - Only relations the transcript actually articulates — with evidence.
3. ATTRIBUTION — UNTAGGED quizzes only: 3. ATTRIBUTION — every QUIZ line gets a mapping: toolCallId → the proposition id it was really testing.
- QUIZ lines whose question carries a tree nodeId resolve automatically; do not attribute those. Attribute RETROACTIVELY — a quiz often tests a proposition the session only articulates later; the
- The others are PROBE-phase questions (asked before the tree was initialized) that test prerequisite question text is your guide. If you truly cannot tell, use null.
understanding. Attribute them RETROACTIVELY to the proposition they were really probing — even if that
proposition is only articulated later in the session. The QUIZ question text is your guide. If you truly
cannot tell, use null.
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides. transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
@@ -205,6 +210,16 @@ function createExtractor(model: LanguageModel) {
// ─── the auditor ─────────────────────────────────────────────────────────── // ─── the auditor ───────────────────────────────────────────────────────────
const auditSchema = findingsSchema.extend({ const auditSchema = findingsSchema.extend({
propositions: z.array(
propositionSchema.extend({
linkTo: z
.string()
.optional()
.describe(
"The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. When unsure, omit: duplicates over wrong merges.",
),
}),
),
dropped: z dropped: z
.array( .array(
z.object({ z.object({
@@ -216,27 +231,46 @@ const auditSchema = findingsSchema.extend({
}); });
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
ONE session transcript and a set of proposed findings (propositions, edges, event attribution). Your job is the ONE session transcript, a set of proposed findings (propositions, edges, quiz attribution), and the CURRENT
faithfulness net: verify every candidate against the transcript, keep only what is backed, and return the knowledge graph. Your jobs: the faithfulness net, and identity.
verified subset.
Verify: FAITHFULNESS — verify every candidate against the transcript:
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement must - PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement
be a faithful reading of it. The node theory applies: a statement must be an OBJECTIVE, world-facing fact — must be a faithful reading of it. The node theory applies: a statement must be a TRUE, self-contained,
reword learner- or session-relative statements ("the learner understands X" → "X") or drop them when the world-facing fact — reword learner- or session-relative statements ("the learner understands X" → "X"), state
objective fact is not actually backed. Drop invented or bloated statements. Conservative: when two proposed propositions misconceptions as the true correction, or drop them when no objective fact is backed. Drop invented or
are really one, keep the better-stated one and drop the other. bloated statements. Conservative: when two proposed propositions are really one, keep the better-stated one
- EDGES: the evidence must actually articulate the relation, in the direction claimed (from = prerequisite). and drop the other.
Drop direction-swapped or speculative edges. - EDGES: the evidence must actually articulate the relation, in the direction claimed. Drop
- ATTRIBUTION: only UNTAGGED quizzes carry attribution — the QUIZ line's question must genuinely test the direction-swapped or speculative edges.
attributed proposition. Drop wrong attributions to null, or re-point them to the proposition the question - ATTRIBUTION: the QUIZ line's question must genuinely test the attributed proposition. Drop wrong
actually tests. attributions to null, or re-point them to the proposition the question actually tests.
- UNDERSTANDING notes: verify against the quiz outcomes and the session's evidence; reword to a named-word
scale or drop.
IDENTITY — link-or-create, per proposition:
- You see the current graph (every node: id, statement, description, aliases). For each verified proposition,
decide: does an EXISTING node already state this same proposition?
- If yes: set "linkTo" to that node's id. The session's phrasings will join its aliases and its provenance.
- If no: omit "linkTo" and a new node is created.
- When unsure, omit — duplicates are recoverable later; wrong merges are not.
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
well-evidenced proposition is still knowledge. well-evidenced proposition is still knowledge.
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
(the same shapes you received) plus the dropped list.`; (the same shapes you received, with linkTo added) plus the dropped list.`;
/** The current graph, compact — the auditor's identity context. */
function graphView(nodes: GraphNode[]): string {
if (nodes.length === 0) return "(the graph is currently empty — every proposition is a new node)";
const lines = nodes.map(
(n) =>
`- ${n.id} :: ${n.statement}\n description: ${n.description}` +
(n.aliases.length > 0 ? `\n aliases: ${n.aliases.join(" | ")}` : ""),
);
return lines.join("\n");
}
function createAuditor(model: LanguageModel) { function createAuditor(model: LanguageModel) {
return new ToolLoopAgent({ return new ToolLoopAgent({
@@ -244,7 +278,7 @@ function createAuditor(model: LanguageModel) {
instructions: AUDITOR_INSTRUCTIONS, instructions: AUDITOR_INSTRUCTIONS,
tools: { tools: {
submit_review: tool({ submit_review: tool({
description: "Submit the verified findings subset + the dropped list.", description: "Submit the verified findings subset (with identity verdicts) + the dropped list.",
inputSchema: auditSchema, inputSchema: auditSchema,
execute: async (review) => review, execute: async (review) => review,
}), }),
@@ -295,17 +329,20 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
throw new Error("the extractor never submitted findings"); throw new Error("the extractor never submitted findings");
} }
async function audit(chatId: string, findings: SessionFindings): Promise<SessionFindings> { async function audit(chatId: string, findings: SessionFindings): Promise<AuditResult> {
const record = chatStore.getChat(chatId); const record = chatStore.getChat(chatId);
if (!record) throw new Error(`Chat not found: ${chatId}`); if (!record) throw new Error(`Chat not found: ${chatId}`);
const view = transcriptView(record); const view = transcriptView(record);
const graph = getGraph(graphStore);
console.info( console.info(
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` + `[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
`${Object.keys(findings.attribution.quiz).length} attributions)`, `${Object.keys(findings.attribution.quiz).length} attributions, ${graph.nodes.length} graph nodes)`,
); );
const review = await auditorAgent.generate({ const review = await auditorAgent.generate({
prompt: `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}`, prompt:
`Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}\n\n` +
`Current knowledge graph (${graph.nodes.length} nodes):\n\n${graphView(graph.nodes)}`,
}); });
let auditResult: AuditResult | null = null; let auditResult: AuditResult | null = null;
for (const part of review.content) { for (const part of review.content) {
@@ -320,12 +357,12 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "), auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
); );
} }
const { dropped: _dropped, ...verified } = auditResult;
console.info( console.info(
`[extraction ${chatId}] audit kept ${verified.propositions.length}/${findings.propositions.length} propositions, ` + `[extraction ${chatId}] audit kept ${auditResult.propositions.length}/${findings.propositions.length} propositions, ` +
`${verified.edges.length}/${findings.edges.length} edges`, `${auditResult.edges.length}/${findings.edges.length} edges, ` +
`${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes`,
); );
return verified; return auditResult;
} }
async function runExtraction(chatId: string): Promise<EmissionStats> { async function runExtraction(chatId: string): Promise<EmissionStats> {
@@ -334,10 +371,10 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
try { try {
console.info(`[extraction ${chatId}] pipeline start`); console.info(`[extraction ${chatId}] pipeline start`);
const findings = await extract(chatId); const findings = await extract(chatId);
const verified = await audit(chatId, findings); const auditResult = await audit(chatId, findings);
// The merger re-reads the chat record itself — events are rebuilt // The merger re-reads the chat record itself — events are rebuilt
// from the transcript, never from stale copies. // from the transcript, never from stale copies.
const stats = mergeFindings(chatId, verified, { chatStore, graphStore }); const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore });
console.info(`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events`); console.info(`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events`);
return stats; return stats;
} finally { } finally {
@@ -347,3 +384,4 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
return { runExtraction }; return { runExtraction };
} }
+129 -78
View File
@@ -1,50 +1,69 @@
/** /**
* Knowledge-graph contracts — nodes, ledgers, extraction findings * Knowledge-graph contracts — nodes, ledgers, extraction findings
* *
* The minimal implementation of the graph vision (docs/vision.md §1, the
* 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
* (derivable) and supports (grounds without entailing); provenance is
* message-granular; extraction is free-form and post-hoc, fully decoupled
* from the session's plan tree; the event ledger records quiz outcomes plus
* the extractor's per-proposition understanding observations. Anything not
* yet decided (the state model, strength derivation, understanding
* treatment, selection strategies) is deliberately NOT here — transitional
* stubs carry the renderer until those land.
*
* Part of the domain-split contracts (src/core/contracts/): each domain file * Part of the domain-split contracts (src/core/contracts/): each domain file
* is owned outright by its stream — cross-domain schema changes go through * is owned outright by its stream — cross-domain schema changes go through
* master. Pure types + protocol docs + tiny pure helpers; type-only imports * master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md.
* of "ai" (safe everywhere: server, client, tests). See docs/vision.md.
*/ */
// ─── Knowledge graph (vision §1 — global, cross-session) ──────────────────── // ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
export type EdgeType = "requires" | "supports"; /** Two relations only (DECIDED 2026-09-02): follows = b is derivable from a;
* supports = a grounds b without entailing it. No scalars, no encoded
* strength — strength is derived at read time from topology. */
export type EdgeType = "follows" | "supports";
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
* edges record every (chat, message) where they came up, deep-linkable to
* the exact transcript point. */
export interface ProvenancePoint {
chatId: string;
messageId: string;
}
export interface GraphNode { export interface GraphNode {
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */ /** Opaque stable id, store-assigned: "n_" + nanoid-like. */
id: string; id: string;
/** Short name / header for the fact (2-6 words) — the rendering handle. /** The minimal proposition — one true, self-contained statement. This is
* NODE THEORY: a node is an OBJECTIVE fact, never learner- or * BOTH the rendering label AND the identity key: the auditor recognizes
* session-relative. No "the learner understands…", no "this session * a re-encounter by judgment (link-or-create), not by string matching. */
* covered…" — just the fact, world-facing. The learner's relationship
* to the fact lives ENTIRELY in the event ledger: state labels are
* derived from quiz outcomes, never written into the node. */
name: string;
/** Canonical proposition statement — the full fact, one reasoning step,
* also objective (no learner / teacher / session references). */
statement: string; statement: string;
/** Verbatim alternate statements from sessions (merge target audit trail). */ /** The elaborated version — required. Same proposition, more words:
* scope, precision, the "what exactly does it mean". */
description: string;
/** Verbatim phrasings sessions used for this proposition (the identity
* audit trail — how the world said what the node says). */
aliases: string[]; aliases: string[];
kind: "truth" | "misconception";
createdAt: string; createdAt: string;
/** Every (chat, message) point where this proposition came up. */
provenance: ProvenancePoint[];
} }
export interface GraphEdge { export interface GraphEdge {
/** Prerequisite GraphNode id. */
from: string; from: string;
to: string; to: string;
type: EdgeType; type: EdgeType;
/** Chat ids that asserted this edge (teacher may simplify — auditable). */ /** Every (chat, message) point where this relation was asserted. */
provenance: string[]; provenance: ProvenancePoint[];
} }
/** mappings.jsonl line — the session→graph identity map: the extraction /** mappings.jsonl line — the session→graph identity map: the extraction
* pipeline's proposition (identified by its source id) → the graph node it * pipeline's proposition (identified by its run-local source id) → the
* merged into. The merge audit trail: one line per (chat, graph node). */ * graph node it resolved to (created or linked). The audit trail of the
* auditor's identity verdicts; also the chat-scoped resolution table the
* merger uses to point events and edges at node ids. */
export interface NodeMapping { export interface NodeMapping {
chatId: string; chatId: string;
/** The writer's local id for the proposition — a plan-tree node id when /** The extractor's run-local proposition id ("p1", …). */
* tree-derived, else the extractor's proposition id ("p1", …). */
sourceId: string; sourceId: string;
graphNodeId: string; graphNodeId: string;
} }
@@ -57,23 +76,26 @@ export interface SessionCheckpoint {
foldedAt: string; foldedAt: string;
} }
export type LedgerEventKind = /**
| "probe-correct" * Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
| "probe-miss" * dont-know — plus `observation`, the extractor's per-proposition
| "dont-know" * understanding note joining the ledger as a session-scoped observation
| "misconception-signaled" * (named-word scale; treatment pending the state-model ideation). No
| "taught" * `taught` events at all; no misconception kind — a misconception is a
| "check-correct" * TRUE correction in the graph, and the wrongness lives here, in the
| "check-incorrect"; * ledger, until the state model exists.
*/
export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
export interface LedgerEvent { export interface LedgerEvent {
/** null = not yet attributed (untagged probe events; consolidation assigns). */ /** null = not yet attributed. */
nodeId: string | null; nodeId: string | null;
kind: LedgerEventKind; kind: LedgerEventKind;
chatId: string; chatId: string;
at: string; at: string;
/** Idempotency key: the quiz part's toolCallId or `taught:<treeNodeId>` — /** Idempotency key: the quiz part's toolCallId, or `obs:<sourceId>` for
* re-emission of the same chat must never double-record an event. */ * understanding observations — re-emission of the same chat must never
* double-record an event. */
ref?: string; ref?: string;
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */ /** The learner's quiz note — diagnostic gold (vision §1 calibration). */
note?: string; note?: string;
@@ -83,91 +105,121 @@ export interface LedgerEvent {
/** /**
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES * Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
* (nodes/edges/events/mappings/sessions). The derived graph is rebuilt by * (nodes/edges/events/mappings/sessions), all append-only.
* reading the ledgers — store events, derive state. sessions.jsonl is the *
* extraction checkpoint: which chats the extraction pipeline has processed, * nodes.jsonl one line per node — the FIRST line for an id defines
* and through which updatedAt (never extract the same turns twice). * statement/description/createdAt; later lines for the
* same id are link-appends (a session's re-encounter):
* their statements/aliases join `aliases`, their
* provenance unions. getGraph merges by id.
* edges.jsonl one line per edge ASSERTION; readers merge by
* (from,to,type), unioning provenance.
*
* The derived graph is rebuilt by reading the ledgers — store events,
* derive state. sessions.jsonl is the extraction checkpoint: which chats
* the extraction pipeline has processed, and through which updatedAt
* (never extract the same turns twice).
* *
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor → * The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
* merger): the extractor proposes propositions/edges/attribution with * merger), tree-blind and post-hoc: the extractor reads only the transcript
* verbatim evidence, the auditor drops anything unbacked, and the merger * and proposes propositions/edges/quiz-attribution with verbatim evidence;
* (core/graph.ts, deterministic) resolves identity against the existing * the auditor sees the extractor's output PLUS the current graph, owns
* ledgers and appends. See SessionFindings below. * faithfulness (evidence check) and identity (link-or-create: point each
* proposition at the existing node it is, or let it be created — recording
* the session's phrasings as aliases); the merger (core/graph.ts,
* deterministic) applies the verdicts and appends.
*/ */
// ─── Extraction findings (extractor output → audit → merger input) ─────────── // ─── Extraction findings (extractor output → audit → merger input) ───────────
/** One proposed proposition — a node candidate. */ /** One proposed proposition — a node candidate (or a re-encounter of one). */
export interface ExtractedProposition { export interface ExtractedProposition {
/** Extractor-local id ("p1", …) referenced by edges + attribution. */ /** Extractor-local id ("p1", …) referenced by edges + attribution. */
id: string; id: string;
/** Short name / header for the fact (2-6 words) — the rendering handle. */ /** The minimal proposition — a true, self-contained, world-facing
name: string; * statement. No learner / teacher / session references: misconceptions
/** Canonical one-reasoning-step statement — an objective, world-facing * are stated as the true correction, and wrongness lives in the event
* fact (no learner / teacher / session references; the learner's * ledger, never in the node. */
* relationship to it is carried only by the event attribution). */
statement: string; statement: string;
/** Verbatim alternate phrasings from the session (merge aliases). */ /** The elaborated version — required. */
description: string;
/** Verbatim phrasings the session used. */
aliases: string[]; aliases: string[];
kind: "truth" | "misconception"; /** The message ids where this proposition came up — provenance. */
/** The plan-tree node this came from, when it did (mapping provenance). */ messages: string[];
treeNodeId?: string;
/** Verbatim transcript quotes backing the proposition. */ /** Verbatim transcript quotes backing the proposition. */
evidence: string[]; evidence: string[];
/** The extractor's understanding note for this proposition IN THIS
* session — a named-word scale ("solid", "shaky", "accepted",
* "resisted"), no numbers. Joins the ledger as an observation event;
* treatment pending the state-model ideation. */
understanding?: string;
} }
/** One proposed relation between two propositions. */ /** One proposed relation between two propositions. */
export interface ExtractedEdge { export interface ExtractedEdge {
/** Proposition id that is the prerequisite / support. */ /** Proposition id that is followed / supported. */
from: string; from: string;
/** Proposition id that depends on / is supported by `from`. */
to: string; to: string;
type: EdgeType; type: EdgeType;
/** The message ids where this relation was asserted — provenance. */
messages: string[];
/** Verbatim transcript quotes backing the relation. */ /** Verbatim transcript quotes backing the relation. */
evidence: string[]; evidence: string[];
} }
/** The extractor's event attribution — its ONLY say about events. The kinds, /**
* timestamps, idempotency refs, notes and question texts are all rebuilt * The extractor's quiz attribution — its ONLY say about quiz events. Event
* deterministically by the merger from the transcript's quiz / mark_done
* tool parts; the LLM supplies exactly the judgment call: WHICH proposition
* each one is about (vision open question: probes attribute retroactively,
* against the whole transcript — a pre-tree probe can map to a proposition
* the tree only articulates later). */
/** The extractor's event attribution — its ONLY say about events. Event
* kinds, timestamps, idempotency refs, notes and question texts are all * kinds, timestamps, idempotency refs, notes and question texts are all
* rebuilt deterministically by the merger from the transcript's tool * rebuilt deterministically by the merger from the transcript's quiz tool
* parts, and TAGGED events (quiz nodeId, mark_done tree node) resolve * parts; the LLM supplies exactly the judgment call: WHICH proposition each
* deterministically through the proposition mapping. The LLM supplies * quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
* exactly the one judgment call that needs it — the vision's open * mapping to lean on) — retroactively against the whole transcript.
* probe-attribution question: WHICH proposition did an UNTAGGED probe */
* test? Answered retroactively against the whole transcript — a pre-tree
* probe can map to a proposition the tree only articulates later. */
export interface ExtractionAttribution { export interface ExtractionAttribution {
/** UNTAGGED quiz toolCallId → proposition id; missing / null = /** quiz toolCallId → proposition id; null / missing = unattributed (the
* unattributed (the event lands with nodeId null + the question text, * event lands with nodeId null + the question text, a later-
* exactly like unattributable probes always did). */ * attribution input). */
quiz: Record<string, string | null>; quiz: Record<string, string | null>;
} }
/** The extractor's — and, filtered, the auditor's — output shape. */ /** The extractor's output shape. */
export interface SessionFindings { export interface SessionFindings {
propositions: ExtractedProposition[]; propositions: ExtractedProposition[];
edges: ExtractedEdge[]; edges: ExtractedEdge[];
attribution: ExtractionAttribution; attribution: ExtractionAttribution;
} }
/** The auditor's verdict: the verified subset, plus the drops (for the log — /** The auditor's verdict on one proposition: the findings, plus `linkTo` —
* the audit trail of what was rejected and why). */ * the identity judgment. Absent = create a new node. */
export interface AuditResult extends SessionFindings { export interface AuditedProposition extends ExtractedProposition {
/** The existing GraphNode id this proposition IS — the auditor's
* identity verdict. On link, the session's phrasings (statement,
* aliases) join the node's aliases and the provenance unions.
* Conservative: when unsure, leave it out — duplicates over wrong
* merges. */
linkTo?: string;
}
/** The auditor's output: the verified findings (with identity verdicts),
* plus the drops (the audit trail of what was rejected and why). */
export interface AuditResult {
propositions: AuditedProposition[];
edges: ExtractedEdge[];
attribution: ExtractionAttribution;
/** Dropped candidates: proposition id, edge "from→to", or attribution /** Dropped candidates: proposition id, edge "from→to", or attribution
* toolCallId — with the reason. */ * toolCallId — with the reason. */
dropped: { ref: string; reason: string }[]; dropped: { ref: string; reason: string }[];
} }
/** Per-node state class for the home view (vision §1 "Home page") — derived, /**
* never stored: taught if ANY taught event; else the LATEST quiz event decides * Per-node state class for the home view — TRANSITIONAL (kept so the
* (known / missed / dont-know); a node with no events is a candidate. */ * renderer survives the contract change untouched). Derived, never stored:
* the LATEST quiz event decides (known / missed / dont-know); a node with
* no quiz events is a candidate. `taught` is now unreachable (no taught
* events are emitted anymore) — the real state model (dependency depth +
* topology-derived strength + quiz events) is pending ideation.
*/
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate"; export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */ /** The GET /api/graph payload — everything the home view's PURE renderer needs. */
@@ -187,4 +239,3 @@ export function titleFromPrompt(prompt: string): string {
const oneLine = prompt.replace(/\s+/g, " ").trim(); const oneLine = prompt.replace(/\s+/g, " ").trim();
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine; return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
} }
+16 -22
View File
@@ -15,10 +15,9 @@
* the block is absent (new learner). * the block is absent (new learner).
* *
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE * Selection strategy (§1 "the selection problem", simplest first): the WHOLE
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know / * graph while it is small; past MAX_NODES, hotspots (missed / dont-know
* misconception nodes) plus their 1-hop `requires` neighborhood — a live * nodes) plus their 1-hop `follows` neighborhood — the frontier is where
* misconception is always relevant context, and the frontier is where the * the next session should probe.
* next session should probe.
*/ */
import type { GraphEdge, GraphNode } from "./contracts.ts"; import type { GraphEdge, GraphNode } from "./contracts.ts";
@@ -31,22 +30,18 @@ export const GRAPH_CONTEXT_HEADER = "LEARNER KNOWLEDGE GRAPH (auto-generated)";
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */ /** Above this node count only hotspots + their 1-hop neighborhood are shown. */
export const MAX_CONTEXT_NODES = 150; export const MAX_CONTEXT_NODES = 150;
/** Group order for node lines: diagnostic value first (vision §1 hotspots). /** Group order for node lines: diagnostic value first (vision §1 hotspots). */
* A misconception-kind node is a hotspot whatever its state — a live
* misconception is always relevant context. */
const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]); const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]);
function groupOf(node: GraphNode, state: NodeStateClass): number { function groupOf(node: GraphNode, state: NodeStateClass): number {
if (HOTSPOT_STATES.has(state) || node.kind === "misconception") return 0; if (HOTSPOT_STATES.has(state)) return 0;
if (state === "known") return 1; if (state === "known") return 1;
if (state === "taught") return 2; return 2; // candidate
return 3; // candidate
} }
const GROUP_LABELS = [ const GROUP_LABELS = [
"Hotspots (missed / dont-know / misconceptions) — likely misconception sites and unexplored ground:", "Hotspots (missed / dont-know) — likely misconception sites and unexplored ground:",
"Known (answered correctly):", "Known (answered correctly):",
"Taught (lessons covered them):",
"Candidates (never probed — assume nothing):", "Candidates (never probed — assume nothing):",
] as const; ] as const;
@@ -76,9 +71,9 @@ export function renderGraphContext(
truncated = true; truncated = true;
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n))); const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
const keep = new Set(hotspots.map((n) => n.id)); const keep = new Set(hotspots.map((n) => n.id));
// 1-hop requires neighborhood: prerequisites AND dependents. // 1-hop follows neighborhood: premises AND derivations.
for (const edge of edges) { for (const edge of edges) {
if (edge.type !== "requires") continue; if (edge.type !== "follows") continue;
if (keep.has(edge.from) || keep.has(edge.to)) { if (keep.has(edge.from) || keep.has(edge.to)) {
keep.add(edge.from); keep.add(edge.from);
keep.add(edge.to); keep.add(edge.to);
@@ -91,7 +86,7 @@ export function renderGraphContext(
// ── node lines, grouped/sorted by diagnostic priority ────────────────────────── // ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
const lines: string[] = [ const lines: string[] = [
`===== ${GRAPH_CONTEXT_HEADER} =====`, `===== ${GRAPH_CONTEXT_HEADER} =====`,
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "requires").length} requires edges`, `${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "follows").length} follows edges`,
]; ];
for (const [group, label] of GROUP_LABELS.entries()) { for (const [group, label] of GROUP_LABELS.entries()) {
@@ -101,19 +96,18 @@ export function renderGraphContext(
if (groupNodes.length === 0) continue; if (groupNodes.length === 0) continue;
lines.push("", label); lines.push("", label);
for (const node of groupNodes) { for (const node of groupNodes) {
const kind = node.kind === "misconception" ? " · misconception" : ""; lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})`);
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})${kind}`);
} }
} }
// ── requires edges, merged into chains ────────────────────────────────── // ── requires edges, merged into chains ──────────────────────────────────
const byId = new Map(selected.map((n) => [n.id, n.statement])); const byId = new Map(selected.map((n) => [n.id, n.statement]));
const requires = edges.filter( const follows = edges.filter(
(e) => e.type === "requires" && byId.has(e.from) && byId.has(e.to), (e) => e.type === "follows" && byId.has(e.from) && byId.has(e.to),
); );
if (requires.length > 0) { if (follows.length > 0) {
lines.push("", "Requires (prerequisite → dependent):"); lines.push("", "Follows (premise → derivation):");
for (const chain of edgeChains(requires)) { for (const chain of edgeChains(follows)) {
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → ")); lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
} }
} }
+164 -160
View File
@@ -2,22 +2,34 @@
* The knowledge graph: merger (the deterministic final stage of the * The knowledge graph: merger (the deterministic final stage of the
* extraction pipeline — vision §1 construction) + derivation (reads). * extraction pipeline — vision §1 construction) + derivation (reads).
* *
* The extraction pipeline is the SOLE graph writer: * The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
* *
* extractor (LLM) → SessionFindings: propositions, edges, and event * extractor (LLM) → SessionFindings: propositions, edges, quiz
* attribution, each with verbatim evidence * attribution, per-proposition understanding notes —
* auditor (LLM) → the verified subset (unbacked candidates dropped) * each with verbatim evidence, from the transcript ALONE
* auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
* VERDICT per proposition (linkTo an existing node, or
* create), and the drops for the log
* merger (HERE) → mergeFindings(): deterministic ledger appends * merger (HERE) → mergeFindings(): deterministic ledger appends
* *
* The merger is deliberately the ONLY stage that touches the ledgers, and it * The merger is deliberately the ONLY stage that touches the ledgers, and it
* is pure code: the LLM supplies exactly the judgment calls (what was * is pure code: the LLM supplies exactly the judgment calls (what the
* taught, what depends on what, which proposition a quiz tested), while * propositions are, what follows/supports what, which proposition a quiz
* event kinds/timestamps/refs — anything the model could silently * tested, whether a proposition IS an existing node), while event
* mis-copy — are rebuilt deterministically from the transcript's tool * kinds/timestamps/refs/provenance — anything the model could silently
* parts. Idempotent + incremental: statement/alias matching dedupes nodes, * mis-copy — are rebuilt deterministically from the transcript's tool parts
* (from,to,type) dedupes edge assertions, chat-scoped refs dedupe events, * and the message ids in the findings. Idempotent + incremental: refs are
* one mapping line per (chat, graph node), and sessions.jsonl checkpoints * chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
* which chats have been extracted (never the same turns twice). * asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
* sessions.jsonl checkpoints which chats have been extracted (never the
* same turns twice).
*
* Identity across re-encounters is the AUDITOR's judgment (link-or-create),
* not string matching: the conservative v1 exact-statement merge was
* replaced by the 2026-09-02 redesign. A re-extracted session's
* propositions link to the existing nodes; the link appends a node line
* whose phrasings join `aliases` and whose provenance unions (getGraph
* merges lines by id).
* *
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads * Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
* only — everything is recomputed from the ledgers. Every function takes * only — everything is recomputed from the ledgers. Every function takes
@@ -29,14 +41,14 @@
import { randomBytes } from "node:crypto"; import { randomBytes } from "node:crypto";
import { isToolUIPart, type UIMessage } from "ai"; import { isToolUIPart, type UIMessage } from "ai";
import type { import type {
AuditResult,
GraphEdge, GraphEdge,
GraphNode, GraphNode,
ProvenancePoint,
GraphSnapshot, GraphSnapshot,
LedgerEvent, LedgerEvent,
LedgerEventKind, LedgerEventKind,
NodeStateClass, NodeStateClass,
PlanToolResult,
SessionFindings,
} from "./contracts.ts"; } from "./contracts.ts";
import type { QuestionAnswerData } from "./question-core.ts"; import type { QuestionAnswerData } from "./question-core.ts";
import type { ChatStore, GraphStore } from "./ports.ts"; import type { ChatStore, GraphStore } from "./ports.ts";
@@ -80,19 +92,25 @@ function toolParts(messages: UIMessage[]): ToolPartView[] {
return out; return out;
} }
/** A plan tool part's output, if it is a well-formed PlanToolResult. */ /** (chatId, messageIds) → provenance points; order-preserving, deduped. */
function planResult(part: ToolPartView): PlanToolResult | null { function pointsOf(chatId: string, messages: string[]): ProvenancePoint[] {
if (part.state !== "output-available") return null; const seen = new Set<string>();
const out = part.output; const out: ProvenancePoint[] = [];
if (typeof out !== "object" || out === null) return null; for (const messageId of messages) {
if (!("tree" in out) || typeof (out as PlanToolResult).ok !== "boolean") return null; if (messageId.length === 0 || seen.has(messageId)) continue;
return out as PlanToolResult; seen.add(messageId);
out.push({ chatId, messageId });
} }
return out;
}
const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
a.chatId === b.chatId && a.messageId === b.messageId;
// ─── the merger (extraction pipeline stage 3 — deterministic) ──────────────── // ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
export interface EmissionStats { export interface EmissionStats {
/** New graph nodes appended (and mappings with them). */ /** New graph nodes appended. */
nodes: number; nodes: number;
/** New edge assertions appended. */ /** New edge assertions appended. */
edges: number; edges: number;
@@ -102,156 +120,135 @@ export interface EmissionStats {
/** /**
* Merge one chat's audited findings into the graph ledgers — the pipeline's * Merge one chat's audited findings into the graph ledgers — the pipeline's
* apply stage. Deterministic + idempotent: re-merging the same findings (or * apply stage. Deterministic + idempotent: re-merging the same verdicts (or
* a later re-extraction of the same chat) appends nothing but the new. * a later re-extraction of the same chat) appends nothing but the new.
* *
* Identity resolution (v1: conservative): a proposition whose statement or * Identity is the auditor's link-or-create verdict: `linkTo` an existing
* alias EXACTLY matches (case-insensitive, trimmed) an existing node's * node (validated against the ledger — an unknown id falls back to create)
* statement or alias merges onto it; everything else is a new node. One * appends a LINK line (the session's phrasings join the node's aliases and
* mapping line per (chat, graph node) — the merge audit trail. Fuzzy / * the provenance unions; getGraph merges lines by id); absent → a new node.
* embedding adjudication stays deferred (vision §1 consolidation). * Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
* a proposition id already mapped for this chat reuses its resolution.
* *
* Events are rebuilt deterministically from the transcript's quiz and * Events are rebuilt deterministically from the transcript: quiz outcomes
* plan_mark_done parts (kinds, timestamps, notes, refs). TAGGED events * (correct / miss / dont-know; kinds + timestamps + refs from the tool
* (quiz nodeId, mark_done tree node) resolve through the proposition * parts, nodeId from the extractor's attribution through this run's
* mapping — deterministic; the findings' attribution is consulted ONLY for * mapping) plus one observation event per proposition carrying an
* untagged probe quizzes: the retroactive probe-attribution resolution (a * understanding note (ref `obs:<sourceId>`). No taught events, ever.
* pre-tree probe maps to a proposition the tree only articulates later).
* Unattributed quizzes land with nodeId null + the question text, exactly
* like the unattributable ones always have.
*/ */
export function mergeFindings(chatId: string, findings: SessionFindings, deps: GraphDeps): EmissionStats { export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps): EmissionStats {
const { graphStore: ledgers, chatStore } = deps; const { graphStore: ledgers, chatStore } = deps;
const record = chatStore.getChat(chatId); const record = chatStore.getChat(chatId);
if (!record) return { nodes: 0, edges: 0, events: 0 }; if (!record) return { nodes: 0, edges: 0, events: 0 };
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 }; const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
// 1. Identity resolution. mapping: proposition id (and tree node id) // 1. Resolution: this run's propositions → graph node ids. Existing
// → graph node id — from this chat's existing mapping lines, then // mapping lines for THIS chat are loaded first — the audit trail doubles
// extended as findings resolve. chatNodeIds: graph nodes this chat has // as the resolution table, which makes re-merging the same chat
// already mapped onto — the (chat, graph node) mapping dedupe. // idempotent (a re-extraction re-merges through the auditor's linkTo
// verdicts instead). Fresh sourceIds resolve as the verdicts say.
const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
const mapping = new Map<string, string>(); const mapping = new Map<string, string>();
const chatNodeIds = new Set<string>(); const seenMappingKeys = new Set<string>();
for (const m of ledgers.readMappings()) { for (const m of ledgers.readMappings()) {
if (m.chatId !== chatId || typeof m.sourceId !== "string") continue; if (m.chatId !== chatId) continue;
mapping.set(m.sourceId, m.graphNodeId); mapping.set(m.sourceId, m.graphNodeId);
chatNodeIds.add(m.graphNodeId); seenMappingKeys.add(m.sourceId);
}
// 2. Propositions → graph nodes. Exact statement/alias match against the
// whole ledger first (cross-session reuse), then against propositions
// created earlier in THIS merge (intra-findings dedupe).
const norm = (s: string): string => s.trim().toLowerCase();
const existingByTitle = new Map<string, string>();
for (const node of ledgers.readNodes()) {
for (const title of [node.statement, ...node.aliases]) {
const key = norm(title);
if (key && !existingByTitle.has(key)) existingByTitle.set(key, node.id);
}
} }
const now = new Date().toISOString(); const now = new Date().toISOString();
for (const p of findings.propositions) {
if (mapping.has(p.id)) continue; for (const p of audit.propositions) {
const keys = [p.statement, ...p.aliases].map(norm).filter((k) => k.length > 0); if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
const matchId = keys.map((k) => existingByTitle.get(k)).find((id) => id !== undefined); const points = pointsOf(chatId, p.messages);
let graphNodeId: string; let graphNodeId: string;
if (matchId !== undefined) { if (p.linkTo !== undefined && existingIds.has(p.linkTo)) {
graphNodeId = matchId; // reuse — conservative merge, duplicates over wrong merges // Identity verdict: this proposition IS that node. Append a link
} else { // line — the session's phrasings (statement + aliases) join the
const graphNode: GraphNode = { // node's aliases, the provenance unions. Reads merge by id.
id: newGraphNodeId(), graphNodeId = p.linkTo;
name: p.name, ledgers.appendNode({
id: graphNodeId,
statement: p.statement, statement: p.statement,
description: p.description,
aliases: [...p.aliases], aliases: [...p.aliases],
kind: p.kind,
createdAt: now, createdAt: now,
provenance: points,
});
} else {
const node: GraphNode = {
id: newGraphNodeId(),
statement: p.statement,
description: p.description,
aliases: [...p.aliases],
createdAt: now,
provenance: points,
}; };
ledgers.appendNode(graphNode); ledgers.appendNode(node);
stats.nodes++; stats.nodes++;
graphNodeId = graphNode.id; graphNodeId = node.id;
} existingIds.add(graphNodeId);
for (const key of keys) if (!existingByTitle.has(key)) existingByTitle.set(key, graphNodeId);
if (!chatNodeIds.has(graphNodeId)) {
ledgers.appendMapping({ chatId, sourceId: p.treeNodeId ?? p.id, graphNodeId });
chatNodeIds.add(graphNodeId);
} }
mapping.set(p.id, graphNodeId); mapping.set(p.id, graphNodeId);
if (p.treeNodeId !== undefined) mapping.set(p.treeNodeId, graphNodeId); if (!seenMappingKeys.has(p.id)) {
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
seenMappingKeys.add(p.id);
}
} }
// 3. Edges. Dedupe by (from,to,type) across the whole ledger: an existing // 2. Edges. Resolve endpoints through the mapping; dedupe by
// assertion just gains this chatId as provenance (appended as a new // (from,to,type) already asserted by THIS chat (an existing assertion
// line; reads merge). // by another chat stays untouched — provenance unions at read time).
const asserted = new Map<string, Set<string>>(); const assertedByChat = new Set<string>();
for (const line of ledgers.readEdges()) { for (const line of ledgers.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`; const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
let provenance = asserted.get(key); if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
if (!provenance) {
provenance = new Set();
asserted.set(key, provenance);
} }
for (const c of line.provenance) provenance.add(c); for (const edge of audit.edges) {
}
for (const edge of findings.edges) {
const from = mapping.get(edge.from); const from = mapping.get(edge.from);
const to = mapping.get(edge.to); const to = mapping.get(edge.to);
if (!from || !to || from === to) continue; // unmapped endpoint — skip if (!from || !to || from === to) continue; // unmapped endpoint — skip
const key = `${from}\u0000${to}\u0000${edge.type}`; const key = `${from}\u0000${to}\u0000${edge.type}`;
const provenance = asserted.get(key); if (assertedByChat.has(key)) continue;
if (provenance?.has(chatId)) continue; // already asserted by this chat ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
ledgers.appendEdge({ from, to, type: edge.type, provenance: [chatId] }); assertedByChat.add(key);
if (provenance) provenance.add(chatId);
else asserted.set(key, new Set([chatId]));
stats.edges++; stats.edges++;
} }
// 4. Events, rebuilt deterministically in transcript order so the // 3. Events, rebuilt deterministically in transcript order. Refs are
// probe/check boundary is exact: quizzes BEFORE the first plan_init // chat-scoped idempotency keys (quiz toolCallId, obs:<sourceId>) —
// output are probes. Refs are chat-scoped (a quiz toolCallId or // a ref already in the ledger means this evidence was extracted before.
// taught:<treeNodeId>) — a ref already in the ledger means this turn
// was extracted before. The findings' attribution supplies the
// nodeId, nothing else.
const seenRefs = new Set<string>(); const seenRefs = new Set<string>();
for (const event of ledgers.readEvents()) { for (const event of ledgers.readEvents()) {
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref); if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
} }
let planSeen = false;
for (const part of toolParts(record.messages)) { for (const part of toolParts(record.messages)) {
// The boundary flips on the first plan_init OUTPUT, whatever it says if (part.name !== "quiz" || part.state !== "output-available") continue;
// (an ok:false "tree already initialized" still means a tree exists). const event = quizEvent(part, chatId, audit, mapping, now);
if (part.name === "plan_init" && part.state === "output-available") planSeen = true;
if (part.name === "quiz" && part.state === "output-available") {
const event = quizEvent(part, chatId, mapping, findings, planSeen, now);
if (event?.ref !== undefined && !seenRefs.has(event.ref)) { if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
ledgers.appendEvent(event); ledgers.appendEvent(event);
seenRefs.add(event.ref); seenRefs.add(event.ref);
stats.events++; stats.events++;
} }
} }
// Understanding observations: one per proposition carrying a note,
if (part.name === "plan_mark_done" && part.state === "output-available") { // resolved to the node the verdict mapped it to.
const result = planResult(part); for (const p of audit.propositions) {
const input = part.input as { id?: unknown } | undefined; if (p.understanding === undefined || p.understanding.length === 0) continue;
if (result?.ok && typeof input?.id === "string") { const ref = `obs:${p.id}`;
const ref = `taught:${input.id}`; if (seenRefs.has(ref)) continue;
if (!seenRefs.has(ref)) {
// mark_done is tagged with the tree node id — deterministic.
ledgers.appendEvent({ ledgers.appendEvent({
nodeId: mapping.get(input.id) ?? null, nodeId: mapping.get(p.id) ?? null,
kind: "taught", kind: "observation",
chatId, chatId,
at: now, at: now,
ref, ref,
note: p.understanding,
}); });
seenRefs.add(ref); seenRefs.add(ref);
stats.events++; stats.events++;
} }
}
}
}
// 5. Checkpoint: this chat is extracted through its current updatedAt. // 4. Checkpoint: this chat is extracted through its current updatedAt.
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt }); ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
return stats; return stats;
} }
@@ -261,9 +258,8 @@ export function mergeFindings(chatId: string, findings: SessionFindings, deps: G
function quizEvent( function quizEvent(
part: ToolPartView, part: ToolPartView,
chatId: string, chatId: string,
audit: AuditResult,
mapping: Map<string, string>, mapping: Map<string, string>,
findings: SessionFindings,
planSeen: boolean,
now: string, now: string,
): LedgerEvent | null { ): LedgerEvent | null {
const out = part.output; const out = part.output;
@@ -274,16 +270,13 @@ function quizEvent(
let kind: LedgerEventKind; let kind: LedgerEventKind;
if (answer.dontKnow === true) kind = "dont-know"; if (answer.dontKnow === true) kind = "dont-know";
else if (answer.correct === true) kind = planSeen ? "check-correct" : "probe-correct"; else if (answer.correct === true) kind = "correct";
else kind = planSeen ? "check-incorrect" : "probe-miss"; else kind = "miss";
// Tagged (input.nodeId) resolves deterministically through the mapping; // Tree-blind: EVERY quiz attributes through the extractor's judgment —
// attribution is the LLM judgment call for UNTAGGED probes only — the // the vision's retroactive attribution, with no tree mapping to lean on.
// vision's open probe-attribution question, resolved retroactively. const attributedId = audit.attribution.quiz[part.toolCallId];
const input = part.input as { nodeId?: unknown } | undefined; const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
const tagged = typeof input?.nodeId === "string" ? mapping.get(input.nodeId) : undefined;
const attributedId = findings.attribution.quiz[part.toolCallId];
const nodeId = tagged ?? (attributedId ? (mapping.get(attributedId) ?? null) : null);
const event: LedgerEvent = { const event: LedgerEvent = {
nodeId, nodeId,
@@ -293,7 +286,7 @@ function quizEvent(
ref: part.toolCallId, ref: part.toolCallId,
}; };
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note; if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
// The question text is the attribution input for unattributed probes. // The question text is the attribution input for unattributed quizzes.
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText; if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
return event; return event;
} }
@@ -335,20 +328,39 @@ export interface DerivedGraph {
} }
/** /**
* The whole graph from the ledgers. Edge LINES are assertions; the returned * The whole graph from the ledgers. Node LINES may repeat an id (link-
* edges are merged by (from,to,type) with unioned provenance, first-seen * appends): the FIRST line defines statement/description/createdAt; later
* order. `store events, derive state` — this is the only reader that matters. * lines' statements and aliases join `aliases`, provenance unions. Edge
* LINES are assertions, merged by (from,to,type) with unioned provenance,
* first-seen order. `store events, derive state` — this is the only reader
* that matters.
*/ */
export function getGraph(graphStore: GraphStore): DerivedGraph { export function getGraph(graphStore: GraphStore): DerivedGraph {
const nodes = graphStore.readNodes(); // Merge node lines by id (first line defines; later lines add phrasings).
const byId = new Map<string, GraphNode>();
for (const line of graphStore.readNodes()) {
const existing = byId.get(line.id);
if (!existing) {
byId.set(line.id, { ...line, aliases: [...line.aliases], provenance: [...line.provenance] });
continue;
}
// A later line's statement is a phrasing, not a replacement.
for (const phrasing of [line.statement, ...line.aliases]) {
if (phrasing.length > 0 && !existing.aliases.includes(phrasing)) existing.aliases.push(phrasing);
}
for (const pt of line.provenance) {
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
}
}
const merged: GraphEdge[] = []; const merged: GraphEdge[] = [];
const byKey = new Map<string, GraphEdge>(); const byKey = new Map<string, GraphEdge>();
for (const line of graphStore.readEdges()) { for (const line of graphStore.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`; const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
const existing = byKey.get(key); const existing = byKey.get(key);
if (existing) { if (existing) {
for (const c of line.provenance) { for (const pt of line.provenance) {
if (!existing.provenance.includes(c)) existing.provenance.push(c); if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
} }
} else { } else {
const edge: GraphEdge = { ...line, provenance: [...line.provenance] }; const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
@@ -356,34 +368,27 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
merged.push(edge); merged.push(edge);
} }
} }
return { nodes, edges: merged, events: graphStore.readEvents() }; return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
} }
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set([ const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set(["correct", "miss", "dont-know"]);
"probe-correct",
"check-correct",
"probe-miss",
"check-incorrect",
"dont-know",
]);
/** /**
* Derive node states from events (never stored): taught if ANY taught event; * Derive node states from events (never stored): the LATEST quiz event
* else the LATEST quiz event decides (known / missed / dont-know); a node * decides (known / missed / dont-know); a node with no quiz events is a
* with no events is a candidate. * candidate. TRANSITIONAL (2026-09-02 redesign): `taught` is unreachable
* now that no taught events are emitted, and observation events are NOT
* state inputs — the real state model (dependency depth + topology-derived
* strength + quiz events) is pending ideation; this keeps the renderer
* alive until it lands.
*/ */
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> { export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> {
const states: Record<string, NodeStateClass> = {}; const states: Record<string, NodeStateClass> = {};
for (const node of nodes) states[node.id] = "candidate"; for (const node of nodes) states[node.id] = "candidate";
const taught = new Set<string>();
const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>(); const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>();
events.forEach((event, index) => { events.forEach((event, index) => {
if (event.nodeId === null) return; if (event.nodeId === null) return;
if (event.kind === "taught") {
taught.add(event.nodeId);
return;
}
if (!QUIZ_EVENT_KINDS.has(event.kind)) return; if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
const current = latest.get(event.nodeId); const current = latest.get(event.nodeId);
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) { if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
@@ -392,13 +397,9 @@ export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Rec
}); });
for (const node of nodes) { for (const node of nodes) {
if (taught.has(node.id)) {
states[node.id] = "taught";
continue;
}
const last = latest.get(node.id); const last = latest.get(node.id);
if (!last) continue; // candidate if (!last) continue; // candidate
if (last.kind === "probe-correct" || last.kind === "check-correct") states[node.id] = "known"; if (last.kind === "correct") states[node.id] = "known";
else if (last.kind === "dont-know") states[node.id] = "dont-know"; else if (last.kind === "dont-know") states[node.id] = "dont-know";
else states[node.id] = "missed"; else states[node.id] = "missed";
} }
@@ -422,10 +423,13 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
chats.add(chatId); chats.add(chatId);
}; };
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId); for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
for (const node of nodes) {
for (const pt of node.provenance) touch(node.id, pt.chatId);
}
for (const edge of edges) { for (const edge of edges) {
for (const chatId of edge.provenance) { for (const pt of edge.provenance) {
touch(edge.from, chatId); touch(edge.from, pt.chatId);
touch(edge.to, chatId); touch(edge.to, pt.chatId);
} }
} }
for (const event of events) { for (const event of events) {
+1 -1
View File
@@ -56,7 +56,7 @@ export interface GraphStore {
appendEdge(assertion: GraphEdge): void; appendEdge(assertion: GraphEdge): void;
readEdges(): GraphEdge[]; readEdges(): GraphEdge[];
/** Append one quiz outcome / taught event. */ /** Append one quiz outcome / understanding observation. */
appendEvent(event: LedgerEvent): void; appendEvent(event: LedgerEvent): void;
readEvents(): LedgerEvent[]; readEvents(): LedgerEvent[];
+28 -35
View File
@@ -1,20 +1,26 @@
/** /**
* GraphStore — the fs JSONL adapter: five append-only ledgers at * GraphStore — the fs JSONL adapter: five append-only ledgers at
* config.LEDGER_FILES (contracts.ts "v1 emission"). * config.LEDGER_FILES (contracts.ts "v1 emission").
* nodes.jsonl GraphNode — one line per graph node * nodes.jsonl GraphNode — one line per node write: the FIRST
* line for an id defines it; a later
* line with a REPEATED id is a
* link-append (session phrasings join
* aliases, provenance unions); reads
* merge by id (core/graph.ts getGraph)
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second * edges.jsonl GraphEdge — one line per edge ASSERTION (a second
* session asserting the same (from,to,type) * session asserting the same (from,to,type)
* appends another line; reads merge) * appends another line; reads merge)
* events.jsonl LedgerEvent — quiz outcomes + taught events * events.jsonl LedgerEvent — quiz outcomes + understanding observations
* mappings.jsonl NodeMapping — the session→graph identity map: * mappings.jsonl NodeMapping — the session→graph identity map:
* (chatId, sourceId) → graphNodeId * (chatId, sourceId) → graphNodeId
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt) * sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
* *
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this * Implements core/ports.ts GraphStore. data/graph/* is exclusively this
* adapter's. Reads are strict against the contracts.ts shapes: corrupt lines * adapter's. Reads are strict against the contracts.ts shapes (the 2026-09-02
* are skipped, never guessed at — the ledgers are the source of truth and * graph redesign): corrupt lines are skipped, never guessed at — the ledgers
* everything else is derived (core/graph.ts, which takes this port as a * are the source of truth and everything else is derived (core/graph.ts,
* parameter). Same idioms as src/store/chats.ts: sync fs, no framework. * which takes this port as a parameter). Same idioms as src/store/chats.ts:
* sync fs, no framework.
*/ */
import * as fs from "node:fs"; import * as fs from "node:fs";
@@ -26,6 +32,7 @@ import type {
LedgerEvent, LedgerEvent,
LedgerEventKind, LedgerEventKind,
NodeMapping, NodeMapping,
ProvenancePoint,
SessionCheckpoint, SessionCheckpoint,
} from "../core/contracts.ts"; } from "../core/contracts.ts";
import type { GraphStore } from "../core/ports.ts"; import type { GraphStore } from "../core/ports.ts";
@@ -69,26 +76,22 @@ function readValid<T>(file: string, isValid: (v: unknown) => v is T): T[] {
return readRawLines(file).filter(isValid); return readRawLines(file).filter(isValid);
} }
/** Pre-name ledger line (the first extraction runs) — read tolerantly. */ /** Provenance points are (chatId, messageId) pairs — message-granular. */
type StoredGraphNode = Omit<GraphNode, "name"> & { name?: string }; const isProvenancePoint = (v: unknown): v is ProvenancePoint =>
isRecord(v) && isString(v.chatId) && isString(v.messageId);
const isStoredGraphNode = (v: unknown): v is StoredGraphNode => const isGraphNode = (v: unknown): v is GraphNode =>
isRecord(v) && isRecord(v) &&
isString(v.id) && isString(v.id) &&
isString(v.statement) && isString(v.statement) &&
isString(v.description) &&
Array.isArray(v.aliases) && Array.isArray(v.aliases) &&
(v.kind === "truth" || v.kind === "misconception") && v.aliases.every(isString) &&
isString(v.createdAt) && isString(v.createdAt) &&
(v.name === undefined || isString(v.name)); Array.isArray(v.provenance) &&
v.provenance.every(isProvenancePoint);
/** Derived header for legacy node lines written before `name` existed: const EDGE_TYPES = ["follows", "supports"] as const;
* the first clause of the statement, capped — good enough to render. */
function nameFromStatement(statement: string): string {
const clause = statement.split(/[.;:—]/)[0]!.trim();
return clause.length > 60 ? `${clause.slice(0, 57)}…` : clause || statement;
}
const EDGE_TYPES = ["requires", "supports"] as const;
const isGraphEdge = (v: unknown): v is GraphEdge => const isGraphEdge = (v: unknown): v is GraphEdge =>
isRecord(v) && isRecord(v) &&
@@ -96,17 +99,9 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
isString(v.to) && isString(v.to) &&
(EDGE_TYPES as readonly string[]).includes(v.type as string) && (EDGE_TYPES as readonly string[]).includes(v.type as string) &&
Array.isArray(v.provenance) && Array.isArray(v.provenance) &&
v.provenance.every(isString); v.provenance.every(isProvenancePoint);
const EVENT_KINDS: readonly LedgerEventKind[] = [ const EVENT_KINDS: readonly LedgerEventKind[] = ["correct", "miss", "dont-know", "observation"];
"probe-correct",
"probe-miss",
"dont-know",
"misconception-signaled",
"taught",
"check-correct",
"check-incorrect",
];
const isLedgerEvent = (v: unknown): v is LedgerEvent => const isLedgerEvent = (v: unknown): v is LedgerEvent =>
isRecord(v) && isRecord(v) &&
@@ -127,13 +122,11 @@ export const graphStore: GraphStore = {
appendNode(node: GraphNode): void { appendNode(node: GraphNode): void {
appendLine(LEDGER_FILES.nodes, node); appendLine(LEDGER_FILES.nodes, node);
}, },
/** Raw node lines in append order (ids are unique — no merging needed). /** Raw node lines in append order. A node id may appear on more than
* Legacy lines without a `name` get one derived from the statement. */ * one line (a later session's LINK-append: phrasings + provenance); the
* derivation (core/graph.ts getGraph) merges lines by id. */
readNodes(): GraphNode[] { readNodes(): GraphNode[] {
return readValid(LEDGER_FILES.nodes, isStoredGraphNode).map((n) => ({ return readValid(LEDGER_FILES.nodes, isGraphNode);
...n,
name: n.name ?? nameFromStatement(n.statement),
}));
}, },
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */ /** Append one edge ASSERTION; readers merge lines by (from,to,type). */
+89 -71
View File
@@ -1,13 +1,16 @@
/** /**
* The extraction pipeline tests (vision §1 construction): extractor → * The extraction pipeline tests (vision §1 construction, 2026-09-02 redesign):
* auditor → merger, the sole graph writer. * extractor → auditor → merger, the sole graph writer — tree-blind,
* free-form, with the auditor owning faithfulness and identity.
* *
* transcriptView — the compact view both LLM stages see (pure) * transcriptView — the compact view both LLM stages see (pure):
* message-id-tagged verbatim text + quiz lines, NO tree
* pipeline — MockLanguageModelV4 drives both agents: findings out, * pipeline — MockLanguageModelV4 drives both agents: findings out,
* audit verdict in, deterministic merge applied; failure * audit verdict (with linkTo identity) in, deterministic
* paths checkpoint nothing; single-flight per chat * merge applied; failure paths checkpoint nothing;
* single-flight per chat
* *
* The merger itself (identity resolution, event rebuild, idempotency) is * The merger itself (identity verdicts, event rebuild, idempotency) is
* tested in tests/graph.test.ts directly against mergeFindings — here it is * tested in tests/graph.test.ts directly against mergeFindings — here it is
* the integration: what one full pipeline run does to the ledgers. * the integration: what one full pipeline run does to the ledgers.
* *
@@ -21,9 +24,16 @@ import * as fs from "node:fs";
import { after, describe, it } from "node:test"; import { after, describe, it } from "node:test";
import { MockLanguageModelV4 } from "ai/test"; import { MockLanguageModelV4 } from "ai/test";
import type { UIMessage } from "ai"; import type { UIMessage } from "ai";
import type { GraphEdge, GraphNode, LedgerEvent, NodeMapping, SessionCheckpoint, SessionFindings } from "../src/core/contracts.ts"; import type {
AuditResult,
GraphEdge,
GraphNode,
LedgerEvent,
NodeMapping,
SessionCheckpoint,
SessionFindings,
} from "../src/core/contracts.ts";
import type { GraphStore } from "../src/core/ports.ts"; import type { GraphStore } from "../src/core/ports.ts";
import { createTree, markDone } from "../src/core/plan-tree.ts";
import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts"; import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts";
import { chatStore } from "../src/store/chats.ts"; import { chatStore } from "../src/store/chats.ts";
import { PATHS } from "../config.ts"; import { PATHS } from "../config.ts";
@@ -72,7 +82,7 @@ function quizPart(toolCallId: string, input: Record<string, unknown>, output: Re
return { type: "tool-quiz", toolCallId, state: "output-available", input, output }; return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
} }
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: ReturnType<typeof createTree>): unknown { function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: unknown): unknown {
return { return {
type: `tool-${name}`, type: `tool-${name}`,
toolCallId, toolCallId,
@@ -82,21 +92,11 @@ function planPart(toolCallId: string, name: string, input: Record<string, unknow
}; };
} }
/** A session-shaped fixture chat: probe → plan → check → mark_done. */ /** A session-shaped fixture chat: probe → teaching text → check. Tree-blind
* world: the plan tree is NOT part of extraction — it is noise. */
function fixtureChat() { function fixtureChat() {
const tree = markDone(
createTree(
[
{ id: "cr", title: "Chain rule intuition" },
{ id: "apply", title: "Applying the chain rule" },
],
[{ from: "cr", to: "apply" }],
),
"cr",
);
const chat = chatStore.createChat("Extraction fixture"); const chat = chatStore.createChat("Extraction fixture");
createdChats.push(chat.id); createdChats.push(chat.id);
chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, [ chatStore.saveMessages(chat.id, [
user("u1", "teach me the chain rule"), user("u1", "teach me the chain rule"),
assistant("a1", [ assistant("a1", [
@@ -107,15 +107,14 @@ function fixtureChat() {
]), ]),
assistant("a2", [ assistant("a2", [
{ type: "text", text: "The derivative measures how fast a function changes." }, { type: "text", text: "The derivative measures how fast a function changes." },
planPart("p1", "plan_init", { nodes: [], edges: [] }, tree), planPart("p1", "plan_init", { nodes: [], edges: [] }, { nodes: [], edges: [], currentNodeId: null, doneNodeIds: [], version: 1 }),
]), ]),
assistant("a3", [ assistant("a3", [
quizPart("check-1", { question: "Check", nodeId: "apply" }, { quizPart("check-1", { question: "Check" }, {
questionId: "check-1", kind: "quiz", questionText: "Check", correct: true, dontKnow: false, questionId: "check-1", kind: "quiz", questionText: "Check", correct: true, dontKnow: false,
answeredAt: "2026-08-22T02:00:00.000Z", answeredAt: "2026-08-22T02:00:00.000Z",
}), }),
]), ]),
assistant("a4", [planPart("p2", "plan_mark_done", { id: "cr" }, tree)]),
]); ]);
return { chatId: chat.id, record: chatStore.getChat(chat.id)! }; return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
} }
@@ -130,30 +129,38 @@ after(() => {
} }
}); });
/** What the extractor would produce for the fixture session. */ /** What the extractor would produce for the fixture session — tree-blind,
* free-form, with message-granular provenance + an understanding note. */
const FINDINGS: SessionFindings = { const FINDINGS: SessionFindings = {
propositions: [ propositions: [
{ {
id: "p1", id: "p1",
name: "Chain rule", statement: "The derivative measures how fast a function changes",
statement: "Chain rule intuition", description: "The derivative of a function at a point is its instantaneous rate of change there.",
aliases: [], aliases: [],
kind: "truth", messages: ["a2"],
treeNodeId: "cr", evidence: ["TEACHER [a2]: The derivative measures how fast a function changes."],
evidence: ["TEACHER: The derivative measures how fast a function changes."], understanding: "solid",
}, },
{ {
id: "p2", id: "p2",
name: "Applying chain rule", statement: "Chain rule intuition",
statement: "Applying the chain rule", description: "The chain rule differentiates composed functions by multiplying local rates.",
aliases: [], aliases: [],
kind: "truth", messages: ["a2"],
treeNodeId: "apply", evidence: ["LEARNER [u1]: teach me the chain rule"],
evidence: ["FINAL PLAN TREE (node ids): cr = Chain rule intuition; apply = Applying the chain rule"],
}, },
], ],
edges: [{ from: "p1", to: "p2", type: "requires", evidence: ["TREE EDGES: cr requires apply"] }], edges: [
attribution: { quiz: { "probe-1": "p1" } }, {
from: "p1",
to: "p2",
type: "follows",
messages: ["a2"],
evidence: ["LEARNER [u1]: teach me the chain rule"],
},
],
attribution: { quiz: { "probe-1": "p1", "check-1": "p2" } },
}; };
/** A model whose ONE tool call is `name` with the given input; then text. */ /** A model whose ONE tool call is `name` with the given input; then text. */
@@ -202,28 +209,26 @@ function silentModel(): MockLanguageModelV4 {
// ─── transcriptView (pure) ─────────────────────────────────────────────────── // ─── transcriptView (pure) ───────────────────────────────────────────────────
describe("transcriptView (pure)", () => { describe("transcriptView (pure)", () => {
it("shows the final tree, verbatim conversation, structured events — plan ops folded", () => { it("shows message-id-tagged verbatim conversation + quiz lines — tree-blind, plan ops folded", () => {
const { record } = fixtureChat(); const { record } = fixtureChat();
const view = transcriptView(record); const view = transcriptView(record);
// The final tree with node ids (the extractor's backbone + the
// pruned-out stays pruned — only the FINAL tree is shown).
assert.match(view, /SESSION: Extraction fixture/); assert.match(view, /SESSION: Extraction fixture/);
assert.match(view, /FINAL PLAN TREE \(node ids\): cr = Chain rule intuition; apply = Applying the chain rule/);
assert.match(view, /TREE EDGES: cr requires apply/);
// Conversation verbatim — evidence quotes can come from it. // Conversation verbatim, each line tagged with its message id —
assert.match(view, /LEARNER: teach me the chain rule/); // the provenance anchors the extractor reports back.
assert.match(view, /TEACHER: The derivative measures how fast a function changes\./); assert.match(view, /LEARNER \[u1\] teach me the chain rule/);
assert.match(view, /TEACHER \[a2\] The derivative measures how fast a function changes\./);
// Structured events, attribution-ready. // Structured events, attribution-ready.
assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/); assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/);
assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/); assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/);
assert.match(view, /TEACHER marked tree node done \[p2\]: cr/);
// plan_init / plan_mark_done tool parts themselves are folded into // TREE-BLIND: no plan tree, no tree edges, no mark_done lines — plan
// the tree line — no raw JSON noise. // ops fold to the generic one-liner like every other non-quiz tool.
assert.doesNotMatch(view, /plan_init/); assert.doesNotMatch(view, /FINAL PLAN TREE/);
assert.doesNotMatch(view, /TREE EDGES/);
assert.match(view, /\(plan_init tool result delivered\)/);
assert.doesNotMatch(view, /output-available/); assert.doesNotMatch(view, /output-available/);
}); });
@@ -253,48 +258,61 @@ describe("transcriptView (pure)", () => {
// ─── the pipeline ─────────────────────────────────────────────────────────── // ─── the pipeline ───────────────────────────────────────────────────────────
describe("the extraction pipeline (extractor → auditor → merger)", () => { describe("the extraction pipeline (extractor → auditor → merger)", () => {
it("runs end to end: findings → audit verdict → deterministic merge + checkpoint", async () => { it("runs end to end: findings → audit verdict (identity) → deterministic merge + checkpoint", async () => {
const { chatId } = fixtureChat(); const { chatId } = fixtureChat();
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
// The auditor passes p1 through, drops p2 (unbacked), re-points the // The auditor passes p1 through and LINKS it to a pre-existing node
// probe attribution to p1, and records the drop. // (identity verdict), drops p2 (unbacked) + the edge with it.
const verified: SessionFindings = { const existingId = "n_existing";
propositions: [FINDINGS.propositions[0]!], ledgers.appendNode({
id: existingId,
statement: "Derivatives measure rates of change",
description: "Pre-existing node from an earlier session.",
aliases: [],
createdAt: "2026-08-01T00:00:00.000Z",
provenance: [],
});
const verdict: AuditResult = {
propositions: [
{ ...FINDINGS.propositions[0]!, linkTo: existingId },
],
edges: [], edges: [],
attribution: { quiz: { "probe-1": "p1" } }, attribution: { quiz: { "probe-1": "p1" } },
dropped: [{ ref: "p2", reason: "evidence not found" }, { ref: "p1→p2", reason: "endpoint dropped" }],
}; };
const pipeline = createExtractionPipeline({ const pipeline = createExtractionPipeline({
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: submitModel("submit_findings", FINDINGS), extractor: submitModel("submit_findings", FINDINGS),
auditor: submitModel("submit_review", { ...verified, dropped: [{ ref: "p2", reason: "evidence not found" }] }), auditor: submitModel("submit_review", verdict),
}); });
const stats = await pipeline.runExtraction(chatId); const stats = await pipeline.runExtraction(chatId);
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 3 }); // The linked proposition created NO new node; the understanding note
// became an observation; both quizzes became events.
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 3 });
// Only the audited proposition became a node; mapping provenance kept. // Link-append: the phrasing joined the existing node's aliases and
assert.equal(ledgers.readNodes().length, 1); // the provenance unioned; no second node line defines a new node.
assert.equal(ledgers.readNodes()[0]!.statement, "Chain rule intuition"); assert.equal(ledgers.readNodes().length, 2); // the pre-seeded line + the link line
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "cr", graphNodeId: ledgers.readNodes()[0]!.id }]); assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "p1", graphNodeId: existingId }]);
// Events rebuilt deterministically from the transcript — the probe // Events rebuilt deterministically: quizzes in transcript order, then
// retroactively attributed, the check tagged, mark_done mapped. // the observation. check-1's attribution pointed at the DROPPED p2 →
// unresolvable → null (the question text is kept for later attribution).
const events = ledgers.readEvents(); const events = ledgers.readEvents();
assert.deepEqual( assert.deepEqual(
events.map((e) => [e.kind, e.ref]), events.map((e) => [e.kind, e.ref, e.nodeId]),
[ [
["probe-miss", "probe-1"], ["miss", "probe-1", existingId],
["check-correct", "check-1"], ["correct", "check-1", null],
["taught", "taught:cr"], ["observation", "obs:p1", existingId],
], ],
); );
const nodeId = ledgers.readNodes()[0]!.id; assert.equal(events[0]!.nodeId, existingId, "probe attributed via the verdict's mapping");
assert.equal(events[0]!.nodeId, nodeId, "probe attributed via audit"); assert.equal(events[2]!.note, "solid", "the understanding note joins the ledger");
assert.equal(events[1]!.nodeId, null, "tagged to a dropped proposition → null");
assert.equal(events[2]!.nodeId, nodeId, "taught via tree-node mapping");
// Checkpointed through the record's updatedAt — never the same turns twice. // Checkpointed through the record's updatedAt — never the same turns twice.
assert.equal(ledgers.readCheckpoints().length, 1); assert.equal(ledgers.readCheckpoints().length, 1);
@@ -318,7 +336,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
chatStore, chatStore,
graphStore: ledgers, graphStore: ledgers,
extractor: silentModel(), extractor: silentModel(),
auditor: submitModel("submit_review", FINDINGS), auditor: submitModel("submit_review", { ...FINDINGS, dropped: [] }),
}); });
await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/); await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/);
assert.equal(ledgers.readCheckpoints().length, 0); assert.equal(ledgers.readCheckpoints().length, 0);
+117 -138
View File
@@ -1,13 +1,14 @@
/** /**
* Graph-as-context tests (W8; vision §1 "the graph as session context"). * Graph-as-context tests (vision §1 "the graph as session context", 2026-09-02
* redesign vocabulary: follows edges, no node kinds).
* *
* renderGraphContext — pure: empty / small / cap-truncation rendering * renderGraphContext — pure: empty / small / cap-truncation rendering
* buildGraphContext — per-request entry point (derived-graph → block) * buildGraphContext — per-request entry point (derived-graph → block)
* injection plumbing — chat-flow style: the teacher's instructions carry * injection plumbing — chat-flow style: the teacher's instructions carry
* the block (MockLanguageModelV4 captures the prompt) * the block (MockLanguageModelV4 captures the prompt)
* exact-match merge — the merger (mergeFindings) dedupes * cross-chat edges — the merger (mergeFindings) appends one ASSERTION
* case/whitespace-identical statements onto existing * line per session; reads merge with unioned
* nodes (no fuzzy matching) * message-granular provenance
* *
* Ledger isolation: NONE needed for the merge tests — the merger's ledger is * Ledger isolation: NONE needed for the merge tests — the merger's ledger is
* injectable (in-memory), so the live data/graph files are only ever READ * injectable (in-memory), so the live data/graph files are only ever READ
@@ -22,7 +23,14 @@ import * as path from "node:path";
import { after, describe, it } from "node:test"; import { after, describe, it } from "node:test";
import { MockLanguageModelV4, simulateReadableStream } from "ai/test"; import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider"; import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
import type { GraphEdge, GraphNode, LedgerEvent, SessionFindings } from "../src/core/contracts.ts"; import type {
AuditResult,
GraphEdge,
GraphNode,
LedgerEvent,
NodeMapping,
SessionCheckpoint,
} from "../src/core/contracts.ts";
import type { NodeStateClass } from "../src/core/contracts.ts"; import type { NodeStateClass } from "../src/core/contracts.ts";
import { getGraph, mergeFindings } from "../src/core/graph.ts"; import { getGraph, mergeFindings } from "../src/core/graph.ts";
import type { GraphStore } from "../src/core/ports.ts"; import type { GraphStore } from "../src/core/ports.ts";
@@ -35,19 +43,25 @@ import {
import { createTeacherAgent } from "../src/agent/teacher-agent.ts"; import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
import { handleChatRequest } from "../src/server/chat-handler.ts"; import { handleChatRequest } from "../src/server/chat-handler.ts";
import { chatStore } from "../src/store/chats.ts"; import { chatStore } from "../src/store/chats.ts";
import type { NodeMapping, SessionCheckpoint } from "../src/core/contracts.ts";
import { graphStore } from "../src/store/graph.ts"; import { graphStore } from "../src/store/graph.ts";
import { createTree } from "../src/core/plan-tree.ts";
import { PATHS } from "../config.ts"; import { PATHS } from "../config.ts";
// ─── fixtures ──────────────────────────────────────────────────────────────── // ─── fixtures ────────────────────────────────────────────────────────────────
function node(id: string, statement: string, over: Partial<GraphNode> = {}): GraphNode { function node(id: string, statement: string, over: Partial<GraphNode> = {}): GraphNode {
return { id, name: statement.split(";")[0]!, statement, aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z", ...over }; return {
id,
statement,
description: `${statement} — elaborated.`,
aliases: [],
createdAt: "2026-08-22T00:00:00.000Z",
provenance: [],
...over,
};
} }
function edge(from: string, to: string): GraphEdge { function edge(from: string, to: string): GraphEdge {
return { from, to, type: "requires", provenance: ["chat_x"] }; return { from, to, type: "follows", provenance: [{ chatId: "chat_x", messageId: "m1" }] };
} }
function states(...entries: [string, NodeStateClass][]): Record<string, NodeStateClass> { function states(...entries: [string, NodeStateClass][]): Record<string, NodeStateClass> {
@@ -61,7 +75,7 @@ describe("renderGraphContext (pure)", () => {
assert.equal(renderGraphContext([], [], {}), null); assert.equal(renderGraphContext([], [], {}), null);
}); });
it("renders a header, state-grouped node lines (hotspots first), and chained requires edges", () => { it("renders a header, state-grouped node lines (hotspots first), and chained follows edges", () => {
const nodes = [ const nodes = [
node("n_a", "Chain rule intuition"), node("n_a", "Chain rule intuition"),
node("n_b", "Quotient rule"), node("n_b", "Quotient rule"),
@@ -73,26 +87,26 @@ describe("renderGraphContext (pure)", () => {
["n_a", "known"], ["n_a", "known"],
["n_b", "missed"], ["n_b", "missed"],
["n_c", "candidate"], ["n_c", "candidate"],
["n_d", "taught"], ["n_d", "dont-know"],
); );
const out = renderGraphContext(nodes, edges, st)!; const out = renderGraphContext(nodes, edges, st)!;
assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present"); assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present");
assert.match(out, /4 nodes, 3 requires edges/); assert.match(out, /4 nodes, 3 follows edges/);
// Node lines: "• [state] statement (id)". // Node lines: "• [state] statement (id)".
assert.ok(out.includes("• [missed] Quotient rule (n_b)")); assert.ok(out.includes("• [missed] Quotient rule (n_b)"));
assert.ok(out.includes("• [known] Chain rule intuition (n_a)")); assert.ok(out.includes("• [known] Chain rule intuition (n_a)"));
assert.ok(out.includes("• [taught] Composite functions (n_d)")); assert.ok(out.includes("• [dont-know] Composite functions (n_d)"));
assert.ok(out.includes("• [candidate] Product rule (n_c)")); assert.ok(out.includes("• [candidate] Product rule (n_c)"));
// Groups in the decided order: hotspots, known, taught, candidates. // Groups in the decided order: hotspots, known, candidates.
const order = ["Hotspots", "Known", "Taught", "Candidates"].map((g) => out.indexOf(g)); const order = ["Hotspots", "Known", "Candidates"].map((g) => out.indexOf(g));
assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order"); assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order");
// Edges merged into chains: d → a → b → c (one line, three arrows). // Edges merged into chains: d → a → b → c (one line, three arrows).
assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule")); assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule"));
assert.ok(out.includes("Requires (prerequisite → dependent)")); assert.ok(out.includes("Follows (premise → derivation)"));
assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter"); assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter");
}); });
@@ -104,15 +118,8 @@ describe("renderGraphContext (pure)", () => {
assert.ok(!out.includes("B → C"), "no edge was invented between siblings"); assert.ok(!out.includes("B → C"), "no edge was invented between siblings");
}); });
it("marks misconception-kind nodes as hotspots even when their state is candidate", () => { it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop follows neighborhood only, with a truncation note", () => {
const nodes = [node("n_m", "Division by zero is fine", { kind: "misconception" })]; // 3 hotspots, each with a premise and a derivation (1-hop), plus a
const out = renderGraphContext(nodes, [], states(["n_m", "candidate"]))!;
assert.match(out, /• \[candidate\] Division by zero is fine \(n_m\) · misconception/);
assert.ok(out.includes("Hotspots"), "misconception nodes join the hotspot group");
});
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop requires neighborhood only, with a truncation note", () => {
// 3 hotspots, each with a prerequisite and a dependent (1-hop), plus a
// long tail of disconnected candidates that must drop out. // long tail of disconnected candidates that must drop out.
const nodes: GraphNode[] = []; const nodes: GraphNode[] = [];
const edges: GraphEdge[] = []; const edges: GraphEdge[] = [];
@@ -121,7 +128,7 @@ describe("renderGraphContext (pure)", () => {
const h = `n_h${i}`; const h = `n_h${i}`;
const pre = `n_p${i}`; const pre = `n_p${i}`;
const dep = `n_d${i}`; const dep = `n_d${i}`;
nodes.push(node(pre, `Prerequisite ${i}`), node(h, `Hotspot ${i}`), node(dep, `Dependent ${i}`)); nodes.push(node(pre, `Premise ${i}`), node(h, `Hotspot ${i}`), node(dep, `Derivation ${i}`));
edges.push(edge(pre, h), edge(h, dep)); edges.push(edge(pre, h), edge(h, dep));
st[pre] = "known"; st[pre] = "known";
st[h] = "missed"; st[h] = "missed";
@@ -139,11 +146,11 @@ describe("renderGraphContext (pure)", () => {
const out = renderGraphContext(nodes, edges, st)!; const out = renderGraphContext(nodes, edges, st)!;
assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay"); assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay");
assert.ok(out.includes("• [known] Prerequisite 1 (n_p1)"), "1-hop prerequisites stay"); assert.ok(out.includes("• [known] Premise 1 (n_p1)"), "1-hop premises stay");
assert.ok(out.includes("• [candidate] Dependent 1 (n_d1)"), "1-hop dependents stay"); assert.ok(out.includes("• [candidate] Derivation 1 (n_d1)"), "1-hop derivations stay");
assert.ok(!out.includes("Far away"), "disconnected nodes drop out"); assert.ok(!out.includes("Far away"), "disconnected nodes drop out");
assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted"); assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted");
assert.ok(out.includes(", 6 requires edges"), "edges among the selected nodes stay"); assert.ok(out.includes(", 6 follows edges"), "edges among the selected nodes stay");
}); });
it("keeps the full graph at exactly the cap (no truncation note)", () => { it("keeps the full graph at exactly the cap (no truncation note)", () => {
@@ -169,11 +176,11 @@ describe("buildGraphContext", () => {
it("derives states from events and renders the block", () => { it("derives states from events and renders the block", () => {
const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")]; const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")];
const events: LedgerEvent[] = [ const events: LedgerEvent[] = [
{ nodeId: "n_b", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" }, { nodeId: "n_b", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
]; ];
const out = buildGraphContext({ nodes, edges: [], events })!; const out = buildGraphContext({ nodes, edges: [], events })!;
assert.ok(out.includes(GRAPH_CONTEXT_HEADER)); assert.ok(out.includes(GRAPH_CONTEXT_HEADER));
assert.ok(out.includes("• [known] Checked node (n_b)"), "check-correct → known"); assert.ok(out.includes("• [known] Checked node (n_b)"), "correct → known");
assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate"); assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate");
}); });
@@ -244,7 +251,7 @@ describe("graph-context injection (chat-flow style)", () => {
nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")], nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")],
edges: [edge("n_ctx2", "n_ctx")], edges: [edge("n_ctx2", "n_ctx")],
events: [ events: [
{ nodeId: "n_ctx2", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" }, { nodeId: "n_ctx2", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
] as LedgerEvent[], ] as LedgerEvent[],
}; };
const context = buildGraphContext(graph); const context = buildGraphContext(graph);
@@ -296,7 +303,7 @@ describe("graph-context injection (chat-flow style)", () => {
}); });
}); });
// ─── exact-match merge (merger identity resolution) ────────────────────────── // ─── cross-chat edge assertions (merger + read-time merging) ────────────────
/** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */ /** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */
function memoryLedgers() { function memoryLedgers() {
@@ -317,128 +324,100 @@ function memoryLedgers() {
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c), appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
readCheckpoints: () => checkpoints.map((c) => ({ ...c })), readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
// Test accessors (not part of the GraphStore surface). // Test accessors (not part of the GraphStore surface).
counts: { get nodes() { return nodes.length; }, get mappings() { return mappings.length; } }, counts: { get nodes() { return nodes.length; } },
} satisfies GraphStore & { counts: { nodes: number; mappings: number } }; } satisfies GraphStore & { counts: { nodes: number } };
} }
/** Findings as the extractor would file them for a plan tree: one /** A minimal chat record with one user message (the merger reads the real chat store). */
* proposition per node (statement = title, treeNodeId set), one `requires` function fixtureChat(title: string) {
* edge per tree edge — the merge tests drive the merger with these. */
function treeFindings(tree: ReturnType<typeof createTree>): SessionFindings {
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
return {
propositions: tree.nodes.map((n, i) => ({
id: `p${i + 1}`,
name: n.title,
statement: n.title,
aliases: [],
kind: "truth" as const,
treeNodeId: n.id,
evidence: ["fixture evidence"],
})),
edges: tree.edges.map((e) => ({
from: idByNodeId.get(e.from)!,
to: idByNodeId.get(e.to)!,
type: "requires" as const,
evidence: ["fixture evidence"],
})),
attribution: { quiz: {} },
};
}
/** A real chat record with a stored tree (the merger reads the real chat store). */
function fixtureChat(title: string, tree: ReturnType<typeof createTree>) {
const chat = chatStore.createChat(title); const chat = chatStore.createChat(title);
createdChats.push(chat.id); createdChats.push(chat.id);
chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]); chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]);
return chatStore.getChat(chat.id)!; return chatStore.getChat(chat.id)!;
} }
describe("exact-match merge (merger identity resolution)", () => { describe("cross-chat edge assertions (merger + read-time merging)", () => {
it("maps a case/whitespace-identical title onto the existing node instead of duplicating", () => { it("a second session asserting the same edge appends a second line; reads merge with unioned provenance", () => {
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
const chat1 = fixtureChat("Merge A", tree1);
let stats = mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 0 });
const firstId = ledgers.readNodes()[0]!.id;
// Second session, same proposition: different casing + padding. // Session 1: two fresh propositions + a follows edge between them.
const tree2 = createTree([{ id: "cr2", title: " chain Rule INTUITION " }], []); const chat1 = fixtureChat("Edge A");
const chat2 = fixtureChat("Merge B", tree2); const verdict1: AuditResult = {
stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers }); propositions: [
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "no duplicate node appended"); {
id: "p1",
assert.equal(ledgers.counts.nodes, 1, "still exactly one graph node"); statement: "Derivative as a limit",
assert.equal(ledgers.readNodes()[0]!.id, firstId); description: "The derivative is the limit of difference quotients.",
// Provenance stays honest: one mapping line per (chat, graph node). aliases: [],
assert.equal(ledgers.counts.mappings, 2); messages: ["u1"],
assert.deepEqual(ledgers.readMappings()[1], { chatId: chat2.id, sourceId: "cr2", graphNodeId: firstId }); evidence: ["e"],
}); },
{
it("matches aliases too (exact, case-insensitive)", () => { id: "p2",
const ledgers = memoryLedgers(); statement: "Chain rule intuition",
ledgers.appendNode({ description: "Composed functions differentiate by multiplying local rates.",
id: "n_alias", aliases: [],
name: "Differentiation", messages: ["u1"],
statement: "Differentiation basics", evidence: ["e"],
aliases: ["the derivative"], },
kind: "truth",
createdAt: "2026-08-22T00:00:00.000Z",
});
const tree = createTree([{ id: "d", title: "The Derivative" }], []);
const chat = fixtureChat("Merge alias", tree);
const stats = mergeFindings(chat.id, treeFindings(tree), { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "alias match → no new node");
assert.deepEqual(ledgers.readMappings(), [{ chatId: chat.id, sourceId: "d", graphNodeId: "n_alias" }]);
});
it("does NOT fuzzy-match near-identical titles (that stays deferred)", () => {
const ledgers = memoryLedgers();
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
const chat1 = fixtureChat("Fuzzy A", tree1);
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
const tree2 = createTree([{ id: "cr2", title: "Chain rules intuition" }], []);
const chat2 = fixtureChat("Fuzzy B", tree2);
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
assert.equal(stats.nodes, 1, "near-miss statements stay separate nodes");
assert.equal(ledgers.counts.nodes, 2);
});
it("merged nodes unify the second chat's edges onto the existing graph node", () => {
const ledgers = memoryLedgers();
const tree1 = createTree(
[
{ id: "base", title: "Derivative as a limit" },
{ id: "chain", title: "Chain rule intuition" },
], ],
[{ from: "base", to: "chain" }], edges: [{ from: "p1", to: "p2", type: "follows", messages: ["u1"], evidence: ["e"] }],
); attribution: { quiz: {} },
const chat1 = fixtureChat("Edge A", tree1); dropped: [],
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers }); };
let stats = mergeFindings(chat1.id, verdict1, { chatStore, graphStore: ledgers });
assert.deepEqual(stats, { nodes: 2, edges: 1, events: 0 });
const base = ledgers.readNodes()[0]!.id;
const chain = ledgers.readNodes()[1]!.id;
// Second session re-teaches the same two propositions (exact statements // Session 2: the auditor links both propositions to the SAME nodes
// modulo case) and asserts the same edge — it must dedupe onto the // (identity verdicts) and asserts the same relation.
// existing nodes. const chat2 = fixtureChat("Edge B");
const tree2 = createTree( const verdict2: AuditResult = {
[ propositions: [
{ id: "base", title: "derivative as a limit" }, {
{ id: "chain", title: "chain rule intuition" }, id: "q1",
statement: "The derivative as a limit of quotients", // different phrasing
description: "…",
aliases: [],
messages: ["u1"],
evidence: ["e"],
linkTo: base,
},
{
id: "q2",
statement: "Chain rule, intuitively",
description: "…",
aliases: [],
messages: ["u1"],
evidence: ["e"],
linkTo: chain,
},
], ],
[{ from: "base", to: "chain" }], edges: [{ from: "q1", to: "q2", type: "follows", messages: ["u1"], evidence: ["e"] }],
); attribution: { quiz: {} },
const chat2 = fixtureChat("Edge B", tree2); dropped: [],
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers }); };
// A new ASSERTION line is appended (provenance is honest) — reads merge stats = mergeFindings(chat2.id, verdict2, { chatStore, graphStore: ledgers });
// by (from,to,type), so the derived graph still has one edge with both
// sessions in its provenance. // No new nodes; ONE new assertion line (per-session honest provenance).
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line"); assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line");
assert.equal(ledgers.counts.nodes, 2); assert.equal(ledgers.counts.nodes, 4, "2 defining lines + 2 link-appends for the linked propositions");
const lines = ledgers.readEdges(); const lines = ledgers.readEdges();
assert.equal(lines.length, 2, "two assertion lines for the same edge"); assert.equal(lines.length, 2, "two assertion lines for the same edge");
assert.equal(lines[0]!.from, lines[1]!.from); assert.equal(lines[0]!.from, lines[1]!.from);
assert.equal(lines[0]!.to, lines[1]!.to); assert.equal(lines[0]!.to, lines[1]!.to);
assert.deepEqual(lines.map((l) => l.provenance), [[chat1.id], [chat2.id]]); assert.deepEqual(lines.map((l) => l.provenance), [
[{ chatId: chat1.id, messageId: "u1" }],
[{ chatId: chat2.id, messageId: "u1" }],
]);
// Read-time merging: one derived edge, both sessions in its provenance.
const derived = getGraph(ledgers).edges;
assert.equal(derived.length, 1);
assert.deepEqual(derived[0]!.provenance, [
{ chatId: chat1.id, messageId: "u1" },
{ chatId: chat2.id, messageId: "u1" },
]);
}); });
}); });
+220 -164
View File
@@ -1,9 +1,9 @@
/** /**
* Graph merger + trigger + derivation tests (vision §1; contracts.ts * Graph merger + trigger + derivation tests (vision §1, 2026-09-02 redesign).
* "extraction pipeline"). The merger (core/graph.ts mergeFindings) is the * The merger (core/graph.ts mergeFindings) is the pipeline's deterministic
* pipeline's deterministic apply stage: findings in, ledger appends out — * apply stage: AUDITED VERDICTS in, ledger appends out — link-or-create
* identity resolution, retroactive probe attribution, event rebuild, and * identity (the auditor's linkTo, never string matching), deterministic quiz
* the idempotency guarantees. * events, understanding observations, and the idempotency guarantees.
* *
* Ledger isolation: PATHS is fixed at import time, so the five data/graph * Ledger isolation: PATHS is fixed at import time, so the five data/graph
* ledgers are backed up and cleared before the suite and restored after * ledgers are backed up and cleared before the suite and restored after
@@ -18,9 +18,8 @@ import assert from "node:assert/strict";
import * as fs from "node:fs"; import * as fs from "node:fs";
import { after, describe, it } from "node:test"; import { after, describe, it } from "node:test";
import type { UIMessage } from "ai"; import type { UIMessage } from "ai";
import type { PlanTree, PlanToolResult, SessionFindings } from "../src/core/contracts.ts"; import type { AuditResult } from "../src/core/contracts.ts";
import type { QuestionAnswerData } from "../src/core/question-core.ts"; import type { QuestionAnswerData } from "../src/core/question-core.ts";
import { createTree, markDone } from "../src/core/plan-tree.ts";
import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts"; import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
import { graphStore } from "../src/store/graph.ts"; import { graphStore } from "../src/store/graph.ts";
import { chatStore } from "../src/store/chats.ts"; import { chatStore } from "../src/store/chats.ts";
@@ -80,16 +79,6 @@ function quizPart(
return { type: "tool-quiz", toolCallId, state: "output-available", input, output }; return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
} }
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: PlanTree): unknown {
return {
type: `tool-${name}`,
toolCallId,
state: "output-available",
input,
output: { ok: true, tree } satisfies PlanToolResult,
};
}
function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData { function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
return { return {
questionId: "q", questionId: "q",
@@ -102,59 +91,17 @@ function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
}; };
} }
/** Create a chat in the real store with a tree and messages; returns the record. */ /** Create a chat in the real store with messages; returns the record.
function fixtureChat(title: string, tree: PlanTree | null, messages: UIMessage[]) { * Tree-blind world: the plan tree is irrelevant to the merger — no tree. */
function fixtureChat(title: string, messages: UIMessage[]) {
const chat = chatStore.createChat(title); const chat = chatStore.createChat(title);
createdChats.push(chat.id); createdChats.push(chat.id);
if (tree) chatStore.saveTree(chat.id, tree);
chatStore.saveMessages(chat.id, messages); chatStore.saveMessages(chat.id, messages);
return chatStore.getChat(chat.id)!; return chatStore.getChat(chat.id)!;
} }
/** The fixture tree: alpha → beta, gamma free-floating, alpha done. */ /** The full fixture transcript: five graded quizzes + a dismissed one + an ask. */
function fixtureTree(): PlanTree { function fixtureMessages(): UIMessage[] {
return markDone(
createTree(
[
{ id: "alpha", title: "Alpha intuition" },
{ id: "beta", title: "Beta rule" },
{ id: "gamma", title: "Gamma application" },
],
[{ from: "alpha", to: "beta" }],
),
"alpha",
);
}
/**
* Findings shaped exactly like what the extractor would produce for a
* session's plan tree: one proposition per tree node (statement = title,
* treeNodeId set), one `requires` edge per tree edge.
*/
function treeFindings(tree: PlanTree): SessionFindings {
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
return {
propositions: tree.nodes.map((n, i) => ({
id: `p${i + 1}`,
name: n.title,
statement: n.title,
aliases: [],
kind: "truth" as const,
treeNodeId: n.id,
evidence: ["fixture evidence"],
})),
edges: tree.edges.map((e) => ({
from: idByNodeId.get(e.from)!,
to: idByNodeId.get(e.to)!,
type: "requires" as const,
evidence: ["fixture evidence"],
})),
attribution: { quiz: {} },
};
}
/** The full fixture transcript: probes → plan → checks → mark_done → noise. */
function fixtureMessages(tree: PlanTree): UIMessage[] {
return [ return [
user("u1", "teach me the fixture topic"), user("u1", "teach me the fixture topic"),
assistant("a1", [ assistant("a1", [
@@ -171,32 +118,30 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }), answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }),
), ),
]), ]),
assistant("a3", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]), assistant("a3", [
assistant("a4", [ // input.nodeId "alpha" is deliberately present and IGNORED —
// attribution is the only quiz→node path in the tree-blind world.
quizPart( quizPart(
"call-3", "call-3",
{ question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" }, { question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }), answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }),
), ),
]), ]),
assistant("a5", [ assistant("a4", [
quizPart( quizPart(
"call-4", "call-4",
{ question: "Q4", nodeId: "beta", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" }, { question: "Q4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }), answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }),
), ),
]), ]),
assistant("a6", [ assistant("a5", [
// nodeId present but NOT in the tree → tagged-but-unmapped: the
// attribution can rescue it, or it lands null + question text.
quizPart( quizPart(
"call-5", "call-5",
{ question: "Q5", nodeId: "ghost", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" }, { question: "Q5", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }), answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }),
), ),
]), ]),
assistant("a7", [planPart("p2", "plan_mark_done", { id: "alpha" }, tree)]), assistant("a6", [
assistant("a8", [
// dismissed quiz + an answered ask — neither becomes an event // dismissed quiz + an answered ask — neither becomes an event
quizPart( quizPart(
"call-6", "call-6",
@@ -214,25 +159,58 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
]; ];
} }
/** /** The auditor's verdict for the fixture chat: three verified propositions
* The extractor's findings for the fixture chat: the tree as propositions + * (no linkTo — the graph starts empty), one follows edge, all quizzes
* the probe attribution. The audit stage is a no-op pass-through in these * attributed, one understanding note. */
* merger tests — its behavior is tested with the pipeline (extraction.test). function fixtureAudit(): AuditResult {
*/ return {
function fixtureFindings(): SessionFindings { propositions: [
const findings = treeFindings(fixtureTree()); {
// Untagged probe quizzes: call-1 attributed retroactively to Alpha (the id: "p1",
// vision's open question — resolved by the extractor); call-2 genuinely statement: "Alpha intuition",
// unattributable; call-5's ghost nodeId is rescued onto Gamma. description: "Alpha intuition, elaborated: what alpha means and how it behaves in practice.",
findings.attribution.quiz = { "call-1": "p1", "call-2": null, "call-5": "p3" }; aliases: ["alpha feel"],
return findings; messages: ["a1", "a3"],
evidence: ["fixture evidence"],
understanding: "solid after the check",
},
{
id: "p2",
statement: "Beta rule",
description: "The beta rule and when it applies.",
aliases: [],
messages: ["a4"],
evidence: ["fixture evidence"],
},
{
id: "p3",
statement: "Gamma application",
description: "Applying the whole framework to gamma cases.",
aliases: [],
messages: ["a5"],
evidence: ["fixture evidence"],
},
],
edges: [
{
from: "p1",
to: "p2",
type: "follows",
messages: ["a4"],
evidence: ["fixture evidence"],
},
],
attribution: { quiz: { "call-1": "p1", "call-2": null, "call-3": "p1", "call-4": "p2", "call-5": "p3" } },
dropped: [],
};
} }
/** The chat every merger test folds (shared, so ordering is intentional). */ /** The chat every merger test folds (shared, so ordering is intentional). */
const chat1 = fixtureChat("Graph fixture", fixtureTree(), fixtureMessages(fixtureTree())); const chat1 = fixtureChat("Graph fixture", fixtureMessages());
/** Set by the cross-session test; the snapshot test (which runs after it) reads it. */ /** Set by the cross-session tests; the snapshot test (which runs after) reads them. */
let chat2Id = ""; let chat2Id = "";
let chat3Id = "";
// ─── the merger ────────────────────────────────────────────────────────────── // ─── the merger ──────────────────────────────────────────────────────────────
@@ -248,71 +226,83 @@ function scopedGraph(chatIds: string[]) {
); );
return { return {
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)), nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
edges: graph.edges.filter((e) => e.provenance.some((c) => chatIds.includes(c))), edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
events: graph.events.filter((e) => chatIds.includes(e.chatId)), events: graph.events.filter((e) => chatIds.includes(e.chatId)),
}; };
} }
describe("the merger (mergeFindings)", () => { describe("the merger (mergeFindings)", () => {
it("merges propositions/edges by identity and rebuilds events with retroactive probe attribution", () => { it("creates nodes from the verdicts, rebuilds quiz events deterministically, records observations", () => {
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps); const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 }); assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 });
const graph = scopedGraph([chat1.id]); const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3); assert.equal(graph.nodes.length, 3);
const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id])); const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id]));
for (const node of graph.nodes) { for (const node of graph.nodes) {
assert.equal(node.kind, "truth");
assert.deepEqual(node.aliases, []);
assert.match(node.id, /^n_/); assert.match(node.id, /^n_/);
assert.ok(node.description.length > 0, "description is required");
assert.equal("kind" in node, false, "no kind field");
assert.ok(Array.isArray(node.provenance), "message-granular provenance");
} }
// Provenance is message-granular: p1 came up in a1 and a3.
const alpha = graph.nodes.find((n) => n.statement === "Alpha intuition")!;
assert.deepEqual(alpha.provenance, [
{ chatId: chat1.id, messageId: "a1" },
{ chatId: chat1.id, messageId: "a3" },
]);
// The tree edge alpha→beta becomes a `requires` edge on the mapped nodes. // The follows edge, provenance attached.
assert.equal(graph.edges.length, 1); assert.equal(graph.edges.length, 1);
const edge = graph.edges[0]!; const edge = graph.edges[0]!;
assert.equal(edge.from, idByStatement.get("Alpha intuition")); assert.equal(edge.from, idByStatement.get("Alpha intuition"));
assert.equal(edge.to, idByStatement.get("Beta rule")); assert.equal(edge.to, idByStatement.get("Beta rule"));
assert.equal(edge.type, "requires"); assert.equal(edge.type, "follows");
assert.deepEqual(edge.provenance, [chat1.id]); assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "a4" }]);
// Mapping lines: one per (chat, sourceId).
assert.deepEqual(
graphStore.readMappings().map((m) => m.sourceId),
["p1", "p2", "p3"],
);
// Events — kinds, refs, notes, and the attribution paths. // Events — kinds, refs, notes, and the attribution paths.
assert.equal(graph.events.length, 6); assert.equal(graph.events.length, 6);
const byRef = new Map(graph.events.map((e) => [e.ref, e])); const byRef = new Map(graph.events.map((e) => [e.ref, e]));
const probeMiss = byRef.get("call-1")!; const miss = byRef.get("call-1")!;
assert.equal(probeMiss.kind, "probe-miss"); // quiz BEFORE plan_init assert.equal(miss.kind, "miss");
assert.equal(probeMiss.nodeId, idByStatement.get("Alpha intuition"), "retroactive probe attribution"); assert.equal(miss.nodeId, idByStatement.get("Alpha intuition"), "attributed quiz");
assert.equal(probeMiss.question, undefined, "attributed → no question text needed");
const probeDontKnow = byRef.get("call-2")!; const dontKnow = byRef.get("call-2")!;
assert.equal(probeDontKnow.kind, "dont-know"); // same kind, both phases assert.equal(dontKnow.kind, "dont-know");
assert.equal(probeDontKnow.nodeId, null, "unattributable probe → null"); assert.equal(dontKnow.nodeId, null, "unattributable → null");
assert.equal(probeDontKnow.question, "Q2 text"); assert.equal(dontKnow.question, "Q2 text");
const check = byRef.get("call-3")!; const correct = byRef.get("call-3")!;
assert.equal(check.kind, "check-correct"); // quiz AFTER plan_init assert.equal(correct.kind, "correct");
assert.equal(check.nodeId, idByStatement.get("Alpha intuition"), "tagged quiz → deterministic"); assert.equal(correct.nodeId, idByStatement.get("Alpha intuition"), "attribution wins over the ignored input.nodeId");
assert.equal(check.note, "recalled the diagram"); assert.equal(correct.note, "recalled the diagram");
const checkMiss = byRef.get("call-4")!; const miss2 = byRef.get("call-4")!;
assert.equal(checkMiss.kind, "check-incorrect"); assert.equal(miss2.kind, "miss");
assert.equal(checkMiss.nodeId, idByStatement.get("Beta rule")); assert.equal(miss2.nodeId, idByStatement.get("Beta rule"));
const ghost = byRef.get("call-5")!; const dontKnow2 = byRef.get("call-5")!;
assert.equal(ghost.kind, "dont-know"); assert.equal(dontKnow2.kind, "dont-know");
assert.equal(ghost.nodeId, idByStatement.get("Gamma application"), "tagged-but-unmapped rescued by attribution"); assert.equal(dontKnow2.nodeId, idByStatement.get("Gamma application"));
assert.equal(ghost.question, undefined);
const taught = byRef.get("taught:alpha")!; const observation = byRef.get("obs:p1")!;
assert.equal(taught.kind, "taught"); assert.equal(observation.kind, "observation");
assert.equal(taught.nodeId, idByStatement.get("Alpha intuition"), "mark_done → tree-node mapping"); assert.equal(observation.nodeId, idByStatement.get("Alpha intuition"));
assert.equal(observation.note, "solid after the check");
// The dismissed quiz and the ask question became nothing. // The dismissed quiz and the ask question became nothing.
assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0); assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0);
}); });
it("is idempotent — re-merging the same findings appends nothing", () => { it("is idempotent — re-merging the same verdicts appends nothing", () => {
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps); const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }); assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 });
const graph = scopedGraph([chat1.id]); const graph = scopedGraph([chat1.id]);
assert.equal(graph.nodes.length, 3); assert.equal(graph.nodes.length, 3);
@@ -322,38 +312,63 @@ describe("the merger (mergeFindings)", () => {
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2); assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
}); });
it("merges a second session onto the same nodes and accumulates edge provenance", () => { it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
// chat2 re-teaches the same two propositions (identical statements) — const alphaId = scopedGraph([chat1.id]).nodes.find((n) => n.statement === "Alpha intuition")!.id;
// the conservative identity resolution recognizes them. const chat2 = fixtureChat("Graph fixture 2", [
const tree2 = createTree(
[
{ id: "a", title: "Alpha intuition" },
{ id: "b", title: "Beta rule" },
],
[{ from: "a", to: "b" }],
);
const chat2 = fixtureChat("Graph fixture 2", tree2, [
user("u1", "again"), user("u1", "again"),
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree2)]), assistant("a1", [
quizPart(
"call-9",
{ question: "Q9", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-9", questionText: "Q9 text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
),
]),
]); ]);
const stats = mergeFindings(chat2.id, treeFindings(tree2), graphDeps);
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 });
chat2Id = chat2.id; chat2Id = chat2.id;
// The auditor's identity verdict: this proposition IS the alpha node.
const stats = mergeFindings(
chat2.id,
{
propositions: [
{
id: "q1",
statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings
description: "The intuitive grasp of alpha.",
aliases: ["alpha intuition"],
messages: ["a1"],
evidence: ["fixture evidence"],
linkTo: alphaId,
},
],
edges: [],
attribution: { quiz: { "call-9": "q1" } },
dropped: [],
},
graphDeps,
);
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 1 });
const after = scopedGraph([chat1.id, chat2.id]); const after = scopedGraph([chat1.id, chat2.id]);
assert.equal(after.nodes.length, 3); // no duplicate nodes assert.equal(after.nodes.length, 3, "no duplicate node");
assert.equal(after.edges.length, 1); // one merged edge… const alpha = getGraph(graphStore).nodes.find((n) => n.id === alphaId)!;
assert.deepEqual(after.edges[0]!.provenance, [chat1.id, chat2.id]); // …two sessions assert.ok(alpha.aliases.includes("Alpha feel"), "the session's phrasing joins the aliases");
assert.ok(alpha.aliases.includes("alpha intuition"));
assert.ok(
alpha.provenance.some((pt) => pt.chatId === chat2.id && pt.messageId === "a1"),
"provenance unions",
);
// The mapping records the verdict: (chat2, q1) → the linked node.
const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
}); });
it("derives node states, sessions, and node provenance in graphSnapshot", () => { it("derives node states and sessions in graphSnapshot", () => {
const snap = graphSnapshot(graphDeps); const snap = graphSnapshot(graphDeps);
const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id])); const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id]));
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "taught"); // Alpha's latest quiz is call-9 (chat2, correct) → known.
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "known");
assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed"); assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed");
// Gamma: no taught/checks — but the rescued dont-know probe (call-5)
// makes its latest event a dont-know.
assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know"); assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know");
assert.equal(snap.sessions[chat1.id], "Graph fixture"); assert.equal(snap.sessions[chat1.id], "Graph fixture");
@@ -366,8 +381,35 @@ describe("the merger (mergeFindings)", () => {
assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]); assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]);
}); });
it("never string-matches without a verdict — an identical statement without linkTo is a NEW node", () => {
const chat3 = fixtureChat("Graph fixture 3", [user("u1", "third")]);
chat3Id = chat3.id;
const stats = mergeFindings(
chat3.id,
{
propositions: [
{
id: "r1",
statement: "Alpha intuition", // IDENTICAL to the existing node's statement
description: "Same words, no verdict.",
aliases: [],
messages: ["u1"],
evidence: ["fixture evidence"],
// no linkTo — identity is the auditor's call alone
},
],
edges: [],
attribution: { quiz: {} },
dropped: [],
},
graphDeps,
);
assert.equal(stats.nodes, 1, "created — the merger does not match strings");
assert.equal(getGraph(graphStore).nodes.filter((n) => n.statement === "Alpha intuition").length, 2);
});
it("returns zeros for unknown chats", () => { it("returns zeros for unknown chats", () => {
assert.deepEqual(mergeFindings("chat_does_not_exist", treeFindings(fixtureTree()), graphDeps), { assert.deepEqual(mergeFindings("chat_does_not_exist", fixtureAudit(), graphDeps), {
nodes: 0, nodes: 0,
edges: 0, edges: 0,
events: 0, events: 0,
@@ -379,14 +421,12 @@ describe("the merger (mergeFindings)", () => {
describe("the trigger helper (pendingQuizAnswers)", () => { describe("the trigger helper (pendingQuizAnswers)", () => {
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => { it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
const tree = createTree([{ id: "solo", title: "Solo node" }], []); const chat = fixtureChat("Trigger fixture", [
const chat = fixtureChat("Trigger fixture", tree, [
user("u1", "go"), user("u1", "go"),
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]), assistant("a1", [
assistant("a2", [
quizPart( quizPart(
"call-s", "call-s",
{ question: "QS", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" }, { question: "QS", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }), answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
), ),
quizPart( quizPart(
@@ -407,7 +447,9 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending"); assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0); assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
mergeFindings(chat.id, treeFindings(tree), graphDeps); mergeFindings(chat.id, fixtureAudit(), graphDeps);
// call-s is unattributed here (fixtureAudit has no call-s attribution)
// — but it still becomes an event, so nothing stays pending.
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending"); assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
// A NEW answered quiz raises the count again (the re-trigger path). // A NEW answered quiz raises the count again (the re-trigger path).
@@ -416,7 +458,7 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
assistant("a3", [ assistant("a3", [
quizPart( quizPart(
"call-s4", "call-s4",
{ question: "QS4", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" }, { question: "QS4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }), answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
), ),
]), ]),
@@ -429,29 +471,43 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
describe("graph ledger store", () => { describe("graph ledger store", () => {
it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => { it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => {
const node = { id: "n_storetest", name: "Store test", statement: "Store test", aliases: [], kind: "truth" as const, createdAt: "2026-08-22T00:00:00.000Z" }; const node = {
id: "n_storetest",
statement: "Store test",
description: "Store test, elaborated.",
aliases: [] as string[],
createdAt: "2026-08-22T00:00:00.000Z",
provenance: [{ chatId: "chat_c", messageId: "m1" }],
};
graphStore.appendNode(node); graphStore.appendNode(node);
// A torn append and a legacy V0-shaped node (different schema) — both skipped. // A torn append and a legacy pre-redesign node (old schema) — both skipped.
fs.appendFileSync( fs.appendFileSync(
LEDGER_FILES.nodes, LEDGER_FILES.nodes,
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", domain: "x", state: "known" })}\n`, `{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", kind: "truth", name: "Legacy", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
"utf-8", "utf-8",
); );
const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy"); const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy");
assert.equal(nodes.length, 1); assert.equal(nodes.length, 1);
assert.equal(nodes[0]!.statement, "Store test"); assert.equal(nodes[0]!.statement, "Store test");
assert.equal(nodes[0]!.name, "Store test"); assert.deepEqual(nodes[0]!.provenance, [{ chatId: "chat_c", messageId: "m1" }]);
}); });
it("derives a name for pre-name ledger lines (legacy reads)", () => { it("returns RAW node lines — repeated ids (link-appends) are each their own line; getGraph merges", () => {
// A full, valid node line written before `name` existed. const line = (msgId: string) =>
fs.appendFileSync( JSON.stringify({
LEDGER_FILES.nodes, id: "n_linktest",
`${JSON.stringify({ id: "n_prename", statement: "Derivatives measure instantaneous change; they are limits.", aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z" })}\n`, statement: `Phrasing ${msgId}`,
"utf-8", description: "Elaboration.",
); aliases: [],
const node = graphStore.readNodes().find((n) => n.id === "n_prename"); createdAt: "2026-08-22T00:00:00.000Z",
assert.ok(node); provenance: [{ chatId: "chat_c", messageId: msgId }],
assert.equal(node.name, "Derivatives measure instantaneous change"); });
fs.appendFileSync(LEDGER_FILES.nodes, `${line("m1")}\n${line("m2")}\n`, "utf-8");
const lines = graphStore.readNodes().filter((n) => n.id === "n_linktest");
assert.equal(lines.length, 2, "the store returns raw lines, one per append");
const merged = getGraph(graphStore).nodes.find((n) => n.id === "n_linktest")!;
assert.equal(merged.statement, "Phrasing m1", "the FIRST line defines the node");
assert.ok(merged.aliases.includes("Phrasing m2"), "later phrasings join the aliases");
assert.deepEqual(merged.provenance.map((p) => p.messageId), ["m1", "m2"]);
}); });
}); });