Sync main (graph redesign b07c907): extraction observability ported onto the redesigned pipeline
Conflict resolution: extraction.ts takes main's redesign wholesale and
re-applies 072671f's observability deltas (stage durations, pipeline
failure line) — log shape only, no behavior change. graph-view.tsx's
legend/filters (094bafa) rekeyed from node.name (removed) to
statement/description, matching the new GraphNode shape.
This commit is contained in:
@@ -114,7 +114,7 @@ graph TB
|
||||
LEDG --> SNAP --> RGRAPH
|
||||
```
|
||||
|
||||
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer. The LLM stages propose and verify (propositions, edges, probe attribution — each with verbatim evidence); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs) from the transcript. The teacher stays a teacher; everything below the ledgers is derived.
|
||||
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer — tree-blind and post-hoc. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs, provenance) from the transcript. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger. The teacher stays a teacher; everything below the ledgers is derived.
|
||||
|
||||
## Layout
|
||||
|
||||
@@ -165,7 +165,7 @@ data/ chats/ · graph/ · jobs/ — runtime state, git
|
||||
|
||||
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts).
|
||||
|
||||
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM: propositions, edges, probe attribution, each with verbatim evidence) → auditor (LLM: drops anything unbacked) → merger (`mergeFindings`, deterministic: exact/alias identity resolution, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
|
||||
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM, tree-blind: propositions as true statements + descriptions, follows/supports edges, quiz attribution, understanding notes) → auditor (LLM: drops anything unbacked, stamps each survivor with an identity verdict — link to an existing node or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
|
||||
|
||||
## Testing
|
||||
|
||||
|
||||
@@ -81,11 +81,11 @@ const STATES_ORDER: NodeStateClass[] = ["taught", "known", "missed", "dont-know"
|
||||
|
||||
/**
|
||||
* Pure filter predicate for the legend/filters panel: a node is visible
|
||||
* when its state is toggled on AND (no query OR name/statement matches).
|
||||
* when its state is toggled on AND (no query OR statement matches).
|
||||
* Exported for tests; the sigma reducers just consume its verdict.
|
||||
*/
|
||||
export function isNodeVisible(
|
||||
node: Pick<GraphNode, "name" | "statement">,
|
||||
node: Pick<GraphNode, "statement" | "description">,
|
||||
state: NodeStateClass,
|
||||
hiddenStates: ReadonlySet<NodeStateClass>,
|
||||
query: string,
|
||||
@@ -93,7 +93,7 @@ export function isNodeVisible(
|
||||
if (hiddenStates.has(state)) return false;
|
||||
const q = query.trim().toLowerCase();
|
||||
if (!q) return true;
|
||||
return node.name.toLowerCase().includes(q) || node.statement.toLowerCase().includes(q);
|
||||
return node.statement.toLowerCase().includes(q) || node.description.toLowerCase().includes(q);
|
||||
}
|
||||
|
||||
// ---- Label drawing (canvas) ---------------------------------------------
|
||||
@@ -434,7 +434,7 @@ export function GraphView() {
|
||||
// frozen layout stable for a given node set across remounts.
|
||||
const rand = seededRandom(node.id);
|
||||
graph.addNode(node.id, {
|
||||
label: node.name,
|
||||
label: node.statement,
|
||||
size: STATE_TOKENS[state].size,
|
||||
color: stateColors.get(state) ?? edgeColor,
|
||||
x: (rand() - 0.5) * 100,
|
||||
@@ -694,8 +694,8 @@ export function GraphView() {
|
||||
style={{ visibility: "hidden", left: 0, top: 0 }}
|
||||
className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg"
|
||||
>
|
||||
<div className="text-sm font-medium leading-snug">{selected.node.name}</div>
|
||||
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.statement}</div>
|
||||
<div className="text-sm font-medium leading-snug">{selected.node.statement}</div>
|
||||
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.description}</div>
|
||||
<div className="mt-1 text-xs uppercase tracking-wide text-muted-foreground">
|
||||
{STATE_LABELS[selected.state]}
|
||||
</div>
|
||||
|
||||
@@ -75,11 +75,11 @@ chat record ──▶ EXTRACTOR (LLM) ──▶ AUDITOR (LLM) ──▶ MERGER (
|
||||
proposes verifies applies
|
||||
```
|
||||
|
||||
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned* (propositions, incl. misconceptions and prose-only `supports` edges), *what depends on what*, and *which proposition an untagged probe tested*. Everything the model could silently mis-copy — event kinds, timestamps, idempotency refs, notes, question texts — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (the faithfulness net) drops anything unbacked and logs every drop. Conservative throughout: duplicates over wrong merges, drop over invent.
|
||||
**Tree-blind and post-hoc** (2026-09-02 redesign, docs/vision.md §1): the extractor reads ONLY the transcript — never the session's plan tree — and proposes free-form propositions (true statements with descriptions, misconception-free: a misconception is stated as its true correction), `follows`/`supports` edges, per-proposition understanding notes, and message-granular provenance. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger.
|
||||
|
||||
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — statement/alias exact-match identity resolution (fuzzy/embedding adjudication deferred), one mapping line per (chat, graph node) as the merge audit trail, edge dedupe by `(from, to, type)` with per-chat provenance, event dedupe by chat-scoped ref, and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
|
||||
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned*, *what follows/supports what*, *which proposition each quiz tested* (retroactively, against the whole transcript), and — the auditor's — *identity*: whether a proposition IS an existing graph node (link-or-create). Everything the model could silently mis-copy — event kinds (`correct` / `miss` / `dont-know`), timestamps, idempotency refs, notes, provenance points — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (which sees the extractor's output PLUS the current graph) drops anything unbacked, logs every drop, and stamps each surviving proposition with its identity verdict (`linkTo`). Conservative throughout: duplicates over wrong merges, drop over invent.
|
||||
|
||||
**Probe attribution** (vision open question, resolved structurally): tagged events (quiz `nodeId`, `plan_mark_done`'s tree node) resolve deterministically through the proposition mapping. The extractor's judgment is consulted only for **untagged probe quizzes** — retroactively, against the whole transcript, so a pre-tree probe maps to a proposition the tree only articulates later. Attribution can also rescue a tagged-but-unknown id; genuinely unattributable probes land `nodeId: null` with the question text, exactly as they always have.
|
||||
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — identity comes from the auditor's `linkTo` verdict alone (validated against the ledger; never string matching), one mapping line per (chat, sourceId) as the verdict audit trail (which also makes re-merging the same chat idempotent), link-appends: a linked proposition appends a node line whose phrasings join the node's `aliases` and whose provenance unions (reads merge by id), edge dedupe by `(from, to, type)`-already-asserted-by-this-chat with message-granular provenance, event dedupe by chat-scoped ref (quiz toolCallId, `obs:<sourceId>` for understanding observations), and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
|
||||
|
||||
**The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes.
|
||||
|
||||
|
||||
@@ -1,20 +1,19 @@
|
||||
# The learner's knowledge graph
|
||||
|
||||
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes and plan trees, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Requires** means *a* must be understood before *b*.
|
||||
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
|
||||
|
||||
The states, exactly:
|
||||
|
||||
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
|
||||
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
|
||||
- **[known]** — answered correctly. Foundation you can build on and refer to by name.
|
||||
- **[taught]** — a lesson covered it. Check that it stuck before stacking anything heavy on it.
|
||||
- **[candidate]** — asserted into a plan but never probed. Assume nothing.
|
||||
- **[candidate]** — present in the graph but never probed. Assume nothing.
|
||||
|
||||
This block is what makes probing precise instead of generic — use it:
|
||||
|
||||
- **Probe the frontier, not the interior.** [known] and [taught] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
|
||||
- **Probe the frontier, not the interior.** [known] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
|
||||
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
|
||||
- **Read the Requires chains as scaffolding.** They show what the learner's knowledge rests on. If a prerequisite under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
|
||||
- **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
|
||||
|
||||
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
|
||||
|
||||
|
||||
+156
-118
@@ -1,23 +1,28 @@
|
||||
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
|
||||
import { z } from "zod";
|
||||
import { modelFor } from "../models.ts";
|
||||
import type { AuditResult, ChatRecord, SessionFindings } from "../../core/contracts.ts";
|
||||
import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
|
||||
import type { AuditResult, ChatRecord, GraphNode, SessionFindings } from "../../core/contracts.ts";
|
||||
import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
|
||||
|
||||
/**
|
||||
* The extraction pipeline (vision §1 construction) — the sole graph writer.
|
||||
* Three stages, one orchestrator:
|
||||
* The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
|
||||
* sole graph writer. Three stages, one orchestrator:
|
||||
*
|
||||
* extractor (LLM) transcript → SessionFindings (propositions, edges,
|
||||
* event attribution — each with verbatim evidence)
|
||||
* auditor (LLM) findings → AuditResult (the verified subset + drops)
|
||||
* extractor (LLM) transcript → SessionFindings (free-form propositions,
|
||||
* follows/supports edges, quiz attribution, understanding
|
||||
* notes — each with verbatim evidence). TREE-BLIND: it
|
||||
* sees only the transcript, never the plan tree.
|
||||
* auditor (LLM) findings + THE CURRENT GRAPH → AuditResult: the
|
||||
* verified subset with an identity verdict per
|
||||
* proposition (linkTo an existing node, or create), plus
|
||||
* the drops for the log.
|
||||
* merger (core) mergeFindings() → deterministic ledger appends
|
||||
*
|
||||
* The LLM stages supply only the judgment calls; the deterministic rebuild
|
||||
* in the merger owns everything that must never be mis-copied (event kinds,
|
||||
* timestamps, refs). Both agents are single-tool-call workers: their whole
|
||||
* deliverable is one zod-validated submit call — a ToolLoopAgent purely so
|
||||
* the model can retry a schema-invalid call.
|
||||
* timestamps, refs, provenance points). Both agents are single-tool-call
|
||||
* workers: their whole deliverable is one zod-validated submit call — a
|
||||
* ToolLoopAgent purely so the model can retry a schema-invalid call.
|
||||
*
|
||||
* Created via createExtractionPipeline(deps) — the composition root
|
||||
* (chat-handler) injects the storage ports + models; this module never
|
||||
@@ -32,7 +37,7 @@ import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/gr
|
||||
* stats (or failed) for every run.
|
||||
*/
|
||||
|
||||
// ─── the transcript view (what the LLM sees) ────────────────────────────────
|
||||
// ─── the transcript view (what the extractor + auditor see) ─────────────────
|
||||
|
||||
/**
|
||||
* A quiz tool part's outcome, one line: the attribution evidence.
|
||||
@@ -49,46 +54,35 @@ function quizLine(part: { toolCallId: string; input: unknown; output: unknown })
|
||||
}
|
||||
|
||||
/**
|
||||
* The compact transcript the extractor + auditor both see: user/teacher
|
||||
* text verbatim (evidence quotes must match it), the final plan tree, and
|
||||
* the structured tool events (quiz outcomes, mark_done). Job tool parts
|
||||
* (research/demo/visual) are noise for extraction — their outcomes are
|
||||
* summarized as one line each.
|
||||
* The compact transcript both LLM stages see: user/teacher text VERBATIM,
|
||||
* each tagged with its message id (provenance anchors — the extractor
|
||||
* reports them back per proposition/edge), and quiz outcome lines.
|
||||
* TREE-BLIND by design (2026-09-02): no plan tree, no mark_done —
|
||||
* extraction is free-form, decoupled from the session's plan. Job tool
|
||||
* parts (research/demo/visual) are noise for extraction — their outcomes
|
||||
* are summarized as one line each.
|
||||
*/
|
||||
export function transcriptView(record: ChatRecord): string {
|
||||
const lines: string[] = [];
|
||||
lines.push(`SESSION: ${record.title}`);
|
||||
|
||||
if (record.tree && record.tree.nodes.length > 0) {
|
||||
const nodes = record.tree.nodes.map((n) => `${n.id} = ${n.title}`).join("; ");
|
||||
lines.push(`FINAL PLAN TREE (node ids): ${nodes}`);
|
||||
if (record.tree.edges.length > 0) {
|
||||
const edges = record.tree.edges.map((e) => `${e.from} requires ${e.to}`).join("; ");
|
||||
lines.push(`TREE EDGES: ${edges}`);
|
||||
}
|
||||
}
|
||||
|
||||
for (const message of record.messages as UIMessage[]) {
|
||||
record.messages.forEach((message, index) => {
|
||||
const who = message.role === "user" ? "LEARNER" : "TEACHER";
|
||||
for (const part of message.parts) {
|
||||
const mid = (message as UIMessage).id || `m${index}`;
|
||||
for (const part of (message as UIMessage).parts) {
|
||||
if (part.type === "text" && part.text.trim()) {
|
||||
lines.push(`${who}: ${part.text.trim()}`);
|
||||
lines.push(`${who} [${mid}] ${part.text.trim()}`);
|
||||
} else if (part.type.startsWith("tool-")) {
|
||||
const name = part.type.slice("tool-".length);
|
||||
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
|
||||
if (name === "quiz" && p.state === "output-available") {
|
||||
lines.push(quizLine(p));
|
||||
} else if (name === "plan_mark_done" && p.state === "output-available") {
|
||||
const input = p.input as { id?: unknown } | undefined;
|
||||
lines.push(`TEACHER marked tree node done [${p.toolCallId}]: ${typeof input?.id === "string" ? input.id : "?"}`);
|
||||
} else if (name.startsWith("plan_")) {
|
||||
// plan ops are folded into the FINAL PLAN TREE line above
|
||||
} else if (p.state === "output-available") {
|
||||
lines.push(`(${name} tool result delivered)`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
@@ -99,87 +93,98 @@ const evidence = z
|
||||
.min(1)
|
||||
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
|
||||
|
||||
const messagesField = z
|
||||
.array(z.string())
|
||||
.describe("The message ids (from the [id] tags in the transcript) where this came up — provenance.");
|
||||
|
||||
const propositionSchema = z.object({
|
||||
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
|
||||
statement: z
|
||||
.string()
|
||||
.describe(
|
||||
"The proposition as one reasoning step — a TRUE, self-contained, world-facing statement. " +
|
||||
"Never reference the learner, the teacher, or the session: not \"the learner understands X\" but " +
|
||||
'"X". A misconception is stated as the TRUE correction ("Nietzsche did not endorse ' +
|
||||
'might-makes-right social Darwinism") — the wrongness lives in the quiz outcomes, never here.',
|
||||
),
|
||||
description: z
|
||||
.string()
|
||||
.describe(
|
||||
"The elaborated version of the same proposition — more words: scope, precision, what exactly it means. Still true, still world-facing.",
|
||||
),
|
||||
aliases: z.array(z.string()).describe("Verbatim phrasings this session used for the same proposition, if any."),
|
||||
messages: messagesField,
|
||||
evidence,
|
||||
understanding: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
'One note about THIS session\'s evidence of the learner\'s grasp of this proposition, on a named-word scale ' +
|
||||
'("solid", "shaky", "accepted", "resisted", "not yet engaged") — no numbers, no predictions.',
|
||||
),
|
||||
});
|
||||
|
||||
const edgeSchema = z.object({
|
||||
from: z.string().describe("Proposition id that is followed / supported."),
|
||||
to: z.string().describe("Proposition id that follows / is supported by `from`."),
|
||||
type: z
|
||||
.enum(["follows", "supports"])
|
||||
.describe('"follows" = to is derivable from from; "supports" = from grounds to without entailing it.'),
|
||||
messages: messagesField,
|
||||
evidence,
|
||||
});
|
||||
|
||||
const findingsSchema = z.object({
|
||||
propositions: z
|
||||
.array(
|
||||
z.object({
|
||||
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
|
||||
name: z.string().describe(
|
||||
"A short 2-6 word name for the fact — its header, used for rendering. Not a sentence.",
|
||||
),
|
||||
statement: z
|
||||
.string()
|
||||
.describe(
|
||||
"The proposition as one reasoning step — an OBJECTIVE, world-facing fact. " +
|
||||
"Never reference the learner, the teacher, or the session: not \"the learner " +
|
||||
'understands X\" but \"X\"; not \"the learner thinks Y\" but the wrong idea itself, ' +
|
||||
'stated as a claim (\"Reading: Nietzsche endorsed might-makes-right social Darwinism\"), ' +
|
||||
"with kind \"misconception\" carrying that it is wrong.",
|
||||
),
|
||||
aliases: z.array(z.string()).describe("Verbatim alternate phrasings from this session, if any."),
|
||||
kind: z.enum(["truth", "misconception"]).describe("misconception = a wrong idea the learner holds or was warned about."),
|
||||
treeNodeId: z.string().optional().describe("The plan-tree node id this came from, when it did."),
|
||||
evidence,
|
||||
}),
|
||||
)
|
||||
.describe("Every distinct proposition the session touched — from the plan tree AND from prose."),
|
||||
.array(propositionSchema)
|
||||
.describe("Every distinct proposition the session touched — true statements, misconception-free."),
|
||||
edges: z
|
||||
.array(
|
||||
z.object({
|
||||
from: z.string().describe("Proposition id that is the prerequisite / support."),
|
||||
to: z.string().describe("Proposition id that depends on / is supported by `from`."),
|
||||
type: z.enum(["requires", "supports"]),
|
||||
evidence,
|
||||
}),
|
||||
)
|
||||
.describe("Relations between propositions. `requires` = prerequisite; `supports` = helpful but not prerequisite. Include relations the plan tree does NOT show when the transcript articulates them."),
|
||||
.array(edgeSchema)
|
||||
.describe("Relations between propositions: follows (derivable) / supports (grounds without entailing)."),
|
||||
attribution: z
|
||||
.object({
|
||||
quiz: z
|
||||
.record(z.string(), z.string().nullable())
|
||||
.describe("toolCallId of every UNTAGGED quiz line → the proposition id it was really testing. Probe questions asked BEFORE the tree existed are attributed RETROACTIVELY — the whole transcript is visible; map them to the proposition they were probing (often articulated only later). null when genuinely unattributable. Tagged quizzes (with a tree nodeId in the transcript view) resolve automatically — do not include them."),
|
||||
.describe(
|
||||
"toolCallId of EVERY quiz line → the proposition id it was really testing. Attribute RETROACTIVELY — the whole transcript is visible; map each quiz to the proposition it tests (often articulated elsewhere in the session). null when genuinely unattributable.",
|
||||
),
|
||||
})
|
||||
.describe("Probe attribution — your only event judgment."),
|
||||
.describe("Quiz attribution — your only event judgment."),
|
||||
});
|
||||
|
||||
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
|
||||
transcript and report what was learned — as structured findings for the learner's global knowledge graph.
|
||||
|
||||
The transcript view shows: the session's final plan tree (dependency map, with node ids), the conversation
|
||||
verbatim (LEARNER / TEACHER), and structured events (QUIZ lines with the learner's outcome, mark_done lines).
|
||||
The transcript view shows: the conversation verbatim (LEARNER / TEACHER lines, each tagged with its message id)
|
||||
and QUIZ outcome lines.
|
||||
|
||||
Report, with verbatim evidence quotes for everything:
|
||||
|
||||
NODE THEORY — the graph is a set of objective facts, full stop. A node is what is true (or falsely
|
||||
believed) about the world, never anything about the learner or the session. The learner's
|
||||
relationship to a fact — whether they know it, missed it, or hold it as a misconception — is
|
||||
expressed ENTIRELY by the quiz-outcome events you attribute; it never leaks into the node itself.
|
||||
So no "the learner understands X", no "discussed in this session": the node is just X. A
|
||||
misconception node states the wrong idea itself as a claim, with kind "misconception" marking it
|
||||
wrong — never "the learner believes Y".
|
||||
NODE THEORY — the graph is a set of TRUE propositions about the world, full stop. A node is never anything
|
||||
about the learner or the session. The learner's relationship to a proposition — whether they know it, missed
|
||||
it, resisted it — is expressed ENTIRELY by the quiz outcomes and your understanding note; it never leaks into
|
||||
the node itself. So no "the learner understands X", no "discussed in this session": the node is just X. A
|
||||
misconception is NOT a node of its own wrong claim — state the TRUE correction as the proposition, and let the
|
||||
quiz outcomes carry that the learner believed the wrong version.
|
||||
|
||||
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
|
||||
- Each carries a "name": a short 2-6 word header (the rendering handle), and a "statement": the
|
||||
full fact as one objective reasoning step.
|
||||
- Plan-tree nodes become propositions (pass their treeNodeId) — statement from the node title, enriched to a
|
||||
complete proposition if the prose elaborates it.
|
||||
- Propositions that appeared only in prose (teacher explanations, learner questions) get fresh ids.
|
||||
- Include MISCONCEPTIONS — wrong ideas the learner showed or the teacher warned against — with kind
|
||||
"misconception".
|
||||
- Each is a self-contained TRUE statement ("statement"), plus its elaboration ("description" — more
|
||||
words, same proposition: scope, precision, what exactly it means).
|
||||
- Include corrections of misconceptions the teacher articulated, and true facts explained in prose.
|
||||
- One reasoning step per proposition: never a topic blob, never a sub-step.
|
||||
- aliases: verbatim alternate phrasings this session used for the same thing.
|
||||
- "messages": the [id] tags of every message where the proposition came up — provenance, report it.
|
||||
- "aliases": verbatim phrasings this session used for the same proposition.
|
||||
- "understanding": ONE named-word-scale note on the evidence of the learner's grasp, for this session
|
||||
only ("solid", "shaky", "accepted", "resisted", "not yet engaged"). No numbers, no predictions.
|
||||
|
||||
2. EDGES between propositions:
|
||||
- The tree's dependency edges are the backbone: from = prerequisite, to = dependent, type "requires".
|
||||
- Also include relations articulated only in prose. Use "supports" when the transcript presents something as
|
||||
helpful but NOT a prerequisite.
|
||||
- "follows": to is DERIVABLE from from (understanding from makes understanding to achievable).
|
||||
- "supports": from GROUNDS to without entailing it (helpful context, not a prerequisite).
|
||||
- Only relations the transcript actually articulates — with evidence.
|
||||
|
||||
3. ATTRIBUTION — UNTAGGED quizzes only:
|
||||
- QUIZ lines whose question carries a tree nodeId resolve automatically; do not attribute those.
|
||||
- The others are PROBE-phase questions (asked before the tree was initialized) that test prerequisite
|
||||
understanding. Attribute them RETROACTIVELY to the proposition they were really probing — even if that
|
||||
proposition is only articulated later in the session. The QUIZ question text is your guide. If you truly
|
||||
cannot tell, use null.
|
||||
3. ATTRIBUTION — every QUIZ line gets a mapping: toolCallId → the proposition id it was really testing.
|
||||
Attribute RETROACTIVELY — a quiz often tests a proposition the session only articulates later; the
|
||||
question text is your guide. If you truly cannot tell, use null.
|
||||
|
||||
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
|
||||
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
|
||||
@@ -210,6 +215,16 @@ function createExtractor(model: LanguageModel) {
|
||||
// ─── the auditor ───────────────────────────────────────────────────────────
|
||||
|
||||
const auditSchema = findingsSchema.extend({
|
||||
propositions: z.array(
|
||||
propositionSchema.extend({
|
||||
linkTo: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
"The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. When unsure, omit: duplicates over wrong merges.",
|
||||
),
|
||||
}),
|
||||
),
|
||||
dropped: z
|
||||
.array(
|
||||
z.object({
|
||||
@@ -221,27 +236,46 @@ const auditSchema = findingsSchema.extend({
|
||||
});
|
||||
|
||||
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
|
||||
ONE session transcript and a set of proposed findings (propositions, edges, event attribution). Your job is the
|
||||
faithfulness net: verify every candidate against the transcript, keep only what is backed, and return the
|
||||
verified subset.
|
||||
ONE session transcript, a set of proposed findings (propositions, edges, quiz attribution), and the CURRENT
|
||||
knowledge graph. Your jobs: the faithfulness net, and identity.
|
||||
|
||||
Verify:
|
||||
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement must
|
||||
be a faithful reading of it. The node theory applies: a statement must be an OBJECTIVE, world-facing fact —
|
||||
reword learner- or session-relative statements ("the learner understands X" → "X") or drop them when the
|
||||
objective fact is not actually backed. Drop invented or bloated statements. Conservative: when two proposed propositions
|
||||
are really one, keep the better-stated one and drop the other.
|
||||
- EDGES: the evidence must actually articulate the relation, in the direction claimed (from = prerequisite).
|
||||
Drop direction-swapped or speculative edges.
|
||||
- ATTRIBUTION: only UNTAGGED quizzes carry attribution — the QUIZ line's question must genuinely test the
|
||||
attributed proposition. Drop wrong attributions to null, or re-point them to the proposition the question
|
||||
actually tests.
|
||||
FAITHFULNESS — verify every candidate against the transcript:
|
||||
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement
|
||||
must be a faithful reading of it. The node theory applies: a statement must be a TRUE, self-contained,
|
||||
world-facing fact — reword learner- or session-relative statements ("the learner understands X" → "X"), state
|
||||
misconceptions as the true correction, or drop them when no objective fact is backed. Drop invented or
|
||||
bloated statements. Conservative: when two proposed propositions are really one, keep the better-stated one
|
||||
and drop the other.
|
||||
- EDGES: the evidence must actually articulate the relation, in the direction claimed. Drop
|
||||
direction-swapped or speculative edges.
|
||||
- ATTRIBUTION: the QUIZ line's question must genuinely test the attributed proposition. Drop wrong
|
||||
attributions to null, or re-point them to the proposition the question actually tests.
|
||||
- UNDERSTANDING notes: verify against the quiz outcomes and the session's evidence; reword to a named-word
|
||||
scale or drop.
|
||||
|
||||
IDENTITY — link-or-create, per proposition:
|
||||
- You see the current graph (every node: id, statement, description, aliases). For each verified proposition,
|
||||
decide: does an EXISTING node already state this same proposition?
|
||||
- If yes: set "linkTo" to that node's id. The session's phrasings will join its aliases and its provenance.
|
||||
- If no: omit "linkTo" and a new node is created.
|
||||
- When unsure, omit — duplicates are recoverable later; wrong merges are not.
|
||||
|
||||
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
|
||||
well-evidenced proposition is still knowledge.
|
||||
|
||||
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
|
||||
(the same shapes you received) plus the dropped list.`;
|
||||
(the same shapes you received, with linkTo added) plus the dropped list.`;
|
||||
|
||||
/** The current graph, compact — the auditor's identity context. */
|
||||
function graphView(nodes: GraphNode[]): string {
|
||||
if (nodes.length === 0) return "(the graph is currently empty — every proposition is a new node)";
|
||||
const lines = nodes.map(
|
||||
(n) =>
|
||||
`- ${n.id} :: ${n.statement}\n description: ${n.description}` +
|
||||
(n.aliases.length > 0 ? `\n aliases: ${n.aliases.join(" | ")}` : ""),
|
||||
);
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function createAuditor(model: LanguageModel) {
|
||||
return new ToolLoopAgent({
|
||||
@@ -249,7 +283,7 @@ function createAuditor(model: LanguageModel) {
|
||||
instructions: AUDITOR_INSTRUCTIONS,
|
||||
tools: {
|
||||
submit_review: tool({
|
||||
description: "Submit the verified findings subset + the dropped list.",
|
||||
description: "Submit the verified findings subset (with identity verdicts) + the dropped list.",
|
||||
inputSchema: auditSchema,
|
||||
execute: async (review) => review,
|
||||
}),
|
||||
@@ -302,18 +336,21 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
||||
throw new Error("the extractor never submitted findings");
|
||||
}
|
||||
|
||||
async function audit(chatId: string, findings: SessionFindings): Promise<SessionFindings> {
|
||||
async function audit(chatId: string, findings: SessionFindings): Promise<AuditResult> {
|
||||
const record = chatStore.getChat(chatId);
|
||||
if (!record) throw new Error(`Chat not found: ${chatId}`);
|
||||
const view = transcriptView(record);
|
||||
const graph = getGraph(graphStore);
|
||||
console.info(
|
||||
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
|
||||
`${Object.keys(findings.attribution.quiz).length} attributions)`,
|
||||
`${Object.keys(findings.attribution.quiz).length} attributions, ${graph.nodes.length} graph nodes)`,
|
||||
);
|
||||
const t0 = performance.now();
|
||||
|
||||
const review = await auditorAgent.generate({
|
||||
prompt: `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}`,
|
||||
prompt:
|
||||
`Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}\n\n` +
|
||||
`Current knowledge graph (${graph.nodes.length} nodes):\n\n${graphView(graph.nodes)}`,
|
||||
});
|
||||
let auditResult: AuditResult | null = null;
|
||||
for (const part of review.content) {
|
||||
@@ -328,13 +365,13 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
||||
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
|
||||
);
|
||||
}
|
||||
const { dropped: _dropped, ...verified } = auditResult;
|
||||
console.info(
|
||||
`[extraction ${chatId}] audit kept ${verified.propositions.length}/${findings.propositions.length} propositions, ` +
|
||||
`${verified.edges.length}/${findings.edges.length} edges`,
|
||||
`[extraction ${chatId}] audit kept ${auditResult.propositions.length}/${findings.propositions.length} propositions, ` +
|
||||
`${auditResult.edges.length}/${findings.edges.length} edges, ` +
|
||||
`${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes`,
|
||||
);
|
||||
console.info(`[extraction ${chatId}] auditor done in ${Math.round(performance.now() - t0)}ms`);
|
||||
return verified;
|
||||
return auditResult;
|
||||
}
|
||||
|
||||
async function runExtraction(chatId: string): Promise<EmissionStats> {
|
||||
@@ -344,10 +381,10 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
||||
try {
|
||||
console.info(`[extraction ${chatId}] pipeline start`);
|
||||
const findings = await extract(chatId);
|
||||
const verified = await audit(chatId, findings);
|
||||
const auditResult = await audit(chatId, findings);
|
||||
// The merger re-reads the chat record itself — events are rebuilt
|
||||
// from the transcript, never from stale copies.
|
||||
const stats = mergeFindings(chatId, verified, { chatStore, graphStore });
|
||||
const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore });
|
||||
console.info(
|
||||
`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events ` +
|
||||
`(${Math.round(performance.now() - t0)}ms total)`,
|
||||
@@ -365,3 +402,4 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
||||
|
||||
return { runExtraction };
|
||||
}
|
||||
|
||||
|
||||
+130
-79
@@ -1,50 +1,69 @@
|
||||
/**
|
||||
* Knowledge-graph contracts — nodes, ledgers, extraction findings
|
||||
*
|
||||
* The minimal implementation of the graph vision (docs/vision.md §1, the
|
||||
* 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
|
||||
* (derivable) and supports (grounds without entailing); provenance is
|
||||
* message-granular; extraction is free-form and post-hoc, fully decoupled
|
||||
* from the session's plan tree; the event ledger records quiz outcomes plus
|
||||
* the extractor's per-proposition understanding observations. Anything not
|
||||
* yet decided (the state model, strength derivation, understanding
|
||||
* treatment, selection strategies) is deliberately NOT here — transitional
|
||||
* stubs carry the renderer until those land.
|
||||
*
|
||||
* Part of the domain-split contracts (src/core/contracts/): each domain file
|
||||
* is owned outright by its stream — cross-domain schema changes go through
|
||||
* master. Pure types + protocol docs + tiny pure helpers; type-only imports
|
||||
* of "ai" (safe everywhere: server, client, tests). See docs/vision.md.
|
||||
* master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md.
|
||||
*/
|
||||
// ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
|
||||
|
||||
export type EdgeType = "requires" | "supports";
|
||||
/** Two relations only (DECIDED 2026-09-02): follows = b is derivable from a;
|
||||
* supports = a grounds b without entailing it. No scalars, no encoded
|
||||
* strength — strength is derived at read time from topology. */
|
||||
export type EdgeType = "follows" | "supports";
|
||||
|
||||
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
|
||||
* edges record every (chat, message) where they came up, deep-linkable to
|
||||
* the exact transcript point. */
|
||||
export interface ProvenancePoint {
|
||||
chatId: string;
|
||||
messageId: string;
|
||||
}
|
||||
|
||||
export interface GraphNode {
|
||||
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */
|
||||
id: string;
|
||||
/** Short name / header for the fact (2-6 words) — the rendering handle.
|
||||
* NODE THEORY: a node is an OBJECTIVE fact, never learner- or
|
||||
* session-relative. No "the learner understands…", no "this session
|
||||
* covered…" — just the fact, world-facing. The learner's relationship
|
||||
* to the fact lives ENTIRELY in the event ledger: state labels are
|
||||
* derived from quiz outcomes, never written into the node. */
|
||||
name: string;
|
||||
/** Canonical proposition statement — the full fact, one reasoning step,
|
||||
* also objective (no learner / teacher / session references). */
|
||||
/** The minimal proposition — one true, self-contained statement. This is
|
||||
* BOTH the rendering label AND the identity key: the auditor recognizes
|
||||
* a re-encounter by judgment (link-or-create), not by string matching. */
|
||||
statement: string;
|
||||
/** Verbatim alternate statements from sessions (merge target audit trail). */
|
||||
/** The elaborated version — required. Same proposition, more words:
|
||||
* scope, precision, the "what exactly does it mean". */
|
||||
description: string;
|
||||
/** Verbatim phrasings sessions used for this proposition (the identity
|
||||
* audit trail — how the world said what the node says). */
|
||||
aliases: string[];
|
||||
kind: "truth" | "misconception";
|
||||
createdAt: string;
|
||||
/** Every (chat, message) point where this proposition came up. */
|
||||
provenance: ProvenancePoint[];
|
||||
}
|
||||
|
||||
export interface GraphEdge {
|
||||
/** Prerequisite GraphNode id. */
|
||||
from: string;
|
||||
to: string;
|
||||
type: EdgeType;
|
||||
/** Chat ids that asserted this edge (teacher may simplify — auditable). */
|
||||
provenance: string[];
|
||||
/** Every (chat, message) point where this relation was asserted. */
|
||||
provenance: ProvenancePoint[];
|
||||
}
|
||||
|
||||
/** mappings.jsonl line — the session→graph identity map: the extraction
|
||||
* pipeline's proposition (identified by its source id) → the graph node it
|
||||
* merged into. The merge audit trail: one line per (chat, graph node). */
|
||||
* pipeline's proposition (identified by its run-local source id) → the
|
||||
* graph node it resolved to (created or linked). The audit trail of the
|
||||
* auditor's identity verdicts; also the chat-scoped resolution table the
|
||||
* merger uses to point events and edges at node ids. */
|
||||
export interface NodeMapping {
|
||||
chatId: string;
|
||||
/** The writer's local id for the proposition — a plan-tree node id when
|
||||
* tree-derived, else the extractor's proposition id ("p1", …). */
|
||||
/** The extractor's run-local proposition id ("p1", …). */
|
||||
sourceId: string;
|
||||
graphNodeId: string;
|
||||
}
|
||||
@@ -57,23 +76,26 @@ export interface SessionCheckpoint {
|
||||
foldedAt: string;
|
||||
}
|
||||
|
||||
export type LedgerEventKind =
|
||||
| "probe-correct"
|
||||
| "probe-miss"
|
||||
| "dont-know"
|
||||
| "misconception-signaled"
|
||||
| "taught"
|
||||
| "check-correct"
|
||||
| "check-incorrect";
|
||||
/**
|
||||
* Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
|
||||
* dont-know — plus `observation`, the extractor's per-proposition
|
||||
* understanding note joining the ledger as a session-scoped observation
|
||||
* (named-word scale; treatment pending the state-model ideation). No
|
||||
* `taught` events at all; no misconception kind — a misconception is a
|
||||
* TRUE correction in the graph, and the wrongness lives here, in the
|
||||
* ledger, until the state model exists.
|
||||
*/
|
||||
export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
|
||||
|
||||
export interface LedgerEvent {
|
||||
/** null = not yet attributed (untagged probe events; consolidation assigns). */
|
||||
/** null = not yet attributed. */
|
||||
nodeId: string | null;
|
||||
kind: LedgerEventKind;
|
||||
chatId: string;
|
||||
at: string;
|
||||
/** Idempotency key: the quiz part's toolCallId or `taught:<treeNodeId>` —
|
||||
* re-emission of the same chat must never double-record an event. */
|
||||
/** Idempotency key: the quiz part's toolCallId, or `obs:<sourceId>` for
|
||||
* understanding observations — re-emission of the same chat must never
|
||||
* double-record an event. */
|
||||
ref?: string;
|
||||
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */
|
||||
note?: string;
|
||||
@@ -83,91 +105,121 @@ export interface LedgerEvent {
|
||||
|
||||
/**
|
||||
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
|
||||
* (nodes/edges/events/mappings/sessions). The derived graph is rebuilt by
|
||||
* reading the ledgers — store events, derive state. sessions.jsonl is the
|
||||
* extraction checkpoint: which chats the extraction pipeline has processed,
|
||||
* and through which updatedAt (never extract the same turns twice).
|
||||
* (nodes/edges/events/mappings/sessions), all append-only.
|
||||
*
|
||||
* nodes.jsonl one line per node — the FIRST line for an id defines
|
||||
* statement/description/createdAt; later lines for the
|
||||
* same id are link-appends (a session's re-encounter):
|
||||
* their statements/aliases join `aliases`, their
|
||||
* provenance unions. getGraph merges by id.
|
||||
* edges.jsonl one line per edge ASSERTION; readers merge by
|
||||
* (from,to,type), unioning provenance.
|
||||
*
|
||||
* The derived graph is rebuilt by reading the ledgers — store events,
|
||||
* derive state. sessions.jsonl is the extraction checkpoint: which chats
|
||||
* the extraction pipeline has processed, and through which updatedAt
|
||||
* (never extract the same turns twice).
|
||||
*
|
||||
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
|
||||
* merger): the extractor proposes propositions/edges/attribution with
|
||||
* verbatim evidence, the auditor drops anything unbacked, and the merger
|
||||
* (core/graph.ts, deterministic) resolves identity against the existing
|
||||
* ledgers and appends. See SessionFindings below.
|
||||
* merger), tree-blind and post-hoc: the extractor reads only the transcript
|
||||
* and proposes propositions/edges/quiz-attribution with verbatim evidence;
|
||||
* the auditor sees the extractor's output PLUS the current graph, owns
|
||||
* faithfulness (evidence check) and identity (link-or-create: point each
|
||||
* proposition at the existing node it is, or let it be created — recording
|
||||
* the session's phrasings as aliases); the merger (core/graph.ts,
|
||||
* deterministic) applies the verdicts and appends.
|
||||
*/
|
||||
|
||||
// ─── Extraction findings (extractor output → audit → merger input) ───────────
|
||||
|
||||
/** One proposed proposition — a node candidate. */
|
||||
/** One proposed proposition — a node candidate (or a re-encounter of one). */
|
||||
export interface ExtractedProposition {
|
||||
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
|
||||
id: string;
|
||||
/** Short name / header for the fact (2-6 words) — the rendering handle. */
|
||||
name: string;
|
||||
/** Canonical one-reasoning-step statement — an objective, world-facing
|
||||
* fact (no learner / teacher / session references; the learner's
|
||||
* relationship to it is carried only by the event attribution). */
|
||||
/** The minimal proposition — a true, self-contained, world-facing
|
||||
* statement. No learner / teacher / session references: misconceptions
|
||||
* are stated as the true correction, and wrongness lives in the event
|
||||
* ledger, never in the node. */
|
||||
statement: string;
|
||||
/** Verbatim alternate phrasings from the session (merge aliases). */
|
||||
/** The elaborated version — required. */
|
||||
description: string;
|
||||
/** Verbatim phrasings the session used. */
|
||||
aliases: string[];
|
||||
kind: "truth" | "misconception";
|
||||
/** The plan-tree node this came from, when it did (mapping provenance). */
|
||||
treeNodeId?: string;
|
||||
/** The message ids where this proposition came up — provenance. */
|
||||
messages: string[];
|
||||
/** Verbatim transcript quotes backing the proposition. */
|
||||
evidence: string[];
|
||||
/** The extractor's understanding note for this proposition IN THIS
|
||||
* session — a named-word scale ("solid", "shaky", "accepted",
|
||||
* "resisted"), no numbers. Joins the ledger as an observation event;
|
||||
* treatment pending the state-model ideation. */
|
||||
understanding?: string;
|
||||
}
|
||||
|
||||
/** One proposed relation between two propositions. */
|
||||
export interface ExtractedEdge {
|
||||
/** Proposition id that is the prerequisite / support. */
|
||||
/** Proposition id that is followed / supported. */
|
||||
from: string;
|
||||
/** Proposition id that depends on / is supported by `from`. */
|
||||
to: string;
|
||||
type: EdgeType;
|
||||
/** The message ids where this relation was asserted — provenance. */
|
||||
messages: string[];
|
||||
/** Verbatim transcript quotes backing the relation. */
|
||||
evidence: string[];
|
||||
}
|
||||
|
||||
/** The extractor's event attribution — its ONLY say about events. The kinds,
|
||||
* timestamps, idempotency refs, notes and question texts are all rebuilt
|
||||
* deterministically by the merger from the transcript's quiz / mark_done
|
||||
* tool parts; the LLM supplies exactly the judgment call: WHICH proposition
|
||||
* each one is about (vision open question: probes attribute retroactively,
|
||||
* against the whole transcript — a pre-tree probe can map to a proposition
|
||||
* the tree only articulates later). */
|
||||
/** The extractor's event attribution — its ONLY say about events. Event
|
||||
* kinds, timestamps, idempotency refs, notes and question texts are all
|
||||
* rebuilt deterministically by the merger from the transcript's tool
|
||||
* parts, and TAGGED events (quiz nodeId, mark_done tree node) resolve
|
||||
* deterministically through the proposition mapping. The LLM supplies
|
||||
* exactly the one judgment call that needs it — the vision's open
|
||||
* probe-attribution question: WHICH proposition did an UNTAGGED probe
|
||||
* test? Answered retroactively against the whole transcript — a pre-tree
|
||||
* probe can map to a proposition the tree only articulates later. */
|
||||
/**
|
||||
* The extractor's quiz attribution — its ONLY say about quiz events. Event
|
||||
* kinds, timestamps, idempotency refs, notes and question texts are all
|
||||
* rebuilt deterministically by the merger from the transcript's quiz tool
|
||||
* parts; the LLM supplies exactly the judgment call: WHICH proposition each
|
||||
* quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
|
||||
* mapping to lean on) — retroactively against the whole transcript.
|
||||
*/
|
||||
export interface ExtractionAttribution {
|
||||
/** UNTAGGED quiz toolCallId → proposition id; missing / null =
|
||||
* unattributed (the event lands with nodeId null + the question text,
|
||||
* exactly like unattributable probes always did). */
|
||||
/** quiz toolCallId → proposition id; null / missing = unattributed (the
|
||||
* event lands with nodeId null + the question text, a later-
|
||||
* attribution input). */
|
||||
quiz: Record<string, string | null>;
|
||||
}
|
||||
|
||||
/** The extractor's — and, filtered, the auditor's — output shape. */
|
||||
/** The extractor's output shape. */
|
||||
export interface SessionFindings {
|
||||
propositions: ExtractedProposition[];
|
||||
edges: ExtractedEdge[];
|
||||
attribution: ExtractionAttribution;
|
||||
}
|
||||
|
||||
/** The auditor's verdict: the verified subset, plus the drops (for the log —
|
||||
* the audit trail of what was rejected and why). */
|
||||
export interface AuditResult extends SessionFindings {
|
||||
/** The auditor's verdict on one proposition: the findings, plus `linkTo` —
|
||||
* the identity judgment. Absent = create a new node. */
|
||||
export interface AuditedProposition extends ExtractedProposition {
|
||||
/** The existing GraphNode id this proposition IS — the auditor's
|
||||
* identity verdict. On link, the session's phrasings (statement,
|
||||
* aliases) join the node's aliases and the provenance unions.
|
||||
* Conservative: when unsure, leave it out — duplicates over wrong
|
||||
* merges. */
|
||||
linkTo?: string;
|
||||
}
|
||||
|
||||
/** The auditor's output: the verified findings (with identity verdicts),
|
||||
* plus the drops (the audit trail of what was rejected and why). */
|
||||
export interface AuditResult {
|
||||
propositions: AuditedProposition[];
|
||||
edges: ExtractedEdge[];
|
||||
attribution: ExtractionAttribution;
|
||||
/** Dropped candidates: proposition id, edge "from→to", or attribution
|
||||
* toolCallId — with the reason. */
|
||||
dropped: { ref: string; reason: string }[];
|
||||
}
|
||||
|
||||
/** Per-node state class for the home view (vision §1 "Home page") — derived,
|
||||
* never stored: taught if ANY taught event; else the LATEST quiz event decides
|
||||
* (known / missed / dont-know); a node with no events is a candidate. */
|
||||
/**
|
||||
* Per-node state class for the home view — TRANSITIONAL (kept so the
|
||||
* renderer survives the contract change untouched). Derived, never stored:
|
||||
* the LATEST quiz event decides (known / missed / dont-know); a node with
|
||||
* no quiz events is a candidate. `taught` is now unreachable (no taught
|
||||
* events are emitted anymore) — the real state model (dependency depth +
|
||||
* topology-derived strength + quiz events) is pending ideation.
|
||||
*/
|
||||
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
|
||||
|
||||
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
|
||||
@@ -187,4 +239,3 @@ export function titleFromPrompt(prompt: string): string {
|
||||
const oneLine = prompt.replace(/\s+/g, " ").trim();
|
||||
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
|
||||
}
|
||||
|
||||
|
||||
+16
-22
@@ -15,10 +15,9 @@
|
||||
* the block is absent (new learner).
|
||||
*
|
||||
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE
|
||||
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know /
|
||||
* misconception nodes) plus their 1-hop `requires` neighborhood — a live
|
||||
* misconception is always relevant context, and the frontier is where the
|
||||
* next session should probe.
|
||||
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know
|
||||
* nodes) plus their 1-hop `follows` neighborhood — the frontier is where
|
||||
* the next session should probe.
|
||||
*/
|
||||
|
||||
import type { GraphEdge, GraphNode } from "./contracts.ts";
|
||||
@@ -31,22 +30,18 @@ export const GRAPH_CONTEXT_HEADER = "LEARNER KNOWLEDGE GRAPH (auto-generated)";
|
||||
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */
|
||||
export const MAX_CONTEXT_NODES = 150;
|
||||
|
||||
/** Group order for node lines: diagnostic value first (vision §1 hotspots).
|
||||
* A misconception-kind node is a hotspot whatever its state — a live
|
||||
* misconception is always relevant context. */
|
||||
/** Group order for node lines: diagnostic value first (vision §1 hotspots). */
|
||||
const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]);
|
||||
|
||||
function groupOf(node: GraphNode, state: NodeStateClass): number {
|
||||
if (HOTSPOT_STATES.has(state) || node.kind === "misconception") return 0;
|
||||
if (HOTSPOT_STATES.has(state)) return 0;
|
||||
if (state === "known") return 1;
|
||||
if (state === "taught") return 2;
|
||||
return 3; // candidate
|
||||
return 2; // candidate
|
||||
}
|
||||
|
||||
const GROUP_LABELS = [
|
||||
"Hotspots (missed / dont-know / misconceptions) — likely misconception sites and unexplored ground:",
|
||||
"Hotspots (missed / dont-know) — likely misconception sites and unexplored ground:",
|
||||
"Known (answered correctly):",
|
||||
"Taught (lessons covered them):",
|
||||
"Candidates (never probed — assume nothing):",
|
||||
] as const;
|
||||
|
||||
@@ -76,9 +71,9 @@ export function renderGraphContext(
|
||||
truncated = true;
|
||||
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
|
||||
const keep = new Set(hotspots.map((n) => n.id));
|
||||
// 1-hop requires neighborhood: prerequisites AND dependents.
|
||||
// 1-hop follows neighborhood: premises AND derivations.
|
||||
for (const edge of edges) {
|
||||
if (edge.type !== "requires") continue;
|
||||
if (edge.type !== "follows") continue;
|
||||
if (keep.has(edge.from) || keep.has(edge.to)) {
|
||||
keep.add(edge.from);
|
||||
keep.add(edge.to);
|
||||
@@ -91,7 +86,7 @@ export function renderGraphContext(
|
||||
// ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
|
||||
const lines: string[] = [
|
||||
`===== ${GRAPH_CONTEXT_HEADER} =====`,
|
||||
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "requires").length} requires edges`,
|
||||
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "follows").length} follows edges`,
|
||||
];
|
||||
|
||||
for (const [group, label] of GROUP_LABELS.entries()) {
|
||||
@@ -101,19 +96,18 @@ export function renderGraphContext(
|
||||
if (groupNodes.length === 0) continue;
|
||||
lines.push("", label);
|
||||
for (const node of groupNodes) {
|
||||
const kind = node.kind === "misconception" ? " · misconception" : "";
|
||||
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})${kind}`);
|
||||
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})`);
|
||||
}
|
||||
}
|
||||
|
||||
// ── requires edges, merged into chains ──────────────────────────────────
|
||||
const byId = new Map(selected.map((n) => [n.id, n.statement]));
|
||||
const requires = edges.filter(
|
||||
(e) => e.type === "requires" && byId.has(e.from) && byId.has(e.to),
|
||||
const follows = edges.filter(
|
||||
(e) => e.type === "follows" && byId.has(e.from) && byId.has(e.to),
|
||||
);
|
||||
if (requires.length > 0) {
|
||||
lines.push("", "Requires (prerequisite → dependent):");
|
||||
for (const chain of edgeChains(requires)) {
|
||||
if (follows.length > 0) {
|
||||
lines.push("", "Follows (premise → derivation):");
|
||||
for (const chain of edgeChains(follows)) {
|
||||
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
|
||||
}
|
||||
}
|
||||
|
||||
+176
-172
@@ -2,22 +2,34 @@
|
||||
* The knowledge graph: merger (the deterministic final stage of the
|
||||
* extraction pipeline — vision §1 construction) + derivation (reads).
|
||||
*
|
||||
* The extraction pipeline is the SOLE graph writer:
|
||||
* The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
|
||||
*
|
||||
* extractor (LLM) → SessionFindings: propositions, edges, and event
|
||||
* attribution, each with verbatim evidence
|
||||
* auditor (LLM) → the verified subset (unbacked candidates dropped)
|
||||
* extractor (LLM) → SessionFindings: propositions, edges, quiz
|
||||
* attribution, per-proposition understanding notes —
|
||||
* each with verbatim evidence, from the transcript ALONE
|
||||
* auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
|
||||
* VERDICT per proposition (linkTo an existing node, or
|
||||
* create), and the drops for the log
|
||||
* merger (HERE) → mergeFindings(): deterministic ledger appends
|
||||
*
|
||||
* The merger is deliberately the ONLY stage that touches the ledgers, and it
|
||||
* is pure code: the LLM supplies exactly the judgment calls (what was
|
||||
* taught, what depends on what, which proposition a quiz tested), while
|
||||
* event kinds/timestamps/refs — anything the model could silently
|
||||
* mis-copy — are rebuilt deterministically from the transcript's tool
|
||||
* parts. Idempotent + incremental: statement/alias matching dedupes nodes,
|
||||
* (from,to,type) dedupes edge assertions, chat-scoped refs dedupe events,
|
||||
* one mapping line per (chat, graph node), and sessions.jsonl checkpoints
|
||||
* which chats have been extracted (never the same turns twice).
|
||||
* is pure code: the LLM supplies exactly the judgment calls (what the
|
||||
* propositions are, what follows/supports what, which proposition a quiz
|
||||
* tested, whether a proposition IS an existing node), while event
|
||||
* kinds/timestamps/refs/provenance — anything the model could silently
|
||||
* mis-copy — are rebuilt deterministically from the transcript's tool parts
|
||||
* and the message ids in the findings. Idempotent + incremental: refs are
|
||||
* chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
|
||||
* asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
|
||||
* sessions.jsonl checkpoints which chats have been extracted (never the
|
||||
* same turns twice).
|
||||
*
|
||||
* Identity across re-encounters is the AUDITOR's judgment (link-or-create),
|
||||
* not string matching: the conservative v1 exact-statement merge was
|
||||
* replaced by the 2026-09-02 redesign. A re-extracted session's
|
||||
* propositions link to the existing nodes; the link appends a node line
|
||||
* whose phrasings join `aliases` and whose provenance unions (getGraph
|
||||
* merges lines by id).
|
||||
*
|
||||
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
|
||||
* only — everything is recomputed from the ledgers. Every function takes
|
||||
@@ -29,14 +41,14 @@
|
||||
import { randomBytes } from "node:crypto";
|
||||
import { isToolUIPart, type UIMessage } from "ai";
|
||||
import type {
|
||||
AuditResult,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
ProvenancePoint,
|
||||
GraphSnapshot,
|
||||
LedgerEvent,
|
||||
LedgerEventKind,
|
||||
NodeStateClass,
|
||||
PlanToolResult,
|
||||
SessionFindings,
|
||||
} from "./contracts.ts";
|
||||
import type { QuestionAnswerData } from "./question-core.ts";
|
||||
import type { ChatStore, GraphStore } from "./ports.ts";
|
||||
@@ -80,19 +92,25 @@ function toolParts(messages: UIMessage[]): ToolPartView[] {
|
||||
return out;
|
||||
}
|
||||
|
||||
/** A plan tool part's output, if it is a well-formed PlanToolResult. */
|
||||
function planResult(part: ToolPartView): PlanToolResult | null {
|
||||
if (part.state !== "output-available") return null;
|
||||
const out = part.output;
|
||||
if (typeof out !== "object" || out === null) return null;
|
||||
if (!("tree" in out) || typeof (out as PlanToolResult).ok !== "boolean") return null;
|
||||
return out as PlanToolResult;
|
||||
/** (chatId, messageIds) → provenance points; order-preserving, deduped. */
|
||||
function pointsOf(chatId: string, messages: string[]): ProvenancePoint[] {
|
||||
const seen = new Set<string>();
|
||||
const out: ProvenancePoint[] = [];
|
||||
for (const messageId of messages) {
|
||||
if (messageId.length === 0 || seen.has(messageId)) continue;
|
||||
seen.add(messageId);
|
||||
out.push({ chatId, messageId });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
|
||||
a.chatId === b.chatId && a.messageId === b.messageId;
|
||||
|
||||
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
|
||||
|
||||
export interface EmissionStats {
|
||||
/** New graph nodes appended (and mappings with them). */
|
||||
/** New graph nodes appended. */
|
||||
nodes: number;
|
||||
/** New edge assertions appended. */
|
||||
edges: number;
|
||||
@@ -102,156 +120,135 @@ export interface EmissionStats {
|
||||
|
||||
/**
|
||||
* Merge one chat's audited findings into the graph ledgers — the pipeline's
|
||||
* apply stage. Deterministic + idempotent: re-merging the same findings (or
|
||||
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or
|
||||
* a later re-extraction of the same chat) appends nothing but the new.
|
||||
*
|
||||
* Identity resolution (v1: conservative): a proposition whose statement or
|
||||
* alias EXACTLY matches (case-insensitive, trimmed) an existing node's
|
||||
* statement or alias merges onto it; everything else is a new node. One
|
||||
* mapping line per (chat, graph node) — the merge audit trail. Fuzzy /
|
||||
* embedding adjudication stays deferred (vision §1 consolidation).
|
||||
* Identity is the auditor's link-or-create verdict: `linkTo` an existing
|
||||
* node (validated against the ledger — an unknown id falls back to create)
|
||||
* appends a LINK line (the session's phrasings join the node's aliases and
|
||||
* the provenance unions; getGraph merges lines by id); absent → a new node.
|
||||
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
|
||||
* a proposition id already mapped for this chat reuses its resolution.
|
||||
*
|
||||
* Events are rebuilt deterministically from the transcript's quiz and
|
||||
* plan_mark_done parts (kinds, timestamps, notes, refs). TAGGED events
|
||||
* (quiz nodeId, mark_done tree node) resolve through the proposition
|
||||
* mapping — deterministic; the findings' attribution is consulted ONLY for
|
||||
* untagged probe quizzes: the retroactive probe-attribution resolution (a
|
||||
* pre-tree probe maps to a proposition the tree only articulates later).
|
||||
* Unattributed quizzes land with nodeId null + the question text, exactly
|
||||
* like the unattributable ones always have.
|
||||
* Events are rebuilt deterministically from the transcript: quiz outcomes
|
||||
* (correct / miss / dont-know; kinds + timestamps + refs from the tool
|
||||
* parts, nodeId from the extractor's attribution through this run's
|
||||
* mapping) plus one observation event per proposition carrying an
|
||||
* understanding note (ref `obs:<sourceId>`). No taught events, ever.
|
||||
*/
|
||||
export function mergeFindings(chatId: string, findings: SessionFindings, deps: GraphDeps): EmissionStats {
|
||||
export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps): EmissionStats {
|
||||
const { graphStore: ledgers, chatStore } = deps;
|
||||
const record = chatStore.getChat(chatId);
|
||||
if (!record) return { nodes: 0, edges: 0, events: 0 };
|
||||
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
|
||||
|
||||
// 1. Identity resolution. mapping: proposition id (and tree node id)
|
||||
// → graph node id — from this chat's existing mapping lines, then
|
||||
// extended as findings resolve. chatNodeIds: graph nodes this chat has
|
||||
// already mapped onto — the (chat, graph node) mapping dedupe.
|
||||
// 1. Resolution: this run's propositions → graph node ids. Existing
|
||||
// mapping lines for THIS chat are loaded first — the audit trail doubles
|
||||
// as the resolution table, which makes re-merging the same chat
|
||||
// idempotent (a re-extraction re-merges through the auditor's linkTo
|
||||
// verdicts instead). Fresh sourceIds resolve as the verdicts say.
|
||||
const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
|
||||
const mapping = new Map<string, string>();
|
||||
const chatNodeIds = new Set<string>();
|
||||
const seenMappingKeys = new Set<string>();
|
||||
for (const m of ledgers.readMappings()) {
|
||||
if (m.chatId !== chatId || typeof m.sourceId !== "string") continue;
|
||||
if (m.chatId !== chatId) continue;
|
||||
mapping.set(m.sourceId, m.graphNodeId);
|
||||
chatNodeIds.add(m.graphNodeId);
|
||||
}
|
||||
|
||||
// 2. Propositions → graph nodes. Exact statement/alias match against the
|
||||
// whole ledger first (cross-session reuse), then against propositions
|
||||
// created earlier in THIS merge (intra-findings dedupe).
|
||||
const norm = (s: string): string => s.trim().toLowerCase();
|
||||
const existingByTitle = new Map<string, string>();
|
||||
for (const node of ledgers.readNodes()) {
|
||||
for (const title of [node.statement, ...node.aliases]) {
|
||||
const key = norm(title);
|
||||
if (key && !existingByTitle.has(key)) existingByTitle.set(key, node.id);
|
||||
}
|
||||
seenMappingKeys.add(m.sourceId);
|
||||
}
|
||||
const now = new Date().toISOString();
|
||||
for (const p of findings.propositions) {
|
||||
if (mapping.has(p.id)) continue;
|
||||
const keys = [p.statement, ...p.aliases].map(norm).filter((k) => k.length > 0);
|
||||
const matchId = keys.map((k) => existingByTitle.get(k)).find((id) => id !== undefined);
|
||||
|
||||
for (const p of audit.propositions) {
|
||||
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
|
||||
const points = pointsOf(chatId, p.messages);
|
||||
let graphNodeId: string;
|
||||
if (matchId !== undefined) {
|
||||
graphNodeId = matchId; // reuse — conservative merge, duplicates over wrong merges
|
||||
} else {
|
||||
const graphNode: GraphNode = {
|
||||
id: newGraphNodeId(),
|
||||
name: p.name,
|
||||
if (p.linkTo !== undefined && existingIds.has(p.linkTo)) {
|
||||
// Identity verdict: this proposition IS that node. Append a link
|
||||
// line — the session's phrasings (statement + aliases) join the
|
||||
// node's aliases, the provenance unions. Reads merge by id.
|
||||
graphNodeId = p.linkTo;
|
||||
ledgers.appendNode({
|
||||
id: graphNodeId,
|
||||
statement: p.statement,
|
||||
description: p.description,
|
||||
aliases: [...p.aliases],
|
||||
kind: p.kind,
|
||||
createdAt: now,
|
||||
provenance: points,
|
||||
});
|
||||
} else {
|
||||
const node: GraphNode = {
|
||||
id: newGraphNodeId(),
|
||||
statement: p.statement,
|
||||
description: p.description,
|
||||
aliases: [...p.aliases],
|
||||
createdAt: now,
|
||||
provenance: points,
|
||||
};
|
||||
ledgers.appendNode(graphNode);
|
||||
ledgers.appendNode(node);
|
||||
stats.nodes++;
|
||||
graphNodeId = graphNode.id;
|
||||
}
|
||||
for (const key of keys) if (!existingByTitle.has(key)) existingByTitle.set(key, graphNodeId);
|
||||
if (!chatNodeIds.has(graphNodeId)) {
|
||||
ledgers.appendMapping({ chatId, sourceId: p.treeNodeId ?? p.id, graphNodeId });
|
||||
chatNodeIds.add(graphNodeId);
|
||||
graphNodeId = node.id;
|
||||
existingIds.add(graphNodeId);
|
||||
}
|
||||
mapping.set(p.id, graphNodeId);
|
||||
if (p.treeNodeId !== undefined) mapping.set(p.treeNodeId, graphNodeId);
|
||||
if (!seenMappingKeys.has(p.id)) {
|
||||
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
|
||||
seenMappingKeys.add(p.id);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Edges. Dedupe by (from,to,type) across the whole ledger: an existing
|
||||
// assertion just gains this chatId as provenance (appended as a new
|
||||
// line; reads merge).
|
||||
const asserted = new Map<string, Set<string>>();
|
||||
// 2. Edges. Resolve endpoints through the mapping; dedupe by
|
||||
// (from,to,type) already asserted by THIS chat (an existing assertion
|
||||
// by another chat stays untouched — provenance unions at read time).
|
||||
const assertedByChat = new Set<string>();
|
||||
for (const line of ledgers.readEdges()) {
|
||||
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
||||
let provenance = asserted.get(key);
|
||||
if (!provenance) {
|
||||
provenance = new Set();
|
||||
asserted.set(key, provenance);
|
||||
}
|
||||
for (const c of line.provenance) provenance.add(c);
|
||||
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
|
||||
}
|
||||
for (const edge of findings.edges) {
|
||||
for (const edge of audit.edges) {
|
||||
const from = mapping.get(edge.from);
|
||||
const to = mapping.get(edge.to);
|
||||
if (!from || !to || from === to) continue; // unmapped endpoint — skip
|
||||
const key = `${from}\u0000${to}\u0000${edge.type}`;
|
||||
const provenance = asserted.get(key);
|
||||
if (provenance?.has(chatId)) continue; // already asserted by this chat
|
||||
ledgers.appendEdge({ from, to, type: edge.type, provenance: [chatId] });
|
||||
if (provenance) provenance.add(chatId);
|
||||
else asserted.set(key, new Set([chatId]));
|
||||
if (assertedByChat.has(key)) continue;
|
||||
ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
|
||||
assertedByChat.add(key);
|
||||
stats.edges++;
|
||||
}
|
||||
|
||||
// 4. Events, rebuilt deterministically in transcript order so the
|
||||
// probe/check boundary is exact: quizzes BEFORE the first plan_init
|
||||
// output are probes. Refs are chat-scoped (a quiz toolCallId or
|
||||
// taught:<treeNodeId>) — a ref already in the ledger means this turn
|
||||
// was extracted before. The findings' attribution supplies the
|
||||
// nodeId, nothing else.
|
||||
// 3. Events, rebuilt deterministically in transcript order. Refs are
|
||||
// chat-scoped idempotency keys (quiz toolCallId, obs:<sourceId>) —
|
||||
// a ref already in the ledger means this evidence was extracted before.
|
||||
const seenRefs = new Set<string>();
|
||||
for (const event of ledgers.readEvents()) {
|
||||
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
|
||||
}
|
||||
let planSeen = false;
|
||||
for (const part of toolParts(record.messages)) {
|
||||
// The boundary flips on the first plan_init OUTPUT, whatever it says
|
||||
// (an ok:false "tree already initialized" still means a tree exists).
|
||||
if (part.name === "plan_init" && part.state === "output-available") planSeen = true;
|
||||
|
||||
if (part.name === "quiz" && part.state === "output-available") {
|
||||
const event = quizEvent(part, chatId, mapping, findings, planSeen, now);
|
||||
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
|
||||
ledgers.appendEvent(event);
|
||||
seenRefs.add(event.ref);
|
||||
stats.events++;
|
||||
}
|
||||
}
|
||||
|
||||
if (part.name === "plan_mark_done" && part.state === "output-available") {
|
||||
const result = planResult(part);
|
||||
const input = part.input as { id?: unknown } | undefined;
|
||||
if (result?.ok && typeof input?.id === "string") {
|
||||
const ref = `taught:${input.id}`;
|
||||
if (!seenRefs.has(ref)) {
|
||||
// mark_done is tagged with the tree node id — deterministic.
|
||||
ledgers.appendEvent({
|
||||
nodeId: mapping.get(input.id) ?? null,
|
||||
kind: "taught",
|
||||
chatId,
|
||||
at: now,
|
||||
ref,
|
||||
});
|
||||
seenRefs.add(ref);
|
||||
stats.events++;
|
||||
}
|
||||
}
|
||||
if (part.name !== "quiz" || part.state !== "output-available") continue;
|
||||
const event = quizEvent(part, chatId, audit, mapping, now);
|
||||
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
|
||||
ledgers.appendEvent(event);
|
||||
seenRefs.add(event.ref);
|
||||
stats.events++;
|
||||
}
|
||||
}
|
||||
// Understanding observations: one per proposition carrying a note,
|
||||
// resolved to the node the verdict mapped it to.
|
||||
for (const p of audit.propositions) {
|
||||
if (p.understanding === undefined || p.understanding.length === 0) continue;
|
||||
const ref = `obs:${p.id}`;
|
||||
if (seenRefs.has(ref)) continue;
|
||||
ledgers.appendEvent({
|
||||
nodeId: mapping.get(p.id) ?? null,
|
||||
kind: "observation",
|
||||
chatId,
|
||||
at: now,
|
||||
ref,
|
||||
note: p.understanding,
|
||||
});
|
||||
seenRefs.add(ref);
|
||||
stats.events++;
|
||||
}
|
||||
|
||||
// 5. Checkpoint: this chat is extracted through its current updatedAt.
|
||||
// 4. Checkpoint: this chat is extracted through its current updatedAt.
|
||||
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
|
||||
return stats;
|
||||
}
|
||||
@@ -261,9 +258,8 @@ export function mergeFindings(chatId: string, findings: SessionFindings, deps: G
|
||||
function quizEvent(
|
||||
part: ToolPartView,
|
||||
chatId: string,
|
||||
audit: AuditResult,
|
||||
mapping: Map<string, string>,
|
||||
findings: SessionFindings,
|
||||
planSeen: boolean,
|
||||
now: string,
|
||||
): LedgerEvent | null {
|
||||
const out = part.output;
|
||||
@@ -274,16 +270,13 @@ function quizEvent(
|
||||
|
||||
let kind: LedgerEventKind;
|
||||
if (answer.dontKnow === true) kind = "dont-know";
|
||||
else if (answer.correct === true) kind = planSeen ? "check-correct" : "probe-correct";
|
||||
else kind = planSeen ? "check-incorrect" : "probe-miss";
|
||||
else if (answer.correct === true) kind = "correct";
|
||||
else kind = "miss";
|
||||
|
||||
// Tagged (input.nodeId) resolves deterministically through the mapping;
|
||||
// attribution is the LLM judgment call for UNTAGGED probes only — the
|
||||
// vision's open probe-attribution question, resolved retroactively.
|
||||
const input = part.input as { nodeId?: unknown } | undefined;
|
||||
const tagged = typeof input?.nodeId === "string" ? mapping.get(input.nodeId) : undefined;
|
||||
const attributedId = findings.attribution.quiz[part.toolCallId];
|
||||
const nodeId = tagged ?? (attributedId ? (mapping.get(attributedId) ?? null) : null);
|
||||
// Tree-blind: EVERY quiz attributes through the extractor's judgment —
|
||||
// the vision's retroactive attribution, with no tree mapping to lean on.
|
||||
const attributedId = audit.attribution.quiz[part.toolCallId];
|
||||
const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
|
||||
|
||||
const event: LedgerEvent = {
|
||||
nodeId,
|
||||
@@ -293,7 +286,7 @@ function quizEvent(
|
||||
ref: part.toolCallId,
|
||||
};
|
||||
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
|
||||
// The question text is the attribution input for unattributed probes.
|
||||
// The question text is the attribution input for unattributed quizzes.
|
||||
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
|
||||
return event;
|
||||
}
|
||||
@@ -335,20 +328,39 @@ export interface DerivedGraph {
|
||||
}
|
||||
|
||||
/**
|
||||
* The whole graph from the ledgers. Edge LINES are assertions; the returned
|
||||
* edges are merged by (from,to,type) with unioned provenance, first-seen
|
||||
* order. `store events, derive state` — this is the only reader that matters.
|
||||
* The whole graph from the ledgers. Node LINES may repeat an id (link-
|
||||
* appends): the FIRST line defines statement/description/createdAt; later
|
||||
* lines' statements and aliases join `aliases`, provenance unions. Edge
|
||||
* LINES are assertions, merged by (from,to,type) with unioned provenance,
|
||||
* first-seen order. `store events, derive state` — this is the only reader
|
||||
* that matters.
|
||||
*/
|
||||
export function getGraph(graphStore: GraphStore): DerivedGraph {
|
||||
const nodes = graphStore.readNodes();
|
||||
// Merge node lines by id (first line defines; later lines add phrasings).
|
||||
const byId = new Map<string, GraphNode>();
|
||||
for (const line of graphStore.readNodes()) {
|
||||
const existing = byId.get(line.id);
|
||||
if (!existing) {
|
||||
byId.set(line.id, { ...line, aliases: [...line.aliases], provenance: [...line.provenance] });
|
||||
continue;
|
||||
}
|
||||
// A later line's statement is a phrasing, not a replacement.
|
||||
for (const phrasing of [line.statement, ...line.aliases]) {
|
||||
if (phrasing.length > 0 && !existing.aliases.includes(phrasing)) existing.aliases.push(phrasing);
|
||||
}
|
||||
for (const pt of line.provenance) {
|
||||
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
|
||||
}
|
||||
}
|
||||
|
||||
const merged: GraphEdge[] = [];
|
||||
const byKey = new Map<string, GraphEdge>();
|
||||
for (const line of graphStore.readEdges()) {
|
||||
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
||||
const existing = byKey.get(key);
|
||||
if (existing) {
|
||||
for (const c of line.provenance) {
|
||||
if (!existing.provenance.includes(c)) existing.provenance.push(c);
|
||||
for (const pt of line.provenance) {
|
||||
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
|
||||
}
|
||||
} else {
|
||||
const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
|
||||
@@ -356,34 +368,27 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
|
||||
merged.push(edge);
|
||||
}
|
||||
}
|
||||
return { nodes, edges: merged, events: graphStore.readEvents() };
|
||||
return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
|
||||
}
|
||||
|
||||
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set([
|
||||
"probe-correct",
|
||||
"check-correct",
|
||||
"probe-miss",
|
||||
"check-incorrect",
|
||||
"dont-know",
|
||||
]);
|
||||
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set(["correct", "miss", "dont-know"]);
|
||||
|
||||
/**
|
||||
* Derive node states from events (never stored): taught if ANY taught event;
|
||||
* else the LATEST quiz event decides (known / missed / dont-know); a node
|
||||
* with no events is a candidate.
|
||||
* Derive node states from events (never stored): the LATEST quiz event
|
||||
* decides (known / missed / dont-know); a node with no quiz events is a
|
||||
* candidate. TRANSITIONAL (2026-09-02 redesign): `taught` is unreachable
|
||||
* now that no taught events are emitted, and observation events are NOT
|
||||
* state inputs — the real state model (dependency depth + topology-derived
|
||||
* strength + quiz events) is pending ideation; this keeps the renderer
|
||||
* alive until it lands.
|
||||
*/
|
||||
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> {
|
||||
const states: Record<string, NodeStateClass> = {};
|
||||
for (const node of nodes) states[node.id] = "candidate";
|
||||
|
||||
const taught = new Set<string>();
|
||||
const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>();
|
||||
events.forEach((event, index) => {
|
||||
if (event.nodeId === null) return;
|
||||
if (event.kind === "taught") {
|
||||
taught.add(event.nodeId);
|
||||
return;
|
||||
}
|
||||
if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
|
||||
const current = latest.get(event.nodeId);
|
||||
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
|
||||
@@ -392,13 +397,9 @@ export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Rec
|
||||
});
|
||||
|
||||
for (const node of nodes) {
|
||||
if (taught.has(node.id)) {
|
||||
states[node.id] = "taught";
|
||||
continue;
|
||||
}
|
||||
const last = latest.get(node.id);
|
||||
if (!last) continue; // candidate
|
||||
if (last.kind === "probe-correct" || last.kind === "check-correct") states[node.id] = "known";
|
||||
if (last.kind === "correct") states[node.id] = "known";
|
||||
else if (last.kind === "dont-know") states[node.id] = "dont-know";
|
||||
else states[node.id] = "missed";
|
||||
}
|
||||
@@ -422,10 +423,13 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
chats.add(chatId);
|
||||
};
|
||||
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
|
||||
for (const node of nodes) {
|
||||
for (const pt of node.provenance) touch(node.id, pt.chatId);
|
||||
}
|
||||
for (const edge of edges) {
|
||||
for (const chatId of edge.provenance) {
|
||||
touch(edge.from, chatId);
|
||||
touch(edge.to, chatId);
|
||||
for (const pt of edge.provenance) {
|
||||
touch(edge.from, pt.chatId);
|
||||
touch(edge.to, pt.chatId);
|
||||
}
|
||||
}
|
||||
for (const event of events) {
|
||||
|
||||
+1
-1
@@ -56,7 +56,7 @@ export interface GraphStore {
|
||||
appendEdge(assertion: GraphEdge): void;
|
||||
readEdges(): GraphEdge[];
|
||||
|
||||
/** Append one quiz outcome / taught event. */
|
||||
/** Append one quiz outcome / understanding observation. */
|
||||
appendEvent(event: LedgerEvent): void;
|
||||
readEvents(): LedgerEvent[];
|
||||
|
||||
|
||||
+28
-35
@@ -1,20 +1,26 @@
|
||||
/**
|
||||
* GraphStore — the fs JSONL adapter: five append-only ledgers at
|
||||
* config.LEDGER_FILES (contracts.ts "v1 emission").
|
||||
* nodes.jsonl GraphNode — one line per graph node
|
||||
* nodes.jsonl GraphNode — one line per node write: the FIRST
|
||||
* line for an id defines it; a later
|
||||
* line with a REPEATED id is a
|
||||
* link-append (session phrasings join
|
||||
* aliases, provenance unions); reads
|
||||
* merge by id (core/graph.ts getGraph)
|
||||
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
|
||||
* session asserting the same (from,to,type)
|
||||
* appends another line; reads merge)
|
||||
* events.jsonl LedgerEvent — quiz outcomes + taught events
|
||||
* events.jsonl LedgerEvent — quiz outcomes + understanding observations
|
||||
* mappings.jsonl NodeMapping — the session→graph identity map:
|
||||
* (chatId, sourceId) → graphNodeId
|
||||
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
|
||||
*
|
||||
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this
|
||||
* adapter's. Reads are strict against the contracts.ts shapes: corrupt lines
|
||||
* are skipped, never guessed at — the ledgers are the source of truth and
|
||||
* everything else is derived (core/graph.ts, which takes this port as a
|
||||
* parameter). Same idioms as src/store/chats.ts: sync fs, no framework.
|
||||
* adapter's. Reads are strict against the contracts.ts shapes (the 2026-09-02
|
||||
* graph redesign): corrupt lines are skipped, never guessed at — the ledgers
|
||||
* are the source of truth and everything else is derived (core/graph.ts,
|
||||
* which takes this port as a parameter). Same idioms as src/store/chats.ts:
|
||||
* sync fs, no framework.
|
||||
*/
|
||||
|
||||
import * as fs from "node:fs";
|
||||
@@ -26,6 +32,7 @@ import type {
|
||||
LedgerEvent,
|
||||
LedgerEventKind,
|
||||
NodeMapping,
|
||||
ProvenancePoint,
|
||||
SessionCheckpoint,
|
||||
} from "../core/contracts.ts";
|
||||
import type { GraphStore } from "../core/ports.ts";
|
||||
@@ -69,26 +76,22 @@ function readValid<T>(file: string, isValid: (v: unknown) => v is T): T[] {
|
||||
return readRawLines(file).filter(isValid);
|
||||
}
|
||||
|
||||
/** Pre-name ledger line (the first extraction runs) — read tolerantly. */
|
||||
type StoredGraphNode = Omit<GraphNode, "name"> & { name?: string };
|
||||
/** Provenance points are (chatId, messageId) pairs — message-granular. */
|
||||
const isProvenancePoint = (v: unknown): v is ProvenancePoint =>
|
||||
isRecord(v) && isString(v.chatId) && isString(v.messageId);
|
||||
|
||||
const isStoredGraphNode = (v: unknown): v is StoredGraphNode =>
|
||||
const isGraphNode = (v: unknown): v is GraphNode =>
|
||||
isRecord(v) &&
|
||||
isString(v.id) &&
|
||||
isString(v.statement) &&
|
||||
isString(v.description) &&
|
||||
Array.isArray(v.aliases) &&
|
||||
(v.kind === "truth" || v.kind === "misconception") &&
|
||||
v.aliases.every(isString) &&
|
||||
isString(v.createdAt) &&
|
||||
(v.name === undefined || isString(v.name));
|
||||
Array.isArray(v.provenance) &&
|
||||
v.provenance.every(isProvenancePoint);
|
||||
|
||||
/** Derived header for legacy node lines written before `name` existed:
|
||||
* the first clause of the statement, capped — good enough to render. */
|
||||
function nameFromStatement(statement: string): string {
|
||||
const clause = statement.split(/[.;:—]/)[0]!.trim();
|
||||
return clause.length > 60 ? `${clause.slice(0, 57)}…` : clause || statement;
|
||||
}
|
||||
|
||||
const EDGE_TYPES = ["requires", "supports"] as const;
|
||||
const EDGE_TYPES = ["follows", "supports"] as const;
|
||||
|
||||
const isGraphEdge = (v: unknown): v is GraphEdge =>
|
||||
isRecord(v) &&
|
||||
@@ -96,17 +99,9 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
|
||||
isString(v.to) &&
|
||||
(EDGE_TYPES as readonly string[]).includes(v.type as string) &&
|
||||
Array.isArray(v.provenance) &&
|
||||
v.provenance.every(isString);
|
||||
v.provenance.every(isProvenancePoint);
|
||||
|
||||
const EVENT_KINDS: readonly LedgerEventKind[] = [
|
||||
"probe-correct",
|
||||
"probe-miss",
|
||||
"dont-know",
|
||||
"misconception-signaled",
|
||||
"taught",
|
||||
"check-correct",
|
||||
"check-incorrect",
|
||||
];
|
||||
const EVENT_KINDS: readonly LedgerEventKind[] = ["correct", "miss", "dont-know", "observation"];
|
||||
|
||||
const isLedgerEvent = (v: unknown): v is LedgerEvent =>
|
||||
isRecord(v) &&
|
||||
@@ -127,13 +122,11 @@ export const graphStore: GraphStore = {
|
||||
appendNode(node: GraphNode): void {
|
||||
appendLine(LEDGER_FILES.nodes, node);
|
||||
},
|
||||
/** Raw node lines in append order (ids are unique — no merging needed).
|
||||
* Legacy lines without a `name` get one derived from the statement. */
|
||||
/** Raw node lines in append order. A node id may appear on more than
|
||||
* one line (a later session's LINK-append: phrasings + provenance); the
|
||||
* derivation (core/graph.ts getGraph) merges lines by id. */
|
||||
readNodes(): GraphNode[] {
|
||||
return readValid(LEDGER_FILES.nodes, isStoredGraphNode).map((n) => ({
|
||||
...n,
|
||||
name: n.name ?? nameFromStatement(n.statement),
|
||||
}));
|
||||
return readValid(LEDGER_FILES.nodes, isGraphNode);
|
||||
},
|
||||
|
||||
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */
|
||||
|
||||
+89
-71
@@ -1,13 +1,16 @@
|
||||
/**
|
||||
* The extraction pipeline tests (vision §1 construction): extractor →
|
||||
* auditor → merger, the sole graph writer.
|
||||
* The extraction pipeline tests (vision §1 construction, 2026-09-02 redesign):
|
||||
* extractor → auditor → merger, the sole graph writer — tree-blind,
|
||||
* free-form, with the auditor owning faithfulness and identity.
|
||||
*
|
||||
* transcriptView — the compact view both LLM stages see (pure)
|
||||
* transcriptView — the compact view both LLM stages see (pure):
|
||||
* message-id-tagged verbatim text + quiz lines, NO tree
|
||||
* pipeline — MockLanguageModelV4 drives both agents: findings out,
|
||||
* audit verdict in, deterministic merge applied; failure
|
||||
* paths checkpoint nothing; single-flight per chat
|
||||
* audit verdict (with linkTo identity) in, deterministic
|
||||
* merge applied; failure paths checkpoint nothing;
|
||||
* single-flight per chat
|
||||
*
|
||||
* The merger itself (identity resolution, event rebuild, idempotency) is
|
||||
* The merger itself (identity verdicts, event rebuild, idempotency) is
|
||||
* tested in tests/graph.test.ts directly against mergeFindings — here it is
|
||||
* the integration: what one full pipeline run does to the ledgers.
|
||||
*
|
||||
@@ -21,9 +24,16 @@ import * as fs from "node:fs";
|
||||
import { after, describe, it } from "node:test";
|
||||
import { MockLanguageModelV4 } from "ai/test";
|
||||
import type { UIMessage } from "ai";
|
||||
import type { GraphEdge, GraphNode, LedgerEvent, NodeMapping, SessionCheckpoint, SessionFindings } from "../src/core/contracts.ts";
|
||||
import type {
|
||||
AuditResult,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
LedgerEvent,
|
||||
NodeMapping,
|
||||
SessionCheckpoint,
|
||||
SessionFindings,
|
||||
} from "../src/core/contracts.ts";
|
||||
import type { GraphStore } from "../src/core/ports.ts";
|
||||
import { createTree, markDone } from "../src/core/plan-tree.ts";
|
||||
import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
import { PATHS } from "../config.ts";
|
||||
@@ -72,7 +82,7 @@ function quizPart(toolCallId: string, input: Record<string, unknown>, output: Re
|
||||
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
|
||||
}
|
||||
|
||||
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: ReturnType<typeof createTree>): unknown {
|
||||
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: unknown): unknown {
|
||||
return {
|
||||
type: `tool-${name}`,
|
||||
toolCallId,
|
||||
@@ -82,21 +92,11 @@ function planPart(toolCallId: string, name: string, input: Record<string, unknow
|
||||
};
|
||||
}
|
||||
|
||||
/** A session-shaped fixture chat: probe → plan → check → mark_done. */
|
||||
/** A session-shaped fixture chat: probe → teaching text → check. Tree-blind
|
||||
* world: the plan tree is NOT part of extraction — it is noise. */
|
||||
function fixtureChat() {
|
||||
const tree = markDone(
|
||||
createTree(
|
||||
[
|
||||
{ id: "cr", title: "Chain rule intuition" },
|
||||
{ id: "apply", title: "Applying the chain rule" },
|
||||
],
|
||||
[{ from: "cr", to: "apply" }],
|
||||
),
|
||||
"cr",
|
||||
);
|
||||
const chat = chatStore.createChat("Extraction fixture");
|
||||
createdChats.push(chat.id);
|
||||
chatStore.saveTree(chat.id, tree);
|
||||
chatStore.saveMessages(chat.id, [
|
||||
user("u1", "teach me the chain rule"),
|
||||
assistant("a1", [
|
||||
@@ -107,15 +107,14 @@ function fixtureChat() {
|
||||
]),
|
||||
assistant("a2", [
|
||||
{ type: "text", text: "The derivative measures how fast a function changes." },
|
||||
planPart("p1", "plan_init", { nodes: [], edges: [] }, tree),
|
||||
planPart("p1", "plan_init", { nodes: [], edges: [] }, { nodes: [], edges: [], currentNodeId: null, doneNodeIds: [], version: 1 }),
|
||||
]),
|
||||
assistant("a3", [
|
||||
quizPart("check-1", { question: "Check", nodeId: "apply" }, {
|
||||
quizPart("check-1", { question: "Check" }, {
|
||||
questionId: "check-1", kind: "quiz", questionText: "Check", correct: true, dontKnow: false,
|
||||
answeredAt: "2026-08-22T02:00:00.000Z",
|
||||
}),
|
||||
]),
|
||||
assistant("a4", [planPart("p2", "plan_mark_done", { id: "cr" }, tree)]),
|
||||
]);
|
||||
return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
|
||||
}
|
||||
@@ -130,30 +129,38 @@ after(() => {
|
||||
}
|
||||
});
|
||||
|
||||
/** What the extractor would produce for the fixture session. */
|
||||
/** What the extractor would produce for the fixture session — tree-blind,
|
||||
* free-form, with message-granular provenance + an understanding note. */
|
||||
const FINDINGS: SessionFindings = {
|
||||
propositions: [
|
||||
{
|
||||
id: "p1",
|
||||
name: "Chain rule",
|
||||
statement: "Chain rule intuition",
|
||||
statement: "The derivative measures how fast a function changes",
|
||||
description: "The derivative of a function at a point is its instantaneous rate of change there.",
|
||||
aliases: [],
|
||||
kind: "truth",
|
||||
treeNodeId: "cr",
|
||||
evidence: ["TEACHER: The derivative measures how fast a function changes."],
|
||||
messages: ["a2"],
|
||||
evidence: ["TEACHER [a2]: The derivative measures how fast a function changes."],
|
||||
understanding: "solid",
|
||||
},
|
||||
{
|
||||
id: "p2",
|
||||
name: "Applying chain rule",
|
||||
statement: "Applying the chain rule",
|
||||
statement: "Chain rule intuition",
|
||||
description: "The chain rule differentiates composed functions by multiplying local rates.",
|
||||
aliases: [],
|
||||
kind: "truth",
|
||||
treeNodeId: "apply",
|
||||
evidence: ["FINAL PLAN TREE (node ids): cr = Chain rule intuition; apply = Applying the chain rule"],
|
||||
messages: ["a2"],
|
||||
evidence: ["LEARNER [u1]: teach me the chain rule"],
|
||||
},
|
||||
],
|
||||
edges: [{ from: "p1", to: "p2", type: "requires", evidence: ["TREE EDGES: cr requires apply"] }],
|
||||
attribution: { quiz: { "probe-1": "p1" } },
|
||||
edges: [
|
||||
{
|
||||
from: "p1",
|
||||
to: "p2",
|
||||
type: "follows",
|
||||
messages: ["a2"],
|
||||
evidence: ["LEARNER [u1]: teach me the chain rule"],
|
||||
},
|
||||
],
|
||||
attribution: { quiz: { "probe-1": "p1", "check-1": "p2" } },
|
||||
};
|
||||
|
||||
/** A model whose ONE tool call is `name` with the given input; then text. */
|
||||
@@ -202,28 +209,26 @@ function silentModel(): MockLanguageModelV4 {
|
||||
// ─── transcriptView (pure) ───────────────────────────────────────────────────
|
||||
|
||||
describe("transcriptView (pure)", () => {
|
||||
it("shows the final tree, verbatim conversation, structured events — plan ops folded", () => {
|
||||
it("shows message-id-tagged verbatim conversation + quiz lines — tree-blind, plan ops folded", () => {
|
||||
const { record } = fixtureChat();
|
||||
const view = transcriptView(record);
|
||||
|
||||
// The final tree with node ids (the extractor's backbone + the
|
||||
// pruned-out stays pruned — only the FINAL tree is shown).
|
||||
assert.match(view, /SESSION: Extraction fixture/);
|
||||
assert.match(view, /FINAL PLAN TREE \(node ids\): cr = Chain rule intuition; apply = Applying the chain rule/);
|
||||
assert.match(view, /TREE EDGES: cr requires apply/);
|
||||
|
||||
// Conversation verbatim — evidence quotes can come from it.
|
||||
assert.match(view, /LEARNER: teach me the chain rule/);
|
||||
assert.match(view, /TEACHER: The derivative measures how fast a function changes\./);
|
||||
// Conversation verbatim, each line tagged with its message id —
|
||||
// the provenance anchors the extractor reports back.
|
||||
assert.match(view, /LEARNER \[u1\] teach me the chain rule/);
|
||||
assert.match(view, /TEACHER \[a2\] The derivative measures how fast a function changes\./);
|
||||
|
||||
// Structured events, attribution-ready.
|
||||
assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/);
|
||||
assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/);
|
||||
assert.match(view, /TEACHER marked tree node done \[p2\]: cr/);
|
||||
|
||||
// plan_init / plan_mark_done tool parts themselves are folded into
|
||||
// the tree line — no raw JSON noise.
|
||||
assert.doesNotMatch(view, /plan_init/);
|
||||
// TREE-BLIND: no plan tree, no tree edges, no mark_done lines — plan
|
||||
// ops fold to the generic one-liner like every other non-quiz tool.
|
||||
assert.doesNotMatch(view, /FINAL PLAN TREE/);
|
||||
assert.doesNotMatch(view, /TREE EDGES/);
|
||||
assert.match(view, /\(plan_init tool result delivered\)/);
|
||||
assert.doesNotMatch(view, /output-available/);
|
||||
});
|
||||
|
||||
@@ -253,48 +258,61 @@ describe("transcriptView (pure)", () => {
|
||||
// ─── the pipeline ───────────────────────────────────────────────────────────
|
||||
|
||||
describe("the extraction pipeline (extractor → auditor → merger)", () => {
|
||||
it("runs end to end: findings → audit verdict → deterministic merge + checkpoint", async () => {
|
||||
it("runs end to end: findings → audit verdict (identity) → deterministic merge + checkpoint", async () => {
|
||||
const { chatId } = fixtureChat();
|
||||
const ledgers = memoryLedgers();
|
||||
|
||||
// The auditor passes p1 through, drops p2 (unbacked), re-points the
|
||||
// probe attribution to p1, and records the drop.
|
||||
const verified: SessionFindings = {
|
||||
propositions: [FINDINGS.propositions[0]!],
|
||||
// The auditor passes p1 through and LINKS it to a pre-existing node
|
||||
// (identity verdict), drops p2 (unbacked) + the edge with it.
|
||||
const existingId = "n_existing";
|
||||
ledgers.appendNode({
|
||||
id: existingId,
|
||||
statement: "Derivatives measure rates of change",
|
||||
description: "Pre-existing node from an earlier session.",
|
||||
aliases: [],
|
||||
createdAt: "2026-08-01T00:00:00.000Z",
|
||||
provenance: [],
|
||||
});
|
||||
const verdict: AuditResult = {
|
||||
propositions: [
|
||||
{ ...FINDINGS.propositions[0]!, linkTo: existingId },
|
||||
],
|
||||
edges: [],
|
||||
attribution: { quiz: { "probe-1": "p1" } },
|
||||
dropped: [{ ref: "p2", reason: "evidence not found" }, { ref: "p1→p2", reason: "endpoint dropped" }],
|
||||
};
|
||||
|
||||
const pipeline = createExtractionPipeline({
|
||||
chatStore,
|
||||
graphStore: ledgers,
|
||||
extractor: submitModel("submit_findings", FINDINGS),
|
||||
auditor: submitModel("submit_review", { ...verified, dropped: [{ ref: "p2", reason: "evidence not found" }] }),
|
||||
auditor: submitModel("submit_review", verdict),
|
||||
});
|
||||
const stats = await pipeline.runExtraction(chatId);
|
||||
|
||||
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 3 });
|
||||
// The linked proposition created NO new node; the understanding note
|
||||
// became an observation; both quizzes became events.
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 3 });
|
||||
|
||||
// Only the audited proposition became a node; mapping provenance kept.
|
||||
assert.equal(ledgers.readNodes().length, 1);
|
||||
assert.equal(ledgers.readNodes()[0]!.statement, "Chain rule intuition");
|
||||
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "cr", graphNodeId: ledgers.readNodes()[0]!.id }]);
|
||||
// Link-append: the phrasing joined the existing node's aliases and
|
||||
// the provenance unioned; no second node line defines a new node.
|
||||
assert.equal(ledgers.readNodes().length, 2); // the pre-seeded line + the link line
|
||||
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "p1", graphNodeId: existingId }]);
|
||||
|
||||
// Events rebuilt deterministically from the transcript — the probe
|
||||
// retroactively attributed, the check tagged, mark_done mapped.
|
||||
// Events rebuilt deterministically: quizzes in transcript order, then
|
||||
// the observation. check-1's attribution pointed at the DROPPED p2 →
|
||||
// unresolvable → null (the question text is kept for later attribution).
|
||||
const events = ledgers.readEvents();
|
||||
assert.deepEqual(
|
||||
events.map((e) => [e.kind, e.ref]),
|
||||
events.map((e) => [e.kind, e.ref, e.nodeId]),
|
||||
[
|
||||
["probe-miss", "probe-1"],
|
||||
["check-correct", "check-1"],
|
||||
["taught", "taught:cr"],
|
||||
["miss", "probe-1", existingId],
|
||||
["correct", "check-1", null],
|
||||
["observation", "obs:p1", existingId],
|
||||
],
|
||||
);
|
||||
const nodeId = ledgers.readNodes()[0]!.id;
|
||||
assert.equal(events[0]!.nodeId, nodeId, "probe attributed via audit");
|
||||
assert.equal(events[1]!.nodeId, null, "tagged to a dropped proposition → null");
|
||||
assert.equal(events[2]!.nodeId, nodeId, "taught via tree-node mapping");
|
||||
assert.equal(events[0]!.nodeId, existingId, "probe attributed via the verdict's mapping");
|
||||
assert.equal(events[2]!.note, "solid", "the understanding note joins the ledger");
|
||||
|
||||
// Checkpointed through the record's updatedAt — never the same turns twice.
|
||||
assert.equal(ledgers.readCheckpoints().length, 1);
|
||||
@@ -318,7 +336,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
|
||||
chatStore,
|
||||
graphStore: ledgers,
|
||||
extractor: silentModel(),
|
||||
auditor: submitModel("submit_review", FINDINGS),
|
||||
auditor: submitModel("submit_review", { ...FINDINGS, dropped: [] }),
|
||||
});
|
||||
await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/);
|
||||
assert.equal(ledgers.readCheckpoints().length, 0);
|
||||
|
||||
+117
-138
@@ -1,13 +1,14 @@
|
||||
/**
|
||||
* Graph-as-context tests (W8; vision §1 "the graph as session context").
|
||||
* Graph-as-context tests (vision §1 "the graph as session context", 2026-09-02
|
||||
* redesign vocabulary: follows edges, no node kinds).
|
||||
*
|
||||
* renderGraphContext — pure: empty / small / cap-truncation rendering
|
||||
* buildGraphContext — per-request entry point (derived-graph → block)
|
||||
* injection plumbing — chat-flow style: the teacher's instructions carry
|
||||
* the block (MockLanguageModelV4 captures the prompt)
|
||||
* exact-match merge — the merger (mergeFindings) dedupes
|
||||
* case/whitespace-identical statements onto existing
|
||||
* nodes (no fuzzy matching)
|
||||
* cross-chat edges — the merger (mergeFindings) appends one ASSERTION
|
||||
* line per session; reads merge with unioned
|
||||
* message-granular provenance
|
||||
*
|
||||
* Ledger isolation: NONE needed for the merge tests — the merger's ledger is
|
||||
* injectable (in-memory), so the live data/graph files are only ever READ
|
||||
@@ -22,7 +23,14 @@ import * as path from "node:path";
|
||||
import { after, describe, it } from "node:test";
|
||||
import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
|
||||
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
|
||||
import type { GraphEdge, GraphNode, LedgerEvent, SessionFindings } from "../src/core/contracts.ts";
|
||||
import type {
|
||||
AuditResult,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
LedgerEvent,
|
||||
NodeMapping,
|
||||
SessionCheckpoint,
|
||||
} from "../src/core/contracts.ts";
|
||||
import type { NodeStateClass } from "../src/core/contracts.ts";
|
||||
import { getGraph, mergeFindings } from "../src/core/graph.ts";
|
||||
import type { GraphStore } from "../src/core/ports.ts";
|
||||
@@ -35,19 +43,25 @@ import {
|
||||
import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
|
||||
import { handleChatRequest } from "../src/server/chat-handler.ts";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
import type { NodeMapping, SessionCheckpoint } from "../src/core/contracts.ts";
|
||||
import { graphStore } from "../src/store/graph.ts";
|
||||
import { createTree } from "../src/core/plan-tree.ts";
|
||||
import { PATHS } from "../config.ts";
|
||||
|
||||
// ─── fixtures ────────────────────────────────────────────────────────────────
|
||||
|
||||
function node(id: string, statement: string, over: Partial<GraphNode> = {}): GraphNode {
|
||||
return { id, name: statement.split(";")[0]!, statement, aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z", ...over };
|
||||
return {
|
||||
id,
|
||||
statement,
|
||||
description: `${statement} — elaborated.`,
|
||||
aliases: [],
|
||||
createdAt: "2026-08-22T00:00:00.000Z",
|
||||
provenance: [],
|
||||
...over,
|
||||
};
|
||||
}
|
||||
|
||||
function edge(from: string, to: string): GraphEdge {
|
||||
return { from, to, type: "requires", provenance: ["chat_x"] };
|
||||
return { from, to, type: "follows", provenance: [{ chatId: "chat_x", messageId: "m1" }] };
|
||||
}
|
||||
|
||||
function states(...entries: [string, NodeStateClass][]): Record<string, NodeStateClass> {
|
||||
@@ -61,7 +75,7 @@ describe("renderGraphContext (pure)", () => {
|
||||
assert.equal(renderGraphContext([], [], {}), null);
|
||||
});
|
||||
|
||||
it("renders a header, state-grouped node lines (hotspots first), and chained requires edges", () => {
|
||||
it("renders a header, state-grouped node lines (hotspots first), and chained follows edges", () => {
|
||||
const nodes = [
|
||||
node("n_a", "Chain rule intuition"),
|
||||
node("n_b", "Quotient rule"),
|
||||
@@ -73,26 +87,26 @@ describe("renderGraphContext (pure)", () => {
|
||||
["n_a", "known"],
|
||||
["n_b", "missed"],
|
||||
["n_c", "candidate"],
|
||||
["n_d", "taught"],
|
||||
["n_d", "dont-know"],
|
||||
);
|
||||
const out = renderGraphContext(nodes, edges, st)!;
|
||||
|
||||
assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present");
|
||||
assert.match(out, /4 nodes, 3 requires edges/);
|
||||
assert.match(out, /4 nodes, 3 follows edges/);
|
||||
|
||||
// Node lines: "• [state] statement (id)".
|
||||
assert.ok(out.includes("• [missed] Quotient rule (n_b)"));
|
||||
assert.ok(out.includes("• [known] Chain rule intuition (n_a)"));
|
||||
assert.ok(out.includes("• [taught] Composite functions (n_d)"));
|
||||
assert.ok(out.includes("• [dont-know] Composite functions (n_d)"));
|
||||
assert.ok(out.includes("• [candidate] Product rule (n_c)"));
|
||||
|
||||
// Groups in the decided order: hotspots, known, taught, candidates.
|
||||
const order = ["Hotspots", "Known", "Taught", "Candidates"].map((g) => out.indexOf(g));
|
||||
// Groups in the decided order: hotspots, known, candidates.
|
||||
const order = ["Hotspots", "Known", "Candidates"].map((g) => out.indexOf(g));
|
||||
assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order");
|
||||
|
||||
// Edges merged into chains: d → a → b → c (one line, three arrows).
|
||||
assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule"));
|
||||
assert.ok(out.includes("Requires (prerequisite → dependent)"));
|
||||
assert.ok(out.includes("Follows (premise → derivation)"));
|
||||
assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter");
|
||||
});
|
||||
|
||||
@@ -104,15 +118,8 @@ describe("renderGraphContext (pure)", () => {
|
||||
assert.ok(!out.includes("B → C"), "no edge was invented between siblings");
|
||||
});
|
||||
|
||||
it("marks misconception-kind nodes as hotspots even when their state is candidate", () => {
|
||||
const nodes = [node("n_m", "Division by zero is fine", { kind: "misconception" })];
|
||||
const out = renderGraphContext(nodes, [], states(["n_m", "candidate"]))!;
|
||||
assert.match(out, /• \[candidate\] Division by zero is fine \(n_m\) · misconception/);
|
||||
assert.ok(out.includes("Hotspots"), "misconception nodes join the hotspot group");
|
||||
});
|
||||
|
||||
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop requires neighborhood only, with a truncation note", () => {
|
||||
// 3 hotspots, each with a prerequisite and a dependent (1-hop), plus a
|
||||
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop follows neighborhood only, with a truncation note", () => {
|
||||
// 3 hotspots, each with a premise and a derivation (1-hop), plus a
|
||||
// long tail of disconnected candidates that must drop out.
|
||||
const nodes: GraphNode[] = [];
|
||||
const edges: GraphEdge[] = [];
|
||||
@@ -121,7 +128,7 @@ describe("renderGraphContext (pure)", () => {
|
||||
const h = `n_h${i}`;
|
||||
const pre = `n_p${i}`;
|
||||
const dep = `n_d${i}`;
|
||||
nodes.push(node(pre, `Prerequisite ${i}`), node(h, `Hotspot ${i}`), node(dep, `Dependent ${i}`));
|
||||
nodes.push(node(pre, `Premise ${i}`), node(h, `Hotspot ${i}`), node(dep, `Derivation ${i}`));
|
||||
edges.push(edge(pre, h), edge(h, dep));
|
||||
st[pre] = "known";
|
||||
st[h] = "missed";
|
||||
@@ -139,11 +146,11 @@ describe("renderGraphContext (pure)", () => {
|
||||
|
||||
const out = renderGraphContext(nodes, edges, st)!;
|
||||
assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay");
|
||||
assert.ok(out.includes("• [known] Prerequisite 1 (n_p1)"), "1-hop prerequisites stay");
|
||||
assert.ok(out.includes("• [candidate] Dependent 1 (n_d1)"), "1-hop dependents stay");
|
||||
assert.ok(out.includes("• [known] Premise 1 (n_p1)"), "1-hop premises stay");
|
||||
assert.ok(out.includes("• [candidate] Derivation 1 (n_d1)"), "1-hop derivations stay");
|
||||
assert.ok(!out.includes("Far away"), "disconnected nodes drop out");
|
||||
assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted");
|
||||
assert.ok(out.includes(", 6 requires edges"), "edges among the selected nodes stay");
|
||||
assert.ok(out.includes(", 6 follows edges"), "edges among the selected nodes stay");
|
||||
});
|
||||
|
||||
it("keeps the full graph at exactly the cap (no truncation note)", () => {
|
||||
@@ -169,11 +176,11 @@ describe("buildGraphContext", () => {
|
||||
it("derives states from events and renders the block", () => {
|
||||
const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")];
|
||||
const events: LedgerEvent[] = [
|
||||
{ nodeId: "n_b", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
||||
{ nodeId: "n_b", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
||||
];
|
||||
const out = buildGraphContext({ nodes, edges: [], events })!;
|
||||
assert.ok(out.includes(GRAPH_CONTEXT_HEADER));
|
||||
assert.ok(out.includes("• [known] Checked node (n_b)"), "check-correct → known");
|
||||
assert.ok(out.includes("• [known] Checked node (n_b)"), "correct → known");
|
||||
assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate");
|
||||
});
|
||||
|
||||
@@ -244,7 +251,7 @@ describe("graph-context injection (chat-flow style)", () => {
|
||||
nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")],
|
||||
edges: [edge("n_ctx2", "n_ctx")],
|
||||
events: [
|
||||
{ nodeId: "n_ctx2", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
||||
{ nodeId: "n_ctx2", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
||||
] as LedgerEvent[],
|
||||
};
|
||||
const context = buildGraphContext(graph);
|
||||
@@ -296,7 +303,7 @@ describe("graph-context injection (chat-flow style)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── exact-match merge (merger identity resolution) ──────────────────────────
|
||||
// ─── cross-chat edge assertions (merger + read-time merging) ────────────────
|
||||
|
||||
/** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */
|
||||
function memoryLedgers() {
|
||||
@@ -317,128 +324,100 @@ function memoryLedgers() {
|
||||
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
|
||||
readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
|
||||
// Test accessors (not part of the GraphStore surface).
|
||||
counts: { get nodes() { return nodes.length; }, get mappings() { return mappings.length; } },
|
||||
} satisfies GraphStore & { counts: { nodes: number; mappings: number } };
|
||||
counts: { get nodes() { return nodes.length; } },
|
||||
} satisfies GraphStore & { counts: { nodes: number } };
|
||||
}
|
||||
|
||||
/** Findings as the extractor would file them for a plan tree: one
|
||||
* proposition per node (statement = title, treeNodeId set), one `requires`
|
||||
* edge per tree edge — the merge tests drive the merger with these. */
|
||||
function treeFindings(tree: ReturnType<typeof createTree>): SessionFindings {
|
||||
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
|
||||
return {
|
||||
propositions: tree.nodes.map((n, i) => ({
|
||||
id: `p${i + 1}`,
|
||||
name: n.title,
|
||||
statement: n.title,
|
||||
aliases: [],
|
||||
kind: "truth" as const,
|
||||
treeNodeId: n.id,
|
||||
evidence: ["fixture evidence"],
|
||||
})),
|
||||
edges: tree.edges.map((e) => ({
|
||||
from: idByNodeId.get(e.from)!,
|
||||
to: idByNodeId.get(e.to)!,
|
||||
type: "requires" as const,
|
||||
evidence: ["fixture evidence"],
|
||||
})),
|
||||
attribution: { quiz: {} },
|
||||
};
|
||||
}
|
||||
|
||||
/** A real chat record with a stored tree (the merger reads the real chat store). */
|
||||
function fixtureChat(title: string, tree: ReturnType<typeof createTree>) {
|
||||
/** A minimal chat record with one user message (the merger reads the real chat store). */
|
||||
function fixtureChat(title: string) {
|
||||
const chat = chatStore.createChat(title);
|
||||
createdChats.push(chat.id);
|
||||
chatStore.saveTree(chat.id, tree);
|
||||
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]);
|
||||
return chatStore.getChat(chat.id)!;
|
||||
}
|
||||
|
||||
describe("exact-match merge (merger identity resolution)", () => {
|
||||
it("maps a case/whitespace-identical title onto the existing node instead of duplicating", () => {
|
||||
describe("cross-chat edge assertions (merger + read-time merging)", () => {
|
||||
it("a second session asserting the same edge appends a second line; reads merge with unioned provenance", () => {
|
||||
const ledgers = memoryLedgers();
|
||||
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
|
||||
const chat1 = fixtureChat("Merge A", tree1);
|
||||
let stats = mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
|
||||
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 0 });
|
||||
const firstId = ledgers.readNodes()[0]!.id;
|
||||
|
||||
// Second session, same proposition: different casing + padding.
|
||||
const tree2 = createTree([{ id: "cr2", title: " chain Rule INTUITION " }], []);
|
||||
const chat2 = fixtureChat("Merge B", tree2);
|
||||
stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "no duplicate node appended");
|
||||
|
||||
assert.equal(ledgers.counts.nodes, 1, "still exactly one graph node");
|
||||
assert.equal(ledgers.readNodes()[0]!.id, firstId);
|
||||
// Provenance stays honest: one mapping line per (chat, graph node).
|
||||
assert.equal(ledgers.counts.mappings, 2);
|
||||
assert.deepEqual(ledgers.readMappings()[1], { chatId: chat2.id, sourceId: "cr2", graphNodeId: firstId });
|
||||
});
|
||||
|
||||
it("matches aliases too (exact, case-insensitive)", () => {
|
||||
const ledgers = memoryLedgers();
|
||||
ledgers.appendNode({
|
||||
id: "n_alias",
|
||||
name: "Differentiation",
|
||||
statement: "Differentiation basics",
|
||||
aliases: ["the derivative"],
|
||||
kind: "truth",
|
||||
createdAt: "2026-08-22T00:00:00.000Z",
|
||||
});
|
||||
const tree = createTree([{ id: "d", title: "The Derivative" }], []);
|
||||
const chat = fixtureChat("Merge alias", tree);
|
||||
const stats = mergeFindings(chat.id, treeFindings(tree), { chatStore, graphStore: ledgers });
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "alias match → no new node");
|
||||
assert.deepEqual(ledgers.readMappings(), [{ chatId: chat.id, sourceId: "d", graphNodeId: "n_alias" }]);
|
||||
});
|
||||
|
||||
it("does NOT fuzzy-match near-identical titles (that stays deferred)", () => {
|
||||
const ledgers = memoryLedgers();
|
||||
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
|
||||
const chat1 = fixtureChat("Fuzzy A", tree1);
|
||||
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
|
||||
const tree2 = createTree([{ id: "cr2", title: "Chain rules intuition" }], []);
|
||||
const chat2 = fixtureChat("Fuzzy B", tree2);
|
||||
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
|
||||
assert.equal(stats.nodes, 1, "near-miss statements stay separate nodes");
|
||||
assert.equal(ledgers.counts.nodes, 2);
|
||||
});
|
||||
|
||||
it("merged nodes unify the second chat's edges onto the existing graph node", () => {
|
||||
const ledgers = memoryLedgers();
|
||||
const tree1 = createTree(
|
||||
[
|
||||
{ id: "base", title: "Derivative as a limit" },
|
||||
{ id: "chain", title: "Chain rule intuition" },
|
||||
// Session 1: two fresh propositions + a follows edge between them.
|
||||
const chat1 = fixtureChat("Edge A");
|
||||
const verdict1: AuditResult = {
|
||||
propositions: [
|
||||
{
|
||||
id: "p1",
|
||||
statement: "Derivative as a limit",
|
||||
description: "The derivative is the limit of difference quotients.",
|
||||
aliases: [],
|
||||
messages: ["u1"],
|
||||
evidence: ["e"],
|
||||
},
|
||||
{
|
||||
id: "p2",
|
||||
statement: "Chain rule intuition",
|
||||
description: "Composed functions differentiate by multiplying local rates.",
|
||||
aliases: [],
|
||||
messages: ["u1"],
|
||||
evidence: ["e"],
|
||||
},
|
||||
],
|
||||
[{ from: "base", to: "chain" }],
|
||||
);
|
||||
const chat1 = fixtureChat("Edge A", tree1);
|
||||
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
|
||||
edges: [{ from: "p1", to: "p2", type: "follows", messages: ["u1"], evidence: ["e"] }],
|
||||
attribution: { quiz: {} },
|
||||
dropped: [],
|
||||
};
|
||||
let stats = mergeFindings(chat1.id, verdict1, { chatStore, graphStore: ledgers });
|
||||
assert.deepEqual(stats, { nodes: 2, edges: 1, events: 0 });
|
||||
const base = ledgers.readNodes()[0]!.id;
|
||||
const chain = ledgers.readNodes()[1]!.id;
|
||||
|
||||
// Second session re-teaches the same two propositions (exact statements
|
||||
// modulo case) and asserts the same edge — it must dedupe onto the
|
||||
// existing nodes.
|
||||
const tree2 = createTree(
|
||||
[
|
||||
{ id: "base", title: "derivative as a limit" },
|
||||
{ id: "chain", title: "chain rule intuition" },
|
||||
// Session 2: the auditor links both propositions to the SAME nodes
|
||||
// (identity verdicts) and asserts the same relation.
|
||||
const chat2 = fixtureChat("Edge B");
|
||||
const verdict2: AuditResult = {
|
||||
propositions: [
|
||||
{
|
||||
id: "q1",
|
||||
statement: "The derivative as a limit of quotients", // different phrasing
|
||||
description: "…",
|
||||
aliases: [],
|
||||
messages: ["u1"],
|
||||
evidence: ["e"],
|
||||
linkTo: base,
|
||||
},
|
||||
{
|
||||
id: "q2",
|
||||
statement: "Chain rule, intuitively",
|
||||
description: "…",
|
||||
aliases: [],
|
||||
messages: ["u1"],
|
||||
evidence: ["e"],
|
||||
linkTo: chain,
|
||||
},
|
||||
],
|
||||
[{ from: "base", to: "chain" }],
|
||||
);
|
||||
const chat2 = fixtureChat("Edge B", tree2);
|
||||
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
|
||||
// A new ASSERTION line is appended (provenance is honest) — reads merge
|
||||
// by (from,to,type), so the derived graph still has one edge with both
|
||||
// sessions in its provenance.
|
||||
edges: [{ from: "q1", to: "q2", type: "follows", messages: ["u1"], evidence: ["e"] }],
|
||||
attribution: { quiz: {} },
|
||||
dropped: [],
|
||||
};
|
||||
stats = mergeFindings(chat2.id, verdict2, { chatStore, graphStore: ledgers });
|
||||
|
||||
// No new nodes; ONE new assertion line (per-session honest provenance).
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line");
|
||||
assert.equal(ledgers.counts.nodes, 2);
|
||||
assert.equal(ledgers.counts.nodes, 4, "2 defining lines + 2 link-appends for the linked propositions");
|
||||
const lines = ledgers.readEdges();
|
||||
assert.equal(lines.length, 2, "two assertion lines for the same edge");
|
||||
assert.equal(lines[0]!.from, lines[1]!.from);
|
||||
assert.equal(lines[0]!.to, lines[1]!.to);
|
||||
assert.deepEqual(lines.map((l) => l.provenance), [[chat1.id], [chat2.id]]);
|
||||
assert.deepEqual(lines.map((l) => l.provenance), [
|
||||
[{ chatId: chat1.id, messageId: "u1" }],
|
||||
[{ chatId: chat2.id, messageId: "u1" }],
|
||||
]);
|
||||
|
||||
// Read-time merging: one derived edge, both sessions in its provenance.
|
||||
const derived = getGraph(ledgers).edges;
|
||||
assert.equal(derived.length, 1);
|
||||
assert.deepEqual(derived[0]!.provenance, [
|
||||
{ chatId: chat1.id, messageId: "u1" },
|
||||
{ chatId: chat2.id, messageId: "u1" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -9,7 +9,10 @@ import { isNodeVisible } from "../components/graph-view.tsx";
|
||||
import type { NodeStateClass } from "../src/core/index.ts";
|
||||
|
||||
const none: ReadonlySet<NodeStateClass> = new Set();
|
||||
const node = { name: "Money multiplier", statement: "The money multiplier is the maximum ratio of broad money to base money." };
|
||||
const node = {
|
||||
statement: "The money multiplier is the maximum ratio of broad money to base money.",
|
||||
description: "How much broad money banks create from each unit of base money.",
|
||||
};
|
||||
|
||||
describe("isNodeVisible (legend/filters predicate)", () => {
|
||||
it("is visible with no filters", () => {
|
||||
@@ -26,11 +29,11 @@ describe("isNodeVisible (legend/filters predicate)", () => {
|
||||
assert.equal(isNodeVisible(node, "taught", hidden, ""), true);
|
||||
});
|
||||
|
||||
it("search matches the name case-insensitively", () => {
|
||||
it("search matches the statement case-insensitively", () => {
|
||||
assert.equal(isNodeVisible(node, "candidate", none, "MONEY multiplier"), true);
|
||||
});
|
||||
|
||||
it("search matches the statement when the name does not", () => {
|
||||
it("search matches the description when the statement does not", () => {
|
||||
assert.equal(isNodeVisible(node, "candidate", none, "maximum ratio"), true);
|
||||
assert.equal(isNodeVisible(node, "candidate", none, " velocity"), false);
|
||||
});
|
||||
|
||||
+220
-164
@@ -1,9 +1,9 @@
|
||||
/**
|
||||
* Graph merger + trigger + derivation tests (vision §1; contracts.ts
|
||||
* "extraction pipeline"). The merger (core/graph.ts mergeFindings) is the
|
||||
* pipeline's deterministic apply stage: findings in, ledger appends out —
|
||||
* identity resolution, retroactive probe attribution, event rebuild, and
|
||||
* the idempotency guarantees.
|
||||
* Graph merger + trigger + derivation tests (vision §1, 2026-09-02 redesign).
|
||||
* The merger (core/graph.ts mergeFindings) is the pipeline's deterministic
|
||||
* apply stage: AUDITED VERDICTS in, ledger appends out — link-or-create
|
||||
* identity (the auditor's linkTo, never string matching), deterministic quiz
|
||||
* events, understanding observations, and the idempotency guarantees.
|
||||
*
|
||||
* Ledger isolation: PATHS is fixed at import time, so the five data/graph
|
||||
* ledgers are backed up and cleared before the suite and restored after
|
||||
@@ -18,9 +18,8 @@ import assert from "node:assert/strict";
|
||||
import * as fs from "node:fs";
|
||||
import { after, describe, it } from "node:test";
|
||||
import type { UIMessage } from "ai";
|
||||
import type { PlanTree, PlanToolResult, SessionFindings } from "../src/core/contracts.ts";
|
||||
import type { AuditResult } from "../src/core/contracts.ts";
|
||||
import type { QuestionAnswerData } from "../src/core/question-core.ts";
|
||||
import { createTree, markDone } from "../src/core/plan-tree.ts";
|
||||
import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
|
||||
import { graphStore } from "../src/store/graph.ts";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
@@ -80,16 +79,6 @@ function quizPart(
|
||||
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
|
||||
}
|
||||
|
||||
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: PlanTree): unknown {
|
||||
return {
|
||||
type: `tool-${name}`,
|
||||
toolCallId,
|
||||
state: "output-available",
|
||||
input,
|
||||
output: { ok: true, tree } satisfies PlanToolResult,
|
||||
};
|
||||
}
|
||||
|
||||
function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
|
||||
return {
|
||||
questionId: "q",
|
||||
@@ -102,59 +91,17 @@ function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
|
||||
};
|
||||
}
|
||||
|
||||
/** Create a chat in the real store with a tree and messages; returns the record. */
|
||||
function fixtureChat(title: string, tree: PlanTree | null, messages: UIMessage[]) {
|
||||
/** Create a chat in the real store with messages; returns the record.
|
||||
* Tree-blind world: the plan tree is irrelevant to the merger — no tree. */
|
||||
function fixtureChat(title: string, messages: UIMessage[]) {
|
||||
const chat = chatStore.createChat(title);
|
||||
createdChats.push(chat.id);
|
||||
if (tree) chatStore.saveTree(chat.id, tree);
|
||||
chatStore.saveMessages(chat.id, messages);
|
||||
return chatStore.getChat(chat.id)!;
|
||||
}
|
||||
|
||||
/** The fixture tree: alpha → beta, gamma free-floating, alpha done. */
|
||||
function fixtureTree(): PlanTree {
|
||||
return markDone(
|
||||
createTree(
|
||||
[
|
||||
{ id: "alpha", title: "Alpha intuition" },
|
||||
{ id: "beta", title: "Beta rule" },
|
||||
{ id: "gamma", title: "Gamma application" },
|
||||
],
|
||||
[{ from: "alpha", to: "beta" }],
|
||||
),
|
||||
"alpha",
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Findings shaped exactly like what the extractor would produce for a
|
||||
* session's plan tree: one proposition per tree node (statement = title,
|
||||
* treeNodeId set), one `requires` edge per tree edge.
|
||||
*/
|
||||
function treeFindings(tree: PlanTree): SessionFindings {
|
||||
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
|
||||
return {
|
||||
propositions: tree.nodes.map((n, i) => ({
|
||||
id: `p${i + 1}`,
|
||||
name: n.title,
|
||||
statement: n.title,
|
||||
aliases: [],
|
||||
kind: "truth" as const,
|
||||
treeNodeId: n.id,
|
||||
evidence: ["fixture evidence"],
|
||||
})),
|
||||
edges: tree.edges.map((e) => ({
|
||||
from: idByNodeId.get(e.from)!,
|
||||
to: idByNodeId.get(e.to)!,
|
||||
type: "requires" as const,
|
||||
evidence: ["fixture evidence"],
|
||||
})),
|
||||
attribution: { quiz: {} },
|
||||
};
|
||||
}
|
||||
|
||||
/** The full fixture transcript: probes → plan → checks → mark_done → noise. */
|
||||
function fixtureMessages(tree: PlanTree): UIMessage[] {
|
||||
/** The full fixture transcript: five graded quizzes + a dismissed one + an ask. */
|
||||
function fixtureMessages(): UIMessage[] {
|
||||
return [
|
||||
user("u1", "teach me the fixture topic"),
|
||||
assistant("a1", [
|
||||
@@ -171,32 +118,30 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
|
||||
answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
assistant("a3", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
|
||||
assistant("a4", [
|
||||
assistant("a3", [
|
||||
// input.nodeId "alpha" is deliberately present and IGNORED —
|
||||
// attribution is the only quiz→node path in the tree-blind world.
|
||||
quizPart(
|
||||
"call-3",
|
||||
{ question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
assistant("a5", [
|
||||
assistant("a4", [
|
||||
quizPart(
|
||||
"call-4",
|
||||
{ question: "Q4", nodeId: "beta", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
{ question: "Q4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
assistant("a6", [
|
||||
// nodeId present but NOT in the tree → tagged-but-unmapped: the
|
||||
// attribution can rescue it, or it lands null + question text.
|
||||
assistant("a5", [
|
||||
quizPart(
|
||||
"call-5",
|
||||
{ question: "Q5", nodeId: "ghost", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
{ question: "Q5", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
assistant("a7", [planPart("p2", "plan_mark_done", { id: "alpha" }, tree)]),
|
||||
assistant("a8", [
|
||||
assistant("a6", [
|
||||
// dismissed quiz + an answered ask — neither becomes an event
|
||||
quizPart(
|
||||
"call-6",
|
||||
@@ -214,25 +159,58 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
|
||||
];
|
||||
}
|
||||
|
||||
/**
|
||||
* The extractor's findings for the fixture chat: the tree as propositions +
|
||||
* the probe attribution. The audit stage is a no-op pass-through in these
|
||||
* merger tests — its behavior is tested with the pipeline (extraction.test).
|
||||
*/
|
||||
function fixtureFindings(): SessionFindings {
|
||||
const findings = treeFindings(fixtureTree());
|
||||
// Untagged probe quizzes: call-1 attributed retroactively to Alpha (the
|
||||
// vision's open question — resolved by the extractor); call-2 genuinely
|
||||
// unattributable; call-5's ghost nodeId is rescued onto Gamma.
|
||||
findings.attribution.quiz = { "call-1": "p1", "call-2": null, "call-5": "p3" };
|
||||
return findings;
|
||||
/** The auditor's verdict for the fixture chat: three verified propositions
|
||||
* (no linkTo — the graph starts empty), one follows edge, all quizzes
|
||||
* attributed, one understanding note. */
|
||||
function fixtureAudit(): AuditResult {
|
||||
return {
|
||||
propositions: [
|
||||
{
|
||||
id: "p1",
|
||||
statement: "Alpha intuition",
|
||||
description: "Alpha intuition, elaborated: what alpha means and how it behaves in practice.",
|
||||
aliases: ["alpha feel"],
|
||||
messages: ["a1", "a3"],
|
||||
evidence: ["fixture evidence"],
|
||||
understanding: "solid after the check",
|
||||
},
|
||||
{
|
||||
id: "p2",
|
||||
statement: "Beta rule",
|
||||
description: "The beta rule and when it applies.",
|
||||
aliases: [],
|
||||
messages: ["a4"],
|
||||
evidence: ["fixture evidence"],
|
||||
},
|
||||
{
|
||||
id: "p3",
|
||||
statement: "Gamma application",
|
||||
description: "Applying the whole framework to gamma cases.",
|
||||
aliases: [],
|
||||
messages: ["a5"],
|
||||
evidence: ["fixture evidence"],
|
||||
},
|
||||
],
|
||||
edges: [
|
||||
{
|
||||
from: "p1",
|
||||
to: "p2",
|
||||
type: "follows",
|
||||
messages: ["a4"],
|
||||
evidence: ["fixture evidence"],
|
||||
},
|
||||
],
|
||||
attribution: { quiz: { "call-1": "p1", "call-2": null, "call-3": "p1", "call-4": "p2", "call-5": "p3" } },
|
||||
dropped: [],
|
||||
};
|
||||
}
|
||||
|
||||
/** The chat every merger test folds (shared, so ordering is intentional). */
|
||||
const chat1 = fixtureChat("Graph fixture", fixtureTree(), fixtureMessages(fixtureTree()));
|
||||
const chat1 = fixtureChat("Graph fixture", fixtureMessages());
|
||||
|
||||
/** Set by the cross-session test; the snapshot test (which runs after it) reads it. */
|
||||
/** Set by the cross-session tests; the snapshot test (which runs after) reads them. */
|
||||
let chat2Id = "";
|
||||
let chat3Id = "";
|
||||
|
||||
// ─── the merger ──────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -248,71 +226,83 @@ function scopedGraph(chatIds: string[]) {
|
||||
);
|
||||
return {
|
||||
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
|
||||
edges: graph.edges.filter((e) => e.provenance.some((c) => chatIds.includes(c))),
|
||||
edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
|
||||
events: graph.events.filter((e) => chatIds.includes(e.chatId)),
|
||||
};
|
||||
}
|
||||
|
||||
describe("the merger (mergeFindings)", () => {
|
||||
it("merges propositions/edges by identity and rebuilds events with retroactive probe attribution", () => {
|
||||
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
|
||||
it("creates nodes from the verdicts, rebuilds quiz events deterministically, records observations", () => {
|
||||
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
|
||||
assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 });
|
||||
|
||||
const graph = scopedGraph([chat1.id]);
|
||||
assert.equal(graph.nodes.length, 3);
|
||||
const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id]));
|
||||
for (const node of graph.nodes) {
|
||||
assert.equal(node.kind, "truth");
|
||||
assert.deepEqual(node.aliases, []);
|
||||
assert.match(node.id, /^n_/);
|
||||
assert.ok(node.description.length > 0, "description is required");
|
||||
assert.equal("kind" in node, false, "no kind field");
|
||||
assert.ok(Array.isArray(node.provenance), "message-granular provenance");
|
||||
}
|
||||
// Provenance is message-granular: p1 came up in a1 and a3.
|
||||
const alpha = graph.nodes.find((n) => n.statement === "Alpha intuition")!;
|
||||
assert.deepEqual(alpha.provenance, [
|
||||
{ chatId: chat1.id, messageId: "a1" },
|
||||
{ chatId: chat1.id, messageId: "a3" },
|
||||
]);
|
||||
|
||||
// The tree edge alpha→beta becomes a `requires` edge on the mapped nodes.
|
||||
// The follows edge, provenance attached.
|
||||
assert.equal(graph.edges.length, 1);
|
||||
const edge = graph.edges[0]!;
|
||||
assert.equal(edge.from, idByStatement.get("Alpha intuition"));
|
||||
assert.equal(edge.to, idByStatement.get("Beta rule"));
|
||||
assert.equal(edge.type, "requires");
|
||||
assert.deepEqual(edge.provenance, [chat1.id]);
|
||||
assert.equal(edge.type, "follows");
|
||||
assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "a4" }]);
|
||||
|
||||
// Mapping lines: one per (chat, sourceId).
|
||||
assert.deepEqual(
|
||||
graphStore.readMappings().map((m) => m.sourceId),
|
||||
["p1", "p2", "p3"],
|
||||
);
|
||||
|
||||
// Events — kinds, refs, notes, and the attribution paths.
|
||||
assert.equal(graph.events.length, 6);
|
||||
const byRef = new Map(graph.events.map((e) => [e.ref, e]));
|
||||
|
||||
const probeMiss = byRef.get("call-1")!;
|
||||
assert.equal(probeMiss.kind, "probe-miss"); // quiz BEFORE plan_init
|
||||
assert.equal(probeMiss.nodeId, idByStatement.get("Alpha intuition"), "retroactive probe attribution");
|
||||
assert.equal(probeMiss.question, undefined, "attributed → no question text needed");
|
||||
const miss = byRef.get("call-1")!;
|
||||
assert.equal(miss.kind, "miss");
|
||||
assert.equal(miss.nodeId, idByStatement.get("Alpha intuition"), "attributed quiz");
|
||||
|
||||
const probeDontKnow = byRef.get("call-2")!;
|
||||
assert.equal(probeDontKnow.kind, "dont-know"); // same kind, both phases
|
||||
assert.equal(probeDontKnow.nodeId, null, "unattributable probe → null");
|
||||
assert.equal(probeDontKnow.question, "Q2 text");
|
||||
const dontKnow = byRef.get("call-2")!;
|
||||
assert.equal(dontKnow.kind, "dont-know");
|
||||
assert.equal(dontKnow.nodeId, null, "unattributable → null");
|
||||
assert.equal(dontKnow.question, "Q2 text");
|
||||
|
||||
const check = byRef.get("call-3")!;
|
||||
assert.equal(check.kind, "check-correct"); // quiz AFTER plan_init
|
||||
assert.equal(check.nodeId, idByStatement.get("Alpha intuition"), "tagged quiz → deterministic");
|
||||
assert.equal(check.note, "recalled the diagram");
|
||||
const correct = byRef.get("call-3")!;
|
||||
assert.equal(correct.kind, "correct");
|
||||
assert.equal(correct.nodeId, idByStatement.get("Alpha intuition"), "attribution wins over the ignored input.nodeId");
|
||||
assert.equal(correct.note, "recalled the diagram");
|
||||
|
||||
const checkMiss = byRef.get("call-4")!;
|
||||
assert.equal(checkMiss.kind, "check-incorrect");
|
||||
assert.equal(checkMiss.nodeId, idByStatement.get("Beta rule"));
|
||||
const miss2 = byRef.get("call-4")!;
|
||||
assert.equal(miss2.kind, "miss");
|
||||
assert.equal(miss2.nodeId, idByStatement.get("Beta rule"));
|
||||
|
||||
const ghost = byRef.get("call-5")!;
|
||||
assert.equal(ghost.kind, "dont-know");
|
||||
assert.equal(ghost.nodeId, idByStatement.get("Gamma application"), "tagged-but-unmapped rescued by attribution");
|
||||
assert.equal(ghost.question, undefined);
|
||||
const dontKnow2 = byRef.get("call-5")!;
|
||||
assert.equal(dontKnow2.kind, "dont-know");
|
||||
assert.equal(dontKnow2.nodeId, idByStatement.get("Gamma application"));
|
||||
|
||||
const taught = byRef.get("taught:alpha")!;
|
||||
assert.equal(taught.kind, "taught");
|
||||
assert.equal(taught.nodeId, idByStatement.get("Alpha intuition"), "mark_done → tree-node mapping");
|
||||
const observation = byRef.get("obs:p1")!;
|
||||
assert.equal(observation.kind, "observation");
|
||||
assert.equal(observation.nodeId, idByStatement.get("Alpha intuition"));
|
||||
assert.equal(observation.note, "solid after the check");
|
||||
|
||||
// The dismissed quiz and the ask question became nothing.
|
||||
assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0);
|
||||
});
|
||||
|
||||
it("is idempotent — re-merging the same findings appends nothing", () => {
|
||||
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
|
||||
it("is idempotent — re-merging the same verdicts appends nothing", () => {
|
||||
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 });
|
||||
const graph = scopedGraph([chat1.id]);
|
||||
assert.equal(graph.nodes.length, 3);
|
||||
@@ -322,38 +312,63 @@ describe("the merger (mergeFindings)", () => {
|
||||
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
|
||||
});
|
||||
|
||||
it("merges a second session onto the same nodes and accumulates edge provenance", () => {
|
||||
// chat2 re-teaches the same two propositions (identical statements) —
|
||||
// the conservative identity resolution recognizes them.
|
||||
const tree2 = createTree(
|
||||
[
|
||||
{ id: "a", title: "Alpha intuition" },
|
||||
{ id: "b", title: "Beta rule" },
|
||||
],
|
||||
[{ from: "a", to: "b" }],
|
||||
);
|
||||
const chat2 = fixtureChat("Graph fixture 2", tree2, [
|
||||
it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
|
||||
const alphaId = scopedGraph([chat1.id]).nodes.find((n) => n.statement === "Alpha intuition")!.id;
|
||||
const chat2 = fixtureChat("Graph fixture 2", [
|
||||
user("u1", "again"),
|
||||
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree2)]),
|
||||
assistant("a1", [
|
||||
quizPart(
|
||||
"call-9",
|
||||
{ question: "Q9", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-9", questionText: "Q9 text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
]);
|
||||
|
||||
const stats = mergeFindings(chat2.id, treeFindings(tree2), graphDeps);
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 });
|
||||
chat2Id = chat2.id;
|
||||
|
||||
// The auditor's identity verdict: this proposition IS the alpha node.
|
||||
const stats = mergeFindings(
|
||||
chat2.id,
|
||||
{
|
||||
propositions: [
|
||||
{
|
||||
id: "q1",
|
||||
statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings
|
||||
description: "The intuitive grasp of alpha.",
|
||||
aliases: ["alpha intuition"],
|
||||
messages: ["a1"],
|
||||
evidence: ["fixture evidence"],
|
||||
linkTo: alphaId,
|
||||
},
|
||||
],
|
||||
edges: [],
|
||||
attribution: { quiz: { "call-9": "q1" } },
|
||||
dropped: [],
|
||||
},
|
||||
graphDeps,
|
||||
);
|
||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 1 });
|
||||
|
||||
const after = scopedGraph([chat1.id, chat2.id]);
|
||||
assert.equal(after.nodes.length, 3); // no duplicate nodes
|
||||
assert.equal(after.edges.length, 1); // one merged edge…
|
||||
assert.deepEqual(after.edges[0]!.provenance, [chat1.id, chat2.id]); // …two sessions
|
||||
assert.equal(after.nodes.length, 3, "no duplicate node");
|
||||
const alpha = getGraph(graphStore).nodes.find((n) => n.id === alphaId)!;
|
||||
assert.ok(alpha.aliases.includes("Alpha feel"), "the session's phrasing joins the aliases");
|
||||
assert.ok(alpha.aliases.includes("alpha intuition"));
|
||||
assert.ok(
|
||||
alpha.provenance.some((pt) => pt.chatId === chat2.id && pt.messageId === "a1"),
|
||||
"provenance unions",
|
||||
);
|
||||
// The mapping records the verdict: (chat2, q1) → the linked node.
|
||||
const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
|
||||
assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
|
||||
});
|
||||
|
||||
it("derives node states, sessions, and node provenance in graphSnapshot", () => {
|
||||
it("derives node states and sessions in graphSnapshot", () => {
|
||||
const snap = graphSnapshot(graphDeps);
|
||||
const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id]));
|
||||
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "taught");
|
||||
// Alpha's latest quiz is call-9 (chat2, correct) → known.
|
||||
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "known");
|
||||
assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed");
|
||||
// Gamma: no taught/checks — but the rescued dont-know probe (call-5)
|
||||
// makes its latest event a dont-know.
|
||||
assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know");
|
||||
|
||||
assert.equal(snap.sessions[chat1.id], "Graph fixture");
|
||||
@@ -366,8 +381,35 @@ describe("the merger (mergeFindings)", () => {
|
||||
assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]);
|
||||
});
|
||||
|
||||
it("never string-matches without a verdict — an identical statement without linkTo is a NEW node", () => {
|
||||
const chat3 = fixtureChat("Graph fixture 3", [user("u1", "third")]);
|
||||
chat3Id = chat3.id;
|
||||
const stats = mergeFindings(
|
||||
chat3.id,
|
||||
{
|
||||
propositions: [
|
||||
{
|
||||
id: "r1",
|
||||
statement: "Alpha intuition", // IDENTICAL to the existing node's statement
|
||||
description: "Same words, no verdict.",
|
||||
aliases: [],
|
||||
messages: ["u1"],
|
||||
evidence: ["fixture evidence"],
|
||||
// no linkTo — identity is the auditor's call alone
|
||||
},
|
||||
],
|
||||
edges: [],
|
||||
attribution: { quiz: {} },
|
||||
dropped: [],
|
||||
},
|
||||
graphDeps,
|
||||
);
|
||||
assert.equal(stats.nodes, 1, "created — the merger does not match strings");
|
||||
assert.equal(getGraph(graphStore).nodes.filter((n) => n.statement === "Alpha intuition").length, 2);
|
||||
});
|
||||
|
||||
it("returns zeros for unknown chats", () => {
|
||||
assert.deepEqual(mergeFindings("chat_does_not_exist", treeFindings(fixtureTree()), graphDeps), {
|
||||
assert.deepEqual(mergeFindings("chat_does_not_exist", fixtureAudit(), graphDeps), {
|
||||
nodes: 0,
|
||||
edges: 0,
|
||||
events: 0,
|
||||
@@ -379,14 +421,12 @@ describe("the merger (mergeFindings)", () => {
|
||||
|
||||
describe("the trigger helper (pendingQuizAnswers)", () => {
|
||||
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
|
||||
const tree = createTree([{ id: "solo", title: "Solo node" }], []);
|
||||
const chat = fixtureChat("Trigger fixture", tree, [
|
||||
const chat = fixtureChat("Trigger fixture", [
|
||||
user("u1", "go"),
|
||||
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
|
||||
assistant("a2", [
|
||||
assistant("a1", [
|
||||
quizPart(
|
||||
"call-s",
|
||||
{ question: "QS", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
{ question: "QS", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
|
||||
),
|
||||
quizPart(
|
||||
@@ -407,7 +447,9 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
|
||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
|
||||
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
|
||||
|
||||
mergeFindings(chat.id, treeFindings(tree), graphDeps);
|
||||
mergeFindings(chat.id, fixtureAudit(), graphDeps);
|
||||
// call-s is unattributed here (fixtureAudit has no call-s attribution)
|
||||
// — but it still becomes an event, so nothing stays pending.
|
||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
|
||||
|
||||
// A NEW answered quiz raises the count again (the re-trigger path).
|
||||
@@ -416,7 +458,7 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
|
||||
assistant("a3", [
|
||||
quizPart(
|
||||
"call-s4",
|
||||
{ question: "QS4", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
{ question: "QS4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
@@ -429,29 +471,43 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
|
||||
|
||||
describe("graph ledger store", () => {
|
||||
it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => {
|
||||
const node = { id: "n_storetest", name: "Store test", statement: "Store test", aliases: [], kind: "truth" as const, createdAt: "2026-08-22T00:00:00.000Z" };
|
||||
const node = {
|
||||
id: "n_storetest",
|
||||
statement: "Store test",
|
||||
description: "Store test, elaborated.",
|
||||
aliases: [] as string[],
|
||||
createdAt: "2026-08-22T00:00:00.000Z",
|
||||
provenance: [{ chatId: "chat_c", messageId: "m1" }],
|
||||
};
|
||||
graphStore.appendNode(node);
|
||||
// A torn append and a legacy V0-shaped node (different schema) — both skipped.
|
||||
// A torn append and a legacy pre-redesign node (old schema) — both skipped.
|
||||
fs.appendFileSync(
|
||||
LEDGER_FILES.nodes,
|
||||
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", domain: "x", state: "known" })}\n`,
|
||||
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", kind: "truth", name: "Legacy", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
|
||||
"utf-8",
|
||||
);
|
||||
const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy");
|
||||
assert.equal(nodes.length, 1);
|
||||
assert.equal(nodes[0]!.statement, "Store test");
|
||||
assert.equal(nodes[0]!.name, "Store test");
|
||||
assert.deepEqual(nodes[0]!.provenance, [{ chatId: "chat_c", messageId: "m1" }]);
|
||||
});
|
||||
|
||||
it("derives a name for pre-name ledger lines (legacy reads)", () => {
|
||||
// A full, valid node line written before `name` existed.
|
||||
fs.appendFileSync(
|
||||
LEDGER_FILES.nodes,
|
||||
`${JSON.stringify({ id: "n_prename", statement: "Derivatives measure instantaneous change; they are limits.", aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
|
||||
"utf-8",
|
||||
);
|
||||
const node = graphStore.readNodes().find((n) => n.id === "n_prename");
|
||||
assert.ok(node);
|
||||
assert.equal(node.name, "Derivatives measure instantaneous change");
|
||||
it("returns RAW node lines — repeated ids (link-appends) are each their own line; getGraph merges", () => {
|
||||
const line = (msgId: string) =>
|
||||
JSON.stringify({
|
||||
id: "n_linktest",
|
||||
statement: `Phrasing ${msgId}`,
|
||||
description: "Elaboration.",
|
||||
aliases: [],
|
||||
createdAt: "2026-08-22T00:00:00.000Z",
|
||||
provenance: [{ chatId: "chat_c", messageId: msgId }],
|
||||
});
|
||||
fs.appendFileSync(LEDGER_FILES.nodes, `${line("m1")}\n${line("m2")}\n`, "utf-8");
|
||||
const lines = graphStore.readNodes().filter((n) => n.id === "n_linktest");
|
||||
assert.equal(lines.length, 2, "the store returns raw lines, one per append");
|
||||
const merged = getGraph(graphStore).nodes.find((n) => n.id === "n_linktest")!;
|
||||
assert.equal(merged.statement, "Phrasing m1", "the FIRST line defines the node");
|
||||
assert.ok(merged.aliases.includes("Phrasing m2"), "later phrasings join the aliases");
|
||||
assert.deepEqual(merged.provenance.map((p) => p.messageId), ["m1", "m2"]);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user