Graph redesign: true propositions, follows/supports, auditor-owned identity (vision §1 minimal implementation)
The minimal implementation of the 2026-09-02 graph vision (docs/vision.md §1):
- contracts/graph.ts: every node is a TRUE proposition (statement is both
rendering label and identity key; required description; no kind, no name).
Edges are follows (derivable) / supports (grounds without entailing) with
message-granular provenance points {chatId, messageId}. Events reduce to
correct/miss/dont-know plus observation (understanding notes); no taught
events, no misconception kind.
- extraction.ts: extractor is TREE-BLIND (transcript only, no plan tree) and
free-form; auditor sees findings + the current graph, owns faithfulness and
identity (link-or-create verdict per proposition, recorded as linkTo);
transcriptView tags every line with its message id (provenance anchors).
- core/graph.ts merger: applies verdicts deterministically — linkTo appends
a link line (phrasings join aliases, provenance unions; reads merge by id),
absent creates; NO string matching anywhere. Quiz events rebuilt from tool
parts with attribution-only nodeId resolution; understanding notes become
observation events (obs:<sourceId>); mapping lines are the verdict audit
trail and make re-merges idempotent.
- deriveNodeStates: transitional — latest quiz event wins (known/missed/
dont-know, else candidate); taught unreachable until the state-model
ideation lands.
- graph-context.ts + 50-graph-context.md: follows chains, hotspot grouping,
state vocabulary updated for the new event kinds.
- graph-view.tsx: node label is the statement; detail shows statement +
description.
- tests: merger verdicts (link, no-string-match, cross-chat assertions),
tree-blind pipeline end-to-end, follows vocabulary in context tests.
276/276 green.
This commit is contained in:
@@ -114,7 +114,7 @@ graph TB
|
|||||||
LEDG --> SNAP --> RGRAPH
|
LEDG --> SNAP --> RGRAPH
|
||||||
```
|
```
|
||||||
|
|
||||||
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer. The LLM stages propose and verify (propositions, edges, probe attribution — each with verbatim evidence); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs) from the transcript. The teacher stays a teacher; everything below the ledgers is derived.
|
**Core invariant:** the extraction pipeline (extractor → auditor → merger) is the sole graph writer — tree-blind and post-hoc. The LLM stages propose, verify, and judge identity (link-or-create against the current graph; each finding carries verbatim evidence and message-granular provenance); the deterministic merger owns every ledger append and rebuilds anything the model could silently mis-copy (event kinds, timestamps, idempotency refs, provenance) from the transcript. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger. The teacher stays a teacher; everything below the ledgers is derived.
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
@@ -165,7 +165,7 @@ data/ chats/ · graph/ · jobs/ — runtime state, git
|
|||||||
|
|
||||||
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts).
|
**The async sub-agent protocol (ack-and-steer).** `research` / `demo` / `visual` are server-executed tools: the call resolves immediately mid-turn with a fire-and-forget ack and the teacher keeps teaching; a background runner works in parallel (the job store tracks a live activity line; the client polls `/api/jobs`). On completion the runner builds a **steer message** — a user-role `UIMessage` whose text part is the model-facing result (data parts are dropped by provider conversion) and whose `data-subagent-artifact` part carries the full result for the client — and the **wake pipeline** (`server/jobs.ts`) delivers it: a per-chat serialized queue waits for the turn to end and no question to be pending, then fires a server-side teacher turn that weaves the result into the lesson. The client adopts the wake turn through the same `turnInFlight` machinery as returning mid-stream (busy shimmer, composer gate) and renders steers as artifacts — never a bubble. Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its HTML in system Chrome before delivery; the visual-maker renders each SVG and looks at it (needs a vision-capable model — `MODELS.visualMaker` in config.ts).
|
||||||
|
|
||||||
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM: propositions, edges, probe attribution, each with verbatim evidence) → auditor (LLM: drops anything unbacked) → merger (`mergeFindings`, deterministic: exact/alias identity resolution, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
|
**The knowledge graph (the two channels).** The write channel: the extraction pipeline — extractor (LLM, tree-blind: propositions as true statements + descriptions, follows/supports edges, quiz attribution, understanding notes) → auditor (LLM: drops anything unbacked, stamps each survivor with an identity verdict — link to an existing node or create) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, event rebuild from the transcript's tool parts, dedupe, checkpoint). Triggered once a chat accumulates `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) new answered quizzes — a v1 placeholder trigger isolated in one seam (`turn-service.ts`), swappable without touching the pipeline. Failure checkpoints nothing; the next trigger re-runs. The read channel: the home view derives node states from the ledgers (`GET /api/graph` is a pure read — forceAtlas2 run-then-freeze, no idle simulation), and at session start the derived graph is rendered into a compact markdown block appended to the teacher's instructions (`buildGraphContext`) — whole graph while small, hotspots + 1-hop neighborhood past 150 nodes.
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
|
|||||||
@@ -363,7 +363,7 @@ export function GraphView() {
|
|||||||
// frozen layout stable for a given node set across remounts.
|
// frozen layout stable for a given node set across remounts.
|
||||||
const rand = seededRandom(node.id);
|
const rand = seededRandom(node.id);
|
||||||
graph.addNode(node.id, {
|
graph.addNode(node.id, {
|
||||||
label: node.name,
|
label: node.statement,
|
||||||
size: STATE_TOKENS[state].size,
|
size: STATE_TOKENS[state].size,
|
||||||
color: stateColors.get(state) ?? edgeColor,
|
color: stateColors.get(state) ?? edgeColor,
|
||||||
x: (rand() - 0.5) * 100,
|
x: (rand() - 0.5) * 100,
|
||||||
@@ -552,8 +552,8 @@ export function GraphView() {
|
|||||||
style={{ visibility: "hidden", left: 0, top: 0 }}
|
style={{ visibility: "hidden", left: 0, top: 0 }}
|
||||||
className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg"
|
className="absolute z-10 w-80 max-w-[calc(100%-1rem)] rounded-xl border bg-card p-4 text-left shadow-lg"
|
||||||
>
|
>
|
||||||
<div className="text-sm font-medium leading-snug">{selected.node.name}</div>
|
<div className="text-sm font-medium leading-snug">{selected.node.statement}</div>
|
||||||
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.statement}</div>
|
<div className="mt-1 text-xs leading-relaxed text-muted-foreground">{selected.node.description}</div>
|
||||||
<div className="mt-1 text-xs uppercase tracking-wide text-muted-foreground">
|
<div className="mt-1 text-xs uppercase tracking-wide text-muted-foreground">
|
||||||
{STATE_LABELS[selected.state]}
|
{STATE_LABELS[selected.state]}
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -75,11 +75,11 @@ chat record ──▶ EXTRACTOR (LLM) ──▶ AUDITOR (LLM) ──▶ MERGER (
|
|||||||
proposes verifies applies
|
proposes verifies applies
|
||||||
```
|
```
|
||||||
|
|
||||||
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned* (propositions, incl. misconceptions and prose-only `supports` edges), *what depends on what*, and *which proposition an untagged probe tested*. Everything the model could silently mis-copy — event kinds, timestamps, idempotency refs, notes, question texts — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (the faithfulness net) drops anything unbacked and logs every drop. Conservative throughout: duplicates over wrong merges, drop over invent.
|
**Tree-blind and post-hoc** (2026-09-02 redesign, docs/vision.md §1): the extractor reads ONLY the transcript — never the session's plan tree — and proposes free-form propositions (true statements with descriptions, misconception-free: a misconception is stated as its true correction), `follows`/`supports` edges, per-proposition understanding notes, and message-granular provenance. Every node is an objective, world-facing fact; the learner's relationship to it lives entirely in the event ledger.
|
||||||
|
|
||||||
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — statement/alias exact-match identity resolution (fuzzy/embedding adjudication deferred), one mapping line per (chat, graph node) as the merge audit trail, edge dedupe by `(from, to, type)` with per-chat provenance, event dedupe by chat-scoped ref, and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
|
**Division of trust** (the pipeline's core rule): the LLM stages supply only judgment — *what was learned*, *what follows/supports what*, *which proposition each quiz tested* (retroactively, against the whole transcript), and — the auditor's — *identity*: whether a proposition IS an existing graph node (link-or-create). Everything the model could silently mis-copy — event kinds (`correct` / `miss` / `dont-know`), timestamps, idempotency refs, notes, provenance points — is rebuilt deterministically by the merger from the transcript's tool parts. The extractor files verbatim evidence quotes; the auditor (which sees the extractor's output PLUS the current graph) drops anything unbacked, logs every drop, and stamps each surviving proposition with its identity verdict (`linkTo`). Conservative throughout: duplicates over wrong merges, drop over invent.
|
||||||
|
|
||||||
**Probe attribution** (vision open question, resolved structurally): tagged events (quiz `nodeId`, `plan_mark_done`'s tree node) resolve deterministically through the proposition mapping. The extractor's judgment is consulted only for **untagged probe quizzes** — retroactively, against the whole transcript, so a pre-tree probe maps to a proposition the tree only articulates later. Attribution can also rescue a tagged-but-unknown id; genuinely unattributable probes land `nodeId: null` with the question text, exactly as they always have.
|
**Merger invariants** (`src/core/graph.ts`, `mergeFindings`): idempotent and incremental — identity comes from the auditor's `linkTo` verdict alone (validated against the ledger; never string matching), one mapping line per (chat, sourceId) as the verdict audit trail (which also makes re-merging the same chat idempotent), link-appends: a linked proposition appends a node line whose phrasings join the node's `aliases` and whose provenance unions (reads merge by id), edge dedupe by `(from, to, type)`-already-asserted-by-this-chat with message-granular provenance, event dedupe by chat-scoped ref (quiz toolCallId, `obs:<sourceId>` for understanding observations), and a sessions.jsonl checkpoint (`foldedAt`) marking which turns are processed. Failure anywhere before the merger checkpoints nothing; the next trigger re-runs.
|
||||||
|
|
||||||
**The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes.
|
**The trigger is a v1 placeholder, isolated in ONE seam.** `src/server/turn-service.ts` (`triggerExtraction`) counts answered quizzes not yet in the ledger (`pendingQuizAnswers`, core) and fires the pipeline async once a chat crosses `EXTRACTION.everyNQuizAnswers` (default 3, config.ts) — single-flight per chat. The pipeline is trigger-agnostic: when a better mechanism lands (session end, idle, explicit flush), only this seam changes.
|
||||||
|
|
||||||
|
|||||||
@@ -1,20 +1,19 @@
|
|||||||
# The learner's knowledge graph
|
# The learner's knowledge graph
|
||||||
|
|
||||||
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes and plan trees, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Requires** means *a* must be understood before *b*.
|
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
|
||||||
|
|
||||||
The states, exactly:
|
The states, exactly:
|
||||||
|
|
||||||
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
|
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
|
||||||
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
|
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
|
||||||
- **[known]** — answered correctly. Foundation you can build on and refer to by name.
|
- **[known]** — answered correctly. Foundation you can build on and refer to by name.
|
||||||
- **[taught]** — a lesson covered it. Check that it stuck before stacking anything heavy on it.
|
- **[candidate]** — present in the graph but never probed. Assume nothing.
|
||||||
- **[candidate]** — asserted into a plan but never probed. Assume nothing.
|
|
||||||
|
|
||||||
This block is what makes probing precise instead of generic — use it:
|
This block is what makes probing precise instead of generic — use it:
|
||||||
|
|
||||||
- **Probe the frontier, not the interior.** [known] and [taught] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
|
- **Probe the frontier, not the interior.** [known] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
|
||||||
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
|
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
|
||||||
- **Read the Requires chains as scaffolding.** They show what the learner's knowledge rests on. If a prerequisite under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
|
- **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
|
||||||
|
|
||||||
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
|
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
|
||||||
|
|
||||||
|
|||||||
+152
-114
@@ -1,23 +1,28 @@
|
|||||||
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
|
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
|
||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
import { modelFor } from "../models.ts";
|
import { modelFor } from "../models.ts";
|
||||||
import type { AuditResult, ChatRecord, SessionFindings } from "../../core/contracts.ts";
|
import type { AuditResult, ChatRecord, GraphNode, SessionFindings } from "../../core/contracts.ts";
|
||||||
import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
|
import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The extraction pipeline (vision §1 construction) — the sole graph writer.
|
* The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
|
||||||
* Three stages, one orchestrator:
|
* sole graph writer. Three stages, one orchestrator:
|
||||||
*
|
*
|
||||||
* extractor (LLM) transcript → SessionFindings (propositions, edges,
|
* extractor (LLM) transcript → SessionFindings (free-form propositions,
|
||||||
* event attribution — each with verbatim evidence)
|
* follows/supports edges, quiz attribution, understanding
|
||||||
* auditor (LLM) findings → AuditResult (the verified subset + drops)
|
* notes — each with verbatim evidence). TREE-BLIND: it
|
||||||
|
* sees only the transcript, never the plan tree.
|
||||||
|
* auditor (LLM) findings + THE CURRENT GRAPH → AuditResult: the
|
||||||
|
* verified subset with an identity verdict per
|
||||||
|
* proposition (linkTo an existing node, or create), plus
|
||||||
|
* the drops for the log.
|
||||||
* merger (core) mergeFindings() → deterministic ledger appends
|
* merger (core) mergeFindings() → deterministic ledger appends
|
||||||
*
|
*
|
||||||
* The LLM stages supply only the judgment calls; the deterministic rebuild
|
* The LLM stages supply only the judgment calls; the deterministic rebuild
|
||||||
* in the merger owns everything that must never be mis-copied (event kinds,
|
* in the merger owns everything that must never be mis-copied (event kinds,
|
||||||
* timestamps, refs). Both agents are single-tool-call workers: their whole
|
* timestamps, refs, provenance points). Both agents are single-tool-call
|
||||||
* deliverable is one zod-validated submit call — a ToolLoopAgent purely so
|
* workers: their whole deliverable is one zod-validated submit call — a
|
||||||
* the model can retry a schema-invalid call.
|
* ToolLoopAgent purely so the model can retry a schema-invalid call.
|
||||||
*
|
*
|
||||||
* Created via createExtractionPipeline(deps) — the composition root
|
* Created via createExtractionPipeline(deps) — the composition root
|
||||||
* (chat-handler) injects the storage ports + models; this module never
|
* (chat-handler) injects the storage ports + models; this module never
|
||||||
@@ -27,7 +32,7 @@ import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/gr
|
|||||||
* the next trigger re-runs the whole thing.
|
* the next trigger re-runs the whole thing.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
// ─── the transcript view (what the LLM sees) ────────────────────────────────
|
// ─── the transcript view (what the extractor + auditor see) ─────────────────
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A quiz tool part's outcome, one line: the attribution evidence.
|
* A quiz tool part's outcome, one line: the attribution evidence.
|
||||||
@@ -44,46 +49,35 @@ function quizLine(part: { toolCallId: string; input: unknown; output: unknown })
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The compact transcript the extractor + auditor both see: user/teacher
|
* The compact transcript both LLM stages see: user/teacher text VERBATIM,
|
||||||
* text verbatim (evidence quotes must match it), the final plan tree, and
|
* each tagged with its message id (provenance anchors — the extractor
|
||||||
* the structured tool events (quiz outcomes, mark_done). Job tool parts
|
* reports them back per proposition/edge), and quiz outcome lines.
|
||||||
* (research/demo/visual) are noise for extraction — their outcomes are
|
* TREE-BLIND by design (2026-09-02): no plan tree, no mark_done —
|
||||||
* summarized as one line each.
|
* extraction is free-form, decoupled from the session's plan. Job tool
|
||||||
|
* parts (research/demo/visual) are noise for extraction — their outcomes
|
||||||
|
* are summarized as one line each.
|
||||||
*/
|
*/
|
||||||
export function transcriptView(record: ChatRecord): string {
|
export function transcriptView(record: ChatRecord): string {
|
||||||
const lines: string[] = [];
|
const lines: string[] = [];
|
||||||
lines.push(`SESSION: ${record.title}`);
|
lines.push(`SESSION: ${record.title}`);
|
||||||
|
|
||||||
if (record.tree && record.tree.nodes.length > 0) {
|
record.messages.forEach((message, index) => {
|
||||||
const nodes = record.tree.nodes.map((n) => `${n.id} = ${n.title}`).join("; ");
|
|
||||||
lines.push(`FINAL PLAN TREE (node ids): ${nodes}`);
|
|
||||||
if (record.tree.edges.length > 0) {
|
|
||||||
const edges = record.tree.edges.map((e) => `${e.from} requires ${e.to}`).join("; ");
|
|
||||||
lines.push(`TREE EDGES: ${edges}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
for (const message of record.messages as UIMessage[]) {
|
|
||||||
const who = message.role === "user" ? "LEARNER" : "TEACHER";
|
const who = message.role === "user" ? "LEARNER" : "TEACHER";
|
||||||
for (const part of message.parts) {
|
const mid = (message as UIMessage).id || `m${index}`;
|
||||||
|
for (const part of (message as UIMessage).parts) {
|
||||||
if (part.type === "text" && part.text.trim()) {
|
if (part.type === "text" && part.text.trim()) {
|
||||||
lines.push(`${who}: ${part.text.trim()}`);
|
lines.push(`${who} [${mid}] ${part.text.trim()}`);
|
||||||
} else if (part.type.startsWith("tool-")) {
|
} else if (part.type.startsWith("tool-")) {
|
||||||
const name = part.type.slice("tool-".length);
|
const name = part.type.slice("tool-".length);
|
||||||
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
|
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
|
||||||
if (name === "quiz" && p.state === "output-available") {
|
if (name === "quiz" && p.state === "output-available") {
|
||||||
lines.push(quizLine(p));
|
lines.push(quizLine(p));
|
||||||
} else if (name === "plan_mark_done" && p.state === "output-available") {
|
|
||||||
const input = p.input as { id?: unknown } | undefined;
|
|
||||||
lines.push(`TEACHER marked tree node done [${p.toolCallId}]: ${typeof input?.id === "string" ? input.id : "?"}`);
|
|
||||||
} else if (name.startsWith("plan_")) {
|
|
||||||
// plan ops are folded into the FINAL PLAN TREE line above
|
|
||||||
} else if (p.state === "output-available") {
|
} else if (p.state === "output-available") {
|
||||||
lines.push(`(${name} tool result delivered)`);
|
lines.push(`(${name} tool result delivered)`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
});
|
||||||
return lines.join("\n");
|
return lines.join("\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -94,87 +88,98 @@ const evidence = z
|
|||||||
.min(1)
|
.min(1)
|
||||||
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
|
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
|
||||||
|
|
||||||
const findingsSchema = z.object({
|
const messagesField = z
|
||||||
propositions: z
|
.array(z.string())
|
||||||
.array(
|
.describe("The message ids (from the [id] tags in the transcript) where this came up — provenance.");
|
||||||
z.object({
|
|
||||||
|
const propositionSchema = z.object({
|
||||||
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
|
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
|
||||||
name: z.string().describe(
|
|
||||||
"A short 2-6 word name for the fact — its header, used for rendering. Not a sentence.",
|
|
||||||
),
|
|
||||||
statement: z
|
statement: z
|
||||||
.string()
|
.string()
|
||||||
.describe(
|
.describe(
|
||||||
"The proposition as one reasoning step — an OBJECTIVE, world-facing fact. " +
|
"The proposition as one reasoning step — a TRUE, self-contained, world-facing statement. " +
|
||||||
"Never reference the learner, the teacher, or the session: not \"the learner " +
|
"Never reference the learner, the teacher, or the session: not \"the learner understands X\" but " +
|
||||||
'understands X\" but \"X\"; not \"the learner thinks Y\" but the wrong idea itself, ' +
|
'"X". A misconception is stated as the TRUE correction ("Nietzsche did not endorse ' +
|
||||||
'stated as a claim (\"Reading: Nietzsche endorsed might-makes-right social Darwinism\"), ' +
|
'might-makes-right social Darwinism") — the wrongness lives in the quiz outcomes, never here.',
|
||||||
"with kind \"misconception\" carrying that it is wrong.",
|
|
||||||
),
|
),
|
||||||
aliases: z.array(z.string()).describe("Verbatim alternate phrasings from this session, if any."),
|
description: z
|
||||||
kind: z.enum(["truth", "misconception"]).describe("misconception = a wrong idea the learner holds or was warned about."),
|
.string()
|
||||||
treeNodeId: z.string().optional().describe("The plan-tree node id this came from, when it did."),
|
.describe(
|
||||||
|
"The elaborated version of the same proposition — more words: scope, precision, what exactly it means. Still true, still world-facing.",
|
||||||
|
),
|
||||||
|
aliases: z.array(z.string()).describe("Verbatim phrasings this session used for the same proposition, if any."),
|
||||||
|
messages: messagesField,
|
||||||
evidence,
|
evidence,
|
||||||
}),
|
understanding: z
|
||||||
)
|
.string()
|
||||||
.describe("Every distinct proposition the session touched — from the plan tree AND from prose."),
|
.optional()
|
||||||
|
.describe(
|
||||||
|
'One note about THIS session\'s evidence of the learner\'s grasp of this proposition, on a named-word scale ' +
|
||||||
|
'("solid", "shaky", "accepted", "resisted", "not yet engaged") — no numbers, no predictions.',
|
||||||
|
),
|
||||||
|
});
|
||||||
|
|
||||||
|
const edgeSchema = z.object({
|
||||||
|
from: z.string().describe("Proposition id that is followed / supported."),
|
||||||
|
to: z.string().describe("Proposition id that follows / is supported by `from`."),
|
||||||
|
type: z
|
||||||
|
.enum(["follows", "supports"])
|
||||||
|
.describe('"follows" = to is derivable from from; "supports" = from grounds to without entailing it.'),
|
||||||
|
messages: messagesField,
|
||||||
|
evidence,
|
||||||
|
});
|
||||||
|
|
||||||
|
const findingsSchema = z.object({
|
||||||
|
propositions: z
|
||||||
|
.array(propositionSchema)
|
||||||
|
.describe("Every distinct proposition the session touched — true statements, misconception-free."),
|
||||||
edges: z
|
edges: z
|
||||||
.array(
|
.array(edgeSchema)
|
||||||
z.object({
|
.describe("Relations between propositions: follows (derivable) / supports (grounds without entailing)."),
|
||||||
from: z.string().describe("Proposition id that is the prerequisite / support."),
|
|
||||||
to: z.string().describe("Proposition id that depends on / is supported by `from`."),
|
|
||||||
type: z.enum(["requires", "supports"]),
|
|
||||||
evidence,
|
|
||||||
}),
|
|
||||||
)
|
|
||||||
.describe("Relations between propositions. `requires` = prerequisite; `supports` = helpful but not prerequisite. Include relations the plan tree does NOT show when the transcript articulates them."),
|
|
||||||
attribution: z
|
attribution: z
|
||||||
.object({
|
.object({
|
||||||
quiz: z
|
quiz: z
|
||||||
.record(z.string(), z.string().nullable())
|
.record(z.string(), z.string().nullable())
|
||||||
.describe("toolCallId of every UNTAGGED quiz line → the proposition id it was really testing. Probe questions asked BEFORE the tree existed are attributed RETROACTIVELY — the whole transcript is visible; map them to the proposition they were probing (often articulated only later). null when genuinely unattributable. Tagged quizzes (with a tree nodeId in the transcript view) resolve automatically — do not include them."),
|
.describe(
|
||||||
|
"toolCallId of EVERY quiz line → the proposition id it was really testing. Attribute RETROACTIVELY — the whole transcript is visible; map each quiz to the proposition it tests (often articulated elsewhere in the session). null when genuinely unattributable.",
|
||||||
|
),
|
||||||
})
|
})
|
||||||
.describe("Probe attribution — your only event judgment."),
|
.describe("Quiz attribution — your only event judgment."),
|
||||||
});
|
});
|
||||||
|
|
||||||
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
|
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
|
||||||
transcript and report what was learned — as structured findings for the learner's global knowledge graph.
|
transcript and report what was learned — as structured findings for the learner's global knowledge graph.
|
||||||
|
|
||||||
The transcript view shows: the session's final plan tree (dependency map, with node ids), the conversation
|
The transcript view shows: the conversation verbatim (LEARNER / TEACHER lines, each tagged with its message id)
|
||||||
verbatim (LEARNER / TEACHER), and structured events (QUIZ lines with the learner's outcome, mark_done lines).
|
and QUIZ outcome lines.
|
||||||
|
|
||||||
Report, with verbatim evidence quotes for everything:
|
Report, with verbatim evidence quotes for everything:
|
||||||
|
|
||||||
NODE THEORY — the graph is a set of objective facts, full stop. A node is what is true (or falsely
|
NODE THEORY — the graph is a set of TRUE propositions about the world, full stop. A node is never anything
|
||||||
believed) about the world, never anything about the learner or the session. The learner's
|
about the learner or the session. The learner's relationship to a proposition — whether they know it, missed
|
||||||
relationship to a fact — whether they know it, missed it, or hold it as a misconception — is
|
it, resisted it — is expressed ENTIRELY by the quiz outcomes and your understanding note; it never leaks into
|
||||||
expressed ENTIRELY by the quiz-outcome events you attribute; it never leaks into the node itself.
|
the node itself. So no "the learner understands X", no "discussed in this session": the node is just X. A
|
||||||
So no "the learner understands X", no "discussed in this session": the node is just X. A
|
misconception is NOT a node of its own wrong claim — state the TRUE correction as the proposition, and let the
|
||||||
misconception node states the wrong idea itself as a claim, with kind "misconception" marking it
|
quiz outcomes carry that the learner believed the wrong version.
|
||||||
wrong — never "the learner believes Y".
|
|
||||||
|
|
||||||
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
|
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
|
||||||
- Each carries a "name": a short 2-6 word header (the rendering handle), and a "statement": the
|
- Each is a self-contained TRUE statement ("statement"), plus its elaboration ("description" — more
|
||||||
full fact as one objective reasoning step.
|
words, same proposition: scope, precision, what exactly it means).
|
||||||
- Plan-tree nodes become propositions (pass their treeNodeId) — statement from the node title, enriched to a
|
- Include corrections of misconceptions the teacher articulated, and true facts explained in prose.
|
||||||
complete proposition if the prose elaborates it.
|
|
||||||
- Propositions that appeared only in prose (teacher explanations, learner questions) get fresh ids.
|
|
||||||
- Include MISCONCEPTIONS — wrong ideas the learner showed or the teacher warned against — with kind
|
|
||||||
"misconception".
|
|
||||||
- One reasoning step per proposition: never a topic blob, never a sub-step.
|
- One reasoning step per proposition: never a topic blob, never a sub-step.
|
||||||
- aliases: verbatim alternate phrasings this session used for the same thing.
|
- "messages": the [id] tags of every message where the proposition came up — provenance, report it.
|
||||||
|
- "aliases": verbatim phrasings this session used for the same proposition.
|
||||||
|
- "understanding": ONE named-word-scale note on the evidence of the learner's grasp, for this session
|
||||||
|
only ("solid", "shaky", "accepted", "resisted", "not yet engaged"). No numbers, no predictions.
|
||||||
|
|
||||||
2. EDGES between propositions:
|
2. EDGES between propositions:
|
||||||
- The tree's dependency edges are the backbone: from = prerequisite, to = dependent, type "requires".
|
- "follows": to is DERIVABLE from from (understanding from makes understanding to achievable).
|
||||||
- Also include relations articulated only in prose. Use "supports" when the transcript presents something as
|
- "supports": from GROUNDS to without entailing it (helpful context, not a prerequisite).
|
||||||
helpful but NOT a prerequisite.
|
- Only relations the transcript actually articulates — with evidence.
|
||||||
|
|
||||||
3. ATTRIBUTION — UNTAGGED quizzes only:
|
3. ATTRIBUTION — every QUIZ line gets a mapping: toolCallId → the proposition id it was really testing.
|
||||||
- QUIZ lines whose question carries a tree nodeId resolve automatically; do not attribute those.
|
Attribute RETROACTIVELY — a quiz often tests a proposition the session only articulates later; the
|
||||||
- The others are PROBE-phase questions (asked before the tree was initialized) that test prerequisite
|
question text is your guide. If you truly cannot tell, use null.
|
||||||
understanding. Attribute them RETROACTIVELY to the proposition they were really probing — even if that
|
|
||||||
proposition is only articulated later in the session. The QUIZ question text is your guide. If you truly
|
|
||||||
cannot tell, use null.
|
|
||||||
|
|
||||||
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
|
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
|
||||||
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
|
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
|
||||||
@@ -205,6 +210,16 @@ function createExtractor(model: LanguageModel) {
|
|||||||
// ─── the auditor ───────────────────────────────────────────────────────────
|
// ─── the auditor ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
const auditSchema = findingsSchema.extend({
|
const auditSchema = findingsSchema.extend({
|
||||||
|
propositions: z.array(
|
||||||
|
propositionSchema.extend({
|
||||||
|
linkTo: z
|
||||||
|
.string()
|
||||||
|
.optional()
|
||||||
|
.describe(
|
||||||
|
"The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. When unsure, omit: duplicates over wrong merges.",
|
||||||
|
),
|
||||||
|
}),
|
||||||
|
),
|
||||||
dropped: z
|
dropped: z
|
||||||
.array(
|
.array(
|
||||||
z.object({
|
z.object({
|
||||||
@@ -216,27 +231,46 @@ const auditSchema = findingsSchema.extend({
|
|||||||
});
|
});
|
||||||
|
|
||||||
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
|
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
|
||||||
ONE session transcript and a set of proposed findings (propositions, edges, event attribution). Your job is the
|
ONE session transcript, a set of proposed findings (propositions, edges, quiz attribution), and the CURRENT
|
||||||
faithfulness net: verify every candidate against the transcript, keep only what is backed, and return the
|
knowledge graph. Your jobs: the faithfulness net, and identity.
|
||||||
verified subset.
|
|
||||||
|
|
||||||
Verify:
|
FAITHFULNESS — verify every candidate against the transcript:
|
||||||
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement must
|
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement
|
||||||
be a faithful reading of it. The node theory applies: a statement must be an OBJECTIVE, world-facing fact —
|
must be a faithful reading of it. The node theory applies: a statement must be a TRUE, self-contained,
|
||||||
reword learner- or session-relative statements ("the learner understands X" → "X") or drop them when the
|
world-facing fact — reword learner- or session-relative statements ("the learner understands X" → "X"), state
|
||||||
objective fact is not actually backed. Drop invented or bloated statements. Conservative: when two proposed propositions
|
misconceptions as the true correction, or drop them when no objective fact is backed. Drop invented or
|
||||||
are really one, keep the better-stated one and drop the other.
|
bloated statements. Conservative: when two proposed propositions are really one, keep the better-stated one
|
||||||
- EDGES: the evidence must actually articulate the relation, in the direction claimed (from = prerequisite).
|
and drop the other.
|
||||||
Drop direction-swapped or speculative edges.
|
- EDGES: the evidence must actually articulate the relation, in the direction claimed. Drop
|
||||||
- ATTRIBUTION: only UNTAGGED quizzes carry attribution — the QUIZ line's question must genuinely test the
|
direction-swapped or speculative edges.
|
||||||
attributed proposition. Drop wrong attributions to null, or re-point them to the proposition the question
|
- ATTRIBUTION: the QUIZ line's question must genuinely test the attributed proposition. Drop wrong
|
||||||
actually tests.
|
attributions to null, or re-point them to the proposition the question actually tests.
|
||||||
|
- UNDERSTANDING notes: verify against the quiz outcomes and the session's evidence; reword to a named-word
|
||||||
|
scale or drop.
|
||||||
|
|
||||||
|
IDENTITY — link-or-create, per proposition:
|
||||||
|
- You see the current graph (every node: id, statement, description, aliases). For each verified proposition,
|
||||||
|
decide: does an EXISTING node already state this same proposition?
|
||||||
|
- If yes: set "linkTo" to that node's id. The session's phrasings will join its aliases and its provenance.
|
||||||
|
- If no: omit "linkTo" and a new node is created.
|
||||||
|
- When unsure, omit — duplicates are recoverable later; wrong merges are not.
|
||||||
|
|
||||||
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
|
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
|
||||||
well-evidenced proposition is still knowledge.
|
well-evidenced proposition is still knowledge.
|
||||||
|
|
||||||
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
|
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
|
||||||
(the same shapes you received) plus the dropped list.`;
|
(the same shapes you received, with linkTo added) plus the dropped list.`;
|
||||||
|
|
||||||
|
/** The current graph, compact — the auditor's identity context. */
|
||||||
|
function graphView(nodes: GraphNode[]): string {
|
||||||
|
if (nodes.length === 0) return "(the graph is currently empty — every proposition is a new node)";
|
||||||
|
const lines = nodes.map(
|
||||||
|
(n) =>
|
||||||
|
`- ${n.id} :: ${n.statement}\n description: ${n.description}` +
|
||||||
|
(n.aliases.length > 0 ? `\n aliases: ${n.aliases.join(" | ")}` : ""),
|
||||||
|
);
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
function createAuditor(model: LanguageModel) {
|
function createAuditor(model: LanguageModel) {
|
||||||
return new ToolLoopAgent({
|
return new ToolLoopAgent({
|
||||||
@@ -244,7 +278,7 @@ function createAuditor(model: LanguageModel) {
|
|||||||
instructions: AUDITOR_INSTRUCTIONS,
|
instructions: AUDITOR_INSTRUCTIONS,
|
||||||
tools: {
|
tools: {
|
||||||
submit_review: tool({
|
submit_review: tool({
|
||||||
description: "Submit the verified findings subset + the dropped list.",
|
description: "Submit the verified findings subset (with identity verdicts) + the dropped list.",
|
||||||
inputSchema: auditSchema,
|
inputSchema: auditSchema,
|
||||||
execute: async (review) => review,
|
execute: async (review) => review,
|
||||||
}),
|
}),
|
||||||
@@ -295,17 +329,20 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
|||||||
throw new Error("the extractor never submitted findings");
|
throw new Error("the extractor never submitted findings");
|
||||||
}
|
}
|
||||||
|
|
||||||
async function audit(chatId: string, findings: SessionFindings): Promise<SessionFindings> {
|
async function audit(chatId: string, findings: SessionFindings): Promise<AuditResult> {
|
||||||
const record = chatStore.getChat(chatId);
|
const record = chatStore.getChat(chatId);
|
||||||
if (!record) throw new Error(`Chat not found: ${chatId}`);
|
if (!record) throw new Error(`Chat not found: ${chatId}`);
|
||||||
const view = transcriptView(record);
|
const view = transcriptView(record);
|
||||||
|
const graph = getGraph(graphStore);
|
||||||
console.info(
|
console.info(
|
||||||
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
|
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
|
||||||
`${Object.keys(findings.attribution.quiz).length} attributions)`,
|
`${Object.keys(findings.attribution.quiz).length} attributions, ${graph.nodes.length} graph nodes)`,
|
||||||
);
|
);
|
||||||
|
|
||||||
const review = await auditorAgent.generate({
|
const review = await auditorAgent.generate({
|
||||||
prompt: `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}`,
|
prompt:
|
||||||
|
`Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}\n\n` +
|
||||||
|
`Current knowledge graph (${graph.nodes.length} nodes):\n\n${graphView(graph.nodes)}`,
|
||||||
});
|
});
|
||||||
let auditResult: AuditResult | null = null;
|
let auditResult: AuditResult | null = null;
|
||||||
for (const part of review.content) {
|
for (const part of review.content) {
|
||||||
@@ -320,12 +357,12 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
|||||||
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
|
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
const { dropped: _dropped, ...verified } = auditResult;
|
|
||||||
console.info(
|
console.info(
|
||||||
`[extraction ${chatId}] audit kept ${verified.propositions.length}/${findings.propositions.length} propositions, ` +
|
`[extraction ${chatId}] audit kept ${auditResult.propositions.length}/${findings.propositions.length} propositions, ` +
|
||||||
`${verified.edges.length}/${findings.edges.length} edges`,
|
`${auditResult.edges.length}/${findings.edges.length} edges, ` +
|
||||||
|
`${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes`,
|
||||||
);
|
);
|
||||||
return verified;
|
return auditResult;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function runExtraction(chatId: string): Promise<EmissionStats> {
|
async function runExtraction(chatId: string): Promise<EmissionStats> {
|
||||||
@@ -334,10 +371,10 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
|||||||
try {
|
try {
|
||||||
console.info(`[extraction ${chatId}] pipeline start`);
|
console.info(`[extraction ${chatId}] pipeline start`);
|
||||||
const findings = await extract(chatId);
|
const findings = await extract(chatId);
|
||||||
const verified = await audit(chatId, findings);
|
const auditResult = await audit(chatId, findings);
|
||||||
// The merger re-reads the chat record itself — events are rebuilt
|
// The merger re-reads the chat record itself — events are rebuilt
|
||||||
// from the transcript, never from stale copies.
|
// from the transcript, never from stale copies.
|
||||||
const stats = mergeFindings(chatId, verified, { chatStore, graphStore });
|
const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore });
|
||||||
console.info(`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events`);
|
console.info(`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events`);
|
||||||
return stats;
|
return stats;
|
||||||
} finally {
|
} finally {
|
||||||
@@ -347,3 +384,4 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
|
|||||||
|
|
||||||
return { runExtraction };
|
return { runExtraction };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+129
-78
@@ -1,50 +1,69 @@
|
|||||||
/**
|
/**
|
||||||
* Knowledge-graph contracts — nodes, ledgers, extraction findings
|
* Knowledge-graph contracts — nodes, ledgers, extraction findings
|
||||||
*
|
*
|
||||||
|
* The minimal implementation of the graph vision (docs/vision.md §1, the
|
||||||
|
* 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
|
||||||
|
* (derivable) and supports (grounds without entailing); provenance is
|
||||||
|
* message-granular; extraction is free-form and post-hoc, fully decoupled
|
||||||
|
* from the session's plan tree; the event ledger records quiz outcomes plus
|
||||||
|
* the extractor's per-proposition understanding observations. Anything not
|
||||||
|
* yet decided (the state model, strength derivation, understanding
|
||||||
|
* treatment, selection strategies) is deliberately NOT here — transitional
|
||||||
|
* stubs carry the renderer until those land.
|
||||||
|
*
|
||||||
* Part of the domain-split contracts (src/core/contracts/): each domain file
|
* Part of the domain-split contracts (src/core/contracts/): each domain file
|
||||||
* is owned outright by its stream — cross-domain schema changes go through
|
* is owned outright by its stream — cross-domain schema changes go through
|
||||||
* master. Pure types + protocol docs + tiny pure helpers; type-only imports
|
* master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md.
|
||||||
* of "ai" (safe everywhere: server, client, tests). See docs/vision.md.
|
|
||||||
*/
|
*/
|
||||||
// ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
|
// ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
|
||||||
|
|
||||||
export type EdgeType = "requires" | "supports";
|
/** Two relations only (DECIDED 2026-09-02): follows = b is derivable from a;
|
||||||
|
* supports = a grounds b without entailing it. No scalars, no encoded
|
||||||
|
* strength — strength is derived at read time from topology. */
|
||||||
|
export type EdgeType = "follows" | "supports";
|
||||||
|
|
||||||
|
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
|
||||||
|
* edges record every (chat, message) where they came up, deep-linkable to
|
||||||
|
* the exact transcript point. */
|
||||||
|
export interface ProvenancePoint {
|
||||||
|
chatId: string;
|
||||||
|
messageId: string;
|
||||||
|
}
|
||||||
|
|
||||||
export interface GraphNode {
|
export interface GraphNode {
|
||||||
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */
|
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */
|
||||||
id: string;
|
id: string;
|
||||||
/** Short name / header for the fact (2-6 words) — the rendering handle.
|
/** The minimal proposition — one true, self-contained statement. This is
|
||||||
* NODE THEORY: a node is an OBJECTIVE fact, never learner- or
|
* BOTH the rendering label AND the identity key: the auditor recognizes
|
||||||
* session-relative. No "the learner understands…", no "this session
|
* a re-encounter by judgment (link-or-create), not by string matching. */
|
||||||
* covered…" — just the fact, world-facing. The learner's relationship
|
|
||||||
* to the fact lives ENTIRELY in the event ledger: state labels are
|
|
||||||
* derived from quiz outcomes, never written into the node. */
|
|
||||||
name: string;
|
|
||||||
/** Canonical proposition statement — the full fact, one reasoning step,
|
|
||||||
* also objective (no learner / teacher / session references). */
|
|
||||||
statement: string;
|
statement: string;
|
||||||
/** Verbatim alternate statements from sessions (merge target audit trail). */
|
/** The elaborated version — required. Same proposition, more words:
|
||||||
|
* scope, precision, the "what exactly does it mean". */
|
||||||
|
description: string;
|
||||||
|
/** Verbatim phrasings sessions used for this proposition (the identity
|
||||||
|
* audit trail — how the world said what the node says). */
|
||||||
aliases: string[];
|
aliases: string[];
|
||||||
kind: "truth" | "misconception";
|
|
||||||
createdAt: string;
|
createdAt: string;
|
||||||
|
/** Every (chat, message) point where this proposition came up. */
|
||||||
|
provenance: ProvenancePoint[];
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface GraphEdge {
|
export interface GraphEdge {
|
||||||
/** Prerequisite GraphNode id. */
|
|
||||||
from: string;
|
from: string;
|
||||||
to: string;
|
to: string;
|
||||||
type: EdgeType;
|
type: EdgeType;
|
||||||
/** Chat ids that asserted this edge (teacher may simplify — auditable). */
|
/** Every (chat, message) point where this relation was asserted. */
|
||||||
provenance: string[];
|
provenance: ProvenancePoint[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** mappings.jsonl line — the session→graph identity map: the extraction
|
/** mappings.jsonl line — the session→graph identity map: the extraction
|
||||||
* pipeline's proposition (identified by its source id) → the graph node it
|
* pipeline's proposition (identified by its run-local source id) → the
|
||||||
* merged into. The merge audit trail: one line per (chat, graph node). */
|
* graph node it resolved to (created or linked). The audit trail of the
|
||||||
|
* auditor's identity verdicts; also the chat-scoped resolution table the
|
||||||
|
* merger uses to point events and edges at node ids. */
|
||||||
export interface NodeMapping {
|
export interface NodeMapping {
|
||||||
chatId: string;
|
chatId: string;
|
||||||
/** The writer's local id for the proposition — a plan-tree node id when
|
/** The extractor's run-local proposition id ("p1", …). */
|
||||||
* tree-derived, else the extractor's proposition id ("p1", …). */
|
|
||||||
sourceId: string;
|
sourceId: string;
|
||||||
graphNodeId: string;
|
graphNodeId: string;
|
||||||
}
|
}
|
||||||
@@ -57,23 +76,26 @@ export interface SessionCheckpoint {
|
|||||||
foldedAt: string;
|
foldedAt: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type LedgerEventKind =
|
/**
|
||||||
| "probe-correct"
|
* Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
|
||||||
| "probe-miss"
|
* dont-know — plus `observation`, the extractor's per-proposition
|
||||||
| "dont-know"
|
* understanding note joining the ledger as a session-scoped observation
|
||||||
| "misconception-signaled"
|
* (named-word scale; treatment pending the state-model ideation). No
|
||||||
| "taught"
|
* `taught` events at all; no misconception kind — a misconception is a
|
||||||
| "check-correct"
|
* TRUE correction in the graph, and the wrongness lives here, in the
|
||||||
| "check-incorrect";
|
* ledger, until the state model exists.
|
||||||
|
*/
|
||||||
|
export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
|
||||||
|
|
||||||
export interface LedgerEvent {
|
export interface LedgerEvent {
|
||||||
/** null = not yet attributed (untagged probe events; consolidation assigns). */
|
/** null = not yet attributed. */
|
||||||
nodeId: string | null;
|
nodeId: string | null;
|
||||||
kind: LedgerEventKind;
|
kind: LedgerEventKind;
|
||||||
chatId: string;
|
chatId: string;
|
||||||
at: string;
|
at: string;
|
||||||
/** Idempotency key: the quiz part's toolCallId or `taught:<treeNodeId>` —
|
/** Idempotency key: the quiz part's toolCallId, or `obs:<sourceId>` for
|
||||||
* re-emission of the same chat must never double-record an event. */
|
* understanding observations — re-emission of the same chat must never
|
||||||
|
* double-record an event. */
|
||||||
ref?: string;
|
ref?: string;
|
||||||
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */
|
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */
|
||||||
note?: string;
|
note?: string;
|
||||||
@@ -83,91 +105,121 @@ export interface LedgerEvent {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
|
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
|
||||||
* (nodes/edges/events/mappings/sessions). The derived graph is rebuilt by
|
* (nodes/edges/events/mappings/sessions), all append-only.
|
||||||
* reading the ledgers — store events, derive state. sessions.jsonl is the
|
*
|
||||||
* extraction checkpoint: which chats the extraction pipeline has processed,
|
* nodes.jsonl one line per node — the FIRST line for an id defines
|
||||||
* and through which updatedAt (never extract the same turns twice).
|
* statement/description/createdAt; later lines for the
|
||||||
|
* same id are link-appends (a session's re-encounter):
|
||||||
|
* their statements/aliases join `aliases`, their
|
||||||
|
* provenance unions. getGraph merges by id.
|
||||||
|
* edges.jsonl one line per edge ASSERTION; readers merge by
|
||||||
|
* (from,to,type), unioning provenance.
|
||||||
|
*
|
||||||
|
* The derived graph is rebuilt by reading the ledgers — store events,
|
||||||
|
* derive state. sessions.jsonl is the extraction checkpoint: which chats
|
||||||
|
* the extraction pipeline has processed, and through which updatedAt
|
||||||
|
* (never extract the same turns twice).
|
||||||
*
|
*
|
||||||
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
|
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
|
||||||
* merger): the extractor proposes propositions/edges/attribution with
|
* merger), tree-blind and post-hoc: the extractor reads only the transcript
|
||||||
* verbatim evidence, the auditor drops anything unbacked, and the merger
|
* and proposes propositions/edges/quiz-attribution with verbatim evidence;
|
||||||
* (core/graph.ts, deterministic) resolves identity against the existing
|
* the auditor sees the extractor's output PLUS the current graph, owns
|
||||||
* ledgers and appends. See SessionFindings below.
|
* faithfulness (evidence check) and identity (link-or-create: point each
|
||||||
|
* proposition at the existing node it is, or let it be created — recording
|
||||||
|
* the session's phrasings as aliases); the merger (core/graph.ts,
|
||||||
|
* deterministic) applies the verdicts and appends.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
// ─── Extraction findings (extractor output → audit → merger input) ───────────
|
// ─── Extraction findings (extractor output → audit → merger input) ───────────
|
||||||
|
|
||||||
/** One proposed proposition — a node candidate. */
|
/** One proposed proposition — a node candidate (or a re-encounter of one). */
|
||||||
export interface ExtractedProposition {
|
export interface ExtractedProposition {
|
||||||
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
|
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
|
||||||
id: string;
|
id: string;
|
||||||
/** Short name / header for the fact (2-6 words) — the rendering handle. */
|
/** The minimal proposition — a true, self-contained, world-facing
|
||||||
name: string;
|
* statement. No learner / teacher / session references: misconceptions
|
||||||
/** Canonical one-reasoning-step statement — an objective, world-facing
|
* are stated as the true correction, and wrongness lives in the event
|
||||||
* fact (no learner / teacher / session references; the learner's
|
* ledger, never in the node. */
|
||||||
* relationship to it is carried only by the event attribution). */
|
|
||||||
statement: string;
|
statement: string;
|
||||||
/** Verbatim alternate phrasings from the session (merge aliases). */
|
/** The elaborated version — required. */
|
||||||
|
description: string;
|
||||||
|
/** Verbatim phrasings the session used. */
|
||||||
aliases: string[];
|
aliases: string[];
|
||||||
kind: "truth" | "misconception";
|
/** The message ids where this proposition came up — provenance. */
|
||||||
/** The plan-tree node this came from, when it did (mapping provenance). */
|
messages: string[];
|
||||||
treeNodeId?: string;
|
|
||||||
/** Verbatim transcript quotes backing the proposition. */
|
/** Verbatim transcript quotes backing the proposition. */
|
||||||
evidence: string[];
|
evidence: string[];
|
||||||
|
/** The extractor's understanding note for this proposition IN THIS
|
||||||
|
* session — a named-word scale ("solid", "shaky", "accepted",
|
||||||
|
* "resisted"), no numbers. Joins the ledger as an observation event;
|
||||||
|
* treatment pending the state-model ideation. */
|
||||||
|
understanding?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** One proposed relation between two propositions. */
|
/** One proposed relation between two propositions. */
|
||||||
export interface ExtractedEdge {
|
export interface ExtractedEdge {
|
||||||
/** Proposition id that is the prerequisite / support. */
|
/** Proposition id that is followed / supported. */
|
||||||
from: string;
|
from: string;
|
||||||
/** Proposition id that depends on / is supported by `from`. */
|
|
||||||
to: string;
|
to: string;
|
||||||
type: EdgeType;
|
type: EdgeType;
|
||||||
|
/** The message ids where this relation was asserted — provenance. */
|
||||||
|
messages: string[];
|
||||||
/** Verbatim transcript quotes backing the relation. */
|
/** Verbatim transcript quotes backing the relation. */
|
||||||
evidence: string[];
|
evidence: string[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The extractor's event attribution — its ONLY say about events. The kinds,
|
/**
|
||||||
* timestamps, idempotency refs, notes and question texts are all rebuilt
|
* The extractor's quiz attribution — its ONLY say about quiz events. Event
|
||||||
* deterministically by the merger from the transcript's quiz / mark_done
|
|
||||||
* tool parts; the LLM supplies exactly the judgment call: WHICH proposition
|
|
||||||
* each one is about (vision open question: probes attribute retroactively,
|
|
||||||
* against the whole transcript — a pre-tree probe can map to a proposition
|
|
||||||
* the tree only articulates later). */
|
|
||||||
/** The extractor's event attribution — its ONLY say about events. Event
|
|
||||||
* kinds, timestamps, idempotency refs, notes and question texts are all
|
* kinds, timestamps, idempotency refs, notes and question texts are all
|
||||||
* rebuilt deterministically by the merger from the transcript's tool
|
* rebuilt deterministically by the merger from the transcript's quiz tool
|
||||||
* parts, and TAGGED events (quiz nodeId, mark_done tree node) resolve
|
* parts; the LLM supplies exactly the judgment call: WHICH proposition each
|
||||||
* deterministically through the proposition mapping. The LLM supplies
|
* quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
|
||||||
* exactly the one judgment call that needs it — the vision's open
|
* mapping to lean on) — retroactively against the whole transcript.
|
||||||
* probe-attribution question: WHICH proposition did an UNTAGGED probe
|
*/
|
||||||
* test? Answered retroactively against the whole transcript — a pre-tree
|
|
||||||
* probe can map to a proposition the tree only articulates later. */
|
|
||||||
export interface ExtractionAttribution {
|
export interface ExtractionAttribution {
|
||||||
/** UNTAGGED quiz toolCallId → proposition id; missing / null =
|
/** quiz toolCallId → proposition id; null / missing = unattributed (the
|
||||||
* unattributed (the event lands with nodeId null + the question text,
|
* event lands with nodeId null + the question text, a later-
|
||||||
* exactly like unattributable probes always did). */
|
* attribution input). */
|
||||||
quiz: Record<string, string | null>;
|
quiz: Record<string, string | null>;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The extractor's — and, filtered, the auditor's — output shape. */
|
/** The extractor's output shape. */
|
||||||
export interface SessionFindings {
|
export interface SessionFindings {
|
||||||
propositions: ExtractedProposition[];
|
propositions: ExtractedProposition[];
|
||||||
edges: ExtractedEdge[];
|
edges: ExtractedEdge[];
|
||||||
attribution: ExtractionAttribution;
|
attribution: ExtractionAttribution;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The auditor's verdict: the verified subset, plus the drops (for the log —
|
/** The auditor's verdict on one proposition: the findings, plus `linkTo` —
|
||||||
* the audit trail of what was rejected and why). */
|
* the identity judgment. Absent = create a new node. */
|
||||||
export interface AuditResult extends SessionFindings {
|
export interface AuditedProposition extends ExtractedProposition {
|
||||||
|
/** The existing GraphNode id this proposition IS — the auditor's
|
||||||
|
* identity verdict. On link, the session's phrasings (statement,
|
||||||
|
* aliases) join the node's aliases and the provenance unions.
|
||||||
|
* Conservative: when unsure, leave it out — duplicates over wrong
|
||||||
|
* merges. */
|
||||||
|
linkTo?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The auditor's output: the verified findings (with identity verdicts),
|
||||||
|
* plus the drops (the audit trail of what was rejected and why). */
|
||||||
|
export interface AuditResult {
|
||||||
|
propositions: AuditedProposition[];
|
||||||
|
edges: ExtractedEdge[];
|
||||||
|
attribution: ExtractionAttribution;
|
||||||
/** Dropped candidates: proposition id, edge "from→to", or attribution
|
/** Dropped candidates: proposition id, edge "from→to", or attribution
|
||||||
* toolCallId — with the reason. */
|
* toolCallId — with the reason. */
|
||||||
dropped: { ref: string; reason: string }[];
|
dropped: { ref: string; reason: string }[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Per-node state class for the home view (vision §1 "Home page") — derived,
|
/**
|
||||||
* never stored: taught if ANY taught event; else the LATEST quiz event decides
|
* Per-node state class for the home view — TRANSITIONAL (kept so the
|
||||||
* (known / missed / dont-know); a node with no events is a candidate. */
|
* renderer survives the contract change untouched). Derived, never stored:
|
||||||
|
* the LATEST quiz event decides (known / missed / dont-know); a node with
|
||||||
|
* no quiz events is a candidate. `taught` is now unreachable (no taught
|
||||||
|
* events are emitted anymore) — the real state model (dependency depth +
|
||||||
|
* topology-derived strength + quiz events) is pending ideation.
|
||||||
|
*/
|
||||||
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
|
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
|
||||||
|
|
||||||
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
|
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
|
||||||
@@ -187,4 +239,3 @@ export function titleFromPrompt(prompt: string): string {
|
|||||||
const oneLine = prompt.replace(/\s+/g, " ").trim();
|
const oneLine = prompt.replace(/\s+/g, " ").trim();
|
||||||
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
|
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+16
-22
@@ -15,10 +15,9 @@
|
|||||||
* the block is absent (new learner).
|
* the block is absent (new learner).
|
||||||
*
|
*
|
||||||
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE
|
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE
|
||||||
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know /
|
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know
|
||||||
* misconception nodes) plus their 1-hop `requires` neighborhood — a live
|
* nodes) plus their 1-hop `follows` neighborhood — the frontier is where
|
||||||
* misconception is always relevant context, and the frontier is where the
|
* the next session should probe.
|
||||||
* next session should probe.
|
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import type { GraphEdge, GraphNode } from "./contracts.ts";
|
import type { GraphEdge, GraphNode } from "./contracts.ts";
|
||||||
@@ -31,22 +30,18 @@ export const GRAPH_CONTEXT_HEADER = "LEARNER KNOWLEDGE GRAPH (auto-generated)";
|
|||||||
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */
|
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */
|
||||||
export const MAX_CONTEXT_NODES = 150;
|
export const MAX_CONTEXT_NODES = 150;
|
||||||
|
|
||||||
/** Group order for node lines: diagnostic value first (vision §1 hotspots).
|
/** Group order for node lines: diagnostic value first (vision §1 hotspots). */
|
||||||
* A misconception-kind node is a hotspot whatever its state — a live
|
|
||||||
* misconception is always relevant context. */
|
|
||||||
const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]);
|
const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]);
|
||||||
|
|
||||||
function groupOf(node: GraphNode, state: NodeStateClass): number {
|
function groupOf(node: GraphNode, state: NodeStateClass): number {
|
||||||
if (HOTSPOT_STATES.has(state) || node.kind === "misconception") return 0;
|
if (HOTSPOT_STATES.has(state)) return 0;
|
||||||
if (state === "known") return 1;
|
if (state === "known") return 1;
|
||||||
if (state === "taught") return 2;
|
return 2; // candidate
|
||||||
return 3; // candidate
|
|
||||||
}
|
}
|
||||||
|
|
||||||
const GROUP_LABELS = [
|
const GROUP_LABELS = [
|
||||||
"Hotspots (missed / dont-know / misconceptions) — likely misconception sites and unexplored ground:",
|
"Hotspots (missed / dont-know) — likely misconception sites and unexplored ground:",
|
||||||
"Known (answered correctly):",
|
"Known (answered correctly):",
|
||||||
"Taught (lessons covered them):",
|
|
||||||
"Candidates (never probed — assume nothing):",
|
"Candidates (never probed — assume nothing):",
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
@@ -76,9 +71,9 @@ export function renderGraphContext(
|
|||||||
truncated = true;
|
truncated = true;
|
||||||
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
|
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
|
||||||
const keep = new Set(hotspots.map((n) => n.id));
|
const keep = new Set(hotspots.map((n) => n.id));
|
||||||
// 1-hop requires neighborhood: prerequisites AND dependents.
|
// 1-hop follows neighborhood: premises AND derivations.
|
||||||
for (const edge of edges) {
|
for (const edge of edges) {
|
||||||
if (edge.type !== "requires") continue;
|
if (edge.type !== "follows") continue;
|
||||||
if (keep.has(edge.from) || keep.has(edge.to)) {
|
if (keep.has(edge.from) || keep.has(edge.to)) {
|
||||||
keep.add(edge.from);
|
keep.add(edge.from);
|
||||||
keep.add(edge.to);
|
keep.add(edge.to);
|
||||||
@@ -91,7 +86,7 @@ export function renderGraphContext(
|
|||||||
// ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
|
// ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
|
||||||
const lines: string[] = [
|
const lines: string[] = [
|
||||||
`===== ${GRAPH_CONTEXT_HEADER} =====`,
|
`===== ${GRAPH_CONTEXT_HEADER} =====`,
|
||||||
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "requires").length} requires edges`,
|
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "follows").length} follows edges`,
|
||||||
];
|
];
|
||||||
|
|
||||||
for (const [group, label] of GROUP_LABELS.entries()) {
|
for (const [group, label] of GROUP_LABELS.entries()) {
|
||||||
@@ -101,19 +96,18 @@ export function renderGraphContext(
|
|||||||
if (groupNodes.length === 0) continue;
|
if (groupNodes.length === 0) continue;
|
||||||
lines.push("", label);
|
lines.push("", label);
|
||||||
for (const node of groupNodes) {
|
for (const node of groupNodes) {
|
||||||
const kind = node.kind === "misconception" ? " · misconception" : "";
|
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})`);
|
||||||
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})${kind}`);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── requires edges, merged into chains ──────────────────────────────────
|
// ── requires edges, merged into chains ──────────────────────────────────
|
||||||
const byId = new Map(selected.map((n) => [n.id, n.statement]));
|
const byId = new Map(selected.map((n) => [n.id, n.statement]));
|
||||||
const requires = edges.filter(
|
const follows = edges.filter(
|
||||||
(e) => e.type === "requires" && byId.has(e.from) && byId.has(e.to),
|
(e) => e.type === "follows" && byId.has(e.from) && byId.has(e.to),
|
||||||
);
|
);
|
||||||
if (requires.length > 0) {
|
if (follows.length > 0) {
|
||||||
lines.push("", "Requires (prerequisite → dependent):");
|
lines.push("", "Follows (premise → derivation):");
|
||||||
for (const chain of edgeChains(requires)) {
|
for (const chain of edgeChains(follows)) {
|
||||||
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
|
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+164
-160
@@ -2,22 +2,34 @@
|
|||||||
* The knowledge graph: merger (the deterministic final stage of the
|
* The knowledge graph: merger (the deterministic final stage of the
|
||||||
* extraction pipeline — vision §1 construction) + derivation (reads).
|
* extraction pipeline — vision §1 construction) + derivation (reads).
|
||||||
*
|
*
|
||||||
* The extraction pipeline is the SOLE graph writer:
|
* The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
|
||||||
*
|
*
|
||||||
* extractor (LLM) → SessionFindings: propositions, edges, and event
|
* extractor (LLM) → SessionFindings: propositions, edges, quiz
|
||||||
* attribution, each with verbatim evidence
|
* attribution, per-proposition understanding notes —
|
||||||
* auditor (LLM) → the verified subset (unbacked candidates dropped)
|
* each with verbatim evidence, from the transcript ALONE
|
||||||
|
* auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
|
||||||
|
* VERDICT per proposition (linkTo an existing node, or
|
||||||
|
* create), and the drops for the log
|
||||||
* merger (HERE) → mergeFindings(): deterministic ledger appends
|
* merger (HERE) → mergeFindings(): deterministic ledger appends
|
||||||
*
|
*
|
||||||
* The merger is deliberately the ONLY stage that touches the ledgers, and it
|
* The merger is deliberately the ONLY stage that touches the ledgers, and it
|
||||||
* is pure code: the LLM supplies exactly the judgment calls (what was
|
* is pure code: the LLM supplies exactly the judgment calls (what the
|
||||||
* taught, what depends on what, which proposition a quiz tested), while
|
* propositions are, what follows/supports what, which proposition a quiz
|
||||||
* event kinds/timestamps/refs — anything the model could silently
|
* tested, whether a proposition IS an existing node), while event
|
||||||
* mis-copy — are rebuilt deterministically from the transcript's tool
|
* kinds/timestamps/refs/provenance — anything the model could silently
|
||||||
* parts. Idempotent + incremental: statement/alias matching dedupes nodes,
|
* mis-copy — are rebuilt deterministically from the transcript's tool parts
|
||||||
* (from,to,type) dedupes edge assertions, chat-scoped refs dedupe events,
|
* and the message ids in the findings. Idempotent + incremental: refs are
|
||||||
* one mapping line per (chat, graph node), and sessions.jsonl checkpoints
|
* chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
|
||||||
* which chats have been extracted (never the same turns twice).
|
* asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
|
||||||
|
* sessions.jsonl checkpoints which chats have been extracted (never the
|
||||||
|
* same turns twice).
|
||||||
|
*
|
||||||
|
* Identity across re-encounters is the AUDITOR's judgment (link-or-create),
|
||||||
|
* not string matching: the conservative v1 exact-statement merge was
|
||||||
|
* replaced by the 2026-09-02 redesign. A re-extracted session's
|
||||||
|
* propositions link to the existing nodes; the link appends a node line
|
||||||
|
* whose phrasings join `aliases` and whose provenance unions (getGraph
|
||||||
|
* merges lines by id).
|
||||||
*
|
*
|
||||||
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
|
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
|
||||||
* only — everything is recomputed from the ledgers. Every function takes
|
* only — everything is recomputed from the ledgers. Every function takes
|
||||||
@@ -29,14 +41,14 @@
|
|||||||
import { randomBytes } from "node:crypto";
|
import { randomBytes } from "node:crypto";
|
||||||
import { isToolUIPart, type UIMessage } from "ai";
|
import { isToolUIPart, type UIMessage } from "ai";
|
||||||
import type {
|
import type {
|
||||||
|
AuditResult,
|
||||||
GraphEdge,
|
GraphEdge,
|
||||||
GraphNode,
|
GraphNode,
|
||||||
|
ProvenancePoint,
|
||||||
GraphSnapshot,
|
GraphSnapshot,
|
||||||
LedgerEvent,
|
LedgerEvent,
|
||||||
LedgerEventKind,
|
LedgerEventKind,
|
||||||
NodeStateClass,
|
NodeStateClass,
|
||||||
PlanToolResult,
|
|
||||||
SessionFindings,
|
|
||||||
} from "./contracts.ts";
|
} from "./contracts.ts";
|
||||||
import type { QuestionAnswerData } from "./question-core.ts";
|
import type { QuestionAnswerData } from "./question-core.ts";
|
||||||
import type { ChatStore, GraphStore } from "./ports.ts";
|
import type { ChatStore, GraphStore } from "./ports.ts";
|
||||||
@@ -80,19 +92,25 @@ function toolParts(messages: UIMessage[]): ToolPartView[] {
|
|||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** A plan tool part's output, if it is a well-formed PlanToolResult. */
|
/** (chatId, messageIds) → provenance points; order-preserving, deduped. */
|
||||||
function planResult(part: ToolPartView): PlanToolResult | null {
|
function pointsOf(chatId: string, messages: string[]): ProvenancePoint[] {
|
||||||
if (part.state !== "output-available") return null;
|
const seen = new Set<string>();
|
||||||
const out = part.output;
|
const out: ProvenancePoint[] = [];
|
||||||
if (typeof out !== "object" || out === null) return null;
|
for (const messageId of messages) {
|
||||||
if (!("tree" in out) || typeof (out as PlanToolResult).ok !== "boolean") return null;
|
if (messageId.length === 0 || seen.has(messageId)) continue;
|
||||||
return out as PlanToolResult;
|
seen.add(messageId);
|
||||||
|
out.push({ chatId, messageId });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
|
||||||
|
a.chatId === b.chatId && a.messageId === b.messageId;
|
||||||
|
|
||||||
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
|
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
|
||||||
|
|
||||||
export interface EmissionStats {
|
export interface EmissionStats {
|
||||||
/** New graph nodes appended (and mappings with them). */
|
/** New graph nodes appended. */
|
||||||
nodes: number;
|
nodes: number;
|
||||||
/** New edge assertions appended. */
|
/** New edge assertions appended. */
|
||||||
edges: number;
|
edges: number;
|
||||||
@@ -102,156 +120,135 @@ export interface EmissionStats {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Merge one chat's audited findings into the graph ledgers — the pipeline's
|
* Merge one chat's audited findings into the graph ledgers — the pipeline's
|
||||||
* apply stage. Deterministic + idempotent: re-merging the same findings (or
|
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or
|
||||||
* a later re-extraction of the same chat) appends nothing but the new.
|
* a later re-extraction of the same chat) appends nothing but the new.
|
||||||
*
|
*
|
||||||
* Identity resolution (v1: conservative): a proposition whose statement or
|
* Identity is the auditor's link-or-create verdict: `linkTo` an existing
|
||||||
* alias EXACTLY matches (case-insensitive, trimmed) an existing node's
|
* node (validated against the ledger — an unknown id falls back to create)
|
||||||
* statement or alias merges onto it; everything else is a new node. One
|
* appends a LINK line (the session's phrasings join the node's aliases and
|
||||||
* mapping line per (chat, graph node) — the merge audit trail. Fuzzy /
|
* the provenance unions; getGraph merges lines by id); absent → a new node.
|
||||||
* embedding adjudication stays deferred (vision §1 consolidation).
|
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
|
||||||
|
* a proposition id already mapped for this chat reuses its resolution.
|
||||||
*
|
*
|
||||||
* Events are rebuilt deterministically from the transcript's quiz and
|
* Events are rebuilt deterministically from the transcript: quiz outcomes
|
||||||
* plan_mark_done parts (kinds, timestamps, notes, refs). TAGGED events
|
* (correct / miss / dont-know; kinds + timestamps + refs from the tool
|
||||||
* (quiz nodeId, mark_done tree node) resolve through the proposition
|
* parts, nodeId from the extractor's attribution through this run's
|
||||||
* mapping — deterministic; the findings' attribution is consulted ONLY for
|
* mapping) plus one observation event per proposition carrying an
|
||||||
* untagged probe quizzes: the retroactive probe-attribution resolution (a
|
* understanding note (ref `obs:<sourceId>`). No taught events, ever.
|
||||||
* pre-tree probe maps to a proposition the tree only articulates later).
|
|
||||||
* Unattributed quizzes land with nodeId null + the question text, exactly
|
|
||||||
* like the unattributable ones always have.
|
|
||||||
*/
|
*/
|
||||||
export function mergeFindings(chatId: string, findings: SessionFindings, deps: GraphDeps): EmissionStats {
|
export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps): EmissionStats {
|
||||||
const { graphStore: ledgers, chatStore } = deps;
|
const { graphStore: ledgers, chatStore } = deps;
|
||||||
const record = chatStore.getChat(chatId);
|
const record = chatStore.getChat(chatId);
|
||||||
if (!record) return { nodes: 0, edges: 0, events: 0 };
|
if (!record) return { nodes: 0, edges: 0, events: 0 };
|
||||||
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
|
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
|
||||||
|
|
||||||
// 1. Identity resolution. mapping: proposition id (and tree node id)
|
// 1. Resolution: this run's propositions → graph node ids. Existing
|
||||||
// → graph node id — from this chat's existing mapping lines, then
|
// mapping lines for THIS chat are loaded first — the audit trail doubles
|
||||||
// extended as findings resolve. chatNodeIds: graph nodes this chat has
|
// as the resolution table, which makes re-merging the same chat
|
||||||
// already mapped onto — the (chat, graph node) mapping dedupe.
|
// idempotent (a re-extraction re-merges through the auditor's linkTo
|
||||||
|
// verdicts instead). Fresh sourceIds resolve as the verdicts say.
|
||||||
|
const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
|
||||||
const mapping = new Map<string, string>();
|
const mapping = new Map<string, string>();
|
||||||
const chatNodeIds = new Set<string>();
|
const seenMappingKeys = new Set<string>();
|
||||||
for (const m of ledgers.readMappings()) {
|
for (const m of ledgers.readMappings()) {
|
||||||
if (m.chatId !== chatId || typeof m.sourceId !== "string") continue;
|
if (m.chatId !== chatId) continue;
|
||||||
mapping.set(m.sourceId, m.graphNodeId);
|
mapping.set(m.sourceId, m.graphNodeId);
|
||||||
chatNodeIds.add(m.graphNodeId);
|
seenMappingKeys.add(m.sourceId);
|
||||||
}
|
|
||||||
|
|
||||||
// 2. Propositions → graph nodes. Exact statement/alias match against the
|
|
||||||
// whole ledger first (cross-session reuse), then against propositions
|
|
||||||
// created earlier in THIS merge (intra-findings dedupe).
|
|
||||||
const norm = (s: string): string => s.trim().toLowerCase();
|
|
||||||
const existingByTitle = new Map<string, string>();
|
|
||||||
for (const node of ledgers.readNodes()) {
|
|
||||||
for (const title of [node.statement, ...node.aliases]) {
|
|
||||||
const key = norm(title);
|
|
||||||
if (key && !existingByTitle.has(key)) existingByTitle.set(key, node.id);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
const now = new Date().toISOString();
|
const now = new Date().toISOString();
|
||||||
for (const p of findings.propositions) {
|
|
||||||
if (mapping.has(p.id)) continue;
|
for (const p of audit.propositions) {
|
||||||
const keys = [p.statement, ...p.aliases].map(norm).filter((k) => k.length > 0);
|
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
|
||||||
const matchId = keys.map((k) => existingByTitle.get(k)).find((id) => id !== undefined);
|
const points = pointsOf(chatId, p.messages);
|
||||||
let graphNodeId: string;
|
let graphNodeId: string;
|
||||||
if (matchId !== undefined) {
|
if (p.linkTo !== undefined && existingIds.has(p.linkTo)) {
|
||||||
graphNodeId = matchId; // reuse — conservative merge, duplicates over wrong merges
|
// Identity verdict: this proposition IS that node. Append a link
|
||||||
} else {
|
// line — the session's phrasings (statement + aliases) join the
|
||||||
const graphNode: GraphNode = {
|
// node's aliases, the provenance unions. Reads merge by id.
|
||||||
id: newGraphNodeId(),
|
graphNodeId = p.linkTo;
|
||||||
name: p.name,
|
ledgers.appendNode({
|
||||||
|
id: graphNodeId,
|
||||||
statement: p.statement,
|
statement: p.statement,
|
||||||
|
description: p.description,
|
||||||
aliases: [...p.aliases],
|
aliases: [...p.aliases],
|
||||||
kind: p.kind,
|
|
||||||
createdAt: now,
|
createdAt: now,
|
||||||
|
provenance: points,
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
const node: GraphNode = {
|
||||||
|
id: newGraphNodeId(),
|
||||||
|
statement: p.statement,
|
||||||
|
description: p.description,
|
||||||
|
aliases: [...p.aliases],
|
||||||
|
createdAt: now,
|
||||||
|
provenance: points,
|
||||||
};
|
};
|
||||||
ledgers.appendNode(graphNode);
|
ledgers.appendNode(node);
|
||||||
stats.nodes++;
|
stats.nodes++;
|
||||||
graphNodeId = graphNode.id;
|
graphNodeId = node.id;
|
||||||
}
|
existingIds.add(graphNodeId);
|
||||||
for (const key of keys) if (!existingByTitle.has(key)) existingByTitle.set(key, graphNodeId);
|
|
||||||
if (!chatNodeIds.has(graphNodeId)) {
|
|
||||||
ledgers.appendMapping({ chatId, sourceId: p.treeNodeId ?? p.id, graphNodeId });
|
|
||||||
chatNodeIds.add(graphNodeId);
|
|
||||||
}
|
}
|
||||||
mapping.set(p.id, graphNodeId);
|
mapping.set(p.id, graphNodeId);
|
||||||
if (p.treeNodeId !== undefined) mapping.set(p.treeNodeId, graphNodeId);
|
if (!seenMappingKeys.has(p.id)) {
|
||||||
|
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
|
||||||
|
seenMappingKeys.add(p.id);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 3. Edges. Dedupe by (from,to,type) across the whole ledger: an existing
|
// 2. Edges. Resolve endpoints through the mapping; dedupe by
|
||||||
// assertion just gains this chatId as provenance (appended as a new
|
// (from,to,type) already asserted by THIS chat (an existing assertion
|
||||||
// line; reads merge).
|
// by another chat stays untouched — provenance unions at read time).
|
||||||
const asserted = new Map<string, Set<string>>();
|
const assertedByChat = new Set<string>();
|
||||||
for (const line of ledgers.readEdges()) {
|
for (const line of ledgers.readEdges()) {
|
||||||
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
||||||
let provenance = asserted.get(key);
|
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
|
||||||
if (!provenance) {
|
|
||||||
provenance = new Set();
|
|
||||||
asserted.set(key, provenance);
|
|
||||||
}
|
}
|
||||||
for (const c of line.provenance) provenance.add(c);
|
for (const edge of audit.edges) {
|
||||||
}
|
|
||||||
for (const edge of findings.edges) {
|
|
||||||
const from = mapping.get(edge.from);
|
const from = mapping.get(edge.from);
|
||||||
const to = mapping.get(edge.to);
|
const to = mapping.get(edge.to);
|
||||||
if (!from || !to || from === to) continue; // unmapped endpoint — skip
|
if (!from || !to || from === to) continue; // unmapped endpoint — skip
|
||||||
const key = `${from}\u0000${to}\u0000${edge.type}`;
|
const key = `${from}\u0000${to}\u0000${edge.type}`;
|
||||||
const provenance = asserted.get(key);
|
if (assertedByChat.has(key)) continue;
|
||||||
if (provenance?.has(chatId)) continue; // already asserted by this chat
|
ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
|
||||||
ledgers.appendEdge({ from, to, type: edge.type, provenance: [chatId] });
|
assertedByChat.add(key);
|
||||||
if (provenance) provenance.add(chatId);
|
|
||||||
else asserted.set(key, new Set([chatId]));
|
|
||||||
stats.edges++;
|
stats.edges++;
|
||||||
}
|
}
|
||||||
|
|
||||||
// 4. Events, rebuilt deterministically in transcript order so the
|
// 3. Events, rebuilt deterministically in transcript order. Refs are
|
||||||
// probe/check boundary is exact: quizzes BEFORE the first plan_init
|
// chat-scoped idempotency keys (quiz toolCallId, obs:<sourceId>) —
|
||||||
// output are probes. Refs are chat-scoped (a quiz toolCallId or
|
// a ref already in the ledger means this evidence was extracted before.
|
||||||
// taught:<treeNodeId>) — a ref already in the ledger means this turn
|
|
||||||
// was extracted before. The findings' attribution supplies the
|
|
||||||
// nodeId, nothing else.
|
|
||||||
const seenRefs = new Set<string>();
|
const seenRefs = new Set<string>();
|
||||||
for (const event of ledgers.readEvents()) {
|
for (const event of ledgers.readEvents()) {
|
||||||
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
|
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
|
||||||
}
|
}
|
||||||
let planSeen = false;
|
|
||||||
for (const part of toolParts(record.messages)) {
|
for (const part of toolParts(record.messages)) {
|
||||||
// The boundary flips on the first plan_init OUTPUT, whatever it says
|
if (part.name !== "quiz" || part.state !== "output-available") continue;
|
||||||
// (an ok:false "tree already initialized" still means a tree exists).
|
const event = quizEvent(part, chatId, audit, mapping, now);
|
||||||
if (part.name === "plan_init" && part.state === "output-available") planSeen = true;
|
|
||||||
|
|
||||||
if (part.name === "quiz" && part.state === "output-available") {
|
|
||||||
const event = quizEvent(part, chatId, mapping, findings, planSeen, now);
|
|
||||||
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
|
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
|
||||||
ledgers.appendEvent(event);
|
ledgers.appendEvent(event);
|
||||||
seenRefs.add(event.ref);
|
seenRefs.add(event.ref);
|
||||||
stats.events++;
|
stats.events++;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Understanding observations: one per proposition carrying a note,
|
||||||
if (part.name === "plan_mark_done" && part.state === "output-available") {
|
// resolved to the node the verdict mapped it to.
|
||||||
const result = planResult(part);
|
for (const p of audit.propositions) {
|
||||||
const input = part.input as { id?: unknown } | undefined;
|
if (p.understanding === undefined || p.understanding.length === 0) continue;
|
||||||
if (result?.ok && typeof input?.id === "string") {
|
const ref = `obs:${p.id}`;
|
||||||
const ref = `taught:${input.id}`;
|
if (seenRefs.has(ref)) continue;
|
||||||
if (!seenRefs.has(ref)) {
|
|
||||||
// mark_done is tagged with the tree node id — deterministic.
|
|
||||||
ledgers.appendEvent({
|
ledgers.appendEvent({
|
||||||
nodeId: mapping.get(input.id) ?? null,
|
nodeId: mapping.get(p.id) ?? null,
|
||||||
kind: "taught",
|
kind: "observation",
|
||||||
chatId,
|
chatId,
|
||||||
at: now,
|
at: now,
|
||||||
ref,
|
ref,
|
||||||
|
note: p.understanding,
|
||||||
});
|
});
|
||||||
seenRefs.add(ref);
|
seenRefs.add(ref);
|
||||||
stats.events++;
|
stats.events++;
|
||||||
}
|
}
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// 5. Checkpoint: this chat is extracted through its current updatedAt.
|
// 4. Checkpoint: this chat is extracted through its current updatedAt.
|
||||||
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
|
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
|
||||||
return stats;
|
return stats;
|
||||||
}
|
}
|
||||||
@@ -261,9 +258,8 @@ export function mergeFindings(chatId: string, findings: SessionFindings, deps: G
|
|||||||
function quizEvent(
|
function quizEvent(
|
||||||
part: ToolPartView,
|
part: ToolPartView,
|
||||||
chatId: string,
|
chatId: string,
|
||||||
|
audit: AuditResult,
|
||||||
mapping: Map<string, string>,
|
mapping: Map<string, string>,
|
||||||
findings: SessionFindings,
|
|
||||||
planSeen: boolean,
|
|
||||||
now: string,
|
now: string,
|
||||||
): LedgerEvent | null {
|
): LedgerEvent | null {
|
||||||
const out = part.output;
|
const out = part.output;
|
||||||
@@ -274,16 +270,13 @@ function quizEvent(
|
|||||||
|
|
||||||
let kind: LedgerEventKind;
|
let kind: LedgerEventKind;
|
||||||
if (answer.dontKnow === true) kind = "dont-know";
|
if (answer.dontKnow === true) kind = "dont-know";
|
||||||
else if (answer.correct === true) kind = planSeen ? "check-correct" : "probe-correct";
|
else if (answer.correct === true) kind = "correct";
|
||||||
else kind = planSeen ? "check-incorrect" : "probe-miss";
|
else kind = "miss";
|
||||||
|
|
||||||
// Tagged (input.nodeId) resolves deterministically through the mapping;
|
// Tree-blind: EVERY quiz attributes through the extractor's judgment —
|
||||||
// attribution is the LLM judgment call for UNTAGGED probes only — the
|
// the vision's retroactive attribution, with no tree mapping to lean on.
|
||||||
// vision's open probe-attribution question, resolved retroactively.
|
const attributedId = audit.attribution.quiz[part.toolCallId];
|
||||||
const input = part.input as { nodeId?: unknown } | undefined;
|
const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
|
||||||
const tagged = typeof input?.nodeId === "string" ? mapping.get(input.nodeId) : undefined;
|
|
||||||
const attributedId = findings.attribution.quiz[part.toolCallId];
|
|
||||||
const nodeId = tagged ?? (attributedId ? (mapping.get(attributedId) ?? null) : null);
|
|
||||||
|
|
||||||
const event: LedgerEvent = {
|
const event: LedgerEvent = {
|
||||||
nodeId,
|
nodeId,
|
||||||
@@ -293,7 +286,7 @@ function quizEvent(
|
|||||||
ref: part.toolCallId,
|
ref: part.toolCallId,
|
||||||
};
|
};
|
||||||
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
|
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
|
||||||
// The question text is the attribution input for unattributed probes.
|
// The question text is the attribution input for unattributed quizzes.
|
||||||
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
|
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
|
||||||
return event;
|
return event;
|
||||||
}
|
}
|
||||||
@@ -335,20 +328,39 @@ export interface DerivedGraph {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The whole graph from the ledgers. Edge LINES are assertions; the returned
|
* The whole graph from the ledgers. Node LINES may repeat an id (link-
|
||||||
* edges are merged by (from,to,type) with unioned provenance, first-seen
|
* appends): the FIRST line defines statement/description/createdAt; later
|
||||||
* order. `store events, derive state` — this is the only reader that matters.
|
* lines' statements and aliases join `aliases`, provenance unions. Edge
|
||||||
|
* LINES are assertions, merged by (from,to,type) with unioned provenance,
|
||||||
|
* first-seen order. `store events, derive state` — this is the only reader
|
||||||
|
* that matters.
|
||||||
*/
|
*/
|
||||||
export function getGraph(graphStore: GraphStore): DerivedGraph {
|
export function getGraph(graphStore: GraphStore): DerivedGraph {
|
||||||
const nodes = graphStore.readNodes();
|
// Merge node lines by id (first line defines; later lines add phrasings).
|
||||||
|
const byId = new Map<string, GraphNode>();
|
||||||
|
for (const line of graphStore.readNodes()) {
|
||||||
|
const existing = byId.get(line.id);
|
||||||
|
if (!existing) {
|
||||||
|
byId.set(line.id, { ...line, aliases: [...line.aliases], provenance: [...line.provenance] });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// A later line's statement is a phrasing, not a replacement.
|
||||||
|
for (const phrasing of [line.statement, ...line.aliases]) {
|
||||||
|
if (phrasing.length > 0 && !existing.aliases.includes(phrasing)) existing.aliases.push(phrasing);
|
||||||
|
}
|
||||||
|
for (const pt of line.provenance) {
|
||||||
|
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const merged: GraphEdge[] = [];
|
const merged: GraphEdge[] = [];
|
||||||
const byKey = new Map<string, GraphEdge>();
|
const byKey = new Map<string, GraphEdge>();
|
||||||
for (const line of graphStore.readEdges()) {
|
for (const line of graphStore.readEdges()) {
|
||||||
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
||||||
const existing = byKey.get(key);
|
const existing = byKey.get(key);
|
||||||
if (existing) {
|
if (existing) {
|
||||||
for (const c of line.provenance) {
|
for (const pt of line.provenance) {
|
||||||
if (!existing.provenance.includes(c)) existing.provenance.push(c);
|
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
|
const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
|
||||||
@@ -356,34 +368,27 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
|
|||||||
merged.push(edge);
|
merged.push(edge);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return { nodes, edges: merged, events: graphStore.readEvents() };
|
return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
|
||||||
}
|
}
|
||||||
|
|
||||||
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set([
|
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set(["correct", "miss", "dont-know"]);
|
||||||
"probe-correct",
|
|
||||||
"check-correct",
|
|
||||||
"probe-miss",
|
|
||||||
"check-incorrect",
|
|
||||||
"dont-know",
|
|
||||||
]);
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Derive node states from events (never stored): taught if ANY taught event;
|
* Derive node states from events (never stored): the LATEST quiz event
|
||||||
* else the LATEST quiz event decides (known / missed / dont-know); a node
|
* decides (known / missed / dont-know); a node with no quiz events is a
|
||||||
* with no events is a candidate.
|
* candidate. TRANSITIONAL (2026-09-02 redesign): `taught` is unreachable
|
||||||
|
* now that no taught events are emitted, and observation events are NOT
|
||||||
|
* state inputs — the real state model (dependency depth + topology-derived
|
||||||
|
* strength + quiz events) is pending ideation; this keeps the renderer
|
||||||
|
* alive until it lands.
|
||||||
*/
|
*/
|
||||||
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> {
|
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> {
|
||||||
const states: Record<string, NodeStateClass> = {};
|
const states: Record<string, NodeStateClass> = {};
|
||||||
for (const node of nodes) states[node.id] = "candidate";
|
for (const node of nodes) states[node.id] = "candidate";
|
||||||
|
|
||||||
const taught = new Set<string>();
|
|
||||||
const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>();
|
const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>();
|
||||||
events.forEach((event, index) => {
|
events.forEach((event, index) => {
|
||||||
if (event.nodeId === null) return;
|
if (event.nodeId === null) return;
|
||||||
if (event.kind === "taught") {
|
|
||||||
taught.add(event.nodeId);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
|
if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
|
||||||
const current = latest.get(event.nodeId);
|
const current = latest.get(event.nodeId);
|
||||||
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
|
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
|
||||||
@@ -392,13 +397,9 @@ export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Rec
|
|||||||
});
|
});
|
||||||
|
|
||||||
for (const node of nodes) {
|
for (const node of nodes) {
|
||||||
if (taught.has(node.id)) {
|
|
||||||
states[node.id] = "taught";
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
const last = latest.get(node.id);
|
const last = latest.get(node.id);
|
||||||
if (!last) continue; // candidate
|
if (!last) continue; // candidate
|
||||||
if (last.kind === "probe-correct" || last.kind === "check-correct") states[node.id] = "known";
|
if (last.kind === "correct") states[node.id] = "known";
|
||||||
else if (last.kind === "dont-know") states[node.id] = "dont-know";
|
else if (last.kind === "dont-know") states[node.id] = "dont-know";
|
||||||
else states[node.id] = "missed";
|
else states[node.id] = "missed";
|
||||||
}
|
}
|
||||||
@@ -422,10 +423,13 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
|||||||
chats.add(chatId);
|
chats.add(chatId);
|
||||||
};
|
};
|
||||||
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
|
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
|
||||||
|
for (const node of nodes) {
|
||||||
|
for (const pt of node.provenance) touch(node.id, pt.chatId);
|
||||||
|
}
|
||||||
for (const edge of edges) {
|
for (const edge of edges) {
|
||||||
for (const chatId of edge.provenance) {
|
for (const pt of edge.provenance) {
|
||||||
touch(edge.from, chatId);
|
touch(edge.from, pt.chatId);
|
||||||
touch(edge.to, chatId);
|
touch(edge.to, pt.chatId);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for (const event of events) {
|
for (const event of events) {
|
||||||
|
|||||||
+1
-1
@@ -56,7 +56,7 @@ export interface GraphStore {
|
|||||||
appendEdge(assertion: GraphEdge): void;
|
appendEdge(assertion: GraphEdge): void;
|
||||||
readEdges(): GraphEdge[];
|
readEdges(): GraphEdge[];
|
||||||
|
|
||||||
/** Append one quiz outcome / taught event. */
|
/** Append one quiz outcome / understanding observation. */
|
||||||
appendEvent(event: LedgerEvent): void;
|
appendEvent(event: LedgerEvent): void;
|
||||||
readEvents(): LedgerEvent[];
|
readEvents(): LedgerEvent[];
|
||||||
|
|
||||||
|
|||||||
+28
-35
@@ -1,20 +1,26 @@
|
|||||||
/**
|
/**
|
||||||
* GraphStore — the fs JSONL adapter: five append-only ledgers at
|
* GraphStore — the fs JSONL adapter: five append-only ledgers at
|
||||||
* config.LEDGER_FILES (contracts.ts "v1 emission").
|
* config.LEDGER_FILES (contracts.ts "v1 emission").
|
||||||
* nodes.jsonl GraphNode — one line per graph node
|
* nodes.jsonl GraphNode — one line per node write: the FIRST
|
||||||
|
* line for an id defines it; a later
|
||||||
|
* line with a REPEATED id is a
|
||||||
|
* link-append (session phrasings join
|
||||||
|
* aliases, provenance unions); reads
|
||||||
|
* merge by id (core/graph.ts getGraph)
|
||||||
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
|
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
|
||||||
* session asserting the same (from,to,type)
|
* session asserting the same (from,to,type)
|
||||||
* appends another line; reads merge)
|
* appends another line; reads merge)
|
||||||
* events.jsonl LedgerEvent — quiz outcomes + taught events
|
* events.jsonl LedgerEvent — quiz outcomes + understanding observations
|
||||||
* mappings.jsonl NodeMapping — the session→graph identity map:
|
* mappings.jsonl NodeMapping — the session→graph identity map:
|
||||||
* (chatId, sourceId) → graphNodeId
|
* (chatId, sourceId) → graphNodeId
|
||||||
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
|
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
|
||||||
*
|
*
|
||||||
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this
|
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this
|
||||||
* adapter's. Reads are strict against the contracts.ts shapes: corrupt lines
|
* adapter's. Reads are strict against the contracts.ts shapes (the 2026-09-02
|
||||||
* are skipped, never guessed at — the ledgers are the source of truth and
|
* graph redesign): corrupt lines are skipped, never guessed at — the ledgers
|
||||||
* everything else is derived (core/graph.ts, which takes this port as a
|
* are the source of truth and everything else is derived (core/graph.ts,
|
||||||
* parameter). Same idioms as src/store/chats.ts: sync fs, no framework.
|
* which takes this port as a parameter). Same idioms as src/store/chats.ts:
|
||||||
|
* sync fs, no framework.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import * as fs from "node:fs";
|
import * as fs from "node:fs";
|
||||||
@@ -26,6 +32,7 @@ import type {
|
|||||||
LedgerEvent,
|
LedgerEvent,
|
||||||
LedgerEventKind,
|
LedgerEventKind,
|
||||||
NodeMapping,
|
NodeMapping,
|
||||||
|
ProvenancePoint,
|
||||||
SessionCheckpoint,
|
SessionCheckpoint,
|
||||||
} from "../core/contracts.ts";
|
} from "../core/contracts.ts";
|
||||||
import type { GraphStore } from "../core/ports.ts";
|
import type { GraphStore } from "../core/ports.ts";
|
||||||
@@ -69,26 +76,22 @@ function readValid<T>(file: string, isValid: (v: unknown) => v is T): T[] {
|
|||||||
return readRawLines(file).filter(isValid);
|
return readRawLines(file).filter(isValid);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Pre-name ledger line (the first extraction runs) — read tolerantly. */
|
/** Provenance points are (chatId, messageId) pairs — message-granular. */
|
||||||
type StoredGraphNode = Omit<GraphNode, "name"> & { name?: string };
|
const isProvenancePoint = (v: unknown): v is ProvenancePoint =>
|
||||||
|
isRecord(v) && isString(v.chatId) && isString(v.messageId);
|
||||||
|
|
||||||
const isStoredGraphNode = (v: unknown): v is StoredGraphNode =>
|
const isGraphNode = (v: unknown): v is GraphNode =>
|
||||||
isRecord(v) &&
|
isRecord(v) &&
|
||||||
isString(v.id) &&
|
isString(v.id) &&
|
||||||
isString(v.statement) &&
|
isString(v.statement) &&
|
||||||
|
isString(v.description) &&
|
||||||
Array.isArray(v.aliases) &&
|
Array.isArray(v.aliases) &&
|
||||||
(v.kind === "truth" || v.kind === "misconception") &&
|
v.aliases.every(isString) &&
|
||||||
isString(v.createdAt) &&
|
isString(v.createdAt) &&
|
||||||
(v.name === undefined || isString(v.name));
|
Array.isArray(v.provenance) &&
|
||||||
|
v.provenance.every(isProvenancePoint);
|
||||||
|
|
||||||
/** Derived header for legacy node lines written before `name` existed:
|
const EDGE_TYPES = ["follows", "supports"] as const;
|
||||||
* the first clause of the statement, capped — good enough to render. */
|
|
||||||
function nameFromStatement(statement: string): string {
|
|
||||||
const clause = statement.split(/[.;:—]/)[0]!.trim();
|
|
||||||
return clause.length > 60 ? `${clause.slice(0, 57)}…` : clause || statement;
|
|
||||||
}
|
|
||||||
|
|
||||||
const EDGE_TYPES = ["requires", "supports"] as const;
|
|
||||||
|
|
||||||
const isGraphEdge = (v: unknown): v is GraphEdge =>
|
const isGraphEdge = (v: unknown): v is GraphEdge =>
|
||||||
isRecord(v) &&
|
isRecord(v) &&
|
||||||
@@ -96,17 +99,9 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
|
|||||||
isString(v.to) &&
|
isString(v.to) &&
|
||||||
(EDGE_TYPES as readonly string[]).includes(v.type as string) &&
|
(EDGE_TYPES as readonly string[]).includes(v.type as string) &&
|
||||||
Array.isArray(v.provenance) &&
|
Array.isArray(v.provenance) &&
|
||||||
v.provenance.every(isString);
|
v.provenance.every(isProvenancePoint);
|
||||||
|
|
||||||
const EVENT_KINDS: readonly LedgerEventKind[] = [
|
const EVENT_KINDS: readonly LedgerEventKind[] = ["correct", "miss", "dont-know", "observation"];
|
||||||
"probe-correct",
|
|
||||||
"probe-miss",
|
|
||||||
"dont-know",
|
|
||||||
"misconception-signaled",
|
|
||||||
"taught",
|
|
||||||
"check-correct",
|
|
||||||
"check-incorrect",
|
|
||||||
];
|
|
||||||
|
|
||||||
const isLedgerEvent = (v: unknown): v is LedgerEvent =>
|
const isLedgerEvent = (v: unknown): v is LedgerEvent =>
|
||||||
isRecord(v) &&
|
isRecord(v) &&
|
||||||
@@ -127,13 +122,11 @@ export const graphStore: GraphStore = {
|
|||||||
appendNode(node: GraphNode): void {
|
appendNode(node: GraphNode): void {
|
||||||
appendLine(LEDGER_FILES.nodes, node);
|
appendLine(LEDGER_FILES.nodes, node);
|
||||||
},
|
},
|
||||||
/** Raw node lines in append order (ids are unique — no merging needed).
|
/** Raw node lines in append order. A node id may appear on more than
|
||||||
* Legacy lines without a `name` get one derived from the statement. */
|
* one line (a later session's LINK-append: phrasings + provenance); the
|
||||||
|
* derivation (core/graph.ts getGraph) merges lines by id. */
|
||||||
readNodes(): GraphNode[] {
|
readNodes(): GraphNode[] {
|
||||||
return readValid(LEDGER_FILES.nodes, isStoredGraphNode).map((n) => ({
|
return readValid(LEDGER_FILES.nodes, isGraphNode);
|
||||||
...n,
|
|
||||||
name: n.name ?? nameFromStatement(n.statement),
|
|
||||||
}));
|
|
||||||
},
|
},
|
||||||
|
|
||||||
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */
|
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */
|
||||||
|
|||||||
+89
-71
@@ -1,13 +1,16 @@
|
|||||||
/**
|
/**
|
||||||
* The extraction pipeline tests (vision §1 construction): extractor →
|
* The extraction pipeline tests (vision §1 construction, 2026-09-02 redesign):
|
||||||
* auditor → merger, the sole graph writer.
|
* extractor → auditor → merger, the sole graph writer — tree-blind,
|
||||||
|
* free-form, with the auditor owning faithfulness and identity.
|
||||||
*
|
*
|
||||||
* transcriptView — the compact view both LLM stages see (pure)
|
* transcriptView — the compact view both LLM stages see (pure):
|
||||||
|
* message-id-tagged verbatim text + quiz lines, NO tree
|
||||||
* pipeline — MockLanguageModelV4 drives both agents: findings out,
|
* pipeline — MockLanguageModelV4 drives both agents: findings out,
|
||||||
* audit verdict in, deterministic merge applied; failure
|
* audit verdict (with linkTo identity) in, deterministic
|
||||||
* paths checkpoint nothing; single-flight per chat
|
* merge applied; failure paths checkpoint nothing;
|
||||||
|
* single-flight per chat
|
||||||
*
|
*
|
||||||
* The merger itself (identity resolution, event rebuild, idempotency) is
|
* The merger itself (identity verdicts, event rebuild, idempotency) is
|
||||||
* tested in tests/graph.test.ts directly against mergeFindings — here it is
|
* tested in tests/graph.test.ts directly against mergeFindings — here it is
|
||||||
* the integration: what one full pipeline run does to the ledgers.
|
* the integration: what one full pipeline run does to the ledgers.
|
||||||
*
|
*
|
||||||
@@ -21,9 +24,16 @@ import * as fs from "node:fs";
|
|||||||
import { after, describe, it } from "node:test";
|
import { after, describe, it } from "node:test";
|
||||||
import { MockLanguageModelV4 } from "ai/test";
|
import { MockLanguageModelV4 } from "ai/test";
|
||||||
import type { UIMessage } from "ai";
|
import type { UIMessage } from "ai";
|
||||||
import type { GraphEdge, GraphNode, LedgerEvent, NodeMapping, SessionCheckpoint, SessionFindings } from "../src/core/contracts.ts";
|
import type {
|
||||||
|
AuditResult,
|
||||||
|
GraphEdge,
|
||||||
|
GraphNode,
|
||||||
|
LedgerEvent,
|
||||||
|
NodeMapping,
|
||||||
|
SessionCheckpoint,
|
||||||
|
SessionFindings,
|
||||||
|
} from "../src/core/contracts.ts";
|
||||||
import type { GraphStore } from "../src/core/ports.ts";
|
import type { GraphStore } from "../src/core/ports.ts";
|
||||||
import { createTree, markDone } from "../src/core/plan-tree.ts";
|
|
||||||
import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts";
|
import { createExtractionPipeline, transcriptView } from "../src/agent/subagents/extraction.ts";
|
||||||
import { chatStore } from "../src/store/chats.ts";
|
import { chatStore } from "../src/store/chats.ts";
|
||||||
import { PATHS } from "../config.ts";
|
import { PATHS } from "../config.ts";
|
||||||
@@ -72,7 +82,7 @@ function quizPart(toolCallId: string, input: Record<string, unknown>, output: Re
|
|||||||
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
|
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
|
||||||
}
|
}
|
||||||
|
|
||||||
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: ReturnType<typeof createTree>): unknown {
|
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: unknown): unknown {
|
||||||
return {
|
return {
|
||||||
type: `tool-${name}`,
|
type: `tool-${name}`,
|
||||||
toolCallId,
|
toolCallId,
|
||||||
@@ -82,21 +92,11 @@ function planPart(toolCallId: string, name: string, input: Record<string, unknow
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/** A session-shaped fixture chat: probe → plan → check → mark_done. */
|
/** A session-shaped fixture chat: probe → teaching text → check. Tree-blind
|
||||||
|
* world: the plan tree is NOT part of extraction — it is noise. */
|
||||||
function fixtureChat() {
|
function fixtureChat() {
|
||||||
const tree = markDone(
|
|
||||||
createTree(
|
|
||||||
[
|
|
||||||
{ id: "cr", title: "Chain rule intuition" },
|
|
||||||
{ id: "apply", title: "Applying the chain rule" },
|
|
||||||
],
|
|
||||||
[{ from: "cr", to: "apply" }],
|
|
||||||
),
|
|
||||||
"cr",
|
|
||||||
);
|
|
||||||
const chat = chatStore.createChat("Extraction fixture");
|
const chat = chatStore.createChat("Extraction fixture");
|
||||||
createdChats.push(chat.id);
|
createdChats.push(chat.id);
|
||||||
chatStore.saveTree(chat.id, tree);
|
|
||||||
chatStore.saveMessages(chat.id, [
|
chatStore.saveMessages(chat.id, [
|
||||||
user("u1", "teach me the chain rule"),
|
user("u1", "teach me the chain rule"),
|
||||||
assistant("a1", [
|
assistant("a1", [
|
||||||
@@ -107,15 +107,14 @@ function fixtureChat() {
|
|||||||
]),
|
]),
|
||||||
assistant("a2", [
|
assistant("a2", [
|
||||||
{ type: "text", text: "The derivative measures how fast a function changes." },
|
{ type: "text", text: "The derivative measures how fast a function changes." },
|
||||||
planPart("p1", "plan_init", { nodes: [], edges: [] }, tree),
|
planPart("p1", "plan_init", { nodes: [], edges: [] }, { nodes: [], edges: [], currentNodeId: null, doneNodeIds: [], version: 1 }),
|
||||||
]),
|
]),
|
||||||
assistant("a3", [
|
assistant("a3", [
|
||||||
quizPart("check-1", { question: "Check", nodeId: "apply" }, {
|
quizPart("check-1", { question: "Check" }, {
|
||||||
questionId: "check-1", kind: "quiz", questionText: "Check", correct: true, dontKnow: false,
|
questionId: "check-1", kind: "quiz", questionText: "Check", correct: true, dontKnow: false,
|
||||||
answeredAt: "2026-08-22T02:00:00.000Z",
|
answeredAt: "2026-08-22T02:00:00.000Z",
|
||||||
}),
|
}),
|
||||||
]),
|
]),
|
||||||
assistant("a4", [planPart("p2", "plan_mark_done", { id: "cr" }, tree)]),
|
|
||||||
]);
|
]);
|
||||||
return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
|
return { chatId: chat.id, record: chatStore.getChat(chat.id)! };
|
||||||
}
|
}
|
||||||
@@ -130,30 +129,38 @@ after(() => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
/** What the extractor would produce for the fixture session. */
|
/** What the extractor would produce for the fixture session — tree-blind,
|
||||||
|
* free-form, with message-granular provenance + an understanding note. */
|
||||||
const FINDINGS: SessionFindings = {
|
const FINDINGS: SessionFindings = {
|
||||||
propositions: [
|
propositions: [
|
||||||
{
|
{
|
||||||
id: "p1",
|
id: "p1",
|
||||||
name: "Chain rule",
|
statement: "The derivative measures how fast a function changes",
|
||||||
statement: "Chain rule intuition",
|
description: "The derivative of a function at a point is its instantaneous rate of change there.",
|
||||||
aliases: [],
|
aliases: [],
|
||||||
kind: "truth",
|
messages: ["a2"],
|
||||||
treeNodeId: "cr",
|
evidence: ["TEACHER [a2]: The derivative measures how fast a function changes."],
|
||||||
evidence: ["TEACHER: The derivative measures how fast a function changes."],
|
understanding: "solid",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "p2",
|
id: "p2",
|
||||||
name: "Applying chain rule",
|
statement: "Chain rule intuition",
|
||||||
statement: "Applying the chain rule",
|
description: "The chain rule differentiates composed functions by multiplying local rates.",
|
||||||
aliases: [],
|
aliases: [],
|
||||||
kind: "truth",
|
messages: ["a2"],
|
||||||
treeNodeId: "apply",
|
evidence: ["LEARNER [u1]: teach me the chain rule"],
|
||||||
evidence: ["FINAL PLAN TREE (node ids): cr = Chain rule intuition; apply = Applying the chain rule"],
|
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
edges: [{ from: "p1", to: "p2", type: "requires", evidence: ["TREE EDGES: cr requires apply"] }],
|
edges: [
|
||||||
attribution: { quiz: { "probe-1": "p1" } },
|
{
|
||||||
|
from: "p1",
|
||||||
|
to: "p2",
|
||||||
|
type: "follows",
|
||||||
|
messages: ["a2"],
|
||||||
|
evidence: ["LEARNER [u1]: teach me the chain rule"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
attribution: { quiz: { "probe-1": "p1", "check-1": "p2" } },
|
||||||
};
|
};
|
||||||
|
|
||||||
/** A model whose ONE tool call is `name` with the given input; then text. */
|
/** A model whose ONE tool call is `name` with the given input; then text. */
|
||||||
@@ -202,28 +209,26 @@ function silentModel(): MockLanguageModelV4 {
|
|||||||
// ─── transcriptView (pure) ───────────────────────────────────────────────────
|
// ─── transcriptView (pure) ───────────────────────────────────────────────────
|
||||||
|
|
||||||
describe("transcriptView (pure)", () => {
|
describe("transcriptView (pure)", () => {
|
||||||
it("shows the final tree, verbatim conversation, structured events — plan ops folded", () => {
|
it("shows message-id-tagged verbatim conversation + quiz lines — tree-blind, plan ops folded", () => {
|
||||||
const { record } = fixtureChat();
|
const { record } = fixtureChat();
|
||||||
const view = transcriptView(record);
|
const view = transcriptView(record);
|
||||||
|
|
||||||
// The final tree with node ids (the extractor's backbone + the
|
|
||||||
// pruned-out stays pruned — only the FINAL tree is shown).
|
|
||||||
assert.match(view, /SESSION: Extraction fixture/);
|
assert.match(view, /SESSION: Extraction fixture/);
|
||||||
assert.match(view, /FINAL PLAN TREE \(node ids\): cr = Chain rule intuition; apply = Applying the chain rule/);
|
|
||||||
assert.match(view, /TREE EDGES: cr requires apply/);
|
|
||||||
|
|
||||||
// Conversation verbatim — evidence quotes can come from it.
|
// Conversation verbatim, each line tagged with its message id —
|
||||||
assert.match(view, /LEARNER: teach me the chain rule/);
|
// the provenance anchors the extractor reports back.
|
||||||
assert.match(view, /TEACHER: The derivative measures how fast a function changes\./);
|
assert.match(view, /LEARNER \[u1\] teach me the chain rule/);
|
||||||
|
assert.match(view, /TEACHER \[a2\] The derivative measures how fast a function changes\./);
|
||||||
|
|
||||||
// Structured events, attribution-ready.
|
// Structured events, attribution-ready.
|
||||||
assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/);
|
assert.match(view, /QUIZ \[probe-1\] What is a derivative, roughly\? — the learner answered incorrectly/);
|
||||||
assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/);
|
assert.match(view, /QUIZ \[check-1\] Check — the learner answered correctly/);
|
||||||
assert.match(view, /TEACHER marked tree node done \[p2\]: cr/);
|
|
||||||
|
|
||||||
// plan_init / plan_mark_done tool parts themselves are folded into
|
// TREE-BLIND: no plan tree, no tree edges, no mark_done lines — plan
|
||||||
// the tree line — no raw JSON noise.
|
// ops fold to the generic one-liner like every other non-quiz tool.
|
||||||
assert.doesNotMatch(view, /plan_init/);
|
assert.doesNotMatch(view, /FINAL PLAN TREE/);
|
||||||
|
assert.doesNotMatch(view, /TREE EDGES/);
|
||||||
|
assert.match(view, /\(plan_init tool result delivered\)/);
|
||||||
assert.doesNotMatch(view, /output-available/);
|
assert.doesNotMatch(view, /output-available/);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -253,48 +258,61 @@ describe("transcriptView (pure)", () => {
|
|||||||
// ─── the pipeline ───────────────────────────────────────────────────────────
|
// ─── the pipeline ───────────────────────────────────────────────────────────
|
||||||
|
|
||||||
describe("the extraction pipeline (extractor → auditor → merger)", () => {
|
describe("the extraction pipeline (extractor → auditor → merger)", () => {
|
||||||
it("runs end to end: findings → audit verdict → deterministic merge + checkpoint", async () => {
|
it("runs end to end: findings → audit verdict (identity) → deterministic merge + checkpoint", async () => {
|
||||||
const { chatId } = fixtureChat();
|
const { chatId } = fixtureChat();
|
||||||
const ledgers = memoryLedgers();
|
const ledgers = memoryLedgers();
|
||||||
|
|
||||||
// The auditor passes p1 through, drops p2 (unbacked), re-points the
|
// The auditor passes p1 through and LINKS it to a pre-existing node
|
||||||
// probe attribution to p1, and records the drop.
|
// (identity verdict), drops p2 (unbacked) + the edge with it.
|
||||||
const verified: SessionFindings = {
|
const existingId = "n_existing";
|
||||||
propositions: [FINDINGS.propositions[0]!],
|
ledgers.appendNode({
|
||||||
|
id: existingId,
|
||||||
|
statement: "Derivatives measure rates of change",
|
||||||
|
description: "Pre-existing node from an earlier session.",
|
||||||
|
aliases: [],
|
||||||
|
createdAt: "2026-08-01T00:00:00.000Z",
|
||||||
|
provenance: [],
|
||||||
|
});
|
||||||
|
const verdict: AuditResult = {
|
||||||
|
propositions: [
|
||||||
|
{ ...FINDINGS.propositions[0]!, linkTo: existingId },
|
||||||
|
],
|
||||||
edges: [],
|
edges: [],
|
||||||
attribution: { quiz: { "probe-1": "p1" } },
|
attribution: { quiz: { "probe-1": "p1" } },
|
||||||
|
dropped: [{ ref: "p2", reason: "evidence not found" }, { ref: "p1→p2", reason: "endpoint dropped" }],
|
||||||
};
|
};
|
||||||
|
|
||||||
const pipeline = createExtractionPipeline({
|
const pipeline = createExtractionPipeline({
|
||||||
chatStore,
|
chatStore,
|
||||||
graphStore: ledgers,
|
graphStore: ledgers,
|
||||||
extractor: submitModel("submit_findings", FINDINGS),
|
extractor: submitModel("submit_findings", FINDINGS),
|
||||||
auditor: submitModel("submit_review", { ...verified, dropped: [{ ref: "p2", reason: "evidence not found" }] }),
|
auditor: submitModel("submit_review", verdict),
|
||||||
});
|
});
|
||||||
const stats = await pipeline.runExtraction(chatId);
|
const stats = await pipeline.runExtraction(chatId);
|
||||||
|
|
||||||
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 3 });
|
// The linked proposition created NO new node; the understanding note
|
||||||
|
// became an observation; both quizzes became events.
|
||||||
|
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 3 });
|
||||||
|
|
||||||
// Only the audited proposition became a node; mapping provenance kept.
|
// Link-append: the phrasing joined the existing node's aliases and
|
||||||
assert.equal(ledgers.readNodes().length, 1);
|
// the provenance unioned; no second node line defines a new node.
|
||||||
assert.equal(ledgers.readNodes()[0]!.statement, "Chain rule intuition");
|
assert.equal(ledgers.readNodes().length, 2); // the pre-seeded line + the link line
|
||||||
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "cr", graphNodeId: ledgers.readNodes()[0]!.id }]);
|
assert.deepEqual(ledgers.readMappings(), [{ chatId, sourceId: "p1", graphNodeId: existingId }]);
|
||||||
|
|
||||||
// Events rebuilt deterministically from the transcript — the probe
|
// Events rebuilt deterministically: quizzes in transcript order, then
|
||||||
// retroactively attributed, the check tagged, mark_done mapped.
|
// the observation. check-1's attribution pointed at the DROPPED p2 →
|
||||||
|
// unresolvable → null (the question text is kept for later attribution).
|
||||||
const events = ledgers.readEvents();
|
const events = ledgers.readEvents();
|
||||||
assert.deepEqual(
|
assert.deepEqual(
|
||||||
events.map((e) => [e.kind, e.ref]),
|
events.map((e) => [e.kind, e.ref, e.nodeId]),
|
||||||
[
|
[
|
||||||
["probe-miss", "probe-1"],
|
["miss", "probe-1", existingId],
|
||||||
["check-correct", "check-1"],
|
["correct", "check-1", null],
|
||||||
["taught", "taught:cr"],
|
["observation", "obs:p1", existingId],
|
||||||
],
|
],
|
||||||
);
|
);
|
||||||
const nodeId = ledgers.readNodes()[0]!.id;
|
assert.equal(events[0]!.nodeId, existingId, "probe attributed via the verdict's mapping");
|
||||||
assert.equal(events[0]!.nodeId, nodeId, "probe attributed via audit");
|
assert.equal(events[2]!.note, "solid", "the understanding note joins the ledger");
|
||||||
assert.equal(events[1]!.nodeId, null, "tagged to a dropped proposition → null");
|
|
||||||
assert.equal(events[2]!.nodeId, nodeId, "taught via tree-node mapping");
|
|
||||||
|
|
||||||
// Checkpointed through the record's updatedAt — never the same turns twice.
|
// Checkpointed through the record's updatedAt — never the same turns twice.
|
||||||
assert.equal(ledgers.readCheckpoints().length, 1);
|
assert.equal(ledgers.readCheckpoints().length, 1);
|
||||||
@@ -318,7 +336,7 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
|
|||||||
chatStore,
|
chatStore,
|
||||||
graphStore: ledgers,
|
graphStore: ledgers,
|
||||||
extractor: silentModel(),
|
extractor: silentModel(),
|
||||||
auditor: submitModel("submit_review", FINDINGS),
|
auditor: submitModel("submit_review", { ...FINDINGS, dropped: [] }),
|
||||||
});
|
});
|
||||||
await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/);
|
await assert.rejects(failingExtractor.runExtraction(chatId), /the extractor never submitted findings/);
|
||||||
assert.equal(ledgers.readCheckpoints().length, 0);
|
assert.equal(ledgers.readCheckpoints().length, 0);
|
||||||
|
|||||||
+117
-138
@@ -1,13 +1,14 @@
|
|||||||
/**
|
/**
|
||||||
* Graph-as-context tests (W8; vision §1 "the graph as session context").
|
* Graph-as-context tests (vision §1 "the graph as session context", 2026-09-02
|
||||||
|
* redesign vocabulary: follows edges, no node kinds).
|
||||||
*
|
*
|
||||||
* renderGraphContext — pure: empty / small / cap-truncation rendering
|
* renderGraphContext — pure: empty / small / cap-truncation rendering
|
||||||
* buildGraphContext — per-request entry point (derived-graph → block)
|
* buildGraphContext — per-request entry point (derived-graph → block)
|
||||||
* injection plumbing — chat-flow style: the teacher's instructions carry
|
* injection plumbing — chat-flow style: the teacher's instructions carry
|
||||||
* the block (MockLanguageModelV4 captures the prompt)
|
* the block (MockLanguageModelV4 captures the prompt)
|
||||||
* exact-match merge — the merger (mergeFindings) dedupes
|
* cross-chat edges — the merger (mergeFindings) appends one ASSERTION
|
||||||
* case/whitespace-identical statements onto existing
|
* line per session; reads merge with unioned
|
||||||
* nodes (no fuzzy matching)
|
* message-granular provenance
|
||||||
*
|
*
|
||||||
* Ledger isolation: NONE needed for the merge tests — the merger's ledger is
|
* Ledger isolation: NONE needed for the merge tests — the merger's ledger is
|
||||||
* injectable (in-memory), so the live data/graph files are only ever READ
|
* injectable (in-memory), so the live data/graph files are only ever READ
|
||||||
@@ -22,7 +23,14 @@ import * as path from "node:path";
|
|||||||
import { after, describe, it } from "node:test";
|
import { after, describe, it } from "node:test";
|
||||||
import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
|
import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
|
||||||
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
|
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
|
||||||
import type { GraphEdge, GraphNode, LedgerEvent, SessionFindings } from "../src/core/contracts.ts";
|
import type {
|
||||||
|
AuditResult,
|
||||||
|
GraphEdge,
|
||||||
|
GraphNode,
|
||||||
|
LedgerEvent,
|
||||||
|
NodeMapping,
|
||||||
|
SessionCheckpoint,
|
||||||
|
} from "../src/core/contracts.ts";
|
||||||
import type { NodeStateClass } from "../src/core/contracts.ts";
|
import type { NodeStateClass } from "../src/core/contracts.ts";
|
||||||
import { getGraph, mergeFindings } from "../src/core/graph.ts";
|
import { getGraph, mergeFindings } from "../src/core/graph.ts";
|
||||||
import type { GraphStore } from "../src/core/ports.ts";
|
import type { GraphStore } from "../src/core/ports.ts";
|
||||||
@@ -35,19 +43,25 @@ import {
|
|||||||
import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
|
import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
|
||||||
import { handleChatRequest } from "../src/server/chat-handler.ts";
|
import { handleChatRequest } from "../src/server/chat-handler.ts";
|
||||||
import { chatStore } from "../src/store/chats.ts";
|
import { chatStore } from "../src/store/chats.ts";
|
||||||
import type { NodeMapping, SessionCheckpoint } from "../src/core/contracts.ts";
|
|
||||||
import { graphStore } from "../src/store/graph.ts";
|
import { graphStore } from "../src/store/graph.ts";
|
||||||
import { createTree } from "../src/core/plan-tree.ts";
|
|
||||||
import { PATHS } from "../config.ts";
|
import { PATHS } from "../config.ts";
|
||||||
|
|
||||||
// ─── fixtures ────────────────────────────────────────────────────────────────
|
// ─── fixtures ────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
function node(id: string, statement: string, over: Partial<GraphNode> = {}): GraphNode {
|
function node(id: string, statement: string, over: Partial<GraphNode> = {}): GraphNode {
|
||||||
return { id, name: statement.split(";")[0]!, statement, aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z", ...over };
|
return {
|
||||||
|
id,
|
||||||
|
statement,
|
||||||
|
description: `${statement} — elaborated.`,
|
||||||
|
aliases: [],
|
||||||
|
createdAt: "2026-08-22T00:00:00.000Z",
|
||||||
|
provenance: [],
|
||||||
|
...over,
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
function edge(from: string, to: string): GraphEdge {
|
function edge(from: string, to: string): GraphEdge {
|
||||||
return { from, to, type: "requires", provenance: ["chat_x"] };
|
return { from, to, type: "follows", provenance: [{ chatId: "chat_x", messageId: "m1" }] };
|
||||||
}
|
}
|
||||||
|
|
||||||
function states(...entries: [string, NodeStateClass][]): Record<string, NodeStateClass> {
|
function states(...entries: [string, NodeStateClass][]): Record<string, NodeStateClass> {
|
||||||
@@ -61,7 +75,7 @@ describe("renderGraphContext (pure)", () => {
|
|||||||
assert.equal(renderGraphContext([], [], {}), null);
|
assert.equal(renderGraphContext([], [], {}), null);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("renders a header, state-grouped node lines (hotspots first), and chained requires edges", () => {
|
it("renders a header, state-grouped node lines (hotspots first), and chained follows edges", () => {
|
||||||
const nodes = [
|
const nodes = [
|
||||||
node("n_a", "Chain rule intuition"),
|
node("n_a", "Chain rule intuition"),
|
||||||
node("n_b", "Quotient rule"),
|
node("n_b", "Quotient rule"),
|
||||||
@@ -73,26 +87,26 @@ describe("renderGraphContext (pure)", () => {
|
|||||||
["n_a", "known"],
|
["n_a", "known"],
|
||||||
["n_b", "missed"],
|
["n_b", "missed"],
|
||||||
["n_c", "candidate"],
|
["n_c", "candidate"],
|
||||||
["n_d", "taught"],
|
["n_d", "dont-know"],
|
||||||
);
|
);
|
||||||
const out = renderGraphContext(nodes, edges, st)!;
|
const out = renderGraphContext(nodes, edges, st)!;
|
||||||
|
|
||||||
assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present");
|
assert.ok(out.includes(GRAPH_CONTEXT_HEADER), "header line present");
|
||||||
assert.match(out, /4 nodes, 3 requires edges/);
|
assert.match(out, /4 nodes, 3 follows edges/);
|
||||||
|
|
||||||
// Node lines: "• [state] statement (id)".
|
// Node lines: "• [state] statement (id)".
|
||||||
assert.ok(out.includes("• [missed] Quotient rule (n_b)"));
|
assert.ok(out.includes("• [missed] Quotient rule (n_b)"));
|
||||||
assert.ok(out.includes("• [known] Chain rule intuition (n_a)"));
|
assert.ok(out.includes("• [known] Chain rule intuition (n_a)"));
|
||||||
assert.ok(out.includes("• [taught] Composite functions (n_d)"));
|
assert.ok(out.includes("• [dont-know] Composite functions (n_d)"));
|
||||||
assert.ok(out.includes("• [candidate] Product rule (n_c)"));
|
assert.ok(out.includes("• [candidate] Product rule (n_c)"));
|
||||||
|
|
||||||
// Groups in the decided order: hotspots, known, taught, candidates.
|
// Groups in the decided order: hotspots, known, candidates.
|
||||||
const order = ["Hotspots", "Known", "Taught", "Candidates"].map((g) => out.indexOf(g));
|
const order = ["Hotspots", "Known", "Candidates"].map((g) => out.indexOf(g));
|
||||||
assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order");
|
assert.deepEqual([...order].sort((a, b) => a - b), order, "groups appear in priority order");
|
||||||
|
|
||||||
// Edges merged into chains: d → a → b → c (one line, three arrows).
|
// Edges merged into chains: d → a → b → c (one line, three arrows).
|
||||||
assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule"));
|
assert.ok(out.includes("Composite functions → Chain rule intuition → Quotient rule → Product rule"));
|
||||||
assert.ok(out.includes("Requires (prerequisite → dependent)"));
|
assert.ok(out.includes("Follows (premise → derivation)"));
|
||||||
assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter");
|
assert.ok(out.includes("===== end of knowledge graph ====="), "clear end delimiter");
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -104,15 +118,8 @@ describe("renderGraphContext (pure)", () => {
|
|||||||
assert.ok(!out.includes("B → C"), "no edge was invented between siblings");
|
assert.ok(!out.includes("B → C"), "no edge was invented between siblings");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("marks misconception-kind nodes as hotspots even when their state is candidate", () => {
|
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop follows neighborhood only, with a truncation note", () => {
|
||||||
const nodes = [node("n_m", "Division by zero is fine", { kind: "misconception" })];
|
// 3 hotspots, each with a premise and a derivation (1-hop), plus a
|
||||||
const out = renderGraphContext(nodes, [], states(["n_m", "candidate"]))!;
|
|
||||||
assert.match(out, /• \[candidate\] Division by zero is fine \(n_m\) · misconception/);
|
|
||||||
assert.ok(out.includes("Hotspots"), "misconception nodes join the hotspot group");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("caps at MAX_CONTEXT_NODES: hotspots + their 1-hop requires neighborhood only, with a truncation note", () => {
|
|
||||||
// 3 hotspots, each with a prerequisite and a dependent (1-hop), plus a
|
|
||||||
// long tail of disconnected candidates that must drop out.
|
// long tail of disconnected candidates that must drop out.
|
||||||
const nodes: GraphNode[] = [];
|
const nodes: GraphNode[] = [];
|
||||||
const edges: GraphEdge[] = [];
|
const edges: GraphEdge[] = [];
|
||||||
@@ -121,7 +128,7 @@ describe("renderGraphContext (pure)", () => {
|
|||||||
const h = `n_h${i}`;
|
const h = `n_h${i}`;
|
||||||
const pre = `n_p${i}`;
|
const pre = `n_p${i}`;
|
||||||
const dep = `n_d${i}`;
|
const dep = `n_d${i}`;
|
||||||
nodes.push(node(pre, `Prerequisite ${i}`), node(h, `Hotspot ${i}`), node(dep, `Dependent ${i}`));
|
nodes.push(node(pre, `Premise ${i}`), node(h, `Hotspot ${i}`), node(dep, `Derivation ${i}`));
|
||||||
edges.push(edge(pre, h), edge(h, dep));
|
edges.push(edge(pre, h), edge(h, dep));
|
||||||
st[pre] = "known";
|
st[pre] = "known";
|
||||||
st[h] = "missed";
|
st[h] = "missed";
|
||||||
@@ -139,11 +146,11 @@ describe("renderGraphContext (pure)", () => {
|
|||||||
|
|
||||||
const out = renderGraphContext(nodes, edges, st)!;
|
const out = renderGraphContext(nodes, edges, st)!;
|
||||||
assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay");
|
assert.ok(out.includes("• [missed] Hotspot 1 (n_h1)"), "hotspots stay");
|
||||||
assert.ok(out.includes("• [known] Prerequisite 1 (n_p1)"), "1-hop prerequisites stay");
|
assert.ok(out.includes("• [known] Premise 1 (n_p1)"), "1-hop premises stay");
|
||||||
assert.ok(out.includes("• [candidate] Dependent 1 (n_d1)"), "1-hop dependents stay");
|
assert.ok(out.includes("• [candidate] Derivation 1 (n_d1)"), "1-hop derivations stay");
|
||||||
assert.ok(!out.includes("Far away"), "disconnected nodes drop out");
|
assert.ok(!out.includes("Far away"), "disconnected nodes drop out");
|
||||||
assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted");
|
assert.match(out, new RegExp(`showing 9 of ${nodes.length} nodes`), "truncation is noted");
|
||||||
assert.ok(out.includes(", 6 requires edges"), "edges among the selected nodes stay");
|
assert.ok(out.includes(", 6 follows edges"), "edges among the selected nodes stay");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps the full graph at exactly the cap (no truncation note)", () => {
|
it("keeps the full graph at exactly the cap (no truncation note)", () => {
|
||||||
@@ -169,11 +176,11 @@ describe("buildGraphContext", () => {
|
|||||||
it("derives states from events and renders the block", () => {
|
it("derives states from events and renders the block", () => {
|
||||||
const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")];
|
const nodes = [node("n_a", "Solo node"), node("n_b", "Checked node")];
|
||||||
const events: LedgerEvent[] = [
|
const events: LedgerEvent[] = [
|
||||||
{ nodeId: "n_b", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
{ nodeId: "n_b", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
||||||
];
|
];
|
||||||
const out = buildGraphContext({ nodes, edges: [], events })!;
|
const out = buildGraphContext({ nodes, edges: [], events })!;
|
||||||
assert.ok(out.includes(GRAPH_CONTEXT_HEADER));
|
assert.ok(out.includes(GRAPH_CONTEXT_HEADER));
|
||||||
assert.ok(out.includes("• [known] Checked node (n_b)"), "check-correct → known");
|
assert.ok(out.includes("• [known] Checked node (n_b)"), "correct → known");
|
||||||
assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate");
|
assert.ok(out.includes("• [candidate] Solo node (n_a)"), "no events → candidate");
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -244,7 +251,7 @@ describe("graph-context injection (chat-flow style)", () => {
|
|||||||
nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")],
|
nodes: [node("n_ctx", "Chain rule intuition"), node("n_ctx2", "Derivative as a limit")],
|
||||||
edges: [edge("n_ctx2", "n_ctx")],
|
edges: [edge("n_ctx2", "n_ctx")],
|
||||||
events: [
|
events: [
|
||||||
{ nodeId: "n_ctx2", kind: "check-correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
{ nodeId: "n_ctx2", kind: "correct", chatId: "c1", at: "2026-08-22T00:00:00.000Z", ref: "r1" },
|
||||||
] as LedgerEvent[],
|
] as LedgerEvent[],
|
||||||
};
|
};
|
||||||
const context = buildGraphContext(graph);
|
const context = buildGraphContext(graph);
|
||||||
@@ -296,7 +303,7 @@ describe("graph-context injection (chat-flow style)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
// ─── exact-match merge (merger identity resolution) ──────────────────────────
|
// ─── cross-chat edge assertions (merger + read-time merging) ────────────────
|
||||||
|
|
||||||
/** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */
|
/** An in-memory stand-in for the five ledgers (the GraphStore port, core/ports.ts). */
|
||||||
function memoryLedgers() {
|
function memoryLedgers() {
|
||||||
@@ -317,128 +324,100 @@ function memoryLedgers() {
|
|||||||
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
|
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
|
||||||
readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
|
readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
|
||||||
// Test accessors (not part of the GraphStore surface).
|
// Test accessors (not part of the GraphStore surface).
|
||||||
counts: { get nodes() { return nodes.length; }, get mappings() { return mappings.length; } },
|
counts: { get nodes() { return nodes.length; } },
|
||||||
} satisfies GraphStore & { counts: { nodes: number; mappings: number } };
|
} satisfies GraphStore & { counts: { nodes: number } };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Findings as the extractor would file them for a plan tree: one
|
/** A minimal chat record with one user message (the merger reads the real chat store). */
|
||||||
* proposition per node (statement = title, treeNodeId set), one `requires`
|
function fixtureChat(title: string) {
|
||||||
* edge per tree edge — the merge tests drive the merger with these. */
|
|
||||||
function treeFindings(tree: ReturnType<typeof createTree>): SessionFindings {
|
|
||||||
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
|
|
||||||
return {
|
|
||||||
propositions: tree.nodes.map((n, i) => ({
|
|
||||||
id: `p${i + 1}`,
|
|
||||||
name: n.title,
|
|
||||||
statement: n.title,
|
|
||||||
aliases: [],
|
|
||||||
kind: "truth" as const,
|
|
||||||
treeNodeId: n.id,
|
|
||||||
evidence: ["fixture evidence"],
|
|
||||||
})),
|
|
||||||
edges: tree.edges.map((e) => ({
|
|
||||||
from: idByNodeId.get(e.from)!,
|
|
||||||
to: idByNodeId.get(e.to)!,
|
|
||||||
type: "requires" as const,
|
|
||||||
evidence: ["fixture evidence"],
|
|
||||||
})),
|
|
||||||
attribution: { quiz: {} },
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** A real chat record with a stored tree (the merger reads the real chat store). */
|
|
||||||
function fixtureChat(title: string, tree: ReturnType<typeof createTree>) {
|
|
||||||
const chat = chatStore.createChat(title);
|
const chat = chatStore.createChat(title);
|
||||||
createdChats.push(chat.id);
|
createdChats.push(chat.id);
|
||||||
chatStore.saveTree(chat.id, tree);
|
|
||||||
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]);
|
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] }]);
|
||||||
return chatStore.getChat(chat.id)!;
|
return chatStore.getChat(chat.id)!;
|
||||||
}
|
}
|
||||||
|
|
||||||
describe("exact-match merge (merger identity resolution)", () => {
|
describe("cross-chat edge assertions (merger + read-time merging)", () => {
|
||||||
it("maps a case/whitespace-identical title onto the existing node instead of duplicating", () => {
|
it("a second session asserting the same edge appends a second line; reads merge with unioned provenance", () => {
|
||||||
const ledgers = memoryLedgers();
|
const ledgers = memoryLedgers();
|
||||||
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
|
|
||||||
const chat1 = fixtureChat("Merge A", tree1);
|
|
||||||
let stats = mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
|
|
||||||
assert.deepEqual(stats, { nodes: 1, edges: 0, events: 0 });
|
|
||||||
const firstId = ledgers.readNodes()[0]!.id;
|
|
||||||
|
|
||||||
// Second session, same proposition: different casing + padding.
|
// Session 1: two fresh propositions + a follows edge between them.
|
||||||
const tree2 = createTree([{ id: "cr2", title: " chain Rule INTUITION " }], []);
|
const chat1 = fixtureChat("Edge A");
|
||||||
const chat2 = fixtureChat("Merge B", tree2);
|
const verdict1: AuditResult = {
|
||||||
stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
|
propositions: [
|
||||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "no duplicate node appended");
|
{
|
||||||
|
id: "p1",
|
||||||
assert.equal(ledgers.counts.nodes, 1, "still exactly one graph node");
|
statement: "Derivative as a limit",
|
||||||
assert.equal(ledgers.readNodes()[0]!.id, firstId);
|
description: "The derivative is the limit of difference quotients.",
|
||||||
// Provenance stays honest: one mapping line per (chat, graph node).
|
aliases: [],
|
||||||
assert.equal(ledgers.counts.mappings, 2);
|
messages: ["u1"],
|
||||||
assert.deepEqual(ledgers.readMappings()[1], { chatId: chat2.id, sourceId: "cr2", graphNodeId: firstId });
|
evidence: ["e"],
|
||||||
});
|
},
|
||||||
|
{
|
||||||
it("matches aliases too (exact, case-insensitive)", () => {
|
id: "p2",
|
||||||
const ledgers = memoryLedgers();
|
statement: "Chain rule intuition",
|
||||||
ledgers.appendNode({
|
description: "Composed functions differentiate by multiplying local rates.",
|
||||||
id: "n_alias",
|
aliases: [],
|
||||||
name: "Differentiation",
|
messages: ["u1"],
|
||||||
statement: "Differentiation basics",
|
evidence: ["e"],
|
||||||
aliases: ["the derivative"],
|
},
|
||||||
kind: "truth",
|
|
||||||
createdAt: "2026-08-22T00:00:00.000Z",
|
|
||||||
});
|
|
||||||
const tree = createTree([{ id: "d", title: "The Derivative" }], []);
|
|
||||||
const chat = fixtureChat("Merge alias", tree);
|
|
||||||
const stats = mergeFindings(chat.id, treeFindings(tree), { chatStore, graphStore: ledgers });
|
|
||||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 }, "alias match → no new node");
|
|
||||||
assert.deepEqual(ledgers.readMappings(), [{ chatId: chat.id, sourceId: "d", graphNodeId: "n_alias" }]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("does NOT fuzzy-match near-identical titles (that stays deferred)", () => {
|
|
||||||
const ledgers = memoryLedgers();
|
|
||||||
const tree1 = createTree([{ id: "cr", title: "Chain rule intuition" }], []);
|
|
||||||
const chat1 = fixtureChat("Fuzzy A", tree1);
|
|
||||||
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
|
|
||||||
const tree2 = createTree([{ id: "cr2", title: "Chain rules intuition" }], []);
|
|
||||||
const chat2 = fixtureChat("Fuzzy B", tree2);
|
|
||||||
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
|
|
||||||
assert.equal(stats.nodes, 1, "near-miss statements stay separate nodes");
|
|
||||||
assert.equal(ledgers.counts.nodes, 2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("merged nodes unify the second chat's edges onto the existing graph node", () => {
|
|
||||||
const ledgers = memoryLedgers();
|
|
||||||
const tree1 = createTree(
|
|
||||||
[
|
|
||||||
{ id: "base", title: "Derivative as a limit" },
|
|
||||||
{ id: "chain", title: "Chain rule intuition" },
|
|
||||||
],
|
],
|
||||||
[{ from: "base", to: "chain" }],
|
edges: [{ from: "p1", to: "p2", type: "follows", messages: ["u1"], evidence: ["e"] }],
|
||||||
);
|
attribution: { quiz: {} },
|
||||||
const chat1 = fixtureChat("Edge A", tree1);
|
dropped: [],
|
||||||
mergeFindings(chat1.id, treeFindings(tree1), { chatStore, graphStore: ledgers });
|
};
|
||||||
|
let stats = mergeFindings(chat1.id, verdict1, { chatStore, graphStore: ledgers });
|
||||||
|
assert.deepEqual(stats, { nodes: 2, edges: 1, events: 0 });
|
||||||
|
const base = ledgers.readNodes()[0]!.id;
|
||||||
|
const chain = ledgers.readNodes()[1]!.id;
|
||||||
|
|
||||||
// Second session re-teaches the same two propositions (exact statements
|
// Session 2: the auditor links both propositions to the SAME nodes
|
||||||
// modulo case) and asserts the same edge — it must dedupe onto the
|
// (identity verdicts) and asserts the same relation.
|
||||||
// existing nodes.
|
const chat2 = fixtureChat("Edge B");
|
||||||
const tree2 = createTree(
|
const verdict2: AuditResult = {
|
||||||
[
|
propositions: [
|
||||||
{ id: "base", title: "derivative as a limit" },
|
{
|
||||||
{ id: "chain", title: "chain rule intuition" },
|
id: "q1",
|
||||||
|
statement: "The derivative as a limit of quotients", // different phrasing
|
||||||
|
description: "…",
|
||||||
|
aliases: [],
|
||||||
|
messages: ["u1"],
|
||||||
|
evidence: ["e"],
|
||||||
|
linkTo: base,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "q2",
|
||||||
|
statement: "Chain rule, intuitively",
|
||||||
|
description: "…",
|
||||||
|
aliases: [],
|
||||||
|
messages: ["u1"],
|
||||||
|
evidence: ["e"],
|
||||||
|
linkTo: chain,
|
||||||
|
},
|
||||||
],
|
],
|
||||||
[{ from: "base", to: "chain" }],
|
edges: [{ from: "q1", to: "q2", type: "follows", messages: ["u1"], evidence: ["e"] }],
|
||||||
);
|
attribution: { quiz: {} },
|
||||||
const chat2 = fixtureChat("Edge B", tree2);
|
dropped: [],
|
||||||
const stats = mergeFindings(chat2.id, treeFindings(tree2), { chatStore, graphStore: ledgers });
|
};
|
||||||
// A new ASSERTION line is appended (provenance is honest) — reads merge
|
stats = mergeFindings(chat2.id, verdict2, { chatStore, graphStore: ledgers });
|
||||||
// by (from,to,type), so the derived graph still has one edge with both
|
|
||||||
// sessions in its provenance.
|
// No new nodes; ONE new assertion line (per-session honest provenance).
|
||||||
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line");
|
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 }, "no new nodes, one re-assertion line");
|
||||||
assert.equal(ledgers.counts.nodes, 2);
|
assert.equal(ledgers.counts.nodes, 4, "2 defining lines + 2 link-appends for the linked propositions");
|
||||||
const lines = ledgers.readEdges();
|
const lines = ledgers.readEdges();
|
||||||
assert.equal(lines.length, 2, "two assertion lines for the same edge");
|
assert.equal(lines.length, 2, "two assertion lines for the same edge");
|
||||||
assert.equal(lines[0]!.from, lines[1]!.from);
|
assert.equal(lines[0]!.from, lines[1]!.from);
|
||||||
assert.equal(lines[0]!.to, lines[1]!.to);
|
assert.equal(lines[0]!.to, lines[1]!.to);
|
||||||
assert.deepEqual(lines.map((l) => l.provenance), [[chat1.id], [chat2.id]]);
|
assert.deepEqual(lines.map((l) => l.provenance), [
|
||||||
|
[{ chatId: chat1.id, messageId: "u1" }],
|
||||||
|
[{ chatId: chat2.id, messageId: "u1" }],
|
||||||
|
]);
|
||||||
|
|
||||||
|
// Read-time merging: one derived edge, both sessions in its provenance.
|
||||||
|
const derived = getGraph(ledgers).edges;
|
||||||
|
assert.equal(derived.length, 1);
|
||||||
|
assert.deepEqual(derived[0]!.provenance, [
|
||||||
|
{ chatId: chat1.id, messageId: "u1" },
|
||||||
|
{ chatId: chat2.id, messageId: "u1" },
|
||||||
|
]);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
+220
-164
@@ -1,9 +1,9 @@
|
|||||||
/**
|
/**
|
||||||
* Graph merger + trigger + derivation tests (vision §1; contracts.ts
|
* Graph merger + trigger + derivation tests (vision §1, 2026-09-02 redesign).
|
||||||
* "extraction pipeline"). The merger (core/graph.ts mergeFindings) is the
|
* The merger (core/graph.ts mergeFindings) is the pipeline's deterministic
|
||||||
* pipeline's deterministic apply stage: findings in, ledger appends out —
|
* apply stage: AUDITED VERDICTS in, ledger appends out — link-or-create
|
||||||
* identity resolution, retroactive probe attribution, event rebuild, and
|
* identity (the auditor's linkTo, never string matching), deterministic quiz
|
||||||
* the idempotency guarantees.
|
* events, understanding observations, and the idempotency guarantees.
|
||||||
*
|
*
|
||||||
* Ledger isolation: PATHS is fixed at import time, so the five data/graph
|
* Ledger isolation: PATHS is fixed at import time, so the five data/graph
|
||||||
* ledgers are backed up and cleared before the suite and restored after
|
* ledgers are backed up and cleared before the suite and restored after
|
||||||
@@ -18,9 +18,8 @@ import assert from "node:assert/strict";
|
|||||||
import * as fs from "node:fs";
|
import * as fs from "node:fs";
|
||||||
import { after, describe, it } from "node:test";
|
import { after, describe, it } from "node:test";
|
||||||
import type { UIMessage } from "ai";
|
import type { UIMessage } from "ai";
|
||||||
import type { PlanTree, PlanToolResult, SessionFindings } from "../src/core/contracts.ts";
|
import type { AuditResult } from "../src/core/contracts.ts";
|
||||||
import type { QuestionAnswerData } from "../src/core/question-core.ts";
|
import type { QuestionAnswerData } from "../src/core/question-core.ts";
|
||||||
import { createTree, markDone } from "../src/core/plan-tree.ts";
|
|
||||||
import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
|
import { getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
|
||||||
import { graphStore } from "../src/store/graph.ts";
|
import { graphStore } from "../src/store/graph.ts";
|
||||||
import { chatStore } from "../src/store/chats.ts";
|
import { chatStore } from "../src/store/chats.ts";
|
||||||
@@ -80,16 +79,6 @@ function quizPart(
|
|||||||
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
|
return { type: "tool-quiz", toolCallId, state: "output-available", input, output };
|
||||||
}
|
}
|
||||||
|
|
||||||
function planPart(toolCallId: string, name: string, input: Record<string, unknown>, tree: PlanTree): unknown {
|
|
||||||
return {
|
|
||||||
type: `tool-${name}`,
|
|
||||||
toolCallId,
|
|
||||||
state: "output-available",
|
|
||||||
input,
|
|
||||||
output: { ok: true, tree } satisfies PlanToolResult,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
|
function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
|
||||||
return {
|
return {
|
||||||
questionId: "q",
|
questionId: "q",
|
||||||
@@ -102,59 +91,17 @@ function answer(over: Partial<QuestionAnswerData>): QuestionAnswerData {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Create a chat in the real store with a tree and messages; returns the record. */
|
/** Create a chat in the real store with messages; returns the record.
|
||||||
function fixtureChat(title: string, tree: PlanTree | null, messages: UIMessage[]) {
|
* Tree-blind world: the plan tree is irrelevant to the merger — no tree. */
|
||||||
|
function fixtureChat(title: string, messages: UIMessage[]) {
|
||||||
const chat = chatStore.createChat(title);
|
const chat = chatStore.createChat(title);
|
||||||
createdChats.push(chat.id);
|
createdChats.push(chat.id);
|
||||||
if (tree) chatStore.saveTree(chat.id, tree);
|
|
||||||
chatStore.saveMessages(chat.id, messages);
|
chatStore.saveMessages(chat.id, messages);
|
||||||
return chatStore.getChat(chat.id)!;
|
return chatStore.getChat(chat.id)!;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The fixture tree: alpha → beta, gamma free-floating, alpha done. */
|
/** The full fixture transcript: five graded quizzes + a dismissed one + an ask. */
|
||||||
function fixtureTree(): PlanTree {
|
function fixtureMessages(): UIMessage[] {
|
||||||
return markDone(
|
|
||||||
createTree(
|
|
||||||
[
|
|
||||||
{ id: "alpha", title: "Alpha intuition" },
|
|
||||||
{ id: "beta", title: "Beta rule" },
|
|
||||||
{ id: "gamma", title: "Gamma application" },
|
|
||||||
],
|
|
||||||
[{ from: "alpha", to: "beta" }],
|
|
||||||
),
|
|
||||||
"alpha",
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Findings shaped exactly like what the extractor would produce for a
|
|
||||||
* session's plan tree: one proposition per tree node (statement = title,
|
|
||||||
* treeNodeId set), one `requires` edge per tree edge.
|
|
||||||
*/
|
|
||||||
function treeFindings(tree: PlanTree): SessionFindings {
|
|
||||||
const idByNodeId = new Map(tree.nodes.map((n, i) => [n.id, `p${i + 1}`]));
|
|
||||||
return {
|
|
||||||
propositions: tree.nodes.map((n, i) => ({
|
|
||||||
id: `p${i + 1}`,
|
|
||||||
name: n.title,
|
|
||||||
statement: n.title,
|
|
||||||
aliases: [],
|
|
||||||
kind: "truth" as const,
|
|
||||||
treeNodeId: n.id,
|
|
||||||
evidence: ["fixture evidence"],
|
|
||||||
})),
|
|
||||||
edges: tree.edges.map((e) => ({
|
|
||||||
from: idByNodeId.get(e.from)!,
|
|
||||||
to: idByNodeId.get(e.to)!,
|
|
||||||
type: "requires" as const,
|
|
||||||
evidence: ["fixture evidence"],
|
|
||||||
})),
|
|
||||||
attribution: { quiz: {} },
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The full fixture transcript: probes → plan → checks → mark_done → noise. */
|
|
||||||
function fixtureMessages(tree: PlanTree): UIMessage[] {
|
|
||||||
return [
|
return [
|
||||||
user("u1", "teach me the fixture topic"),
|
user("u1", "teach me the fixture topic"),
|
||||||
assistant("a1", [
|
assistant("a1", [
|
||||||
@@ -171,32 +118,30 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
|
|||||||
answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }),
|
answer({ questionId: "call-2", questionText: "Q2 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T01:01:00.000Z" }),
|
||||||
),
|
),
|
||||||
]),
|
]),
|
||||||
assistant("a3", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
|
assistant("a3", [
|
||||||
assistant("a4", [
|
// input.nodeId "alpha" is deliberately present and IGNORED —
|
||||||
|
// attribution is the only quiz→node path in the tree-blind world.
|
||||||
quizPart(
|
quizPart(
|
||||||
"call-3",
|
"call-3",
|
||||||
{ question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
{ question: "Q3", nodeId: "alpha", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||||
answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }),
|
answer({ questionId: "call-3", questionText: "Q3 text", correct: true, note: "recalled the diagram", answeredAt: "2026-08-22T02:00:00.000Z" }),
|
||||||
),
|
),
|
||||||
]),
|
]),
|
||||||
assistant("a5", [
|
assistant("a4", [
|
||||||
quizPart(
|
quizPart(
|
||||||
"call-4",
|
"call-4",
|
||||||
{ question: "Q4", nodeId: "beta", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
{ question: "Q4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||||
answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }),
|
answer({ questionId: "call-4", questionText: "Q4 text", correct: false, answeredAt: "2026-08-22T02:01:00.000Z" }),
|
||||||
),
|
),
|
||||||
]),
|
]),
|
||||||
assistant("a6", [
|
assistant("a5", [
|
||||||
// nodeId present but NOT in the tree → tagged-but-unmapped: the
|
|
||||||
// attribution can rescue it, or it lands null + question text.
|
|
||||||
quizPart(
|
quizPart(
|
||||||
"call-5",
|
"call-5",
|
||||||
{ question: "Q5", nodeId: "ghost", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
{ question: "Q5", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||||
answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }),
|
answer({ questionId: "call-5", questionText: "Q5 text", dontKnow: true, correct: false, answeredAt: "2026-08-22T02:02:00.000Z" }),
|
||||||
),
|
),
|
||||||
]),
|
]),
|
||||||
assistant("a7", [planPart("p2", "plan_mark_done", { id: "alpha" }, tree)]),
|
assistant("a6", [
|
||||||
assistant("a8", [
|
|
||||||
// dismissed quiz + an answered ask — neither becomes an event
|
// dismissed quiz + an answered ask — neither becomes an event
|
||||||
quizPart(
|
quizPart(
|
||||||
"call-6",
|
"call-6",
|
||||||
@@ -214,25 +159,58 @@ function fixtureMessages(tree: PlanTree): UIMessage[] {
|
|||||||
];
|
];
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/** The auditor's verdict for the fixture chat: three verified propositions
|
||||||
* The extractor's findings for the fixture chat: the tree as propositions +
|
* (no linkTo — the graph starts empty), one follows edge, all quizzes
|
||||||
* the probe attribution. The audit stage is a no-op pass-through in these
|
* attributed, one understanding note. */
|
||||||
* merger tests — its behavior is tested with the pipeline (extraction.test).
|
function fixtureAudit(): AuditResult {
|
||||||
*/
|
return {
|
||||||
function fixtureFindings(): SessionFindings {
|
propositions: [
|
||||||
const findings = treeFindings(fixtureTree());
|
{
|
||||||
// Untagged probe quizzes: call-1 attributed retroactively to Alpha (the
|
id: "p1",
|
||||||
// vision's open question — resolved by the extractor); call-2 genuinely
|
statement: "Alpha intuition",
|
||||||
// unattributable; call-5's ghost nodeId is rescued onto Gamma.
|
description: "Alpha intuition, elaborated: what alpha means and how it behaves in practice.",
|
||||||
findings.attribution.quiz = { "call-1": "p1", "call-2": null, "call-5": "p3" };
|
aliases: ["alpha feel"],
|
||||||
return findings;
|
messages: ["a1", "a3"],
|
||||||
|
evidence: ["fixture evidence"],
|
||||||
|
understanding: "solid after the check",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "p2",
|
||||||
|
statement: "Beta rule",
|
||||||
|
description: "The beta rule and when it applies.",
|
||||||
|
aliases: [],
|
||||||
|
messages: ["a4"],
|
||||||
|
evidence: ["fixture evidence"],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "p3",
|
||||||
|
statement: "Gamma application",
|
||||||
|
description: "Applying the whole framework to gamma cases.",
|
||||||
|
aliases: [],
|
||||||
|
messages: ["a5"],
|
||||||
|
evidence: ["fixture evidence"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
edges: [
|
||||||
|
{
|
||||||
|
from: "p1",
|
||||||
|
to: "p2",
|
||||||
|
type: "follows",
|
||||||
|
messages: ["a4"],
|
||||||
|
evidence: ["fixture evidence"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
attribution: { quiz: { "call-1": "p1", "call-2": null, "call-3": "p1", "call-4": "p2", "call-5": "p3" } },
|
||||||
|
dropped: [],
|
||||||
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The chat every merger test folds (shared, so ordering is intentional). */
|
/** The chat every merger test folds (shared, so ordering is intentional). */
|
||||||
const chat1 = fixtureChat("Graph fixture", fixtureTree(), fixtureMessages(fixtureTree()));
|
const chat1 = fixtureChat("Graph fixture", fixtureMessages());
|
||||||
|
|
||||||
/** Set by the cross-session test; the snapshot test (which runs after it) reads it. */
|
/** Set by the cross-session tests; the snapshot test (which runs after) reads them. */
|
||||||
let chat2Id = "";
|
let chat2Id = "";
|
||||||
|
let chat3Id = "";
|
||||||
|
|
||||||
// ─── the merger ──────────────────────────────────────────────────────────────
|
// ─── the merger ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
@@ -248,71 +226,83 @@ function scopedGraph(chatIds: string[]) {
|
|||||||
);
|
);
|
||||||
return {
|
return {
|
||||||
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
|
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)),
|
||||||
edges: graph.edges.filter((e) => e.provenance.some((c) => chatIds.includes(c))),
|
edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
|
||||||
events: graph.events.filter((e) => chatIds.includes(e.chatId)),
|
events: graph.events.filter((e) => chatIds.includes(e.chatId)),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
describe("the merger (mergeFindings)", () => {
|
describe("the merger (mergeFindings)", () => {
|
||||||
it("merges propositions/edges by identity and rebuilds events with retroactive probe attribution", () => {
|
it("creates nodes from the verdicts, rebuilds quiz events deterministically, records observations", () => {
|
||||||
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
|
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
|
||||||
assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 });
|
assert.deepEqual(stats, { nodes: 3, edges: 1, events: 6 });
|
||||||
|
|
||||||
const graph = scopedGraph([chat1.id]);
|
const graph = scopedGraph([chat1.id]);
|
||||||
assert.equal(graph.nodes.length, 3);
|
assert.equal(graph.nodes.length, 3);
|
||||||
const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id]));
|
const idByStatement = new Map(graph.nodes.map((n) => [n.statement, n.id]));
|
||||||
for (const node of graph.nodes) {
|
for (const node of graph.nodes) {
|
||||||
assert.equal(node.kind, "truth");
|
|
||||||
assert.deepEqual(node.aliases, []);
|
|
||||||
assert.match(node.id, /^n_/);
|
assert.match(node.id, /^n_/);
|
||||||
|
assert.ok(node.description.length > 0, "description is required");
|
||||||
|
assert.equal("kind" in node, false, "no kind field");
|
||||||
|
assert.ok(Array.isArray(node.provenance), "message-granular provenance");
|
||||||
}
|
}
|
||||||
|
// Provenance is message-granular: p1 came up in a1 and a3.
|
||||||
|
const alpha = graph.nodes.find((n) => n.statement === "Alpha intuition")!;
|
||||||
|
assert.deepEqual(alpha.provenance, [
|
||||||
|
{ chatId: chat1.id, messageId: "a1" },
|
||||||
|
{ chatId: chat1.id, messageId: "a3" },
|
||||||
|
]);
|
||||||
|
|
||||||
// The tree edge alpha→beta becomes a `requires` edge on the mapped nodes.
|
// The follows edge, provenance attached.
|
||||||
assert.equal(graph.edges.length, 1);
|
assert.equal(graph.edges.length, 1);
|
||||||
const edge = graph.edges[0]!;
|
const edge = graph.edges[0]!;
|
||||||
assert.equal(edge.from, idByStatement.get("Alpha intuition"));
|
assert.equal(edge.from, idByStatement.get("Alpha intuition"));
|
||||||
assert.equal(edge.to, idByStatement.get("Beta rule"));
|
assert.equal(edge.to, idByStatement.get("Beta rule"));
|
||||||
assert.equal(edge.type, "requires");
|
assert.equal(edge.type, "follows");
|
||||||
assert.deepEqual(edge.provenance, [chat1.id]);
|
assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "a4" }]);
|
||||||
|
|
||||||
|
// Mapping lines: one per (chat, sourceId).
|
||||||
|
assert.deepEqual(
|
||||||
|
graphStore.readMappings().map((m) => m.sourceId),
|
||||||
|
["p1", "p2", "p3"],
|
||||||
|
);
|
||||||
|
|
||||||
// Events — kinds, refs, notes, and the attribution paths.
|
// Events — kinds, refs, notes, and the attribution paths.
|
||||||
assert.equal(graph.events.length, 6);
|
assert.equal(graph.events.length, 6);
|
||||||
const byRef = new Map(graph.events.map((e) => [e.ref, e]));
|
const byRef = new Map(graph.events.map((e) => [e.ref, e]));
|
||||||
|
|
||||||
const probeMiss = byRef.get("call-1")!;
|
const miss = byRef.get("call-1")!;
|
||||||
assert.equal(probeMiss.kind, "probe-miss"); // quiz BEFORE plan_init
|
assert.equal(miss.kind, "miss");
|
||||||
assert.equal(probeMiss.nodeId, idByStatement.get("Alpha intuition"), "retroactive probe attribution");
|
assert.equal(miss.nodeId, idByStatement.get("Alpha intuition"), "attributed quiz");
|
||||||
assert.equal(probeMiss.question, undefined, "attributed → no question text needed");
|
|
||||||
|
|
||||||
const probeDontKnow = byRef.get("call-2")!;
|
const dontKnow = byRef.get("call-2")!;
|
||||||
assert.equal(probeDontKnow.kind, "dont-know"); // same kind, both phases
|
assert.equal(dontKnow.kind, "dont-know");
|
||||||
assert.equal(probeDontKnow.nodeId, null, "unattributable probe → null");
|
assert.equal(dontKnow.nodeId, null, "unattributable → null");
|
||||||
assert.equal(probeDontKnow.question, "Q2 text");
|
assert.equal(dontKnow.question, "Q2 text");
|
||||||
|
|
||||||
const check = byRef.get("call-3")!;
|
const correct = byRef.get("call-3")!;
|
||||||
assert.equal(check.kind, "check-correct"); // quiz AFTER plan_init
|
assert.equal(correct.kind, "correct");
|
||||||
assert.equal(check.nodeId, idByStatement.get("Alpha intuition"), "tagged quiz → deterministic");
|
assert.equal(correct.nodeId, idByStatement.get("Alpha intuition"), "attribution wins over the ignored input.nodeId");
|
||||||
assert.equal(check.note, "recalled the diagram");
|
assert.equal(correct.note, "recalled the diagram");
|
||||||
|
|
||||||
const checkMiss = byRef.get("call-4")!;
|
const miss2 = byRef.get("call-4")!;
|
||||||
assert.equal(checkMiss.kind, "check-incorrect");
|
assert.equal(miss2.kind, "miss");
|
||||||
assert.equal(checkMiss.nodeId, idByStatement.get("Beta rule"));
|
assert.equal(miss2.nodeId, idByStatement.get("Beta rule"));
|
||||||
|
|
||||||
const ghost = byRef.get("call-5")!;
|
const dontKnow2 = byRef.get("call-5")!;
|
||||||
assert.equal(ghost.kind, "dont-know");
|
assert.equal(dontKnow2.kind, "dont-know");
|
||||||
assert.equal(ghost.nodeId, idByStatement.get("Gamma application"), "tagged-but-unmapped rescued by attribution");
|
assert.equal(dontKnow2.nodeId, idByStatement.get("Gamma application"));
|
||||||
assert.equal(ghost.question, undefined);
|
|
||||||
|
|
||||||
const taught = byRef.get("taught:alpha")!;
|
const observation = byRef.get("obs:p1")!;
|
||||||
assert.equal(taught.kind, "taught");
|
assert.equal(observation.kind, "observation");
|
||||||
assert.equal(taught.nodeId, idByStatement.get("Alpha intuition"), "mark_done → tree-node mapping");
|
assert.equal(observation.nodeId, idByStatement.get("Alpha intuition"));
|
||||||
|
assert.equal(observation.note, "solid after the check");
|
||||||
|
|
||||||
// The dismissed quiz and the ask question became nothing.
|
// The dismissed quiz and the ask question became nothing.
|
||||||
assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0);
|
assert.equal([...byRef.keys()].filter((r) => r === "call-6" || r === "call-7").length, 0);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("is idempotent — re-merging the same findings appends nothing", () => {
|
it("is idempotent — re-merging the same verdicts appends nothing", () => {
|
||||||
const stats = mergeFindings(chat1.id, fixtureFindings(), graphDeps);
|
const stats = mergeFindings(chat1.id, fixtureAudit(), graphDeps);
|
||||||
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 });
|
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 0 });
|
||||||
const graph = scopedGraph([chat1.id]);
|
const graph = scopedGraph([chat1.id]);
|
||||||
assert.equal(graph.nodes.length, 3);
|
assert.equal(graph.nodes.length, 3);
|
||||||
@@ -322,38 +312,63 @@ describe("the merger (mergeFindings)", () => {
|
|||||||
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
|
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("merges a second session onto the same nodes and accumulates edge provenance", () => {
|
it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
|
||||||
// chat2 re-teaches the same two propositions (identical statements) —
|
const alphaId = scopedGraph([chat1.id]).nodes.find((n) => n.statement === "Alpha intuition")!.id;
|
||||||
// the conservative identity resolution recognizes them.
|
const chat2 = fixtureChat("Graph fixture 2", [
|
||||||
const tree2 = createTree(
|
|
||||||
[
|
|
||||||
{ id: "a", title: "Alpha intuition" },
|
|
||||||
{ id: "b", title: "Beta rule" },
|
|
||||||
],
|
|
||||||
[{ from: "a", to: "b" }],
|
|
||||||
);
|
|
||||||
const chat2 = fixtureChat("Graph fixture 2", tree2, [
|
|
||||||
user("u1", "again"),
|
user("u1", "again"),
|
||||||
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree2)]),
|
assistant("a1", [
|
||||||
|
quizPart(
|
||||||
|
"call-9",
|
||||||
|
{ question: "Q9", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||||
|
answer({ questionId: "call-9", questionText: "Q9 text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
|
||||||
|
),
|
||||||
|
]),
|
||||||
]);
|
]);
|
||||||
|
|
||||||
const stats = mergeFindings(chat2.id, treeFindings(tree2), graphDeps);
|
|
||||||
assert.deepEqual(stats, { nodes: 0, edges: 1, events: 0 });
|
|
||||||
chat2Id = chat2.id;
|
chat2Id = chat2.id;
|
||||||
|
|
||||||
|
// The auditor's identity verdict: this proposition IS the alpha node.
|
||||||
|
const stats = mergeFindings(
|
||||||
|
chat2.id,
|
||||||
|
{
|
||||||
|
propositions: [
|
||||||
|
{
|
||||||
|
id: "q1",
|
||||||
|
statement: "Alpha feel", // a DIFFERENT phrasing — identity is the verdict, not strings
|
||||||
|
description: "The intuitive grasp of alpha.",
|
||||||
|
aliases: ["alpha intuition"],
|
||||||
|
messages: ["a1"],
|
||||||
|
evidence: ["fixture evidence"],
|
||||||
|
linkTo: alphaId,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
attribution: { quiz: { "call-9": "q1" } },
|
||||||
|
dropped: [],
|
||||||
|
},
|
||||||
|
graphDeps,
|
||||||
|
);
|
||||||
|
assert.deepEqual(stats, { nodes: 0, edges: 0, events: 1 });
|
||||||
|
|
||||||
const after = scopedGraph([chat1.id, chat2.id]);
|
const after = scopedGraph([chat1.id, chat2.id]);
|
||||||
assert.equal(after.nodes.length, 3); // no duplicate nodes
|
assert.equal(after.nodes.length, 3, "no duplicate node");
|
||||||
assert.equal(after.edges.length, 1); // one merged edge…
|
const alpha = getGraph(graphStore).nodes.find((n) => n.id === alphaId)!;
|
||||||
assert.deepEqual(after.edges[0]!.provenance, [chat1.id, chat2.id]); // …two sessions
|
assert.ok(alpha.aliases.includes("Alpha feel"), "the session's phrasing joins the aliases");
|
||||||
|
assert.ok(alpha.aliases.includes("alpha intuition"));
|
||||||
|
assert.ok(
|
||||||
|
alpha.provenance.some((pt) => pt.chatId === chat2.id && pt.messageId === "a1"),
|
||||||
|
"provenance unions",
|
||||||
|
);
|
||||||
|
// The mapping records the verdict: (chat2, q1) → the linked node.
|
||||||
|
const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
|
||||||
|
assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
|
||||||
});
|
});
|
||||||
|
|
||||||
it("derives node states, sessions, and node provenance in graphSnapshot", () => {
|
it("derives node states and sessions in graphSnapshot", () => {
|
||||||
const snap = graphSnapshot(graphDeps);
|
const snap = graphSnapshot(graphDeps);
|
||||||
const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id]));
|
const idByStatement = new Map(snap.nodes.map((n) => [n.statement, n.id]));
|
||||||
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "taught");
|
// Alpha's latest quiz is call-9 (chat2, correct) → known.
|
||||||
|
assert.equal(snap.nodeStates[idByStatement.get("Alpha intuition")!], "known");
|
||||||
assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed");
|
assert.equal(snap.nodeStates[idByStatement.get("Beta rule")!], "missed");
|
||||||
// Gamma: no taught/checks — but the rescued dont-know probe (call-5)
|
|
||||||
// makes its latest event a dont-know.
|
|
||||||
assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know");
|
assert.equal(snap.nodeStates[idByStatement.get("Gamma application")!], "dont-know");
|
||||||
|
|
||||||
assert.equal(snap.sessions[chat1.id], "Graph fixture");
|
assert.equal(snap.sessions[chat1.id], "Graph fixture");
|
||||||
@@ -366,8 +381,35 @@ describe("the merger (mergeFindings)", () => {
|
|||||||
assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]);
|
assert.deepEqual(snap.nodeSessions[idByStatement.get("Gamma application")!], [chat1.id]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("never string-matches without a verdict — an identical statement without linkTo is a NEW node", () => {
|
||||||
|
const chat3 = fixtureChat("Graph fixture 3", [user("u1", "third")]);
|
||||||
|
chat3Id = chat3.id;
|
||||||
|
const stats = mergeFindings(
|
||||||
|
chat3.id,
|
||||||
|
{
|
||||||
|
propositions: [
|
||||||
|
{
|
||||||
|
id: "r1",
|
||||||
|
statement: "Alpha intuition", // IDENTICAL to the existing node's statement
|
||||||
|
description: "Same words, no verdict.",
|
||||||
|
aliases: [],
|
||||||
|
messages: ["u1"],
|
||||||
|
evidence: ["fixture evidence"],
|
||||||
|
// no linkTo — identity is the auditor's call alone
|
||||||
|
},
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
attribution: { quiz: {} },
|
||||||
|
dropped: [],
|
||||||
|
},
|
||||||
|
graphDeps,
|
||||||
|
);
|
||||||
|
assert.equal(stats.nodes, 1, "created — the merger does not match strings");
|
||||||
|
assert.equal(getGraph(graphStore).nodes.filter((n) => n.statement === "Alpha intuition").length, 2);
|
||||||
|
});
|
||||||
|
|
||||||
it("returns zeros for unknown chats", () => {
|
it("returns zeros for unknown chats", () => {
|
||||||
assert.deepEqual(mergeFindings("chat_does_not_exist", treeFindings(fixtureTree()), graphDeps), {
|
assert.deepEqual(mergeFindings("chat_does_not_exist", fixtureAudit(), graphDeps), {
|
||||||
nodes: 0,
|
nodes: 0,
|
||||||
edges: 0,
|
edges: 0,
|
||||||
events: 0,
|
events: 0,
|
||||||
@@ -379,14 +421,12 @@ describe("the merger (mergeFindings)", () => {
|
|||||||
|
|
||||||
describe("the trigger helper (pendingQuizAnswers)", () => {
|
describe("the trigger helper (pendingQuizAnswers)", () => {
|
||||||
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
|
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
|
||||||
const tree = createTree([{ id: "solo", title: "Solo node" }], []);
|
const chat = fixtureChat("Trigger fixture", [
|
||||||
const chat = fixtureChat("Trigger fixture", tree, [
|
|
||||||
user("u1", "go"),
|
user("u1", "go"),
|
||||||
assistant("a1", [planPart("p1", "plan_init", { nodes: [], edges: [] }, tree)]),
|
assistant("a1", [
|
||||||
assistant("a2", [
|
|
||||||
quizPart(
|
quizPart(
|
||||||
"call-s",
|
"call-s",
|
||||||
{ question: "QS", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
{ question: "QS", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||||
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
|
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
|
||||||
),
|
),
|
||||||
quizPart(
|
quizPart(
|
||||||
@@ -407,7 +447,9 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
|
|||||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
|
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
|
||||||
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
|
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
|
||||||
|
|
||||||
mergeFindings(chat.id, treeFindings(tree), graphDeps);
|
mergeFindings(chat.id, fixtureAudit(), graphDeps);
|
||||||
|
// call-s is unattributed here (fixtureAudit has no call-s attribution)
|
||||||
|
// — but it still becomes an event, so nothing stays pending.
|
||||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
|
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
|
||||||
|
|
||||||
// A NEW answered quiz raises the count again (the re-trigger path).
|
// A NEW answered quiz raises the count again (the re-trigger path).
|
||||||
@@ -416,7 +458,7 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
|
|||||||
assistant("a3", [
|
assistant("a3", [
|
||||||
quizPart(
|
quizPart(
|
||||||
"call-s4",
|
"call-s4",
|
||||||
{ question: "QS4", nodeId: "solo", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
{ question: "QS4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||||
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
|
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
|
||||||
),
|
),
|
||||||
]),
|
]),
|
||||||
@@ -429,29 +471,43 @@ describe("the trigger helper (pendingQuizAnswers)", () => {
|
|||||||
|
|
||||||
describe("graph ledger store", () => {
|
describe("graph ledger store", () => {
|
||||||
it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => {
|
it("round-trips lines and skips corrupt / foreign-shaped lines (strict reads)", () => {
|
||||||
const node = { id: "n_storetest", name: "Store test", statement: "Store test", aliases: [], kind: "truth" as const, createdAt: "2026-08-22T00:00:00.000Z" };
|
const node = {
|
||||||
|
id: "n_storetest",
|
||||||
|
statement: "Store test",
|
||||||
|
description: "Store test, elaborated.",
|
||||||
|
aliases: [] as string[],
|
||||||
|
createdAt: "2026-08-22T00:00:00.000Z",
|
||||||
|
provenance: [{ chatId: "chat_c", messageId: "m1" }],
|
||||||
|
};
|
||||||
graphStore.appendNode(node);
|
graphStore.appendNode(node);
|
||||||
// A torn append and a legacy V0-shaped node (different schema) — both skipped.
|
// A torn append and a legacy pre-redesign node (old schema) — both skipped.
|
||||||
fs.appendFileSync(
|
fs.appendFileSync(
|
||||||
LEDGER_FILES.nodes,
|
LEDGER_FILES.nodes,
|
||||||
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", domain: "x", state: "known" })}\n`,
|
`{not json}\n${JSON.stringify({ id: "legacy", statement: "Legacy shape", kind: "truth", name: "Legacy", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
|
||||||
"utf-8",
|
"utf-8",
|
||||||
);
|
);
|
||||||
const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy");
|
const nodes = graphStore.readNodes().filter((n) => n.id === "n_storetest" || n.id === "legacy");
|
||||||
assert.equal(nodes.length, 1);
|
assert.equal(nodes.length, 1);
|
||||||
assert.equal(nodes[0]!.statement, "Store test");
|
assert.equal(nodes[0]!.statement, "Store test");
|
||||||
assert.equal(nodes[0]!.name, "Store test");
|
assert.deepEqual(nodes[0]!.provenance, [{ chatId: "chat_c", messageId: "m1" }]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("derives a name for pre-name ledger lines (legacy reads)", () => {
|
it("returns RAW node lines — repeated ids (link-appends) are each their own line; getGraph merges", () => {
|
||||||
// A full, valid node line written before `name` existed.
|
const line = (msgId: string) =>
|
||||||
fs.appendFileSync(
|
JSON.stringify({
|
||||||
LEDGER_FILES.nodes,
|
id: "n_linktest",
|
||||||
`${JSON.stringify({ id: "n_prename", statement: "Derivatives measure instantaneous change; they are limits.", aliases: [], kind: "truth", createdAt: "2026-08-22T00:00:00.000Z" })}\n`,
|
statement: `Phrasing ${msgId}`,
|
||||||
"utf-8",
|
description: "Elaboration.",
|
||||||
);
|
aliases: [],
|
||||||
const node = graphStore.readNodes().find((n) => n.id === "n_prename");
|
createdAt: "2026-08-22T00:00:00.000Z",
|
||||||
assert.ok(node);
|
provenance: [{ chatId: "chat_c", messageId: msgId }],
|
||||||
assert.equal(node.name, "Derivatives measure instantaneous change");
|
});
|
||||||
|
fs.appendFileSync(LEDGER_FILES.nodes, `${line("m1")}\n${line("m2")}\n`, "utf-8");
|
||||||
|
const lines = graphStore.readNodes().filter((n) => n.id === "n_linktest");
|
||||||
|
assert.equal(lines.length, 2, "the store returns raw lines, one per append");
|
||||||
|
const merged = getGraph(graphStore).nodes.find((n) => n.id === "n_linktest")!;
|
||||||
|
assert.equal(merged.statement, "Phrasing m1", "the FIRST line defines the node");
|
||||||
|
assert.ok(merged.aliases.includes("Phrasing m2"), "later phrasings join the aliases");
|
||||||
|
assert.deepEqual(merged.provenance.map((p) => p.messageId), ["m1", "m2"]);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user