sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Graph redesign: true propositions, follows/supports, auditor-owned identity (vision §1 minimal implementation)

Browse Source 
The minimal implementation of the 2026-09-02 graph vision (docs/vision.md §1):

- contracts/graph.ts: every node is a TRUE proposition (statement is both
  rendering label and identity key; required description; no kind, no name).
  Edges are follows (derivable) / supports (grounds without entailing) with
  message-granular provenance points {chatId, messageId}. Events reduce to
  correct/miss/dont-know plus observation (understanding notes); no taught
  events, no misconception kind.
- extraction.ts: extractor is TREE-BLIND (transcript only, no plan tree) and
  free-form; auditor sees findings + the current graph, owns faithfulness and
  identity (link-or-create verdict per proposition, recorded as linkTo);
  transcriptView tags every line with its message id (provenance anchors).
- core/graph.ts merger: applies verdicts deterministically — linkTo appends
  a link line (phrasings join aliases, provenance unions; reads merge by id),
  absent creates; NO string matching anywhere. Quiz events rebuilt from tool
  parts with attribution-only nodeId resolution; understanding notes become
  observation events (obs:<sourceId>); mapping lines are the verdict audit
  trail and make re-merges idempotent.
- deriveNodeStates: transitional — latest quiz event wins (known/missed/
  dont-know, else candidate); taught unreachable until the state-model
  ideation lands.
- graph-context.ts + 50-graph-context.md: follows chains, hotspot grouping,
  state vocabulary updated for the new event kinds.
- graph-view.tsx: node label is the statement; detail shows statement +
  description.
- tests: merger verdicts (link, no-string-match, cross-chat assertions),
  tree-blind pipeline end-to-end, follows vocabulary in context tests.
  276/276 green.
This commit is contained in:
amosblomqvist
2026-09-08 22:19:37 +03:00
parent e322d775e4
commit b07c907ffd
13 changed files with 945 additions and 813 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/prompts/50-graph-context.md
+4 -5
View File
@@ -1,20 +1,19 @@
# The learner's knowledge graph
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes and plan trees, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Requires** means *a* must be understood before *b*.
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition in its current state (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
The states, exactly:
- **[missed]** — answered wrong. Treat as a live misconception site until a check proves otherwise.
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
- **[known]** — answered correctly. Foundation you can build on and refer to by name.
- **[taught]** — a lesson covered it. Check that it stuck before stacking anything heavy on it.
- **[candidate]** — asserted into a plan but never probed. Assume nothing.
- **[candidate]** — present in the graph but never probed. Assume nothing.
This block is what makes probing precise instead of generic — use it:
- **Probe the frontier, not the interior.** [known] and [taught] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
- **Probe the frontier, not the interior.** [known] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [missed] and [dont-know] nodes.
- **[missed] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
- **Read the Requires chains as scaffolding.** They show what the learner's knowledge rests on. If a prerequisite under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
- **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [missed] or [dont-know], that is where teaching starts — not at the topic itself.
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
src/agent/subagents/extraction.ts
+156 -118
View File
@@ -1,23 +1,28 @@
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type UIMessage } from "ai";
import { z } from "zod";
import { modelFor } from "../models.ts";
import type { AuditResult, ChatRecord, SessionFindings } from "../../core/contracts.ts";
import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
import type { AuditResult, ChatRecord, GraphNode, SessionFindings } from "../../core/contracts.ts";
import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../../core/graph.ts";
/**
* The extraction pipeline (vision §1 construction) — the sole graph writer.
* Three stages, one orchestrator:
* The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
* sole graph writer. Three stages, one orchestrator:
*
* extractor (LLM) transcript → SessionFindings (propositions, edges,
* event attribution — each with verbatim evidence)
* auditor (LLM) findings → AuditResult (the verified subset + drops)
* extractor (LLM) transcript → SessionFindings (free-form propositions,
* follows/supports edges, quiz attribution, understanding
* notes — each with verbatim evidence). TREE-BLIND: it
* sees only the transcript, never the plan tree.
* auditor (LLM) findings + THE CURRENT GRAPH → AuditResult: the
* verified subset with an identity verdict per
* proposition (linkTo an existing node, or create), plus
* the drops for the log.
* merger (core) mergeFindings() → deterministic ledger appends
*
* The LLM stages supply only the judgment calls; the deterministic rebuild
* in the merger owns everything that must never be mis-copied (event kinds,
* timestamps, refs). Both agents are single-tool-call workers: their whole
* deliverable is one zod-validated submit call — a ToolLoopAgent purely so
* the model can retry a schema-invalid call.
* timestamps, refs, provenance points). Both agents are single-tool-call
* workers: their whole deliverable is one zod-validated submit call — a
* ToolLoopAgent purely so the model can retry a schema-invalid call.
*
* Created via createExtractionPipeline(deps) — the composition root
* (chat-handler) injects the storage ports + models; this module never
@@ -27,7 +32,7 @@ import { mergeFindings, type EmissionStats, type GraphDeps } from "../../core/gr
* the next trigger re-runs the whole thing.
*/
// ─── the transcript view (what the LLM sees) ────────────────────────────────
// ─── the transcript view (what the extractor + auditor see) ─────────────────
/**
* A quiz tool part's outcome, one line: the attribution evidence.
@@ -44,46 +49,35 @@ function quizLine(part: { toolCallId: string; input: unknown; output: unknown })
}
/**
* The compact transcript the extractor + auditor both see: user/teacher
* text verbatim (evidence quotes must match it), the final plan tree, and
* the structured tool events (quiz outcomes, mark_done). Job tool parts
* (research/demo/visual) are noise for extraction — their outcomes are
* summarized as one line each.
* The compact transcript both LLM stages see: user/teacher text VERBATIM,
* each tagged with its message id (provenance anchors — the extractor
* reports them back per proposition/edge), and quiz outcome lines.
* TREE-BLIND by design (2026-09-02): no plan tree, no mark_done —
* extraction is free-form, decoupled from the session's plan. Job tool
* parts (research/demo/visual) are noise for extraction — their outcomes
* are summarized as one line each.
*/
export function transcriptView(record: ChatRecord): string {
const lines: string[] = [];
lines.push(`SESSION: ${record.title}`);
if (record.tree && record.tree.nodes.length > 0) {
const nodes = record.tree.nodes.map((n) => `${n.id} = ${n.title}`).join("; ");
lines.push(`FINAL PLAN TREE (node ids): ${nodes}`);
if (record.tree.edges.length > 0) {
const edges = record.tree.edges.map((e) => `${e.from} requires ${e.to}`).join("; ");
lines.push(`TREE EDGES: ${edges}`);
}
}
for (const message of record.messages as UIMessage[]) {
record.messages.forEach((message, index) => {
const who = message.role === "user" ? "LEARNER" : "TEACHER";
for (const part of message.parts) {
const mid = (message as UIMessage).id || `m${index}`;
for (const part of (message as UIMessage).parts) {
if (part.type === "text" && part.text.trim()) {
lines.push(`${who}: ${part.text.trim()}`);
lines.push(`${who} [${mid}] ${part.text.trim()}`);
} else if (part.type.startsWith("tool-")) {
const name = part.type.slice("tool-".length);
const p = part as unknown as { toolCallId: string; state: string; input: unknown; output: unknown };
if (name === "quiz" && p.state === "output-available") {
lines.push(quizLine(p));
} else if (name === "plan_mark_done" && p.state === "output-available") {
const input = p.input as { id?: unknown } | undefined;
lines.push(`TEACHER marked tree node done [${p.toolCallId}]: ${typeof input?.id === "string" ? input.id : "?"}`);
} else if (name.startsWith("plan_")) {
// plan ops are folded into the FINAL PLAN TREE line above
} else if (p.state === "output-available") {
lines.push(`(${name} tool result delivered)`);
}
}
}
}
});
return lines.join("\n");
}
@@ -94,87 +88,98 @@ const evidence = z
.min(1)
.describe("Verbatim quotes from the transcript that back this. The auditor will check every quote.");
const messagesField = z
.array(z.string())
.describe("The message ids (from the [id] tags in the transcript) where this came up — provenance.");
const propositionSchema = z.object({
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
statement: z
.string()
.describe(
"The proposition as one reasoning step — a TRUE, self-contained, world-facing statement. " +
"Never reference the learner, the teacher, or the session: not \"the learner understands X\" but " +
'"X". A misconception is stated as the TRUE correction ("Nietzsche did not endorse ' +
'might-makes-right social Darwinism") — the wrongness lives in the quiz outcomes, never here.',
),
description: z
.string()
.describe(
"The elaborated version of the same proposition — more words: scope, precision, what exactly it means. Still true, still world-facing.",
),
aliases: z.array(z.string()).describe("Verbatim phrasings this session used for the same proposition, if any."),
messages: messagesField,
evidence,
understanding: z
.string()
.optional()
.describe(
'One note about THIS session\'s evidence of the learner\'s grasp of this proposition, on a named-word scale ' +
'("solid", "shaky", "accepted", "resisted", "not yet engaged") — no numbers, no predictions.',
),
});
const edgeSchema = z.object({
from: z.string().describe("Proposition id that is followed / supported."),
to: z.string().describe("Proposition id that follows / is supported by `from`."),
type: z
.enum(["follows", "supports"])
.describe('"follows" = to is derivable from from; "supports" = from grounds to without entailing it.'),
messages: messagesField,
evidence,
});
const findingsSchema = z.object({
propositions: z
.array(
z.object({
id: z.string().describe("Local id: \"p1\", \"p2\", … — referenced by edges and attribution."),
name: z.string().describe(
"A short 2-6 word name for the fact — its header, used for rendering. Not a sentence.",
),
statement: z
.string()
.describe(
"The proposition as one reasoning step — an OBJECTIVE, world-facing fact. " +
"Never reference the learner, the teacher, or the session: not \"the learner " +
'understands X\" but \"X\"; not \"the learner thinks Y\" but the wrong idea itself, ' +
'stated as a claim (\"Reading: Nietzsche endorsed might-makes-right social Darwinism\"), ' +
"with kind \"misconception\" carrying that it is wrong.",
),
aliases: z.array(z.string()).describe("Verbatim alternate phrasings from this session, if any."),
kind: z.enum(["truth", "misconception"]).describe("misconception = a wrong idea the learner holds or was warned about."),
treeNodeId: z.string().optional().describe("The plan-tree node id this came from, when it did."),
evidence,
}),
)
.describe("Every distinct proposition the session touched — from the plan tree AND from prose."),
.array(propositionSchema)
.describe("Every distinct proposition the session touched — true statements, misconception-free."),
edges: z
.array(
z.object({
from: z.string().describe("Proposition id that is the prerequisite / support."),
to: z.string().describe("Proposition id that depends on / is supported by `from`."),
type: z.enum(["requires", "supports"]),
evidence,
}),
)
.describe("Relations between propositions. `requires` = prerequisite; `supports` = helpful but not prerequisite. Include relations the plan tree does NOT show when the transcript articulates them."),
.array(edgeSchema)
.describe("Relations between propositions: follows (derivable) / supports (grounds without entailing)."),
attribution: z
.object({
quiz: z
.record(z.string(), z.string().nullable())
.describe("toolCallId of every UNTAGGED quiz line → the proposition id it was really testing. Probe questions asked BEFORE the tree existed are attributed RETROACTIVELY — the whole transcript is visible; map them to the proposition they were probing (often articulated only later). null when genuinely unattributable. Tagged quizzes (with a tree nodeId in the transcript view) resolve automatically — do not include them."),
.describe(
"toolCallId of EVERY quiz line → the proposition id it was really testing. Attribute RETROACTIVELY — the whole transcript is visible; map each quiz to the proposition it tests (often articulated elsewhere in the session). null when genuinely unattributable.",
),
})
.describe("Probe attribution — your only event judgment."),
.describe("Quiz attribution — your only event judgment."),
});
const EXTRACTOR_INSTRUCTIONS = `You are the extraction specialist in a learning application. You read ONE session
transcript and report what was learned — as structured findings for the learner's global knowledge graph.
The transcript view shows: the session's final plan tree (dependency map, with node ids), the conversation
verbatim (LEARNER / TEACHER), and structured events (QUIZ lines with the learner's outcome, mark_done lines).
The transcript view shows: the conversation verbatim (LEARNER / TEACHER lines, each tagged with its message id)
and QUIZ outcome lines.
Report, with verbatim evidence quotes for everything:
NODE THEORY — the graph is a set of objective facts, full stop. A node is what is true (or falsely
believed) about the world, never anything about the learner or the session. The learner's
relationship to a fact — whether they know it, missed it, or hold it as a misconception — is
expressed ENTIRELY by the quiz-outcome events you attribute; it never leaks into the node itself.
So no "the learner understands X", no "discussed in this session": the node is just X. A
misconception node states the wrong idea itself as a claim, with kind "misconception" marking it
wrong — never "the learner believes Y".
NODE THEORY — the graph is a set of TRUE propositions about the world, full stop. A node is never anything
about the learner or the session. The learner's relationship to a proposition — whether they know it, missed
it, resisted it — is expressed ENTIRELY by the quiz outcomes and your understanding note; it never leaks into
the node itself. So no "the learner understands X", no "discussed in this session": the node is just X. A
misconception is NOT a node of its own wrong claim — state the TRUE correction as the proposition, and let the
quiz outcomes carry that the learner believed the wrong version.
1. PROPOSITIONS — every distinct reasoning-step-sized proposition the session touched:
- Each carries a "name": a short 2-6 word header (the rendering handle), and a "statement": the
full fact as one objective reasoning step.
- Plan-tree nodes become propositions (pass their treeNodeId) — statement from the node title, enriched to a
complete proposition if the prose elaborates it.
- Propositions that appeared only in prose (teacher explanations, learner questions) get fresh ids.
- Include MISCONCEPTIONS — wrong ideas the learner showed or the teacher warned against — with kind
"misconception".
- Each is a self-contained TRUE statement ("statement"), plus its elaboration ("description" — more
words, same proposition: scope, precision, what exactly it means).
- Include corrections of misconceptions the teacher articulated, and true facts explained in prose.
- One reasoning step per proposition: never a topic blob, never a sub-step.
- aliases: verbatim alternate phrasings this session used for the same thing.
- "messages": the [id] tags of every message where the proposition came up — provenance, report it.
- "aliases": verbatim phrasings this session used for the same proposition.
- "understanding": ONE named-word-scale note on the evidence of the learner's grasp, for this session
only ("solid", "shaky", "accepted", "resisted", "not yet engaged"). No numbers, no predictions.
2. EDGES between propositions:
- The tree's dependency edges are the backbone: from = prerequisite, to = dependent, type "requires".
- Also include relations articulated only in prose. Use "supports" when the transcript presents something as
helpful but NOT a prerequisite.
- "follows": to is DERIVABLE from from (understanding from makes understanding to achievable).
- "supports": from GROUNDS to without entailing it (helpful context, not a prerequisite).
- Only relations the transcript actually articulates — with evidence.
3. ATTRIBUTION — UNTAGGED quizzes only:
- QUIZ lines whose question carries a tree nodeId resolve automatically; do not attribute those.
- The others are PROBE-phase questions (asked before the tree was initialized) that test prerequisite
understanding. Attribute them RETROACTIVELY to the proposition they were really probing — even if that
proposition is only articulated later in the session. The QUIZ question text is your guide. If you truly
cannot tell, use null.
3. ATTRIBUTION — every QUIZ line gets a mapping: toolCallId → the proposition id it was really testing.
Attribute RETROACTIVELY — a quiz often tests a proposition the session only articulates later; the
question text is your guide. If you truly cannot tell, use null.
Evidence discipline: every quote must appear VERBATIM in the transcript view (a QUIZ line counts as the
transcript). The auditor rejects anything unbacked. When in doubt, report; the auditor decides.
@@ -205,6 +210,16 @@ function createExtractor(model: LanguageModel) {
// ─── the auditor ───────────────────────────────────────────────────────────
const auditSchema = findingsSchema.extend({
propositions: z.array(
propositionSchema.extend({
linkTo: z
.string()
.optional()
.describe(
"The id of the EXISTING graph node this proposition IS — your identity verdict. Absent = create a new node. When unsure, omit: duplicates over wrong merges.",
),
}),
),
dropped: z
.array(
z.object({
@@ -216,27 +231,46 @@ const auditSchema = findingsSchema.extend({
});
const AUDITOR_INSTRUCTIONS = `You are the audit stage of a learning application's extraction pipeline. You receive
ONE session transcript and a set of proposed findings (propositions, edges, event attribution). Your job is the
faithfulness net: verify every candidate against the transcript, keep only what is backed, and return the
verified subset.
ONE session transcript, a set of proposed findings (propositions, edges, quiz attribution), and the CURRENT
knowledge graph. Your jobs: the faithfulness net, and identity.
Verify:
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement must
be a faithful reading of it. The node theory applies: a statement must be an OBJECTIVE, world-facing fact —
reword learner- or session-relative statements ("the learner understands X" → "X") or drop them when the
objective fact is not actually backed. Drop invented or bloated statements. Conservative: when two proposed propositions
are really one, keep the better-stated one and drop the other.
- EDGES: the evidence must actually articulate the relation, in the direction claimed (from = prerequisite).
Drop direction-swapped or speculative edges.
- ATTRIBUTION: only UNTAGGED quizzes carry attribution — the QUIZ line's question must genuinely test the
attributed proposition. Drop wrong attributions to null, or re-point them to the proposition the question
actually tests.
FAITHFULNESS — verify every candidate against the transcript:
- PROPOSITIONS: every evidence quote must appear (near-)verbatim in the transcript view, and the statement
must be a faithful reading of it. The node theory applies: a statement must be a TRUE, self-contained,
world-facing fact — reword learner- or session-relative statements ("the learner understands X" → "X"), state
misconceptions as the true correction, or drop them when no objective fact is backed. Drop invented or
bloated statements. Conservative: when two proposed propositions are really one, keep the better-stated one
and drop the other.
- EDGES: the evidence must actually articulate the relation, in the direction claimed. Drop
direction-swapped or speculative edges.
- ATTRIBUTION: the QUIZ line's question must genuinely test the attributed proposition. Drop wrong
attributions to null, or re-point them to the proposition the question actually tests.
- UNDERSTANDING notes: verify against the quiz outcomes and the session's evidence; reword to a named-word
scale or drop.
IDENTITY — link-or-create, per proposition:
- You see the current graph (every node: id, statement, description, aliases). For each verified proposition,
decide: does an EXISTING node already state this same proposition?
- If yes: set "linkTo" to that node's id. The session's phrasings will join its aliases and its provenance.
- If no: omit "linkTo" and a new node is created.
- When unsure, omit — duplicates are recoverable later; wrong merges are not.
Also drop propositions not referenced by anything else ONLY if they have no other backing — an unreferenced but
well-evidenced proposition is still knowledge.
Never invent new candidates. Your entire deliverable is ONE submit_review call containing the verified findings
(the same shapes you received) plus the dropped list.`;
(the same shapes you received, with linkTo added) plus the dropped list.`;
/** The current graph, compact — the auditor's identity context. */
function graphView(nodes: GraphNode[]): string {
if (nodes.length === 0) return "(the graph is currently empty — every proposition is a new node)";
const lines = nodes.map(
(n) =>
`- ${n.id} :: ${n.statement}\n description: ${n.description}` +
(n.aliases.length > 0 ? `\n aliases: ${n.aliases.join(" | ")}` : ""),
);
return lines.join("\n");
}
function createAuditor(model: LanguageModel) {
return new ToolLoopAgent({
@@ -244,7 +278,7 @@ function createAuditor(model: LanguageModel) {
instructions: AUDITOR_INSTRUCTIONS,
tools: {
submit_review: tool({
description: "Submit the verified findings subset + the dropped list.",
description: "Submit the verified findings subset (with identity verdicts) + the dropped list.",
inputSchema: auditSchema,
execute: async (review) => review,
}),
@@ -295,17 +329,20 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
throw new Error("the extractor never submitted findings");
}
async function audit(chatId: string, findings: SessionFindings): Promise<SessionFindings> {
async function audit(chatId: string, findings: SessionFindings): Promise<AuditResult> {
const record = chatStore.getChat(chatId);
if (!record) throw new Error(`Chat not found: ${chatId}`);
const view = transcriptView(record);
const graph = getGraph(graphStore);
console.info(
`[extraction ${chatId}] auditor starting (${findings.propositions.length} propositions, ${findings.edges.length} edges, ` +
`${Object.keys(findings.attribution.quiz).length} attributions)`,
`${Object.keys(findings.attribution.quiz).length} attributions, ${graph.nodes.length} graph nodes)`,
);
const review = await auditorAgent.generate({
prompt: `Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}`,
prompt:
`Transcript:\n\n${view}\n\nProposed findings:\n\n${JSON.stringify(findings, null, 2)}\n\n` +
`Current knowledge graph (${graph.nodes.length} nodes):\n\n${graphView(graph.nodes)}`,
});
let auditResult: AuditResult | null = null;
for (const part of review.content) {
@@ -320,12 +357,12 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
auditResult.dropped.map((d) => `${d.ref} (${d.reason})`).join("; "),
);
}
const { dropped: _dropped, ...verified } = auditResult;
console.info(
`[extraction ${chatId}] audit kept ${verified.propositions.length}/${findings.propositions.length} propositions, ` +
`${verified.edges.length}/${findings.edges.length} edges`,
`[extraction ${chatId}] audit kept ${auditResult.propositions.length}/${findings.propositions.length} propositions, ` +
`${auditResult.edges.length}/${findings.edges.length} edges, ` +
`${auditResult.propositions.filter((p) => p.linkTo !== undefined).length} linked to existing nodes`,
);
return verified;
return auditResult;
}
async function runExtraction(chatId: string): Promise<EmissionStats> {
@@ -334,10 +371,10 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
try {
console.info(`[extraction ${chatId}] pipeline start`);
const findings = await extract(chatId);
const verified = await audit(chatId, findings);
const auditResult = await audit(chatId, findings);
// The merger re-reads the chat record itself — events are rebuilt
// from the transcript, never from stale copies.
const stats = mergeFindings(chatId, verified, { chatStore, graphStore });
const stats = mergeFindings(chatId, auditResult, { chatStore, graphStore });
console.info(`[extraction ${chatId}] merged: ${stats.nodes} nodes, ${stats.edges} edges, ${stats.events} events`);
return stats;
} finally {
@@ -347,3 +384,4 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
return { runExtraction };
}
src/core/contracts/graph.ts
+130 -79
View File
@@ -1,50 +1,69 @@
/**
* Knowledge-graph contracts — nodes, ledgers, extraction findings
*
* The minimal implementation of the graph vision (docs/vision.md §1, the
* 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
* (derivable) and supports (grounds without entailing); provenance is
* message-granular; extraction is free-form and post-hoc, fully decoupled
* from the session's plan tree; the event ledger records quiz outcomes plus
* the extractor's per-proposition understanding observations. Anything not
* yet decided (the state model, strength derivation, understanding
* treatment, selection strategies) is deliberately NOT here — transitional
* stubs carry the renderer until those land.
*
* Part of the domain-split contracts (src/core/contracts/): each domain file
* is owned outright by its stream — cross-domain schema changes go through
* master. Pure types + protocol docs + tiny pure helpers; type-only imports
* of "ai" (safe everywhere: server, client, tests). See docs/vision.md.
* master. Pure types + protocol docs + tiny pure helpers. See docs/vision.md.
*/
// ─── Knowledge graph (vision §1 — global, cross-session) ────────────────────
export type EdgeType = "requires" | "supports";
/** Two relations only (DECIDED 2026-09-02): follows = b is derivable from a;
* supports = a grounds b without entailing it. No scalars, no encoded
* strength — strength is derived at read time from topology. */
export type EdgeType = "follows" | "supports";
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
* edges record every (chat, message) where they came up, deep-linkable to
* the exact transcript point. */
export interface ProvenancePoint {
chatId: string;
messageId: string;
}
export interface GraphNode {
/** Opaque stable id, store-assigned: "n_" + nanoid-like. */
id: string;
/** Short name / header for the fact (2-6 words) — the rendering handle.
* NODE THEORY: a node is an OBJECTIVE fact, never learner- or
* session-relative. No "the learner understands…", no "this session
* covered…" — just the fact, world-facing. The learner's relationship
* to the fact lives ENTIRELY in the event ledger: state labels are
* derived from quiz outcomes, never written into the node. */
name: string;
/** Canonical proposition statement — the full fact, one reasoning step,
* also objective (no learner / teacher / session references). */
/** The minimal proposition — one true, self-contained statement. This is
* BOTH the rendering label AND the identity key: the auditor recognizes
* a re-encounter by judgment (link-or-create), not by string matching. */
statement: string;
/** Verbatim alternate statements from sessions (merge target audit trail). */
/** The elaborated version — required. Same proposition, more words:
* scope, precision, the "what exactly does it mean". */
description: string;
/** Verbatim phrasings sessions used for this proposition (the identity
* audit trail — how the world said what the node says). */
aliases: string[];
kind: "truth" | "misconception";
createdAt: string;
/** Every (chat, message) point where this proposition came up. */
provenance: ProvenancePoint[];
}
export interface GraphEdge {
/** Prerequisite GraphNode id. */
from: string;
to: string;
type: EdgeType;
/** Chat ids that asserted this edge (teacher may simplify — auditable). */
provenance: string[];
/** Every (chat, message) point where this relation was asserted. */
provenance: ProvenancePoint[];
}
/** mappings.jsonl line — the session→graph identity map: the extraction
* pipeline's proposition (identified by its source id) → the graph node it
* merged into. The merge audit trail: one line per (chat, graph node). */
* pipeline's proposition (identified by its run-local source id) → the
* graph node it resolved to (created or linked). The audit trail of the
* auditor's identity verdicts; also the chat-scoped resolution table the
* merger uses to point events and edges at node ids. */
export interface NodeMapping {
chatId: string;
/** The writer's local id for the proposition — a plan-tree node id when
* tree-derived, else the extractor's proposition id ("p1", …). */
/** The extractor's run-local proposition id ("p1", …). */
sourceId: string;
graphNodeId: string;
}
@@ -57,23 +76,26 @@ export interface SessionCheckpoint {
foldedAt: string;
}
export type LedgerEventKind =
| "probe-correct"
| "probe-miss"
| "dont-know"
| "misconception-signaled"
| "taught"
| "check-correct"
| "check-incorrect";
/**
* Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
* dont-know — plus `observation`, the extractor's per-proposition
* understanding note joining the ledger as a session-scoped observation
* (named-word scale; treatment pending the state-model ideation). No
* `taught` events at all; no misconception kind — a misconception is a
* TRUE correction in the graph, and the wrongness lives here, in the
* ledger, until the state model exists.
*/
export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
export interface LedgerEvent {
/** null = not yet attributed (untagged probe events; consolidation assigns). */
/** null = not yet attributed. */
nodeId: string | null;
kind: LedgerEventKind;
chatId: string;
at: string;
/** Idempotency key: the quiz part's toolCallId or `taught:<treeNodeId>` —
* re-emission of the same chat must never double-record an event. */
/** Idempotency key: the quiz part's toolCallId, or `obs:<sourceId>` for
* understanding observations — re-emission of the same chat must never
* double-record an event. */
ref?: string;
/** The learner's quiz note — diagnostic gold (vision §1 calibration). */
note?: string;
@@ -83,91 +105,121 @@ export interface LedgerEvent {
/**
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
* (nodes/edges/events/mappings/sessions). The derived graph is rebuilt by
* reading the ledgers — store events, derive state. sessions.jsonl is the
* extraction checkpoint: which chats the extraction pipeline has processed,
* and through which updatedAt (never extract the same turns twice).
* (nodes/edges/events/mappings/sessions), all append-only.
*
* nodes.jsonl one line per node — the FIRST line for an id defines
* statement/description/createdAt; later lines for the
* same id are link-appends (a session's re-encounter):
* their statements/aliases join `aliases`, their
* provenance unions. getGraph merges by id.
* edges.jsonl one line per edge ASSERTION; readers merge by
* (from,to,type), unioning provenance.
*
* The derived graph is rebuilt by reading the ledgers — store events,
* derive state. sessions.jsonl is the extraction checkpoint: which chats
* the extraction pipeline has processed, and through which updatedAt
* (never extract the same turns twice).
*
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
* merger): the extractor proposes propositions/edges/attribution with
* verbatim evidence, the auditor drops anything unbacked, and the merger
* (core/graph.ts, deterministic) resolves identity against the existing
* ledgers and appends. See SessionFindings below.
* merger), tree-blind and post-hoc: the extractor reads only the transcript
* and proposes propositions/edges/quiz-attribution with verbatim evidence;
* the auditor sees the extractor's output PLUS the current graph, owns
* faithfulness (evidence check) and identity (link-or-create: point each
* proposition at the existing node it is, or let it be created — recording
* the session's phrasings as aliases); the merger (core/graph.ts,
* deterministic) applies the verdicts and appends.
*/
// ─── Extraction findings (extractor output → audit → merger input) ───────────
/** One proposed proposition — a node candidate. */
/** One proposed proposition — a node candidate (or a re-encounter of one). */
export interface ExtractedProposition {
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
id: string;
/** Short name / header for the fact (2-6 words) — the rendering handle. */
name: string;
/** Canonical one-reasoning-step statement — an objective, world-facing
* fact (no learner / teacher / session references; the learner's
* relationship to it is carried only by the event attribution). */
/** The minimal proposition — a true, self-contained, world-facing
* statement. No learner / teacher / session references: misconceptions
* are stated as the true correction, and wrongness lives in the event
* ledger, never in the node. */
statement: string;
/** Verbatim alternate phrasings from the session (merge aliases). */
/** The elaborated version — required. */
description: string;
/** Verbatim phrasings the session used. */
aliases: string[];
kind: "truth" | "misconception";
/** The plan-tree node this came from, when it did (mapping provenance). */
treeNodeId?: string;
/** The message ids where this proposition came up — provenance. */
messages: string[];
/** Verbatim transcript quotes backing the proposition. */
evidence: string[];
/** The extractor's understanding note for this proposition IN THIS
* session — a named-word scale ("solid", "shaky", "accepted",
* "resisted"), no numbers. Joins the ledger as an observation event;
* treatment pending the state-model ideation. */
understanding?: string;
}
/** One proposed relation between two propositions. */
export interface ExtractedEdge {
/** Proposition id that is the prerequisite / support. */
/** Proposition id that is followed / supported. */
from: string;
/** Proposition id that depends on / is supported by `from`. */
to: string;
type: EdgeType;
/** The message ids where this relation was asserted — provenance. */
messages: string[];
/** Verbatim transcript quotes backing the relation. */
evidence: string[];
}
/** The extractor's event attribution — its ONLY say about events. The kinds,
* timestamps, idempotency refs, notes and question texts are all rebuilt
* deterministically by the merger from the transcript's quiz / mark_done
* tool parts; the LLM supplies exactly the judgment call: WHICH proposition
* each one is about (vision open question: probes attribute retroactively,
* against the whole transcript — a pre-tree probe can map to a proposition
* the tree only articulates later). */
/** The extractor's event attribution — its ONLY say about events. Event
* kinds, timestamps, idempotency refs, notes and question texts are all
* rebuilt deterministically by the merger from the transcript's tool
* parts, and TAGGED events (quiz nodeId, mark_done tree node) resolve
* deterministically through the proposition mapping. The LLM supplies
* exactly the one judgment call that needs it — the vision's open
* probe-attribution question: WHICH proposition did an UNTAGGED probe
* test? Answered retroactively against the whole transcript — a pre-tree
* probe can map to a proposition the tree only articulates later. */
/**
* The extractor's quiz attribution — its ONLY say about quiz events. Event
* kinds, timestamps, idempotency refs, notes and question texts are all
* rebuilt deterministically by the merger from the transcript's quiz tool
* parts; the LLM supplies exactly the judgment call: WHICH proposition each
* quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
* mapping to lean on) — retroactively against the whole transcript.
*/
export interface ExtractionAttribution {
/** UNTAGGED quiz toolCallId → proposition id; missing / null =
* unattributed (the event lands with nodeId null + the question text,
* exactly like unattributable probes always did). */
/** quiz toolCallId → proposition id; null / missing = unattributed (the
* event lands with nodeId null + the question text, a later-
* attribution input). */
quiz: Record<string, string | null>;
}
/** The extractor's — and, filtered, the auditor's — output shape. */
/** The extractor's output shape. */
export interface SessionFindings {
propositions: ExtractedProposition[];
edges: ExtractedEdge[];
attribution: ExtractionAttribution;
}
/** The auditor's verdict: the verified subset, plus the drops (for the log —
* the audit trail of what was rejected and why). */
export interface AuditResult extends SessionFindings {
/** The auditor's verdict on one proposition: the findings, plus `linkTo` —
* the identity judgment. Absent = create a new node. */
export interface AuditedProposition extends ExtractedProposition {
/** The existing GraphNode id this proposition IS — the auditor's
* identity verdict. On link, the session's phrasings (statement,
* aliases) join the node's aliases and the provenance unions.
* Conservative: when unsure, leave it out — duplicates over wrong
* merges. */
linkTo?: string;
}
/** The auditor's output: the verified findings (with identity verdicts),
* plus the drops (the audit trail of what was rejected and why). */
export interface AuditResult {
propositions: AuditedProposition[];
edges: ExtractedEdge[];
attribution: ExtractionAttribution;
/** Dropped candidates: proposition id, edge "from→to", or attribution
* toolCallId — with the reason. */
dropped: { ref: string; reason: string }[];
}
/** Per-node state class for the home view (vision §1 "Home page") — derived,
* never stored: taught if ANY taught event; else the LATEST quiz event decides
* (known / missed / dont-know); a node with no events is a candidate. */
/**
* Per-node state class for the home view — TRANSITIONAL (kept so the
* renderer survives the contract change untouched). Derived, never stored:
* the LATEST quiz event decides (known / missed / dont-know); a node with
* no quiz events is a candidate. `taught` is now unreachable (no taught
* events are emitted anymore) — the real state model (dependency depth +
* topology-derived strength + quiz events) is pending ideation.
*/
export type NodeStateClass = "taught" | "known" | "missed" | "dont-know" | "candidate";
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
@@ -187,4 +239,3 @@ export function titleFromPrompt(prompt: string): string {
const oneLine = prompt.replace(/\s+/g, " ").trim();
return oneLine.length > 72 ? `${oneLine.slice(0, 72)}…` : oneLine;
}
src/core/graph-context.ts
+16 -22
View File
@@ -15,10 +15,9 @@
* the block is absent (new learner).
*
* Selection strategy (§1 "the selection problem", simplest first): the WHOLE
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know /
* misconception nodes) plus their 1-hop `requires` neighborhood — a live
* misconception is always relevant context, and the frontier is where the
* next session should probe.
* graph while it is small; past MAX_NODES, hotspots (missed / dont-know
* nodes) plus their 1-hop `follows` neighborhood — the frontier is where
* the next session should probe.
*/
import type { GraphEdge, GraphNode } from "./contracts.ts";
@@ -31,22 +30,18 @@ export const GRAPH_CONTEXT_HEADER = "LEARNER KNOWLEDGE GRAPH (auto-generated)";
/** Above this node count only hotspots + their 1-hop neighborhood are shown. */
export const MAX_CONTEXT_NODES = 150;
/** Group order for node lines: diagnostic value first (vision §1 hotspots).
* A misconception-kind node is a hotspot whatever its state — a live
* misconception is always relevant context. */
/** Group order for node lines: diagnostic value first (vision §1 hotspots). */
const HOTSPOT_STATES: ReadonlySet<NodeStateClass> = new Set(["missed", "dont-know"]);
function groupOf(node: GraphNode, state: NodeStateClass): number {
if (HOTSPOT_STATES.has(state) || node.kind === "misconception") return 0;
if (HOTSPOT_STATES.has(state)) return 0;
if (state === "known") return 1;
if (state === "taught") return 2;
return 3; // candidate
return 2; // candidate
}
const GROUP_LABELS = [
"Hotspots (missed / dont-know / misconceptions) — likely misconception sites and unexplored ground:",
"Hotspots (missed / dont-know) — likely misconception sites and unexplored ground:",
"Known (answered correctly):",
"Taught (lessons covered them):",
"Candidates (never probed — assume nothing):",
] as const;
@@ -76,9 +71,9 @@ export function renderGraphContext(
truncated = true;
const hotspots = nodes.filter((n) => isHotspot(n, stateOf(n)));
const keep = new Set(hotspots.map((n) => n.id));
// 1-hop requires neighborhood: prerequisites AND dependents.
// 1-hop follows neighborhood: premises AND derivations.
for (const edge of edges) {
if (edge.type !== "requires") continue;
if (edge.type !== "follows") continue;
if (keep.has(edge.from) || keep.has(edge.to)) {
keep.add(edge.from);
keep.add(edge.to);
@@ -91,7 +86,7 @@ export function renderGraphContext(
// ── node lines, grouped/sorted by diagnostic priority ──────────────────────────
const lines: string[] = [
`===== ${GRAPH_CONTEXT_HEADER} =====`,
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "requires").length} requires edges`,
`${truncated ? `showing ${selected.length} of ${nodes.length} nodes (hotspots + their 1-hop neighborhood)` : `${nodes.length} nodes`}, ${edges.filter((e) => e.type === "follows").length} follows edges`,
];
for (const [group, label] of GROUP_LABELS.entries()) {
@@ -101,19 +96,18 @@ export function renderGraphContext(
if (groupNodes.length === 0) continue;
lines.push("", label);
for (const node of groupNodes) {
const kind = node.kind === "misconception" ? " · misconception" : "";
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})${kind}`);
lines.push(`• [${stateOf(node)}] ${node.statement} (${node.id})`);
}
}
// ── requires edges, merged into chains ──────────────────────────────────
const byId = new Map(selected.map((n) => [n.id, n.statement]));
const requires = edges.filter(
(e) => e.type === "requires" && byId.has(e.from) && byId.has(e.to),
const follows = edges.filter(
(e) => e.type === "follows" && byId.has(e.from) && byId.has(e.to),
);
if (requires.length > 0) {
lines.push("", "Requires (prerequisite → dependent):");
for (const chain of edgeChains(requires)) {
if (follows.length > 0) {
lines.push("", "Follows (premise → derivation):");
for (const chain of edgeChains(follows)) {
lines.push(chain.map((id) => byId.get(id) ?? id).join(" → "));
}
}
src/core/graph.ts
+176 -172
View File
@@ -2,22 +2,34 @@
* The knowledge graph: merger (the deterministic final stage of the
* extraction pipeline — vision §1 construction) + derivation (reads).
*
* The extraction pipeline is the SOLE graph writer:
* The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
*
* extractor (LLM) → SessionFindings: propositions, edges, and event
* attribution, each with verbatim evidence
* auditor (LLM) → the verified subset (unbacked candidates dropped)
* extractor (LLM) → SessionFindings: propositions, edges, quiz
* attribution, per-proposition understanding notes —
* each with verbatim evidence, from the transcript ALONE
* auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
* VERDICT per proposition (linkTo an existing node, or
* create), and the drops for the log
* merger (HERE) → mergeFindings(): deterministic ledger appends
*
* The merger is deliberately the ONLY stage that touches the ledgers, and it
* is pure code: the LLM supplies exactly the judgment calls (what was
* taught, what depends on what, which proposition a quiz tested), while
* event kinds/timestamps/refs — anything the model could silently
* mis-copy — are rebuilt deterministically from the transcript's tool
* parts. Idempotent + incremental: statement/alias matching dedupes nodes,
* (from,to,type) dedupes edge assertions, chat-scoped refs dedupe events,
* one mapping line per (chat, graph node), and sessions.jsonl checkpoints
* which chats have been extracted (never the same turns twice).
* is pure code: the LLM supplies exactly the judgment calls (what the
* propositions are, what follows/supports what, which proposition a quiz
* tested, whether a proposition IS an existing node), while event
* kinds/timestamps/refs/provenance — anything the model could silently
* mis-copy — are rebuilt deterministically from the transcript's tool parts
* and the message ids in the findings. Idempotent + incremental: refs are
* chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
* asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
* sessions.jsonl checkpoints which chats have been extracted (never the
* same turns twice).
*
* Identity across re-encounters is the AUDITOR's judgment (link-or-create),
* not string matching: the conservative v1 exact-statement merge was
* replaced by the 2026-09-02 redesign. A re-extracted session's
* propositions link to the existing nodes; the link appends a node line
* whose phrasings join `aliases` and whose provenance unions (getGraph
* merges lines by id).
*
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
* only — everything is recomputed from the ledgers. Every function takes
@@ -29,14 +41,14 @@
import { randomBytes } from "node:crypto";
import { isToolUIPart, type UIMessage } from "ai";
import type {
AuditResult,
GraphEdge,
GraphNode,
ProvenancePoint,
GraphSnapshot,
LedgerEvent,
LedgerEventKind,
NodeStateClass,
PlanToolResult,
SessionFindings,
} from "./contracts.ts";
import type { QuestionAnswerData } from "./question-core.ts";
import type { ChatStore, GraphStore } from "./ports.ts";
@@ -80,19 +92,25 @@ function toolParts(messages: UIMessage[]): ToolPartView[] {
return out;
}
/** A plan tool part's output, if it is a well-formed PlanToolResult. */
function planResult(part: ToolPartView): PlanToolResult | null {
if (part.state !== "output-available") return null;
const out = part.output;
if (typeof out !== "object" || out === null) return null;
if (!("tree" in out) || typeof (out as PlanToolResult).ok !== "boolean") return null;
return out as PlanToolResult;
/** (chatId, messageIds) → provenance points; order-preserving, deduped. */
function pointsOf(chatId: string, messages: string[]): ProvenancePoint[] {
const seen = new Set<string>();
const out: ProvenancePoint[] = [];
for (const messageId of messages) {
if (messageId.length === 0 || seen.has(messageId)) continue;
seen.add(messageId);
out.push({ chatId, messageId });
}
return out;
}
const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
a.chatId === b.chatId && a.messageId === b.messageId;
// ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
export interface EmissionStats {
/** New graph nodes appended (and mappings with them). */
/** New graph nodes appended. */
nodes: number;
/** New edge assertions appended. */
edges: number;
@@ -102,156 +120,135 @@ export interface EmissionStats {
/**
* Merge one chat's audited findings into the graph ledgers — the pipeline's
* apply stage. Deterministic + idempotent: re-merging the same findings (or
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or
* a later re-extraction of the same chat) appends nothing but the new.
*
* Identity resolution (v1: conservative): a proposition whose statement or
* alias EXACTLY matches (case-insensitive, trimmed) an existing node's
* statement or alias merges onto it; everything else is a new node. One
* mapping line per (chat, graph node) — the merge audit trail. Fuzzy /
* embedding adjudication stays deferred (vision §1 consolidation).
* Identity is the auditor's link-or-create verdict: `linkTo` an existing
* node (validated against the ledger — an unknown id falls back to create)
* appends a LINK line (the session's phrasings join the node's aliases and
* the provenance unions; getGraph merges lines by id); absent → a new node.
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
* a proposition id already mapped for this chat reuses its resolution.
*
* Events are rebuilt deterministically from the transcript's quiz and
* plan_mark_done parts (kinds, timestamps, notes, refs). TAGGED events
* (quiz nodeId, mark_done tree node) resolve through the proposition
* mapping — deterministic; the findings' attribution is consulted ONLY for
* untagged probe quizzes: the retroactive probe-attribution resolution (a
* pre-tree probe maps to a proposition the tree only articulates later).
* Unattributed quizzes land with nodeId null + the question text, exactly
* like the unattributable ones always have.
* Events are rebuilt deterministically from the transcript: quiz outcomes
* (correct / miss / dont-know; kinds + timestamps + refs from the tool
* parts, nodeId from the extractor's attribution through this run's
* mapping) plus one observation event per proposition carrying an
* understanding note (ref `obs:<sourceId>`). No taught events, ever.
*/
export function mergeFindings(chatId: string, findings: SessionFindings, deps: GraphDeps): EmissionStats {
export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps): EmissionStats {
const { graphStore: ledgers, chatStore } = deps;
const record = chatStore.getChat(chatId);
if (!record) return { nodes: 0, edges: 0, events: 0 };
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
// 1. Identity resolution. mapping: proposition id (and tree node id)
// → graph node id — from this chat's existing mapping lines, then
// extended as findings resolve. chatNodeIds: graph nodes this chat has
// already mapped onto — the (chat, graph node) mapping dedupe.
// 1. Resolution: this run's propositions → graph node ids. Existing
// mapping lines for THIS chat are loaded first — the audit trail doubles
// as the resolution table, which makes re-merging the same chat
// idempotent (a re-extraction re-merges through the auditor's linkTo
// verdicts instead). Fresh sourceIds resolve as the verdicts say.
const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
const mapping = new Map<string, string>();
const chatNodeIds = new Set<string>();
const seenMappingKeys = new Set<string>();
for (const m of ledgers.readMappings()) {
if (m.chatId !== chatId || typeof m.sourceId !== "string") continue;
if (m.chatId !== chatId) continue;
mapping.set(m.sourceId, m.graphNodeId);
chatNodeIds.add(m.graphNodeId);
}
// 2. Propositions → graph nodes. Exact statement/alias match against the
// whole ledger first (cross-session reuse), then against propositions
// created earlier in THIS merge (intra-findings dedupe).
const norm = (s: string): string => s.trim().toLowerCase();
const existingByTitle = new Map<string, string>();
for (const node of ledgers.readNodes()) {
for (const title of [node.statement, ...node.aliases]) {
const key = norm(title);
if (key && !existingByTitle.has(key)) existingByTitle.set(key, node.id);
}
seenMappingKeys.add(m.sourceId);
}
const now = new Date().toISOString();
for (const p of findings.propositions) {
if (mapping.has(p.id)) continue;
const keys = [p.statement, ...p.aliases].map(norm).filter((k) => k.length > 0);
const matchId = keys.map((k) => existingByTitle.get(k)).find((id) => id !== undefined);
for (const p of audit.propositions) {
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent
const points = pointsOf(chatId, p.messages);
let graphNodeId: string;
if (matchId !== undefined) {
graphNodeId = matchId; // reuse — conservative merge, duplicates over wrong merges
} else {
const graphNode: GraphNode = {
id: newGraphNodeId(),
name: p.name,
if (p.linkTo !== undefined && existingIds.has(p.linkTo)) {
// Identity verdict: this proposition IS that node. Append a link
// line — the session's phrasings (statement + aliases) join the
// node's aliases, the provenance unions. Reads merge by id.
graphNodeId = p.linkTo;
ledgers.appendNode({
id: graphNodeId,
statement: p.statement,
description: p.description,
aliases: [...p.aliases],
kind: p.kind,
createdAt: now,
provenance: points,
});
} else {
const node: GraphNode = {
id: newGraphNodeId(),
statement: p.statement,
description: p.description,
aliases: [...p.aliases],
createdAt: now,
provenance: points,
};
ledgers.appendNode(graphNode);
ledgers.appendNode(node);
stats.nodes++;
graphNodeId = graphNode.id;
}
for (const key of keys) if (!existingByTitle.has(key)) existingByTitle.set(key, graphNodeId);
if (!chatNodeIds.has(graphNodeId)) {
ledgers.appendMapping({ chatId, sourceId: p.treeNodeId ?? p.id, graphNodeId });
chatNodeIds.add(graphNodeId);
graphNodeId = node.id;
existingIds.add(graphNodeId);
}
mapping.set(p.id, graphNodeId);
if (p.treeNodeId !== undefined) mapping.set(p.treeNodeId, graphNodeId);
if (!seenMappingKeys.has(p.id)) {
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
seenMappingKeys.add(p.id);
}
}
// 3. Edges. Dedupe by (from,to,type) across the whole ledger: an existing
// assertion just gains this chatId as provenance (appended as a new
// line; reads merge).
const asserted = new Map<string, Set<string>>();
// 2. Edges. Resolve endpoints through the mapping; dedupe by
// (from,to,type) already asserted by THIS chat (an existing assertion
// by another chat stays untouched — provenance unions at read time).
const assertedByChat = new Set<string>();
for (const line of ledgers.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
let provenance = asserted.get(key);
if (!provenance) {
provenance = new Set();
asserted.set(key, provenance);
}
for (const c of line.provenance) provenance.add(c);
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
}
for (const edge of findings.edges) {
for (const edge of audit.edges) {
const from = mapping.get(edge.from);
const to = mapping.get(edge.to);
if (!from || !to || from === to) continue; // unmapped endpoint — skip
const key = `${from}\u0000${to}\u0000${edge.type}`;
const provenance = asserted.get(key);
if (provenance?.has(chatId)) continue; // already asserted by this chat
ledgers.appendEdge({ from, to, type: edge.type, provenance: [chatId] });
if (provenance) provenance.add(chatId);
else asserted.set(key, new Set([chatId]));
if (assertedByChat.has(key)) continue;
ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
assertedByChat.add(key);
stats.edges++;
}
// 4. Events, rebuilt deterministically in transcript order so the
// probe/check boundary is exact: quizzes BEFORE the first plan_init
// output are probes. Refs are chat-scoped (a quiz toolCallId or
// taught:<treeNodeId>) — a ref already in the ledger means this turn
// was extracted before. The findings' attribution supplies the
// nodeId, nothing else.
// 3. Events, rebuilt deterministically in transcript order. Refs are
// chat-scoped idempotency keys (quiz toolCallId, obs:<sourceId>) —
// a ref already in the ledger means this evidence was extracted before.
const seenRefs = new Set<string>();
for (const event of ledgers.readEvents()) {
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
}
let planSeen = false;
for (const part of toolParts(record.messages)) {
// The boundary flips on the first plan_init OUTPUT, whatever it says
// (an ok:false "tree already initialized" still means a tree exists).
if (part.name === "plan_init" && part.state === "output-available") planSeen = true;
if (part.name === "quiz" && part.state === "output-available") {
const event = quizEvent(part, chatId, mapping, findings, planSeen, now);
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
ledgers.appendEvent(event);
seenRefs.add(event.ref);
stats.events++;
}
}
if (part.name === "plan_mark_done" && part.state === "output-available") {
const result = planResult(part);
const input = part.input as { id?: unknown } | undefined;
if (result?.ok && typeof input?.id === "string") {
const ref = `taught:${input.id}`;
if (!seenRefs.has(ref)) {
// mark_done is tagged with the tree node id — deterministic.
ledgers.appendEvent({
nodeId: mapping.get(input.id) ?? null,
kind: "taught",
chatId,
at: now,
ref,
});
seenRefs.add(ref);
stats.events++;
}
}
if (part.name !== "quiz" || part.state !== "output-available") continue;
const event = quizEvent(part, chatId, audit, mapping, now);
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
ledgers.appendEvent(event);
seenRefs.add(event.ref);
stats.events++;
}
}
// Understanding observations: one per proposition carrying a note,
// resolved to the node the verdict mapped it to.
for (const p of audit.propositions) {
if (p.understanding === undefined || p.understanding.length === 0) continue;
const ref = `obs:${p.id}`;
if (seenRefs.has(ref)) continue;
ledgers.appendEvent({
nodeId: mapping.get(p.id) ?? null,
kind: "observation",
chatId,
at: now,
ref,
note: p.understanding,
});
seenRefs.add(ref);
stats.events++;
}
// 5. Checkpoint: this chat is extracted through its current updatedAt.
// 4. Checkpoint: this chat is extracted through its current updatedAt.
ledgers.appendCheckpoint({ chatId, foldedAt: record.updatedAt });
return stats;
}
@@ -261,9 +258,8 @@ export function mergeFindings(chatId: string, findings: SessionFindings, deps: G
function quizEvent(
part: ToolPartView,
chatId: string,
audit: AuditResult,
mapping: Map<string, string>,
findings: SessionFindings,
planSeen: boolean,
now: string,
): LedgerEvent | null {
const out = part.output;
@@ -274,16 +270,13 @@ function quizEvent(
let kind: LedgerEventKind;
if (answer.dontKnow === true) kind = "dont-know";
else if (answer.correct === true) kind = planSeen ? "check-correct" : "probe-correct";
else kind = planSeen ? "check-incorrect" : "probe-miss";
else if (answer.correct === true) kind = "correct";
else kind = "miss";
// Tagged (input.nodeId) resolves deterministically through the mapping;
// attribution is the LLM judgment call for UNTAGGED probes only — the
// vision's open probe-attribution question, resolved retroactively.
const input = part.input as { nodeId?: unknown } | undefined;
const tagged = typeof input?.nodeId === "string" ? mapping.get(input.nodeId) : undefined;
const attributedId = findings.attribution.quiz[part.toolCallId];
const nodeId = tagged ?? (attributedId ? (mapping.get(attributedId) ?? null) : null);
// Tree-blind: EVERY quiz attributes through the extractor's judgment —
// the vision's retroactive attribution, with no tree mapping to lean on.
const attributedId = audit.attribution.quiz[part.toolCallId];
const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
const event: LedgerEvent = {
nodeId,
@@ -293,7 +286,7 @@ function quizEvent(
ref: part.toolCallId,
};
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
// The question text is the attribution input for unattributed probes.
// The question text is the attribution input for unattributed quizzes.
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
return event;
}
@@ -335,20 +328,39 @@ export interface DerivedGraph {
}
/**
* The whole graph from the ledgers. Edge LINES are assertions; the returned
* edges are merged by (from,to,type) with unioned provenance, first-seen
* order. `store events, derive state` — this is the only reader that matters.
* The whole graph from the ledgers. Node LINES may repeat an id (link-
* appends): the FIRST line defines statement/description/createdAt; later
* lines' statements and aliases join `aliases`, provenance unions. Edge
* LINES are assertions, merged by (from,to,type) with unioned provenance,
* first-seen order. `store events, derive state` — this is the only reader
* that matters.
*/
export function getGraph(graphStore: GraphStore): DerivedGraph {
const nodes = graphStore.readNodes();
// Merge node lines by id (first line defines; later lines add phrasings).
const byId = new Map<string, GraphNode>();
for (const line of graphStore.readNodes()) {
const existing = byId.get(line.id);
if (!existing) {
byId.set(line.id, { ...line, aliases: [...line.aliases], provenance: [...line.provenance] });
continue;
}
// A later line's statement is a phrasing, not a replacement.
for (const phrasing of [line.statement, ...line.aliases]) {
if (phrasing.length > 0 && !existing.aliases.includes(phrasing)) existing.aliases.push(phrasing);
}
for (const pt of line.provenance) {
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
}
}
const merged: GraphEdge[] = [];
const byKey = new Map<string, GraphEdge>();
for (const line of graphStore.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
const existing = byKey.get(key);
if (existing) {
for (const c of line.provenance) {
if (!existing.provenance.includes(c)) existing.provenance.push(c);
for (const pt of line.provenance) {
if (!existing.provenance.some((q) => samePoint(q, pt))) existing.provenance.push(pt);
}
} else {
const edge: GraphEdge = { ...line, provenance: [...line.provenance] };
@@ -356,34 +368,27 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
merged.push(edge);
}
}
return { nodes, edges: merged, events: graphStore.readEvents() };
return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
}
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set([
"probe-correct",
"check-correct",
"probe-miss",
"check-incorrect",
"dont-know",
]);
const QUIZ_EVENT_KINDS: ReadonlySet<string> = new Set(["correct", "miss", "dont-know"]);
/**
* Derive node states from events (never stored): taught if ANY taught event;
* else the LATEST quiz event decides (known / missed / dont-know); a node
* with no events is a candidate.
* Derive node states from events (never stored): the LATEST quiz event
* decides (known / missed / dont-know); a node with no quiz events is a
* candidate. TRANSITIONAL (2026-09-02 redesign): `taught` is unreachable
* now that no taught events are emitted, and observation events are NOT
* state inputs — the real state model (dependency depth + topology-derived
* strength + quiz events) is pending ideation; this keeps the renderer
* alive until it lands.
*/
export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Record<string, NodeStateClass> {
const states: Record<string, NodeStateClass> = {};
for (const node of nodes) states[node.id] = "candidate";
const taught = new Set<string>();
const latest = new Map<string, { at: string; kind: LedgerEventKind; index: number }>();
events.forEach((event, index) => {
if (event.nodeId === null) return;
if (event.kind === "taught") {
taught.add(event.nodeId);
return;
}
if (!QUIZ_EVENT_KINDS.has(event.kind)) return;
const current = latest.get(event.nodeId);
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
@@ -392,13 +397,9 @@ export function deriveNodeStates(nodes: GraphNode[], events: LedgerEvent[]): Rec
});
for (const node of nodes) {
if (taught.has(node.id)) {
states[node.id] = "taught";
continue;
}
const last = latest.get(node.id);
if (!last) continue; // candidate
if (last.kind === "probe-correct" || last.kind === "check-correct") states[node.id] = "known";
if (last.kind === "correct") states[node.id] = "known";
else if (last.kind === "dont-know") states[node.id] = "dont-know";
else states[node.id] = "missed";
}
@@ -422,10 +423,13 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
chats.add(chatId);
};
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
for (const node of nodes) {
for (const pt of node.provenance) touch(node.id, pt.chatId);
}
for (const edge of edges) {
for (const chatId of edge.provenance) {
touch(edge.from, chatId);
touch(edge.to, chatId);
for (const pt of edge.provenance) {
touch(edge.from, pt.chatId);
touch(edge.to, pt.chatId);
}
}
for (const event of events) {
src/core/ports.ts
+1 -1
View File
@@ -56,7 +56,7 @@ export interface GraphStore {
appendEdge(assertion: GraphEdge): void;
readEdges(): GraphEdge[];
/** Append one quiz outcome / taught event. */
/** Append one quiz outcome / understanding observation. */
appendEvent(event: LedgerEvent): void;
readEvents(): LedgerEvent[];
src/store/graph.ts
+28 -35
View File
@@ -1,20 +1,26 @@
/**
* GraphStore — the fs JSONL adapter: five append-only ledgers at
* config.LEDGER_FILES (contracts.ts "v1 emission").
* nodes.jsonl GraphNode — one line per graph node
* nodes.jsonl GraphNode — one line per node write: the FIRST
* line for an id defines it; a later
* line with a REPEATED id is a
* link-append (session phrasings join
* aliases, provenance unions); reads
* merge by id (core/graph.ts getGraph)
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second
* session asserting the same (from,to,type)
* appends another line; reads merge)
* events.jsonl LedgerEvent — quiz outcomes + taught events
* events.jsonl LedgerEvent — quiz outcomes + understanding observations
* mappings.jsonl NodeMapping — the session→graph identity map:
* (chatId, sourceId) → graphNodeId
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
*
* Implements core/ports.ts GraphStore. data/graph/* is exclusively this
* adapter's. Reads are strict against the contracts.ts shapes: corrupt lines
* are skipped, never guessed at — the ledgers are the source of truth and
* everything else is derived (core/graph.ts, which takes this port as a
* parameter). Same idioms as src/store/chats.ts: sync fs, no framework.
* adapter's. Reads are strict against the contracts.ts shapes (the 2026-09-02
* graph redesign): corrupt lines are skipped, never guessed at — the ledgers
* are the source of truth and everything else is derived (core/graph.ts,
* which takes this port as a parameter). Same idioms as src/store/chats.ts:
* sync fs, no framework.
*/
import * as fs from "node:fs";
@@ -26,6 +32,7 @@ import type {
LedgerEvent,
LedgerEventKind,
NodeMapping,
ProvenancePoint,
SessionCheckpoint,
} from "../core/contracts.ts";
import type { GraphStore } from "../core/ports.ts";
@@ -69,26 +76,22 @@ function readValid<T>(file: string, isValid: (v: unknown) => v is T): T[] {
return readRawLines(file).filter(isValid);
}
/** Pre-name ledger line (the first extraction runs) — read tolerantly. */
type StoredGraphNode = Omit<GraphNode, "name"> & { name?: string };
/** Provenance points are (chatId, messageId) pairs — message-granular. */
const isProvenancePoint = (v: unknown): v is ProvenancePoint =>
isRecord(v) && isString(v.chatId) && isString(v.messageId);
const isStoredGraphNode = (v: unknown): v is StoredGraphNode =>
const isGraphNode = (v: unknown): v is GraphNode =>
isRecord(v) &&
isString(v.id) &&
isString(v.statement) &&
isString(v.description) &&
Array.isArray(v.aliases) &&
(v.kind === "truth" || v.kind === "misconception") &&
v.aliases.every(isString) &&
isString(v.createdAt) &&
(v.name === undefined || isString(v.name));
Array.isArray(v.provenance) &&
v.provenance.every(isProvenancePoint);
/** Derived header for legacy node lines written before `name` existed:
* the first clause of the statement, capped — good enough to render. */
function nameFromStatement(statement: string): string {
const clause = statement.split(/[.;:—]/)[0]!.trim();
return clause.length > 60 ? `${clause.slice(0, 57)}…` : clause || statement;
}
const EDGE_TYPES = ["requires", "supports"] as const;
const EDGE_TYPES = ["follows", "supports"] as const;
const isGraphEdge = (v: unknown): v is GraphEdge =>
isRecord(v) &&
@@ -96,17 +99,9 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
isString(v.to) &&
(EDGE_TYPES as readonly string[]).includes(v.type as string) &&
Array.isArray(v.provenance) &&
v.provenance.every(isString);
v.provenance.every(isProvenancePoint);
const EVENT_KINDS: readonly LedgerEventKind[] = [
"probe-correct",
"probe-miss",
"dont-know",
"misconception-signaled",
"taught",
"check-correct",
"check-incorrect",
];
const EVENT_KINDS: readonly LedgerEventKind[] = ["correct", "miss", "dont-know", "observation"];
const isLedgerEvent = (v: unknown): v is LedgerEvent =>
isRecord(v) &&
@@ -127,13 +122,11 @@ export const graphStore: GraphStore = {
appendNode(node: GraphNode): void {
appendLine(LEDGER_FILES.nodes, node);
},
/** Raw node lines in append order (ids are unique — no merging needed).
* Legacy lines without a `name` get one derived from the statement. */
/** Raw node lines in append order. A node id may appear on more than
* one line (a later session's LINK-append: phrasings + provenance); the
* derivation (core/graph.ts getGraph) merges lines by id. */
readNodes(): GraphNode[] {
return readValid(LEDGER_FILES.nodes, isStoredGraphNode).map((n) => ({
...n,
name: n.name ?? nameFromStatement(n.statement),
}));
return readValid(LEDGER_FILES.nodes, isGraphNode);
},
/** Append one edge ASSERTION; readers merge lines by (from,to,type). */