extraction cleanup: events ledger dies, markdown source view, auditor-as-linker (2026-09-27 rulings)
The graph becomes raw reasoning chains only. Quiz outcomes and understanding observations are no longer formalized; assessment moves to the future practice module. Events ledger removed end-to-end: LedgerEvent/LedgerEventKind/ UnderstandingLevel/ExtractionAttribution/AuditResult.dropped and snapshot nodeObservations deleted from contracts; quizEvent builder and the events step of mergeFindings gone; appendEvent/readEvents dropped from the port and store; LEDGER_FILES loses events.jsonl with a one-time purge of a stale file next to the nodes ledger; graph-deletion and the delete dialog lose event purge + quiz-answer impact counts; GraphSessionProposition becomes a pure birth list. Whole-session markdown source view: new core/session-markdown.ts is the shared formatting engine (id-anchored speaker blocks, quiz/ recall/question blocks, tool one-liners) — the extractor's prompt view and the future user-facing export render from one engine. Coverage fingerprints deliberately stay on the old line format (content-hash semantics; re-formatting never re-opens covered history). The whole session goes to the extractor — no phase cut. Auditor refocused as the identity linker: never sees the source; explores the graph via new auditor-specific tools in agent/tools/graph.ts (find — glob over name/statement/aliases; list_sessions; list_propositions with full statement text; no timestamps) instead of an injected full-graph dump; equivalence bias — paraphrases merge, only genuinely distinct propositions stay apart; no drop channel, the extractor is fully trusted; may draw bridging edges (typed follows/supports) between existing nodes and the batch, resolved by the merger with chat-scoped message-less provenance. Agentic loop, maxTurns 16. UI: the node detail card gains an edges section above the sessions list (follows-from / supported-by / what-follows / what-it-supports, rendered only when present, click-to-select neighbors); the understanding-note chips are gone. Prober list_propositions reworded to what a session covered — names only, no levels. docs/architecture.md + README updated to the as-built pipeline. Gates: typecheck, 1059/1059, lint, build.
This commit is contained in:
@@ -9,7 +9,7 @@ import type { ThinkingLevel } from "@earendil-works/pi-agent-core";
|
||||
import type { LhMessage } from "../native-messages.ts";
|
||||
import type { Phase3 } from "../phases.ts";
|
||||
import type { PlanNodeStatus, PlanTree } from "./plan.ts";
|
||||
import type { EmissionStats, UnderstandingLevel } from "./graph.ts";
|
||||
import type { EmissionStats } from "./graph.ts";
|
||||
|
||||
// ─── Chat records (the session store's on-disk format) ───────────────────────
|
||||
|
||||
|
||||
+43
-98
@@ -5,10 +5,11 @@
|
||||
* 2026-09-02 redesign): every node is a TRUE proposition; edges are follows
|
||||
* (derivable) and supports (grounds without entailing); provenance is
|
||||
* message-granular; extraction is free-form and post-hoc, fully decoupled
|
||||
* from the session's plan tree; the event ledger records quiz outcomes plus
|
||||
* the extractor's per-proposition understanding observations. Anything not
|
||||
* yet decided (the state model, strength derivation, understanding
|
||||
* treatment, selection strategies) is deliberately NOT here — transitional
|
||||
* from the session's plan tree. The graph is an extraction of the reasoning
|
||||
* chains from all teaching sessions (and imported notes) — NOTHING else:
|
||||
* no quiz outcomes, no understanding assessments (2026-09-27 cleanup —
|
||||
* understanding becomes practice-module territory). Anything not
|
||||
* yet decided (the state model, strength derivation, selection strategies) is deliberately NOT here — transitional
|
||||
* stubs carry the renderer until those land.
|
||||
*
|
||||
* Part of the domain-split contracts (src/core/contracts/): each domain file
|
||||
@@ -31,13 +32,13 @@ export interface EmissionStats {
|
||||
nodes: number;
|
||||
/** New edge assertions appended. */
|
||||
edges: number;
|
||||
/** Ledger events rebuilt (quiz outcomes + observations). */
|
||||
events: number;
|
||||
}
|
||||
|
||||
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
|
||||
* edges record every (chat, message) where they came up, deep-linkable to
|
||||
* the exact transcript point. */
|
||||
* the exact transcript point. Auditor-drawn bridging edges carry the chat
|
||||
* but NO message anchor (nothing in the transcript asserts them — the
|
||||
* auditor is the asserter); messageId is "" there. */
|
||||
export interface ProvenancePoint {
|
||||
/** The chat this point anchors to — or a notes source id ("src_…"):
|
||||
* imports carry no message anchors (provenance is the file itself,
|
||||
@@ -50,11 +51,10 @@ export interface ProvenancePoint {
|
||||
* (DECIDED 2026-09-13): personal notes in ARBITRARY formats run through the
|
||||
* same extractor → auditor → merger pipeline, with the file itself as the
|
||||
* provenance anchor. There is no chat: the detail card shows the file path
|
||||
* as a STATIC entry (not a link), and understanding observations still
|
||||
* apply (judged from the author's own voice in the notes). The run's usage
|
||||
* log lives here, mirroring ChatRecord.extractions. Ids are "src_" + hash
|
||||
* of the file path — deterministic, so re-importing the same file re-runs
|
||||
* extraction against the same source id (mappings/events dedupe by it). */
|
||||
* as a STATIC entry (not a link). The run's usage log lives here,
|
||||
* mirroring ChatRecord.extractions. Ids are "src_" + hash of the file
|
||||
* path — deterministic, so re-importing the same file re-runs extraction
|
||||
* against the same source id (mappings dedupe by it). */
|
||||
export interface NotesSource {
|
||||
id: string;
|
||||
filePath: string;
|
||||
@@ -149,39 +149,9 @@ export interface CoverageEntry {
|
||||
foldedAt: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Event kinds (DECIDED 2026-09-02): quiz outcomes only — correct / miss /
|
||||
* dont-know — plus `observation`, the extractor's per-proposition
|
||||
* understanding note joining the ledger as a session-scoped observation
|
||||
* (named-word scale; treatment pending the state-model ideation). No
|
||||
* `taught` events at all; no misconception kind — a misconception is a
|
||||
* TRUE correction in the graph, and the wrongness lives here, in the
|
||||
* ledger, until the state model exists.
|
||||
*/
|
||||
export type LedgerEventKind = "correct" | "miss" | "dont-know" | "observation";
|
||||
|
||||
export interface LedgerEvent {
|
||||
/** null = not yet attributed. */
|
||||
nodeId: string | null;
|
||||
kind: LedgerEventKind;
|
||||
/** The chat that produced this evidence — or a notes source id
|
||||
* ("src_…") for imported-notes observations. */
|
||||
chatId: string;
|
||||
at: string;
|
||||
/** Idempotency key: the quiz part's toolCallId, or `obs:<sourceId>` for
|
||||
* understanding observations — re-emission of the same chat must never
|
||||
* double-record an event. */
|
||||
ref?: string;
|
||||
/** The learner's quiz note (diagnostic gold), or the understanding
|
||||
* LEVEL word for observations — a session-relative judgment. */
|
||||
note?: string;
|
||||
/** The quiz question text, when nodeId is null (later attribution input). */
|
||||
question?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Store format (src/store/graph.ts): JSONL ledgers at config.LEDGER_FILES
|
||||
* (nodes/edges/events/mappings/sessions), all append-only.
|
||||
* (nodes/edges/mappings/sessions), all append-only.
|
||||
*
|
||||
* nodes.jsonl one line per node — the FIRST line for an id defines
|
||||
* name/statement/createdAt; later lines for the
|
||||
@@ -198,27 +168,23 @@ export interface LedgerEvent {
|
||||
* (never extract the same turns twice).
|
||||
*
|
||||
* The EXTRACTION PIPELINE is the sole graph writer (extractor → auditor →
|
||||
* merger), tree-blind and post-hoc: the extractor reads only the transcript
|
||||
* and proposes propositions/edges/quiz-attribution with verbatim evidence;
|
||||
* the auditor sees the extractor's output PLUS the current graph, owns
|
||||
* faithfulness (evidence check) and identity (link-or-create: point each
|
||||
* proposition at the existing node it is, or let it be created — recording
|
||||
* the session's phrasings as aliases); the merger (core/graph.ts,
|
||||
* deterministic) applies the verdicts and appends.
|
||||
* merger), tree-blind and post-hoc (2026-09-27 cleanup — clear separation
|
||||
* of work): the extractor is the ONLY stage that sees the raw source
|
||||
* (whole session as readable markdown, or raw notes) and extracts the
|
||||
* reasoning chains — propositions (lonely relevant ones included) plus
|
||||
* follows/supports edges, with verbatim evidence; the auditor NEVER sees
|
||||
* the source — its job is preparing the merge: identity verdicts
|
||||
* (link-or-create, merging equivalent forms of the same proposition) via
|
||||
* graph-exploration tools (find / list_sessions / list_propositions), and
|
||||
* it may draw BRIDGING edges from existing graph nodes to the new
|
||||
* propositions; the merger (core/graph.ts, deterministic) applies the
|
||||
* verdicts and appends. Quiz events and understanding observations are
|
||||
* GONE — the pipeline is the only session→graph bridge, and it carries
|
||||
* propositions + edges, nothing else.
|
||||
*/
|
||||
|
||||
// ─── Extraction findings (extractor output → audit → merger input) ───────────
|
||||
|
||||
/**
|
||||
* The extractor's standardized per-session understanding scale (DECIDED
|
||||
* 2026-09-08): a CLOSED named-word list — no numbers, no prose, no
|
||||
* justifications (a justification, if the agent wants one, is dropped;
|
||||
* it is not stored). The level is session- and extraction-run-relative
|
||||
* judgment about the learner, shown next to its session — never a node
|
||||
* property.
|
||||
*/
|
||||
export type UnderstandingLevel = "solid" | "shaky" | "accepted" | "resisted" | "not yet engaged";
|
||||
|
||||
/** One proposed proposition — a node candidate (or a re-encounter of one). */
|
||||
export interface ExtractedProposition {
|
||||
/** Extractor-local id ("p1", …) referenced by edges + attribution. */
|
||||
@@ -231,7 +197,8 @@ export interface ExtractedProposition {
|
||||
/** The minimal proposition — a true, self-contained, world-facing
|
||||
* statement, at full precision. No learner / teacher / session
|
||||
* references: misconceptions are stated as the true correction, and
|
||||
* wrongness lives in the event ledger, never in the node. */
|
||||
* wrongness is never recorded in the graph (2026-09-27: the event
|
||||
* ledger is gone). */
|
||||
statement: string;
|
||||
/** Verbatim phrasings the session used. */
|
||||
aliases: string[];
|
||||
@@ -239,17 +206,17 @@ export interface ExtractedProposition {
|
||||
messages: string[];
|
||||
/** Verbatim transcript quotes backing the proposition. */
|
||||
evidence: string[];
|
||||
/** The extractor's understanding level for this proposition IN THIS
|
||||
* session — one word from the UnderstandingLevel scale. Joins the
|
||||
* ledger as an observation event (note = the level word); treatment
|
||||
* pending the state-model ideation. */
|
||||
understanding?: UnderstandingLevel;
|
||||
}
|
||||
|
||||
/** One proposed relation between two propositions. */
|
||||
/** One proposed relation. Endpoints are extractor-local proposition ids
|
||||
* ("p1", …); the AUDITOR may also emit bridging edges whose from/to is an
|
||||
* EXISTING graph node id (connecting the new content to what the graph
|
||||
* already holds) — the merger resolves both. */
|
||||
export interface ExtractedEdge {
|
||||
/** Proposition id that is followed / supported. */
|
||||
/** Proposition id (or, for auditor-drawn bridges, a graph node id)
|
||||
* that is followed / supported. */
|
||||
from: string;
|
||||
/** Proposition id (or graph node id) that follows / is supported. */
|
||||
to: string;
|
||||
type: EdgeType;
|
||||
/** The message ids where this relation was asserted — provenance. */
|
||||
@@ -258,26 +225,10 @@ export interface ExtractedEdge {
|
||||
evidence: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* The extractor's quiz attribution — its ONLY say about quiz events. Event
|
||||
* kinds, timestamps, idempotency refs, notes and question texts are all
|
||||
* rebuilt deterministically by the merger from the transcript's quiz tool
|
||||
* parts; the LLM supplies exactly the judgment call: WHICH proposition each
|
||||
* quiz tested. Tree-blind, so EVERY quiz attributes here (there is no tree
|
||||
* mapping to lean on) — retroactively against the whole transcript.
|
||||
*/
|
||||
export interface ExtractionAttribution {
|
||||
/** quiz toolCallId → proposition id; null / missing = unattributed (the
|
||||
* event lands with nodeId null + the question text, a later-
|
||||
* attribution input). */
|
||||
quiz: Record<string, string | null>;
|
||||
}
|
||||
|
||||
/** The extractor's output shape. */
|
||||
/** The extractor's output shape: the raw reasoning chains. */
|
||||
export interface SessionFindings {
|
||||
propositions: ExtractedProposition[];
|
||||
edges: ExtractedEdge[];
|
||||
attribution: ExtractionAttribution;
|
||||
}
|
||||
|
||||
/** The auditor's verdict on one proposition: the findings, plus `linkTo` —
|
||||
@@ -286,20 +237,19 @@ export interface AuditedProposition extends ExtractedProposition {
|
||||
/** The existing GraphNode id this proposition IS — the auditor's
|
||||
* identity verdict. On link, the session's phrasings (statement,
|
||||
* aliases) join the node's aliases and the provenance unions.
|
||||
* Conservative: when unsure, leave it out — duplicates over wrong
|
||||
* merges. */
|
||||
* EQUIVALENCE BIAS (2026-09-27): paraphrases, rewordings, and
|
||||
* compressions of the same claim MERGE — do not create a duplicate
|
||||
* because the wording differs. Only genuinely distinct propositions
|
||||
* stay apart. */
|
||||
linkTo?: string;
|
||||
}
|
||||
|
||||
/** The auditor's output: the verified findings (with identity verdicts),
|
||||
* plus the drops (the audit trail of what was rejected and why). */
|
||||
/** The auditor's output: the findings with identity verdicts, plus any
|
||||
* bridging edges drawn to existing graph nodes. No drop channel — the
|
||||
* auditor never rejects; the extractor is fully trusted (2026-09-27). */
|
||||
export interface AuditResult {
|
||||
propositions: AuditedProposition[];
|
||||
edges: ExtractedEdge[];
|
||||
attribution: ExtractionAttribution;
|
||||
/** Dropped candidates: proposition id, edge "from→to", or attribution
|
||||
* toolCallId — with the reason. */
|
||||
dropped: { ref: string; reason: string }[];
|
||||
}
|
||||
|
||||
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
|
||||
@@ -328,11 +278,6 @@ export interface GraphSnapshot {
|
||||
fileSources: Record<string, string>;
|
||||
/** nodeId → chatIds that touched the node (click-through provenance). */
|
||||
nodeSessions: Record<string, string[]>;
|
||||
/** nodeId → chatId → that session's understanding observation (the
|
||||
* extractor's per-session named-word note; session-relative data,
|
||||
* shown next to the session list — the confidence history). Multiple
|
||||
* notes for one (node, chat) collapse to the latest. */
|
||||
nodeObservations: Record<string, Record<string, string>>;
|
||||
/** Structural communities (Louvain over follows+supports), size ≥ 3,
|
||||
* joined with persisted LLM labelings. Purely structural — never
|
||||
* depends on layout. */
|
||||
|
||||
@@ -13,17 +13,16 @@
|
||||
*
|
||||
* preserve-graph the chat's extraction bookkeeping dies — its coverage
|
||||
* rows and its legacy checkpoint lines — and NOTHING else:
|
||||
* nodes, edges, events, mappings stay byte-identical.
|
||||
* nodes, edges, mappings stay byte-identical.
|
||||
* The graph keeps the session's contributions; the
|
||||
* snapshot simply no longer resolves the dead chatId to
|
||||
* a title (nodeSessions/nodeObservations filter unknown
|
||||
* ids at read time — no UI ghosting).
|
||||
* a title (nodeSessions filters unknown ids at read
|
||||
* time — no UI ghosting).
|
||||
*
|
||||
* delete-nodes "this session never happened": every trace of the
|
||||
* chat leaves the ledgers — its provenance points on
|
||||
* every node and edge line, its link lines (their
|
||||
* phrasings die with them), its quiz events and
|
||||
* observations, its mapping lines (the audit trail), its
|
||||
* phrasings die with them), its mapping lines (the audit trail), its
|
||||
* coverage/checkpoint rows — and orphans die: a node
|
||||
* whose references all came from this chat disappears
|
||||
* entirely, pruned from the positions and embedding
|
||||
@@ -58,7 +57,6 @@ import type {
|
||||
CoverageEntry,
|
||||
GraphEdge,
|
||||
GraphNode,
|
||||
LedgerEvent,
|
||||
NodeMapping,
|
||||
SessionCheckpoint,
|
||||
} from "./contracts.ts";
|
||||
@@ -76,8 +74,6 @@ export interface ChatDeletionStats {
|
||||
nodesRemoved: number;
|
||||
/** Edge assertion LINES removed (empty-provenance or dead endpoint). */
|
||||
edgesRemoved: number;
|
||||
/** Events removed (quiz outcomes + observations). */
|
||||
eventsRemoved: number;
|
||||
/** Mapping lines removed (the chat's audit trail). */
|
||||
mappingsRemoved: number;
|
||||
/** Coverage rows removed. */
|
||||
@@ -89,7 +85,6 @@ export interface ChatDeletionStats {
|
||||
const EMPTY_STATS: ChatDeletionStats = {
|
||||
nodesRemoved: 0,
|
||||
edgesRemoved: 0,
|
||||
eventsRemoved: 0,
|
||||
mappingsRemoved: 0,
|
||||
coverageRemoved: 0,
|
||||
checkpointsRemoved: 0,
|
||||
@@ -126,7 +121,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
// ── delete-nodes: the full purge ──────────────────────────────────────
|
||||
const nodeLines = graphStore.readNodes();
|
||||
const edgeLines = graphStore.readEdges();
|
||||
const events = graphStore.readEvents();
|
||||
const mappings = graphStore.readMappings();
|
||||
|
||||
// Does the chat touch the graph at all? (A never-extracted chat still
|
||||
@@ -136,7 +130,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
const touched =
|
||||
nodeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) ||
|
||||
edgeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) ||
|
||||
events.some((event) => event.chatId === chatId) ||
|
||||
mappings.some((m) => m.chatId === chatId);
|
||||
if (!touched) {
|
||||
if (stats.coverageRemoved > 0 || stats.checkpointsRemoved > 0) {
|
||||
@@ -149,7 +142,7 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
// remains — another chat/src's provenance point on any of its lines,
|
||||
// a surviving mapping line (notes-front-door nodes carry EMPTY
|
||||
// provenance: the file IS the anchor, recorded through the mapping
|
||||
// line), or a surviving event naming it. (Edge endpoints are deliberately
|
||||
// line). (Edge endpoints are deliberately
|
||||
// NOT survival references: an edge's asserting session always left
|
||||
// provenance on its endpoints, so endpoint-only liveness cannot
|
||||
// occur in a consistent ledger — and counting it would keep nodes
|
||||
@@ -163,9 +156,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
for (const mapping of mappings) {
|
||||
if (mapping.chatId !== chatId) live.add(mapping.graphNodeId);
|
||||
}
|
||||
for (const event of events) {
|
||||
if (event.chatId !== chatId && event.nodeId !== null) live.add(event.nodeId);
|
||||
}
|
||||
|
||||
// 2. Node lines. Per-line (each line is ONE extraction run's append, so
|
||||
// per-session alias provenance exists at line granularity):
|
||||
@@ -219,19 +209,16 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
outEdges.push(points.length === line.provenance.length ? line : { ...line, provenance: points });
|
||||
}
|
||||
|
||||
// 4. Events, mappings: the chat's rows die outright.
|
||||
const outEvents = events.filter((event) => event.chatId !== chatId);
|
||||
// 4. Mappings: the chat's rows die outright.
|
||||
const outMappings = mappings.filter((m) => m.chatId !== chatId);
|
||||
|
||||
stats.nodesRemoved = [...new Set(nodeLines.map((line) => line.id))].filter((id) => !live.has(id)).length;
|
||||
stats.edgesRemoved = edgeLines.length - outEdges.length;
|
||||
stats.eventsRemoved = events.length - outEvents.length;
|
||||
stats.mappingsRemoved = mappings.length - outMappings.length;
|
||||
|
||||
graphStore.rewriteLedgers({
|
||||
nodes: outNodes,
|
||||
edges: outEdges,
|
||||
events: outEvents,
|
||||
mappings: outMappings,
|
||||
coverage,
|
||||
checkpoints,
|
||||
@@ -264,8 +251,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
*
|
||||
* - every line of the node (create + link lines)
|
||||
* - every EDGE touching it (from or to), whatever session asserted it
|
||||
* - every EVENT naming it (quiz outcomes + observations), from every
|
||||
* session — the evidence history dies with the thing it was about
|
||||
* - every mapping line that anchored it (the audit trail; for a
|
||||
* notes-born node the file IS the anchor — its mapping line is the
|
||||
* only one)
|
||||
@@ -287,20 +272,16 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
|
||||
export interface NodeDeletionStats {
|
||||
/** Edge assertion LINES removed (every edge touching the node). */
|
||||
edgesRemoved: number;
|
||||
/** Events removed (quiz outcomes + observations naming the node). */
|
||||
eventsRemoved: number;
|
||||
/** Mapping lines removed (the node's audit trail). */
|
||||
mappingsRemoved: number;
|
||||
}
|
||||
|
||||
/** The confirm dialog's pre-flight facts (the GET on the same node id):
|
||||
* what full death would take with it. sessions is the deduped set of
|
||||
* chats/notes sources that reference the node — provenance points,
|
||||
* mapping lines, and events all count. */
|
||||
* chats/notes sources that reference the node — provenance points and
|
||||
* mapping lines count. */
|
||||
export interface NodeDeletionImpact {
|
||||
edges: number;
|
||||
quizEvents: number;
|
||||
observations: number;
|
||||
sessions: string[];
|
||||
}
|
||||
|
||||
@@ -313,7 +294,6 @@ function nodeLinesOf(graphStore: GraphStore, nodeId: string): GraphNode[] {
|
||||
export function nodeDeletionImpact(graphStore: GraphStore, nodeId: string): NodeDeletionImpact | null {
|
||||
if (nodeLinesOf(graphStore, nodeId).length === 0) return null;
|
||||
const edges = graphStore.readEdges().filter((line) => line.from === nodeId || line.to === nodeId);
|
||||
const events = graphStore.readEvents().filter((event) => event.nodeId === nodeId);
|
||||
const sessions = new Set<string>();
|
||||
for (const line of nodeLinesOf(graphStore, nodeId)) {
|
||||
for (const pt of line.provenance) sessions.add(pt.chatId);
|
||||
@@ -321,11 +301,8 @@ export function nodeDeletionImpact(graphStore: GraphStore, nodeId: string): Node
|
||||
for (const mapping of graphStore.readMappings()) {
|
||||
if (mapping.graphNodeId === nodeId) sessions.add(mapping.chatId);
|
||||
}
|
||||
for (const event of events) sessions.add(event.chatId);
|
||||
return {
|
||||
edges: edges.length,
|
||||
quizEvents: events.filter((event) => event.kind !== "observation").length,
|
||||
observations: events.filter((event) => event.kind === "observation").length,
|
||||
sessions: [...sessions].sort(),
|
||||
};
|
||||
}
|
||||
@@ -341,11 +318,9 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
|
||||
if (nodeLines.length === 0) return null;
|
||||
|
||||
const edgeLines = graphStore.readEdges();
|
||||
const events = graphStore.readEvents();
|
||||
const mappings = graphStore.readMappings();
|
||||
|
||||
const outEdges = edgeLines.filter((line) => line.from !== nodeId && line.to !== nodeId);
|
||||
const outEvents = events.filter((event) => event.nodeId !== nodeId);
|
||||
const outMappings = mappings.filter((m) => m.graphNodeId !== nodeId);
|
||||
|
||||
// Only the touched ledgers: the node's own lines always die (its file
|
||||
@@ -354,7 +329,6 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
|
||||
graphStore.rewriteLedgers({
|
||||
nodes: outNodes,
|
||||
...(outEdges.length !== edgeLines.length ? { edges: outEdges } : {}),
|
||||
...(outEvents.length !== events.length ? { events: outEvents } : {}),
|
||||
...(outMappings.length !== mappings.length ? { mappings: outMappings } : {}),
|
||||
});
|
||||
|
||||
@@ -367,7 +341,6 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
|
||||
|
||||
return {
|
||||
edgesRemoved: edgeLines.length - outEdges.length,
|
||||
eventsRemoved: events.length - outEvents.length,
|
||||
mappingsRemoved: mappings.length - outMappings.length,
|
||||
};
|
||||
}
|
||||
|
||||
+73
-245
@@ -2,45 +2,45 @@
|
||||
* The knowledge graph: merger (the deterministic final stage of the
|
||||
* extraction pipeline — vision §1 construction) + derivation (reads).
|
||||
*
|
||||
* The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc:
|
||||
* The extraction pipeline is the SOLE graph writer, tree-blind and post-hoc
|
||||
* (2026-09-27 cleanup — clear separation of work):
|
||||
*
|
||||
* extractor (LLM) → SessionFindings: propositions, edges, quiz
|
||||
* attribution, per-proposition understanding notes —
|
||||
* each with verbatim evidence, from the transcript ALONE
|
||||
* auditor (LLM) → AuditResult: the verified subset, with the IDENTITY
|
||||
* VERDICT per proposition (linkTo an existing node, or
|
||||
* create), and the drops for the log
|
||||
* extractor (LLM) → SessionFindings: the raw reasoning chains —
|
||||
* propositions (lonely relevant ones included) and
|
||||
* follows/supports edges, each with verbatim evidence,
|
||||
* from the source ALONE (whole session as markdown,
|
||||
* or raw notes). Fully trusted: no stage re-checks it.
|
||||
* auditor (LLM) → AuditResult: identity verdicts (linkTo an existing
|
||||
* node, or create) via graph-exploration tools, plus
|
||||
* any BRIDGING edges it draws between existing graph
|
||||
* nodes and the new propositions. Never sees the
|
||||
* source; never drops.
|
||||
* merger (HERE) → mergeFindings(): deterministic ledger appends
|
||||
*
|
||||
* The merger is deliberately the ONLY stage that touches the ledgers, and it
|
||||
* is pure code: the LLM supplies exactly the judgment calls (what the
|
||||
* propositions are, what follows/supports what, which proposition a quiz
|
||||
* tested, whether a proposition IS an existing node), while event
|
||||
* kinds/timestamps/refs/provenance — anything the model could silently
|
||||
* mis-copy — are rebuilt deterministically from the transcript's tool parts
|
||||
* and the message ids in the findings. Idempotent + incremental: refs are
|
||||
* chat-scoped idempotency keys, edges dedupe by (from,to,type)-already-
|
||||
* asserted-by-this-chat, mapping lines dedupe by (chat, sourceId), and
|
||||
* sessions.jsonl checkpoints which chats have been extracted (never the
|
||||
* same turns twice).
|
||||
* propositions are, what follows/supports what, whether a proposition IS an
|
||||
* existing node), while timestamps/provenance — anything the model could
|
||||
* silently mis-copy — are rebuilt deterministically. Idempotent +
|
||||
* incremental: edges dedupe by (from,to,type)-already-asserted-by-this-chat,
|
||||
* mapping lines dedupe by (chat, sourceId), and coverage entries /
|
||||
* sessions.jsonl checkpoints which messages and chats have been extracted
|
||||
* (never the same turns twice). No events: the events ledger is GONE
|
||||
* (2026-09-27) — the graph carries propositions + edges, nothing else.
|
||||
*
|
||||
* Identity across re-encounters is the AUDITOR's judgment (link-or-create),
|
||||
* not string matching: the conservative v1 exact-statement merge was
|
||||
* replaced by the 2026-09-02 redesign. A re-extracted session's
|
||||
* propositions link to the existing nodes; the link appends a node line
|
||||
* whose phrasings join `aliases` and whose provenance unions (getGraph
|
||||
* merges lines by id).
|
||||
* Identity across re-encounters is the AUDITOR's judgment (link-or-create
|
||||
* with an equivalence bias — paraphrases merge), not string matching.
|
||||
* A re-extracted session's propositions link to the existing nodes; the
|
||||
* link appends a node line whose phrasings join `aliases` and whose
|
||||
* provenance unions (getGraph merges lines by id).
|
||||
*
|
||||
* Derivation below (getGraph / deriveNodeStates / graphSnapshot) is reads
|
||||
* only — everything is recomputed from the ledgers. Every function takes
|
||||
* the storage ports (core/ports.ts) as explicit parameters — pure core, no
|
||||
* adapter imports; the composition root (src/server/stores.ts) wires the fs
|
||||
* adapters in.
|
||||
* Derivation below (getGraph / graphSnapshot) is reads only — everything
|
||||
* is recomputed from the ledgers. Every function takes the storage ports
|
||||
* (core/ports.ts) as explicit parameters — pure core, no adapter imports;
|
||||
* the composition root (src/server/stores.ts) wires the fs adapters in.
|
||||
*/
|
||||
|
||||
import { randomBytes } from "node:crypto";
|
||||
import type { LhMessage } from "./native-messages.ts";
|
||||
import type { ToolResultMessage } from "@earendil-works/pi-ai";
|
||||
import type { CoverageEntry } from "./contracts/graph.ts";
|
||||
import type {
|
||||
AuditResult,
|
||||
@@ -50,13 +50,9 @@ import type {
|
||||
GraphNode,
|
||||
ProvenancePoint,
|
||||
GraphSnapshot,
|
||||
LedgerEvent,
|
||||
LedgerEventKind,
|
||||
UnderstandingLevel,
|
||||
} from "./contracts.ts";
|
||||
import { MIN_CLUSTER_SIZE } from "./clusters.ts";
|
||||
import { birthMappingIndex, birthSessionOf, birthSessionPartition } from "./session-clusters.ts";
|
||||
import type { QuestionAnswerData } from "./question-core.ts";
|
||||
import type { ChatStore, GraphStore } from "./ports.ts";
|
||||
|
||||
/** The storage ports every graph operation needs, passed explicitly. */
|
||||
@@ -71,71 +67,22 @@ function newGraphNodeId(): string {
|
||||
}
|
||||
|
||||
/** What one merge run merges FROM (the caller supplies it — the merger never
|
||||
* reads the chat store): a session's transcript + its coverage checkpoint,
|
||||
* or — for the format-agnostic notes front door (2026-09-13) — `messages:
|
||||
* null`: raw notes carry no tool parts, so no quiz events are rebuilt (quiz
|
||||
* outcomes inside notes surface only through the extractor's understanding
|
||||
* observations), and the whole-file checkpoint stamps the import.
|
||||
* reads the chat store): the coverage checkpoint for a session batch, or —
|
||||
* for the format-agnostic notes front door (2026-09-13) — no coverage at
|
||||
* all: the whole-file checkpoint stamps the import.
|
||||
*
|
||||
* Coverage (vision-extraction-coverage.md, settled 2026-09-15): a chat merge
|
||||
* passes the BATCH it extracted — the caller computed the fingerprints
|
||||
* before the run, from the same snapshot the merger sees — and step 4
|
||||
* appends one CoverageEntry per message on success ("covered" means
|
||||
* exactly this merge happened). `batchTag` prefixes the observation refs so
|
||||
* re-runs of the SAME batch dedupe while a re-opened (mutated) message's
|
||||
* fresh understanding observations land as new events; the notes path
|
||||
* passes neither and keeps the whole-file checkpoint. */
|
||||
* before the run, from the same snapshot the merger sees — and the final
|
||||
* step appends one CoverageEntry per message on success ("covered" means
|
||||
* exactly this merge happened). The notes path passes neither and keeps
|
||||
* the whole-file checkpoint. */
|
||||
export interface MergeSource {
|
||||
messages: LhMessage[] | null;
|
||||
foldedAt: string;
|
||||
/** The batch being merged (chat path): one entry per message in
|
||||
* `messages`, same order — undefined (notes path / coverage-less
|
||||
* callers) → the legacy whole-chat checkpoint is written instead. */
|
||||
/** The batch being merged (chat path): one entry per message in the
|
||||
* extracted batch — undefined (notes path / coverage-less callers) →
|
||||
* the legacy whole-chat checkpoint is written instead. */
|
||||
coverage?: CoverageEntry[];
|
||||
/** The batch's deterministic tag (core/coverage.ts batchTag) — obs refs. */
|
||||
batchTag?: string;
|
||||
}
|
||||
|
||||
// ─── transcript scanning (shared by the merger + trigger) ─────────────────────
|
||||
|
||||
/** A flattened, order-preserving view of every tool call (+ its result
|
||||
* state) in a native transcript. Wave D: the pairing is native — calls
|
||||
* come from assistant content blocks; a result map built from the
|
||||
* toolResult messages carries the outcome (a dangling call is
|
||||
* input-available, an error result is output-error). */
|
||||
interface ToolPartView {
|
||||
name: string;
|
||||
toolCallId: string;
|
||||
state: string;
|
||||
input: unknown;
|
||||
output: unknown;
|
||||
}
|
||||
|
||||
function toolParts(messages: LhMessage[]): ToolPartView[] {
|
||||
const results = new Map<string, ToolResultMessage>();
|
||||
for (const m of messages) {
|
||||
if ((m as { role?: string }).role === "toolResult") {
|
||||
results.set((m as ToolResultMessage).toolCallId, m as ToolResultMessage);
|
||||
}
|
||||
}
|
||||
const out: ToolPartView[] = [];
|
||||
for (const m of messages) {
|
||||
if ((m as { role?: string }).role !== "assistant") continue;
|
||||
const content = (m as { content?: unknown }).content;
|
||||
if (!Array.isArray(content)) continue;
|
||||
for (const block of content as Array<{ type?: string; id?: string; name?: string; arguments?: unknown }>) {
|
||||
if (block.type !== "toolCall" || typeof block.id !== "string") continue;
|
||||
const result = results.get(block.id);
|
||||
out.push({
|
||||
name: block.name ?? "",
|
||||
toolCallId: block.id,
|
||||
state: result ? (result.isError ? "output-error" : "output-available") : "input-available",
|
||||
input: block.arguments,
|
||||
output: result && !result.isError ? result.details : undefined,
|
||||
});
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** (chatId, messageIds) → provenance points; order-preserving, deduped. */
|
||||
@@ -167,21 +114,14 @@ const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
|
||||
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent:
|
||||
* a proposition id already mapped for this chat reuses its resolution.
|
||||
*
|
||||
* Events are rebuilt deterministically from the transcript: quiz outcomes
|
||||
* (correct / miss / dont-know; kinds + timestamps + refs from the tool
|
||||
* parts, nodeId from the extractor's attribution through this run's
|
||||
* mapping) plus one observation event per proposition carrying an
|
||||
* understanding note (ref `obs:<sourceId>`). No taught events, ever.
|
||||
*
|
||||
* `chatId` is the run's source key: a real chat, or a notes source id
|
||||
* ("src_…") for the format-agnostic import path — same ledgers, same
|
||||
* idempotency, one graph. The `source` parameter carries the transcript
|
||||
* being merged (null messages = raw notes: no quiz events) and the
|
||||
* checkpoint stamp; the merger itself is chat-store-free.
|
||||
* idempotency, one graph. The `source` parameter carries the checkpoint
|
||||
* stamp; the merger itself is chat-store-free.
|
||||
*/
|
||||
export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDeps, source: MergeSource): EmissionStats {
|
||||
const { graphStore: ledgers } = deps;
|
||||
const stats: EmissionStats = { nodes: 0, edges: 0, events: 0 };
|
||||
const stats: EmissionStats = { nodes: 0, edges: 0 };
|
||||
|
||||
// 1. Resolution: this run's propositions → graph node ids. Existing
|
||||
// mapping lines for THIS chat are loaded first — the audit trail doubles
|
||||
@@ -238,63 +178,41 @@ export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDep
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Edges. Resolve endpoints through the mapping; dedupe by
|
||||
// (from,to,type) already asserted by THIS chat (an existing assertion
|
||||
// by another chat stays untouched — provenance unions at read time).
|
||||
// 2. Edges. Resolve endpoints through the mapping — with BRIDGING
|
||||
// (2026-09-27): an endpoint the extractor never proposed but that
|
||||
// names an EXISTING graph node is the auditor drawing a connection
|
||||
// between the graph it explored and this batch's propositions. Such
|
||||
// edges carry chat-scoped provenance with NO message anchor (nothing
|
||||
// in the transcript asserts them — the auditor is the asserter);
|
||||
// transcript-articulated edges keep message-granular provenance.
|
||||
// Dedupe by (from,to,type) already asserted by THIS chat (an existing
|
||||
// assertion by another chat stays untouched — provenance unions at
|
||||
// read time).
|
||||
const assertedByChat = new Set<string>();
|
||||
for (const line of ledgers.readEdges()) {
|
||||
const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
|
||||
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
|
||||
}
|
||||
for (const edge of audit.edges) {
|
||||
const from = mapping.get(edge.from);
|
||||
const to = mapping.get(edge.to);
|
||||
if (!from || !to || from === to) continue; // unmapped endpoint — skip
|
||||
const resolve = (endpoint: string): string | undefined =>
|
||||
mapping.get(endpoint) ?? (existingIds.has(endpoint) ? endpoint : undefined);
|
||||
const from = resolve(edge.from);
|
||||
const to = resolve(edge.to);
|
||||
if (!from || !to || from === to) continue; // unresolvable endpoint — skip
|
||||
const bridged = !mapping.has(edge.from) || !mapping.has(edge.to);
|
||||
const key = `${from}\u0000${to}\u0000${edge.type}`;
|
||||
if (assertedByChat.has(key)) continue;
|
||||
ledgers.appendEdge({ from, to, type: edge.type, provenance: pointsOf(chatId, edge.messages) });
|
||||
ledgers.appendEdge({
|
||||
from,
|
||||
to,
|
||||
type: edge.type,
|
||||
provenance: bridged ? [{ chatId, messageId: "" }] : pointsOf(chatId, edge.messages),
|
||||
});
|
||||
assertedByChat.add(key);
|
||||
stats.edges++;
|
||||
}
|
||||
|
||||
// 3. Events, rebuilt deterministically in transcript order. Refs are
|
||||
// chat-scoped idempotency keys (quiz toolCallId, obs:<sourceId>) —
|
||||
// a ref already in the ledger means this evidence was extracted before.
|
||||
const seenRefs = new Set<string>();
|
||||
for (const event of ledgers.readEvents()) {
|
||||
if (event.chatId === chatId && event.ref) seenRefs.add(event.ref);
|
||||
}
|
||||
for (const part of source.messages ? toolParts(source.messages) : []) {
|
||||
if (part.name !== "quiz" || part.state !== "output-available") continue;
|
||||
const event = quizEvent(part, chatId, audit, mapping, now);
|
||||
if (event?.ref !== undefined && !seenRefs.has(event.ref)) {
|
||||
ledgers.appendEvent(event);
|
||||
seenRefs.add(event.ref);
|
||||
stats.events++;
|
||||
}
|
||||
}
|
||||
// Understanding observations: one per proposition carrying a note,
|
||||
// resolved to the node the verdict mapped it to. Refs carry the batch
|
||||
// tag when there is one: run-local source ids ("p1"…) shift position
|
||||
// between runs, so a bare obs:p1 could never both dedupe a re-run of
|
||||
// the same batch AND let a re-opened message's fresh level through.
|
||||
for (const p of audit.propositions) {
|
||||
if (p.understanding === undefined || p.understanding.length === 0) continue;
|
||||
const ref = source.batchTag !== undefined ? `obs:${source.batchTag}:${p.id}` : `obs:${p.id}`;
|
||||
if (seenRefs.has(ref)) continue;
|
||||
ledgers.appendEvent({
|
||||
nodeId: mapping.get(p.id) ?? null,
|
||||
kind: "observation",
|
||||
chatId,
|
||||
at: now,
|
||||
ref,
|
||||
note: p.understanding,
|
||||
});
|
||||
seenRefs.add(ref);
|
||||
stats.events++;
|
||||
}
|
||||
|
||||
// 4. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md):
|
||||
// 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md):
|
||||
// only now, after every append succeeded, is the batch covered. The
|
||||
// chat path appends one CoverageEntry per message (per-batch atomic —
|
||||
// an abort or failure mid-run leaves earlier batches covered, this
|
||||
@@ -308,50 +226,11 @@ export function mergeFindings(chatId: string, audit: AuditResult, deps: GraphDep
|
||||
return stats;
|
||||
}
|
||||
|
||||
/** Build the ledger event for one answered quiz tool part (or null to skip).
|
||||
* Deterministic in everything except the attribution-supplied nodeId. */
|
||||
function quizEvent(
|
||||
part: ToolPartView,
|
||||
chatId: string,
|
||||
audit: AuditResult,
|
||||
mapping: Map<string, string>,
|
||||
now: string,
|
||||
): LedgerEvent | null {
|
||||
const out = part.output;
|
||||
if (typeof out !== "object" || out === null) return null;
|
||||
const answer = out as Partial<QuestionAnswerData>;
|
||||
if (answer.kind !== "quiz") return null; // ask questions are not graded
|
||||
if (answer.dismissed === true) return null; // no answer — no evidence
|
||||
|
||||
let kind: LedgerEventKind;
|
||||
if (answer.dontKnow === true) kind = "dont-know";
|
||||
else if (answer.correct === true) kind = "correct";
|
||||
else kind = "miss";
|
||||
|
||||
// Tree-blind: EVERY quiz attributes through the extractor's judgment —
|
||||
// the vision's retroactive attribution, with no tree mapping to lean on.
|
||||
const attributedId = audit.attribution.quiz[part.toolCallId];
|
||||
const nodeId = attributedId ? (mapping.get(attributedId) ?? null) : null;
|
||||
|
||||
const event: LedgerEvent = {
|
||||
nodeId,
|
||||
kind,
|
||||
chatId,
|
||||
at: typeof answer.answeredAt === "string" ? answer.answeredAt : now,
|
||||
ref: part.toolCallId,
|
||||
};
|
||||
if (typeof answer.note === "string" && answer.note.length > 0) event.note = answer.note;
|
||||
// The question text is the attribution input for unattributed quizzes.
|
||||
if (nodeId === null && typeof answer.questionText === "string") event.question = answer.questionText;
|
||||
return event;
|
||||
}
|
||||
|
||||
// ─── derivation (reads only — everything below is recomputable) ─────────────
|
||||
|
||||
export interface DerivedGraph {
|
||||
nodes: GraphNode[];
|
||||
edges: GraphEdge[];
|
||||
events: LedgerEvent[];
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -396,14 +275,14 @@ export function getGraph(graphStore: GraphStore): DerivedGraph {
|
||||
merged.push(edge);
|
||||
}
|
||||
}
|
||||
return { nodes: [...byId.values()], edges: merged, events: graphStore.readEvents() };
|
||||
return { nodes: [...byId.values()], edges: merged };
|
||||
}
|
||||
|
||||
// ─── the /api/graph payload ──────────────────────────────────────────────────
|
||||
|
||||
/** Everything the home graph view needs, in one fetch (shape: contracts). */
|
||||
export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
const { nodes, edges, events } = getGraph(deps.graphStore);
|
||||
const { nodes, edges } = getGraph(deps.graphStore);
|
||||
|
||||
const nodeChats = new Map<string, Set<string>>();
|
||||
const touch = (nodeId: string, chatId: string): void => {
|
||||
@@ -424,9 +303,6 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
touch(edge.to, pt.chatId);
|
||||
}
|
||||
}
|
||||
for (const event of events) {
|
||||
if (event.nodeId !== null) touch(event.nodeId, event.chatId);
|
||||
}
|
||||
|
||||
// Labels for every source the graph can point back at: real chats
|
||||
// (title, deep-linkable) and imported notes files (path, static — the
|
||||
@@ -453,27 +329,6 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
if (ids.length > 0) nodeSessions[nodeId] = ids;
|
||||
}
|
||||
|
||||
// Understanding observations per (node, chat) — the extractor's
|
||||
// session-relative note (latest wins within a chat). Read-time join for
|
||||
// the detail card: the confidence history next to the session list.
|
||||
const nodeObservations: Record<string, Record<string, string>> = {};
|
||||
const latestObs = new Map<string, { at: string; note: string; index: number }>();
|
||||
events.forEach((event, index) => {
|
||||
if (event.kind !== "observation" || event.nodeId === null) return;
|
||||
if (!event.note) return;
|
||||
const key = `${event.nodeId}\u0000${event.chatId}`;
|
||||
const current = latestObs.get(key);
|
||||
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
|
||||
latestObs.set(key, { at: event.at, note: event.note, index });
|
||||
}
|
||||
});
|
||||
for (const [key, { note }] of latestObs) {
|
||||
const sep = key.indexOf("\u0000");
|
||||
const nodeId = key.slice(0, sep);
|
||||
const chatId = key.slice(sep + 1);
|
||||
(nodeObservations[nodeId] ??= {})[chatId] = note;
|
||||
}
|
||||
|
||||
// BIRTH-SESSION CLUSTERS (wave 7, owner-locked 2026-09-23): embeddings
|
||||
// are dead — a cluster is the SESSION THAT CREATED ITS PROPOSITIONS,
|
||||
// derived on read from provenance (core/session-clusters.ts). The label
|
||||
@@ -502,7 +357,6 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
|
||||
sessions,
|
||||
fileSources,
|
||||
nodeSessions,
|
||||
nodeObservations,
|
||||
clusters,
|
||||
...(Object.keys(positions).length > 0 ? { positions } : {}),
|
||||
};
|
||||
@@ -577,48 +431,22 @@ export function graphSessionIndex(deps: GraphDeps): GraphSessionEntry[] {
|
||||
);
|
||||
}
|
||||
|
||||
/** One birth proposition of one session, for the prober's per-session
|
||||
* view. `understanding` is the extractor's latest observation for
|
||||
* (node, session) — the named-word scale (solid/shaky/accepted/resisted/
|
||||
* not yet engaged) — and `evaluatedAt` is THAT event's own timestamp
|
||||
* (owner ruling 2026-09-23: quiz outcomes are evidence in the ledger,
|
||||
* but they never move the evaluation date — the level's event does). */
|
||||
/** One birth proposition of one session — WHAT WAS COVERED there (2026-09-27
|
||||
* cleanup: no understanding levels; assessment is practice-module
|
||||
* territory and nothing writes it until that module exists). */
|
||||
export interface GraphSessionProposition {
|
||||
nodeId: string;
|
||||
/** The compact handle — the same name the canvas shows. */
|
||||
name: string;
|
||||
understanding: UnderstandingLevel | null;
|
||||
/** The `at` of the event that minted `understanding`; null when the
|
||||
* session never evaluated this proposition. */
|
||||
evaluatedAt: string | null;
|
||||
}
|
||||
|
||||
/** The propositions BORN in one session, with the session's latest
|
||||
* understanding of each. Latest observation wins by `at`, then by
|
||||
* ledger order (the same tie rule the snapshot's nodeObservations
|
||||
* uses). Members sorted by node id — deterministic. */
|
||||
/** The propositions BORN in one session — what that session first
|
||||
* articulated. Members sorted by node id — deterministic. */
|
||||
export function graphSessionPropositions(deps: GraphDeps, sessionId: string): GraphSessionProposition[] {
|
||||
const mappingFallback = birthMappingIndex(deps.graphStore.readMappings());
|
||||
const born = getGraph(deps.graphStore).nodes.filter((node) => birthSessionOf(node, mappingFallback) === sessionId);
|
||||
const latest = new Map<string, { at: string; note: string; index: number }>();
|
||||
deps.graphStore.readEvents().forEach((event, index) => {
|
||||
if (event.kind !== "observation" || event.nodeId === null || event.chatId !== sessionId) return;
|
||||
if (!event.note) return;
|
||||
const current = latest.get(event.nodeId);
|
||||
if (!current || event.at > current.at || (event.at === current.at && index > current.index)) {
|
||||
latest.set(event.nodeId, { at: event.at, note: event.note, index });
|
||||
}
|
||||
});
|
||||
return born
|
||||
.map((node) => {
|
||||
const observation = latest.get(node.id);
|
||||
return {
|
||||
nodeId: node.id,
|
||||
name: node.name,
|
||||
understanding: (observation?.note as UnderstandingLevel | undefined) ?? null,
|
||||
evaluatedAt: observation?.at ?? null,
|
||||
};
|
||||
})
|
||||
.map((node) => ({ nodeId: node.id, name: node.name }))
|
||||
.sort((a, b) => (a.nodeId < b.nodeId ? -1 : 1));
|
||||
}
|
||||
|
||||
|
||||
@@ -18,7 +18,6 @@ import type {
|
||||
GraphNode,
|
||||
JobKind,
|
||||
JobRecord,
|
||||
LedgerEvent,
|
||||
ExtractionRunUsage,
|
||||
NodeMapping,
|
||||
PlanTree,
|
||||
@@ -84,8 +83,6 @@ export interface GraphStore {
|
||||
readEdges(): GraphEdge[];
|
||||
|
||||
/** Append one quiz outcome / understanding observation. */
|
||||
appendEvent(event: LedgerEvent): void;
|
||||
readEvents(): LedgerEvent[];
|
||||
|
||||
/** Append one identity-map line (chat's tree node → graph node). */
|
||||
appendMapping(mapping: NodeMapping): void;
|
||||
@@ -130,7 +127,6 @@ export interface GraphStore {
|
||||
rewriteLedgers(next: {
|
||||
nodes?: GraphNode[];
|
||||
edges?: GraphEdge[];
|
||||
events?: LedgerEvent[];
|
||||
mappings?: NodeMapping[];
|
||||
coverage?: CoverageEntry[];
|
||||
checkpoints?: SessionCheckpoint[];
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
/**
|
||||
* Session → markdown — the ONE formatting engine (owner ruling 2026-09-27).
|
||||
*
|
||||
* The extractor's view of a teaching session and the future user-facing
|
||||
* session export share this engine: a session is rendered as clean,
|
||||
* readable markdown — speaker headers, quiz/check-in blocks, tool notes —
|
||||
* with the message-id anchors preserved (extraction provenance depends on
|
||||
* the extractor reporting the `[id]` tags it saw; `anchors: false` mints
|
||||
* the same document without them, for the user export).
|
||||
*
|
||||
* Pure core: native messages in (LhMessage), string out. Content parity
|
||||
* with the coverage line builder (core/coverage.ts messageViewLines) is
|
||||
* deliberate — every message that produces view lines produces a markdown
|
||||
* block, in the same order — but the formats are SEPARATE by design
|
||||
* (2026-09-27): coverage fingerprints hash the line format and stay stable
|
||||
* (no mass re-extraction when presentation changes); markdown is
|
||||
* presentation. A failed tool call renders nothing (retry context, not
|
||||
* learning evidence); custom markers are invisible, same as the view.
|
||||
*/
|
||||
|
||||
import type { LhMessage } from "./native-messages.ts";
|
||||
import type { ToolResultMessage } from "@earendil-works/pi-ai";
|
||||
|
||||
/** The user-facing text blocks of a native message (string or text parts). */
|
||||
function textOf(content: unknown): string[] {
|
||||
if (typeof content === "string") return content.trim() ? [content] : [];
|
||||
if (!Array.isArray(content)) return [];
|
||||
return content
|
||||
.filter((c): c is { type: "text"; text: string } => (c as { type?: string }).type === "text")
|
||||
.map((c) => c.text)
|
||||
.filter((t) => t.trim().length > 0);
|
||||
}
|
||||
|
||||
/** One tool result's learner-visible outcome, rendered as body lines. */
|
||||
function toolResultBody(result: ToolResultMessage): { heading: string; body: string[] } {
|
||||
const out = result.details as
|
||||
| Partial<{
|
||||
questionText: string;
|
||||
correct: boolean;
|
||||
dontKnow: boolean;
|
||||
note: string;
|
||||
text: string;
|
||||
answers: string[];
|
||||
other: string;
|
||||
dismissed: boolean;
|
||||
}>
|
||||
| undefined;
|
||||
|
||||
if (result.toolName === "quiz" || result.toolName === "explain") {
|
||||
const heading = result.toolName === "quiz" ? "Quiz" : "Recall check";
|
||||
const question = typeof out?.questionText === "string" && out.questionText ? out.questionText : "(no question text)";
|
||||
const body = [`**Q:** ${question}`];
|
||||
if (out?.dismissed === true) body.push("The learner skipped this check.");
|
||||
else if (out?.dontKnow === true) body.push("The learner did not know.");
|
||||
else if (out?.correct === true) body.push("The learner answered correctly.");
|
||||
else if (out?.correct === false) body.push("The learner answered incorrectly.");
|
||||
else if (typeof out?.text === "string" && out.text.trim()) body.push(`The learner answered in his own words: ${out.text.trim()}`);
|
||||
if (typeof out?.note === "string" && out.note) body.push(`*(his thinking: ${out.note})*`);
|
||||
return { heading, body };
|
||||
}
|
||||
|
||||
if (result.toolName === "ask_user_question") {
|
||||
const question = typeof out?.questionText === "string" && out.questionText ? out.questionText : "(no question text)";
|
||||
const body = [`**Q:** ${question}`];
|
||||
const chosen = [
|
||||
...(Array.isArray(out?.answers) ? out!.answers! : []),
|
||||
...(typeof out?.other === "string" && out.other.trim() ? [out.other.trim()] : []),
|
||||
];
|
||||
if (typeof out?.text === "string" && out.text.trim()) body.push(`The learner answered: ${out.text.trim()}`);
|
||||
else if (chosen.length > 0) body.push(`The learner chose: ${chosen.join(", ")}`);
|
||||
else if (out?.dismissed === true) body.push("The learner skipped this question.");
|
||||
if (typeof out?.note === "string" && out.note) body.push(`*(his thinking: ${out.note})*`);
|
||||
return { heading: "Question", body };
|
||||
}
|
||||
|
||||
// Parity with the line builder: other clean tool results are one line.
|
||||
return { heading: "Tool", body: [`*${result.toolName} result delivered*`] };
|
||||
}
|
||||
|
||||
/**
|
||||
* ONE message as a markdown block — `null` when the message renders
|
||||
* nothing (a failed tool call, a bare marker, an empty text part set).
|
||||
* `mid` is the provenance anchor shown when `anchors` is on.
|
||||
*/
|
||||
export function messageMarkdown(message: LhMessage, mid: string, anchors = true): string | null {
|
||||
const role = (message as { role?: string }).role;
|
||||
const anchor = anchors ? ` · ${mid}` : "";
|
||||
const parts: string[] = [];
|
||||
|
||||
if (role === "user") {
|
||||
for (const text of textOf((message as { content: unknown }).content)) {
|
||||
parts.push(`### Learner${anchor}\n\n${text.trim()}`);
|
||||
}
|
||||
} else if (role === "assistant") {
|
||||
for (const text of textOf((message as { content: unknown }).content)) {
|
||||
parts.push(`### Teacher${anchor}\n\n${text.trim()}`);
|
||||
}
|
||||
} else if (role === "toolResult") {
|
||||
const result = message as ToolResultMessage;
|
||||
if (result.isError) return null; // retry context, not learning evidence
|
||||
const { heading, body } = toolResultBody(result);
|
||||
parts.push(`### ${heading}${anchor}\n\n${body.join("\n\n")}`);
|
||||
}
|
||||
|
||||
// custom markers are invisible, same as the extraction view.
|
||||
return parts.length > 0 ? parts.join("\n\n") : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* A whole session (or one extraction BATCH — same engine, subset of
|
||||
* messages) as one markdown document. Blocks render in message order;
|
||||
* messages that render nothing are skipped silently. The optional title
|
||||
* mints the document header (`# <title>`).
|
||||
*/
|
||||
export function sessionMarkdown(
|
||||
messages: ReadonlyArray<LhMessage>,
|
||||
opts: { title?: string; anchors?: boolean } = {},
|
||||
): string {
|
||||
const blocks: string[] = [];
|
||||
if (opts.title !== undefined && opts.title.length > 0) blocks.push(`# ${opts.title}`);
|
||||
for (const message of messages) {
|
||||
const block = messageMarkdown(message, (message as { id?: string }).id ?? "", opts.anchors ?? true);
|
||||
if (block !== null) blocks.push(block);
|
||||
}
|
||||
return blocks.join("\n\n");
|
||||
}
|
||||
Reference in New Issue
Block a user