delete the mappings ledger — per-job in-memory resolution + notes fingerprint guard

The persisted (chatId, sourceId)→node table was a shared namespace across
independent chunk/batch jobs: each chunk's extractor mints local ids fresh
(p1..pN), so later chunks collided with earlier ones — propositions silently
skipped as 'already resolved' (coverage checkpointed anyway: permanent silent
loss) and edges resolved onto the wrong nodes (the 'everything links to
everything' mush). Owner ruling 2026-09-29: mergeFindings resolves THIS job's
ids in memory, nothing persists between jobs; cross-job identity is the
auditor's linkTo verdicts alone.

- contracts: NodeMapping dies; NotesSource.contentFp — sha256 of the text at
  the last run, the re-import guard (unchanged re-import = TRUE no-op, no
  model calls, no usage; changed text re-runs; legacy unstamped sources
  re-run once)
- merger: job-local resolution map, in-submit duplicate ids skip once,
  edges resolve extractor-local ids only
- ports/store: appendMapping/readMappings, LEDGER_FILES.mappings,
  rewriteLedgers.mappings gone (stale data/graph/mappings.jsonl just stops
  being read)
- session-clusters: birthSessionOf is provenance-only — the legacy
  mapping-line fallback for provenance-empty notes nodes dies
- deletion: no mapping rows to strip; stats lose mappingsRemoved
- replay-layout batches nodes by first provenance point
- graph.test pins the new independence semantics (re-merge re-creates;
  duplicate ids within one submit resolve once); extraction tests pin
  per-chunk merges + the fingerprint guard
This commit is contained in:
amosblomqvist
2026-09-29 02:25:32 +03:00
parent 92146248d2
commit 8355fbabdb
17 changed files with 182 additions and 304 deletions
-1
View File
@@ -58,7 +58,6 @@ export type ModelRole =
export const LEDGER_FILES = { export const LEDGER_FILES = {
nodes: path.join(PATHS.graph, "nodes.jsonl"), nodes: path.join(PATHS.graph, "nodes.jsonl"),
edges: path.join(PATHS.graph, "edges.jsonl"), edges: path.join(PATHS.graph, "edges.jsonl"),
mappings: path.join(PATHS.graph, "mappings.jsonl"),
sessions: path.join(PATHS.graph, "sessions.jsonl"), sessions: path.join(PATHS.graph, "sessions.jsonl"),
/** Per-message extraction coverage (contracts/graph.ts CoverageEntry; /** Per-message extraction coverage (contracts/graph.ts CoverageEntry;
* the checkpoint's replacement for chat extractions — notes keep * the checkpoint's replacement for chat extractions — notes keep
+4 -4
View File
@@ -329,7 +329,7 @@ Sub-agents **verify, never one-shot**: the demo-builder headlessly executes its
```mermaid ```mermaid
flowchart TB flowchart TB
TRIG["trigger seam (turn-service): coverageOf(record, coverageLedger)<br/>— uncovered tokens ≥ graph.extractionChunkTokens (settings)?"] -->|"yes, single-flight per source"| RUN["extraction.runExtraction(chatId)"] TRIG["trigger seam (turn-service): coverageOf(record, coverageLedger)<br/>— uncovered tokens ≥ graph.extractionChunkTokens (settings)?"] -->|"yes, single-flight per source"| RUN["extraction.runExtraction(chatId)"]
IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path) — idempotent re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN IMPORT["POST /api/graph/import (notes front door)<br/>deterministic src id = sha256(path); content-fp guard skips unchanged re-imports"] -->|"importNotes: shared pipeline, second extractor prompt"| RUN
RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"] RUN --> BATCH["batchRuns — greedy token-budgeted batches of oldest uncovered TEACHER-PROSE messages"]
BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"] BATCH --> EXT["EXTRACTOR (LLM) — sees ONLY the session markdown view<br/>(tree-blind, graph-blind; shared engine: core/session-markdown.ts)"]
EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity ONLY (link-or-create, equivalence bias) — never mints edges"] EXT --> AUD["AUDITOR (LLM, agentic loop) — sees findings ONLY, never the source;<br/>explores the graph via find / list_sessions / list_propositions tools,<br/>judges identity ONLY (link-or-create, equivalence bias) — never mints edges"]
@@ -342,11 +342,11 @@ flowchart TB
**Serialized audit, parallel extracts — per-chunk jobs** (owner ruling 2026-09-28): every chunk (notes file) and every batch (chat coverage) is its own JOB — registry row, usage record, failure domain. Extracts fan out in parallel across all jobs and sources, throttled by a process-wide 3-permit extract semaphore on `globalThis` (the provider-facing rate stance; a job waiting for a permit shows `queued`). The audit→merge section remains a process-wide FIFO critical section — a mutex on `globalThis` (the same module-instance idiom as the run registries): before any auditor LLM call or graph read, a job waits (`waiting` phase) until every earlier job's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. Failure isolation: one job failing (bad chunk, degraded model output) is absorbed at the fan-out — its messages stay uncovered and honestly re-runnable while its siblings still merge; the run returns what actually merged. Abort is source-wide: every job of a source's fan-out dies together, and merged jobs keep their checkpoints. The status feed shows one row per job (`file.md · 2/5` suffixes for chunked sources), and each job persists its own usage record (failed jobs bill their partial spend too). **Serialized audit, parallel extracts — per-chunk jobs** (owner ruling 2026-09-28): every chunk (notes file) and every batch (chat coverage) is its own JOB — registry row, usage record, failure domain. Extracts fan out in parallel across all jobs and sources, throttled by a process-wide 3-permit extract semaphore on `globalThis` (the provider-facing rate stance; a job waiting for a permit shows `queued`). The audit→merge section remains a process-wide FIFO critical section — a mutex on `globalThis` (the same module-instance idiom as the run registries): before any auditor LLM call or graph read, a job waits (`waiting` phase) until every earlier job's merge has landed, so each auditor explores the graph as it actually stands — two concurrent audits would each link into a graph the other is about to change, the structural mint of duplicate nodes. Failure isolation: one job failing (bad chunk, degraded model output) is absorbed at the fan-out — its messages stay uncovered and honestly re-runnable while its siblings still merge; the run returns what actually merged. Abort is source-wide: every job of a source's fan-out dies together, and merged jobs keep their checkpoints. The status feed shows one row per job (`file.md · 2/5` suffixes for chunked sources), and each job persists its own usage record (failed jobs bill their partial spend too).
**Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never mints edges — cross-source connection happens solely through identified equivalences, and relations between propositions come from the extractor alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Division of trust** (the pipeline's core rule): the extractor is fully trusted and is the only stage that ever sees the source. The auditor never sees the transcript — it receives the findings JSON alone and explores the existing graph through read-only tools (`find`, `list_sessions`, `list_propositions` — statement text included, no timestamps) to reach its verdicts. VERDICTS-ONLY (owner ruling 2026-09-28): the auditor's submit is `{ links }` — identity links only (which findings propositions ARE existing nodes). The edges argument is GONE (owner, same evening): the auditor never mints edges — cross-source connection happens solely through identified equivalences, and relations between propositions come from the extractor alone. It never restates the findings; the merger joins findings + verdicts, so an order of magnitude less is emitted and there is no structural way for the auditor to alter or drop extractor output. Equivalence bias (owner ruling 2026-09-27): paraphrases, rewordings, and compressions of the same claim MERGE; only genuinely distinct propositions stay apart. The deterministic merger validates and appends: link resolution, edge dedupe by `(from, to, type)`, coverage checkpoints. There is no drop channel — nothing rejects the extractor's output. **Per-job in-memory resolution** (owner, 2026-09-29): the mappings ledger is deleted — each job resolves its own proposition ids against the graph in memory and merges independently, so no job can see or collide with another's ids (the persisted table once silently dropped same-id propositions across chunks and welded wrong edges). Cross-job and cross-run identity is the auditor's links alone, plus the coverage ledger (nothing re-extracts) and the notes content-fingerprint guard.
**Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort. **Coverage, not quiz-counting** (supersedes the v1 every-N-quiz-answers trigger): each message's *view* (the compact line format `core/coverage.ts` has always hashed) is hashed into a per-chat fingerprint ledger (`coverage.jsonl`); "covered" means exactly "a run committed on this content." The prompt view is the richer markdown document — a deliberate 2026-09-27 split: re-formatting presentation never re-opens covered history. Uncovered = fingerprint diff, never dual-tracked; runs are greedy token-budgeted batches (`graph.extractionChunkTokens` — a settings knob, default 4096, floor 500, read fresh each turn) of the oldest uncovered messages — quiz-less sessions extract too. A coarse `sessions.jsonl` checkpoint still marks each merged run. The trigger seam (`turn-service.ts`) is trigger-agnostic from the pipeline's side: when a better mechanism lands (session end, idle, explicit flush), only the seam changes. Single-flight per source; the run registry powers the coverage UI and true abort.
**Two front doors, one pipeline**: chat transcripts (quiz-triggered) and raw personal notes (`POST /api/graph/import` — each file becomes a `NotesSource` playing the `chatId` role, deterministic id from the path hash). Notes chunk like chats (owner, 2026-09-28): large files split at line boundaries under the same token budget, and each chunk/batch fans out as its own job exactly like a chat run — parallel extracts under the semaphore, serialized audits under the gate. No single model call carries a whole file. Both submit tools require their top-level fields (`edges`/`links` present even when empty): a payload missing them fails validation rather than silently merging structure-less nodes. **Two front doors, one pipeline**: chat transcripts (quiz-triggered) and raw personal notes (`POST /api/graph/import` — each file becomes a `NotesSource` playing the `chatId` role, deterministic id from the path hash; a content-fingerprint stamp makes an unchanged re-import a TRUE no-op — no model calls, no usage — while changed text re-runs the whole file, owner 2026-09-29). Notes chunk like chats (owner, 2026-09-28): large files split at line boundaries under the same token budget, and each chunk/batch fans out as its own job exactly like a chat run — parallel extracts under the semaphore, serialized audits under the gate. No single model call carries a whole file. Both submit tools require their top-level fields (`edges`/`links` present even when empty): a payload missing them fails validation rather than silently merging structure-less nodes.
**The read channel is pure derivation.** `GET /api/graph` reads the ledgers (`graphSnapshot`: merged nodes/edges, birth-session clusters, positions) — the home view never triggers a write. Node detail cards show a node's edges — outgoing under "What follows from this" / "What this supports", incoming under "Follows from" / "Supported by" (an edge A→B means A implies or supports B, owner semantics 2026-09-28) — and its birth sessions. **The read channel is pure derivation.** `GET /api/graph` reads the ledgers (`graphSnapshot`: merged nodes/edges, birth-session clusters, positions) — the home view never triggers a write. Node detail cards show a node's edges — outgoing under "What follows from this" / "What this supports", incoming under "Follows from" / "Supported by" (an edge A→B means A implies or supports B, owner semantics 2026-09-28) — and its birth sessions.
@@ -409,7 +409,7 @@ The Electron main is a launcher + security guard, never a host for app logic: si
| **One writer per ledger** — the merger is the sole graph writer; chat records written only by the turn pipeline; job records only by the runner; positions PUT is the only sanctioned graph-adjacent exception | | **One writer per ledger** — the merger is the sole graph writer; chat records written only by the turn pipeline; job records only by the runner; positions PUT is the only sanctioned graph-adjacent exception |
| **The store never decides** — adapters persist and shape-guard; merge/dedup/derivation semantics are reader-side, in core | | **The store never decides** — adapters persist and shape-guard; merge/dedup/derivation semantics are reader-side, in core |
| **Derived, never dual-tracked** — append-only evidence; every state recomputed at read (snapshot, plan statuses, phases, costs, coverage) | | **Derived, never dual-tracked** — append-only evidence; every state recomputed at read (snapshot, plan statuses, phases, costs, coverage) |
| **Idempotency via deterministic keys** — steer ids, ledger refs, coverage fingerprints, mapping `(chatId, sourceId)`, cluster content hashes; re-running anything converges | | **Idempotency via deterministic keys** — steer ids, ledger refs, coverage fingerprints, source content fingerprints, cluster content hashes; re-running anything converges |
| **LLMs judge; code mechanizes** — grading, merging, fingerprints, layout are deterministic; a failed LLM stage checkpoints nothing | | **LLMs judge; code mechanizes** — grading, merging, fingerprints, layout are deterministic; a failed LLM stage checkpoints nothing |
| **The persisted record is the only source of truth** — agents are stateless per-turn rebuilds; no unanswered tool call reaches a provider | | **The persisted record is the only source of truth** — agents are stateless per-turn rebuilds; no unanswered tool call reaches a provider |
| **Failure containment** — side channels (context, extraction, usage, naming) are fire-and-forget behind try/catch seams; a side-channel failure never touches the response or the persist; the turn registry clears on every path | | **Failure containment** — side channels (context, extraction, usage, naming) are fire-and-forget behind try/catch seams; a side-channel failure never touches the response or the persist; the turn registry clears on every path |
+7 -7
View File
@@ -19,7 +19,7 @@ const depthMode: "off" | "vertical" | "radial" =
depthArg === "--depth=vertical" ? "vertical" : depthArg === "--depth=radial" ? "radial" : "off"; depthArg === "--depth=vertical" ? "vertical" : depthArg === "--depth=radial" ? "radial" : "off";
import { incrementalLayout } from "../src/core/layout.ts"; import { incrementalLayout } from "../src/core/layout.ts";
import { birthSessionPartition } from "../src/core/session-clusters.ts"; import { birthSessionPartition } from "../src/core/session-clusters.ts";
import type { GraphNode, NodeMapping, Position } from "../src/core/contracts.ts"; import type { GraphNode, Position } from "../src/core/contracts.ts";
function main() { function main() {
const t0 = Date.now(); const t0 = Date.now();
@@ -35,17 +35,17 @@ function main() {
const edges = graphStore.readEdges(); const edges = graphStore.readEdges();
// The cluster source (wave 7): the birth-session partition, derived on // The cluster source (wave 7): the birth-session partition, derived on
// read — same as the real settle (core/layout-settle.ts). // read — same as the real settle (core/layout-settle.ts).
const birthClusters = birthSessionPartition(nodes, graphStore.readMappings()); const birthClusters = birthSessionPartition(nodes);
// Batches: each imported file, in real import order (sources.json // Batches: each imported file, in real import order (sources.json
// importedAt). A node belongs to the file whose mapping first claims it // importedAt). A node belongs to the session of its FIRST provenance
// (mapping line order = arrival); nodes no mapping covers (legacy) ride // point (line order = arrival); nodes with no provenance (legacy) ride
// the final batch. // the final batch.
const sources = [...graphStore.readSources()].sort((a, b) => a.importedAt.localeCompare(b.importedAt)); const sources = [...graphStore.readSources()].sort((a, b) => a.importedAt.localeCompare(b.importedAt));
const mappings: NodeMapping[] = graphStore.readMappings();
const batchOf = new Map<string, string>(); // nodeId → srcId const batchOf = new Map<string, string>(); // nodeId → srcId
for (const m of mappings) { for (const node of nodes) {
if (!batchOf.has(m.graphNodeId)) batchOf.set(m.graphNodeId, m.chatId); const birth = node.provenance[0]?.chatId;
if (birth !== undefined && !batchOf.has(node.id)) batchOf.set(node.id, birth);
} }
const delivered = new Set<string>(); const delivered = new Set<string>();
+10 -2
View File
@@ -1221,11 +1221,19 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
/** Import ONE raw notes file: register the source (deterministic id per /** Import ONE raw notes file: register the source (deterministic id per
* file path — re-import keeps the same identity and its usage history) * file path — re-import keeps the same identity and its usage history)
* and run the pipeline over it. */ * and run the pipeline over it. Re-import guard (owner, 2026-09-29):
* the file's content fingerprint decides — unchanged text is a TRUE
* no-op (the mappings ledger used to dedupe this at merge time by
* accident); changed text re-runs the whole file. */
async function importNotes(input: NotesImportInput): Promise<EmissionStats> { async function importNotes(input: NotesImportInput): Promise<EmissionStats> {
const id = `src_${crypto.createHash("sha256").update(input.filePath).digest("hex").slice(0, 12)}`; const id = `src_${crypto.createHash("sha256").update(input.filePath).digest("hex").slice(0, 12)}`;
const contentFp = crypto.createHash("sha256").update(input.text).digest("hex").slice(0, 16);
if (graphStore.readSources().find((s) => s.id === id)?.contentFp === contentFp) {
console.info(`[extraction ${id}] re-import with unchanged content (fp match) — skipping`);
return { nodes: 0, edges: 0 };
}
const importedAt = new Date().toISOString(); const importedAt = new Date().toISOString();
graphStore.upsertSource({ id, filePath: input.filePath, importedAt }); graphStore.upsertSource({ id, filePath: input.filePath, importedAt, contentFp });
return runNotesExtraction({ id, ...input, importedAt }); return runNotesExtraction({ id, ...input, importedAt });
} }
+6 -12
View File
@@ -58,6 +58,12 @@ export interface NotesSource {
filePath: string; filePath: string;
/** ISO — when this file was imported (the checkpoint's foldedAt). */ /** ISO — when this file was imported (the checkpoint's foldedAt). */
importedAt: string; importedAt: string;
/** sha256 of the imported text at the last EXTRACTION run — the
* same-path re-import guard (owner, 2026-09-29): a matching
* fingerprint is a TRUE no-op (no model calls, no usage record);
* a mismatch or a legacy missing stamp re-runs the whole file.
* The mappings ledger used to give this idempotency by accident. */
contentFp?: string;
/** The run's usage log — same append-only discipline as /** The run's usage log — same append-only discipline as
* ChatRecord.extractions, idempotent per startedAt. */ * ChatRecord.extractions, idempotent per startedAt. */
extractions?: ExtractionRunUsage[]; extractions?: ExtractionRunUsage[];
@@ -109,18 +115,6 @@ export interface GraphEdge {
provenance: ProvenancePoint[]; provenance: ProvenancePoint[];
} }
/** mappings.jsonl line — the session→graph identity map: the extraction
* pipeline's proposition (identified by its run-local source id) → the
* graph node it resolved to (created or linked). The audit trail of the
* auditor's identity verdicts; also the chat-scoped resolution table the
* merger uses to point events and edges at node ids. */
export interface NodeMapping {
chatId: string;
/** The extractor's run-local proposition id ("p1", …). */
sourceId: string;
graphNodeId: string;
}
/** sessions.jsonl line — the emission checkpoint for one chat (which chats /** sessions.jsonl line — the emission checkpoint for one chat (which chats
* have been folded, and through which updatedAt — never fold twice). */ * have been folded, and through which updatedAt — never fold twice). */
export interface SessionCheckpoint { export interface SessionCheckpoint {
+5 -30
View File
@@ -13,7 +13,7 @@
* *
* preserve-graph the chat's extraction bookkeeping dies — its coverage * preserve-graph the chat's extraction bookkeeping dies — its coverage
* rows and its legacy checkpoint lines — and NOTHING else: * rows and its legacy checkpoint lines — and NOTHING else:
* nodes, edges, mappings stay byte-identical. * nodes and edges stay byte-identical.
* The graph keeps the session's contributions; the * The graph keeps the session's contributions; the
* snapshot simply no longer resolves the dead chatId to * snapshot simply no longer resolves the dead chatId to
* a title (nodeSessions filters unknown ids at read * a title (nodeSessions filters unknown ids at read
@@ -22,8 +22,8 @@
* delete-nodes "this session never happened": every trace of the * delete-nodes "this session never happened": every trace of the
* chat leaves the ledgers — its provenance points on * chat leaves the ledgers — its provenance points on
* every node and edge line, its link lines (their * every node and edge line, its link lines (their
* phrasings die with them), its mapping lines (the audit trail), its * phrasings die with them), its coverage/checkpoint
* coverage/checkpoint rows — and orphans die: a node * rows — and orphans die: a node
* whose references all came from this chat disappears * whose references all came from this chat disappears
* entirely, pruned from the positions and embedding * entirely, pruned from the positions and embedding
* ledgers too (dead members would break cluster hulls). * ledgers too (dead members would break cluster hulls).
@@ -57,7 +57,6 @@ import type {
CoverageEntry, CoverageEntry,
GraphEdge, GraphEdge,
GraphNode, GraphNode,
NodeMapping,
SessionCheckpoint, SessionCheckpoint,
} from "./contracts.ts"; } from "./contracts.ts";
import type { GraphStore } from "./ports.ts"; import type { GraphStore } from "./ports.ts";
@@ -74,8 +73,6 @@ export interface ChatDeletionStats {
nodesRemoved: number; nodesRemoved: number;
/** Edge assertion LINES removed (empty-provenance or dead endpoint). */ /** Edge assertion LINES removed (empty-provenance or dead endpoint). */
edgesRemoved: number; edgesRemoved: number;
/** Mapping lines removed (the chat's audit trail). */
mappingsRemoved: number;
/** Coverage rows removed. */ /** Coverage rows removed. */
coverageRemoved: number; coverageRemoved: number;
/** Checkpoint lines removed. */ /** Checkpoint lines removed. */
@@ -85,7 +82,6 @@ export interface ChatDeletionStats {
const EMPTY_STATS: ChatDeletionStats = { const EMPTY_STATS: ChatDeletionStats = {
nodesRemoved: 0, nodesRemoved: 0,
edgesRemoved: 0, edgesRemoved: 0,
mappingsRemoved: 0,
coverageRemoved: 0, coverageRemoved: 0,
checkpointsRemoved: 0, checkpointsRemoved: 0,
}; };
@@ -121,7 +117,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
// ── delete-nodes: the full purge ────────────────────────────────────── // ── delete-nodes: the full purge ──────────────────────────────────────
const nodeLines = graphStore.readNodes(); const nodeLines = graphStore.readNodes();
const edgeLines = graphStore.readEdges(); const edgeLines = graphStore.readEdges();
const mappings = graphStore.readMappings();
// Does the chat touch the graph at all? (A never-extracted chat still // Does the chat touch the graph at all? (A never-extracted chat still
// needs its coverage rows dropped above, but no ledger rewrite — keeps // needs its coverage rows dropped above, but no ledger rewrite — keeps
@@ -129,8 +124,7 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
// changed.) // changed.)
const touched = const touched =
nodeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) || nodeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) ||
edgeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)) || edgeLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId));
mappings.some((m) => m.chatId === chatId);
if (!touched) { if (!touched) {
if (stats.coverageRemoved > 0 || stats.checkpointsRemoved > 0) { if (stats.coverageRemoved > 0 || stats.checkpointsRemoved > 0) {
graphStore.rewriteLedgers({ coverage, checkpoints }); graphStore.rewriteLedgers({ coverage, checkpoints });
@@ -153,9 +147,6 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
for (const line of nodeLines) { for (const line of nodeLines) {
if (survives(line.provenance)) live.add(line.id); if (survives(line.provenance)) live.add(line.id);
} }
for (const mapping of mappings) {
if (mapping.chatId !== chatId) live.add(mapping.graphNodeId);
}
// 2. Node lines. Per-line (each line is ONE extraction run's append, so // 2. Node lines. Per-line (each line is ONE extraction run's append, so
// per-session alias provenance exists at line granularity): // per-session alias provenance exists at line granularity):
@@ -209,17 +200,12 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
outEdges.push(points.length === line.provenance.length ? line : { ...line, provenance: points }); outEdges.push(points.length === line.provenance.length ? line : { ...line, provenance: points });
} }
// 4. Mappings: the chat's rows die outright.
const outMappings = mappings.filter((m) => m.chatId !== chatId);
stats.nodesRemoved = [...new Set(nodeLines.map((line) => line.id))].filter((id) => !live.has(id)).length; stats.nodesRemoved = [...new Set(nodeLines.map((line) => line.id))].filter((id) => !live.has(id)).length;
stats.edgesRemoved = edgeLines.length - outEdges.length; stats.edgesRemoved = edgeLines.length - outEdges.length;
stats.mappingsRemoved = mappings.length - outMappings.length;
graphStore.rewriteLedgers({ graphStore.rewriteLedgers({
nodes: outNodes, nodes: outNodes,
edges: outEdges, edges: outEdges,
mappings: outMappings,
coverage, coverage,
checkpoints, checkpoints,
}); });
@@ -272,14 +258,11 @@ export function deleteChatTraces(graphStore: GraphStore, chatId: string, mode: C
export interface NodeDeletionStats { export interface NodeDeletionStats {
/** Edge assertion LINES removed (every edge touching the node). */ /** Edge assertion LINES removed (every edge touching the node). */
edgesRemoved: number; edgesRemoved: number;
/** Mapping lines removed (the node's audit trail). */
mappingsRemoved: number;
} }
/** The confirm dialog's pre-flight facts (the GET on the same node id): /** The confirm dialog's pre-flight facts (the GET on the same node id):
* what full death would take with it. sessions is the deduped set of * what full death would take with it. sessions is the deduped set of
* chats/notes sources that reference the node — provenance points and * chats/notes sources that reference the node — provenance points count. */
* mapping lines count. */
export interface NodeDeletionImpact { export interface NodeDeletionImpact {
edges: number; edges: number;
sessions: string[]; sessions: string[];
@@ -298,9 +281,6 @@ export function nodeDeletionImpact(graphStore: GraphStore, nodeId: string): Node
for (const line of nodeLinesOf(graphStore, nodeId)) { for (const line of nodeLinesOf(graphStore, nodeId)) {
for (const pt of line.provenance) sessions.add(pt.chatId); for (const pt of line.provenance) sessions.add(pt.chatId);
} }
for (const mapping of graphStore.readMappings()) {
if (mapping.graphNodeId === nodeId) sessions.add(mapping.chatId);
}
return { return {
edges: edges.length, edges: edges.length,
sessions: [...sessions].sort(), sessions: [...sessions].sort(),
@@ -318,10 +298,7 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
if (nodeLines.length === 0) return null; if (nodeLines.length === 0) return null;
const edgeLines = graphStore.readEdges(); const edgeLines = graphStore.readEdges();
const mappings = graphStore.readMappings();
const outEdges = edgeLines.filter((line) => line.from !== nodeId && line.to !== nodeId); const outEdges = edgeLines.filter((line) => line.from !== nodeId && line.to !== nodeId);
const outMappings = mappings.filter((m) => m.graphNodeId !== nodeId);
// Only the touched ledgers: the node's own lines always die (its file // Only the touched ledgers: the node's own lines always die (its file
// must move), everything else rewrites only when rows actually drop. // must move), everything else rewrites only when rows actually drop.
@@ -329,7 +306,6 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
graphStore.rewriteLedgers({ graphStore.rewriteLedgers({
nodes: outNodes, nodes: outNodes,
...(outEdges.length !== edgeLines.length ? { edges: outEdges } : {}), ...(outEdges.length !== edgeLines.length ? { edges: outEdges } : {}),
...(outMappings.length !== mappings.length ? { mappings: outMappings } : {}),
}); });
// The resting position dies with the node. // The resting position dies with the node.
@@ -341,6 +317,5 @@ export function deleteNodeTraces(graphStore: GraphStore, nodeId: string): NodeDe
return { return {
edgesRemoved: edgeLines.length - outEdges.length, edgesRemoved: edgeLines.length - outEdges.length,
mappingsRemoved: mappings.length - outMappings.length,
}; };
} }
+33 -44
View File
@@ -55,7 +55,7 @@ import type {
GraphSnapshot, GraphSnapshot,
} from "./contracts.ts"; } from "./contracts.ts";
import { MIN_CLUSTER_SIZE } from "./clusters.ts"; import { MIN_CLUSTER_SIZE } from "./clusters.ts";
import { birthMappingIndex, birthSessionOf, birthSessionPartition } from "./session-clusters.ts"; import { birthSessionOf, birthSessionPartition } from "./session-clusters.ts";
import type { ChatStore, GraphStore } from "./ports.ts"; import type { ChatStore, GraphStore } from "./ports.ts";
/** The storage ports every graph operation needs, passed explicitly. */ /** The storage ports every graph operation needs, passed explicitly. */
@@ -101,21 +101,23 @@ const samePoint = (a: ProvenancePoint, b: ProvenancePoint): boolean =>
// ─── the merger (extraction pipeline stage 3 — deterministic) ──────────────── // ─── the merger (extraction pipeline stage 3 — deterministic) ────────────────
/** /**
* Merge one chat's audited findings into the graph ledgers — the pipeline's * Merge one job's audited findings into the graph ledgers — the pipeline's
* apply stage. Deterministic + idempotent: re-merging the same verdicts (or * apply stage. Deterministic. Each chunk/batch job merges INDEPENDENTLY
* a later re-extraction of the same chat) appends nothing but the new. * (owner, 2026-09-29): resolution is a job-local in-memory map from this
* submit's proposition ids to graph node ids — nothing persists between
* jobs, so no job can see (or collide with) another's ids. Cross-job
* identity is the auditor's alone: it links this job's propositions to
* existing nodes before this function is ever called.
* *
* Identity is the auditor's link-or-create verdict: `linkTo` an existing * Identity inside the job is the auditor's link-or-create verdict: `linkTo`
* node (validated against the ledger — an unknown id falls back to create) * an existing node (validated against the ledger — an unknown id falls
* appends a LINK line (the session's phrasings join the node's aliases and * back to create) appends a LINK line (the job's phrasings join the node's
* the provenance unions; getGraph merges lines by id); absent → a new node. * aliases and the provenance unions; getGraph merges lines by id);
* Mapping lines record each verdict: one per (chat, sourceId). Idempotent: * absent → a new node.
* a proposition id already mapped for this chat reuses its resolution.
* *
* `chatId` is the run's source key: a real chat, or a notes source id * `chatId` is the job's source key: a real chat, or a notes source id
* ("src_…") for the format-agnostic import path — same ledgers, same * ("src_…") for the format-agnostic import path. The `source` parameter
* idempotency, one graph. The `source` parameter carries the checkpoint * carries the checkpoint stamp; the merger itself is chat-store-free.
* stamp; the merger itself is chat-store-free.
*/ */
export function mergeFindings( export function mergeFindings(
chatId: string, chatId: string,
@@ -127,26 +129,22 @@ export function mergeFindings(
const { graphStore: ledgers } = deps; const { graphStore: ledgers } = deps;
const stats: EmissionStats = { nodes: 0, edges: 0 }; const stats: EmissionStats = { nodes: 0, edges: 0 };
// 1. Resolution: this run's propositions → graph node ids. Existing // 1. Resolution: THIS job's propositions → graph node ids, built
// mapping lines for THIS chat are loaded first — the audit trail doubles // in-memory as the propositions resolve (owner, 2026-09-29: the
// as the resolution table, which makes re-merging the same chat // mappings ledger is gone — a persisted table was a shared namespace
// idempotent (a re-extraction re-merges through the auditor's linkTo // across independent jobs, and cross-job id collisions silently
// verdicts instead). Fresh sourceIds resolve as the verdicts say. // dropped propositions and welded wrong edges). Dedupe across runs
// is the coverage ledger (nothing re-extracts) and the auditor's
// equivalence links (re-encountered knowledge merges in).
const existingIds = new Set(ledgers.readNodes().map((n) => n.id)); const existingIds = new Set(ledgers.readNodes().map((n) => n.id));
const mapping = new Map<string, string>(); const mapping = new Map<string, string>();
const seenMappingKeys = new Set<string>();
for (const m of ledgers.readMappings()) {
if (m.chatId !== chatId) continue;
mapping.set(m.sourceId, m.graphNodeId);
seenMappingKeys.add(m.sourceId);
}
const now = new Date().toISOString(); const now = new Date().toISOString();
const linkOf = new Map(verdicts.links.map((l) => [l.id, l.linkTo])); const linkOf = new Map(verdicts.links.map((l) => [l.id, l.linkTo]));
// VERDICTS-ONLY (owner, 2026-09-28): the propositions come from the // VERDICTS-ONLY (owner, 2026-09-28): the propositions come from the
// extractor's findings verbatim; the auditor's links join here by id. // extractor's findings verbatim; the auditor's links join here by id.
for (const p of findings.propositions) { for (const p of findings.propositions) {
if (mapping.has(p.id)) continue; // already resolved for this chat — idempotent if (mapping.has(p.id)) continue; // duplicate id WITHIN this submit — first resolution stands
const points = pointOf(chatId); const points = pointOf(chatId);
const linkTo = linkOf.get(p.id); const linkTo = linkOf.get(p.id);
let graphNodeId: string; let graphNodeId: string;
@@ -180,20 +178,14 @@ export function mergeFindings(
existingIds.add(graphNodeId); existingIds.add(graphNodeId);
} }
mapping.set(p.id, graphNodeId); mapping.set(p.id, graphNodeId);
if (!seenMappingKeys.has(p.id)) {
ledgers.appendMapping({ chatId, sourceId: p.id, graphNodeId });
seenMappingKeys.add(p.id);
}
} }
// 2. Edges — the extractor's own first, then the auditor's BRIDGING // 2. Edges — the extractor's own, the ONLY edge source (the auditor's
// edges (endpoints may be local ids and/or EXISTING graph node ids — // edges argument is gone, owner 2026-09-28 evening). Endpoints are
// the auditor welding the batch into the graph it explored). Both // this job's local proposition ids and resolve through the job-local
// resolve through the mapping, both carry the run's chat-level // map; edges carry the run's chat-level provenance point and dedupe
// provenance point, and both dedupe by (from,to,type) already // by (from,to,type) already asserted by THIS chat — an assertion by
// asserted by THIS chat: an echoed duplicate (a weak auditor // another chat stays untouched (provenance unions at read time).
// restating an extractor edge) merges to one append; an assertion
// by another chat stays untouched (provenance unions at read time).
const assertedByChat = new Set<string>(); const assertedByChat = new Set<string>();
for (const line of ledgers.readEdges()) { for (const line of ledgers.readEdges()) {
const key = `${line.from}\u0000${line.to}\u0000${line.type}`; const key = `${line.from}\u0000${line.to}\u0000${line.type}`;
@@ -295,7 +287,6 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
} }
chats.add(chatId); chats.add(chatId);
}; };
for (const m of deps.graphStore.readMappings()) touch(m.graphNodeId, m.chatId);
for (const node of nodes) { for (const node of nodes) {
for (const pt of node.provenance) touch(node.id, pt.chatId); for (const pt of node.provenance) touch(node.id, pt.chatId);
} }
@@ -339,7 +330,7 @@ export function graphSnapshot(deps: GraphDeps): GraphSnapshot {
// presentation rule (a region needs >= 3 propositions for a hull). A // presentation rule (a region needs >= 3 propositions for a hull). A
// birth session whose record is gone (deleted chat) keeps its members // birth session whose record is gone (deleted chat) keeps its members
// but renders unlabeled — honest, never guessed. // but renders unlabeled — honest, never guessed.
const communities = birthSessionPartition(nodes, deps.graphStore.readMappings()); const communities = birthSessionPartition(nodes);
const clusters: Cluster[] = communities const clusters: Cluster[] = communities
.filter((community) => community.members.length >= MIN_CLUSTER_SIZE) .filter((community) => community.members.length >= MIN_CLUSTER_SIZE)
.map((community) => ({ .map((community) => ({
@@ -396,8 +387,7 @@ export interface GraphSessionEntry {
export function graphSessionIndex(deps: GraphDeps): GraphSessionEntry[] { export function graphSessionIndex(deps: GraphDeps): GraphSessionEntry[] {
const counts = new Map<string, number>(); const counts = new Map<string, number>();
const born = getGraph(deps.graphStore); const born = getGraph(deps.graphStore);
const mappingFallback = birthMappingIndex(deps.graphStore.readMappings()); for (const cluster of birthSessionPartition(born.nodes)) {
for (const cluster of birthSessionPartition(born.nodes, deps.graphStore.readMappings())) {
counts.set(cluster.id, cluster.members.length); counts.set(cluster.id, cluster.members.length);
} }
const chats = new Map(deps.chatStore.listChats().map((chat) => [chat.id, chat])); const chats = new Map(deps.chatStore.listChats().map((chat) => [chat.id, chat]));
@@ -445,8 +435,7 @@ export interface GraphSessionProposition {
/** The propositions BORN in one session — what that session first /** The propositions BORN in one session — what that session first
* articulated. Members sorted by node id — deterministic. */ * articulated. Members sorted by node id — deterministic. */
export function graphSessionPropositions(deps: GraphDeps, sessionId: string): GraphSessionProposition[] { export function graphSessionPropositions(deps: GraphDeps, sessionId: string): GraphSessionProposition[] {
const mappingFallback = birthMappingIndex(deps.graphStore.readMappings()); const born = getGraph(deps.graphStore).nodes.filter((node) => birthSessionOf(node) === sessionId);
const born = getGraph(deps.graphStore).nodes.filter((node) => birthSessionOf(node, mappingFallback) === sessionId);
return born return born
.map((node) => ({ nodeId: node.id, name: node.name })) .map((node) => ({ nodeId: node.id, name: node.name }))
.sort((a, b) => (a.nodeId < b.nodeId ? -1 : 1)); .sort((a, b) => (a.nodeId < b.nodeId ? -1 : 1));
+2 -3
View File
@@ -1,4 +1,4 @@
import type { GraphEdge, GraphNode, NodeMapping, Position } from "./contracts.ts"; import type { GraphEdge, GraphNode, Position } from "./contracts.ts";
import { incrementalLayout } from "./layout.ts"; import { incrementalLayout } from "./layout.ts";
import { birthSessionPartition } from "./session-clusters.ts"; import { birthSessionPartition } from "./session-clusters.ts";
@@ -19,7 +19,6 @@ import { birthSessionPartition } from "./session-clusters.ts";
export function settleGraphLayout(graphStore: { export function settleGraphLayout(graphStore: {
readNodes(): GraphNode[]; readNodes(): GraphNode[];
readEdges(): GraphEdge[]; readEdges(): GraphEdge[];
readMappings(): NodeMapping[];
readPositions(): Record<string, Position>; readPositions(): Record<string, Position>;
writePositions(positions: Record<string, Position>): void; writePositions(positions: Record<string, Position>): void;
}): void { }): void {
@@ -39,7 +38,7 @@ export function settleGraphLayout(graphStore: {
// core/session-clusters.ts, wave 7) — one cluster concept everywhere. // core/session-clusters.ts, wave 7) — one cluster concept everywhere.
// The unique list is the deduped set the partition needs (lines may // The unique list is the deduped set the partition needs (lines may
// repeat an id — first defines). // repeat an id — first defines).
const communities = birthSessionPartition(unique, graphStore.readMappings()); const communities = birthSessionPartition(unique);
const next = incrementalLayout(unique, edges, previous, { clusters: communities }); const next = incrementalLayout(unique, edges, previous, { clusters: communities });
graphStore.writePositions(Object.fromEntries(next)); graphStore.writePositions(Object.fromEntries(next));
} }
-6
View File
@@ -19,7 +19,6 @@ import type {
JobKind, JobKind,
JobRecord, JobRecord,
ExtractionRunUsage, ExtractionRunUsage,
NodeMapping,
PlanTree, PlanTree,
Position, Position,
CoverageEntry, CoverageEntry,
@@ -84,10 +83,6 @@ export interface GraphStore {
/** Append one quiz outcome / understanding observation. */ /** Append one quiz outcome / understanding observation. */
/** Append one identity-map line (chat's tree node → graph node). */
appendMapping(mapping: NodeMapping): void;
readMappings(): NodeMapping[];
/** Append one emission checkpoint. */ /** Append one emission checkpoint. */
appendCheckpoint(checkpoint: SessionCheckpoint): void; appendCheckpoint(checkpoint: SessionCheckpoint): void;
readCheckpoints(): SessionCheckpoint[]; readCheckpoints(): SessionCheckpoint[];
@@ -127,7 +122,6 @@ export interface GraphStore {
rewriteLedgers(next: { rewriteLedgers(next: {
nodes?: GraphNode[]; nodes?: GraphNode[];
edges?: GraphEdge[]; edges?: GraphEdge[];
mappings?: NodeMapping[];
coverage?: CoverageEntry[]; coverage?: CoverageEntry[];
checkpoints?: SessionCheckpoint[]; checkpoints?: SessionCheckpoint[];
}): void; }): void;
+14 -38
View File
@@ -21,24 +21,18 @@
* *
* Notes imports are birth-sessions too (owner ruling 2026-09-23): their * Notes imports are birth-sessions too (owner ruling 2026-09-23): their
* nodes carry a "src_…" source id instead of a chat id; the caller labels * nodes carry a "src_…" source id instead of a chat id; the caller labels
* them with the file's path (kind: file in the session views). The * them with the file's path (kind: file in the session views). The old
* mapping-line fallback (owner-approved 2026-09-23, surfaced by the * mapping-line fallback for provenance-empty notes nodes is GONE (owner,
* interlude's live smoke): production notes imports create provenance- * 2026-09-29): every merge writes provenance points, and the mappings
* EMPTY nodes (the notes front door passes messages: null — the file's * ledger itself died with per-job in-memory resolution — provenance is
* anchor is the mapping line), so a provenance-empty node's birth session * the one birth record.
* is its FIRST mapping line's chatId in ledger order. Only notes-born
* nodes ever hit the fallback — chat extraction always writes provenance
* points, and a re-encountered node's link-append adds one, which
* re-opens the provenance rule above. Deletion-safe: a deleted chat's
* mapping lines are stripped alongside its provenance; notes mappings
* survive because files are never deleted.
* *
* Browser-safe pure logic (no node builtins, no fs): same contract style * Browser-safe pure logic (no node builtins, no fs): same contract style
* as the rest of the core graph derivation. Not re-exported through the * as the rest of the core graph derivation. Not re-exported through the
* core barrel — the shell reads clusters from the snapshot; only the * core barrel — the shell reads clusters from the snapshot; only the
* server-side derivations (snapshot, settle) and tests import this. * server-side derivations (snapshot, settle) and tests import this.
*/ */
import type { GraphNode, NodeMapping } from "./contracts.ts"; import type { GraphNode } from "./contracts.ts";
/** One birth-session cluster: the session id + the SORTED member node /** One birth-session cluster: the session id + the SORTED member node
* ids born in it. The shape is deliberately the { members } contract * ids born in it. The shape is deliberately the { members } contract
@@ -51,44 +45,26 @@ export interface BirthSessionCluster {
members: string[]; members: string[];
} }
/** A node's birth session: its first surviving provenance point's chatId; /** A node's birth session: its first surviving provenance point's chatId.
* for a provenance-empty node, its first mapping line's chatId (the notes * null when the node carries no provenance at all (legacy data). */
* fallback — see the module doc). null when neither exists. */ export function birthSessionOf(node: GraphNode): string | null {
export function birthSessionOf(node: GraphNode, mappingFallback?: ReadonlyMap<string, string>): string | null { return node.provenance[0]?.chatId ?? null;
const first = node.provenance[0];
if (first !== undefined) return first.chatId;
return mappingFallback?.get(node.id) ?? null;
}
/** The notes fallback index: nodeId → the chatId of the node's FIRST
* mapping line in ledger order. */
export function birthMappingIndex(mappings: readonly NodeMapping[]): Map<string, string> {
const index = new Map<string, string>();
for (const mapping of mappings) {
if (!index.has(mapping.graphNodeId)) index.set(mapping.graphNodeId, mapping.chatId);
}
return index;
} }
/** /**
* The birth-session partition over the given node LINES: dedupes by id * The birth-session partition over the given node LINES: dedupes by id
* (first line defines — the getGraph rule), groups by birth session, and * (first line defines — the getGraph rule), groups by birth session, and
* returns the groups size-desc with members sorted (deterministic output: * returns the groups size-desc with members sorted (deterministic output:
* ties break by session id ascending). Nodes with no surviving provenance * ties break by session id ascending). A node with no provenance is left
* fall back to their first mapping line (notes-born); a node with neither * unclustered — it belongs to no session the graph can name.
* is left unclustered — it belongs to no session the graph can name.
*/ */
export function birthSessionPartition( export function birthSessionPartition(nodes: readonly GraphNode[]): BirthSessionCluster[] {
nodes: readonly GraphNode[],
mappings: readonly NodeMapping[] = [],
): BirthSessionCluster[] {
const mappingFallback = birthMappingIndex(mappings);
const members = new Map<string, string[]>(); const members = new Map<string, string[]>();
const seen = new Set<string>(); const seen = new Set<string>();
for (const node of nodes) { for (const node of nodes) {
if (seen.has(node.id)) continue; if (seen.has(node.id)) continue;
seen.add(node.id); seen.add(node.id);
const birth = birthSessionOf(node, mappingFallback); const birth = birthSessionOf(node);
if (birth === null) continue; if (birth === null) continue;
const bucket = members.get(birth); const bucket = members.get(birth);
if (bucket) bucket.push(node.id); if (bucket) bucket.push(node.id);
-1
View File
@@ -57,7 +57,6 @@ export function graphRevision(): number {
for (const file of [ for (const file of [
LEDGER_FILES.nodes, LEDGER_FILES.nodes,
LEDGER_FILES.edges, LEDGER_FILES.edges,
LEDGER_FILES.mappings,
LEDGER_FILES.sessions, LEDGER_FILES.sessions,
LEDGER_FILES.coverage, LEDGER_FILES.coverage,
POSITIONS_FILE, POSITIONS_FILE,
-15
View File
@@ -10,8 +10,6 @@
* edges.jsonl GraphEdge — one line per edge ASSERTION (a second * edges.jsonl GraphEdge — one line per edge ASSERTION (a second
* session asserting the same (from,to,type) * session asserting the same (from,to,type)
* appends another line; reads merge) * appends another line; reads merge)
* mappings.jsonl NodeMapping — the session→graph identity map:
* (chatId, sourceId) → graphNodeId
* sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt) * sessions.jsonl SessionCheckpoint— emission checkpoints (foldedAt)
* *
* Plus config.SOURCES_FILE (sources.json): the imported-notes bookkeeping * Plus config.SOURCES_FILE (sources.json): the imported-notes bookkeeping
@@ -34,7 +32,6 @@ import type {
GraphNode, GraphNode,
ExtractionRunUsage, ExtractionRunUsage,
NotesSource, NotesSource,
NodeMapping,
Position, Position,
ProvenancePoint, ProvenancePoint,
CoverageEntry, CoverageEntry,
@@ -107,9 +104,6 @@ const isGraphEdge = (v: unknown): v is GraphEdge =>
Array.isArray(v.provenance) && Array.isArray(v.provenance) &&
v.provenance.every(isProvenancePoint); v.provenance.every(isProvenancePoint);
const isNodeMapping = (v: unknown): v is NodeMapping =>
isRecord(v) && isString(v.chatId) && isString(v.sourceId) && isString(v.graphNodeId);
const isCheckpoint = (v: unknown): v is SessionCheckpoint => const isCheckpoint = (v: unknown): v is SessionCheckpoint =>
isRecord(v) && isString(v.chatId) && isString(v.foldedAt); isRecord(v) && isString(v.chatId) && isString(v.foldedAt);
@@ -137,13 +131,6 @@ export const graphStore: GraphStore = {
return readValid(LEDGER_FILES.edges, isGraphEdge); return readValid(LEDGER_FILES.edges, isGraphEdge);
}, },
appendMapping(mapping: NodeMapping): void {
appendLine(LEDGER_FILES.mappings, mapping);
},
readMappings(): NodeMapping[] {
return readValid(LEDGER_FILES.mappings, isNodeMapping);
},
appendCheckpoint(checkpoint: SessionCheckpoint): void { appendCheckpoint(checkpoint: SessionCheckpoint): void {
appendLine(LEDGER_FILES.sessions, checkpoint); appendLine(LEDGER_FILES.sessions, checkpoint);
}, },
@@ -198,14 +185,12 @@ export const graphStore: GraphStore = {
rewriteLedgers(next: { rewriteLedgers(next: {
nodes?: GraphNode[]; nodes?: GraphNode[];
edges?: GraphEdge[]; edges?: GraphEdge[];
mappings?: NodeMapping[];
coverage?: CoverageEntry[]; coverage?: CoverageEntry[];
checkpoints?: SessionCheckpoint[]; checkpoints?: SessionCheckpoint[];
}): void { }): void {
const files = { const files = {
nodes: LEDGER_FILES.nodes, nodes: LEDGER_FILES.nodes,
edges: LEDGER_FILES.edges, edges: LEDGER_FILES.edges,
mappings: LEDGER_FILES.mappings,
coverage: LEDGER_FILES.coverage, coverage: LEDGER_FILES.coverage,
checkpoints: LEDGER_FILES.sessions, checkpoints: LEDGER_FILES.sessions,
} as const; } as const;
+36 -26
View File
@@ -35,7 +35,7 @@ import type {
ExtractionRunUsage, ExtractionRunUsage,
GraphEdge, GraphEdge,
GraphNode, GraphNode,
NodeMapping,
NotesSource, NotesSource,
SessionCheckpoint, SessionCheckpoint,
SessionFindings, SessionFindings,
@@ -65,7 +65,6 @@ function fauxEngine(steps: FauxResponseStep[]): PiEngine {
function memoryLedgers() { function memoryLedgers() {
const nodes: GraphNode[] = []; const nodes: GraphNode[] = [];
const edges: GraphEdge[] = []; const edges: GraphEdge[] = [];
const mappings: NodeMapping[] = [];
const checkpoints: SessionCheckpoint[] = []; const checkpoints: SessionCheckpoint[] = [];
const coverage: CoverageEntry[] = []; const coverage: CoverageEntry[] = [];
const sources = new Map<string, NotesSource>(); const sources = new Map<string, NotesSource>();
@@ -75,8 +74,6 @@ function memoryLedgers() {
readNodes: () => nodes.map((n) => ({ ...n })), readNodes: () => nodes.map((n) => ({ ...n })),
appendEdge: (e: GraphEdge) => void edges.push(e), appendEdge: (e: GraphEdge) => void edges.push(e),
readEdges: () => edges.map((e) => ({ ...e })), readEdges: () => edges.map((e) => ({ ...e })),
appendMapping: (m: NodeMapping) => void mappings.push(m),
readMappings: () => mappings.map((m) => ({ ...m })),
appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c), appendCheckpoint: (c: SessionCheckpoint) => void checkpoints.push(c),
readCheckpoints: () => checkpoints.map((c) => ({ ...c })), readCheckpoints: () => checkpoints.map((c) => ({ ...c })),
// Extraction coverage (vision-extraction-coverage.md) — append-only; // Extraction coverage (vision-extraction-coverage.md) — append-only;
@@ -108,7 +105,6 @@ function memoryLedgers() {
rewriteLedgers: (next) => { rewriteLedgers: (next) => {
if (next.nodes) nodes.splice(0, nodes.length, ...next.nodes.map((n) => ({ ...n, provenance: [...n.provenance], aliases: [...n.aliases] }))); if (next.nodes) nodes.splice(0, nodes.length, ...next.nodes.map((n) => ({ ...n, provenance: [...n.provenance], aliases: [...n.aliases] })));
if (next.edges) edges.splice(0, edges.length, ...next.edges.map((e) => ({ ...e, provenance: [...e.provenance] }))); if (next.edges) edges.splice(0, edges.length, ...next.edges.map((e) => ({ ...e, provenance: [...e.provenance] })));
if (next.mappings) mappings.splice(0, mappings.length, ...next.mappings.map((m) => ({ ...m })));
if (next.coverage) coverage.splice(0, coverage.length, ...next.coverage.map((c) => ({ ...c }))); if (next.coverage) coverage.splice(0, coverage.length, ...next.coverage.map((c) => ({ ...c })));
if (next.checkpoints) checkpoints.splice(0, checkpoints.length, ...next.checkpoints.map((c) => ({ ...c }))); if (next.checkpoints) checkpoints.splice(0, checkpoints.length, ...next.checkpoints.map((c) => ({ ...c })));
}, },
@@ -419,11 +415,11 @@ describe("the extraction pipeline (extractor → auditor → merger)", () => {
// Link-append: the phrasing joined the existing node's aliases and // Link-append: the phrasing joined the existing node's aliases and
// the provenance unioned; p2's create line is its own node. // the provenance unioned; p2's create line is its own node.
assert.equal(ledgers.readNodes().length, 3); // pre-seeded + link line + p2's create assert.equal(ledgers.readNodes().length, 3); // pre-seeded + link line + p2's create
const mappings = ledgers.readMappings(); const linkedLines = ledgers.readNodes().filter((n) => n.id === existingId);
assert.equal(mappings.length, 2); assert.ok(
assert.deepEqual(mappings[0], { chatId, sourceId: "p1", graphNodeId: existingId }); linkedLines.some((line) => line.provenance.some((pt) => pt.chatId === chatId)),
assert.equal(mappings[1]!.sourceId, "p2"); "the link-append unions provenance",
assert.match(mappings[1]!.graphNodeId, /^n_/); );
// COVERAGE checkpointed per message (not a whole-chat checkpoint): // COVERAGE checkpointed per message (not a whole-chat checkpoint):
// one entry per transcript message, fingerprinted content each — // one entry per transcript message, fingerprinted content each —
@@ -1021,7 +1017,6 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
const node = ledgers.readNodes()[0]!; const node = ledgers.readNodes()[0]!;
assert.equal(node.statement, "Vim starts in Normal mode, where keys are commands."); assert.equal(node.statement, "Vim starts in Normal mode, where keys are commands.");
assert.deepEqual(node.provenance, [{ chatId: source.id, messageId: "" }]); assert.deepEqual(node.provenance, [{ chatId: source.id, messageId: "" }]);
assert.deepEqual(ledgers.readMappings(), [{ chatId: source.id, sourceId: "p1", graphNodeId: node.id }]);
// The checkpoint stamps the import. // The checkpoint stamps the import.
assert.equal(ledgers.readCheckpoints()[0]!.chatId, source.id); assert.equal(ledgers.readCheckpoints()[0]!.chatId, source.id);
}); });
@@ -1062,10 +1057,12 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
assert.ok(calls >= 2, `expected ≥2 extractor calls, got ${calls}`); assert.ok(calls >= 2, `expected ≥2 extractor calls, got ${calls}`);
assert.match(views[0]!, new RegExp(`part 1/${calls}`)); assert.match(views[0]!, new RegExp(`part 1/${calls}`));
assert.match(views.at(-1)!, new RegExp(`part ${calls}/${calls}`)); assert.match(views.at(-1)!, new RegExp(`part ${calls}/${calls}`));
// Same findings in every chunk → ONE node total (mapping dedupe // Same findings in every chunk → one node PER CHUNK: jobs merge
// across chunks), stats never double-count. // independently (owner, 2026-09-29 — the mappings ledger is gone);
assert.deepEqual(stats, { nodes: 1, edges: 0 }); // cross-chunk identity is the auditor's linkTo verdict, and these
assert.equal(ledgers.readNodes().length, 1); // fixtures submit none.
assert.deepEqual(stats, { nodes: calls, edges: 0 });
assert.equal(ledgers.readNodes().length, calls);
}); });
it("required top-level fields (owner, 2026-09-28): a findings submit MISSING edges fails validation — no silent edge-less merge", async () => { it("required top-level fields (owner, 2026-09-28): a findings submit MISSING edges fails validation — no silent edge-less merge", async () => {
@@ -1098,7 +1095,7 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
assert.equal(ledgers.readCoverage().length, 0, "nothing checkpointed — the messages stay re-runnable"); assert.equal(ledgers.readCoverage().length, 0, "nothing checkpointed — the messages stay re-runnable");
}); });
it("re-importing the same file is idempotent: same source identity, refs dedupe, usage history grows", async () => { it("re-importing is fingerprint-guarded: unchanged text is a TRUE no-op; changed text re-runs (owner, 2026-09-29)", async () => {
const ledgers = memoryLedgers(); const ledgers = memoryLedgers();
const mk = () => const mk = () =>
createExtractionPipeline({ createExtractionPipeline({
@@ -1110,12 +1107,23 @@ describe("the notes front door (importNotes — format-agnostic)", () => {
const first = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT }); const first = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT });
assert.deepEqual(first, { nodes: 1, edges: 0 }); assert.deepEqual(first, { nodes: 1, edges: 0 });
// Unchanged text: the content fingerprint matches — no extraction
// at all (no model calls, no usage record; the mappings ledger
// used to give this no-op at merge time by accident).
const second = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT }); const second = await mk().importNotes({ filePath: "/vault/learn vim.md", text: NOTES_TEXT });
assert.deepEqual(second, { nodes: 0, edges: 0 }, "the same source id dedupes everything"); assert.deepEqual(second, { nodes: 0, edges: 0 }, "unchanged content is a true no-op");
assert.equal(ledgers.readSources().length, 1, "one source identity — the path hashes to the same id"); assert.equal(ledgers.readSources().length, 1, "one source identity — the path hashes to the same id");
assert.equal(ledgers.readSources()[0]!.extractions?.length, 2, "both runs billed their usage"); assert.equal(ledgers.readSources()[0]!.extractions?.length, 1, "only the real run billed its usage");
assert.equal(ledgers.readNodes().length, 1, "no duplicate node"); assert.equal(ledgers.readNodes().length, 1, "no duplicate node");
// Changed text under the same path: the fingerprint misses — the
// whole file re-runs (fresh merge; the auditor's links would dedupe
// identity on a real model).
const third = await mk().importNotes({ filePath: "/vault/learn vim.md", text: `${NOTES_TEXT}\nA new closing line.` });
assert.deepEqual(third, { nodes: 1, edges: 0 }, "changed content re-extracts");
assert.equal(ledgers.readSources()[0]!.extractions?.length, 2, "the re-run bills its usage");
assert.equal(ledgers.readNodes().length, 2);
}); });
it("aborts like a chat run: true cancellation, nothing merged, partial usage in the source record", async () => { it("aborts like a chat run: true cancellation, nothing merged, partial usage in the source record", async () => {
@@ -1221,9 +1229,11 @@ describe("the chunked pipeline (extraction coverage)", () => {
assert.ok(views[0]!.startsWith("# Chunky coverage fixture")); assert.ok(views[0]!.startsWith("# Chunky coverage fixture"));
// Batch 1 merged the full findings (verdicts-only: the extractor's // Batch 1 merged the full findings (verdicts-only: the extractor's
// output always fully merges); batch 2's identical ids mapped to // output always fully merges); batch 2's identical ids are a
// the same nodes (chat-scoped mapping) — nothing new. // SEPARATE job's namespace (owner, 2026-09-29 — no mapping table)
assert.deepEqual(stats, { nodes: 2, edges: 1 }); // — its propositions merge again as fresh nodes. Cross-batch
// identity is the auditor's linkTo verdict, never the merger's.
assert.deepEqual(stats, { nodes: 4, edges: 2 });
// Per-batch coverage: every message checkpointed, nothing legacy. // Per-batch coverage: every message checkpointed, nothing legacy.
const coverage = ledgers.readCoverage(); const coverage = ledgers.readCoverage();
@@ -1464,10 +1474,11 @@ describe("per-chunk parallel jobs (owner, 2026-09-28)", () => {
assert.equal(extractPeak, 3, "8 chunks, 3 permits — the peak pins both the fan-out and the cap"); assert.equal(extractPeak, 3, "8 chunks, 3 permits — the peak pins both the fan-out and the cap");
assert.equal(auditPeak, 1, "audits serialize behind the gate — never two at once"); assert.equal(auditPeak, 1, "audits serialize behind the gate — never two at once");
// Identical findings in every chunk: one merge, then dedupe — the // Identical findings in every chunk, one merge PER chunk (jobs are
// aggregate counts real emissions, never double. // independent — owner, 2026-09-29): 8 chunks × 2 propositions,
assert.deepEqual(stats, { nodes: 2, edges: 1 }); // 8 × 1 edge. The auditor's links, not the merger, dedupe identity.
assert.equal(ledgers.readNodes().length, 2); assert.deepEqual(stats, { nodes: 16, edges: 8 });
assert.equal(ledgers.readNodes().length, 16);
}); });
}); });
@@ -1496,7 +1507,6 @@ describe("the session-deletion guard (chat deleted mid-run)", () => {
// trace for a later snapshot to resurrect. // trace for a later snapshot to resurrect.
assert.equal(ledgers.readNodes().length, 0); assert.equal(ledgers.readNodes().length, 0);
assert.equal(ledgers.readEdges().length, 0); assert.equal(ledgers.readEdges().length, 0);
assert.equal(ledgers.readMappings().length, 0);
assert.equal(ledgers.readCoverage().length, 0); assert.equal(ledgers.readCoverage().length, 0);
assert.equal(ledgers.readCheckpoints().length, 0); assert.equal(ledgers.readCheckpoints().length, 0);
+1 -8
View File
@@ -37,14 +37,12 @@ function fakeStores(
nodes: GraphNode[], nodes: GraphNode[],
over: { over: {
chats?: Array<Pick<ChatSummary, "id" | "title" | "createdAt" | "updatedAt">>; chats?: Array<Pick<ChatSummary, "id" | "title" | "createdAt" | "updatedAt">>;
mappings?: Array<{ chatId: string; sourceId: string; graphNodeId: string }>;
} = {}, } = {},
): { chatStore: ChatStore; graphStore: GraphStore } { ): { chatStore: ChatStore; graphStore: GraphStore } {
const edges: GraphEdge[] = []; const edges: GraphEdge[] = [];
const graphStore: GraphStore = { const graphStore: GraphStore = {
readNodes: () => nodes, readNodes: () => nodes,
readEdges: () => edges, readEdges: () => edges,
readMappings: () => over.mappings ?? [],
readCheckpoints: () => [], readCheckpoints: () => [],
readCoverage: () => [], readCoverage: () => [],
readSources: () => [], readSources: () => [],
@@ -217,12 +215,7 @@ describe("auditor graph tools", () => {
}); });
it("list_propositions: FULL statement text (identity work), no dates", async () => { it("list_propositions: FULL statement text (identity work), no dates", async () => {
const { chatStore: cs, graphStore: gs } = fakeStores(nodes, { const { chatStore: cs, graphStore: gs } = fakeStores(nodes);
mappings: [
{ chatId: "c1", sourceId: "p1", graphNodeId: "n1" },
{ chatId: "c1", sourceId: "p2", graphNodeId: "n2" },
],
});
const tools = auditorGraphTools({ chatStore: cs, graphStore: gs }); const tools = auditorGraphTools({ chatStore: cs, graphStore: gs });
const out = (await tools.list_propositions!.execute!("tc", { session: "c1" })).details as string; const out = (await tools.list_propositions!.execute!("tc", { session: "c1" })).details as string;
assert.match(out, /2 proposition\(s\) born in this session:/); assert.match(out, /2 proposition\(s\) born in this session:/);
+57 -61
View File
@@ -272,14 +272,8 @@ let chat3Id = "";
* emissions run concurrently against the same files — scope, never assume). */ * emissions run concurrently against the same files — scope, never assume). */
function scopedGraph(chatIds: string[]) { function scopedGraph(chatIds: string[]) {
const graph = getGraph(graphStore); const graph = getGraph(graphStore);
const nodeIds = new Set(
graphStore
.readMappings()
.filter((m) => chatIds.includes(m.chatId))
.map((m) => m.graphNodeId),
);
return { return {
nodes: graph.nodes.filter((n) => nodeIds.has(n.id)), nodes: graph.nodes.filter((n) => n.provenance.some((pt) => chatIds.includes(pt.chatId))),
edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))), edges: graph.edges.filter((e) => e.provenance.some((pt) => chatIds.includes(pt.chatId))),
}; };
} }
@@ -312,21 +306,45 @@ describe("the merger (mergeFindings)", () => {
assert.equal(edge.type, "follows"); assert.equal(edge.type, "follows");
assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "" }]); assert.deepEqual(edge.provenance, [{ chatId: chat1.id, messageId: "" }]);
// Mapping lines: one per (chat, sourceId).
assert.deepEqual(
graphStore.readMappings().map((m) => m.sourceId),
["p1", "p2", "p3"],
);
}); });
it("is idempotent — re-merging the same verdicts appends nothing", () => { it("merges INDEPENDENTLY per call — no cross-run identity (owner, 2026-09-29)", () => {
const stats = mergeFindings(chat1.id, fixtureFindings(), fixtureVerdicts(), graphDeps, sessionSource(chat1.id)); // The mappings ledger is gone: resolution is a job-local in-memory
assert.deepEqual(stats, { nodes: 0, edges: 0 }); // map, so a second merge of the same verdicts knows nothing of the
const graph = scopedGraph([chat1.id]); // first. Cross-run identity is the AUDITOR's job (linkTo) plus the
assert.equal(graph.nodes.length, 3); // coverage ledger (nothing re-extracts) — never the merger's.
assert.equal(graph.edges.length, 1); // (Own chat + unique statements — this suite stacks on shared
// files, so nothing here may touch the other tests' counts.)
const rep = fixtureChat("Independence fixture", [user("u1", "rep")].flat());
const findings: SessionFindings = {
propositions: [
{ id: "p1", name: "Rep one", statement: "Independence one statement", aliases: [] },
{ id: "p2", name: "Rep two", statement: "Independence two statement", aliases: [] },
],
edges: [{ from: "p1", to: "p2", type: "follows" }],
};
const first = mergeFindings(rep.id, findings, { links: [] }, graphDeps, sessionSource(rep.id));
assert.deepEqual(first, { nodes: 2, edges: 1 });
const second = mergeFindings(rep.id, findings, { links: [] }, graphDeps, sessionSource(rep.id));
assert.deepEqual(second, { nodes: 2, edges: 1 }, "nothing remembered — everything re-creates");
const graph = scopedGraph([rep.id]);
assert.equal(graph.nodes.length, 4, "the propositions merge again as fresh nodes");
assert.equal(graph.edges.length, 2, "fresh endpoints — fresh edge assertions");
// exactly one checkpoint line per merge, though — checkpoints are a log // exactly one checkpoint line per merge, though — checkpoints are a log
assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === chat1.id).length, 2); assert.equal(graphStore.readCheckpoints().filter((c) => c.chatId === rep.id).length, 2);
});
it("is idempotent WITHIN one submit — a repeated proposition id resolves once", () => {
const dup = fixtureChat("Duplicate-id fixture", [user("u1", "dup")].flat());
const findings: SessionFindings = {
propositions: [
{ id: "p1", name: "Dup", statement: "Dup statement", aliases: [] },
{ id: "p1", name: "Dup", statement: "Dup statement", aliases: [] },
],
edges: [],
};
const stats = mergeFindings(dup.id, findings, { links: [] }, graphDeps, sessionSource(dup.id));
assert.deepEqual(stats, { nodes: 1, edges: 0 }, "the duplicate id is skipped — first resolution stands");
}); });
it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => { it("applies a linkTo verdict: phrasings join aliases, provenance unions, no duplicate node", () => {
@@ -374,9 +392,6 @@ describe("the merger (mergeFindings)", () => {
alpha.provenance.some((pt) => pt.chatId === chat2.id), alpha.provenance.some((pt) => pt.chatId === chat2.id),
"provenance unions", "provenance unions",
); );
// The mapping records the verdict: (chat2, q1) → the linked node.
const m = graphStore.readMappings().find((x) => x.chatId === chat2.id)!;
assert.deepEqual(m, { chatId: chat2.id, sourceId: "q1", graphNodeId: alphaId });
}); });
it("resolves imported-notes ids to their file paths in the snapshot (static provenance)", () => { it("resolves imported-notes ids to their file paths in the snapshot (static provenance)", () => {
@@ -514,11 +529,6 @@ describe("the merger (mergeFindings)", () => {
const node = getGraph(graphStore).nodes.find((n) => n.statement.startsWith("Vim starts"))!; const node = getGraph(graphStore).nodes.find((n) => n.statement.startsWith("Vim starts"))!;
assert.deepEqual(node.provenance, [{ chatId: "src_notesfixture", messageId: "" }], "one chat-level point — the source key, no message anchors"); assert.deepEqual(node.provenance, [{ chatId: "src_notesfixture", messageId: "" }], "one chat-level point — the source key, no message anchors");
assert.deepEqual(graphStore.readMappings().find((m) => m.chatId === "src_notesfixture"), {
chatId: "src_notesfixture",
sourceId: "p1",
graphNodeId: node.id,
});
// The checkpoint stamps the import time. // The checkpoint stamps the import time.
assert.equal(graphStore.readCheckpoints().find((c) => c.chatId === "src_notesfixture")?.foldedAt, "2026-09-13T00:00:00.000Z"); assert.equal(graphStore.readCheckpoints().find((c) => c.chatId === "src_notesfixture")?.foldedAt, "2026-09-13T00:00:00.000Z");
}); });
@@ -591,7 +601,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
* asserts X→Y; dB links X (its own phrasings), creates Z, asserts X→Z. * asserts X→Y; dB links X (its own phrasings), creates Z, asserts X→Z.
* Deleting dA must: keep X (dB's link line survives, dA's create-line * Deleting dA must: keep X (dB's link line survives, dA's create-line
* aliases stripped — the identity stays), kill Y (orphan), keep Z and * aliases stripped — the identity stays), kill Y (orphan), keep Z and
* dB's edge, kill dA's events/mappings/coverage/checkpoints. */ * dB's edge, kill dA's coverage/checkpoints. */
function buildTwoSessionScenario() { function buildTwoSessionScenario() {
const dA = fixtureChat("Deletion session A", [ const dA = fixtureChat("Deletion session A", [
user("du1", "teach me the deletion topic"), user("du1", "teach me the deletion topic"),
@@ -611,13 +621,12 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
edges: [{ from: "p1", to: "p2", type: "follows", }], edges: [{ from: "p1", to: "p2", type: "follows", }],
}; };
mergeFindings(dA.id, findingsA, { links: [] }, graphDeps, sessionSource(dA.id)); mergeFindings(dA.id, findingsA, { links: [] }, graphDeps, sessionSource(dA.id));
// Resolve through THIS session's mappings — the shared ledger holds // Resolve through THIS session's provenance — the shared ledger
// lookalike statements from earlier tests, so statement-lookup would // holds lookalike statements from earlier tests, so scope by chat.
// link dB to the wrong node. const bornOf = (chatId: string, statement: string): string =>
const mappedId = (sourceId: string): string => getGraph(graphStore).nodes.find((n) => n.statement === statement && n.provenance.some((pt) => pt.chatId === chatId))!.id;
graphStore.readMappings().find((m) => m.chatId === dA.id && m.sourceId === sourceId)!.graphNodeId; const xId = bornOf(dA.id, "Shared core statement");
const xId = mappedId("p1"); const yId = bornOf(dA.id, "Solo fact statement");
const yId = mappedId("p2");
const dB = fixtureChat("Deletion session B", [ const dB = fixtureChat("Deletion session B", [
user("eu1", "more on the shared core"), user("eu1", "more on the shared core"),
@@ -637,7 +646,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
edges: [{ from: "q1", to: "q2", type: "follows", }], edges: [{ from: "q1", to: "q2", type: "follows", }],
}; };
mergeFindings(dB.id, findingsB, { links: [{ id: "q1", linkTo: xId }] }, graphDeps, sessionSource(dB.id)); mergeFindings(dB.id, findingsB, { links: [{ id: "q1", linkTo: xId }] }, graphDeps, sessionSource(dB.id));
const zId = graphStore.readMappings().find((m) => m.chatId === dB.id && m.sourceId === "q2")!.graphNodeId; const zId = bornOf(dB.id, "B extra statement");
// Coverage rows (both chats) + positions garnish — the deletion // Coverage rows (both chats) + positions garnish — the deletion
// prunes both ledgers. // prunes both ledgers.
@@ -658,7 +667,6 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
assert.deepEqual(stats, { assert.deepEqual(stats, {
nodesRemoved: 1, nodesRemoved: 1,
edgesRemoved: 1, edgesRemoved: 1,
mappingsRemoved: 2,
coverageRemoved: 2, coverageRemoved: 2,
checkpointsRemoved: 1, checkpointsRemoved: 1,
} satisfies ChatDeletionStats); } satisfies ChatDeletionStats);
@@ -680,9 +688,6 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
// Edges: dA's assertion X→Y died; dB's X→Z survived. // Edges: dA's assertion X→Y died; dB's X→Z survived.
const scoped = scopedGraph([dA.id, dB.id]); const scoped = scopedGraph([dA.id, dB.id]);
assert.deepEqual(scoped.edges, [{ from: xId, to: zId, type: "follows", provenance: [{ chatId: dB.id, messageId: "" }] }]); assert.deepEqual(scoped.edges, [{ from: xId, to: zId, type: "follows", provenance: [{ chatId: dB.id, messageId: "" }] }]);
// Mappings: dA's audit trail died; dB's stays.
const mappings = graphStore.readMappings().filter((m) => [dA.id, dB.id].includes(m.chatId));
assert.deepEqual(mappings, [{ chatId: dB.id, sourceId: "q1", graphNodeId: xId }, { chatId: dB.id, sourceId: "q2", graphNodeId: zId }]);
// Positions: the orphan pruned. // Positions: the orphan pruned.
const positions = graphStore.readPositions(); const positions = graphStore.readPositions();
assert.equal(positions[yId], undefined); assert.equal(positions[yId], undefined);
@@ -699,7 +704,7 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
graphStore.appendNode({ id: "n_del_append_probe", name: "Probe", statement: "Probe statement", aliases: [], createdAt: "2026-09-23T00:04:00.000Z", provenance: [{ chatId: "chat_foreign_probe", messageId: "" }] }); graphStore.appendNode({ id: "n_del_append_probe", name: "Probe", statement: "Probe statement", aliases: [], createdAt: "2026-09-23T00:04:00.000Z", provenance: [{ chatId: "chat_foreign_probe", messageId: "" }] });
assert.equal(graphStore.readNodes().length, nodesBefore + 1); assert.equal(graphStore.readNodes().length, nodesBefore + 1);
// Idempotent: a second run finds nothing. // Idempotent: a second run finds nothing.
assert.deepEqual(deleteChatTraces(graphStore, dA.id, "delete-nodes"), { ...stats, nodesRemoved: 0, edgesRemoved: 0, mappingsRemoved: 0, coverageRemoved: 0, checkpointsRemoved: 0 } satisfies ChatDeletionStats); assert.deepEqual(deleteChatTraces(graphStore, dA.id, "delete-nodes"), { ...stats, nodesRemoved: 0, edgesRemoved: 0, coverageRemoved: 0, checkpointsRemoved: 0 } satisfies ChatDeletionStats);
}); });
it("preserve-graph: only the extraction bookkeeping dies — the graph ledgers are byte-identical", () => { it("preserve-graph: only the extraction bookkeeping dies — the graph ledgers are byte-identical", () => {
@@ -707,20 +712,17 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
const before = { const before = {
nodes: graphStore.readNodes(), nodes: graphStore.readNodes(),
edges: graphStore.readEdges(), edges: graphStore.readEdges(),
mappings: graphStore.readMappings(),
positions: graphStore.readPositions(), positions: graphStore.readPositions(),
}; };
const stats = deleteChatTraces(graphStore, dA.id, "preserve-graph"); const stats = deleteChatTraces(graphStore, dA.id, "preserve-graph");
assert.deepEqual(stats, { assert.deepEqual(stats, {
nodesRemoved: 0, nodesRemoved: 0,
edgesRemoved: 0, edgesRemoved: 0,
mappingsRemoved: 0,
coverageRemoved: 2, coverageRemoved: 2,
checkpointsRemoved: 1, checkpointsRemoved: 1,
} satisfies ChatDeletionStats); } satisfies ChatDeletionStats);
assert.deepEqual(graphStore.readNodes(), before.nodes); assert.deepEqual(graphStore.readNodes(), before.nodes);
assert.deepEqual(graphStore.readEdges(), before.edges); assert.deepEqual(graphStore.readEdges(), before.edges);
assert.deepEqual(graphStore.readMappings(), before.mappings);
assert.deepEqual(graphStore.readPositions(), before.positions); assert.deepEqual(graphStore.readPositions(), before.positions);
assert.equal(graphStore.readCoverage().some((row) => row.chatId === dA.id), false); assert.equal(graphStore.readCoverage().some((row) => row.chatId === dA.id), false);
assert.equal(graphStore.readCoverage().some((row) => row.chatId === dB.id), true); assert.equal(graphStore.readCoverage().some((row) => row.chatId === dB.id), true);
@@ -765,15 +767,15 @@ describe("session deletion (deleteChatTraces + deleteChatSession, wave 6)", () =
// ─── single-node deletion (node interlude, full death) ─────────────────────── // ─── single-node deletion (node interlude, full death) ───────────────────────
describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlude)", () => { describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlude)", () => {
/** The full-death scenario: node X is born in a notes file (the /** The full-death scenario: node X is born in a notes file (EMPTY
* production orphan shape — EMPTY provenance, the mapping line is the * provenance — a legacy line; the mappings anchor is gone, owner
* anchor), re-encountered by a chat (link line: alias + provenance), * 2026-09-29), re-encountered by a chat (link line: alias +
* asserted in an edge, and evaluated by two sessions. */ * provenance), and asserted in edges by two sessions. */
function buildNodeScenario() { function buildNodeScenario() {
// Clean slate per test: each test in this describe builds its own // Clean slate per test: each test in this describe builds its own
// scenario from empty ledgers (the suite's isolation covers the // scenario from empty ledgers (the suite's isolation covers the
// whole file; this covers describe-internal stacking). // whole file; this covers describe-internal stacking).
graphStore.rewriteLedgers({ nodes: [], edges: [], mappings: [] }); graphStore.rewriteLedgers({ nodes: [], edges: [] });
graphStore.writePositions({}); graphStore.writePositions({});
graphStore.appendNode({ graphStore.appendNode({
id: "n_dead", id: "n_dead",
@@ -804,10 +806,6 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
graphStore.appendEdge({ from: "n_dead", to: "n_keep", type: "follows", provenance: [{ chatId: "chat_node_a", messageId: "m1" }] }); graphStore.appendEdge({ from: "n_dead", to: "n_keep", type: "follows", provenance: [{ chatId: "chat_node_a", messageId: "m1" }] });
graphStore.appendEdge({ from: "n_keep", to: "n_dead", type: "supports", provenance: [{ chatId: "chat_node_b", messageId: "m3" }] }); graphStore.appendEdge({ from: "n_keep", to: "n_dead", type: "supports", provenance: [{ chatId: "chat_node_b", messageId: "m3" }] });
graphStore.appendEdge({ from: "n_keep", to: "n_other", type: "follows", provenance: [{ chatId: "chat_node_b", messageId: "m3" }] }); graphStore.appendEdge({ from: "n_keep", to: "n_other", type: "follows", provenance: [{ chatId: "chat_node_b", messageId: "m3" }] });
// Mappings: the notes anchor + the chat's audit trail.
graphStore.appendMapping({ chatId: "src_nodefile", sourceId: "p1", graphNodeId: "n_dead" });
graphStore.appendMapping({ chatId: "chat_node_a", sourceId: "p1", graphNodeId: "n_dead" });
graphStore.appendMapping({ chatId: "chat_node_a", sourceId: "p2", graphNodeId: "n_keep" });
// Resting positions for both. // Resting positions for both.
graphStore.writePositions({ n_dead: { x: 1, y: 1 }, n_keep: { x: 2, y: 2 } }); graphStore.writePositions({ n_dead: { x: 1, y: 1 }, n_keep: { x: 2, y: 2 } });
} }
@@ -816,9 +814,9 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
buildNodeScenario(); buildNodeScenario();
assert.deepEqual(nodeDeletionImpact(graphStore, "n_dead"), { assert.deepEqual(nodeDeletionImpact(graphStore, "n_dead"), {
edges: 2, edges: 2,
// The file anchor + the re-encounter — every referencing // The re-encounter — the only session provenance can name (the
// session, deduped and sorted. // legacy empty-provenance line names none).
sessions: ["chat_node_a", "src_nodefile"], sessions: ["chat_node_a"],
}); });
// Unknown node: null (the route's 404). // Unknown node: null (the route's 404).
assert.equal(nodeDeletionImpact(graphStore, "n_missing"), null); assert.equal(nodeDeletionImpact(graphStore, "n_missing"), null);
@@ -828,7 +826,7 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
buildNodeScenario(); buildNodeScenario();
const keepLineBefore = JSON.stringify(graphStore.readNodes().find((n) => n.id === "n_keep")); const keepLineBefore = JSON.stringify(graphStore.readNodes().find((n) => n.id === "n_keep"));
const stats = deleteNodeTraces(graphStore, "n_dead"); const stats = deleteNodeTraces(graphStore, "n_dead");
assert.deepEqual(stats, { edgesRemoved: 2, mappingsRemoved: 2 } satisfies NodeDeletionStats); assert.deepEqual(stats, { edgesRemoved: 2 } satisfies NodeDeletionStats);
// The node's EVERY line is gone; the neighbor is byte-identical // The node's EVERY line is gone; the neighbor is byte-identical
// (full death is node-scoped — other nodes' sessions keep their // (full death is node-scoped — other nodes' sessions keep their
@@ -844,8 +842,6 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
graphStore.readEdges().map((e) => [e.from, e.to]), graphStore.readEdges().map((e) => [e.from, e.to]),
[["n_keep", "n_other"]], [["n_keep", "n_other"]],
); );
// The audit trail: X's mappings gone, the neighbor's stay.
assert.deepEqual(graphStore.readMappings().map((m) => m.graphNodeId), ["n_keep"]);
// The resting position died with the node. // The resting position died with the node.
const positions = graphStore.readPositions(); const positions = graphStore.readPositions();
assert.equal(positions["n_dead"], undefined); assert.equal(positions["n_dead"], undefined);
@@ -866,7 +862,7 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
provenance: [{ chatId: "chat_node_solo", messageId: "m1" }], provenance: [{ chatId: "chat_node_solo", messageId: "m1" }],
}); });
const stats = deleteNodeTraces(graphStore, "n_solo"); const stats = deleteNodeTraces(graphStore, "n_solo");
assert.deepEqual(stats, { edgesRemoved: 0, mappingsRemoved: 0 } satisfies NodeDeletionStats); assert.deepEqual(stats, { edgesRemoved: 0 } satisfies NodeDeletionStats);
assert.equal(graphStore.readNodes().some((n) => n.id === "n_solo"), false); assert.equal(graphStore.readNodes().some((n) => n.id === "n_solo"), false);
}); });
@@ -916,7 +912,7 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
const revisionBefore = graphRevision(); const revisionBefore = graphRevision();
const outcome = deleteGraphNode("n_orch"); const outcome = deleteGraphNode("n_orch");
assert.ok(outcome.ok); assert.ok(outcome.ok);
assert.deepEqual(outcome.stats, { edgesRemoved: 0, mappingsRemoved: 0 }); assert.deepEqual(outcome.stats, { edgesRemoved: 0 });
assert.ok(outcome.graphRevision > 0); assert.ok(outcome.graphRevision > 0);
assert.ok(outcome.graphRevision >= revisionBefore, "the version stamp moved"); assert.ok(outcome.graphRevision >= revisionBefore, "the version stamp moved");
assert.equal(graphStore.readNodes().some((n) => n.id === "n_orch"), false); assert.equal(graphStore.readNodes().some((n) => n.id === "n_orch"), false);
-2
View File
@@ -108,7 +108,6 @@ describe("settleGraphLayout", () => {
const store = { const store = {
readNodes: () => lines.map((l) => ({ ...l })), readNodes: () => lines.map((l) => ({ ...l })),
readEdges: () => [edge("a", "b")], readEdges: () => [edge("a", "b")],
readMappings: () => [],
readPositions: () => ({} as Record<string, Position>), readPositions: () => ({} as Record<string, Position>),
writePositions: (p: Record<string, Position>) => void written.push(p), writePositions: (p: Record<string, Position>) => void written.push(p),
}; };
@@ -122,7 +121,6 @@ describe("settleGraphLayout", () => {
settleGraphLayout({ settleGraphLayout({
readNodes: () => [], readNodes: () => [],
readEdges: () => [], readEdges: () => [],
readMappings: () => [],
readPositions: () => ({} as Record<string, Position>), readPositions: () => ({} as Record<string, Position>),
writePositions: (p) => void written.push(p), writePositions: (p) => void written.push(p),
}); });
+7 -44
View File
@@ -38,14 +38,12 @@ function fakeStores(
over: { over: {
chats?: Array<Pick<ChatSummary, "id" | "title" | "createdAt" | "updatedAt">>; chats?: Array<Pick<ChatSummary, "id" | "title" | "createdAt" | "updatedAt">>;
sources?: Array<Pick<NotesSource, "id" | "filePath" | "importedAt">>; sources?: Array<Pick<NotesSource, "id" | "filePath" | "importedAt">>;
mappings?: Array<{ chatId: string; sourceId: string; graphNodeId: string }>;
} = {}, } = {},
): GraphDeps { ): GraphDeps {
const edges: GraphEdge[] = []; const edges: GraphEdge[] = [];
const graphStore: GraphStore = { const graphStore: GraphStore = {
readNodes: () => nodes, readNodes: () => nodes,
readEdges: () => edges, readEdges: () => edges,
readMappings: () => over.mappings ?? [],
readCheckpoints: () => [], readCheckpoints: () => [],
readCoverage: () => [], readCoverage: () => [],
readSources: () => readSources: () =>
@@ -103,46 +101,15 @@ describe("birthSessionPartition", () => {
}); });
}); });
// ─── the notes mapping fallback (owner-approved 2026-09-23) ───────────────── // ─── notes files as birth sessions (production shape) ────────────────────────
describe("the notes mapping-line fallback", () => {
// The PRODUCTION notes shape: importNotes passes messages: null, so
// nodes are born with EMPTY provenance — the file's anchor is the
// mapping line (interlude live-smoke finding: the provenance-only rule
// left every imported file unclustered and invisible to the prober).
it("empty-provenance nodes birth under their first mapping line's session", () => {
const nodes = [
node("a", "Alpha", []),
node("b", "Beta", []),
node("c", "Gamma", []),
node("d", "Delta", [pt("chat1")]), // provenance wins — mapping ignored
];
const mappings = [
{ chatId: "src_notes", sourceId: "p1", graphNodeId: "a" },
{ chatId: "src_notes", sourceId: "p2", graphNodeId: "b" },
{ chatId: "src_notes", sourceId: "p3", graphNodeId: "c" },
// A chat mapping for a node that ALSO has provenance: never consulted.
{ chatId: "chat2", sourceId: "p4", graphNodeId: "d" },
// A mapping for a node that does not exist: ignored.
{ chatId: "src_notes", sourceId: "p5", graphNodeId: "ghost" },
];
const clusters = birthSessionPartition(nodes, mappings);
assert.deepEqual(clusters, [
{ id: "src_notes", members: ["a", "b", "c"] },
{ id: "chat1", members: ["d"] },
], "size-desc, then id asc — src_notes first");
// No mapping and no provenance: unclustered.
assert.deepEqual(birthSessionPartition([node("x", "X", [])], mappings), []);
});
describe("notes files as birth sessions", () => {
// The mapping-line fallback is GONE (owner, 2026-09-29): the mappings
// ledger itself died with per-job in-memory resolution, and every merge
// writes provenance points — the src id IS the birth session.
it("graphSessionIndex lists imported files as kind:file with birth counts (production shape)", () => { it("graphSessionIndex lists imported files as kind:file with birth counts (production shape)", () => {
const nodes = [node("a", "Alpha", []), node("b", "Beta", []), node("c", "Gamma", [])]; const nodes = [node("a", "Alpha", [pt("src_abc")]), node("b", "Beta", [pt("src_abc")]), node("c", "Gamma", [pt("src_abc")])];
const deps = fakeStores(nodes, { const deps = fakeStores(nodes, {
mappings: [
{ chatId: "src_abc", sourceId: "p1", graphNodeId: "a" },
{ chatId: "src_abc", sourceId: "p2", graphNodeId: "b" },
{ chatId: "src_abc", sourceId: "p3", graphNodeId: "c" },
],
sources: [{ id: "src_abc", filePath: "/vault/learn vim.md", importedAt: "2026-09-22T00:00:00.000Z" }], sources: [{ id: "src_abc", filePath: "/vault/learn vim.md", importedAt: "2026-09-22T00:00:00.000Z" }],
}); });
const index = graphSessionIndex(deps); const index = graphSessionIndex(deps);
@@ -159,11 +126,7 @@ describe("the notes mapping-line fallback", () => {
}); });
it("graphSessionPropositions lists the file's births under the src id (a pure birth list, 2026-09-27)", () => { it("graphSessionPropositions lists the file's births under the src id (a pure birth list, 2026-09-27)", () => {
const deps = fakeStores([node("a", "Alpha", []), node("b", "Beta", [])], { const deps = fakeStores([node("a", "Alpha", [pt("src_abc")]), node("b", "Beta", [pt("src_abc")])], {
mappings: [
{ chatId: "src_abc", sourceId: "p1", graphNodeId: "a" },
{ chatId: "src_abc", sourceId: "p2", graphNodeId: "b" },
],
sources: [{ id: "src_abc", filePath: "/vault/learn vim.md", importedAt: "2026-09-22T00:00:00.000Z" }], sources: [{ id: "src_abc", filePath: "/vault/learn vim.md", importedAt: "2026-09-22T00:00:00.000Z" }],
}); });
assert.deepEqual(graphSessionPropositions(deps, "src_abc"), [ assert.deepEqual(graphSessionPropositions(deps, "src_abc"), [