feat: extraction coverage — message-granular ledger + token-batched runs (d132373 vision)
THE MESSAGE IS THE UNIT OF COVERAGE (vision-extraction-coverage.md): a per-chat append-only ledger (coverage.jsonl) records which messages the pipeline has extracted, by content fingerprint over exactly what the extraction view sees (one shared line builder — core/coverage.ts); the uncovered remainder is DERIVED at read time, never dual-tracked. A quiz answer merging into the asking message re-opens only that message; plan-block data-part attach never re-opens anything. - The trigger (turn-service) replaces every-3-quiz-answers with uncovered-tokens >= EXTRACTION.chunkTokens (4096) — quiz-less sessions extract too, content size is the honest signal. - runExtraction is now chunked: token-budgeted batches of the oldest uncovered consecutive messages (a tool call + result never split), each merged batch checkpointing its messages' coverage atomically — an abort or error loses at most the in-flight batch; merged batches stay covered. - Observation refs are batch-tagged (obs:<batchTag>:<sourceId>): re-runs of the same batch dedupe; a re-opened message's fresh understanding lands as a new event. Fixes the old obs:p1 position-collision. - New API: GET /api/graph/coverage (exact per-session derived coverage + running flags, most-uncovered first) and POST /api/graph/coverage/extract (manual run for one session, 409 while in flight). - Graph view chrome: the coverage button opens a panel with every session's covered/uncovered tokens and a per-session Extract button — the manual answer to aborted runs, errors, and dormant tails. Renders even when the graph is empty: an empty graph with uncovered sessions is exactly when it matters. Polls only while open. - Notes imports keep the whole-file checkpoint path (coverage undefined). - Tests: pure-core coverage suite (11), chunked-pipeline suite (3: multi-batch + no-op, abort-keeps-earlier-batches + remainder re-run, re-open alone + data-part invisibility), panel render suite (5); pendingQuizAnswers and its tests deleted as dead code. 486/486. Live-smoked on real GLM: manual extract of "teach me modal realism" ran one batch, wrote 4 coverage entries, panel flipped to "1.3k covered · complete" with the button disabled, graph refetched (nodes 11), usage persisted with real stats.
This commit is contained in:
+1
-50
@@ -20,7 +20,7 @@ import { after, describe, it } from "node:test";
|
||||
import type { UIMessage } from "ai";
|
||||
import type { AuditResult } from "../src/core/contracts.ts";
|
||||
import type { QuestionAnswerData } from "../src/core/question-core.ts";
|
||||
import { clusterContentId, getGraph, graphSnapshot, mergeFindings, pendingQuizAnswers, type GraphDeps } from "../src/core/graph.ts";
|
||||
import { clusterContentId, getGraph, graphSnapshot, mergeFindings, type GraphDeps } from "../src/core/graph.ts";
|
||||
import { graphStore } from "../src/store/graph.ts";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
import { LEDGER_FILES, PATHS, SOURCES_FILE, EMBEDDINGS_FILE } from "../config.ts";
|
||||
@@ -564,55 +564,6 @@ describe("the merger (mergeFindings)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── the trigger helper ──────────────────────────────────────────────────────
|
||||
|
||||
describe("the trigger helper (pendingQuizAnswers)", () => {
|
||||
it("counts answered quizzes not yet in the ledger — dismissed/ask never count", () => {
|
||||
const chat = fixtureChat("Trigger fixture", [
|
||||
user("u1", "go"),
|
||||
assistant("a1", [
|
||||
quizPart(
|
||||
"call-s",
|
||||
{ question: "QS", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-s", questionText: "QS text", correct: true, answeredAt: "2026-08-22T03:00:00.000Z" }),
|
||||
),
|
||||
quizPart(
|
||||
"call-s2",
|
||||
{ question: "QS2", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-s2", questionText: "QS2 text", dismissed: true }),
|
||||
),
|
||||
{
|
||||
type: "tool-ask_user_question",
|
||||
toolCallId: "call-s3",
|
||||
state: "output-available",
|
||||
input: { question: "free?" },
|
||||
output: answer({ questionId: "call-s3", kind: "ask", questionText: "free?", text: "y" }),
|
||||
},
|
||||
]),
|
||||
]);
|
||||
|
||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1, "one answered quiz pending");
|
||||
assert.equal(pendingQuizAnswers("chat_does_not_exist", graphDeps), 0);
|
||||
|
||||
mergeFindings(chat.id, fixtureAudit(), graphDeps, sessionSource(chat.id));
|
||||
// call-s is unattributed here (fixtureAudit has no call-s attribution)
|
||||
// — but it still becomes an event, so nothing stays pending.
|
||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 0, "extracted → nothing pending");
|
||||
|
||||
// A NEW answered quiz raises the count again (the re-trigger path).
|
||||
chatStore.saveMessages(chat.id, [
|
||||
...chatStore.getChat(chat.id)!.messages,
|
||||
assistant("a3", [
|
||||
quizPart(
|
||||
"call-s4",
|
||||
{ question: "QS4", options: [{ label: "a" }, { label: "b" }], correctAnswer: "a", explanation: "e" },
|
||||
answer({ questionId: "call-s4", questionText: "QS4 text", correct: false, answeredAt: "2026-08-22T03:01:00.000Z" }),
|
||||
),
|
||||
]),
|
||||
]);
|
||||
assert.equal(pendingQuizAnswers(chat.id, graphDeps), 1);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── store primitives ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
Reference in New Issue
Block a user