Extraction-UX data layer: the chunk budget becomes a graph setting; the chat view gets a per-session coverage read
Owner-locked extraction-UX redesign (dispatch 2026-09-24), part 1 (data
layer; the UI chunk follows).
THRESHOLD -> SETTING
- config.ts's EXTRACTION.chunkTokens dies; the default moves into
FACTORY_SETTINGS.graph.extractionChunkTokens (4096, the old constant's
value). GraphSettings gains the optional field, Required at resolution
via the standard ?? FACTORY pattern — pre-redesign records resolve the
default cleanly.
- FLOOR 500 (the implementer's call, documented on the field): every run
pays the graph-carrying prompt overhead out of the write channel's
money — a smaller budget fires penny-batches that cost more per
extracted token than the content is worth. The sanitizer CLAMPS up
(bd68486 clamp-never-reject discipline: fractions floor, junk drops);
the settings PUT REJECTS below-floor with the rationale in the message.
No ceiling: a large budget just delays extraction — the user's stated
cost/thoroughness tradeoff, not a misconfiguration to police.
- The trigger (turn-service.triggerExtraction) takes the budget as a
parameter — chat-handler passes effective.graph.extractionChunkTokens,
the fresh per-turn read, same as every other gate.
- The pipeline's BATCH side threads through the same number:
ExtractionPipelineDeps gains an OPTIONAL getChunkTokens getter with the
factory default (Teacher's 24 test fixtures stay untouched; the two
call-site lines inside extraction.ts are the only cross-band edits,
flagged). chat-handler + backfill/import-notes wire the live
effectiveSettings — trigger and batches share one budget, always.
PER-SESSION COVERAGE READ
- sessionCoverage(chatId) (server/chats.ts): the chat view's ring-meter
block — totalMessages/coveredCount/uncoveredCount/coveredTokens/
uncoveredTokens (the same coverageOf derivation /api/graph/coverage
serves, scoped to one chat), running (chat-kind extraction registered
for this id), and lastFoldedAt (the chat's newest coverage row's fold
— a cheap single pass; null before any fold). GET /api/chats/[id]
carries it as coverage: {...}; unknown chat -> null (the route 404s
first).
- /api/graph/coverage (the all-sessions list) compiles untouched — the
per-session read is a scoping of it, not a supersession; the UI's
sidebar/ring chunk decides which surface it consumes.
Tests: factory/sanitizer pins (clamp-up, fraction-floor, junk-drop),
resolution pins (absent -> 4096, stored 2000 wins), settings-body pins
(valid/absent/fraction/below-floor 400/junk 400), sessionCoverage pins
(derived split, newest-fold scoping, cross-session isolation, running via
the registry, null on unknown chat). 893/893.
This commit is contained in:
@@ -890,7 +890,7 @@ function passThroughAuditStep(): FauxResponseStep {
|
||||
}
|
||||
|
||||
/** A session with UNCOVERED-multiple-batch mass: five long distinct messages
|
||||
* (~1200 tok each — 3600+1200 > EXTRACTION.chunkTokens 4096 → batches of
|
||||
* (~1200 tok each — 3600+1200 > the factory chunk budget 4096 → batches of
|
||||
* [m1,m2,m3] and [m4,m5]). */
|
||||
function chunkyChat() {
|
||||
const chat = chatStore.createChat("Chunky coverage fixture");
|
||||
|
||||
+70
-1
@@ -21,6 +21,7 @@ import type { UIMessage } from "ai";
|
||||
import type { AuditResult, NamingRunUsage } from "../src/core/contracts.ts";
|
||||
import type { QuestionAnswerData } from "../src/core/question-core.ts";
|
||||
import { getGraph, graphSnapshot, mergeFindings, type GraphDeps } from "../src/core/graph.ts";
|
||||
import { coverageSlots } from "../src/core/coverage.ts";
|
||||
import { graphStore } from "../src/store/graph.ts";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
import { LEDGER_FILES, PATHS, SOURCES_FILE } from "../config.ts";
|
||||
@@ -618,7 +619,7 @@ describe("graph ledger store", () => {
|
||||
// ─── session deletion (wave 6 — the first destructive ledger operation) ─────
|
||||
|
||||
import { deleteChatTraces, type ChatDeletionStats } from "../src/core/graph-deletion.ts";
|
||||
import { deleteChatSession } from "../src/server/chats.ts";
|
||||
import { deleteChatSession, sessionCoverage } from "../src/server/chats.ts";
|
||||
import { beginTurn, endTurn } from "../src/server/turn-liveness.ts";
|
||||
import { jobStore } from "../src/store/jobs.ts";
|
||||
import {
|
||||
@@ -989,3 +990,71 @@ describe("single-node deletion (deleteNodeTraces + deleteGraphNode, node interlu
|
||||
assert.deepEqual(deleteGraphNode("n_orch"), { ok: false, status: 404, code: "not_found", error: "No such node" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("session coverage read (extraction-UX redesign, 2026-09-24)", () => {
|
||||
it("one session's block: derived uncovered, the newest fold, honest nulls", () => {
|
||||
// Empty coverage: everything uncovered, no fold yet, not running.
|
||||
const chat = fixtureChat("coverage read", [
|
||||
user("cov_u1", "tell me about coverage reads"),
|
||||
user("cov_u2", "and about per-session blocks"),
|
||||
user("cov_u3", "a third distinct message"),
|
||||
]);
|
||||
const empty = sessionCoverage(chat.id)!;
|
||||
assert.ok(empty);
|
||||
assert.equal(empty.totalMessages, 3);
|
||||
assert.equal(empty.coveredCount, 0);
|
||||
assert.equal(empty.uncoveredCount, 3);
|
||||
assert.equal(empty.coveredTokens, 0);
|
||||
assert.ok(empty.uncoveredTokens > 0);
|
||||
assert.equal(empty.running, false);
|
||||
assert.equal(empty.lastFoldedAt, null, "no fold yet");
|
||||
|
||||
// Fold the first two messages: the read derives the split + the
|
||||
// newest fold timestamp from the ledger rows.
|
||||
const slots = coverageSlots(chatStore.getChat(chat.id)!.messages as UIMessage[]);
|
||||
graphStore.appendCoverage({ chatId: chat.id, fp: slots[0]!.fp, tokens: slots[0]!.tokens, foldedAt: "2026-09-24T10:00:00.000Z" });
|
||||
graphStore.appendCoverage({ chatId: chat.id, fp: slots[1]!.fp, tokens: slots[1]!.tokens, foldedAt: "2026-09-24T11:30:00.000Z" });
|
||||
// Another session's row must not leak into this read.
|
||||
graphStore.appendCoverage({ chatId: "chat_other", fp: slots[2]!.fp, tokens: slots[2]!.tokens, foldedAt: "2026-09-24T23:00:00.000Z" });
|
||||
|
||||
const cov = sessionCoverage(chat.id)!;
|
||||
assert.equal(cov.totalMessages, 3);
|
||||
assert.equal(cov.coveredCount, 2);
|
||||
assert.equal(cov.uncoveredCount, 1);
|
||||
assert.equal(cov.coveredTokens, slots[0]!.tokens + slots[1]!.tokens);
|
||||
assert.equal(cov.uncoveredTokens, slots[2]!.tokens);
|
||||
assert.equal(cov.lastFoldedAt, "2026-09-24T11:30:00.000Z", "the NEWEST fold, and scoped to this session's rows");
|
||||
});
|
||||
|
||||
it("running: true only for THIS session's chat-kind run (the registry pin)", () => {
|
||||
const chat = fixtureChat("coverage running", [user("cov_r1", "running pin")]);
|
||||
const g = globalThis as { __lhExtractionRuns?: unknown };
|
||||
const realRuns = g.__lhExtractionRuns;
|
||||
try {
|
||||
g.__lhExtractionRuns = {
|
||||
running: new Map([
|
||||
// a notes run for the same id is NOT this session's chat run
|
||||
["src_notes", { chatId: "src_notes", kind: "notes", phase: "extracting", startedAt: 1, usage: {}, controller: new AbortController() }],
|
||||
["other_chat", { chatId: "other_chat", kind: "chat", phase: "auditing", startedAt: 2, usage: {}, controller: new AbortController() }],
|
||||
]),
|
||||
inFlight: new Set(),
|
||||
lastStartedAt: 0,
|
||||
};
|
||||
assert.equal(sessionCoverage(chat.id)!.running, false, "no chat-kind run for this id");
|
||||
g.__lhExtractionRuns = {
|
||||
running: new Map([
|
||||
[chat.id, { chatId: chat.id, kind: "chat", phase: "extracting", startedAt: 3, usage: {}, controller: new AbortController() }],
|
||||
]),
|
||||
inFlight: new Set(),
|
||||
lastStartedAt: 0,
|
||||
};
|
||||
assert.equal(sessionCoverage(chat.id)!.running, true);
|
||||
} finally {
|
||||
g.__lhExtractionRuns = realRuns;
|
||||
}
|
||||
});
|
||||
|
||||
it("unknown chat → null (the caller owns the 404)", () => {
|
||||
assert.equal(sessionCoverage("chat_never_was"), null);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -162,6 +162,28 @@ describe("parseGraphSettingsBody", () => {
|
||||
assert.equal(parseGraphSettingsBody("nope").ok, false);
|
||||
});
|
||||
|
||||
it("extractionChunkTokens: whole tokens ≥ the 500 floor (extraction-UX redesign, 2026-09-24)", () => {
|
||||
const ok = parseGraphSettingsBody({ enabled: true, extractionChunkTokens: 2048 });
|
||||
assert.ok(ok.ok);
|
||||
assert.deepEqual(ok.settings, { enabled: true, extractionChunkTokens: 2048 });
|
||||
// absent = unset — replace-whole keeps the factory default at resolution
|
||||
const absent = parseGraphSettingsBody({ enabled: true });
|
||||
assert.ok(absent.ok);
|
||||
assert.deepEqual(absent.settings, { enabled: true });
|
||||
// fractions floor to whole tokens
|
||||
const fraction = parseGraphSettingsBody({ extractionChunkTokens: 1024.9 });
|
||||
assert.ok(fraction.ok);
|
||||
assert.equal(fraction.settings.extractionChunkTokens, 1024);
|
||||
// below the cost floor: 400 with the floor named (penny-batches)
|
||||
const tiny = parseGraphSettingsBody({ extractionChunkTokens: 100 });
|
||||
assert.equal(tiny.ok, false, "below-floor rejected");
|
||||
assert.match(tiny.ok === false ? tiny.error : "", /at least 500/);
|
||||
// junk shapes: 400
|
||||
assert.equal(parseGraphSettingsBody({ extractionChunkTokens: "4096" }).ok, false);
|
||||
assert.equal(parseGraphSettingsBody({ extractionChunkTokens: NaN }).ok, false);
|
||||
assert.equal(parseGraphSettingsBody({ extractionChunkTokens: null }).ok, false);
|
||||
});
|
||||
|
||||
it("the dead wave-6 rungs in a body are ignored — the section collapses on write", () => {
|
||||
// Old clients may still send {embedding: true, embeddingModel: …};
|
||||
// the parser reads ONLY the master switch — the dead keys are not
|
||||
|
||||
@@ -42,7 +42,9 @@ describe("resolveSettings", () => {
|
||||
assert.equal(eff.appearance.proseFontSize, 16);
|
||||
assert.equal(eff.appearance.sessionNameSerif, true);
|
||||
assert.equal(eff.appearance.mdRendering.headingWeight, 900);
|
||||
assert.deepEqual(eff.graph, { enabled: true }); // wave 7 collapse: the master switch ONLY
|
||||
// wave 7 collapse: the master switch ONLY + the redesign's chunk
|
||||
// budget — absent record resolves the factory 4096 (the old config constant)
|
||||
assert.deepEqual(eff.graph, { enabled: true, extractionChunkTokens: 4096 });
|
||||
});
|
||||
|
||||
it("explicit record values win over every factory default", () => {
|
||||
@@ -75,8 +77,12 @@ describe("resolveSettings", () => {
|
||||
assert.equal(eff.appearance.mode, "light");
|
||||
assert.equal(eff.appearance.proseFontSize, 20);
|
||||
assert.equal(eff.appearance.mdRendering.headingWeight, 600);
|
||||
// wave 7 collapse: the section IS the master switch.
|
||||
assert.deepEqual(eff.graph, { enabled: false });
|
||||
// wave 7 collapse: the section IS the master switch (absent budget
|
||||
// field = factory 4096 — a pre-redesign record resolves cleanly).
|
||||
assert.deepEqual(eff.graph, { enabled: false, extractionChunkTokens: 4096 });
|
||||
// The chunk budget rides the same resolution: stored value wins.
|
||||
const budgeted = resolveSettings({ version: 1, graph: { extractionChunkTokens: 2000 } });
|
||||
assert.deepEqual(budgeted.graph, { enabled: true, extractionChunkTokens: 2000 });
|
||||
});
|
||||
|
||||
it("partial role override: thinkingLevel alone borrows the SLOT's model (or nothing, without one)", () => {
|
||||
|
||||
+18
-1
@@ -21,7 +21,9 @@ describe("FACTORY_SETTINGS (the compiled factory layer)", () => {
|
||||
it("pins the owner-decided defaults", () => {
|
||||
assert.equal(CURRENT_SETTINGS_VERSION, 1);
|
||||
assert.deepEqual(FACTORY_SETTINGS.general, { autoUpdate: true });
|
||||
assert.deepEqual(FACTORY_SETTINGS.graph, { enabled: true }); // wave 7 collapse: the master switch ONLY
|
||||
// wave 7 collapse: the master switch ONLY + the extraction-UX
|
||||
// redesign's chunk budget (2026-09-24) — default = the old config constant
|
||||
assert.deepEqual(FACTORY_SETTINGS.graph, { enabled: true, extractionChunkTokens: 4096 });
|
||||
const a = FACTORY_SETTINGS.appearance;
|
||||
assert.equal(a.mode, "dark");
|
||||
assert.equal(a.background, null); // the mode's own default, not a factory hex
|
||||
@@ -106,6 +108,21 @@ describe("sanitizeUserSettings", () => {
|
||||
assert.deepEqual(out.agent, {});
|
||||
assert.deepEqual(out.appearance, {});
|
||||
assert.deepEqual(out.graph, {});
|
||||
|
||||
// extractionChunkTokens (extraction-UX redesign, 2026-09-24):
|
||||
// clamp-never-reject — a finite positive value survives (fractions
|
||||
// floor), below-floor values clamp UP to the cost floor, junk drops
|
||||
// (the factory default resolves at read time).
|
||||
const budget = sanitizeUserSettings({ version: 1, graph: { enabled: true, extractionChunkTokens: 2048 } });
|
||||
assert.deepEqual(budget?.graph, { enabled: true, extractionChunkTokens: 2048 });
|
||||
const fraction = sanitizeUserSettings({ version: 1, graph: { extractionChunkTokens: 1024.9 } });
|
||||
assert.equal(fraction?.graph?.extractionChunkTokens, 1024, "fractions floor to whole tokens");
|
||||
const tiny = sanitizeUserSettings({ version: 1, graph: { extractionChunkTokens: 10 } });
|
||||
assert.equal(tiny?.graph?.extractionChunkTokens, 500, "below the floor clamps UP — penny-batches are a cost bug, not a preference");
|
||||
for (const junk of [0, -1000, NaN, Infinity, "4096", null, true]) {
|
||||
const dropped = sanitizeUserSettings({ version: 1, graph: { extractionChunkTokens: junk } });
|
||||
assert.equal(dropped?.graph?.extractionChunkTokens, undefined, `junk ${String(junk)} drops`);
|
||||
}
|
||||
});
|
||||
|
||||
it("out-of-range numbers CLAMP (proseFontSize, headingWeight)", () => {
|
||||
|
||||
Reference in New Issue
Block a user