Extraction-UX data layer: the chunk budget becomes a graph setting; the chat view gets a per-session coverage read

Owner-locked extraction-UX redesign (dispatch 2026-09-24), part 1 (data
layer; the UI chunk follows).

THRESHOLD -> SETTING
- config.ts's EXTRACTION.chunkTokens dies; the default moves into
  FACTORY_SETTINGS.graph.extractionChunkTokens (4096, the old constant's
  value). GraphSettings gains the optional field, Required at resolution
  via the standard ?? FACTORY pattern — pre-redesign records resolve the
  default cleanly.
- FLOOR 500 (the implementer's call, documented on the field): every run
  pays the graph-carrying prompt overhead out of the write channel's
  money — a smaller budget fires penny-batches that cost more per
  extracted token than the content is worth. The sanitizer CLAMPS up
  (bd68486 clamp-never-reject discipline: fractions floor, junk drops);
  the settings PUT REJECTS below-floor with the rationale in the message.
  No ceiling: a large budget just delays extraction — the user's stated
  cost/thoroughness tradeoff, not a misconfiguration to police.
- The trigger (turn-service.triggerExtraction) takes the budget as a
  parameter — chat-handler passes effective.graph.extractionChunkTokens,
  the fresh per-turn read, same as every other gate.
- The pipeline's BATCH side threads through the same number:
  ExtractionPipelineDeps gains an OPTIONAL getChunkTokens getter with the
  factory default (Teacher's 24 test fixtures stay untouched; the two
  call-site lines inside extraction.ts are the only cross-band edits,
  flagged). chat-handler + backfill/import-notes wire the live
  effectiveSettings — trigger and batches share one budget, always.

PER-SESSION COVERAGE READ
- sessionCoverage(chatId) (server/chats.ts): the chat view's ring-meter
  block — totalMessages/coveredCount/uncoveredCount/coveredTokens/
  uncoveredTokens (the same coverageOf derivation /api/graph/coverage
  serves, scoped to one chat), running (chat-kind extraction registered
  for this id), and lastFoldedAt (the chat's newest coverage row's fold
  — a cheap single pass; null before any fold). GET /api/chats/[id]
  carries it as coverage: {...}; unknown chat -> null (the route 404s
  first).
- /api/graph/coverage (the all-sessions list) compiles untouched — the
  per-session read is a scoping of it, not a supersession; the UI's
  sidebar/ring chunk decides which surface it consumes.

Tests: factory/sanitizer pins (clamp-up, fraction-floor, junk-drop),
resolution pins (absent -> 4096, stored 2000 wins), settings-body pins
(valid/absent/fraction/below-floor 400/junk 400), sessionCoverage pins
(derived split, newest-fold scoping, cross-session isolation, running via
the registry, null on unknown chat). 893/893.

src/agent/subagents/extraction.ts

@@ -4,7 +4,7 @@ import type { Usage } from "@earendil-works/pi-ai";
 import { z } from "zod";
 import { lhTool, toPiTools } from "../lh-tool.ts";
 import type { ModelRole } from "../../../config.ts";
-import { ModelsNotConfiguredError, ROLE_SLOT, type ModelSlotName } from "../../core/settings.ts";
+import { FACTORY_SETTINGS, ModelsNotConfiguredError, ROLE_SLOT, type ModelSlotName } from "../../core/settings.ts";
 import type { PiEngine } from "../models.ts";
 import { runSubagent } from "./pi-run.ts";
 import type {
@@ -20,7 +20,6 @@ import { getGraph, mergeFindings, type EmissionStats, type GraphDeps } from "../
 import type { GraphStore } from "../../core/ports.ts";
 import { settleGraphLayout } from "../../core/layout-settle.ts";
 import { batchRuns, batchTag, coverageOf, messageViewLines } from "../../core/coverage.ts";
-import { EXTRACTION } from "../../../config.ts";
 
 /**
  * The extraction pipeline (vision §1 construction, 2026-09-02 redesign) — the
@@ -569,7 +568,15 @@ export interface ExtractionModels {
 	engineFor?: (role: ModelRole) => PiEngine | undefined;
 }
 
-export interface ExtractionPipelineDeps extends GraphDeps, ExtractionModels {}
+export interface ExtractionPipelineDeps extends GraphDeps, ExtractionModels {
+	/** The chunk budget — the transcript-token share of the extractor's
+	 *  context; batches fill to it (the trigger fires on the same number,
+	 *  read fresh per turn by the composition root). OPTIONAL with the
+	 *  factory default (4096) so test fixtures stay minimal; the server
+	 *  callers wire the live effective settings (extraction-UX redesign,
+	 *  2026-09-24). */
+	getChunkTokens?: () => number;
+}
 
 export interface NotesImportInput {
 	/** The source's file path — its identity (deterministic src id) AND its
@@ -760,10 +767,11 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
 				console.info(`[extraction ${chatId}] nothing uncovered — no-op`);
 				return { nodes: 0, edges: 0, events: 0 };
 			}
-			const batches = batchRuns(covered.uncovered, EXTRACTION.chunkTokens);
+			const chunkTokens = deps.getChunkTokens?.() ?? FACTORY_SETTINGS.graph.extractionChunkTokens;
+			const batches = batchRuns(covered.uncovered, chunkTokens);
 			console.info(
 				`[extraction ${chatId}] pipeline start — ${covered.uncoveredTokens} uncovered tok, ` +
-					`${covered.uncoveredCount} messages, ${batches.length} batch(es) of ≤${EXTRACTION.chunkTokens} tok`,
+					`${covered.uncoveredCount} messages, ${batches.length} batch(es) of ≤${chunkTokens} tok`,
 			);
 			for (const batch of batches) {
 				// The batch's prompt view: SESSION header + the batch's lines,

src/core/settings-resolution.ts

@@ -205,6 +205,10 @@ export function resolveSettings(record: UserSettings | null): EffectiveSettings
 			// the embedding/clustering/labeling rungs and the BYOK embeddingModel
 			// are gone (stale record keys are simply never read here).
 			enabled: graph.enabled ?? FACTORY_SETTINGS.graph.enabled,
+			// The chunk budget (extraction-UX redesign, 2026-09-24): absent
+			// (pre-redesign record, fresh install) resolves to the factory
+			// 4096; the sanitizer has already clamped/floored any stored value.
+			extractionChunkTokens: graph.extractionChunkTokens ?? FACTORY_SETTINGS.graph.extractionChunkTokens,
 		},
 	};
 }
@@ -216,7 +220,7 @@ export function resolveSettings(record: UserSettings | null): EffectiveSettings
  * switch. Used by the chat-handler's post-turn trigger. (Wave 7: the master
  * switch is the ONLY graph setting left — the section collapsed.)
  */
-export function extractionEnabled(graph: Required<GraphSettings>): boolean {
+export function extractionEnabled(graph: { enabled: boolean }): boolean {
 	return graph.enabled;
 }
 
@@ -228,7 +232,7 @@ export function extractionEnabled(graph: Required<GraphSettings>): boolean {
  * Teacher's tool factories consume this at registration time.
  */
 export function graphToolsEnabled(effective: {
-	graph: Required<GraphSettings>;
+	graph: { enabled: boolean };
 	agent: { graphTools: { enabled?: boolean } };
 }): boolean {
 	return effective.graph.enabled && (effective.agent.graphTools.enabled ?? true);

src/core/settings.ts

@@ -255,6 +255,26 @@ export interface GraphSettings {
 	 *  names, and the prober graph tools read the ledgers directly — there
 	 *  is nothing left to configure. Data stays stored either way. */
 	enabled?: boolean;
+	/** The chunk budget (extraction-UX redesign, owner 2026-09-24): the
+	 *  transcript-token share of the extractor's context — a run fires
+	 *  when a chat's UNCOVERED tokens reach it, and the run batches the
+	 *  oldest uncovered messages into runs of ≤ this many tokens. It
+	 *  bounds both cost and chunk size (the extractor prompt also
+	 *  carries the whole graph + instructions, so a smaller budget means
+	 *  more, smaller runs — each paying that fixed overhead).
+	 *
+	 *  FACTORY DEFAULT 4096 (the pre-setting constant, config.ts's
+	 *  EXTRACTION.chunkTokens — vision-extraction-coverage.md, 2026-09-15).
+	 *
+	 *  FLOOR 500 (EXTRACTION_CHUNK_TOKENS_FLOOR, the implementer's call
+	 *  per the owner's dispatch): every run pays the graph-carrying
+	 *  prompt overhead out of the write channel's money — a budget below
+	 *  ~500 tokens would fire penny-batches that cost more per extracted
+	 *  token than the content is worth, so the sanitizer clamps and the
+	 *  PUT rejects below it. NO CEILING on purpose: a large budget just
+	 *  delays extraction — the user's stated cost/thoroughness tradeoff,
+	 *  not a misconfiguration to police. */
+	extractionChunkTokens?: number;
 }
 
 /** THE record. Every section optional (delta); unknown top-level keys are
@@ -307,9 +327,14 @@ export const FACTORY_SETTINGS: FactorySettings = {
 		sessionNameSerif: true,
 		mdRendering: { headingWeight: 900, boldColor: null, italicColor: null },
 	},
-	graph: { enabled: true },
+	graph: { enabled: true, extractionChunkTokens: 4096 },
 };
 
+/** The chunk budget's validated floor (GraphSettings.extractionChunkTokens
+ *  — see its doc for the penny-batch rationale; the settings PUT names this
+ *  in its rejection, the sanitizer clamps to it). */
+export const EXTRACTION_CHUNK_TOKENS_FLOOR = 500;
+
 // ─── Sanitizer ─────────────────────────────────────────────────────────────
 
 const PROSE_FONT_MIN = 12;
@@ -462,9 +487,20 @@ function sanitizeGraph(raw: Record<string, unknown>): GraphSettings {
 	// this reconstruction — the migration story: the next graph-section
 	// write (PUT is replace-whole) persists the collapsed shape, and no
 	// active code ever looks at the stale fields.
+	//
+	// extractionChunkTokens (extraction-UX redesign, 2026-09-24): the chunk
+	// budget — clamp-never-reject (the bd68486 discipline): a finite
+	// positive number floors to EXTRACTION_CHUNK_TOKENS_FLOOR (fractions
+	// floor to integers), anything else drops to the factory default at
+	// resolution. A disk record below the floor (older app, hand edit)
+	// clamps UP — the floor is a cost protection, not data to honor.
 	const out: GraphSettings = {};
 	const enabled = boolOrDrop(raw.enabled);
 	if (enabled !== undefined) out.enabled = enabled;
+	const rawTokens = raw.extractionChunkTokens;
+	if (typeof rawTokens === "number" && Number.isFinite(rawTokens) && rawTokens > 0) {
+		out.extractionChunkTokens = Math.max(EXTRACTION_CHUNK_TOKENS_FLOOR, Math.floor(rawTokens));
+	}
 	return out;
 }
 

src/server/chat-handler.ts

@@ -317,7 +317,10 @@ export async function handleChatRequest(
 				// pipeline knobs — embedding/clustering/labeling — gate inside the
 				// pipeline and its consumers, later waves, via the same resolved view.)
 				if (extractionEnabled(effective.graph)) {
-					triggerExtraction(stores, extraction, chatId);
+					// Fresh per-turn read of the graph section's chunk budget
+					// (extraction-UX redesign, 2026-09-24): the trigger compares
+					// against it and the pipeline's batches share it.
+					triggerExtraction(stores, extraction, chatId, effective.graph.extractionChunkTokens);
 				}
 				// LAST: a poller that sees not-in-flight must read an already-persisted
 				// record — endTurn only after persistTurn has done its job.

src/server/chats.ts

@@ -1,6 +1,8 @@
 import { abortExtraction, extractionStatuses } from "../agent/subagents/extraction.ts";
+import { coverageOf } from "../core/coverage.ts";
 import { deleteChatTraces, type ChatDeleteMode, type ChatDeletionStats } from "../core/graph-deletion.ts";
 import { titleFromPrompt } from "../core/contracts.ts";
+import type { UIMessage } from "ai";
 import { graphRevision } from "./graph.ts";
 import { stores } from "./stores.ts";
 import { isTurnRunning } from "./turn-liveness.ts";
@@ -110,3 +112,47 @@ export async function deleteChatSession(chatId: string, mode: ChatDeleteMode): P
 	stores.chatStore.deleteChat(chatId);
 	return { ok: true, mode, stats, jobsRemoved, graphRevision: graphRevision() };
 }
+
+/** One session's coverage read (extraction-UX redesign, owner dispatch
+ *  2026-09-24): the chat view's ring meter + popover data — the same
+ *  derivation the all-sessions list serves (/api/graph/coverage), scoped
+ *  to one chat. "Uncovered" stays DERIVED per request (fingerprint the
+ *  transcript, diff the coverage ledger — never dual-tracked).
+ *
+ *  lastFoldedAt: the newest coverage row's fold timestamp — a cheap single
+ *  pass over the chat's ledger rows (the ring popover's "last extraction"
+ *  line); null before the session's first fold. */
+export interface SessionCoverage {
+	totalMessages: number;
+	coveredCount: number;
+	uncoveredCount: number;
+	coveredTokens: number;
+	uncoveredTokens: number;
+	/** True while a chat-kind extraction run is registered for this session. */
+	running: boolean;
+	/** ISO of the newest fold in this session's coverage rows; null before any. */
+	lastFoldedAt: string | null;
+}
+
+/** The per-session coverage block (see SessionCoverage). Null when the
+ *  chat doesn't exist — callers own the 404. */
+export function sessionCoverage(chatId: string): SessionCoverage | null {
+	const record = stores.chatStore.getChat(chatId);
+	if (!record) return null;
+	const entries = stores.graphStore.readCoverage();
+	const cov = coverageOf(record.messages as UIMessage[], entries, chatId);
+	let lastFoldedAt: string | null = null;
+	for (const entry of entries) {
+		if (entry.chatId !== chatId) continue;
+		if (lastFoldedAt === null || entry.foldedAt > lastFoldedAt) lastFoldedAt = entry.foldedAt;
+	}
+	return {
+		totalMessages: cov.slots.length,
+		coveredCount: cov.coveredCount,
+		uncoveredCount: cov.uncoveredCount,
+		coveredTokens: cov.coveredTokens,
+		uncoveredTokens: cov.uncoveredTokens,
+		running: extractionStatuses().some((run) => run.kind === "chat" && run.chatId === chatId),
+		lastFoldedAt,
+	};
+}

src/server/settings-body.ts

@@ -22,7 +22,7 @@ import type {
 	ModelsSettings,
 	RoleModelOverride,
 } from "../core/settings.ts";
-import { BACKGROUND_MODEL_ROLES, MODEL_SLOTS } from "../core/settings.ts";
+import { BACKGROUND_MODEL_ROLES, EXTRACTION_CHUNK_TOKENS_FLOOR, MODEL_SLOTS } from "../core/settings.ts";
 import { TOGGLEABLE_TOOLS } from "../core/pending.ts";
 
 /** Parsed-or-rejected — `ok: false` carries the 400's error message. */
@@ -283,15 +283,33 @@ export function parseAppearanceSettingsBody(body: unknown): SettingsBodyResult<A
 export function parseGraphSettingsBody(body: unknown): SettingsBodyResult<GraphSettings> {
 	const raw = asRecord(body);
 	if (!raw) return { ok: false, error: "Invalid JSON body" };
-	// Wave 7 collapse (owner ruling 2026-09-23): the section is the master
-	// switch ONLY. PUT is replace-whole — a body like {enabled: true} IS the
-	// whole section now; the dead rungs' keys in old records are unread
-	// (the sanitizer drops them; see sanitizeGraph).
+	// Wave 7 collapse (owner ruling 2026-09-23) + the extraction-UX
+	// redesign (2026-09-24): the section is the master switch and the
+	// chunk budget. PUT is replace-whole — a body like {enabled: true} IS
+	// the whole section (extractionChunkTokens absent = factory default at
+	// resolution); the dead rungs' keys in old records are unread (the
+	// sanitizer drops them; see sanitizeGraph).
 	const settings: GraphSettings = {};
 	if (raw.enabled !== undefined) {
 		if (typeof raw.enabled !== "boolean") return { ok: false, error: "enabled must be a boolean" };
 		settings.enabled = raw.enabled;
 	}
+	if (raw.extractionChunkTokens !== undefined) {
+		if (typeof raw.extractionChunkTokens !== "number" || !Number.isFinite(raw.extractionChunkTokens)) {
+			return { ok: false, error: "extractionChunkTokens must be a number" };
+		}
+		const floored = Math.floor(raw.extractionChunkTokens);
+		if (floored < EXTRACTION_CHUNK_TOKENS_FLOOR) {
+			// the floor's rationale lives on GraphSettings.extractionChunkTokens:
+			// every run pays the graph-carrying prompt overhead — penny-batches
+			// cost more per extracted token than the content is worth.
+			return {
+				ok: false,
+				error: `extractionChunkTokens must be at least ${EXTRACTION_CHUNK_TOKENS_FLOOR} (smaller batches waste the per-run graph overhead)`,
+			};
+		}
+		settings.extractionChunkTokens = floored;
+	}
 	return { ok: true, settings };
 }
 

src/server/turn-service.ts

@@ -19,7 +19,6 @@ import type { UIMessage } from "ai";
 import { withPlanBlocks } from "../core/plan-blocks.ts";
 import { coverageOf } from "../core/coverage.ts";
 import type { ChatStore, GraphStore } from "../core/ports.ts";
-import { EXTRACTION } from "../../config.ts";
 import type { ExtractionPipeline } from "../agent/subagents/extraction.ts";
 import { ModelsNotConfiguredError } from "../core/settings.ts";
 
@@ -74,11 +73,14 @@ export function persistTurn(chatStore: ChatStore, chatId: string, messages: UIMe
 /**
  * The graph write channel (vision §1 construction): the extraction pipeline
  * (extractor → auditor → merger — the sole graph writer) runs ASYNC once a
- * chat's UNCOVERED transcript tokens reach EXTRACTION.chunkTokens
+ * chat's UNCOVERED transcript tokens reach the chunk budget
  * (vision-extraction-coverage.md, 2026-09-15 — replacing the old
  * every-3-quiz-answers count: content size is the honest signal, and
- * quiz-less sessions extract too). The pipeline itself extracts only the
- * uncovered batches. Fire-and-forget — never blocks the turn; failures are
+ * quiz-less sessions extract too). The budget is the graph section's
+ * extractionChunkTokens setting (extraction-UX redesign, 2026-09-24) —
+ * the composition root passes the per-turn effective read (chat-handler
+ * resolves fresh per turn, same as every other gate). The pipeline
+ * itself extracts only the uncovered batches. Fire-and-forget — never blocks the turn; failures are
  * logged by the pipeline and leave coverage untouched, so the next trigger
  * (or the manual extract button) re-runs what's left.
  *
@@ -90,13 +92,14 @@ export function triggerExtraction(
 	deps: { chatStore: ChatStore; graphStore: GraphStore },
 	extraction: ExtractionPipeline,
 	chatId: string,
+	chunkTokens: number,
 ): void {
 	if (process.env.NODE_ENV === "test") return;
 	try {
 		const record = deps.chatStore.getChat(chatId);
 		if (!record) return;
 		const covered = coverageOf(record.messages as UIMessage[], deps.graphStore.readCoverage(), chatId);
-		if (covered.uncoveredTokens < EXTRACTION.chunkTokens) return;
+		if (covered.uncoveredTokens < chunkTokens) return;
 		void extraction.runExtraction(chatId).catch((err) => {
 			// BYOK (owner, 2026-09-22): an unconfigured write channel SKIPS
 			// (the coverage ledger keeps the messages uncovered — the next