Extraction observability: usage capture, live status payload, true abort (Wave 4)
Extraction is real spend and background work; the dock needs to see and stop it. User decisions: persist all cost data (chat record), per-stage usage granularity, and TRUE cancellation for abort. USAGE CAPTURE (live + persisted): - extraction.ts: per-call onStepFinish captures each billed step's usage (SDK LanguageModelUsage -> CallUsage: input/output/cache read+write/reasoning) into the registry's live per-stage usage (extractor/auditor; the merger is deterministic — no model call). - /api/graph/status runs now carry usage (model + steps per stage); cost/context folds are the consumer's (core/prices.ts — nothing stores dollars). - ChatRecord.extractions: append-only per-run usage log (mirrors the turns log): startedAt key, at, durationMs, merge stats when the merger ran, stages. Persisted on success AND failure — a failed run still spent money. Idempotent per startedAt; the list payload (ChatSummary) never leaks it. - EmissionStats moved to contracts/graph.ts (its contract home); core/graph.ts re-exports for existing importers. - Verified end-to-end against the real pipeline (backfill one chat): per-step token counts, reasoning tokens, and the auditor's growing prompt-cache reads captured and persisted. TRUE ABORT (the dock's abort button, user decision): - RunningExtraction owns one AbortController per run, shared across phases; abortExtraction(chatId) fires it and returns whether a run was aborted (idempotent no-op otherwise). The pending model call receives the signal — the provider cancels the request, spend stops. throwIfAborted() at phase boundaries ends runs aborted in sync gaps. - Abort is a deliberate stop, not a failure: same ledger discipline (nothing merged, nothing checkpointed, partial usage persisted) but the run reports zero emissions instead of throwing. Exported via src/server/observability.ts for master's /api/activity/abort route. - Tests: mid-run abort (hanging mock that rejects on the signal), live-accumulation + persisted-record assertions, store-seam idempotency, and the summary-payload leak guard.
This commit is contained in:
+32
-1
@@ -12,7 +12,7 @@ import * as fs from "node:fs";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
import { PATHS } from "../config.ts";
|
||||
import { callCost, contextWindow, jobCost, sessionCost, sumSteps, turnCost } from "../src/core/prices.ts";
|
||||
import type { CallUsage, JobRecord, TurnUsage } from "../src/core/contracts.ts";
|
||||
import type { CallUsage, ExtractionRunUsage, JobRecord, TurnUsage } from "../src/core/contracts.ts";
|
||||
|
||||
const call = (over: Partial<CallUsage> = {}): CallUsage => ({
|
||||
inputTokens: 0,
|
||||
@@ -134,3 +134,34 @@ test("appendTurnUsage: appends, is idempotent per messageId, survives saveMessag
|
||||
|
||||
fs.unlinkSync(`${PATHS.chats}/${record.id}.json`);
|
||||
});
|
||||
|
||||
test("appendExtractionUsage: appends, idempotent per startedAt, not in list payload", () => {
|
||||
const record = chatStore.createChat("extraction usage test chat");
|
||||
const runEntry = (startedAt: number): ExtractionRunUsage => ({
|
||||
startedAt,
|
||||
at: "2026-09-09T00:00:00Z",
|
||||
durationMs: 42_000,
|
||||
stats: { nodes: 1, edges: 2, events: 3 },
|
||||
stages: {
|
||||
extractor: { model: "z-ai/glm-5.3-flash", steps: [call({ inputTokens: 100, outputTokens: 10 })] },
|
||||
auditor: { model: "z-ai/glm-5.3-flash", steps: [call({ inputTokens: 50, outputTokens: 5 })] },
|
||||
},
|
||||
});
|
||||
chatStore.appendExtractionUsage(record.id, runEntry(1));
|
||||
chatStore.appendExtractionUsage(record.id, runEntry(1)); // same run appended twice → no-op
|
||||
chatStore.appendExtractionUsage(record.id, runEntry(2)); // a retried run is a NEW spend
|
||||
let reread = chatStore.getChat(record.id);
|
||||
assert.equal(reread?.extractions?.length, 2);
|
||||
// failed runs (stats absent) persist with usage — money was spent
|
||||
chatStore.appendExtractionUsage(record.id, { ...runEntry(3), stats: undefined });
|
||||
reread = chatStore.getChat(record.id);
|
||||
assert.equal(reread?.extractions?.length, 3);
|
||||
assert.equal(reread?.extractions?.at(-1)?.stats, undefined);
|
||||
|
||||
// the list payload must NOT leak the extractions log
|
||||
const listed = chatStore.listChats().find((c) => c.id === record.id);
|
||||
assert.ok(listed);
|
||||
assert.equal("extractions" in listed, false);
|
||||
|
||||
fs.unlinkSync(`${PATHS.chats}/${record.id}.json`);
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user