Extraction is real spend and background work; the dock needs to see
and stop it. User decisions: persist all cost data (chat record),
per-stage usage granularity, and TRUE cancellation for abort.
USAGE CAPTURE (live + persisted):
- extraction.ts: per-call onStepFinish captures each billed step's
usage (SDK LanguageModelUsage -> CallUsage: input/output/cache
read+write/reasoning) into the registry's live per-stage usage
(extractor/auditor; the merger is deterministic — no model call).
- /api/graph/status runs now carry usage (model + steps per stage);
cost/context folds are the consumer's (core/prices.ts — nothing
stores dollars).
- ChatRecord.extractions: append-only per-run usage log (mirrors the
turns log): startedAt key, at, durationMs, merge stats when the
merger ran, stages. Persisted on success AND failure — a failed run
still spent money. Idempotent per startedAt; the list payload
(ChatSummary) never leaks it.
- EmissionStats moved to contracts/graph.ts (its contract home);
core/graph.ts re-exports for existing importers.
- Verified end-to-end against the real pipeline (backfill one chat):
per-step token counts, reasoning tokens, and the auditor's growing
prompt-cache reads captured and persisted.
TRUE ABORT (the dock's abort button, user decision):
- RunningExtraction owns one AbortController per run, shared across
phases; abortExtraction(chatId) fires it and returns whether a run
was aborted (idempotent no-op otherwise). The pending model call
receives the signal — the provider cancels the request, spend
stops. throwIfAborted() at phase boundaries ends runs aborted in
sync gaps.
- Abort is a deliberate stop, not a failure: same ledger discipline
(nothing merged, nothing checkpointed, partial usage persisted) but
the run reports zero emissions instead of throwing. Exported via
src/server/observability.ts for master's /api/activity/abort route.
- Tests: mid-run abort (hanging mock that rejects on the signal),
live-accumulation + persisted-record assertions, store-seam
idempotency, and the summary-payload leak guard.
The 2026-09 observability wave's type-level foundation, landed before the
streams so nothing renders against guessed shapes:
- contracts/chat.ts: CallUsage + TurnUsage (per-model-call token accounting,
one entry per teacher/wake turn keyed by messageId) and ChatRecord.turns —
a DELIBERATELY parallel append-only usage log, not per-message metadata:
the client re-POSTs its message list every turn and its copies never
carry server-stamped fields, so anything inside messages gets clobbered
by the next persist (the wave-3 merge-rule lesson).
- contracts/chat.ts: PlanBlockData — the persisted data-plan-block part
(full DAG on plan_init, one-level neighborhood on plan updates, snapshot
semantics so history renders what was true then).
- contracts/jobs.ts: JobUsage + JobRecord.usage — a sub-agent run's
accumulated accounting, written by the runner.
- core/ports.ts + store/chats.ts: ChatStore.appendTurnUsage — idempotent
per messageId (abort replays / wake retries append once); listChats no
longer leaks turns into the list payload.
- core/prices.ts: PRICES + CONTEXT_LIMITS (generated from OpenRouter's
published pricing via scripts/sync-pricing.ts) + pure folds: callCost,
turnCost, sessionCost, contextWindow, jobCost. Unknown model or absent
fields render as absent, never zero.
- server/observability.ts: barrel-approved surface for the transport layer
(fixes the import-law violation in the graph status route that slipped
through the wave-3 merge verification).
Capture wiring (agent onStepFinish -> persist seam, plan-block emission)
lands next as master seam work; the streams start against these shapes.
The background extraction pipeline is invisible on the home surface —
essential observability per the user's directive: the graph view must
show what is currently running (extracting / auditing / merging) and
for which chat.
- extraction.ts: module-level registry of in-flight runs (chatId,
phase, startedAt), updated at each phase transition in runExtraction
and cleared in finally (done or failed). extractionStatuses() is the
read; one process, one registry.
- GET /api/graph/status: thin transport over the registry, chat titles
joined from the chat store.
- graph-view: polls every 2s and shows one chip per running pipeline
(spinner + phase + title + elapsed seconds, bottom-left); when a run
finishes (some -> none) the graph refetches, so merges land live.
Status is cosmetic — poll errors never surface.
- Unit test drives a mocked pipeline run and observes the registry
mid-flight: 'extracting' during the extractor's generate, 'auditing'
during the auditor's, empty after.
amosblomqvist