compaction: the usage-anchored trigger + visible session state (round 15)
The relational-databases incident, diagnosed and fixed: TRIGGER — pi's estimator law, adopted exactly: anchor on the last assistant message's provider-reported usage (totalTokens || the field sum) and chars/4-estimate ONLY what was appended after it. The anchor scan skips aborted/zero-usage assistants and STOPS at the newest compaction marker (pre-cut usage described context the next request will not carry — pi's distrust rule). The old estimator (JSON.stringify of every message /4) tripled the true context (measured: 84,412 'tokens' estimated vs 38,799 provider-reported on the incident record) — scaffolding, escaped LaTeX, thinking blocks and app-side details all counted. estimateMessageTokens now counts wire text only (text + thinking + toolCall name/args; content text for user/toolResult/custom). The unanchored fallback runs the plain heuristic over the CUT-HONORED projection plus the additive prompt term; the anchored path adds nothing (usage already billed prompt and tools). OBSERVABILITY — compaction is a visible session-wide state (owner spec): the transcript-bottom placeholder register gains COMPACTING CONTEXT… (BrailleSpinner + Shimmer, the THINKING placeholder's twin, mutually exclusive by construction — the gate precedes beginTurn), and the sidebar row shows 'Compacting context · <ago>' with the spinner (sessionMeta outranks busy; compacting-changed joins turn-started/turn-settled as refresh triggers). Liveness rides the API like turn-busy: ChatListItem.compacting + compactionInFlight on both GET /api/chats and GET /api/chats/[id] (initiallyCompacting covers the navigate-back-mid-compaction mount — the event already fired), exported through the server barrel. Also: the relational-databases record rolled back to pre-compaction (marker, post-marker turns, tree reset; backup in /tmp) for a clean retest. The summarizer-usage 'gap' was a misread — steps[] was populated all along.
This commit is contained in:
@@ -34,6 +34,7 @@ import {
|
||||
buildCompactionPrompt,
|
||||
compactionCutIndex,
|
||||
COMPACTION_RESERVE_TOKENS,
|
||||
estimateContextTokens,
|
||||
estimateTokens,
|
||||
newCompactionMarker,
|
||||
planTreeToText,
|
||||
@@ -142,13 +143,19 @@ export function effectiveThresholdTokens(settingTokens: number | undefined, cont
|
||||
|
||||
// ─── the estimate over the ACTIVE projection ────────────────────────────────
|
||||
|
||||
/** The projected context estimate: what the next LLM call would carry —
|
||||
* toLlmMessages output (cut honored) + the system prompt. The record
|
||||
* path carries no leading system message (settleWith filters it), so
|
||||
* the chrome rides as the additive term instead: prompt text here,
|
||||
* tool declarations absorbed by the 16k reserve (the declarations are
|
||||
* a few k and stable per session). chars/4. */
|
||||
/** The projected context estimate: what the next LLM call would carry.
|
||||
* THE TRIGGER LAW (pi's, verified 2026-10-05): anchor on the last
|
||||
* assistant message's provider-reported usage — it already counted
|
||||
* the system prompt, tool declarations, and the provider's own
|
||||
* tokenizer — and chars/4-estimate ONLY messages appended after it.
|
||||
* estimateContextTokens refuses anchors that predate a compaction
|
||||
* marker (that usage described context the next request will NOT
|
||||
* carry), so the fallback path runs the plain heuristic over the
|
||||
* CUT-HONORED projection (never the dropped span) plus the additive
|
||||
* prompt term the anchor would have covered. */
|
||||
export function projectedContextTokens(messages: readonly LhMessage[], systemPromptTokens: number): number {
|
||||
const est = estimateContextTokens(messages);
|
||||
if (est.lastUsageIndex !== null) return est.tokens;
|
||||
return estimateTokens(toLlmMessages(messages as readonly AgentMessage[]) as unknown as readonly LhMessage[]) + systemPromptTokens;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user