extraction: the spine stage — cross-stretch assembly between extract and audit

Per-batch pipeline is now extract → spine → audit → merge (owner order,
2026-09-30), the spine inside the serialized audit section so it reads
the graph exactly as the following audit will. Measured motive: 0 of 133
chains in the live corpus crossed a stretch boundary — no stage minted
cross-stretch chains.

- src/agent/subagents/spine.ts: SESSION/NOTES spine preambles + shared
  SPINE_CORE (span doctrine; extraction doctrines carried over), the
  submit_spine TypeBox schema, validateSpine (mechanical span law: every
  chain touches >=1 fresh endpoint AND >=1 existing digest endpoint;
  unknown ids, self-reference, and id collisions are loud), the digest
  prompt builder, and the runner (strong tier, retry-only).
- extraction.ts: 'assembling' phase, spine stage in the per-need gate,
  usage/billing stage, assemble() inside withAuditLock for both front
  doors; first stretch of a source skips the call (empty digest, no
  spend); the audit sees COMBINED propositions (chunk + connectives),
  never chains; merger receives the combined findings.
- core/graph.ts mergeFindings: chain endpoints resolve through the
  job-local map OR pass through verbatim as existing n_ ids (the spine's
  mixed namespace).
- EXTRACTION_CORE: steps-of-reasoning selection bar (examples, notation
  glosses, and restatements that no argument needs are not steps).
- Factory chunk budget 4096 -> 8192 (fewer seams, more local context).
- healStringifiedArrays moved to the shared submit seam (submit-agent.ts).
- Cost panel: Assembly row; usage records carry a spine stage (optional
  on legacy records).
- Tests +5: prompt pins, span-law matrix, digest rendering, seeded
  end-to-end flow (spanning chain + connective, verbatim n_ endpoint,
  spine billed as its own stage), non-spanning refused + retried.
This commit is contained in:
amosblomqvist
2026-09-30 16:38:45 +03:00
parent 5ed724c224
commit 52c02533d1
18 changed files with 787 additions and 87 deletions
+16 -2
View File
@@ -61,6 +61,7 @@ export interface GraphCostSummary {
* historical spend on either is simply not displayed.) */
stages: {
extraction: StageCost;
spine: StageCost;
audit: StageCost;
};
/** Newest first, capped at `recentCap` (default 10). */
@@ -112,11 +113,16 @@ function mergeTokens(into: CallUsage, steps: readonly CallUsage[]): void {
* same output — no clock, no fs, no store reads. */
export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
const extraction = emptyStage();
const spine = emptyStage();
const audit = emptyStage();
const runs: RecentRun[] = [];
const foldExtractionRun = (run: ExtractionRunUsage, title: string) => {
const stages = run.stages ?? { extractor: { model: "unknown", steps: [] }, auditor: { model: "unknown", steps: [] } };
// The spine stage is OPTIONAL on the record (2026-09-30): records
// written before it existed (and runs whose digest was empty, so
// the call was skipped) carry no entry — default to empty steps.
const spineStage = stages.spine ?? { model: "unknown", steps: [] };
// Per-stage: tokens always count; dollars only when priced.
extraction.runs++;
mergeTokens(extraction.tokens, stages.extractor.steps);
@@ -124,6 +130,14 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
if (extDollars === null) extraction.unpricedRuns++;
else extraction.dollars += extDollars;
if (spineStage.steps.length > 0) {
spine.runs++;
mergeTokens(spine.tokens, spineStage.steps);
const spDollars = stageDollars(spineStage.model, spineStage.steps);
if (spDollars === null) spine.unpricedRuns++;
else spine.dollars += spDollars;
}
audit.runs++;
mergeTokens(audit.tokens, stages.auditor.steps);
const audDollars = stageDollars(stages.auditor.model, stages.auditor.steps);
@@ -135,7 +149,7 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
const runDollars = allStages.every((d) => d === null)
? null
: allStages.reduce<number>((a, d) => a + (d ?? 0), 0);
const allSteps = [...stages.extractor.steps, ...stages.auditor.steps];
const allSteps = [...stages.extractor.steps, ...spineStage.steps, ...stages.auditor.steps];
runs.push({
startedAt: run.startedAt,
at: run.at,
@@ -163,7 +177,7 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
return {
totalDollars: extraction.dollars + audit.dollars,
unpricedRuns,
stages: { extraction, audit },
stages: { extraction, spine, audit },
runs: capped,
};
}