Demos execute JS and need a browser; the app's own binary now provides
one: main.cjs boots a hidden window + CDP port under --browser-service
(before the single-instance lock - a helper is deliberately a second
instance of this binary), lifetime tied to the server's stdin (EOF =
quit, no orphans). headless.ts connects over CDP as PRIMARY (owner
ruling 2026-10-10), system Chrome stays the fallback, graceful refusal
when neither. Electron's CDP has no Target.createTarget, so the helper's
pre-created window is THE page, reused and reset per job behind a
serialized queue. PDFs and SVGs never reach a browser anymore (native
since stages 1-2); stale Chrome wording in model-facing tool copy made
engine-neutral. Live-proofed end-to-end from the dev tree (helper seen
by process watch, js-ran 42 probe, no orphan). Gates 1523/1523.
Visual-maker SVGs never execute script, so a browser added nothing but a
process. resvg rasterizes in-process now (white background and fitTo
width mirror the old Chrome-page semantics; svgViewport unchanged,
quirks included for parity). The old Chrome path stays verbatim as the
fallback for exotic-SVG parse failures - it was the only path before,
so nothing regresses.
The resvg binary (4MB per platform, one per build) rides the standalone
via the same tracing pattern. Gates 1519/1519; native path pinned
against real SVGs (intrinsic size, viewBox-only 900 default, opts
override, system-font text).
The browser was never the PDF engine - pdfjs always painted the pages;
Chrome only hosted the canvas tab. Now pdfjs-dist v6's own Node
integration (@napi-rs/canvas, its optionalDependency) hosts in-process:
same paint code, Skia canvas, zero browsers, zero temp files, no base64
hop. The whole pdf-reader runs browser-free - the Chrome gate on
pdf_render is gone.
The Skia binary (27MB per platform, exactly one per build - npm installs
only the building machine's) rides the standalone via the same
outputFileTracingIncludes pattern as pdfjs-dist. Verified: builds green
(1515), native render pinned against the real fixture (canonical
single-IDAT PNGs, dpi/72 arithmetic, page-range skipping), and the
canvas loads + paints from inside .next/standalone.
Stages 2-3 (resvg for SVG, the --browser-service demo helper) follow.
The whitehead incident (2026-10-09): glm-5.3 emitted quiz type with
quotes baked in ('"predict"') — rejected ten times, byte-identical
retries, the lesson never started — and correctAnswer with the JSON
key-value colon glued on ([':parmenides']). Third GLM wobble incident,
and each prior heal was wired only where its incident landed; this one
landed precisely where the practice heal never reached.
Layer 1 — healEncodedArguments moves to core/argument-heal (the display
seam needs it too — the tool-parts law) with withArgumentHeal +
healToolRecord (agent/tools/heal) wired at EVERY registry: prober,
teacher, practice, bareQuestionTools (it had no prepareArguments at
all), demo-builder, and submit-agent (composing after
healStringifiedArrays; the auditor's extraTools wrapped).
Layer 2 — stripModelValueDebris (core/value-debris): leading/trailing
colons, quotes, backticks, whitespace strip at the MEMBERSHIP seams —
coerceCorrectAnswer (boundary check + display + grading in one),
normalizeQuizType (persisted wobbled records keep their role),
resolveTargetId (copy-syntax on targets).
Layer 3 — schema description hygiene: the quiz type and practice
mode/outcome/rung/missClass values now read as bare words — the quoted
JSON-key style invited quote-copying.
Display mirror — parseQuizInput heals the raw record on read, so a
boundary-healed quiz whose record keeps raw args renders instead of
becoming an invisible pending quiz (learner lockout).
Pins: tests/tool-heal.test.ts (the incident shapes end-to-end through
pi-ai validation, the wiring, the strips, the mirror). The stringy-
options pins flip to boundary-parity per tool-parts' own law; the
chat-flow garbage pin keeps its honest-reject intent with an unhealable
shape. 1471 tests, typecheck, lint green.
web_fetch_pdf downloads (25MB cap, probe-validated) into a per-job temp
dir and returns the triage summary — page count, size class (the pdf
doctrine's <=10/<=40 thresholds), flagged math/image-heavy pages. The
researcher's toolset then gains pdf_info / pdf_extract / pdf_search over
that dir, scoped by a resolve-and-startswith wall: the researcher's
filesystem ends at the temp dir. Render is deliberately absent — the
researcher slot has no vision; visual reading stays the pdf job's job.
web_fetch: a PDF URL now returns a POINTER to web_fetch_pdf instead of
dead-ending, and every output carries 'Source: <final url after
redirects> (HTTP <status>)' — the 'ill-formed request' lesson: a
200-served error body must be distinguishable from a real page.
The runner owns the temp lifecycle (mkdtemp before, recursive cleanup in
the job's finally — never enters the record) and web_fetch_pdf urls ride
the sources ledger. Doctrine step 5 added: fetch, triage, extract only
the pages that matter, cite as (p. N).
Tests build a REAL minimal PDF programmatically (correct xref offsets —
pdfjs parses it like any download): zero network, real parse.
Owner ruling 2026-10-04: the module is a sunk cost, retrievable from
history if ever revisited. Removed: core practice contracts + pipeline
helpers, the PracticeStore port + adapter + data dir path, server
service + API routes (sets, clusters picker, practice aborts), the
practice pages + dialog/exam/list UI, chain-table's Generate handoff,
sidebar nav, activity-dock practice rows, and the partitioner/
problemGenerator/grader model roles (BACKGROUND_MODEL_ROLES 8 → 5).
Survivors, moved to core/chains.ts (pure + browser-safe): ReasoningChain,
chainsForSelection, formatChainLine — the chain table and graph
emission still consume them.
pi-ai 1.0's onProviderStreamEvent, at the layer our incident class lives
(GLM arg truncation, DeepSeek id resets) — env-gated (owner choice:
no app surface, off by default; raw events carry full prompts), one
JSONL per chat under data/debug/provider-events/ with a 20MB cap and a
run-start marker for record correlation.
Wired at every agent construction: teacher/prober turns (AgentBuildOptions.
chatId), jobs via LoopContext (researcher, demo-builder through
runSubagent; visual-maker + pdf-reader through their streamOnce), and
the extraction pipeline (extractor + auditor keyed by source id).
practice deliberately unwired (sunk-cost module).
Authoring refinements (quizInputProblems, nodeId checks, ask/explain
refinements, submit-agent pipeline laws) now return invalidInputResult —
same model-facing failure and retry semantics pi minted from the throw,
but details keeps {tool, problems} as data instead of the throw path's
empty {}. ToolInputError deleted; wire shape unchanged (verified against
pi's own thrown-error conversion in agent-loop).
The partitioner was told to reference chain ids, but no rendered line
ever showed one — chainLine emitted bare 'premises → conclusion', so
the model re-typed whole rendered lines into submit_partition and every
call failed 'unknown chain ids' validation (owner live report,
2026-09-30 evening). The generator's YOUR TARGET block had the same
blindness.
- chainLine now leads with the chain's id: [arg_x] n_a, n_b → n_c;
a bare proposition repeats its own id ([n_y] n_y (no premises)) —
mechanically teaching that its chain id IS its proposition id.
- PARTITIONER_INSTRUCTIONS rewritten to chains-primary vocabulary
(was still 'follows-edges, bijective re-encoding' — pre-chains
framing) with the id-only doctrine and singular propositions
first-class.
- partitionSchema/problemSchema gained field descriptions: ids
VERBATIM, never re-type, never invent.
- Generator OUTPUT bullet + target-block header say the same.
Design ruling en route: partitioner stays an LLM partitioning chains
(deterministic partitioning and propositions-as-targets both
considered and declined by the owner).
Owner live-test verdict on the spine (2026-09-30): no perceptible gain —
the real problems (restatement nodes, shallow re-derivations, same
conclusion from many angles) are mint-time issues a post-hoc chain
minter cannot fix. This arc moves assembly INTO the extractor and adds
the deterministic dedupe the owner specified:
- Digest-aware extraction: each stretch's extractor prompt carries THE
GRAPH SO FAR (its own source's merged propositions + chains, ids
verbatim); build on existing ids instead of restating, lean on
established conclusions as premises. Mixed-namespace chain endpoints
(fresh p-ids OR existing n_-ids) validated at the submit seam.
- Serialized per source: a source's batches/chunks run strictly in
order (each extract reads every earlier merge); cross-source runs
stay parallel; source-wide abort stops the loop. The spine stage,
role, phase, usage stage, cost row, and setup roster are gone.
- Core-ideas prompt: goal reframed as uncovering core ideas and showing
how everything derives from the foundations; direction upward, prefer
lean derivations, never re-derive what the graph already holds.
- Chain simplification (merger maintenance): simplifyChains drops
provably redundant chains at every merge — prefer-fewest-premises +
ancestry-cover ([A,B]⟹X with [X,C]⟹Y covers [A,B,C]⟹Y, transitive);
same-key provenance lines drop together; rewriteLedgers under the
audit lock; droppedChains counted separately in stats.
Chunk budget stays 8192 (owner: keep for now, maybe lower later).
Per-batch pipeline is now extract → spine → audit → merge (owner order,
2026-09-30), the spine inside the serialized audit section so it reads
the graph exactly as the following audit will. Measured motive: 0 of 133
chains in the live corpus crossed a stretch boundary — no stage minted
cross-stretch chains.
- src/agent/subagents/spine.ts: SESSION/NOTES spine preambles + shared
SPINE_CORE (span doctrine; extraction doctrines carried over), the
submit_spine TypeBox schema, validateSpine (mechanical span law: every
chain touches >=1 fresh endpoint AND >=1 existing digest endpoint;
unknown ids, self-reference, and id collisions are loud), the digest
prompt builder, and the runner (strong tier, retry-only).
- extraction.ts: 'assembling' phase, spine stage in the per-need gate,
usage/billing stage, assemble() inside withAuditLock for both front
doors; first stretch of a source skips the call (empty digest, no
spend); the audit sees COMBINED propositions (chunk + connectives),
never chains; merger receives the combined findings.
- core/graph.ts mergeFindings: chain endpoints resolve through the
job-local map OR pass through verbatim as existing n_ ids (the spine's
mixed namespace).
- EXTRACTION_CORE: steps-of-reasoning selection bar (examples, notation
glosses, and restatements that no argument needs are not steps).
- Factory chunk budget 4096 -> 8192 (fewer seams, more local context).
- healStringifiedArrays moved to the shared submit seam (submit-agent.ts).
- Cost panel: Assembly row; usage records carry a spine stage (optional
on legacy records).
- Tests +5: prompt pins, span-law matrix, digest rendering, seeded
end-to-end flow (spanning chain + connective, verbatim n_ endpoint,
spine billed as its own stage), non-spanning refused + retried.
Data analysis of the five-source corpus (490 nodes, 244 chains) found
three pathologies the goal-first core let through:
1. Narrative-adjacency chains — premises that occasioned rather than
entailed conclusions (concentrated in the polemical sources).
2. Session-scaffolding minted as knowledge — the source's own axiom
numbering (A1/F2/UT2), arc labels ('the crux') persisted as nodes.
3. Full-discourse volume — glossary and argument-arc material at equal
rank with load-bearing ideas (~200 nodes per source).
Three doctrine lines, same terse register as the core:
- entailment is the only link that earns a chain; historical/narrative
connections keep their pieces as bare propositions or drop them
- load-bearing ideas only (what a reference on the subject would teach)
- propositions must stand alone for a reader who never saw the source;
no leaning on the source's own scaffolding
Substring pins for all three; shared by session + notes extractors by
construction.
- architecture.md section 6 rewritten: stored truth = propositions + chains
(chains.jsonl); deriveView as the lossy image (implication nodes +
untyped arrows, rebuilt per read); the bijection framing (independent
arguments for one conclusion stay distinct — the corruption the edge
store invited); new deletion-semantics paragraph (proposition death
cascades to every referencing chain; arg_ death = one argument);
supports retirement stated plainly; auditor's propositions-only scope;
clusters stated as birth-session partition truth (no Louvain claim)
- section 7 practice chain model: stored-chains inclusion law replaces
follows-edge grouping; chainsToGraph bijection claim deleted
- README: graph clauses (stored chains, derived view, chain dedupe),
practice clause, ledger inventory, and two long-stale line-5 claims
(ledger events, semantic top-K) corrected
- contracts/graph.ts GraphSnapshot comment: Louvain claim replaced with
birth-session truth; extraction.ts comments no longer narrate the
dead expandFindings/chainsToEdges seam
Owner rulings 2026-09-30: reasoning chains are the stored truth; the graph
canvas renders a lossy derived image. No migration, no backcompat — old
graph data was wiped.
- contracts: GraphChain (arg_-minted id, premises, conclusion) replaces
typed edges; GraphEdge survives only as the untyped derived arrow;
GraphSnapshot carries stored chains; supports edges retired entirely
- store: chains.jsonl ledger (edges.jsonl purged at load alongside legacy
events.jsonl); readChains/appendChain seam; merge dedupes by
(sorted premises, conclusion), re-assertions append under the same arg_ id
- core: deriveView mints implication nodes (name ⟹, statement via
formatChainLine) plus premise→imp→conclusion arrows; clusters stay
proposition-only (birthSessionPartition); chainsForSelection now groups
STORED chains (chain rides iff every member selected) — independent
arguments for one conclusion never union; chainsToGraph/chainsToEdges die
- extraction: extractor submits chains natively (expandFindings deleted);
auditor receives propositions only; supports fan-out gone
- deletion: full death of a proposition kills every referencing chain;
arg_-prefixed node deletes kill that chain; chainsRemoved stats
- UI: every arrow renders the ⟹ double shaft; supports toggle + thickness
slider and detail-card edge sections deleted (card keeps name +
proposition); chain table + practice pipeline read stored chains
- tests: 1188 green across the new seams
- dialog: optional 'Anything else?' textarea (typing-only, 2000-char mirror)
- API: body gains note? — trimmed, whitespace-only omitted, hard 400 over 2000
- pipeline: note persisted on the set and threaded to both agent stages
- agents: THE LEARNER'S NOTE prompt block (present only when set) + the
weighed-never-owed stance pinned in both doctrines
- fix en route: doubled THE PARAMETERS heading in both prompts since slice 3
(renderParams owns the heading; the literal blocks removed)
Rulings: each problem generation is its own visible dock job (the
extraction per-chunk pattern), and practice shares extraction's
semaphore — one process-wide 3-permit pool for ALL background LLM
jobs; total background LLM concurrency bounded at 3.
A. SHARED POOL: new src/agent/subagents/bg-llm-semaphore.ts —
globalThis __lhLlmSlots, PERMIT_COUNT 3 (the ruling), the
fresh-take/handed-off-permit deadlock discipline preserved for
BOTH callers. extraction.ts's withExtractSlot is now a thin
wrapper (job-row signal → withBgLlmSlot); __lhExtractSlots and
__lhPracticeSlots both die. The partitioner now acquires a permit
too (it's a background LLM job — it didn't before).
B. REGISTRY ROWS: the aggregate setId:generation / setId:grading rows
are replaced by setId:partition (phase partitioning, label
'Partitioning — title'), setId:gen:problemId (queued→generating,
'Problem i/N — title'), setId:grade:problemId (queued→grading,
'Grading i/N — title'). Each row carries its OWN abort controller
and its OWN usage (one PracticeStepLog per row — the aggregate
partitioner/generator/grader folding dies). Row key = feed id =
abort key (the dock dup-key law).
C. ABORT SEMANTICS: abortPracticeRow(key) routes by shape — gen row:
THAT problem only (error 'Aborted', siblings continue, set settles
ready with one errored problem); partition row: the whole run;
grade row: that grade drops (answer kept, re-submittable).
abortPractice(setId) composes from row aborts + the synchronous
persisted marks (a set-abort's verdict stands before the settles
re-read it — the settle step keeps an existing error status).
/api/activity/abort kind practice now takes the ROW key (charset
admits ':').
D. FEED + DOCK: ActivityPractice/FeedPractice carry per-row identity
(id = row key, problemIndex/problemTotal); dock blobs render per
row, labels carry the i/N; the aggregate row is gone.
E. Boot sweep unchanged at set grain; the live-row check is now a
key-prefix scan (any live row of the set protects it). The
updatedAt heartbeat covers the new row scheme as-is.
Tests (20 in the pipeline suite): per-problem row shape/uniqueness/
i-N labels (partition row frozen behind a gated partitioner; per-row
usage ISOLATION pin — b's log never shows on c's row); abort-one-gen
(sibling completes, set ready with one errored problem); partition-
row abort; queued grade-row abort (dead-ticket path: answer kept,
re-submittable, sibling lands); shared-pool identity + contention
(practice queues behind extraction-style holders); registry/dock
feed mapping. Test isolation hardened: unique per-store set ids,
afterEach registry/lock/pool reset (a timed-out test can no longer
poison the rest).
architecture.md: jobs-pattern paragraph rewritten (per-problem rows,
shared pool, abort routing).
1149/1149, tsc 0, lint 0, build 0.
Prior state: zero formatting guidance anywhere in the agent module —
models wrote prose-only by default.
GENERATOR_DOCTRINE Part III gains the FORMATTING bullet (after the
THE PARAMETERS bullet, before PREVIOUS PROBLEMS): statement, solution,
and rubric render as full Markdown — math always LaTeX (inline $…$,
display $$…$$), code/pseudocode/formal systems in fenced blocks with
a language tag, identifiers inline as `code`, tables for parallel
structure, lists for enumerations; prose-only renderings of
mathematical or structured content are a defect.
GRADER_INSTRUCTIONS gains the feedback-formatting paragraph (before
the OUTPUT section): full Markdown, LaTeX math, fenced code, lists/
tables where they carry structure — teach the boundary in properly
formatted prose, not plain text.
PARTITIONER gets nothing (machine-shaped output).
Tests: substring pins on the exported doctrine constants — the exact
FORMATTING sentence + ordering (after presentation, before previous
problems), the exact grader paragraph + before-OUTPUT ordering.
Architecture.md §7 one clause (the formatting doctrine in the
generator's prompt inventory).
1144/1144, tsc 0, lint 0, build 0.
Owner report: created a set, 'Interrupted (server restarted)' showed
on list + set views, generation proceeded and overwrote it. The
sweep's 'generating + no registry row' test had a false-positive
mode: it ran at EVERY pipeline instantiation, and an instance could
evaluate against a registry whose fresh row it couldn't observe.
Three guards, per ruling:
1. HEARTBEAT: ProblemSet.updatedAt? (contract + doc) — bumped on
every store write; createSet stamps it = createdAt; saveSet
re-stamps (validator tolerates absence for legacy rows).
2. ONCE PER PROCESS: the sweep guards on globalThis
.__lhPracticeSweepDone — no instance re-evaluates after the
process's first sweep.
3. GRACE WINDOW: only sets whose heartbeat (updatedAt ?? createdAt)
is older than 120s are stamped. A live run saves at every phase/
problem transition, so it is structurally unreachable; a true
orphan is minutes stale at next boot. One [practice <id>] log
line per stamped set (the extraction/jobs console.error idiom).
Tests: stale orphan stamped; FRESH generating set untouched (grace);
once-guard (second factory instantiation never re-sweeps); live-row
protection re-armed via resetSweep() test seam; store pins for
updatedAt birth-stamp + re-stamp + legacy tolerance. Architecture.md
sweep sentence updated (once-per-process + grace semantics).
1142/1142, tsc 0, lint 0, build 0.
Contracts: LadderRung 0-4, CueDistance 0-2, Presentation theory|
application; PracticeParams carries the three named levels. Doc
comments hold the three definition tables verbatim (rung names +
definitions from the ladder doctrine; cue levels; presentation
levels).
Core: LADDER_RUNGS/CUE_LEVELS/PRESENTATIONS tables in src/core/
practice.ts (definitions verbatim) — ONE source spoken by the agents'
prompts, the dialog's stepped controls, and the exam header; core
barrel re-exports practice.ts (pure domain, browser-safe).
Agents: Part I's ladder intro now 'five rungs, numbered; the request
arrives as a rung' with bullets numbered 0-4 (definitions untouched);
Part III's parameters block replaced with the named-level doctrine
(rung = exact class, no continuum; cue counts lesson examples AND
previous problems' situations; presentation dresses, never changes);
partitioner's RESPECT THE PARAMETERS replaced (low rungs = many small
assignments, high = larger; far cue makes confusables-together MORE
important; presentation does not shape the partition). renderParams
outputs the exact named-level block. Anchor/0..1 language swept dead.
Service + store: strict union validation — floats between levels,
out-of-range, wrong strings, legacy dial shapes all 400 at the API
and fail isProblemSet (legacy sets skipped from lists; no migration —
data/practice is disposable scratch, owner wipes before testing).
UI: dialog = 5-detent ladder slider + 3-detent cue slider (each shows
the level NAME + one-line definition) + Theory|Application segmented
toggle; defaults rung 2 / cue 1 / theory; Create posts the new shape.
Exam header gains the levels line ('Direct exercise · Adjacent
context · Theory').
Tests: renderParams exact-block pin; strict-validation pins (new
tests/practice-service.test.ts — bad unions 400, legacy shape 400);
store pins (legacy floats + 2.5 rung fail); render pins (detents +
names + definitions, defaults, detent-swap, toggle active segment,
levels line); every params fixture moved off floats. 1138/1138, tsc
0, lint 0, build 0.
Docs: architecture.md — discretization paragraph (compact tables,
named-level invocation, defaults, one-table source), UI mentions;
README axes sentence.
Owner ruling: 'problem sets always take in nodes — graph objects' —
practice is graph-domain, not chat-domain. No session-created sets.
API: POST /api/practice/sets takes {nodeIds, params, title?} ONLY —
the chatId branch, UnknownChatError, and the any-provenance filter are
deleted; the selector is nodeIds (readNodes filtered to the selection,
unknown ids drop, empty live intersection → 400 empty_selection);
title optional from the UI (the picker's cluster label), dated default
stays. chatStore leaves the pipeline deps entirely (labels resolve
store-side in the clusters service).
GET /api/graph/clusters (new): the birth-session groups the graph view
renders — graphSnapshot's clusters verbatim (same ordering, same
MIN_CLUSTER_SIZE presentation filter), mapped to {key, kind
chat|notes, label (null when the naming record is gone — honest,
never guessed), nodeIds, nodeCount}; corrupt/empty graph → empty
list. GraphClusterGroup contract in core/contracts/graph.ts (client-
safe — the dialog imports it).
practice-dialog: session picker → cluster picker (rows show label +
proposition count, notes clusters marked); Create posts the cluster's
nodeIds + label as title + params. 400 empty-selection copy reworded
for the vanished-nodes edge ('Nothing practicable is left in that
cluster — the graph changed.'); 503 deep link unchanged. Preselected
mode gone — the dialog is the only creation entry for now (the graph-
view cluster entry comes later with free-form selection).
Sidebar: session right-click 'Practice this session's propositions'
removed (item + state + dialog wiring); the /practice nav row stays.
Tests: pipeline — chatId/any-provenance tests die, nodeIds selection
pins added (unknown-id filtering, dated-default title, title hint
rides, empty/vanished → EmptySelectionError); render — cluster picker
pins (label+count rows, notes kind, unlabeled cluster muted, empty
graph line), preselected pins removed; 16 render + 13 pipeline.
Docs: architecture.md §7 API+UI paragraph rewritten (graph-domain
ruling, /api/graph/clusters, cluster picker as the only entry, no
404-unknown-chat); README creation-flow line updated.
1130/1130, tsc 0, lint 0, build 0.
src/agent/subagents/practice.ts: createPracticeAgents(deps) with the
ExtractionModels mirror (per-stage engine overrides + engineFor +
providerKeyMissing + refuseIfStagesUnready, all three roles strong-tier).
Generator doctrine = the owner's problem-engineering framework verbatim
(minus Sources) + Part III wiring section per the green-lit amendments
(FULL SET input with minimal-use doctrine; degradation never in
meta-commentary). Partitioner: true-partition doctrine, validate via
chainCoverage + exactly-once + unknown-ids, error-back retries through
runSubmitAgent's new validate hook (ToolInputError → error tool result).
Generator: echo law (chains set === target). Grader: verdict union +
nonempty feedback (schema-enforced). renderChains/renderPropositions/
renderParams/renderPreviousProblems. BACKGROUND_MODEL_ROLES extended with
the practice trio (settings override symmetry). runSubmitAgent gains the
validate seam (null = accept). 16-test faux-engine suite incl. retry
paths, refusal gates, neverSubmitted.
runSubmitAgent moved verbatim from extraction.ts to src/agent/subagents/
submit-agent.ts (exported; extraction imports it — zero behavior change,
extraction suites green untouched). New optional prepareArguments seam:
extraction wires its healStringifiedArrays heal; default is identity
pass-through. ModelRole += partitioner | problemGenerator | grader;
ROLE_SLOT pins all three to strong (quality-critical trio, plan default).
Note: BACKGROUND_MODEL_ROLES (per-role settings overrides) deliberately
NOT extended — settings-surface decision, flagged for the planning
session.
Observed live: the auditor looped find for a very long time, never touching
list_sessions/list_propositions. Root cause was the prompt: its only
procedure text was 'For each proposition, search the graph (its key words,
its synonyms, its phrasings)' — a per-proposition find loop with no
starting point, no sense of the space's extent, and no termination rule.
And find returned id+name+statement with no session pointer, so results
never composed with the listing tools (owner diagnosis, 2026-09-29).
- AUDITOR_INSTRUCTIONS now prescribes the process: (1) list_sessions ONCE —
the graph is small and finite, this shows its entire extent; (2) read
plausibly-related sessions with list_propositions (full text) — every
realistic candidate seen in a few calls; (3) find as the targeted sweep;
(4) SUBMIT once related sessions are read — never re-search seen ground
- find hits now carry 'born in: <session id>' — results point at sessions
the auditor can read whole (legacy no-provenance nodes name none)
- identity doctrine untouched: equivalence bias, linkTo mechanics,
verdicts-only deliverable
Diagnosis on tonight's data (4 sources, 173 nodes, 216 edges): supports is
62-83% of edges, topology is hub-and-spoke (34/94 libertarianism nodes are
degree-1 leaves), hub edges read as essay-flow not logic, 33-41% of nodes
are degree<=1 across sources. The old doctrines asked for exactly this:
'every distinct proposition' + 'lonely nodes = failed extraction' +
'COMPLETENESS... omitting is the main way extraction fails'.
Owner rulings, prompt-only (data model untouched, both prompts):
- strict reasoning bar: a proposition earns its node by being a premise,
step, or conclusion of an argument the source actually makes; isolated
facts, name-drops, asides, trivia, example enumerations are dropped
- supports tightened: ONLY explicit grounding acts in the text, rare
exception, genuinely argumentative relations are almost always follows
- completeness inverted: precision over recall, WHEN IN DOUBT OMIT,
never connect merely-related propositions
- MUST-connect clause dropped; corrections of misconceptions stay in
(argumentative by nature)
Identity doctrines untouched (established-terminology names, MERGE YOUR
OWN OUTPUT, aliases) — zero near-duplicate statements proves they work.
The persisted (chatId, sourceId)→node table was a shared namespace across
independent chunk/batch jobs: each chunk's extractor mints local ids fresh
(p1..pN), so later chunks collided with earlier ones — propositions silently
skipped as 'already resolved' (coverage checkpointed anyway: permanent silent
loss) and edges resolved onto the wrong nodes (the 'everything links to
everything' mush). Owner ruling 2026-09-29: mergeFindings resolves THIS job's
ids in memory, nothing persists between jobs; cross-job identity is the
auditor's linkTo verdicts alone.
- contracts: NodeMapping dies; NotesSource.contentFp — sha256 of the text at
the last run, the re-import guard (unchanged re-import = TRUE no-op, no
model calls, no usage; changed text re-runs; legacy unstamped sources
re-run once)
- merger: job-local resolution map, in-submit duplicate ids skip once,
edges resolve extractor-local ids only
- ports/store: appendMapping/readMappings, LEDGER_FILES.mappings,
rewriteLedgers.mappings gone (stale data/graph/mappings.jsonl just stops
being read)
- session-clusters: birthSessionOf is provenance-only — the legacy
mapping-line fallback for provenance-empty notes nodes dies
- deletion: no mapping rows to strip; stats lose mappingsRemoved
- replay-layout batches nodes by first provenance point
- graph.test pins the new independence semantics (re-merge re-creates;
duplicate ids within one submit resolve once); extraction tests pin
per-chunk merges + the fingerprint guard
Hypothesis under test by the owner: coherent extractions turn to mush
after merging because the auditor minted too many bridging edges of its
own — inference, not identity, and out of the auditor's scope. The graph
now connects across sources solely through identified equivalences;
relations between propositions come from the extractor alone.
- auditSchema drops 'edges' (submit_review = links only); the submit-seam
required-field doctrine reworded per tool
- AUDITOR_INSTRUCTIONS: the BRIDGING EDGES section dies; deliverable is
the identity verdicts only
- AuditVerdicts drops edges; ExtractedEdge endpoints are extractor-local
ids only; mergeFindings stops consuming verdict edges and its resolve()
drops the raw-node-id branch (dead — the extractor never sees graph ids)
- extractor untouched: submit_findings keeps propositions + follows/
supports edges, required as before
- tests: 17 submit fixtures + 9 typed verdict literals slimmed to links
- docs: architecture.md division-of-trust + pipeline diagram updated
Every notes chunk and chat coverage batch is its own job — registry row,
usage record, failure domain. Extracts fan out process-wide behind a
3-permit extract semaphore on globalThis (new 'queued' phase); audits and
merges stay serialized behind the existing FIFO gate ('waiting'), so each
auditor explores the graph after every earlier job's merge.
- ExtractionPhase gains 'queued'; RunningExtraction gains part {i,n};
the feed shows one row per job with a '· i/N' suffix (status route)
- abortExtraction is source-wide: every job of a fan-out dies together;
merged jobs keep their coverage checkpoints
- failure isolation at the fan-out (Promise.allSettled + aggregation):
a failed chunk no longer kills its siblings — the run returns what
actually merged; job-level aborts log, pre-fan-out errors still throw
- usage ledgers: one record per job (source-keyed), failed jobs bill
their partial spend
- runJob/aggregateJobs helpers; setPhase/recordStep/extract/audit/
notesExtract key the registry by jobKey (src#i) — notesExtract's
signal lookup was the one real bug the new tests caught (source-id
lookup meant fan-out aborts never reached the model call)
- semaphore is handoff-correct (fresh takes increment; handed-off
permits don't double-count — an inflated counter deadlocks at 3)
- tests: isolation pins rewritten (stage failure now resolves zeros),
single-flight/registry tails updated, per-job usage counts, and a new
describe pinning the fan-out: 3 extracting + 5 queued, parts 1..8,
source-wide abort, per-job usage on failure, extract peak == 3,
audit peak == 1
- docs/architecture.md: serialized-audit paragraph rewritten for
per-chunk jobs; two-front-doors wording aligned
Owner ruling 2026-09-28: the auditor has no control over the
extractor's output and never alters it, so re-emitting the full
findings per submit was pure payload cost — the retry collapse the
schema slim attacked, halved again. submit_review is now links
(local proposition id -> existing graph node; EQUIVALENCE BIAS
carried in the linkTo description) + bridging edges ONLY; the
merger joins extractor findings + verdicts.
mergeFindings(chatId, findings, verdicts, ...) — propositions come
from findings verbatim, links join by id, edges emit extractor's
then auditor's through one dedup-preserving emitter. The old echo
form hid an accidental drop channel: an auditor that omitted a
proposition from its echo silently dropped it from the merge.
Verdicts-only removes it structurally — extractor output always
fully merges (tests updated to full-merge expectations).
AuditResult/AuditedProposition -> IdentityLink/AuditVerdicts;
audit() logs links/bridges; batch view renders anchorless like
transcriptView; auditor prompt rewritten to the verdicts-only
deliverable; architecture.md division-of-trust updated.
OpenRouter log diagnosis (owner, 2026-09-28): extraction time was
dominated by tool-validation retries. The failing call had the whole
propositions array emitted as ONE JSON-STRING ('propositions.0: must
be object'), a truncated placeholder artifact inside it, and
word-salad statements late in the array — payload-size collapse.
Field audit: 'evidence' was never persisted anywhere and its
faithfulness-checking consumer died with the fully-trusted ruling;
'messages' fed messageId granularity that NOTHING downstream reads,
and the model sent [] anyway (chat-level provenance in practice).
'aliases' stays (optional now): the auditor's find tool globs them.
Schema: propositions = {id, name, statement, aliases?}; edges =
{from, to, type}. Prompts lose the evidence/messages doctrine and
[id]-anchor guidance, gain a compact faithfulness line;
transcriptView renders anchorless. The auditor echo instruction
drops the evidence/messages mention, so its payload halves too.
Heal: healStringifiedArrays on the submit tool's prepareArguments
(pi-agent-core's sanctioned pre-TypeBox shim) parses a stringified
propositions/edges back into arrays — the LaTeX-restore doctrine:
heal weak-model wire behavior at the seam. Test drives the real
agent-loop seam with a stringified submit.
Extraction may run in parallel across sources (extractors are
source-local and graph-blind), but the auditor explores the LIVE
graph through its tools and stamps identity verdicts against what
it finds — two concurrent audits would each link into a graph the
other is about to change, the structural mint of duplicate nodes.
withAuditLock: a FIFO promise-chain mutex on globalThis (the same
module-instance idiom as the run registries — a module-level lock
would be per-route-instance and useless in dev). The check runs
before any LLM is invoked or the graph read; the lock spans audit
through merge inclusive (the merge is the graph mutation) and is
reacquired per batch, so interleaved multi-batch runs each audit
against the latest graph instead of starving each other. Applied at
both front doors — chat batches and notes imports.
New 'waiting' phase (extracting → waiting → auditing → merging):
visible in the status feed so a queued run never looks stalled in
extracting. Abort-aware on both sides of the queue: a run aborted
while waiting leaves without auditing (its chain link still
releases — successors never hang); a run aborted mid-audit releases
in finally.
Tests pin: a second run queues at the gate and its auditor is
invoked only after the first run's merge is in the ledgers (the
post-merge-graph invariant itself); a run aborted while waiting
never audits, checkpoints nothing, and leaves a healthy chain for a
third run. Gates: typecheck, 1061/1061, lint, build.
Owner's first live test of the relinked pipeline surfaced three issues,
diagnosed against the ledgers:
1. Detail-card edge labels were backwards (UI only — data and canvas
arrows were correct). Semantics pinned 2026-09-28: an edge A→B
means A implies B (follows) or A supports B (supports). Outgoing
edges now list under 'What follows from this' / 'What this
supports', incoming under 'Follows from' / 'Supported by'.
Verified against the ledger: Invariance --supports--> Twin
paradox / Relativity of simultaneity; Argumentation ethics
--follows--> Homesteading — arrows correct, only the words were
swapped. docs/architecture.md wording fixed to match.
2. Session-local terminology leaked into node names ('Criterion C',
'Action axiom (F1)'). Both extractor prompts gain a naming
doctrine: names use established, discipline-canonical terminology
(glossary-headword test); never source-local bookkeeping (letters,
numbering, positional labels); local shorthand goes to aliases.
3. Near-duplicate propositions and lonely nodes (~40% of fresh-run
nodes edge-less; 'Argumentation ethics' vs 'A priori of
argumentation' duplicated despite the auditor having linked that
phrase once). Root cause is upstream of the auditor: the extractor
had no self-merge or edge-completeness duty. Both prompts gain:
MERGE YOUR OWN OUTPUT (same claim at different precision = one
proposition, phrasings to aliases; nothing downstream dedupes) and
edge COMPLETENESS (a proposition that arose as evidence for,
consequence of, or prerequisite of another must carry the edge —
a graph of lonely nodes is a failed extraction).
Owner rulings on the open question: keep two edge types (no
'relates to' junk drawer); re-measure loneliness after these fixes.
Gates: typecheck, 1059/1059, lint, build.
amosblomqvist