1345599ff71be6586d9241f13f1265d506123258
70
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 amosblomqvist
|
91341a93da |
digest-aware serialized extraction + chain simplification replace the spine stage
Owner live-test verdict on the spine (2026-09-30): no perceptible gain — the real problems (restatement nodes, shallow re-derivations, same conclusion from many angles) are mint-time issues a post-hoc chain minter cannot fix. This arc moves assembly INTO the extractor and adds the deterministic dedupe the owner specified: - Digest-aware extraction: each stretch's extractor prompt carries THE GRAPH SO FAR (its own source's merged propositions + chains, ids verbatim); build on existing ids instead of restating, lean on established conclusions as premises. Mixed-namespace chain endpoints (fresh p-ids OR existing n_-ids) validated at the submit seam. - Serialized per source: a source's batches/chunks run strictly in order (each extract reads every earlier merge); cross-source runs stay parallel; source-wide abort stops the loop. The spine stage, role, phase, usage stage, cost row, and setup roster are gone. - Core-ideas prompt: goal reframed as uncovering core ideas and showing how everything derives from the foundations; direction upward, prefer lean derivations, never re-derive what the graph already holds. - Chain simplification (merger maintenance): simplifyChains drops provably redundant chains at every merge — prefer-fewest-premises + ancestry-cover ([A,B]⟹X with [X,C]⟹Y covers [A,B,C]⟹Y, transitive); same-key provenance lines drop together; rewriteLedgers under the audit lock; droppedChains counted separately in stats. Chunk budget stays 8192 (owner: keep for now, maybe lower later). |
||
|
amosblomqvist
|
52c02533d1 |
extraction: the spine stage — cross-stretch assembly between extract and audit
Per-batch pipeline is now extract → spine → audit → merge (owner order, 2026-09-30), the spine inside the serialized audit section so it reads the graph exactly as the following audit will. Measured motive: 0 of 133 chains in the live corpus crossed a stretch boundary — no stage minted cross-stretch chains. - src/agent/subagents/spine.ts: SESSION/NOTES spine preambles + shared SPINE_CORE (span doctrine; extraction doctrines carried over), the submit_spine TypeBox schema, validateSpine (mechanical span law: every chain touches >=1 fresh endpoint AND >=1 existing digest endpoint; unknown ids, self-reference, and id collisions are loud), the digest prompt builder, and the runner (strong tier, retry-only). - extraction.ts: 'assembling' phase, spine stage in the per-need gate, usage/billing stage, assemble() inside withAuditLock for both front doors; first stretch of a source skips the call (empty digest, no spend); the audit sees COMBINED propositions (chunk + connectives), never chains; merger receives the combined findings. - core/graph.ts mergeFindings: chain endpoints resolve through the job-local map OR pass through verbatim as existing n_ ids (the spine's mixed namespace). - EXTRACTION_CORE: steps-of-reasoning selection bar (examples, notation glosses, and restatements that no argument needs are not steps). - Factory chunk budget 4096 -> 8192 (fewer seams, more local context). - healStringifiedArrays moved to the shared submit seam (submit-agent.ts). - Cost panel: Assembly row; usage records carry a spine stage (optional on legacy records). - Tests +5: prompt pins, span-law matrix, digest rendering, seeded end-to-end flow (spanning chain + connective, verbatim n_ endpoint, spine billed as its own stage), non-spanning refused + retried. |
||
|
amosblomqvist
|
35d8bc1a0c |
physics: imp nodes get the MIDPOINT depth anchor between premises and conclusion
Owner ruling (2026-09-30, live-testing loop — supersedes 02a58db's exemption): implication nodes ARE depth-anchored after all. Their anchor is the average of the HIGHEST premise row and the conclusion's row — depth(imp) = (max(premise depths) + depth(conclusion)) / 2 — so the junction always sits strictly above every premise and strictly below the conclusion (a conclusion is >= highest premise + 1 in the logical graph). Uniform across modes: a half-row in vertical, a ring midway in radial (owner confirmed both). Implementation: - layout.ts: new ImpLink type; SimOptions/incrementalLayout gain impLinks (the derived chain linkage). The rank map is now computed once alongside the orphan mask: proposition ranks via Tarjan over the logical edges, then each supplied link adds its midpoint entry. An imp node WITH an anchor is structured (leaves the orphan mask — gravity projections treat it like any ranked node); WITHOUT supplied links, arg_ ids keep the legacy 02a58db exemption (honest fallback for callers that predate the field). The depth force reuses the precomputed map — one pass, shared. - deriveView (core/graph.ts) emits impLinks (one per multi-premise live chain — single-premise chains mint no imp node); GraphSnapshot carries them; graph-view maps them into CanvasData; settleGraphLayout and replay-layout forward them (replay activates a junction's link exactly when its chain is delivered). Tests (+3, 1 rewritten): the 02a58db exemption pin became the MIDPOINT pin (premises row 0, conclusion row -500, junction pinned to -250); mixed-depth pin (premises at 0 and 1, conclusion 2, junction at the 1.5 half-row — highest premise, not average-of-premises); legacy fallback pin (no impLinks = never depth-anchored); deriveView impLinks emission pin (multi-premise only). The horizontal-pole and ORPHAN RULE pins from 5554c3a survive unchanged — proposition orphans are untouched. |
||
|
amosblomqvist
|
af666aaa3f |
physics: bundles join their cluster; depth runs over the logical graph
Owner-approved brief (orchestrator, physics arc): two laws.
1. IMP-NODE CLUSTER ASSIGNMENT. Imp nodes were unclustered (the partition
runs over stored propositions only) — no centroid pull, outside the
painted hulls, bundles sitting on the periphery pointing outward.
Now: a derived imp node's cluster is a function of its derivation —
MAJORITY among its PREMISES' birth sessions (owner: premises, NOT the
conclusion), ties to the chain's OWN birth (its provenance's first
point — the asserting session; deterministic, coincides with the
unanimous common case). Imp nodes are always re-derived, never
birthed, so the assignment must be exactly this.
Seam chosen: core/session-clusters.ts exports impClusterKey + a
shared attachImpToCommunities that mutates a communities list's
members IN PLACE. graphSnapshot attaches AFTER the hull filter (a
2-proposition community + its bundle never resurrects a hull — the
filter counts propositions); settleGraphLayout attaches to the
unfiltered partition (settle pulls every community; hulls are a
canvas-only concern). Both cluster-membership consumers — the sim's
centroid pull and buildRegions' rasterization — read it through the
members lists that went through the one helper, so the resting
ledger and the live sim agree on where bundles belong.
2. DEPTHS OVER THE LOGICAL GRAPH. inferenceDepths ran Tarjan over the
DRAWN edges (premise→imp→conclusion), inflating every multi-premise
conclusion one hop (depth(imp)+1 instead of depth(premises)+1) and
ranking the geometry point itself. Now:
- deriveView additionally emits logicalEdges — every chain contracted
to direct premise→conclusion arrows over PROPOSITION ids only
(per-premise, deduped — a conclusion's rank is max over premises,
which a single arbitrary premise edge could not express).
- GraphSnapshot carries logicalEdges; createGraphSimulation /
incrementalLayout accept a logicalEdges option: the depth
machinery (orphan detection AND the rank pass) reads it, springs
keep the drawn edges. Absent = edges (legacy behavior).
- Imp nodes join the ORPHAN-EXEMPTION mask explicitly — never pinned
to a level, keep the full regular forces (gravity, cluster pull,
springs), settle naturally between premise and conclusion levels;
the rank map is computed over propositions only, so arg_ ids are
absent from it by construction.
Wiring: settle + canvas (CanvasData.logicalEdges → buildSim) +
replay-layout all pass the logical set.
EXTRA FIX (in-slice discovery, one line, graph-server domain):
server/graph.ts graphClusters — the practice picker's /api/graph/
clusters — partitioned snap.NODES (the canvas image, imp nodes riding
it since chains-primary), leaking arg_ ids into nodeIds: pickable ids
that silently vanish inside chainsForSelection while nodeCount lied.
Now partitions snap.propositions — same purity law as the chain
table's 767a559 fix.
Tests (+13): impClusterKey pins (unanimous; cross-session majority; tie
→ chain's own birth, not the conclusion's); attach pins (members gain
the imp id, sorted; single-premise attaches nothing — no node exists;
dangling mints nothing; home absent from the list stays unattached);
snapshot pins (imp id rides cluster members to both consumers; the hull
filter counts propositions); logical-depth pins (multi-premise
conclusion at premises'+1 NOT +2; imp absent from the rank map);
depth-force exemption pin (rows pin crisply, arg_x floats unpinned —
old behavior stamped it onto the conclusion's row); deriveView logical
arrows (per-premise, deduped, proposition ids only).
|
||
|
amosblomqvist
|
8d72de596a |
fix: chain table reads the stored propositions, not the derived canvas image
The owner's live test surfaced it: after importing notes, the chains view listed implication nodes as bogus singleton "chains" — and each carried the same arg_ id as the real stored chain, tripping React's duplicate-key error (chain-table.tsx:372). Root cause: graph-view passed snapshot.nodes — the derived canvas view (propositions + implication nodes) — into ChainTableOverlay, whose chainsForSelection turns every unchained node into a singleton chain. An implication node is never a chain member (members are proposition ids), so every imp node minted a bogus singleton with a colliding id; its row was even selectable, leaking arg_ ids into practice input. Fix (owner-approved shape): GraphSnapshot gains `propositions` — the stored-truth node list riding beside `chains` for exactly this class of consumer. The chain table now receives snapshot.propositions; the canvas keeps snapshot.nodes (imp nodes are meant to be drawn). Practice pipeline unaffected (it already read getGraph, which is propositions-only). Test: the snapshot split pinned as an invariant in graph.test.ts — image = truth + one imp node per fully-live stored chain; truth never carries an arg_ id. |
||
|
amosblomqvist
|
a2cb1447f7 |
docs: chains-primary reality in architecture.md + README; two stale comments fixed
- architecture.md section 6 rewritten: stored truth = propositions + chains (chains.jsonl); deriveView as the lossy image (implication nodes + untyped arrows, rebuilt per read); the bijection framing (independent arguments for one conclusion stay distinct — the corruption the edge store invited); new deletion-semantics paragraph (proposition death cascades to every referencing chain; arg_ death = one argument); supports retirement stated plainly; auditor's propositions-only scope; clusters stated as birth-session partition truth (no Louvain claim) - section 7 practice chain model: stored-chains inclusion law replaces follows-edge grouping; chainsToGraph bijection claim deleted - README: graph clauses (stored chains, derived view, chain dedupe), practice clause, ledger inventory, and two long-stale line-5 claims (ledger events, semantic top-K) corrected - contracts/graph.ts GraphSnapshot comment: Louvain claim replaced with birth-session truth; extraction.ts comments no longer narrate the dead expandFindings/chainsToEdges seam |
||
|
amosblomqvist
|
4985b6f9fa |
chains-primary: stored chains replace typed edges; edges become a derived view
Owner rulings 2026-09-30: reasoning chains are the stored truth; the graph canvas renders a lossy derived image. No migration, no backcompat — old graph data was wiped. - contracts: GraphChain (arg_-minted id, premises, conclusion) replaces typed edges; GraphEdge survives only as the untyped derived arrow; GraphSnapshot carries stored chains; supports edges retired entirely - store: chains.jsonl ledger (edges.jsonl purged at load alongside legacy events.jsonl); readChains/appendChain seam; merge dedupes by (sorted premises, conclusion), re-assertions append under the same arg_ id - core: deriveView mints implication nodes (name ⟹, statement via formatChainLine) plus premise→imp→conclusion arrows; clusters stay proposition-only (birthSessionPartition); chainsForSelection now groups STORED chains (chain rides iff every member selected) — independent arguments for one conclusion never union; chainsToGraph/chainsToEdges die - extraction: extractor submits chains natively (expandFindings deleted); auditor receives propositions only; supports fan-out gone - deletion: full death of a proposition kills every referencing chain; arg_-prefixed node deletes kill that chain; chainsRemoved stats - UI: every arrow renders the ⟹ double shaft; supports toggle + thickness slider and detail-card edge sections deleted (card keeps name + proposition); chain table + practice pipeline read stored chains - tests: 1188 green across the new seams |
||
|
amosblomqvist
|
06032769d2 |
practice: the learner's creation note — optional free-form input carried to partitioner + generators (owner, 2026-09-29)
- dialog: optional 'Anything else?' textarea (typing-only, 2000-char mirror) - API: body gains note? — trimmed, whitespace-only omitted, hard 400 over 2000 - pipeline: note persisted on the set and threaded to both agent stages - agents: THE LEARNER'S NOTE prompt block (present only when set) + the weighed-never-owed stance pinned in both doctrines - fix en route: doubled THE PARAMETERS heading in both prompts since slice 3 (renderParams owns the heading; the literal blocks removed) |
||
|
amosblomqvist
|
cf0ec4a2f0 | practice UI round — top bars + sidebar reopen, one-attempt per-problem submit with instant reveal, score-only grading (owner 2026-09-29) | ||
|
amosblomqvist
|
8d39f3717c |
practice bug fix — the boot sweep stamped a LIVE run (owner live test 2026-09-29)
Owner report: created a set, 'Interrupted (server restarted)' showed on list + set views, generation proceeded and overwrote it. The sweep's 'generating + no registry row' test had a false-positive mode: it ran at EVERY pipeline instantiation, and an instance could evaluate against a registry whose fresh row it couldn't observe. Three guards, per ruling: 1. HEARTBEAT: ProblemSet.updatedAt? (contract + doc) — bumped on every store write; createSet stamps it = createdAt; saveSet re-stamps (validator tolerates absence for legacy rows). 2. ONCE PER PROCESS: the sweep guards on globalThis .__lhPracticeSweepDone — no instance re-evaluates after the process's first sweep. 3. GRACE WINDOW: only sets whose heartbeat (updatedAt ?? createdAt) is older than 120s are stamped. A live run saves at every phase/ problem transition, so it is structurally unreachable; a true orphan is minutes stale at next boot. One [practice <id>] log line per stamped set (the extraction/jobs console.error idiom). Tests: stale orphan stamped; FRESH generating set untouched (grace); once-guard (second factory instantiation never re-sweeps); live-row protection re-armed via resetSweep() test seam; store pins for updatedAt birth-stamp + re-stamp + legacy tolerance. Architecture.md sweep sentence updated (once-per-process + grace semantics). 1142/1142, tsc 0, lint 0, build 0. |
||
|
amosblomqvist
|
e92679254d |
practice — discretized parameter axes: named levels, never a number (owner-approved 2026-09-29)
Contracts: LadderRung 0-4, CueDistance 0-2, Presentation theory|
application; PracticeParams carries the three named levels. Doc
comments hold the three definition tables verbatim (rung names +
definitions from the ladder doctrine; cue levels; presentation
levels).
Core: LADDER_RUNGS/CUE_LEVELS/PRESENTATIONS tables in src/core/
practice.ts (definitions verbatim) — ONE source spoken by the agents'
prompts, the dialog's stepped controls, and the exam header; core
barrel re-exports practice.ts (pure domain, browser-safe).
Agents: Part I's ladder intro now 'five rungs, numbered; the request
arrives as a rung' with bullets numbered 0-4 (definitions untouched);
Part III's parameters block replaced with the named-level doctrine
(rung = exact class, no continuum; cue counts lesson examples AND
previous problems' situations; presentation dresses, never changes);
partitioner's RESPECT THE PARAMETERS replaced (low rungs = many small
assignments, high = larger; far cue makes confusables-together MORE
important; presentation does not shape the partition). renderParams
outputs the exact named-level block. Anchor/0..1 language swept dead.
Service + store: strict union validation — floats between levels,
out-of-range, wrong strings, legacy dial shapes all 400 at the API
and fail isProblemSet (legacy sets skipped from lists; no migration —
data/practice is disposable scratch, owner wipes before testing).
UI: dialog = 5-detent ladder slider + 3-detent cue slider (each shows
the level NAME + one-line definition) + Theory|Application segmented
toggle; defaults rung 2 / cue 1 / theory; Create posts the new shape.
Exam header gains the levels line ('Direct exercise · Adjacent
context · Theory').
Tests: renderParams exact-block pin; strict-validation pins (new
tests/practice-service.test.ts — bad unions 400, legacy shape 400);
store pins (legacy floats + 2.5 rung fail); render pins (detents +
names + definitions, defaults, detent-swap, toggle active segment,
levels line); every params fixture moved off floats. 1138/1138, tsc
0, lint 0, build 0.
Docs: architecture.md — discretization paragraph (compact tables,
named-level invocation, defaults, one-table source), UI mentions;
README axes sentence.
|
||
|
amosblomqvist
|
a53127a764 |
practice fix follow-up — the picker lists EVERY birth-session group (owner ruling)
Owner escalation on deviation 1: practice availability must not
inherit a visual-rendering threshold. graphClusters() now runs
birthSessionPartition(snap.nodes) directly (same size-desc/id-asc
ordering) instead of reading the snapshot's MIN_CLUSTER_SIZE-filtered
clusters array — MIN_CLUSTER_SIZE stays a hull-drawing rule only.
Label join mirrors the snapshot's (chat title / notes path, null when
the naming record is gone). Contract doc + architecture.md sentence
corrected to match; empty-picker copy reworded ('once a session births
its first proposition' — no 3+ mechanics). Render pins: new empty line
+ a 1- and 2-proposition group listing pin. 1130/1130, tsc 0, lint 0,
build 0.
|
||
|
amosblomqvist
|
f39e1b4294 |
practice fix — creation is graph-domain: cluster picker, nodeIds-only API (owner ruling 2026-09-29)
Owner ruling: 'problem sets always take in nodes — graph objects' —
practice is graph-domain, not chat-domain. No session-created sets.
API: POST /api/practice/sets takes {nodeIds, params, title?} ONLY —
the chatId branch, UnknownChatError, and the any-provenance filter are
deleted; the selector is nodeIds (readNodes filtered to the selection,
unknown ids drop, empty live intersection → 400 empty_selection);
title optional from the UI (the picker's cluster label), dated default
stays. chatStore leaves the pipeline deps entirely (labels resolve
store-side in the clusters service).
GET /api/graph/clusters (new): the birth-session groups the graph view
renders — graphSnapshot's clusters verbatim (same ordering, same
MIN_CLUSTER_SIZE presentation filter), mapped to {key, kind
chat|notes, label (null when the naming record is gone — honest,
never guessed), nodeIds, nodeCount}; corrupt/empty graph → empty
list. GraphClusterGroup contract in core/contracts/graph.ts (client-
safe — the dialog imports it).
practice-dialog: session picker → cluster picker (rows show label +
proposition count, notes clusters marked); Create posts the cluster's
nodeIds + label as title + params. 400 empty-selection copy reworded
for the vanished-nodes edge ('Nothing practicable is left in that
cluster — the graph changed.'); 503 deep link unchanged. Preselected
mode gone — the dialog is the only creation entry for now (the graph-
view cluster entry comes later with free-form selection).
Sidebar: session right-click 'Practice this session's propositions'
removed (item + state + dialog wiring); the /practice nav row stays.
Tests: pipeline — chatId/any-provenance tests die, nodeIds selection
pins added (unknown-id filtering, dated-default title, title hint
rides, empty/vanished → EmptySelectionError); render — cluster picker
pins (label+count rows, notes kind, unlabeled cluster muted, empty
graph line), preselected pins removed; 16 render + 13 pipeline.
Docs: architecture.md §7 API+UI paragraph rewritten (graph-domain
ruling, /api/graph/clusters, cluster picker as the only entry, no
404-unknown-chat); README creation-flow line updated.
1130/1130, tsc 0, lint 0, build 0.
|
||
|
amosblomqvist
|
b7a19609cb |
practice slice 2 — store, paths, self-contained sets
Contract amendment (planning ruling): ProblemSet gains propositions
(nodeId → {name, statement}) and chains (the frozen derivation) — sets
are documents of what WAS selected; grading/dedup/coverage never touch
the live graph. config.ts PATHS.practice → data/practice/. PracticeStore
port (createSet assigns ps_<ts36>_<rand>, getSet, listSets createdAt-desc,
saveSet replace-whole, deleteSet). src/store/practice.ts: one JSON per
set, hand-written isProblemSet read guard (types-only; corrupt/missing →
null/skipped, never thrown; saveSet refuses invalid records). Wired in
src/server/stores.ts. 10-test suite: round-trip, corrupt-tolerance,
missing-dir, validator garbage-reject + optional-tolerate + verdict
union, ps_ id shape.
|
||
|
amosblomqvist
|
05dcaafc68 |
practice slice 1 — contracts, the chain bijection, coverage
src/core/contracts/practice.ts (new domain): ReasoningChain (group by conclusion, id = conclusion node id), PracticeParams (three 0..1 dials), ProblemSet/Problem lifecycle, feedback-only ProblemGrade. src/core/ practice.ts: chainsForSelection (follows-only, both-ends-in-selection, group-by-conclusion, singletons for edge-less nodes), chainsToGraph (the exact inverse — bijection pinned), chainCoverage (the completeness law: union of problems' chains = the set's chains; duplicated allowed). 13-test suite: diamond, premise-only, singleton, supports-invisible, outside-endpoint, dedupe, determinism, round-trips, coverage cases. |
||
|
amosblomqvist
|
8355fbabdb |
delete the mappings ledger — per-job in-memory resolution + notes fingerprint guard
The persisted (chatId, sourceId)→node table was a shared namespace across independent chunk/batch jobs: each chunk's extractor mints local ids fresh (p1..pN), so later chunks collided with earlier ones — propositions silently skipped as 'already resolved' (coverage checkpointed anyway: permanent silent loss) and edges resolved onto the wrong nodes (the 'everything links to everything' mush). Owner ruling 2026-09-29: mergeFindings resolves THIS job's ids in memory, nothing persists between jobs; cross-job identity is the auditor's linkTo verdicts alone. - contracts: NodeMapping dies; NotesSource.contentFp — sha256 of the text at the last run, the re-import guard (unchanged re-import = TRUE no-op, no model calls, no usage; changed text re-runs; legacy unstamped sources re-run once) - merger: job-local resolution map, in-submit duplicate ids skip once, edges resolve extractor-local ids only - ports/store: appendMapping/readMappings, LEDGER_FILES.mappings, rewriteLedgers.mappings gone (stale data/graph/mappings.jsonl just stops being read) - session-clusters: birthSessionOf is provenance-only — the legacy mapping-line fallback for provenance-empty notes nodes dies - deletion: no mapping rows to strip; stats lose mappingsRemoved - replay-layout batches nodes by first provenance point - graph.test pins the new independence semantics (re-merge re-creates; duplicate ids within one submit resolve once); extraction tests pin per-chunk merges + the fingerprint guard |
||
|
amosblomqvist
|
3d9d20794c |
Auditor submits links ONLY — the edges argument is gone (owner, 2026-09-28 evening)
Hypothesis under test by the owner: coherent extractions turn to mush after merging because the auditor minted too many bridging edges of its own — inference, not identity, and out of the auditor's scope. The graph now connects across sources solely through identified equivalences; relations between propositions come from the extractor alone. - auditSchema drops 'edges' (submit_review = links only); the submit-seam required-field doctrine reworded per tool - AUDITOR_INSTRUCTIONS: the BRIDGING EDGES section dies; deliverable is the identity verdicts only - AuditVerdicts drops edges; ExtractedEdge endpoints are extractor-local ids only; mergeFindings stops consuming verdict edges and its resolve() drops the raw-node-id branch (dead — the extractor never sees graph ids) - extractor untouched: submit_findings keeps propositions + follows/ supports edges, required as before - tests: 17 submit fixtures + 9 typed verdict literals slimmed to links - docs: architecture.md division-of-trust + pipeline diagram updated |
||
|
amosblomqvist
|
f11b1664dc |
verdicts-only auditor: submit_review is {links, bridging edges} — the findings echo dies
Owner ruling 2026-09-28: the auditor has no control over the extractor's output and never alters it, so re-emitting the full findings per submit was pure payload cost — the retry collapse the schema slim attacked, halved again. submit_review is now links (local proposition id -> existing graph node; EQUIVALENCE BIAS carried in the linkTo description) + bridging edges ONLY; the merger joins extractor findings + verdicts. mergeFindings(chatId, findings, verdicts, ...) — propositions come from findings verbatim, links join by id, edges emit extractor's then auditor's through one dedup-preserving emitter. The old echo form hid an accidental drop channel: an auditor that omitted a proposition from its echo silently dropped it from the merge. Verdicts-only removes it structurally — extractor output always fully merges (tests updated to full-merge expectations). AuditResult/AuditedProposition -> IdentityLink/AuditVerdicts; audit() logs links/bridges; batch view renders anchorless like transcriptView; auditor prompt rewritten to the verdicts-only deliverable; architecture.md division-of-trust updated. |
||
|
amosblomqvist
|
bd9694b0e4 |
slim the submit schemas: evidence and messages die; stringified-payload heal at the pre-validation seam
OpenRouter log diagnosis (owner, 2026-09-28): extraction time was
dominated by tool-validation retries. The failing call had the whole
propositions array emitted as ONE JSON-STRING ('propositions.0: must
be object'), a truncated placeholder artifact inside it, and
word-salad statements late in the array — payload-size collapse.
Field audit: 'evidence' was never persisted anywhere and its
faithfulness-checking consumer died with the fully-trusted ruling;
'messages' fed messageId granularity that NOTHING downstream reads,
and the model sent [] anyway (chat-level provenance in practice).
'aliases' stays (optional now): the auditor's find tool globs them.
Schema: propositions = {id, name, statement, aliases?}; edges =
{from, to, type}. Prompts lose the evidence/messages doctrine and
[id]-anchor guidance, gain a compact faithfulness line;
transcriptView renders anchorless. The auditor echo instruction
drops the evidence/messages mention, so its payload halves too.
Heal: healStringifiedArrays on the submit tool's prepareArguments
(pi-agent-core's sanctioned pre-TypeBox shim) parses a stringified
propositions/edges back into arrays — the LaTeX-restore doctrine:
heal weak-model wire behavior at the seam. Test drives the real
agent-loop seam with a stringified submit.
|
||
|
amosblomqvist
|
1b50013551 |
extraction cleanup: events ledger dies, markdown source view, auditor-as-linker (2026-09-27 rulings)
The graph becomes raw reasoning chains only. Quiz outcomes and understanding observations are no longer formalized; assessment moves to the future practice module. Events ledger removed end-to-end: LedgerEvent/LedgerEventKind/ UnderstandingLevel/ExtractionAttribution/AuditResult.dropped and snapshot nodeObservations deleted from contracts; quizEvent builder and the events step of mergeFindings gone; appendEvent/readEvents dropped from the port and store; LEDGER_FILES loses events.jsonl with a one-time purge of a stale file next to the nodes ledger; graph-deletion and the delete dialog lose event purge + quiz-answer impact counts; GraphSessionProposition becomes a pure birth list. Whole-session markdown source view: new core/session-markdown.ts is the shared formatting engine (id-anchored speaker blocks, quiz/ recall/question blocks, tool one-liners) — the extractor's prompt view and the future user-facing export render from one engine. Coverage fingerprints deliberately stay on the old line format (content-hash semantics; re-formatting never re-opens covered history). The whole session goes to the extractor — no phase cut. Auditor refocused as the identity linker: never sees the source; explores the graph via new auditor-specific tools in agent/tools/graph.ts (find — glob over name/statement/aliases; list_sessions; list_propositions with full statement text; no timestamps) instead of an injected full-graph dump; equivalence bias — paraphrases merge, only genuinely distinct propositions stay apart; no drop channel, the extractor is fully trusted; may draw bridging edges (typed follows/supports) between existing nodes and the batch, resolved by the merger with chat-scoped message-less provenance. Agentic loop, maxTurns 16. UI: the node detail card gains an edges section above the sessions list (follows-from / supported-by / what-follows / what-it-supports, rendered only when present, click-to-select neighbors); the understanding-note chips are gone. Prober list_propositions reworded to what a session covered — names only, no levels. docs/architecture.md + README updated to the as-built pipeline. Gates: typecheck, 1059/1059, lint, build. |
||
|
amosblomqvist
|
2f39766528 |
free-recall explain tool: the quizzing-mechanic switch (owner 2026-09-27)
Sycophancy mitigation as a structural commitment: the model writes its full correctAnswer (plus explanation, in that order) BEFORE the learner sees the question; the commitment rides the persisted call args, is revealed on submit, and grades his free text in the model's next reply — the client never grades free recall. - tools.explain setting (segmented 'Quizzing — Multiple choice | Free recall' in Agent settings' Tools group): free arms explain and drops quiz for BOTH agents (prober variant carries no nodeId, same pattern as the probe quiz); mechanics keys never pass filterEnabledTools — quiz stays unwritable (the 2026-09-19 wedge ruling stands) - doctrine lives pi-natively in the tool descriptions (owner ruling — no conditional system-prompt chunks): explain carries the commitment rule; the MC option-building craft moved out of the prompt into the quiz description; shared/20-question-tools.md slims to the quiz-agnostic core - display seams: parseExplainInput (LaTeX-debris restoration included), dock text-mode with an 'I don't know' row (kbd row 0), AnsweredCard reveal (The answer / Why, no verdict chips), [Explain answer] format - phases/plan seams recognize both quizzing tools' nodeId tags; explain answers deliberately feed no ledger evidence (no client grading to key on — a future ruling can parse the model's grade) 1049/1049 tests, typecheck/lint/build green. |
||
|
amosblomqvist
|
07b25c4ffd |
de-meta the system prompts; method leads the composition (owner, 2026-09-27)
The owner's capture-vs-composition check confirmed the wire sends exactly
what composes (byte-identical save the nodeId paragraph ebfe058 had since
changed). This wave removes the implementation meta that leaked into the
prompts and reorders so the method carries weight:
- base: owner's new opener verbatim (install understanding; the
understanding definition — which absorbs the old 'Understanding is
structure' paragraph); Voice moved in from shared/10 (deleted); the
philosophy keeps edge-dependence + logistics ('standard learning'
dropped per owner); intro's two-agents/fragments sentence gone.
- teacher/05-method opens with the owner's methodology bridge ('No other
teaching methods come close') — paragraph 3 of the opener, placed here
so the shared base never points the prober at methodology it doesn't
carry.
- deleted: teacher/60-probing-in-brief (self-evident from shared history;
its mid-lesson mini-probe move salvaged into 20-teaching as one
sentence), the 'Session shape — the durable artifact' section
(extraction-pipeline meta), the fragments-meta lines in shared/20 and
05-method, 'the transcript's seam' phrasing, prober's 'the teacher
that takes over' / 'the handoff is automatic'.
- composition reorder (listSourceSegments): base -> the agent's own
fragments -> shared/ appendix. Teacher method now starts at 12% in
(was 62%); prober doctrine at 34%. shared/20 opens with the owner's
decision rule (quiz only for known-correct-answer, ask for everything
else).
- HANDOFF_TEXT: the owner's user-message version — first-person, paces
the planning (unconditional truths, motivated derivations), ends at
plan_init + accept-or-changes.
Teacher 31.4k -> 28.7k chars; prober 10.0k -> 9.6k. Gates: typecheck 0,
1018/1018, lint clean, build green. (Suite runs 16s; the earlier 10-min
timeout was stale node_modules after master's df1328e 0.87.1 sync —
npm install fixed it.)
|
||
|
amosblomqvist
|
d6d7c09e2e | quiz nodeId required (owner ruling 2026-09-26): no untagged teach quizzes — boundary rejects, model retries tagged; off-map checks route to ask_user_question; anti-rush doctrine in schema+prompt; docs+comments updated | ||
|
amosblomqvist
|
145b470752 |
fix: verdict death — plan→teach seam inferred from first nodeId-tagged quiz
The Accept/Decline verdict gate is gone (owner ruling 2026-09-26): presenting the plan ends the teacher's turn; the learner's next message is a natural reply. Teaching begins at the first quiz call tagged with a nodeId (probe quizzes carry none) — both folds (view phaseAt3, native phaseAt3Lh) advance on the call itself, and the teach divider renders before that whole assistant block. No backwards compat: verdict markers, the verdict input event, optimistic appends, composer buttons, and projection all deleted; prompts and docs rewritten to the natural flow. |
||
|
amosblomqvist
|
f3003d357a |
Wave D: native AgentEvent wire + client session rewrite (SDK purged)
Server: ui-stream.ts is verbatim pi AgentEvent passthrough (SSE data lines + [DONE]); only app-level events are the loop-level error and turn-settled terminal; onError carries raw provider text (owner ruling). Client: useChat + the AI-SDK transport replaced by components/session-core.ts (SessionModel: 409 identity-retry, optimistic c_ appends, absorbed() adoption, needsContinuation, shared consume for POST + reattach) and the session.ts hook; the golden-master grouping canon moved client-side into core/view.ts (projectView); scanners read the native record (phaseAt3Lh, danglingQuestionOf, graph toolParts, plan-blocks, runner, extraction). Coverage: the native message is the unit (fork 3) with the call+result inseparability bond rule; view-line-less messages (markers, error results) are invisible to coverage and never checkpointed. Deletes: chat-transport, projection, ui-message, their tests (ported to tests/session-core.test.ts), and the ai + @ai-sdk/react dependencies — the Vercel SDK is fully out of the tree. Wire pins in chat-flow rewritten onto the AgentEvent dialect (eventsOf parser). Gates: tsc clean, 994/994 tests, eslint clean, build green; live smoke on PORT=3004 (verbatim wire → native record). |
||
|
amosblomqvist
|
7ab47d4fcb |
Wave B: the native message record — input events, projection, adapter death
The chat record is now pi-native agent history (LhMessage[]), the AI SDK's
UIMessage transcript contract is gone from the server, and every client turn
arrives as ONE explicit input event.
Core (new):
- src/core/native-messages.ts — the LhMessage taxonomy (user/assistant/
toolResult + lhMarker customs: verdict, probe-handoff, subagent-steer,
subagent-abort, plan-block), constructors, and toLlmMessages — the single
model-facing seam (convertToLlm): strips ids, flattens markers to user
text, drops UI-only plan-blocks, synthesizes INTERRUPTED results for
dangling aborted calls, never throws.
- src/core/projection.ts — projectNative: the one-way native → UIMessage
projection (consecutive assistant+toolResult runs group into one
assistant view message; toolResults attach backward by toolCallId;
markers project to data parts; plan-blocks splice after the last
successful plan_init). Every existing consumer reads through it.
- src/core/input-events.ts — the ChatInput union (text/answer/verdict/
continue), parseChatInput validation, and inputAppends (answer targets
the dangling question call or 400s; verdict = optional note user
message + native verdict marker).
Server:
- chat-handler.ts — the turn root takes {chatId, input}; write-ahead
persists input appends + claimed steers before the turn; claimSteers
coalesces the whole queue with the handler's steerDelivered filter as
the dedupe seam; teacher-vs-prober branch on hasProbeDone.
- turn-service.ts — persistTurn stamps ids, drops clean question
placeholder toolResults (the record ends at the unanswered call),
emits plan-block markers idempotently (plan-blocks.ts rewrite), and
persists abort partials.
- ui-stream.ts — PiTurnSpec takes the native base; the wire dialect is
unchanged. jobs.ts steers are native markers (steer_<jobId> dedup ids);
waitForIdle reads danglingQuestionOf. teacher-agent.ts wires
convertToLlm: toLlmMessages. record-adapter.ts DELETED wholesale.
- GET /api/chats/[id] serves messages: projectNative(record.messages).
Client:
- chat-transport.ts — the 409 retry is IDENTITY (re-POST the same event
body after the turn settles — the event was never persisted
pre-acceptance; the merge logic is gone). The transport consumes the
stashed input event via prepareSendMessagesRequest and throws loudly on
a missing stash.
- chat.tsx — every send call site stashes exactly one ChatInput event
(text, answer, verdict, continue) before invoking the SDK send.
Tests: full native migration (chat-flow rewritten to input events and
native record pins; jobs/extraction/demos/graph/usage/plan-blocks/
probe-done/reattach-transport converted; record-adapter.test.ts deleted;
chat-transport tests pin the identity retry). 982/982, typecheck, lint,
build all green.
|
||
|
amosblomqvist
|
b87a6925af |
Wave A: pi-native tool layer — delete LhTool/toPiTool, TypeBox everywhere, terminate on questions
- src/agent/lh-tool.ts deleted: the zod-wrapped LhTool + toPiTool bridge (z.toJSONSchema per call, clientQuestion flag, hand-rolled result mapping) is gone. Tools are pi-agent-core native AgentTools now. - New src/agent/tool-execute.ts owns what the wrapper added: the abort guard (runWithAbortGuard: run signal + 30s per-call timeout + 250ms grace, 'Operation aborted' / timeout wording preserved), ToolInputError (the retryable '- problem at path' format), result mapping (string verbatim / object JSON / image parts, details rides the record's tool part output), and questionResult() — placeholder + terminate:true (owner Option A: belt of terminate + slim makeQuestionStopper with the clean-question rule; maxTurns deleted from the teaching turn entirely per owner). - All 11 tool files + tools.ts registries converted to TypeBox parameters (typebox@1.3.7, pinned to pi-agent-core's copy). zod refinements (quizInputProblems, askOptionProblems, live-nodeId) moved from schemas into execute — the model sees the same retryable error shape. - question-core + contracts/probe TypeBox-authored (single source shared with the UI); tool-parts parse seam runs the boundary's full chain (Convert → Check → Clean → refinement), fixing the live-smoke lockout: pi's boundary Value.Converts correctAnswer: 17 → '17' but the record keeps raw args, so a parse without Convert rejected a posed quiz (invisible pending question, turn ended). Pinned with the real smoke emission shape. - Subagents converted: demo-builder/researcher AgentTool-native, visual-maker/pdf-reader declaration-only pi-ai Tools with local execute fns, extraction submit schemas re-authored in TypeBox (owner option 2: zod defaults → Type.Optional + fill at the submit seam). - Accepted delta, flagged: TypeBox-native schemas skip pi's JSON-Schema primitive coercion fast path... (boundary still Value.Converts), and prober-quiz unknown keys ride through (Value.Clean strips at display). - Tests: 988 passing (was 967 at the recon baseline); tool-abort and read-tool rewritten against the native seams; question-core/question- tools/tool-parts/demos/web-search/graph-tools/plan-tree/probe-done/ tool-toggles converted to Value.Check/execute pins. Live smoke (PORT=3004, real model): native ls turn streams end-to-end; a real quiz pose (first attempt missing correctAnswer → tool-input-error → model retry → posed) terminates the turn cleanly and persists pending. |
||
|
amosblomqvist
|
95bc8d2261 |
launch message: reasoning before the plan (owner ruling, 2026-09-26)
The probe→teach handoff wake read as a dispatch order ("Author the session's
plan map now") and the teacher skipped the deliberation the planning fragment
asks for. The wake now paces it: take your time, read the assessment and the
probing transcript, reason out the optimal way to teach this person this
thing — floors, ceilings, misconceptions, the minimal tree so every step
lands as obviously true — and only then call plan_init. Machinery honest:
plan_init is still the first tool, the plan map still the first artifact, and
the tail now says to wait for his verdict (which is what presenting actually
does — the old "then begin teaching" tail skipped the verdict seam).
|
||
|
amosblomqvist
|
73027f49bf |
Cost channel cull: the cluster-naming fold dies — two stages, two channels
Off UI's d651f93 flag (displayCostTotals already isolates the display to extraction + audit; the naming stage row and naming runs were filtered out client-side). This culls the server side of the same ruling. THE STANDING RULING, REVERSED BY ITS OWN PREMISE: NamingRunUsage / readNamingRuns / appendNamingUsage were kept at the wave-7 namer kill ONLY because cost-summary's HISTORICAL naming stage read them. With the fold culled, that premise dies — so the whole ledger read path goes too, following the wave-7 embeddings precedent chain: consumer-less ledger seams are deleted, and the on-disk naming-runs.jsonl stays INERT (real trees keep their historical spend bytes; they are simply not displayed anywhere — the same treatment as historical stages.embedding spend). - core/cost-summary.ts: two channels in (chat + notes front doors), two stages out; RecentRun.kind collapses to "extraction" (labels field gone); CostSummaryInput.naming gone; totalDollars = extraction + audit. - ports/store/contracts/config: appendNamingUsage + readNamingRuns ports, store impls + guards, the NamingRunUsage interface, and LEDGER_FILES.namingRuns die. graphRevision no longer counts the file (a smaller revision value is harmless — reads never compare). - components/graph-view.tsx (mechanical, flagged): the defensive .filter(run.kind !== "naming") guards a kind that no longer exists. - Tests: cost-summary's both-channels fold pins; the naming fixtures and the appendNamingUsage store pin die with the seams. |
||
|
amosblomqvist
|
4ded547cb5 |
Context-meter amendment: CONTEXT_LIMITS dies — pi's live catalog is the only limit source
Owner ruling (2026-09-23, amendment to 04e336d): no hardcoded fallback exists anywhere. contextWindow(record, limits) resolves the denominator ONLY from the live index (provider:id -> contextWindow straight from pi's catalog); a model missing from it (historical session, package update, failed fetch) reads limit null and the meter renders usage-only — the honest-render precedent, never a fake window. - src/core/prices.ts: the 1354-entry CONTEXT_LIMITS table (65KB) and its generation apparatus deleted; the fold returns limit: number | null. PRICES untouched (regen verified byte-identical via the reworked sync-pricing script). - scripts/sync-pricing.ts: generates PRICES only; contextWindow leaves the priced map, the guard, and the template. - The dock meter's owner-selected treatment: the activity feed carries the limit server-side. catalogContextLimits() (server/model-catalog.ts) builds the index over the same keyed providers the picker federates; activityFeed() joins it onto every running job (ActivityJob.contextLimit, null when the model is missing — the dock reads it off the feed, no client-side table, no regression while the UI's catalog-join chunk lands). - components/activity-dock.tsx: toBlobs reads job.contextLimit (import of CONTEXT_LIMITS gone); stats-bar renders usage-only on null limit (mechanical interim — UI stream's join chunk follows). - tests: usage.test.ts pins the no-fallback contract (index hit / absent / empty / missing-model -> null); activity-feed.test.ts pins the server join (catalog model -> 1048576, unknown -> null, no-usage -> null); render pins updated to the interim usage-only render. |
||
|
amosblomqvist
|
d1b46132b9 |
Wave 7 close-out: the embedding ledger seam dies; notes birth via mapping lines
Part 1 — the ledger + remnants interlude (master-dispatched, owner ruling 2026-09-23: nothing configures the graph beyond chat models): - The embedding ledger surface is gone: readEmbeddings/writeEmbeddings ports, the EmbeddingsLedger contract, EMBEDDINGS_FILE (config, store impls, graphRevision source list), and graph-deletion's vector-pruning step. On-disk data files stay untouched (inert history). - The extraction embedder seam dies wholesale: ExtractionPipelineDeps drops the embedder dep (chat-handler + backfill/import-notes stubs go with it), the 'embedding' phase leaves ExtractionPhase, the usage capture/persist helpers die, and stages.embedding leaves ExtractionRunUsage (historical records keep their shape — reads are lenient). Extraction is two stages again, like the local-embedder era. - Consumer-less remnants: latestQuizOutcomes + QUIZ_EVENT_KINDS, clusterContentId, matchClusterLabelings + Community + ClusterLabeling, the readClusterLabels/appendClusterLabel port seams + store impls + LEDGER_FILES.clusterLabels. NamingRunUsage/readNamingRuns STAY (standing ruling — the cost pill renders historical naming spend); clusterHue + MIN_CLUSTER_SIZE stay (live). - The 'labeler' role dies: BACKGROUND_MODEL_ROLES, ROLE_SLOT, ModelRole, the catalog pin projections, and the UI roster strings. RolePins keys shrink with the union. The notes birth fallback (owner-approved 2026-09-23, surfaced by the interlude's live smoke: production notes imports create provenance-EMPTY nodes (importNotes passes messages: null — the file's anchor is the mapping line), so the provenance-only birth rule left every imported file unclustered and invisible to the prober's session list, against the locked file-birth ruling. A provenance-empty node now births under its FIRST mapping line's chatId (ledger order); only notes-born nodes ever hit the fallback (chat nodes always carry provenance; a re-encountered node's link-append re-opens the provenance rule). Deletion-safe: mapping lines of deleted chats are stripped alongside their provenance. Live-verified on port 3004: a fresh notes import clusters under its src_ id with the file path as the label, end to end with no embedder. 845/845 tests, typecheck/lint/build green. EOF ) |
||
|
amosblomqvist
|
6b13583e96 |
wave 7: the prober reads its map — graph-context injection dies whole
The owner's zero-configuration ruling (2026-09-23): nothing configures the graph beyond chat models. The prior-knowledge injection — its budget dial, its runner, its marker part, its dock/dispatch entries — is removed across all bands; the prober now reads the learner's persistent knowledge map on demand through two gated tools instead: - list_sessions: every birth session, names + kinds + counts + dates (graphSessionIndex, Platform's core seam) - list_propositions(session): what was born there + the session's own latest understanding level + that evaluation's date (graphSessionPropositions) — names/levels/dating ONLY, no statement text (owner ruling: the map's organization is the prober's business, its prose is not) Registered via createProberTools when chat-handler passes graphStore (the graphToolsEnabled gate decided server-side, per-turn fresh); the prober doctrine gains the map-review nudge at session start. Also: runner's JobKind special-casing collapses (KIND_ROLE/KIND_RUNNERS are plain records again), semantic-clusters.ts dies with its test, the embedder dep's doc tells the wave-7 stub story, and ports.ts's readClusterLabels/readNamingRuns surface stays untouched for Platform's ledger-port interlude. 848/848 tests, typecheck/lint/build green. |
||
|
amosblomqvist
|
0a4a9a11de |
BYOK provider embeddings (wave 6 chunk B): kill local transformers
Replace the @huggingface/transformers local embedder (~316MB ONNX cache) with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id" slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per call from settings + the auth store so key rotations land immediately. - src/server/embeddings.ts rewritten: slug grammar, injected-fetch adapters (OpenRouter input_type, Gemini taskType batching, OpenAI index-ordered rows), honest resolution (null = skip-with-info, set model with missing key = honest error), embedNewNodes with model-change re-embed wipe + ghost pruning + usage steps, federated catalog (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse path 404s; OpenAI text-embedding- filter; Gemini embedContent filter). - GET /api/models/embeddings: federated catalog, empty when no keys. - Extraction tail records stages.embedding usage into the run's money channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the sync-generated block), callCost falls back to it; cost-summary folds the fourth stage; graph-context injection resolves its embedder per run, fails honestly when unconfigured, records query tokens. - Settings: GraphSettings.embeddingModel (null factory), section parser validates slug shape (id may contain colons — OpenRouter :free suffixes, live catalog fact). - Removal: @huggingface/transformers uninstalled, PATHS.modelCache and onnx tracing entries deleted, @huggingface tracing-excludes guard added, stale sharp/libvips comments updated to historical. Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter + 3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5 model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects bad slugs with 400. Gates: typecheck, lint, 895/895 tests, build. |
||
|
amosblomqvist
|
bfa906e2f8 |
feat(models): BYOK all the way down — factory pins deleted, per-need refusal gates
Owner decision 2026-09-22: 'it doesn't make sense to have any default
model configured anywhere; the application is bring your own keys.'
config.ts: MODELS deleted; ModelRole becomes an explicit 9-role union;
DEFAULT_THINKING_ORDER (medium, low, high, minimal, xhigh, max) bakes
the unset-level default — first entry the model supports; explicitly
set levels keep pi's native nearest-clamp.
core/settings-resolution: the factory layer dissolves — the chain is
role override → slot → NOTHING. resolveSettings(record) takes no
factory. Empty chains resolve to null/undefined; a thinkingLevel-only
slot is inert (stored, never run). resolveConversationSpec: chat.model
override over the strong slot, null when both empty. resolveRoleSpec
spans all nine roles for the job/extraction engines.
Refusal envelope (core/settings.ts): modelsNotConfigured(missing) —
{error, code: 'models_not_configured', missing: [slot tiers]} +
ModelsNotConfiguredError with .envelope. Per-need gates, each honest:
- chat-handler: universal turn gate (new + continuing turns) returns
503 before beginTurn — no liveness, no write-ahead persist, no graph
trigger before the gate fires.
- runner.runJob: unresolvable engine lands the job as error with the
envelope wording; no phantom loop, no result steer.
- extraction: refuseIfStagesUnconfigured throws before any stage
(turn-service info-skips so coverage retries next turn; the import
route surfaces per-file code+missing); labeler-less runs skip
cluster naming with a console line and still merge.
Write side: slot entries must carry a model — level-only slot writes
are 400s (absent slot key = clear). piEngineForSpec resolves unset
thinking levels via DEFAULT_THINKING_ORDER; piEngineFor now throws
honestly; thinkingLevelFor/modelIdFor deleted; teacher-agent, pi-run,
visual-maker, pdf-reader all read engine.level.
model-catalog: defaults/slots/rolePins null when the chain is empty
(contracts nullable) — no phantom ids in payloads. Stores drop
MODEL_PINS entirely.
Tests: pin-leaners converted to explicit injection (per-chat model
pins on scripted chats — exactly what the setup flow writes; faux
engines for agent-construction tests). New coverage: the 503 gate
envelope, the runner refusal, the extraction refusal + labeler skip.
Gates: typecheck, lint, 815/815, build.
|
||
|
amosblomqvist
|
f9e3a4c43d |
feat(models): strong/cheap/vision model slots + multi-provider engine federation
Model slots wave (2026-09-20), chunk B:
- settings: new `models` section {strong, cheap, vision} slots (each
field independent, same shape as a role override); ROLE_SLOT maps all
nine roles to tiers (strong=prober+teacher, cheap=researcher/
demoBuilder/extractor/auditor/labeler, vision=visualMaker+pdfReader);
resolution chain per role: override → slot → factory pin.
- config: ModelSpec gains provider (all pins openrouter, the BYOK
default); MODEL_PINS in stores.ts now spans all nine roles (the strong
slot's factory rep IS the prober pin).
- engine: piModels() registers EVERY builtin provider catalog;
piEngineForSpec(resolved spec) is the new engine factory (pi-ai
resolves per-provider keys from env itself); piEngineFor stays as the
config-pin fallback. chat-handler resolves the conversation chain
(chat override → strong slot → pin) and injects engineFor into the job
runner, closing the old gap where per-role overrides never reached
job engines.
- /api/models: models federate ONLY keyed providers (providers surface
with keyPresent, env-wins, fresh per call); slots (resolved tier pins)
join rolePins; catalogDefaultsFor inherits the strong slot; the models
settings section gets the same catalog gate as chat.
- pricing: PRICES/CONTEXT_LIMITS uniformly keyed provider:id
(regenerated from ALL pi-ai catalogs, 1354 keys); usage stamps
(TurnUsage, JobUsage, extraction stages) switch to provider:id —
owner decision: no legacy compat, old stamps render unpriced.
|
||
|
amosblomqvist
|
a3b6a495d3 | Merge branch 'feat/edge-glyph' | ||
|
amosblomqvist
|
cc1e3193f4 |
feat(models): rolePins on GET /api/models — the background roles' resolved pins
Settings interlude (owner request, 2026-09-19): the route now also returns
rolePins: Record<BackgroundModelRole, {model: {provider, id},
thinkingLevel}> — the six background roles' RESOLVED models (the user's
agent-models override falling back to the config.ts MODELS pin), so the
settings UI shows each role row's 'Default: <actual pinned model>' instead
of a generic config caption. Same resolution the agent factories use
(effectiveSettings), so the caption can never drift from what runs.
Pure projection (rolePinsFor) + the fs wrapper (catalogRolePins); pins
tested against config defaults and model/thinkingLevel-only overrides.
BACKGROUND_MODEL_PINS exported from stores.ts for the test (was local).
|
||
|
amosblomqvist
|
3e263a18cc |
Graph pipeline cost metrics: capture naming spend, fold all channels, cost pill + panel
- Cluster namer bills the money channel now: per-step usage + model captured via onStep, persisted as NamingRunUsage records to data/graph/naming-runs.jsonl (idempotent per startedAt; failed/aborted runs persist too — money spent either way). callUsage promoted to extraction.ts module scope for reuse. - core/cost-summary.ts: the pure fold behind the pill — chat runs, notes-source runs, and naming runs into stage totals (extraction / audit / naming) + newest-first recent runs. Dollars INFERRED read-time via prices.ts, never stored; unpriced models excluded-but-counted (render '—', never a silent zero); tokens always sum. - GET /api/graph/costs (src/server graphCosts composition, thin route). - Graph view: bottom-left cost pill (cumulative dollars headline) + expandable breakdown panel (stage rows with tokens secondary, recent runs with time/title/merged/cost); fetched on mount, refetched when a pipeline run finishes; merged into one bottom-left stack with the running-pipeline pills. - Tests: pure-fold suite, naming capture wiring (full importNotes -> naming path), store idempotency + strict reads. |
||
|
amosblomqvist
|
1b9122c88b |
fix(pdf): split pagesRead from pagesConsulted — the honest badge count
Live-smoke follow-up the user approved (2026-09-19): the GEB read showed
"78 pages consulted" when the reader genuinely READ ~12 - pdf_search hit
snippets inflated the count. PdfResult now carries both:
- pagesRead: extract/render output entered the loop - the honest count
the badge and brief footer show ("read N pages", "Pages read: ...").
- pagesConsulted: every page whose content entered the loop, search-hit
context lines included - unchanged, still the citation audit set (the
reader's instructions still say "Pages consulted: N, N, ..." and the
footer still lists it when it exceeds the read set).
The unread-finish gate deliberately stays on CONSULTED: search hits are
genuine consultation (anti-fabrication duty), so a search-only brief is
honest and must deliver - pagesRead is a reporting refinement, not a
delivery gate. Pinned by the search-only faux test (read [], consulted
[1], delivered).
UI tolerates pre-split persisted records via ?? fallbacks (old steers
lack pagesRead).
|
||
|
amosblomqvist
|
9a90dd51d9 |
feat(T2b): sources v0 — read tool + pdf-reader sub-agent
Learner source files, on the teacher's own terms (nothing auto-injects): - read tool (src/agent/tools/read.ts): pi-read port — path/offset/limit, 2000-line/50KB truncation with use-offset continuation notices, image files (jpg/png/gif/webp/bmp) returned as native image content parts (new LhToolImageResult escape hatch in lh-tool.ts). Always-on in both the prober and teacher registries (user-decided 2026-09). - pdf-reader sub-agent (JobKind 'pdf', toggleable, ack-and-steer): triage via pdf_info (pdfjs text layer: per-page text/image/math-density), targeted pdf_extract, regex pdf_search (hits count as consultation), and pdf_render — pages painted by pdfjs onto a canvas shim in headless Chrome (the pdfium viewer doesn't paint headless), screenshot fed back image-in-the-loop (visual-maker idiom, providerSafePng applied). The unread-finish gate bounces a zero-consultation brief once. Teacher gets a text brief with page citations only; renders never leave the loop. - MODELS.pdfReader = qwen/qwen3.8-flash (user override 2026-09-19: the look-at-renders workload wants the vision-proven model; catalog listed glm-5.3-flash as image-capable but qwen is the known-good daily driver). - UI: PdfCompleteBadge + PdfBriefCard (steer-card, job-card), dispatch card + activity-dock kind entries (type-forced, flagged for UI review). - Prompts: 'Learner source files' section in teacher/30-jobs.md; read line in prober/10-probe-doctrine.md. Doctrine rides tool descriptions per the self-contained-tools principle. |
||
|
amosblomqvist
|
5c380de669 |
Merge lh-ui refinement wave (feat/ui-polish @ 8163437) into main
11 commits on fbeca36: minimal-color rounds 1-2 (phase color retired, the braille spinner is the one working indicator; the chip is label-only), typography rounds 1-5 (three-font rule enforced — CMU titles, Source Sans session titles, JetBrains mono; snapped type ladder in globals.css with the xxs rung merged into xs; lh-input caret at the font-metric floor; session titles at prose size, message counts and active-row weight dropped, ISO dates), header rounds 6/6b/6c/6d (60px top bar aligned to the sidebar LH band; the open-sidebar toggle matches the close glyph at size-4 both ways; SidebarToggle always-mounted with a lockstep animated slot — no more one-frame title jump; title at text-3xl). Semantic change folded in: `busy` in the chats list now means exactly "a teach turn is in flight" — background jobs no longer spin the session row (the sidebar's Background agents section is their channel). Merge fallout fixed in the merge: components/graph-coverage.tsx (main's coverage panel) held the last three text-xxs uses; with the rung deleted they silently lost their size — mechanically swapped to text-xs, matching lh-ui's own swap in graph-context-marker.tsx. No file overlap with main's waves since fbeca36 (extraction coverage, lh-graph merge, world-scale bake) — clean auto-merge. Gates: typecheck, eslint, 496/496 tests, build. Live-verified on the dev server: 60px header with 32px CMU title, label-only chip, snapped ladder active (--lh-prose-size 16px, no --text-xxs rule anywhere), sidebar rows with ISO dates, animated sidebar toggle with no header jump. |
||
|
amosblomqvist
|
2678f04e7a |
feat: extraction coverage — message-granular ledger + token-batched runs (d132373 vision)
THE MESSAGE IS THE UNIT OF COVERAGE (vision-extraction-coverage.md): a per-chat append-only ledger (coverage.jsonl) records which messages the pipeline has extracted, by content fingerprint over exactly what the extraction view sees (one shared line builder — core/coverage.ts); the uncovered remainder is DERIVED at read time, never dual-tracked. A quiz answer merging into the asking message re-opens only that message; plan-block data-part attach never re-opens anything. - The trigger (turn-service) replaces every-3-quiz-answers with uncovered-tokens >= EXTRACTION.chunkTokens (4096) — quiz-less sessions extract too, content size is the honest signal. - runExtraction is now chunked: token-budgeted batches of the oldest uncovered consecutive messages (a tool call + result never split), each merged batch checkpointing its messages' coverage atomically — an abort or error loses at most the in-flight batch; merged batches stay covered. - Observation refs are batch-tagged (obs:<batchTag>:<sourceId>): re-runs of the same batch dedupe; a re-opened message's fresh understanding lands as a new event. Fixes the old obs:p1 position-collision. - New API: GET /api/graph/coverage (exact per-session derived coverage + running flags, most-uncovered first) and POST /api/graph/coverage/extract (manual run for one session, 409 while in flight). - Graph view chrome: the coverage button opens a panel with every session's covered/uncovered tokens and a per-session Extract button — the manual answer to aborted runs, errors, and dormant tails. Renders even when the graph is empty: an empty graph with uncovered sessions is exactly when it matters. Polls only while open. - Notes imports keep the whole-file checkpoint path (coverage undefined). - Tests: pure-core coverage suite (11), chunked-pipeline suite (3: multi-batch + no-op, abort-keeps-earlier-batches + remainder re-run, re-open alone + data-part invisibility), panel render suite (5); pendingQuizAnswers and its tests deleted as dead code. 486/486. Live-smoked on real GLM: manual extract of "teach me modal realism" ran one batch, wrote 4 coverage entries, panel flipped to "1.3k covered · complete" with the button disabled, graph refetched (nodes 11), usage persisted with real stats. |
||
|
amosblomqvist
|
326222f53c |
Minimal-color round 1 (owner): phase color retired — the braille spinner is the one working indicator
Nothing exists purely for decoration: phase color-coding is retired app-wide (phases only advance — every session ends Teaching — so phase-as-signal was decoration with a one-time function), and what state genuinely deserved — 'a turn is in flight' — is now carried by MOTION alone, in the terminal's own glyph language. - BrailleSpinner (components/braille-spinner.tsx): the canonical cli-spinner sequence (⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏) as exact dot bitmaps in a 2×4 braille cell, 80ms/frame (800ms cycle). Pure CSS: one shared keyframes, frames staggered via animation-delay; no JS timer, no re-render per frame — cheaper than the old breathing dot. Static contexts (SSR, prefers-reduced-motion) show frame 0 — a legible glyph, not a blank. - Sidebar rows: spinner ONLY while a teach turn is in flight; idle and question-pending render NOTHING (a resting state has no scan value — the meta line's 'Waiting for your answer' text carries it). The three-state phase dot (pulse/solid/hollow) and PHASE_FILL/RING maps are gone. - Top bar PhaseChip: label-only pill; the old dot slot becomes the spinner's home while a turn is in flight (busy prop wired from ChatView's busy = localStream || turnInFlight). Transcript PhaseDividers drop phase colors for monochrome rules + labels. - API narrowing (app/api/chats/route.ts + ChatListItem contract): busy = isTurnRunning ONLY — background jobs no longer count (the sidebar's Background agents section is their channel; a per-row spinner would double-signal them). - globals.css: --phase-* tokens + lh-dot-breathe removed (dead after the sweep — no remaining users). Live-verified: idle/pending rows render no indicator, /chat/new chip is label-only, 10 CSSAnimations at 800ms with correct 80ms stagger, static frame 0 under reduced motion. Gates: typecheck, lint, 470/470, build. Lane crossings to flag at merge: app/api/chats/route.ts + the ChatListItem contract comment (src/core) — the busy-semantics narrowing; components/chat.tsx one-line busy prop. |
||
|
amosblomqvist
|
4022e3f0d0 |
fix: the knowledge-context slider is a discrete 6-stop scale — off, then powers of four from 64
The continuous 0-4096 dial was precision theater (a 128-token budget is ~2 propositions; nobody wants that granularity). Decided stops (owner, 2026-09-15): 0 / 64 / 256 / 1024 / 4096 / 16384 — the native range walks positions 0-5 and maps through GRAPH_CONTEXT_TOKENS_STOPS in contracts. Max rises to 16384 (route range follows); the server still accepts any integer in range so a stale client cannot brick settings; non-stop legacy values snap to the nearest stop for the thumb, readout stays honest. |
||
|
amosblomqvist
|
1d45d77303 |
feat: graph context injection — the session's prior knowledge, steered in at session start (vision-context-injection.md)
One shot per chat, fired at the first turn's onFinish: the session's topic (its only typed message) is embedded as an EmbeddingGemma retrieval query; the semantically closest graph propositions are selected under the session's token budget (pure budget, no similarity threshold — the decided doctrine) and delivered as a marker-only steer through the wake pipeline, queuing behind pending questions like every sub-agent result. - contracts: ChatSettings.graphContextTokens (default 1024, max 4096, step 128, 0 = off); GraphContextData + graphContextText; the graph-context JobKind + buildGraphContextSteer (id steer_ctx_<chatId>) - core/graph-context.ts: brute-force cosine top-K, greedy chars/4 fill under budget, most-recent UnderstandingLevel fold, count cap 64, parse-don't-cast at the marker seam - server: embedQuery (role-asymmetric query format, same warm singleton — vector-space parity by construction); the runner (startGraphContext- Injection one-shot gates; runGraphContextJob validates ledger parity); wake branches (chatIdOf/jobIdOf, finalizeJob done, recoverStaleJobs re-deliver or contextRespawn); abortJob kills context jobs SILENTLY (nothing was promised in the lesson); turn-service trigger seam - adapter: data-graph-context flattens to the canonical block (malformed parts have no model face) - UI: the budget slider (home dock + /chat/new draft, the session-data strip); the quiet SystemLine "~537 · 10 propositions"; dock kind "Knowledge context"; model-picker PUT now spreads settings (a model pick can no longer clear the sibling override) Live-smoked end to end on the real embedder + real GLM: seeded ledger (10 propositions), session on "modal logic: possible worlds and the necessity box" — job in the dock, queued behind the prober's question, delivered on the answer, SystemLine in the transcript, wake turn reads the block and continues the probe. Gates: tsc, lint, 459/459, build. |
||
|
amosblomqvist
|
f5edc7f0a5 |
Semantic topic clusters: local embeddings replace Louvain
Louvain only ever found the reasoning chains (the graph is disconnected by nature — every connected component became its own cluster) and edge-orphans could never join anything. Clusters are now RELATED TOPICS: propositions grouped by meaning. - Embedder (src/server/embeddings.ts): in-process transformers.js v4 running EmbeddingGemma-300M q8 (MTEB 68.4, ~300MB one-time download), MRL-truncated to 512d, "title: none | text:" prefix on 100% of inputs, L2-normalized so cosine = dot. Model/dim mismatch voids the ledger (full re-embed). Injected at the composition root — the agent layer never imports the server layer (architecture law). - Ledger (data/graph/embeddings.json): model + dim + vectors + the persisted cluster partition. Reclustering runs at the END of every embedding batch (post-merge hook, both front doors) — never per read (the agglomerative pass is ~1.5s at 338 nodes). - Clustering (src/core/semantic-clusters.ts): hand-rolled average-linkage agglomerative over cosine (npm HDBSCAN is an abandoned alpha). Stragglers stay singletons by default; attachStragglers is the all-in toggle. Threshold 0.60 swept on the real graph: 26 topics, 74/90 edge-orphans land in real topics, 10 stragglers. - Consumers (snapshot, settle, namer, replay) read the persisted partition — one cluster concept everywhere; core-survival label matching unchanged (20/26 topics kept labels). - Louvain retired: detectCommunities + graphology* deps deleted (graphology-louvain had leaked into the client bundle since the clusters wave); scripts/bench-graph.ts (dead FA2 bench) removed. - scripts/embed-graph.ts: backfill/cold-start (338 nodes in 150s including the one-time model download). |
||
|
amosblomqvist
|
0c14ea425d |
Option A: incremental anchored force layout, persisted server-side
The layout now EVOLVES instead of reshuffling on every extraction. A pure
core module (src/core/layout.ts) warm-starts forceAtlas2 from the positions
ledger (data/graph/positions.json, read-modify-write like sources.json,
updatedAt stamp for future live-physics conflict handling), re-blends every
previously-positioned node toward its anchor each iteration (retention 0.85
- old placements stay recognizable, strong forces still win), seeds new
nodes near their positioned neighbors, and seeds orphans on the rim outside
the core so they read as unconnected knowledge.
- Port/store: GraphStore.readPositions/writePositions; snapshot carries
positions when the ledger exists (optional - fresh graphs unchanged).
- Extraction pipeline settles synchronously after every merge (chat and
notes paths), before usage persist - the next /api/graph fetch already
reflects new nodes.
- View: the ledger IS the layout; the local one-shot FA2 pass survives
only as a fallback when no ledger exists.
- scripts/replay-layout.ts: simulates organic growth by replaying the
current 338-node graph one imported file at a time (real import order,
edges activate when both endpoints exist). Real run: 338 positions in
0.3s, 90-degree-0 orphans collecting at mean r=338 vs connected r=194.
- Also restores the cluster region fills (hull background colors) that
|
||
|
amosblomqvist
|
f0267a5fd9 |
Clusters phases B+C: labeling records, the naming call, snapshot payload
Core-survival labelings (vision section 1, DECIDED 2026-09-13): ClusterLabeling records in a new append-only ledger (data/graph/cluster-labels.jsonl), immutable, id = clu_ + sha256 of the sorted core so re-naming races and re-runs dedupe at the store seam. matchClusterLabelings (core/clusters.ts, pure): a community keeps a record's label while at least half its core survives in it; both sides single-successor (a split keeps one name; greedy by overlap, ties to the newer record). The namer (src/agent/subagents/cluster-namer.ts): ONE singular cheap LLM call (new 'labeler' role, glm-flash/low) names every new/changed community at once, given member names + statements (capped at 12 members per cluster). refreshClusterLabels runs POST-EXTRACTION (user-decided) from both front doors, fire-and-forget — naming failure never fails the run; refreshes serialize behind a chain so parallel imports never mint duplicate labels. import-notes.ts awaits clusterNamingSettled before exit. Payload: GraphSnapshot.clusters (>= MIN_CLUSTER_SIZE 3) joins detection with labelings at read time — claimed clusters carry the record's label and id; unlabeled ones carry a member-hash id and null label. Detection is deterministic so no memo in v1 (ms at current scale; revisit at vault scale). Tests (372/372): matcher half-rule/split/merge/greedy/tie cases; naming end-to-end through importNotes (one call, one record, claim on re-refresh), naming failure non-fatal; snapshot derivation with labeling join; 11 test pipelines now pin an explicit labeler seam. |
||
|
amosblomqvist
|
543cf33789 |
Notes front door: format-agnostic import into the graph pipeline
The same extractor→auditor→merger pipeline now runs over raw personal
notes in arbitrary formats (vision §1, DECIDED 2026-09-13):
- NotesSource records (data/graph/sources.json, deterministic id from
path hash → idempotent re-imports) play the chatId role in mappings,
checkpoints, and observation events; usage lands on the source record
(appendSourceUsage, idempotent per startedAt, kept across upserts).
- mergeFindings is chat-store-free: callers pass MergeSource
{messages, foldedAt} — sessions pass the chat record, notes pass
messages:null (no quiz-event rebuild; a deterministic rebuild needs
structured quiz tool parts).
- Second extractor prompt (NOTES_EXTRACTOR_INSTRUCTIONS) for general-
purpose extraction; understanding judges the AUTHOR's grasp. Auditor
wording input-agnostic. extract/audit share one generic runSubmitAgent.
- graphSnapshot.fileSources (srcId→filePath); notes sources render as
static path entries in the detail card (FileTextIcon, never a link).
- One shared pipeline instance (exported from chat-handler): same
single-flight registry, live status chips (label = file path), usage
accounting, and true abort for both front doors.
- Transport: POST /api/graph/import (multipart, 3-wide worker pool) +
Import notes… button in the graph view; scripts/import-notes.ts for
batch runs (concurrency 4).
- real-data-extraction/ gitignored (raw import sources never committed).
355/355 tests; typecheck, lint, build green.
|
||
|
amosblomqvist
|
de0c286311 |
feat: model + thinking control — the stats bar's picker, on pi-ai's catalog
The model-control wave (user-decided design, 2026-09-12): ONE controller
for the conversation agents (prober/teacher are functionally one agent);
background sub-agents stay config-pinned. pi-ai owns the registry — the
shipped 366-model OpenRouter catalog (cost, context, modality,
thinkingLevelMap), getSupportedThinkingLevels/clampThinkingLevel for the
per-model thinking truth, and streamSimple's automatic clamp. No second
registry to drift out of sync.
Seams:
- config.ts: ReasoningEffort retired for pi's ThinkingLevel
(off|minimal|low|medium|high|xhigh|max). GLM roles go honest at "low" —
pi's catalog says GLM-5.3-flash accepts exactly low|high|max, and the old
"minimal" was silently clamped to low by streamSimple since the swap;
nothing changes behaviorally. visualMaker → qwen/qwen3.8-flash (fixes
the dead qwen/qwen3.8-max slug — the catalog only ships -0902; also the
user's cost pick, 0.15/0.47 per-1M).
- src/agent/models.ts: piEngineFor(role, settings?) — provider+id resolved
from the catalog, thinking level from settings ?? config, CLAMPED to the
model's supported list (a settings change between models can never send
an invalid level). PiEngine gains `level` (observability/tests).
- ChatRecord.settings {model?: {provider, id}, thinkingLevel?} — written
by PUT /api/chats/[id]/settings (replace-whole, no updatedAt churn),
read by chat-handler at turn composition: agents are stateless per turn,
so the change is next-turn-effective by construction. The turns log
still records the ACTUAL billed model (the honest witness).
- GET /api/models: the configured-provider catalog as client-safe DTOs
(core/contracts/models.ts) + config defaults; a second provider joins
the picker by registering it in piModels() — nothing else changes.
- PUT settings validation gate (src/server/model-catalog.ts): unknown
provider/model or unsupported level → 400 carrying the supported list,
never a silent clamp.
- scripts/sync-pricing.ts: regenerated PRICES/CONTEXT_LIMITS from pi-ai's
shipped openrouter.json (offline; same source as the engine + picker).
All 366 models priced; glm's stale 0.15/0.5 → pi's 0.075/0.25, limit
1310720 → 1048576 — cost is inferred at read time, so history re-prices.
- UI (components/model-picker.tsx): stats-bar right side becomes model +
thinking buttons; menus open ABOVE; fuzzy search over "provider id
name" (core/fuzzy.ts — pi's /model pattern); model rows show name/id/
price/context; the thinking menu lists exactly the chosen model's
supported levels (GLM: low|high|max — no dead options, no "off").
Model switch drops an incompatible pinned level (falls back to config,
which clamps honestly) rather than carrying a 400.
Tests (350/350, +15): catalog projection + defaults, the settings gate,
pi thinking truth pinned against upgrades (glm low|high|max; minimal→low;
off→low — GLM cannot disable thinking), the engine factory's
override/fallback/clamp/throw matrix, fuzzy ranking, store round-trip,
and the re-priced folds (usage, stats bar, activity dock).
Live smoke (disposable driver, deleted): catalog route 366 + defaults;
glm+minimal → 400 + supported; turn 1 billed qwen/qwen3.8-flash under
the override; mid-session switch flipped turn 2 to glm; clear-overrides
restored the config pin. ALL CHECKS PASS on real OpenRouter.
|