Commit Graph
75 Commits
Author SHA1 Message Date
sorokin.oleg.edu a50ca4b867 feat: add Russian localization and independent STT languages
CI / Typecheck, tests, lint (push) Canceled after 0s
CI / Standalone (mac-arm64) (push) Canceled after 0s
CI / Standalone (linux-x64) (push) Canceled after 0s
CI / Standalone (windows-x64) (push) Canceled after 0s
2026-10-25 17:18:44 +03:00
amosblomqvist 624c91a437 fix: align compaction cuts and native usage accounting 2026-10-24 19:05:02 +03:00
amosblomqvist 0fc6d4ab94 feat: add verified desktop ChatGPT subscription connection 2026-10-24 12:17:08 +03:00
amosblomqvist 2d67376b76 fix: always run explicit notes imports and surface failures 2026-10-23 04:43:56 +03:00
amosblomqvist 643e7596f0 streaming: connection diet — the parallel-session freeze, fixed
Root cause (postmortem, verified live): Chromium caps 6 concurrent
HTTP/1.1 connections per origin. Every running turn pinned slots for
its whole life — the POST /api/chat response stream plus a reattach
GET stream per visit that never closed on navigate-away — so two
concurrent thinking sessions starved navigation RSC fetches in the
browser queue (147s measured) until the turns settled. Server was
exonerated (3ms curls during the freeze); main thread idle throughout.

- COMPACTION LAW + WIRE DIET (ui-stream): store the states, not the
  chunks — replace-in-place snapshot retention keyed by updateSlots,
  ~100ms coalesced latest-wins fan-out, slim wire drops pi's duplicate
  assistantMessageEvent payload.
- DIET B: POST /api/chat acks and closes ({ok:true}) — the turn wire
  is GET /stream for everyone, submitter included; send() reattaches
  via refetch-first resumeStream(true).
- DIET A: unmount cleanup calls model.stop() — navigating away aborts
  the view connection (turns are server-side background objects).
- Answer-flicker fix: resumeStream's eager tail clear moved off the
  refetch path (the refetch swaps base+tail atomically; clean-slate
  guard restored on refetch failure) — the optimistic answer card
  survives refetch latency (1.7s stalls measured on dev).
- Tests converted to the ack contract (drainTurn/drainRunning helpers,
  dietFetch mock) + a pin that the POST body reads while the turn runs.

Gates: 1489/1489, typecheck, lint. Live: dual streaming sessions,
navigations 335-648ms, in-page settings 19ms (was 147s).
2026-10-10 09:42:47 +03:00
amosblomqvist 34f498b128 artifacts: the height ratchet fixed, invert switch persisted + restyled (release polish 7.2)
- bridge measures the BODY (scrollHeight + rect + margins) instead of the
  viewport-clamped documentElement — documentElement.scrollHeight latches
  at the parent's own height, so demos could grow but never shrink on
  window resize (owner-reported); smoke: 1036 → 2386 → 1036 px
- invert now PERSISTS as details.ui.inverted on the steer record, written
  by the silent artifact-ui route (dismissal pattern: no wake, no SSE) and
  seeded through the projection (SubagentArtifactData.ui); survives
  navigation + reload, never enters teacher context or exports
- invert restyled as the house Toggle switch (settings controls) on both
  cards; demo header is the caption grammar: 'INTERACTIVE DEMO — title'
  left (title normal-case, same size/tracking), switch + download far
  right; figure header: label left, switch/download/open far right

1439 tests green (route + pure-seam pins added); live smoke passed
2026-10-07 19:13:24 +03:00
amosblomqvist 0b2e4e03fb sessions: export as markdown — the sidebar right-click, the route, the practice join (release polish 3)
Owner round (2026-10-06): markdown export of sessions, actioned from
the sidebar context menu — "Export as Markdown…" sits with the
management actions, above Delete (the destructive action stays last).

- GET /api/chats/:id/markdown (404 unknown): the whole session as one
  attachment — text/markdown; charset=utf-8, filename from the slugified
  title ("relational-databases-joins-normal-forms.md", non-ascii titles
  fall back to "session.md"). Both session kinds export.
- The client is a same-origin anchor download; the route's
  Content-Disposition names the file.
- sessionMarkdown's user-facing debut — minted ANCHORLESS (message-id
  anchors are extraction provenance, not learner-facing). The engine
  joins the core barrel (pure, browser-safe) for its first consumer.

The practice join (owner-approved mid-round): practice tool results are
bare acks — statement, outcome, verdict, and the model solution all
live in the assistant's call arguments. The engine now folds them the
same way the view seam folds cards: Problem blocks render the pose's
statement (solution never leaks at pose time); Concluded blocks render
the outcome as prose ("Solved with hints.") + verdict, then the
pose-committed model solution as a blockquoted reveal via
problemInputForConclude (clean-poses-only, its refusal law). A
dismissed problem concludes as "The learner skipped this problem." —
no reveal. Errored poses render nothing (the failed-tool law).

1427 tests (route pins: headers, slug, anchorless body, 404; engine
pins: pose-time statement, the reveal, the dismissal refusal, the
errored-pose silence); tsc, lint, build green.
2026-10-06 10:23:16 +03:00
amosblomqvist a2db80b72d compaction: the usage-anchored trigger + visible session state (round 15)
The relational-databases incident, diagnosed and fixed:

TRIGGER — pi's estimator law, adopted exactly: anchor on the last
assistant message's provider-reported usage (totalTokens || the field
sum) and chars/4-estimate ONLY what was appended after it. The anchor
scan skips aborted/zero-usage assistants and STOPS at the newest
compaction marker (pre-cut usage described context the next request
will not carry — pi's distrust rule). The old estimator
(JSON.stringify of every message /4) tripled the true context
(measured: 84,412 'tokens' estimated vs 38,799 provider-reported on
the incident record) — scaffolding, escaped LaTeX, thinking blocks
and app-side details all counted. estimateMessageTokens now counts
wire text only (text + thinking + toolCall name/args; content text
for user/toolResult/custom). The unanchored fallback runs the plain
heuristic over the CUT-HONORED projection plus the additive prompt
term; the anchored path adds nothing (usage already billed prompt
and tools).

OBSERVABILITY — compaction is a visible session-wide state (owner
spec): the transcript-bottom placeholder register gains COMPACTING
CONTEXT… (BrailleSpinner + Shimmer, the THINKING placeholder's twin,
mutually exclusive by construction — the gate precedes beginTurn),
and the sidebar row shows 'Compacting context · <ago>' with the
spinner (sessionMeta outranks busy; compacting-changed joins
turn-started/turn-settled as refresh triggers). Liveness rides the
API like turn-busy: ChatListItem.compacting + compactionInFlight on
both GET /api/chats and GET /api/chats/[id] (initiallyCompacting
covers the navigate-back-mid-compaction mount — the event already
fired), exported through the server barrel.

Also: the relational-databases record rolled back to pre-compaction
(marker, post-marker turns, tree reset; backup in /tmp) for a clean
retest. The summarizer-usage 'gap' was a misread — steps[] was
populated all along.
2026-10-06 00:38:15 +03:00
amosblomqvist ffc4df1938 compaction: the surface — settings row, two ruled dividers, summary dialog, send gate (round 14 slice 3)
- settings: compaction.thresholdTokens row on the Agent tab (owner pick)
  in a Context group — CommitInput mirroring the graph budget row, hint
  resolved through the SAME core resolver the trigger reads; PUT rejects
  below the 32k floor with the anti-churn rationale; own section
  (compaction) in SETTINGS_SECTIONS + the settings-signal union
- view fold: one marker → two dividers — the minimal MOMENT divider at
  the marker (retrying-message register, no rule lines) and the longer
  CUT divider emitted immediately before the first kept message
  (phase-divider geometry, chip carries '· Nk tok'); both carry the full
  summary payload; full history stays visible to the user
- components/compaction-divider.tsx: the dividers + summary dialog
  (pure exported body for SSR pins, the delete-dialog pattern; Esc +
  Close dismiss, backdrop does not)
- chat.tsx: compaction-changed SSE flips the running state — composer
  canSend, onSend/doSend/doSteerOrSend, and both docks' submit + hint
  ('compacting session…') gate on it (the server's registry mutex is the
  structural gate; this is the affordance); dialog mounts beside the
  abort dialog
- pins: settings-body (4), view fold (2), divider SSR (4)
2026-10-05 22:51:08 +03:00
amosblomqvist 85022ae143 practice hardening: weak-model argument heal + parked dismissals mid-turn
The GLM incident (2026-10-04): glm-5.3 sent mode as a JSON-string-encoded
enum ('"recall"') on 19 consecutive problem calls; TypeBox's error never
named the expected values, so the model re-emitted the identical payload
until the dev server was killed. And mid-turn the learner was trapped:
Esc's question/problem rungs only dismissed — the dismiss route 409'd
against the turn-owned record and nothing could stop the running turn.

- healEncodedArguments (practice.ts, the prepareArguments seam): a STRING
  where the schema expects a literal union / boolean / null / number /
  array is JSON.parse-d and kept when it satisfies the schema. Valid args
  pass through untouched (same reference); unhealable values stay for
  validation to reject honestly. Wired on BOTH problem and conclude.
- dismissal-park.ts: the dismiss route parks a mid-turn dismissal (202);
  the turn's settle path (onFinish, after persistTurn, before endTurn —
  the same ordering law as the record write itself) applies it against
  the persisted record, only if the dismissed thing really is pending.
- confirmAbort: question/problem discard while busy now ALSO aborts the
  in-flight turn; the dialog line says so (busy prop).

No error-result cap (owner ruling). 16 new pins (heal incl. pi-ai
validateToolArguments end-to-end; park registry, apply paths, the 202
route contract).
2026-10-04 23:57:00 +03:00
amosblomqvist 90893ae236 graph context: a separate marker message before the first user turn
The map picks no longer glue into the opening prompt as seed text
(owner, 2026-10-04 late). POST /api/chats accepts graphContext
{propositions}; the route resolves the picks against the live graph,
derives the reasoning chains among them (chainsForSelection →
formatChainLine), and writes a graph-context custom marker as
message[0] — before the learner's first user message, in either mode.

- Core: newGraphContextMessage (model-facing content = the
  contextualization framing + proposition lines + chain lines;
  details = the structured pick); toLlmMessages flattens it to a user
  message like every marker, so the model reads it for free.
- View: projects to a user-side data-graph-context part.
- UI: GraphContextCard — minimal, muted, dashed, collapsed by default
  (counts only); expand reveals name — statement rows and the chain
  lines. Same compact-expandable pattern as the practice cards.
- Home: sends graphContext with the creation whenever picks exist;
  the prompt param carries the learner's own words only.
2026-10-04 23:07:13 +03:00
amosblomqvist 7dcb006cf3 session kind: ChatRecord.kind teach|practice (absent = teach)
Practice sessions (owner, 2026-10-04): the marker rides the record and
the normalized summary (absent field reads as teach — pre-practice
records never migrate); POST /api/chats accepts {kind: 'practice'}.
The agent branch, tools, and UI land in the following slices.
2026-10-04 21:36:14 +03:00
amosblomqvist 814c2ad412 delete the batch practice-problems module (superseded by practice sessions)
Owner ruling 2026-10-04: the module is a sunk cost, retrievable from
history if ever revisited. Removed: core practice contracts + pipeline
helpers, the PracticeStore port + adapter + data dir path, server
service + API routes (sets, clusters picker, practice aborts), the
practice pages + dialog/exam/list UI, chain-table's Generate handoff,
sidebar nav, activity-dock practice rows, and the partitioner/
problemGenerator/grader model roles (BACKGROUND_MODEL_ROLES 8 → 5).

Survivors, moved to core/chains.ts (pure + browser-safe): ReasoningChain,
chainsForSelection, formatChainLine — the chain table and graph
emission still consume them.
2026-10-04 21:34:24 +03:00
amosblomqvist 5271ca0e4a composer steering: the general steering machinery (owner arc 2026-10-03)
STEERING, NOT FOLLOW-UPS — pi-agent-core's two rails, used deliberately:
agent.steer() injects at the next response boundary INSIDE the live run
(one logical turn, the model changes course); followUp() would only extend
an ended run — not built. The same rail job markers already ride
(steeringMode 'all', the 2026-09-26 doctrine).

- POST /api/chats/[id]/steer: 404/400 honest guards; no live turn → 409
  and the client silently falls back to a plain text send (the settle
  race); live → steerLiveChat hands the message to the live agent's
  steering queue AND parks it in the userSteers registry — the stranding
  safety net. A steer stranded behind a question termination or abort
  stays parked and rides the chat's NEXT run (handOffEligible), the same
  deliver-or-requeue lifecycle as job markers. Reconciliation at settle:
  delivered ids drop (reconcileUserSteers).
- Client: steer(text) queues + POSTs without flipping any turn state;
  the drain's verbatim message events upsert the real user card by the
  client-minted id (steer_<localid>) and the indicator fold drops the
  line. canSend no longer gates on turnInFlight — Enter mid-turn steers;
  Enter with an empty composer still stops; the square button stays Stop.
- The queued-steer indicator (owner spec): quiet one-liners above the
  composer shell — right-aligned to where its corner rounding starts,
  truncated with an ellipsis, fixed max width, ~half opacity, no
  background, no container. Tunable later.
- Verified from pi-agent-core source: steering drains per completed turn
  inside the loop; injected messages emit message_start/end verbatim;
  finishTurn {action:'end'} (our question stopper) exits BEFORE the
  steering poll — hence the registry.
2026-10-03 03:01:51 +03:00
amosblomqvist 673a430d6e question dismissal, A2: the silent write + the dismissal pause (owner design 2026-10-03)
- {type:"dismiss"} input event: parse + pure appends with the answer-event
  orphan discipline (dangling-call 400, kind match, answer must be dismissed)
- POST /api/chats/[id]/dismiss: the no-turn composition — write-ahead append,
  409 while a turn runs, emits record-changed (new bus word; idle views sync)
- needsContinuation: an all-dismissed trailing batch demands nothing;
  endsInDismissal = the derived pause predicate (tail-based, no latch)
- wake pipeline: while the record ends in a dismissal the pump holds every
  delivery in the steer queue — the user's next send is the wake (until the
  text-turn steer claim lands, settlement re-pump delivers right after it)
- session-core.dismissQuestion: optimistic skipped card + silent POST;
  chat.tsx confirmAbort question-mode swaps submitAnswer for it; ESC-abort
  popup everywhere (AbortTurnDialog) with popup-Esc claimed via state gate
- parallel question calls dismiss call-by-call (results bind by id)
2026-10-03 01:51:13 +03:00
amosblomqvist 26131b3599 push architecture slice 7: graph-status + practice surfaces ride the events channel; coverage polls accepted-by-design; stale poll comments + architecture.md observability section rewritten to the push model 2026-10-02 18:03:47 +03:00
amosblomqvist 78c7393909 push architecture slice 1: core event bus + /api/events SSE channel (notify->fetch infra, no consumers yet) 2026-10-02 17:04:22 +03:00
amosblomqvist e7f4923159 deletion screen minimal + 'node' as THE UI word (owner ruling)
Owner 2026-09-30: the confirm dialog's pre-flight impact read had to
go — what full death takes is obvious, the per-open fetch flickered,
and richer facts would only confuse as deletion semantics evolve
(rewiring is on the table). The screen is now: Delete node? / Delete
N nodes?, the names in the mono block, one permanence line, Cancel /
Delete. No loading state, no gone state (a 404 mid-run lands in the
honest failure line), no sessions join — the joins prop died too.

The dead path is deleted end to end: GET /api/graph/nodes/[id],
graphNodeImpact (server), nodeDeletionImpact (core) and their tests.

WORDING: node/node(s) is THE UI word for graph objects, standardized
across every surface's copy (owner, same ruling): the dialog titles,
the chain table's counts/search/summary/delete aria-labels, the
practice dialog's cluster counts, selection line, and empty-cluster
hint. Internal identifiers (selection.propositions) unchanged.

Rewritten dialog pins (tests/delete-node-dialog.test.tsx): minimal
contract — no facts/loading/fetch copy, singular/plural titles, the
cap, 409 dock phrasing, busy states, no 'proposition' left in the
dialog.
2026-09-30 01:19:03 +03:00
amosblomqvist 06032769d2 practice: the learner's creation note — optional free-form input carried to partitioner + generators (owner, 2026-09-29)
- dialog: optional 'Anything else?' textarea (typing-only, 2000-char mirror)
- API: body gains note? — trimmed, whitespace-only omitted, hard 400 over 2000
- pipeline: note persisted on the set and threaded to both agent stages
- agents: THE LEARNER'S NOTE prompt block (present only when set) + the
  weighed-never-owed stance pinned in both doctrines
- fix en route: doubled THE PARAMETERS heading in both prompts since slice 3
  (renderParams owns the heading; the literal blocks removed)
2026-09-29 22:16:28 +03:00
amosblomqvist cf0ec4a2f0 practice UI round — top bars + sidebar reopen, one-attempt per-problem submit with instant reveal, score-only grading (owner 2026-09-29) 2026-09-29 21:54:32 +03:00
amosblomqvist 88eb4b2ec0 practice restructure — per-problem jobs + ONE shared background-LLM pool (owner ruling 2026-09-29)
Rulings: each problem generation is its own visible dock job (the
extraction per-chunk pattern), and practice shares extraction's
semaphore — one process-wide 3-permit pool for ALL background LLM
jobs; total background LLM concurrency bounded at 3.

A. SHARED POOL: new src/agent/subagents/bg-llm-semaphore.ts —
   globalThis __lhLlmSlots, PERMIT_COUNT 3 (the ruling), the
   fresh-take/handed-off-permit deadlock discipline preserved for
   BOTH callers. extraction.ts's withExtractSlot is now a thin
   wrapper (job-row signal → withBgLlmSlot); __lhExtractSlots and
   __lhPracticeSlots both die. The partitioner now acquires a permit
   too (it's a background LLM job — it didn't before).

B. REGISTRY ROWS: the aggregate setId:generation / setId:grading rows
   are replaced by setId:partition (phase partitioning, label
   'Partitioning — title'), setId:gen:problemId (queued→generating,
   'Problem i/N — title'), setId:grade:problemId (queued→grading,
   'Grading i/N — title'). Each row carries its OWN abort controller
   and its OWN usage (one PracticeStepLog per row — the aggregate
   partitioner/generator/grader folding dies). Row key = feed id =
   abort key (the dock dup-key law).

C. ABORT SEMANTICS: abortPracticeRow(key) routes by shape — gen row:
   THAT problem only (error 'Aborted', siblings continue, set settles
   ready with one errored problem); partition row: the whole run;
   grade row: that grade drops (answer kept, re-submittable).
   abortPractice(setId) composes from row aborts + the synchronous
   persisted marks (a set-abort's verdict stands before the settles
   re-read it — the settle step keeps an existing error status).
   /api/activity/abort kind practice now takes the ROW key (charset
   admits ':').

D. FEED + DOCK: ActivityPractice/FeedPractice carry per-row identity
   (id = row key, problemIndex/problemTotal); dock blobs render per
   row, labels carry the i/N; the aggregate row is gone.

E. Boot sweep unchanged at set grain; the live-row check is now a
   key-prefix scan (any live row of the set protects it). The
   updatedAt heartbeat covers the new row scheme as-is.

Tests (20 in the pipeline suite): per-problem row shape/uniqueness/
i-N labels (partition row frozen behind a gated partitioner; per-row
usage ISOLATION pin — b's log never shows on c's row); abort-one-gen
(sibling completes, set ready with one errored problem); partition-
row abort; queued grade-row abort (dead-ticket path: answer kept,
re-submittable, sibling lands); shared-pool identity + contention
(practice queues behind extraction-style holders); registry/dock
feed mapping. Test isolation hardened: unique per-store set ids,
afterEach registry/lock/pool reset (a timed-out test can no longer
poison the rest).

architecture.md: jobs-pattern paragraph rewritten (per-problem rows,
shared pool, abort routing).

1149/1149, tsc 0, lint 0, build 0.
2026-09-29 20:42:33 +03:00
amosblomqvist f39e1b4294 practice fix — creation is graph-domain: cluster picker, nodeIds-only API (owner ruling 2026-09-29)
Owner ruling: 'problem sets always take in nodes — graph objects' —
practice is graph-domain, not chat-domain. No session-created sets.

API: POST /api/practice/sets takes {nodeIds, params, title?} ONLY —
the chatId branch, UnknownChatError, and the any-provenance filter are
deleted; the selector is nodeIds (readNodes filtered to the selection,
unknown ids drop, empty live intersection → 400 empty_selection);
title optional from the UI (the picker's cluster label), dated default
stays. chatStore leaves the pipeline deps entirely (labels resolve
store-side in the clusters service).

GET /api/graph/clusters (new): the birth-session groups the graph view
renders — graphSnapshot's clusters verbatim (same ordering, same
MIN_CLUSTER_SIZE presentation filter), mapped to {key, kind
chat|notes, label (null when the naming record is gone — honest,
never guessed), nodeIds, nodeCount}; corrupt/empty graph → empty
list. GraphClusterGroup contract in core/contracts/graph.ts (client-
safe — the dialog imports it).

practice-dialog: session picker → cluster picker (rows show label +
proposition count, notes clusters marked); Create posts the cluster's
nodeIds + label as title + params. 400 empty-selection copy reworded
for the vanished-nodes edge ('Nothing practicable is left in that
cluster — the graph changed.'); 503 deep link unchanged. Preselected
mode gone — the dialog is the only creation entry for now (the graph-
view cluster entry comes later with free-form selection).

Sidebar: session right-click 'Practice this session's propositions'
removed (item + state + dialog wiring); the /practice nav row stays.

Tests: pipeline — chatId/any-provenance tests die, nodeIds selection
pins added (unknown-id filtering, dated-default title, title hint
rides, empty/vanished → EmptySelectionError); render — cluster picker
pins (label+count rows, notes kind, unlabeled cluster muted, empty
graph line), preselected pins removed; 16 render + 13 pipeline.

Docs: architecture.md §7 API+UI paragraph rewritten (graph-domain
ruling, /api/graph/clusters, cluster picker as the only entry, no
404-unknown-chat); README creation-flow line updated.

1130/1130, tsc 0, lint 0, build 0.
2026-09-29 19:26:37 +03:00
amosblomqvist 27c5d9df19 practice slice 4 — pipeline, registry, API, feed
src/agent/subagents/practice-pipeline.ts (sibling of practice.ts —
orchestration vs calls): createPracticePipeline(graphStore+practiceStore
+chatStore+PracticeModels). Fire-and-forget front doors returning
{set, done}: createProblemSet (refuse → resolve selection (chatId =
ANY-provenance membership per brief; nodeIds raw) → chains → snapshot →
createSet partitioning → registry row → runPartitioner → stubs land
(generating) → ONE generator per problem behind the process-wide
3-permit abort-aware semaphore (__lhPracticeSlots, extraction idiom) →
per-problem failure isolation → ready (errored problems included);
abortPractice (idempotent: controllers + 'Aborted' marks; dropped
grades keep answers, re-submittable); submitAnswers (ready-only,
already-graded rejects, one grading job per answer under the same
semaphore); deleteProblemSet (abort-first). Per-set write mutex on
globalThis (__lhPracticeLocks, audit-lock idiom); registry
__lhPracticeRuns one row per set per kind; boot sweep marks orphaned
partitioning/generating sets 'Interrupted (server restarted)'. Typed
errors: EmptySelection/UnknownChat/SetNotFound/SetNotReady/BadAnswers.
Usage folds stage logs into the registry row (v1: completion-folded).

src/server/practice.ts: thin service — strict params parsing, typed
error → response mapping (503 envelope family, 400/404/409), list/get.
Routes: POST+GET /api/practice/sets (202 {id}), GET+DELETE
/api/practice/sets/[id], POST /api/practice/sets/[id]/answers (202).
Composition root: chat-handler wires createPracticePipeline (engineFor
fresh + providerKeyMissing gate) exported as ; barrel re-
exports. Feed: observability activityFeed() += practice rows
(ActivityPractice); dock blobs (process 'practice', GraduationCap
icon, set-keyed abort); /api/activity/abort kind 'practice'.

tests/practice-pipeline.test.ts — 13 tests: end-to-end (stubs mid-
flight via gated generators, coverage complete), failure isolation,
3-permit semaphore pin, abort mid-generation, registry shape pins,
refusal-before-create, chatId any-provenance selection, typed errors,
answers→grades land + graded rejects, boot sweep (orphan + live-row),
delete. 1114/1114, tsc 0, lint 0, build 0.
2026-09-29 18:31:53 +03:00
amosblomqvist 4001c3465b Graph view: unique pill keys for chunk jobs (second duplicate-key consumer)
The graph page's pipeline pills keyed rows by run.chatId — the source id,
shared by every chunk job of a fan-out (the dock got the same fix in
8f66f47; this consumer was missed). /api/graph/status rows now carry an
id (the registry job key: source id, or 'src#i'), and the pills key by
it. chatId stays the SOURCE id — the reveal refetch and per-run cost
refresh track chatId SETS, which dedupe naturally and fire when the
source's LAST row leaves the feed.
2026-09-29 01:26:23 +03:00
amosblomqvist d047e7b2dc Per-chunk jobs: parallel extracts under a 3-slot semaphore, serialized audits (owner 2026-09-28)
Every notes chunk and chat coverage batch is its own job — registry row,
usage record, failure domain. Extracts fan out process-wide behind a
3-permit extract semaphore on globalThis (new 'queued' phase); audits and
merges stay serialized behind the existing FIFO gate ('waiting'), so each
auditor explores the graph after every earlier job's merge.

- ExtractionPhase gains 'queued'; RunningExtraction gains part {i,n};
  the feed shows one row per job with a '· i/N' suffix (status route)
- abortExtraction is source-wide: every job of a fan-out dies together;
  merged jobs keep their coverage checkpoints
- failure isolation at the fan-out (Promise.allSettled + aggregation):
  a failed chunk no longer kills its siblings — the run returns what
  actually merged; job-level aborts log, pre-fan-out errors still throw
- usage ledgers: one record per job (source-keyed), failed jobs bill
  their partial spend
- runJob/aggregateJobs helpers; setPhase/recordStep/extract/audit/
  notesExtract key the registry by jobKey (src#i) — notesExtract's
  signal lookup was the one real bug the new tests caught (source-id
  lookup meant fan-out aborts never reached the model call)
- semaphore is handoff-correct (fresh takes increment; handed-off
  permits don't double-count — an inflated counter deadlocks at 3)
- tests: isolation pins rewritten (stage failure now resolves zeros),
  single-flight/registry tails updated, per-job usage counts, and a new
  describe pinning the fan-out: 3 extracting + 5 queued, parts 1..8,
  source-wide abort, per-job usage on failure, extract peak == 3,
  audit peak == 1
- docs/architecture.md: serialized-audit paragraph rewritten for
  per-chunk jobs; two-front-doors wording aligned
2026-09-29 00:34:21 +03:00
amosblomqvist 3e9a85d60f per-session models: every session owns its model + thinking level from birth
The reported bug (2026-09-28): every chat ran glm-5.3-flash regardless of
any pick, while composer and settings UIs agreed with each other and lied
about the run. Root cause: a fossil default-settings record (settings.json
chat section, frozen pre-slots-wave) was stamped verbatim into every new
chat at birth by the new-session flow, and resolveConversationSpec prefers
the per-chat record over the strong slot forever.

Owner redesign — models are per session, always; the strong slot is the
new-session default:

- POST /api/chats stamps the newborn verbatim from the strong slot
  (model + thinking level); no client settings PUT on create anymore
- the composer control is dual-bound: in-session it reads/writes the
  session's own record (GET /api/chats/:id, PUT .../settings); on the
  draft surfaces it binds the strong slot (picking there = changing the
  default)
- thinking levels follow the identical mechanics (confirmed by owner)
- the zombie default-settings system dies whole: useDefaultSettings,
  /api/settings/defaults, src/store/default-settings.ts, the chat
  section of UserSettings (sanitizer strips fossil copies), the legacy
  default-settings.json fold, and DEFAULT_SETTINGS_FILE
- catalogDefaultsFor collapses to the strong slot alone
- thinking display for a legacy unstamped session resolves exactly like
  the engine (session ?? strong ?? model ladder)

Tests: new chat-birth-stamp suite pins the stamp, the unconfigured boot,
the unset level, and the copy-not-live-view regression; defaults pins
rewritten strong-only; legacy-migration suite deleted.
2026-09-28 23:30:11 +03:00
amosblomqvist f3003d357a Wave D: native AgentEvent wire + client session rewrite (SDK purged)
Server: ui-stream.ts is verbatim pi AgentEvent passthrough (SSE data
lines + [DONE]); only app-level events are the loop-level error and
turn-settled terminal; onError carries raw provider text (owner ruling).

Client: useChat + the AI-SDK transport replaced by components/session-core.ts
(SessionModel: 409 identity-retry, optimistic c_ appends, absorbed()
adoption, needsContinuation, shared consume for POST + reattach) and the
session.ts hook; the golden-master grouping canon moved client-side into
core/view.ts (projectView); scanners read the native record (phaseAt3Lh,
danglingQuestionOf, graph toolParts, plan-blocks, runner, extraction).

Coverage: the native message is the unit (fork 3) with the call+result
inseparability bond rule; view-line-less messages (markers, error
results) are invisible to coverage and never checkpointed.

Deletes: chat-transport, projection, ui-message, their tests (ported to
tests/session-core.test.ts), and the ai + @ai-sdk/react dependencies —
the Vercel SDK is fully out of the tree. Wire pins in chat-flow rewritten
onto the AgentEvent dialect (eventsOf parser).

Gates: tsc clean, 994/994 tests, eslint clean, build green; live smoke
on PORT=3004 (verbatim wire → native record).
2026-09-26 01:52:47 +03:00
amosblomqvist 7ab47d4fcb Wave B: the native message record — input events, projection, adapter death
The chat record is now pi-native agent history (LhMessage[]), the AI SDK's
UIMessage transcript contract is gone from the server, and every client turn
arrives as ONE explicit input event.

Core (new):
- src/core/native-messages.ts — the LhMessage taxonomy (user/assistant/
  toolResult + lhMarker customs: verdict, probe-handoff, subagent-steer,
  subagent-abort, plan-block), constructors, and toLlmMessages — the single
  model-facing seam (convertToLlm): strips ids, flattens markers to user
  text, drops UI-only plan-blocks, synthesizes INTERRUPTED results for
  dangling aborted calls, never throws.
- src/core/projection.ts — projectNative: the one-way native → UIMessage
  projection (consecutive assistant+toolResult runs group into one
  assistant view message; toolResults attach backward by toolCallId;
  markers project to data parts; plan-blocks splice after the last
  successful plan_init). Every existing consumer reads through it.
- src/core/input-events.ts — the ChatInput union (text/answer/verdict/
  continue), parseChatInput validation, and inputAppends (answer targets
  the dangling question call or 400s; verdict = optional note user
  message + native verdict marker).

Server:
- chat-handler.ts — the turn root takes {chatId, input}; write-ahead
  persists input appends + claimed steers before the turn; claimSteers
  coalesces the whole queue with the handler's steerDelivered filter as
  the dedupe seam; teacher-vs-prober branch on hasProbeDone.
- turn-service.ts — persistTurn stamps ids, drops clean question
  placeholder toolResults (the record ends at the unanswered call),
  emits plan-block markers idempotently (plan-blocks.ts rewrite), and
  persists abort partials.
- ui-stream.ts — PiTurnSpec takes the native base; the wire dialect is
  unchanged. jobs.ts steers are native markers (steer_<jobId> dedup ids);
  waitForIdle reads danglingQuestionOf. teacher-agent.ts wires
  convertToLlm: toLlmMessages. record-adapter.ts DELETED wholesale.
- GET /api/chats/[id] serves messages: projectNative(record.messages).

Client:
- chat-transport.ts — the 409 retry is IDENTITY (re-POST the same event
  body after the turn settles — the event was never persisted
  pre-acceptance; the merge logic is gone). The transport consumes the
  stashed input event via prepareSendMessagesRequest and throws loudly on
  a missing stash.
- chat.tsx — every send call site stashes exactly one ChatInput event
  (text, answer, verdict, continue) before invoking the SDK send.

Tests: full native migration (chat-flow rewritten to input events and
native record pins; jobs/extraction/demos/graph/usage/plan-blocks/
probe-done/reattach-transport converted; record-adapter.test.ts deleted;
chat-transport tests pin the identity retry). 982/982, typecheck, lint,
build all green.
2026-09-26 00:37:46 +03:00
amosblomqvist e79f85ebe4 Prompt-override orphan trap: delete clears the switch, writes heal orphans
Contract-fix interlude (2026-09-26), two bugs from UI's prompts-overhaul
(6edc220):

(a) ORPHAN-400 TRAP: an ON switch whose override file dies orphaned the
record; every agent-section write then 400'd (ON-must-have-file) while
the disabled toggle couldn't reach the OFF escape hatch — a user stuck
unable to write settings. Verified live on 3004 before the fix.
Two-layer fix:
- DELETE /api/settings/prompts/[...segment] now clears the record's
  switch in the same operation (switch-first order: a crash between the
  two leaves the valid off+file state, never an orphan). Response gains
  switchCleared (additive). Overrules the 2026-09-19 switch-survives-
  delete note — the delete route and the record agree by construction.
- PUT /api/settings/[section] agent writes HEAL an orphaned ON to OFF
  (key dropped) inside the write — never a 400 (defense in depth: disk
  wipes outside the app can still orphan). A valid ON with its file
  stays valid; the invariant: a user can never get stuck unable to
  write settings, and the record never contradicts file reality for
  long. Heal logs at info.

(b) TEST WIPES REAL DATA: tests/prompt-overrides.test.ts wiped
data/prompts/ (REAL user override files) in after() and its empty-list
case — every npm test killed actual overrides and orphaned their ON
switches (feeding trap a). Isolated via the established LH_DATA_DIR
seam (mkdtemp + env set before the first config.ts-loading import; all
app-module imports dynamic). after() now snapshots the REAL repo
data/prompts before/after and fails the suite if it changed; the seam is
pinned (writes land under the temp dir).

Pins: delete-clears-switch (incl. idempotent no-op), the exact repro
(disk-level wipe → mirror-the-orphan + tool toggle → 200, healed key,
rest of the write landed), vanished-fragment ON heals, valid ON stays.
Live: the 400-forever write now lands 200, heal info-logged, record
clean.

Gates: 967/967 (+5), typecheck, lint, build green.
2026-09-25 19:07:12 +03:00
amosblomqvist 7cfd8b5bc7 Background turn runner: every turn is a server-side process, clients are views
Owner ruling 2026-09-25. runPiTurn executes a turn decoupled from any HTTP
response: an unbounded memory-only event log (finite by construction; the
30-min liveness TTL bounds the pathological case) plus a subscriber set.
The POST /api/chat stream and the new GET /api/chats/[id]/stream are both
subscribers — identical SSE dialect; late subscribers get FULL replay
(starting at the start chunk) then live until finish + [DONE]. A view
cancelling (reload, navigate) now detaches ONLY — never aborts the turn.
The remote kill is the abort channel: POST /api/activity/abort gains
kind: "turn" (id = chatId; not_running 409 like jobs); handle.abort sets
sawAbort + agent.abort(), every subscriber closes cleanly, the partial
persists through the usual settle (onFinish runs once, then endTurn,
then unregisterTurn LAST so a view that finds nothing finds the record
current). No running turn on reattach: the cheap terminal — one
{"type":"turn-settled"} event + [DONE] (settledTurnResponse).

The live smoke caught what the unit suite had masked: the agent_end
close() had been left behind in the restructure — ReferenceError every
turn, steered into the error path; fixed and pinned (clean turns must
never emit an error chunk; finishReason stop present).
2026-09-25 17:56:12 +03:00
amosblomqvist e4049e4bf8 Key-presence refusal at every gate: a pinned model is not enough
Owner-diagnosed: the turn gate checked only model coordinates, so a
pinned slot whose provider key was deleted passed the gate and died
mid-stream masked to 'An error occurred.' The key is now gated where
the model is — by what would RUN (env-first keyValue truth, so an
env-shadowed key passes).

- core/settings.ts: the provider_key_missing envelope family (code,
  provider, slots, human line naming the provider from AUTH_PROVIDER_KEYS
  labels) + ProviderKeyMissingError + isProviderKeyRefusal matcher.
- chat-handler turn gate: refuses 503 before beginTurn when the resolved
  conversation's provider key is gone (slots: [strong]).
- Job runner: same check via an injected dep (agent layer cannot read
  the key store) — a keyless engine refuses the spawn through the
  existing failure-shape convention, no phantom run.
- Extraction pipeline: refuseIfStagesUnready refuses BEFORE any batch
  (first offender: extractor before auditor); the trigger skips info-
  level, /api/graph/import surfaces the 503 envelope per file.
- Tests: 8 new pins (turn gate both ways, runner refusal, pipeline
  first-offender + pass-through, envelope factory, matcher); the three
  turn-driving suites seed OPENROUTER_API_KEY like any real setup.
- Doc fix: stale EXTRACTION.chunkTokens comment in the manual-extract
  route (left over from eed4b4e).

Known residual (pre-existing, flagged for master): the manual extract
route logs BOTH config refusals as console.error (fire-and-forget path);
the coverage ledger keeps everything uncovered either way.
2026-09-24 22:51:12 +03:00
amosblomqvist 3327309e8b The all-sessions coverage GET dies: the per-session block is the coverage surface
Extraction-UX close-out interlude (master dispatch 2026-09-24, off UI's
ff1ec7a confirmation): nothing client-side consumes GET /api/graph/coverage
— the sidebar block and the chat-bar ring both read GET /api/chats/[id]'s
coverage block. The dead route (app/api/graph/coverage/route.ts) is
deleted; the manual-extract front door (POST /api/graph/coverage/extract)
stays by design.

No server helper was orphaned — the route was composition-only over
shared seams (chatStore, extractionStatuses, stores, coverageOf). The
sessionCoverage doc + the chats/[id] route comment now point at the
per-session block as THE coverage surface.

Pin (tests/coverage.test.ts): the route file's absence (prompt-overrides'
structural-pin precedent) + the Extract POST's survival — a merge cannot
resurrect the dead surface silently.
2026-09-24 15:29:01 +03:00
amosblomqvist 2e9fd01c0c Extraction-UX data layer: the chunk budget becomes a graph setting; the chat view gets a per-session coverage read
Owner-locked extraction-UX redesign (dispatch 2026-09-24), part 1 (data
layer; the UI chunk follows).

THRESHOLD -> SETTING
- config.ts's EXTRACTION.chunkTokens dies; the default moves into
  FACTORY_SETTINGS.graph.extractionChunkTokens (4096, the old constant's
  value). GraphSettings gains the optional field, Required at resolution
  via the standard ?? FACTORY pattern — pre-redesign records resolve the
  default cleanly.
- FLOOR 500 (the implementer's call, documented on the field): every run
  pays the graph-carrying prompt overhead out of the write channel's
  money — a smaller budget fires penny-batches that cost more per
  extracted token than the content is worth. The sanitizer CLAMPS up
  (bd68486 clamp-never-reject discipline: fractions floor, junk drops);
  the settings PUT REJECTS below-floor with the rationale in the message.
  No ceiling: a large budget just delays extraction — the user's stated
  cost/thoroughness tradeoff, not a misconfiguration to police.
- The trigger (turn-service.triggerExtraction) takes the budget as a
  parameter — chat-handler passes effective.graph.extractionChunkTokens,
  the fresh per-turn read, same as every other gate.
- The pipeline's BATCH side threads through the same number:
  ExtractionPipelineDeps gains an OPTIONAL getChunkTokens getter with the
  factory default (Teacher's 24 test fixtures stay untouched; the two
  call-site lines inside extraction.ts are the only cross-band edits,
  flagged). chat-handler + backfill/import-notes wire the live
  effectiveSettings — trigger and batches share one budget, always.

PER-SESSION COVERAGE READ
- sessionCoverage(chatId) (server/chats.ts): the chat view's ring-meter
  block — totalMessages/coveredCount/uncoveredCount/coveredTokens/
  uncoveredTokens (the same coverageOf derivation /api/graph/coverage
  serves, scoped to one chat), running (chat-kind extraction registered
  for this id), and lastFoldedAt (the chat's newest coverage row's fold
  — a cheap single pass; null before any fold). GET /api/chats/[id]
  carries it as coverage: {...}; unknown chat -> null (the route 404s
  first).
- /api/graph/coverage (the all-sessions list) compiles untouched — the
  per-session read is a scoping of it, not a supersession; the UI's
  sidebar/ring chunk decides which surface it consumes.

Tests: factory/sanitizer pins (clamp-up, fraction-floor, junk-drop),
resolution pins (absent -> 4096, stored 2000 wins), settings-body pins
(valid/absent/fraction/below-floor 400/junk 400), sessionCoverage pins
(derived split, newest-fold scoping, cross-session isolation, running via
the registry, null on unknown chat). 893/893.
2026-09-24 15:12:24 +03:00
amosblomqvist 2d133f1133 Single-node deletion: full death by node id (node interlude, owner-locked)
The remedy for undeletable nodes (preserve-graph orphans from session
deletion; notes-import nodes with no chat behind them): DELETE
/api/graph/nodes/[id], full-death semantics — the user pointing at the
node IS the intent; references from other sessions do NOT preserve it.

- core/graph-deletion.ts deleteNodeTraces: the node's every line, every
  edge touching it, every quiz/observation event naming it, its mapping
  lines, and its resting position — one synchronous atomic rewriteLedgers
  call; only the ledgers that changed rewrite (mtimes stay honest). Blast
  radius is exactly the rows naming the node id; notes source records stay
  (deleting a proposition is not deleting the file). Derived data
  self-heals on read: the birth cluster shrinks below MIN_CLUSTER_SIZE and
  stops rendering.
- nodeDeletionImpact (GET /api/graph/nodes/[id]): the confirm dialog's
  pre-flight facts — edge count, quiz/observation history, referencing
  sessions. The snapshot's edges + nodeSessions cover part of this, but
  the raw event history was nowhere on the client — hence the read.
- Guards (server/graph.ts deleteGraphNode), decided + documented: NO
  turn/jobs gates (they never write graph ledgers; the orchestration is
  synchronous end to end, nothing interleaves). THE EXTRACTION GATE: 409
  while ANY pipeline run is registered — a mid-flight run holds
  pre-deletion findings and could re-reference the dying node. The run is
  never aborted (its spend belongs to another session — outside this
  deletion's blast radius; the UI retries once the dock drains. A run
  started after the deletion is legitimate re-learning: fresh ledgers,
  fresh node id, and mergeFindings' existingIds guard ignores linkTo ids
  that no longer exist — no dangling edges, no resurrection.

Live-verified on dev data: impact read, full-death stats, honest 404 on
repeat, and the file cluster fell 3→2 and unrendered itself.

859/859 tests, typecheck/lint/build green.
EOF
)
2026-09-23 16:28:24 +03:00
amosblomqvist c86430e8bf Wave 7 chunk B: remove the embeddings machinery — the ladder collapses
The owner's wave-7 ruling: embeddings die entirely (git history is the
stash); clustering is birth-session derived (chunk A, 950ddfd), labels
are session names, and the prober's graph tools replace injection.

- settings: GraphSettings collapses to { enabled } — embedding,
  embeddingModel, clustering, labeling all die. Migration story: stale
  record keys are UNREAD and dropped by the sanitizer's reconstruction;
  the next graph-section PUT (replace-whole) persists the collapsed
  shape. The settings-panel GraphSection renders one master row.
- deleted: src/server/embeddings.ts (adapters, resolveEmbeddingPortFor,
  embedNewNodes, embeddingPort, embeddingModelCatalog), the
  GET /api/models/embeddings route, scripts/embed-graph.ts, tests for
  all three, and the /api/models/embeddings picker surfaces in the UI.
- prices: EMBEDDING_PRICES overlay + callCost fallback die. cost-summary:
  the embedding stage fold dies — back to three stages (extraction/
  audit/naming) per the owner ruling; historical stages.embedding spend
  in old run records is simply not displayed.
- ports: EmbeddingPort/EmbeddingResult/EmbeddingTask die. The embeddings
  LEDGER surface (readEmbeddings/writeEmbeddings + EmbeddingsLedger +
  EMBEDDINGS_FILE + graph-deletion's vector pruning) STAYS — inert data,
  kept only so graph-context.ts compiles until Teacher's pass removes
  it; the seam dies in the follow-up interlude.
- chat-handler (approved cross-lane unwiring, wave-6 precedent): the
  extraction pipeline's embedder becomes a documented no-op (nothing
  embedded, nothing spent, ledger untouched) until Teacher's pass
  removes the phase; the graph-context injection dispatch gate, the
  graphContext deps wiring, the setContextRespawn seam, and
  turn-service's triggerGraphContext are gone — injection is dead
  code pending Teacher's file deletions.
- scripts: backfill-graph/import-notes pass the same no-op embedder;
  replay-layout derives clusters from birthSessionPartition (the same
  partition the real settle uses) instead of the embeddings ledger.
- settings-body: parseGraphSettingsBody reads the master switch only —
  old-shape keys in a body are ignored, not rejected; bad shapes 400.
- next.config: the @huggingface tracing-exclusion guards die with the
  last reference.

Cross-lane flags: extraction.ts/graph-context.ts keep stale embed
comments + the embed phase until Teacher's full pass (their parallel
interlude owns those files — zero overlap kept by design);
semantic-clusters.ts + its test stay one more round (extraction.test.ts,
Teacher's file, still imports it — they delete it with the namer cases);
ChatSettings.graphContextTokens + the graph-context-control UI stay
until Teacher+UI.

Gates: tsc clean, eslint clean, 885/885 tests, build compiles.
Live-verified on the port-3004 dev tree: PUT /api/settings/graph with
dead keys collapses on write, bad shapes 400, GET reads {enabled: true},
/api/models/embeddings 404s.
2026-09-23 15:02:55 +03:00
amosblomqvist 0a4a9a11de BYOK provider embeddings (wave 6 chunk B): kill local transformers
Replace the @huggingface/transformers local embedder (~316MB ONNX cache)
with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id"
slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per
call from settings + the auth store so key rotations land immediately.

- src/server/embeddings.ts rewritten: slug grammar, injected-fetch
  adapters (OpenRouter input_type, Gemini taskType batching, OpenAI
  index-ordered rows), honest resolution (null = skip-with-info, set
  model with missing key = honest error), embedNewNodes with model-change
  re-embed wipe + ghost pruning + usage steps, federated catalog
  (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse
  path 404s; OpenAI text-embedding- filter; Gemini embedContent filter).
- GET /api/models/embeddings: federated catalog, empty when no keys.
- Extraction tail records stages.embedding usage into the run's money
  channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the
  sync-generated block), callCost falls back to it; cost-summary folds
  the fourth stage; graph-context injection resolves its embedder per
  run, fails honestly when unconfigured, records query tokens.
- Settings: GraphSettings.embeddingModel (null factory), section parser
  validates slug shape (id may contain colons — OpenRouter :free
  suffixes, live catalog fact).
- Removal: @huggingface/transformers uninstalled, PATHS.modelCache and
  onnx tracing entries deleted, @huggingface tracing-excludes guard
  added, stale sharp/libvips comments updated to historical.

Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter +
3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5
model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects
bad slugs with 400.

Gates: typecheck, lint, 895/895 tests, build.
2026-09-23 03:53:57 +03:00
amosblomqvist a6338511d3 feat(graph): two-mode session deletion — the first destructive ledger operation
DELETE /api/chats/[id]?mode=preserve-graph|delete-nodes (mode required):
- preserve-graph: only extraction bookkeeping dies (coverage rows,
  legacy checkpoints) — graph ledgers byte-untouched
- delete-nodes: the chat's ledger traces die. Node survival = surviving
  provenance point / mapping line (notes imports carry empty
  provenance) / surviving event. The creator's aliases + provenance are
  stripped from a surviving node's create line (owner ruling
  2026-09-23: name/statement/createdAt are the shared identity);
  orphan nodes, their edges, events, mappings, positions, embedding
  vectors and cluster members die; cluster labels stay (immutable)

Gates: 404 unknown, 409 turn_in_flight / jobs_running (with jobIds),
503 extraction_winding_down after a 10s abort-and-poll; the purge is
one sync block (ledgers → jobs → chat record last). rewriteLedgers is
atomic (tmp + rename). graphRevision (max mtime over graph data files)
rides the activity feed + the response as the change stamp.

Anti-resurrection: the chat pipeline re-checks chat liveness at each
merge — a run that slips past the wind-down aborts with nothing merged
or checkpointed; its partial spend dies with the chat.
2026-09-23 02:41:04 +03:00
amosblomqvist 9387418864 feat(models): clamp-on-write for pinned thinking levels + DEFAULT_THINKING_ORDER in the core barrel
Fix 1 — clamp, never reject (owner ruling, interlude 2026-09-22): a
model swap with a pinned thinking level must JUST WORK. The owner hit
the write-side 400 live (Claude Fable 5.1/xhigh → GLM 5.3 flash:
"z-ai/glm-5.3-flash does not support thinking level xhigh"). The
supported-level verdicts in validateChatSettings and
validateModelsSettings are DEAD — the WRITE clamps the pin to the new
model's nearest supported level (pi's clampThinkingLevel, the same
translation piEngineForSpec applies at runtime) and stores the clamped
result; the server is the truth — the record shows the clamped level
after write.

New pure helpers in model-catalog.ts, applied by ALL THREE write
routes (section, defaults, per-chat — one verdict seam, so every
surface that used the old 400 gets the clamp uniformly):
- clampModelsLevels: every slot's level clamps against the SLOT'S OWN
  model (the parser already requires a model on each slot write).
- clampChatLevel: the body's own model, else the record's resolved
  strong slot (the chain's fallback); no model anywhere → rides
  unchecked (runtime clamps at engine build; the turn gate refuses
  the model-less chat anyway).

Fix 2 — DEFAULT_THINKING_ORDER moves from config.ts to core/settings
(core barrel, AUTH_KEYS precedent — one source of truth, no drift).
The UI imports it to auto-populate a thinking level on FRESH model
picks; the ordering has no other client role. Legacy unset records
keep the runtime defaultThinkingLevel fallback at engine build
untouched. Import sites updated: agent/models.ts, tests.

Verified live on 3004: the owner's exact case stores max (not 400);
a bare chat-level write clamps to low against the strong slot's glm;
unknown provider/model still 400s with the provider list.

Tests: the level-verdict suites became clamp suites (the owner's case
pinned verbatim: xhigh→max, minimal→low, off→low, supported
pass-through, level-less slots ride); the machine-aware
expectBareLevelVerdict chain is retired — clampChatLevel takes the
strong slot as an explicit param, so the tests are pure again.

Gates: typecheck, lint, 839/839, build.
2026-09-22 21:44:43 +03:00
amosblomqvist bfa906e2f8 feat(models): BYOK all the way down — factory pins deleted, per-need refusal gates
Owner decision 2026-09-22: 'it doesn't make sense to have any default
model configured anywhere; the application is bring your own keys.'

config.ts: MODELS deleted; ModelRole becomes an explicit 9-role union;
DEFAULT_THINKING_ORDER (medium, low, high, minimal, xhigh, max) bakes
the unset-level default — first entry the model supports; explicitly
set levels keep pi's native nearest-clamp.

core/settings-resolution: the factory layer dissolves — the chain is
role override → slot → NOTHING. resolveSettings(record) takes no
factory. Empty chains resolve to null/undefined; a thinkingLevel-only
slot is inert (stored, never run). resolveConversationSpec: chat.model
override over the strong slot, null when both empty. resolveRoleSpec
spans all nine roles for the job/extraction engines.

Refusal envelope (core/settings.ts): modelsNotConfigured(missing) —
{error, code: 'models_not_configured', missing: [slot tiers]} +
ModelsNotConfiguredError with .envelope. Per-need gates, each honest:
- chat-handler: universal turn gate (new + continuing turns) returns
  503 before beginTurn — no liveness, no write-ahead persist, no graph
  trigger before the gate fires.
- runner.runJob: unresolvable engine lands the job as error with the
  envelope wording; no phantom loop, no result steer.
- extraction: refuseIfStagesUnconfigured throws before any stage
  (turn-service info-skips so coverage retries next turn; the import
  route surfaces per-file code+missing); labeler-less runs skip
  cluster naming with a console line and still merge.

Write side: slot entries must carry a model — level-only slot writes
are 400s (absent slot key = clear). piEngineForSpec resolves unset
thinking levels via DEFAULT_THINKING_ORDER; piEngineFor now throws
honestly; thinkingLevelFor/modelIdFor deleted; teacher-agent, pi-run,
visual-maker, pdf-reader all read engine.level.

model-catalog: defaults/slots/rolePins null when the chain is empty
(contracts nullable) — no phantom ids in payloads. Stores drop
MODEL_PINS entirely.

Tests: pin-leaners converted to explicit injection (per-chat model
pins on scripted chats — exactly what the setup flow writes; faux
engines for agent-construction tests). New coverage: the 503 gate
envelope, the runner refusal, the extraction refusal + labeler skip.

Gates: typecheck, lint, 815/815, build.
2026-09-22 16:59:16 +03:00
amosblomqvist f9e3a4c43d feat(models): strong/cheap/vision model slots + multi-provider engine federation
Model slots wave (2026-09-20), chunk B:

- settings: new `models` section {strong, cheap, vision} slots (each
  field independent, same shape as a role override); ROLE_SLOT maps all
  nine roles to tiers (strong=prober+teacher, cheap=researcher/
  demoBuilder/extractor/auditor/labeler, vision=visualMaker+pdfReader);
  resolution chain per role: override → slot → factory pin.
- config: ModelSpec gains provider (all pins openrouter, the BYOK
  default); MODEL_PINS in stores.ts now spans all nine roles (the strong
  slot's factory rep IS the prober pin).
- engine: piModels() registers EVERY builtin provider catalog;
  piEngineForSpec(resolved spec) is the new engine factory (pi-ai
  resolves per-provider keys from env itself); piEngineFor stays as the
  config-pin fallback. chat-handler resolves the conversation chain
  (chat override → strong slot → pin) and injects engineFor into the job
  runner, closing the old gap where per-role overrides never reached
  job engines.
- /api/models: models federate ONLY keyed providers (providers surface
  with keyPresent, env-wins, fresh per call); slots (resolved tier pins)
  join rolePins; catalogDefaultsFor inherits the strong slot; the models
  settings section gets the same catalog gate as chat.
- pricing: PRICES/CONTEXT_LIMITS uniformly keyed provider:id
  (regenerated from ALL pi-ai catalogs, 1354 keys); usage stamps
  (TurnUsage, JobUsage, extraction stages) switch to provider:id —
  owner decision: no legacy compat, old stamps render unpriced.
2026-09-22 00:14:56 +03:00
amosblomqvist f1373d5de5 feat(search): Tavily + Brave behind a search-provider port; Google CSE retires
Search port + model slots wave, chunk A (owner design, 2026-09-20):
- core/search.ts: SearchHit + SearchProvider port + SearchStatus —
  pure, browser-safe; adapters arrive as injected dependencies.
- server/search.ts: Tavily + Brave adapters (wire shapes VERIFIED
  against docs.tavily.com's OpenAPI + Brave's dashboard docs — Tavily
  auth is the Bearer header, not the legacy body api_key; Brave is
  X-Subscription-Token) + engine resolution per owner decisions: a key
  ALONE activates its engine (no engine choice needed), explicit
  agent.search.engine pins the preference (both keys: Tavily first),
  a pinned engine with a missing key is honestly unavailable — no
  silent fallback, no default engine.
- Curated keys catalog: GOOGLE_SEARCH_API_KEY/GOOGLE_CSE_ID retire
  (CSE closed to new customers, dies Jan 2027 — stored values become
  preserved unknown vars); TAVILY_API_KEY (tvly- shape) + BRAVE_API_KEY
  (length guard) join. UI keys-section TOOL_IDS now derives from the
  catalog — cannot drift.
- GET /api/search: {available, engine, reason} — the honest status for
  the settings gray-out; researcher-disable semantics ride the same
  signal (Teacher stream rewires the tool onto the port).
- agent.search settings field (schema + sanitizer + strict parser);
  web.ts's WebSearchResult is now the core SearchHit (re-exported).
2026-09-22 00:02:07 +03:00
amosblomqvist ec38f91a2b feat(keys): free-form model-provider keys — pi-ai provider table on /api/keys
Provider-keys interlude (owner spec, 2026-09-19): any pi-compatible
provider's key is manageable, not just the curated set.

GET /api/keys now also returns providers: {id, envVar, label}[] — every
pi-ai provider that takes a plain api-key env var (33 today), derived
from pi-ai's own provider→env-var table. No-drift mechanism: the table
is generated from pi-ai (builtinProviders + findEnvKeys over a
truthy-for-every-var proxy env) and pinned by
tests/auth-provider-keys.test.ts, which re-derives the truth at test
time and asserts EXACT equality — a pi-ai upgrade fails the gate loudly.

Owner-spec exclusions: github-copilot (OAuth), amazon-bedrock (AWS),
google-vertex (ADC), azure-openai-responses (base URL), the cloudflare
pair; anthropic keeps ANTHROPIC_API_KEY only. OPENROUTER_API_KEY
graduates from the curated catalog to the provider table (dedupe — the
UI pins it always-visible); curated stays Deepgram + the Google search
pair. Statuses: curated + present vars + unknown file vars — absent
provider keys never spam rows; the dropdown is discovery.

Write side: PUT/DELETE accept provider env vars (generic truncation
guard — trim, no whitespace, ≥12; family checks stay on the curated
three). Present-in-env provider vars surface as status rows so
env-wins/shadow semantics stay honest.
2026-09-19 22:52:40 +03:00
amosblomqvist a3b6a495d3 Merge branch 'feat/edge-glyph' 2026-09-19 22:39:03 +03:00
amosblomqvist 3e263a18cc Graph pipeline cost metrics: capture naming spend, fold all channels, cost pill + panel
- Cluster namer bills the money channel now: per-step usage + model
  captured via onStep, persisted as NamingRunUsage records to
  data/graph/naming-runs.jsonl (idempotent per startedAt; failed/aborted
  runs persist too — money spent either way). callUsage promoted to
  extraction.ts module scope for reuse.
- core/cost-summary.ts: the pure fold behind the pill — chat runs,
  notes-source runs, and naming runs into stage totals (extraction /
  audit / naming) + newest-first recent runs. Dollars INFERRED read-time
  via prices.ts, never stored; unpriced models excluded-but-counted
  (render '—', never a silent zero); tokens always sum.
- GET /api/graph/costs (src/server graphCosts composition, thin route).
- Graph view: bottom-left cost pill (cumulative dollars headline) +
  expandable breakdown panel (stage rows with tokens secondary, recent
  runs with time/title/merged/cost); fetched on mount, refetched when a
  pipeline run finishes; merged into one bottom-left stack with the
  running-pipeline pills.
- Tests: pure-fold suite, naming capture wiring (full importNotes ->
  naming path), store idempotency + strict reads.
2026-09-19 20:55:35 +03:00
amosblomqvist cbaf6df454 feat(keys): first-class BYOK key management — store, catalog, /api/keys
The auth/keys wave (2026-09-19): the server now owns the flat auth.json
map directly (LH_AUTH_FILE from main.cjs; repo-root auth.json in dev,
gitignored). The store reflects itself into the child's env at boot with
fill-gap precedence — real env (incl. .env.local in dev) always wins —
so every consumer keeps its plain process.env reads and saved keys take
effect immediately, no restart; main.cjs's spawn-time injection retired
(an env copy would shadow every settings-page change).

Curated 4-key catalog (OpenRouter, Deepgram, Google search pair) with
shape-only validation (no live pings — owner decision), 400s on bad
pastes, last-4+length masking (plaintext never leaves the store),
unknown file vars preserved + surfaced. Routes: GET /api/keys,
PUT/DELETE /api/keys/:id. Tests: catalog/store/composition, with the
env-sanitizing prologue (a dev shell may export real keys; tests use
throwaway values only).
2026-09-19 20:54:16 +03:00
amosblomqvist cb4fe25c89 wave 2 (platform stream): settings storage & plumbing foundation (P2)
- src/core/settings.ts: versioned DELTA record schema + sanitizer (unknown keys preserved)
  — general/chat/agent{models,tools,prompts}/appearance/graph sections
- src/core/settings-resolution.ts: pure resolveSettings (factory defaults → user delta),
  extraction/graph-context channel predicates
- src/store/settings.ts + prompt-overrides.ts: SettingsStore/PromptOverrideStore adapters
  wired through stores.ts (effectiveSettings() fresh per call); lazy migration of
  default-settings.json into record.chat
- app/api/settings/** routes (record, section bodies, prompt overrides per segment)
- chat-handler: graph gates at trigger call-sites + toolToggles injection seam
- config.ts: SETTINGS_FILE + PROMPTS_OVERRIDE_DIR paths

587/587 tests; typecheck/lint/build clean
2026-09-19 18:05:30 +03:00
amosblomqvist 487f5d4cea wave 1 (platform stream): background-observability accuracy + owner-driven follow-ups
- observability: liveness-gated stale-job recovery, extraction run registry on globalThis with
  explicit 'embedding' phase, /api/jobs force-dynamic, client refresh nudges at every seam
- headless: providerSafePng canonicalizer (multi-IDAT Chrome PNGs → single-IDAT) fixing qwen 400s
- composer: Enter routes through pure enterAction — accepts a pending plan verdict (owner decision)
- markdown chrome: session-scoped mermaid overlay + md-blocks component set + pan-zoom math,
  corner-copy (clipboard grant in electron main), math/table/code block fixes, heading weight 900

543/543 tests; typecheck/lint/build clean
2026-09-19 13:16:34 +03:00
amosblomqvist bd3086194f feat: no baked-in default model — the new-session controls ARE the default settings page
Owner reframe (2026-09-18): the model + thinking + context controls on
the home graph view and /chat/new are not a draft overlay on a baked-in
default — they ARE the settings page for the default model, and the
user's last choice IS the default. BYOK first boot has no keys and
possibly no OpenRouter, so there is NO default to show: the controls
render 'no model' ('—') until the first pick.

The flash was the deeper symptom: the surfaces rendered from client
state that started empty and fetched truth afterward, so every visit
flashed the config pin (GLM/low — a dev artifact) for a few frames.
Fix at the root: both pages are now force-dynamic server components
that read the record (readDefaultSettings → initialSettings) and hand
it to the client — the first frame shows truth. Dynamic routes aren't
cached client-side (staleTimes.dynamic = 0), so every navigation
re-reads the record; the useDefaultSettings hook keeps only save-side
state (optimistic set + PUT). The GET-on-mount and its touch-guard are
gone.

ModelControl gains noDefaultModel (the default surfaces only): unset
settings render no model/level, never the catalog pin; the thinking
menu no-ops until a model exists. Chat surfaces keep the catalog
fallback — the engine runs the config pin when a session record has no
override, so the control keeps showing what will actually run.

Renamed draft→default throughout: data/default-settings.json,
GET/PUT /api/settings/defaults, useDefaultSettings. New sessions
still inherit the default on creation exactly as before.

Live-verified: first-boot 'no model' with zero flash, picker +
Shift+Tab cycle, values held across session→/chat/new→home roundtrips
and a reload (10×100ms samples, never the old default), zero console
errors.
2026-09-18 09:37:20 +03:00
amosblomqvist a94a108e74 feat: draft settings persistence — shared model/thinking/context record for the new-session surfaces
Both new-session pages (the home graph view's topic dock and /chat/new)
held their model + thinking + knowledge-context choices in plain
useState(undefined) — every navigation reset them to config defaults.

Now both surfaces read/write ONE server-side record
(data/draft-settings.json via GET/PUT /api/settings/draft, the
render-settings pattern): whatever the learner last left the controls on
is what every later visit shows. useDraftSettings loads on mount with a
touch-guard (a slow GET never clobbers a fast choice), saves
optimistically. Session creation still carries the choice into the new
chat exactly as before; per-session settings are untouched.

The PUT validation is the exact per-chat contract — parseSettingsBody
extracted verbatim from app/api/chats/[id]/settings and shared by both
routes (unknown model/level → 400 with the supported list, never a
silent clamp). 400s on garbage, {} = the clear-overrides write.

Live-verified: Cmd+K+Enter model pick, Shift+Tab thinking cycle, context
slider — all held across home → session → home → /chat/new, and carried
into a created session's record.
2026-09-18 09:25:40 +03:00
amosblomqvist 573e369db5 fix: dictation cloud path — bearer subprotocol, access_token grant field
Two live-verified Deepgram facts from the first real mic-test round:
- /v1/auth/grant returns { access_token }, not { token } — the route now
  parses the right field.
- Minted JWTs authenticate over the WebSocket via the "bearer"
  subprotocol; ["token", jwt] is rejected at the handshake (token is
  for raw API keys only, as in the pi dictate extension).

Also documents the env-precedence gotcha in docs/research/stt.md: a
DEEPGRAM_API_KEY exported in the shell (e.g. .zshrc) silently beats
.env.local in next dev and the desktop authEnv() seam.
2026-09-17 19:44:35 +03:00