812ea234198bdf5a696e975e854bd4be03dd1181
79
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 amosblomqvist
|
487f5d4cea |
wave 1 (platform stream): background-observability accuracy + owner-driven follow-ups
- observability: liveness-gated stale-job recovery, extraction run registry on globalThis with explicit 'embedding' phase, /api/jobs force-dynamic, client refresh nudges at every seam - headless: providerSafePng canonicalizer (multi-IDAT Chrome PNGs → single-IDAT) fixing qwen 400s - composer: Enter routes through pure enterAction — accepts a pending plan verdict (owner decision) - markdown chrome: session-scoped mermaid overlay + md-blocks component set + pan-zoom math, corner-copy (clipboard grant in electron main), math/table/code block fixes, heading weight 900 543/543 tests; typecheck/lint/build clean |
||
|
amosblomqvist
|
bd3086194f |
feat: no baked-in default model — the new-session controls ARE the default settings page
Owner reframe (2026-09-18): the model + thinking + context controls on
the home graph view and /chat/new are not a draft overlay on a baked-in
default — they ARE the settings page for the default model, and the
user's last choice IS the default. BYOK first boot has no keys and
possibly no OpenRouter, so there is NO default to show: the controls
render 'no model' ('—') until the first pick.
The flash was the deeper symptom: the surfaces rendered from client
state that started empty and fetched truth afterward, so every visit
flashed the config pin (GLM/low — a dev artifact) for a few frames.
Fix at the root: both pages are now force-dynamic server components
that read the record (readDefaultSettings → initialSettings) and hand
it to the client — the first frame shows truth. Dynamic routes aren't
cached client-side (staleTimes.dynamic = 0), so every navigation
re-reads the record; the useDefaultSettings hook keeps only save-side
state (optimistic set + PUT). The GET-on-mount and its touch-guard are
gone.
ModelControl gains noDefaultModel (the default surfaces only): unset
settings render no model/level, never the catalog pin; the thinking
menu no-ops until a model exists. Chat surfaces keep the catalog
fallback — the engine runs the config pin when a session record has no
override, so the control keeps showing what will actually run.
Renamed draft→default throughout: data/default-settings.json,
GET/PUT /api/settings/defaults, useDefaultSettings. New sessions
still inherit the default on creation exactly as before.
Live-verified: first-boot 'no model' with zero flash, picker +
Shift+Tab cycle, values held across session→/chat/new→home roundtrips
and a reload (10×100ms samples, never the old default), zero console
errors.
|
||
|
amosblomqvist
|
a94a108e74 |
feat: draft settings persistence — shared model/thinking/context record for the new-session surfaces
Both new-session pages (the home graph view's topic dock and /chat/new)
held their model + thinking + knowledge-context choices in plain
useState(undefined) — every navigation reset them to config defaults.
Now both surfaces read/write ONE server-side record
(data/draft-settings.json via GET/PUT /api/settings/draft, the
render-settings pattern): whatever the learner last left the controls on
is what every later visit shows. useDraftSettings loads on mount with a
touch-guard (a slow GET never clobbers a fast choice), saves
optimistically. Session creation still carries the choice into the new
chat exactly as before; per-session settings are untouched.
The PUT validation is the exact per-chat contract — parseSettingsBody
extracted verbatim from app/api/chats/[id]/settings and shared by both
routes (unknown model/level → 400 with the supported list, never a
silent clamp). 400s on garbage, {} = the clear-overrides write.
Live-verified: Cmd+K+Enter model pick, Shift+Tab thinking cycle, context
slider — all held across home → session → home → /chat/new, and carried
into a created session's record.
|
||
|
amosblomqvist
|
573e369db5 |
fix: dictation cloud path — bearer subprotocol, access_token grant field
Two live-verified Deepgram facts from the first real mic-test round:
- /v1/auth/grant returns { access_token }, not { token } — the route now
parses the right field.
- Minted JWTs authenticate over the WebSocket via the "bearer"
subprotocol; ["token", jwt] is rejected at the handshake (token is
for raw API keys only, as in the pi dictate extension).
Also documents the env-precedence gotcha in docs/research/stt.md: a
DEEPGRAM_API_KEY exported in the shell (e.g. .zshrc) silently beats
.env.local in next dev and the desktop authEnv() seam.
|
||
|
amosblomqvist
|
e3bdbc2f2c |
feat: voice dictation (STT wave 1 — Deepgram cloud)
Two-band architecture from docs/research/stt.md: renderer captures mic audio in an AudioWorklet (linear16 16kHz mono, ~128ms frames, per-quantum RMS for the meter) and streams it directly to Deepgram over WebSocket — audio never loops through our server. The real key stays server-side; /api/stt/token mints a short-lived grant token (TTL 300s) per run. Patterns ported from the owner's pi dictate extension (tuned in real use): generation counter guarding every async handler, the latest-RMS-never-resets meter (-50..-10dB window, 60ms tick), mic-first stop + CloseStream + 3s safety net, endpointing=300 with the utterance_end_ms=1000 backstop. UI (owner placement): mic button in the stats bar's middle slot on all three surfaces (home graph-view, chat, draft); while recording the meter unfurls to fill the row's free width with the interim ghost at its flank. Esc ends dictation from the composer. Token mint comes before getUserMedia so a keyless app fails fast without a TCC prompt; one automatic reconnect on abnormal socket close, errors surface inline. Live-smoked the full error path in the dev shell (bar renders on all three pages, 403 grant failure shows inline, mic untouched). The full audio path needs a Deepgram key with Member permission to mint tokens. |
||
|
amosblomqvist
|
769a3da186 |
feat: session rename + graph chrome polish (popovers dismiss, no-shift no-match, coverage reseat)
Rename (owner ask): clicking the header title opens edit mode with the whole title selected — typing replaces, arrows move inside. Enter or a click away commits (PUT /api/chats/:id); Escape and an empty commit discard. renameChat joins the ChatStore port; like saveSettings it is NOT conversation activity — updatedAt stays put so sidebar order never churns on a rename. The sidebar patch is instant: the header dispatches lh:chat-renamed and SessionsSidebar folds it into its snapshot (the 2.5s poll is only a backstop). Sidebar right-click rename is DELIBERATELY deferred: it needs a real context-menu system, which gets its own pass later (owner decision, 2026-09-17). Graph chrome: - The extraction-coverage button moves between search and import; the gear keeps the far-right corner. Coverage still renders on an EMPTY graph (that is exactly when it matters). - The no-match search note no longer shifts its siblings: the chrome row is items-start (every button is 44px tall, so top alignment pins the row while the search container grows downward). - Popovers dismiss properly: pointerdown anywhere outside the control's root (button + panel) closes it, and so does Escape. Two panels can never be open at once — the sibling chrome buttons are outside the open root, so pointerdown closes the first before the click opens the second. New shared hook components/popover-dismiss.ts, wired into both the settings gear and the coverage control. Gates: typecheck, eslint, 498/498 (+2 rename store tests), build. Live-verified: select-all/focus/Enter/Escape/empty/blur round-trip, updatedAt untouched, sidebar instant patch; chrome order + frozen positions under a no-match query; popover close via canvas, search input, Esc, mutual exclusion, toggle intact. |
||
|
amosblomqvist
|
5c380de669 |
Merge lh-ui refinement wave (feat/ui-polish @ 8163437) into main
11 commits on fbeca36: minimal-color rounds 1-2 (phase color retired, the braille spinner is the one working indicator; the chip is label-only), typography rounds 1-5 (three-font rule enforced — CMU titles, Source Sans session titles, JetBrains mono; snapped type ladder in globals.css with the xxs rung merged into xs; lh-input caret at the font-metric floor; session titles at prose size, message counts and active-row weight dropped, ISO dates), header rounds 6/6b/6c/6d (60px top bar aligned to the sidebar LH band; the open-sidebar toggle matches the close glyph at size-4 both ways; SidebarToggle always-mounted with a lockstep animated slot — no more one-frame title jump; title at text-3xl). Semantic change folded in: `busy` in the chats list now means exactly "a teach turn is in flight" — background jobs no longer spin the session row (the sidebar's Background agents section is their channel). Merge fallout fixed in the merge: components/graph-coverage.tsx (main's coverage panel) held the last three text-xxs uses; with the rung deleted they silently lost their size — mechanically swapped to text-xs, matching lh-ui's own swap in graph-context-marker.tsx. No file overlap with main's waves since fbeca36 (extraction coverage, lh-graph merge, world-scale bake) — clean auto-merge. Gates: typecheck, eslint, 496/496 tests, build. Live-verified on the dev server: 60px header with 32px CMU title, label-only chip, snapped ladder active (--lh-prose-size 16px, no --text-xxs rule anywhere), sidebar rows with ISO dates, animated sidebar toggle with no header jump. |
||
|
amosblomqvist
|
2678f04e7a |
feat: extraction coverage — message-granular ledger + token-batched runs (d132373 vision)
THE MESSAGE IS THE UNIT OF COVERAGE (vision-extraction-coverage.md): a per-chat append-only ledger (coverage.jsonl) records which messages the pipeline has extracted, by content fingerprint over exactly what the extraction view sees (one shared line builder — core/coverage.ts); the uncovered remainder is DERIVED at read time, never dual-tracked. A quiz answer merging into the asking message re-opens only that message; plan-block data-part attach never re-opens anything. - The trigger (turn-service) replaces every-3-quiz-answers with uncovered-tokens >= EXTRACTION.chunkTokens (4096) — quiz-less sessions extract too, content size is the honest signal. - runExtraction is now chunked: token-budgeted batches of the oldest uncovered consecutive messages (a tool call + result never split), each merged batch checkpointing its messages' coverage atomically — an abort or error loses at most the in-flight batch; merged batches stay covered. - Observation refs are batch-tagged (obs:<batchTag>:<sourceId>): re-runs of the same batch dedupe; a re-opened message's fresh understanding lands as a new event. Fixes the old obs:p1 position-collision. - New API: GET /api/graph/coverage (exact per-session derived coverage + running flags, most-uncovered first) and POST /api/graph/coverage/extract (manual run for one session, 409 while in flight). - Graph view chrome: the coverage button opens a panel with every session's covered/uncovered tokens and a per-session Extract button — the manual answer to aborted runs, errors, and dormant tails. Renders even when the graph is empty: an empty graph with uncovered sessions is exactly when it matters. Polls only while open. - Notes imports keep the whole-file checkpoint path (coverage undefined). - Tests: pure-core coverage suite (11), chunked-pipeline suite (3: multi-batch + no-op, abort-keeps-earlier-batches + remainder re-run, re-open alone + data-part invisibility), panel render suite (5); pendingQuizAnswers and its tests deleted as dead code. 486/486. Live-smoked on real GLM: manual extract of "teach me modal realism" ran one batch, wrote 4 coverage entries, panel flipped to "1.3k covered · complete" with the button disabled, graph refetched (nodes 11), usage persisted with real stats. |
||
|
amosblomqvist
|
326222f53c |
Minimal-color round 1 (owner): phase color retired — the braille spinner is the one working indicator
Nothing exists purely for decoration: phase color-coding is retired app-wide (phases only advance — every session ends Teaching — so phase-as-signal was decoration with a one-time function), and what state genuinely deserved — 'a turn is in flight' — is now carried by MOTION alone, in the terminal's own glyph language. - BrailleSpinner (components/braille-spinner.tsx): the canonical cli-spinner sequence (⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏) as exact dot bitmaps in a 2×4 braille cell, 80ms/frame (800ms cycle). Pure CSS: one shared keyframes, frames staggered via animation-delay; no JS timer, no re-render per frame — cheaper than the old breathing dot. Static contexts (SSR, prefers-reduced-motion) show frame 0 — a legible glyph, not a blank. - Sidebar rows: spinner ONLY while a teach turn is in flight; idle and question-pending render NOTHING (a resting state has no scan value — the meta line's 'Waiting for your answer' text carries it). The three-state phase dot (pulse/solid/hollow) and PHASE_FILL/RING maps are gone. - Top bar PhaseChip: label-only pill; the old dot slot becomes the spinner's home while a turn is in flight (busy prop wired from ChatView's busy = localStream || turnInFlight). Transcript PhaseDividers drop phase colors for monochrome rules + labels. - API narrowing (app/api/chats/route.ts + ChatListItem contract): busy = isTurnRunning ONLY — background jobs no longer count (the sidebar's Background agents section is their channel; a per-row spinner would double-signal them). - globals.css: --phase-* tokens + lh-dot-breathe removed (dead after the sweep — no remaining users). Live-verified: idle/pending rows render no indicator, /chat/new chip is label-only, 10 CSSAnimations at 800ms with correct 80ms stagger, static frame 0 under reduced motion. Gates: typecheck, lint, 470/470, build. Lane crossings to flag at merge: app/api/chats/route.ts + the ChatListItem contract comment (src/core) — the busy-semantics narrowing; components/chat.tsx one-line busy prop. |
||
|
amosblomqvist
|
1d45d77303 |
feat: graph context injection — the session's prior knowledge, steered in at session start (vision-context-injection.md)
One shot per chat, fired at the first turn's onFinish: the session's topic (its only typed message) is embedded as an EmbeddingGemma retrieval query; the semantically closest graph propositions are selected under the session's token budget (pure budget, no similarity threshold — the decided doctrine) and delivered as a marker-only steer through the wake pipeline, queuing behind pending questions like every sub-agent result. - contracts: ChatSettings.graphContextTokens (default 1024, max 4096, step 128, 0 = off); GraphContextData + graphContextText; the graph-context JobKind + buildGraphContextSteer (id steer_ctx_<chatId>) - core/graph-context.ts: brute-force cosine top-K, greedy chars/4 fill under budget, most-recent UnderstandingLevel fold, count cap 64, parse-don't-cast at the marker seam - server: embedQuery (role-asymmetric query format, same warm singleton — vector-space parity by construction); the runner (startGraphContext- Injection one-shot gates; runGraphContextJob validates ledger parity); wake branches (chatIdOf/jobIdOf, finalizeJob done, recoverStaleJobs re-deliver or contextRespawn); abortJob kills context jobs SILENTLY (nothing was promised in the lesson); turn-service trigger seam - adapter: data-graph-context flattens to the canonical block (malformed parts have no model face) - UI: the budget slider (home dock + /chat/new draft, the session-data strip); the quiet SystemLine "~537 · 10 propositions"; dock kind "Knowledge context"; model-picker PUT now spreads settings (a model pick can no longer clear the sibling override) Live-smoked end to end on the real embedder + real GLM: seeded ledger (10 propositions), session on "modal logic: possible worlds and the necessity box" — job in the dock, queued behind the prober's question, delivered on the answer, SystemLine in the transcript, wake turn reads the block and continues the probe. Gates: tsc, lint, 459/459, build. |
||
|
amosblomqvist
|
05712037b2 |
Live physics drag + splat cluster regions
The renderer grows a live d3 sim (createGraphSimulation, shared with the server settle): pointerdown on a node pins fx/fy and pointer capture (preventDefault kills the compat mousedown, so d3-zoom never pans); the first CLICK_SLOP-exceeding move reheats to alphaTarget 0.3; release un-pins and the sim anneals (release-and-relax) — alpha < 0.005 settles it, a 250ms debounced onCommit PUTs the rest state. Verified live: 123fps during a 320ms drag (regions rebuilding every 3rd frame), 85-tick settle, then zero idle frames. A slop-free press stays a click (select, no reheat, no commit). PUT /api/graph/positions sanitizes (finite coords, known ids) and if the graph grew mid-drag runs ONE incremental settle with the client's arrangement as anchors (50-id PUT test: anchored drift 0.15). Cluster regions swap convex hulls for a low-res world-space SPLAT bitmap: per-member radial-gradient discs (alpha 0.2) + intra-cluster edges as round-cap strokes (Minkowski dilation — concave shapes finally render), upscaled with bilinear smoothing (the upscale IS the blur, no ctx.filter: Safari). One tiny blit per frame replaces 32 clusters x 4 offset Path2D polygons re-rasterized at DPR 2 (the old GPU coil-whine path); during the live sim the bitmap rebuilds every 3rd frame. cluster-geometry.ts retires convexHull/offsetPolygon/traceRoundedPath (tests follow; clusterHue stays). Region labels stay (centroid + clusterHue, inverse-gated on zoom). |
||
|
amosblomqvist
|
ecadededd9 |
World-space rendering: geometry cached once, zoom pays only rasterization
Fixes the zoom frame-rate collapse and rebuilds draw() around a single camera transform, per the user's Obsidian-style sizing proposal. Bug fix first: the eased zoom leaked a queued rAF callback per frame -- advanceZoom applies each step through d3-zoom, whose zoom event called markDirty() from inside the tick loop, scheduling an extra draw while the loop scheduled its own. Draws multiplied per displayed frame; sustained wheel ran at ~20fps against a 120Hz display. markDirty now skips scheduling while inTick (verified: 62 frames/500ms during a 28x zoom burst, matching pan). World-space model: every draw call now runs in graph coordinates under one setTransform(k). Sizes are defined as screen-px-at-fit and converted once per geometry build (/ kFit), so nodes, node labels, arrowheads and the hover halo scale with the graph as you zoom (0.25x painted area at half zoom, exactly k^2) while edge widths and hull borders stay viewer-relative hairlines (/ k). Cluster hull geometry (convex hull + three offset fills + solid boundary -- dashes dropped for now) is built as cached Path2Ds per data/visibility/settings change; camera moves do zero geometry math, making zoom per-frame cost identical to pan. Label wrapping runs once per build, not per frame. Region labels stay viewer-relative (18px at any zoom) as UI chrome. Picking radius scales with the world-sized discs. Glide feel: ZOOM_EASE_MS 120 -> 45, convergence 0.1% -> 0.2%; a notch now settles in ~260ms instead of ~750ms. Resize still paints synchronously. Also fixes barrel-law lint violations from the settings wave (import from @/src/core and @/src/server, not deep module paths). |
||
|
amosblomqvist
|
8e2d2118a6 |
Graph view refinements: server-persisted settings, label size, eased zoom, resize-blank fix
Five refinements to the hand-rolled renderer, all user-directed: 1. Settings are SERVER-persisted (data/render-settings.json via new GET/PUT /api/graph/render-settings) - one source of truth for every browser, replacing localStorage. The client loads once after mount, saves debounced on user changes, and never lets a slow GET clobber a faster user (or mounting PUT defaults over the server copy). The shape/defaults/sanitizer moved to src/core/render-settings.ts - pure, shared by client, server store, and tests; every disk/network value passes through sanitizeRenderSettings. 2. New "Label size" control (default 12px - the previous hardcoded size). Font, wrap width, and line height all scale; verified live: +76k px with the label gate open. 3. "Labels until zoom" slider reworked: log-scaled over 0.0625-4 so the useful band (0.25-0.5 camera-ratio units) sits at 33%/50% of the track instead of crammed at the far left of the old linear 0.25-4 slider. Display units are normalized: the default 0.5 reads as "1x", 0.25 as "0.5x". The underlying labelZoom semantics are unchanged, so existing values survive. 4. Smooth wheel zoom: d3-zoom instant per-notch jumps (x1.2, bang) are replaced by an eased glide - wheel events accumulate a cursor-anchored target scale and each tick eases the camera toward it (exponential, 120ms constant, frame-rate independent). Pan/pinch stay d3-owned; a pan cancels a glide in progress. Verified live: 8-frame glide. 5. Resize blanking fixed (graph vanished for the duration of a window drag, reappearing ~300ms after). Cause: every resize event reallocated the canvas backing store - which CLEARS it - and the redraw was deferred to rAF, which live resizes starve via relayout, so the canvas sat wiped for the whole drag. Fix: the ResizeObserver callback now redraws SYNCHRONOUSLY after reallocating (verified via a probe RO firing in the same delivery batch: canvas fully painted). kFit (the label gate 1x reference) is also recomputed on resize from the now-cached world bbox - the camera itself still persists. tests/render-settings.test.ts: 10 tests pinning the sanitizer (garbage/ partial/clamps, legacy no-labelSize records) and the slider mapping (default dead-center at 500, monotonic, exact round-trips, display units). 402/402 green; typecheck, lint, build pass. |
||
|
amosblomqvist
|
543cf33789 |
Notes front door: format-agnostic import into the graph pipeline
The same extractor→auditor→merger pipeline now runs over raw personal
notes in arbitrary formats (vision §1, DECIDED 2026-09-13):
- NotesSource records (data/graph/sources.json, deterministic id from
path hash → idempotent re-imports) play the chatId role in mappings,
checkpoints, and observation events; usage lands on the source record
(appendSourceUsage, idempotent per startedAt, kept across upserts).
- mergeFindings is chat-store-free: callers pass MergeSource
{messages, foldedAt} — sessions pass the chat record, notes pass
messages:null (no quiz-event rebuild; a deterministic rebuild needs
structured quiz tool parts).
- Second extractor prompt (NOTES_EXTRACTOR_INSTRUCTIONS) for general-
purpose extraction; understanding judges the AUTHOR's grasp. Auditor
wording input-agnostic. extract/audit share one generic runSubmitAgent.
- graphSnapshot.fileSources (srcId→filePath); notes sources render as
static path entries in the detail card (FileTextIcon, never a link).
- One shared pipeline instance (exported from chat-handler): same
single-flight registry, live status chips (label = file path), usage
accounting, and true abort for both front doors.
- Transport: POST /api/graph/import (multipart, 3-wide worker pool) +
Import notes… button in the graph view; scripts/import-notes.ts for
batch runs (concurrency 4).
- real-data-extraction/ gitignored (raw import sources never committed).
355/355 tests; typecheck, lint, build green.
|
||
|
amosblomqvist
|
de0c286311 |
feat: model + thinking control — the stats bar's picker, on pi-ai's catalog
The model-control wave (user-decided design, 2026-09-12): ONE controller
for the conversation agents (prober/teacher are functionally one agent);
background sub-agents stay config-pinned. pi-ai owns the registry — the
shipped 366-model OpenRouter catalog (cost, context, modality,
thinkingLevelMap), getSupportedThinkingLevels/clampThinkingLevel for the
per-model thinking truth, and streamSimple's automatic clamp. No second
registry to drift out of sync.
Seams:
- config.ts: ReasoningEffort retired for pi's ThinkingLevel
(off|minimal|low|medium|high|xhigh|max). GLM roles go honest at "low" —
pi's catalog says GLM-5.3-flash accepts exactly low|high|max, and the old
"minimal" was silently clamped to low by streamSimple since the swap;
nothing changes behaviorally. visualMaker → qwen/qwen3.8-flash (fixes
the dead qwen/qwen3.8-max slug — the catalog only ships -0902; also the
user's cost pick, 0.15/0.47 per-1M).
- src/agent/models.ts: piEngineFor(role, settings?) — provider+id resolved
from the catalog, thinking level from settings ?? config, CLAMPED to the
model's supported list (a settings change between models can never send
an invalid level). PiEngine gains `level` (observability/tests).
- ChatRecord.settings {model?: {provider, id}, thinkingLevel?} — written
by PUT /api/chats/[id]/settings (replace-whole, no updatedAt churn),
read by chat-handler at turn composition: agents are stateless per turn,
so the change is next-turn-effective by construction. The turns log
still records the ACTUAL billed model (the honest witness).
- GET /api/models: the configured-provider catalog as client-safe DTOs
(core/contracts/models.ts) + config defaults; a second provider joins
the picker by registering it in piModels() — nothing else changes.
- PUT settings validation gate (src/server/model-catalog.ts): unknown
provider/model or unsupported level → 400 carrying the supported list,
never a silent clamp.
- scripts/sync-pricing.ts: regenerated PRICES/CONTEXT_LIMITS from pi-ai's
shipped openrouter.json (offline; same source as the engine + picker).
All 366 models priced; glm's stale 0.15/0.5 → pi's 0.075/0.25, limit
1310720 → 1048576 — cost is inferred at read time, so history re-prices.
- UI (components/model-picker.tsx): stats-bar right side becomes model +
thinking buttons; menus open ABOVE; fuzzy search over "provider id
name" (core/fuzzy.ts — pi's /model pattern); model rows show name/id/
price/context; the thinking menu lists exactly the chosen model's
supported levels (GLM: low|high|max — no dead options, no "off").
Model switch drops an incompatible pinned level (falls back to config,
which clamps honestly) rather than carrying a 400.
Tests (350/350, +15): catalog projection + defaults, the settings gate,
pi thinking truth pinned against upgrades (glm low|high|max; minimal→low;
off→low — GLM cannot disable thinking), the engine factory's
override/fallback/clamp/throw matrix, fuzzy ranking, store round-trip,
and the re-priced folds (usage, stats bar, activity dock).
Live smoke (disposable driver, deleted): catalog route 366 + defaults;
glm+minimal → 400 + supported; turn 1 billed qwen/qwen3.8-flash under
the override; mid-session switch flipped turn 2 to glm; clear-overrides
restored the config pin. ALL CHECKS PASS on real OpenRouter.
|
||
|
amosblomqvist
|
fde46c28fd | Activity abort route: POST /api/activity/abort — uniform END control (jobs + extraction) | ||
|
amosblomqvist
|
37c4638feb |
Merge feat/graph-rendering: wave 4 — extraction usage capture (live + persisted), true abort
# Conflicts: # src/server/observability.ts |
||
|
amosblomqvist
|
9174cde55e |
Extraction observability: usage capture, live status payload, true abort (Wave 4)
Extraction is real spend and background work; the dock needs to see and stop it. User decisions: persist all cost data (chat record), per-stage usage granularity, and TRUE cancellation for abort. USAGE CAPTURE (live + persisted): - extraction.ts: per-call onStepFinish captures each billed step's usage (SDK LanguageModelUsage -> CallUsage: input/output/cache read+write/reasoning) into the registry's live per-stage usage (extractor/auditor; the merger is deterministic — no model call). - /api/graph/status runs now carry usage (model + steps per stage); cost/context folds are the consumer's (core/prices.ts — nothing stores dollars). - ChatRecord.extractions: append-only per-run usage log (mirrors the turns log): startedAt key, at, durationMs, merge stats when the merger ran, stages. Persisted on success AND failure — a failed run still spent money. Idempotent per startedAt; the list payload (ChatSummary) never leaks it. - EmissionStats moved to contracts/graph.ts (its contract home); core/graph.ts re-exports for existing importers. - Verified end-to-end against the real pipeline (backfill one chat): per-step token counts, reasoning tokens, and the auditor's growing prompt-cache reads captured and persisted. TRUE ABORT (the dock's abort button, user decision): - RunningExtraction owns one AbortController per run, shared across phases; abortExtraction(chatId) fires it and returns whether a run was aborted (idempotent no-op otherwise). The pending model call receives the signal — the provider cancels the request, spend stops. throwIfAborted() at phase boundaries ends runs aborted in sync gaps. - Abort is a deliberate stop, not a failure: same ledger discipline (nothing merged, nothing checkpointed, partial usage persisted) but the run reports zero emissions instead of throwing. Exported via src/server/observability.ts for master's /api/activity/abort route. - Tests: mid-run abort (hanging mock that rejects on the signal), live-accumulation + persisted-record assertions, store-seam idempotency, and the summary-payload leak guard. |
||
|
amosblomqvist
|
f8ba5b1628 |
Activity feed: one dock-shaped GET /api/activity (cross-chat processes)
The dock is visible in every view, so it needs every running process — sub-agent jobs across ALL chats (not just the open one) plus the global extraction runs — in one poll. /api/jobs stays chat-scoped and untouched: it is the transcript machinery's poll and carries turnInFlight for the re-attach seam (chat-transport), the seam we spent three live rounds stabilizing; making it polymorphic would fork its response shape by mode and put regression risk on exactly the wrong path. The dock is a different consumer with a different shape (cross-chat, running-only, cost-bearing), served by a thin route over server/observability.ts: - activityFeed(): running jobs from every chat (title-joined, oldest-first on last activity) + extraction runs + busyChats (chats with a teacher turn in flight — the biggest spender; data made available, rendering is a dock decision). - GET /api/activity — the dock's single poll everywhere. - Cost math stays client-side (core/prices.ts folds); raw usage rides the records and the transport stays thin. Decided with the user over the polymorphic-/api/jobs and separate-all-jobs-endpoint alternatives. |
||
|
amosblomqvist
|
83a00800e9 |
Observability contracts: usage accounting, plan blocks, price table (Phase 0)
The 2026-09 observability wave's type-level foundation, landed before the streams so nothing renders against guessed shapes: - contracts/chat.ts: CallUsage + TurnUsage (per-model-call token accounting, one entry per teacher/wake turn keyed by messageId) and ChatRecord.turns — a DELIBERATELY parallel append-only usage log, not per-message metadata: the client re-POSTs its message list every turn and its copies never carry server-stamped fields, so anything inside messages gets clobbered by the next persist (the wave-3 merge-rule lesson). - contracts/chat.ts: PlanBlockData — the persisted data-plan-block part (full DAG on plan_init, one-level neighborhood on plan updates, snapshot semantics so history renders what was true then). - contracts/jobs.ts: JobUsage + JobRecord.usage — a sub-agent run's accumulated accounting, written by the runner. - core/ports.ts + store/chats.ts: ChatStore.appendTurnUsage — idempotent per messageId (abort replays / wake retries append once); listChats no longer leaks turns into the list payload. - core/prices.ts: PRICES + CONTEXT_LIMITS (generated from OpenRouter's published pricing via scripts/sync-pricing.ts) + pure folds: callCost, turnCost, sessionCost, contextWindow, jobCost. Unknown model or absent fields render as absent, never zero. - server/observability.ts: barrel-approved surface for the transport layer (fixes the import-law violation in the graph status route that slipped through the wave-3 merge verification). Capture wiring (agent onStepFinish -> persist seam, plan-block emission) lands next as master seam work; the streams start against these shapes. |
||
|
amosblomqvist
|
0c12d53fca |
Background pipeline status: registry + /api/graph/status + graph-view chips
The background extraction pipeline is invisible on the home surface — essential observability per the user's directive: the graph view must show what is currently running (extracting / auditing / merging) and for which chat. - extraction.ts: module-level registry of in-flight runs (chatId, phase, startedAt), updated at each phase transition in runExtraction and cleared in finally (done or failed). extractionStatuses() is the read; one process, one registry. - GET /api/graph/status: thin transport over the registry, chat titles joined from the chat store. - graph-view: polls every 2s and shows one chip per running pipeline (spinner + phase + title + elapsed seconds, bottom-left); when a run finishes (some -> none) the graph refetches, so merges land live. Status is cosmetic — poll errors never surface. - Unit test drives a mocked pipeline run and observes the registry mid-flight: 'extracting' during the extractor's generate, 'auditing' during the auditor's, empty after. |
||
|
amosblomqvist
|
25df0edd5b |
Merge feat/ui-polish: three-phase session model, plan graph, layout restructure
chat.tsx woven by hand: UI's session-surface layout (transcript column + right sidebar as full-height siblings, h-10 header with PhaseChip) wrapped in sub-agents' JobsContext.Provider; the message map carries both the steer ternary (steers bypass the Message wrapper — the model-facing glue text never renders) and UI's PhaseDivider emission (steers are verified phase-boundary no-ops). Right sidebar = sub-agents' fleet panel column (SubAgentsPanel above TreePanel, xl-only) carried onto UI's skeleton. contracts.ts auto-merged (ChatSummary.phase/pendingQuestion + ChatListItem, additive above the async-protocol section). Smoke-verified on the merged tree: steer renders bare as its artifact badge with no learner bubble, no rendered glue text, and no spurious phase divider at the steer boundary. |
||
|
amosblomqvist
|
2d7b004fda |
fix: live sub-agent observability — dispatch cards, steer artifacts, sidebar panel
The pi-style ack protocol (previous commit) left the client describing the
old world: artifact cards read the ack string as a result ("could not be
displayed"), the only job poller rendered on input-available parts that no
longer exist (so wake turns landed invisibly until re-navigation), steers
rendered as learner bubbles with no SVG anywhere, and server-persisted
assistant messages carried empty SDK ids (duplicate React keys).
- persistTurn (turn-service): fill empty message ids at the canonical write
— server-only messages (last turn's, wake turns') now key uniquely;
client-authored ids untouched.
- chat.tsx: ONE central /api/jobs poll (fast while jobs run, slow discovery
cadence, off for job-less sessions) drives every job surface, re-syncs
once when a job settles (the record already holds the steer + wake
response at flip time), and adopts server-side wake turns via the same
turnInFlight machinery as returning mid-turn. Steers branch to the
artifact card — never a bubble. Composer no longer gates on jobs.
- /api/jobs now reports turnInFlight alongside the job list.
- dispatch-card: the tool-call box — live activity by toolCallId while
running, quiet "delivered"/"failed" one-liner after; the ack is visible
observability, the artifact lives in the steer.
- steer-card: renders the delivered artifact by kind (research brief /
sanitized SVG / sandboxed demo iframe) from the data-subagent-artifact
part; result renderers extracted and exported from the artifact cards.
- sub-agents-panel: right-sidebar section above the plan tree listing
running sub-agents (kind, elapsed, live activity) — vanishes when idle.
- tests: dispatch/steer/panel render tests re-target the new protocol;
persistTurn id-hygiene suite. 229/229 green, build green.
Out-of-lane touches for master review: chat.tsx + components/** (cleared
this wave), app/api/jobs/route.ts, src/server/turn-service.ts, contracts
protocol doc §2. Live-verified on :3003 (full loop, SSR DOM checked).
|
||
|
Amos
|
1c3d288d08 |
Three-phase session status: derived summaries for the picker
- core/phases.ts: Phase3 (probe/plan/teach) + pure derivations — phaseOfTree (point-in-time, from the stored plan tree: no nodes → probe, no occupied node → plan, occupied → teach) and phaseAt3 (monotonic transcript fold for the dividers: failed plan_init never marks a seam; phase only advances). - contracts.ts: ChatSummary gains record-derived phase + pendingQuestion; ChatListItem adds liveness-merged busy. - store/chats.ts listChats computes the record-derived fields (tree + pending question scan). - api/chats route merges busy = turn running || running background job. Lane crossing granted by master: contracts/store/api (schema arbitration). Data-only inference: pre-plan planning reads as probe until the plan lands. |
||
|
Amos
|
a7ca33548b |
Sessions keep running across navigation: turn liveness + write-ahead persist
The bug: answering a quiz and navigating away mid-turn, then returning, re-prompted the same question. Root causes: (1) the answer was durable only at onFinish — every mid-turn view read the stale pre-answer record; (2) the server had no notion of a turn in flight; (3) remount treated the stale record as current, with no re-attach path. Worse, the stale dock let the quiz be answered again, racing two turns into a last-writer-wins saveMessages clobber. Fix (four pieces): - Write-ahead persist in chat-handler: the request body IS the client's canonical state — saved to disk BEFORE the stream starts. The answer is durable the instant the request arrives; crashes recover via the existing mount continuation. - Turn liveness registry (src/server/turn-liveness.ts): in-memory chatId → in-flight map with a TTL leak guard; beginTurn rejects concurrent turns (409) so one turn ever owns a chat. endTurn runs in onFinish AFTER persistTurn, so pollers that see not-in-flight always read a finished record; a turn-start failure clears the registry (no lockout). - GET /api/chats/[id] reports turnInFlight; the chat page passes initiallyInFlight to the client. - Client re-attach (components/chat.tsx): while in flight, busy shimmer + composer gated + mount continuation suppressed; poll /api/chats/[id] until the turn finishes, then syncFromServer adopts the completed transcript. Stop stays local-stream only (a server turn has no client request to abort). Verified by scripted-model tests: concurrent turn 409 + registry lifecycle, write-ahead durability mid-turn, failed-turn lockout guard. |
||
|
Amos
|
e92140f993 |
Shell/core import law: public API barrels + ESLint enforcement (stage 2)
The shell now imports through exactly two barrels, enforced by
import/no-restricted-paths (npm run lint):
components/** -> "@/src/core" (pure core API, browser-safe)
app/** -> "@/src/core" + "@/src/server" (server entries)
Zone violations are lint errors with architecture-pointer messages; the
pure-core modules (contracts, question-core, plan-tree, phases, pending,
math-delims, ui-message) may never import agent/server/store.
Supporting moves (direction: IO layers -> core):
- core/index.ts: the core public API barrel
- core/ui-message.ts: moved from src/agent (it was shell-facing all along)
- quiz/ask input zod schemas moved agent/tools -> core/question-core
(the question contract, alongside the grading logic)
- ChatRecord/ChatSummary moved store/chats -> core/contracts (single schema)
- GraphSnapshot/NodeStateClass moved core/graph -> core/contracts (the
/api/graph payload IS a contract; the pure renderer needs only these)
- src/server/{chats,graph,jobs}.ts + index.ts: thin server entries for the
transport layer; GET /api/graph's fold side-effect now lives behind
graphView() instead of inline in the route
Gates: tsc clean, 190/190 tests, eslint clean (law verified to fire on
store/agent/subpath violations), production build clean.
(Repo repair en route: a git object (app/chat/[id] tree) was missing from
the object store — reconstructed byte-exactly via mktree from its blob;
the broken node_modules that stalled the build was cleanly reinstalled.)
|
||
|
Amos
|
8728573253 | W4: graph store + emission + home graph view | ||
|
Amos
|
df16750560 |
W2: async job infra + researcher sub-agent
- src/store/jobs.ts: per-job JSON store in data/jobs (create/get/update/list, dedup by toolCallId) - src/agent/tools/jobs.ts: research tool — client-side (no execute), turn ends at the call; input per contracts ResearchToolInput - src/agent/tools/web.ts: server-side web_search (Google CSE) + web_fetch (Readability/Turndown/linkedom, Jina fallback) — errors return as strings, never thrown into the agent loop. Deps: @mozilla/readability, linkedom, turndown (+@types/turndown dev) - src/agent/subagents/researcher.ts: isolated ToolLoopAgent (web tools only, 12-step cap), researcher prompt ported from pi - src/agent/subagents/runner.ts: spawnPendingJobs hook + runResearchJob — fire-and-forget, activity line per tool call, fulfills the part in the LATEST chat record (output-available) before flipping job status; failure still fulfills the part so the teacher learns the outcome - src/server/chat-handler.ts: onFinish spawn hook after saveMessages - app/api/jobs/route.ts: GET ?chatId= polling endpoint - components/artifacts/job-card.tsx + registry entry: polling activity card (2s) → syncFromServer on finish; research-brief card (markdown + sources) - src/agent/prompts/20-jobs.md: research doctrine for the teacher - tests/jobs.test.ts: store CRUD, spawn hook via scripted mock teacher, runner completion/failure with fake model + stubbed fetch |
||
|
Amos
|
a72c902d30 |
v3: AI SDK pivot — pi removed, Next.js 16 + AI SDK 7 + AI Elements teacher app
The teaching agent no longer runs on pi. Stack: - ToolLoopAgent (ai@7) + OpenRouter (OPENROUTER_API_KEY), model via LH_MODEL, default anthropic/claude-sonnet-4.5 with reasoning tokens - quiz/ask_user_question = client-side tools (no server execute): the call lands as a typed tool part in input-available state; the card grades via src/core/question-core.ts; addToolOutput sends the verdict; the teacher's reaction continues the same assistant message - resume-mid-quiz is free: the pending tool part persists in the transcript - persistence: one JSON per chat (data/chats/), full UIMessage transcripts, saved onFinish (also on abort) - UI: AI Elements (owned copy-paste components, pruned to the five in use) + button-based QuizCard/AnsweredCard (ask answers never graded) - pi fully removed: TUI, extensions, Fastify+WS server, bridge, custom protocol all deleted (recoverable from git history) - tests: question-core units carry over; chat-flow integration drives handleChatRequest with MockLanguageModelV4 (48 tests, zero tokens); render smoke tests guard the cards (renderToStaticMarkup) Carried over unchanged: teacher system prompt, question-core, schema/ ledger/derive/extract pipeline, data model. |