487f5d4ceadabe2b2cdf30b0afbb84e1d50bb74b
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 amosblomqvist
|
487f5d4cea |
wave 1 (platform stream): background-observability accuracy + owner-driven follow-ups
- observability: liveness-gated stale-job recovery, extraction run registry on globalThis with explicit 'embedding' phase, /api/jobs force-dynamic, client refresh nudges at every seam - headless: providerSafePng canonicalizer (multi-IDAT Chrome PNGs → single-IDAT) fixing qwen 400s - composer: Enter routes through pure enterAction — accepts a pending plan verdict (owner decision) - markdown chrome: session-scoped mermaid overlay + md-blocks component set + pan-zoom math, corner-copy (clipboard grant in electron main), math/table/code block fixes, heading weight 900 543/543 tests; typecheck/lint/build clean |
||
|
amosblomqvist
|
597b3f43e1 |
fix: wake-turn steer delivered twice — claimSteers re-claimed the steer the pump's request already carried
The wake pump fires its turn with the steer INSIDE the request body but dequeues the queue entry only after the turn returns 200 — so the entry is still queued when the handler claims. The record doesn't carry the steer yet (write-ahead persist hasn't run), so claimSteers claimed it a SECOND time and the handler appended it: outgoing = [...body, ...claimed] held two copies of the same deterministic steer id, and the write-ahead persist saved that for the whole wake turn. The client's adoption poll rendered both -> React duplicate-key warning (steer_ctx_<chatId>). withMessageIds deduped only at onFinish, so the final record looked clean. Fix: claimSteers(chatId, incoming) — a queued steer whose id is already in the turn's incoming list is being delivered BY this very turn: finalized and resolved, never appended again. |
||
|
amosblomqvist
|
740293c361 |
feat: custom message types for system turns — wake and verdict messages
The custom-message scheme (user-decided 2026-09-12): system-authored
user-role messages are NOT human messages — they carry marker data parts
only, the adapter owns their model-facing wording, and the UI renders
them under control. Nothing fake is ever persisted as human speech.
The inventory (all migrated):
- probe→teacher handoff wake: was a text-bearing user message rendering
as a fake user bubble; now data-probe-handoff marker-only — the
adapter flattens it to HANDOFF_TEXT (core/contracts/chat.ts), the UI
renders a quiet 'Preparing your plan…' system line.
- plan accept/decline: was boilerplate text ('The plan is accepted…',
'I'd like the plan revised…') masquerading as learner speech — the
decline wasn't even marked; now data-verdict {accepted, at} on its own
message, adapter flattens to verdictText(), UI renders the quiet
verdict marker ('✓ Plan accepted' / '↻ Plan revision requested').
sendVerdict sends TWO REAL MESSAGES when the learner typed a note
(the note as a genuine user message, then the verdict) in ONE turn
via setMessages + sendMessage() — no 409 dance, no double teacher turn.
- sub-agent steers: the abort steer gains its data-subagent-aborted
marker part; the UI keys steer rendering off PARTS (artifact/aborted),
not metadata — metadata stays as pump mechanics only.
- record-adapter: systemTextOf() flattens marker-only messages; steers
keep their text (payload — briefs are content, not boilerplate).
phases.ts: plan→teach advances on data-verdict{accepted:true} only;
learner text alone can never sanction teaching (pinned). data-plan-
accepted retired, no back-compat (user-decided: wiped-data doctrine).
Pinned: record-adapter.test.ts (verdict/handoff flattening, note+verdict
order, steer payload kept, UI-only parts skipped), phases.test.ts
(verdict marker semantics, out-of-order, text-never-fakes), jobs.test.ts
(abort marker part), demos.test.tsx (badge renders off the part, not
metadata).
Live GLM smoke (disposable driver, deleted): probe→conclude→wake→plan→
decline-with-note (revised, still plan)→accept-with-note (teach) — ALL
CHECKS PASS, including marker-only handoff/verdict on the wire and zero
fake human speech in the transcript.
Gates: tsc, lint, build, 335/335.
|
||
|
amosblomqvist
|
2d717fd932 |
Swap the server engine to pi-ai/pi-agent-core (Band B+C of docs/pi-engine-swap.md)
The Vercel AI SDK survives only as Band D (useChat client + UIMessage
record types); @openrouter/ai-sdk-provider is uninstalled.
- src/agent/models.ts: PiEngine {model, streamFn} via createModels +
openrouterProvider subpath; thinkingLevelFor maps config effort none->off
- src/agent/lh-tool.ts: neutral LhTool (zod authoring, optional execute,
clientQuestion flag); toPiTool converts via z.toJSONSchema and re-runs
zod inside execute so refinements ride the error-result retry path
- src/server/pi/record-adapter.ts: UIMessage records -> pi Messages
(dangling tool calls get synthesized interrupted results)
- src/server/pi/ui-stream.ts: SSE bridge emitting AI-SDK-shaped chunks so
the useChat client parses unchanged (continuation id rule, question
tools never emit output, tool-input-error for bad authoring)
- teacher-agent.ts: stateless per-turn pi Agents; shouldStopAfterTurn
stops only when every question-tool call resolved cleanly (errors retry)
- chat-handler.ts: buildAgent(outgoing) factory seam (stateless doctrine);
assertValidUiMessages owns the retired SDK validation gate
- sub-agents via shared runSubagent (pi-run.ts); extraction stages stop
after a clean submit; visual-maker keeps its hand-rolled loop over
direct streamSimple calls; aborted model calls bill no usage
- tests: all model seams ported to pi-ai faux provider (scripted
FauxResponseSteps, agent factories); handleChatRequest second param
is now an agent factory
Gates: tsc, lint, build, 305/305. Live GLM smoke (disposable driver,
deleted): prober streaming + persistence, question seam, turn-2
continuation same message id, per-turn real token usage - ALL PASS.
Data wiped at the pivot (chats/graph/jobs, .gitkeeps kept).
|
||
|
amosblomqvist
|
5653c77a29 |
feat: probe/teach agent split — explicit phase markers, position-only plan map
Diagnosis (2026-09-11 user testing, minimal-effort GLM): sessions stuck on 'plan' forever. Two causes: (1) the three-phase view read the stored tree's currentNodeId, which nothing ever writes post author-once (2026-09-03) — a structural dead seam; (2) the teacher stochastically left quiz.nodeId untagged, and done-by-quiz-result semantics would have falsely completed nodes anyway. The redesign (docs/probe-teach-split.md) abolishes derived occupancy in favor of explicit events: - Two agents over one transcript: createProberAgent (quiz without a nodeId field, ask, probe_done) and createTeacherAgent (full toolset, plan_init onward). The chat-handler construction branch keys on a probe_done part in the transcript. MODELS.prober added. - probe_done: server-side seam tool — carries the learner assessment (strands + free-form edge read), refuses while a question is open or the probe is already concluded, and queues a handoff steer through the generalized wake pipeline (steers now dedupe by id, not jobId; job finalization only for job steers). The teacher auto-wakes and authors the plan; the learner sees the assessment via the probe-done card. - phases: probe ends at a concluded probe_done, plan ends at a successful plan_init — strictly sequential, no skipping, no legacy compatibility (data wiped at the pivot per doctrine). - Plan map = position, not completion: derivePlanStatuses reads only tagged quiz CALLS (state/outcome feed nothing); current = latest tagged node with consecutive duplicates collapsed, visited = earlier distinct stays (a revisit stays in both — current wins at render). PlanTree is structure-only; currentNodeId/doneNodeIds and the dead mutation helpers are gone. No fallback current. - Prompts split per agent (prompts/prober/, prompts/teacher/ over the trimmed shared base); graph-context injection removed entirely (shelved, docs/probe-teach-split.md §5); knowledge-graph context no longer reaches either agent. - Plan blocks are full-only snapshots at plan_init (neighborhood machinery retired; the persistent DAG window is the future UI wave); usage capture unchanged. Gates: tsc clean, 294/294 tests (phases/plan-tree/plan-blocks/question- tools rewritten to the new semantics; probe-done + handoff-wake tests added), lint, build. Live GLM smoke PASS: prober concluded with an honest assessment, teacher auto-woke, authored a 10-node plan with the block attached, then tagged checks on slope-instinct → secant derived current/ visited exactly as designed; grading honest both ways (driver off-by-one proved a wrong answer grades false). |
||
|
amosblomqvist
|
17ed37a471 |
feat: abort sub-agent jobs (real kill) + runner usage capture into JobRecord.usage
Wave 4 (agent-side observability + control), reconciled with the dock contract (master relay 2026-09): - abortJob in src/server/jobs.ts, exported through the server barrel for POST /api/activity/abort (master's thin route): fires the job's AbortController (real kill — the in-flight model call aborts, not just the next loop check), drops any queued result steer, flips the record to error "Aborted by learner" + activity "aborted" synchronously, and queues a one-line abort steer (buildAbortSteer) through the same wake pipeline so the teacher's promise doesn't hang in the lesson. Shares the result steer's deterministic id space, so abort + racing result can never both deliver (steerDelivered dedupe). - Abort registry (src/agent/subagents/aborts.ts) on globalThis, mirroring the wake pipeline's module-state pattern; runner registers/unregisters per run, so the route's module instance sees the loop's controller. - Runner usage capture: LoopContext (abortSignal + onUsage) threads into all three kind loops; per-step usage folds via accumulateJobUsage into JobRecord.usage (model from config MODELS, absent-never-zero), flushed in runJob's finally so an aborted job's money record survives the kill. - Contracts: JobUsage token fields optional + lastStep (most recent call's accounting, for the dock's per-blob context line vs CONTEXT_LIMITS); SubagentResultMeta.aborted; AbortSteerMessage; buildAbortSteer; SteerMessage union generalized. - demo-builder/visual-maker accept abortSignal + onUsage (visual folds generateText usage per iteration). - stillRunning/finishJob/jobContext guards: an aborted job is never overwritten by the runner (terminal no matter what the wake does); failJob and the pump's finalizeJob keep the flip. - UI: abort steer renders a quiet aborted chip (job-aborted-badge). - Prompt 20-jobs.md: one line about aborted jobs. Tests: accumulateJobUsage folds (undefined-preservation, lastStep), runner usage capture, real-kill abort (sync flip, silent unwind, abort steer wakes), usage-so-far survives the kill, aborting a delivering job drops the queued result steer, outcome reasons, abort-chip render. 330/330, typecheck/lint/build green. |
||
|
amosblomqvist
|
2535d7ad35 |
fix: defer the jobs settle-sync until the turn is over (coalescing follow-up 2)
With continuation coalescing, a job flips done at CLAIM time — while the learner's own answer turn is still streaming. The poller's settle re-sync fired on the flip, swapping in a mid-turn record snapshot mid-stream; the steer after the assistant message broke the SDK's in-place continuation merge, and the streaming response landed as a SECOND message with the same id — duplicate React keys in the live transcript (the user's live test hit it; the record itself stayed clean). A settle during a busy turn (local stream OR server turnInFlight) now marks a pending sync; the first idle poll tick adopts the finished record. This also fixes the latent multi-job race: steer B settling while steer A's wake turn is in flight would have grabbed the same mid-turn snapshot. Out-of-lane surgical touch (flagged for master review): the wave-1 poller block in components/chat.tsx. |
||
|
amosblomqvist
|
91ae2e137d |
fix: dedupe same-id continuation copies in persistTurn (coalescing follow-up)
With a steer coalesced into a quiz-answer continuation, the steer sits AFTER the assistant message being continued, so the SDK's onFinish list carries the input copy AND the extended response copy as two entries with the SAME id — persistTurn saved both, the jobs poller's syncFromServer imported the dup, and React keyed a single message twice (the user's live test hit it). withMessageIds now dedupes by id keeping the LAST occurrence — the SDK's final merge state (a superset of the earlier copy) — which also yields the right semantic shape: [user, steer, assistant(extended)]. Out-of-lane surgical touch (flagged for master review): turn-service.ts. Squashed the live session's already-corrupted record by hand (same dedupe). |
||
|
amosblomqvist
|
e33b83ca2d |
fix: steer-delivery starvation — continuation coalescing (claimSteers)
The bug (seen live): a teacher that ends every turn with a new pending quiz keeps the wake pipeline's idle gates permanently closed — the steer sits in 'delivering result' forever while the learner answers check after check. - claimSteers (server/jobs.ts): the chat-handler claims every queued, undelivered steer the instant a client turn wins the turn registry, and appends it after the learner's answer — conversion-safe, the answered quiz carries its output. Every answer flushes pending steers immediately. - The pump keeps its post-idle duplicate-guard (re-check steerDelivered before firing), so a claim racing the pump converges to exactly one delivery either way. - The stale-job sweep now skips chats with a turn in flight (a healthy but slow wake no longer gets falsely re-queued as 'recovering'). - Out-of-lane surgical touch (flagged for master review): the 3-line handleChatRequest seam (claim after beginTurn, persist+stream 'outgoing'). - Verified live end-to-end on the stuck session: answering the pending quiz fired the steer + wake turn within a second; job flipped done/delivered. |
||
|
amosblomqvist
|
2f8cf71f52 |
Wave 2: 409 merge-retry transport, dead pendingJob code, evidence-based timing, prompt-size guard
- components/chat-transport.ts (NEW): createChatTransport(chatId) — the chat.tsx seam is now a one-line swap. Its fetch turns a 409 on POST /api/chat (a learner send racing the wake pipeline's ~1s adoption window) into: wait out the in-flight turn (poll turnInFlight, 30s cap), refetch the record, MERGE the outgoing messages onto the fresh server list, re-POST. A bare retry would clobber the steer + wake messages via the write-ahead persist (steerDelivered makes that loss permanent) — merging is the whole point. Unit-tested: pass-through, wait+merge+reorder, dedupe by id, repeated-409 loop, bounded give-up. - src/core/pending.ts: findPendingJobCall deleted (dead since wave 1 — job tools resolve server-side with an ack, nothing pends client-side). JOB_TOOLS kept (runner.ts still uses it). Header doc rewritten. - src/server/jobs.ts: GRACE_MS 1500 → 750, with the evidence in the comment: the race window is only the learner's answer→POST flight (sendAutomaticallyWhen dispatches synchronously; ~10ms localhost, <300ms realistic RTT — 750 is 2.5× worst case) and misses self-heal via the pump's gate recheck, the 409 backoff ramp, and the new transport merge-retry. POLL_MS=400 kept (sampling granularity only). - tests/jobs.test.ts: prompt-size guard — every fragment ≤8k chars, composed prompt ≤60k — the README-leak class (815 bytes ≈ 200 tokens per teacher turn before wave 1's fix) can't silently recur. Gates: typecheck, 279/279 tests, lint, build — all green. |
||
|
amosblomqvist
|
28ccc76594 |
fix: README.md leaked into the teacher's system prompt
composeInstructions loaded EVERY .md in src/agent/prompts — including README.md, the directory's human documentation, which rode into the system prompt as its own section (visible in OpenRouter logs, old POC paths and all). The loader now enforces the documented convention: only numeric-prefix fragments (\d+-*.md) compose. README.md stays in the directory as docs and can never reach the model again. README's rule text updated to match; composeInstructions exported for tests; regression test asserts the README never leaks and every numeric fragment composes verbatim. |
||
|
amosblomqvist
|
2d7b004fda |
fix: live sub-agent observability — dispatch cards, steer artifacts, sidebar panel
The pi-style ack protocol (previous commit) left the client describing the
old world: artifact cards read the ack string as a result ("could not be
displayed"), the only job poller rendered on input-available parts that no
longer exist (so wake turns landed invisibly until re-navigation), steers
rendered as learner bubbles with no SVG anywhere, and server-persisted
assistant messages carried empty SDK ids (duplicate React keys).
- persistTurn (turn-service): fill empty message ids at the canonical write
— server-only messages (last turn's, wake turns') now key uniquely;
client-authored ids untouched.
- chat.tsx: ONE central /api/jobs poll (fast while jobs run, slow discovery
cadence, off for job-less sessions) drives every job surface, re-syncs
once when a job settles (the record already holds the steer + wake
response at flip time), and adopts server-side wake turns via the same
turnInFlight machinery as returning mid-turn. Steers branch to the
artifact card — never a bubble. Composer no longer gates on jobs.
- /api/jobs now reports turnInFlight alongside the job list.
- dispatch-card: the tool-call box — live activity by toolCallId while
running, quiet "delivered"/"failed" one-liner after; the ack is visible
observability, the artifact lives in the steer.
- steer-card: renders the delivered artifact by kind (research brief /
sanitized SVG / sandboxed demo iframe) from the data-subagent-artifact
part; result renderers extracted and exported from the artifact cards.
- sub-agents-panel: right-sidebar section above the plan tree listing
running sub-agents (kind, elapsed, live activity) — vanishes when idle.
- tests: dispatch/steer/panel render tests re-target the new protocol;
persistTurn id-hygiene suite. 229/229 green, build green.
Out-of-lane touches for master review: chat.tsx + components/** (cleared
this wave), app/api/jobs/route.ts, src/server/turn-service.ts, contracts
protocol doc §2. Live-verified on :3003 (full loop, SSR DOM checked).
|
||
|
Amos
|
99fa3f0847 |
Pi-style ack-and-steer sub-agent protocol: dispatch mid-turn, steer via server wake turn
The job tools (research/demo/visual) switch from client-side turn-breaking calls to pi's model: execute resolves IMMEDIATELY with a fire-and-forget ack, the teacher's turn continues, and the result arrives later as an injected user-role steer message that wakes the teacher in a new server-side turn. - contracts.ts: THE ASYNC PROTOCOL rewritten (ack now, steer later); steer types (SubagentSteerMessage, metadata.subagentResult + data-subagent- artifact part), pure buildSubagentSteer/steerDelivered/isSubagentFailure - tools/jobs.ts: jobTools(ctx) factory — server-side execute creates the job (deduped by toolCallId), fires spawn with the dispatch-time input (the part isn't persisted yet at execute — a transcript scan would race) - runner.ts: runJob(jobId, input?) builds the steer and hands it to the injected deliver seam; pipeline owns the done/error flip (a polling client re-syncs once, at the end); failures still steer - server/jobs.ts (new pipeline): per-chat serialized queues on globalThis, idle gates (turn liveness + pending question, 1.5s grace), 409 retry with backoff, idempotent per jobId, stale-job recovery sweep (re-deliver or re-spawn); wake turns run through the same handleChatRequest composition via setTurnRunner injection - chat-handler.ts: wires runner+deliver+tools ctx; drops spawnTurnJobs from onFinish; teacher default registry = question tools only - plan.ts: PlanToolContext split out (plan tools need no job wiring) - visual-maker: edit_svg added (pi parity — exact-match single replacement, fuzzy context on a miss); researcher: pi's kept/dropped source accounting restored - prompts 20-jobs/40-demos: async discipline rewritten (announce and keep teaching, no waiting/polling/fabricating, parallel dispatch, teach with the artifact when it arrives) - tests: jobs.test.ts rewritten (dispatch ack mid-turn, runner steers, wake pipeline: deferral/409/idempotency/failure), demos.test.tsx runner tests ported to runJob + fake deliver; 222 green FLAGGED surgical touches outside the stream's owned files, for master review at merge: src/core/contracts.ts (protocol doc + steer types), src/core/ui-message.ts (job tool + data-part types), src/server/ chat-handler.ts (wiring), src/server/turn-service.ts (spawnTurnJobs retirement), src/agent/teacher-agent.ts (default tools), src/agent/tools/ plan.ts (context type split). Verified live on PORT=3003 with real models: teacher dispatched research mid-turn, researcher ran real searches/fetches, steer injected on quiz answer, wake turn wove the brief in citing its sources, job flipped done; a client/wake race correctly 409'd. |
||
|
Amos
|
59e1fee850 |
Turn services + agent DI: chat-handler becomes the composition root (stage 4)
The chat turn is now composed, not implemented, by chat-handler.ts:
- src/server/turn-service.ts: one named, error-contained seam per turn
responsibility — buildTeacherContext (W8 read channel), persistTurn
(canonical state write), spawnTurnJobs (async protocol kick-off),
foldTurnIntoGraph (session→graph consolidation; the extraction-pipeline
stage swaps this body behind the same seam). Each takes its deps
explicitly; failures log one [chat <id>] line and never touch the
response.
- chat-handler.ts wires: storage ports (stores), the job runner, the
teacher tools (ctx = chatId + injected ChatStore), and the services.
It no longer contains any logic beyond request validation.
Agent layer now receives dependencies instead of importing them — a new
lint zone enforces it (src/agent never imports src/server or src/store;
verified to fire both ways):
- runner.ts: createJobRunner({ chatStore, jobStore }) factory returning
the spawn hook + the three job runners; the model-override test seam
stays module-level. Bodies otherwise verbatim.
- plan tools: persist through the ctx-injected ChatStore
(TeacherToolContext = { chatId, chatStore }).
- teacher-agent's dependency-free default registry is now the true
client-side set: questionTools + jobTools (jobTools was ctx-parameter-
shaped but never used its ctx — now honestly context-free). Production
always passes createTeacherTools({ chatId, chatStore }); tests that
drive a bare mock model get the client-side set unchanged.
Zero behavior change: tsc clean, 190/190 tests, eslint clean, production
build clean. Tests inject the same fs ports the chat-handler does.
|
||
|
Amos
|
1167157c23 |
Storage ports: ChatStore/GraphStore/JobStore + adapters + core inversion (stage 3)
Per-entity semantic ports (src/core/ports.ts), one adapter family
(src/store/*, fs JSON/JSONL), exactly one composition point
(src/server/stores.ts). Dependency direction is now machine-checked:
adapters implement core interfaces; core never imports an adapter.
The inversion:
- core/graph.ts (emitChat/getGraph/graphSnapshot/foldStaleChats) is now
pure — it takes the storage ports as explicit parameters (GraphDeps).
No store imports; newGraphNodeId is local; node id format unchanged.
- core/graph-context.ts buildGraphContext() no longer defaults to the
live ledgers: the chat-handler hands in getGraph(graphStore) per turn.
- store adapters export exactly one object each implementing their port
(chatStore/graphStore/jobStore); bodies otherwise verbatim.
NodeMapping + SessionCheckpoint moved store/graph -> core/contracts
(they ARE contract shapes); CreateJobInit/JobPatch live in ports.
- titleFromPrompt moved store/chats -> core/contracts (pure helper,
planNodeStatus precedent). Store modules now contain persistence only.
- server/stores.ts is the composition root; server/{chats,graph,jobs}.ts
and chat-handler wire from it; agent (plan tools, sub-agent runner)
call the port objects. The shell surface (@/src/server exports) is
unchanged.
Lint law extended: the whole of src/core is now a purity zone (was a
file glob with a graph/graph-context exemption). Verified it fires on
core->store imports; tsc clean, 190/190 tests, eslint clean, prod build
clean (prerender 6/6). Tests pass deps explicitly — graph.test uses the
same fs-adapter pair stores.ts wires in production; graph-context.test
keeps its in-memory ledgers as a GraphStore implementation.
|
||
|
Amos
|
df16750560 |
W2: async job infra + researcher sub-agent
- src/store/jobs.ts: per-job JSON store in data/jobs (create/get/update/list, dedup by toolCallId) - src/agent/tools/jobs.ts: research tool — client-side (no execute), turn ends at the call; input per contracts ResearchToolInput - src/agent/tools/web.ts: server-side web_search (Google CSE) + web_fetch (Readability/Turndown/linkedom, Jina fallback) — errors return as strings, never thrown into the agent loop. Deps: @mozilla/readability, linkedom, turndown (+@types/turndown dev) - src/agent/subagents/researcher.ts: isolated ToolLoopAgent (web tools only, 12-step cap), researcher prompt ported from pi - src/agent/subagents/runner.ts: spawnPendingJobs hook + runResearchJob — fire-and-forget, activity line per tool call, fulfills the part in the LATEST chat record (output-available) before flipping job status; failure still fulfills the part so the teacher learns the outcome - src/server/chat-handler.ts: onFinish spawn hook after saveMessages - app/api/jobs/route.ts: GET ?chatId= polling endpoint - components/artifacts/job-card.tsx + registry entry: polling activity card (2s) → syncFromServer on finish; research-brief card (markdown + sources) - src/agent/prompts/20-jobs.md: research doctrine for the teacher - tests/jobs.test.ts: store CRUD, spawn hook via scripted mock teacher, runner completion/failure with fake model + stubbed fetch |