The old lines were our hand-rolled shortcut: every arrow from the
source's bottom-center to the target's top-center as a single S-bezier.
Dagre actually routes each edge as border-intersected waypoints that
leave/enter nodes on whatever side the route needs — mermaid's
behavior. We now keep that routing (owner: adopt it).
blockLayout also returns edgePts — dagre's per-edge waypoint lists.
edgePathD draws them through d3's curveBasis, the exact recipe
dagre-d3's own renderer uses (and mermaid's default flowchart curve):
endpoints repeat so the line starts at the source border and the END
pulls back by the tip overhang so the arrowhead's tip lands on the
target border with its mouth at the path's end. Pinned in tests.
Dangling/unroutable edges draw nothing. d3-shape added as a direct
dependency (^3.2.0, already in the tree via dagre-d3-es — lane touch,
flagged to master with dagre-d3-es).
The owner chose: adopt the same layout engine regular mermaid uses, keep
our SVG skin. That engine is dagre (Sugiyama-style layered layout) — and
mermaid 11 ships dagre-d3-es 7.0.14, so we import exactly that fork, via
the same deep subpath mermaid itself uses. No new code shipped; pinned
verbatim in package.json (lane touch — flagged to master).
blockLayout keeps its signature ({ x, y, nodeH, width, height }, all
consumers unchanged): it feeds dagre our geometry (NODE_W boxes, GAP_X/
GAP_Y separations, zero margin, per-node grown heights from round 7),
runs the layout (deterministic, synchronous, DOM-free — SSR and render
tests stay exact), and converts dagre's node centers back to top-lefts.
Dangling edges are ignored defensively (a ghost endpoint would spawn an
unmeasured node and break the measurer).
topoFor stays exported for order/depth semantics (tests); placement no
longer calls it. dagre's network-simplex ranker + crossing minimization
now handle multi-parent DAGs, so future plan shapes route like mermaid
diagrams do. On the existing tree fixtures dagre's numbers agree exactly
with the old hand-rolled layout — every render pin survives verbatim.
Louvain only ever found the reasoning chains (the graph is disconnected
by nature — every connected component became its own cluster) and
edge-orphans could never join anything. Clusters are now RELATED TOPICS:
propositions grouped by meaning.
- Embedder (src/server/embeddings.ts): in-process transformers.js v4
running EmbeddingGemma-300M q8 (MTEB 68.4, ~300MB one-time download),
MRL-truncated to 512d, "title: none | text:" prefix on 100% of inputs,
L2-normalized so cosine = dot. Model/dim mismatch voids the ledger
(full re-embed). Injected at the composition root — the agent layer
never imports the server layer (architecture law).
- Ledger (data/graph/embeddings.json): model + dim + vectors + the
persisted cluster partition. Reclustering runs at the END of every
embedding batch (post-merge hook, both front doors) — never per read
(the agglomerative pass is ~1.5s at 338 nodes).
- Clustering (src/core/semantic-clusters.ts): hand-rolled average-linkage
agglomerative over cosine (npm HDBSCAN is an abandoned alpha). Stragglers
stay singletons by default; attachStragglers is the all-in toggle.
Threshold 0.60 swept on the real graph: 26 topics, 74/90 edge-orphans
land in real topics, 10 stragglers.
- Consumers (snapshot, settle, namer, replay) read the persisted
partition — one cluster concept everywhere; core-survival label
matching unchanged (20/26 topics kept labels).
- Louvain retired: detectCommunities + graphology* deps deleted
(graphology-louvain had leaked into the client bundle since the
clusters wave); scripts/bench-graph.ts (dead FA2 bench) removed.
- scripts/embed-graph.ts: backfill/cold-start (338 nodes in 150s
including the one-time model download).
One engine owns the equilibrium: the merge settle and (next commit)
the live drag both run createGraphSimulation — d3-force link springs,
1/d charge, weak centroid gravity — so a drag's rest state IS the
server's rest state. The FA2-era anchor mechanism survives as a
post-tick POSITION-SPACE re-blend (retention 0.85): d3's forceLink is
a Hooke spring (strength*(d-L)*alpha), which overpowers any force-space
anchor at long range — the re-blend is alpha-independent and always
wins as the sim anneals (the 675-unit drift bug). Gravity retuned for
d3's 1/d many-body falloff (bootstrap 0.2 / incremental 0.12: cluster
spacing lands ~400 world units, the FA2-era density). Rim orphans seed
(and anchor) at 1.7x the CONNECTED core's mean radius — a max-span-based
rim fed back on itself and ran the layout away to 6000 units. Replay:
338 positions in 1.6s, old-node shift within a batch 4.8 (FA2: 23).
Implements Option B from docs/research/graph-rendering.md. sigma.js v3
(WebGL) is replaced by one canvas, one rAF loop with a dirty flag, and
d3-zoom for the camera — eliminating the entire bug class from the
sigma-era bolt-on architecture:
- Hull/node desync during pan (two independent render loops) is now
structurally impossible: hull fills, dashed hull strokes, edges with
arrowheads, node discs, region labels, node labels, and hover glow
are all drawn in ONE frame under ONE camera.
- The settings-change canvas-sizing race (everything shifting up-left)
is gone: settings changes only redraw; the canvas and camera persist
(verified live: camera identical to 15 decimals across a slider drag).
- The GPU floor is gone: idle schedules NO frames at all (verified via
pixel-identity over 600ms idle); pan redraws exactly once per frame.
components/graph-canvas.ts: GraphCanvasRenderer class plus pure helpers
(zoomRatioOf, labelAlpha, fitTransform, arrowGeometry, pickNearest).
The sigma camera-ratio convention (1 = fit, smaller = zoomed in) is kept
verbatim so persisted labelZoom settings survive the swap; node labels
and cluster region labels cross-fade over the same LABELS_FADE_SPAN
(0.5, unchanged). Picking is a nearest-node distance check with the
10px pad that replaces the old padded-picking vertex shader. Hull math
stays in components/cluster-geometry.ts, reused as-is. No ctx.filter,
ever.
graph-view.tsx keeps every feature: fetch/refetch on runs-empty, FA2
run-then-freeze layout, localStorage render settings + sanitizers,
search filtering with edge drops and card-close, detail card with
flipped anchoring and per-source provenance, settings panel, import
button, pipeline status chips. Only the renderer plumbing changed.
tests/graph-canvas.test.ts pins the pure math: ratio direction, label
gate + inverse complement (alphas sum to 1), fit transform padding,
arrowhead tip-on-disc-edge and symmetry, picking pad/ties/null cases.
Live-verified end-to-end on the real 338-node graph (dev server
port 3002): single canvas, paint at fit, hover cursor+glow+fade, click
card open/anchored/close, stage click, search filter to zero with hint,
label gate opens via slider and zoom, cluster-labels toggle, zoom-to-
cursor 0.56px anchor drift, camera persistence across settings, zero
idle repaints. sigma dependency removed from package.json; d3-zoom and
d3-selection added.
Note: the settings feed effect needed isEmpty in its deps — without it
the renderer mounted with settings=null and every draw early-returned
(blank canvas); caught live, not by typecheck.
detectCommunities(nodes, edges) in src/core/clusters.ts (new dep
graphology-communities-louvain): Louvain modularity over the undirected,
unweighted follows+supports structure (user-decided 2026-09-13). The
random-walk traversal runs on an inlined seeded mulberry32 so the same
graph always yields the same partition; community indices are still
unstable across edits, which is exactly why labelings (phase B) key to
sorted MEMBER SETS. follows+supports between the same pair collapse to
one edge; self-loops and dangling edges are skipped defensively; no
min-size filter here (the >=3 render threshold is a presentation rule).
Tests: two-cliques-plus-bridge split, determinism, singleton/empty
graphs, pair collapse, defensive skips, member sorting. 363/363.
Real-data smoke over the current notes graph: 338 nodes / 211 edges ->
138 communities, modularity 0.95, and the big ones are exactly the
learn-session topics (databases, differential forms, sales, servers,
vim, copulas...). 32 communities clear the >=3 render threshold.
The Vercel AI SDK survives only as Band D (useChat client + UIMessage
record types); @openrouter/ai-sdk-provider is uninstalled.
- src/agent/models.ts: PiEngine {model, streamFn} via createModels +
openrouterProvider subpath; thinkingLevelFor maps config effort none->off
- src/agent/lh-tool.ts: neutral LhTool (zod authoring, optional execute,
clientQuestion flag); toPiTool converts via z.toJSONSchema and re-runs
zod inside execute so refinements ride the error-result retry path
- src/server/pi/record-adapter.ts: UIMessage records -> pi Messages
(dangling tool calls get synthesized interrupted results)
- src/server/pi/ui-stream.ts: SSE bridge emitting AI-SDK-shaped chunks so
the useChat client parses unchanged (continuation id rule, question
tools never emit output, tool-input-error for bad authoring)
- teacher-agent.ts: stateless per-turn pi Agents; shouldStopAfterTurn
stops only when every question-tool call resolved cleanly (errors retry)
- chat-handler.ts: buildAgent(outgoing) factory seam (stateless doctrine);
assertValidUiMessages owns the retired SDK validation gate
- sub-agents via shared runSubagent (pi-run.ts); extraction stages stop
after a clean submit; visual-maker keeps its hand-rolled loop over
direct streamSimple calls; aborted model calls bill no usage
- tests: all model seams ported to pi-ai faux provider (scripted
FauxResponseSteps, agent factories); handleChatRequest second param
is now an agent factory
Gates: tsc, lint, build, 305/305. Live GLM smoke (disposable driver,
deleted): prober streaming + persistence, question seam, turn-2
continuation same message id, per-turn real token usage - ALL PASS.
Data wiped at the pivot (chats/graph/jobs, .gitkeeps kept).
The shell now imports through exactly two barrels, enforced by
import/no-restricted-paths (npm run lint):
components/** -> "@/src/core" (pure core API, browser-safe)
app/** -> "@/src/core" + "@/src/server" (server entries)
Zone violations are lint errors with architecture-pointer messages; the
pure-core modules (contracts, question-core, plan-tree, phases, pending,
math-delims, ui-message) may never import agent/server/store.
Supporting moves (direction: IO layers -> core):
- core/index.ts: the core public API barrel
- core/ui-message.ts: moved from src/agent (it was shell-facing all along)
- quiz/ask input zod schemas moved agent/tools -> core/question-core
(the question contract, alongside the grading logic)
- ChatRecord/ChatSummary moved store/chats -> core/contracts (single schema)
- GraphSnapshot/NodeStateClass moved core/graph -> core/contracts (the
/api/graph payload IS a contract; the pure renderer needs only these)
- src/server/{chats,graph,jobs}.ts + index.ts: thin server entries for the
transport layer; GET /api/graph's fold side-effect now lives behind
graphView() instead of inline in the route
Gates: tsc clean, 190/190 tests, eslint clean (law verified to fire on
store/agent/subpath violations), production build clean.
(Repo repair en route: a git object (app/chat/[id] tree) was missing from
the object store — reconstructed byte-exactly via mktree from its blob;
the broken node_modules that stalled the build was cleanly reinstalled.)
Delete src/extract/ (pi-session normalize/extract/merge stubs), src/ledger.ts,
src/schema.ts, src/derive.ts, src/queries/, src/cli.ts (npm run ingest/graph/
what-next), data/legacy-graph/, data/agent/ (pi agentDir vestige), and their
config.ts/package.json references.
The retroactive-bootstrap goal is dropped: the live chat store (AI SDK
UIMessages) is the only session format now, and the old pi JSONLs in
data/sessions/ stay on disk, inert. contracts.ts is the single schema.
- src/store/jobs.ts: per-job JSON store in data/jobs (create/get/update/list,
dedup by toolCallId)
- src/agent/tools/jobs.ts: research tool — client-side (no execute), turn ends
at the call; input per contracts ResearchToolInput
- src/agent/tools/web.ts: server-side web_search (Google CSE) + web_fetch
(Readability/Turndown/linkedom, Jina fallback) — errors return as strings,
never thrown into the agent loop. Deps: @mozilla/readability, linkedom,
turndown (+@types/turndown dev)
- src/agent/subagents/researcher.ts: isolated ToolLoopAgent (web tools only,
12-step cap), researcher prompt ported from pi
- src/agent/subagents/runner.ts: spawnPendingJobs hook + runResearchJob —
fire-and-forget, activity line per tool call, fulfills the part in the
LATEST chat record (output-available) before flipping job status; failure
still fulfills the part so the teacher learns the outcome
- src/server/chat-handler.ts: onFinish spawn hook after saveMessages
- app/api/jobs/route.ts: GET ?chatId= polling endpoint
- components/artifacts/job-card.tsx + registry entry: polling activity card
(2s) → syncFromServer on finish; research-brief card (markdown + sources)
- src/agent/prompts/20-jobs.md: research doctrine for the teacher
- tests/jobs.test.ts: store CRUD, spawn hook via scripted mock teacher,
runner completion/failure with fake model + stubbed fetch
Layout:
- Home = knowledge-graph placeholder (main view) with new-session input
docked below; session list moved out of home into a hamburger drawer
- SessionDrawer: slide-over session navigation (backdrop/ESC close, active
highlight, fetches /api/chats on open) on both home and chat headers
- Chat header loses the back button — the drawer is the navigation
Math (inline $ was silently not rendering):
- @streamdown/math defaults singleDollarTextMath:false — now
createMathPlugin({ singleDollarTextMath: true }) in message + reasoning
- katex.min.css imported in layout (streamdown does not inject plugin
styles; display math only looked right via browser MathML fallback)
- QuizCard/AnsweredCard route question/options/explanation through
MessageResponse; user chat bubbles render markdown+math too
- Currency hazard: micromark-extension-math@3.1 lacks pandoc's
closing-digit rule, so bare '$5 and $10' parses as math — defense is at
the source: system prompt now instructs escaped \$5 / 'USD' for money
Hydration fix (React #418):
- QuizCard shuffled options with Math.random() — server/client orders
diverged. shuffleOptions now takes an injectable RNG; QuizCard passes
seededRandom(toolCallId) (mulberry32/FNV-1a in question-core). Same
question → same stable order across reloads; still varies per question.
Tests: 53 pass (5 new math-rendering pins incl. fenced-vs-one-line $$
behavior and the escaped-currency contract). tsc + next build clean.
Verified live in browser: inline/display math in text + cards, drawer
open/navigate/active-highlight/ESC, no console errors, no font 404s.
The teaching agent no longer runs on pi. Stack:
- ToolLoopAgent (ai@7) + OpenRouter (OPENROUTER_API_KEY), model via LH_MODEL,
default anthropic/claude-sonnet-4.5 with reasoning tokens
- quiz/ask_user_question = client-side tools (no server execute): the call
lands as a typed tool part in input-available state; the card grades via
src/core/question-core.ts; addToolOutput sends the verdict; the teacher's
reaction continues the same assistant message
- resume-mid-quiz is free: the pending tool part persists in the transcript
- persistence: one JSON per chat (data/chats/), full UIMessage transcripts,
saved onFinish (also on abort)
- UI: AI Elements (owned copy-paste components, pruned to the five in use)
+ button-based QuizCard/AnsweredCard (ask answers never graded)
- pi fully removed: TUI, extensions, Fastify+WS server, bridge, custom
protocol all deleted (recoverable from git history)
- tests: question-core units carry over; chat-flow integration drives
handleChatRequest with MockLanguageModelV4 (48 tests, zero tokens);
render smoke tests guard the cards (renderToStaticMarkup)
Carried over unchanged: teacher system prompt, question-core, schema/
ledger/derive/extract pipeline, data model.
amosblomqvist