- Arrows into not-yet-visited nodes are grayed out like the nodes
(stroke-border line + a fill-border copy of the arrowhead); the
in-chat block stays all-white.
- The yellow glow is static — the pulse is gone (owner).
- Smaller blur: stdDeviation 7 → 5, glow rect margin 10 → 8.
- The glow clipped at the frame on narrow graphs: its visible reach
(GLOW_MARGIN + 3·GLOW_BLUR = 23 units) far exceeded GRAPH_PAD (8).
Surfaces that show the glow now pad by GLOW_REACH — the popup's
viewBox AND its fit box both use POPUP_PAD = GLOW_REACH, so the blur
never clips; the monochrome in-chat block stays tight at GRAPH_PAD.
arrowhead_1.svg (the CM/MJX long-arrow head) baked in — transform chain
collapsed to a clean scale 0.2 + y-flip: viewBox 0 0 50.2 104, tip at
(50.2, 52), notch mouth at x 27.2, tail opening 9.6 tall (≈10 at the
owner's full height 104).
Sizing per the owner's spec: line width stays 1.5; head height 16 makes
the mouth opening tuck the line flush (the LaTeX proportion). markerUnits
userSpaceOnUse, refX at the mouth (27.2) so the line terminates at the
notch — never at the narrow tip, where it would show through. Edges
shorten by the tip overhang (≈3.54 units) so the head's tip lands
exactly on the target node's border.
Look (rectangular, white language — owner spec):
- Boxes: no corner rounding (rx dropped everywhere).
- Visited: fill-none, white stroke, white text. Upcoming: today's gray
(fill-card, border stroke, muted text) — kept per owner. Current:
white stroke/text + a faint yellow highlight — a blurred yellow fill
(feGaussianBlur) that bleeds out over the node's edges, still breathing.
- Edges + interim arrowhead: white (marker swaps to the owner's SVG when
it lands: right-pointing, solid fill, tight viewBox, tip at right edge).
- The IN-CHAT plan block renders monochrome — every node white, no
status language, no glow; the popup/plan-window show the progression.
Popup entrance fix: the transition from round 3 animated the first fit
(graph flew in from the top-left zoom:1 state). The transition now stays
off until the first fit has been applied — the popup paints settled.
- ZOOM_MAX 2× → 4×: node titles reach ~50px rendered, comfortable for
close reading. Single tunable constant.
- Smooth zoom: a 180ms ease-out CSS transition on the transform wrapper —
CSS retargets from the current computed value, so chained wheel ticks
and the fit/button zooms all glide instead of snapping. Disabled while
dragging (per-frame pans must not rubber-band). panning is React state
toggled at drag start/end.
Root cause of the popup's fit/center failure: PlanDiagram rendered
w-full inside the popup's shrink-to-fit absolute wrapper — the svg
resolved to the popup's width regardless of its viewBox extent, so the
transform math (1 svg unit = 1 px at zoom 1) was off and the 'fit floor'
neither fitted nor centered.
- naturalSize: the popup's diagram renders at the viewBox width in px,
so fit/pan/zoom math holds.
- The popup's fit box is now the PADDED ink box (GRAPH_PAD all around):
centered strokes and the glow halo never clip at the frame edge.
- fitZoomFor lost its 0.15 artificial floor — for wide graphs it sat
ABOVE the true fit and broke 'fully zoomed out = the whole graph
visible, whichever axis binds' (owner's rule). The fit itself is the
floor; a 0.02 epsilon only guards degenerate viewports.
In-chat full plan block:
- TEXT_CAP_SCALE (16.5/12.5): node text renders at text-xs in svg units;
the block now carries maxWidth = extent × cap + mx-auto, so it still
fills the column whenever the column is narrower than the cap, but
never scales node text past the chat prose size (--lh-prose-size).
- GRAPH_PAD (8 svg units) around the full-extent viewBox: centered
strokes (half outside the geometry) and the −4 glow halo were clipping
at the frame edges — the padded origin (-8,-8) keeps all ink visible.
Only the full-extent render pads; the plan window's override viewport
and the popup's fit margins are untouched.
Test pins updated: mx-auto + max-width:232.32px + viewBox='-8 -8 176 152'.
Per-stroke alpha baked into every stroke/disc accumulated through
source-over compositing: one shape 0.35, an orphan disc over anything
1-0.65^2 ~ 0.58, hubs toward 1.0 — a lone node's glow read as a
darker double-counted spot on top of other regions. The pipeline is
now dilate at full opacity -> blur -> scale the whole field by
edgeAlpha once at the end. Painted at 1, the alpha channel is a pure
coverage indicator (overlaps of any count contribute exactly 1.0),
the blur smooths it into a coverage falloff, and the final scaling
gives a constant interior alpha everywhere covered, varying only
across the blur edges. The blur-off path also runs the readback now
(radius 0 = copy) so the end-alpha applies uniformly without blur.
Interaction change (user-flagged, accepted for now): different
clusters' overlapping halos no longer blend hues proportionally —
the later-painted cluster covers the earlier, ownership boundaries
softened only by the blur reach. Verified: old build spread 36k of
55k tinted pixels above alpha 100 (stacking); new build concentrates
at 70-90 (0.35*255=89 interior with falloff), and an orphan disc now
reads no brighter than the skeleton it sits on.
The stroke width is anchored to kFit (screen px at fit zoom — the
camera persists across searches), but the blur radius was anchored to
cell, which scales with the FILTERED world span: a tight search shrank
cell, and the blur's screen reach (blur*cell*kFit) collapsed with it
while the stroke stayed put — the blur visibly went away for
close-together matches and the knob did nothing (verified: blur 6 and
blur 10 produced identical output under a tight search). The blur now
uses the stroke's anchoring: its world reach is blur*cellRef, cellRef
being the full-graph cell from worldBBox (the same bbox kFit comes
from) — span-independent, and with nothing filtered cell == cellRef
so the unfiltered look is unchanged.
The cell also gains a floor: a single matching node (span 0) zeroed
the cell and blew the bitmap dims to Infinity, throwing the reported
getImageData TypeError. The floor is the padded rect's own reach, so
dims stay finite however tight the search (and the old bitmap
ballooning — ~1040px bitmap for a 3-node search — is gone too).
The settings fetch resolves before the graph snapshot creates the
renderer, so the load-time run of the region/physics feed effects hit
a null rendererRef and never re-fired — every reload silently kept
the DEFAULT region/physics tuning (a stored blur 10 rendered as 6).
Feeds #4/#5 now carry isEmpty in their deps, the same guard Feed #2
(setSettings) already had. Sizes were unaffected only because their
stored values happened to equal the defaults.
buildRegions filtered everything by visibility except the
intra-cluster edge strokes: with a search active, edges between
filtered-out nodes were still stroked into the bitmap — which had
already shrunk to the visible members — and segments crossing that
rect cut diagonal hue streaks from other clusters across the
filtered regions. Strokes now require both endpoints visible, same
law as the world rect, member discs, and label anchors. The
no-double-counting rule survives: a member touched only by a
partially-invisible edge gets its disc.
Non-integer blur radii broke the rendering (the log slider mapped to
fractional values), so the control is now integer-only by design:
REGION_BLUR_OPTIONS with snap-to-nearest sanitization for any input,
same law as the bitmap resolution. The old off zone (blur 0) is gone;
stored 0s snap back into the set.
All graph panel controls are now user-facing and server-persisted:
the old session-only dev number inputs are gone. GraphRenderSettings
gains nested regions/physics sections (backward compatible: old flat
files load with defaults), and RegionTuning/PhysicsTuning moved from
the renderer into core/render-settings.ts — one shape, one sanitizer,
one source for the store, the panel, and the renderer.
Physics/region clamps are sweep-validated safe ranges (each field at
both ends + all four corners settles cleanly on the real graph: no
NaN, no explosion — linkStrength caps at 4 since 8 explodes and 5
wobbles; friction floors at 0.1 since 0 never damps). Panel: log
sliders (default = geometric mean = position 50, step 5), off-zone
sliders for blur/cluster pull (position 0 = exactly 0), a fixed-set
dropdown for bitmap resolution, depth-anchor modes promoted to the
select.
Marching squares over each cluster's own contribution to the shared
blurred bitmap, recovered by projecting premultiplied pixels onto the
cluster's paint color (blends split proportionally between overlapping
halos — the natural divider). Boundary level = fraction of the
cluster's peak field, user-adjustable in the settings panel; 0 = off.
One paint, one blur, one readback; contouring is cheap typed-array
math over small per-cluster crops (stride-2 sampling: measured 1.2ms
for all 26 clusters, vs 7.3ms naive). During the live sim boundaries
recompute every 2nd region rebuild and stale paths ride along — drag
cadence matches the pre-boundary baseline (avg 8.46ms vs 8.37ms).
Two follow-ups on the semantic-cluster wave, both user-reported:
1. Region coloring: edge-only dilation (the anti-double-counting rule
from the Louvain era) left 114 of 338 cluster members completely
uncolored — semantic clusters contain members that share no
intra-cluster edge with anyone (90 edge-orphans + isolated
micro-topics), and 20 of 45 clusters painted nothing at all. Fix:
a disc (radius = dilation, same as a round cap) for exactly the
members no intra-cluster edge stroke reached. Every member is
painted by its edges' caps OR one disc, never both — the
no-double-counting rule survives. Verified live: all 74
orphans-in-clusters sit on tinted region pixels.
2. Depth modes (ORPHAN RULE): a node with no edges has no dependency
structure, so depth-0 stamping put 90 unconnected propositions on
the vertical baseline / radial base ring — meaning they don't have.
Orphans are now exempt from the depth force and get the regular
view's unprojected forces instead (full center gravity + full
cluster pull). Structured-node projections unchanged (x-only
gravity in vertical, none in radial; tangential-only cluster pull
in radial). Gravity in depth modes is a custom force with EXACTLY
d3's forceX/forceY math (strength × alpha — the first draft
forgot the alpha and shifted the settle). Verified live: radial
orphans follow their topics instead of the base ring; vertical
orphans spread 1006 world units (6/90 near the baseline band vs
90/90 before).
Louvain only ever found the reasoning chains (the graph is disconnected
by nature — every connected component became its own cluster) and
edge-orphans could never join anything. Clusters are now RELATED TOPICS:
propositions grouped by meaning.
- Embedder (src/server/embeddings.ts): in-process transformers.js v4
running EmbeddingGemma-300M q8 (MTEB 68.4, ~300MB one-time download),
MRL-truncated to 512d, "title: none | text:" prefix on 100% of inputs,
L2-normalized so cosine = dot. Model/dim mismatch voids the ledger
(full re-embed). Injected at the composition root — the agent layer
never imports the server layer (architecture law).
- Ledger (data/graph/embeddings.json): model + dim + vectors + the
persisted cluster partition. Reclustering runs at the END of every
embedding batch (post-merge hook, both front doors) — never per read
(the agglomerative pass is ~1.5s at 338 nodes).
- Clustering (src/core/semantic-clusters.ts): hand-rolled average-linkage
agglomerative over cosine (npm HDBSCAN is an abandoned alpha). Stragglers
stay singletons by default; attachStragglers is the all-in toggle.
Threshold 0.60 swept on the real graph: 26 topics, 74/90 edge-orphans
land in real topics, 10 stragglers.
- Consumers (snapshot, settle, namer, replay) read the persisted
partition — one cluster concept everywhere; core-survival label
matching unchanged (20/26 topics kept labels).
- Louvain retired: detectCommunities + graphology* deps deleted
(graphology-louvain had leaked into the client bundle since the
clusters wave); scripts/bench-graph.ts (dead FA2 bench) removed.
- scripts/embed-graph.ts: backfill/cold-start (338 nodes in 150s
including the one-time model download).
The graph's follows/supports edges encode derivation (premise ->
conclusion); the layout can now express that rank spatially, as a
session-only playground toggle (nothing promoted yet):
- inferenceDepths: longest chain from a base node, CYCLE-SAFE by
SCC condensation (Tarjan, iterative) — the DAG property is typical,
not guaranteed; cycle members share their component's depth (mutual
support = equal footing). O(V+E), pure, deterministic.
- DepthAnchor force: per-node target scalar — vertical mode rows
(baseline at the anchor centroid, arrows up), radial mode rings
(foundations center, outer boundary = edge of knowledge). The rank
is computed, not simulated: pairwise up-forces can't give baselines.
- Mode projections: gravity x-only in vertical / dropped in radial;
cluster pull x-only in vertical / tangential-only in radial; the
anchor re-blend preserves user arrangement in the FREE dimension
only (x / angle) — meaning owns the constrained axis, memory owns
the free one.
- setPhysicsTuning reheats (alpha 1): toggling a mode re-forms the
graph live and commits the new rest state; drag release is springy
along the constrained axis (node returns toward its level/ring).
- Sweep on the real graph: strength 0.8 = 215/216 arrows correct with
organic sag; 1.5 = crisp rows. 174 base nodes, max chain depth 7,
zero cycles in current data. replay-layout --depth=<mode> bakes a
mode into the ledger for rest-state experiments.
User-facing live-sim controls (settings panel, Physics section):
node repulsion, edge attraction, edge length, central pull, cluster
pull, friction — session-only number inputs feeding createGraphSimulation
(the SAME engine the server settle runs, so the feel IS the physics).
Changing a value rebuilds the sim from current positions; drag pins
(fx/fy live on the node objects) survive; a mid-settle change re-anneals.
The graph-view FA2 bootstrap fallback is rewritten on incrementalLayout
(d3-force): the last graphology import in the shell is gone.
The 'virtual node' realized as a centroid force: two allocation-free
O(n) passes per tick (membership resolved once at sim creation;
Float64Array sums). Measured indistinguishable from force-off at 338
nodes (271ms vs 272ms per 400-tick settle).
- createGraphSimulation: SimOptions.clusterPull + linkStrength +
velocityDecay (physics controls plumbing); degree-scaled link
stiffness so linkStrength=1 is exactly d3's default
- settleGraphLayout split into layout-settle.ts (it computes the
Louvain partition for the pull); layout.ts stays browser-barrel-pure
- barrel no longer re-exports clusters.ts: graphology + louvain are
fully out of the client bundle (verified against built chunks)
- replay passes the per-batch partition; CLUSTER_PULL_STRENGTH=0.15
tuned by sweep: cluster spread 63->36 world units, centroid
separation and total span unchanged (no blob merge, no collapse)
User feedback round 1 on the splat regions:
- Double counting: per-node gradient discs stacked with the edge-stroke
dilation wherever nodes sat on edges (hub hot spots). A cluster is its
CONNECTED skeleton — the region is now the dilated intra-cluster edge
network ONLY (round caps make line-end points discs), per the user's
definition. No per-node discs at all.
- The 'blur' was no blur: bilinear upscale of a 256px bitmap produced
tent-ramp chunkiness at zoom (the low-quality-JPEG look). Now a REAL
blur: 3 box-blur rounds on the premultiplied bitmap (separable running
sum, pure + unit-tested; premultiply keeps halos out of transparent
space). The upscale now has nothing left to make chunky.
- Live dev controls: number inputs (not sliders) in the settings panel —
dilation (screen px at fit), blur (bitmap px), bitmap resolution,
edge alpha. Session-only React state, never persisted; the renderer's
sanitizeRegionTuning makes bad/empty values safe. Verified end-to-end
through the real input (dilation 28 → 60 took effect).
- Premultiply un-recovery bug caught live: c = pm/a*255, not pm/a (the
first version rounded every color to black; regions rendered as
dark blobs with alpha 185).
- Region canvas: willReadFrequently (Chrome keeps it on the CPU for the
per-rebuild getImageData); defaults dilation 55 → 28 after measuring
screen coverage (63% → 55%).
- Perf: 120fps sustained through a drag with blur + rebuild-every-3rd-
frame (45 frames / 360ms measured), zero idle frames after settle.
The renderer grows a live d3 sim (createGraphSimulation, shared with
the server settle): pointerdown on a node pins fx/fy and pointer capture
(preventDefault kills the compat mousedown, so d3-zoom never pans); the
first CLICK_SLOP-exceeding move reheats to alphaTarget 0.3; release
un-pins and the sim anneals (release-and-relax) — alpha < 0.005 settles
it, a 250ms debounced onCommit PUTs the rest state. Verified live: 123fps
during a 320ms drag (regions rebuilding every 3rd frame), 85-tick settle,
then zero idle frames. A slop-free press stays a click (select, no reheat,
no commit). PUT /api/graph/positions sanitizes (finite coords, known ids)
and if the graph grew mid-drag runs ONE incremental settle with the
client's arrangement as anchors (50-id PUT test: anchored drift 0.15).
Cluster regions swap convex hulls for a low-res world-space SPLAT bitmap:
per-member radial-gradient discs (alpha 0.2) + intra-cluster edges as
round-cap strokes (Minkowski dilation — concave shapes finally render),
upscaled with bilinear smoothing (the upscale IS the blur, no ctx.filter:
Safari). One tiny blit per frame replaces 32 clusters x 4 offset
Path2D polygons re-rasterized at DPR 2 (the old GPU coil-whine path);
during the live sim the bitmap rebuilds every 3rd frame. cluster-geometry.ts
retires convexHull/offsetPolygon/traceRoundedPath (tests follow; clusterHue
stays). Region labels stay (centroid + clusterHue, inverse-gated on zoom).
One engine owns the equilibrium: the merge settle and (next commit)
the live drag both run createGraphSimulation — d3-force link springs,
1/d charge, weak centroid gravity — so a drag's rest state IS the
server's rest state. The FA2-era anchor mechanism survives as a
post-tick POSITION-SPACE re-blend (retention 0.85): d3's forceLink is
a Hooke spring (strength*(d-L)*alpha), which overpowers any force-space
anchor at long range — the re-blend is alpha-independent and always
wins as the sim anneals (the 675-unit drift bug). Gravity retuned for
d3's 1/d many-body falloff (bootstrap 0.2 / incremental 0.12: cluster
spacing lands ~400 world units, the FA2-era density). Rim orphans seed
(and anchor) at 1.7x the CONNECTED core's mean radius — a max-span-based
rim fed back on itself and ran the layout away to 6000 units. Replay:
338 positions in 1.6s, old-node shift within a batch 4.8 (FA2: 23).
The layout now EVOLVES instead of reshuffling on every extraction. A pure
core module (src/core/layout.ts) warm-starts forceAtlas2 from the positions
ledger (data/graph/positions.json, read-modify-write like sources.json,
updatedAt stamp for future live-physics conflict handling), re-blends every
previously-positioned node toward its anchor each iteration (retention 0.85
- old placements stay recognizable, strong forces still win), seeds new
nodes near their positioned neighbors, and seeds orphans on the rim outside
the core so they read as unconnected knowledge.
- Port/store: GraphStore.readPositions/writePositions; snapshot carries
positions when the ledger exists (optional - fresh graphs unchanged).
- Extraction pipeline settles synchronously after every merge (chat and
notes paths), before usage persist - the next /api/graph fetch already
reflects new nodes.
- View: the ledger IS the layout; the local one-shot FA2 pass survives
only as a fallback when no ledger exists.
- scripts/replay-layout.ts: simulates organic growth by replaying the
current 338-node graph one imported file at a time (real import order,
edges activate when both endpoints exist). Real run: 338 positions in
0.3s, 90-degree-0 orphans collecting at mean r=338 vs connected r=194.
- Also restores the cluster region fills (hull background colors) that
6692b19 had removed on a misreading of "no glow".
Live-verified: fit zoom derives from the ledger bbox, pickAt aligns with
ledger positions at every probe, hover/click/detail-card work, 408/408
tests, typecheck/lint/build green.
Future live physics stays cheap by design: the same pure function is the
merge-time settler and the future client simulator; a pinned node is just
an anchor with full retention.
- LABEL_GAP 6 -> 2 (user: labels sit way too far below their nodes).
- Hover glow is now PROPORTIONAL to the disc — outer radius = node radius
x 1.7 (GLOW_SCALE), matching the blur-a-copy-of-the-disc mental model,
instead of the old constant +6px pad that read huge on small nodes and
thin on big ones. GLOW_ALPHA 0.45 -> 0.3.
- Hover-OUT fade was broken by construction: draw() gated the glow on
hoverId, which nulls instantly on leave, so the 180ms fade-out ran
invisibly and the glow popped off in one frame. New hoverNode field
outlives hoverId through the outgoing fade (cleared when the fade
completes); verified live: 8 distinct frames over ~200ms then stable.
- Sliders rescaled per spec: ALL five are now log-scaled 0-100 with the
default dead-center at 50 (ranges chosen as default/4..default*4:
node/arrow/label size 0.5-8, edge width 0.25-4, labelZoom 0.0625-4),
coarse step 5 (21 stops; labels-until-zoom was "way too granular" at
201). Generic toSliderPosition/fromSliderPosition replace the
labelZoom-only pair; sanitize clamps follow the same range constants;
size sliders display compact values ("2", "1.6", "0.75").
EOF
- Defaults per user: nodeSize 2, edgeWidth 1, arrowLength 2, labelSize 2
(labelZoom 0.5 stays — reads as the "1x" they confirmed good).
- Label size slider floor 3 -> 1 so the default 2 is not the smallest
value; sanitize clamp widened to match.
- Cluster hull glow (layered low-alpha fills) removed for now: it read as
jagged up close (straight-line hull approximation at high zoom) and
does not belong until the layout settles. The solid viewer-relative
border and the inverse-gated region labels stay. HULL_FILL_LAYERS
constant deleted; hullShapes cache carries only the stroke path now.
- Saved render-settings.json reset so the new defaults apply on next load.
EOF
The defaults were calibrated in the screen-fixed era; world-scaling renders
sizes at setting x k/kFit, so at the default label gate (2x fit) everything
read ~2x too big ("labels way too large even at the minimum"). Fit-zoom
normalizes layout scale away, so the fix belongs in sizes, not layout:
spreading nodes further apart in world units changes nothing at fit.
- Defaults: nodeSize 6 -> 4, arrowLength 5 -> 4, labelSize 12 -> 7.
- Clamps/slider ranges widened at the low end: nodeSize 1-14 (step 0.5),
edgeWidth 0.25-4 (step 0.25), arrowLength 1-15 (step 0.5),
labelSize 3-24 (step 0.5).
- GLOW_PAD 9 -> 6 (halo dominated small discs), REGION_LABEL_SIZE 18 -> 14.
- Tests updated for the new clamp bounds plus a low-end pass-through case
(403/403). The saved render-settings.json still shadows defaults by
design; the local copy was reset so the new defaults showed immediately.
Fixes the zoom frame-rate collapse and rebuilds draw() around a single
camera transform, per the user's Obsidian-style sizing proposal.
Bug fix first: the eased zoom leaked a queued rAF callback per frame --
advanceZoom applies each step through d3-zoom, whose zoom event called
markDirty() from inside the tick loop, scheduling an extra draw while the
loop scheduled its own. Draws multiplied per displayed frame; sustained
wheel ran at ~20fps against a 120Hz display. markDirty now skips
scheduling while inTick (verified: 62 frames/500ms during a 28x zoom
burst, matching pan).
World-space model: every draw call now runs in graph coordinates under
one setTransform(k). Sizes are defined as screen-px-at-fit and converted
once per geometry build (/ kFit), so nodes, node labels, arrowheads and
the hover halo scale with the graph as you zoom (0.25x painted area at
half zoom, exactly k^2) while edge widths and hull borders stay
viewer-relative hairlines (/ k). Cluster hull geometry (convex hull +
three offset fills + solid boundary -- dashes dropped for now) is built
as cached Path2Ds per data/visibility/settings change; camera moves do
zero geometry math, making zoom per-frame cost identical to pan. Label
wrapping runs once per build, not per frame. Region labels stay
viewer-relative (18px at any zoom) as UI chrome. Picking radius scales
with the world-sized discs.
Glide feel: ZOOM_EASE_MS 120 -> 45, convergence 0.1% -> 0.2%; a notch now
settles in ~260ms instead of ~750ms. Resize still paints synchronously.
Also fixes barrel-law lint violations from the settings wave (import
from @/src/core and @/src/server, not deep module paths).
Five refinements to the hand-rolled renderer, all user-directed:
1. Settings are SERVER-persisted (data/render-settings.json via new
GET/PUT /api/graph/render-settings) - one source of truth for every
browser, replacing localStorage. The client loads once after mount,
saves debounced on user changes, and never lets a slow GET clobber a
faster user (or mounting PUT defaults over the server copy). The
shape/defaults/sanitizer moved to src/core/render-settings.ts - pure,
shared by client, server store, and tests; every disk/network value
passes through sanitizeRenderSettings.
2. New "Label size" control (default 12px - the previous hardcoded size).
Font, wrap width, and line height all scale; verified live: +76k px
with the label gate open.
3. "Labels until zoom" slider reworked: log-scaled over 0.0625-4 so the
useful band (0.25-0.5 camera-ratio units) sits at 33%/50% of the
track instead of crammed at the far left of the old linear 0.25-4
slider. Display units are normalized: the default 0.5 reads as "1x",
0.25 as "0.5x". The underlying labelZoom semantics are unchanged,
so existing values survive.
4. Smooth wheel zoom: d3-zoom instant per-notch jumps (x1.2, bang) are
replaced by an eased glide - wheel events accumulate a cursor-anchored
target scale and each tick eases the camera toward it (exponential,
120ms constant, frame-rate independent). Pan/pinch stay d3-owned; a
pan cancels a glide in progress. Verified live: 8-frame glide.
5. Resize blanking fixed (graph vanished for the duration of a window
drag, reappearing ~300ms after). Cause: every resize event
reallocated the canvas backing store - which CLEARS it - and the
redraw was deferred to rAF, which live resizes starve via relayout,
so the canvas sat wiped for the whole drag. Fix: the ResizeObserver
callback now redraws SYNCHRONOUSLY after reallocating (verified via
a probe RO firing in the same delivery batch: canvas fully painted).
kFit (the label gate 1x reference) is also recomputed on resize from
the now-cached world bbox - the camera itself still persists.
tests/render-settings.test.ts: 10 tests pinning the sanitizer (garbage/
partial/clamps, legacy no-labelSize records) and the slider mapping
(default dead-center at 500, monotonic, exact round-trips, display
units). 402/402 green; typecheck, lint, build pass.
Implements Option B from docs/research/graph-rendering.md. sigma.js v3
(WebGL) is replaced by one canvas, one rAF loop with a dirty flag, and
d3-zoom for the camera — eliminating the entire bug class from the
sigma-era bolt-on architecture:
- Hull/node desync during pan (two independent render loops) is now
structurally impossible: hull fills, dashed hull strokes, edges with
arrowheads, node discs, region labels, node labels, and hover glow
are all drawn in ONE frame under ONE camera.
- The settings-change canvas-sizing race (everything shifting up-left)
is gone: settings changes only redraw; the canvas and camera persist
(verified live: camera identical to 15 decimals across a slider drag).
- The GPU floor is gone: idle schedules NO frames at all (verified via
pixel-identity over 600ms idle); pan redraws exactly once per frame.
components/graph-canvas.ts: GraphCanvasRenderer class plus pure helpers
(zoomRatioOf, labelAlpha, fitTransform, arrowGeometry, pickNearest).
The sigma camera-ratio convention (1 = fit, smaller = zoomed in) is kept
verbatim so persisted labelZoom settings survive the swap; node labels
and cluster region labels cross-fade over the same LABELS_FADE_SPAN
(0.5, unchanged). Picking is a nearest-node distance check with the
10px pad that replaces the old padded-picking vertex shader. Hull math
stays in components/cluster-geometry.ts, reused as-is. No ctx.filter,
ever.
graph-view.tsx keeps every feature: fetch/refetch on runs-empty, FA2
run-then-freeze layout, localStorage render settings + sanitizers,
search filtering with edge drops and card-close, detail card with
flipped anchoring and per-source provenance, settings panel, import
button, pipeline status chips. Only the renderer plumbing changed.
tests/graph-canvas.test.ts pins the pure math: ratio direction, label
gate + inverse complement (alphas sum to 1), fit transform padding,
arrowhead tip-on-disc-edge and symmetry, picking pad/ties/null cases.
Live-verified end-to-end on the real 338-node graph (dev server
port 3002): single canvas, paint at fit, hover cursor+glow+fade, click
card open/anchored/close, stage click, search filter to zero with hint,
label gate opens via slider and zoom, cluster-labels toggle, zoom-to-
cursor 0.56px anchor drift, camera persistence across settings, zero
idle repaints. sigma dependency removed from package.json; d3-zoom and
d3-selection added.
Note: the settings feed effect needed isEmpty in its deps — without it
the renderer mounted with settings=null and every draw early-returned
(blank canvas); caught live, not by typecheck.
Four research passes (two library fact-checkers, two prior-art explorers)
with all claims verified against shipped package code (npm tarballs of
sigma@3.0.3 and @sigma/layer-webgl@3.0.0), the sigma.js storybook source,
and primary docs. Nothing implemented yet — this is the decision document.
Headline verified facts:
- sigma v3.0.3 ships a PUBLIC createCanvas/createLayer API; our DOM
injection was improvising a worse version of it. Sigma's own
cluster-label storybook demonstrates our exact minimal-fix pattern
(custom layer + afterRender + graphToViewport).
- @sigma/layer-webgl contours render METABALL fields, not hull polygons;
no dashed strokes; one WebGL context per community (~16 browser cap).
Our visual is not achievable with shipped sigma layer programs.
- Nobody in the graph-tool world ships hull regions (Gephi, Gephi Lite,
Obsidian: node coloring only). Aesthetic references: ggforce
geom_mark_hull, Bostock's thick-stroke hull trick, Excalidraw frames.
- Canvas 2D at 338→low-thousands of elements is comfortably within
demonstrated 60fps territory; the cost driver is repaint count x
pixel area, not element count.
- Cluster-first layout: two-level pattern (quotient anchors -> per-cluster
layout -> compose) is well-trodden; LinLog FA2 is the one-afternoon A/B
candidate; no turnkey JS package composes it (~150 lines of glue).
Options scored: sigma + public 2D canvas layer (7/10, ~1d minimal fix),
hand-rolled d3-zoom + single Canvas 2D (8/10, recommended target),
PixiJS v8 (7/10). Full matrix, facts-not-to-rely-on list, and open items
in the document.
Three problems from the first cut (user report 2026-09-13), all rooted
and fixed:
1. Boundaries reshaping on zoom — an inward offset normal. In y-down
screen coordinates, monotone chain yields clockwise-on-screen hulls,
whose OUTWARD normal is (dy, -dx); the code used (-dy, dx), so the
14px padding pulled every hull vertex 14-20px INWARD. Zoomed in this
mostly passed; zoomed out, once a cluster projected smaller than ~2x
the pad, vertices crossed the centroid and the polygon self-
intersected — spiky garbage re-forming every frame. Geometry moved to
components/cluster-geometry.ts (exported, unit-tested) with the
corrected normal; tests pin the direction (offset moves every vertex
AWAY from the centroid; stays sane when the hull is smaller than the
pad; hull orientation asserted clockwise-on-screen).
2. GPU ~100% + audible coil whine on any interaction — the hull redraw
was hooked to sigma's afterRender, which fires for EVERY render
including ~a dozen hover-animation frames per hover, and each redraw
rasterized ~32 fills through ctx.filter blur(12px), a GPU-heavy
full-canvas filter. Now: redraws happen ONLY on camera updates and
resizes (hover leaves the hull canvas untouched — verified live),
and the blur is replaced with layered low-alpha fills (soft edge at
a fraction of the cost, no canvas filter at all).
3. Cluster labels cluttering the view — a persisted 'Cluster labels'
checkbox in the rendering settings (hulls stay, text toggles).
377/377 tests, typecheck, lint, build green. Live-verified: hover does
not repaint hulls, zoom does; far zoom-out yields 3 contiguous tinted
regions (no fragmentation); toggle clears/returns labels and persists.
Two custom canvas layers around sigma's own stack (DOM order: hulls
behind everything; region labels above nodes, below node labels).
Per cluster (vision section 1): convex hull (monotone chain) of member
viewport positions, offset outward 14px, rounded corners, a blurred
low-opacity fill (ctx.filter, graceful degrade to flat fill) plus a
dashed boundary stroke, and the LLM label at the centroid. Stable hue
per cluster id (content hash) — a region keeps its color across
reshuffles because the id only changes when the members do.
Geometry is a pure function of positions at draw time: every
afterRender re-derives hulls through the current camera, so pan/zoom/
relayout can never invalidate it (the boundary decision pays off: no
stored geometry, nothing to recompute after a future physics change).
The inverse gate: the cluster label layer's opacity is the exact
complement of the node-label layer's over the fade span of THE zoom
control (labelZoom, default 0.5x) — the two label systems never
coexist.
Live-verified in the browser: hulls painting behind edges; region
labels above nodes; zoom-in crossfades cluster 1 -> 0 as node labels
0 -> 1 (0.176 + 0.824 mid-fade); zoom-out restores; hull pixel count
changes with zoom (geometry follows the camera); zero console errors;
layers removed on renderer teardown. 372/372 tests, typecheck, lint,
build green.
Also named the real graph: one 11.5s labeler call minted all 32
region labels for the 338-node notes graph (Relational Databases & SQL,
Differential Forms, Sales as Uncertainty Reduction, ...).
Core-survival labelings (vision section 1, DECIDED 2026-09-13):
ClusterLabeling records in a new append-only ledger
(data/graph/cluster-labels.jsonl), immutable, id = clu_ + sha256 of the
sorted core so re-naming races and re-runs dedupe at the store seam.
matchClusterLabelings (core/clusters.ts, pure): a community keeps a
record's label while at least half its core survives in it; both sides
single-successor (a split keeps one name; greedy by overlap, ties to
the newer record).
The namer (src/agent/subagents/cluster-namer.ts): ONE singular cheap LLM
call (new 'labeler' role, glm-flash/low) names every new/changed
community at once, given member names + statements (capped at 12
members per cluster). refreshClusterLabels runs POST-EXTRACTION
(user-decided) from both front doors, fire-and-forget — naming failure
never fails the run; refreshes serialize behind a chain so parallel
imports never mint duplicate labels. import-notes.ts awaits
clusterNamingSettled before exit.
Payload: GraphSnapshot.clusters (>= MIN_CLUSTER_SIZE 3) joins detection
with labelings at read time — claimed clusters carry the record's label
and id; unlabeled ones carry a member-hash id and null label. Detection
is deterministic so no memo in v1 (ms at current scale; revisit at
vault scale).
Tests (372/372): matcher half-rule/split/merge/greedy/tie cases; naming
end-to-end through importNotes (one call, one record, claim on
re-refresh), naming failure non-fatal; snapshot derivation with
labeling join; 11 test pipelines now pin an explicit labeler seam.
detectCommunities(nodes, edges) in src/core/clusters.ts (new dep
graphology-communities-louvain): Louvain modularity over the undirected,
unweighted follows+supports structure (user-decided 2026-09-13). The
random-walk traversal runs on an inlined seeded mulberry32 so the same
graph always yields the same partition; community indices are still
unstable across edits, which is exactly why labelings (phase B) key to
sorted MEMBER SETS. follows+supports between the same pair collapse to
one edge; self-loops and dangling edges are skipped defensively; no
min-size filter here (the >=3 render threshold is a presentation rule).
Tests: two-cliques-plus-bridge split, determinism, singleton/empty
graphs, pair collapse, defensive skips, member sorting. 363/363.
Real-data smoke over the current notes graph: 338 nodes / 211 edges ->
138 communities, modularity 0.95, and the big ones are exactly the
learn-session topics (databases, differential forms, sales, servers,
vim, copulas...). 32 communities clear the >=3 render threshold.
The most suitable default is the stricter end — labels only once
zoomed in past 2x fit (user-decided 2026-09-13). The slider window
extends below it (0.25x) so the default is no longer the floor.
The label-visibility zoom threshold (was the LABELS_CAMERA_RATIO=1
constant) becomes the fourth persisted setting: labels appear when the
camera ratio is at/below it (1x = fit zoom; slider 0.5x-4x, shown with
an x suffix). Deliberately THE single zoom knob per the cluster design
(vision section 1): cluster labels will fade in over the same span as
node labels fade out, gated inversely on this same setting.
Live-verified: at a zoomed-out camera (labels previously hidden),
raising the threshold to 4x rebuilds and re-shows labels without any
camera movement; resetting restores the fade. 355/355 tests,
typecheck, lint clean.
Three sliders behind a gear in the search panel (user-decided
2026-09-13), persisted per browser in localStorage with clamped
sanitizing on load. Default node size dropped 8 → 6 for dense graphs.
Mechanics: the mount effect is split — the data effect builds the
graphology graph + one frozen forceAtlas2 pass per snapshot; the render
effect mounts sigma and re-runs on settings change only. Sizes are
stamped onto the same graph instance (layout never recomputed), and the
arrowhead program is constructed per rebuild with ratios solved from
the settings (head px = thickness × lengthToThicknessRatio × 2; wideness
keeps the stock 0.8 shape). The camera position carries across rebuilds
via a ref, so a slider tweak never re-fits or re-lays-out the view;
useDeferredValue smooths slider drags.
Live-verified in the browser: sliders persist to localStorage, the
renderer rebuilds with zero console errors, hover/picking survives the
rebuild, and a zoomed-out camera stays zoomed out across a settings
change (labels-gate opacity as the observable). 355/355 tests,
typecheck, lint, build green.
Diagnosis from the real 25-file notes import (user-pasted session logs):
almost every run hit 'Validation failed for tool submit_findings/review:
edges must have required properties edges' — the model (glm flash, low
thinking) omits top-level fields when they're empty, and pi-ai's
structural JSON-Schema validation runs BEFORE the zod layer, so each
omission cost one of the extractor's maxTurns 4 (full payload
regeneration) and occasionally burned out the run entirely (the two
FAILED imports).
The model is technically wrong, but the schema provokes it:
required-but-usually-empty arrays are a known weak-model anti-pattern.
- lhTool's model-facing declaration now converts with io: "input"
(the declaration describes what the MODEL sends): .default() fields
drop out of required. No-op for teacher tools (none use defaults).
- findingsSchema.edges / auditSchema.dropped: .default([]) — omission
means "none"; attribution: .default({quiz:{}}) for quiz-free sessions.
- Prompts hardened in all three agents: include every top-level field,
never omit because empty.
Structural probe: validateToolArguments now passes the exact pasted
failure shape; zod re-parse in execute applies the defaults. 355/355
tests, typecheck, lint, build green. (lh-tool.ts is the shared engine
seam — flagged to master: io:"input" is the correct mode for every
model-facing declaration, teacher path verified unaffected.)
\ding{55} = U+2717 BALLOT X — I'd extracted U+2715 (\ding{53}, a
different multiplication cross) last round by mistake. Marks now match
pifont exactly: check \ding{51} (U+2713), x \ding{55} (U+2717).
The same extractor→auditor→merger pipeline now runs over raw personal
notes in arbitrary formats (vision §1, DECIDED 2026-09-13):
- NotesSource records (data/graph/sources.json, deterministic id from
path hash → idempotent re-imports) play the chatId role in mappings,
checkpoints, and observation events; usage lands on the source record
(appendSourceUsage, idempotent per startedAt, kept across upserts).
- mergeFindings is chat-store-free: callers pass MergeSource
{messages, foldedAt} — sessions pass the chat record, notes pass
messages:null (no quiz-event rebuild; a deterministic rebuild needs
structured quiz tool parts).
- Second extractor prompt (NOTES_EXTRACTOR_INSTRUCTIONS) for general-
purpose extraction; understanding judges the AUTHOR's grasp. Auditor
wording input-agnostic. extract/audit share one generic runSubmitAgent.
- graphSnapshot.fileSources (srcId→filePath); notes sources render as
static path entries in the detail card (FileTextIcon, never a link).
- One shared pipeline instance (exported from chat-handler): same
single-flight registry, live status chips (label = file path), usage
accounting, and true abort for both front doors.
- Transport: POST /api/graph/import (multipart, 3-wide worker pool) +
Import notes… button in the graph view; scripts/import-notes.ts for
batch runs (concurrency 4).
- real-data-extraction/ gitignored (raw import sources never committed).
355/355 tests; typecheck, lint, build green.
components/ui/glyph-mark.tsx: extracted vector outlines from the system
Zapf Dingbats font — U+2713 (LaTeX \ding{51}, the hand-drawn check) and
U+2715 (\ding{55}, the bulky cross) — as static SVG geometry on an
ink-bounded viewBox with a y-flip path transform (inverse of font axes).
This matches the pifont shape LaTeX renders, not a random glyph. Colors
stay: green/red like the verdict chips; the span keeps data-mark pins.
- Option rows: items-start → items-center — the icon span's top-misalign
came from the row alignment; now marks center across the row.
- The literal ✓/✗ look (owner preference over the round-3 lucide set)
re-rendered as hand-rolled SVG with thick rounded strokes (strokeWidth
3, round caps/joins) — the glyph shape, as a true vector, sized like
the row text. Test pins updated to data-mark=check|x.
The muted 'Why' box's px-3 double-inset with card p-4 — the W-label sat
deeper than the option-box edges. px removed (py-2 stays for vertical
air); the label now aligns with everything else on the left edge. The
rounded bg-muted box is bg-matched so its padding was invisible anyway.
Community detection as the graph's at-a-glance structure: Louvain on
graphology, seeded; structure-based (never layout-based); entirely
inferred from the graph, entirely separate from extraction; size-
gated (>= ~3 nodes); named by one cheap cached LLM call per new/changed
cluster.
Labeling persistence — the core-survival rule (user-decided): each
append-only labeling record stores the label + the member set it was
produced with (the core); a recomputed community keeps its label
while >= half its members are from that core, relabels under half
(new record appended, old records stay as history); single successor on
splits/merges; color keys to the labeling record.
Rendering: dashed hull + blurred low-opacity fill behind everything,
per-cluster stable hue, cluster labels inverse-gated against the node
label span (domain view zoomed out / proposition view zoomed in);
boundary geometry computed at draw time (pure function of member
positions) so future physics never invalidates it; inter-cluster edges
stay visible (meaningful bridges).
Multi-scale clusters (hierarchical Louvain dendrogram) deliberately
parked. Open questions: boundary aesthetic/label placement (visual
testing), threshold tuning (synthetic benchmark — now the primary
clustering-quality validation, sparsity being the honest risk).
- ✓/✗ in the answered card: replaced literal glyphs with lucide CheckIcon/
XIcon (strokeWidth 2.5) inside a fixed size-4 flex box — text elements
never carried the right size/vertical alignment, vectors do. Test pins
updated (lucide-check/lucide-x classes).
- All user-typed INPUT text now matches markdown output: lh-prose class
replaces the earlier font-prose (family-only) band-aid on the composer,
question-dock text/thinking/Other fields, home topic input, model search
box, and answered-card 'your thinking'/free-text records. lh-prose sets
family+size (16.5px) so placeholders inherit correctly.
- Home view: StatsBar under the composer (model+thinking control) — the
DOCK geometry matches the chat view (max-w-[46rem] px-4) so nothing
shifts when a session starts; pre-chat selection persists via deferred
save (ModelControl chatId-empty) + PUT after POST /api/chats.
- Answer marks (✓/✗): same visual size as row text, vertically centered;
colored green-500/red-500 matching the verdict chips.
- All user-typed inputs (composer, question-dock text/thinking/Other,
answered-card 'your thinking' and free-text records) now render
Source Sans 3 via the font-prose utility. Placeholders inherit the
family via the element.
- Answered-card explanation: markdown wrapper made truly inline so the
'Why:' label + the paragraph sit on one line — removes the oversized
leading gap.
- New session: literal '+' glyph removed — one lucide PlusIcon (the
duplicated glyph was PlusIcon + text '+').
- Top bar: a small pointer-events-none gradient under the header eases
the scroller into the bar exactly like the dock's bottom fade.
- Option subtext (opt.description): .lh-prose — Source Sans 3; 'I don't
know' copy stays JetBrains Mono.
- 'I don't know': now layout-matched with a radio circle; dashed border
preserved as the deliberate non-answer.
- Separator spacing: options group pb-1 → pb-2 so the gap before the
divider equals the 8px inter-option gap.
amosblomqvist