Louvain only ever found the reasoning chains (the graph is disconnected
by nature — every connected component became its own cluster) and
edge-orphans could never join anything. Clusters are now RELATED TOPICS:
propositions grouped by meaning.
- Embedder (src/server/embeddings.ts): in-process transformers.js v4
running EmbeddingGemma-300M q8 (MTEB 68.4, ~300MB one-time download),
MRL-truncated to 512d, "title: none | text:" prefix on 100% of inputs,
L2-normalized so cosine = dot. Model/dim mismatch voids the ledger
(full re-embed). Injected at the composition root — the agent layer
never imports the server layer (architecture law).
- Ledger (data/graph/embeddings.json): model + dim + vectors + the
persisted cluster partition. Reclustering runs at the END of every
embedding batch (post-merge hook, both front doors) — never per read
(the agglomerative pass is ~1.5s at 338 nodes).
- Clustering (src/core/semantic-clusters.ts): hand-rolled average-linkage
agglomerative over cosine (npm HDBSCAN is an abandoned alpha). Stragglers
stay singletons by default; attachStragglers is the all-in toggle.
Threshold 0.60 swept on the real graph: 26 topics, 74/90 edge-orphans
land in real topics, 10 stragglers.
- Consumers (snapshot, settle, namer, replay) read the persisted
partition — one cluster concept everywhere; core-survival label
matching unchanged (20/26 topics kept labels).
- Louvain retired: detectCommunities + graphology* deps deleted
(graphology-louvain had leaked into the client bundle since the
clusters wave); scripts/bench-graph.ts (dead FA2 bench) removed.
- scripts/embed-graph.ts: backfill/cold-start (338 nodes in 150s
including the one-time model download).
The graph's follows/supports edges encode derivation (premise ->
conclusion); the layout can now express that rank spatially, as a
session-only playground toggle (nothing promoted yet):
- inferenceDepths: longest chain from a base node, CYCLE-SAFE by
SCC condensation (Tarjan, iterative) — the DAG property is typical,
not guaranteed; cycle members share their component's depth (mutual
support = equal footing). O(V+E), pure, deterministic.
- DepthAnchor force: per-node target scalar — vertical mode rows
(baseline at the anchor centroid, arrows up), radial mode rings
(foundations center, outer boundary = edge of knowledge). The rank
is computed, not simulated: pairwise up-forces can't give baselines.
- Mode projections: gravity x-only in vertical / dropped in radial;
cluster pull x-only in vertical / tangential-only in radial; the
anchor re-blend preserves user arrangement in the FREE dimension
only (x / angle) — meaning owns the constrained axis, memory owns
the free one.
- setPhysicsTuning reheats (alpha 1): toggling a mode re-forms the
graph live and commits the new rest state; drag release is springy
along the constrained axis (node returns toward its level/ring).
- Sweep on the real graph: strength 0.8 = 215/216 arrows correct with
organic sag; 1.5 = crisp rows. 174 base nodes, max chain depth 7,
zero cycles in current data. replay-layout --depth=<mode> bakes a
mode into the ledger for rest-state experiments.
The 'virtual node' realized as a centroid force: two allocation-free
O(n) passes per tick (membership resolved once at sim creation;
Float64Array sums). Measured indistinguishable from force-off at 338
nodes (271ms vs 272ms per 400-tick settle).
- createGraphSimulation: SimOptions.clusterPull + linkStrength +
velocityDecay (physics controls plumbing); degree-scaled link
stiffness so linkStrength=1 is exactly d3's default
- settleGraphLayout split into layout-settle.ts (it computes the
Louvain partition for the pull); layout.ts stays browser-barrel-pure
- barrel no longer re-exports clusters.ts: graphology + louvain are
fully out of the client bundle (verified against built chunks)
- replay passes the per-batch partition; CLUSTER_PULL_STRENGTH=0.15
tuned by sweep: cluster spread 63->36 world units, centroid
separation and total span unchanged (no blob merge, no collapse)
One engine owns the equilibrium: the merge settle and (next commit)
the live drag both run createGraphSimulation — d3-force link springs,
1/d charge, weak centroid gravity — so a drag's rest state IS the
server's rest state. The FA2-era anchor mechanism survives as a
post-tick POSITION-SPACE re-blend (retention 0.85): d3's forceLink is
a Hooke spring (strength*(d-L)*alpha), which overpowers any force-space
anchor at long range — the re-blend is alpha-independent and always
wins as the sim anneals (the 675-unit drift bug). Gravity retuned for
d3's 1/d many-body falloff (bootstrap 0.2 / incremental 0.12: cluster
spacing lands ~400 world units, the FA2-era density). Rim orphans seed
(and anchor) at 1.7x the CONNECTED core's mean radius — a max-span-based
rim fed back on itself and ran the layout away to 6000 units. Replay:
338 positions in 1.6s, old-node shift within a batch 4.8 (FA2: 23).
The layout now EVOLVES instead of reshuffling on every extraction. A pure
core module (src/core/layout.ts) warm-starts forceAtlas2 from the positions
ledger (data/graph/positions.json, read-modify-write like sources.json,
updatedAt stamp for future live-physics conflict handling), re-blends every
previously-positioned node toward its anchor each iteration (retention 0.85
- old placements stay recognizable, strong forces still win), seeds new
nodes near their positioned neighbors, and seeds orphans on the rim outside
the core so they read as unconnected knowledge.
- Port/store: GraphStore.readPositions/writePositions; snapshot carries
positions when the ledger exists (optional - fresh graphs unchanged).
- Extraction pipeline settles synchronously after every merge (chat and
notes paths), before usage persist - the next /api/graph fetch already
reflects new nodes.
- View: the ledger IS the layout; the local one-shot FA2 pass survives
only as a fallback when no ledger exists.
- scripts/replay-layout.ts: simulates organic growth by replaying the
current 338-node graph one imported file at a time (real import order,
edges activate when both endpoints exist). Real run: 338 positions in
0.3s, 90-degree-0 orphans collecting at mean r=338 vs connected r=194.
- Also restores the cluster region fills (hull background colors) that
6692b19 had removed on a misreading of "no glow".
Live-verified: fit zoom derives from the ledger bbox, pickAt aligns with
ledger positions at every probe, hover/click/detail-card work, 408/408
tests, typecheck/lint/build green.
Future live physics stays cheap by design: the same pure function is the
merge-time settler and the future client simulator; a pinned node is just
an anchor with full retention.
amosblomqvist