From 0513b1555cd6eed9577abca870b0ac67cba34f72 Mon Sep 17 00:00:00 2001 From: amosblomqvist Date: Sun, 13 Sep 2026 04:59:56 +0300 Subject: [PATCH] Research: graph rendering & cluster-first layout (decision basis) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four research passes (two library fact-checkers, two prior-art explorers) with all claims verified against shipped package code (npm tarballs of sigma@3.0.3 and @sigma/layer-webgl@3.0.0), the sigma.js storybook source, and primary docs. Nothing implemented yet — this is the decision document. Headline verified facts: - sigma v3.0.3 ships a PUBLIC createCanvas/createLayer API; our DOM injection was improvising a worse version of it. Sigma's own cluster-label storybook demonstrates our exact minimal-fix pattern (custom layer + afterRender + graphToViewport). - @sigma/layer-webgl contours render METABALL fields, not hull polygons; no dashed strokes; one WebGL context per community (~16 browser cap). Our visual is not achievable with shipped sigma layer programs. - Nobody in the graph-tool world ships hull regions (Gephi, Gephi Lite, Obsidian: node coloring only). Aesthetic references: ggforce geom_mark_hull, Bostock's thick-stroke hull trick, Excalidraw frames. - Canvas 2D at 338→low-thousands of elements is comfortably within demonstrated 60fps territory; the cost driver is repaint count x pixel area, not element count. - Cluster-first layout: two-level pattern (quotient anchors -> per-cluster layout -> compose) is well-trodden; LinLog FA2 is the one-afternoon A/B candidate; no turnkey JS package composes it (~150 lines of glue). Options scored: sigma + public 2D canvas layer (7/10, ~1d minimal fix), hand-rolled d3-zoom + single Canvas 2D (8/10, recommended target), PixiJS v8 (7/10). Full matrix, facts-not-to-rely-on list, and open items in the document. --- docs/research/graph-rendering.md | 227 +++++++++++++++++++++++++++++++ 1 file changed, 227 insertions(+) create mode 100644 docs/research/graph-rendering.md diff --git a/docs/research/graph-rendering.md b/docs/research/graph-rendering.md new file mode 100644 index 0000000..78273e6 --- /dev/null +++ b/docs/research/graph-rendering.md @@ -0,0 +1,227 @@ +# Graph rendering & cluster-first layout — research findings + +*Compiled 2026-09-13 from four research passes (library fact-check ×2, prior-art ×2), all +claims source-verified by subagents against shipped package code (`npm pack` tarballs of +`sigma@3.0.3`, `@sigma/layer-webgl@3.0.0`), the sigma.js GitHub storybook source, and +primary docs. Speculation is marked where it appears. This document is the decision +basis for the rendering rebuild; nothing has been implemented yet.* + +--- + +## 1. Requirements (as of this research) + +Current symptoms to eliminate: +- Hull layers lag behind nodes during pan (desync). +- Settings changes shift hulls up-left until the next interaction (canvas-sizing race). +- GPU ~50–100% during pan/zoom (redundant full-canvas 2D repaints per camera event + + WebGL re-render per frame; the earlier `ctx.filter: blur` per frame caused coil whine). +- Stale/garbage boundary shapes at extreme zoom-out. + +Feature set: +- Circles + directed arrows; precomputed layout (no runtime physics). +- Cluster regions: hull boundary around each community, dashed stroke, soft translucent + fill, stable color per region, region name label. +- Inverse zoom gates: node labels when zoomed IN, region labels when zoomed OUT, + cross-fading over a shared span. +- Pan, zoom-to-cursor, hover glow, click → detail card, edge-type filtering, camera + survives UI changes. 60fps, minimal GPU, ordinary laptop. + +New direction (user, 2026-09-13): +- **Cluster-first placement**: layout depends on clustering — clusters placed as units, + nodes within a cluster close together (compact, natural hull shapes as a consequence). +- **Freeform experimentation**: a dev-only playground to try layouts/shapes/styles + against real and synthetic data; renderer must be dumb, data-driven, swappable. + +Scale: 338 nodes / 211 edges today; low thousands expected (notes imports). + +## 2. Verified facts about the current stack (sigma.js v3) + +- **v3.0.3 ships a PUBLIC custom-layer API** — `createLayer` / `createCanvas` / + `createCanvasContext` / `killLayer` (typed in shipped `.d.ts`). Our current + `firstChild` DOM injection was improvising a worse version of a supported API. +- **Sigma sizes and DPR-scales ALL registered canvases** in its own `resize()` — + using the public API eliminates the canvas-sizing race by construction. +- **`afterRender` fires once per rendered frame** (rAF-debounced `scheduleRender()`). + Camera motion → full re-render each rAF. **Hover is cheap**: it only redraws the + `hoverNodes` WebGL layer + `edgeLabels` canvas via a separate rAF; no `afterRender`, + no full re-render. Sigma is strictly event-driven: no continuous loop, no rendering + when idle (verified: zero `setInterval`, all rAF sites are schedulers). +- **`renderer.graphToViewport()` is public and typed** — the sanctioned graph→pixel + path. `renderer.matrix` is **private**: building a `ctx.setTransform()` from it would + be guessing at internals; per-vertex `graphToViewport()` is the supported route. +- **The official cluster-label storybook example is exactly our pattern**: + custom DOM layer + `afterRender` + per-cluster `graphToViewport()` re-projection, + maintained in the sigma repo. Our minimal-change fix path is a demonstrated usage. +- Real v3 layer order (verified in source, NOT in docs): + `edges` (WebGL) → `edgeLabels` → `nodes` (WebGL) → `labels` → `hovers` → + `hoverNodes` (WebGL) → `mouse`. + +### `@sigma/layer-webgl@3.0.0` (official background-layer package) +- `bindWebGLLayer` inserts a background WebGL layer (`beforeLayer: "edges"`), redrawn + on `afterRender` with the same camera matrix — fully in-loop, no desync. +- **`createContoursProgram` renders METABALL-style implicit fields, not hull polygons** + (verified from the fragment shader: fullscreen quad, summed radial falloffs, + analytic contour lines via `dFdx/dFdy`). Translucent fill: yes (arbitrary CSS color + with alpha). **Dashed stroke: NO** — solid anti-aliased line only; no dash parameter + exists. Per-community color: yes, but one program + one WebGL context per community, + and each runs a fullscreen fragment pass per frame (browsers cap ~16 WebGL contexts — + a problem for dozens of regions). +- **Decisive fact**: our exact visual (hull polygon + translucent fill + dashed + stroke) is **NOT achievable with shipped layer-webgl programs**. It would require a + custom `WebGLLayerProgram` (supported extension point, moderate effort, dashes DIY in + GLSL) — whereas Canvas 2D gives it for free with `ctx.setLineDash()`. +- v4-era programs (heatmap, density-splat, dashed/fill layer factories) are NOT in the + v3 package. + +### sigma v4 +- Real but **beta** (4.0.0-beta.5; docs site self-describes as alpha). Single `stage` + canvas + public `createLayer`, GPU picking with downsizing, SDF labels, depth + layers. Breaking migration. Not a production bet today. + +### Pan GPU cost at our scale +- Structural facts verified (event-driven idle; full re-render per camera frame; + synchronous `gl.readPixels` picking per mousemove pre-#1546, since fixed). +- **Absolute GPU cost of panning 338–3k nodes in v3: UNVERIFIED** — no public + benchmark exists; nobody has measured it. If it matters, we measure it ourselves. + +## 3. Verified facts about the candidates + +| Candidate | Renderer | True hull fill+dash | Picking | Camera | Maintenance | License | Fit | +|---|---|---|---|---|---|---|---| +| sigma v3 + public 2D canvas layer | WebGL (+our 2D) | yes (our canvas, free) | GPU color-pick | built-in | active (3.0.3, 2026-04) | MIT | 7/10 | +| sigma v3 + layer-webgl contours | WebGL | **no** (metaballs, solid border) | — | built-in | active | MIT | 6/10 | +| sigma v4 + layers | WebGL, SDF | via custom program (beta risk) | GPU picking | built-in | **beta** | MIT | 6 (risky) | +| d3-zoom + Canvas 2D (hand-rolled loop) | Canvas 2D | yes, trivially | distance check | d3-zoom (verified math) | d3 very active | ISC | **8/10** | +| PixiJS v8 + pixi-viewport | WebGL/WebGPU/Canvas | yes (Graphics; dashes via shim `pixi-dashed-line`) | built-in events | pixi-viewport 6.0.3 | very active | MIT | 7/10 | +| cytoscape.js 3.34 | Canvas 2D | via bubblesets/layers extensions | built-in | built-in | active | MIT | 5/10 | +| vis-network 10.1 | Canvas 2D | **no** — clustering = supernodes (collapse, opposite of our regions) | built-in | built-in | sporadic | MIT | 2/10 | +| @cosmos.gl/graph (Cosmograph engine) | WebGL (luma.gl) | no (cluster = layout force only) | GPU | built-in | active | MIT | 3/10 | + +Fact nuances we must not over-trust: +- "Canvas 2D is GPU-rasterized": **half-true**. Skia may raster on CPU or GPU depending + on browser/OS config. Either way the cost driver is **repaint count × pixel area + (DPR²)**, not "Skia = GPU". +- Canvas 2D at 60fps for hundreds–low-thousands of elements: **supported by practitioner + evidence** (cytoscape is fully canvas-based into the thousands; 10k animated canvas + elements ≈ 55fps in one 2025/26 benchmark). Pixel area matters more than node count + at our scale. +- Obsidian's graph view renderer: **UNVERIFIED** — no primary source. Do not cite. +- PixiJS exact bundle size: unmeasured. yFiles: commercial, unverified details. + +## 4. Prior art for our specific features + +### Region/hull visuals +- Nobody in the graph-tool world ships our region aesthetic: Gephi desktop, Gephi Lite + (the flagship sigma app — zero hull code in its tree), Obsidian — all partition by + **node coloring only**. The region feature is a differentiator, not a copied pattern. +- The aesthetic reference: ggforce `geom_mark_hull` (expand + round the hull, low-alpha + fill, integrated label placement). +- **Bostock's thick-stroke hull trick**: one path, wide stroke (e.g. 32px) + + `linejoin: round` = padded, rounded hull with zero polygon-offset math. Replaces our + offset-normal code outright. +- Named-region UX validation: Excalidraw frames / tldraw regions (dashed titled + container around content). +- Closest feature twin: **Cosmograph** — documents "cluster labels when zoomed out, + point labels when zoomed in" as a first-class use case (our exact inverse gate). + +### Zoom-gated labels +- Consensus pattern = map-style per-layer zoom ranges (Mapbox/MapLibre + `minzoom`/`maxzoom` on symbol layers). State labels at far zoom, city labels at near + zoom is the standard cartographic solution; our inverse gates are the same mechanism + with opposite ranges. Sigma's own `labelRenderedSizeThreshold` is the zoom-in gate. +- Cross-fade + hysteresis over a shared span: no single canonical writeup (medium + confidence); nearest established practice is map label collision systems. + +### The sync pattern (fixes our desync class structurally) +- Canonical and repeatedly documented: **one rAF loop with a dirty flag; input/DOM + events only mutate state.** All layers drawn in the same frame, under the same + camera transform — either same canvas (`setTransform` once, draw back-to-front) or + stacked canvases all redrawn by the same loop. + Sources: browser.engineering, MDN "Optimizing canvas", web.dev canvas performance. + +### Picking +- Canvas 2D at our scale: nearest-node distance check per pointer event is the norm, + trivially fast (µs for 3k nodes; spatial grid only past ~10k). + +## 5. Cluster-first layout (the new direction) + +- **Two-level pattern is well-trodden**: place cluster anchors first (quotient graph — + one super-node per community, Gephi's "Groups by Partition" plugin is the desktop + prior art), layout nodes within each cluster, compose by translating local layouts + to cluster anchors. Bostock's Clustered Force Layout lineage; cosmos.gl's cluster + force verified from source (per-frame center-of-mass pull, or user-pinned positions — + cluster anchors as data independent of member layout). +- **graphology FA2 has NO cluster-aware settings** (full settings list read; gravity is + global). **LinLog mode** (`linLogMode: true`) is the documented "communities separate + tightly" switch — a one-afternoon A/B candidate before any custom pipeline. +- **No turnkey JS package does the composed two-level pipeline** — assemble from + graphology primitives (~150 lines of glue: quotient build, per-cluster FA2 with + circular init, compose transform, `noverlap` cleanup). The combination is our + synthesis from verified primitives. +- Deterministic given fixed init positions (no RNG dependency in FA2). +- **Server-side at extraction time is the better fit** for our architecture: positions + become data, rendering stays dumb. Browser worker fallback for live re-layout. +- Harness precedents: yFiles layout playground (polished), Observable/bl.ocks lineage + (notebook = graph + sliders + canvas), react-sigma layouts example. No canonical OSS + playground for our stack — copy the config-driven pattern (algorithm dropdown + + parameter sliders + seed input + live canvas). + +## 6. Options (the decision) + +**Option A — Stay on sigma v3; mount the hull canvas via the public +`createCanvas` API; redraw on `afterRender` via `graphToViewport()`.** +Supported, minimal-change, demonstrated by sigma's own cluster-label storybook. Kills +the desync, the sizing race, and the multi-event redraws (once per frame, sigma-sized). +Our exact visuals stay (dashes free in 2D). Keeps sigma's camera/labels/picking +machinery. Residual: full-canvas 2D repaint per interaction frame remains (raster cost, +mitigable with move-gating like sigma's own `hideLabelsOnMove`); WebGL + 2D raster +still both run per frame. Est: ~1 day. + +**Option A′ — sigma + custom `WebGLLayerProgram` for hulls.** Supported extension +point, but dashes are DIY GLSL and we'd rebuild in shaders what Canvas 2D gives free. +No reason given A exists. Est: several days, for a worse iteration loop. + +**Option B — d3-zoom + hand-rolled single Canvas 2D renderer.** +One canvas, one rAF + dirty flag, world-space geometry + one transform, all layers +back-to-front in the same frame. Lowest GPU/CPU, total control, perfectly matches the +new requirements: renderer is a dumb draw function over data (layout experiments can't +fight it), playground is trivial, cluster-first layout slots in as pure data. Costs +~600 lines + camera feel + picking + label gating owned by us. Est: ~1 focused day. + +**Option C — PixiJS v8 + pixi-viewport.** Cleanest scene-graph answer (one ticker, +sync by construction), WebGPU headroom. But: dashes need a third-party shim, no graph +semantics, bundle, more machinery than Canvas 2D needs at our scale. Est: 1–2 days. + +**Recommendation**: **Option B as the target** — it is the only option that structurally +matches all three requirement sets (bugs, cluster-first placement, freeform +experimentation), scored highest (8/10), and removes the vendor-internals class of +problems permanently. **Option A is a legitimate ~1-day stabilization** if we want the +current bugs dead immediately while deferring the rebuild; it reuses all our existing +visual code and is the officially sanctioned pattern, but it keeps us on sigma's +opinionated machinery and the experimentation direction will keep pushing against it. + +**Layout work rides along independent of renderer choice**: first A/B +LinLog-only FA2 (one afternoon), then the two-level pipeline +(quotient → per-cluster FA2 → compose → noverlap) if needed, developed in the playground. + +## 7. Facts we should NOT rely on + +- "Obsidian's graph view uses renderer X" — unverified. +- "Sigma's documented canvas DOM order" — source-only, implementation detail, changed + in v4. +- "Canvas 2D is always GPU-rasterized" — false as stated. +- "Sigma v4 is ready" — beta. +- "Sigma pan cost at our scale is X%" — nobody has measured it publicly. +- "@cosmograph/cosmos is the maintained Cosmograph engine" — stale npm channel (Dec + 2024); current engine is `@cosmos.gl/graph` v3. +- Any PixiJS bundle-size number — unmeasured. + +## 8. Open items for the implementation phase + +- Measure pan GPU/CPU ourselves (Chrome profiler) before/after whichever path is chosen. +- Playground design: `/playground` dev-only route, config-driven (layout algorithm + dropdown, parameter sliders, seed input, synthetic + real datasets), same renderer. +- Hull styling decision inside Option B: Bostock thick-stroke trick vs offset polygons + (geometry already unit-tested in `components/cluster-geometry.ts`). +- Label-gate hysteresis: small dead-band in log-zoom to avoid threshold flicker. +- Whether layout runs server-side at extraction (persisted as data) or in-browser.