The same extractor→auditor→merger pipeline now runs over raw personal
notes in arbitrary formats (vision §1, DECIDED 2026-09-13):
- NotesSource records (data/graph/sources.json, deterministic id from
path hash → idempotent re-imports) play the chatId role in mappings,
checkpoints, and observation events; usage lands on the source record
(appendSourceUsage, idempotent per startedAt, kept across upserts).
- mergeFindings is chat-store-free: callers pass MergeSource
{messages, foldedAt} — sessions pass the chat record, notes pass
messages:null (no quiz-event rebuild; a deterministic rebuild needs
structured quiz tool parts).
- Second extractor prompt (NOTES_EXTRACTOR_INSTRUCTIONS) for general-
purpose extraction; understanding judges the AUTHOR's grasp. Auditor
wording input-agnostic. extract/audit share one generic runSubmitAgent.
- graphSnapshot.fileSources (srcId→filePath); notes sources render as
static path entries in the detail card (FileTextIcon, never a link).
- One shared pipeline instance (exported from chat-handler): same
single-flight registry, live status chips (label = file path), usage
accounting, and true abort for both front doors.
- Transport: POST /api/graph/import (multipart, 3-wide worker pool) +
Import notes… button in the graph view; scripts/import-notes.ts for
batch runs (concurrency 4).
- real-data-extraction/ gitignored (raw import sources never committed).
355/355 tests; typecheck, lint, build green.
Community detection as the graph's at-a-glance structure: Louvain on
graphology, seeded; structure-based (never layout-based); entirely
inferred from the graph, entirely separate from extraction; size-
gated (>= ~3 nodes); named by one cheap cached LLM call per new/changed
cluster.
Labeling persistence — the core-survival rule (user-decided): each
append-only labeling record stores the label + the member set it was
produced with (the core); a recomputed community keeps its label
while >= half its members are from that core, relabels under half
(new record appended, old records stay as history); single successor on
splits/merges; color keys to the labeling record.
Rendering: dashed hull + blurred low-opacity fill behind everything,
per-cluster stable hue, cluster labels inverse-gated against the node
label span (domain view zoomed out / proposition view zoomed in);
boundary geometry computed at draw time (pure function of member
positions) so future physics never invalidates it; inter-cluster edges
stay visible (meaningful bridges).
Multi-scale clusters (hierarchical Louvain dendrogram) deliberately
parked. Open questions: boundary aesthetic/label placement (visual
testing), threshold tuning (synthetic benchmark — now the primary
clustering-quality validation, sparsity being the honest risk).
The extractor's per-session understanding notes were free-form strings;
the model freely appended em-dash justifications ('shaky — the learner
could state the headline distinction but calls the machinery behind it
foggy'), which overflowed the detail card's session rows and squeezed
the session names.
The note becomes a CLOSED named-word scale — solid / shaky / accepted /
resisted / not yet engaged (DECIDED 2026-09-08, no numbers) — enforced
by the zod enum in propositionSchema: a wrong-shaped note is rejected
at parse time and the agent retries. Justifications are dropped, not
stored (user decision: not necessary). Extractor and auditor
instructions updated to demand exactly one word.
Renderer: the session row keeps the session NAME as the main thing
(flex-1, truncates) and the level chip is bounded (max-w-28,
whitespace-nowrap, truncate) — overflow impossible even for legacy
prose notes in old ledgers.
Vision doc: §1 extractor bullet + event-ledger line updated; state
model line notes the five-class derivation was removed; amendments
table row added.
With real extracted data, full-sentence statements as canvas labels proved
unworkable: 10-line label blocks, illegible overprint clusters, bottom-edge
clipping. Per the user's 2026-09-08 decision:
- GraphNode/ExtractedProposition gain `name` — a very short, glanceable
label (glossary-headword style: "Will to power"), presentational only:
not unique, not identity, never precise. Set at creation; link-appends
never change it (first line defines; labels stay stable across sessions).
- `statement` keeps its full precision and becomes the identity anchor;
the detail card shows it under the name. `description` is dropped.
- Extractor coins the name (freshest session context); auditor carries it
through on create, rewording only clearly unglanceable names; linked
propositions keep the existing node's name.
- Renderer: canvas label = name; search matches name + statement;
detail card = name (bold) + statement (muted).
- Store guard requires name (pre-pivot node lines are skipped as legacy);
vision.md node-schema bullet + amendments table updated (2026-09-08).
Records the 2026-09-02 graph philosophy decisions, replacing the
2026-08-22 live-emission construction model (second reversal — back to
post-hoc, now deliberately tree-blind):
- Node theory: every node is a TRUE proposition (no kind field);
statement (minimal, context-relative) + description (required) +
aliases (verbatim phrasings); contradictions coexist — the
surrounding subgraph is the context; misconceptions enter as true
corrections, wrongness lives in the event ledger.
- Edges: follows (derivable) | supports (grounds without entailing);
no scalars, no encoded strength — derived at read time.
- Provenance: message-granular (chat, message) points on nodes and
edges, deep-linkable.
- Construction: free-form extraction fully decoupled from the
session's formalism. Extractor sees ONLY the session transcript
(tree-blind, message ids); auditor sees extractor output + the
global graph (faithfulness net + identity judgment, link-or-create,
aliases as phrasings); merger stays deterministic append-only.
- Event ledger: quiz outcomes deterministic from tool parts; NO
taught events; extractor records per-proposition understanding
notes (named-word scale, no numbers) — treatment TBD.
- Understanding: inferred, never encoded; state model (session-
independent, computed from dependency depth/strength) is pending
ideation — current five-class derivation is transitional.
Also updates: §2 plan tree (session scaffolding only), graph-as-context
(follows-edge subtree, event-derived hotspots), home page (message
provenance, transitional states), roadmap item 5, open questions
#2/#3 resolved, amendments table (+6 rows).
- Every session starts with the learner's knowledge graph as base
context (precision probing); selection problem noted: pedagogical
relevance != semantic closeness (structural neighborhood, diagnostic
hotspots, frontier strip); strategies to tune on data.
- Sub-agent architecture: async background workers with live activity
indicators; demo-builder TESTS code headlessly; visual-maker verifies
by looking (port of vault svg-maker); mermaid needs no sub-agent;
researcher tool for fact verification (port of pi web-search/web-fetch
+ interactive-subagents researcher prompt).
- Not-a-chat presentation principle marked LOW PRIORITY per user.
Margin model flaws (user feedback): fragment placement needs unreliable
routing; scattering kills the receipt moment and chronology; session
value is in the moment, revision is rare, HTML export deprioritized.
Evolved model: one chronological spine; learner blocks are intact typed
'interjection' cards; teacher replies as woven Q&A interludes
(quote-then-answer); anchoring is a link overlay (backlinks), margins
demoted to a computed view. Chat-ness dies by type asymmetry, not
relocation.
Asymmetry of artifact: the teacher writes the document (whiteboard), the
learner writes in the margins (voice/floor). Bubbles die; the unit is the
block; chronology yields to tree structure; Q&A becomes document
furniture; the composer becomes a response field. Presentation-layer
change only — the interaction model stays conversational.
Captures 2026-08-22 decisions: hybrid live-emission + consolidation graph
construction (reversing the vault doc's teacher-untouched rule), one
substrate merged across sessions, structured plan-tree tools (mermaid
demoted to ad-hoc), probe/teach phase separation with collapsed teach
checks, right-panel tree view, LaTeX delimiter switch plan, and a
dependency-ordered roadmap.
amosblomqvist