7a17230f2633559ea8bde7243fc9ccd046a959b8
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 amosblomqvist
|
c86430e8bf |
Wave 7 chunk B: remove the embeddings machinery — the ladder collapses
The owner's wave-7 ruling: embeddings die entirely (git history is the
stash); clustering is birth-session derived (chunk A, 950ddfd), labels
are session names, and the prober's graph tools replace injection.
- settings: GraphSettings collapses to { enabled } — embedding,
embeddingModel, clustering, labeling all die. Migration story: stale
record keys are UNREAD and dropped by the sanitizer's reconstruction;
the next graph-section PUT (replace-whole) persists the collapsed
shape. The settings-panel GraphSection renders one master row.
- deleted: src/server/embeddings.ts (adapters, resolveEmbeddingPortFor,
embedNewNodes, embeddingPort, embeddingModelCatalog), the
GET /api/models/embeddings route, scripts/embed-graph.ts, tests for
all three, and the /api/models/embeddings picker surfaces in the UI.
- prices: EMBEDDING_PRICES overlay + callCost fallback die. cost-summary:
the embedding stage fold dies — back to three stages (extraction/
audit/naming) per the owner ruling; historical stages.embedding spend
in old run records is simply not displayed.
- ports: EmbeddingPort/EmbeddingResult/EmbeddingTask die. The embeddings
LEDGER surface (readEmbeddings/writeEmbeddings + EmbeddingsLedger +
EMBEDDINGS_FILE + graph-deletion's vector pruning) STAYS — inert data,
kept only so graph-context.ts compiles until Teacher's pass removes
it; the seam dies in the follow-up interlude.
- chat-handler (approved cross-lane unwiring, wave-6 precedent): the
extraction pipeline's embedder becomes a documented no-op (nothing
embedded, nothing spent, ledger untouched) until Teacher's pass
removes the phase; the graph-context injection dispatch gate, the
graphContext deps wiring, the setContextRespawn seam, and
turn-service's triggerGraphContext are gone — injection is dead
code pending Teacher's file deletions.
- scripts: backfill-graph/import-notes pass the same no-op embedder;
replay-layout derives clusters from birthSessionPartition (the same
partition the real settle uses) instead of the embeddings ledger.
- settings-body: parseGraphSettingsBody reads the master switch only —
old-shape keys in a body are ignored, not rejected; bad shapes 400.
- next.config: the @huggingface tracing-exclusion guards die with the
last reference.
Cross-lane flags: extraction.ts/graph-context.ts keep stale embed
comments + the embed phase until Teacher's full pass (their parallel
interlude owns those files — zero overlap kept by design);
semantic-clusters.ts + its test stay one more round (extraction.test.ts,
Teacher's file, still imports it — they delete it with the namer cases);
ChatSettings.graphContextTokens + the graph-context-control UI stay
until Teacher+UI.
Gates: tsc clean, eslint clean, 885/885 tests, build compiles.
Live-verified on the port-3004 dev tree: PUT /api/settings/graph with
dead keys collapses on write, bad shapes 400, GET reads {enabled: true},
/api/models/embeddings 404s.
|
||
|
amosblomqvist
|
0a4a9a11de |
BYOK provider embeddings (wave 6 chunk B): kill local transformers
Replace the @huggingface/transformers local embedder (~316MB ONNX cache) with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id" slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per call from settings + the auth store so key rotations land immediately. - src/server/embeddings.ts rewritten: slug grammar, injected-fetch adapters (OpenRouter input_type, Gemini taskType batching, OpenAI index-ordered rows), honest resolution (null = skip-with-info, set model with missing key = honest error), embedNewNodes with model-change re-embed wipe + ghost pruning + usage steps, federated catalog (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse path 404s; OpenAI text-embedding- filter; Gemini embedContent filter). - GET /api/models/embeddings: federated catalog, empty when no keys. - Extraction tail records stages.embedding usage into the run's money channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the sync-generated block), callCost falls back to it; cost-summary folds the fourth stage; graph-context injection resolves its embedder per run, fails honestly when unconfigured, records query tokens. - Settings: GraphSettings.embeddingModel (null factory), section parser validates slug shape (id may contain colons — OpenRouter :free suffixes, live catalog fact). - Removal: @huggingface/transformers uninstalled, PATHS.modelCache and onnx tracing entries deleted, @huggingface tracing-excludes guard added, stale sharp/libvips comments updated to historical. Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter + 3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5 model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects bad slugs with 400. Gates: typecheck, lint, 895/895 tests, build. |
||
|
amosblomqvist
|
f9e3a4c43d |
feat(models): strong/cheap/vision model slots + multi-provider engine federation
Model slots wave (2026-09-20), chunk B:
- settings: new `models` section {strong, cheap, vision} slots (each
field independent, same shape as a role override); ROLE_SLOT maps all
nine roles to tiers (strong=prober+teacher, cheap=researcher/
demoBuilder/extractor/auditor/labeler, vision=visualMaker+pdfReader);
resolution chain per role: override → slot → factory pin.
- config: ModelSpec gains provider (all pins openrouter, the BYOK
default); MODEL_PINS in stores.ts now spans all nine roles (the strong
slot's factory rep IS the prober pin).
- engine: piModels() registers EVERY builtin provider catalog;
piEngineForSpec(resolved spec) is the new engine factory (pi-ai
resolves per-provider keys from env itself); piEngineFor stays as the
config-pin fallback. chat-handler resolves the conversation chain
(chat override → strong slot → pin) and injects engineFor into the job
runner, closing the old gap where per-role overrides never reached
job engines.
- /api/models: models federate ONLY keyed providers (providers surface
with keyPresent, env-wins, fresh per call); slots (resolved tier pins)
join rolePins; catalogDefaultsFor inherits the strong slot; the models
settings section gets the same catalog gate as chat.
- pricing: PRICES/CONTEXT_LIMITS uniformly keyed provider:id
(regenerated from ALL pi-ai catalogs, 1354 keys); usage stamps
(TurnUsage, JobUsage, extraction stages) switch to provider:id —
owner decision: no legacy compat, old stamps render unpriced.
|
||
|
amosblomqvist
|
3e263a18cc |
Graph pipeline cost metrics: capture naming spend, fold all channels, cost pill + panel
- Cluster namer bills the money channel now: per-step usage + model captured via onStep, persisted as NamingRunUsage records to data/graph/naming-runs.jsonl (idempotent per startedAt; failed/aborted runs persist too — money spent either way). callUsage promoted to extraction.ts module scope for reuse. - core/cost-summary.ts: the pure fold behind the pill — chat runs, notes-source runs, and naming runs into stage totals (extraction / audit / naming) + newest-first recent runs. Dollars INFERRED read-time via prices.ts, never stored; unpriced models excluded-but-counted (render '—', never a silent zero); tokens always sum. - GET /api/graph/costs (src/server graphCosts composition, thin route). - Graph view: bottom-left cost pill (cumulative dollars headline) + expandable breakdown panel (stage rows with tokens secondary, recent runs with time/title/merged/cost); fetched on mount, refetched when a pipeline run finishes; merged into one bottom-left stack with the running-pipeline pills. - Tests: pure-fold suite, naming capture wiring (full importNotes -> naming path), store idempotency + strict reads. |