2 Commits

Include renames

Author	SHA1	Message	Date
amosblomqvist	b1d4eadb93	embedNewNodes: usage-on-error + pay-once (partial-failure accounting) ... Teacher's Wave-6 review flagged a money-ledger hole: a mid-batch embed throw discarded the spend of batches that had already landed (extraction only sees success results). Owner ruling 2026-09-23 (option A, pay-once): the landed vectors persist too — a retry embeds only what remains, so each text is billed exactly once. - embedNewNodes never throws on embed failures: per-batch try/catch, result gains `failed` (error message), `embedded` counts what actually landed, and usage carries the completed batches' steps either way (usage-on-error). - Zero-landed failures write nothing — the old ledger (its own model/dim) survives untouched. Stale-wipe semantics unchanged. - scripts/embed-graph.ts: honest failure output (lands count, not the in-memory total), exit 1 on failure. - tests: 4 new cases — mid-run failure (32 land, 8 remain; retry resumes and re-bills only the rest), first-batch failure (no write, no spend, old vectors survive), success carries no failed key. Live-verified on port 3004: bogus-model 400 from OpenRouter → exit 1, ledger untouched; healthy re-run embeds 0 (no re-spend); reset scratch setting to null. Gates: typecheck, lint, 899/899 tests, build.	2026-09-23 12:21:36 +03:00
amosblomqvist	0a4a9a11de	BYOK provider embeddings (wave 6 chunk B): kill local transformers ... Replace the @huggingface/transformers local embedder (~316MB ONNX cache) with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id" slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per call from settings + the auth store so key rotations land immediately. - src/server/embeddings.ts rewritten: slug grammar, injected-fetch adapters (OpenRouter input_type, Gemini taskType batching, OpenAI index-ordered rows), honest resolution (null = skip-with-info, set model with missing key = honest error), embedNewNodes with model-change re-embed wipe + ghost pruning + usage steps, federated catalog (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse path 404s; OpenAI text-embedding- filter; Gemini embedContent filter). - GET /api/models/embeddings: federated catalog, empty when no keys. - Extraction tail records stages.embedding usage into the run's money channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the sync-generated block), callCost falls back to it; cost-summary folds the fourth stage; graph-context injection resolves its embedder per run, fails honestly when unconfigured, records query tokens. - Settings: GraphSettings.embeddingModel (null factory), section parser validates slug shape (id may contain colons — OpenRouter :free suffixes, live catalog fact). - Removal: @huggingface/transformers uninstalled, PATHS.modelCache and onnx tracing entries deleted, @huggingface tracing-excludes guard added, stale sharp/libvips comments updated to historical. Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter + 3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5 model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects bad slugs with 400. Gates: typecheck, lint, 895/895 tests, build.	2026-09-23 03:53:57 +03:00