sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

BYOK provider embeddings (wave 6 chunk B): kill local transformers

Browse Source 
Replace the @huggingface/transformers local embedder (~316MB ONNX cache)
with an EmbeddingPort seam: graph.embeddingModel (nullable "provider:id"
slug) selects an OpenRouter/OpenAI/Gemini adapter, resolved fresh per
call from settings + the auth store so key rotations land immediately.

- src/server/embeddings.ts rewritten: slug grammar, injected-fetch
  adapters (OpenRouter input_type, Gemini taskType batching, OpenAI
  index-ordered rows), honest resolution (null = skip-with-info, set
  model with missing key = honest error), embedNewNodes with model-change
  re-embed wipe + ghost pruning + usage steps, federated catalog
  (OpenRouter /api/v1/embeddings/models — VERIFIED live, the inverse
  path 404s; OpenAI text-embedding- filter; Gemini embedContent filter).
- GET /api/models/embeddings: federated catalog, empty when no keys.
- Extraction tail records stages.embedding usage into the run's money
  channel; EMBEDDING_PRICES overlay in core/prices.ts (outside the
  sync-generated block), callCost falls back to it; cost-summary folds
  the fourth stage; graph-context injection resolves its embedder per
  run, fails honestly when unconfigured, records query tokens.
- Settings: GraphSettings.embeddingModel (null factory), section parser
  validates slug shape (id may contain colons — OpenRouter :free
  suffixes, live catalog fact).
- Removal: @huggingface/transformers uninstalled, PATHS.modelCache and
  onnx tracing entries deleted, @huggingface tracing-excludes guard
  added, stale sharp/libvips comments updated to historical.

Live-verified on port 3004: catalog federates 36 rows (33 OpenRouter +
3 OpenAI), embedNewNodes embedded 10 nodes via the free liquid/lfm-2.5
model (dim 1024, 435 prompt tokens, 1 call, $0), settings PUT rejects
bad slugs with 400.

Gates: typecheck, lint, 895/895 tests, build.
This commit is contained in:
amosblomqvist
2026-09-23 03:53:57 +03:00
parent a6338511d3
commit 0a4a9a11de
29 changed files with 1233 additions and 841 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/subagents/extraction.ts
+38 -11
View File
@@ -505,8 +505,14 @@ export interface RunningExtraction {
phase: ExtractionPhase;
startedAt: number;
/** Live per-stage usage, accumulated as steps bill (the dock's cost /
* context-window blobs read this through /api/graph/status). */
usage: { extractor?: ExtractionStageUsage; auditor?: ExtractionStageUsage };
* context-window blobs read this through /api/graph/status). Wave 6:
* the provider-embedding tail became a billable stage — recorded from
* the embedder's reported usage when present. */
usage: {
extractor?: ExtractionStageUsage;
auditor?: ExtractionStageUsage;
embedding?: ExtractionStageUsage;
};
/** True-cancellation handle (the dock's abort button): one controller per
* run, shared across phases; abort() cancels the in-flight model call. */
controller: AbortController;
@@ -574,9 +580,12 @@ export interface ExtractionModels {
export interface ExtractionPipelineDeps extends GraphDeps, ExtractionModels {
/** The post-merge embedding batch (semantic clusters run on it).
* INJECTED by the composition root (the agent layer never imports
* the server layer — architecture law): production passes the real
* local-model embedder (server/embeddings.ts embedNewNodes), tests
* fake it. Failure inside it is non-fatal by contract. */
* the server layer — architecture law): production passes the
* provider-embedder batch (server/embeddings.ts embedNewNodes — BYOK
* wave 6; null model = it skips itself with an info log), tests fake
* it. Failure inside it is non-fatal by contract. When the result
* carries a `usage` stage (the EmbedBatchResult contract), it is
* recorded into the run's money-channel capture. */
embedder: (graphStore: GraphStore) => Promise<unknown>;
}
@@ -664,6 +673,27 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
run.usage[stage]!.steps.push(usage);
}
/** The embedding tail's failure log (non-fatal by contract). */
function recordEmbedFailure(chatId: string): (err: unknown) => undefined {
return (err) => {
console.error(`[extraction ${chatId}] embedding failed (non-fatal):`, err);
return undefined;
};
}
/** Wave 6: fold the provider-embedding tail's reported usage into the
* run's capture — one ExtractionStageUsage per run (the embedder
* already aggregated its API calls into steps). Absent/skipped/failed
* embeds contribute nothing (a run that embedded nothing stays
* two-stage, like the whole local-embedder era). */
function captureEmbedUsage(chatId: string, result: unknown): void {
const stage = (result as { usage?: ExtractionStageUsage } | undefined)?.usage;
if (!stage) return;
const run = runningExtractions.get(chatId);
if (!run) return;
run.usage.embedding = stage;
}
async function extract(chatId: string, view: string): Promise<SessionFindings> {
const signal = runningExtractions.get(chatId)?.controller.signal;
@@ -739,6 +769,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
stages: {
extractor: run.usage.extractor ?? { model: "unknown", steps: [] },
auditor: run.usage.auditor ?? { model: "unknown", steps: [] },
...(run.usage.embedding ? { embedding: run.usage.embedding } : {}),
},
};
if (run.kind === "notes") graphStore.appendSourceUsage(chatId, entry);
@@ -830,9 +861,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
// MEANING, so embed the new propositions BEFORE the settle and
// the namer look at clusters. Failure is non-fatal — clustering
// runs without the new nodes until the next merge retries.
await embed(graphStore).catch((err) => {
console.error(`[extraction ${chatId}] embedding failed (non-fatal):`, err);
});
await captureEmbedUsage(chatId, await embed(graphStore).catch(recordEmbedFailure(chatId)));
// Option A: the layout ledger evolves with the merge (sync — the
// next snapshot fetch must already reflect the new nodes).
settleGraphLayout(graphStore);
@@ -963,9 +992,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
// (topics from meaning), then the settle, then the namer —
// under their own phase, exactly like the chat path.
setPhase(id, "embedding", "notes", input.filePath);
await embed(graphStore).catch((err) => {
console.error(`[extraction ${id}] embedding failed (non-fatal):`, err);
});
await captureEmbedUsage(id, await embed(graphStore).catch(recordEmbedFailure(id)));
// Option A: same post-merge settle as the chat path.
settleGraphLayout(graphStore);
persistRunUsage(id, Math.round(performance.now() - t0), stats);
src/agent/subagents/graph-context.ts
+48 -14
View File
@@ -12,9 +12,10 @@
* quiz/ask calls, so delivery lands mid-probing, before plan_init — where
* prior knowledge matters most.
*
* No model call, no usage: the job's only cost is one embedding pass over
* the warm singleton (tens–~200ms) plus a brute-force cosine scan. It still
* carries a JobRecord (kind "graph-context") so the dock shows it, and it
* No teacher-model call: the job's only model spend is ONE embedding
* query (the local embedder was free, provider ones are not — wave 6
* records its prompt tokens on the job record). It still carries a
* JobRecord (kind "graph-context") so the dock shows it, and it
* registers its AbortController so the dock's END kills it — but an abort
* steers NOTHING: nothing was ever promised in the lesson (the injection is
* server-initiated scaffolding, unlike a teacher-dispatched sub-agent).
@@ -27,6 +28,7 @@
*/
import {
accumulateJobUsage,
buildGraphContextSteer,
steerDelivered,
type JobRecord,
@@ -41,20 +43,25 @@ import type { UIMessage } from "ai";
import { registerJobAbort, unregisterJobAbort } from "./aborts.ts";
import type { ChatStore, GraphStore, JobStore } from "../../core/ports.ts";
/** The embedder as the runner needs it: the model/dim the ledger must
* match (parity: one vector space) + the query seam. Injected so tests
* run without the transformer model. */
/** The embedder as the runner needs it (BYOK wave 6): the model slug the
* ledger must match (parity: one vector space) + the query seam.
* Resolved FRESH per run by the composition root (deps.embedder) so a
* settings PUT or a key save between dispatch and spawn lands; null
* model = no embedding configured — the run skips honestly. The query
* carries its prompt-token count so the job records its spend (the
* local embedder was free; provider ones are not). */
export interface GraphContextEmbedder {
model: string;
dim: number;
embedQuery: (text: string) => Promise<number[]>;
model: string | null;
embedQuery: (text: string) => Promise<{ vector: number[]; promptTokens?: number }>;
}
export interface GraphContextDeps {
chatStore: Pick<ChatStore, "getChat">;
jobStore: Pick<JobStore, "createJob" | "getJob" | "updateJob" | "findJobByToolCallId">;
graphStore: Pick<GraphStore, "readNodes" | "readEvents" | "readEmbeddings">;
embedder: GraphContextEmbedder;
/** The embedder RESOLVER (wave 6): called once at run start — fresh
* setting + key reads each run. null = no embedding model configured. */
embedder: () => GraphContextEmbedder | null;
/** The wake seam — jobs.ts deliverJobResult (resolves on delivery). */
deliver: (steer: SteerMessage) => Promise<void>;
}
@@ -126,7 +133,7 @@ export function startGraphContextInjection(deps: GraphContextDeps, chatId: strin
* crash-recovery respawn seam: a job record orphaned "running" by a
* process crash re-runs from here (the record is the only input needed). */
export async function runGraphContextJob(deps: GraphContextDeps, jobId: string): Promise<void> {
const { chatStore, jobStore, graphStore, embedder, deliver } = deps;
const { chatStore, jobStore, graphStore, deliver } = deps;
const job = jobStore.getJob(jobId);
if (!job || job.status !== "running") return;
@@ -135,6 +142,13 @@ export async function runGraphContextJob(deps: GraphContextDeps, jobId: string):
};
const chat = chatStore.getChat(job.chatId);
if (!chat) return fail(`chat not found: ${job.chatId}`);
// BYOK (wave 6): fresh resolution per run; null = honestly off (the
// dispatch gate already skips unconfigured sessions — this only fires
// on a config change racing a respawn, and the error says exactly why).
const embedder = deps.embedder();
if (!embedder || embedder.model === null) {
return fail("no embedding model configured — injection skipped (graph.embeddingModel is null)");
}
const topic = topicOf(chat.messages);
if (topic === null) return fail("no injectable topic (session past its start)");
const budget = effectiveContextBudget(chat.settings);
@@ -146,19 +160,39 @@ export async function runGraphContextJob(deps: GraphContextDeps, jobId: string):
try {
jobStore.updateJob(jobId, { activity: "embedding topic" });
const ledger = graphStore.readEmbeddings();
if (ledger.model !== embedder.model || ledger.dim !== embedder.dim) {
return fail("embeddings ledger is stale (model/dim mismatch) — injection skipped");
// Staleness is the MODEL identity (wave 6): a given model serves one
// dim by default, so model parity is space parity; the dim is checked
// defensively against the served vector below.
if (ledger.model !== embedder.model) {
return fail("embeddings ledger is stale (model mismatch) — injection skipped");
}
if (Object.keys(ledger.vectors).length === 0) {
return fail("the knowledge graph has no embeddings yet — nothing to inject");
}
const query = await embedder.embedQuery(topic);
if (controller.signal.aborted) return;
// The defensive dim half of the old parity check: a provider-side dim
// change (rare) must fail the query, never silently mismatch.
if (query.vector.length !== ledger.dim) {
return fail(
`embedding dim mismatch: query is ${query.vector.length}, ledger holds ${ledger.dim}`,
);
}
// Money channel (wave 6): the one embedding call's prompt tokens fold
// into the job record — priced via the EMBEDDING_PRICES overlay.
if (query.promptTokens !== undefined) {
jobStore.updateJob(jobId, {
usage: accumulateJobUsage(job.usage, embedder.model, {
inputTokens: query.promptTokens,
outputTokens: 0,
}),
});
}
jobStore.updateJob(jobId, { activity: "selecting propositions" });
const data = selectGraphContext({
chatId: job.chatId,
query,
query: query.vector,
vectors: ledger.vectors,
nodes: graphStore.readNodes(),
events: graphStore.readEvents(),
src/agent/subagents/headless.ts
+7 -5
View File
@@ -205,11 +205,13 @@ function svgViewport(svg: string, opts: RenderSvgOptions): { width: number; heig
* are an arbitrary SPLIT of one zlib stream, so merging them is the more
* standard shape, byte-identical stream, zero re-encoding.
*
* Pure Node on purpose (no sharp): the process already loads transformers'
* nested sharp + libvips, and importing the root sharp (Next's optional
* dependency) put a SECOND libvips into the process — macOS's ObjC runtime
* warns (duplicate GNotificationCenterDelegate, "mysterious crashes").
* One native image lib per process; app code carries none. */
* Pure Node on purpose (no sharp): historically the process already carried
* a native image lib via @huggingface/transformers' nested sharp + libvips,
* and importing the root sharp (Next's optional dependency) put a SECOND
* libvips into the process — macOS's ObjC runtime warns (duplicate
* GNotificationCenterDelegate, "mysterious crashes"). The transformers
* embedder died with wave 6 (2026-09-23) but the rule lives on: one native
* image lib per process; app code carries none. */
export function providerSafePng(png: Buffer): Buffer {
const SIG_LEN = 8;
if (png.length < SIG_LEN + 12 || png.subarray(0, SIG_LEN).toString("hex") !== "89504e470d0a1a0a") return png;