Wave 7 chunk B: remove the embeddings machinery — the ladder collapses
The owner's wave-7 ruling: embeddings die entirely (git history is the
stash); clustering is birth-session derived (chunk A, 950ddfd), labels
are session names, and the prober's graph tools replace injection.
- settings: GraphSettings collapses to { enabled } — embedding,
embeddingModel, clustering, labeling all die. Migration story: stale
record keys are UNREAD and dropped by the sanitizer's reconstruction;
the next graph-section PUT (replace-whole) persists the collapsed
shape. The settings-panel GraphSection renders one master row.
- deleted: src/server/embeddings.ts (adapters, resolveEmbeddingPortFor,
embedNewNodes, embeddingPort, embeddingModelCatalog), the
GET /api/models/embeddings route, scripts/embed-graph.ts, tests for
all three, and the /api/models/embeddings picker surfaces in the UI.
- prices: EMBEDDING_PRICES overlay + callCost fallback die. cost-summary:
the embedding stage fold dies — back to three stages (extraction/
audit/naming) per the owner ruling; historical stages.embedding spend
in old run records is simply not displayed.
- ports: EmbeddingPort/EmbeddingResult/EmbeddingTask die. The embeddings
LEDGER surface (readEmbeddings/writeEmbeddings + EmbeddingsLedger +
EMBEDDINGS_FILE + graph-deletion's vector pruning) STAYS — inert data,
kept only so graph-context.ts compiles until Teacher's pass removes
it; the seam dies in the follow-up interlude.
- chat-handler (approved cross-lane unwiring, wave-6 precedent): the
extraction pipeline's embedder becomes a documented no-op (nothing
embedded, nothing spent, ledger untouched) until Teacher's pass
removes the phase; the graph-context injection dispatch gate, the
graphContext deps wiring, the setContextRespawn seam, and
turn-service's triggerGraphContext are gone — injection is dead
code pending Teacher's file deletions.
- scripts: backfill-graph/import-notes pass the same no-op embedder;
replay-layout derives clusters from birthSessionPartition (the same
partition the real settle uses) instead of the embeddings ledger.
- settings-body: parseGraphSettingsBody reads the master switch only —
old-shape keys in a body are ignored, not rejected; bad shapes 400.
- next.config: the @huggingface tracing-exclusion guards die with the
last reference.
Cross-lane flags: extraction.ts/graph-context.ts keep stale embed
comments + the embed phase until Teacher's full pass (their parallel
interlude owns those files — zero overlap kept by design);
semantic-clusters.ts + its test stay one more round (extraction.test.ts,
Teacher's file, still imports it — they delete it with the namer cases);
ChatSettings.graphContextTokens + the graph-context-control UI stay
until Teacher+UI.
Gates: tsc clean, eslint clean, 885/885 tests, build compiles.
Live-verified on the port-3004 dev tree: PUT /api/settings/graph with
dead keys collapses on write, bad shapes 400, GET reads {enabled: true},
/api/models/embeddings 404s.
This commit is contained in:
@@ -1,17 +0,0 @@
|
|||||||
/**
|
|
||||||
* GET /api/models/embeddings — the BYOK embedding-model picker's data
|
|
||||||
* (wave 6, owner ruling 2026-09-23): every KEYED provider's embedding
|
|
||||||
* catalog, federated LIVE (unlike /api/models — pi-ai's static catalog
|
|
||||||
* carries no embedding models). Rows are provider-qualified slugs, the
|
|
||||||
* exact shape graph.embeddingModel stores. Empty array when no provider
|
|
||||||
* is keyed (or every catalog fetch failed — per-provider failures are
|
|
||||||
* logged and skipped, never 500s).
|
|
||||||
*/
|
|
||||||
import { embeddingModelCatalog } from "@/src/server";
|
|
||||||
|
|
||||||
export const dynamic = "force-dynamic";
|
|
||||||
|
|
||||||
export async function GET() {
|
|
||||||
const models = await embeddingModelCatalog();
|
|
||||||
return Response.json(models);
|
|
||||||
}
|
|
||||||
@@ -279,7 +279,6 @@ function GraphCostPanel({ costs }: { costs: GraphCostSummary }) {
|
|||||||
["Extraction", costs.stages.extraction],
|
["Extraction", costs.stages.extraction],
|
||||||
["Audit", costs.stages.audit],
|
["Audit", costs.stages.audit],
|
||||||
["Cluster naming", costs.stages.naming],
|
["Cluster naming", costs.stages.naming],
|
||||||
["Embeddings", costs.stages.embedding],
|
|
||||||
];
|
];
|
||||||
return (
|
return (
|
||||||
<div className="w-80 max-w-[calc(100vw-1.5rem)] rounded-xl border bg-card/95 p-3 text-xs shadow-lg backdrop-blur">
|
<div className="w-80 max-w-[calc(100vw-1.5rem)] rounded-xl border bg-card/95 p-3 text-xs shadow-lg backdrop-blur">
|
||||||
|
|||||||
@@ -16,39 +16,10 @@
|
|||||||
* persist but resolve inert — the switch shows stored, the dim effective).
|
* persist but resolve inert — the switch shows stored, the dim effective).
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
import type { AppearanceSettings, GraphSettings } from "@/src/core";
|
||||||
import { MenuShell } from "@/components/menu-shell";
|
|
||||||
import { useSettingsSaved } from "@/components/settings-signal";
|
|
||||||
import { EMBEDDING_PRICES, type AppearanceSettings, type GraphSettings } from "@/src/core";
|
|
||||||
import { useAppearance } from "./appearance.tsx";
|
import { useAppearance } from "./appearance.tsx";
|
||||||
import { useSettingsDialog } from "./settings-dialog.tsx";
|
|
||||||
import { CommitInput, SettingsRow, ThinSelect, Toggle } from "./controls.tsx";
|
import { CommitInput, SettingsRow, ThinSelect, Toggle } from "./controls.tsx";
|
||||||
|
|
||||||
/** One row of GET /api/models/embeddings (the live-federated BYOK
|
|
||||||
* catalog). The server twin is src/server/embeddings.ts's
|
|
||||||
* EmbeddingCatalogModel — components may not import the server layer,
|
|
||||||
* and the shape is exactly these two strings. */
|
|
||||||
interface EmbeddingCatalogModel {
|
|
||||||
id: string;
|
|
||||||
name: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Pure: the price line for a catalog row — USD per 1M INPUT tokens
|
|
||||||
* (embeddings emit no output). Unlisted models stay honestly "—"
|
|
||||||
* (never zero). The label convention is the model menu's. */
|
|
||||||
export function embeddingPriceLabel(id: string): string {
|
|
||||||
const price = EMBEDDING_PRICES[id];
|
|
||||||
if (!price) return "—";
|
|
||||||
const fmt = (n: number) => `$${n < 1 ? n.toFixed(2).replace(/0$/, "") : n.toFixed(2)}`;
|
|
||||||
return `${fmt(price.input)}/1M in`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Pure: which ladder rungs need a configured embedding model to do
|
|
||||||
* anything — everything but the master switch. */
|
|
||||||
export function needsEmbeddingModel(key: "enabled" | "embedding" | "clustering" | "labeling"): boolean {
|
|
||||||
return key !== "enabled";
|
|
||||||
}
|
|
||||||
|
|
||||||
export function AppearanceSection({
|
export function AppearanceSection({
|
||||||
appearance,
|
appearance,
|
||||||
onSectionChange,
|
onSectionChange,
|
||||||
@@ -167,17 +138,6 @@ export function AppearanceSection({
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
const LADDER: readonly {
|
|
||||||
key: "enabled" | "embedding" | "clustering" | "labeling";
|
|
||||||
label: string;
|
|
||||||
needs: "enabled" | "embedding" | "clustering" | "labeling" | null;
|
|
||||||
}[] = [
|
|
||||||
{ key: "enabled", label: "Graph", needs: null },
|
|
||||||
{ key: "embedding", label: "Embedding", needs: "enabled" },
|
|
||||||
{ key: "clustering", label: "Clustering", needs: "embedding" },
|
|
||||||
{ key: "labeling", label: "Labeling", needs: "clustering" },
|
|
||||||
];
|
|
||||||
|
|
||||||
export function GraphSection({
|
export function GraphSection({
|
||||||
graph,
|
graph,
|
||||||
onSectionChange,
|
onSectionChange,
|
||||||
@@ -185,280 +145,19 @@ export function GraphSection({
|
|||||||
graph: GraphSettings | undefined;
|
graph: GraphSettings | undefined;
|
||||||
onSectionChange: (next: GraphSettings) => void;
|
onSectionChange: (next: GraphSettings) => void;
|
||||||
}) {
|
}) {
|
||||||
const { open: openSettingsSection } = useSettingsDialog();
|
// Wave 7 collapse (owner ruling 2026-09-23): the section is the MASTER
|
||||||
// The BYOK embedding catalog — null until the first fetch lands.
|
// SWITCH ONLY. The embedding/clustering/labeling rungs and the BYOK
|
||||||
const [catalog, setCatalog] = useState<EmbeddingCatalogModel[] | null>(null);
|
// embedding-model picker are gone — clustering is the free birth-session
|
||||||
const [menuOpen, setMenuOpen] = useState(false);
|
// partition derived on read, labels ARE session names, and the prober's
|
||||||
const anchorRef = useRef<HTMLButtonElement>(null);
|
// graph tools read the ledgers directly. There is nothing left to
|
||||||
// The re-embed gate: a pick that differs from the stored model, held
|
// configure but on/off.
|
||||||
// until the user answers the confirm strip (or the graph turns out to
|
|
||||||
// be empty — nothing to re-embed, the write lands directly).
|
|
||||||
const [pending, setPending] = useState<EmbeddingCatalogModel | null>(null);
|
|
||||||
// null while the count fetch is in flight; -1 when it failed (the
|
|
||||||
// confirm line then omits the number rather than guessing one).
|
|
||||||
const [pendingCount, setPendingCount] = useState<number | null>(null);
|
|
||||||
|
|
||||||
// Stored values (the switches show these) vs EFFECTIVE values (a
|
|
||||||
// stored-on rung under a broken chain is inert — the schema's ladder).
|
|
||||||
const stored = {
|
|
||||||
enabled: graph?.enabled ?? true,
|
|
||||||
embedding: graph?.embedding ?? true,
|
|
||||||
clustering: graph?.clustering ?? true,
|
|
||||||
labeling: graph?.labeling ?? true,
|
|
||||||
};
|
|
||||||
const effective = {
|
|
||||||
enabled: stored.enabled,
|
|
||||||
embedding: stored.enabled && stored.embedding,
|
|
||||||
clustering: stored.enabled && stored.embedding && stored.clustering,
|
|
||||||
labeling: stored.enabled && stored.embedding && stored.clustering && stored.labeling,
|
|
||||||
};
|
|
||||||
// No embedding model = the whole ladder under the master switch is
|
|
||||||
// inert (the embedding port refuses without one) — those rows gray
|
|
||||||
// out with a hint pointing at the picker below (owner, wave 6).
|
|
||||||
const modelUnset = !graph?.embeddingModel;
|
|
||||||
|
|
||||||
const commitPickRef = useRef(onSectionChange);
|
|
||||||
commitPickRef.current = onSectionChange;
|
|
||||||
const commitRef = useRef<(id: string | null) => void>(null);
|
|
||||||
commitRef.current = (id) => commitPickRef.current({ ...graph, embeddingModel: id });
|
|
||||||
|
|
||||||
// Catalog: fetch on mount and re-read on any settings save — keying a
|
|
||||||
// provider changes what this list can offer.
|
|
||||||
const refreshCatalog = useCallback(async () => {
|
|
||||||
try {
|
|
||||||
const resp = await fetch("/api/models/embeddings");
|
|
||||||
if (!resp.ok) return;
|
|
||||||
const list = (await resp.json()) as EmbeddingCatalogModel[];
|
|
||||||
if (Array.isArray(list)) setCatalog(list);
|
|
||||||
} catch {
|
|
||||||
// leave the last known list; the empty-add path stays clickable
|
|
||||||
}
|
|
||||||
}, []);
|
|
||||||
useEffect(() => {
|
|
||||||
void refreshCatalog();
|
|
||||||
}, [refreshCatalog]);
|
|
||||||
useSettingsSaved(() => {
|
|
||||||
void refreshCatalog();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Dismissal: the agent-section contract — outside pointerdown closes
|
|
||||||
// unless it lands inside the menu or on the anchor; Escape is CLAIMED
|
|
||||||
// so the dialog behind stays put.
|
|
||||||
useEffect(() => {
|
|
||||||
if (!menuOpen) return;
|
|
||||||
const onPointer = (e: PointerEvent) => {
|
|
||||||
const t = e.target as HTMLElement;
|
|
||||||
if (t.closest("[data-lh-menu]") || anchorRef.current?.contains(t)) return;
|
|
||||||
setMenuOpen(false);
|
|
||||||
};
|
|
||||||
const onKey = (e: KeyboardEvent) => {
|
|
||||||
if (e.key === "Escape") {
|
|
||||||
e.stopPropagation();
|
|
||||||
setMenuOpen(false);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
document.addEventListener("pointerdown", onPointer);
|
|
||||||
document.addEventListener("keydown", onKey, true);
|
|
||||||
return () => {
|
|
||||||
document.removeEventListener("pointerdown", onPointer);
|
|
||||||
document.removeEventListener("keydown", onKey, true);
|
|
||||||
};
|
|
||||||
}, [menuOpen]);
|
|
||||||
|
|
||||||
// A held pick: count the graph's propositions. Zero = nothing to
|
|
||||||
// re-embed, so the write lands without a confirm (the strip would be
|
|
||||||
// noise). Otherwise the strip shows the honest number.
|
|
||||||
useEffect(() => {
|
|
||||||
if (!pending) return;
|
|
||||||
let cancelled = false;
|
|
||||||
(async () => {
|
|
||||||
let n = -1;
|
|
||||||
try {
|
|
||||||
const resp = await fetch("/api/graph");
|
|
||||||
const snap = (await resp.json()) as { nodes?: unknown };
|
|
||||||
if (Array.isArray(snap.nodes)) n = snap.nodes.length;
|
|
||||||
} catch {
|
|
||||||
// count unknown — the strip reads "all propositions"
|
|
||||||
}
|
|
||||||
if (cancelled) return;
|
|
||||||
if (n === 0) {
|
|
||||||
setPending(null);
|
|
||||||
setPendingCount(null);
|
|
||||||
commitRef.current?.(pending.id);
|
|
||||||
} else {
|
|
||||||
setPendingCount(n);
|
|
||||||
}
|
|
||||||
})();
|
|
||||||
return () => {
|
|
||||||
cancelled = true;
|
|
||||||
};
|
|
||||||
}, [pending]);
|
|
||||||
|
|
||||||
const pick = (m: EmbeddingCatalogModel | null) => {
|
|
||||||
setMenuOpen(false);
|
|
||||||
if (m === null) {
|
|
||||||
// Unset never re-embeds — the port just stops resolving.
|
|
||||||
if (graph?.embeddingModel != null) commitRef.current?.(null);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
if (m.id === graph?.embeddingModel) return;
|
|
||||||
setPending(m);
|
|
||||||
setPendingCount(null);
|
|
||||||
};
|
|
||||||
|
|
||||||
// The trigger's label: the catalog row's name when known, else the raw
|
|
||||||
// slug (a key may have been removed after the model was set).
|
|
||||||
const currentName = useMemo(() => {
|
|
||||||
const id = graph?.embeddingModel ?? null;
|
|
||||||
if (!id) return null;
|
|
||||||
return catalog?.find((m) => m.id === id)?.name ?? id;
|
|
||||||
}, [graph?.embeddingModel, catalog]);
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<>
|
<SettingsRow label="Graph" testid="graph-row-enabled">
|
||||||
{LADDER.map(({ key, label, needs }) => {
|
<Toggle
|
||||||
// Touchable only when every ancestor is EFFECTIVELY on, and
|
checked={graph?.enabled ?? true}
|
||||||
// (below the master switch) only when a model is configured.
|
label="graph enabled"
|
||||||
const parentOn = needs === null ? true : effective[needs];
|
onChange={(on) => onSectionChange({ enabled: on })}
|
||||||
const inert = !parentOn || (needsEmbeddingModel(key) && modelUnset);
|
/>
|
||||||
return (
|
</SettingsRow>
|
||||||
<SettingsRow
|
|
||||||
key={key}
|
|
||||||
label={label}
|
|
||||||
testid={`graph-row-${key}`}
|
|
||||||
dimmed={inert}
|
|
||||||
hint={parentOn && needsEmbeddingModel(key) && modelUnset ? "Needs an embedding model" : undefined}
|
|
||||||
>
|
|
||||||
<Toggle
|
|
||||||
checked={stored[key]}
|
|
||||||
disabled={inert}
|
|
||||||
label={`graph ${key}`}
|
|
||||||
onChange={(on) => onSectionChange({ ...graph, [key]: on })}
|
|
||||||
/>
|
|
||||||
</SettingsRow>
|
|
||||||
);
|
|
||||||
})}
|
|
||||||
|
|
||||||
{/* The embedding-model picker — the setting every rung above
|
|
||||||
depends on. Empty catalog means no provider key is set: the
|
|
||||||
trigger BECOMES the deep link to the Keys section. */}
|
|
||||||
<SettingsRow label="Embedding model" testid="graph-row-embedding-model">
|
|
||||||
{catalog === null ? (
|
|
||||||
<span className="text-xs text-muted-foreground">Loading…</span>
|
|
||||||
) : catalog.length === 0 ? (
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
data-testid="embedding-model-empty"
|
|
||||||
onClick={() => openSettingsSection("keys")}
|
|
||||||
className="rounded-lg border border-border/60 px-2 py-1 text-xs text-muted-foreground transition-colors hover:bg-muted"
|
|
||||||
>
|
|
||||||
Add a provider key first…
|
|
||||||
</button>
|
|
||||||
) : (
|
|
||||||
<button
|
|
||||||
ref={anchorRef}
|
|
||||||
type="button"
|
|
||||||
data-testid="embedding-model-trigger"
|
|
||||||
onClick={() => setMenuOpen((o) => !o)}
|
|
||||||
className="max-w-56 truncate rounded-lg border border-border/60 px-2 py-1 text-xs text-card-foreground transition-colors hover:bg-muted"
|
|
||||||
title={graph?.embeddingModel ?? undefined}
|
|
||||||
>
|
|
||||||
{currentName ?? "—"}
|
|
||||||
</button>
|
|
||||||
)}
|
|
||||||
</SettingsRow>
|
|
||||||
|
|
||||||
{/* The re-embed confirm — switching models rewrites every vector
|
|
||||||
in the graph (the staleness machinery, next background pass).
|
|
||||||
Honesty over comfort: the count is the user's decision input. */}
|
|
||||||
{pending && (
|
|
||||||
<div
|
|
||||||
data-testid="embedding-reembed-confirm"
|
|
||||||
className="rounded-xl border border-border/60 bg-background p-3 text-xs"
|
|
||||||
>
|
|
||||||
<p className="text-card-foreground">Switch to {pending.name}?</p>
|
|
||||||
<p className="mt-1 text-muted-foreground">
|
|
||||||
{pendingCount === null
|
|
||||||
? "Counting the graph’s propositions…"
|
|
||||||
: pendingCount > 0
|
|
||||||
? `All ${pendingCount.toLocaleString()} propositions in the graph will be re-embedded with the new model (runs with the next background pass).`
|
|
||||||
: "All propositions in the graph will be re-embedded with the new model (runs with the next background pass)."}
|
|
||||||
</p>
|
|
||||||
{pendingCount !== null && (
|
|
||||||
<div className="mt-2 flex gap-2">
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
data-testid="embedding-reembed-yes"
|
|
||||||
onClick={() => {
|
|
||||||
commitRef.current?.(pending.id);
|
|
||||||
setPending(null);
|
|
||||||
setPendingCount(null);
|
|
||||||
}}
|
|
||||||
className="rounded-lg border border-border/60 px-2.5 py-1 text-card-foreground transition-colors hover:bg-muted"
|
|
||||||
>
|
|
||||||
Switch model
|
|
||||||
</button>
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
data-testid="embedding-reembed-no"
|
|
||||||
onClick={() => {
|
|
||||||
setPending(null);
|
|
||||||
setPendingCount(null);
|
|
||||||
}}
|
|
||||||
className="rounded-lg border border-border/60 px-2.5 py-1 text-muted-foreground transition-colors hover:bg-muted"
|
|
||||||
>
|
|
||||||
Cancel
|
|
||||||
</button>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{menuOpen && catalog !== null && catalog.length > 0 && (
|
|
||||||
<MenuShell
|
|
||||||
anchorRef={anchorRef}
|
|
||||||
testid="embedding-model-menu"
|
|
||||||
preferredSide="down"
|
|
||||||
align="right"
|
|
||||||
menuClass="flex w-[20rem] flex-col gap-0.5 rounded-xl border bg-card p-2 text-xs shadow-xl"
|
|
||||||
>
|
|
||||||
<button
|
|
||||||
type="button"
|
|
||||||
data-testid="embedding-option-unset"
|
|
||||||
onClick={() => pick(null)}
|
|
||||||
className={`flex items-center justify-between gap-2 rounded-lg px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted ${
|
|
||||||
modelUnset ? "bg-muted/60" : ""
|
|
||||||
}`}
|
|
||||||
>
|
|
||||||
<span className="text-card-foreground">Unset</span>
|
|
||||||
<span className="text-muted-foreground">stop embedding</span>
|
|
||||||
</button>
|
|
||||||
{catalog.map((m) => (
|
|
||||||
<button
|
|
||||||
key={m.id}
|
|
||||||
type="button"
|
|
||||||
data-testid={`embedding-option-${m.id.replace(/[^a-z0-9-]/gi, "_")}`}
|
|
||||||
onClick={() => pick(m)}
|
|
||||||
className={`flex flex-col gap-0.5 rounded-lg px-2 py-1.5 text-left outline-none transition-colors hover:bg-muted ${
|
|
||||||
m.id === graph?.embeddingModel ? "bg-muted/60" : ""
|
|
||||||
}`}
|
|
||||||
>
|
|
||||||
<span className="flex items-center justify-between gap-2">
|
|
||||||
<span className="truncate text-card-foreground">{m.name}</span>
|
|
||||||
<span
|
|
||||||
className="shrink-0 tabular-nums text-muted-foreground"
|
|
||||||
title="USD per 1M input tokens"
|
|
||||||
>
|
|
||||||
{embeddingPriceLabel(m.id)}
|
|
||||||
</span>
|
|
||||||
</span>
|
|
||||||
{/* The full slug — ids are colon-safe (openrouter:openai/…),
|
|
||||||
never split, shown as stored. */}
|
|
||||||
<span className="truncate text-muted-foreground">{m.id}</span>
|
|
||||||
</button>
|
|
||||||
))}
|
|
||||||
</MenuShell>
|
|
||||||
)}
|
|
||||||
</>
|
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
@@ -150,10 +150,11 @@ export function SettingsDialogBody({
|
|||||||
};
|
};
|
||||||
}, []);
|
}, []);
|
||||||
const [active, setActive] = useState<SectionId>(initialSection);
|
const [active, setActive] = useState<SectionId>(initialSection);
|
||||||
// A deep-link fired while the dialog is ALREADY open (wave 6: the
|
// A deep-link fired while the dialog is ALREADY open (first exercised by
|
||||||
// embedding picker's "Add a provider key first…" lands on Keys from
|
// the wave-6 embedding picker's "Add a provider key first…" link — the
|
||||||
// inside the dialog) changes initialSection under our feet — follow it,
|
// picker itself died with wave 7, the mechanism is general) changes
|
||||||
// or the link silently does nothing.
|
// initialSection under our feet — follow it, or the link silently does
|
||||||
|
// nothing.
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
setActive(initialSection);
|
setActive(initialSection);
|
||||||
}, [initialSection]);
|
}, [initialSection]);
|
||||||
|
|||||||
@@ -14,10 +14,6 @@ const nextConfig: NextConfig = {
|
|||||||
outputFileTracingExcludes: {
|
outputFileTracingExcludes: {
|
||||||
"*": [
|
"*": [
|
||||||
"data/**/*",
|
"data/**/*",
|
||||||
// Wave 6 (2026-09-23): the local transformers embeddings died — any
|
|
||||||
// @huggingface copy (or its ~300MB model cache, if one ever
|
|
||||||
// reappears) must never sweep into the standalone bundle again.
|
|
||||||
"node_modules/@huggingface/**/*",
|
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
// Empirically verified gaps (2026-09-17 standalone smoke): the tracer
|
// Empirically verified gaps (2026-09-17 standalone smoke): the tracer
|
||||||
|
|||||||
@@ -8,14 +8,13 @@
|
|||||||
import { chatStore } from "../src/store/chats.ts";
|
import { chatStore } from "../src/store/chats.ts";
|
||||||
import { graphStore } from "../src/store/graph.ts";
|
import { graphStore } from "../src/store/graph.ts";
|
||||||
import { createExtractionPipeline } from "../src/agent/subagents/extraction.ts";
|
import { createExtractionPipeline } from "../src/agent/subagents/extraction.ts";
|
||||||
import { embedNewNodes } from "../src/server/embeddings.ts";
|
|
||||||
import { getGraph } from "../src/core/graph.ts";
|
import { getGraph } from "../src/core/graph.ts";
|
||||||
|
|
||||||
async function main() {
|
async function main() {
|
||||||
const only = process.argv[2]; // optional: extract just one chat id
|
const only = process.argv[2]; // optional: extract just one chat id
|
||||||
const chats = only ? chatStore.listChats().filter((c) => c.id === only) : chatStore.listChats();
|
const chats = only ? chatStore.listChats().filter((c) => c.id === only) : chatStore.listChats();
|
||||||
console.log(`[backfill] ${chats.length} chats to extract`);
|
console.log(`[backfill] ${chats.length} chats to extract`);
|
||||||
const pipeline = createExtractionPipeline({ chatStore, graphStore, embedder: embedNewNodes });
|
const pipeline = createExtractionPipeline({ chatStore, graphStore, embedder: async () => ({ embedded: 0, total: 0, skipped: "embeddings removed (wave 7)" }) });
|
||||||
for (const chat of chats) {
|
for (const chat of chats) {
|
||||||
const t0 = Date.now();
|
const t0 = Date.now();
|
||||||
try {
|
try {
|
||||||
|
|||||||
@@ -1,47 +0,0 @@
|
|||||||
/**
|
|
||||||
* Embed the whole graph (backfill / cold start / model change): one-time
|
|
||||||
* pass that fills the embedding ledger for every node — the same
|
|
||||||
* embedNewNodes the post-merge hook runs, just with an empty (or
|
|
||||||
* stale-model) ledger so every node is "new".
|
|
||||||
*
|
|
||||||
* BYOK (wave 6, 2026-09-23): vectors come from the configured provider
|
|
||||||
* (graph.embeddingModel, keys from the auth store) — this script resolves
|
|
||||||
* the live state up front so an unconfigured machine says so before
|
|
||||||
* anything bills. A model CHANGE re-embeds everything (ledger staleness).
|
|
||||||
*
|
|
||||||
* npx tsx scripts/embed-graph.ts
|
|
||||||
*/
|
|
||||||
import { embedNewNodes, embeddingPort } from "../src/server/embeddings.ts";
|
|
||||||
import { graphStore } from "../src/store/graph.ts";
|
|
||||||
|
|
||||||
async function main() {
|
|
||||||
const resolved = embeddingPort();
|
|
||||||
if (!resolved.ok) {
|
|
||||||
console.error(`[embed] cannot run — ${resolved.reason}`);
|
|
||||||
process.exit(1);
|
|
||||||
}
|
|
||||||
const t0 = Date.now();
|
|
||||||
console.log(`[embed] model: ${resolved.port.model} (provider embeddings, batch 32)`);
|
|
||||||
const { embedded, total, usage, failed } = await embedNewNodes(graphStore);
|
|
||||||
const secs = ((Date.now() - t0) / 1000).toFixed(1);
|
|
||||||
if (usage) {
|
|
||||||
const tokens = usage.steps.reduce((sum, s) => sum + (s.inputTokens ?? 0), 0);
|
|
||||||
console.log(`[embed] embedding spend: ${tokens} prompt tokens over ${usage.steps.length} call(s)`);
|
|
||||||
}
|
|
||||||
// Usage-on-error + pay-once (owner ruling 2026-09-23): a mid-run provider
|
|
||||||
// failure still reports the landed batches' spend above; the landed
|
|
||||||
// vectors are persisted, so the honest count is what LANDED — the
|
|
||||||
// in-memory `total` would read as a ledger claim it can't make.
|
|
||||||
if (failed) {
|
|
||||||
console.error(
|
|
||||||
`[embed] failed after ${embedded} node(s) landed (${secs}s) — the ledger keeps exactly those (a retry embeds only the rest): ${failed}`,
|
|
||||||
);
|
|
||||||
process.exit(1);
|
|
||||||
}
|
|
||||||
console.log(`[embed] embedded ${embedded} nodes → ledger holds ${total} vectors (${secs}s)`);
|
|
||||||
}
|
|
||||||
|
|
||||||
main().catch((err) => {
|
|
||||||
console.error("[embed] failed:", err);
|
|
||||||
process.exit(1);
|
|
||||||
});
|
|
||||||
@@ -15,7 +15,6 @@
|
|||||||
import * as fs from "node:fs";
|
import * as fs from "node:fs";
|
||||||
import * as path from "node:path";
|
import * as path from "node:path";
|
||||||
import { createExtractionPipeline } from "../src/agent/subagents/extraction.ts";
|
import { createExtractionPipeline } from "../src/agent/subagents/extraction.ts";
|
||||||
import { embedNewNodes } from "../src/server/embeddings.ts";
|
|
||||||
import { clusterNamingSettled } from "../src/agent/subagents/cluster-namer.ts";
|
import { clusterNamingSettled } from "../src/agent/subagents/cluster-namer.ts";
|
||||||
import { getGraph } from "../src/core/graph.ts";
|
import { getGraph } from "../src/core/graph.ts";
|
||||||
import { stores } from "../src/server/stores.ts";
|
import { stores } from "../src/server/stores.ts";
|
||||||
@@ -29,7 +28,7 @@ async function main() {
|
|||||||
.sort();
|
.sort();
|
||||||
console.log(`[import-notes] ${files.length} note files from ${dir} (concurrency ${limit})`);
|
console.log(`[import-notes] ${files.length} note files from ${dir} (concurrency ${limit})`);
|
||||||
|
|
||||||
const pipeline = createExtractionPipeline({ ...stores, embedder: embedNewNodes });
|
const pipeline = createExtractionPipeline({ ...stores, embedder: async () => ({ embedded: 0, total: 0, skipped: "embeddings removed (wave 7)" }) });
|
||||||
const queue = [...files];
|
const queue = [...files];
|
||||||
const totals = { nodes: 0, edges: 0, events: 0, failed: 0 };
|
const totals = { nodes: 0, edges: 0, events: 0, failed: 0 };
|
||||||
const workers = Array.from({ length: Math.min(limit, files.length) }, async () => {
|
const workers = Array.from({ length: Math.min(limit, files.length) }, async () => {
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ const depthArg = process.argv.find((a) => a.startsWith("--depth="));
|
|||||||
const depthMode: "off" | "vertical" | "radial" =
|
const depthMode: "off" | "vertical" | "radial" =
|
||||||
depthArg === "--depth=vertical" ? "vertical" : depthArg === "--depth=radial" ? "radial" : "off";
|
depthArg === "--depth=vertical" ? "vertical" : depthArg === "--depth=radial" ? "radial" : "off";
|
||||||
import { incrementalLayout } from "../src/core/layout.ts";
|
import { incrementalLayout } from "../src/core/layout.ts";
|
||||||
|
import { birthSessionPartition } from "../src/core/session-clusters.ts";
|
||||||
import type { GraphNode, NodeMapping, Position } from "../src/core/contracts.ts";
|
import type { GraphNode, NodeMapping, Position } from "../src/core/contracts.ts";
|
||||||
|
|
||||||
function main() {
|
function main() {
|
||||||
@@ -32,12 +33,9 @@ function main() {
|
|||||||
nodes.push(line);
|
nodes.push(line);
|
||||||
}
|
}
|
||||||
const edges = graphStore.readEdges();
|
const edges = graphStore.readEdges();
|
||||||
// The semantic-topic source: the embedding ledger (run
|
// The cluster source (wave 7): the birth-session partition, derived on
|
||||||
// scripts/embed-graph.ts first if missing/stale).
|
// read — same as the real settle (core/layout-settle.ts).
|
||||||
const embeddings = graphStore.readEmbeddings();
|
const birthClusters = birthSessionPartition(nodes);
|
||||||
if (Object.keys(embeddings.vectors).length === 0) {
|
|
||||||
console.warn("[replay] embedding ledger is empty — clusters will be absent (run scripts/embed-graph.ts)");
|
|
||||||
}
|
|
||||||
|
|
||||||
// Batches: each imported file, in real import order (sources.json
|
// Batches: each imported file, in real import order (sources.json
|
||||||
// importedAt). A node belongs to the file whose mapping first claims it
|
// importedAt). A node belongs to the file whose mapping first claims it
|
||||||
@@ -85,9 +83,8 @@ function main() {
|
|||||||
const batchNodes = nodes.filter((n) => delivered.has(n.id));
|
const batchNodes = nodes.filter((n) => delivered.has(n.id));
|
||||||
positions = incrementalLayout(batchNodes, activeEdges, positions, {
|
positions = incrementalLayout(batchNodes, activeEdges, positions, {
|
||||||
// The same partition the real settle would see for this batch:
|
// The same partition the real settle would see for this batch:
|
||||||
// the persisted semantic topics, filtered to delivered nodes
|
// the birth-session clusters, filtered to delivered nodes.
|
||||||
// (never recomputed — the agglomerative pass is ~1.5s at scale).
|
clusters: birthClusters
|
||||||
clusters: embeddings.clusters
|
|
||||||
.map((c) => ({ members: c.members.filter((id) => delivered.has(id)) }))
|
.map((c) => ({ members: c.members.filter((id) => delivered.has(id)) }))
|
||||||
.filter((c) => c.members.length > 0),
|
.filter((c) => c.members.length > 0),
|
||||||
...(depthMode === "off"
|
...(depthMode === "off"
|
||||||
|
|||||||
@@ -55,12 +55,13 @@ export interface GraphCostSummary {
|
|||||||
totalDollars: number;
|
totalDollars: number;
|
||||||
/** Runs where EVERY stage was unpriced (totalDollars excludes them). */
|
/** Runs where EVERY stage was unpriced (totalDollars excludes them). */
|
||||||
unpricedRuns: number;
|
unpricedRuns: number;
|
||||||
/** The billable pipeline stages (wave 6: the embedder became one —
|
/** The billable pipeline stages. (Wave 7, owner ruling 2026-09-23:
|
||||||
* provider embeddings cost money; merger is the only free stage). */
|
* back to three — the embedding stage is gone with the provider
|
||||||
|
* embedder; historical records' stages.embedding spend is simply not
|
||||||
|
* displayed.) */
|
||||||
stages: {
|
stages: {
|
||||||
extraction: StageCost;
|
extraction: StageCost;
|
||||||
audit: StageCost;
|
audit: StageCost;
|
||||||
embedding: StageCost;
|
|
||||||
naming: StageCost;
|
naming: StageCost;
|
||||||
};
|
};
|
||||||
/** Newest first, capped at `recentCap` (default 10). */
|
/** Newest first, capped at `recentCap` (default 10). */
|
||||||
@@ -115,7 +116,6 @@ function mergeTokens(into: CallUsage, steps: readonly CallUsage[]): void {
|
|||||||
export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
|
export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
|
||||||
const extraction = emptyStage();
|
const extraction = emptyStage();
|
||||||
const audit = emptyStage();
|
const audit = emptyStage();
|
||||||
const embedding = emptyStage();
|
|
||||||
const naming = emptyStage();
|
const naming = emptyStage();
|
||||||
const runs: RecentRun[] = [];
|
const runs: RecentRun[] = [];
|
||||||
|
|
||||||
@@ -134,25 +134,12 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
|
|||||||
if (audDollars === null) audit.unpricedRuns++;
|
if (audDollars === null) audit.unpricedRuns++;
|
||||||
else audit.dollars += audDollars;
|
else audit.dollars += audDollars;
|
||||||
|
|
||||||
// Wave 6: the embedding tail — present only when an embedding model
|
|
||||||
// ran (absent stage = the run embedded nothing, never "free").
|
|
||||||
let embDollars: number | null = null;
|
|
||||||
let embSteps: CallUsage[] = [];
|
|
||||||
if (stages.embedding) {
|
|
||||||
embedding.runs++;
|
|
||||||
embSteps = stages.embedding.steps;
|
|
||||||
mergeTokens(embedding.tokens, embSteps);
|
|
||||||
embDollars = stageDollars(stages.embedding.model, embSteps);
|
|
||||||
if (embDollars === null) embedding.unpricedRuns++;
|
|
||||||
else embedding.dollars += embDollars;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Run-level: null only when EVERY stage is unpriced.
|
// Run-level: null only when EVERY stage is unpriced.
|
||||||
const allStages: Array<number | null> = [extDollars, audDollars, embDollars];
|
const allStages: Array<number | null> = [extDollars, audDollars];
|
||||||
const runDollars = allStages.every((d) => d === null)
|
const runDollars = allStages.every((d) => d === null)
|
||||||
? null
|
? null
|
||||||
: allStages.reduce<number>((a, d) => a + (d ?? 0), 0);
|
: allStages.reduce<number>((a, d) => a + (d ?? 0), 0);
|
||||||
const allSteps = [...stages.extractor.steps, ...stages.auditor.steps, ...embSteps];
|
const allSteps = [...stages.extractor.steps, ...stages.auditor.steps];
|
||||||
runs.push({
|
runs.push({
|
||||||
startedAt: run.startedAt,
|
startedAt: run.startedAt,
|
||||||
at: run.at,
|
at: run.at,
|
||||||
@@ -195,9 +182,9 @@ export function graphCostSummary(input: CostSummaryInput): GraphCostSummary {
|
|||||||
// A run is "unpriced" when it billed nothing countable at all.
|
// A run is "unpriced" when it billed nothing countable at all.
|
||||||
const unpricedRuns = runs.filter((r) => r.dollars === null).length;
|
const unpricedRuns = runs.filter((r) => r.dollars === null).length;
|
||||||
return {
|
return {
|
||||||
totalDollars: extraction.dollars + audit.dollars + embedding.dollars + naming.dollars,
|
totalDollars: extraction.dollars + audit.dollars + naming.dollars,
|
||||||
unpricedRuns,
|
unpricedRuns,
|
||||||
stages: { extraction, audit, embedding, naming },
|
stages: { extraction, audit, naming },
|
||||||
runs: capped,
|
runs: capped,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
+8
-49
@@ -128,11 +128,15 @@ export interface GraphStore {
|
|||||||
/** Replace the whole ledger (the settle is a full-graph computation). */
|
/** Replace the whole ledger (the settle is a full-graph computation). */
|
||||||
writePositions(positions: Record<string, Position>): void;
|
writePositions(positions: Record<string, Position>): void;
|
||||||
|
|
||||||
/** The embedding ledger (config.EMBEDDINGS_FILE). WRITTEN ONLY by
|
/** The embeddings ledger (config.EMBEDDINGS_FILE) — INERT DATA since
|
||||||
* the embedder (server/embeddings.ts) after a merge — read-only for
|
* wave 7 (owner ruling 2026-09-23: the provider embedder is dead;
|
||||||
* clustering consumers (snapshot, settle, namer). */
|
* clustering is birth-session, labels are session names). Kept ONLY
|
||||||
|
* because the graph-context injection (agent band, Teacher's pass
|
||||||
|
* removes it) still reads it for vector-space staleness — the seam
|
||||||
|
* dies with it in the follow-up interlude. graph-deletion keeps
|
||||||
|
* pruning dead vectors meanwhile (harmless housekeeping). */
|
||||||
readEmbeddings(): EmbeddingsLedger;
|
readEmbeddings(): EmbeddingsLedger;
|
||||||
/** Replace the whole ledger (embedding is a full-batch computation). */
|
/** Replace the whole ledger (graph-deletion's vector pruning only). */
|
||||||
writeEmbeddings(ledger: EmbeddingsLedger): void;
|
writeEmbeddings(ledger: EmbeddingsLedger): void;
|
||||||
|
|
||||||
/** THE DESTRUCTIVE SEAM (session-deletion wave 6, owner 2026-09-22 —
|
/** THE DESTRUCTIVE SEAM (session-deletion wave 6, owner 2026-09-22 —
|
||||||
@@ -244,48 +248,3 @@ export interface AuthStore {
|
|||||||
deleteAuthVar(name: string): void;
|
deleteAuthVar(name: string): void;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── EmbeddingPort — BYOK provider embeddings (wave 6, 2026-09-23) ───────────
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The provider-embedding seam (the local transformers embedder died with
|
|
||||||
* this wave): vectors come from the user's keyed provider — OpenRouter,
|
|
||||||
* OpenAI, or Google — named by graph.embeddingModel ("provider:id", null =
|
|
||||||
* honestly off: embedding/clustering/injection skip with an info log).
|
|
||||||
*
|
|
||||||
* Owner decisions (2026-09-23) baked in:
|
|
||||||
* - The adapters are resolved FRESH per call from the setting + the auth
|
|
||||||
* store (key rotations land immediately — the search port's treatment).
|
|
||||||
* - Vectors are stored as the provider returns them (default dims, no
|
|
||||||
* truncation, no normalization); the ledger is self-describing on
|
|
||||||
* model + dim, and any model change voids it (full re-embed).
|
|
||||||
* - Usage (prompt tokens) flows back to the money channel: adapters
|
|
||||||
* report it, the caller records it as a CallUsage step.
|
|
||||||
*
|
|
||||||
* Pure types only (browser-safe): the adapters are server-side
|
|
||||||
* (src/server/embeddings.ts) and arrive as injected dependencies.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/** Which side of retrieval the text is — providers bill identically but
|
|
||||||
* quality-optimize differently (OpenRouter's input_type, Gemini's
|
|
||||||
* taskType); every input in one call shares the task. */
|
|
||||||
export type EmbeddingTask = "document" | "query";
|
|
||||||
|
|
||||||
/** One embedding batch's result: vectors in input order, the model's
|
|
||||||
* identity + dim (the ledger stamps), and the provider-reported prompt
|
|
||||||
* token count (undefined = provider didn't report; cost stays honest
|
|
||||||
* "—" via the existing unpriced path). */
|
|
||||||
export interface EmbeddingResult {
|
|
||||||
vectors: number[][];
|
|
||||||
/** The full "provider:id" slug — ledger/cost keys use it verbatim. */
|
|
||||||
model: string;
|
|
||||||
dim: number;
|
|
||||||
promptTokens?: number;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** A live provider embedding adapter. `embed` throws on failure (adapters
|
|
||||||
* map HTTP errors to clear Error messages); callers treat failure as
|
|
||||||
* non-fatal per the extraction contract. */
|
|
||||||
export interface EmbeddingPort {
|
|
||||||
model: string;
|
|
||||||
embed(texts: readonly string[], task: EmbeddingTask): Promise<EmbeddingResult>;
|
|
||||||
}
|
|
||||||
|
|||||||
+1
-26
@@ -2752,30 +2752,6 @@ export const CONTEXT_LIMITS: Record<string, number> = {
|
|||||||
|
|
||||||
// ─── Pure helpers (folds + cost inference) ───────────────────────────────────
|
// ─── Pure helpers (folds + cost inference) ───────────────────────────────────
|
||||||
|
|
||||||
// ─── Embedding prices (wave 6, 2026-09-23) ─────────────────────────────────
|
|
||||||
|
|
||||||
/** Hand-maintained BYOK embedding prices, USD per 1M INPUT tokens — the
|
|
||||||
* sync script never touches this block (pi-ai's catalogs carry no
|
|
||||||
* embedding models, so the generated PRICES table has none of these).
|
|
||||||
* VERIFIED against provider pricing pages 2026-09-23: OpenAI's model
|
|
||||||
* docs ($0.02 3-small / $0.13 3-large / $0.10 ada-002), Google's pricing
|
|
||||||
* page ($0.15 gemini-embedding-001 / $0.20 Gemini Embedding 2 text),
|
|
||||||
* OpenRouter passes provider pricing through. output: 0 — embeddings
|
|
||||||
* emit no output tokens. Unlisted models stay unpriced ("—", never
|
|
||||||
* zero); tokens still get captured. */
|
|
||||||
export const EMBEDDING_PRICES: Record<string, ModelPrice> = {
|
|
||||||
"openai:text-embedding-3-small": { input: 0.02, output: 0 },
|
|
||||||
"openai:text-embedding-3-large": { input: 0.13, output: 0 },
|
|
||||||
"openai:text-embedding-ada-002": { input: 0.1, output: 0 },
|
|
||||||
"google:gemini-embedding-001": { input: 0.15, output: 0 },
|
|
||||||
"google:gemini-embedding-2": { input: 0.2, output: 0 },
|
|
||||||
"openrouter:openai/text-embedding-3-small": { input: 0.02, output: 0 },
|
|
||||||
"openrouter:openai/text-embedding-3-large": { input: 0.13, output: 0 },
|
|
||||||
"openrouter:openai/text-embedding-ada-002": { input: 0.1, output: 0 },
|
|
||||||
"openrouter:google/gemini-embedding-001": { input: 0.15, output: 0 },
|
|
||||||
};
|
|
||||||
|
|
||||||
|
|
||||||
/** Sum a list of per-step usages into one CallUsage. A field stays
|
/** Sum a list of per-step usages into one CallUsage. A field stays
|
||||||
* undefined when EVERY step left it unreported — absent is not zero. */
|
* undefined when EVERY step left it unreported — absent is not zero. */
|
||||||
export function sumSteps(steps: readonly CallUsage[]): CallUsage {
|
export function sumSteps(steps: readonly CallUsage[]): CallUsage {
|
||||||
@@ -2797,8 +2773,7 @@ export function callCost(
|
|||||||
usage: { inputTokens?: number; outputTokens?: number; cacheReadTokens?: number; cacheWriteTokens?: number },
|
usage: { inputTokens?: number; outputTokens?: number; cacheReadTokens?: number; cacheWriteTokens?: number },
|
||||||
model: string,
|
model: string,
|
||||||
): number | null {
|
): number | null {
|
||||||
// Wave 6: the hand-maintained embedding overlay backs the lookup too.
|
const price = PRICES[model];
|
||||||
const price = PRICES[model] ?? EMBEDDING_PRICES[model];
|
|
||||||
if (!price) return null;
|
if (!price) return null;
|
||||||
if (usage.inputTokens === undefined || usage.outputTokens === undefined) return null;
|
if (usage.inputTokens === undefined || usage.outputTokens === undefined) return null;
|
||||||
const cacheRead = usage.cacheReadTokens ?? 0;
|
const cacheRead = usage.cacheReadTokens ?? 0;
|
||||||
|
|||||||
@@ -201,13 +201,10 @@ export function resolveSettings(record: UserSettings | null): EffectiveSettings
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
graph: {
|
graph: {
|
||||||
|
// Wave 7 collapse (owner ruling 2026-09-23): the master switch ONLY —
|
||||||
|
// the embedding/clustering/labeling rungs and the BYOK embeddingModel
|
||||||
|
// are gone (stale record keys are simply never read here).
|
||||||
enabled: graph.enabled ?? FACTORY_SETTINGS.graph.enabled,
|
enabled: graph.enabled ?? FACTORY_SETTINGS.graph.enabled,
|
||||||
embedding: graph.embedding ?? FACTORY_SETTINGS.graph.embedding,
|
|
||||||
clustering: graph.clustering ?? FACTORY_SETTINGS.graph.clustering,
|
|
||||||
labeling: graph.labeling ?? FACTORY_SETTINGS.graph.labeling,
|
|
||||||
// BYOK (wave 6): null is the factory floor and a VALID resolved
|
|
||||||
// value — ?? only fills the absent case, null passes through.
|
|
||||||
embeddingModel: graph.embeddingModel ?? FACTORY_SETTINGS.graph.embeddingModel,
|
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -216,23 +213,13 @@ export function resolveSettings(record: UserSettings | null): EffectiveSettings
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* The WRITE channel gate (extraction + its auditor): dies with the master
|
* The WRITE channel gate (extraction + its auditor): dies with the master
|
||||||
* switch. Used by the chat-handler's post-turn trigger; deeper pipeline
|
* switch. Used by the chat-handler's post-turn trigger. (Wave 7: the master
|
||||||
* consumers (embedder, clustering, labeling — later waves) gate on the
|
* switch is the ONLY graph setting left — the section collapsed.)
|
||||||
* deeper rungs the same way.
|
|
||||||
*/
|
*/
|
||||||
export function extractionEnabled(graph: Required<GraphSettings>): boolean {
|
export function extractionEnabled(graph: Required<GraphSettings>): boolean {
|
||||||
return graph.enabled;
|
return graph.enabled;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* The READ channel gate (graph-context injection): needs the master switch
|
|
||||||
* AND the embedding rung — injection is a vector query, so a graph without
|
|
||||||
* embeddings has nothing to inject.
|
|
||||||
*/
|
|
||||||
export function graphContextEnabled(graph: Required<GraphSettings>): boolean {
|
|
||||||
return graph.enabled && graph.embedding;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The PROBER GRAPH TOOLS gate (wave 7, owner ruling 2026-09-23): the
|
* The PROBER GRAPH TOOLS gate (wave 7, owner ruling 2026-09-23): the
|
||||||
* toggle is an agent-capability switch (it gates tool registration), but
|
* toggle is an agent-capability switch (it gates tool registration), but
|
||||||
@@ -246,3 +233,5 @@ export function graphToolsEnabled(effective: {
|
|||||||
}): boolean {
|
}): boolean {
|
||||||
return effective.graph.enabled && (effective.agent.graphTools.enabled ?? true);
|
return effective.graph.enabled && (effective.agent.graphTools.enabled ?? true);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+14
-31
@@ -249,23 +249,14 @@ export interface AppearanceSettings {
|
|||||||
* meaningful when its parent is on (the UI grays dependents out; stored
|
* meaningful when its parent is on (the UI grays dependents out; stored
|
||||||
* values persist but resolve inert). All default true. */
|
* values persist but resolve inert). All default true. */
|
||||||
export interface GraphSettings {
|
export interface GraphSettings {
|
||||||
/** MASTER: off = no extraction, no auditor, no embedding, no
|
/** MASTER (and, since wave 7, ONLY): off = no extraction, no auditor.
|
||||||
* graph-context injection, no clustering. Data stays stored. */
|
* The wave-7 collapse (owner ruling 2026-09-23) deleted the three
|
||||||
|
* subordinate rungs — embedding/clustering/labeling — and the BYOK
|
||||||
|
* embeddingModel: clustering is the free birth-session partition
|
||||||
|
* derived on read (core/session-clusters.ts), labels ARE session
|
||||||
|
* names, and the prober graph tools read the ledgers directly — there
|
||||||
|
* is nothing left to configure. Data stays stored either way. */
|
||||||
enabled?: boolean;
|
enabled?: boolean;
|
||||||
/** Vector embedding + graph-context injection (consumes vectors even
|
|
||||||
* without clustering — hence its own rung). */
|
|
||||||
embedding?: boolean;
|
|
||||||
/** Cluster computation + settle (needs embedding). */
|
|
||||||
clustering?: boolean;
|
|
||||||
/** The LLM cluster-namer (needs clustering). */
|
|
||||||
labeling?: boolean;
|
|
||||||
/** BYOK embeddings (wave 6, 2026-09-23): the provider embedding model
|
|
||||||
* as a "provider:id" slug (openrouter:…, openai:…, google:…). null =
|
|
||||||
* no model — embedding/clustering/injection skip honestly (info log,
|
|
||||||
* graph stays textual); no factory floor exists. Keys live in the auth
|
|
||||||
* store, resolved fresh per call. A model CHANGE re-embeds everything
|
|
||||||
* (the ledger's staleness semantics). */
|
|
||||||
embeddingModel?: string | null;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** THE record. Every section optional (delta); unknown top-level keys are
|
/** THE record. Every section optional (delta); unknown top-level keys are
|
||||||
@@ -318,7 +309,7 @@ export const FACTORY_SETTINGS: FactorySettings = {
|
|||||||
sessionNameSerif: true,
|
sessionNameSerif: true,
|
||||||
mdRendering: { headingWeight: 900, boldColor: null, italicColor: null },
|
mdRendering: { headingWeight: 900, boldColor: null, italicColor: null },
|
||||||
},
|
},
|
||||||
graph: { enabled: true, embedding: true, clustering: true, labeling: true, embeddingModel: null },
|
graph: { enabled: true },
|
||||||
};
|
};
|
||||||
|
|
||||||
// ─── Sanitizer ─────────────────────────────────────────────────────────────
|
// ─── Sanitizer ─────────────────────────────────────────────────────────────
|
||||||
@@ -467,23 +458,15 @@ function sanitizeAppearance(raw: Record<string, unknown>): AppearanceSettings {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function sanitizeGraph(raw: Record<string, unknown>): GraphSettings {
|
function sanitizeGraph(raw: Record<string, unknown>): GraphSettings {
|
||||||
|
// Wave 7 collapse (owner ruling 2026-09-23): the section is the master
|
||||||
|
// switch ONLY. The dead rungs' keys (embedding/clustering/labeling/
|
||||||
|
// embeddingModel) in existing records are simply UNREAD and DROPPED by
|
||||||
|
// this reconstruction — the migration story: the next graph-section
|
||||||
|
// write (PUT is replace-whole) persists the collapsed shape, and no
|
||||||
|
// active code ever looks at the stale fields.
|
||||||
const out: GraphSettings = {};
|
const out: GraphSettings = {};
|
||||||
const enabled = boolOrDrop(raw.enabled);
|
const enabled = boolOrDrop(raw.enabled);
|
||||||
if (enabled !== undefined) out.enabled = enabled;
|
if (enabled !== undefined) out.enabled = enabled;
|
||||||
const embedding = boolOrDrop(raw.embedding);
|
|
||||||
if (embedding !== undefined) out.embedding = embedding;
|
|
||||||
const clustering = boolOrDrop(raw.clustering);
|
|
||||||
if (clustering !== undefined) out.clustering = clustering;
|
|
||||||
const labeling = boolOrDrop(raw.labeling);
|
|
||||||
if (labeling !== undefined) out.labeling = labeling;
|
|
||||||
// BYOK embeddings (wave 6): null or a provider-qualified slug; anything
|
|
||||||
// else (bad type, bad shape) DROPS to absent — an invalid write never
|
|
||||||
// becomes a live model (the PUT parser 400s it first; this is the
|
|
||||||
// read-side belt-and-suspenders for hand-edited files).
|
|
||||||
if (raw.embeddingModel === null) out.embeddingModel = null;
|
|
||||||
else if (typeof raw.embeddingModel === "string" && /^\S+:\S+$/.test(raw.embeddingModel)) {
|
|
||||||
out.embeddingModel = raw.embeddingModel;
|
|
||||||
}
|
|
||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -4,23 +4,20 @@ import { createProberAgent, createTeacherAgent, type AgentBuildOptions, type Tea
|
|||||||
import { piEngineForSpec, type PiEngine } from "../agent/models.ts";
|
import { piEngineForSpec, type PiEngine } from "../agent/models.ts";
|
||||||
import { createJobRunner } from "../agent/subagents/runner.ts";
|
import { createJobRunner } from "../agent/subagents/runner.ts";
|
||||||
import { createExtractionPipeline } from "../agent/subagents/extraction.ts";
|
import { createExtractionPipeline } from "../agent/subagents/extraction.ts";
|
||||||
import { runGraphContextJob, type GraphContextDeps } from "../agent/subagents/graph-context.ts";
|
|
||||||
import { embedNewNodes, embeddingPort } from "./embeddings.ts";
|
|
||||||
import type { ModelRole } from "../../config.ts";
|
import type { ModelRole } from "../../config.ts";
|
||||||
import { claimSteers, deliverJobResult, setContextRespawn, setJobRespawn, setTurnRunner } from "./jobs.ts";
|
import { claimSteers, deliverJobResult, setJobRespawn, setTurnRunner } from "./jobs.ts";
|
||||||
import { recordToPiMessages } from "./pi/record-adapter.ts";
|
import { recordToPiMessages } from "./pi/record-adapter.ts";
|
||||||
import { createPiTurnResponse } from "./pi/ui-stream.ts";
|
import { createPiTurnResponse } from "./pi/ui-stream.ts";
|
||||||
import { effectiveSettings, stores } from "./stores.ts";
|
import { effectiveSettings, stores } from "./stores.ts";
|
||||||
import { searchAvailable, searchProvider } from "./search.ts";
|
import { searchAvailable, searchProvider } from "./search.ts";
|
||||||
import {
|
import {
|
||||||
extractionEnabled,
|
extractionEnabled,
|
||||||
graphContextEnabled,
|
|
||||||
resolveConversationSpec,
|
resolveConversationSpec,
|
||||||
resolveRoleSpec,
|
resolveRoleSpec,
|
||||||
} from "../core/settings-resolution.ts";
|
} from "../core/settings-resolution.ts";
|
||||||
import { modelsNotConfigured } from "../core/settings.ts";
|
import { modelsNotConfigured } from "../core/settings.ts";
|
||||||
import { beginTurn, endTurn } from "./turn-liveness.ts";
|
import { beginTurn, endTurn } from "./turn-liveness.ts";
|
||||||
import { persistTurn, triggerExtraction, triggerGraphContext, withMessageIds } from "./turn-service.ts";
|
import { persistTurn, triggerExtraction, withMessageIds } from "./turn-service.ts";
|
||||||
import type { CallUsage } from "../core/contracts.ts";
|
import type { CallUsage } from "../core/contracts.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -63,9 +60,11 @@ const jobRunner = createJobRunner({
|
|||||||
* (/api/graph/import) hit the same single-flight registry. */
|
* (/api/graph/import) hit the same single-flight registry. */
|
||||||
export const extraction = createExtractionPipeline({
|
export const extraction = createExtractionPipeline({
|
||||||
...stores,
|
...stores,
|
||||||
// The provider-embedder batch (BYOK wave 6): post-merge embedding —
|
// Wave 7 (owner ruling 2026-09-23): embeddings are DEAD — the pipeline's
|
||||||
// resolves the model fresh per call; null model = info-skip inside.
|
// embed seam becomes a no-op until Teacher's pass removes the phase
|
||||||
embedder: embedNewNodes,
|
// outright. Nothing embedded, nothing spent, the ledger untouched
|
||||||
|
// (its data is inert anyway: nothing reads it but the dying injection).
|
||||||
|
embedder: async () => ({ embedded: 0, total: 0, skipped: "embeddings removed (wave 7)" }),
|
||||||
// BYOK (owner, 2026-09-22): the extraction stages resolve the chain —
|
// BYOK (owner, 2026-09-22): the extraction stages resolve the chain —
|
||||||
// role override → slot → NOTHING — read FRESH per stage call; an empty
|
// role override → slot → NOTHING — read FRESH per stage call; an empty
|
||||||
// chain flows through as undefined and the pipeline refuses the run
|
// chain flows through as undefined and the pipeline refuses the run
|
||||||
@@ -74,30 +73,6 @@ export const extraction = createExtractionPipeline({
|
|||||||
resolveEngine(role),
|
resolveEngine(role),
|
||||||
});
|
});
|
||||||
|
|
||||||
/** Graph context injection (vision-context-injection.md): the runner's
|
|
||||||
* deps — same storage ports, the provider-embedder RESOLVER as the query
|
|
||||||
* seam (BYOK wave 6: fresh setting + key reads per run, so vector-space
|
|
||||||
* parity with the ledger is checked against the CURRENT model; a key or
|
|
||||||
* settings PUT between dispatch and spawn lands), and the wake pipeline
|
|
||||||
* as the delivery seam. */
|
|
||||||
export const graphContext: GraphContextDeps = {
|
|
||||||
chatStore: stores.chatStore,
|
|
||||||
jobStore: stores.jobStore,
|
|
||||||
graphStore: stores.graphStore,
|
|
||||||
embedder: () => {
|
|
||||||
const resolved = embeddingPort();
|
|
||||||
if (!resolved.ok) return null;
|
|
||||||
const port = resolved.port;
|
|
||||||
return {
|
|
||||||
model: port.model,
|
|
||||||
embedQuery: async (text: string) => {
|
|
||||||
const result = await port.embed([text], "query");
|
|
||||||
return { vector: result.vectors[0]!, promptTokens: result.promptTokens };
|
|
||||||
},
|
|
||||||
};
|
|
||||||
},
|
|
||||||
deliver: deliverJobResult,
|
|
||||||
};
|
|
||||||
|
|
||||||
// Wake-pipeline wiring (contracts.ts "THE ASYNC PROTOCOL" §4): the pipeline
|
// Wake-pipeline wiring (contracts.ts "THE ASYNC PROTOCOL" §4): the pipeline
|
||||||
// fires its server-side teacher turns through THIS composition root, and
|
// fires its server-side teacher turns through THIS composition root, and
|
||||||
@@ -125,7 +100,6 @@ function resolveEngine(role: ModelRole): PiEngine | undefined {
|
|||||||
}
|
}
|
||||||
|
|
||||||
setJobRespawn((jobId) => void jobRunner.runJob(jobId));
|
setJobRespawn((jobId) => void jobRunner.runJob(jobId));
|
||||||
setContextRespawn((jobId) => void runGraphContextJob(graphContext, jobId));
|
|
||||||
|
|
||||||
/** The message-shape gate the SDK's UIMessageValidator owned: a malformed
|
/** The message-shape gate the SDK's UIMessageValidator owned: a malformed
|
||||||
* body must fail BEFORE the stream exists — the path where no onFinish will
|
* body must fail BEFORE the stream exists — the path where no onFinish will
|
||||||
@@ -219,27 +193,7 @@ export async function handleChatRequest(
|
|||||||
console.error(`[chat ${chatId}] write-ahead persist failed:`, err);
|
console.error(`[chat ${chatId}] write-ahead persist failed:`, err);
|
||||||
}
|
}
|
||||||
|
|
||||||
// THE GRAPH READ CHANNEL (vision-context-injection.md): one shot per
|
|
||||||
// chat, at session start — fired HERE, at turn start, not in onFinish:
|
|
||||||
// the injection's only input is the first typed message (the topic),
|
|
||||||
// which the write-ahead persist above just made durable, so it runs
|
|
||||||
// CONCURRENT with the prober's turn (embedding + cosine scan, no model
|
|
||||||
// call) and the dock shows the job the moment the session starts. Its
|
|
||||||
// steer then waits in the wake queue through the turn (waitForIdle) —
|
|
||||||
// exactly the same hold as a sub-agent result finishing mid-turn. All
|
|
||||||
// gates (exactly-one-text-message topic, budget, one-shot) live in the
|
|
||||||
// runner module; for every session past its start this is a no-op.
|
|
||||||
// Extraction stays in onFinish by contrast: it depends on quiz
|
|
||||||
// answers accumulated DURING the turn.
|
|
||||||
// GRAPH GATES (settings-storage wave, 2026-09-19): the read channel
|
|
||||||
// needs BOTH the master switch and the embedding rung (injection is a
|
|
||||||
// vector query); with graph off there is no context job at all.
|
|
||||||
// BYOK (wave 6): an unconfigured embedding model means no vectors to
|
|
||||||
// query — the dispatch skips honestly (the runner's own resolution is
|
|
||||||
// the belt-and-suspenders for respawn races).
|
|
||||||
if (graphContextEnabled(effective.graph) && effective.graph.embeddingModel !== null) {
|
|
||||||
triggerGraphContext(graphContext, chatId);
|
|
||||||
}
|
|
||||||
|
|
||||||
// MONEY CHANNEL (wave 4): one collector per request, fed by the agent's
|
// MONEY CHANNEL (wave 4): one collector per request, fed by the agent's
|
||||||
// message_end events. Every model call's usage lands in the chat record's
|
// message_end events. Every model call's usage lands in the chat record's
|
||||||
|
|||||||
@@ -1,493 +0,0 @@
|
|||||||
/**
|
|
||||||
* BYOK PROVIDER EMBEDDINGS (wave 6, 2026-09-23): the local transformers
|
|
||||||
* embedder is dead — vectors now come from the user's keyed provider
|
|
||||||
* (OpenRouter / OpenAI / Google), named by graph.embeddingModel
|
|
||||||
* ("provider:id", null = honestly off; skip with an info log, the graph
|
|
||||||
* stays textual — owner ruling 2026-09-23).
|
|
||||||
*
|
|
||||||
* Owner decisions baked in:
|
|
||||||
* - NO factory model (BYOK, like the model slots): null is the floor.
|
|
||||||
* - Adapters resolve FRESH per call from the setting + the auth store
|
|
||||||
* (the search port's treatment — key rotations land immediately); a
|
|
||||||
* key missing for a SET model is an honest error, never a fallback.
|
|
||||||
* - Vectors stored as the provider returns them (default dims, no
|
|
||||||
* truncation, no normalization). The ledger is self-describing on
|
|
||||||
* model + dim; a model change voids it (full re-embed, minutes at
|
|
||||||
* current scale — the local embedder's same staleness rule).
|
|
||||||
* - Usage (prompt tokens) flows to the money channel: every API call
|
|
||||||
* reports one CallUsage step (outputTokens: 0 — embeddings emit no
|
|
||||||
* output; priced via the hand-maintained EMBEDDING_PRICES overlay).
|
|
||||||
*
|
|
||||||
* The adapters are injected-fetch dependencies so tests never touch the
|
|
||||||
* network; wire protocols VERIFIED against provider docs 2026-09-23:
|
|
||||||
* - OpenRouter: POST /api/v1/embeddings (Bearer), input_type
|
|
||||||
* search_document|search_query; catalog GET /api/v1/embeddings/models
|
|
||||||
* (VERIFIED live 2026-09-23 — the inverse path /api/v1/models/
|
|
||||||
* embeddings 404s).
|
|
||||||
* - OpenAI: POST /v1/embeddings (Bearer); catalog GET /v1/models.
|
|
||||||
* - Google: POST /v1beta/models/{m}:batchEmbedContents (x-goog-api-key),
|
|
||||||
* taskType RETRIEVAL_DOCUMENT|RETRIEVAL_QUERY; catalog GET /v1beta/models
|
|
||||||
* filtered on supportedGenerationMethods ∋ embedContent.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import type { CallUsage, EmbeddingsLedger, GraphNode } from "../core/contracts.ts";
|
|
||||||
import type { ExtractionStageUsage } from "../core/contracts/chat.ts";
|
|
||||||
import type { EmbeddingPort, EmbeddingResult, EmbeddingTask } from "../core/ports.ts";
|
|
||||||
import { semanticClusters, SEMANTIC_SIM_THRESHOLD } from "../core/semantic-clusters.ts";
|
|
||||||
import { keyValue } from "./keys.ts";
|
|
||||||
import { effectiveSettings } from "./stores.ts";
|
|
||||||
|
|
||||||
/** The wire-callable shape the adapters close over (tests inject a stub). */
|
|
||||||
type FetchLike = (input: string, init: RequestInit) => Promise<Response>;
|
|
||||||
|
|
||||||
/** One provider embedding API call — kept small enough to never trip
|
|
||||||
* per-request input limits (propositions are tiny; 32 was the local
|
|
||||||
* embedder's batch and it stays). */
|
|
||||||
const EMBED_BATCH = 32;
|
|
||||||
/** Per-request timeout: a hung provider call must not hang an extraction
|
|
||||||
* run (the wave-1 hung-job lesson). */
|
|
||||||
const REQUEST_TIMEOUT_MS = 60_000;
|
|
||||||
|
|
||||||
/** The providers this wave ships + their auth vars (the curated key
|
|
||||||
* catalog, same ids the auth store and /api/keys use). */
|
|
||||||
const PROVIDER_KEYS = {
|
|
||||||
openrouter: "OPENROUTER_API_KEY",
|
|
||||||
openai: "OPENAI_API_KEY",
|
|
||||||
google: "GEMINI_API_KEY",
|
|
||||||
} as const;
|
|
||||||
export type EmbeddingProviderId = keyof typeof PROVIDER_KEYS;
|
|
||||||
|
|
||||||
/** "provider:id" → {provider, id}; null when malformed (no guessing). */
|
|
||||||
export function parseEmbeddingModelSlug(slug: string): { provider: EmbeddingProviderId; id: string } | null {
|
|
||||||
const colon = slug.indexOf(":");
|
|
||||||
if (colon <= 0 || colon === slug.length - 1) return null;
|
|
||||||
const provider = slug.slice(0, colon) as EmbeddingProviderId;
|
|
||||||
if (!(provider in PROVIDER_KEYS)) return null;
|
|
||||||
return { provider, id: slug.slice(colon + 1) };
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Adapters (one per provider; pure injected-fetch factories) ──────────────
|
|
||||||
|
|
||||||
/** Shared: HTTP-error → clear Error (the search adapters' mapping). */
|
|
||||||
async function demandOk(resp: Response, label: string): Promise<void> {
|
|
||||||
if (resp.ok) return;
|
|
||||||
const body = await resp.text();
|
|
||||||
throw new Error(`${label} API ${resp.status}: ${body.slice(0, 200)}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Shared: normalize a provider's {embedding, index} rows to input order. */
|
|
||||||
function rowsInOrder(
|
|
||||||
data: Array<{ embedding?: unknown; index?: unknown }>,
|
|
||||||
count: number,
|
|
||||||
label: string,
|
|
||||||
): number[][] {
|
|
||||||
const byIndex = new Map<number, number[]>();
|
|
||||||
for (const row of data) {
|
|
||||||
if (
|
|
||||||
Array.isArray(row.embedding) &&
|
|
||||||
row.embedding.every((x) => typeof x === "number") &&
|
|
||||||
typeof row.index === "number"
|
|
||||||
) {
|
|
||||||
byIndex.set(row.index, row.embedding as number[]);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
const out: number[][] = [];
|
|
||||||
for (let i = 0; i < count; i++) {
|
|
||||||
const vec = byIndex.get(i);
|
|
||||||
if (!vec) throw new Error(`${label} API returned ${byIndex.size}/${count} vectors`);
|
|
||||||
out.push(vec);
|
|
||||||
}
|
|
||||||
return out;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** One API call's money-channel step: embeddings emit no output, cache
|
|
||||||
* nothing, reason never — the fields exist (CallUsage requires them) as
|
|
||||||
* absent-is-not-zero undefineds. */
|
|
||||||
function embeddingStep(promptTokens: number): CallUsage {
|
|
||||||
return {
|
|
||||||
inputTokens: promptTokens,
|
|
||||||
outputTokens: 0,
|
|
||||||
cacheReadTokens: undefined,
|
|
||||||
cacheWriteTokens: undefined,
|
|
||||||
reasoningTokens: undefined,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** OpenRouter: POST https://openrouter.ai/api/v1/embeddings — Bearer auth,
|
|
||||||
* input_type hints the retrieval side (verified against the docs 2026-09-23). */
|
|
||||||
export function openrouterEmbeddingProvider(model: string, apiKey: string, fetchFn: FetchLike = fetch): EmbeddingPort {
|
|
||||||
const parsed = parseEmbeddingModelSlug(model);
|
|
||||||
if (!parsed) throw new Error(`invalid embedding model slug: ${model}`);
|
|
||||||
return {
|
|
||||||
model,
|
|
||||||
async embed(texts: readonly string[], task: EmbeddingTask): Promise<EmbeddingResult> {
|
|
||||||
const resp = await fetchFn("https://openrouter.ai/api/v1/embeddings", {
|
|
||||||
method: "POST",
|
|
||||||
headers: { "content-type": "application/json", authorization: `Bearer ${apiKey}` },
|
|
||||||
body: JSON.stringify({
|
|
||||||
model: parsed.id,
|
|
||||||
input: texts,
|
|
||||||
input_type: task === "query" ? "search_query" : "search_document",
|
|
||||||
}),
|
|
||||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
|
||||||
});
|
|
||||||
await demandOk(resp, "OpenRouter embeddings");
|
|
||||||
const data = (await resp.json()) as {
|
|
||||||
data?: Array<{ embedding?: unknown; index?: unknown }>;
|
|
||||||
usage?: { prompt_tokens?: number };
|
|
||||||
};
|
|
||||||
return {
|
|
||||||
vectors: rowsInOrder(data.data ?? [], texts.length, "OpenRouter embeddings"),
|
|
||||||
model,
|
|
||||||
dim: 0, // filled by the caller from the first vector (below)
|
|
||||||
promptTokens: data.usage?.prompt_tokens,
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** OpenAI: POST https://api.openai.com/v1/embeddings — Bearer auth; the
|
|
||||||
* response's usage carries prompt_tokens (verified 2026-09-23). */
|
|
||||||
export function openaiEmbeddingProvider(model: string, apiKey: string, fetchFn: FetchLike = fetch): EmbeddingPort {
|
|
||||||
const parsed = parseEmbeddingModelSlug(model);
|
|
||||||
if (!parsed) throw new Error(`invalid embedding model slug: ${model}`);
|
|
||||||
return {
|
|
||||||
model,
|
|
||||||
async embed(texts: readonly string[], task: EmbeddingTask): Promise<EmbeddingResult> {
|
|
||||||
// OpenAI's embeddings API takes no task hint (one format per
|
|
||||||
// model); the argument exists for seam parity and is unused here.
|
|
||||||
void task;
|
|
||||||
const resp = await fetchFn("https://api.openai.com/v1/embeddings", {
|
|
||||||
method: "POST",
|
|
||||||
headers: { "content-type": "application/json", authorization: `Bearer ${apiKey}` },
|
|
||||||
body: JSON.stringify({ model: parsed.id, input: texts }),
|
|
||||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
|
||||||
});
|
|
||||||
await demandOk(resp, "OpenAI embeddings");
|
|
||||||
const data = (await resp.json()) as {
|
|
||||||
data?: Array<{ embedding?: unknown; index?: unknown }>;
|
|
||||||
usage?: { prompt_tokens?: number };
|
|
||||||
};
|
|
||||||
return {
|
|
||||||
vectors: rowsInOrder(data.data ?? [], texts.length, "OpenAI embeddings"),
|
|
||||||
model,
|
|
||||||
dim: 0,
|
|
||||||
promptTokens: data.usage?.prompt_tokens,
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Google: POST …/models/{m}:batchEmbedContents — x-goog-api-key, one
|
|
||||||
* request per text with taskType RETRIEVAL_QUERY|RETRIEVAL_DOCUMENT
|
|
||||||
* (verified against the Gemini API docs 2026-09-23). */
|
|
||||||
export function googleEmbeddingProvider(model: string, apiKey: string, fetchFn: FetchLike = fetch): EmbeddingPort {
|
|
||||||
const parsed = parseEmbeddingModelSlug(model);
|
|
||||||
if (!parsed) throw new Error(`invalid embedding model slug: ${model}`);
|
|
||||||
return {
|
|
||||||
model,
|
|
||||||
async embed(texts: readonly string[], task: EmbeddingTask): Promise<EmbeddingResult> {
|
|
||||||
const resp = await fetchFn(
|
|
||||||
`https://generativelanguage.googleapis.com/v1beta/models/${encodeURIComponent(parsed.id)}:batchEmbedContents`,
|
|
||||||
{
|
|
||||||
method: "POST",
|
|
||||||
headers: { "content-type": "application/json", "x-goog-api-key": apiKey },
|
|
||||||
body: JSON.stringify({
|
|
||||||
requests: texts.map((text) => ({
|
|
||||||
model: `models/${parsed.id}`,
|
|
||||||
content: { parts: [{ text }] },
|
|
||||||
taskType: task === "query" ? "RETRIEVAL_QUERY" : "RETRIEVAL_DOCUMENT",
|
|
||||||
})),
|
|
||||||
}),
|
|
||||||
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
|
|
||||||
},
|
|
||||||
);
|
|
||||||
await demandOk(resp, "Google embeddings");
|
|
||||||
const data = (await resp.json()) as {
|
|
||||||
embeddings?: Array<{ values?: unknown }>;
|
|
||||||
/** embedContent responses carry usageMetadata; the batch
|
|
||||||
* shape may too — parsed opportunistically, never guessed. */
|
|
||||||
usageMetadata?: { promptTokenCount?: number };
|
|
||||||
};
|
|
||||||
const vectors = (data.embeddings ?? []).map((row, i) => {
|
|
||||||
if (!Array.isArray(row.values) || !row.values.every((x) => typeof x === "number")) {
|
|
||||||
throw new Error(`Google embeddings API returned a bad vector at index ${i}`);
|
|
||||||
}
|
|
||||||
return row.values as number[];
|
|
||||||
});
|
|
||||||
if (vectors.length !== texts.length) {
|
|
||||||
throw new Error(`Google embeddings API returned ${vectors.length}/${texts.length} vectors`);
|
|
||||||
}
|
|
||||||
return { vectors, model, dim: 0, promptTokens: data.usageMetadata?.promptTokenCount };
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── Resolution (fresh per call — the search port's treatment) ───────────────
|
|
||||||
|
|
||||||
/** The resolved state: a live port, or the honest reason it isn't. */
|
|
||||||
export type EmbeddingResolution =
|
|
||||||
| { ok: true; port: EmbeddingPort }
|
|
||||||
| { ok: false; reason: string };
|
|
||||||
|
|
||||||
/** Pure resolution from a setting slug + a key lookup (testable): null
|
|
||||||
* slug = off; a set slug with no key = honest unavailability. */
|
|
||||||
export function resolveEmbeddingPortFor(
|
|
||||||
slug: string | null | undefined,
|
|
||||||
keyOf: (envVar: string) => string | undefined,
|
|
||||||
fetchFn?: FetchLike,
|
|
||||||
): EmbeddingResolution {
|
|
||||||
if (slug === null || slug === undefined || slug === "") {
|
|
||||||
return { ok: false, reason: "no embedding model configured (graph.embeddingModel is null)" };
|
|
||||||
}
|
|
||||||
const parsed = parseEmbeddingModelSlug(slug);
|
|
||||||
if (!parsed) {
|
|
||||||
return { ok: false, reason: `invalid embedding model slug: ${slug}` };
|
|
||||||
}
|
|
||||||
const apiKey = keyOf(PROVIDER_KEYS[parsed.provider]);
|
|
||||||
if (!apiKey) {
|
|
||||||
return {
|
|
||||||
ok: false,
|
|
||||||
reason: `embedding model is set to "${slug}" but its ${PROVIDER_KEYS[parsed.provider]} key is missing`,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
const factory =
|
|
||||||
parsed.provider === "openrouter"
|
|
||||||
? openrouterEmbeddingProvider
|
|
||||||
: parsed.provider === "openai"
|
|
||||||
? openaiEmbeddingProvider
|
|
||||||
: googleEmbeddingProvider;
|
|
||||||
return { ok: true, port: factory(slug, apiKey, fetchFn) };
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The LIVE resolution — reads the settings record + auth store fresh on
|
|
||||||
* every call, so a settings PUT or a key save lands on the next call. */
|
|
||||||
export function embeddingPort(): EmbeddingResolution {
|
|
||||||
return resolveEmbeddingPortFor(effectiveSettings().graph.embeddingModel, keyValue);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ─── The post-merge batch (the extraction pipeline's embedding tail) ────────
|
|
||||||
|
|
||||||
/** What the embedding tail reports back: counts for logs, the skip reason
|
|
||||||
* when nothing ran (owner ruling: null model = info skip, not error), the
|
|
||||||
* failure message when it died partway (owner ruling 2026-09-23:
|
|
||||||
* usage-on-error — a multi-batch embed that fails mid-run still reports
|
|
||||||
* the spend of the batches that landed, so the money channel never
|
|
||||||
* loses a billed call), and the money-channel stage when it billed (one
|
|
||||||
* CallUsage step per API call, outputTokens: 0 — priced via
|
|
||||||
* EMBEDDING_PRICES). */
|
|
||||||
export interface EmbedBatchResult {
|
|
||||||
embedded: number;
|
|
||||||
total: number;
|
|
||||||
skipped?: string;
|
|
||||||
/** Set when an embed batch failed after others may have landed. The
|
|
||||||
* landed vectors are already persisted (pay-once, owner ruling A
|
|
||||||
* 2026-09-23) — the next run embeds only what remains. */
|
|
||||||
failed?: string;
|
|
||||||
usage?: ExtractionStageUsage;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Embed every node the ledger is missing — the post-merge hook (new
|
|
||||||
* propositions only, typically a handful) AND the cold-start backfill
|
|
||||||
* (empty ledger = every node) in one function. A model change voids the
|
|
||||||
* ledger first (full re-embed — vectors from different models live in
|
|
||||||
* incompatible spaces). Also prunes vectors for nodes that no longer
|
|
||||||
* exist, then re-clusters ONCE per batch (consumers read the persisted
|
|
||||||
* partition). Non-fatal by contract: the caller treats failure as
|
|
||||||
* "clustering runs without the new nodes until the next merge retries".
|
|
||||||
* Usage-on-error + pay-once (owner ruling 2026-09-23): an embed batch
|
|
||||||
* that throws never propagates — the batches that already landed are
|
|
||||||
* persisted (each text is paid for exactly once; a retry continues with
|
|
||||||
* the remaining nodes), their spend still reaches the money channel via
|
|
||||||
* usage, and `failed` carries the error for logs and callers.
|
|
||||||
*/
|
|
||||||
export async function embedNewNodes(
|
|
||||||
graphStore: {
|
|
||||||
readNodes(): GraphNode[];
|
|
||||||
readEmbeddings(): EmbeddingsLedger;
|
|
||||||
writeEmbeddings(ledger: EmbeddingsLedger): void;
|
|
||||||
},
|
|
||||||
/** Test seam: the live resolution by default; tests inject a scripted
|
|
||||||
* port (they never read the machine's settings/auth files). */
|
|
||||||
resolvePort: () => EmbeddingResolution = embeddingPort,
|
|
||||||
): Promise<EmbedBatchResult> {
|
|
||||||
const resolved = resolvePort();
|
|
||||||
const ledger = graphStore.readEmbeddings();
|
|
||||||
if (!resolved.ok) {
|
|
||||||
console.info(`[embeddings] skipped — ${resolved.reason}`);
|
|
||||||
return { embedded: 0, total: Object.keys(ledger.vectors).length, skipped: resolved.reason };
|
|
||||||
}
|
|
||||||
const port = resolved.port;
|
|
||||||
const stale = ledger.model !== port.model;
|
|
||||||
const vectors: Record<string, number[]> = stale ? {} : { ...ledger.vectors };
|
|
||||||
const steps: NonNullable<EmbedBatchResult["usage"]>["steps"] = [];
|
|
||||||
let dim = ledger.dim;
|
|
||||||
|
|
||||||
// Node LINES may repeat an id — dedupe, first defines (getGraph rule).
|
|
||||||
const seen = new Set<string>();
|
|
||||||
const todo: GraphNode[] = [];
|
|
||||||
for (const node of graphStore.readNodes()) {
|
|
||||||
if (seen.has(node.id)) continue;
|
|
||||||
seen.add(node.id);
|
|
||||||
if (!vectors[node.id]) todo.push(node);
|
|
||||||
}
|
|
||||||
|
|
||||||
let landed = 0;
|
|
||||||
let failed: string | undefined;
|
|
||||||
for (let i = 0; i < todo.length; i += EMBED_BATCH) {
|
|
||||||
const batch = todo.slice(i, i + EMBED_BATCH);
|
|
||||||
let result: EmbeddingResult;
|
|
||||||
try {
|
|
||||||
result = await port.embed(
|
|
||||||
batch.map((n) => n.statement),
|
|
||||||
"document",
|
|
||||||
);
|
|
||||||
} catch (err) {
|
|
||||||
// Usage-on-error + pay-once (owner ruling 2026-09-23): stop
|
|
||||||
// here, keep the batches that landed, report the failure — the
|
|
||||||
// throw never escapes this function.
|
|
||||||
failed = err instanceof Error ? err.message : String(err);
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
for (let j = 0; j < batch.length; j++) {
|
|
||||||
vectors[batch[j]!.id] = result.vectors[j]!;
|
|
||||||
landed++;
|
|
||||||
}
|
|
||||||
if (result.vectors[0]) dim = result.vectors[0].length;
|
|
||||||
if (result.promptTokens !== undefined) {
|
|
||||||
// outputTokens: 0 — embeddings emit no output; callCost needs it
|
|
||||||
// defined to price the input at all.
|
|
||||||
steps.push(embeddingStep(result.promptTokens));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (failed !== undefined) {
|
|
||||||
console.error(
|
|
||||||
`[embeddings] embedding failed after ${landed}/${todo.length} nodes (non-fatal; landed vectors persist, the next run embeds only what remains): ${failed}`,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Prune ghosts: vectors for node ids that no longer exist.
|
|
||||||
let pruned = 0;
|
|
||||||
for (const id of Object.keys(vectors)) {
|
|
||||||
if (!seen.has(id)) {
|
|
||||||
delete vectors[id];
|
|
||||||
pruned++;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// RECLUSTER AT THE END OF THE BATCH (user-decided 2026-09-13): the
|
|
||||||
// agglomerative pass runs HERE — once per embedding batch — and
|
|
||||||
// consumers (snapshot, settle, namer) read the persisted partition.
|
|
||||||
const clusters = semanticClusters(vectors, { similarityThreshold: SEMANTIC_SIM_THRESHOLD });
|
|
||||||
|
|
||||||
const wrote =
|
|
||||||
failed !== undefined
|
|
||||||
? landed > 0
|
|
||||||
: todo.length > 0 || pruned > 0 || stale || ledger.clusters.length !== clusters.length;
|
|
||||||
if (wrote) {
|
|
||||||
graphStore.writeEmbeddings({
|
|
||||||
model: port.model,
|
|
||||||
dim,
|
|
||||||
vectors,
|
|
||||||
clusters,
|
|
||||||
updatedAt: new Date().toISOString(),
|
|
||||||
});
|
|
||||||
}
|
|
||||||
const result: EmbedBatchResult = {
|
|
||||||
embedded: landed,
|
|
||||||
total: Object.keys(vectors).length,
|
|
||||||
usage: steps.length > 0 ? { model: port.model, steps } : undefined,
|
|
||||||
};
|
|
||||||
if (failed !== undefined) result.failed = failed;
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
// ─── The federated embedding catalog (wave 6, owner ruling 2026-09-23) ──────
|
|
||||||
|
|
||||||
/** One catalog row: provider-qualified id (the setting's exact slug
|
|
||||||
* format) + display name. */
|
|
||||||
export interface EmbeddingCatalogModel {
|
|
||||||
/** The full "provider:id" slug — what graph.embeddingModel stores. */
|
|
||||||
id: string;
|
|
||||||
/** Human label for the picker (the provider's name, when given). */
|
|
||||||
name: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The embedding models of every KEYED provider, fetched live (the
|
|
||||||
* chat-models picker is a static pi-ai catalog, but pi-ai carries no
|
|
||||||
* embedding models — so this endpoint federates the providers' own
|
|
||||||
* lists). A provider without a key contributes nothing; a provider whose
|
|
||||||
* catalog call fails is skipped with an error log (never poisons the
|
|
||||||
* rest, never fails the endpoint). Empty array = nothing keyed. */
|
|
||||||
export async function embeddingModelCatalog(
|
|
||||||
fetchFn: FetchLike = fetch,
|
|
||||||
/** Test seam: the live key lookup by default; tests inject scripted
|
|
||||||
* keys (they never read the machine's auth file). */
|
|
||||||
keyOf: (id: string) => string | undefined = keyValue,
|
|
||||||
): Promise<EmbeddingCatalogModel[]> {
|
|
||||||
const out: EmbeddingCatalogModel[] = [];
|
|
||||||
const fetchJson = async (url: string, headers: Record<string, string>): Promise<unknown> => {
|
|
||||||
const resp = await fetchFn(url, { headers, signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS) });
|
|
||||||
await demandOk(resp, "Embeddings catalog");
|
|
||||||
return resp.json();
|
|
||||||
};
|
|
||||||
|
|
||||||
// OpenRouter: GET /api/v1/embeddings/models — the dedicated embedding
|
|
||||||
// list (VERIFIED live 2026-09-23: the inverse path /api/v1/models/
|
|
||||||
// embeddings 404s; this one returns {data: [{id, name, …}]}). Paginated
|
|
||||||
// — limit=500 covers the whole list at current size (33).
|
|
||||||
if (keyOf(PROVIDER_KEYS.openrouter)) {
|
|
||||||
try {
|
|
||||||
const data = (await fetchJson("https://openrouter.ai/api/v1/embeddings/models?limit=500", {
|
|
||||||
authorization: `Bearer ${keyOf(PROVIDER_KEYS.openrouter)}`,
|
|
||||||
})) as { data?: Array<{ id?: unknown; name?: unknown }> };
|
|
||||||
for (const m of data.data ?? []) {
|
|
||||||
if (typeof m.id !== "string" || !m.id) continue;
|
|
||||||
out.push({ id: `openrouter:${m.id}`, name: typeof m.name === "string" && m.name ? m.name : m.id });
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
console.error("[embeddings] OpenRouter catalog fetch failed:", err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// OpenAI: GET /v1/models — no embedding-only list, so filter the
|
|
||||||
// generic one on the embedding naming convention.
|
|
||||||
if (keyOf(PROVIDER_KEYS.openai)) {
|
|
||||||
try {
|
|
||||||
const data = (await fetchJson("https://api.openai.com/v1/models", {
|
|
||||||
authorization: `Bearer ${keyOf(PROVIDER_KEYS.openai)}`,
|
|
||||||
})) as { data?: Array<{ id?: unknown }> };
|
|
||||||
for (const m of data.data ?? []) {
|
|
||||||
if (typeof m.id !== "string" || !/^text-embedding-/.test(m.id)) continue;
|
|
||||||
out.push({ id: `openai:${m.id}`, name: m.id });
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
console.error("[embeddings] OpenAI catalog fetch failed:", err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Google: GET /v1beta/models — filter on the methods the model
|
|
||||||
// actually serves (embedContent present).
|
|
||||||
if (keyOf(PROVIDER_KEYS.google)) {
|
|
||||||
try {
|
|
||||||
const data = (await fetchJson("https://generativelanguage.googleapis.com/v1beta/models?pageSize=1000", {
|
|
||||||
"x-goog-api-key": keyOf(PROVIDER_KEYS.google)!,
|
|
||||||
})) as {
|
|
||||||
models?: Array<{ name?: unknown; displayName?: unknown; supportedGenerationMethods?: unknown }>;
|
|
||||||
};
|
|
||||||
for (const m of data.models ?? []) {
|
|
||||||
if (typeof m.name !== "string" || !m.name.startsWith("models/")) continue;
|
|
||||||
if (!Array.isArray(m.supportedGenerationMethods) || !m.supportedGenerationMethods.includes("embedContent")) continue;
|
|
||||||
const id = m.name.slice("models/".length);
|
|
||||||
out.push({
|
|
||||||
id: `google:${id}`,
|
|
||||||
name: typeof m.displayName === "string" && m.displayName ? m.displayName : id,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
console.error("[embeddings] Google catalog fetch failed:", err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return out;
|
|
||||||
}
|
|
||||||
@@ -6,7 +6,6 @@
|
|||||||
|
|
||||||
export * from "./chat-handler.ts";
|
export * from "./chat-handler.ts";
|
||||||
export * from "./chats.ts";
|
export * from "./chats.ts";
|
||||||
export * from "./embeddings.ts";
|
|
||||||
export * from "./graph.ts";
|
export * from "./graph.ts";
|
||||||
export * from "./commit-positions.ts";
|
export * from "./commit-positions.ts";
|
||||||
export * from "./jobs.ts";
|
export * from "./jobs.ts";
|
||||||
|
|||||||
+12
-31
@@ -289,41 +289,22 @@ export function parseAppearanceSettingsBody(body: unknown): SettingsBodyResult<A
|
|||||||
return { ok: true, settings };
|
return { ok: true, settings };
|
||||||
}
|
}
|
||||||
|
|
||||||
/** graph: the four-rung ladder { enabled?, embedding?, clustering?, labeling? }
|
/** graph: the collapsed section { enabled? } (wave 7, owner ruling
|
||||||
* + embeddingModel? — the BYOK provider embedding model (wave 6,
|
* 2026-09-23): the master switch ONLY — the embedding/clustering/labeling
|
||||||
* 2026-09-23): null or a "provider:id" slug of a shipped provider.
|
* rungs and the BYOK embeddingModel died with the provider embedder
|
||||||
* Catalog validity is network-dependent — the parser checks SHAPE only
|
* (clustering is the free birth-session partition, labels are session
|
||||||
* (the federated /api/models/embeddings gives the picker real options). */
|
* names). Old-shape keys in a body are ignored, not rejected. */
|
||||||
export function parseGraphSettingsBody(body: unknown): SettingsBodyResult<GraphSettings> {
|
export function parseGraphSettingsBody(body: unknown): SettingsBodyResult<GraphSettings> {
|
||||||
const raw = asRecord(body);
|
const raw = asRecord(body);
|
||||||
if (!raw) return { ok: false, error: "Invalid JSON body" };
|
if (!raw) return { ok: false, error: "Invalid JSON body" };
|
||||||
|
// Wave 7 collapse (owner ruling 2026-09-23): the section is the master
|
||||||
|
// switch ONLY. PUT is replace-whole — a body like {enabled: true} IS the
|
||||||
|
// whole section now; the dead rungs' keys in old records are unread
|
||||||
|
// (the sanitizer drops them; see sanitizeGraph).
|
||||||
const settings: GraphSettings = {};
|
const settings: GraphSettings = {};
|
||||||
for (const key of ["enabled", "embedding", "clustering", "labeling"] as const) {
|
if (raw.enabled !== undefined) {
|
||||||
if (raw[key] !== undefined) {
|
if (typeof raw.enabled !== "boolean") return { ok: false, error: "enabled must be a boolean" };
|
||||||
if (typeof raw[key] !== "boolean") {
|
settings.enabled = raw.enabled;
|
||||||
return { ok: false, error: `${key} must be a boolean` };
|
|
||||||
}
|
|
||||||
settings[key] = raw[key];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (raw.embeddingModel !== undefined) {
|
|
||||||
if (raw.embeddingModel !== null) {
|
|
||||||
// NOTE: the id may itself contain colons (OpenRouter suffixes like
|
|
||||||
// "liquid/lfm-2.5-embedding-350m:free") — the provider is the part
|
|
||||||
// before the FIRST colon, so the id half just forbids whitespace.
|
|
||||||
if (
|
|
||||||
typeof raw.embeddingModel !== "string" ||
|
|
||||||
!/^(openrouter|openai|google):\S+$/.test(raw.embeddingModel)
|
|
||||||
) {
|
|
||||||
return {
|
|
||||||
ok: false,
|
|
||||||
error: 'embeddingModel must be null or a "openrouter|openai|google:model-id" slug',
|
|
||||||
};
|
|
||||||
}
|
|
||||||
settings.embeddingModel = raw.embeddingModel;
|
|
||||||
} else {
|
|
||||||
settings.embeddingModel = null;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
return { ok: true, settings };
|
return { ok: true, settings };
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -21,7 +21,6 @@ import { coverageOf } from "../core/coverage.ts";
|
|||||||
import type { ChatStore, GraphStore } from "../core/ports.ts";
|
import type { ChatStore, GraphStore } from "../core/ports.ts";
|
||||||
import { EXTRACTION } from "../../config.ts";
|
import { EXTRACTION } from "../../config.ts";
|
||||||
import type { ExtractionPipeline } from "../agent/subagents/extraction.ts";
|
import type { ExtractionPipeline } from "../agent/subagents/extraction.ts";
|
||||||
import { startGraphContextInjection, type GraphContextDeps } from "../agent/subagents/graph-context.ts";
|
|
||||||
import { ModelsNotConfiguredError } from "../core/settings.ts";
|
import { ModelsNotConfiguredError } from "../core/settings.ts";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -113,21 +112,3 @@ export function triggerExtraction(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* The graph READ channel (vision-context-injection.md): one shot per chat,
|
|
||||||
* at session start — the first typed message is the session's topic; the
|
|
||||||
* semantically closest propositions are selected under the session's token
|
|
||||||
* budget and steered in through the wake pipeline. Fire-and-forget like
|
|
||||||
* triggerExtraction; the one-shot gates (topic, budget, prior injection)
|
|
||||||
* live in the runner module, not here — this seam only decides WHEN to ask.
|
|
||||||
* Skipped under tests, same doctrine: the runner is tested directly with a
|
|
||||||
* scripted embedder.
|
|
||||||
*/
|
|
||||||
export function triggerGraphContext(deps: GraphContextDeps, chatId: string): void {
|
|
||||||
if (process.env.NODE_ENV === "test") return;
|
|
||||||
try {
|
|
||||||
startGraphContextInjection(deps, chatId);
|
|
||||||
} catch (err) {
|
|
||||||
console.error(`[chat ${chatId}] graph-context trigger failed:`, err);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -96,39 +96,6 @@ describe("graphCostSummary (the cost pill's fold)", () => {
|
|||||||
assert.equal(s.unpricedRuns, 1); // run-level: only the both-stages run
|
assert.equal(s.unpricedRuns, 1); // run-level: only the both-stages run
|
||||||
});
|
});
|
||||||
|
|
||||||
test("the embedding stage (wave 6): folds when present, absent when the run embedded nothing", () => {
|
|
||||||
const EMBED_MODEL = "openai:text-embedding-3-small";
|
|
||||||
const embStep = callCost(step(5000, 0), EMBED_MODEL)!;
|
|
||||||
assert.ok(embStep > 0, "the overlay prices the known embedding models");
|
|
||||||
const withEmbed = extractionRun(100, PRICED, PRICED);
|
|
||||||
withEmbed.stages.embedding = { model: EMBED_MODEL, steps: [step(5000, 0)] };
|
|
||||||
const s = graphCostSummary({
|
|
||||||
chats: [{ title: "a", extractions: [withEmbed] }, { title: "b", extractions: [extractionRun(200, PRICED, PRICED)] }],
|
|
||||||
sources: [],
|
|
||||||
naming: [],
|
|
||||||
});
|
|
||||||
assert.equal(s.stages.embedding.runs, 1, "only the run that embedded");
|
|
||||||
assert.equal(s.stages.embedding.dollars, embStep);
|
|
||||||
assert.equal(s.stages.embedding.tokens.inputTokens, 5000);
|
|
||||||
assert.equal(s.stages.extraction.runs, 2, "the two-stage era still folds");
|
|
||||||
assert.equal(s.stages.extraction.runs, 2);
|
|
||||||
// Run-level: the embedding dollars ride the run's total.
|
|
||||||
const extStep = callCost(step(1000, 200), PRICED)!;
|
|
||||||
const audStep = callCost(step(2000, 100), PRICED)!;
|
|
||||||
assert.equal(s.runs.find((r) => r.startedAt === 100)!.dollars, extStep + audStep + embStep);
|
|
||||||
assert.equal(s.runs.find((r) => r.startedAt === 200)!.dollars, extStep + audStep);
|
|
||||||
assert.equal(s.totalDollars, (extStep + audStep) * 2 + embStep);
|
|
||||||
});
|
|
||||||
|
|
||||||
test("an UNPRICED embedding model stays honest: counted, never a silent zero; a priced sibling saves the run total", () => {
|
|
||||||
const unpricedEmbed = extractionRun(100, PRICED, "unknown-model");
|
|
||||||
unpricedEmbed.stages.embedding = { model: "some/new-embed", steps: [step(900, 0)] };
|
|
||||||
const s = graphCostSummary({ chats: [{ title: "x", extractions: [unpricedEmbed] }], sources: [], naming: [] });
|
|
||||||
assert.equal(s.stages.embedding.unpricedRuns, 1);
|
|
||||||
assert.equal(s.stages.embedding.dollars, 0);
|
|
||||||
assert.notEqual(s.runs[0]!.dollars, null, "the priced extractor saves the run total");
|
|
||||||
});
|
|
||||||
|
|
||||||
test("recent runs cap: newest survive", () => {
|
test("recent runs cap: newest survive", () => {
|
||||||
const extractions = [1, 2, 3, 4].map((n) => extractionRun(n * 100, PRICED, PRICED));
|
const extractions = [1, 2, 3, 4].map((n) => extractionRun(n * 100, PRICED, PRICED));
|
||||||
const s = graphCostSummary({
|
const s = graphCostSummary({
|
||||||
|
|||||||
@@ -1,91 +0,0 @@
|
|||||||
import React from "react";
|
|
||||||
(globalThis as Record<string, unknown>).React = React;
|
|
||||||
|
|
||||||
import { renderToStaticMarkup } from "react-dom/server";
|
|
||||||
import { describe, it } from "node:test";
|
|
||||||
import assert from "node:assert/strict";
|
|
||||||
import {
|
|
||||||
embeddingPriceLabel,
|
|
||||||
GraphSection,
|
|
||||||
needsEmbeddingModel,
|
|
||||||
} from "../components/settings/appearance-section.tsx";
|
|
||||||
import { EMBEDDING_PRICES, type GraphSettings } from "../src/core/index.ts";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Wave 6 chunk B — the embeddings settings seams: the price line for
|
|
||||||
* catalog rows (EMBEDDING_PRICES lookup, honestly "—" when unlisted),
|
|
||||||
* the model-dependency predicate that drives the ladder gray-out, and
|
|
||||||
* GraphSection's SSR shape with and without a configured model.
|
|
||||||
*
|
|
||||||
* The fetches (catalog, node count) live in effects — dead under
|
|
||||||
* renderToStaticMarkup, so the render pins the STORED-state markup only;
|
|
||||||
* the interactive round-trips are live-verified on the dev server.
|
|
||||||
*/
|
|
||||||
|
|
||||||
describe("embeddingPriceLabel", () => {
|
|
||||||
it("prices a listed model: USD per 1M input tokens", () => {
|
|
||||||
// Straight from the hand-maintained table (openai small = $0.02).
|
|
||||||
assert.equal(embeddingPriceLabel("openai:text-embedding-3-small"), "$0.02/1M in");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("prices every entry in EMBEDDING_PRICES without zero or 'NaN' leaking", () => {
|
|
||||||
for (const id of Object.keys(EMBEDDING_PRICES)) {
|
|
||||||
const label = embeddingPriceLabel(id);
|
|
||||||
assert.match(label, /^\$\d+(\.\d+)?\/1M in$/);
|
|
||||||
assert.notEqual(label, "$0/1M in");
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
it("trims a trailing zero the model-menu way ($0.2, not $0.20)", () => {
|
|
||||||
assert.equal(embeddingPriceLabel("google:gemini-embedding-2"), "$0.2/1M in");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("stays honestly '—' for unlisted models (never zero)", () => {
|
|
||||||
assert.equal(embeddingPriceLabel("openrouter:some/model-not-in-table"), "—");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("needsEmbeddingModel", () => {
|
|
||||||
it("spares only the master switch", () => {
|
|
||||||
assert.equal(needsEmbeddingModel("enabled"), false);
|
|
||||||
assert.equal(needsEmbeddingModel("embedding"), true);
|
|
||||||
assert.equal(needsEmbeddingModel("clustering"), true);
|
|
||||||
assert.equal(needsEmbeddingModel("labeling"), true);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("GraphSection (SSR)", () => {
|
|
||||||
const render = (graph: GraphSettings | undefined) =>
|
|
||||||
renderToStaticMarkup(<GraphSection graph={graph} onSectionChange={() => {}} />);
|
|
||||||
|
|
||||||
it("grays out + hints the model-dependent rungs when no model is set", () => {
|
|
||||||
const html = render({ enabled: true } as GraphSettings);
|
|
||||||
// Master switch: never model-dimmed.
|
|
||||||
const enabledRow = html.split('data-testid="graph-row-enabled"')[1]?.split("</div>")[0] ?? "";
|
|
||||||
assert.ok(!enabledRow.includes("opacity-40"), "master switch must not dim on unset model");
|
|
||||||
// The three dependent rungs dim and each carries the hint.
|
|
||||||
for (const key of ["embedding", "clustering", "labeling"] as const) {
|
|
||||||
const row = html.split(`data-testid="graph-row-${key}"`)[1]?.split('data-testid="graph-row-')[0] ?? "";
|
|
||||||
assert.ok(row.includes("opacity-40"), `${key} row dims without a model`);
|
|
||||||
assert.ok(row.includes("Needs an embedding model"), `${key} row carries the hint`);
|
|
||||||
}
|
|
||||||
assert.ok(html.includes('data-testid="graph-row-embedding-model"'));
|
|
||||||
});
|
|
||||||
|
|
||||||
it("keeps the ladder touchable once a model is set", () => {
|
|
||||||
const html = render({
|
|
||||||
enabled: true,
|
|
||||||
embeddingModel: "openai:text-embedding-3-small",
|
|
||||||
} as GraphSettings);
|
|
||||||
for (const key of ["embedding", "clustering", "labeling"] as const) {
|
|
||||||
const row = html.split(`data-testid="graph-row-${key}"`)[1]?.split('data-testid="graph-row-')[0] ?? "";
|
|
||||||
assert.ok(!row.includes("opacity-40"), `${key} row stays lit with a model`);
|
|
||||||
assert.ok(!row.includes("Needs an embedding model"), "no hint with a model");
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
it("renders the picker row (catalog fetch is an effect — inert in SSR)", () => {
|
|
||||||
const html = render(undefined);
|
|
||||||
assert.ok(html.includes("Loading…"), "pre-fetch trigger state renders");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -1,474 +0,0 @@
|
|||||||
/**
|
|
||||||
* BYOK provider embeddings (wave 6, 2026-09-23): the adapters' wire
|
|
||||||
* protocols (injected fetch — never the network), the slug/keys
|
|
||||||
* resolution rules (null = honestly off; a set model with a missing key
|
|
||||||
* is an honest error), the post-merge batch's staleness/re-embed/skip
|
|
||||||
* semantics (scripted port — never the machine's settings/auth files),
|
|
||||||
* and the federated catalog's federation rules.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import assert from "node:assert/strict";
|
|
||||||
import { describe, it } from "node:test";
|
|
||||||
import { callCost, EMBEDDING_PRICES } from "../src/core/prices.ts";
|
|
||||||
import {
|
|
||||||
embedNewNodes,
|
|
||||||
embeddingModelCatalog,
|
|
||||||
googleEmbeddingProvider,
|
|
||||||
openaiEmbeddingProvider,
|
|
||||||
openrouterEmbeddingProvider,
|
|
||||||
parseEmbeddingModelSlug,
|
|
||||||
resolveEmbeddingPortFor,
|
|
||||||
type EmbeddingResolution,
|
|
||||||
} from "../src/server/embeddings.ts";
|
|
||||||
import type { EmbeddingPort, EmbeddingTask } from "../src/core/ports.ts";
|
|
||||||
import type { EmbeddingsLedger, GraphNode } from "../src/core/contracts/graph.ts";
|
|
||||||
|
|
||||||
// ─── scripted fetch + in-memory graph (no fs, no network) ────────────────────
|
|
||||||
|
|
||||||
type RecordedCall = { url: string; init: RequestInit };
|
|
||||||
|
|
||||||
function jsonResponse(body: unknown, status = 200): Response {
|
|
||||||
return new Response(JSON.stringify(body), {
|
|
||||||
status,
|
|
||||||
headers: { "content-type": "application/json" },
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function recordingFetch(
|
|
||||||
route: (url: string, body: unknown) => Response | Promise<Response>,
|
|
||||||
): { calls: RecordedCall[]; fetch: (input: string, init: RequestInit) => Promise<Response> } {
|
|
||||||
const calls: RecordedCall[] = [];
|
|
||||||
const fetchFn = async (input: string, init: RequestInit): Promise<Response> => {
|
|
||||||
calls.push({ url: input, init });
|
|
||||||
const body = init.body ? (JSON.parse(String(init.body)) as unknown) : null;
|
|
||||||
return route(input, body);
|
|
||||||
};
|
|
||||||
return { calls, fetch: fetchFn };
|
|
||||||
}
|
|
||||||
|
|
||||||
function node(id: string, statement = `statement of ${id}`): GraphNode {
|
|
||||||
return { id, name: id, statement, aliases: [], provenance: [], createdAt: "2026-09-23T00:00:00.000Z" };
|
|
||||||
}
|
|
||||||
|
|
||||||
function fakeGraphStore(start?: Partial<EmbeddingsLedger>): {
|
|
||||||
ledger: EmbeddingsLedger;
|
|
||||||
writes: EmbeddingsLedger[];
|
|
||||||
store: {
|
|
||||||
readNodes(): GraphNode[];
|
|
||||||
readEmbeddings(): EmbeddingsLedger;
|
|
||||||
writeEmbeddings(ledger: EmbeddingsLedger): void;
|
|
||||||
};
|
|
||||||
nodes: GraphNode[];
|
|
||||||
} {
|
|
||||||
const nodes = [node("n1", "alpha fact"), node("n2", "beta fact")];
|
|
||||||
const ledger: EmbeddingsLedger = {
|
|
||||||
model: start?.model ?? "test-embed",
|
|
||||||
dim: start?.dim ?? 2,
|
|
||||||
vectors: { ...start?.vectors },
|
|
||||||
clusters: [...start?.clusters ?? []],
|
|
||||||
updatedAt: start?.updatedAt ?? "2026-09-23T00:00:00.000Z",
|
|
||||||
};
|
|
||||||
const writes: EmbeddingsLedger[] = [];
|
|
||||||
return {
|
|
||||||
ledger,
|
|
||||||
writes,
|
|
||||||
nodes,
|
|
||||||
store: {
|
|
||||||
readNodes: () => nodes,
|
|
||||||
readEmbeddings: () => ledger,
|
|
||||||
writeEmbeddings: (next) => {
|
|
||||||
writes.push(next);
|
|
||||||
Object.assign(ledger, next);
|
|
||||||
},
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/** A scripted port: unit vectors by text hash, records (texts, task) calls. */
|
|
||||||
function scriptedPort(model = "openai:test-embed", dim = 2, tokensPerCall = 120): {
|
|
||||||
calls: Array<{ texts: string[]; task: EmbeddingTask }>;
|
|
||||||
port: EmbeddingPort;
|
|
||||||
} {
|
|
||||||
const calls: Array<{ texts: string[]; task: EmbeddingTask }> = [];
|
|
||||||
return {
|
|
||||||
calls,
|
|
||||||
port: {
|
|
||||||
model,
|
|
||||||
async embed(texts, task) {
|
|
||||||
calls.push({ texts: [...texts], task });
|
|
||||||
return {
|
|
||||||
vectors: texts.map((_, i) => Array.from({ length: dim }, (_, j) => (j === i ? 1 : 0))),
|
|
||||||
model,
|
|
||||||
dim,
|
|
||||||
promptTokens: tokensPerCall,
|
|
||||||
};
|
|
||||||
},
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
const okPort = (port: EmbeddingPort): EmbeddingResolution => ({ ok: true, port });
|
|
||||||
|
|
||||||
// ─── usage-on-error + pay-once (owner ruling 2026-09-23, interlude) ──────────
|
|
||||||
|
|
||||||
describe("embedNewNodes partial-failure accounting (Teacher review follow-up)", () => {
|
|
||||||
it("a mid-run batch failure: landed vectors persist, usage survives, failed carries the error, next run resumes", async () => {
|
|
||||||
// 40 nodes → batch 1 (32) lands, batch 2 (8) throws.
|
|
||||||
const many: GraphNode[] = Array.from({ length: 40 }, (_, i) => node(`n${i}`, `statement ${i}`));
|
|
||||||
const ledger: EmbeddingsLedger = { model: "m", dim: 2, vectors: {}, clusters: [], updatedAt: "" };
|
|
||||||
let writes = 0;
|
|
||||||
let call = 0;
|
|
||||||
const failing: EmbeddingPort = {
|
|
||||||
model: "m",
|
|
||||||
async embed(texts) {
|
|
||||||
call++;
|
|
||||||
if (call === 2) throw new Error("provider exploded mid-run");
|
|
||||||
return {
|
|
||||||
vectors: texts.map((_, i) => (i === 0 ? [1, 0] : [0, 1])),
|
|
||||||
model: "m",
|
|
||||||
dim: 2,
|
|
||||||
promptTokens: 100,
|
|
||||||
};
|
|
||||||
},
|
|
||||||
};
|
|
||||||
const store = {
|
|
||||||
readNodes: () => many,
|
|
||||||
readEmbeddings: () => ledger,
|
|
||||||
writeEmbeddings: (next: EmbeddingsLedger) => {
|
|
||||||
writes++;
|
|
||||||
Object.assign(ledger, next);
|
|
||||||
},
|
|
||||||
};
|
|
||||||
const result = await embedNewNodes(store, () => okPort(failing));
|
|
||||||
// Usage-on-error: batch 1 billed — the money channel keeps it.
|
|
||||||
assert.equal(result.failed, "provider exploded mid-run");
|
|
||||||
assert.equal(result.embedded, 32, "only the landed batch counts as embedded");
|
|
||||||
assert.equal(result.usage?.steps.length, 1);
|
|
||||||
assert.equal(result.usage?.steps[0]?.inputTokens, 100);
|
|
||||||
// Pay-once: the landed vectors are persisted — a retry embeds only the rest.
|
|
||||||
assert.equal(writes, 1);
|
|
||||||
assert.equal(Object.keys(ledger.vectors).length, 32);
|
|
||||||
assert.equal(ledger.model, "m");
|
|
||||||
const resume = await embedNewNodes(store, () => okPort(scriptedPort("m", 2).port));
|
|
||||||
assert.equal(resume.embedded, 8, "the retry resumes — it re-embeds only the 8 remaining");
|
|
||||||
assert.equal(resume.failed, undefined);
|
|
||||||
assert.equal(Object.keys(ledger.vectors).length, 40);
|
|
||||||
assert.equal(resume.usage?.steps[0]?.inputTokens, 120, "scripted port tokens — the retry billed once");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a FIRST-batch failure: nothing written, nothing billed, the old ledger stays intact", async () => {
|
|
||||||
const g = fakeGraphStore({ vectors: { n1: [1, 0] } });
|
|
||||||
const alwaysFails: EmbeddingPort = {
|
|
||||||
model: "m",
|
|
||||||
embed: async () => {
|
|
||||||
throw new Error("total outage");
|
|
||||||
},
|
|
||||||
};
|
|
||||||
const result = await embedNewNodes(g.store, () => okPort(alwaysFails));
|
|
||||||
assert.equal(result.failed, "total outage");
|
|
||||||
assert.equal(result.embedded, 0);
|
|
||||||
assert.equal(result.usage, undefined, "no call landed — nothing to bill");
|
|
||||||
assert.equal(g.writes.length, 0, "the ledger is untouched (its own model/dim still true)");
|
|
||||||
assert.ok(g.ledger.vectors.n1, "the old vector survives a zero-landed failure");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("success still carries no `failed` key and counts every todo node", async () => {
|
|
||||||
const g = fakeGraphStore();
|
|
||||||
const scripted = scriptedPort(g.ledger.model, 2);
|
|
||||||
const result = await embedNewNodes(g.store, () => okPort(scripted.port));
|
|
||||||
assert.equal(result.failed, undefined);
|
|
||||||
assert.equal(result.embedded, 2);
|
|
||||||
assert.equal(result.skipped, undefined);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ─── the money channel's embedding overlay ─────────────────────────────────
|
|
||||||
|
|
||||||
describe("the embedding price overlay (hand-maintained, outside the sync block)", () => {
|
|
||||||
it("prices the known embedding models at their verified input rates", () => {
|
|
||||||
// USD per 1M input tokens, verified against provider pages 2026-09-23.
|
|
||||||
assert.equal(callCost({ inputTokens: 1_000_000, outputTokens: 0 }, "openai:text-embedding-3-small"), 0.02);
|
|
||||||
assert.equal(callCost({ inputTokens: 1_000_000, outputTokens: 0 }, "openai:text-embedding-3-large"), 0.13);
|
|
||||||
assert.equal(callCost({ inputTokens: 1_000_000, outputTokens: 0 }, "google:gemini-embedding-001"), 0.15);
|
|
||||||
assert.equal(callCost({ inputTokens: 1_000_000, outputTokens: 0 }, "google:gemini-embedding-2"), 0.2);
|
|
||||||
assert.equal(callCost({ inputTokens: 1_000_000, outputTokens: 0 }, "openrouter:openai/text-embedding-3-small"), 0.02);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("unknown embedding models stay unpriced (—, never zero), tokens still captured", () => {
|
|
||||||
assert.equal(callCost({ inputTokens: 100, outputTokens: 0 }, "openrouter:mystery/embed"), null);
|
|
||||||
assert.equal(!("openrouter:mystery/embed" in EMBEDDING_PRICES), true);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ─── the slug grammar ─────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
describe("parseEmbeddingModelSlug", () => {
|
|
||||||
it("parses the three shipped providers", () => {
|
|
||||||
assert.deepEqual(parseEmbeddingModelSlug("openai:text-embedding-3-small"), { provider: "openai", id: "text-embedding-3-small" });
|
|
||||||
assert.deepEqual(parseEmbeddingModelSlug("openrouter:openai/text-embedding-3-large"), { provider: "openrouter", id: "openai/text-embedding-3-large" });
|
|
||||||
assert.deepEqual(parseEmbeddingModelSlug("google:gemini-embedding-001"), { provider: "google", id: "gemini-embedding-001" });
|
|
||||||
// OpenRouter ids may carry a :free suffix (live catalog fact) — the
|
|
||||||
// provider is the part before the FIRST colon.
|
|
||||||
assert.deepEqual(parseEmbeddingModelSlug("openrouter:liquid/lfm-2.5-embedding-350m:free"), { provider: "openrouter", id: "liquid/lfm-2.5-embedding-350m:free" });
|
|
||||||
});
|
|
||||||
|
|
||||||
it("rejects malformed slugs (no guessing)", () => {
|
|
||||||
assert.equal(parseEmbeddingModelSlug("text-embedding-3-small"), null, "no provider prefix");
|
|
||||||
assert.equal(parseEmbeddingModelSlug("anthropic:embed"), null, "unshipped provider");
|
|
||||||
assert.equal(parseEmbeddingModelSlug("openai:"), null, "empty id");
|
|
||||||
assert.equal(parseEmbeddingModelSlug(":id"), null, "empty provider");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ─── resolution: BYOK honesty rules ───────────────────────────────────────────
|
|
||||||
|
|
||||||
describe("resolveEmbeddingPortFor", () => {
|
|
||||||
it("null/absent setting = honestly off", () => {
|
|
||||||
for (const slug of [null, undefined, ""]) {
|
|
||||||
const res = resolveEmbeddingPortFor(slug, () => "key");
|
|
||||||
assert.equal(res.ok, false);
|
|
||||||
assert.match((res as { reason: string }).reason, /no embedding model configured/);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a set model with no key is an honest error naming the key", () => {
|
|
||||||
const res = resolveEmbeddingPortFor("openai:text-embedding-3-small", () => undefined);
|
|
||||||
assert.equal(res.ok, false);
|
|
||||||
assert.match((res as { reason: string }).reason, /OPENAI_API_KEY/);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("an invalid slug never resolves", () => {
|
|
||||||
const res = resolveEmbeddingPortFor("bedrock:whatever", () => "key");
|
|
||||||
assert.equal(res.ok, false);
|
|
||||||
assert.match((res as { reason: string }).reason, /invalid embedding model slug/);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a keyed model resolves to its provider adapter", () => {
|
|
||||||
const res = resolveEmbeddingPortFor("openai:text-embedding-3-small", () => "sk-test");
|
|
||||||
assert.equal(res.ok, true);
|
|
||||||
assert.equal((res as { port: EmbeddingPort }).port.model, "openai:text-embedding-3-small");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ─── the adapters' wire protocols (injected fetch) ────────────────────────────
|
|
||||||
|
|
||||||
describe("openrouterEmbeddingProvider", () => {
|
|
||||||
it("POSTs /api/v1/embeddings with Bearer auth and the task input_type", async () => {
|
|
||||||
const { calls, fetch } = recordingFetch(() =>
|
|
||||||
jsonResponse({ data: [{ embedding: [1, 0], index: 0 }], usage: { prompt_tokens: 42 } }),
|
|
||||||
);
|
|
||||||
const port = openrouterEmbeddingProvider("openrouter:openai/text-embedding-3-small", "k", fetch);
|
|
||||||
const result = await port.embed(["hello"], "document");
|
|
||||||
assert.equal(calls.length, 1);
|
|
||||||
assert.equal(calls[0]!.url, "https://openrouter.ai/api/v1/embeddings");
|
|
||||||
assert.equal((calls[0]!.init.headers as Record<string, string>).authorization, "Bearer k");
|
|
||||||
assert.deepEqual(JSON.parse(String(calls[0]!.init.body)), {
|
|
||||||
model: "openai/text-embedding-3-small",
|
|
||||||
input: ["hello"],
|
|
||||||
input_type: "search_document",
|
|
||||||
});
|
|
||||||
assert.deepEqual(result.vectors, [[1, 0]]);
|
|
||||||
assert.equal(result.promptTokens, 42);
|
|
||||||
|
|
||||||
await port.embed(["q"], "query");
|
|
||||||
assert.equal((JSON.parse(String(calls[1]!.init.body)) as { input_type: string }).input_type, "search_query");
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("openaiEmbeddingProvider", () => {
|
|
||||||
it("POSTs /v1/embeddings, Bearer auth, no task hint; index ordering honored", async () => {
|
|
||||||
const { calls, fetch } = recordingFetch(() =>
|
|
||||||
jsonResponse({
|
|
||||||
data: [{ embedding: [0, 1], index: 1 }, { embedding: [1, 0], index: 0 }],
|
|
||||||
usage: { prompt_tokens: 7, total_tokens: 7 },
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
const port = openaiEmbeddingProvider("openai:text-embedding-3-small", "sk", fetch);
|
|
||||||
const result = await port.embed(["a", "b"], "document");
|
|
||||||
assert.equal(calls[0]!.url, "https://api.openai.com/v1/embeddings");
|
|
||||||
assert.equal((calls[0]!.init.headers as Record<string, string>).authorization, "Bearer sk");
|
|
||||||
const body = JSON.parse(String(calls[0]!.init.body)) as Record<string, unknown>;
|
|
||||||
assert.deepEqual(body.input, ["a", "b"]);
|
|
||||||
assert.equal("input_type" in body, false, "OpenAI takes no task hint");
|
|
||||||
// Rows arrive out of order — the adapter must restore input order.
|
|
||||||
assert.deepEqual(result.vectors, [[1, 0], [0, 1]]);
|
|
||||||
assert.equal(result.promptTokens, 7);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("googleEmbeddingProvider", () => {
|
|
||||||
it("POSTs :batchEmbedContents with x-goog-api-key and per-text taskType", async () => {
|
|
||||||
const { calls, fetch } = recordingFetch((_url, body) => {
|
|
||||||
const n = ((body as { requests?: unknown[] } | null)?.requests ?? []).length;
|
|
||||||
return jsonResponse({ embeddings: Array.from({ length: n }, (_, i) => ({ values: i === 0 ? [1, 0] : [0, 1] })) });
|
|
||||||
});
|
|
||||||
const port = googleEmbeddingProvider("google:gemini-embedding-001", "gk", fetch);
|
|
||||||
const result = await port.embed(["a", "b"], "document");
|
|
||||||
assert.equal(calls[0]!.url, "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:batchEmbedContents");
|
|
||||||
assert.equal((calls[0]!.init.headers as Record<string, string>)["x-goog-api-key"], "gk");
|
|
||||||
assert.deepEqual(JSON.parse(String(calls[0]!.init.body)), {
|
|
||||||
requests: [
|
|
||||||
{ model: "models/gemini-embedding-001", content: { parts: [{ text: "a" }] }, taskType: "RETRIEVAL_DOCUMENT" },
|
|
||||||
{ model: "models/gemini-embedding-001", content: { parts: [{ text: "b" }] }, taskType: "RETRIEVAL_DOCUMENT" },
|
|
||||||
],
|
|
||||||
});
|
|
||||||
assert.deepEqual(result.vectors, [[1, 0], [0, 1]]);
|
|
||||||
assert.equal(result.promptTokens, undefined, "batch responses carry no token count we can trust");
|
|
||||||
|
|
||||||
await port.embed(["q"], "query");
|
|
||||||
const second = JSON.parse(String(calls[1]!.init.body)) as { requests: Array<{ taskType: string }> };
|
|
||||||
assert.equal(second.requests[0]!.taskType, "RETRIEVAL_QUERY");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a short/missing vector is an honest error, never a silent misalignment", async () => {
|
|
||||||
const { fetch } = recordingFetch(() => jsonResponse({ embeddings: [{ values: [1, 0] }] }));
|
|
||||||
const port = googleEmbeddingProvider("google:gemini-embedding-001", "gk", fetch);
|
|
||||||
await assert.rejects(port.embed(["a", "b"], "document"), /1\/2 vectors/);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe("adapter error mapping (all providers)", () => {
|
|
||||||
it("HTTP failures become clear Errors carrying the status", async () => {
|
|
||||||
for (const factory of [openrouterEmbeddingProvider, openaiEmbeddingProvider, googleEmbeddingProvider]) {
|
|
||||||
const { fetch } = recordingFetch(() => jsonResponse({ error: { message: "bad key" } }, 401));
|
|
||||||
const port = factory("openai:text-embedding-3-small", "k", fetch);
|
|
||||||
await assert.rejects(port.embed(["x"], "document"), /API 401/);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ─── the post-merge batch (scripted port — the machine's files stay unread) ───
|
|
||||||
|
|
||||||
describe("embedNewNodes (the extraction tail)", () => {
|
|
||||||
it("embeds missing nodes, stamps model/dim, records one usage step per API call", async () => {
|
|
||||||
const g = fakeGraphStore();
|
|
||||||
const scripted = scriptedPort("openai:text-embedding-3-small", 2, 120);
|
|
||||||
const result = await embedNewNodes(g.store, () => okPort(scripted.port));
|
|
||||||
assert.deepEqual(result, {
|
|
||||||
embedded: 2,
|
|
||||||
total: 2,
|
|
||||||
usage: {
|
|
||||||
model: "openai:text-embedding-3-small",
|
|
||||||
steps: [
|
|
||||||
{ inputTokens: 120, outputTokens: 0, cacheReadTokens: undefined, cacheWriteTokens: undefined, reasoningTokens: undefined },
|
|
||||||
],
|
|
||||||
},
|
|
||||||
});
|
|
||||||
assert.equal(g.writes.length, 1, "the ledger is written once per batch");
|
|
||||||
assert.equal(g.ledger.model, "openai:text-embedding-3-small");
|
|
||||||
assert.equal(g.ledger.dim, 2);
|
|
||||||
assert.ok(g.ledger.vectors.n1 && g.ledger.vectors.n2);
|
|
||||||
assert.deepEqual(scripted.calls[0]!.texts, ["alpha fact", "beta fact"]);
|
|
||||||
assert.equal(scripted.calls[0]!.task, "document");
|
|
||||||
assert.equal(result.usage?.steps.length, 1);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a MODEL CHANGE voids the ledger and re-embeds everything", async () => {
|
|
||||||
// The ledger holds a stale-model vector for a node that still exists.
|
|
||||||
const g = fakeGraphStore({ model: "openai:old-embed", vectors: { n1: [9, 9] } });
|
|
||||||
const scripted = scriptedPort("openai:new-embed", 2);
|
|
||||||
const result = await embedNewNodes(g.store, () => okPort(scripted.port));
|
|
||||||
assert.equal(result.embedded, 2, "the stale vector dies — BOTH nodes re-embed");
|
|
||||||
assert.deepEqual(scripted.calls[0]!.texts, ["alpha fact", "beta fact"]);
|
|
||||||
assert.equal(g.ledger.model, "openai:new-embed");
|
|
||||||
assert.ok(!g.ledger.vectors.n1!.every((x, i) => x === [9, 9][i]), "the old vector was replaced");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("null model = info skip: nothing written, nothing embedded, the reason rides the result", async () => {
|
|
||||||
const g = fakeGraphStore({ vectors: { n1: [1, 0] } });
|
|
||||||
const result = await embedNewNodes(g.store, () => ({
|
|
||||||
ok: false,
|
|
||||||
reason: "no embedding model configured (graph.embeddingModel is null)",
|
|
||||||
}));
|
|
||||||
assert.deepEqual(result, { embedded: 0, total: 1, skipped: "no embedding model configured (graph.embeddingModel is null)" });
|
|
||||||
assert.equal(g.writes.length, 0, "a skipped batch never touches the ledger");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a set model with a missing key is the same skip path (its reason says so)", async () => {
|
|
||||||
const g = fakeGraphStore();
|
|
||||||
const result = await embedNewNodes(g.store, () => ({
|
|
||||||
ok: false,
|
|
||||||
reason: 'embedding model is set to "openai:x" but its OPENAI_API_KEY key is missing',
|
|
||||||
}));
|
|
||||||
assert.match(result.skipped ?? "", /OPENAI_API_KEY/);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("prunes ghost vectors for nodes that no longer exist", async () => {
|
|
||||||
const g = fakeGraphStore({ vectors: { n1: [1, 0], ghost: [0, 1] } });
|
|
||||||
const scripted = scriptedPort(g.ledger.model, 2);
|
|
||||||
const result = await embedNewNodes(g.store, () => okPort(scripted.port));
|
|
||||||
assert.equal(result.embedded, 1, "n1 already has a vector; only n2 embeds");
|
|
||||||
assert.ok(!("ghost" in g.ledger.vectors), "the ghost vector is pruned");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("batches at 32 (a 40-node backfill = two API calls)", async () => {
|
|
||||||
const many: GraphNode[] = Array.from({ length: 40 }, (_, i) => node(`n${i}`, `statement ${i}`));
|
|
||||||
const ledger: EmbeddingsLedger = { model: "m", dim: 2, vectors: {}, clusters: [], updatedAt: "" };
|
|
||||||
const writes: EmbeddingsLedger[] = [];
|
|
||||||
const scripted = scriptedPort("m", 2);
|
|
||||||
const result = await embedNewNodes(
|
|
||||||
{
|
|
||||||
readNodes: () => many,
|
|
||||||
readEmbeddings: () => ledger,
|
|
||||||
writeEmbeddings: (next) => writes.push(next),
|
|
||||||
},
|
|
||||||
() => okPort(scripted.port),
|
|
||||||
);
|
|
||||||
assert.equal(result.embedded, 40);
|
|
||||||
assert.equal(scripted.calls.length, 2);
|
|
||||||
assert.equal(scripted.calls[0]!.texts.length, 32);
|
|
||||||
assert.equal(scripted.calls[1]!.texts.length, 8);
|
|
||||||
assert.equal(writes.length, 1);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
// ─── the federated catalog (scripted keys + fetch) ────────────────────────────
|
|
||||||
|
|
||||||
describe("embeddingModelCatalog", () => {
|
|
||||||
it("no keys = empty catalog", async () => {
|
|
||||||
const models = await embeddingModelCatalog(
|
|
||||||
recordingFetch(() => jsonResponse({})).fetch,
|
|
||||||
() => undefined,
|
|
||||||
);
|
|
||||||
assert.deepEqual(models, []);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("federates every keyed provider with its filter rules", async () => {
|
|
||||||
const { fetch, calls } = recordingFetch((url) => {
|
|
||||||
if (url.includes("openrouter.ai/api/v1/embeddings/models")) {
|
|
||||||
return jsonResponse({ data: [{ id: "openai/text-embedding-3-small", name: "Text Embedding 3 Small" }, { id: "other/embed" }] });
|
|
||||||
}
|
|
||||||
if (url.includes("api.openai.com")) {
|
|
||||||
return jsonResponse({ data: [{ id: "text-embedding-3-large" }, { id: "text-embedding-3-small" }, { id: "gpt-6-sol" }] });
|
|
||||||
}
|
|
||||||
if (url.includes("generativelanguage")) {
|
|
||||||
return jsonResponse({
|
|
||||||
models: [
|
|
||||||
{ name: "models/gemini-embedding-001", displayName: "Gemini Embedding", supportedGenerationMethods: ["embedContent", "generateContent"] },
|
|
||||||
{ name: "models/gemini-6-pro", supportedGenerationMethods: ["generateContent"] },
|
|
||||||
],
|
|
||||||
});
|
|
||||||
}
|
|
||||||
return jsonResponse({});
|
|
||||||
});
|
|
||||||
const models = await embeddingModelCatalog(fetch, (id) => (id === "OPENROUTER_API_KEY" || id === "OPENAI_API_KEY" || id === "GEMINI_API_KEY" ? "k" : undefined));
|
|
||||||
assert.equal(calls[0]!.url, "https://openrouter.ai/api/v1/embeddings/models?limit=500", "the real catalog path (the inverse 404s — verified live)");
|
|
||||||
assert.deepEqual(models, [
|
|
||||||
{ id: "openrouter:openai/text-embedding-3-small", name: "Text Embedding 3 Small" },
|
|
||||||
{ id: "openrouter:other/embed", name: "other/embed" },
|
|
||||||
{ id: "openai:text-embedding-3-large", name: "text-embedding-3-large" },
|
|
||||||
{ id: "openai:text-embedding-3-small", name: "text-embedding-3-small" },
|
|
||||||
{ id: "google:gemini-embedding-001", name: "Gemini Embedding" },
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a failing provider never poisons the rest", async () => {
|
|
||||||
const { fetch } = recordingFetch((url) => {
|
|
||||||
if (url.includes("openrouter.ai")) return jsonResponse({}, 500); // their side broke
|
|
||||||
if (url.includes("api.openai.com")) return jsonResponse({ data: [{ id: "text-embedding-3-small" }] });
|
|
||||||
return jsonResponse({});
|
|
||||||
});
|
|
||||||
const models = await embeddingModelCatalog(fetch, (id) => (id === "OPENROUTER_API_KEY" || id === "OPENAI_API_KEY" ? "k" : undefined));
|
|
||||||
assert.deepEqual(models, [{ id: "openai:text-embedding-3-small", name: "text-embedding-3-small" }]);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -134,37 +134,24 @@ describe("AgentSection (thin SSR)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("GraphSection — the dependency ladder", () => {
|
describe("GraphSection — the collapsed master switch (wave 7)", () => {
|
||||||
// Wave 6: without an embedding model the three dependent rungs are
|
// Wave 7 (owner ruling 2026-09-23): the section is the master switch
|
||||||
// inert (dimmed, disabled, hinted) — pinned in embedding-settings.
|
// ONLY — the embedding/clustering/labeling rungs and the BYOK model
|
||||||
// These ladder pins isolate the CHAIN logic with a model configured.
|
// picker are gone (clustering is the free birth-session partition,
|
||||||
const MODEL: GraphSettings = { enabled: true, embeddingModel: "openai:text-embedding-3-small" };
|
// labels ARE session names). One row, one toggle.
|
||||||
|
it("renders the single master-switch row", () => {
|
||||||
it("defaults with a model set: all four rungs on, none disabled", () => {
|
const html = renderToStaticMarkup(<GraphSection graph={undefined} onSectionChange={() => {}} />);
|
||||||
const html = renderToStaticMarkup(<GraphSection graph={MODEL} onSectionChange={() => {}} />);
|
assert.ok(html.includes('data-testid="graph-row-enabled"'));
|
||||||
for (const key of ["enabled", "embedding", "clustering", "labeling"]) {
|
assert.ok(!html.includes("graph-row-embedding"), "the dead rungs render nothing");
|
||||||
assert.ok(html.includes(`data-testid="graph-row-${key}"`), key);
|
assert.ok(!html.includes("Embedding model"), "the picker is gone");
|
||||||
assert.ok(!tagOf(html, `toggle-graph-${key}`).includes("disabled"), `${key} touchable`);
|
assert.ok(!html.includes("graph-row-clustering"));
|
||||||
}
|
assert.ok(!html.includes("graph-row-labeling"));
|
||||||
|
assert.ok(tagOf(html, "toggle-graph-enabled").includes('aria-checked="true"'), "absent record = factory ON");
|
||||||
});
|
});
|
||||||
|
|
||||||
it("master off grays EVERY dependent (stored values persist, resolve inert)", () => {
|
it("a stored off record shows the switch off", () => {
|
||||||
const html = renderToStaticMarkup(
|
const html = renderToStaticMarkup(<GraphSection graph={{ enabled: false }} onSectionChange={() => {}} />);
|
||||||
<GraphSection graph={{ ...MODEL, enabled: false, embedding: true, clustering: true, labeling: true }} onSectionChange={() => {}} />,
|
assert.ok(tagOf(html, "toggle-graph-enabled").includes('aria-checked="false"'));
|
||||||
);
|
|
||||||
for (const key of ["embedding", "clustering", "labeling"]) {
|
|
||||||
assert.ok(tagOf(html, `toggle-graph-${key}`).includes("disabled"), `${key} disabled`);
|
|
||||||
}
|
|
||||||
assert.ok(!tagOf(html, "toggle-graph-enabled").includes("disabled"), "master itself stays touchable");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a mid-ladder break grays only what's below it", () => {
|
|
||||||
const html = renderToStaticMarkup(
|
|
||||||
<GraphSection graph={{ ...MODEL, embedding: true, clustering: false }} onSectionChange={() => {}} />,
|
|
||||||
);
|
|
||||||
assert.ok(!tagOf(html, "toggle-graph-embedding").includes("disabled"), "embedding touchable");
|
|
||||||
assert.ok(!tagOf(html, "toggle-graph-clustering").includes("disabled"), "clustering touchable");
|
|
||||||
assert.ok(tagOf(html, "toggle-graph-labeling").includes("disabled"), "labeling grayed");
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -150,47 +150,25 @@ describe("parseAppearanceSettingsBody", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
describe("parseGraphSettingsBody", () => {
|
describe("parseGraphSettingsBody", () => {
|
||||||
it("parses the four-rung ladder replace-whole", () => {
|
it("parses the collapsed section replace-whole (wave 7: the master switch ONLY)", () => {
|
||||||
const r = parseGraphSettingsBody({ enabled: false, embedding: false });
|
const r = parseGraphSettingsBody({ enabled: false });
|
||||||
assert.ok(r.ok);
|
assert.ok(r.ok);
|
||||||
assert.deepEqual(r.settings, { enabled: false, embedding: false });
|
assert.deepEqual(r.settings, { enabled: false });
|
||||||
assert.equal(parseGraphSettingsBody({}).ok, true);
|
assert.equal(parseGraphSettingsBody({}).ok, true);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("rejects non-boolean rungs and unknown bodies", () => {
|
it("rejects non-boolean switches and unknown bodies", () => {
|
||||||
assert.equal(parseGraphSettingsBody({ enabled: "off" }).ok, false);
|
assert.equal(parseGraphSettingsBody({ enabled: "off" }).ok, false);
|
||||||
assert.equal(parseGraphSettingsBody("nope").ok, false);
|
assert.equal(parseGraphSettingsBody("nope").ok, false);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("the ladder DEPENDENCY (graph → embedding → clustering → labeling) is a UI concern, not a parse concern", () => {
|
it("the dead wave-6 rungs in a body are ignored — the section collapses on write", () => {
|
||||||
// Nothing stops a stored {enabled: false, labeling: true} — the
|
// Old clients may still send {embedding: true, embeddingModel: …};
|
||||||
// values persist but resolve inert (grays out in the panel).
|
// the parser reads ONLY the master switch — the dead keys are not
|
||||||
|
// an error, they are just not part of the section anymore.
|
||||||
const r = parseGraphSettingsBody({ enabled: false, clustering: true, labeling: true });
|
const r = parseGraphSettingsBody({ enabled: false, clustering: true, labeling: true });
|
||||||
assert.ok(r.ok);
|
assert.ok(r.ok);
|
||||||
});
|
assert.deepEqual(r.settings, { enabled: false });
|
||||||
|
|
||||||
it("embeddingModel (wave 6): null or a shipped-provider slug; anything else is a 400", () => {
|
|
||||||
const nulled = parseGraphSettingsBody({ embeddingModel: null });
|
|
||||||
assert.ok(nulled.ok);
|
|
||||||
assert.deepEqual(nulled.settings, { embeddingModel: null });
|
|
||||||
for (const slug of [
|
|
||||||
"openai:text-embedding-3-small",
|
|
||||||
"openrouter:openai/text-embedding-3-large",
|
|
||||||
"google:gemini-embedding-001",
|
|
||||||
// OpenRouter ids may carry a :free-style suffix — the provider is
|
|
||||||
// the part before the FIRST colon (live catalog fact, 2026-09-23).
|
|
||||||
"openrouter:liquid/lfm-2.5-embedding-350m:free",
|
|
||||||
]) {
|
|
||||||
const r = parseGraphSettingsBody({ embeddingModel: slug });
|
|
||||||
assert.ok(r.ok, slug);
|
|
||||||
assert.deepEqual(r.settings, { embeddingModel: slug });
|
|
||||||
}
|
|
||||||
// Bad shapes never become a live model.
|
|
||||||
assert.equal(parseGraphSettingsBody({ embeddingModel: "text-embedding-3-small" }).ok, false, "no provider prefix");
|
|
||||||
assert.equal(parseGraphSettingsBody({ embeddingModel: "bedrock:whatever" }).ok, false, "unshipped provider");
|
|
||||||
assert.equal(parseGraphSettingsBody({ embeddingModel: "openai:" }).ok, false, "empty id");
|
|
||||||
assert.equal(parseGraphSettingsBody({ embeddingModel: 42 }).ok, false, "non-string");
|
|
||||||
assert.equal(parseGraphSettingsBody({ embeddingModel: "openai:has spaces" }).ok, false, "whitespace id");
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -11,7 +11,6 @@ import assert from "node:assert/strict";
|
|||||||
import { describe, it } from "node:test";
|
import { describe, it } from "node:test";
|
||||||
import {
|
import {
|
||||||
extractionEnabled,
|
extractionEnabled,
|
||||||
graphContextEnabled,
|
|
||||||
graphToolsEnabled,
|
graphToolsEnabled,
|
||||||
resolveConversationSpec,
|
resolveConversationSpec,
|
||||||
resolveModelSpec,
|
resolveModelSpec,
|
||||||
@@ -43,7 +42,7 @@ describe("resolveSettings", () => {
|
|||||||
assert.equal(eff.appearance.proseFontSize, 16);
|
assert.equal(eff.appearance.proseFontSize, 16);
|
||||||
assert.equal(eff.appearance.sessionNameSerif, true);
|
assert.equal(eff.appearance.sessionNameSerif, true);
|
||||||
assert.equal(eff.appearance.mdRendering.headingWeight, 900);
|
assert.equal(eff.appearance.mdRendering.headingWeight, 900);
|
||||||
assert.deepEqual(eff.graph, { enabled: true, embedding: true, clustering: true, labeling: true, embeddingModel: null }); // BYOK wave 6: no factory embedding model
|
assert.deepEqual(eff.graph, { enabled: true }); // wave 7 collapse: the master switch ONLY
|
||||||
});
|
});
|
||||||
|
|
||||||
it("explicit record values win over every factory default", () => {
|
it("explicit record values win over every factory default", () => {
|
||||||
@@ -76,9 +75,8 @@ describe("resolveSettings", () => {
|
|||||||
assert.equal(eff.appearance.mode, "light");
|
assert.equal(eff.appearance.mode, "light");
|
||||||
assert.equal(eff.appearance.proseFontSize, 20);
|
assert.equal(eff.appearance.proseFontSize, 20);
|
||||||
assert.equal(eff.appearance.mdRendering.headingWeight, 600);
|
assert.equal(eff.appearance.mdRendering.headingWeight, 600);
|
||||||
// graph: enabled=false explicit; absent rungs stay factory-on (stored
|
// wave 7 collapse: the section IS the master switch.
|
||||||
// but inert — the ladder grays them out in the UI).
|
assert.deepEqual(eff.graph, { enabled: false });
|
||||||
assert.deepEqual(eff.graph, { enabled: false, embedding: true, clustering: true, labeling: true, embeddingModel: null }); // wave 6: unset embeddingModel resolves to the null floor
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("partial role override: thinkingLevel alone borrows the SLOT's model (or nothing, without one)", () => {
|
it("partial role override: thinkingLevel alone borrows the SLOT's model (or nothing, without one)", () => {
|
||||||
@@ -224,29 +222,20 @@ describe("resolveConversationSpec (the turn gate's chain)", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
describe("the graph channel gates (the wave's seam honoring)", () => {
|
describe("the graph channel gates (the wave's seam honoring)", () => {
|
||||||
it("WRITE channel (extraction): dies with the master switch only", () => {
|
it("WRITE channel (extraction): dies with the master switch — the ONLY graph setting since wave 7", () => {
|
||||||
assert.equal(extractionEnabled({ enabled: true, embedding: true, clustering: true, labeling: true, embeddingModel: null }), true);
|
assert.equal(extractionEnabled({ enabled: true }), true);
|
||||||
assert.equal(extractionEnabled({ enabled: true, embedding: false, clustering: false, labeling: false, embeddingModel: null }), true);
|
assert.equal(extractionEnabled({ enabled: false }), false);
|
||||||
assert.equal(extractionEnabled({ enabled: false, embedding: true, clustering: true, labeling: true, embeddingModel: null }), false);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("READ channel (graph-context): needs master AND the embedding rung", () => {
|
it("the write gate rides resolveSettings' factory defaults (all on → open)", () => {
|
||||||
assert.equal(graphContextEnabled({ enabled: true, embedding: true, clustering: false, labeling: false, embeddingModel: null }), true);
|
|
||||||
assert.equal(graphContextEnabled({ enabled: true, embedding: false, clustering: true, labeling: true, embeddingModel: null }), false);
|
|
||||||
assert.equal(graphContextEnabled({ enabled: false, embedding: true, clustering: true, labeling: true, embeddingModel: null }), false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("both gates ride resolveSettings' factory defaults (all on → both open)", () => {
|
|
||||||
const eff = resolveSettings(null);
|
const eff = resolveSettings(null);
|
||||||
assert.equal(extractionEnabled(eff.graph), true);
|
assert.equal(extractionEnabled(eff.graph), true);
|
||||||
assert.equal(graphContextEnabled(eff.graph), true);
|
|
||||||
const off = resolveSettings({ version: 1, graph: { enabled: false } });
|
const off = resolveSettings({ version: 1, graph: { enabled: false } });
|
||||||
assert.equal(extractionEnabled(off.graph), false);
|
assert.equal(extractionEnabled(off.graph), false);
|
||||||
assert.equal(graphContextEnabled(off.graph), false);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("PROBER GRAPH TOOLS gate (wave 7, owner ruling 2026-09-23): the agent toggle AND the graph master switch compose", () => {
|
it("PROBER GRAPH TOOLS gate (wave 7, owner ruling 2026-09-23): the agent toggle AND the graph master switch compose", () => {
|
||||||
const graph = { enabled: true, embedding: true, clustering: true, labeling: true, embeddingModel: null };
|
const graph = { enabled: true };
|
||||||
assert.equal(graphToolsEnabled({ graph, agent: { graphTools: {} } }), true, "absent subsection = ON");
|
assert.equal(graphToolsEnabled({ graph, agent: { graphTools: {} } }), true, "absent subsection = ON");
|
||||||
assert.equal(graphToolsEnabled({ graph, agent: { graphTools: { enabled: true } } }), true);
|
assert.equal(graphToolsEnabled({ graph, agent: { graphTools: { enabled: true } } }), true);
|
||||||
assert.equal(graphToolsEnabled({ graph, agent: { graphTools: { enabled: false } } }), false);
|
assert.equal(graphToolsEnabled({ graph, agent: { graphTools: { enabled: false } } }), false);
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ describe("FACTORY_SETTINGS (the compiled factory layer)", () => {
|
|||||||
it("pins the owner-decided defaults", () => {
|
it("pins the owner-decided defaults", () => {
|
||||||
assert.equal(CURRENT_SETTINGS_VERSION, 1);
|
assert.equal(CURRENT_SETTINGS_VERSION, 1);
|
||||||
assert.deepEqual(FACTORY_SETTINGS.general, { autoUpdate: true });
|
assert.deepEqual(FACTORY_SETTINGS.general, { autoUpdate: true });
|
||||||
assert.deepEqual(FACTORY_SETTINGS.graph, { enabled: true, embedding: true, clustering: true, labeling: true, embeddingModel: null }); // BYOK wave 6: no factory embedding model
|
assert.deepEqual(FACTORY_SETTINGS.graph, { enabled: true }); // wave 7 collapse: the master switch ONLY
|
||||||
const a = FACTORY_SETTINGS.appearance;
|
const a = FACTORY_SETTINGS.appearance;
|
||||||
assert.equal(a.mode, "dark");
|
assert.equal(a.mode, "dark");
|
||||||
assert.equal(a.background, null); // the mode's own default, not a factory hex
|
assert.equal(a.background, null); // the mode's own default, not a factory hex
|
||||||
@@ -75,7 +75,7 @@ describe("sanitizeUserSettings", () => {
|
|||||||
sessionNameSerif: false,
|
sessionNameSerif: false,
|
||||||
mdRendering: { headingWeight: 700, boldColor: "#ff8800", italicColor: null },
|
mdRendering: { headingWeight: 700, boldColor: "#ff8800", italicColor: null },
|
||||||
},
|
},
|
||||||
graph: { enabled: false, embedding: true, clustering: false, labeling: true },
|
graph: { enabled: false },
|
||||||
};
|
};
|
||||||
assert.deepEqual(sanitizeUserSettings(record), record);
|
assert.deepEqual(sanitizeUserSettings(record), record);
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user