feat(models): clamp-on-write for pinned thinking levels + DEFAULT_THINKING_ORDER in the core barrel
Fix 1 — clamp, never reject (owner ruling, interlude 2026-09-22): a model swap with a pinned thinking level must JUST WORK. The owner hit the write-side 400 live (Claude Fable 5.1/xhigh → GLM 5.3 flash: "z-ai/glm-5.3-flash does not support thinking level xhigh"). The supported-level verdicts in validateChatSettings and validateModelsSettings are DEAD — the WRITE clamps the pin to the new model's nearest supported level (pi's clampThinkingLevel, the same translation piEngineForSpec applies at runtime) and stores the clamped result; the server is the truth — the record shows the clamped level after write. New pure helpers in model-catalog.ts, applied by ALL THREE write routes (section, defaults, per-chat — one verdict seam, so every surface that used the old 400 gets the clamp uniformly): - clampModelsLevels: every slot's level clamps against the SLOT'S OWN model (the parser already requires a model on each slot write). - clampChatLevel: the body's own model, else the record's resolved strong slot (the chain's fallback); no model anywhere → rides unchecked (runtime clamps at engine build; the turn gate refuses the model-less chat anyway). Fix 2 — DEFAULT_THINKING_ORDER moves from config.ts to core/settings (core barrel, AUTH_KEYS precedent — one source of truth, no drift). The UI imports it to auto-populate a thinking level on FRESH model picks; the ordering has no other client role. Legacy unset records keep the runtime defaultThinkingLevel fallback at engine build untouched. Import sites updated: agent/models.ts, tests. Verified live on 3004: the owner's exact case stores max (not 400); a bare chat-level write clamps to low against the strong slot's glm; unknown provider/model still 400s with the provider list. Tests: the level-verdict suites became clamp suites (the owner's case pinned verbatim: xhigh→max, minimal→low, off→low, supported pass-through, level-less slots ride); the machine-aware expectBareLevelVerdict chain is retired — clampChatLevel takes the strong slot as an explicit param, so the tests are pure again. Gates: typecheck, lint, 839/839, build.
This commit is contained in:
@@ -1,13 +1,20 @@
|
||||
import { getChat, validateChatSettings, parseSettingsBody, stores } from "@/src/server";
|
||||
import {
|
||||
clampChatLevel,
|
||||
effectiveSettings,
|
||||
getChat,
|
||||
parseSettingsBody,
|
||||
stores,
|
||||
validateChatSettings,
|
||||
} from "@/src/server";
|
||||
import type { ChatSettings } from "@/src/core";
|
||||
|
||||
/**
|
||||
* The per-chat model control (model-control wave): replace-whole settings —
|
||||
* the picker sends the complete desired object ({} clears all overrides and
|
||||
* returns the chat to config.ts's pin). Validation is the catalog gate
|
||||
* (src/server/model-catalog.ts): unknown provider/model or an unsupported
|
||||
* thinking level is a 400 carrying the supported list, never a silent clamp
|
||||
* — the user must see what the model actually accepts. Effective next turn
|
||||
* the picker sends the complete desired object ({} clears all overrides).
|
||||
* Validation is the catalog gate (src/server/model-catalog.ts): unknown
|
||||
* provider/model is a 400. Thinking levels never reject (owner,
|
||||
* interlude 2026-09-22): a pinned level clamps to the named model's
|
||||
* nearest supported and the clamped value is stored. Effective next turn
|
||||
* by construction (agents are stateless per turn).
|
||||
*
|
||||
* The body parsing itself is shared with the draft route
|
||||
@@ -26,8 +33,12 @@ export async function PUT(req: Request, { params }: { params: Promise<{ id: stri
|
||||
return Response.json({ error: verdict.error, supported: verdict.supported }, { status: 400 });
|
||||
}
|
||||
|
||||
// Replace-whole persists the validated object verbatim — no merges, so a
|
||||
// stale client can never resurrect a cleared override.
|
||||
stores.chatStore.saveSettings(id, parsed.settings);
|
||||
return Response.json({ settings: parsed.settings satisfies ChatSettings });
|
||||
// Replace-whole persists the validated, level-clamped object — no
|
||||
// merges, so a stale client can never resurrect a cleared override.
|
||||
// Clamp-on-write (owner, interlude 2026-09-22): a pinned thinking
|
||||
// level never rejects — it clamps to the named model's nearest
|
||||
// supported (body model, else the record's strong slot).
|
||||
const settings = clampChatLevel(parsed.settings, effectiveSettings().models.strong);
|
||||
stores.chatStore.saveSettings(id, settings);
|
||||
return Response.json({ settings });
|
||||
}
|
||||
|
||||
@@ -8,10 +8,13 @@
|
||||
* section ∈ general | chat | agent | appearance | graph (else 404).
|
||||
* Body: the complete desired section object. Parse + validate strictly
|
||||
* (server/settings-body.ts): a bad shape/type is a 400 carrying the
|
||||
* supported list where one exists, never a silent clamp. The chat
|
||||
* section additionally passes the catalog gate (validateChatSettings —
|
||||
* unknown provider/model or unsupported thinking level = 400), same as
|
||||
* the defaults route. The agent section additionally validates prompt
|
||||
* supported list where one exists. The chat and models sections
|
||||
* additionally pass the catalog gate (validateChatSettings /
|
||||
* validateModelsSettings — unknown provider/model = 400). Thinking
|
||||
* levels never reject a write (owner, interlude 2026-09-22): a pinned
|
||||
* level CLAMPS to the named model's nearest supported level and the
|
||||
* clamped value is stored — model swaps just work, same as the
|
||||
* defaults route. The agent section additionally validates prompt
|
||||
* switches: an ON switch must reference a CURRENT source segment with an
|
||||
* existing override file (an OFF switch may reference anything — users
|
||||
* cleaning up orphans must not be blocked).
|
||||
@@ -25,6 +28,9 @@ import type { AgentSettings, ChatSettings, ModelsSettings } from "@/src/core";
|
||||
import {
|
||||
SETTINGS_SECTIONS,
|
||||
allSegmentPaths,
|
||||
clampChatLevel,
|
||||
clampModelsLevels,
|
||||
effectiveSettings,
|
||||
promptOverrides,
|
||||
readSettingsRecord,
|
||||
validateChatSettings,
|
||||
@@ -87,10 +93,24 @@ export async function PUT(request: Request, ctx: { params: Promise<{ section: st
|
||||
}
|
||||
}
|
||||
|
||||
// Clamp-on-write (owner ruling, interlude 2026-09-22): a model swap
|
||||
// with a pinned thinking level NEVER errors — the pin clamps to the
|
||||
// new model's nearest supported level (pi's nearest-clamp, the same
|
||||
// translation piEngineForSpec applies at runtime) and the clamped
|
||||
// value is what lands in the record. The chat section's bare levels
|
||||
// clamp against the record's resolved strong slot (the chain's
|
||||
// fallback); the models section's slots each carry their own model.
|
||||
const sectionOut =
|
||||
section === "chat"
|
||||
? clampChatLevel(parsed.settings as ChatSettings, effectiveSettings().models.strong)
|
||||
: section === "models"
|
||||
? clampModelsLevels(parsed.settings as ModelsSettings)
|
||||
: parsed.settings;
|
||||
|
||||
// Read-modify-write ONE section of the record; everything else rides
|
||||
// along untouched (other sections, unknown keys — sanitize preserves).
|
||||
const record = readSettingsRecord() ?? { version: 1 };
|
||||
const next = { ...record, [section]: parsed.settings };
|
||||
const next = { ...record, [section]: sectionOut };
|
||||
writeSettingsRecord(next);
|
||||
return Response.json({ settings: next });
|
||||
}
|
||||
|
||||
@@ -9,16 +9,25 @@
|
||||
* lives at data/default-settings.json (config.DEFAULT_SETTINGS_FILE).
|
||||
*
|
||||
* GET → 200 {settings} once saved, 200 null on first boot ("no model").
|
||||
* PUT → 200 {settings} — the VALIDATED copy persisted verbatim. The exact
|
||||
* per-chat contract (parseSettingsBody + the catalog gate): unknown
|
||||
* provider/model or an unsupported thinking level is a 400 carrying
|
||||
* the supported list, never a silent clamp.
|
||||
* PUT → 200 {settings} — the VALIDATED copy, level-clamped, persisted.
|
||||
* The exact per-chat contract (parseSettingsBody + the catalog
|
||||
* gate): an unknown provider/model is a 400. Thinking levels
|
||||
* never reject (owner, interlude 2026-09-22): a pinned level
|
||||
* clamps to the named model's nearest supported and the clamped
|
||||
* value is stored — model swaps just work.
|
||||
*
|
||||
* Per-session settings (PUT /api/chats/:id/settings) are a separate record
|
||||
* untouched by this — the default still rides along on session creation
|
||||
* exactly as before.
|
||||
*/
|
||||
import { readDefaultSettings, writeDefaultSettings, parseSettingsBody, validateChatSettings } from "@/src/server";
|
||||
import {
|
||||
clampChatLevel,
|
||||
effectiveSettings,
|
||||
parseSettingsBody,
|
||||
readDefaultSettings,
|
||||
validateChatSettings,
|
||||
writeDefaultSettings,
|
||||
} from "@/src/server";
|
||||
|
||||
export const dynamic = "force-dynamic";
|
||||
|
||||
@@ -34,6 +43,11 @@ export async function PUT(request: Request) {
|
||||
if (!verdict.ok) {
|
||||
return Response.json({ error: verdict.error, supported: verdict.supported }, { status: 400 });
|
||||
}
|
||||
writeDefaultSettings(parsed.settings);
|
||||
return Response.json({ settings: parsed.settings });
|
||||
// Clamp-on-write (owner, interlude 2026-09-22): a pinned thinking
|
||||
// level never rejects the write — it clamps to the named model's
|
||||
// nearest supported level (body model, else the record's strong
|
||||
// slot) and the clamped value is stored.
|
||||
const settings = clampChatLevel(parsed.settings, effectiveSettings().models.strong);
|
||||
writeDefaultSettings(settings);
|
||||
return Response.json({ settings });
|
||||
}
|
||||
|
||||
@@ -57,22 +57,6 @@ export type ModelRole =
|
||||
| "auditor"
|
||||
| "labeler";
|
||||
|
||||
/** The DEFAULT thinking level for UNSET levels only (owner, 2026-09-22):
|
||||
* a slot/override/chat that names a model but no thinking level resolves
|
||||
* to the FIRST entry in this order that the model supports
|
||||
* (getSupportedThinkingLevels); a non-reasoning model resolves "off"
|
||||
* regardless (pi reports exactly ["off"]). EXPLICITLY set levels never
|
||||
* consult this list — they clamp nearest via pi's native
|
||||
* clampThinkingLevel on model switches. */
|
||||
export const DEFAULT_THINKING_ORDER = [
|
||||
"medium",
|
||||
"low",
|
||||
"high",
|
||||
"minimal",
|
||||
"xhigh",
|
||||
"max",
|
||||
] as const satisfies readonly ThinkingLevel[];
|
||||
|
||||
/**
|
||||
* The extraction pipeline (vision §1 construction): the sole graph writer.
|
||||
* Triggered per chat once this many NEW quiz answers have landed since the
|
||||
|
||||
+2
-1
@@ -9,7 +9,8 @@ import {
|
||||
} from "@earendil-works/pi-ai";
|
||||
import { builtinProviders } from "@earendil-works/pi-ai/providers/all";
|
||||
import type { ThinkingLevel } from "@earendil-works/pi-agent-core";
|
||||
import { DEFAULT_THINKING_ORDER, type ModelRole } from "../../config.ts";
|
||||
import { type ModelRole } from "../../config.ts";
|
||||
import { DEFAULT_THINKING_ORDER } from "../core/settings.ts";
|
||||
|
||||
/**
|
||||
* The one engine factory every agent's model comes from (the pi-ai swap,
|
||||
|
||||
@@ -69,6 +69,29 @@ export interface RoleModelOverride {
|
||||
export const MODEL_SLOTS = ["strong", "cheap", "vision"] as const;
|
||||
export type ModelSlotName = (typeof MODEL_SLOTS)[number];
|
||||
|
||||
/** The DEFAULT thinking level for UNSET levels only (owner, 2026-09-22):
|
||||
* a slot/override/chat that names a model but no thinking level resolves
|
||||
* to the FIRST entry in this order that the model supports
|
||||
* (getSupportedThinkingLevels); a non-reasoning model resolves "off"
|
||||
* regardless (pi reports exactly ["off"]). EXPLICITLY set levels never
|
||||
* consult this list — they clamp nearest via pi's native
|
||||
* clampThinkingLevel on model switches.
|
||||
*
|
||||
* Client surface (interlude, 2026-09-22): this constant lives in the CORE
|
||||
* BARREL — the UI imports it to auto-populate a thinking level on a FRESH
|
||||
* model pick (one source of truth, AUTH_KEYS precedent; per-model
|
||||
* supportedLevels already reach the client via the catalog). The ordering
|
||||
* has NO other client role: legacy unset records keep the runtime
|
||||
* defaultThinkingLevel fallback at engine build. */
|
||||
export const DEFAULT_THINKING_ORDER = [
|
||||
"medium",
|
||||
"low",
|
||||
"high",
|
||||
"minimal",
|
||||
"xhigh",
|
||||
"max",
|
||||
] as const satisfies readonly ThinkingLevel[];
|
||||
|
||||
/** The machine-readable NOT-CONFIGURED refusal envelope (owner, 2026-09-22):
|
||||
* every per-need gate — the turn gate in chat-handler, a job spawn, a
|
||||
* notes import — refuses with this exact shape. HTTP callers get 503 +
|
||||
|
||||
+62
-51
@@ -17,7 +17,7 @@
|
||||
* set; nothing here changes.
|
||||
*/
|
||||
|
||||
import { getSupportedThinkingLevels, type Api, type Model } from "@earendil-works/pi-ai";
|
||||
import { clampThinkingLevel, getSupportedThinkingLevels, type Api, type Model } from "@earendil-works/pi-ai";
|
||||
import type { ThinkingLevel } from "@earendil-works/pi-agent-core";
|
||||
import { piModels } from "../agent/models.ts";
|
||||
import { settingsStore } from "../store/settings.ts";
|
||||
@@ -34,7 +34,7 @@ import type {
|
||||
} from "../core/contracts/models.ts";
|
||||
import type { EffectiveModelSpec, EffectiveSlot } from "../core/settings-resolution.ts";
|
||||
import { AUTH_PROVIDER_KEYS } from "../core/auth-keys.ts";
|
||||
import { BACKGROUND_MODEL_ROLES, type BackgroundModelRole, type ModelSlotName, type ModelsSettings } from "../core/settings.ts";
|
||||
import { BACKGROUND_MODEL_ROLES, MODEL_SLOTS, type BackgroundModelRole, type ModelSlotName, type ModelsSettings } from "../core/settings.ts";
|
||||
|
||||
/** The picker's client-safe model entry (contract: core/contracts/models.ts). */
|
||||
export type ModelDto = ModelCatalogEntry;
|
||||
@@ -196,38 +196,17 @@ export function validateChatSettings(settings: ChatSettings): SettingsValidation
|
||||
return { ok: false, error: `No model "${id}" in the ${provider} catalog` };
|
||||
}
|
||||
}
|
||||
if (settings.thinkingLevel !== undefined) {
|
||||
// The conversation chain's fallback is the strong slot — the model
|
||||
// a settings-less chat would run. BYOK (owner, 2026-09-22): with no
|
||||
// model to check against (no chat.model, no strong slot), the level
|
||||
// rides unchecked — it clamps against whatever the chain later holds.
|
||||
const strong = effectiveSettings().models.strong;
|
||||
const provider = settings.model?.provider ?? strong?.provider;
|
||||
const id = settings.model?.id ?? strong?.id;
|
||||
if (!provider || !id) return { ok: true };
|
||||
const model = piModels().getModel(provider, id);
|
||||
if (!model) return { ok: false, error: `No model "${id}" in the ${provider} catalog` };
|
||||
const supported = getSupportedThinkingLevels(model);
|
||||
if (!supported.includes(settings.thinkingLevel)) {
|
||||
return {
|
||||
ok: false,
|
||||
error: `${id} does not support thinking level "${settings.thinkingLevel}"`,
|
||||
supported,
|
||||
};
|
||||
}
|
||||
}
|
||||
// BYOK interlude (owner, 2026-09-22): thinking levels NEVER reject a
|
||||
// write — a model swap with a pinned level JUST WORKS. The WRITE clamps
|
||||
// the pin to the new model's nearest supported level (clampChatLevel
|
||||
// below, applied by the routes) and stores the clamped result; a bare
|
||||
// level with no model anywhere rides unchecked and clamps against
|
||||
// whatever the chain later holds at engine build.
|
||||
return { ok: true };
|
||||
}
|
||||
/** Validate a models-section write (PUT /api/settings/models) against the
|
||||
* catalog — the same gate the chat section gets. A slot must NAME a real
|
||||
* provider+model (owner, 2026-09-22: no factory floor exists, so a level
|
||||
* alone can never run — the parser rejects it, this gate re-checks the
|
||||
* model); a thinkingLevel must be supported by the slot's own model.
|
||||
* keyPresent is NOT gated here (a stored pin for a provider whose key
|
||||
* lapsed stays valid — it fails honestly at stream time, and the picker
|
||||
* stops offering it). */
|
||||
|
||||
export function validateModelsSettings(settings: ModelsSettings): SettingsValidation {
|
||||
for (const name of ["strong", "cheap", "vision"] as const) {
|
||||
for (const name of MODEL_SLOTS) {
|
||||
const slot = settings[name];
|
||||
if (!slot) continue;
|
||||
const verdict = validateSlotSpec(name, slot);
|
||||
@@ -247,25 +226,57 @@ function validateSlotSpec(name: ModelSlotName, slot: { model?: { provider: strin
|
||||
return { ok: false, error: `No model "${id}" in the ${provider} catalog` };
|
||||
}
|
||||
}
|
||||
if (slot.thinkingLevel !== undefined) {
|
||||
// The write-side parser requires slot.model (owner, 2026-09-22 — a
|
||||
// level-only slot can never run), so this fallback is tolerance for
|
||||
// future callers; with no model to check against, accept (the
|
||||
// engine build clamps whatever the chain later holds).
|
||||
const resolved = effectiveSettings().models[name];
|
||||
const provider = slot.model?.provider ?? resolved?.provider;
|
||||
const id = slot.model?.id ?? resolved?.id;
|
||||
if (!provider || !id) return { ok: true };
|
||||
const model = piModels().getModel(provider, id);
|
||||
if (!model) return { ok: false, error: `No model "${id}" in the ${provider} catalog` };
|
||||
const supported = getSupportedThinkingLevels(model);
|
||||
if (!supported.includes(slot.thinkingLevel)) {
|
||||
return {
|
||||
ok: false,
|
||||
error: `${id} does not support thinking level "${slot.thinkingLevel}"`,
|
||||
supported,
|
||||
};
|
||||
}
|
||||
}
|
||||
// BYOK interlude (owner, 2026-09-22): the supported-level verdict is
|
||||
// DEAD — a slot write with a pinned level never rejects; the WRITE
|
||||
// clamps the level to the model's nearest supported (clampModelsLevels
|
||||
// below, applied by the route) and stores the clamped result.
|
||||
return { ok: true };
|
||||
}
|
||||
|
||||
// ─── clamp-on-write (owner ruling, interlude 2026-09-22) ─────────────────────
|
||||
// A model swap with a pinned thinking level must JUST WORK: the pin
|
||||
// translates to the new model's nearest supported level — pi's
|
||||
// clampThinkingLevel, the SAME translation piEngineForSpec applies at
|
||||
// runtime — and the clamped value is what lands in the record. The
|
||||
// server is the truth: the record shows the clamped level after write.
|
||||
|
||||
/** Clamp one level against one model (identity when the model is unknown
|
||||
* or the level already supported — the existence gate rejects unknown
|
||||
* models separately, so this only ever sees real ones in practice). */
|
||||
function clampLevelFor(provider: string, id: string, level: ThinkingLevel): ThinkingLevel {
|
||||
const model = piModels().getModel(provider, id);
|
||||
return model ? clampThinkingLevel(model, level) : level;
|
||||
}
|
||||
|
||||
/** The models-section write's level translation: every slot's pinned
|
||||
* thinkingLevel clamps against THE SLOT'S OWN model (the parser requires
|
||||
* a model on every slot write — owner, 2026-09-22). Pure: catalog in,
|
||||
* clamped section out; the route applies it after validation. */
|
||||
export function clampModelsLevels(settings: ModelsSettings): ModelsSettings {
|
||||
const out: ModelsSettings = {};
|
||||
for (const name of MODEL_SLOTS) {
|
||||
const slot = settings[name];
|
||||
if (!slot?.model) continue;
|
||||
out[name] =
|
||||
slot.thinkingLevel !== undefined
|
||||
? {
|
||||
model: slot.model,
|
||||
thinkingLevel: clampLevelFor(slot.model.provider, slot.model.id, slot.thinkingLevel),
|
||||
}
|
||||
: slot;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The chat-section write's level translation: a pinned thinkingLevel
|
||||
* clamps against the body's own model, else the record's resolved strong
|
||||
* slot (the conversation chain's fallback — what a settings-less chat
|
||||
* would run). No model anywhere → rides unchecked (runtime clamps at
|
||||
* engine build; the turn gate refuses the model-less chat anyway). */
|
||||
export function clampChatLevel(settings: ChatSettings, strong: EffectiveSlot | null): ChatSettings {
|
||||
if (settings.thinkingLevel === undefined) return settings;
|
||||
const provider = settings.model?.provider ?? strong?.provider;
|
||||
const id = settings.model?.id ?? strong?.id;
|
||||
if (!provider || !id) return settings;
|
||||
return { ...settings, thinkingLevel: clampLevelFor(provider, id, settings.thinkingLevel) };
|
||||
}
|
||||
|
||||
+79
-48
@@ -1,9 +1,22 @@
|
||||
import { describe, it } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { modelCatalog, validateChatSettings, validateModelsSettings, catalogDefaultsFor, catalogProviders, catalogSlots, toModelDto, rolePinsFor, catalogRolePins } from "../src/server/model-catalog.ts";
|
||||
import {
|
||||
catalogDefaultsFor,
|
||||
catalogProviders,
|
||||
catalogRolePins,
|
||||
catalogSlots,
|
||||
clampChatLevel,
|
||||
clampModelsLevels,
|
||||
modelCatalog,
|
||||
rolePinsFor,
|
||||
toModelDto,
|
||||
validateChatSettings,
|
||||
validateModelsSettings,
|
||||
} from "../src/server/model-catalog.ts";
|
||||
import type { EffectiveSlot } from "../src/core/settings-resolution.ts";
|
||||
import type { SettingsValidation } from "../src/server/model-catalog.ts";
|
||||
import { clampThinkingLevel, getSupportedThinkingLevels } from "@earendil-works/pi-ai";
|
||||
import { DEFAULT_THINKING_ORDER } from "../config.ts";
|
||||
import { DEFAULT_THINKING_ORDER } from "../src/core/settings.ts";
|
||||
import { piEngineForSpec, piModels } from "../src/agent/models.ts";
|
||||
import { BACKGROUND_MODEL_ROLES } from "../src/core/settings.ts";
|
||||
import { resolveSettings } from "../src/core/settings-resolution.ts";
|
||||
@@ -29,26 +42,11 @@ function glmModel() {
|
||||
return glm!;
|
||||
}
|
||||
|
||||
/** The record's resolved tier as seen by the validation fallbacks —
|
||||
* machine-aware truth (data/settings.json belongs to settings-store.test.ts
|
||||
* under the one-fs-owner discipline, so READ it, never pin it): on a
|
||||
* scratch worktree the record is empty, on the owner's machine slots are
|
||||
* configured; the expected verdicts below follow whichever holds. */
|
||||
/** The machine-aware expected verdict for a bare-level validation: mirrors
|
||||
* the fallback chain exactly — no resolved tier → rides unchecked; a tier
|
||||
* whose model left the catalog → rejected; else the supported-level check. */
|
||||
function expectBareLevelVerdict(verdict: SettingsValidation, tier: "strong" | "cheap" | "vision") {
|
||||
const slot = effectiveSettings().models[tier];
|
||||
if (!slot) {
|
||||
assert.deepEqual(verdict, { ok: true }, "no model to check against → accepted (clamps at run)");
|
||||
return;
|
||||
}
|
||||
const model = piModels().getModel(slot.provider, slot.id);
|
||||
if (!model) {
|
||||
assert.equal(verdict.ok, false, "a vanished model is rejected honestly");
|
||||
return;
|
||||
}
|
||||
assert.equal(verdict.ok, getSupportedThinkingLevels(model).includes("minimal"));
|
||||
/** Another reasoning specimen with a different supported set. */
|
||||
function qwenModel() {
|
||||
const qwen = piModels().getModel("openrouter", "qwen/qwen3.8-max-0902");
|
||||
assert.ok(qwen, "qwen3.8-max in catalog");
|
||||
return qwen!;
|
||||
}
|
||||
|
||||
describe("the model catalog projection", () => {
|
||||
@@ -217,21 +215,46 @@ describe("the models-section validation gate", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("thinkingLevel resolves against the slot's model; level-only slots check nothing", () => {
|
||||
// level checked against the named model
|
||||
assert.equal(
|
||||
validateModelsSettings({ strong: { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "max" } }).ok,
|
||||
true,
|
||||
it("thinking levels never reject a slot write (owner, interlude 2026-09-22)", () => {
|
||||
// clamp, never reject: the model swap with a pinned level JUST
|
||||
// WORKS — validation accepts ANY level; the WRITE clamps to the
|
||||
// new model's nearest supported and stores the clamped result.
|
||||
assert.deepEqual(
|
||||
validateModelsSettings({ strong: { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "minimal" } }),
|
||||
{ ok: true },
|
||||
);
|
||||
assert.equal(
|
||||
validateModelsSettings({ strong: { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "minimal" } }).ok,
|
||||
false,
|
||||
assert.deepEqual(
|
||||
validateModelsSettings({ strong: { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "xhigh" } }),
|
||||
{ ok: true },
|
||||
);
|
||||
// (a level-only slot never passes the WRITE parser — owner, 2026-09-22;
|
||||
// the validation gate falls back to the tier's RESOLVED model —
|
||||
// machine-aware: the record's strong tier decides the verdict)
|
||||
const levelOnly = validateModelsSettings({ strong: { thinkingLevel: "minimal" } });
|
||||
expectBareLevelVerdict(levelOnly, "strong");
|
||||
});
|
||||
|
||||
it("clampModelsLevels: the slot's own model translates the pin (the owner's exact case)", () => {
|
||||
// Claude Fable 5.1 xhigh → GLM 5.3 flash: the pin clamps to glm's
|
||||
// nearest supported (low|high|max) and the CLAMPED value is stored.
|
||||
const glm = { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "xhigh" as const };
|
||||
assert.deepEqual(clampModelsLevels({ strong: glm }), {
|
||||
strong: { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "max" },
|
||||
});
|
||||
assert.deepEqual(clampModelsLevels({ strong: { ...glm, thinkingLevel: "minimal" as const } }).strong, {
|
||||
model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" },
|
||||
thinkingLevel: "low",
|
||||
});
|
||||
// glm cannot disable thinking: off clamps UP to low
|
||||
assert.equal(
|
||||
clampModelsLevels({ strong: { ...glm, thinkingLevel: "off" as const } }).strong?.thinkingLevel,
|
||||
"low",
|
||||
);
|
||||
// a supported level passes through untouched
|
||||
assert.deepEqual(clampModelsLevels({ strong: { ...glm, thinkingLevel: "high" as const } }).strong, {
|
||||
model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" },
|
||||
thinkingLevel: "high",
|
||||
});
|
||||
// a slot with no level rides as-is (client populates on fresh
|
||||
// picks per DEFAULT_THINKING_ORDER — the server never invents one)
|
||||
assert.deepEqual(clampModelsLevels({ strong: { model: glm.model } }), {
|
||||
strong: { model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" } },
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts empty settings (the clear-slots write)", () => {
|
||||
@@ -240,21 +263,29 @@ describe("the models-section validation gate", () => {
|
||||
});
|
||||
|
||||
describe("the settings validation gate", () => {
|
||||
it("accepts a supported level and rejects an unsupported one with the list", () => {
|
||||
const ok = validateChatSettings({ model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "max" });
|
||||
assert.deepEqual(ok, { ok: true });
|
||||
const bad = validateChatSettings({ model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "minimal" });
|
||||
assert.equal(bad.ok, false);
|
||||
if (!bad.ok) {
|
||||
assert.match(bad.error, /does not support thinking level/);
|
||||
assert.deepEqual(bad.supported, ["low", "high", "max"]);
|
||||
}
|
||||
it("thinking levels never reject a chat write (owner, interlude 2026-09-22)", () => {
|
||||
assert.deepEqual(
|
||||
validateChatSettings({ model: { provider: "openrouter", id: "z-ai/glm-5.3-flash" }, thinkingLevel: "minimal" }),
|
||||
{ ok: true },
|
||||
);
|
||||
assert.deepEqual(validateChatSettings({ thinkingLevel: "xhigh" }), { ok: true });
|
||||
});
|
||||
|
||||
it("a bare thinking level checks against the chat model, else the strong slot, else rides unchecked", () => {
|
||||
// machine-aware: with a record-carried strong slot the level checks
|
||||
// against it; with none it rides unchecked (clamps at run).
|
||||
expectBareLevelVerdict(validateChatSettings({ thinkingLevel: "minimal" }), "strong");
|
||||
it("clampChatLevel: the body's model, else the strong slot, translates the pin", () => {
|
||||
const glm = { provider: "openrouter", id: "z-ai/glm-5.3-flash" };
|
||||
const strong: EffectiveSlot = { provider: "openrouter", id: "z-ai/glm-5.3-flash" };
|
||||
// the body's own model wins
|
||||
assert.deepEqual(
|
||||
clampChatLevel({ model: { provider: "openrouter", id: "qwen/qwen3.8-max-0902" }, thinkingLevel: "xhigh" as const }, strong),
|
||||
{ model: { provider: "openrouter", id: "qwen/qwen3.8-max-0902" }, thinkingLevel: clampThinkingLevel(qwenModel(), "xhigh") },
|
||||
);
|
||||
// no body model → the record's strong slot is the fallback
|
||||
assert.equal(clampChatLevel({ thinkingLevel: "xhigh" as const }, strong).thinkingLevel, "max");
|
||||
// no model anywhere → rides unchecked (runtime clamps; the turn
|
||||
// gate refuses the model-less chat anyway)
|
||||
assert.deepEqual(clampChatLevel({ thinkingLevel: "minimal" as const }, null), { thinkingLevel: "minimal" });
|
||||
// no pinned level → untouched
|
||||
assert.deepEqual(clampChatLevel({ model: { ...glm } }, strong), { model: { ...glm } });
|
||||
});
|
||||
|
||||
it("rejects unknown providers and models", () => {
|
||||
|
||||
Reference in New Issue
Block a user