Quiz hardening: cross-field schema validation rejects ungradeable quizzes

quizInputSchema now enforces (via pure quizInputProblems + superRefine) that
correctAnswer resolves against option values, options are non-empty and
unique, and multi-answer requires multiSelect — schema-valid now implies
gradeable. The AI SDK already returns invalid tool input to the model as a
tool-error part (verified in ai@7 run loop), so the teacher retries in the
same turn. chat-handler onError passes tool-input validation messages
through to the client (masked 'An error occurred.' for everything else).
Covers the Gate 3 GLM failure: correctAnswer slug matching no option.
This commit is contained in:
Amos
2026-09-01 01:47:47 +03:00
parent 83e3ab6180
commit 5dab8d12d4
5 changed files with 346 additions and 46 deletions
+12 -1
View File
@@ -1,4 +1,4 @@
import { createAgentUIStreamResponse, type UIMessage } from "ai";
import { createAgentUIStreamResponse, InvalidToolInputError, NoSuchToolError, type UIMessage } from "ai";
import { createTeacherAgent, type TeacherAgent } from "../agent/teacher-agent.ts";
import { createJobRunner } from "../agent/subagents/runner.ts";
import { createExtractionPipeline } from "../agent/subagents/extraction.ts";
@@ -45,6 +45,17 @@ export async function handleChatRequest(req: Request, agent?: TeacherAgent): Pro
buildTeacherContext(stores.graphStore, chatId),
),
uiMessages: body.messages as UIMessage[],
// Tool-call validation failures are messages OUR schemas authored (the
// quiz authoring contract, unknown tools) — safe and useful in the
// transcript: they are exactly what the teacher saw and retried on. The
// tool loop delivers them as plain strings (what the model received) or
// as the original error object; expose both, and keep everything else
// (provider/transport internals) masked, as the SDK defaults.
onError: (error) => {
if (typeof error === "string") return error;
if (InvalidToolInputError.isInstance(error) || NoSuchToolError.isInstance(error)) return error.message;
return "An error occurred.";
},
onFinish: ({ messages, isAborted }) => {
persistTurn(stores.chatStore, chatId, messages as UIMessage[], isAborted);
spawnTurnJobs(jobRunner, chatId, messages as UIMessage[]);