Quiz hardening: cross-field schema validation rejects ungradeable quizzes
quizInputSchema now enforces (via pure quizInputProblems + superRefine) that correctAnswer resolves against option values, options are non-empty and unique, and multi-answer requires multiSelect — schema-valid now implies gradeable. The AI SDK already returns invalid tool input to the model as a tool-error part (verified in ai@7 run loop), so the teacher retries in the same turn. chat-handler onError passes tool-input validation messages through to the client (masked 'An error occurred.' for everything else). Covers the Gate 3 GLM failure: correctAnswer slug matching no option.
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
import { createAgentUIStreamResponse, type UIMessage } from "ai";
|
||||
import { createAgentUIStreamResponse, InvalidToolInputError, NoSuchToolError, type UIMessage } from "ai";
|
||||
import { createTeacherAgent, type TeacherAgent } from "../agent/teacher-agent.ts";
|
||||
import { createJobRunner } from "../agent/subagents/runner.ts";
|
||||
import { createExtractionPipeline } from "../agent/subagents/extraction.ts";
|
||||
@@ -45,6 +45,17 @@ export async function handleChatRequest(req: Request, agent?: TeacherAgent): Pro
|
||||
buildTeacherContext(stores.graphStore, chatId),
|
||||
),
|
||||
uiMessages: body.messages as UIMessage[],
|
||||
// Tool-call validation failures are messages OUR schemas authored (the
|
||||
// quiz authoring contract, unknown tools) — safe and useful in the
|
||||
// transcript: they are exactly what the teacher saw and retried on. The
|
||||
// tool loop delivers them as plain strings (what the model received) or
|
||||
// as the original error object; expose both, and keep everything else
|
||||
// (provider/transport internals) masked, as the SDK defaults.
|
||||
onError: (error) => {
|
||||
if (typeof error === "string") return error;
|
||||
if (InvalidToolInputError.isInstance(error) || NoSuchToolError.isInstance(error)) return error.message;
|
||||
return "An error occurred.";
|
||||
},
|
||||
onFinish: ({ messages, isAborted }) => {
|
||||
persistTurn(stores.chatStore, chatId, messages as UIMessage[], isAborted);
|
||||
spawnTurnJobs(jobRunner, chatId, messages as UIMessage[]);
|
||||
|
||||
Reference in New Issue
Block a user