Quiz hardening: cross-field schema validation rejects ungradeable quizzes

quizInputSchema now enforces (via pure quizInputProblems + superRefine) that
correctAnswer resolves against option values, options are non-empty and
unique, and multi-answer requires multiSelect — schema-valid now implies
gradeable. The AI SDK already returns invalid tool input to the model as a
tool-error part (verified in ai@7 run loop), so the teacher retries in the
same turn. chat-handler onError passes tool-input validation messages
through to the client (masked 'An error occurred.' for everything else).
Covers the Gate 3 GLM failure: correctAnswer slug matching no option.
This commit is contained in:
Amos
2026-09-01 01:47:47 +03:00
parent 83e3ab6180
commit 5dab8d12d4
5 changed files with 346 additions and 46 deletions
+1
View File
@@ -26,6 +26,7 @@ export const quizTool = tool({
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
"Use it to (1) assess what the learner already understands before teaching, and (2) run tight practice/retrieval loops after explaining. " +
"Options-only: single-select or multi-select, plus an automatic 'I don't know' choice so the learner can signal a genuine gap instead of guessing. " +
"correctAnswer must be the EXACT value (or label) of one of the listed options — a correctAnswer that matches no option fails validation and the call is returned to you to fix. " +
"No free-text answers — for non-graded questions use ask_user_question instead. " +
"Ask exactly one question per tool call; after calling, end your turn and wait for the graded result.",
inputSchema: quizInputSchema,