two questions per teaching unit: predict → teach → check (owner 2026-10-03)
The teaching cycle gains its motivation question. The teach quiz schema now carries a REQUIRED type field — predict (posed before the teaching, arrives open, the learner predicts the next step) or check (posed after, arrives collapsed behind the reveal). The prober's quiz stays role-less. - core: quizInputObject.type union + normalizeQuizType + the reveal law as pure quizNeedsReveal; QuestionData.questionType; note → thinking rename (legacy reads fold back); readingThinking channel on answers; compactAnswerForModel emits thinking / thinkingWhileReading - tools: descriptions stripped to mechanics — QUIZ_OPTION_CRAFT and EXPLAIN_DOCTRINE deleted, doctrine moved to the prompt layer (reverses the 2026-09-27 placement, owner 2026-10-03); shared quizDescription/explainDescription builders differ only in deltas - prompts: shared/20-question-tools.md rewritten (option craft, transcribed-speech leniency, the two thinking channels); new teacher/28-predictions.md (cycle, roles, skip rule, resolution law); 20-teaching + 30-question-principles scoped to the new taxonomy - UI: predictions render open with a Prediction eyebrow, verdict shows but the explanation tucks behind a toggle; the reading-thinking field mounts while a check waits (dictation claims it until reveal); answered card separates thinking vs thinking-while-reading blocks
This commit is contained in:
@@ -59,6 +59,7 @@ const validQuiz = {
|
||||
],
|
||||
correctAnswer: "4",
|
||||
explanation: "Arithmetic.",
|
||||
type: "check",
|
||||
nodeId: "arith",
|
||||
};
|
||||
|
||||
@@ -383,3 +384,49 @@ describe("the prober's explain: no nodeId field at all", () => {
|
||||
assert.equal(empty.success, false);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── descriptions are MECHANICS (owner 2026-10-03) ────────────────────────
|
||||
// Question-design doctrine lives in the system prompt (shared craft chunk +
|
||||
// teacher fragments), NOT in tool descriptions. A description that teaches
|
||||
// the model how to write questions drifts the moment a new variant (prober,
|
||||
// future adaptive practice) reuses the tool with a different doctrine.
|
||||
describe("tool descriptions carry mechanics only, no question-craft", () => {
|
||||
const teachQuiz = teachQuizToolFor({ chatId: "chat_x", chatStore: undefined as never });
|
||||
for (const [name, tool] of [
|
||||
["teach quiz", teachQuiz],
|
||||
["probe quiz", probeQuizTool],
|
||||
] as const) {
|
||||
it(`${name}: no craft doctrine, mechanics intact`, () => {
|
||||
const d = tool.description;
|
||||
for (const banned of ["asymmetry", "distractor", "believable", "misconception"]) {
|
||||
assert.ok(!d.toLowerCase().includes(banned), `${name} description leaks craft: "${banned}"`);
|
||||
}
|
||||
for (const mechanical of ["GRADED", "ONE question per turn", "ask_user_question"]) {
|
||||
assert.ok(d.includes(mechanical), `${name} description lost mechanics: "${mechanical}"`);
|
||||
}
|
||||
});
|
||||
}
|
||||
it("the teach quiz declares the `type` field and `nodeId`; the probe quiz declares neither", () => {
|
||||
assert.ok(teachQuiz.description.includes("`type`"), "teach quiz points at the role field");
|
||||
assert.ok(teachQuiz.description.includes("`nodeId`"), "teach quiz points at the plan-map validation");
|
||||
assert.ok(!probeQuizTool.description.includes("`type`"), "the probe quiz has no roles");
|
||||
assert.ok(!probeQuizTool.description.includes("`nodeId`"), "the probe quiz has no plan map");
|
||||
});
|
||||
it("the explain tools keep the COMMITMENT rule (the grading contract) and drop the softer doctrine", () => {
|
||||
for (const [name, tool] of [
|
||||
["teach explain", teachExplainToolFor({ chatId: "chat_x", chatStore: undefined as never })],
|
||||
["probe explain", probeExplainTool],
|
||||
] as const) {
|
||||
assert.ok(tool.description.includes("COMMITTED"), `${name} keeps the commitment rule`);
|
||||
assert.ok(!tool.description.includes("near-miss"), `${name} drops the doctrine (now in the shared prompt)`);
|
||||
}
|
||||
});
|
||||
it("the craft now lives in the shared prompt chunk (both agents compose it)", async () => {
|
||||
const shared = await fs.promises.readFile(
|
||||
path.join(PATHS.root, "src/agent/prompts/shared/20-question-tools.md"),
|
||||
"utf-8",
|
||||
);
|
||||
assert.ok(shared.includes("asymmetry"), "the option craft moved here, not deleted");
|
||||
assert.ok(shared.includes("TRANSCRIBED SPEECH"), "the dictation-leniency note lives here");
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user