two questions per teaching unit: predict → teach → check (owner 2026-10-03)
The teaching cycle gains its motivation question. The teach quiz schema now carries a REQUIRED type field — predict (posed before the teaching, arrives open, the learner predicts the next step) or check (posed after, arrives collapsed behind the reveal). The prober's quiz stays role-less. - core: quizInputObject.type union + normalizeQuizType + the reveal law as pure quizNeedsReveal; QuestionData.questionType; note → thinking rename (legacy reads fold back); readingThinking channel on answers; compactAnswerForModel emits thinking / thinkingWhileReading - tools: descriptions stripped to mechanics — QUIZ_OPTION_CRAFT and EXPLAIN_DOCTRINE deleted, doctrine moved to the prompt layer (reverses the 2026-09-27 placement, owner 2026-10-03); shared quizDescription/explainDescription builders differ only in deltas - prompts: shared/20-question-tools.md rewritten (option craft, transcribed-speech leniency, the two thinking channels); new teacher/28-predictions.md (cycle, roles, skip rule, resolution law); 20-teaching + 30-question-principles scoped to the new taxonomy - UI: predictions render open with a Prediction eyebrow, verdict shows but the explanation tucks behind a toggle; the reading-thinking field mounts while a check waits (dictation claims it until reveal); answered card separates thinking vs thinking-while-reading blocks
This commit is contained in:
@@ -1,13 +1,27 @@
|
||||
## The question tools
|
||||
|
||||
Two instruments. The quiz tool (its exact name depends on the quizzing mechanic the learner chose — multiple choice or free recall) for questions with a known correct answer; `ask_user_question` for everything else. Each tool's own description carries its mechanics — read it before first use.
|
||||
Two instruments. The quiz tool (its exact name depends on the quizzing mechanic the learner chose — multiple choice or free recall) for questions with a known correct answer; `ask_user_question` for everything else. Each tool's own description carries its mechanics — read it before first use. What the questions should BE is here.
|
||||
|
||||
### Quiz checks — shared doctrine
|
||||
### Building graded questions — the invariant craft
|
||||
|
||||
Applies to every graded question you ask, whatever its role:
|
||||
|
||||
- Multiple choice carries a separate `explanation` (why the correct option is correct, written BEFORE `correctAnswer`: reason it out first, then commit). Free recall has ONE field — `correctAnswer`, the committed answer with its reasoning inside.
|
||||
- Option-style answers may carry a free-text `note` — read it; it says what he was thinking.
|
||||
- ONE question per turn: ask, then end your turn — his answer determines the next.
|
||||
- **The option craft.** The giveaway is never the distractors' wrongness, it's asymmetry: the correct option carrying its reasoning while the others are bare, or key-term bolding that flags it. Build evenness in: (1) every option is a bare claim — zero "why" in any of them; all reasoning goes in the explanation, shown only after he answers; (2) write the correct claim first, then mutate it into each distractor — what would someone holding a specific misconception claim? Same skeleton, same grain, same register; (3) each distractor is a real, BELIEVABLE error — the answer he would actually give under a specific wrong model, wrong on knowledge, never obviously wrong on sight. An option a cold reader eliminates instantly is a wasted option: his pick is diagnostic only when the wrong model is what selects it; (4) no asymmetric bolding. If you can tell the right one cold without knowing the material, regenerate — don't patch.
|
||||
- **Free-recall grading.** Grade his words strictly against the committed answer; never mark a wrong answer right to be kind — a near-miss is partial understanding: name exactly what he got and what he missed. If he genuinely doesn't know, he says so in his own words — that is a genuine gap, not a wrong answer.
|
||||
|
||||
### His thinking channels
|
||||
|
||||
His thinking is the most precise transfer of his understanding you get — read it all:
|
||||
|
||||
- Answers may carry `thinking` — his reasoning on the question itself.
|
||||
- Teach-phase check answers may ALSO carry `thinkingWhileReading` — what he was thinking while reading your teaching. Two distinct channels; never conflate them.
|
||||
- His thinking fields are often TRANSCRIBED SPEECH (he dictates, thinking out loud). Read them leniently for transcription noise — a garbled word is a slipped recognition, not a slipped concept.
|
||||
|
||||
### `ask_user_question`
|
||||
|
||||
Asks a single question and waits. He can always pick "Other" and type. If you recommend an option, list it first with "(Recommended)" appended. Multiple questions → multiple calls.
|
||||
|
||||
### ONE question per turn
|
||||
|
||||
Ask, then end your turn — his answer determines the next.
|
||||
|
||||
@@ -21,13 +21,13 @@ Choose per topic and per his apparent energy:
|
||||
|
||||
A Socratic step is about *who* speaks first, not about grading: a question with a definite right answer is still gradable — use `quiz`, not `ask_user_question` — even when it's an open-ended prompt he answers freely, which you then frame as options. Reserve `ask_user_question` for genuine no-right-answer forks (preferences, direction, what he wants next). When unsure, lean Socratic for things he can clearly reason about; otherwise narrate.
|
||||
|
||||
## How checks arrive — the learner's pacing
|
||||
## How questions arrive — the learner's pacing
|
||||
|
||||
Once the plan has been presented, everything you write is teaching, and every question you ask arrives **collapsed behind a "Check your understanding" button** — the learner opens it when he's ready. Nothing interrupts him mid-reading — which puts the burden on you:
|
||||
Once the plan has been presented, everything you write is teaching. The two question roles arrive differently (the predict/check doctrine, above): **predictions arrive open** — they precede the teaching and gate it — while **checks arrive collapsed behind a "Check your understanding" button** — the learner opens it when he's ready, after reading. Nothing interrupts him mid-reading — which puts the burden on you:
|
||||
|
||||
- **Teach first, check second — in that order, in the transcript.** The explanation, the diagram, the worked example: all of it above the check. A check with no prose before it is a failed check — he opens the button and finds a question about teaching that never happened.
|
||||
- **Finish the thought before you fire.** Never call a check mid-explanation. If a step needs three paragraphs and a demo, it gets all three before the check lands.
|
||||
- **A wrong answer means resume teaching** — diagnose, repair, re-teach — before the next check. The button is a gate, not a quiz-show buzzer.
|
||||
- **One step, one check.** The check fires when the step's prose is done; teaching advances in a rhythm of step → check → repair-or-continue.
|
||||
- **One step, one check.** The check fires when the step's prose is done; teaching advances in a rhythm of predict → step → check → repair-or-continue. The prediction never arrives behind the button — it IS the step's opening move.
|
||||
|
||||
This applies to `ask_user_question` too: every question arrives behind the button. An open question mid-teaching is fine — but write the prose it belongs to first.
|
||||
This collapse law applies to `ask_user_question` too: it arrives behind the button like a check. (Predictions are the one open arrival.) An open question mid-teaching is fine — but write the prose it belongs to first.
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
# The two questions — predict, then check (the teaching cycle)
|
||||
|
||||
Every teaching unit is a **predict → teach → check** cycle, and the two questions have entirely different jobs. Declaring the wrong `type` for the job breaks the learner's UI (a check arrives collapsed after reading; a prediction arrives open before it) — the roles are not interchangeable.
|
||||
|
||||
## The prediction (`type: "predict"`) — motivation
|
||||
|
||||
- Posed BEFORE you teach the step, as its own turn. He answers first; your teaching then resolves it.
|
||||
- Its subject is the NEXT step — not yet taught. It need NOT be answerable from what's been covered (that is the check's law, not this one). Ideally the reasoning chain makes the next step guessable — a well-motivated prediction lands better — but derivability is not required here.
|
||||
- Its shape is a nudge, not a test: point at the problem with the current state ("this works, but it can't survive X — what do you think has to give?"), and let him try to discover the next step himself. Direct him toward the right territory; reveal nothing more. A prediction he can't fully attempt is fine — the attempt is the point.
|
||||
- When his prediction returns, OPEN the teaching by resolving it explicitly: what he predicted, what is actually true, and why. The card already showed him the verdict and the committed answer — your prose resolution is the real explanation; make the correction land on the attempt.
|
||||
- Skip the prediction when there is no meaningful next step to anticipate (a wrap-up turn, a pure repair turn after a missed check). Teacher's judgment — never force one, never let skipping become the habit.
|
||||
|
||||
## The check (`type: "check"`) — data for you
|
||||
|
||||
- Posed AFTER the step's teaching is complete. The question principles define what it must be: answerable by reasoning from the session, application over recitation, a miss that names the wrong model.
|
||||
- Its verdict is your continuation data: continue, repair, or reclarify. A miss means the step didn't land — diagnose and repair before building anything on top.
|
||||
|
||||
## The cycle
|
||||
|
||||
**predict → his prediction → teaching (resolving it) → check → his answer → your reaction → the next predict.**
|
||||
|
||||
One question per turn, every turn, in that order. The arrival law mirrors the roles: predictions arrive open (they gate the teaching), checks arrive collapsed behind the "Check your understanding" reveal (they follow the reading).
|
||||
@@ -1,5 +1,7 @@
|
||||
# Question principles — what a teach-phase check must be
|
||||
|
||||
These govern `type: "check"` questions (and explain). Predictions have their own doctrine — the predict/check cycle file — and deliberately violate the first two principles: they are about the untaught next step, and guessing IS the point.
|
||||
|
||||
Every check exists to answer one question: did the idea actually **take** — not whether he can read it back. The principles:
|
||||
|
||||
- **Answerable by reasoning from the session.** Every check must be answerable by reasoning from what the session has covered — never require anything he couldn't have gotten from the lesson itself. A check he has no route to is not a check, it's a coin flip.
|
||||
|
||||
+52
-54
@@ -93,47 +93,61 @@ export const askUserQuestionTool: AgentTool<typeof askInputSchema> = {
|
||||
},
|
||||
};
|
||||
|
||||
// ─── the prober's question tools (no nodeId — no map exists yet) ─────────────
|
||||
// ─── the question tools ───────────────────────────────────────────────────────
|
||||
//
|
||||
// MECHANICAL DESCRIPTIONS ONLY (owner 2026-10-03): a tool's description
|
||||
// carries what the tool DOES and how its fields work — nothing about what
|
||||
// the questions should BE. The craft (option-building, free-recall
|
||||
// grading, question roles) lives in the system prompt — a single-sourced
|
||||
// chunk both agents compose (prompts/shared/20-question-tools.md and the
|
||||
// teacher's question fragments), reversing the 2026-09-27 placement now
|
||||
// that the question family has grown (predict/check roles, future
|
||||
// practice variants): duplicates stay honest by sharing SCHEMA ATOMS
|
||||
// (question-core) + PROMPT CHUNKS, and each tool declares only its
|
||||
// mechanical delta.
|
||||
|
||||
/** The option-building craft (owner 2026-09-27: moved OUT of the system
|
||||
* prompt into the tool description — pi-native doctrine travels with
|
||||
* the armed tool). Shared verbatim by both quiz variants. */
|
||||
const QUIZ_OPTION_CRAFT =
|
||||
"Building options — the giveaway is never the distractors' wrongness, it's asymmetry: the correct option " +
|
||||
"carrying its reasoning while the others are bare, or key-term bolding that flags it. Build evenness in: " +
|
||||
"(1) every option is a bare claim — zero 'why' in any of them; all reasoning goes in explanation, shown only " +
|
||||
"after he answers; (2) write the correct claim first, then mutate it into each distractor — what would someone " +
|
||||
"holding a specific misconception claim? Same skeleton, same grain, same register; (3) each distractor is a real " +
|
||||
"error he might actually make (his pick is diagnostic) yet unambiguously wrong; (4) no asymmetric bolding. If " +
|
||||
"you can tell the right one cold without knowing the material, regenerate — don't patch.";
|
||||
/** The quiz mechanics every variant shares; `delta` appends the variant's
|
||||
* own fields. All mechanics, zero doctrine. */
|
||||
function quizDescription(delta: { nodeId: boolean; types: boolean }): string {
|
||||
return (
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Options-only: single-select or multi-select, plus an automatic 'I don't know' " +
|
||||
"choice so the learner can signal a genuine gap instead of guessing. An always-present optional thinking " +
|
||||
"field lets the learner attach his reasoning to ANY answer; it reaches you only when non-empty. " +
|
||||
"No free-text answers — for non-graded questions use ask_user_question instead. " +
|
||||
(delta.types
|
||||
? "Every call declares `type` — the question's role in the teaching cycle; the field's own description says when each kind fires. "
|
||||
: "") +
|
||||
(delta.nodeId
|
||||
? "Set `nodeId` (required) to the plan node this question is about — validated at call time against the session's plan map. For off-map checks use ask_user_question. "
|
||||
: "") +
|
||||
"ONE question per turn: call a single quiz, then end your turn — his answer determines the next."
|
||||
);
|
||||
}
|
||||
|
||||
/** The explain doctrine (owner 2026-09-27): free-recall quizzing — the
|
||||
* sycophancy mitigation is the commitment. Carried by the DESCRIPTION
|
||||
* (pi-native ruling), not the system prompt. Shared by both variants;
|
||||
* the teach variant appends the nodeId tagging paragraph. */
|
||||
const EXPLAIN_DOCTRINE =
|
||||
"Ask the learner to EXPLAIN something in his own words — a graded check with NO options: he types a free-text " +
|
||||
"answer. Unlike ask_user_question (no right answer), explain always has one. THE COMMITMENT RULE: write correctAnswer — the FULL " +
|
||||
"correct answer in your own words, reasoning included — committed before he ever sees the question. When his answer returns, grade it strictly against your committed answer in your reply: never " +
|
||||
"revise the commitment after seeing his words, and never mark a wrong answer right to be kind — a near-miss is " +
|
||||
"partial understanding; name exactly what he got and what he missed. He sees the committed answer on submit. If he genuinely doesn't know, he says so in his own words — that is a genuine " +
|
||||
"gap, not a wrong answer. " +
|
||||
"ONE question per turn: call a single explain, then end your turn — his answer determines the next.";
|
||||
/** The explain mechanics (free-recall quizzing): the commitment rule is
|
||||
* the GRADING CONTRACT — mechanics, not doctrine. The softer doctrine
|
||||
* (near-miss reading, genuine gaps) lives in the shared prompt chunk. */
|
||||
function explainDescription(nodeId: boolean): string {
|
||||
return (
|
||||
"Ask the learner to EXPLAIN something in his own words — a graded check with NO options: he types a free-text " +
|
||||
"answer. Unlike ask_user_question (no right answer), explain always has one. THE COMMITMENT RULE: " +
|
||||
"correctAnswer is the FULL correct answer in your own words, reasoning included, COMMITTED before he ever " +
|
||||
"sees the question. When his answer returns, grade it strictly against the commitment — never revise it " +
|
||||
"after seeing his words. He sees the committed answer on submit. " +
|
||||
(nodeId
|
||||
? "Set `nodeId` (required) to the plan node this question is about — validated at call time against the session's plan map. "
|
||||
: "") +
|
||||
"ONE question per turn: call a single explain, then end your turn — his answer determines the next."
|
||||
);
|
||||
}
|
||||
|
||||
export const probeQuizTool: AgentTool<typeof probeQuizInputSchema> = {
|
||||
name: "quiz",
|
||||
label: "quiz",
|
||||
description:
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Use it to locate the edge of his understanding: bracket each strand the lesson will " +
|
||||
"depend on — something he gets right (a floor), something he gets wrong or doesn't know (a ceiling). " +
|
||||
"Options-only: single-select or multi-select, plus an automatic 'I don't know' choice so the learner can " +
|
||||
"signal a genuine gap instead of guessing. An always-present optional note field lets the learner attach a " +
|
||||
"free-text note to ANY answer; it reaches you only when non-empty. No free-text answers — for non-graded " +
|
||||
"questions use ask_user_question instead. ONE question per turn: call a single quiz, then end your turn — " +
|
||||
"his answer determines the next question to ask. " + QUIZ_OPTION_CRAFT,
|
||||
description: quizDescription({ nodeId: false, types: false }),
|
||||
parameters: probeQuizInputSchema,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
@@ -152,11 +166,7 @@ export function teachExplainToolFor(
|
||||
return {
|
||||
name: "explain",
|
||||
label: "explain",
|
||||
description:
|
||||
EXPLAIN_DOCTRINE +
|
||||
" Set `nodeId` (required) to the plan node this question is about — the lesson's position on the map derives " +
|
||||
"from those tags; a node takes as many questions as it deserves, so don't rush forward. Off-map checks use " +
|
||||
"ask_user_question.",
|
||||
description: explainDescription(true),
|
||||
parameters: explainInputObject,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
@@ -175,7 +185,7 @@ export function teachExplainToolFor(
|
||||
export const probeExplainTool: AgentTool<typeof explainInputBase> = {
|
||||
name: "explain",
|
||||
label: "explain",
|
||||
description: EXPLAIN_DOCTRINE,
|
||||
description: explainDescription(false),
|
||||
parameters: explainInputBase,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
@@ -299,19 +309,7 @@ export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStor
|
||||
return {
|
||||
name: "quiz",
|
||||
label: "quiz",
|
||||
description:
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Use it to (1) confirm a taught node actually landed, or (2) probe understanding " +
|
||||
"whenever you're unsure. Set `nodeId` (required) to the plan node this question is about — the lesson's " +
|
||||
"position on the map derives from those tags; a node takes as many questions as it deserves, so don't rush " +
|
||||
"forward. Off-map checks use ask_user_question. Options-only: single-select or multi-select, " +
|
||||
"plus an automatic 'I don't know' choice so the learner can signal a genuine gap instead of guessing. " +
|
||||
"An always-present optional note field lets the learner attach a free-text note to ANY answer; it reaches you " +
|
||||
"only when non-empty. No free-text answers — for non-graded questions use ask_user_question instead. " +
|
||||
"ONE question per turn: call a single quiz, then end your turn — his answer determines the next. " +
|
||||
QUIZ_OPTION_CRAFT,
|
||||
description: quizDescription({ nodeId: true, types: true }),
|
||||
parameters: quizInputObject,
|
||||
prepareArguments: prepareQuestionArguments,
|
||||
executionMode: "sequential",
|
||||
|
||||
Reference in New Issue
Block a user