free-recall explain tool: the quizzing-mechanic switch (owner 2026-09-27)

Sycophancy mitigation as a structural commitment: the model writes its
full correctAnswer (plus explanation, in that order) BEFORE the learner
sees the question; the commitment rides the persisted call args, is
revealed on submit, and grades his free text in the model's next reply —
the client never grades free recall.

- tools.explain setting (segmented 'Quizzing — Multiple choice | Free
  recall' in Agent settings' Tools group): free arms explain and drops
  quiz for BOTH agents (prober variant carries no nodeId, same pattern
  as the probe quiz); mechanics keys never pass filterEnabledTools —
  quiz stays unwritable (the 2026-09-19 wedge ruling stands)
- doctrine lives pi-natively in the tool descriptions (owner ruling —
  no conditional system-prompt chunks): explain carries the commitment
  rule; the MC option-building craft moved out of the prompt into the
  quiz description; shared/20-question-tools.md slims to the
  quiz-agnostic core
- display seams: parseExplainInput (LaTeX-debris restoration included),
  dock text-mode with an 'I don't know' row (kbd row 0), AnsweredCard
  reveal (The answer / Why, no verdict chips), [Explain answer] format
- phases/plan seams recognize both quizzing tools' nodeId tags; explain
  answers deliberately feed no ledger evidence (no client grading to
  key on — a future ruling can parse the model's grade)

1049/1049 tests, typecheck/lint/build green.
This commit is contained in:
amosblomqvist
2026-09-27 08:55:13 +03:00
parent 07d2c84461
commit 2f39766528
25 changed files with 784 additions and 68 deletions
@@ -1,6 +1,6 @@
# The probe
Your whole job is two measurements: **what he needs to learn** (his goal — `ask_user_question`) and **what he already knows** (a detailed map of his exact understanding — `quiz`). The whole lesson will be planned from your map — it must be precise.
Your whole job is two measurements: **what he needs to learn** (his goal — `ask_user_question`) and **what he already knows** (a detailed map of his exact understanding — the quizzing tool: `quiz` or `explain`, whichever the learner’s mechanic armed). The whole lesson will be planned from your map — it must be precise.
Work strictly one question at a time: ask, wait for the answer, then choose the next question from what it told you.
@@ -16,7 +16,7 @@ Work strictly one question at a time: ask, wait for the answer, then choose the
From the goal, what will the lesson depend on? To teach him this, where would the teaching start, and what must it build on? Each dependency is a strand — bound by relevance, nothing more.
## 4. Binary-search every strand — `quiz`
## 4. Binary-search every strand — the quizzing tool
The edge of a strand is found by bracketing: something he reliably gets **right** (a floor) and something he gets **wrong** or doesn't know (a ceiling). Narrow the bracket one question at a time:
+6 -20
View File
@@ -1,28 +1,14 @@
## The question tools
Two instruments. `quiz` only for questions with a known correct answer, `ask_user_question` for everything else.
Two instruments. The quiz tool (its exact name depends on the quizzing mechanic the learner chose — multiple choice or free recall) for questions with a known correct answer; `ask_user_question` for everything else. Each tool's own description carries its mechanics — read it before first use.
### `quiz`
### Quiz checks — shared doctrine
Asks a question, grades instantly, shows ✓/✗ + the correct answer + explanation to both of you. Single- or multi-select.
- `correctAnswer` (required) is the option's `value` string — self-checking, never a position number. Multi-select: array of values, graded as an exact-set match.
- `explanation` (required) says why the correct answer is correct. It comes BEFORE `correctAnswer` in the call by design: reason it out first, then commit the answer the reasoning supports.
- An "I don't know" option is added automatically — provide only the real, gradable options (≥2), never your own opt-out. A dontKnow result is a genuine gap, not a wrong answer.
- Answers may carry a free-text `note` from the always-present note field — read it; it says what he was thinking.
- Options display shuffled by default; `shuffle: false` only when order is meaningful (e.g. an 'All/None of the above' that must stay last). `multiSelect: true` only when several options are correct.
- `explanation` (required) says why the correct answer is correct. It comes BEFORE `correctAnswer` in the call by design: reason it out first, then commit to the answer the reasoning supports.
- An "I don't know" choice is always available to the learner — never author your own opt-out. A dontKnow result is a genuine gap, not a wrong answer.
- Answers may carry a free-text `note` — read it; it says what he was thinking.
- ONE question per turn: ask, then end your turn — his answer determines the next.
### `ask_user_question`
Asks a single question and waits. He can always pick "Other" and type. If you recommend an option, list it first with "(Recommended)" appended. Multiple questions → multiple calls.
### Building quiz options
The giveaway is never the distractors' wrongness — it's asymmetry: the correct option carrying its reasoning while the others are bare, or key-term bolding that flags it. Build evenness in, don't audit it in afterwards:
1. Every option is a bare claim — zero "why" in any of them; all reasoning goes in `explanation`, shown only after he answers.
2. Write the correct claim first, then mutate it into each distractor: what would someone holding a specific misconception — or confusing it with a specific neighbour — claim? Same skeleton, same grain, same register: parallelism by construction.
3. Each distractor is a real error he might actually make (so his pick is diagnostic) yet unambiguously wrong.
4. No asymmetric bolding — bold the parallel term in every option or in none.
If you can tell the right one cold without knowing the material, regenerate — don't patch.
+1 -1
View File
@@ -26,7 +26,7 @@ You write the map with `plan_init` (a mermaid `graph TD` block). It is a **map,
The map is yours to rewrite at your own initiative: re-call `plan_init` whenever it's wrong — a detour node for a question worth exploring, a repair node for a caught misconception, prune what proves unnecessary. A map that can't change is a wrong map.
**Every quiz carries `nodeId`** (required — a quiz without it is rejected): that's what moves the map. Its live position is simply the latest tagged node, so asking about the next node IS how the lesson gets there. A tag is a position, not a verdict — nodes are never marked finished. A node takes as many questions as it deserves: stay on one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward — and move on by asking about the next one. Off-map checks (no plan node fits) use ask_user_question.
**Every quizzing call carries `nodeId`** (required — the quizzing tool is one of `quiz` or `explain` depending on the mechanic the learner chose; a call without it is rejected): that's what moves the map. Its live position is simply the latest tagged node, so asking about the next node IS how the lesson gets there. A tag is a position, not a verdict — nodes are never marked finished. A node takes as many questions as it deserves: stay on one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward — and move on by asking about the next one. Off-map checks (no plan node fits) use ask_user_question.
## Presenting the plan — always, before any teaching
@@ -5,5 +5,5 @@ Every check exists to answer one question: did the idea actually **take** — no
- **Answerable by reasoning from the session.** Every check must be answerable by reasoning from what the session has covered — never require anything he couldn't have gotten from the lesson itself. A check he has no route to is not a check, it's a coin flip.
- **Never just reciting your words back.** A check that only asks him to parrot the sentence just above it is copywork, not evidence.
- **Prefer problems that apply the knowledge.** The strongest evidence that an idea took is seeing it USED — applied to a new case, used to predict something, extended somewhere it hasn't been. Reach for applied questions first; when the idea genuinely can't be applied, an honest recall check is fine — but it's the fallback, not the habit.
- **Catch confusion early, and make it nameable.** Design the wrong options as real misconceptions — each the answer he'd give under a specific wrong model. Then a miss TELLS you which wrong model he holds; diagnose and repair before building anything on top of it.
- **Catch confusion early, and make it nameable.** Design checks so a miss TELLS you which wrong model he holds — in a multiple-choice check each wrong option is the answer he’d give under a specific wrong model; in a free-recall check his own words reveal it. Diagnose and repair before building anything on top of it.
- **His answer is signal, not verdict.** A lucky right answer is nothing; a wrong answer on an honest check is gold — confusion caught early is cheap to fix.
+2 -1
View File
@@ -203,7 +203,8 @@ function baseAgentOptions(kind: AgentKind, opts: AgentBuildOptions, tools: Agent
}
/**
* The PROBER (agent A): quiz (no nodeId) + ask + probe_done, lean on
* The PROBER (agent A): the quizzing tool (quiz, or explain under free
* recall — no nodeId either way) + ask + probe_done, lean on
* probing doctrine. Its whole job is bracketing the learner's edge and
* concluding with the assessment (probe_done) — which also queues the
* handoff wake (see tools/probe.ts) and is the transcript seam the
+114 -6
View File
@@ -4,6 +4,9 @@ import { questionResult, serverExecute, ToolInputError } from "./tool-execute.ts
import {
askInputSchema,
askOptionProblems,
explainInputBase,
explainInputObject,
explainInputProblems,
parseOptionsIfString,
probeQuizInputSchema,
quizInputObject,
@@ -92,6 +95,32 @@ export const askUserQuestionTool: AgentTool<typeof askInputSchema> = {
// ─── the prober's question tools (no nodeId — no map exists yet) ─────────────
/** The option-building craft (owner 2026-09-27: moved OUT of the system
* prompt into the tool description — pi-native doctrine travels with
* the armed tool). Shared verbatim by both quiz variants. */
const QUIZ_OPTION_CRAFT =
"Building options — the giveaway is never the distractors' wrongness, it's asymmetry: the correct option " +
"carrying its reasoning while the others are bare, or key-term bolding that flags it. Build evenness in: " +
"(1) every option is a bare claim — zero 'why' in any of them; all reasoning goes in explanation, shown only " +
"after he answers; (2) write the correct claim first, then mutate it into each distractor — what would someone " +
"holding a specific misconception claim? Same skeleton, same grain, same register; (3) each distractor is a real " +
"error he might actually make (his pick is diagnostic) yet unambiguously wrong; (4) no asymmetric bolding. If " +
"you can tell the right one cold without knowing the material, regenerate — don't patch.";
/** The explain doctrine (owner 2026-09-27): free-recall quizzing — the
* sycophancy mitigation is the commitment. Carried by the DESCRIPTION
* (pi-native ruling), not the system prompt. Shared by both variants;
* the teach variant appends the nodeId tagging paragraph. */
const EXPLAIN_DOCTRINE =
"Ask the learner to EXPLAIN something in his own words — a graded check with NO options: he types a free-text " +
"answer. Unlike ask_user_question (no right answer), explain always has one. THE COMMITMENT RULE: write " +
"explanation (your reasoning) first, then correctAnswer — the FULL correct answer, committed before he ever sees " +
"the question. When his answer returns, grade it strictly against your committed answer in your reply: never " +
"revise the commitment after seeing his words, and never mark a wrong answer right to be kind — a near-miss is " +
"partial understanding; name exactly what he got and what he missed. He sees the committed answer and the " +
"explanation on submit. An 'I don't know' choice is always available to him — a genuine gap, not a wrong answer. " +
"ONE question per turn: call a single explain, then end your turn — his answer determines the next.";
export const probeQuizTool: AgentTool<typeof probeQuizInputSchema> = {
name: "quiz",
label: "quiz",
@@ -105,7 +134,7 @@ export const probeQuizTool: AgentTool<typeof probeQuizInputSchema> = {
"signal a genuine gap instead of guessing. An always-present optional note field lets the learner attach a " +
"free-text note to ANY answer; it reaches you only when non-empty. No free-text answers — for non-graded " +
"questions use ask_user_question instead. ONE question per turn: call a single quiz, then end your turn — " +
"his answer determines the next question to ask.",
"his answer determines the next question to ask. " + QUIZ_OPTION_CRAFT,
parameters: probeQuizInputSchema,
prepareArguments: prepareQuestionArguments,
executionMode: "sequential",
@@ -116,6 +145,48 @@ export const probeQuizTool: AgentTool<typeof probeQuizInputSchema> = {
},
};
/** The teacher's free-recall variant (nodeId-validated, same seam as the
* teach quiz — phases/plan recognize BOTH tool names). */
export function teachExplainToolFor(
ctx: Pick<PlanToolContext, "chatId" | "chatStore">,
): AgentTool<typeof explainInputObject> {
return {
name: "explain",
label: "explain",
description:
EXPLAIN_DOCTRINE +
" Set `nodeId` (required) to the plan node this question is about — the lesson's position on the map derives " +
"from those tags; a node takes as many questions as it deserves, so don't rush forward. Off-map checks use " +
"ask_user_question.",
parameters: explainInputObject,
prepareArguments: prepareQuestionArguments,
executionMode: "sequential",
execute: async (_toolCallId, input) => {
const problems = [...explainInputProblems(input)];
const nodeIdProblemText = nodeIdProblem(ctx, input.nodeId);
if (nodeIdProblemText) problems.push(nodeIdProblemText);
if (problems.length > 0) throw new ToolInputError("explain", problems);
return questionResult();
},
};
}
/** The prober's free-recall variant (no nodeId — same pattern as the
* probe quiz). */
export const probeExplainTool: AgentTool<typeof explainInputBase> = {
name: "explain",
label: "explain",
description: EXPLAIN_DOCTRINE,
parameters: explainInputBase,
prepareArguments: prepareQuestionArguments,
executionMode: "sequential",
execute: async (_toolCallId, input) => {
const problems = explainInputProblems(input);
if (problems.length > 0) throw new ToolInputError("explain", problems);
return questionResult();
},
};
/** The prober's registry context: the probe's own needs (probe_done —
* ProbeDoneContext) PLUS the job-dispatch wiring, since the owner ruling
* (2026-09-20) puts research + pdf in the probe phase's registry too. */
@@ -156,10 +227,20 @@ export function createProberTools(
spawn: ctx.spawn,
searchAvailable: ctx.searchAvailable,
});
// Quizzing mechanics (owner 2026-09-27): free-recall swaps the probe
// quiz for the probe explain — one quizzing tool at a time, whichever
// the settings arm. ask stays always-on.
const mechanics = quizMechanics(enabled);
// The armed quizzing tool, widened to the registry's heterogeneous
// AgentTool record BEFORE the filter (a conditional spread of two
// differently-typed literals makes the whole object's type a union —
// unusable as Record<string, AgentTool<any>> downstream).
const quizzing: Record<string, AgentTool<any>> =
mechanics === "free" ? { explain: probeExplainTool } : { quiz: probeQuizTool };
return {
...filterEnabledTools(
{
quiz: probeQuizTool,
...quizzing,
ask_user_question: askUserQuestionTool,
// The learner-filesystem set is always-on (user-decided 2026-09,
// sources v1): the prober inspects and LOCATES learner-referenced
@@ -184,7 +265,7 @@ export function createProberTools(
: {}),
...done,
},
enabled,
withoutMechanics(enabled),
),
...done,
};
@@ -230,7 +311,8 @@ export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStor
"plus an automatic 'I don't know' choice so the learner can signal a genuine gap instead of guessing. " +
"An always-present optional note field lets the learner attach a free-text note to ANY answer; it reaches you " +
"only when non-empty. No free-text answers — for non-graded questions use ask_user_question instead. " +
"ONE question per turn: call a single quiz, then end your turn — his answer determines the next.",
"ONE question per turn: call a single quiz, then end your turn — his answer determines the next. " +
QUIZ_OPTION_CRAFT,
parameters: quizInputObject,
prepareArguments: prepareQuestionArguments,
executionMode: "sequential",
@@ -269,13 +351,39 @@ export interface TeacherToolContext extends PlanToolContext {
* future toggleables (question-tool variants, …) need no change here.
* Absent/undefined = enabled — tests and old records see today's set.
*/
/** The quizzing mechanics (owner 2026-09-27): `tools.explain === true` in
* the resolved settings ⇒ free-recall checks (explain armed, quiz dropped)
* — anything else ⇒ multiple choice, exactly today's set. Mechanics-
* managed, NOT individually toggleable: quiz/explain never pass through
* filterEnabledTools (the 2026-09-19 wedge ruling stands — a quiz-less
* prober collapses its assessment flow). */
export function quizMechanics(enabled?: EnabledTools): "choice" | "free" {
return enabled?.explain === true ? "free" : "choice";
}
/** Strip the mechanics keys so filterEnabledTools can never drop the
* quizzing tools (the derivation above is their only switch). */
function withoutMechanics(enabled?: EnabledTools): EnabledTools | undefined {
if (!enabled) return undefined;
const { explain: _e, quiz: _q, ...rest } = enabled;
return rest;
}
export function createTeacherTools(
ctx: TeacherToolContext,
enabled?: EnabledTools,
): Record<string, AgentTool<any>> {
// Quizzing mechanics (owner 2026-09-27): free-recall swaps the teach
// quiz for the teach explain — one quizzing tool at a time, whichever
// the settings arm. ask stays always-on.
const mechanics = quizMechanics(enabled);
// Same widening as the prober's (see above) — the conditional spread of
// two differently-typed literals collapses into an unusable union type.
const quizzing: Record<string, AgentTool<any>> =
mechanics === "free" ? { explain: teachExplainToolFor(ctx) } : { quiz: teachQuizToolFor(ctx) };
return filterEnabledTools(
{
quiz: teachQuizToolFor(ctx),
...quizzing,
ask_user_question: askUserQuestionTool,
// The learner-filesystem set is always-on (user-decided 2026-09,
// sources v1): the teacher locates and reads learner-referenced
@@ -292,6 +400,6 @@ export function createTeacherTools(
// reaches jobTools' research gate.
...jobTools({ chatId: ctx.chatId, jobStore: ctx.jobStore, spawn: ctx.spawn, searchAvailable: ctx.searchAvailable }),
},
enabled,
withoutMechanics(enabled),
);
}
+4 -2
View File
@@ -85,7 +85,9 @@ export function derivePlanStatuses(tree: PlanTree, messages: readonly PartsCarri
// duplicates collapsed (several checks on one node = one stay on it)
for (const m of messages) {
for (const part of m.parts) {
if (part.type !== "tool-quiz") continue;
// Either quizzing tool (quiz/explain) tags position — the mechanic
// is a settings switch; the fold recognizes both (owner 2026-09-27).
if (part.type !== "tool-quiz" && part.type !== "tool-explain") continue;
const nodeId = (part.input as { nodeId?: unknown } | undefined)?.nodeId;
if (typeof nodeId !== "string" || !ids.has(nodeId)) continue; // untagged/unknown — moves nothing
if (runs.at(-1) !== nodeId) runs.push(nodeId);
@@ -130,7 +132,7 @@ export function derivePlanStatusesLh(tree: PlanTree, messages: readonly LhMessag
const content = (m as { content?: unknown }).content;
if (!Array.isArray(content)) continue;
for (const block of content as Array<{ type?: string; name?: string; arguments?: unknown }>) {
if (block.type !== "toolCall" || block.name !== "quiz") continue;
if (block.type !== "toolCall" || (block.name !== "quiz" && block.name !== "explain")) continue;
const nodeId = (block.arguments as { nodeId?: unknown } | undefined)?.nodeId;
if (typeof nodeId !== "string" || !ids.has(nodeId)) continue;
if (runs.at(-1) !== nodeId) runs.push(nodeId);
+8 -1
View File
@@ -39,7 +39,7 @@ export interface PendingToolCall {
}
/** Tools that render in the question dock and gate the composer. */
export const QUESTION_TOOLS = ["quiz", "ask_user_question"] as const;
export const QUESTION_TOOLS = ["quiz", "ask_user_question", "explain"] as const;
/** Tools fulfilled by background sub-agent jobs (W2/W6). */
export const JOB_TOOLS = ["research", "demo", "visual"] as const;
@@ -62,6 +62,13 @@ export const JOB_TOOLS = ["research", "demo", "visual"] as const;
* 2026-09). */
export const TOGGLEABLE_TOOLS = ["research", "demo", "visual", "pdf"] as const;
/** The keys the settings WRITE side accepts in agent.tools: the plain
* toggleables PLUS the quizzing-mechanics switch (`explain: true` arms
* free-recall quizzing and drops the MC quiz — see quizMechanics in
* agent/tools.ts; owner 2026-09-27). `quiz` itself stays unwritable:
* mechanics-managed, never individually toggleable. */
export const WRITABLE_TOOL_KEYS = [...TOGGLEABLE_TOOLS, "explain"] as const;
/** The resolved settings → factory seam shape (Platform wires chat-handler
* to pass this through from the settings record; storage and defaults are
* Platform's lane). JSON-native: a partial record keyed by tool name.
+8 -2
View File
@@ -83,7 +83,13 @@ function advance(
if (phase === "probe" && part.type === "tool-probe_done" && part.state === "output-available") {
phase = "plan";
}
if (phase === "plan" && part.type === "tool-quiz" && taggedNodeId(part.input)) {
// plan → teach via the first nodeId-tagged quizzing CALL — quiz OR
// explain, whichever mechanic the settings armed (both carry nodeId).
if (
phase === "plan" &&
(part.type === "tool-quiz" || part.type === "tool-explain") &&
taggedNodeId(part.input)
) {
phase = "teach";
}
}
@@ -136,7 +142,7 @@ export function phaseAt3Lh(messages: readonly LhMessage[], upToIndex?: number):
for (const block of content) {
if (
block.type === "toolCall" &&
block.name === "quiz" &&
(block.name === "quiz" || block.name === "explain") &&
taggedNodeId(block.arguments)
) {
phase = "teach";
+89 -4
View File
@@ -217,6 +217,68 @@ export const quizInputObject = Type.Object({
* it ran at the call boundary, where quizInputProblems runs now). */
export const probeQuizInputSchema = quizInputBase;
// ─── the explain tool (free-recall quizzing, owner 2026-09-27) ─────────────
//
// Quiz mechanics WITHOUT options: the learner answers in his own words.
// The SYCOPHANCY MITIGATION is structural — the model commits to the full
// correct answer BEFORE posing (the commitment rides the persisted call
// args, is revealed to the learner on submit, and is restated in the answer
// result the model grades against; the client never grades free text).
// Mechanics doctrine lives in the TOOL DESCRIPTIONS (owner ruling
// 2026-09-27: pi-native — doctrine travels with the armed tool), not in
// the system prompt.
export const explainInputBase = Type.Object({
question: Type.String({
description:
"The single question to ask. ONE question per turn: ask, then end your turn — the answer determines the next.",
}),
explanation: Type.String({
description:
"REQUIRED. Explanation revealed AFTER the learner answers (shown however he did). It is written BEFORE " +
"correctAnswer by design: reason it out first, then commit to the answer the reasoning supports.",
}),
correctAnswer: Type.String({
description:
"REQUIRED. The full correct answer in your own words — COMMITTED before the learner ever sees the question. " +
"The commitment is immutable: when his answer returns, grade it strictly against this text — never revise the " +
"commitment after seeing his words, and never mark a wrong answer right to be kind. Both this and the explanation " +
"are shown to him on submit; your next reply grades the nuance of what he wrote.",
}),
details: Type.Optional(
Type.String({ description: "Optional extra context or instructions shown under the question." }),
),
});
/** The TEACH-side explain object: base + the plan-position field (same
* semantics as the quiz's nodeId — the seam recognizes BOTH tools). */
export const explainInputObject = Type.Object({
...explainInputBase.properties,
nodeId: Type.String({
description:
"REQUIRED plan-tree node id this question is about — must EXACTLY match one of this session's " +
"plan_init node slugs (validated at call time; a mismatched id is rejected back to you). " +
"The lesson's position on the map derives from these tags (the node the latest tagged question is about is " +
"where the map stands; earlier ones count as visited). A node takes as many questions as it deserves: stay on " +
"one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward. " +
"For off-map checks use ask_user_question.",
}),
});
/** The explain authoring contract: the commitment must exist. Empty
* result = an explain the dock can arm and the model can grade against. */
export function explainInputProblems(input: { correctAnswer: string }): string[] {
if (input.correctAnswer.trim().length === 0) {
return ["correctAnswer is required — commit to the full correct answer BEFORE posing the question."];
}
return [];
}
/** The tool names that carry the quizzing mechanic — the teach seam
* (phases), the plan-position fold, and the pending gate all recognize
* whichever is armed. */
export const QUIZ_TOOL_NAMES = ["quiz", "explain"] as const;
/** The TEACHER's quiz declaration (base + the position field). */
export const quizInputSchema = quizInputObject;
@@ -249,6 +311,8 @@ export const askInputSchema = Type.Object({
export type QuizToolInput = Static<typeof quizInputObject>;
export type AskToolInput = Static<typeof askInputSchema>;
export type ExplainToolInput = Static<typeof explainInputObject>;
export type ProbeExplainToolInput = Static<typeof explainInputBase>;
/** A real selected answer, in the order the options were displayed. */
@@ -258,7 +322,7 @@ export interface OptionAnswer {
index: number; // 1-based display index
}
export type QuestionKind = "quiz" | "ask";
export type QuestionKind = "quiz" | "ask" | "explain";
export type SelectMode = "single-select" | "multi-select";
export type QuestionMode = SelectMode | "text";
@@ -272,6 +336,8 @@ export interface QuestionData {
options: QuizOption[]; // display order (post-shuffle)
// quiz only. NEVER send to the client before answering:
correctIndices?: number[];
// explain only. NEVER send to the client before answering:
committedAnswer?: string;
explanation?: string;
createdAt: string;
}
@@ -285,11 +351,12 @@ export interface QuestionAnswerData {
dontKnow: boolean;
answers: OptionAnswer[];
other?: string; // ask: "Other" free text
text?: string; // ask: text-mode answer
text?: string; // ask/explain: text-mode answer
note?: string;
correct?: boolean; // quiz only
correctIndices?: number[]; // quiz only (rendering)
explanation?: string; // quiz only (rendering)
committed?: string; // explain only (rendering): the model's committed answer
explanation?: string; // quiz/explain only (rendering)
dismissed?: boolean;
answeredAt: string;
}
@@ -526,12 +593,18 @@ export function buildAnswer(qd: QuestionData, req: AnswerRequest): QuestionAnswe
base.explanation = qd.explanation;
base.correct = base.dontKnow ? false : isCorrect(base.answers.map((a) => a.index), qd.correctIndices ?? []);
}
if (qd.kind === "explain") {
// No client grading — the model grades in its reply, against the
// commitment. Snapshots ride along for the reveal + rendering.
base.committed = qd.committedAnswer;
base.explanation = qd.explanation;
}
return base;
}
/** The user message that carries the answer into the agent's context. */
export function formatAnswerMessage(ad: QuestionAnswerData): string {
const label = ad.kind === "quiz" ? "Quiz answer" : "Answer to your question";
const label = ad.kind === "quiz" ? "Quiz answer" : ad.kind === "explain" ? "Explain answer" : "Answer to your question";
const head = `[${label}]`;
const lines: string[] = [head];
@@ -553,6 +626,18 @@ export function formatAnswerMessage(ad: QuestionAnswerData): string {
lines.push(`Result: ${verdict}`);
}
lines.push(`Correct: ${correctStr}`);
} else if (ad.kind === "explain") {
if (ad.dontKnow) {
lines.push(
`The learner selected "I don't know" — they did NOT attempt an answer (a genuine knowledge gap, not a wrong guess).`,
);
} else {
lines.push(`The learner answered: ${ad.text ?? "(no answer)"}`);
}
if (ad.committed) {
lines.push(`Your committed correct answer (immutable — grade against it, never revise it): ${ad.committed}`);
}
lines.push("Grade his answer strictly against your committed answer in your reply: name what he got and what he missed.");
} else {
if (ad.text) {
lines.push(`The learner answered: ${ad.text}`);
+12 -1
View File
@@ -29,7 +29,7 @@
import { Value } from "typebox/value";
import type { TSchema } from "typebox";
import { quizInputBase, askInputSchema, askOptionProblems, quizInputProblems, type AskToolInput, type QuizToolInput } from "./question-core.ts";
import { quizInputBase, askInputSchema, askOptionProblems, quizInputProblems, explainInputBase, explainInputProblems, type AskToolInput, type QuizToolInput, type ExplainToolInput } from "./question-core.ts";
import { restoreMathEscapes } from "./math-delims.ts";
import { probeDoneInputSchema, type ProbeDoneInput } from "./contracts/probe.ts";
@@ -103,6 +103,17 @@ export function parseAskInput(input: unknown): AskToolInput | undefined {
return cleaned;
}
/** Explain parse for DISPLAY: the shared authoring base (the teacher's
* nodeId is an unknown key Value.Clean strips, same as the quiz seam) —
* both explain variants share the same fields, and the commitment
* refinement (explainInputProblems) mirrors the boundary chain. */
export function parseExplainInput(input: unknown): ExplainToolInput | undefined {
const cleaned = boundaryParse<ExplainToolInput>(explainInputBase, restoreMathEscapesDeep(input));
if (!cleaned) return undefined;
if (explainInputProblems(cleaned).length > 0) return undefined;
return cleaned;
}
/** probe_done parse — the core-owned schema the tool itself declares. */
export function parseProbeDoneInput(input: unknown): ProbeDoneInput | undefined {
return boundaryParse<ProbeDoneInput>(probeDoneInputSchema, input);
+6 -5
View File
@@ -23,7 +23,7 @@ import type {
RoleModelOverride,
} from "../core/settings.ts";
import { BACKGROUND_MODEL_ROLES, EXTRACTION_CHUNK_TOKENS_FLOOR, MODEL_SLOTS } from "../core/settings.ts";
import { TOGGLEABLE_TOOLS } from "../core/pending.ts";
import { WRITABLE_TOOL_KEYS } from "../core/pending.ts";
/** Parsed-or-rejected — `ok: false` carries the 400's error message. */
export type SettingsBodyResult<T> =
@@ -157,10 +157,11 @@ export function parseAgentSettingsBody(body: unknown): SettingsBodyResult<AgentS
if (tools.map) {
// Unknown tool keys are rejected at WRITE time (user decision
// 2026-09-19): the delta is inert while unread, but once the bridge
// honors it a persisted `quiz: false` would genuinely drop the quiz —
// and the definitive list is the adjudicated job-tools-only set.
// The read-side sanitizer stays lenient for forward-compat.
const supported = TOGGLEABLE_TOOLS as readonly string[];
// honors it a persisted bogus key would drift. The writable set is
// the job tools + the quizzing-mechanics switch (explain, owner
// 2026-09-27). The read-side sanitizer stays lenient for
// forward-compat.
const supported = WRITABLE_TOOL_KEYS as readonly string[];
for (const key of Object.keys(tools.map)) {
if (!supported.includes(key)) {
return { ok: false, error: `unknown tool "${key}" (supported: ${supported.join(", ")})` };