free-recall explain tool: the quizzing-mechanic switch (owner 2026-09-27)

Sycophancy mitigation as a structural commitment: the model writes its
full correctAnswer (plus explanation, in that order) BEFORE the learner
sees the question; the commitment rides the persisted call args, is
revealed on submit, and grades his free text in the model's next reply —
the client never grades free recall.

- tools.explain setting (segmented 'Quizzing — Multiple choice | Free
  recall' in Agent settings' Tools group): free arms explain and drops
  quiz for BOTH agents (prober variant carries no nodeId, same pattern
  as the probe quiz); mechanics keys never pass filterEnabledTools —
  quiz stays unwritable (the 2026-09-19 wedge ruling stands)
- doctrine lives pi-natively in the tool descriptions (owner ruling —
  no conditional system-prompt chunks): explain carries the commitment
  rule; the MC option-building craft moved out of the prompt into the
  quiz description; shared/20-question-tools.md slims to the
  quiz-agnostic core
- display seams: parseExplainInput (LaTeX-debris restoration included),
  dock text-mode with an 'I don't know' row (kbd row 0), AnsweredCard
  reveal (The answer / Why, no verdict chips), [Explain answer] format
- phases/plan seams recognize both quizzing tools' nodeId tags; explain
  answers deliberately feed no ledger evidence (no client grading to
  key on — a future ruling can parse the model's grade)

1049/1049 tests, typecheck/lint/build green.
This commit is contained in:
amosblomqvist
2026-09-27 08:55:13 +03:00
parent 07d2c84461
commit 2f39766528
25 changed files with 784 additions and 68 deletions
+36
View File
@@ -457,3 +457,39 @@ describe("probe-done card", () => {
assert.match(html, /Writing the assessment/);
});
});
// ─── the explain reveal (free-recall, owner 2026-09-27) ────────────────────
describe("AnsweredCard: the explain variant", () => {
const explainQD = (): QuestionData => ({
id: "e1",
kind: "explain",
question: "Explain why the null result matters.",
mode: "text",
options: [],
committedAnswer: "It ruled out the luminiferous aether.",
explanation: "A stationary aether predicts a fringe shift; none was seen.",
createdAt: new Date().toISOString(),
});
it("reveals the committed answer + explanation with NO verdict chips", () => {
const html = renderToStaticMarkup(<AnsweredCard answer={buildAnswer(explainQD(), { text: "Something about relativity?" })} />);
assert.match(html, /Explain/, "labeled as the explain kind");
assert.match(html, /The answer:/);
assert.match(html, /luminiferous aether/);
assert.match(html, /Why:/);
assert.ok(!html.includes("incorrect") && !html.includes("correct</") , "no client grading");
assert.ok(!html.includes('data-mark="x"') && !html.includes('data-mark="check"'));
});
it("renders his free text as the answer", () => {
const html = renderToStaticMarkup(<AnsweredCard answer={buildAnswer(explainQD(), { text: "Something about relativity?" })} />);
assert.match(html, /Something about relativity\?/);
});
it("dontKnow renders as the gap, not a wrong answer", () => {
const html = renderToStaticMarkup(<AnsweredCard answer={buildAnswer(explainQD(), { dontKnow: true })} />);
assert.match(html, /I don&#x27;t know/);
assert.ok(!html.includes("The learner answered:"));
});
});