free-recall explain tool: the quizzing-mechanic switch (owner 2026-09-27)

Sycophancy mitigation as a structural commitment: the model writes its
full correctAnswer (plus explanation, in that order) BEFORE the learner
sees the question; the commitment rides the persisted call args, is
revealed on submit, and grades his free text in the model's next reply —
the client never grades free recall.

- tools.explain setting (segmented 'Quizzing — Multiple choice | Free
  recall' in Agent settings' Tools group): free arms explain and drops
  quiz for BOTH agents (prober variant carries no nodeId, same pattern
  as the probe quiz); mechanics keys never pass filterEnabledTools —
  quiz stays unwritable (the 2026-09-19 wedge ruling stands)
- doctrine lives pi-natively in the tool descriptions (owner ruling —
  no conditional system-prompt chunks): explain carries the commitment
  rule; the MC option-building craft moved out of the prompt into the
  quiz description; shared/20-question-tools.md slims to the
  quiz-agnostic core
- display seams: parseExplainInput (LaTeX-debris restoration included),
  dock text-mode with an 'I don't know' row (kbd row 0), AnsweredCard
  reveal (The answer / Why, no verdict chips), [Explain answer] format
- phases/plan seams recognize both quizzing tools' nodeId tags; explain
  answers deliberately feed no ledger evidence (no client grading to
  key on — a future ruling can parse the model's grade)

1049/1049 tests, typecheck/lint/build green.
This commit is contained in:
amosblomqvist
2026-09-27 08:55:13 +03:00
parent 07d2c84461
commit 2f39766528
25 changed files with 784 additions and 68 deletions
+50
View File
@@ -66,6 +66,7 @@ import {
} from "@/src/core";
import type { ThinkingLevel } from "@earendil-works/pi-agent-core";
import { prettyModel } from "@/components/usage-format";
import { cn } from "@/lib/utils";
import { SettingsGroup, SettingsRow, Toggle } from "./controls.tsx";
import { ModelMenuList, ThinkingMenuList } from "./model-menu";
@@ -110,6 +111,15 @@ export function composeTool(agent: AgentSettings | undefined, id: string, on: bo
return { ...agent, tools: { ...agent?.tools, [id]: on } };
}
/** The quizzing mechanics (owner 2026-09-27): a segmented control, not a
* toggle — exactly one mechanic is ever active. `explain: true` arms
* free-recall (explain in, quiz out); anything else is multiple choice.
* `quiz` itself is never written (mechanics-managed — see quizMechanics
* in agent/tools.ts). Pure — exported for tests. */
export function composeQuizMechanics(agent: AgentSettings | undefined, mechanics: "choice" | "free"): AgentSettings {
return { ...agent, tools: { ...agent?.tools, explain: mechanics === "free" } };
}
/** Compose the prober graph-tools toggle (wave 7): its own key in the
* agent section — never in the tool-toggles map (owner ruling). Absent =
* ON (the toggle discipline); the write is explicit either way, so an
@@ -725,6 +735,46 @@ export function AgentSection({
</SettingsRow>
);
})()}
{/* The quizzing mechanic (owner 2026-09-27): segmented — exactly one
of Multiple choice / Free recall is active; the choice swaps which
quizzing tool BOTH agents carry (prober checks + teach checks).
Persisted as agent.tools.explain; absent = Multiple choice. */}
{(() => {
const free = agent?.tools?.explain === true;
return (
<SettingsRow
label="Quizzing"
hint="How both tutors check your understanding"
testid="quiz-mechanics-row"
>
<div
role="radiogroup"
aria-label="Quizzing mechanics"
className="flex rounded-lg border border-border/60 bg-muted/30 p-0.5"
>
{(["choice", "free"] as const).map((m) => {
const active = m === "free" ? free : !free;
return (
<button
key={m}
type="button"
role="radio"
aria-checked={active}
data-testid={`quiz-mechanics-${m}`}
onClick={() => onSectionChange(composeQuizMechanics(agent, m))}
className={cn(
"h-8 rounded-md px-3 text-sm font-medium transition-colors",
active ? "bg-background text-foreground shadow-sm" : "text-muted-foreground hover:text-foreground",
)}
>
{m === "choice" ? "Multiple choice" : "Free recall"}
</button>
);
})}
</div>
</SettingsRow>
);
})()}
{TOGGLE_TOOLS.map((id) => {
// Research needs a live search engine; the row + toggle gray
// on the same signal the registry drops the tool on (see