quiz answers carry a confidence slider — raw probability, snapped 0/30/50/70/90/100

Owner design round 2026-10-01: every multiple-choice quiz answer now
carries the learner's stated probability that his selected answer is
correct — calibration data for the prober and teacher, the flow itself
unchanged.

Definition (owner ruling, after weighing chance-normalization): RAW
probability. Sub-chance reports are diagnostic gold, not dead space —
'picked an option, believes it wrong' is a real state. The agents get
the sophistication: new shared/25-confidence.md composes for both
agents (chance level, sub-chance = distrusts the pick, correct+low =
fragile, wrong+high = committed misconception, IDK = genuine gap).

- question-core: AnswerRequest/QuestionAnswerData.confidence (quiz
  only, optional — old records valid), buildAnswer clamps and drops it
  on dontKnow, formatAnswerMessage states the line, CONFIDENCE_STOPS +
  DEFAULT_CONFIDENCE_INDEX exported.
- question-dock: snapped slider below 'I don't know', above 'Your
  thinking' — native range over the stop INDEX so ←/→ step the uneven
  grid cleanly, Tab chain rows → slider → thinking, Enter submits from
  the slider too. Rests at 50 and always sends (owner: 50 must mean a
  real coin flip, never 'untouched'). 'I don't know' GRAYS it out —
  layout holds, no value sent.

Deliberately untouched: explain free-recall card (quiz-only scope),
answered-card rendering of confidence, ask flow.
This commit is contained in:
amosblomqvist
2026-10-02 00:24:05 +03:00
parent d8cca61d1d
commit 9b9f6372fa
5 changed files with 137 additions and 6 deletions
+1
View File
@@ -179,6 +179,7 @@ describe("the source-derived segment list", () => {
"system-prompt.md",
"prober/10-probe-doctrine.md",
"shared/20-question-tools.md",
"shared/25-confidence.md",
"shared/30-learner-files.md",
"shared/40-formatting.md",
],
+32
View File
@@ -12,6 +12,8 @@ import {
askOptionProblems,
buildAnswer,
coerceCorrectAnswer,
CONFIDENCE_STOPS,
DEFAULT_CONFIDENCE_INDEX,
findPendingQuestion,
formatAnswerMessage,
isCorrect,
@@ -337,6 +339,13 @@ function quizData(overrides?: Partial<QuestionData>): QuestionData {
};
}
describe("confidence stops", () => {
it("the owner-decided grid (2026-10-01): resolution concentrates at the top", () => {
assert.deepEqual(CONFIDENCE_STOPS, [0, 30, 50, 70, 90, 100]);
assert.equal(CONFIDENCE_STOPS[DEFAULT_CONFIDENCE_INDEX], 50, "rests at a real coin flip");
});
});
describe("buildAnswer", () => {
it("grades a correct selection", () => {
const ad = buildAnswer(quizData(), { values: ["mercury"] });
@@ -368,6 +377,22 @@ describe("buildAnswer", () => {
assert.deepEqual(ad.answers.map((a) => a.value), ["mercury"]);
});
// Confidence (owner, 2026-10-01): the slider's raw probability rides on
// real quiz answers only — never on "I don't know" (no answer, no
// probability), never on ask/explain. Clamped defensively.
it("carries confidence on quiz answers and clamps out-of-range values", () => {
assert.equal(buildAnswer(quizData(), { values: ["mercury"], confidence: 90 }).confidence, 90);
assert.equal(buildAnswer(quizData(), { values: ["mercury"], confidence: 150 }).confidence, 100);
assert.equal(buildAnswer(quizData(), { values: ["mercury"], confidence: -5 }).confidence, 0);
assert.equal(buildAnswer(quizData(), { values: ["mercury"] }).confidence, undefined);
});
it("drops confidence on dontKnow and on non-quiz kinds", () => {
assert.equal(buildAnswer(quizData(), { dontKnow: true, confidence: 70 }).confidence, undefined);
const ask = quizData({ kind: "ask", correctIndices: undefined, explanation: undefined });
assert.equal(buildAnswer(ask, { other: "x", confidence: 70 }).confidence, undefined);
});
it("multi-select grades as exact-set", () => {
const multi = quizData({
mode: "multi-select",
@@ -412,6 +437,13 @@ describe("formatAnswerMessage", () => {
assert.match(formatAnswerMessage(ad), /Note: guessed/);
});
it("states the confidence line on answered quizzes, absent on dontKnow", () => {
const ad = buildAnswer(quizData(), { values: ["mercury"], confidence: 70 });
assert.match(formatAnswerMessage(ad), /Confidence: 70% — the learner's stated probability that his selected answer is correct\./);
const idk = buildAnswer(quizData(), { dontKnow: true, confidence: 70 });
assert.ok(!formatAnswerMessage(idk).includes("Confidence:"));
});
it("formats ask answers", () => {
const ask = quizData({ kind: "ask", correctIndices: undefined, explanation: undefined });
const ad = buildAnswer(ask, { other: "whatever" });