quiz answers carry a confidence slider — raw probability, snapped 0/30/50/70/90/100
Owner design round 2026-10-01: every multiple-choice quiz answer now carries the learner's stated probability that his selected answer is correct — calibration data for the prober and teacher, the flow itself unchanged. Definition (owner ruling, after weighing chance-normalization): RAW probability. Sub-chance reports are diagnostic gold, not dead space — 'picked an option, believes it wrong' is a real state. The agents get the sophistication: new shared/25-confidence.md composes for both agents (chance level, sub-chance = distrusts the pick, correct+low = fragile, wrong+high = committed misconception, IDK = genuine gap). - question-core: AnswerRequest/QuestionAnswerData.confidence (quiz only, optional — old records valid), buildAnswer clamps and drops it on dontKnow, formatAnswerMessage states the line, CONFIDENCE_STOPS + DEFAULT_CONFIDENCE_INDEX exported. - question-dock: snapped slider below 'I don't know', above 'Your thinking' — native range over the stop INDEX so ←/→ step the uneven grid cleanly, Tab chain rows → slider → thinking, Enter submits from the slider too. Rests at 50 and always sends (owner: 50 must mean a real coin flip, never 'untouched'). 'I don't know' GRAYS it out — layout holds, no value sent. Deliberately untouched: explain free-recall card (quiz-only scope), answered-card rendering of confidence, ask flow.
This commit is contained in:
@@ -179,6 +179,7 @@ describe("the source-derived segment list", () => {
|
||||
"system-prompt.md",
|
||||
"prober/10-probe-doctrine.md",
|
||||
"shared/20-question-tools.md",
|
||||
"shared/25-confidence.md",
|
||||
"shared/30-learner-files.md",
|
||||
"shared/40-formatting.md",
|
||||
],
|
||||
|
||||
@@ -12,6 +12,8 @@ import {
|
||||
askOptionProblems,
|
||||
buildAnswer,
|
||||
coerceCorrectAnswer,
|
||||
CONFIDENCE_STOPS,
|
||||
DEFAULT_CONFIDENCE_INDEX,
|
||||
findPendingQuestion,
|
||||
formatAnswerMessage,
|
||||
isCorrect,
|
||||
@@ -337,6 +339,13 @@ function quizData(overrides?: Partial<QuestionData>): QuestionData {
|
||||
};
|
||||
}
|
||||
|
||||
describe("confidence stops", () => {
|
||||
it("the owner-decided grid (2026-10-01): resolution concentrates at the top", () => {
|
||||
assert.deepEqual(CONFIDENCE_STOPS, [0, 30, 50, 70, 90, 100]);
|
||||
assert.equal(CONFIDENCE_STOPS[DEFAULT_CONFIDENCE_INDEX], 50, "rests at a real coin flip");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildAnswer", () => {
|
||||
it("grades a correct selection", () => {
|
||||
const ad = buildAnswer(quizData(), { values: ["mercury"] });
|
||||
@@ -368,6 +377,22 @@ describe("buildAnswer", () => {
|
||||
assert.deepEqual(ad.answers.map((a) => a.value), ["mercury"]);
|
||||
});
|
||||
|
||||
// Confidence (owner, 2026-10-01): the slider's raw probability rides on
|
||||
// real quiz answers only — never on "I don't know" (no answer, no
|
||||
// probability), never on ask/explain. Clamped defensively.
|
||||
it("carries confidence on quiz answers and clamps out-of-range values", () => {
|
||||
assert.equal(buildAnswer(quizData(), { values: ["mercury"], confidence: 90 }).confidence, 90);
|
||||
assert.equal(buildAnswer(quizData(), { values: ["mercury"], confidence: 150 }).confidence, 100);
|
||||
assert.equal(buildAnswer(quizData(), { values: ["mercury"], confidence: -5 }).confidence, 0);
|
||||
assert.equal(buildAnswer(quizData(), { values: ["mercury"] }).confidence, undefined);
|
||||
});
|
||||
|
||||
it("drops confidence on dontKnow and on non-quiz kinds", () => {
|
||||
assert.equal(buildAnswer(quizData(), { dontKnow: true, confidence: 70 }).confidence, undefined);
|
||||
const ask = quizData({ kind: "ask", correctIndices: undefined, explanation: undefined });
|
||||
assert.equal(buildAnswer(ask, { other: "x", confidence: 70 }).confidence, undefined);
|
||||
});
|
||||
|
||||
it("multi-select grades as exact-set", () => {
|
||||
const multi = quizData({
|
||||
mode: "multi-select",
|
||||
@@ -412,6 +437,13 @@ describe("formatAnswerMessage", () => {
|
||||
assert.match(formatAnswerMessage(ad), /Note: guessed/);
|
||||
});
|
||||
|
||||
it("states the confidence line on answered quizzes, absent on dontKnow", () => {
|
||||
const ad = buildAnswer(quizData(), { values: ["mercury"], confidence: 70 });
|
||||
assert.match(formatAnswerMessage(ad), /Confidence: 70% — the learner's stated probability that his selected answer is correct\./);
|
||||
const idk = buildAnswer(quizData(), { dontKnow: true, confidence: 70 });
|
||||
assert.ok(!formatAnswerMessage(idk).includes("Confidence:"));
|
||||
});
|
||||
|
||||
it("formats ask answers", () => {
|
||||
const ask = quizData({ kind: "ask", correctIndices: undefined, explanation: undefined });
|
||||
const ad = buildAnswer(ask, { other: "whatever" });
|
||||
|
||||
Reference in New Issue
Block a user