quiz answers carry a confidence slider — raw probability, snapped 0/30/50/70/90/100

Owner design round 2026-10-01: every multiple-choice quiz answer now
carries the learner's stated probability that his selected answer is
correct — calibration data for the prober and teacher, the flow itself
unchanged.

Definition (owner ruling, after weighing chance-normalization): RAW
probability. Sub-chance reports are diagnostic gold, not dead space —
'picked an option, believes it wrong' is a real state. The agents get
the sophistication: new shared/25-confidence.md composes for both
agents (chance level, sub-chance = distrusts the pick, correct+low =
fragile, wrong+high = committed misconception, IDK = genuine gap).

- question-core: AnswerRequest/QuestionAnswerData.confidence (quiz
  only, optional — old records valid), buildAnswer clamps and drops it
  on dontKnow, formatAnswerMessage states the line, CONFIDENCE_STOPS +
  DEFAULT_CONFIDENCE_INDEX exported.
- question-dock: snapped slider below 'I don't know', above 'Your
  thinking' — native range over the stop INDEX so ←/→ step the uneven
  grid cleanly, Tab chain rows → slider → thinking, Enter submits from
  the slider too. Rests at 50 and always sends (owner: 50 must mean a
  real coin flip, never 'untouched'). 'I don't know' GRAYS it out —
  layout holds, no value sent.

Deliberately untouched: explain free-recall card (quiz-only scope),
answered-card rendering of confidence, ask flow.
This commit is contained in:
amosblomqvist
2026-10-02 00:24:05 +03:00
parent d8cca61d1d
commit 9b9f6372fa
5 changed files with 137 additions and 6 deletions
+11
View File
@@ -0,0 +1,11 @@
## Confidence — reading the learner's self-assessment
Every quiz answer arrives with a `Confidence: N%` line: the learner's stated probability that his selected answer is correct, set on a snapped slider (0, 30, 50, 70, 90, 100). This is calibration data, not decoration. Read it against the verdict:
- 50 is a genuine coin flip. 90–100 claims certainty; 0–30 claims real doubt.
- For an n-option question, chance is 1/n. Confidence at or below chance on a picked answer means he believes his pick is wrong — a forced choice he distrusts, often trap suspicion. Treat it as honesty, not error.
- Correct + high confidence: solid knowledge — move on.
- Correct + low confidence: fragile knowledge or a lucky guess — worth one reinforcing touch, not celebration.
- Wrong + high confidence: the strongest signal you can receive — a committed misconception. Name the crack, probe it gently, and re-teach around it.
- Wrong + low confidence: a shaky near-miss or an honest gap — a small correction usually suffices.
- "I don't know" carries no confidence line: he declined to guess. A genuine gap, never a wrong answer.
+19 -1
View File
@@ -350,6 +350,7 @@ export interface QuestionAnswerData {
other?: string; // ask: "Other" free text
text?: string; // ask/explain: text-mode answer
note?: string;
confidence?: number; // quiz only: the learner's stated probability (0-100, raw) that his selected answer is correct — never sent on "I don't know" (owner, 2026-10-01)
correct?: boolean; // quiz only
correctIndices?: number[]; // quiz only (rendering)
committed?: string; // explain only (rendering): the model's committed answer
@@ -375,8 +376,17 @@ export interface AnswerRequest {
other?: string; // ask "Other"
text?: string; // ask text-mode
note?: string;
confidence?: number; // quiz only: 0-100 raw probability — 50 is a real coin flip, always sent (owner, 2026-10-01)
}
/** The confidence slider's snap stops (owner, 2026-10-01): raw probability
* that the selected answer is correct, uneven on purpose — resolution
* concentrates at the top where the calibration signal lives (90 vs 100
* separates knows from mastered). 50 is a genuine coin flip and the
* resting default — untouched and deliberate 50 are the same number. */
export const CONFIDENCE_STOPS: readonly number[] = [0, 30, 50, 70, 90, 100];
export const DEFAULT_CONFIDENCE_INDEX = 2;
// ─── dock selection (the client state machine, pure for tests) ─────────────
/** The question dock's selection state (0-based indices in display order). */
@@ -571,7 +581,6 @@ export function buildAnswer(qd: QuestionData, req: AnswerRequest): QuestionAnswe
note: req.note?.trim() || undefined,
answeredAt: new Date().toISOString(),
};
// resolve selected option values → OptionAnswer (in display order)
if (req.values && req.values.length > 0) {
const byValue = new Map(qd.options.map((o, i) => [o.value, { ...o, index: i + 1 }]));
@@ -589,6 +598,12 @@ export function buildAnswer(qd: QuestionData, req: AnswerRequest): QuestionAnswe
base.correctIndices = qd.correctIndices ?? [];
base.explanation = qd.explanation;
base.correct = base.dontKnow ? false : isCorrect(base.answers.map((a) => a.index), qd.correctIndices ?? []);
// The confidence slider rides on real quiz answers only — never on
// "I don't know" (no answer, no probability). Clamped defensively;
// the dock snaps to the grid, the record accepts any integer.
if (!base.dontKnow && req.confidence !== undefined) {
base.confidence = Math.max(0, Math.min(100, Math.round(req.confidence)));
}
}
if (qd.kind === "explain") {
// No client grading — the model grades in its reply, against the
@@ -620,6 +635,9 @@ export function formatAnswerMessage(ad: QuestionAnswerData): string {
const selectedStr = ad.answers.map((a) => `${a.index}. ${a.label}`).join(", ");
lines.push(`Selected: ${selectedStr || "(none)"}`);
lines.push(`Result: ${verdict}`);
if (ad.confidence !== undefined) {
lines.push(`Confidence: ${ad.confidence}% — the learner's stated probability that his selected answer is correct.`);
}
}
lines.push(`Correct: ${correctStr}`);
} else if (ad.kind === "explain") {