sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

practice UI round — top bars + sidebar reopen, one-attempt per-problem submit with instant reveal, score-only grading (owner 2026-09-29)

Browse Source
This commit is contained in:
amosblomqvist
2026-09-29 21:54:32 +03:00
parent ddce919f9b
commit cf0ec4a2f0
14 changed files with 811 additions and 554 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -156,7 +156,7 @@ data/ chats/ · graph/ (six ledgers + positions/source
**The knowledge graph (two channels, one writer).** The write channel: the extraction pipeline — extractor (LLM: sees only the session, rendered as readable markdown by the shared `core/session-markdown.ts` engine — propositions as true statements, `follows`/`supports` edges with verbatim evidence; fully trusted, graph-blind) → auditor (LLM, an agentic loop that never sees the source: explores the existing graph through `find` / `list_sessions` / `list_propositions` tools, stamps each proposition with an identity verdict — link to an existing node (equivalence bias: paraphrases merge) or create — and may draw bridging edges between existing nodes and the new batch) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, resolves and dedupes edges, checkpoints coverage) → layout settle. No quiz outcomes or understanding assessments live in the graph (2026-09-27 cleanup); practice (`/practice`) consumes the graph as feedback-only problem sets and writes nothing back. Triggered when a chat's **uncovered tokens** (message-view fingerprints, `coverage.jsonl`) cross `graph.extractionChunkTokens` (a settings knob, default 4096, floor 500 — read fresh per turn); failure checkpoints nothing, the next trigger re-runs. The same pipeline serves the **notes front door** (`POST /api/graph/import`: any `*.md`/`*.txt` becomes a source; re-imports are idempotent by path hash). The read channel: `GET /api/graph` derives the snapshot from the ledgers — a pure read; the home view is a hand-rolled Canvas 2D renderer (one rAF loop, dirty-flag idle, d3-zoom camera, zoom-gated labels, cluster hulls) over a d3-force layout, with server-persisted render settings and drag-committed positions. Clusters are **birth-session groupings**: a node joins the cluster of the session that birthed it (a chat, or an imported notes file as a file-kind session), derived at read — no embedding pass, no naming pass. The teacher reaches the graph on its own terms through the gated prober tools (`list_sessions` / `list_propositions`, the `agent.graphTools` setting) — there is no automatic context injection. **The knowledge graph (two channels, one writer).** The write channel: the extraction pipeline — extractor (LLM: sees only the session, rendered as readable markdown by the shared `core/session-markdown.ts` engine — propositions as true statements, `follows`/`supports` edges with verbatim evidence; fully trusted, graph-blind) → auditor (LLM, an agentic loop that never sees the source: explores the existing graph through `find` / `list_sessions` / `list_propositions` tools, stamps each proposition with an identity verdict — link to an existing node (equivalence bias: paraphrases merge) or create — and may draw bridging edges between existing nodes and the new batch) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, resolves and dedupes edges, checkpoints coverage) → layout settle. No quiz outcomes or understanding assessments live in the graph (2026-09-27 cleanup); practice (`/practice`) consumes the graph as feedback-only problem sets and writes nothing back. Triggered when a chat's **uncovered tokens** (message-view fingerprints, `coverage.jsonl`) cross `graph.extractionChunkTokens` (a settings knob, default 4096, floor 500 — read fresh per turn); failure checkpoints nothing, the next trigger re-runs. The same pipeline serves the **notes front door** (`POST /api/graph/import`: any `*.md`/`*.txt` becomes a source; re-imports are idempotent by path hash). The read channel: `GET /api/graph` derives the snapshot from the ledgers — a pure read; the home view is a hand-rolled Canvas 2D renderer (one rAF loop, dirty-flag idle, d3-zoom camera, zoom-gated labels, cluster hulls) over a d3-force layout, with server-persisted render settings and drag-committed positions. Clusters are **birth-session groupings**: a node joins the cluster of the session that birthed it (a chat, or an imported notes file as a file-kind session), derived at read — no embedding pass, no naming pass. The teacher reaches the graph on its own terms through the gated prober tools (`list_sessions` / `list_propositions`, the `agent.graphTools` setting) — there is no automatic context injection.
**Practice (problem sets).** The graph's `follows`-edges become reasoning chains; the `/practice` creation dialog picks a birth-session cluster (the graph view's own groups) and freezes its propositions + chains into a self-contained `ProblemSet`. Three named-level axes steer generation (ladder rung 0–4, cue distance 0–2, Theory|Application — never a number to interpolate; defaults Direct exercise · Adjacent context · Theory). A partitioner agent covers every chain exactly once, one generator per partition writes problems against the owner's problem-engineering framework (commit-before-pose: statement + solution + rubric together), and grading is **strengthening, not assessment** — one grader per answer, verdict (`solid`/`partial`/`missed`) + feedback, post-grade reveal, nothing ever written back to the graph. Runs are registry-backed background jobs on the activity dock (no steer, never wakes the teacher); sets live in `data/practice`, crash-swept on boot. **Practice (problem sets).** The graph's `follows`-edges become reasoning chains; the `/practice` creation dialog picks a birth-session cluster (the graph view's own groups) and freezes its propositions + chains into a self-contained `ProblemSet`. Three named-level axes steer generation (ladder rung 0–4, cue distance 0–2, Theory|Application — never a number to interpolate; defaults Direct exercise · Adjacent context · Theory). A partitioner agent covers every chain exactly once, one generator per partition writes problems against the owner's problem-engineering framework (commit-before-pose: statement + solution + rubric together), and grading is **strengthening, not assessment** — one attempt per problem with its own submit button, instant solution/rubric reveal, then a score 0–10 with brief reasoning landing live; nothing ever written back to the graph. Runs are registry-backed background jobs on the activity dock (no steer, never wakes the teacher); sets live in `data/practice`, crash-swept on boot.
**Dictation.** Optional speech-to-text: the server mints short-lived Deepgram tokens; audio streams browser→Deepgram over WebSocket, never through the server. **Dictation.** Optional speech-to-text: the server mints short-lived Deepgram tokens; audio streams browser→Deepgram over WebSocket, never through the server.
app/api/practice/sets/[id]/answers/route.ts
+3 -2
View File
@@ -2,8 +2,9 @@
* POST /api/practice/sets/[id]/answers — submit answers for grading * POST /api/practice/sets/[id]/answers — submit answers for grading
* (202; one grading job per answered problem runs in the background). * (202; one grading job per answered problem runs in the background).
* Body: {answers: [{problemId, text}]}. Set must be ready; answers must * Body: {answers: [{problemId, text}]}. Set must be ready; answers must
* be non-empty text on known, not-yet-graded problems (400/409 via the * be non-empty text on known, not-yet-answered problems (400/409 via the
* service's typed-error mapping). * service's typed-error mapping). ONE ATTEMPT (owner 2026-09-29): once a
* problem carries an answer it is done — re-submission is refused.
*/ */
import { submitPracticeAnswersResponse } from "@/src/server"; import { submitPracticeAnswersResponse } from "@/src/server";
components/practice-exam.tsx
+402 -246
View File
@@ -1,282 +1,438 @@
"use client"; "use client";
/** /**
* The exam page (practice slice 5): one problem set, numbered problems, * The exam page (practice slice 5, one-attempt redesign 2026-09-29):
* markdown statements, a textarea + dictation per problem (the bar * one problem set, numbered problems, markdown statements, a textarea +
* renders only on the FOCUSED box — thinking out loud is the primary * dictation per problem (the bar renders only on the FOCUSED box —
* answering mode), one submit for the set, and the post-grade reveal * thinking out loud is the primary answering mode). ONE ATTEMPT: each
* (verdict chip, feedback, model solution, rubric) under the box. * problem has its OWN submit button; once submitted the box locks and
* the MODEL SOLUTION reveals instantly (rubric collapsed by default).
* The grade (score 0–10 + brief reasoning) lands via the poll and
* updates live — no reload.
* *
* DISPLAY-SEAM MATH HEALING: statements/solutions/rubrics/feedback ride * TOP BAR (owner, UI round 2026-09-29): the chat pattern — an h-15
* lh-drag header in flow above the scroller, the sidebar reopen toggle
* first (it clears the traffic lights via .lh-closed-toggle), the set
* title truncating beside it. The levels + status stay in the body.
*
* DISPLAY-SEAM MATH HEALING: statements/solutions/rubrics/reasoning ride
* the same streaming-args parse as quiz/ask strings — restoreMathEscapes * the same streaming-args parse as quiz/ask strings — restoreMathEscapes
* heals them at render (the same helper the tool-part seam uses). * heals them at render (the same helper the tool-part seam uses).
* *
* POLLING: 2s while the set isn't ready (stubs → filled), and while a * POLLING: 2s while the set isn't ready (stubs → filled), and while a
* grading run is live for this set (activity feed's practice rows) with * grading run is live for this set (activity feed practice rows are
* answered-but-ungraded problems outstanding. A settled page goes quiet * keyed setId:grade:problemId — the LIVE rows name exactly which
* (a dropped grade terminates honestly: no live row → no poll). * problems' grades are pending). A settled page goes quiet; a dropped
* grade shows "grade interrupted" honestly.
*/ */
import { useCallback, useEffect, useRef, useState } from "react"; import { useCallback, useEffect, useRef, useState } from "react";
import type { GradeVerdict, Problem, ProblemSet } from "@/src/core"; import type { Problem, ProblemSet } from "@/src/core";
import { CUE_LEVELS, LADDER_RUNGS, PRESENTATIONS } from "@/src/core"; import { CUE_LEVELS, LADDER_RUNGS, PRESENTATIONS } from "@/src/core";
import { restoreMathEscapes } from "@/src/core"; import { restoreMathEscapes } from "@/src/core";
import { MessageResponse } from "@/components/ai-elements/message"; import { MessageResponse } from "@/components/ai-elements/message";
import { BrailleSpinner } from "@/components/braille-spinner"; import { BrailleSpinner } from "@/components/braille-spinner";
import { DictationBar, useDictation } from "@/components/dictation"; import { DictationBar, useDictation } from "@/components/dictation";
import { SidebarToggle } from "@/components/sidebar";
/** The verdict chip's tone — plain, no invented color language. */
export function verdictClass(verdict: GradeVerdict): string {
if (verdict === "solid") return "bg-green-500/15 text-green-500";
if (verdict === "partial") return "bg-amber-500/15 text-amber-600 dark:text-amber-400";
return "bg-destructive/15 text-destructive";
}
/** One answerable problem: textarea + focus-gated dictation bar. */ /** One answerable problem: textarea + focus-gated dictation bar. */
export function ProblemBox({ export function ProblemBox({
problem, problem,
draft, draft,
onDraft, onDraft,
}: { }: {
problem: Pick<Problem, "id">; problem: Pick<Problem, "id">;
draft: string; draft: string;
onDraft: (text: string) => void; onDraft: (text: string) => void;
}) { }) {
const [focused, setFocused] = useState(false); const [focused, setFocused] = useState(false);
// The dictation appends into THIS box only (per-box instance). // The dictation appends into THIS box only (per-box instance).
const dictation = useDictation({ const dictation = useDictation({
append: (text) => onDraft(text), append: (text) => onDraft(text),
}); });
return ( return (
<div className="flex flex-col gap-1.5" data-testid={`problem-box-${problem.id}`}> <div
<textarea className="flex flex-col gap-1.5"
data-testid={`answer-${problem.id}`} data-testid={`problem-box-${problem.id}`}
value={draft} >
rows={5} <textarea
placeholder="Think out loud — dictation or typing, both fine." data-testid={`answer-${problem.id}`}
onChange={(e) => onDraft(e.target.value)} value={draft}
onFocus={() => setFocused(true)} rows={5}
onBlur={() => setFocused(false)} placeholder="Think out loud — dictation or typing, both fine."
className="w-full resize-y rounded-lg border bg-background px-3 py-2 text-sm outline-none focus-visible:ring-1 focus-visible:ring-ring" onChange={(e) => onDraft(e.target.value)}
/> onFocus={() => setFocused(true)}
{focused && <DictationBar dictation={dictation} />} onBlur={() => setFocused(false)}
</div> className="w-full resize-y rounded-lg border bg-background px-3 py-2 text-sm outline-none focus-visible:ring-1 focus-visible:ring-ring"
); />
{focused && <DictationBar dictation={dictation} />}
</div>
);
} }
/** The exam page's pure body — exported for SSR pins. */ /** The exam page's pure body — exported for SSR pins. */
export function PracticeExamBody({ export function PracticeExamBody({
set, set,
drafts, drafts,
onDraft, onDraft,
onSubmit, onSubmit,
submitting, submittingId,
submitError, submitErrors,
grading, liveGradeIds,
}: { }: {
set: ProblemSet; set: ProblemSet;
drafts: Record<string, string>; drafts: Record<string, string>;
onDraft: (problemId: string, text: string) => void; onDraft: (problemId: string, text: string) => void;
onSubmit: () => void; /** Per-problem submit (owner 2026-09-29): ONE attempt per problem. */
submitting: boolean; onSubmit: (problemId: string) => void;
submitError: string | null; /** The problem whose submit POST is in flight. */
grading: boolean; submittingId: string | null;
/** Per-problem submit failures, keyed by problem id. */
submitErrors: Record<string, string>;
/** Problems whose grade rows are LIVE (the poll's activity feed). */
liveGradeIds: ReadonlySet<string>;
}) { }) {
const answerable = set.problems.filter((p) => !p.grade); return (
const anyDraft = answerable.some((p) => (drafts[p.id] ?? p.answer ?? "") !== ""); <div className="flex h-dvh">
return ( <div className="flex min-w-0 flex-1 flex-col">
<div className="h-dvh overflow-y-auto" data-testid="practice-exam-scroll"> {/* The top bar (owner, UI round 2026-09-29): the new-session
<div className="mx-auto flex w-full max-w-3xl flex-col gap-6 p-6" data-testid="practice-exam"> pattern — h-15, lh-drag (the Electron window-drag surface, so
<div className="flex flex-col gap-1"> the traffic lights never cover content), the sidebar reopen
<h1 className="font-title text-3xl tracking-tight">{set.title}</h1> toggle first, the set title truncating beside it. */}
<p className="text-xs text-muted-foreground/80" data-testid="practice-exam-levels"> <header className="lh-drag flex h-15 items-center gap-3 border-b border-border px-3">
{LADDER_RUNGS[set.params.ladderRung].name} · {CUE_LEVELS[set.params.cueDistance].name} ·{" "} <SidebarToggle />
{PRESENTATIONS[set.params.presentation].name} <div
</p> className="min-w-0 flex-1 truncate font-title text-3xl tracking-tight"
<p className="text-sm text-muted-foreground" data-testid="practice-exam-status"> data-testid="practice-exam-title"
{set.status === "partitioning" && "Partitioning the chains…"} >
{set.status === "generating" && `Generating — ${set.problems.filter((p) => p.status === "ready").length}/${set.problems.length} problems ready`} {set.title}
{set.status === "error" && (set.error ?? "Something went wrong")} </div>
{set.status === "ready" && </header>
(grading ? "Grading your answers…" : `${set.problems.length} problems · ${set.problems.filter((p) => p.grade).length} graded`)} <div className="relative flex min-h-0 flex-1">
</p> <div
</div> className="h-full min-w-0 flex-1 overflow-y-auto"
data-testid="practice-exam-scroll"
>
<div
className="mx-auto flex w-full max-w-3xl flex-col gap-6 p-6"
data-testid="practice-exam"
>
<div className="flex flex-col gap-1">
<p
className="text-xs text-muted-foreground/80"
data-testid="practice-exam-levels"
>
{LADDER_RUNGS[set.params.ladderRung].name} ·{" "}
{CUE_LEVELS[set.params.cueDistance].name} ·{" "}
{PRESENTATIONS[set.params.presentation].name}
</p>
<p
className="text-sm text-muted-foreground"
data-testid="practice-exam-status"
>
{set.status === "partitioning" && "Partitioning the chains…"}
{set.status === "generating" &&
`Generating — ${set.problems.filter((p) => p.status === "ready").length}/${set.problems.length} problems ready`}
{set.status === "error" &&
(set.error ?? "Something went wrong")}
{set.status === "ready" &&
(liveGradeIds.size > 0
? "Grading your answers…"
: `${set.problems.length} problems · ${set.problems.filter((p) => p.grade).length} graded`)}
</p>
</div>
{set.problems.map((problem, i) => ( {set.problems.map((problem, i) => (
<section key={problem.id} className="rounded-xl border bg-card p-4" data-testid={`problem-${problem.id}`}> <section
<h2 className="mb-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">Problem {i + 1}</h2> key={problem.id}
{problem.status === "generating" ? ( className="rounded-xl border bg-card p-4"
// The stub: propositions NEVER surface (never-announce law). data-testid={`problem-${problem.id}`}
<div className="flex items-center gap-2 text-sm text-muted-foreground" data-testid={`generating-${problem.id}`}> >
<BrailleSpinner className="size-4" /> <h2 className="mb-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">
Generating problem… Problem {i + 1}
</div> </h2>
) : problem.status === "error" ? ( {problem.status === "generating" ? (
<p className="text-sm text-destructive" data-testid={`problem-error-${problem.id}`}> // The stub: propositions NEVER surface (never-announce law).
This problem failed to generate{problem.error ? ` — ${problem.error}` : ""}. <div
</p> className="flex items-center gap-2 text-sm text-muted-foreground"
) : ( data-testid={`generating-${problem.id}`}
<> >
<div className="lh-prose text-sm"> <BrailleSpinner className="size-4" />
<MessageResponse mode="static">{restoreMathEscapes(problem.statement)}</MessageResponse> Generating problem…
</div> </div>
{problem.grade ? ( ) : problem.status === "error" ? (
// Graded: the answer read-only, the reveal below. <p
<div className="mt-4 flex flex-col gap-3"> className="text-sm text-destructive"
<textarea data-testid={`problem-error-${problem.id}`}
data-testid={`answer-${problem.id}`} >
value={problem.answer ?? ""} This problem failed to generate
readOnly {problem.error ? ` — ${problem.error}` : ""}.
rows={5} </p>
className="w-full resize-y rounded-lg border bg-muted/40 px-3 py-2 text-sm text-muted-foreground" ) : (
/> <>
<div className="rounded-lg border bg-background p-3" data-testid={`reveal-${problem.id}`}> <div className="lh-prose text-sm">
<div className="mb-2 flex items-center gap-2"> <MessageResponse mode="static">
<span className={`rounded px-1.5 py-0.5 text-xs font-medium uppercase ${verdictClass(problem.grade.verdict)}`}> {restoreMathEscapes(problem.statement)}
{problem.grade.verdict} </MessageResponse>
</span> </div>
</div> {problem.answer === undefined ? (
<div className="lh-prose text-sm" data-testid={`feedback-${problem.id}`}> // ANSWERABLE (owner 2026-09-29): the box + its OWN submit
<MessageResponse mode="static">{restoreMathEscapes(problem.grade.feedback)}</MessageResponse> // button — one attempt, submitted the moment it's clicked.
</div> <div className="flex flex-col items-start gap-2">
<div className="mt-3 text-xs uppercase tracking-wide text-muted-foreground">Model solution</div> <div className="w-full">
<div className="lh-prose text-sm" data-testid={`solution-${problem.id}`}> <ProblemBox
<MessageResponse mode="static">{restoreMathEscapes(problem.solution)}</MessageResponse> problem={problem}
</div> draft={drafts[problem.id] ?? ""}
<div className="mt-3 text-xs uppercase tracking-wide text-muted-foreground">Rubric</div> onDraft={(text) => onDraft(problem.id, text)}
<div className="lh-prose text-sm" data-testid={`rubric-${problem.id}`}> />
<MessageResponse mode="static">{restoreMathEscapes(problem.rubric)}</MessageResponse> </div>
</div> <button
</div> type="button"
</div> data-testid={`practice-submit-${problem.id}`}
) : ( onClick={() => onSubmit(problem.id)}
// Answerable: the box (graded problems lock above). disabled={
<div className="mt-4"> set.status !== "ready" ||
<ProblemBox problem={problem} draft={drafts[problem.id] ?? problem.answer ?? ""} onDraft={(text) => onDraft(problem.id, text)} /> (drafts[problem.id] ?? "").trim() === "" ||
</div> submittingId !== null
)} }
</> className="rounded-md bg-primary px-4 py-2 text-sm font-medium text-primary-foreground transition-colors hover:bg-primary/90 disabled:opacity-40"
)} >
</section> {submittingId === problem.id
))} ? "Submitting…"
: "Submit answer"}
{set.status === "ready" && answerable.length > 0 && ( </button>
<div className="flex flex-col items-start gap-2"> {submitErrors[problem.id] && (
<button <p
type="button" data-testid={`practice-submit-error-${problem.id}`}
data-testid="practice-submit" className="text-sm text-destructive"
onClick={onSubmit} >
disabled={!anyDraft || submitting} {submitErrors[problem.id]}
className="rounded-md bg-primary px-4 py-2 text-sm font-medium text-primary-foreground transition-colors hover:bg-primary/90 disabled:opacity-40" </p>
> )}
{submitting ? "Submitting…" : "Submit answers"} </div>
</button> ) : (
{submitError && ( // SUBMITTED (owner 2026-09-29): the answer locks, the
<p data-testid="practice-submit-error" className="text-sm text-destructive"> // model solution reveals INSTANTLY, the rubric stays
{submitError} // collapsed, and the grade lands live when it lands.
</p> <div className="mt-4 flex flex-col gap-3">
)} <textarea
</div> data-testid={`answer-${problem.id}`}
)} value={problem.answer}
</div> readOnly
</div> rows={5}
); className="w-full resize-y rounded-lg border bg-muted/40 px-3 py-2 text-sm text-muted-foreground"
/>
<div
className="rounded-lg border bg-background p-3"
data-testid={`reveal-${problem.id}`}
>
{problem.grade ? (
<div className="flex flex-col gap-1.5">
<div className="flex items-baseline gap-2">
<span
data-testid={`grade-${problem.id}`}
className="font-mono text-sm font-medium"
>
{problem.grade.score}/10
</span>
</div>
<div
className="lh-prose text-sm"
data-testid={`reasoning-${problem.id}`}
>
<MessageResponse mode="static">
{restoreMathEscapes(
problem.grade.reasoning,
)}
</MessageResponse>
</div>
</div>
) : liveGradeIds.has(problem.id) ? (
<div
className="flex items-center gap-2 text-sm text-muted-foreground"
data-testid={`grading-${problem.id}`}
>
<BrailleSpinner className="size-4" />
Grading…
</div>
) : (
<p
className="text-sm text-muted-foreground"
data-testid={`grade-interrupted-${problem.id}`}
>
Grade interrupted — the answer stands.
</p>
)}
<div className="mt-3 text-xs uppercase tracking-wide text-muted-foreground">
Model solution
</div>
<div
className="lh-prose text-sm"
data-testid={`solution-${problem.id}`}
>
<MessageResponse mode="static">
{restoreMathEscapes(problem.solution)}
</MessageResponse>
</div>
<details
className="mt-3"
data-testid={`rubric-${problem.id}`}
>
<summary className="cursor-pointer select-none text-xs uppercase tracking-wide text-muted-foreground">
Rubric
</summary>
<div className="mt-2 lh-prose text-sm">
<MessageResponse mode="static">
{restoreMathEscapes(problem.rubric)}
</MessageResponse>
</div>
</details>
</div>
</div>
)}
</>
)}
</section>
))}
</div>
</div>
</div>
</div>
</div>
);
} }
export function PracticeExam({ initial }: { initial: ProblemSet }) { export function PracticeExam({ initial }: { initial: ProblemSet }) {
const [set, setSet] = useState<ProblemSet>(initial); const [set, setSet] = useState<ProblemSet>(initial);
const [drafts, setDrafts] = useState<Record<string, string>>({}); const [drafts, setDrafts] = useState<Record<string, string>>({});
const [submitting, setSubmitting] = useState(false); const [submittingId, setSubmittingId] = useState<string | null>(null);
const [submitError, setSubmitError] = useState<string | null>(null); const [submitErrors, setSubmitErrors] = useState<Record<string, string>>({});
const [gradingLive, setGradingLive] = useState(false); // Which problems' grade rows are LIVE (activity feed) — drives the
const mounted = useRef(true); // per-problem "Grading…" spinners and the status line.
useEffect(() => { const [liveGradeIds, setLiveGradeIds] = useState<Set<string>>(new Set());
mounted.current = true; // True from a successful submit until the feed shows no live grade
return () => { // rows for this set — keeps the poll alive across refetches.
mounted.current = false; const [gradingLive, setGradingLive] = useState(false);
}; const mounted = useRef(true);
}, []); useEffect(() => {
mounted.current = true;
return () => {
mounted.current = false;
};
}, []);
const refetch = useCallback(async () => { const refetch = useCallback(async () => {
try { try {
const res = await fetch(`/api/practice/sets/${initial.id}`, { cache: "no-store" }); const res = await fetch(`/api/practice/sets/${initial.id}`, {
if (!res.ok) return; cache: "no-store",
const d = (await res.json()) as { set: ProblemSet }; });
if (mounted.current) setSet(d.set); if (!res.ok) return;
} catch { const d = (await res.json()) as { set: ProblemSet };
// transient poll failures keep the last snapshot if (mounted.current) setSet(d.set);
} } catch {
}, [initial.id]); // transient poll failures keep the last snapshot
}
}, [initial.id]);
// The poll: while not ready (generation), or while a grading run is // The poll: while not ready (generation), or while a grading run is
// live for this set with outstanding ungraded answers. The activity // live for this set with outstanding ungraded answers. The activity
// feed is the grading-liveness truth — a dropped grade (no live row) // feed's practice rows are ROW-KEYED (setId:grade:problemId) — the
// settles the page instead of polling forever. // live rows name exactly which grades are pending; a dropped grade
const pendingGrade = set.problems.some((p) => p.answer && !p.grade); // (no live row) settles the page instead of polling forever.
const needsPoll = set.status !== "ready" || (gradingLive && pendingGrade); const pendingGrade = set.problems.some(
useEffect(() => { (p) => p.answer !== undefined && !p.grade,
if (!needsPoll) return; );
let stopped = false; const needsPoll = set.status !== "ready" || (gradingLive && pendingGrade);
const tick = async () => { useEffect(() => {
await refetch(); if (!needsPoll) return;
try { let stopped = false;
const res = await fetch("/api/activity", { cache: "no-store" }); const tick = async () => {
if (!res.ok) return; await refetch();
const feed = (await res.json()) as { practice?: Array<{ id: string; kind: string }> }; try {
if (!stopped && mounted.current) { const res = await fetch("/api/activity", { cache: "no-store" });
setGradingLive((feed.practice ?? []).some((row) => row.id === initial.id)); if (!res.ok) return;
} const feed = (await res.json()) as {
} catch { practice?: Array<{ id: string; kind: string }>;
// keep the last liveness read };
} if (!stopped && mounted.current) {
}; const prefix = `${initial.id}:grade:`;
void tick(); const pids = (feed.practice ?? [])
const t = setInterval(() => void tick(), 2000); .filter((row) => row.id.startsWith(prefix))
return () => { .map((row) => row.id.slice(prefix.length));
stopped = true; setLiveGradeIds(new Set(pids));
clearInterval(t); setGradingLive(pids.length > 0);
}; }
}, [needsPoll, refetch, initial.id]); } catch {
// keep the last liveness read
}
};
void tick();
const t = setInterval(() => void tick(), 2000);
return () => {
stopped = true;
clearInterval(t);
};
}, [needsPoll, refetch, initial.id]);
const submit = useCallback(async () => { // ONE attempt per problem (owner 2026-09-29): the submit posts THAT
if (submitting) return; // problem's answer alone; the server refuses a second submission
const answers = set.problems // forever after. On success the box locks (answer persisted) and the
.filter((p) => !p.grade) // reveal shows instantly; the grade arrives via the poll.
.map((p) => ({ problemId: p.id, text: drafts[p.id] ?? p.answer ?? "" })) const submit = useCallback(
.filter((a) => a.text.trim() !== ""); async (problemId: string) => {
if (answers.length === 0) return; if (submittingId !== null) return;
setSubmitting(true); const p = set.problems.find((q) => q.id === problemId);
setSubmitError(null); if (!p || p.answer !== undefined) return; // done problems never resubmit
try { const text = drafts[problemId] ?? "";
const res = await fetch(`/api/practice/sets/${set.id}/answers`, { if (text.trim() === "") return;
method: "POST", setSubmittingId(problemId);
headers: { "content-type": "application/json" }, setSubmitErrors((prev) => {
body: JSON.stringify({ answers }), const next = { ...prev };
}); delete next[problemId];
if (!res.ok) { return next;
const body = (await res.json().catch(() => ({}))) as { error?: string }; });
setSubmitError(body.error ?? `Submit failed (${res.status})`); try {
setSubmitting(false); const res = await fetch(`/api/practice/sets/${set.id}/answers`, {
return; method: "POST",
} headers: { "content-type": "application/json" },
setSubmitting(false); body: JSON.stringify({ answers: [{ problemId, text }] }),
setGradingLive(true); // grades land via the poll });
await refetch(); if (!res.ok) {
} catch { const body = (await res.json().catch(() => ({}))) as {
setSubmitError("Submit failed — network error. Try again."); error?: string;
setSubmitting(false); };
} setSubmitErrors((prev) => ({
}, [submitting, set, drafts, refetch]); ...prev,
[problemId]: body.error ?? `Submit failed (${res.status})`,
}));
setSubmittingId(null);
return;
}
setSubmittingId(null);
setGradingLive(true); // the grade lands via the poll
await refetch();
} catch {
setSubmitErrors((prev) => ({
...prev,
[problemId]: "Submit failed — network error. Try again.",
}));
setSubmittingId(null);
}
},
[submittingId, set, drafts, refetch],
);
return ( return (
<PracticeExamBody <PracticeExamBody
set={set} set={set}
drafts={drafts} drafts={drafts}
onDraft={(problemId, text) => setDrafts((prev) => ({ ...prev, [problemId]: text }))} onDraft={(problemId, text) =>
onSubmit={() => void submit()} setDrafts((prev) => ({ ...prev, [problemId]: text }))
submitting={submitting} }
submitError={submitError} onSubmit={(problemId) => void submit(problemId)}
grading={gradingLive && pendingGrade} submittingId={submittingId}
/> submitErrors={submitErrors}
); liveGradeIds={liveGradeIds}
/>
);
} }
components/practice-list.tsx
+248 -198
View File
@@ -13,229 +13,279 @@ import { useCallback, useEffect, useState } from "react";
import { useRouter } from "next/navigation"; import { useRouter } from "next/navigation";
import { createPortal } from "react-dom"; import { createPortal } from "react-dom";
import { PlusIcon, TrashIcon } from "lucide-react"; import { PlusIcon, TrashIcon } from "lucide-react";
import { SidebarToggle } from "@/components/sidebar";
import type { ProblemSet } from "@/src/core"; import type { ProblemSet } from "@/src/core";
import { PracticeDialog } from "@/components/practice-dialog"; import { PracticeDialog } from "@/components/practice-dialog";
/** A set's status word — plain and honest, no invented progress. */ /** A set's status word — plain and honest, no invented progress. */
export function setStatusWord(set: Pick<ProblemSet, "status">): string { export function setStatusWord(set: Pick<ProblemSet, "status">): string {
return set.status === "partitioning" ? "partitioning" : set.status === "generating" ? "generating" : set.status; return set.status === "partitioning"
? "partitioning"
: set.status === "generating"
? "generating"
: set.status;
} }
/** The list's pure body — exported for SSR pins. Rows are buttons /** The list's pure body — exported for SSR pins. Rows are buttons
* (click-through); the delete affordance is a nested icon button. */ * (click-through); the delete affordance is a nested icon button. */
export function PracticeListBody({ export function PracticeListBody({
sets, sets,
onOpen, onOpen,
onDelete, onDelete,
onNew, onNew,
}: { }: {
sets: ProblemSet[]; sets: ProblemSet[];
onOpen: (id: string) => void; onOpen: (id: string) => void;
onDelete: (set: ProblemSet) => void; onDelete: (set: ProblemSet) => void;
onNew: () => void; onNew: () => void;
}) { }) {
return ( return (
<div className="h-dvh overflow-y-auto" data-testid="practice-list-scroll"> <div className="flex h-dvh">
<div className="mx-auto flex w-full max-w-3xl flex-col gap-3 p-6" data-testid="practice-list"> <div className="flex min-w-0 flex-1 flex-col">
<div className="flex items-center justify-between"> {/* The top bar (owner, UI round 2026-09-29): the new-session
<h1 className="font-title text-3xl tracking-tight">Practice</h1> pattern — h-15, lh-drag (the Electron window-drag surface),
<button the sidebar reopen toggle first, "Practice" beside it, the
type="button" page's one action at the right edge (lh-no-drag — it must
data-testid="practice-new" stay clickable inside the drag band). */}
onClick={onNew} <header className="lh-drag flex h-15 items-center gap-3 border-b border-border px-3">
className="flex items-center gap-1.5 rounded-md border px-3 py-1.5 text-sm transition-colors hover:bg-accent" <SidebarToggle />
> <div className="min-w-0 flex-1 truncate font-title text-3xl tracking-tight">
<PlusIcon className="size-3.5" /> Practice
New problem set </div>
</button> <button
</div> type="button"
data-testid="practice-new"
onClick={onNew}
className="lh-no-drag flex shrink-0 items-center gap-1.5 rounded-md border px-3 py-1.5 text-sm transition-colors hover:bg-accent"
>
<PlusIcon className="size-3.5" />
New problem set
</button>
</header>
<div className="relative flex min-h-0 flex-1">
<div
className="h-full min-w-0 flex-1 overflow-y-auto"
data-testid="practice-list-scroll"
>
<div
className="mx-auto flex w-full max-w-3xl flex-col gap-3 p-6"
data-testid="practice-list"
>
{sets.length === 0 && (
<div
className="rounded-xl border bg-card p-6 text-sm text-muted-foreground"
data-testid="practice-empty"
>
No problem sets yet — create one from a session.
</div>
)}
{sets.length === 0 && ( <div className="flex flex-col gap-2">
<div className="rounded-xl border bg-card p-6 text-sm text-muted-foreground" data-testid="practice-empty"> {sets.map((set) => {
No problem sets yet — create one from a session. const ready = set.problems.filter(
</div> (p) => p.status === "ready",
)} ).length;
const graded = set.problems.filter((p) => p.grade).length;
<div className="flex flex-col gap-2"> return (
{sets.map((set) => { <div key={set.id} className="flex items-stretch gap-1">
const ready = set.problems.filter((p) => p.status === "ready").length; <button
const graded = set.problems.filter((p) => p.grade).length; type="button"
return ( data-testid={`practice-row-${set.id}`}
<div key={set.id} className="flex items-stretch gap-1"> onClick={() => onOpen(set.id)}
<button className="flex flex-1 items-center justify-between gap-4 rounded-xl border bg-card px-4 py-3 text-left transition-colors hover:bg-accent/50"
type="button" >
data-testid={`practice-row-${set.id}`} <span className="min-w-0">
onClick={() => onOpen(set.id)} <span className="block truncate font-medium">
className="flex flex-1 items-center justify-between gap-4 rounded-xl border bg-card px-4 py-3 text-left transition-colors hover:bg-accent/50" {set.title}
> </span>
<span className="min-w-0"> <span className="block text-xs text-muted-foreground">
<span className="block truncate font-medium">{set.title}</span> {set.createdAt.slice(0, 10)} ·{" "}
<span className="block text-xs text-muted-foreground"> {set.problems.length > 0
{set.createdAt.slice(0, 10)} · {set.problems.length > 0 ? `${ready}/${set.problems.length} problems` : "no problems yet"} ? `${ready}/${set.problems.length} problems`
{graded > 0 ? ` · ${graded} graded` : ""} : "no problems yet"}
</span> {graded > 0 ? ` · ${graded} graded` : ""}
</span> </span>
<span </span>
data-testid={`practice-status-${set.id}`} <span
className={`shrink-0 text-xs uppercase tracking-wide ${ data-testid={`practice-status-${set.id}`}
set.status === "error" ? "text-destructive" : "text-muted-foreground" className={`shrink-0 text-xs uppercase tracking-wide ${
}`} set.status === "error"
> ? "text-destructive"
{setStatusWord(set)} : "text-muted-foreground"
</span> }`}
</button> >
<button {setStatusWord(set)}
type="button" </span>
data-testid={`practice-delete-${set.id}`} </button>
aria-label={`Delete ${set.title}`} <button
onClick={() => onDelete(set)} type="button"
className="flex items-center rounded-xl border px-3 text-muted-foreground/60 transition-colors hover:bg-accent hover:text-foreground" data-testid={`practice-delete-${set.id}`}
> aria-label={`Delete ${set.title}`}
<TrashIcon className="size-4" /> onClick={() => onDelete(set)}
</button> className="flex items-center rounded-xl border px-3 text-muted-foreground/60 transition-colors hover:bg-accent hover:text-foreground"
</div> >
); <TrashIcon className="size-4" />
})} </button>
</div> </div>
</div> );
</div> })}
); </div>
</div>
</div>
</div>
</div>
</div>
);
} }
/** The delete confirm — the delete-session dialog's shape (no mode /** The delete confirm — the delete-session dialog's shape (no mode
* checkbox: a set has no preserve-graph question). Pure body + portal * checkbox: a set has no preserve-graph question). Pure body + portal
* wrapper. */ * wrapper. */
export function DeleteSetBody({ export function DeleteSetBody({
title, title,
busy, busy,
failure, failure,
onClose, onClose,
onRun, onRun,
}: { }: {
title: string; title: string;
busy: boolean; busy: boolean;
failure: string | null; failure: string | null;
onClose: () => void; onClose: () => void;
onRun: () => void; onRun: () => void;
}) { }) {
return ( return (
<div <div
role="dialog" role="dialog"
aria-modal="true" aria-modal="true"
aria-label="Delete problem set" aria-label="Delete problem set"
data-testid="delete-set-dialog" data-testid="delete-set-dialog"
onClick={(e) => e.stopPropagation()} onClick={(e) => e.stopPropagation()}
className="w-[32rem] max-w-[calc(100vw-2rem)] rounded-xl border bg-card p-5 shadow-xl" className="w-[32rem] max-w-[calc(100vw-2rem)] rounded-xl border bg-card p-5 shadow-xl"
> >
<h2 className="font-title text-2xl tracking-tight text-destructive">Delete problem set?</h2> <h2 className="font-title text-2xl tracking-tight text-destructive">
<p className="mt-2 font-mono text-sm">{title}</p> Delete problem set?
<p className="mt-3 text-sm text-muted-foreground"> </h2>
The set and its problems are removed. Nothing in the graph or your sessions changes. <p className="mt-2 font-mono text-sm">{title}</p>
</p> <p className="mt-3 text-sm text-muted-foreground">
{failure && ( The set and its problems are removed. Nothing in the graph or your
<p data-testid="delete-set-error" className="mt-3 text-sm text-destructive"> sessions changes.
{failure} </p>
</p> {failure && (
)} <p
<div className="mt-5 flex justify-end gap-2"> data-testid="delete-set-error"
<button className="mt-3 text-sm text-destructive"
type="button" >
onClick={onClose} {failure}
disabled={busy} </p>
className="rounded-md px-3 py-1.5 text-sm text-muted-foreground transition-colors hover:bg-accent hover:text-foreground" )}
> <div className="mt-5 flex justify-end gap-2">
Cancel <button
</button> type="button"
<button onClick={onClose}
type="button" disabled={busy}
data-testid="delete-set-run" className="rounded-md px-3 py-1.5 text-sm text-muted-foreground transition-colors hover:bg-accent hover:text-foreground"
onClick={onRun} >
disabled={busy} Cancel
className="rounded-md bg-destructive px-3 py-1.5 text-sm font-medium text-destructive-foreground transition-colors hover:bg-destructive/90 disabled:opacity-40" </button>
> <button
{busy ? "Deleting…" : "Delete"} type="button"
</button> data-testid="delete-set-run"
</div> onClick={onRun}
</div> disabled={busy}
); className="rounded-md bg-destructive px-3 py-1.5 text-sm font-medium text-destructive-foreground transition-colors hover:bg-destructive/90 disabled:opacity-40"
>
{busy ? "Deleting…" : "Delete"}
</button>
</div>
</div>
);
} }
export function PracticeListPage({ initialSets }: { initialSets: ProblemSet[] }) { export function PracticeListPage({
const router = useRouter(); initialSets,
const [sets, setSets] = useState<ProblemSet[]>(initialSets); }: {
const [creating, setCreating] = useState(false); initialSets: ProblemSet[];
const [deleting, setDeleting] = useState<ProblemSet | null>(null); }) {
const [deleteBusy, setDeleteBusy] = useState(false); const router = useRouter();
const [deleteFailure, setDeleteFailure] = useState<string | null>(null); const [sets, setSets] = useState<ProblemSet[]>(initialSets);
const [creating, setCreating] = useState(false);
const [deleting, setDeleting] = useState<ProblemSet | null>(null);
const [deleteBusy, setDeleteBusy] = useState(false);
const [deleteFailure, setDeleteFailure] = useState<string | null>(null);
const refresh = useCallback(async () => { const refresh = useCallback(async () => {
try { try {
const res = await fetch("/api/practice/sets", { cache: "no-store" }); const res = await fetch("/api/practice/sets", { cache: "no-store" });
if (!res.ok) return; if (!res.ok) return;
const d = (await res.json()) as { sets: ProblemSet[] }; const d = (await res.json()) as { sets: ProblemSet[] };
setSets(d.sets); setSets(d.sets);
} catch { } catch {
// a transient failure on the poll keeps the last snapshot // a transient failure on the poll keeps the last snapshot
} }
}, []); }, []);
// The reveal poll: while any set is partitioning/generating, statuses // The reveal poll: while any set is partitioning/generating, statuses
// live-update; a settled list goes quiet. // live-update; a settled list goes quiet.
const anyRunning = sets.some((s) => s.status === "partitioning" || s.status === "generating"); const anyRunning = sets.some(
useEffect(() => { (s) => s.status === "partitioning" || s.status === "generating",
if (!anyRunning) return; );
const t = setInterval(() => void refresh(), 2000); useEffect(() => {
return () => clearInterval(t); if (!anyRunning) return;
}, [anyRunning, refresh]); const t = setInterval(() => void refresh(), 2000);
return () => clearInterval(t);
}, [anyRunning, refresh]);
const runDelete = useCallback(async () => { const runDelete = useCallback(async () => {
if (!deleting || deleteBusy) return; if (!deleting || deleteBusy) return;
setDeleteBusy(true); setDeleteBusy(true);
setDeleteFailure(null); setDeleteFailure(null);
try { try {
const res = await fetch(`/api/practice/sets/${deleting.id}`, { method: "DELETE" }); const res = await fetch(`/api/practice/sets/${deleting.id}`, {
if (!res.ok) { method: "DELETE",
const body = (await res.json().catch(() => ({}))) as { error?: string }; });
setDeleteFailure(body.error ?? `Delete failed (${res.status})`); if (!res.ok) {
setDeleteBusy(false); const body = (await res.json().catch(() => ({}))) as { error?: string };
return; setDeleteFailure(body.error ?? `Delete failed (${res.status})`);
} setDeleteBusy(false);
setDeleting(null); return;
setDeleteBusy(false); }
await refresh(); setDeleting(null);
} catch { setDeleteBusy(false);
setDeleteFailure("Delete failed — network error. Try again."); await refresh();
setDeleteBusy(false); } catch {
} setDeleteFailure("Delete failed — network error. Try again.");
}, [deleting, deleteBusy, refresh]); setDeleteBusy(false);
}
}, [deleting, deleteBusy, refresh]);
return ( return (
<> <>
<PracticeListBody <PracticeListBody
sets={sets} sets={sets}
onOpen={(id) => router.push(`/practice/${id}`)} onOpen={(id) => router.push(`/practice/${id}`)}
onDelete={(set) => setDeleting(set)} onDelete={(set) => setDeleting(set)}
onNew={() => setCreating(true)} onNew={() => setCreating(true)}
/> />
{creating && <PracticeDialog onClose={() => setCreating(false)} />} {creating && <PracticeDialog onClose={() => setCreating(false)} />}
{deleting && {deleting &&
createPortal( createPortal(
<div <div
className="fixed inset-0 z-50 flex items-center justify-center bg-background/80" className="fixed inset-0 z-50 flex items-center justify-center bg-background/80"
onClick={() => { onClick={() => {
if (!deleteBusy) setDeleting(null); if (!deleteBusy) setDeleting(null);
}} }}
> >
<DeleteSetBody <DeleteSetBody
title={deleting.title} title={deleting.title}
busy={deleteBusy} busy={deleteBusy}
failure={deleteFailure} failure={deleteFailure}
onClose={() => setDeleting(null)} onClose={() => setDeleting(null)}
onRun={() => void runDelete()} onRun={() => void runDelete()}
/> />
</div>, </div>,
document.body, document.body,
)} )}
</> </>
); );
} }
docs/architecture.md
+6 -6
View File
@@ -152,7 +152,7 @@ flowchart TB
- `contracts/jobs.ts` — the ack-and-steer protocol: job records, steer message builders, deterministic steer ids, usage accumulation - `contracts/jobs.ts` — the ack-and-steer protocol: job records, steer message builders, deterministic steer ids, usage accumulation
- `contracts/probe.ts` — probe assessment + the shared zod schema - `contracts/probe.ts` — probe assessment + the shared zod schema
- `contracts/models.ts` — client-safe model-catalog shapes - `contracts/models.ts` — client-safe model-catalog shapes
- `contracts/practice.ts` — problem sets, reasoning chains, params, grade verdicts - `contracts/practice.ts` — problem sets, reasoning chains, params, grade scores
--- ---
@@ -234,7 +234,7 @@ flowchart TB
end end
subgraph PRAC["PRACTICE — problem sets (owns data/practice)"] subgraph PRAC["PRACTICE — problem sets (owns data/practice)"]
PPART["practice pipeline — partitioner → one generator per problem · one grader per answer<br/>(feedback-only; never steers the teacher)"] PPART["practice pipeline — partitioner → one generator per problem · one grader per answer<br/>(score-only; never steers the teacher)"]
PDATA["problem sets — data/practice/*.json"] PDATA["problem sets — data/practice/*.json"]
end end
@@ -368,7 +368,7 @@ flowchart TB
## 7. Practice — strengthening, not assessment (owner ruling 2026-09-29) ## 7. Practice — strengthening, not assessment (owner ruling 2026-09-29)
Practice turns the graph into problem sets the learner reasons through out loud. The purpose ruling first: this is **strengthening, not assessment**. Grades are feedback for the learner — a verdict, an explanation, a look at the model solution — and nothing else. Grading writes nothing to the graph, no understanding levels exist anywhere, and the vision doc's exam machinery (scores, streaks, spaced repetition over failure rates) stays parked. A set exercises what the learner already met; it certifies nothing. Practice turns the graph into problem sets the learner reasons through out loud. The purpose ruling first: this is **strengthening, not assessment**. Grades are feedback for the learner — a score, a brief justification, a look at the model solution — and nothing else. Grading writes nothing to the graph, no understanding levels exist anywhere, and the vision doc's exam machinery (scores, streaks, spaced repetition over failure rates) stays parked. A set exercises what the learner already met; it certifies nothing.
**The chain model.** Practice sees a node selection as **reasoning chains**: the selection's `follows`-edges grouped by conclusion (`chainsForSelection`, `src/core/practice.ts`). A chain is `premises → conclusion` — one per conclusion node, a singleton for an edge-less node; premise-only nodes belong to no chain. The grouping is a **bijection with the follows-edge set** (`chainsToGraph` is its exact inverse, round-trip test-pinned) — no chain exists that isn't the edges, no edge that isn't in a chain. **Supports edges are invisible to practice** (owner ruling: supports are grounding, not reasoning steps). The partition must cover every chain **exactly once** — a true partition; duplicates fail validation exactly like gaps (owner amendment 2026-09-29: variety accumulates ACROSS sets, never within one — the at-least-once doctrine is dead). `chainCoverage` is the soundness check: only `missing` fails a partition, `duplicated` is informational. **The chain model.** Practice sees a node selection as **reasoning chains**: the selection's `follows`-edges grouped by conclusion (`chainsForSelection`, `src/core/practice.ts`). A chain is `premises → conclusion` — one per conclusion node, a singleton for an edge-less node; premise-only nodes belong to no chain. The grouping is a **bijection with the follows-edge set** (`chainsToGraph` is its exact inverse, round-trip test-pinned) — no chain exists that isn't the edges, no edge that isn't in a chain. **Supports edges are invisible to practice** (owner ruling: supports are grounding, not reasoning steps). The partition must cover every chain **exactly once** — a true partition; duplicates fail validation exactly like gaps (owner amendment 2026-09-29: variety accumulates ACROSS sets, never within one — the at-least-once doctrine is dead). `chainCoverage` is the soundness check: only `missing` fails a partition, `duplicated` is informational.
@@ -378,11 +378,11 @@ Practice turns the graph into problem sets the learner reasons through out loud.
**Two-stage generation.** One **partitioner** agent reads the chains plus the three NAMED-LEVEL parameters and submits the true partition; each partition becomes one problem's target. Then **one generator agent per problem**: its prompt carries the problem-engineering framework verbatim (the owner's doctrine — the three distances, the search-to-solution axis, the rung ladder, reverse/discriminate/probe-the-premises, the standing prohibitions, the formatting doctrine — full Markdown, LaTeX math, fenced code, tables/lists; prose-only renderings of structured content are a defect), **YOUR TARGET** (the assigned chains echoed verbatim — the echo law: the submit's chain set must equal the assignment exactly), and **THE FULL SET** (the whole selection: outside propositions are supporting material the generator may draw on with minimal use and maximal focus; the never-announce law covers ALL propositions). The submit is commit-before-pose — statement, solution, and rubric arrive together (the same sycophancy mitigation the quiz/explain mechanic uses); a problem never exists without its pre-written answer. Generators run in parallel (a process-wide 3-permit semaphore shared with grading) with per-problem failure isolation: one bad problem errors alone; the set still reaches `ready`. **Dedup across sets**: each generator also sees the problems of the ≤5 most recent sets sharing a node, so situations don't repeat between sets; within a set, parallel generators can't see each other — accepted for V1 (owner ruling). **Two-stage generation.** One **partitioner** agent reads the chains plus the three NAMED-LEVEL parameters and submits the true partition; each partition becomes one problem's target. Then **one generator agent per problem**: its prompt carries the problem-engineering framework verbatim (the owner's doctrine — the three distances, the search-to-solution axis, the rung ladder, reverse/discriminate/probe-the-premises, the standing prohibitions, the formatting doctrine — full Markdown, LaTeX math, fenced code, tables/lists; prose-only renderings of structured content are a defect), **YOUR TARGET** (the assigned chains echoed verbatim — the echo law: the submit's chain set must equal the assignment exactly), and **THE FULL SET** (the whole selection: outside propositions are supporting material the generator may draw on with minimal use and maximal focus; the never-announce law covers ALL propositions). The submit is commit-before-pose — statement, solution, and rubric arrive together (the same sycophancy mitigation the quiz/explain mechanic uses); a problem never exists without its pre-written answer. Generators run in parallel (a process-wide 3-permit semaphore shared with grading) with per-problem failure isolation: one bad problem errors alone; the set still reaches `ready`. **Dedup across sets**: each generator also sees the problems of the ≤5 most recent sets sharing a node, so situations don't repeat between sets; within a set, parallel generators can't see each other — accepted for V1 (owner ruling).
**Grading is feedback-only.** Submitting answers fires one **grader** job per answered problem: the grader sees the problem, the propositions its chains touch, the pre-written solution and rubric, and the learner's answer — never the full set (minimal, by ruling). It returns a verdict (`solid`/`partial`/`missed`) and feedback that names the failed recall and teaches the boundary. The verdict lands on the problem and the UI reveals it with the solution and rubric; the answer locks. Nothing reaches the graph, no level, no count, anywhere. A grader failure leaves the answer gradeless and re-submittable. **Grading is score-only (owner redesign, 2026-09-29).** ONE ATTEMPT: each problem has its own submit button; once submitted the answer persists and the problem is done — the route refuses re-submission forever after. The submit reveals INSTANTLY (no waiting on the grader): the answer locks read-only, the model solution shows, the rubric sits collapsed. Grading fires one **grader** job per answered problem — the grader sees the problem, the propositions its chains touch, the pre-written solution and rubric, and the learner's answer, never the full set (minimal, by ruling) — and returns an integer **score 0–10 plus one or two sentences of reasoning** (which rubric points were met or missed, which proposition failed to come to mind). The score lands on the problem via the page's 2s poll and updates live; grades never serialize set-wide (submitting B while A grades is fine — the shared pool bounds parallelism). Nothing reaches the graph, no level, no count, anywhere. A dropped grade leaves the answer standing with an honest “grade interrupted” line — one attempt, no re-submission.
**The jobs pattern — registry, not JobRecords.** Practice runs are NOT Workbench jobs: no `JobStore` records, no steer messages, nothing ever wakes the teacher — a practice run isn't chat-scoped (a set outlives its birth session and has no lesson to re-enter; its result is itself, persisted in its own store and read by its own page). What it shares with the Workbench is the observability surface: a `globalThis` run registry (the module-instance idiom — `__lhPracticeRuns`) with PER-PROBLEM ROWS (owner ruling, 2026-09-29 — matching the extraction per-chunk pattern): `setId:partition` while the partitioner runs ("Partitioning — title"), then one `setId:gen:problemId` per problem ("queued"→"generating", label "Problem i/N — title"), and one `setId:grade:problemId` per graded answer ("queued"→"grading", "Grading i/N — title") — each row its OWN abort controller and its OWN usage accumulation; the row key IS the feed id (the dock dup-key law) and the abort target. ABORT ROUTES BY ROW SHAPE: a gen-row abort kills THAT problem only (siblings continue; the set still settles `ready` with one errored problem), a partition-row abort ends the whole run, a grade-row abort drops that grade (answer kept, re-submittable); set-level abort (delete) composes from row aborts plus the synchronous persisted marks. Model calls — partitioner included — run under the SHARED background-LLM semaphore (`bg-llm-semaphore.ts`): one process-wide pool of 3 permits for ALL background LLM jobs, extraction chunks included (total background LLM concurrency bounded at 3, whoever calls). A boot sweep marks orphaned `partitioning`/`generating` sets as `error "Interrupted (server restarted)"` — once per process, and only when the set's `updatedAt` heartbeat is >120s stale (a live run saves at every transition and is structurally never stamped; the guard trio closed a live-stamping bug from the first live test) — the job runner's crash-honesty at set grain. Store writes serialize behind a per-set mutex; the pipeline is the sole writer of `data/practice`. **The jobs pattern — registry, not JobRecords.** Practice runs are NOT Workbench jobs: no `JobStore` records, no steer messages, nothing ever wakes the teacher — a practice run isn't chat-scoped (a set outlives its birth session and has no lesson to re-enter; its result is itself, persisted in its own store and read by its own page). What it shares with the Workbench is the observability surface: a `globalThis` run registry (the module-instance idiom — `__lhPracticeRuns`) with PER-PROBLEM ROWS (owner ruling, 2026-09-29 — matching the extraction per-chunk pattern): `setId:partition` while the partitioner runs ("Partitioning — title"), then one `setId:gen:problemId` per problem ("queued"→"generating", label "Problem i/N — title"), and one `setId:grade:problemId` per graded answer ("queued"→"grading", "Grading i/N — title") — each row its OWN abort controller and its OWN usage accumulation; the row key IS the feed id (the dock dup-key law) and the abort target. ABORT ROUTES BY ROW SHAPE: a gen-row abort kills THAT problem only (siblings continue; the set still settles `ready` with one errored problem), a partition-row abort ends the whole run, a grade-row abort drops that grade (answer kept — one attempt, the problem stays done); set-level abort (delete) composes from row aborts plus the synchronous persisted marks. Model calls — partitioner included — run under the SHARED background-LLM semaphore (`bg-llm-semaphore.ts`): one process-wide pool of 3 permits for ALL background LLM jobs, extraction chunks included (total background LLM concurrency bounded at 3, whoever calls). A boot sweep marks orphaned `partitioning`/`generating` sets as `error "Interrupted (server restarted)"` — once per process, and only when the set's `updatedAt` heartbeat is >120s stale (a live run's 30s FILE watchdog keeps the heartbeat fresh even across dev's per-worker processes — the registry is not shared between them; the guard trio closed a live-stamping bug from the first live test) — the job runner's crash-honesty at set grain. Store writes serialize behind a per-set mutex; the pipeline is the sole writer of `data/practice`.
**API + UI.** Creation is GRAPH-DOMAIN, not chat-domain (owner ruling 2026-09-29: "problem sets always take in nodes — graph objects"): `POST /api/practice/sets` takes `{nodeIds, params, title?}` ONLY — no chatId, no clusterId; the UI resolves a picked cluster to its member node ids and passes the cluster's label as the title. `GET /api/graph/clusters` serves the picker: the birth-session partition itself (`birthSessionPartition`, same ordering — size desc, ties by id) — EVERY group regardless of size (owner ruling, same evening: MIN_CLUSTER_SIZE is a hull-drawing rule; practice availability must not inherit a visual threshold), each as `{key, kind: "chat"|"notes", label, nodeIds, nodeCount}`; a cluster whose naming record is gone keeps `label: null` — honest, never guessed. 202 `{id}` with generation fire-and-forget; 400 empty selection (empty intersection with live nodes — the vanished-nodes edge) / bad params; 503 model/key refusal. `GET /api/practice/sets` · `GET/DELETE /api/practice/sets/[id]` (delete aborts live runs first) · `POST /api/practice/sets/[id]/answers` (202; 409 when not ready). Surfaces: `/practice` (the list — status/counts rows, delete confirm, the new-set dialog with the named-level controls (a 5-detent ladder slider, a 3-detent cue slider, a Theory|Application toggle — each showing the level's name + definition) and the cluster picker — the ONLY creation entry for now; the graph-view cluster entry comes later with the free-form selection UI) and `/practice/[id]` (the exam page — a levels line in the header, numbered problems, markdown statements with math healing at the display seam, a textarea + focus-gated dictation bar per problem, one Submit per set, the post-grade reveal). Entry: the sidebar's Practice nav row. **API + UI.** Creation is GRAPH-DOMAIN, not chat-domain (owner ruling 2026-09-29: "problem sets always take in nodes — graph objects"): `POST /api/practice/sets` takes `{nodeIds, params, title?}` ONLY — no chatId, no clusterId; the UI resolves a picked cluster to its member node ids and passes the cluster's label as the title. `GET /api/graph/clusters` serves the picker: the birth-session partition itself (`birthSessionPartition`, same ordering — size desc, ties by id) — EVERY group regardless of size (owner ruling, same evening: MIN_CLUSTER_SIZE is a hull-drawing rule; practice availability must not inherit a visual threshold), each as `{key, kind: "chat"|"notes", label, nodeIds, nodeCount}`; a cluster whose naming record is gone keeps `label: null` — honest, never guessed. 202 `{id}` with generation fire-and-forget; 400 empty selection (empty intersection with live nodes — the vanished-nodes edge) / bad params; 503 model/key refusal. `GET /api/practice/sets` · `GET/DELETE /api/practice/sets/[id]` (delete aborts live runs first) · `POST /api/practice/sets/[id]/answers` (202; 409 when not ready; 400 re-answering a submitted problem — one attempt). Surfaces: `/practice` (the list — the chat-pattern top bar with the sidebar reopen toggle and the New action at the right edge, status/counts rows, delete confirm, the new-set dialog with the named-level controls (a 5-detent ladder slider, a 3-detent cue slider, a Theory|Application toggle — each showing the level's name + definition) and the cluster picker — the ONLY creation entry for now; the graph-view cluster entry comes later with the free-form selection UI) and `/practice/[id]` (the exam page — the chat-pattern top bar, a levels + status line, numbered problems, markdown statements with math healing at the display seam, a textarea + focus-gated dictation bar per problem with its OWN Submit button — one attempt per problem, instant solution/rubric reveal, the score landing live). Entry: the sidebar's Practice nav row.
**Model wiring.** Three roles — `partitioner`, `problemGenerator`, `grader` — all pinned to the **strong** slot (quality-critical, one settings edit from cheap) and in `BACKGROUND_MODEL_ROLES` (per-role overrides). The stage agents are single-submit loops on the shared `runSubmitAgent` seam (`submit-agent.ts`, extracted from extraction for exactly this): a `validate` hook turns a bad submit into a tool-input error the model retries against, within `maxTurns 4`; exhaustion fails the stage honestly (the set says so, the chains re-run cleanly on a new set). **Model wiring.** Three roles — `partitioner`, `problemGenerator`, `grader` — all pinned to the **strong** slot (quality-critical, one settings edit from cheap) and in `BACKGROUND_MODEL_ROLES` (per-role overrides). The stage agents are single-submit loops on the shared `runSubmitAgent` seam (`submit-agent.ts`, extracted from extraction for exactly this): a `validate` hook turns a bad submit into a tool-input error the model retries against, within `maxTurns 4`; exhaustion fails the stage honestly (the set says so, the chains re-run cleanly on a new set).
next-env.d.ts
Vendored
+2 -2
View File
@@ -1,7 +1,7 @@
/// <reference types="next" /> /// <reference types="next" />
/// <reference types="next/image-types/global" /> /// <reference types="next/image-types/global" />
import "./.next/types/routes.d.ts"; import "./.next/dev/types/routes.d.ts";
import "./.next/types/root-params.d.ts"; import "./.next/dev/types/root-params.d.ts";
// NOTE: This file should not be edited // NOTE: This file should not be edited
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information. // see https://nextjs.org/docs/app/api-reference/config/typescript for more information.
src/agent/subagents/practice-pipeline.ts
+16 -9
View File
@@ -34,10 +34,12 @@
* background failure paths settle the persisted record (never an * background failure paths settle the persisted record (never an
* unhandled rejection); aborts mark "Aborted". * unhandled rejection); aborts mark "Aborted".
* *
* GRADING FAILURE DISCIPLINE: an aborted or failed grade leaves the * ONE ATTEMPT PER PROBLEM (owner ruling 2026-09-29): a problem accepts
* answer in place, the grade absent — the problem is re-submittable * a single submission — once `answer` is set the problem is done, the
* later (a new grading job). A second submitAnswers while grading is in * route refuses re-submission. Grades are per-problem: submitting
* flight is rejected (single live grading run per set). * another problem while one grades is fine (the shared LLM pool bounds
* parallelism); only the SAME problem's live grade row blocks (which
* the one-attempt rule already prevents client-side).
*/ */
import type { GraphStore, PracticeStore } from "../../core/ports.ts"; import type { GraphStore, PracticeStore } from "../../core/ports.ts";
@@ -476,8 +478,10 @@ export function createPracticePipeline(deps: PracticePipelineDeps) {
answers: Array<{ problemId: string; text: string }>, answers: Array<{ problemId: string; text: string }>,
): Promise<{ set: ProblemSet; done: Promise<void> }> { ): Promise<{ set: ProblemSet; done: Promise<void> }> {
agents.refuseIfStagesUnready(); // grading needs its engine — refuse BEFORE saving answers agents.refuseIfStagesUnready(); // grading needs its engine — refuse BEFORE saving answers
for (const key of practiceRuns().keys()) { for (const a of answers) {
if (key.startsWith(`${setId}:grade:`)) throw new BadAnswersError("a grading run is already in flight for this set"); if (practiceRuns().has(`${setId}:grade:${(a as { problemId?: unknown }).problemId}`)) {
throw new BadAnswersError("this problem's grade is already in flight");
}
} }
const set = await withSetLock(setId, () => { const set = await withSetLock(setId, () => {
const cur = deps.practiceStore.getSet(setId); const cur = deps.practiceStore.getSet(setId);
@@ -488,7 +492,9 @@ export function createPracticePipeline(deps: PracticePipelineDeps) {
if (typeof a.problemId !== "string" || !ids.has(a.problemId)) throw new BadAnswersError(`unknown problem id`); if (typeof a.problemId !== "string" || !ids.has(a.problemId)) throw new BadAnswersError(`unknown problem id`);
if (typeof a.text !== "string" || a.text.trim() === "") throw new BadAnswersError("answers must be non-empty text"); if (typeof a.text !== "string" || a.text.trim() === "") throw new BadAnswersError("answers must be non-empty text");
const p = cur.problems.find((q) => q.id === a.problemId)!; const p = cur.problems.find((q) => q.id === a.problemId)!;
if (p.grade) throw new BadAnswersError(`problem ${a.problemId} is already graded`); // ONE attempt (owner 2026-09-29): an existing answer ends the
// problem — graded or not, it is never re-submitted.
if (p.answer !== undefined) throw new BadAnswersError(`problem ${a.problemId} was already answered`);
} }
for (const a of answers) { for (const a of answers) {
const p = cur.problems.find((q) => q.id === a.problemId)!; const p = cur.problems.find((q) => q.id === a.problemId)!;
@@ -550,12 +556,13 @@ export function createPracticePipeline(deps: PracticePipelineDeps) {
if (!cur) return; if (!cur) return;
const p = cur.problems.find((q) => q.id === problemId); const p = cur.problems.find((q) => q.id === problemId);
if (!p || p.grade) return; if (!p || p.grade) return;
p.grade = { verdict: graded.verdict, feedback: graded.feedback, gradedAt: new Date().toISOString() }; p.grade = { score: graded.score, reasoning: graded.reasoning, gradedAt: new Date().toISOString() };
deps.practiceStore.saveSet(cur); deps.practiceStore.saveSet(cur);
}); });
}); });
} catch (err) { } catch (err) {
// Grading failure: the grade simply stays absent (re-submittable). // Grading failure: the grade simply stays absent (the answer
// stands — one attempt; the UI shows "grade interrupted").
console.error(`[practice] grading failed for ${problemId}:`, isAbortLike(err) ? "aborted" : messageOf(err)); console.error(`[practice] grading failed for ${problemId}:`, isAbortLike(err) ? "aborted" : messageOf(err));
} finally { } finally {
practiceRuns().delete(row.key); practiceRuns().delete(row.key);
src/agent/subagents/practice.ts
+13 -17
View File
@@ -211,7 +211,7 @@ PARTITION DOCTRINE:
OUTPUT — your entire deliverable is ONE submit_partition call: "partitions", a list of { chains: [ids] }, one entry per problem, together containing every input chain exactly once. No empty entries; no unknown ids; no problem without a real target relationship.`; OUTPUT — your entire deliverable is ONE submit_partition call: "partitions", a list of { chains: [ids] }, one entry per problem, together containing every input chain exactly once. No empty entries; no unknown ids; no problem without a real target relationship.`;
export const GRADER_INSTRUCTIONS = `You are the grading stage of a learning application's practice system. One practice problem — built to strengthen the learner's recall of specific propositions — has been answered. Your job is FEEDBACK ONLY: verdict + what actually mattered. You are not assessing a person; you are coaching a recall. export const GRADER_INSTRUCTIONS = `You are the grading stage of a learning application's practice system. One practice problem — built to strengthen the learner's recall of specific propositions — has been answered. Your job is small and fast: an integer score and a one-or-two-sentence justification. You are not assessing a person; you are scoring a recall.
You receive: the problem's statement; its pre-written MODEL SOLUTION and GRADING RUBRIC (committed before the learner ever saw the problem — they are the reference, not you); the propositions (names + statements) the problem was built to exercise; and the learner's ANSWER — often dictated thinking-out-loud prose. Extract the reasoning from the prose; grade the substance, never the form. Do not demand the solution's exact wording — the rubric names what actually matters; nothing else is graded. You receive: the problem's statement; its pre-written MODEL SOLUTION and GRADING RUBRIC (committed before the learner ever saw the problem — they are the reference, not you); the propositions (names + statements) the problem was built to exercise; and the learner's ANSWER — often dictated thinking-out-loud prose. Extract the reasoning from the prose; grade the substance, never the form. Do not demand the solution's exact wording — the rubric names what actually matters; nothing else is graded.
@@ -219,18 +219,14 @@ JUDGE AGAINST THE REFERENCE:
- Work the rubric points: which are met, which are missed, which are half-met in the answer's own terms. - Work the rubric points: which are met, which are missed, which are half-met in the answer's own terms.
- Where the answer diverges from the model solution, ask whether the divergence is a different-but-valid route (the rubric decides) or a wrong turn. - Where the answer diverges from the model solution, ask whether the divergence is a different-but-valid route (the rubric decides) or a wrong turn.
NAME THE FAILED RECALL: when something is missed, identify WHICH proposition failed to come to mind — or came to mind in a wrong or incomplete form. That is the actionable diagnosis; say it plainly. THE SCORE (one integer, 0–10):
- 9–10: the targeted recall(s) happened — the answer meets the rubric's essentials. Peripheral gaps do not lower it.
- 5–8: some of the target came to mind; the answer is materially incomplete or partly wrong.
- 0–4: the targeted proposition(s) did not come to mind — the answer is empty or on the wrong track.
TEACH THE BOUNDARY: feedback should locate the boundary — what would have made the answer right, where the wrong path breaks and why. Not "incorrect"; the reason it is incorrect, in one or two tight paragraphs. The reasoning is plain prose, one or two sentences: which rubric points were met or missed and, when something failed, WHICH proposition failed to come to mind. Mathematics is LaTeX — inline as \\(…\\), display as \\[…\\] (a bare $ is literal currency); no headers, no lists — minimal is the standard.
THE VERDICT (exactly one): OUTPUT — your entire deliverable is ONE submit_grade call: "score" (integer 0–10) and "reasoning" (non-empty, brief).`;
- "solid": the targeted recall(s) happened — the answer meets the rubric's essentials. Peripheral gaps do not lower it.
- "partial": some of the target came to mind; the answer is materially incomplete or partly wrong.
- "missed": the targeted proposition(s) did not come to mind — the answer is empty or on the wrong track.
Feedback renders as full Markdown. Mathematics is always LaTeX — inline as \\(…\\), display as \\[…\\] on its own lines (a bare $ is literal currency). Inside math, LaTeX commands for symbols, not unicode stand-ins. Code in fenced blocks with a language tag, identifiers inline as \`code\`. Lists and tables wherever they carry structure; no headers — feedback is short-form, structure it with paragraphs and bold labels. Name the failed recall and teach the boundary in properly formatted prose, not plain text.
OUTPUT — your entire deliverable is ONE submit_grade call: "verdict" and "feedback" (non-empty; the boundary-teaching text above).`;
// ─── schemas + fills ───────────────────────────────────────────────────────── // ─── schemas + fills ─────────────────────────────────────────────────────────
@@ -250,8 +246,8 @@ const problemSchema = Type.Object({
type ProblemSubmit = Static<typeof problemSchema>; type ProblemSubmit = Static<typeof problemSchema>;
const gradeSchema = Type.Object({ const gradeSchema = Type.Object({
verdict: Type.Union([Type.Literal("solid"), Type.Literal("partial"), Type.Literal("missed")]), score: Type.Integer({ minimum: 0, maximum: 10 }),
feedback: Type.String({ minLength: 1 }), reasoning: Type.String({ minLength: 1 }),
}); });
type GradeSubmit = Static<typeof gradeSchema>; type GradeSubmit = Static<typeof gradeSchema>;
@@ -364,8 +360,8 @@ export interface GraderInput {
signal?: AbortSignal; signal?: AbortSignal;
} }
export interface GraderOutput { export interface GraderOutput {
verdict: "solid" | "partial" | "missed"; score: number;
feedback: string; reasoning: string;
log: PracticeStepLog; log: PracticeStepLog;
} }
@@ -499,7 +495,7 @@ export function createPracticeAgents(deps: PracticeModels) {
const grade = await runSubmitAgent({ const grade = await runSubmitAgent({
role: "grader", role: "grader",
toolName: "submit_grade", toolName: "submit_grade",
toolDescription: "Submit the grade: verdict (solid | partial | missed) and non-empty feedback.", toolDescription: "Submit the grade: an integer score 0-10 and brief non-empty reasoning.",
parameters: gradeSchema, parameters: gradeSchema,
fill: (raw) => raw, fill: (raw) => raw,
systemPrompt: GRADER_INSTRUCTIONS, systemPrompt: GRADER_INSTRUCTIONS,
@@ -509,7 +505,7 @@ export function createPracticeAgents(deps: PracticeModels) {
onStep: stepLogger(log), onStep: stepLogger(log),
neverSubmitted: "the grader never submitted a grade", neverSubmitted: "the grader never submitted a grade",
}); });
return { verdict: grade.verdict, feedback: grade.feedback, log }; return { score: grade.score, reasoning: grade.reasoning, log };
} }
return { refuseIfStagesUnready, runPartitioner, runProblemGenerator, runGrader }; return { refuseIfStagesUnready, runPartitioner, runProblemGenerator, runGrader };
src/core/contracts/practice.ts
+10 -10
View File
@@ -113,16 +113,16 @@ export type ProblemSetStatus = "partitioning" | "generating" | "ready" | "error"
* generator submits. */ * generator submits. */
export type ProblemStatus = "generating" | "ready" | "error"; export type ProblemStatus = "generating" | "ready" | "error";
/** The grader's three-level verdict (feedback-only; nothing derives from /** The grading job's deliverable (owner, 2026-09-29 — one attempt per
* it — no understanding writes, owner 2026-09-29). */ * problem; the grader scores, it does not coach in prose): an integer
export type GradeVerdict = "solid" | "partial" | "missed"; * score against the pre-written rubric plus a minimal justification.
* Nothing derives from it — no understanding writes. */
/** The grading job's deliverable, stored on the problem record. */
export interface ProblemGrade { export interface ProblemGrade {
verdict: GradeVerdict; /** 0–10, integer. */
/** What actually mattered: the failed recall named, the boundary score: number;
* taught — against the pre-written solution + rubric. */ /** Why this score — one or two sentences: which rubric points were
feedback: string; * met or missed, and which proposition failed to come to mind. */
reasoning: string;
gradedAt: string; gradedAt: string;
} }
@@ -151,7 +151,7 @@ export interface Problem {
/** The learner's answer, once submitted (dictation-first — thinking /** The learner's answer, once submitted (dictation-first — thinking
* out loud is the primary answering mode). */ * out loud is the primary answering mode). */
answer?: string; answer?: string;
/** The grading job's verdict, once it lands. */ /** The grader's score + reasoning, once it lands. */
grade?: ProblemGrade; grade?: ProblemGrade;
} }
src/store/practice.ts
+16 -14
View File
@@ -73,7 +73,21 @@ const isPropositions = (v: unknown): boolean => {
const SET_STATUSES = new Set(["partitioning", "generating", "ready", "error"]); const SET_STATUSES = new Set(["partitioning", "generating", "ready", "error"]);
const PROBLEM_STATUSES = new Set(["generating", "ready", "error"]); const PROBLEM_STATUSES = new Set(["generating", "ready", "error"]);
const VERDICTS = new Set(["solid", "partial", "missed"]); function isGrade(g: unknown): boolean {
if (typeof g !== "object" || g === null) return false;
const v = g as Record<string, unknown>;
// Integer 0–10 + non-empty reasoning (one-attempt redesign, owner
// 2026-09-29: the grader scores; verdicts + prose feedback retired).
return (
typeof v.score === "number" &&
Number.isInteger(v.score) &&
v.score >= 0 &&
v.score <= 10 &&
typeof v.reasoning === "string" &&
v.reasoning.length > 0 &&
typeof v.gradedAt === "string"
);
}
const isProblem = (v: unknown): boolean => { const isProblem = (v: unknown): boolean => {
if (typeof v !== "object" || v === null) return false; if (typeof v !== "object" || v === null) return false;
@@ -83,19 +97,7 @@ const isProblem = (v: unknown): boolean => {
if (typeof p.status !== "string" || !PROBLEM_STATUSES.has(p.status)) return false; if (typeof p.status !== "string" || !PROBLEM_STATUSES.has(p.status)) return false;
if (p.error !== undefined && typeof p.error !== "string") return false; if (p.error !== undefined && typeof p.error !== "string") return false;
if (p.answer !== undefined && typeof p.answer !== "string") return false; if (p.answer !== undefined && typeof p.answer !== "string") return false;
if (p.grade !== undefined) { if (p.grade !== undefined && !isGrade(p.grade)) return false;
const g = p.grade as Record<string, unknown>;
if (
typeof g !== "object" ||
g === null ||
typeof g.verdict !== "string" ||
!VERDICTS.has(g.verdict) ||
typeof g.feedback !== "string" ||
typeof g.gradedAt !== "string"
) {
return false;
}
}
return true; return true;
}; };
tests/practice-agents.test.ts
+9 -9
View File
@@ -233,27 +233,27 @@ describe("runGrader", () => {
answer: "Well, if I'm arguing at all, I must own myself…", answer: "Well, if I'm arguing at all, I must own myself…",
}; };
it("returns the verdict + feedback with its step log", async () => { it("returns the score + reasoning with its step log", async () => {
const agents = createPracticeAgents({ const agents = createPracticeAgents({
grader: submitModel("submit_grade", { verdict: "solid", feedback: "The recall happened." }), grader: submitModel("submit_grade", { score: 9, reasoning: "The recall happened." }),
}); });
const out = await agents.runGrader(graderInput); const out = await agents.runGrader(graderInput);
assert.equal(out.verdict, "solid"); assert.equal(out.score, 9);
assert.equal(out.feedback, "The recall happened."); assert.equal(out.reasoning, "The recall happened.");
assert.ok(out.log.model); assert.ok(out.log.model);
assert.equal(out.log.steps.length, 1); assert.equal(out.log.steps.length, 1);
}); });
it("schema-rejects an off-union verdict and retries", async () => { it("schema-rejects an out-of-range score and retries", async () => {
const agents = createPracticeAgents({ const agents = createPracticeAgents({
grader: retryModel( grader: retryModel(
"submit_grade", "submit_grade",
{ verdict: "excellent", feedback: "…" }, { score: 12, reasoning: "…" },
{ verdict: "partial", feedback: "Half the target came to mind." }, { score: 6, reasoning: "Half the target came to mind." },
), ),
}); });
const out = await agents.runGrader(graderInput); const out = await agents.runGrader(graderInput);
assert.equal(out.verdict, "partial"); assert.equal(out.score, 6);
assert.equal(out.log.steps.length, 2); assert.equal(out.log.steps.length, 2);
}); });
@@ -312,7 +312,7 @@ describe("formatting doctrine pins", () => {
it("GRADER_INSTRUCTIONS carries the formatting paragraph before the OUTPUT section", () => { it("GRADER_INSTRUCTIONS carries the formatting paragraph before the OUTPUT section", () => {
assert.ok( assert.ok(
GRADER_INSTRUCTIONS.includes( GRADER_INSTRUCTIONS.includes(
"Feedback renders as full Markdown. Mathematics is always LaTeX — inline as \\(…\\), display as \\[…\\] on its own lines (a bare $ is literal currency). Inside math, LaTeX commands for symbols, not unicode stand-ins. Code in fenced blocks with a language tag, identifiers inline as `code`. Lists and tables wherever they carry structure; no headers — feedback is short-form, structure it with paragraphs and bold labels. Name the failed recall and teach the boundary in properly formatted prose, not plain text.", "The reasoning is plain prose, one or two sentences: which rubric points were met or missed and, when something failed, WHICH proposition failed to come to mind. Mathematics is LaTeX — inline as \\(…\\), display as \\[…\\] (a bare $ is literal currency); no headers, no lists — minimal is the standard.",
), ),
"the exact grader paragraph", "the exact grader paragraph",
); );
tests/practice-pipeline.test.ts
+18 -14
View File
@@ -152,7 +152,7 @@ function echoGenerator(count = 24, failTargets: string[] = []): PiEngine {
return fauxEngine(Array.from({ length: count }, () => factory)); return fauxEngine(Array.from({ length: count }, () => factory));
} }
/** A grader whose verdict encodes in the answer text ("solid" → solid). */ /** A grader whose score encodes in the answer text ("solid" → 9, else 6). */
function echoGrader(count: number): PiEngine { function echoGrader(count: number): PiEngine {
const factory: FauxResponseStep = (context) => { const factory: FauxResponseStep = (context) => {
const answer = lastUserText(context as never).split("THE LEARNER'S ANSWER\n")[1] ?? ""; const answer = lastUserText(context as never).split("THE LEARNER'S ANSWER\n")[1] ?? "";
@@ -161,8 +161,8 @@ function echoGrader(count: number): PiEngine {
fauxToolCall( fauxToolCall(
"submit_grade", "submit_grade",
{ {
verdict: answer.includes("solid") ? "solid" : "partial", score: answer.includes("solid") ? 9 : 6,
feedback: "The boundary.", reasoning: "Which rubric points were met.",
} as ToolCall["arguments"], } as ToolCall["arguments"],
{ id: "grade-call" }, { id: "grade-call" },
), ),
@@ -521,7 +521,7 @@ describe("createProblemSet (generation)", () => {
const gate = new Promise<void>((r) => (release = r)); const gate = new Promise<void>((r) => (release = r));
// hold both grades at the model seam // hold both grades at the model seam
const holdSteps: FauxResponseStep[] = Array.from({ length: 2 }, () => () => const holdSteps: FauxResponseStep[] = Array.from({ length: 2 }, () => () =>
gate.then(() => fauxAssistantMessage([fauxToolCall("submit_grade", { verdict: "solid", feedback: "yes" } as ToolCall["arguments"], { id: "gr" })], { stopReason: "toolUse" })), gate.then(() => fauxAssistantMessage([fauxToolCall("submit_grade", { score: 9, reasoning: "yes" } as ToolCall["arguments"], { id: "gr" })], { stopReason: "toolUse" })),
); );
const f = fixture({ const f = fixture({
partitioner: partitionModel([{ chains: ["b"] }, { chains: ["c"] }]), partitioner: partitionModel([{ chains: ["b"] }, { chains: ["c"] }]),
@@ -554,7 +554,7 @@ describe("createProblemSet (generation)", () => {
await sub.done; await sub.done;
assert.equal(practiceStatuses().length, 0); assert.equal(practiceStatuses().length, 0);
const graded = f.practice.store.getSet(set.id)!; const graded = f.practice.store.getSet(set.id)!;
assert.ok(graded.problems.every((p) => p.grade?.verdict === "solid")); assert.ok(graded.problems.every((p) => p.grade?.score === 9));
}); });
it("abort ONE gen row: that problem dies alone, siblings complete, set reaches ready with one errored", async () => { it("abort ONE gen row: that problem dies alone, siblings complete, set reaches ready with one errored", async () => {
@@ -600,7 +600,7 @@ describe("createProblemSet (generation)", () => {
assert.equal(practiceStatuses().length, 0); assert.equal(practiceStatuses().length, 0);
}); });
it("abort ONE grade row (queued): that grade drops (answer kept, re-submittable), sibling grade lands", async () => { it("abort ONE grade row (queued): that grade drops (answer kept, one attempt — no re-submission), sibling grade lands", async () => {
const f = fixture({ const f = fixture({
partitioner: partitionModel([{ chains: ["b"] }, { chains: ["c"] }]), partitioner: partitionModel([{ chains: ["b"] }, { chains: ["c"] }]),
problemGenerator: echoGenerator(2), problemGenerator: echoGenerator(2),
@@ -645,14 +645,18 @@ describe("createProblemSet (generation)", () => {
assert.equal(dropped.grade, undefined, "the aborted grade is absent"); assert.equal(dropped.grade, undefined, "the aborted grade is absent");
assert.equal(dropped.answer, "first", "the answer is kept"); assert.equal(dropped.answer, "first", "the answer is kept");
const other = graded.problems.find((p) => p.id !== victimId)!; const other = graded.problems.find((p) => p.id !== victimId)!;
assert.equal(other.grade?.verdict, "partial", "the sibling grade landed (echoGrader: no 'solid' in its text)"); assert.equal(other.grade?.score, 6, "the sibling grade landed (echoGrader: no 'solid' in its text)");
assert.equal(practiceStatuses().length, 0); assert.equal(practiceStatuses().length, 0);
// re-submittable: the dropped answer grades on a second submit // ONE attempt (owner 2026-09-29): the dropped grade's answer
const resub = await pipeline.submitAnswers(set.id, [{ problemId: victimId, text: "retry answer" }]); // stands — re-submission is refused, the problem is done.
await resub.done; await assert.rejects(
const retried = f.practice.store.getSet(set.id)!; () => pipeline.submitAnswers(set.id, [{ problemId: victimId, text: "retry answer" }]),
assert.equal(retried.problems.find((p) => p.id === victimId)!.grade?.verdict, "partial", "the retry grades (echoGrader verdict)"); /was already answered/,
);
const after = f.practice.store.getSet(set.id)!;
assert.equal(after.problems.find((p) => p.id === victimId)!.grade, undefined, "still no grade — one attempt");
assert.equal(after.problems.find((p) => p.id === victimId)!.answer, "first");
rh1(); rh1();
rh2(); rh2();
@@ -772,8 +776,8 @@ describe("submitAnswers (grading)", () => {
const final = f.practice.store.getSet(set.id)!; const final = f.practice.store.getSet(set.id)!;
const g1 = final.problems.find((p) => p.id === p1.id)!.grade; const g1 = final.problems.find((p) => p.id === p1.id)!.grade;
const g2 = final.problems.find((p) => p.id === p2.id)!.grade; const g2 = final.problems.find((p) => p.id === p2.id)!.grade;
assert.ok(g1 && g1.verdict === "solid", "verdict encodes from the answer text"); assert.ok(g1 && g1.score === 9, "score encodes from the answer text");
assert.ok(g2 && g2.verdict === "partial"); assert.ok(g2 && g2.score === 6);
assert.ok(g1!.gradedAt >= NOW); assert.ok(g1!.gradedAt >= NOW);
assert.equal(practiceStatuses().length, 0, "the grading row clears"); assert.equal(practiceStatuses().length, 0, "the grading row clears");
tests/practice-store.test.ts
+13 -9
View File
@@ -79,13 +79,13 @@ describe("the practice store", () => {
rubric: "- names the performative contradiction", rubric: "- names the performative contradiction",
status: "ready", status: "ready",
answer: "Because…", answer: "Because…",
grade: { verdict: "solid", feedback: "Named it.", gradedAt: "2026-09-29T13:00:00.000Z" }, grade: { score: 9, reasoning: "Named it.", gradedAt: "2026-09-29T13:00:00.000Z" },
}); });
set.status = "ready"; set.status = "ready";
practiceStore.saveSet(set); practiceStore.saveSet(set);
const reread = practiceStore.getSet(set.id); const reread = practiceStore.getSet(set.id);
assert.equal(reread?.status, "ready"); assert.equal(reread?.status, "ready");
assert.equal(reread?.problems[0]?.grade?.verdict, "solid"); assert.equal(reread?.problems[0]?.grade?.score, 9);
}); });
it("listSets returns createdAt-descending and skips corrupt/unparseable files", () => { it("listSets returns createdAt-descending and skips corrupt/unparseable files", () => {
@@ -157,17 +157,21 @@ describe("isProblemSet", () => {
assert.ok(isProblemSet(minimal)); assert.ok(isProblemSet(minimal));
}); });
it("grade verdicts are the three-level union", () => { it("grades are integer scores 0–10 with non-empty reasoning (one-attempt redesign)", () => {
const withGrade = (verdict: string): unknown => ({ const withGrade = (score: unknown, reasoning = "why"): unknown => ({
...init(), ...init(),
id: "ps_valid", id: "ps_valid",
problems: [ problems: [
{ id: "pp_1", chains: [], statement: "", solution: "", rubric: "", status: "ready", grade: { verdict, feedback: "", gradedAt: "t" } }, { id: "pp_1", chains: [], statement: "", solution: "", rubric: "", status: "ready", grade: { score, reasoning, gradedAt: "t" } },
], ],
}); });
assert.ok(isProblemSet(withGrade("solid"))); assert.ok(isProblemSet(withGrade(0)));
assert.ok(isProblemSet(withGrade("partial"))); assert.ok(isProblemSet(withGrade(10)));
assert.ok(isProblemSet(withGrade("missed"))); assert.ok(!isProblemSet(withGrade(11)), "above the range");
assert.ok(!isProblemSet(withGrade("excellent"))); assert.ok(!isProblemSet(withGrade(-1)), "below the range");
assert.ok(!isProblemSet(withGrade(7.5)), "fractional scores are refused");
assert.ok(!isProblemSet(withGrade("9")), "string scores are refused");
assert.ok(!isProblemSet(withGrade(9, "")), "empty reasoning is refused");
assert.ok(!isProblemSet({ ...init(), id: "ps_valid", problems: [{ id: "pp_1", chains: [], statement: "", solution: "", rubric: "", status: "ready", grade: { verdict: "solid", feedback: "legacy", gradedAt: "t" } }] }), "legacy verdict grades fail (wipe regime)");
}); });
}); });
tests/render-practice.test.tsx
+54 -17
View File
@@ -200,7 +200,7 @@ describe("PracticeListBody — the set list", () => {
it("graded count appears; empty list states it honestly", () => { it("graded count appears; empty list states it honestly", () => {
const s = set({ const s = set({
problems: [ problems: [
readyProblem("pp_1", ["n3"], { grade: { verdict: "solid", feedback: "f", gradedAt: "2026-09-29T13:00:00.000Z" } }), readyProblem("pp_1", ["n3"], { answer: "a", grade: { score: 9, reasoning: "r", gradedAt: "2026-09-29T13:00:00.000Z" } }),
readyProblem("pp_2", ["n1"]), readyProblem("pp_2", ["n1"]),
], ],
}); });
@@ -251,9 +251,9 @@ describe("PracticeExamBody — exam states", () => {
drafts: {} as Record<string, string>, drafts: {} as Record<string, string>,
onDraft: () => {}, onDraft: () => {},
onSubmit: () => {}, onSubmit: () => {},
submitting: false, submittingId: null as string | null,
submitError: null as string | null, submitErrors: {} as Record<string, string>,
grading: false, liveGradeIds: new Set<string>(),
}; };
it("generating: numbered stubs with the placeholder — propositions NEVER surface", () => { it("generating: numbered stubs with the placeholder — propositions NEVER surface", () => {
@@ -271,12 +271,19 @@ describe("PracticeExamBody — exam states", () => {
assert.ok(!html.includes("Each person owns themself")); assert.ok(!html.includes("Each person owns themself"));
assert.ok(!html.includes('data-testid="answer-pp_1"'), "no answer box for a stub"); assert.ok(!html.includes('data-testid="answer-pp_1"'), "no answer box for a stub");
// No submit while generating. // No submit while generating.
assert.ok(!html.includes('data-testid="practice-submit"')); assert.ok(!html.includes('data-testid="practice-submit-'));
}); });
it("the header carries the levels line (named levels, owner 2026-09-29)", () => { it("the top bar carries the title + sidebar toggle; the body carries the levels line (owner, UI round 2026-09-29)", () => {
const s = set({ problems: [readyProblem("pp_1", ["n3"])] }); const s = set({ problems: [readyProblem("pp_1", ["n3"])] });
const html = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />); const html = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
// The chat-pattern top bar: drag surface, toggle slot, truncating title.
assert.ok(html.includes("lh-drag"));
assert.ok(html.includes('aria-label="Open sidebar"'));
assert.ok(html.includes('data-testid="practice-exam-title"'));
assert.ok(html.includes("Ethics lesson"));
// The internal scroller (the sidebar stays fixed when this page scrolls).
assert.ok(html.includes('data-testid="practice-exam-scroll"'));
assert.ok(html.includes('data-testid="practice-exam-levels"')); assert.ok(html.includes('data-testid="practice-exam-levels"'));
assert.ok(html.includes("Direct exercise · Adjacent context · Theory"), "the fixture's rung 2 / cue 1 / theory"); assert.ok(html.includes("Direct exercise · Adjacent context · Theory"), "the fixture's rung 2 / cue 1 / theory");
}); });
@@ -286,37 +293,67 @@ describe("PracticeExamBody — exam states", () => {
const gated = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />); const gated = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
assert.ok(gated.includes("self-ownership")); // markdown strong rendered assert.ok(gated.includes("self-ownership")); // markdown strong rendered
assert.ok(gated.includes('data-testid="answer-pp_1"')); assert.ok(gated.includes('data-testid="answer-pp_1"'));
assert.ok(gated.includes('data-testid="practice-submit"')); // PER-PROBLEM submit (owner 2026-09-29): the button lives under the
// box, gated on a draft; there is NO global submit anywhere.
assert.ok(gated.includes('data-testid="practice-submit-pp_1"'));
assert.ok(gated.includes('disabled=""'), "empty drafts — submit disabled"); assert.ok(gated.includes('disabled=""'), "empty drafts — submit disabled");
const filled = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} drafts={{ pp_1: "because…" }} />); const filled = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} drafts={{ pp_1: "because…" }} />);
assert.ok(!filled.includes('data-testid="practice-submit-error"')); assert.ok(!filled.includes('disabled=""'), "a draft arms the submit");
assert.ok(!filled.includes('data-testid="practice-submit-error-pp_1"'));
}); });
it("graded: answer read-only, verdict chip + feedback + solution + rubric revealed", () => { it("submitted + graded: answer read-only, score + reasoning + solution revealed, rubric collapsed", () => {
const s = set({ const s = set({
problems: [ problems: [
readyProblem("pp_1", ["n3"], { readyProblem("pp_1", ["n3"], {
answer: "because arguing is intentional action", answer: "because arguing is intentional action",
grade: { verdict: "partial", feedback: "Half the target came to mind.", gradedAt: "2026-09-29T13:00:00.000Z" }, grade: { score: 6, reasoning: "Half the target came to mind.", gradedAt: "2026-09-29T13:00:00.000Z" },
}), }),
readyProblem("pp_2", ["n1"]), readyProblem("pp_2", ["n1"]),
], ],
}); });
const html = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />); const html = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
// The locked answer. // The locked answer — one attempt, done.
assert.ok(html.includes("because arguing is intentional action")); assert.ok(html.includes("because arguing is intentional action"));
assert.ok(/readOnly/.test(html), "the graded answer is read-only"); assert.ok(/readOnly/.test(html), "the submitted answer is read-only");
// The reveal. assert.ok(!html.includes('data-testid="practice-submit-pp_1"'), "no re-submission");
assert.ok(html.includes('data-testid="reveal-pp_1"')); // The grade: mono score + brief reasoning.
assert.ok(html.includes("partial")); assert.ok(html.includes('data-testid="grade-pp_1"'));
assert.ok(html.includes('data-testid="feedback-pp_1"')); assert.ok(html.includes("6/10"));
assert.ok(html.includes('data-testid="reasoning-pp_1"'));
assert.ok(html.includes("Half the target came to mind.")); assert.ok(html.includes("Half the target came to mind."));
// The instant reveal.
assert.ok(html.includes('data-testid="reveal-pp_1"'));
assert.ok(html.includes('data-testid="solution-pp_1"')); assert.ok(html.includes('data-testid="solution-pp_1"'));
assert.ok(html.includes("The model solution.")); assert.ok(html.includes("The model solution."));
// The rubric rides a collapsed <details> (owner 2026-09-29).
assert.ok(html.includes("<details"));
assert.ok(!html.includes("<details open"), "collapsed by default");
assert.ok(html.includes('data-testid="rubric-pp_1"')); assert.ok(html.includes('data-testid="rubric-pp_1"'));
assert.ok(html.includes("names self-ownership")); assert.ok(html.includes("names self-ownership"));
// Ungraded sibling stays answerable. // Unanswered sibling stays answerable.
assert.ok(html.includes('data-testid="answer-pp_2"')); assert.ok(html.includes('data-testid="answer-pp_2"'));
assert.ok(html.includes('data-testid="practice-submit-pp_2"'));
});
it("submitted, grade pending: the reveal shows instantly, grading spins per problem", () => {
const s = set({
problems: [readyProblem("pp_1", ["n3"], { answer: "because arguing is intentional action" })],
});
const live = renderToStaticMarkup(
<PracticeExamBody set={s} {...examBase} liveGradeIds={new Set(["pp_1"])} />,
);
// Instant reveal — the solution needs no grade.
assert.ok(live.includes('data-testid="solution-pp_1"'));
assert.ok(live.includes('data-testid="rubric-pp_1"'));
// The live spinner for exactly this problem.
assert.ok(live.includes('data-testid="grading-pp_1"'));
assert.ok(live.includes("Grading"));
assert.ok(!live.includes('data-testid="grade-pp_1"'), "no score yet");
// No live row (dropped grade): honest, no eternal spinner.
const dropped = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
assert.ok(dropped.includes('data-testid="grade-interrupted-pp_1"'));
assert.ok(!dropped.includes('data-testid="grading-pp_1"'));
}); });
it("a failed problem says so; a failed set says so", () => { it("a failed problem says so; a failed set says so", () => {