practice UI round — top bars + sidebar reopen, one-attempt per-problem submit with instant reveal, score-only grading (owner 2026-09-29)
This commit is contained in:
amosblomqvist
@@ -156,7 +156,7 @@ data/ chats/ · graph/ (six ledgers + positions/source
|
||||
|
||||
**The knowledge graph (two channels, one writer).** The write channel: the extraction pipeline — extractor (LLM: sees only the session, rendered as readable markdown by the shared `core/session-markdown.ts` engine — propositions as true statements, `follows`/`supports` edges with verbatim evidence; fully trusted, graph-blind) → auditor (LLM, an agentic loop that never sees the source: explores the existing graph through `find` / `list_sessions` / `list_propositions` tools, stamps each proposition with an identity verdict — link to an existing node (equivalence bias: paraphrases merge) or create — and may draw bridging edges between existing nodes and the new batch) → merger (`mergeFindings`, deterministic: applies the verdicts, link-appends phrasings/provenance, resolves and dedupes edges, checkpoints coverage) → layout settle. No quiz outcomes or understanding assessments live in the graph (2026-09-27 cleanup); practice (`/practice`) consumes the graph as feedback-only problem sets and writes nothing back. Triggered when a chat's **uncovered tokens** (message-view fingerprints, `coverage.jsonl`) cross `graph.extractionChunkTokens` (a settings knob, default 4096, floor 500 — read fresh per turn); failure checkpoints nothing, the next trigger re-runs. The same pipeline serves the **notes front door** (`POST /api/graph/import`: any `*.md`/`*.txt` becomes a source; re-imports are idempotent by path hash). The read channel: `GET /api/graph` derives the snapshot from the ledgers — a pure read; the home view is a hand-rolled Canvas 2D renderer (one rAF loop, dirty-flag idle, d3-zoom camera, zoom-gated labels, cluster hulls) over a d3-force layout, with server-persisted render settings and drag-committed positions. Clusters are **birth-session groupings**: a node joins the cluster of the session that birthed it (a chat, or an imported notes file as a file-kind session), derived at read — no embedding pass, no naming pass. The teacher reaches the graph on its own terms through the gated prober tools (`list_sessions` / `list_propositions`, the `agent.graphTools` setting) — there is no automatic context injection.
|
||||
|
||||
**Practice (problem sets).** The graph's `follows`-edges become reasoning chains; the `/practice` creation dialog picks a birth-session cluster (the graph view's own groups) and freezes its propositions + chains into a self-contained `ProblemSet`. Three named-level axes steer generation (ladder rung 0–4, cue distance 0–2, Theory|Application — never a number to interpolate; defaults Direct exercise · Adjacent context · Theory). A partitioner agent covers every chain exactly once, one generator per partition writes problems against the owner's problem-engineering framework (commit-before-pose: statement + solution + rubric together), and grading is **strengthening, not assessment** — one grader per answer, verdict (`solid`/`partial`/`missed`) + feedback, post-grade reveal, nothing ever written back to the graph. Runs are registry-backed background jobs on the activity dock (no steer, never wakes the teacher); sets live in `data/practice`, crash-swept on boot.
|
||||
**Practice (problem sets).** The graph's `follows`-edges become reasoning chains; the `/practice` creation dialog picks a birth-session cluster (the graph view's own groups) and freezes its propositions + chains into a self-contained `ProblemSet`. Three named-level axes steer generation (ladder rung 0–4, cue distance 0–2, Theory|Application — never a number to interpolate; defaults Direct exercise · Adjacent context · Theory). A partitioner agent covers every chain exactly once, one generator per partition writes problems against the owner's problem-engineering framework (commit-before-pose: statement + solution + rubric together), and grading is **strengthening, not assessment** — one attempt per problem with its own submit button, instant solution/rubric reveal, then a score 0–10 with brief reasoning landing live; nothing ever written back to the graph. Runs are registry-backed background jobs on the activity dock (no steer, never wakes the teacher); sets live in `data/practice`, crash-swept on boot.
|
||||
|
||||
**Dictation.** Optional speech-to-text: the server mints short-lived Deepgram tokens; audio streams browser→Deepgram over WebSocket, never through the server.
|
||||
|
||||
|
||||
@@ -2,8 +2,9 @@
|
||||
* POST /api/practice/sets/[id]/answers — submit answers for grading
|
||||
* (202; one grading job per answered problem runs in the background).
|
||||
* Body: {answers: [{problemId, text}]}. Set must be ready; answers must
|
||||
* be non-empty text on known, not-yet-graded problems (400/409 via the
|
||||
* service's typed-error mapping).
|
||||
* be non-empty text on known, not-yet-answered problems (400/409 via the
|
||||
* service's typed-error mapping). ONE ATTEMPT (owner 2026-09-29): once a
|
||||
* problem carries an answer it is done — re-submission is refused.
|
||||
*/
|
||||
import { submitPracticeAnswersResponse } from "@/src/server";
|
||||
|
||||
|
||||
+267
-111
@@ -1,36 +1,39 @@
|
||||
"use client";
|
||||
|
||||
/**
|
||||
* The exam page (practice slice 5): one problem set, numbered problems,
|
||||
* markdown statements, a textarea + dictation per problem (the bar
|
||||
* renders only on the FOCUSED box — thinking out loud is the primary
|
||||
* answering mode), one submit for the set, and the post-grade reveal
|
||||
* (verdict chip, feedback, model solution, rubric) under the box.
|
||||
* The exam page (practice slice 5, one-attempt redesign 2026-09-29):
|
||||
* one problem set, numbered problems, markdown statements, a textarea +
|
||||
* dictation per problem (the bar renders only on the FOCUSED box —
|
||||
* thinking out loud is the primary answering mode). ONE ATTEMPT: each
|
||||
* problem has its OWN submit button; once submitted the box locks and
|
||||
* the MODEL SOLUTION reveals instantly (rubric collapsed by default).
|
||||
* The grade (score 0–10 + brief reasoning) lands via the poll and
|
||||
* updates live — no reload.
|
||||
*
|
||||
* DISPLAY-SEAM MATH HEALING: statements/solutions/rubrics/feedback ride
|
||||
* TOP BAR (owner, UI round 2026-09-29): the chat pattern — an h-15
|
||||
* lh-drag header in flow above the scroller, the sidebar reopen toggle
|
||||
* first (it clears the traffic lights via .lh-closed-toggle), the set
|
||||
* title truncating beside it. The levels + status stay in the body.
|
||||
*
|
||||
* DISPLAY-SEAM MATH HEALING: statements/solutions/rubrics/reasoning ride
|
||||
* the same streaming-args parse as quiz/ask strings — restoreMathEscapes
|
||||
* heals them at render (the same helper the tool-part seam uses).
|
||||
*
|
||||
* POLLING: 2s while the set isn't ready (stubs → filled), and while a
|
||||
* grading run is live for this set (activity feed's practice rows) with
|
||||
* answered-but-ungraded problems outstanding. A settled page goes quiet
|
||||
* (a dropped grade terminates honestly: no live row → no poll).
|
||||
* grading run is live for this set (activity feed practice rows are
|
||||
* keyed setId:grade:problemId — the LIVE rows name exactly which
|
||||
* problems' grades are pending). A settled page goes quiet; a dropped
|
||||
* grade shows "grade interrupted" honestly.
|
||||
*/
|
||||
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import type { GradeVerdict, Problem, ProblemSet } from "@/src/core";
|
||||
import type { Problem, ProblemSet } from "@/src/core";
|
||||
import { CUE_LEVELS, LADDER_RUNGS, PRESENTATIONS } from "@/src/core";
|
||||
import { restoreMathEscapes } from "@/src/core";
|
||||
import { MessageResponse } from "@/components/ai-elements/message";
|
||||
import { BrailleSpinner } from "@/components/braille-spinner";
|
||||
import { DictationBar, useDictation } from "@/components/dictation";
|
||||
|
||||
/** The verdict chip's tone — plain, no invented color language. */
|
||||
export function verdictClass(verdict: GradeVerdict): string {
|
||||
if (verdict === "solid") return "bg-green-500/15 text-green-500";
|
||||
if (verdict === "partial") return "bg-amber-500/15 text-amber-600 dark:text-amber-400";
|
||||
return "bg-destructive/15 text-destructive";
|
||||
}
|
||||
import { SidebarToggle } from "@/components/sidebar";
|
||||
|
||||
/** One answerable problem: textarea + focus-gated dictation bar. */
|
||||
export function ProblemBox({
|
||||
@@ -48,7 +51,10 @@ export function ProblemBox({
|
||||
append: (text) => onDraft(text),
|
||||
});
|
||||
return (
|
||||
<div className="flex flex-col gap-1.5" data-testid={`problem-box-${problem.id}`}>
|
||||
<div
|
||||
className="flex flex-col gap-1.5"
|
||||
data-testid={`problem-box-${problem.id}`}
|
||||
>
|
||||
<textarea
|
||||
data-testid={`answer-${problem.id}`}
|
||||
value={draft}
|
||||
@@ -70,114 +76,227 @@ export function PracticeExamBody({
|
||||
drafts,
|
||||
onDraft,
|
||||
onSubmit,
|
||||
submitting,
|
||||
submitError,
|
||||
grading,
|
||||
submittingId,
|
||||
submitErrors,
|
||||
liveGradeIds,
|
||||
}: {
|
||||
set: ProblemSet;
|
||||
drafts: Record<string, string>;
|
||||
onDraft: (problemId: string, text: string) => void;
|
||||
onSubmit: () => void;
|
||||
submitting: boolean;
|
||||
submitError: string | null;
|
||||
grading: boolean;
|
||||
/** Per-problem submit (owner 2026-09-29): ONE attempt per problem. */
|
||||
onSubmit: (problemId: string) => void;
|
||||
/** The problem whose submit POST is in flight. */
|
||||
submittingId: string | null;
|
||||
/** Per-problem submit failures, keyed by problem id. */
|
||||
submitErrors: Record<string, string>;
|
||||
/** Problems whose grade rows are LIVE (the poll's activity feed). */
|
||||
liveGradeIds: ReadonlySet<string>;
|
||||
}) {
|
||||
const answerable = set.problems.filter((p) => !p.grade);
|
||||
const anyDraft = answerable.some((p) => (drafts[p.id] ?? p.answer ?? "") !== "");
|
||||
return (
|
||||
<div className="h-dvh overflow-y-auto" data-testid="practice-exam-scroll">
|
||||
<div className="mx-auto flex w-full max-w-3xl flex-col gap-6 p-6" data-testid="practice-exam">
|
||||
<div className="flex h-dvh">
|
||||
<div className="flex min-w-0 flex-1 flex-col">
|
||||
{/* The top bar (owner, UI round 2026-09-29): the new-session
|
||||
pattern — h-15, lh-drag (the Electron window-drag surface, so
|
||||
the traffic lights never cover content), the sidebar reopen
|
||||
toggle first, the set title truncating beside it. */}
|
||||
<header className="lh-drag flex h-15 items-center gap-3 border-b border-border px-3">
|
||||
<SidebarToggle />
|
||||
<div
|
||||
className="min-w-0 flex-1 truncate font-title text-3xl tracking-tight"
|
||||
data-testid="practice-exam-title"
|
||||
>
|
||||
{set.title}
|
||||
</div>
|
||||
</header>
|
||||
<div className="relative flex min-h-0 flex-1">
|
||||
<div
|
||||
className="h-full min-w-0 flex-1 overflow-y-auto"
|
||||
data-testid="practice-exam-scroll"
|
||||
>
|
||||
<div
|
||||
className="mx-auto flex w-full max-w-3xl flex-col gap-6 p-6"
|
||||
data-testid="practice-exam"
|
||||
>
|
||||
<div className="flex flex-col gap-1">
|
||||
<h1 className="font-title text-3xl tracking-tight">{set.title}</h1>
|
||||
<p className="text-xs text-muted-foreground/80" data-testid="practice-exam-levels">
|
||||
{LADDER_RUNGS[set.params.ladderRung].name} · {CUE_LEVELS[set.params.cueDistance].name} ·{" "}
|
||||
<p
|
||||
className="text-xs text-muted-foreground/80"
|
||||
data-testid="practice-exam-levels"
|
||||
>
|
||||
{LADDER_RUNGS[set.params.ladderRung].name} ·{" "}
|
||||
{CUE_LEVELS[set.params.cueDistance].name} ·{" "}
|
||||
{PRESENTATIONS[set.params.presentation].name}
|
||||
</p>
|
||||
<p className="text-sm text-muted-foreground" data-testid="practice-exam-status">
|
||||
<p
|
||||
className="text-sm text-muted-foreground"
|
||||
data-testid="practice-exam-status"
|
||||
>
|
||||
{set.status === "partitioning" && "Partitioning the chains…"}
|
||||
{set.status === "generating" && `Generating — ${set.problems.filter((p) => p.status === "ready").length}/${set.problems.length} problems ready`}
|
||||
{set.status === "error" && (set.error ?? "Something went wrong")}
|
||||
{set.status === "generating" &&
|
||||
`Generating — ${set.problems.filter((p) => p.status === "ready").length}/${set.problems.length} problems ready`}
|
||||
{set.status === "error" &&
|
||||
(set.error ?? "Something went wrong")}
|
||||
{set.status === "ready" &&
|
||||
(grading ? "Grading your answers…" : `${set.problems.length} problems · ${set.problems.filter((p) => p.grade).length} graded`)}
|
||||
(liveGradeIds.size > 0
|
||||
? "Grading your answers…"
|
||||
: `${set.problems.length} problems · ${set.problems.filter((p) => p.grade).length} graded`)}
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{set.problems.map((problem, i) => (
|
||||
<section key={problem.id} className="rounded-xl border bg-card p-4" data-testid={`problem-${problem.id}`}>
|
||||
<h2 className="mb-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">Problem {i + 1}</h2>
|
||||
<section
|
||||
key={problem.id}
|
||||
className="rounded-xl border bg-card p-4"
|
||||
data-testid={`problem-${problem.id}`}
|
||||
>
|
||||
<h2 className="mb-2 text-xs font-medium uppercase tracking-wide text-muted-foreground">
|
||||
Problem {i + 1}
|
||||
</h2>
|
||||
{problem.status === "generating" ? (
|
||||
// The stub: propositions NEVER surface (never-announce law).
|
||||
<div className="flex items-center gap-2 text-sm text-muted-foreground" data-testid={`generating-${problem.id}`}>
|
||||
<div
|
||||
className="flex items-center gap-2 text-sm text-muted-foreground"
|
||||
data-testid={`generating-${problem.id}`}
|
||||
>
|
||||
<BrailleSpinner className="size-4" />
|
||||
Generating problem…
|
||||
</div>
|
||||
) : problem.status === "error" ? (
|
||||
<p className="text-sm text-destructive" data-testid={`problem-error-${problem.id}`}>
|
||||
This problem failed to generate{problem.error ? ` — ${problem.error}` : ""}.
|
||||
<p
|
||||
className="text-sm text-destructive"
|
||||
data-testid={`problem-error-${problem.id}`}
|
||||
>
|
||||
This problem failed to generate
|
||||
{problem.error ? ` — ${problem.error}` : ""}.
|
||||
</p>
|
||||
) : (
|
||||
<>
|
||||
<div className="lh-prose text-sm">
|
||||
<MessageResponse mode="static">{restoreMathEscapes(problem.statement)}</MessageResponse>
|
||||
<MessageResponse mode="static">
|
||||
{restoreMathEscapes(problem.statement)}
|
||||
</MessageResponse>
|
||||
</div>
|
||||
{problem.grade ? (
|
||||
// Graded: the answer read-only, the reveal below.
|
||||
{problem.answer === undefined ? (
|
||||
// ANSWERABLE (owner 2026-09-29): the box + its OWN submit
|
||||
// button — one attempt, submitted the moment it's clicked.
|
||||
<div className="flex flex-col items-start gap-2">
|
||||
<div className="w-full">
|
||||
<ProblemBox
|
||||
problem={problem}
|
||||
draft={drafts[problem.id] ?? ""}
|
||||
onDraft={(text) => onDraft(problem.id, text)}
|
||||
/>
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
data-testid={`practice-submit-${problem.id}`}
|
||||
onClick={() => onSubmit(problem.id)}
|
||||
disabled={
|
||||
set.status !== "ready" ||
|
||||
(drafts[problem.id] ?? "").trim() === "" ||
|
||||
submittingId !== null
|
||||
}
|
||||
className="rounded-md bg-primary px-4 py-2 text-sm font-medium text-primary-foreground transition-colors hover:bg-primary/90 disabled:opacity-40"
|
||||
>
|
||||
{submittingId === problem.id
|
||||
? "Submitting…"
|
||||
: "Submit answer"}
|
||||
</button>
|
||||
{submitErrors[problem.id] && (
|
||||
<p
|
||||
data-testid={`practice-submit-error-${problem.id}`}
|
||||
className="text-sm text-destructive"
|
||||
>
|
||||
{submitErrors[problem.id]}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
) : (
|
||||
// SUBMITTED (owner 2026-09-29): the answer locks, the
|
||||
// model solution reveals INSTANTLY, the rubric stays
|
||||
// collapsed, and the grade lands live when it lands.
|
||||
<div className="mt-4 flex flex-col gap-3">
|
||||
<textarea
|
||||
data-testid={`answer-${problem.id}`}
|
||||
value={problem.answer ?? ""}
|
||||
value={problem.answer}
|
||||
readOnly
|
||||
rows={5}
|
||||
className="w-full resize-y rounded-lg border bg-muted/40 px-3 py-2 text-sm text-muted-foreground"
|
||||
/>
|
||||
<div className="rounded-lg border bg-background p-3" data-testid={`reveal-${problem.id}`}>
|
||||
<div className="mb-2 flex items-center gap-2">
|
||||
<span className={`rounded px-1.5 py-0.5 text-xs font-medium uppercase ${verdictClass(problem.grade.verdict)}`}>
|
||||
{problem.grade.verdict}
|
||||
<div
|
||||
className="rounded-lg border bg-background p-3"
|
||||
data-testid={`reveal-${problem.id}`}
|
||||
>
|
||||
{problem.grade ? (
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<div className="flex items-baseline gap-2">
|
||||
<span
|
||||
data-testid={`grade-${problem.id}`}
|
||||
className="font-mono text-sm font-medium"
|
||||
>
|
||||
{problem.grade.score}/10
|
||||
</span>
|
||||
</div>
|
||||
<div className="lh-prose text-sm" data-testid={`feedback-${problem.id}`}>
|
||||
<MessageResponse mode="static">{restoreMathEscapes(problem.grade.feedback)}</MessageResponse>
|
||||
</div>
|
||||
<div className="mt-3 text-xs uppercase tracking-wide text-muted-foreground">Model solution</div>
|
||||
<div className="lh-prose text-sm" data-testid={`solution-${problem.id}`}>
|
||||
<MessageResponse mode="static">{restoreMathEscapes(problem.solution)}</MessageResponse>
|
||||
</div>
|
||||
<div className="mt-3 text-xs uppercase tracking-wide text-muted-foreground">Rubric</div>
|
||||
<div className="lh-prose text-sm" data-testid={`rubric-${problem.id}`}>
|
||||
<MessageResponse mode="static">{restoreMathEscapes(problem.rubric)}</MessageResponse>
|
||||
<div
|
||||
className="lh-prose text-sm"
|
||||
data-testid={`reasoning-${problem.id}`}
|
||||
>
|
||||
<MessageResponse mode="static">
|
||||
{restoreMathEscapes(
|
||||
problem.grade.reasoning,
|
||||
)}
|
||||
</MessageResponse>
|
||||
</div>
|
||||
</div>
|
||||
) : liveGradeIds.has(problem.id) ? (
|
||||
<div
|
||||
className="flex items-center gap-2 text-sm text-muted-foreground"
|
||||
data-testid={`grading-${problem.id}`}
|
||||
>
|
||||
<BrailleSpinner className="size-4" />
|
||||
Grading…
|
||||
</div>
|
||||
) : (
|
||||
// Answerable: the box (graded problems lock above).
|
||||
<div className="mt-4">
|
||||
<ProblemBox problem={problem} draft={drafts[problem.id] ?? problem.answer ?? ""} onDraft={(text) => onDraft(problem.id, text)} />
|
||||
<p
|
||||
className="text-sm text-muted-foreground"
|
||||
data-testid={`grade-interrupted-${problem.id}`}
|
||||
>
|
||||
Grade interrupted — the answer stands.
|
||||
</p>
|
||||
)}
|
||||
<div className="mt-3 text-xs uppercase tracking-wide text-muted-foreground">
|
||||
Model solution
|
||||
</div>
|
||||
<div
|
||||
className="lh-prose text-sm"
|
||||
data-testid={`solution-${problem.id}`}
|
||||
>
|
||||
<MessageResponse mode="static">
|
||||
{restoreMathEscapes(problem.solution)}
|
||||
</MessageResponse>
|
||||
</div>
|
||||
<details
|
||||
className="mt-3"
|
||||
data-testid={`rubric-${problem.id}`}
|
||||
>
|
||||
<summary className="cursor-pointer select-none text-xs uppercase tracking-wide text-muted-foreground">
|
||||
Rubric
|
||||
</summary>
|
||||
<div className="mt-2 lh-prose text-sm">
|
||||
<MessageResponse mode="static">
|
||||
{restoreMathEscapes(problem.rubric)}
|
||||
</MessageResponse>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</section>
|
||||
))}
|
||||
|
||||
{set.status === "ready" && answerable.length > 0 && (
|
||||
<div className="flex flex-col items-start gap-2">
|
||||
<button
|
||||
type="button"
|
||||
data-testid="practice-submit"
|
||||
onClick={onSubmit}
|
||||
disabled={!anyDraft || submitting}
|
||||
className="rounded-md bg-primary px-4 py-2 text-sm font-medium text-primary-foreground transition-colors hover:bg-primary/90 disabled:opacity-40"
|
||||
>
|
||||
{submitting ? "Submitting…" : "Submit answers"}
|
||||
</button>
|
||||
{submitError && (
|
||||
<p data-testid="practice-submit-error" className="text-sm text-destructive">
|
||||
{submitError}
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
@@ -186,8 +305,13 @@ export function PracticeExamBody({
|
||||
export function PracticeExam({ initial }: { initial: ProblemSet }) {
|
||||
const [set, setSet] = useState<ProblemSet>(initial);
|
||||
const [drafts, setDrafts] = useState<Record<string, string>>({});
|
||||
const [submitting, setSubmitting] = useState(false);
|
||||
const [submitError, setSubmitError] = useState<string | null>(null);
|
||||
const [submittingId, setSubmittingId] = useState<string | null>(null);
|
||||
const [submitErrors, setSubmitErrors] = useState<Record<string, string>>({});
|
||||
// Which problems' grade rows are LIVE (activity feed) — drives the
|
||||
// per-problem "Grading…" spinners and the status line.
|
||||
const [liveGradeIds, setLiveGradeIds] = useState<Set<string>>(new Set());
|
||||
// True from a successful submit until the feed shows no live grade
|
||||
// rows for this set — keeps the poll alive across refetches.
|
||||
const [gradingLive, setGradingLive] = useState(false);
|
||||
const mounted = useRef(true);
|
||||
useEffect(() => {
|
||||
@@ -199,7 +323,9 @@ export function PracticeExam({ initial }: { initial: ProblemSet }) {
|
||||
|
||||
const refetch = useCallback(async () => {
|
||||
try {
|
||||
const res = await fetch(`/api/practice/sets/${initial.id}`, { cache: "no-store" });
|
||||
const res = await fetch(`/api/practice/sets/${initial.id}`, {
|
||||
cache: "no-store",
|
||||
});
|
||||
if (!res.ok) return;
|
||||
const d = (await res.json()) as { set: ProblemSet };
|
||||
if (mounted.current) setSet(d.set);
|
||||
@@ -210,9 +336,12 @@ export function PracticeExam({ initial }: { initial: ProblemSet }) {
|
||||
|
||||
// The poll: while not ready (generation), or while a grading run is
|
||||
// live for this set with outstanding ungraded answers. The activity
|
||||
// feed is the grading-liveness truth — a dropped grade (no live row)
|
||||
// settles the page instead of polling forever.
|
||||
const pendingGrade = set.problems.some((p) => p.answer && !p.grade);
|
||||
// feed's practice rows are ROW-KEYED (setId:grade:problemId) — the
|
||||
// live rows name exactly which grades are pending; a dropped grade
|
||||
// (no live row) settles the page instead of polling forever.
|
||||
const pendingGrade = set.problems.some(
|
||||
(p) => p.answer !== undefined && !p.grade,
|
||||
);
|
||||
const needsPoll = set.status !== "ready" || (gradingLive && pendingGrade);
|
||||
useEffect(() => {
|
||||
if (!needsPoll) return;
|
||||
@@ -222,9 +351,16 @@ export function PracticeExam({ initial }: { initial: ProblemSet }) {
|
||||
try {
|
||||
const res = await fetch("/api/activity", { cache: "no-store" });
|
||||
if (!res.ok) return;
|
||||
const feed = (await res.json()) as { practice?: Array<{ id: string; kind: string }> };
|
||||
const feed = (await res.json()) as {
|
||||
practice?: Array<{ id: string; kind: string }>;
|
||||
};
|
||||
if (!stopped && mounted.current) {
|
||||
setGradingLive((feed.practice ?? []).some((row) => row.id === initial.id));
|
||||
const prefix = `${initial.id}:grade:`;
|
||||
const pids = (feed.practice ?? [])
|
||||
.filter((row) => row.id.startsWith(prefix))
|
||||
.map((row) => row.id.slice(prefix.length));
|
||||
setLiveGradeIds(new Set(pids));
|
||||
setGradingLive(pids.length > 0);
|
||||
}
|
||||
} catch {
|
||||
// keep the last liveness read
|
||||
@@ -238,45 +374,65 @@ export function PracticeExam({ initial }: { initial: ProblemSet }) {
|
||||
};
|
||||
}, [needsPoll, refetch, initial.id]);
|
||||
|
||||
const submit = useCallback(async () => {
|
||||
if (submitting) return;
|
||||
const answers = set.problems
|
||||
.filter((p) => !p.grade)
|
||||
.map((p) => ({ problemId: p.id, text: drafts[p.id] ?? p.answer ?? "" }))
|
||||
.filter((a) => a.text.trim() !== "");
|
||||
if (answers.length === 0) return;
|
||||
setSubmitting(true);
|
||||
setSubmitError(null);
|
||||
// ONE attempt per problem (owner 2026-09-29): the submit posts THAT
|
||||
// problem's answer alone; the server refuses a second submission
|
||||
// forever after. On success the box locks (answer persisted) and the
|
||||
// reveal shows instantly; the grade arrives via the poll.
|
||||
const submit = useCallback(
|
||||
async (problemId: string) => {
|
||||
if (submittingId !== null) return;
|
||||
const p = set.problems.find((q) => q.id === problemId);
|
||||
if (!p || p.answer !== undefined) return; // done problems never resubmit
|
||||
const text = drafts[problemId] ?? "";
|
||||
if (text.trim() === "") return;
|
||||
setSubmittingId(problemId);
|
||||
setSubmitErrors((prev) => {
|
||||
const next = { ...prev };
|
||||
delete next[problemId];
|
||||
return next;
|
||||
});
|
||||
try {
|
||||
const res = await fetch(`/api/practice/sets/${set.id}/answers`, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ answers }),
|
||||
body: JSON.stringify({ answers: [{ problemId, text }] }),
|
||||
});
|
||||
if (!res.ok) {
|
||||
const body = (await res.json().catch(() => ({}))) as { error?: string };
|
||||
setSubmitError(body.error ?? `Submit failed (${res.status})`);
|
||||
setSubmitting(false);
|
||||
const body = (await res.json().catch(() => ({}))) as {
|
||||
error?: string;
|
||||
};
|
||||
setSubmitErrors((prev) => ({
|
||||
...prev,
|
||||
[problemId]: body.error ?? `Submit failed (${res.status})`,
|
||||
}));
|
||||
setSubmittingId(null);
|
||||
return;
|
||||
}
|
||||
setSubmitting(false);
|
||||
setGradingLive(true); // grades land via the poll
|
||||
setSubmittingId(null);
|
||||
setGradingLive(true); // the grade lands via the poll
|
||||
await refetch();
|
||||
} catch {
|
||||
setSubmitError("Submit failed — network error. Try again.");
|
||||
setSubmitting(false);
|
||||
setSubmitErrors((prev) => ({
|
||||
...prev,
|
||||
[problemId]: "Submit failed — network error. Try again.",
|
||||
}));
|
||||
setSubmittingId(null);
|
||||
}
|
||||
}, [submitting, set, drafts, refetch]);
|
||||
},
|
||||
[submittingId, set, drafts, refetch],
|
||||
);
|
||||
|
||||
return (
|
||||
<PracticeExamBody
|
||||
set={set}
|
||||
drafts={drafts}
|
||||
onDraft={(problemId, text) => setDrafts((prev) => ({ ...prev, [problemId]: text }))}
|
||||
onSubmit={() => void submit()}
|
||||
submitting={submitting}
|
||||
submitError={submitError}
|
||||
grading={gradingLive && pendingGrade}
|
||||
onDraft={(problemId, text) =>
|
||||
setDrafts((prev) => ({ ...prev, [problemId]: text }))
|
||||
}
|
||||
onSubmit={(problemId) => void submit(problemId)}
|
||||
submittingId={submittingId}
|
||||
submitErrors={submitErrors}
|
||||
liveGradeIds={liveGradeIds}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -13,12 +13,17 @@ import { useCallback, useEffect, useState } from "react";
|
||||
import { useRouter } from "next/navigation";
|
||||
import { createPortal } from "react-dom";
|
||||
import { PlusIcon, TrashIcon } from "lucide-react";
|
||||
import { SidebarToggle } from "@/components/sidebar";
|
||||
import type { ProblemSet } from "@/src/core";
|
||||
import { PracticeDialog } from "@/components/practice-dialog";
|
||||
|
||||
/** A set's status word — plain and honest, no invented progress. */
|
||||
export function setStatusWord(set: Pick<ProblemSet, "status">): string {
|
||||
return set.status === "partitioning" ? "partitioning" : set.status === "generating" ? "generating" : set.status;
|
||||
return set.status === "partitioning"
|
||||
? "partitioning"
|
||||
: set.status === "generating"
|
||||
? "generating"
|
||||
: set.status;
|
||||
}
|
||||
|
||||
/** The list's pure body — exported for SSR pins. Rows are buttons
|
||||
@@ -35,30 +40,51 @@ export function PracticeListBody({
|
||||
onNew: () => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="h-dvh overflow-y-auto" data-testid="practice-list-scroll">
|
||||
<div className="mx-auto flex w-full max-w-3xl flex-col gap-3 p-6" data-testid="practice-list">
|
||||
<div className="flex items-center justify-between">
|
||||
<h1 className="font-title text-3xl tracking-tight">Practice</h1>
|
||||
<div className="flex h-dvh">
|
||||
<div className="flex min-w-0 flex-1 flex-col">
|
||||
{/* The top bar (owner, UI round 2026-09-29): the new-session
|
||||
pattern — h-15, lh-drag (the Electron window-drag surface),
|
||||
the sidebar reopen toggle first, "Practice" beside it, the
|
||||
page's one action at the right edge (lh-no-drag — it must
|
||||
stay clickable inside the drag band). */}
|
||||
<header className="lh-drag flex h-15 items-center gap-3 border-b border-border px-3">
|
||||
<SidebarToggle />
|
||||
<div className="min-w-0 flex-1 truncate font-title text-3xl tracking-tight">
|
||||
Practice
|
||||
</div>
|
||||
<button
|
||||
type="button"
|
||||
data-testid="practice-new"
|
||||
onClick={onNew}
|
||||
className="flex items-center gap-1.5 rounded-md border px-3 py-1.5 text-sm transition-colors hover:bg-accent"
|
||||
className="lh-no-drag flex shrink-0 items-center gap-1.5 rounded-md border px-3 py-1.5 text-sm transition-colors hover:bg-accent"
|
||||
>
|
||||
<PlusIcon className="size-3.5" />
|
||||
New problem set
|
||||
</button>
|
||||
</div>
|
||||
|
||||
</header>
|
||||
<div className="relative flex min-h-0 flex-1">
|
||||
<div
|
||||
className="h-full min-w-0 flex-1 overflow-y-auto"
|
||||
data-testid="practice-list-scroll"
|
||||
>
|
||||
<div
|
||||
className="mx-auto flex w-full max-w-3xl flex-col gap-3 p-6"
|
||||
data-testid="practice-list"
|
||||
>
|
||||
{sets.length === 0 && (
|
||||
<div className="rounded-xl border bg-card p-6 text-sm text-muted-foreground" data-testid="practice-empty">
|
||||
<div
|
||||
className="rounded-xl border bg-card p-6 text-sm text-muted-foreground"
|
||||
data-testid="practice-empty"
|
||||
>
|
||||
No problem sets yet — create one from a session.
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex flex-col gap-2">
|
||||
{sets.map((set) => {
|
||||
const ready = set.problems.filter((p) => p.status === "ready").length;
|
||||
const ready = set.problems.filter(
|
||||
(p) => p.status === "ready",
|
||||
).length;
|
||||
const graded = set.problems.filter((p) => p.grade).length;
|
||||
return (
|
||||
<div key={set.id} className="flex items-stretch gap-1">
|
||||
@@ -69,16 +95,23 @@ export function PracticeListBody({
|
||||
className="flex flex-1 items-center justify-between gap-4 rounded-xl border bg-card px-4 py-3 text-left transition-colors hover:bg-accent/50"
|
||||
>
|
||||
<span className="min-w-0">
|
||||
<span className="block truncate font-medium">{set.title}</span>
|
||||
<span className="block truncate font-medium">
|
||||
{set.title}
|
||||
</span>
|
||||
<span className="block text-xs text-muted-foreground">
|
||||
{set.createdAt.slice(0, 10)} · {set.problems.length > 0 ? `${ready}/${set.problems.length} problems` : "no problems yet"}
|
||||
{set.createdAt.slice(0, 10)} ·{" "}
|
||||
{set.problems.length > 0
|
||||
? `${ready}/${set.problems.length} problems`
|
||||
: "no problems yet"}
|
||||
{graded > 0 ? ` · ${graded} graded` : ""}
|
||||
</span>
|
||||
</span>
|
||||
<span
|
||||
data-testid={`practice-status-${set.id}`}
|
||||
className={`shrink-0 text-xs uppercase tracking-wide ${
|
||||
set.status === "error" ? "text-destructive" : "text-muted-foreground"
|
||||
set.status === "error"
|
||||
? "text-destructive"
|
||||
: "text-muted-foreground"
|
||||
}`}
|
||||
>
|
||||
{setStatusWord(set)}
|
||||
@@ -99,6 +132,9 @@ export function PracticeListBody({
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -127,13 +163,19 @@ export function DeleteSetBody({
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
className="w-[32rem] max-w-[calc(100vw-2rem)] rounded-xl border bg-card p-5 shadow-xl"
|
||||
>
|
||||
<h2 className="font-title text-2xl tracking-tight text-destructive">Delete problem set?</h2>
|
||||
<h2 className="font-title text-2xl tracking-tight text-destructive">
|
||||
Delete problem set?
|
||||
</h2>
|
||||
<p className="mt-2 font-mono text-sm">{title}</p>
|
||||
<p className="mt-3 text-sm text-muted-foreground">
|
||||
The set and its problems are removed. Nothing in the graph or your sessions changes.
|
||||
The set and its problems are removed. Nothing in the graph or your
|
||||
sessions changes.
|
||||
</p>
|
||||
{failure && (
|
||||
<p data-testid="delete-set-error" className="mt-3 text-sm text-destructive">
|
||||
<p
|
||||
data-testid="delete-set-error"
|
||||
className="mt-3 text-sm text-destructive"
|
||||
>
|
||||
{failure}
|
||||
</p>
|
||||
)}
|
||||
@@ -160,7 +202,11 @@ export function DeleteSetBody({
|
||||
);
|
||||
}
|
||||
|
||||
export function PracticeListPage({ initialSets }: { initialSets: ProblemSet[] }) {
|
||||
export function PracticeListPage({
|
||||
initialSets,
|
||||
}: {
|
||||
initialSets: ProblemSet[];
|
||||
}) {
|
||||
const router = useRouter();
|
||||
const [sets, setSets] = useState<ProblemSet[]>(initialSets);
|
||||
const [creating, setCreating] = useState(false);
|
||||
@@ -181,7 +227,9 @@ export function PracticeListPage({ initialSets }: { initialSets: ProblemSet[] })
|
||||
|
||||
// The reveal poll: while any set is partitioning/generating, statuses
|
||||
// live-update; a settled list goes quiet.
|
||||
const anyRunning = sets.some((s) => s.status === "partitioning" || s.status === "generating");
|
||||
const anyRunning = sets.some(
|
||||
(s) => s.status === "partitioning" || s.status === "generating",
|
||||
);
|
||||
useEffect(() => {
|
||||
if (!anyRunning) return;
|
||||
const t = setInterval(() => void refresh(), 2000);
|
||||
@@ -193,7 +241,9 @@ export function PracticeListPage({ initialSets }: { initialSets: ProblemSet[] })
|
||||
setDeleteBusy(true);
|
||||
setDeleteFailure(null);
|
||||
try {
|
||||
const res = await fetch(`/api/practice/sets/${deleting.id}`, { method: "DELETE" });
|
||||
const res = await fetch(`/api/practice/sets/${deleting.id}`, {
|
||||
method: "DELETE",
|
||||
});
|
||||
if (!res.ok) {
|
||||
const body = (await res.json().catch(() => ({}))) as { error?: string };
|
||||
setDeleteFailure(body.error ?? `Delete failed (${res.status})`);
|
||||
|
||||
@@ -152,7 +152,7 @@ flowchart TB
|
||||
- `contracts/jobs.ts` — the ack-and-steer protocol: job records, steer message builders, deterministic steer ids, usage accumulation
|
||||
- `contracts/probe.ts` — probe assessment + the shared zod schema
|
||||
- `contracts/models.ts` — client-safe model-catalog shapes
|
||||
- `contracts/practice.ts` — problem sets, reasoning chains, params, grade verdicts
|
||||
- `contracts/practice.ts` — problem sets, reasoning chains, params, grade scores
|
||||
|
||||
---
|
||||
|
||||
@@ -234,7 +234,7 @@ flowchart TB
|
||||
end
|
||||
|
||||
subgraph PRAC["PRACTICE — problem sets (owns data/practice)"]
|
||||
PPART["practice pipeline — partitioner → one generator per problem · one grader per answer<br/>(feedback-only; never steers the teacher)"]
|
||||
PPART["practice pipeline — partitioner → one generator per problem · one grader per answer<br/>(score-only; never steers the teacher)"]
|
||||
PDATA["problem sets — data/practice/*.json"]
|
||||
end
|
||||
|
||||
@@ -368,7 +368,7 @@ flowchart TB
|
||||
|
||||
## 7. Practice — strengthening, not assessment (owner ruling 2026-09-29)
|
||||
|
||||
Practice turns the graph into problem sets the learner reasons through out loud. The purpose ruling first: this is **strengthening, not assessment**. Grades are feedback for the learner — a verdict, an explanation, a look at the model solution — and nothing else. Grading writes nothing to the graph, no understanding levels exist anywhere, and the vision doc's exam machinery (scores, streaks, spaced repetition over failure rates) stays parked. A set exercises what the learner already met; it certifies nothing.
|
||||
Practice turns the graph into problem sets the learner reasons through out loud. The purpose ruling first: this is **strengthening, not assessment**. Grades are feedback for the learner — a score, a brief justification, a look at the model solution — and nothing else. Grading writes nothing to the graph, no understanding levels exist anywhere, and the vision doc's exam machinery (scores, streaks, spaced repetition over failure rates) stays parked. A set exercises what the learner already met; it certifies nothing.
|
||||
|
||||
**The chain model.** Practice sees a node selection as **reasoning chains**: the selection's `follows`-edges grouped by conclusion (`chainsForSelection`, `src/core/practice.ts`). A chain is `premises → conclusion` — one per conclusion node, a singleton for an edge-less node; premise-only nodes belong to no chain. The grouping is a **bijection with the follows-edge set** (`chainsToGraph` is its exact inverse, round-trip test-pinned) — no chain exists that isn't the edges, no edge that isn't in a chain. **Supports edges are invisible to practice** (owner ruling: supports are grounding, not reasoning steps). The partition must cover every chain **exactly once** — a true partition; duplicates fail validation exactly like gaps (owner amendment 2026-09-29: variety accumulates ACROSS sets, never within one — the at-least-once doctrine is dead). `chainCoverage` is the soundness check: only `missing` fails a partition, `duplicated` is informational.
|
||||
|
||||
@@ -378,11 +378,11 @@ Practice turns the graph into problem sets the learner reasons through out loud.
|
||||
|
||||
**Two-stage generation.** One **partitioner** agent reads the chains plus the three NAMED-LEVEL parameters and submits the true partition; each partition becomes one problem's target. Then **one generator agent per problem**: its prompt carries the problem-engineering framework verbatim (the owner's doctrine — the three distances, the search-to-solution axis, the rung ladder, reverse/discriminate/probe-the-premises, the standing prohibitions, the formatting doctrine — full Markdown, LaTeX math, fenced code, tables/lists; prose-only renderings of structured content are a defect), **YOUR TARGET** (the assigned chains echoed verbatim — the echo law: the submit's chain set must equal the assignment exactly), and **THE FULL SET** (the whole selection: outside propositions are supporting material the generator may draw on with minimal use and maximal focus; the never-announce law covers ALL propositions). The submit is commit-before-pose — statement, solution, and rubric arrive together (the same sycophancy mitigation the quiz/explain mechanic uses); a problem never exists without its pre-written answer. Generators run in parallel (a process-wide 3-permit semaphore shared with grading) with per-problem failure isolation: one bad problem errors alone; the set still reaches `ready`. **Dedup across sets**: each generator also sees the problems of the ≤5 most recent sets sharing a node, so situations don't repeat between sets; within a set, parallel generators can't see each other — accepted for V1 (owner ruling).
|
||||
|
||||
**Grading is feedback-only.** Submitting answers fires one **grader** job per answered problem: the grader sees the problem, the propositions its chains touch, the pre-written solution and rubric, and the learner's answer — never the full set (minimal, by ruling). It returns a verdict (`solid`/`partial`/`missed`) and feedback that names the failed recall and teaches the boundary. The verdict lands on the problem and the UI reveals it with the solution and rubric; the answer locks. Nothing reaches the graph, no level, no count, anywhere. A grader failure leaves the answer gradeless and re-submittable.
|
||||
**Grading is score-only (owner redesign, 2026-09-29).** ONE ATTEMPT: each problem has its own submit button; once submitted the answer persists and the problem is done — the route refuses re-submission forever after. The submit reveals INSTANTLY (no waiting on the grader): the answer locks read-only, the model solution shows, the rubric sits collapsed. Grading fires one **grader** job per answered problem — the grader sees the problem, the propositions its chains touch, the pre-written solution and rubric, and the learner's answer, never the full set (minimal, by ruling) — and returns an integer **score 0–10 plus one or two sentences of reasoning** (which rubric points were met or missed, which proposition failed to come to mind). The score lands on the problem via the page's 2s poll and updates live; grades never serialize set-wide (submitting B while A grades is fine — the shared pool bounds parallelism). Nothing reaches the graph, no level, no count, anywhere. A dropped grade leaves the answer standing with an honest “grade interrupted” line — one attempt, no re-submission.
|
||||
|
||||
**The jobs pattern — registry, not JobRecords.** Practice runs are NOT Workbench jobs: no `JobStore` records, no steer messages, nothing ever wakes the teacher — a practice run isn't chat-scoped (a set outlives its birth session and has no lesson to re-enter; its result is itself, persisted in its own store and read by its own page). What it shares with the Workbench is the observability surface: a `globalThis` run registry (the module-instance idiom — `__lhPracticeRuns`) with PER-PROBLEM ROWS (owner ruling, 2026-09-29 — matching the extraction per-chunk pattern): `setId:partition` while the partitioner runs ("Partitioning — title"), then one `setId:gen:problemId` per problem ("queued"→"generating", label "Problem i/N — title"), and one `setId:grade:problemId` per graded answer ("queued"→"grading", "Grading i/N — title") — each row its OWN abort controller and its OWN usage accumulation; the row key IS the feed id (the dock dup-key law) and the abort target. ABORT ROUTES BY ROW SHAPE: a gen-row abort kills THAT problem only (siblings continue; the set still settles `ready` with one errored problem), a partition-row abort ends the whole run, a grade-row abort drops that grade (answer kept, re-submittable); set-level abort (delete) composes from row aborts plus the synchronous persisted marks. Model calls — partitioner included — run under the SHARED background-LLM semaphore (`bg-llm-semaphore.ts`): one process-wide pool of 3 permits for ALL background LLM jobs, extraction chunks included (total background LLM concurrency bounded at 3, whoever calls). A boot sweep marks orphaned `partitioning`/`generating` sets as `error "Interrupted (server restarted)"` — once per process, and only when the set's `updatedAt` heartbeat is >120s stale (a live run saves at every transition and is structurally never stamped; the guard trio closed a live-stamping bug from the first live test) — the job runner's crash-honesty at set grain. Store writes serialize behind a per-set mutex; the pipeline is the sole writer of `data/practice`.
|
||||
**The jobs pattern — registry, not JobRecords.** Practice runs are NOT Workbench jobs: no `JobStore` records, no steer messages, nothing ever wakes the teacher — a practice run isn't chat-scoped (a set outlives its birth session and has no lesson to re-enter; its result is itself, persisted in its own store and read by its own page). What it shares with the Workbench is the observability surface: a `globalThis` run registry (the module-instance idiom — `__lhPracticeRuns`) with PER-PROBLEM ROWS (owner ruling, 2026-09-29 — matching the extraction per-chunk pattern): `setId:partition` while the partitioner runs ("Partitioning — title"), then one `setId:gen:problemId` per problem ("queued"→"generating", label "Problem i/N — title"), and one `setId:grade:problemId` per graded answer ("queued"→"grading", "Grading i/N — title") — each row its OWN abort controller and its OWN usage accumulation; the row key IS the feed id (the dock dup-key law) and the abort target. ABORT ROUTES BY ROW SHAPE: a gen-row abort kills THAT problem only (siblings continue; the set still settles `ready` with one errored problem), a partition-row abort ends the whole run, a grade-row abort drops that grade (answer kept — one attempt, the problem stays done); set-level abort (delete) composes from row aborts plus the synchronous persisted marks. Model calls — partitioner included — run under the SHARED background-LLM semaphore (`bg-llm-semaphore.ts`): one process-wide pool of 3 permits for ALL background LLM jobs, extraction chunks included (total background LLM concurrency bounded at 3, whoever calls). A boot sweep marks orphaned `partitioning`/`generating` sets as `error "Interrupted (server restarted)"` — once per process, and only when the set's `updatedAt` heartbeat is >120s stale (a live run's 30s FILE watchdog keeps the heartbeat fresh even across dev's per-worker processes — the registry is not shared between them; the guard trio closed a live-stamping bug from the first live test) — the job runner's crash-honesty at set grain. Store writes serialize behind a per-set mutex; the pipeline is the sole writer of `data/practice`.
|
||||
|
||||
**API + UI.** Creation is GRAPH-DOMAIN, not chat-domain (owner ruling 2026-09-29: "problem sets always take in nodes — graph objects"): `POST /api/practice/sets` takes `{nodeIds, params, title?}` ONLY — no chatId, no clusterId; the UI resolves a picked cluster to its member node ids and passes the cluster's label as the title. `GET /api/graph/clusters` serves the picker: the birth-session partition itself (`birthSessionPartition`, same ordering — size desc, ties by id) — EVERY group regardless of size (owner ruling, same evening: MIN_CLUSTER_SIZE is a hull-drawing rule; practice availability must not inherit a visual threshold), each as `{key, kind: "chat"|"notes", label, nodeIds, nodeCount}`; a cluster whose naming record is gone keeps `label: null` — honest, never guessed. 202 `{id}` with generation fire-and-forget; 400 empty selection (empty intersection with live nodes — the vanished-nodes edge) / bad params; 503 model/key refusal. `GET /api/practice/sets` · `GET/DELETE /api/practice/sets/[id]` (delete aborts live runs first) · `POST /api/practice/sets/[id]/answers` (202; 409 when not ready). Surfaces: `/practice` (the list — status/counts rows, delete confirm, the new-set dialog with the named-level controls (a 5-detent ladder slider, a 3-detent cue slider, a Theory|Application toggle — each showing the level's name + definition) and the cluster picker — the ONLY creation entry for now; the graph-view cluster entry comes later with the free-form selection UI) and `/practice/[id]` (the exam page — a levels line in the header, numbered problems, markdown statements with math healing at the display seam, a textarea + focus-gated dictation bar per problem, one Submit per set, the post-grade reveal). Entry: the sidebar's Practice nav row.
|
||||
**API + UI.** Creation is GRAPH-DOMAIN, not chat-domain (owner ruling 2026-09-29: "problem sets always take in nodes — graph objects"): `POST /api/practice/sets` takes `{nodeIds, params, title?}` ONLY — no chatId, no clusterId; the UI resolves a picked cluster to its member node ids and passes the cluster's label as the title. `GET /api/graph/clusters` serves the picker: the birth-session partition itself (`birthSessionPartition`, same ordering — size desc, ties by id) — EVERY group regardless of size (owner ruling, same evening: MIN_CLUSTER_SIZE is a hull-drawing rule; practice availability must not inherit a visual threshold), each as `{key, kind: "chat"|"notes", label, nodeIds, nodeCount}`; a cluster whose naming record is gone keeps `label: null` — honest, never guessed. 202 `{id}` with generation fire-and-forget; 400 empty selection (empty intersection with live nodes — the vanished-nodes edge) / bad params; 503 model/key refusal. `GET /api/practice/sets` · `GET/DELETE /api/practice/sets/[id]` (delete aborts live runs first) · `POST /api/practice/sets/[id]/answers` (202; 409 when not ready; 400 re-answering a submitted problem — one attempt). Surfaces: `/practice` (the list — the chat-pattern top bar with the sidebar reopen toggle and the New action at the right edge, status/counts rows, delete confirm, the new-set dialog with the named-level controls (a 5-detent ladder slider, a 3-detent cue slider, a Theory|Application toggle — each showing the level's name + definition) and the cluster picker — the ONLY creation entry for now; the graph-view cluster entry comes later with the free-form selection UI) and `/practice/[id]` (the exam page — the chat-pattern top bar, a levels + status line, numbered problems, markdown statements with math healing at the display seam, a textarea + focus-gated dictation bar per problem with its OWN Submit button — one attempt per problem, instant solution/rubric reveal, the score landing live). Entry: the sidebar's Practice nav row.
|
||||
|
||||
**Model wiring.** Three roles — `partitioner`, `problemGenerator`, `grader` — all pinned to the **strong** slot (quality-critical, one settings edit from cheap) and in `BACKGROUND_MODEL_ROLES` (per-role overrides). The stage agents are single-submit loops on the shared `runSubmitAgent` seam (`submit-agent.ts`, extracted from extraction for exactly this): a `validate` hook turns a bad submit into a tool-input error the model retries against, within `maxTurns 4`; exhaustion fails the stage honestly (the set says so, the chains re-run cleanly on a new set).
|
||||
|
||||
|
||||
Vendored
+2
-2
@@ -1,7 +1,7 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
import "./.next/types/routes.d.ts";
|
||||
import "./.next/types/root-params.d.ts";
|
||||
import "./.next/dev/types/routes.d.ts";
|
||||
import "./.next/dev/types/root-params.d.ts";
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information.
|
||||
|
||||
@@ -34,10 +34,12 @@
|
||||
* background failure paths settle the persisted record (never an
|
||||
* unhandled rejection); aborts mark "Aborted".
|
||||
*
|
||||
* GRADING FAILURE DISCIPLINE: an aborted or failed grade leaves the
|
||||
* answer in place, the grade absent — the problem is re-submittable
|
||||
* later (a new grading job). A second submitAnswers while grading is in
|
||||
* flight is rejected (single live grading run per set).
|
||||
* ONE ATTEMPT PER PROBLEM (owner ruling 2026-09-29): a problem accepts
|
||||
* a single submission — once `answer` is set the problem is done, the
|
||||
* route refuses re-submission. Grades are per-problem: submitting
|
||||
* another problem while one grades is fine (the shared LLM pool bounds
|
||||
* parallelism); only the SAME problem's live grade row blocks (which
|
||||
* the one-attempt rule already prevents client-side).
|
||||
*/
|
||||
|
||||
import type { GraphStore, PracticeStore } from "../../core/ports.ts";
|
||||
@@ -476,8 +478,10 @@ export function createPracticePipeline(deps: PracticePipelineDeps) {
|
||||
answers: Array<{ problemId: string; text: string }>,
|
||||
): Promise<{ set: ProblemSet; done: Promise<void> }> {
|
||||
agents.refuseIfStagesUnready(); // grading needs its engine — refuse BEFORE saving answers
|
||||
for (const key of practiceRuns().keys()) {
|
||||
if (key.startsWith(`${setId}:grade:`)) throw new BadAnswersError("a grading run is already in flight for this set");
|
||||
for (const a of answers) {
|
||||
if (practiceRuns().has(`${setId}:grade:${(a as { problemId?: unknown }).problemId}`)) {
|
||||
throw new BadAnswersError("this problem's grade is already in flight");
|
||||
}
|
||||
}
|
||||
const set = await withSetLock(setId, () => {
|
||||
const cur = deps.practiceStore.getSet(setId);
|
||||
@@ -488,7 +492,9 @@ export function createPracticePipeline(deps: PracticePipelineDeps) {
|
||||
if (typeof a.problemId !== "string" || !ids.has(a.problemId)) throw new BadAnswersError(`unknown problem id`);
|
||||
if (typeof a.text !== "string" || a.text.trim() === "") throw new BadAnswersError("answers must be non-empty text");
|
||||
const p = cur.problems.find((q) => q.id === a.problemId)!;
|
||||
if (p.grade) throw new BadAnswersError(`problem ${a.problemId} is already graded`);
|
||||
// ONE attempt (owner 2026-09-29): an existing answer ends the
|
||||
// problem — graded or not, it is never re-submitted.
|
||||
if (p.answer !== undefined) throw new BadAnswersError(`problem ${a.problemId} was already answered`);
|
||||
}
|
||||
for (const a of answers) {
|
||||
const p = cur.problems.find((q) => q.id === a.problemId)!;
|
||||
@@ -550,12 +556,13 @@ export function createPracticePipeline(deps: PracticePipelineDeps) {
|
||||
if (!cur) return;
|
||||
const p = cur.problems.find((q) => q.id === problemId);
|
||||
if (!p || p.grade) return;
|
||||
p.grade = { verdict: graded.verdict, feedback: graded.feedback, gradedAt: new Date().toISOString() };
|
||||
p.grade = { score: graded.score, reasoning: graded.reasoning, gradedAt: new Date().toISOString() };
|
||||
deps.practiceStore.saveSet(cur);
|
||||
});
|
||||
});
|
||||
} catch (err) {
|
||||
// Grading failure: the grade simply stays absent (re-submittable).
|
||||
// Grading failure: the grade simply stays absent (the answer
|
||||
// stands — one attempt; the UI shows "grade interrupted").
|
||||
console.error(`[practice] grading failed for ${problemId}:`, isAbortLike(err) ? "aborted" : messageOf(err));
|
||||
} finally {
|
||||
practiceRuns().delete(row.key);
|
||||
|
||||
@@ -211,7 +211,7 @@ PARTITION DOCTRINE:
|
||||
|
||||
OUTPUT — your entire deliverable is ONE submit_partition call: "partitions", a list of { chains: [ids] }, one entry per problem, together containing every input chain exactly once. No empty entries; no unknown ids; no problem without a real target relationship.`;
|
||||
|
||||
export const GRADER_INSTRUCTIONS = `You are the grading stage of a learning application's practice system. One practice problem — built to strengthen the learner's recall of specific propositions — has been answered. Your job is FEEDBACK ONLY: verdict + what actually mattered. You are not assessing a person; you are coaching a recall.
|
||||
export const GRADER_INSTRUCTIONS = `You are the grading stage of a learning application's practice system. One practice problem — built to strengthen the learner's recall of specific propositions — has been answered. Your job is small and fast: an integer score and a one-or-two-sentence justification. You are not assessing a person; you are scoring a recall.
|
||||
|
||||
You receive: the problem's statement; its pre-written MODEL SOLUTION and GRADING RUBRIC (committed before the learner ever saw the problem — they are the reference, not you); the propositions (names + statements) the problem was built to exercise; and the learner's ANSWER — often dictated thinking-out-loud prose. Extract the reasoning from the prose; grade the substance, never the form. Do not demand the solution's exact wording — the rubric names what actually matters; nothing else is graded.
|
||||
|
||||
@@ -219,18 +219,14 @@ JUDGE AGAINST THE REFERENCE:
|
||||
- Work the rubric points: which are met, which are missed, which are half-met in the answer's own terms.
|
||||
- Where the answer diverges from the model solution, ask whether the divergence is a different-but-valid route (the rubric decides) or a wrong turn.
|
||||
|
||||
NAME THE FAILED RECALL: when something is missed, identify WHICH proposition failed to come to mind — or came to mind in a wrong or incomplete form. That is the actionable diagnosis; say it plainly.
|
||||
THE SCORE (one integer, 0–10):
|
||||
- 9–10: the targeted recall(s) happened — the answer meets the rubric's essentials. Peripheral gaps do not lower it.
|
||||
- 5–8: some of the target came to mind; the answer is materially incomplete or partly wrong.
|
||||
- 0–4: the targeted proposition(s) did not come to mind — the answer is empty or on the wrong track.
|
||||
|
||||
TEACH THE BOUNDARY: feedback should locate the boundary — what would have made the answer right, where the wrong path breaks and why. Not "incorrect"; the reason it is incorrect, in one or two tight paragraphs.
|
||||
The reasoning is plain prose, one or two sentences: which rubric points were met or missed and, when something failed, WHICH proposition failed to come to mind. Mathematics is LaTeX — inline as \\(…\\), display as \\[…\\] (a bare $ is literal currency); no headers, no lists — minimal is the standard.
|
||||
|
||||
THE VERDICT (exactly one):
|
||||
- "solid": the targeted recall(s) happened — the answer meets the rubric's essentials. Peripheral gaps do not lower it.
|
||||
- "partial": some of the target came to mind; the answer is materially incomplete or partly wrong.
|
||||
- "missed": the targeted proposition(s) did not come to mind — the answer is empty or on the wrong track.
|
||||
|
||||
Feedback renders as full Markdown. Mathematics is always LaTeX — inline as \\(…\\), display as \\[…\\] on its own lines (a bare $ is literal currency). Inside math, LaTeX commands for symbols, not unicode stand-ins. Code in fenced blocks with a language tag, identifiers inline as \`code\`. Lists and tables wherever they carry structure; no headers — feedback is short-form, structure it with paragraphs and bold labels. Name the failed recall and teach the boundary in properly formatted prose, not plain text.
|
||||
|
||||
OUTPUT — your entire deliverable is ONE submit_grade call: "verdict" and "feedback" (non-empty; the boundary-teaching text above).`;
|
||||
OUTPUT — your entire deliverable is ONE submit_grade call: "score" (integer 0–10) and "reasoning" (non-empty, brief).`;
|
||||
|
||||
// ─── schemas + fills ─────────────────────────────────────────────────────────
|
||||
|
||||
@@ -250,8 +246,8 @@ const problemSchema = Type.Object({
|
||||
type ProblemSubmit = Static<typeof problemSchema>;
|
||||
|
||||
const gradeSchema = Type.Object({
|
||||
verdict: Type.Union([Type.Literal("solid"), Type.Literal("partial"), Type.Literal("missed")]),
|
||||
feedback: Type.String({ minLength: 1 }),
|
||||
score: Type.Integer({ minimum: 0, maximum: 10 }),
|
||||
reasoning: Type.String({ minLength: 1 }),
|
||||
});
|
||||
type GradeSubmit = Static<typeof gradeSchema>;
|
||||
|
||||
@@ -364,8 +360,8 @@ export interface GraderInput {
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
export interface GraderOutput {
|
||||
verdict: "solid" | "partial" | "missed";
|
||||
feedback: string;
|
||||
score: number;
|
||||
reasoning: string;
|
||||
log: PracticeStepLog;
|
||||
}
|
||||
|
||||
@@ -499,7 +495,7 @@ export function createPracticeAgents(deps: PracticeModels) {
|
||||
const grade = await runSubmitAgent({
|
||||
role: "grader",
|
||||
toolName: "submit_grade",
|
||||
toolDescription: "Submit the grade: verdict (solid | partial | missed) and non-empty feedback.",
|
||||
toolDescription: "Submit the grade: an integer score 0-10 and brief non-empty reasoning.",
|
||||
parameters: gradeSchema,
|
||||
fill: (raw) => raw,
|
||||
systemPrompt: GRADER_INSTRUCTIONS,
|
||||
@@ -509,7 +505,7 @@ export function createPracticeAgents(deps: PracticeModels) {
|
||||
onStep: stepLogger(log),
|
||||
neverSubmitted: "the grader never submitted a grade",
|
||||
});
|
||||
return { verdict: grade.verdict, feedback: grade.feedback, log };
|
||||
return { score: grade.score, reasoning: grade.reasoning, log };
|
||||
}
|
||||
|
||||
return { refuseIfStagesUnready, runPartitioner, runProblemGenerator, runGrader };
|
||||
|
||||
@@ -113,16 +113,16 @@ export type ProblemSetStatus = "partitioning" | "generating" | "ready" | "error"
|
||||
* generator submits. */
|
||||
export type ProblemStatus = "generating" | "ready" | "error";
|
||||
|
||||
/** The grader's three-level verdict (feedback-only; nothing derives from
|
||||
* it — no understanding writes, owner 2026-09-29). */
|
||||
export type GradeVerdict = "solid" | "partial" | "missed";
|
||||
|
||||
/** The grading job's deliverable, stored on the problem record. */
|
||||
/** The grading job's deliverable (owner, 2026-09-29 — one attempt per
|
||||
* problem; the grader scores, it does not coach in prose): an integer
|
||||
* score against the pre-written rubric plus a minimal justification.
|
||||
* Nothing derives from it — no understanding writes. */
|
||||
export interface ProblemGrade {
|
||||
verdict: GradeVerdict;
|
||||
/** What actually mattered: the failed recall named, the boundary
|
||||
* taught — against the pre-written solution + rubric. */
|
||||
feedback: string;
|
||||
/** 0–10, integer. */
|
||||
score: number;
|
||||
/** Why this score — one or two sentences: which rubric points were
|
||||
* met or missed, and which proposition failed to come to mind. */
|
||||
reasoning: string;
|
||||
gradedAt: string;
|
||||
}
|
||||
|
||||
@@ -151,7 +151,7 @@ export interface Problem {
|
||||
/** The learner's answer, once submitted (dictation-first — thinking
|
||||
* out loud is the primary answering mode). */
|
||||
answer?: string;
|
||||
/** The grading job's verdict, once it lands. */
|
||||
/** The grader's score + reasoning, once it lands. */
|
||||
grade?: ProblemGrade;
|
||||
}
|
||||
|
||||
|
||||
+16
-14
@@ -73,7 +73,21 @@ const isPropositions = (v: unknown): boolean => {
|
||||
|
||||
const SET_STATUSES = new Set(["partitioning", "generating", "ready", "error"]);
|
||||
const PROBLEM_STATUSES = new Set(["generating", "ready", "error"]);
|
||||
const VERDICTS = new Set(["solid", "partial", "missed"]);
|
||||
function isGrade(g: unknown): boolean {
|
||||
if (typeof g !== "object" || g === null) return false;
|
||||
const v = g as Record<string, unknown>;
|
||||
// Integer 0–10 + non-empty reasoning (one-attempt redesign, owner
|
||||
// 2026-09-29: the grader scores; verdicts + prose feedback retired).
|
||||
return (
|
||||
typeof v.score === "number" &&
|
||||
Number.isInteger(v.score) &&
|
||||
v.score >= 0 &&
|
||||
v.score <= 10 &&
|
||||
typeof v.reasoning === "string" &&
|
||||
v.reasoning.length > 0 &&
|
||||
typeof v.gradedAt === "string"
|
||||
);
|
||||
}
|
||||
|
||||
const isProblem = (v: unknown): boolean => {
|
||||
if (typeof v !== "object" || v === null) return false;
|
||||
@@ -83,19 +97,7 @@ const isProblem = (v: unknown): boolean => {
|
||||
if (typeof p.status !== "string" || !PROBLEM_STATUSES.has(p.status)) return false;
|
||||
if (p.error !== undefined && typeof p.error !== "string") return false;
|
||||
if (p.answer !== undefined && typeof p.answer !== "string") return false;
|
||||
if (p.grade !== undefined) {
|
||||
const g = p.grade as Record<string, unknown>;
|
||||
if (
|
||||
typeof g !== "object" ||
|
||||
g === null ||
|
||||
typeof g.verdict !== "string" ||
|
||||
!VERDICTS.has(g.verdict) ||
|
||||
typeof g.feedback !== "string" ||
|
||||
typeof g.gradedAt !== "string"
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (p.grade !== undefined && !isGrade(p.grade)) return false;
|
||||
return true;
|
||||
};
|
||||
|
||||
|
||||
@@ -233,27 +233,27 @@ describe("runGrader", () => {
|
||||
answer: "Well, if I'm arguing at all, I must own myself…",
|
||||
};
|
||||
|
||||
it("returns the verdict + feedback with its step log", async () => {
|
||||
it("returns the score + reasoning with its step log", async () => {
|
||||
const agents = createPracticeAgents({
|
||||
grader: submitModel("submit_grade", { verdict: "solid", feedback: "The recall happened." }),
|
||||
grader: submitModel("submit_grade", { score: 9, reasoning: "The recall happened." }),
|
||||
});
|
||||
const out = await agents.runGrader(graderInput);
|
||||
assert.equal(out.verdict, "solid");
|
||||
assert.equal(out.feedback, "The recall happened.");
|
||||
assert.equal(out.score, 9);
|
||||
assert.equal(out.reasoning, "The recall happened.");
|
||||
assert.ok(out.log.model);
|
||||
assert.equal(out.log.steps.length, 1);
|
||||
});
|
||||
|
||||
it("schema-rejects an off-union verdict and retries", async () => {
|
||||
it("schema-rejects an out-of-range score and retries", async () => {
|
||||
const agents = createPracticeAgents({
|
||||
grader: retryModel(
|
||||
"submit_grade",
|
||||
{ verdict: "excellent", feedback: "…" },
|
||||
{ verdict: "partial", feedback: "Half the target came to mind." },
|
||||
{ score: 12, reasoning: "…" },
|
||||
{ score: 6, reasoning: "Half the target came to mind." },
|
||||
),
|
||||
});
|
||||
const out = await agents.runGrader(graderInput);
|
||||
assert.equal(out.verdict, "partial");
|
||||
assert.equal(out.score, 6);
|
||||
assert.equal(out.log.steps.length, 2);
|
||||
});
|
||||
|
||||
@@ -312,7 +312,7 @@ describe("formatting doctrine pins", () => {
|
||||
it("GRADER_INSTRUCTIONS carries the formatting paragraph before the OUTPUT section", () => {
|
||||
assert.ok(
|
||||
GRADER_INSTRUCTIONS.includes(
|
||||
"Feedback renders as full Markdown. Mathematics is always LaTeX — inline as \\(…\\), display as \\[…\\] on its own lines (a bare $ is literal currency). Inside math, LaTeX commands for symbols, not unicode stand-ins. Code in fenced blocks with a language tag, identifiers inline as `code`. Lists and tables wherever they carry structure; no headers — feedback is short-form, structure it with paragraphs and bold labels. Name the failed recall and teach the boundary in properly formatted prose, not plain text.",
|
||||
"The reasoning is plain prose, one or two sentences: which rubric points were met or missed and, when something failed, WHICH proposition failed to come to mind. Mathematics is LaTeX — inline as \\(…\\), display as \\[…\\] (a bare $ is literal currency); no headers, no lists — minimal is the standard.",
|
||||
),
|
||||
"the exact grader paragraph",
|
||||
);
|
||||
|
||||
@@ -152,7 +152,7 @@ function echoGenerator(count = 24, failTargets: string[] = []): PiEngine {
|
||||
return fauxEngine(Array.from({ length: count }, () => factory));
|
||||
}
|
||||
|
||||
/** A grader whose verdict encodes in the answer text ("solid" → solid). */
|
||||
/** A grader whose score encodes in the answer text ("solid" → 9, else 6). */
|
||||
function echoGrader(count: number): PiEngine {
|
||||
const factory: FauxResponseStep = (context) => {
|
||||
const answer = lastUserText(context as never).split("THE LEARNER'S ANSWER\n")[1] ?? "";
|
||||
@@ -161,8 +161,8 @@ function echoGrader(count: number): PiEngine {
|
||||
fauxToolCall(
|
||||
"submit_grade",
|
||||
{
|
||||
verdict: answer.includes("solid") ? "solid" : "partial",
|
||||
feedback: "The boundary.",
|
||||
score: answer.includes("solid") ? 9 : 6,
|
||||
reasoning: "Which rubric points were met.",
|
||||
} as ToolCall["arguments"],
|
||||
{ id: "grade-call" },
|
||||
),
|
||||
@@ -521,7 +521,7 @@ describe("createProblemSet (generation)", () => {
|
||||
const gate = new Promise<void>((r) => (release = r));
|
||||
// hold both grades at the model seam
|
||||
const holdSteps: FauxResponseStep[] = Array.from({ length: 2 }, () => () =>
|
||||
gate.then(() => fauxAssistantMessage([fauxToolCall("submit_grade", { verdict: "solid", feedback: "yes" } as ToolCall["arguments"], { id: "gr" })], { stopReason: "toolUse" })),
|
||||
gate.then(() => fauxAssistantMessage([fauxToolCall("submit_grade", { score: 9, reasoning: "yes" } as ToolCall["arguments"], { id: "gr" })], { stopReason: "toolUse" })),
|
||||
);
|
||||
const f = fixture({
|
||||
partitioner: partitionModel([{ chains: ["b"] }, { chains: ["c"] }]),
|
||||
@@ -554,7 +554,7 @@ describe("createProblemSet (generation)", () => {
|
||||
await sub.done;
|
||||
assert.equal(practiceStatuses().length, 0);
|
||||
const graded = f.practice.store.getSet(set.id)!;
|
||||
assert.ok(graded.problems.every((p) => p.grade?.verdict === "solid"));
|
||||
assert.ok(graded.problems.every((p) => p.grade?.score === 9));
|
||||
});
|
||||
|
||||
it("abort ONE gen row: that problem dies alone, siblings complete, set reaches ready with one errored", async () => {
|
||||
@@ -600,7 +600,7 @@ describe("createProblemSet (generation)", () => {
|
||||
assert.equal(practiceStatuses().length, 0);
|
||||
});
|
||||
|
||||
it("abort ONE grade row (queued): that grade drops (answer kept, re-submittable), sibling grade lands", async () => {
|
||||
it("abort ONE grade row (queued): that grade drops (answer kept, one attempt — no re-submission), sibling grade lands", async () => {
|
||||
const f = fixture({
|
||||
partitioner: partitionModel([{ chains: ["b"] }, { chains: ["c"] }]),
|
||||
problemGenerator: echoGenerator(2),
|
||||
@@ -645,14 +645,18 @@ describe("createProblemSet (generation)", () => {
|
||||
assert.equal(dropped.grade, undefined, "the aborted grade is absent");
|
||||
assert.equal(dropped.answer, "first", "the answer is kept");
|
||||
const other = graded.problems.find((p) => p.id !== victimId)!;
|
||||
assert.equal(other.grade?.verdict, "partial", "the sibling grade landed (echoGrader: no 'solid' in its text)");
|
||||
assert.equal(other.grade?.score, 6, "the sibling grade landed (echoGrader: no 'solid' in its text)");
|
||||
assert.equal(practiceStatuses().length, 0);
|
||||
|
||||
// re-submittable: the dropped answer grades on a second submit
|
||||
const resub = await pipeline.submitAnswers(set.id, [{ problemId: victimId, text: "retry answer" }]);
|
||||
await resub.done;
|
||||
const retried = f.practice.store.getSet(set.id)!;
|
||||
assert.equal(retried.problems.find((p) => p.id === victimId)!.grade?.verdict, "partial", "the retry grades (echoGrader verdict)");
|
||||
// ONE attempt (owner 2026-09-29): the dropped grade's answer
|
||||
// stands — re-submission is refused, the problem is done.
|
||||
await assert.rejects(
|
||||
() => pipeline.submitAnswers(set.id, [{ problemId: victimId, text: "retry answer" }]),
|
||||
/was already answered/,
|
||||
);
|
||||
const after = f.practice.store.getSet(set.id)!;
|
||||
assert.equal(after.problems.find((p) => p.id === victimId)!.grade, undefined, "still no grade — one attempt");
|
||||
assert.equal(after.problems.find((p) => p.id === victimId)!.answer, "first");
|
||||
|
||||
rh1();
|
||||
rh2();
|
||||
@@ -772,8 +776,8 @@ describe("submitAnswers (grading)", () => {
|
||||
const final = f.practice.store.getSet(set.id)!;
|
||||
const g1 = final.problems.find((p) => p.id === p1.id)!.grade;
|
||||
const g2 = final.problems.find((p) => p.id === p2.id)!.grade;
|
||||
assert.ok(g1 && g1.verdict === "solid", "verdict encodes from the answer text");
|
||||
assert.ok(g2 && g2.verdict === "partial");
|
||||
assert.ok(g1 && g1.score === 9, "score encodes from the answer text");
|
||||
assert.ok(g2 && g2.score === 6);
|
||||
assert.ok(g1!.gradedAt >= NOW);
|
||||
assert.equal(practiceStatuses().length, 0, "the grading row clears");
|
||||
|
||||
|
||||
@@ -79,13 +79,13 @@ describe("the practice store", () => {
|
||||
rubric: "- names the performative contradiction",
|
||||
status: "ready",
|
||||
answer: "Because…",
|
||||
grade: { verdict: "solid", feedback: "Named it.", gradedAt: "2026-09-29T13:00:00.000Z" },
|
||||
grade: { score: 9, reasoning: "Named it.", gradedAt: "2026-09-29T13:00:00.000Z" },
|
||||
});
|
||||
set.status = "ready";
|
||||
practiceStore.saveSet(set);
|
||||
const reread = practiceStore.getSet(set.id);
|
||||
assert.equal(reread?.status, "ready");
|
||||
assert.equal(reread?.problems[0]?.grade?.verdict, "solid");
|
||||
assert.equal(reread?.problems[0]?.grade?.score, 9);
|
||||
});
|
||||
|
||||
it("listSets returns createdAt-descending and skips corrupt/unparseable files", () => {
|
||||
@@ -157,17 +157,21 @@ describe("isProblemSet", () => {
|
||||
assert.ok(isProblemSet(minimal));
|
||||
});
|
||||
|
||||
it("grade verdicts are the three-level union", () => {
|
||||
const withGrade = (verdict: string): unknown => ({
|
||||
it("grades are integer scores 0–10 with non-empty reasoning (one-attempt redesign)", () => {
|
||||
const withGrade = (score: unknown, reasoning = "why"): unknown => ({
|
||||
...init(),
|
||||
id: "ps_valid",
|
||||
problems: [
|
||||
{ id: "pp_1", chains: [], statement: "", solution: "", rubric: "", status: "ready", grade: { verdict, feedback: "", gradedAt: "t" } },
|
||||
{ id: "pp_1", chains: [], statement: "", solution: "", rubric: "", status: "ready", grade: { score, reasoning, gradedAt: "t" } },
|
||||
],
|
||||
});
|
||||
assert.ok(isProblemSet(withGrade("solid")));
|
||||
assert.ok(isProblemSet(withGrade("partial")));
|
||||
assert.ok(isProblemSet(withGrade("missed")));
|
||||
assert.ok(!isProblemSet(withGrade("excellent")));
|
||||
assert.ok(isProblemSet(withGrade(0)));
|
||||
assert.ok(isProblemSet(withGrade(10)));
|
||||
assert.ok(!isProblemSet(withGrade(11)), "above the range");
|
||||
assert.ok(!isProblemSet(withGrade(-1)), "below the range");
|
||||
assert.ok(!isProblemSet(withGrade(7.5)), "fractional scores are refused");
|
||||
assert.ok(!isProblemSet(withGrade("9")), "string scores are refused");
|
||||
assert.ok(!isProblemSet(withGrade(9, "")), "empty reasoning is refused");
|
||||
assert.ok(!isProblemSet({ ...init(), id: "ps_valid", problems: [{ id: "pp_1", chains: [], statement: "", solution: "", rubric: "", status: "ready", grade: { verdict: "solid", feedback: "legacy", gradedAt: "t" } }] }), "legacy verdict grades fail (wipe regime)");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -200,7 +200,7 @@ describe("PracticeListBody — the set list", () => {
|
||||
it("graded count appears; empty list states it honestly", () => {
|
||||
const s = set({
|
||||
problems: [
|
||||
readyProblem("pp_1", ["n3"], { grade: { verdict: "solid", feedback: "f", gradedAt: "2026-09-29T13:00:00.000Z" } }),
|
||||
readyProblem("pp_1", ["n3"], { answer: "a", grade: { score: 9, reasoning: "r", gradedAt: "2026-09-29T13:00:00.000Z" } }),
|
||||
readyProblem("pp_2", ["n1"]),
|
||||
],
|
||||
});
|
||||
@@ -251,9 +251,9 @@ describe("PracticeExamBody — exam states", () => {
|
||||
drafts: {} as Record<string, string>,
|
||||
onDraft: () => {},
|
||||
onSubmit: () => {},
|
||||
submitting: false,
|
||||
submitError: null as string | null,
|
||||
grading: false,
|
||||
submittingId: null as string | null,
|
||||
submitErrors: {} as Record<string, string>,
|
||||
liveGradeIds: new Set<string>(),
|
||||
};
|
||||
|
||||
it("generating: numbered stubs with the placeholder — propositions NEVER surface", () => {
|
||||
@@ -271,12 +271,19 @@ describe("PracticeExamBody — exam states", () => {
|
||||
assert.ok(!html.includes("Each person owns themself"));
|
||||
assert.ok(!html.includes('data-testid="answer-pp_1"'), "no answer box for a stub");
|
||||
// No submit while generating.
|
||||
assert.ok(!html.includes('data-testid="practice-submit"'));
|
||||
assert.ok(!html.includes('data-testid="practice-submit-'));
|
||||
});
|
||||
|
||||
it("the header carries the levels line (named levels, owner 2026-09-29)", () => {
|
||||
it("the top bar carries the title + sidebar toggle; the body carries the levels line (owner, UI round 2026-09-29)", () => {
|
||||
const s = set({ problems: [readyProblem("pp_1", ["n3"])] });
|
||||
const html = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
|
||||
// The chat-pattern top bar: drag surface, toggle slot, truncating title.
|
||||
assert.ok(html.includes("lh-drag"));
|
||||
assert.ok(html.includes('aria-label="Open sidebar"'));
|
||||
assert.ok(html.includes('data-testid="practice-exam-title"'));
|
||||
assert.ok(html.includes("Ethics lesson"));
|
||||
// The internal scroller (the sidebar stays fixed when this page scrolls).
|
||||
assert.ok(html.includes('data-testid="practice-exam-scroll"'));
|
||||
assert.ok(html.includes('data-testid="practice-exam-levels"'));
|
||||
assert.ok(html.includes("Direct exercise · Adjacent context · Theory"), "the fixture's rung 2 / cue 1 / theory");
|
||||
});
|
||||
@@ -286,37 +293,67 @@ describe("PracticeExamBody — exam states", () => {
|
||||
const gated = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
|
||||
assert.ok(gated.includes("self-ownership")); // markdown strong rendered
|
||||
assert.ok(gated.includes('data-testid="answer-pp_1"'));
|
||||
assert.ok(gated.includes('data-testid="practice-submit"'));
|
||||
// PER-PROBLEM submit (owner 2026-09-29): the button lives under the
|
||||
// box, gated on a draft; there is NO global submit anywhere.
|
||||
assert.ok(gated.includes('data-testid="practice-submit-pp_1"'));
|
||||
assert.ok(gated.includes('disabled=""'), "empty drafts — submit disabled");
|
||||
const filled = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} drafts={{ pp_1: "because…" }} />);
|
||||
assert.ok(!filled.includes('data-testid="practice-submit-error"'));
|
||||
assert.ok(!filled.includes('disabled=""'), "a draft arms the submit");
|
||||
assert.ok(!filled.includes('data-testid="practice-submit-error-pp_1"'));
|
||||
});
|
||||
|
||||
it("graded: answer read-only, verdict chip + feedback + solution + rubric revealed", () => {
|
||||
it("submitted + graded: answer read-only, score + reasoning + solution revealed, rubric collapsed", () => {
|
||||
const s = set({
|
||||
problems: [
|
||||
readyProblem("pp_1", ["n3"], {
|
||||
answer: "because arguing is intentional action",
|
||||
grade: { verdict: "partial", feedback: "Half the target came to mind.", gradedAt: "2026-09-29T13:00:00.000Z" },
|
||||
grade: { score: 6, reasoning: "Half the target came to mind.", gradedAt: "2026-09-29T13:00:00.000Z" },
|
||||
}),
|
||||
readyProblem("pp_2", ["n1"]),
|
||||
],
|
||||
});
|
||||
const html = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
|
||||
// The locked answer.
|
||||
// The locked answer — one attempt, done.
|
||||
assert.ok(html.includes("because arguing is intentional action"));
|
||||
assert.ok(/readOnly/.test(html), "the graded answer is read-only");
|
||||
// The reveal.
|
||||
assert.ok(html.includes('data-testid="reveal-pp_1"'));
|
||||
assert.ok(html.includes("partial"));
|
||||
assert.ok(html.includes('data-testid="feedback-pp_1"'));
|
||||
assert.ok(/readOnly/.test(html), "the submitted answer is read-only");
|
||||
assert.ok(!html.includes('data-testid="practice-submit-pp_1"'), "no re-submission");
|
||||
// The grade: mono score + brief reasoning.
|
||||
assert.ok(html.includes('data-testid="grade-pp_1"'));
|
||||
assert.ok(html.includes("6/10"));
|
||||
assert.ok(html.includes('data-testid="reasoning-pp_1"'));
|
||||
assert.ok(html.includes("Half the target came to mind."));
|
||||
// The instant reveal.
|
||||
assert.ok(html.includes('data-testid="reveal-pp_1"'));
|
||||
assert.ok(html.includes('data-testid="solution-pp_1"'));
|
||||
assert.ok(html.includes("The model solution."));
|
||||
// The rubric rides a collapsed <details> (owner 2026-09-29).
|
||||
assert.ok(html.includes("<details"));
|
||||
assert.ok(!html.includes("<details open"), "collapsed by default");
|
||||
assert.ok(html.includes('data-testid="rubric-pp_1"'));
|
||||
assert.ok(html.includes("names self-ownership"));
|
||||
// Ungraded sibling stays answerable.
|
||||
// Unanswered sibling stays answerable.
|
||||
assert.ok(html.includes('data-testid="answer-pp_2"'));
|
||||
assert.ok(html.includes('data-testid="practice-submit-pp_2"'));
|
||||
});
|
||||
|
||||
it("submitted, grade pending: the reveal shows instantly, grading spins per problem", () => {
|
||||
const s = set({
|
||||
problems: [readyProblem("pp_1", ["n3"], { answer: "because arguing is intentional action" })],
|
||||
});
|
||||
const live = renderToStaticMarkup(
|
||||
<PracticeExamBody set={s} {...examBase} liveGradeIds={new Set(["pp_1"])} />,
|
||||
);
|
||||
// Instant reveal — the solution needs no grade.
|
||||
assert.ok(live.includes('data-testid="solution-pp_1"'));
|
||||
assert.ok(live.includes('data-testid="rubric-pp_1"'));
|
||||
// The live spinner for exactly this problem.
|
||||
assert.ok(live.includes('data-testid="grading-pp_1"'));
|
||||
assert.ok(live.includes("Grading"));
|
||||
assert.ok(!live.includes('data-testid="grade-pp_1"'), "no score yet");
|
||||
// No live row (dropped grade): honest, no eternal spinner.
|
||||
const dropped = renderToStaticMarkup(<PracticeExamBody set={s} {...examBase} />);
|
||||
assert.ok(dropped.includes('data-testid="grade-interrupted-pp_1"'));
|
||||
assert.ok(!dropped.includes('data-testid="grading-pp_1"'));
|
||||
});
|
||||
|
||||
it("a failed problem says so; a failed set says so", () => {
|
||||
|
||||
Reference in New Issue
Block a user