practice bug fix — the boot sweep stamped a LIVE run (owner live test 2026-09-29)

Owner report: created a set, 'Interrupted (server restarted)' showed
on list + set views, generation proceeded and overwrote it. The
sweep's 'generating + no registry row' test had a false-positive
mode: it ran at EVERY pipeline instantiation, and an instance could
evaluate against a registry whose fresh row it couldn't observe.

Three guards, per ruling:

1. HEARTBEAT: ProblemSet.updatedAt? (contract + doc) — bumped on
   every store write; createSet stamps it = createdAt; saveSet
   re-stamps (validator tolerates absence for legacy rows).

2. ONCE PER PROCESS: the sweep guards on globalThis
   .__lhPracticeSweepDone — no instance re-evaluates after the
   process's first sweep.

3. GRACE WINDOW: only sets whose heartbeat (updatedAt ?? createdAt)
   is older than 120s are stamped. A live run saves at every phase/
   problem transition, so it is structurally unreachable; a true
   orphan is minutes stale at next boot. One [practice <id>] log
   line per stamped set (the extraction/jobs console.error idiom).

Tests: stale orphan stamped; FRESH generating set untouched (grace);
once-guard (second factory instantiation never re-sweeps); live-row
protection re-armed via resetSweep() test seam; store pins for
updatedAt birth-stamp + re-stamp + legacy tolerance. Architecture.md
sweep sentence updated (once-per-process + grace semantics).

1142/1142, tsc 0, lint 0, build 0.
This commit is contained in:
amosblomqvist
2026-09-29 20:19:46 +03:00
parent e92679254d
commit 8d39f3717c
6 changed files with 116 additions and 14 deletions
+23 -9
View File
@@ -264,15 +264,29 @@ export interface PracticePipelineDeps extends PracticeModels {
export function createPracticePipeline(deps: PracticePipelineDeps) {
const agents = createPracticeAgents(deps);
// BOOT SWEEP: a persisted partitioning/generating set with no live row
// is a crash/restart orphan (the registry is memory-only) — mark it,
// idempotently. Runs at every pipeline instantiation (dev route
// entries re-run it; live rows are globalThis-shared and skipped).
for (const set of deps.practiceStore.listSets()) {
if (
(set.status === "partitioning" || set.status === "generating") &&
!practiceRuns().has(`${set.id}:generation`)
) {
// BOOT SWEEP (owner bug fix, live test 2026-09-29 — the sweep once
// stamped a LIVE run): a persisted partitioning/generating set with no
// live row AND a heartbeat older than the grace window is a crash/
// restart orphan (the registry is memory-only) — mark it, idempotently.
// THREE guards against stamping the living:
// 1. ONCE PER PROCESS (globalThis flag) — an instance could otherwise
// evaluate against a registry whose fresh row it can't observe
// (dev route bundles, restart paths).
// 2. GRACE WINDOW (120s over updatedAt ?? createdAt) — a live run
// saves at every phase/problem transition, so its heartbeat is
// structurally never 120s stale; a true orphan is minutes stale at
// the next boot.
// 3. Live-row check (globalThis-shared registry) as before.
const sweepFlag = globalThis as { __lhPracticeSweepDone?: boolean };
if (!sweepFlag.__lhPracticeSweepDone) {
sweepFlag.__lhPracticeSweepDone = true;
const staleBefore = Date.now() - 120_000;
for (const set of deps.practiceStore.listSets()) {
if (set.status !== "partitioning" && set.status !== "generating") continue;
if (practiceRuns().has(`${set.id}:generation`)) continue;
const heartbeat = Date.parse(set.updatedAt ?? set.createdAt);
if (Number.isFinite(heartbeat) && heartbeat > staleBefore) continue; // grace
console.error(`[practice ${set.id}] boot sweep: stale ${set.status} set marked interrupted`);
set.status = "error";
set.error = "Interrupted (server restarted)";
deps.practiceStore.saveSet(set);
+6
View File
@@ -163,6 +163,12 @@ export interface ProblemSet {
* default. */
title: string;
createdAt: string;
/** Bumped on every store write — the live-run heartbeat the boot
* sweep's grace window reads (a live run saves at every phase or
* problem transition, so it is structurally never 120s stale).
* Absent on legacy rows (validator tolerates; readers fall back
* to createdAt). */
updatedAt?: string;
params: PracticeParams;
/** The selected node ids — the raw selection record. The chain
* decomposition derives from these + the follows-edges among them at
+4
View File
@@ -133,6 +133,7 @@ function createSet(init: Omit<ProblemSet, "id">): ProblemSet {
const set: ProblemSet = {
...init,
id: `ps_${Date.now().toString(36)}_${crypto.randomUUID().slice(0, 8)}`,
updatedAt: init.updatedAt ?? init.createdAt,
};
writeSetFile(set);
return set;
@@ -164,6 +165,9 @@ function listSets(): ProblemSet[] {
function saveSet(set: ProblemSet): void {
if (!isProblemSet(set)) throw new Error("problem set record failed validation");
// The heartbeat: every write re-stamps (the boot sweep's grace window
// reads this — a live run is structurally never 120s stale).
set.updatedAt = new Date().toISOString();
writeSetFile(set);
}