sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

feat: probe/teach agent split — explicit phase markers, position-only plan map

Browse Source 
Diagnosis (2026-09-11 user testing, minimal-effort GLM): sessions stuck on
'plan' forever. Two causes: (1) the three-phase view read the stored tree's
currentNodeId, which nothing ever writes post author-once (2026-09-03) — a
structural dead seam; (2) the teacher stochastically left quiz.nodeId
untagged, and done-by-quiz-result semantics would have falsely completed
nodes anyway.

The redesign (docs/probe-teach-split.md) abolishes derived occupancy in
favor of explicit events:

- Two agents over one transcript: createProberAgent (quiz without a
  nodeId field, ask, probe_done) and createTeacherAgent (full toolset,
  plan_init onward). The chat-handler construction branch keys on a
  probe_done part in the transcript. MODELS.prober added.
- probe_done: server-side seam tool — carries the learner assessment
  (strands + free-form edge read), refuses while a question is open or
  the probe is already concluded, and queues a handoff steer through the
  generalized wake pipeline (steers now dedupe by id, not jobId; job
  finalization only for job steers). The teacher auto-wakes and authors
  the plan; the learner sees the assessment via the probe-done card.
- phases: probe ends at a concluded probe_done, plan ends at a
  successful plan_init — strictly sequential, no skipping, no legacy
  compatibility (data wiped at the pivot per doctrine).
- Plan map = position, not completion: derivePlanStatuses reads only
  tagged quiz CALLS (state/outcome feed nothing); current = latest
  tagged node with consecutive duplicates collapsed, visited = earlier
  distinct stays (a revisit stays in both — current wins at render).
  PlanTree is structure-only; currentNodeId/doneNodeIds and the dead
  mutation helpers are gone. No fallback current.
- Prompts split per agent (prompts/prober/, prompts/teacher/ over the
  trimmed shared base); graph-context injection removed entirely
  (shelved, docs/probe-teach-split.md §5); knowledge-graph context no
  longer reaches either agent.
- Plan blocks are full-only snapshots at plan_init (neighborhood
  machinery retired; the persistent DAG window is the future UI wave);
  usage capture unchanged.

Gates: tsc clean, 294/294 tests (phases/plan-tree/plan-blocks/question-
tools rewritten to the new semantics; probe-done + handoff-wake tests
added), lint, build. Live GLM smoke PASS: prober concluded with an honest
assessment, teacher auto-woke, authored a 10-node plan with the block
attached, then tagged checks on slope-instinct → secant derived current/
visited exactly as designed; grading honest both ways (driver off-by-one
proved a wrong answer grades false).
This commit is contained in:
amosblomqvist
2026-09-11 06:44:20 +03:00
parent 741ff416d6
commit 5653c77a29
49 changed files with 1538 additions and 2038 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/server/jobs.ts
+38 -23
View File
@@ -110,15 +110,28 @@ function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function jobIdOf(steer: SteerMessage): string {
return (steer.metadata as { subagentResult: { jobId: string } }).subagentResult.jobId;
/** The job a steer belongs to, if it is a job steer (result/abort);
* probe-handoff steers have no job — null. */
function jobIdOf(steer: SteerMessage): string | null {
const meta = steer.metadata as { subagentResult?: { jobId: string } };
return meta.subagentResult?.jobId ?? null;
}
/**
* Hand a finished job's steer to the pipeline. Resolves when the wake turn
* has completed (and the job status flipped); rejects when delivery fails
* unrecoverably. Never rejects for queueing itself.
*/
/** Finalize a delivered steer's job bookkeeping — job steers only. */
function finalizeSteer(steer: SteerMessage): void {
const jobId = jobIdOf(steer);
if (jobId) finalizeJob(jobId);
}
/** Fail a steer's job on unrecoverable delivery failure — job steers only. */
function failSteer(steer: SteerMessage, err: Error): void {
const jobId = jobIdOf(steer);
if (jobId) failJob(jobId, err);
}
/** Queue ANY steer (result, abort, probe handoff) through the pipeline.
* Resolves when the wake turn has completed; rejects when delivery fails
* unrecoverably. Never rejects for queueing itself. */
export function deliverJobResult(steer: SteerMessage): Promise<void> {
return new Promise<void>((resolve, reject) => {
const s = state();
@@ -155,13 +168,12 @@ export function claimSteers(chatId: string): SteerMessage[] {
if (!record) return []; // the handler validated the chat; defensive only
const claimed: SteerMessage[] = [];
for (const entry of queue) {
const jobId = jobIdOf(entry.steer);
if (steerDelivered(record.messages, jobId)) {
if (steerDelivered(record.messages, entry.steer.id)) {
// A wake already landed it — just finish the bookkeeping.
finalizeJob(jobId);
finalizeSteer(entry.steer);
} else {
claimed.push(entry.steer);
finalizeJob(jobId); // delivery is now the client turn's job
finalizeSteer(entry.steer); // delivery is now the client turn's job
}
entry.resolve();
}
@@ -229,10 +241,13 @@ export function abortJob(jobId: string): AbortOutcome {
}
function chatIdOf(steer: SteerMessage): string {
// The probe handoff carries its chat directly (no job record exists).
const handoff = (steer.metadata as { probeHandoff?: { chatId: string } }).probeHandoff;
if (handoff) return handoff.chatId;
// The chat is resolvable from the job record; the steer's metadata
// carries jobId, which indexes the store.
const jobId = jobIdOf(steer);
const job = stores.jobStore.getJob(jobId);
const job = jobId ? stores.jobStore.getJob(jobId) : undefined;
return job?.chatId ?? "";
}
@@ -253,21 +268,21 @@ async function pump(chatId: string): Promise<void> {
return;
}
const entry = queue[0]!;
const jobId = jobIdOf(entry.steer);
const steerId = entry.steer.id;
// Gone chat: nothing to deliver into.
const record = stores.chatStore.getChat(chatId);
if (!record) {
queue.shift();
entry.reject(new Error(`chat ${chatId} not found — cannot deliver job ${jobId}`));
entry.reject(new Error(`chat ${chatId} not found — cannot deliver steer ${steerId}`));
continue;
}
// Idempotent delivery: the transcript already carries this steer
// (a previous wake completed, or a duplicate deliver raced in).
if (steerDelivered(record.messages, jobId)) {
if (steerDelivered(record.messages, steerId)) {
queue.shift();
finalizeJob(jobId);
finalizeSteer(entry.steer);
entry.resolve();
continue;
}
@@ -280,7 +295,7 @@ async function pump(chatId: string): Promise<void> {
const fresh = stores.chatStore.getChat(chatId);
if (!fresh) {
queue.shift();
entry.reject(new Error(`chat ${chatId} not found — cannot deliver job ${jobId}`));
entry.reject(new Error(`chat ${chatId} not found — cannot deliver steer ${steerId}`));
continue;
}
@@ -289,16 +304,16 @@ async function pump(chatId: string): Promise<void> {
if (!s.turnRunner) {
queue.shift();
const err = new Error("wake pipeline: no turn runner wired (setTurnRunner)");
failJob(jobId, err);
failSteer(entry.steer, err);
entry.reject(err);
continue;
}
// A client continuation may have COALESCED this steer (claimSteers) while
// the gates were closed — re-check before firing: exactly one delivery.
if (steerDelivered(fresh.messages, jobId)) {
if (steerDelivered(fresh.messages, steerId)) {
queue.shift();
finalizeJob(jobId);
finalizeSteer(entry.steer);
entry.resolve();
continue;
}
@@ -306,7 +321,7 @@ async function pump(chatId: string): Promise<void> {
// Fire the server-side teacher turn. A 409 = a client turn raced
// in between idle and firing — back off and retry the same steer.
const status = await s.turnRunner(chatId, [...fresh.messages, entry.steer]).catch((err: unknown) => {
console.error(`[job ${jobId}] wake turn crashed:`, err);
console.error(`[steer ${steerId}] wake turn crashed:`, err);
return -1;
});
if (status === 409) {
@@ -317,11 +332,11 @@ async function pump(chatId: string): Promise<void> {
queue.shift();
if (status === 200) {
finalizeJob(jobId);
finalizeSteer(entry.steer);
entry.resolve();
} else {
const err = new Error(`wake turn failed with status ${status}`);
failJob(jobId, err);
failSteer(entry.steer, err);
entry.reject(err);
}
}