feat: probe/teach agent split — explicit phase markers, position-only plan map
Diagnosis (2026-09-11 user testing, minimal-effort GLM): sessions stuck on 'plan' forever. Two causes: (1) the three-phase view read the stored tree's currentNodeId, which nothing ever writes post author-once (2026-09-03) — a structural dead seam; (2) the teacher stochastically left quiz.nodeId untagged, and done-by-quiz-result semantics would have falsely completed nodes anyway. The redesign (docs/probe-teach-split.md) abolishes derived occupancy in favor of explicit events: - Two agents over one transcript: createProberAgent (quiz without a nodeId field, ask, probe_done) and createTeacherAgent (full toolset, plan_init onward). The chat-handler construction branch keys on a probe_done part in the transcript. MODELS.prober added. - probe_done: server-side seam tool — carries the learner assessment (strands + free-form edge read), refuses while a question is open or the probe is already concluded, and queues a handoff steer through the generalized wake pipeline (steers now dedupe by id, not jobId; job finalization only for job steers). The teacher auto-wakes and authors the plan; the learner sees the assessment via the probe-done card. - phases: probe ends at a concluded probe_done, plan ends at a successful plan_init — strictly sequential, no skipping, no legacy compatibility (data wiped at the pivot per doctrine). - Plan map = position, not completion: derivePlanStatuses reads only tagged quiz CALLS (state/outcome feed nothing); current = latest tagged node with consecutive duplicates collapsed, visited = earlier distinct stays (a revisit stays in both — current wins at render). PlanTree is structure-only; currentNodeId/doneNodeIds and the dead mutation helpers are gone. No fallback current. - Prompts split per agent (prompts/prober/, prompts/teacher/ over the trimmed shared base); graph-context injection removed entirely (shelved, docs/probe-teach-split.md §5); knowledge-graph context no longer reaches either agent. - Plan blocks are full-only snapshots at plan_init (neighborhood machinery retired; the persistent DAG window is the future UI wave); usage capture unchanged. Gates: tsc clean, 294/294 tests (phases/plan-tree/plan-blocks/question- tools rewritten to the new semantics; probe-done + handoff-wake tests added), lint, build. Live GLM smoke PASS: prober concluded with an honest assessment, teacher auto-woke, authored a 10-node plan with the block attached, then tagged checks on slope-instinct → secant derived current/ visited exactly as designed; grading honest both ways (driver off-by-one proved a wrong answer grades false).
This commit is contained in:
amosblomqvist
@@ -0,0 +1,109 @@
|
||||
import { tool, type ToolSet } from "ai";
|
||||
import { z } from "zod";
|
||||
import type { ProbeDoneInput, ProbeDoneResult } from "../../core/contracts.ts";
|
||||
import type { UIMessage } from "ai";
|
||||
import { findPendingToolCall } from "../../core/pending.ts";
|
||||
import type { ChatStore } from "../../core/ports.ts";
|
||||
import { buildProbeHandoffSteer, type ProbeHandoffSteerMessage } from "../../core/contracts.ts";
|
||||
|
||||
/** What the probe_done tool closes over. */
|
||||
export interface ProbeDoneContext {
|
||||
chatId: string;
|
||||
/** The storage port (open-question + already-concluded checks). */
|
||||
chatStore: ChatStore;
|
||||
/** Queue the handoff steer (the wake pipeline — src/server/jobs.ts).
|
||||
* Injected so this module never imports the server seam. */
|
||||
deliver: (steer: ProbeHandoffSteerMessage) => void;
|
||||
}
|
||||
|
||||
/** Does the transcript already carry a concluded probe (any state)? */
|
||||
function probeConcluded(messages: UIMessage[]): boolean {
|
||||
for (const m of messages) {
|
||||
if (m.parts.some((p) => p.type === "tool-probe_done")) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* probe_done — the probe→planning seam (docs/probe-teach-split.md §2).
|
||||
*
|
||||
* SERVER-SIDE. The prober concludes the probe by authoring the learner
|
||||
* assessment (the tool INPUT — it streams to the learner live and is
|
||||
* durable in the transcript as the tool part itself). Three outcomes:
|
||||
*
|
||||
* ok:false + error — a question is still open ("resolve it first"), or
|
||||
* the probe is already concluded. The refusal returns to the prober as
|
||||
* a tool result; it fixes course and retries (the return-to-agent
|
||||
* doctrine, same as nodeId validation).
|
||||
* ok:true — the assessment is recorded, and the HANDOFF is queued: a
|
||||
* steer through the wake pipeline that fires the teacher's first turn
|
||||
* (plan authoring) the moment the prober's turn ends. Auto-wake is the
|
||||
* user-decided trigger — the learner experiences assessment → plan as
|
||||
* one continuous flow.
|
||||
*
|
||||
* The first probe_done in the transcript is also the construction seam:
|
||||
* from that message on, every request builds the TEACHER agent.
|
||||
*/
|
||||
export function probeDoneTool(ctx: ProbeDoneContext): ToolSet {
|
||||
return {
|
||||
probe_done: tool({
|
||||
description:
|
||||
"Conclude the probing phase and hand off to the teaching phase. Call this ONCE, when the probe is " +
|
||||
"complete: every strand the lesson depends on is bracketed (a floor he reliably gets right AND a " +
|
||||
"ceiling where it runs out), and his learning goal is concrete. The input IS the learner assessment — " +
|
||||
"one strand per dependency the lesson will rest on, with your read of where his edge sits on it " +
|
||||
"(floor, ceiling, misconceptions, what to build from). It streams to the learner live and becomes " +
|
||||
"the handoff artifact the teacher plans from. Refused while a question is still open, and after " +
|
||||
"the probe is already concluded. After it succeeds, say your closing sentence and end your turn.",
|
||||
inputSchema: z.object({
|
||||
assessment: z
|
||||
.object({
|
||||
strands: z
|
||||
.array(
|
||||
z.object({
|
||||
strand: z.string().describe("The dependency strand (e.g. 'the derivative as a limit') — a name the plan can hang a node on."),
|
||||
assessment: z
|
||||
.string()
|
||||
.describe(
|
||||
"Where his edge sits on this strand: the floor (what he reliably gets right), the ceiling (where it runs out), " +
|
||||
"misconceptions found, and what the lesson should build from.",
|
||||
),
|
||||
}),
|
||||
)
|
||||
.min(1)
|
||||
.describe("Every strand the planned lesson will depend on, each with your read of his edge on it."),
|
||||
summary: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Optional overall synthesis: his apparent goal and where the teaching should enter."),
|
||||
})
|
||||
.describe("The learner assessment — the probe's distilled output."),
|
||||
}) satisfies z.ZodType<ProbeDoneInput>,
|
||||
execute: async (input): Promise<ProbeDoneResult> => {
|
||||
const record = ctx.chatStore.getChat(ctx.chatId);
|
||||
if (!record) return { ok: false, error: `Chat not found: ${ctx.chatId}` };
|
||||
|
||||
if (findPendingToolCall(record.messages) !== null) {
|
||||
return {
|
||||
ok: false,
|
||||
error:
|
||||
"A question is still open — resolve it (or let him answer) before concluding the probe. " +
|
||||
"probe_done only fires with no question outstanding.",
|
||||
};
|
||||
}
|
||||
if (probeConcluded(record.messages)) {
|
||||
return { ok: false, error: "The probe is already concluded — probe_done fires exactly once." };
|
||||
}
|
||||
try {
|
||||
ctx.deliver(buildProbeHandoffSteer(ctx.chatId));
|
||||
} catch (err) {
|
||||
console.error(`[chat ${ctx.chatId}] probe handoff queueing failed:`, err);
|
||||
// The handoff is recoverable-by-design (the next client message
|
||||
// constructs the teacher off the transcript seam) — record the
|
||||
// assessment anyway; the wake is an optimization, not a gate.
|
||||
}
|
||||
return { ok: true, assessment: input.assessment };
|
||||
},
|
||||
}),
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user