feat: probe/teach agent split — explicit phase markers, position-only plan map
Diagnosis (2026-09-11 user testing, minimal-effort GLM): sessions stuck on 'plan' forever. Two causes: (1) the three-phase view read the stored tree's currentNodeId, which nothing ever writes post author-once (2026-09-03) — a structural dead seam; (2) the teacher stochastically left quiz.nodeId untagged, and done-by-quiz-result semantics would have falsely completed nodes anyway. The redesign (docs/probe-teach-split.md) abolishes derived occupancy in favor of explicit events: - Two agents over one transcript: createProberAgent (quiz without a nodeId field, ask, probe_done) and createTeacherAgent (full toolset, plan_init onward). The chat-handler construction branch keys on a probe_done part in the transcript. MODELS.prober added. - probe_done: server-side seam tool — carries the learner assessment (strands + free-form edge read), refuses while a question is open or the probe is already concluded, and queues a handoff steer through the generalized wake pipeline (steers now dedupe by id, not jobId; job finalization only for job steers). The teacher auto-wakes and authors the plan; the learner sees the assessment via the probe-done card. - phases: probe ends at a concluded probe_done, plan ends at a successful plan_init — strictly sequential, no skipping, no legacy compatibility (data wiped at the pivot per doctrine). - Plan map = position, not completion: derivePlanStatuses reads only tagged quiz CALLS (state/outcome feed nothing); current = latest tagged node with consecutive duplicates collapsed, visited = earlier distinct stays (a revisit stays in both — current wins at render). PlanTree is structure-only; currentNodeId/doneNodeIds and the dead mutation helpers are gone. No fallback current. - Prompts split per agent (prompts/prober/, prompts/teacher/ over the trimmed shared base); graph-context injection removed entirely (shelved, docs/probe-teach-split.md §5); knowledge-graph context no longer reaches either agent. - Plan blocks are full-only snapshots at plan_init (neighborhood machinery retired; the persistent DAG window is the future UI wave); usage capture unchanged. Gates: tsc clean, 294/294 tests (phases/plan-tree/plan-blocks/question- tools rewritten to the new semantics; probe-done + handoff-wake tests added), lint, build. Live GLM smoke PASS: prober concluded with an honest assessment, teacher auto-woke, authored a 10-node plan with the block attached, then tagged checks on slope-instinct → secant derived current/ visited exactly as designed; grading honest both ways (driver off-by-one proved a wrong answer grades false).
This commit is contained in:
amosblomqvist
@@ -124,12 +124,15 @@ export interface TurnUsage {
|
||||
* small container, the current node glowing — clearly a fragment, not the
|
||||
* whole plan. Clicking either opens the full plan in a popup. */
|
||||
export interface PlanBlockData {
|
||||
kind: "full" | "neighborhood";
|
||||
/** Only "full" survives (the 2026-09-11 pivot): the plan renders IN
|
||||
* FLOW once, at authoring. The neighborhood fragments are retired —
|
||||
* position derives live from the transcript (the persistent DAG
|
||||
* window is the successor). */
|
||||
kind: "full";
|
||||
at: string;
|
||||
/** The current node at snapshot time (the glow target; null = none). */
|
||||
current: string | null;
|
||||
/** Included nodes: all of them (full) or the one-level-up/one-level-down
|
||||
* neighborhood of `current` (neighborhood). Statuses are the snapshot's,
|
||||
/** All nodes of the authored tree. Statuses are the snapshot's,
|
||||
* derived at emission time — the block never re-derives at render. */
|
||||
nodes: Array<{ id: string; title: string; status: PlanNodeStatus }>;
|
||||
edges: Array<{ from: string; to: string }>;
|
||||
@@ -141,9 +144,9 @@ export interface PlanBlockData {
|
||||
* them in listChats. Liveness (`busy`) is NOT record-derived — the API
|
||||
* route merges it in (see ChatListItem).
|
||||
*
|
||||
* Phase semantics (three phases, data-only): probe = no plan yet, plan =
|
||||
* tree exists but no node occupied, teach = a node is occupied
|
||||
* (core/phases.ts phaseOfTree).
|
||||
* Phase semantics (three phases, event-driven — docs/probe-teach-split.md
|
||||
* §4): probe ends at the concluded probe_done, plan ends at the successful
|
||||
* plan_init; derived from the transcript (core/phases.ts phaseAt3).
|
||||
*/
|
||||
export interface ChatSummary {
|
||||
id: string;
|
||||
|
||||
@@ -242,7 +242,7 @@ export interface AbortSteerMessage extends UIMessage {
|
||||
}
|
||||
|
||||
/** Anything the wake pipeline may queue and deliver. */
|
||||
export type SteerMessage = SubagentSteerMessage | AbortSteerMessage;
|
||||
export type SteerMessage = SubagentSteerMessage | AbortSteerMessage | ProbeHandoffSteerMessage;
|
||||
|
||||
/** Whether a result is the failure shape (html/svg empty, failure brief). */
|
||||
export function isSubagentFailure(kind: JobKind, result: ResearchResult | DemoResult | VisualResult): boolean {
|
||||
@@ -326,10 +326,39 @@ function abortText(kind: JobKind): string {
|
||||
return "The figure job was aborted by the learner — no figure is coming. Acknowledge it in one line and teach without it (a re-dispatch is fine if the learner wants it).";
|
||||
}
|
||||
|
||||
/** True when the transcript already carries this job's steer (idempotent delivery). */
|
||||
export function steerDelivered(messages: UIMessage[], jobId: string): boolean {
|
||||
return messages.some(
|
||||
(m) => (m.metadata as { subagentResult?: { jobId?: string } } | undefined)?.subagentResult?.jobId === jobId,
|
||||
);
|
||||
// ─── the probe handoff steer (the probe→teach split, docs/probe-teach-split.md §2) ──
|
||||
//
|
||||
// probe_done's execute queues this through the same wake pipeline; the
|
||||
// pump fires it once the prober's turn ends (gates + grace). It carries no
|
||||
// artifact — the assessment it references is durable in the transcript as
|
||||
// the probe_done tool part. Deterministic id per chat: exactly one
|
||||
// handoff, idempotent re-delivery.
|
||||
|
||||
export interface ProbeHandoffSteerMessage extends UIMessage {
|
||||
id: `steer_probe_${string}`;
|
||||
role: "user";
|
||||
metadata: { probeHandoff: { chatId: string } };
|
||||
parts: [{ type: "text"; text: string }];
|
||||
}
|
||||
|
||||
export function buildProbeHandoffSteer(chatId: string): ProbeHandoffSteerMessage {
|
||||
return {
|
||||
id: `steer_probe_${chatId}`,
|
||||
role: "user",
|
||||
metadata: { probeHandoff: { chatId } },
|
||||
parts: [
|
||||
{
|
||||
type: "text",
|
||||
text: "The probe is concluded — the learner assessment is recorded. Author the session's plan map now (call plan_init) and present it, then begin teaching.",
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
/** True when the transcript already carries this steer — idempotent
|
||||
* delivery for ANY steer kind: every steer id is deterministic per its
|
||||
* event (steer_<jobId> for jobs, steer_probe_<chatId> for the handoff). */
|
||||
export function steerDelivered(messages: UIMessage[], steerId: string): boolean {
|
||||
return messages.some((m) => m.id === steerId);
|
||||
}
|
||||
|
||||
|
||||
+36
-51
@@ -29,39 +29,35 @@ export interface PlanEdge {
|
||||
export interface PlanTree {
|
||||
nodes: PlanNode[];
|
||||
edges: PlanEdge[];
|
||||
currentNodeId: string | null;
|
||||
doneNodeIds: string[];
|
||||
/** Increments on every mutation — lets the UI animate/react to changes. */
|
||||
version: number;
|
||||
}
|
||||
|
||||
// NOTE (2026-09-11 pivot, docs/probe-teach-split.md): the tree once carried
|
||||
// `currentNodeId` / `doneNodeIds` — progress fields. They are GONE by
|
||||
// design: the tree carries structure ONLY; position and visited-ness derive
|
||||
// from the transcript at render time (derivePlanStatuses below). There is no
|
||||
// done ledger of any kind. No backwards compatibility for old records.
|
||||
|
||||
/** Derived, never stored: status of a node for the panel (vision §2). */
|
||||
export type PlanNodeStatus = "done" | "current" | "upcoming";
|
||||
export type PlanNodeStatus = "current" | "visited" | "upcoming";
|
||||
|
||||
export function planNodeStatus(tree: PlanTree, nodeId: string): PlanNodeStatus {
|
||||
if (tree.currentNodeId === nodeId) return "current";
|
||||
if (tree.doneNodeIds.includes(nodeId)) return "done";
|
||||
return "upcoming";
|
||||
}
|
||||
|
||||
// ─── derived statuses: the author-once protocol (2026-09-03) ─────────────────────
|
||||
// ─── derived statuses: position from CALLS, never from results ─────────────
|
||||
//
|
||||
// The teacher AUTHORS the map (plan_init, mermaid — once at the seam,
|
||||
// re-callable to rewrite) and never tracks progress: statuses derive from
|
||||
// the transcript, pure and at render time. A check names its node (quiz
|
||||
// input nodeId — validated at the call boundary against the live tree);
|
||||
// the derivation reads outcomes:
|
||||
// done — any check on the node that the learner PASSED (demonstration-
|
||||
// only: "explaining is yours, done is theirs")
|
||||
// current — the node of the last UNANSWERED check; if none, the first
|
||||
// node in authoring order whose prerequisites are all done and
|
||||
// that isn't done itself (forward-looking: the panel leads the
|
||||
// lesson — during teaching it points at what's coming, and holds
|
||||
// through the check that follows)
|
||||
// upcoming — everything else
|
||||
// Untagged checks and nodeIds missing from the current tree are lost signal
|
||||
// (the standing doctrine); a rewritten map keeps honest progress for reused
|
||||
// slugs because the derivation matches ids against the CURRENT tree.
|
||||
// (2026-09-11 pivot — docs/probe-teach-split.md §3.) The teacher AUTHORS the
|
||||
// map (plan_init, mermaid — re-callable rewrite) and never tracks progress.
|
||||
// The quiz `nodeId` means "this question is ABOUT this node" — a position
|
||||
// signal, never a completion signal. Derivation reads CALLS only:
|
||||
// current — the node of the most RECENT tagged quiz call (any state)
|
||||
// visited — every node referenced by an EARLIER tagged call
|
||||
// Consequences: a node spans multiple turns/checks naturally (several
|
||||
// questions may tag the same node — the map holds still); moving to a new
|
||||
// node is the teacher's judgment the previous one is finished, regardless of
|
||||
// check outcomes (a failed check then a move-on is still visited); results
|
||||
// feed nothing; there is NO fallback current (before the first tagged check
|
||||
// nothing is current). Untagged checks are legal ad-hoc checks — they move
|
||||
// nothing. NodeIds missing from the current tree are ignored (a rewrite
|
||||
// may have dropped the node).
|
||||
|
||||
/** Minimal structural message slice — anything with typed parts (mirrors
|
||||
* phases.ts: pure, no runtime deps, safe on server, client, and in tests). */
|
||||
@@ -70,48 +66,37 @@ interface PartsCarrier {
|
||||
}
|
||||
|
||||
export interface DerivedPlanStatuses {
|
||||
/** The node the most recent tagged quiz call is about; null before any. */
|
||||
current: string | null;
|
||||
/** Node ids demonstrated by the learner (passed checks). */
|
||||
done: ReadonlySet<string>;
|
||||
/** Nodes referenced by earlier tagged calls — been there, monotonic. */
|
||||
visited: ReadonlySet<string>;
|
||||
}
|
||||
|
||||
/** Statuses of a node: the derived view the plan panel should render. */
|
||||
export function derivedNodeStatus(statuses: DerivedPlanStatuses, nodeId: string): PlanNodeStatus {
|
||||
if (statuses.current === nodeId) return "current";
|
||||
if (statuses.done.has(nodeId)) return "done";
|
||||
if (statuses.visited.has(nodeId)) return "visited";
|
||||
return "upcoming";
|
||||
}
|
||||
|
||||
export function derivePlanStatuses(tree: PlanTree, messages: readonly PartsCarrier[]): DerivedPlanStatuses {
|
||||
const ids = new Set(tree.nodes.map((n) => n.id));
|
||||
const done = new Set<string>();
|
||||
let current: string | null = null;
|
||||
|
||||
const runs: string[] = []; // distinct tagged nodes in call order, CONSECUTIVE
|
||||
// duplicates collapsed (several checks on one node = one stay on it)
|
||||
for (const m of messages) {
|
||||
for (const part of m.parts) {
|
||||
if (part.type !== "tool-quiz") continue;
|
||||
const nodeId = (part.input as { nodeId?: unknown } | undefined)?.nodeId;
|
||||
if (typeof nodeId !== "string" || !ids.has(nodeId)) continue; // untagged/unknown = lost signal
|
||||
if (part.state === "input-available") {
|
||||
current = nodeId; // a check is out — that's where we are
|
||||
} else if (part.state === "output-available") {
|
||||
if ((part.output as { correct?: unknown } | undefined)?.correct === true) done.add(nodeId);
|
||||
// failed checks change nothing: resume teaching, re-check later
|
||||
}
|
||||
if (typeof nodeId !== "string" || !ids.has(nodeId)) continue; // untagged/unknown — moves nothing
|
||||
if (runs.at(-1) !== nodeId) runs.push(nodeId);
|
||||
}
|
||||
}
|
||||
|
||||
if (current === null) {
|
||||
// No check out: the first node (authoring order) whose prerequisites
|
||||
// are all demonstrated and that isn't done itself — the next lesson.
|
||||
const prereqs = new Map<string, string[]>();
|
||||
for (const e of tree.edges) prereqs.set(e.to, [...(prereqs.get(e.to) ?? []), e.from]);
|
||||
current =
|
||||
tree.nodes.find(
|
||||
(n) => !done.has(n.id) && (prereqs.get(n.id) ?? []).every((p) => done.has(p)),
|
||||
)?.id ?? null;
|
||||
}
|
||||
return { current, done };
|
||||
const current = runs.at(-1) ?? null;
|
||||
// Visited = been there in an EARLIER stay: everything before the current
|
||||
// run. A revisit (a→b→a) puts a in visited AND current — current wins at
|
||||
// render (derivedNodeStatus), visited records it has been left before.
|
||||
const visited = new Set(runs.slice(0, -1));
|
||||
return { current, visited };
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
/**
|
||||
* Probe-phase contracts (the probe→teach split, docs/probe-teach-split.md).
|
||||
*
|
||||
* The prober concludes its phase with `probe_done` — a server-side tool
|
||||
* whose INPUT is the learner assessment (authored by the model, streamed to
|
||||
* the learner live, durable in the transcript by the tool result itself).
|
||||
* The first `probe_done` in the transcript is the probe→planning seam: the
|
||||
* teacher agent is constructed from that point on, and the tool's execute
|
||||
* queues the handoff steer through the wake pipeline so the teacher starts
|
||||
* immediately after the prober's turn ends (auto-wake, user-decided).
|
||||
*
|
||||
* Assessment shape (v1 — refine later): the probe was a binary search per
|
||||
* dependency strand; the assessment formalizes exactly that. A list of the
|
||||
* strands the lesson will depend on, each with a free-form read of where
|
||||
* the learner's edge sits.
|
||||
*/
|
||||
|
||||
/** One dependency strand the planned lesson will rest on. */
|
||||
export interface ProbeStrand {
|
||||
/** Name/description of the strand (e.g. "the derivative as a limit"). */
|
||||
strand: string;
|
||||
/** Free-form: where the learner's edge sits on this strand — floor,
|
||||
* ceiling, misconceptions found, what to build from. */
|
||||
assessment: string;
|
||||
}
|
||||
|
||||
/** The prober's read of the learner, carried by probe_done. */
|
||||
export interface ProbeAssessment {
|
||||
strands: ProbeStrand[];
|
||||
/** Optional overall synthesis: the learner's apparent goal and edge. */
|
||||
summary?: string;
|
||||
}
|
||||
|
||||
/** probe_done tool input (the assessment the prober authors). */
|
||||
export interface ProbeDoneInput {
|
||||
assessment: ProbeAssessment;
|
||||
}
|
||||
|
||||
/** probe_done tool output. */
|
||||
export interface ProbeDoneResult {
|
||||
ok: boolean;
|
||||
/** Refusal reason when ok === false (open question, already concluded). */
|
||||
error?: string;
|
||||
assessment?: ProbeAssessment;
|
||||
}
|
||||
Reference in New Issue
Block a user