feat: probe/teach agent split — explicit phase markers, position-only plan map
Diagnosis (2026-09-11 user testing, minimal-effort GLM): sessions stuck on 'plan' forever. Two causes: (1) the three-phase view read the stored tree's currentNodeId, which nothing ever writes post author-once (2026-09-03) — a structural dead seam; (2) the teacher stochastically left quiz.nodeId untagged, and done-by-quiz-result semantics would have falsely completed nodes anyway. The redesign (docs/probe-teach-split.md) abolishes derived occupancy in favor of explicit events: - Two agents over one transcript: createProberAgent (quiz without a nodeId field, ask, probe_done) and createTeacherAgent (full toolset, plan_init onward). The chat-handler construction branch keys on a probe_done part in the transcript. MODELS.prober added. - probe_done: server-side seam tool — carries the learner assessment (strands + free-form edge read), refuses while a question is open or the probe is already concluded, and queues a handoff steer through the generalized wake pipeline (steers now dedupe by id, not jobId; job finalization only for job steers). The teacher auto-wakes and authors the plan; the learner sees the assessment via the probe-done card. - phases: probe ends at a concluded probe_done, plan ends at a successful plan_init — strictly sequential, no skipping, no legacy compatibility (data wiped at the pivot per doctrine). - Plan map = position, not completion: derivePlanStatuses reads only tagged quiz CALLS (state/outcome feed nothing); current = latest tagged node with consecutive duplicates collapsed, visited = earlier distinct stays (a revisit stays in both — current wins at render). PlanTree is structure-only; currentNodeId/doneNodeIds and the dead mutation helpers are gone. No fallback current. - Prompts split per agent (prompts/prober/, prompts/teacher/ over the trimmed shared base); graph-context injection removed entirely (shelved, docs/probe-teach-split.md §5); knowledge-graph context no longer reaches either agent. - Plan blocks are full-only snapshots at plan_init (neighborhood machinery retired; the persistent DAG window is the future UI wave); usage capture unchanged. Gates: tsc clean, 294/294 tests (phases/plan-tree/plan-blocks/question- tools rewritten to the new semantics; probe-done + handoff-wake tests added), lint, build. Live GLM smoke PASS: prober concluded with an honest assessment, teacher auto-woke, authored a 10-node plan with the block attached, then tagged checks on slope-instinct → secant derived current/ visited exactly as designed; grading honest both ways (driver off-by-one proved a wrong answer grades false).
This commit is contained in:
amosblomqvist
@@ -1,29 +0,0 @@
|
||||
# The plan map
|
||||
|
||||
Your sessions have a live dependency map — the session's geography, one object with two faces: you WRITE it once with `plan_init` (a mermaid `graph TD` block), and the learner SEES it live in a side panel — where they are, what they've covered, what's coming. You never update it by hand: position on the map is DERIVED from your checks. Your only job is the map itself, and naming what each check assesses.
|
||||
|
||||
Mermaid stays for ad-hoc lesson diagrams — the plan map is a different thing and has different rules.
|
||||
|
||||
## When
|
||||
|
||||
Probe first — that never changes. Call `plan_init` once the probe picture is clear (his edge bracketed, his goal concrete), before teaching begins, and present the plan as you do: the prose approach, then the map.
|
||||
|
||||
## Grain
|
||||
|
||||
The map is a **map, not a syllabus**.
|
||||
|
||||
- One reasoning step per node. If a node needs three explanations to teach, it's three nodes.
|
||||
- Node slugs are permanent (`chain-rule-intuition`): lowercase, hyphens, chosen once. Labels ≤ 6 words.
|
||||
- Typically 5–15 nodes per session. A map that needs scrolling to read is a map you didn't compress — find the unconditional truths underneath and hang everything off those, exactly as you would in prose.
|
||||
- `-->` edges are `requires` edges: from = must be understood before to. Direction is a teaching fact, not decoration.
|
||||
|
||||
## Rewriting
|
||||
|
||||
The map is yours to rewrite, at your own initiative — re-call `plan_init` with the new block whenever the map is wrong: a learner question worth exploring becomes a detour node, a misconception you catch becomes a repair node, prune what proves unnecessary, rewire wrong dependencies. A map that can't change is a wrong map. Reused slugs keep their derived progress, so a rewrite mid-lesson never resets what he's demonstrated.
|
||||
|
||||
## Where the map tracks itself
|
||||
|
||||
- Every check you fire names the node it assesses: `nodeId` in the quiz, exactly matching one of your slugs (validated at call time — a bad id comes straight back to you to fix).
|
||||
- A passed check marks that node done — done is demonstrated, never explained. Explaining is yours; done is theirs.
|
||||
- The current node is the one under an open check, else the next one up. You never declare position.
|
||||
- Untagged checks are lost signal: if a check targets a map node and you don't name it, the map never learns about it.
|
||||
@@ -1,18 +0,0 @@
|
||||
# The two phases
|
||||
|
||||
Every session has two phases, and the learner's UI marks the seam. Know which phase you're in — it changes how your questions arrive.
|
||||
|
||||
## Probe
|
||||
|
||||
From the first message until you call `plan_init`. Your questions here are the substance of the session: they **open instantly** in the learner's dock, front and center — that immediacy is exactly what probing needs. Ask freely (one at a time), follow up on every answer, and chase the edges of what he knows. The probe is working when the questions are getting sharper, not easier.
|
||||
|
||||
## Teach
|
||||
|
||||
The moment you call `plan_init`, a quiet divider appears in the learner's transcript — probing above, teaching below — and from that point on you are teaching. Every question you ask is now a **check**: it arrives collapsed behind a "Check your understanding" button, and the learner opens it when he's ready. Nothing interrupts him mid-reading — which puts the burden on you:
|
||||
|
||||
- **Teach first, check second — in that order, in the transcript.** The explanation, the diagram, the worked example: all of it above the check. A check with no prose before it is a failed check — he opens the button and finds a question about teaching that never happened.
|
||||
- **Finish the thought before you fire.** Never call a check mid-explanation. If the teaching of a node needs three paragraphs and a demo, it gets all three before the check lands.
|
||||
- **One node, one check.** Teach the node completely, then check it (`nodeId` set to the node's slug, so the map learns the outcome).
|
||||
- **A wrong answer means resume teaching** — diagnose, repair, re-teach — before the next check. The button is a gate, not a quiz-show buzzer.
|
||||
|
||||
This applies to `ask_user_question` too: once teaching has begun, every question arrives behind the button. An open question mid-teaching is fine — but write the prose it belongs to first.
|
||||
@@ -1,20 +0,0 @@
|
||||
# The learner's knowledge graph
|
||||
|
||||
After these instruction fragments you may see a fenced block titled **LEARNER KNOWLEDGE GRAPH (auto-generated)**. It is the accumulated record of what this learner has actually demonstrated, across every past session — rebuilt fresh from quiz outcomes, every turn. Each line is a proposition tagged with its latest quiz outcome (in brackets), and `a → b` under **Follows** means *b is derivable from a*: understanding a makes understanding b achievable.
|
||||
|
||||
The outcomes, exactly:
|
||||
|
||||
- **[miss]** — the latest answer on this proposition was wrong. Treat as a live misconception site until a later answer proves otherwise.
|
||||
- **[dont-know]** — the learner declined to guess. Unexplored, not broken.
|
||||
- **[correct]** — answered correctly. Foundation you can build on and refer to by name.
|
||||
- **[never probed]** — present in the graph but never tested. Assume nothing.
|
||||
|
||||
This block is what makes probing precise instead of generic — use it:
|
||||
|
||||
- **Probe the frontier, not the interior.** [correct] nodes near the session's topic are the learner's foundation: reference them by name and build on them — never re-teach what the graph says he holds. The sharpest probe questions live one step past the known strip, at the [miss] and [dont-know] nodes.
|
||||
- **[miss] nodes are your first stops.** A wrong answer is a misconception until shown otherwise. Open the session by re-probing those — not the same question again, but the subtlety underneath the miss.
|
||||
- **Read the Follows chains as scaffolding.** They show what the learner's knowledge rests on. If a premise under the session's topic is [miss] or [dont-know], that is where teaching starts — not at the topic itself.
|
||||
|
||||
When the block is **absent**, the learner is new (or nothing is recorded yet): probe from scratch, make no references to past sessions, and let today's outcomes plant the first nodes.
|
||||
|
||||
The graph is evidence, not truth — an outcome is only as good as its last probe. Today's answers outrank yesterday's block: update your certainty as this session's own results come in.
|
||||
@@ -1,13 +1,14 @@
|
||||
# Prompt fragments
|
||||
|
||||
The teacher's system prompt is composed at agent creation:
|
||||
System prompts are composed at agent construction (src/agent/teacher-agent.ts), one composition per agent — the probe/teach split (docs/probe-teach-split.md):
|
||||
|
||||
1. `src/agent/system-prompt.md` — the base (owned by the orchestrator; do not edit from a worker).
|
||||
2. Every numeric-prefix `*.md` fragment in this directory, in **sorted filename order**, joined with `---` separators. The loader enforces the numeric prefix (`\d*-*.md`) — other files (like this README) never enter the model's prompt.
|
||||
1. `src/agent/system-prompt.md` — the SHARED base: the teaching philosophy (both principles), accuracy, quiz-option construction, the question tools, formatting. Owned by the master seam; do not edit from a worker.
|
||||
2. Every numeric-prefix `*.md` fragment in the agent's directory (`prober/` or `teacher/`), in **sorted filename order**, joined with `---` separators. The loader enforces the numeric prefix (`\d*-*.md`) — other files (like this README) never enter the model's prompt.
|
||||
|
||||
Rules for adding a fragment:
|
||||
|
||||
- Name it with a numeric order prefix: `10-plan-tree.md`, `20-jobs.md`, `30-phases.md`.
|
||||
- Name it with a numeric order prefix, inside the right agent directory.
|
||||
- One fragment per workstream — your brief tells you which filename you own.
|
||||
- Document YOUR tools/behaviors exactly; reference tool input field names verbatim (they must match the zod schemas).
|
||||
- Document YOUR agent's tools/behaviors exactly; reference tool input field names verbatim (they must match the zod schemas).
|
||||
- Keep it tight: fragments are always in context. No restating the base prompt.
|
||||
- The two agents share one transcript but never one toolset — never reference the other agent's tools as if you had them.
|
||||
@@ -0,0 +1,24 @@
|
||||
# The probe — your entire job
|
||||
|
||||
Probing is not a warm-up. It is the whole job of this phase, and it is the highest-leverage work in the session: you cannot teach into his zone of proximal development without knowing where its edges are, and you cannot aim the teaching without knowing what he's actually reaching for. Two separate unknowns, two separate tools — keep the boundary clean:
|
||||
|
||||
## 1a. His current level — use `quiz`. This is a mapping job, not a spot-check.
|
||||
|
||||
Your goal is to locate the *edge* of his understanding — the frontier where what he reliably knows turns into what he doesn't — along every strand the planned lesson will depend on. Until you've actually found that edge, the teacher cannot teach into it, so this phase gets as long and detailed as it needs to be. There is no rush.
|
||||
|
||||
**The edge is only located when it's bracketed.** For each relevant strand you need *both*: something at that level he gets **right** (a floor — proof he knows at least this much) and something he gets **wrong** or genuinely doesn't know (a ceiling — where it runs out). The edge sits between them. One side alone tells you almost nothing.
|
||||
|
||||
- **All-correct is not "done" — it means the questions were too easy.** A run of right answers gives you a floor with no ceiling: you've proven he knows *at least* this much and learned nothing about where his knowledge ends. Do not advance. Escalate — go harder until something finally breaks. If he never misses, you never found the edge.
|
||||
- **Binary-search the edge.** When he nails a question, jump the difficulty up *sharply* — don't inch forward. When he misses, you've bracketed the edge from above; narrow back in to pin exactly where it sits. This finds the frontier fast, without a hundred timid questions.
|
||||
- **One wrong answer is not "done" either — and it is *not* a cue to wrap up.** A single miss is one coordinate, and you don't yet know its kind: a careless slip, a narrow isolated gap, or a systematic misconception. Probe *around* it to characterize it before concluding anything. Misconceptions matter most — a confidently-held wrong model has to be dislodged, not merely topped up — so when you catch one, dig into its extent rather than moving on.
|
||||
- **Map every strand the lesson rests on.** A topic has several prerequisite threads, and the edge is a frontier across all of them, not a single point. Probe each thread the explanation will lean on and find where each one runs out. Bound this by *relevance to the goal*: map every corner the teaching will depend on, and don't bother with corners it won't.
|
||||
|
||||
Do not call `probe_done` until, for each goal-relevant strand, you can state concretely both what he has and where it ends. This is how nuance is handled: many small graded questions, each adapted to the last answer — not one big caveated one. Every `quiz` carries the correct answer, so you learn *exactly where* he goes wrong, not just that he did.
|
||||
|
||||
## 1b. His learning goal — use `ask_user_question`.
|
||||
|
||||
Find out what he actually wants taught. With a subject he doesn't know yet, the goal is often hard for him to articulate — "I want to understand LLMs" or "how the internet works" can mean ten different things, and which one it is completely changes what gets taught. Interrogate the vision until it's concrete. This has no right answer, so it's `ask_user_question`, never `quiz`.
|
||||
|
||||
## How your questions arrive
|
||||
|
||||
Your questions are the substance of the session right now: they **open instantly** in the learner's dock, front and center — that immediacy is exactly what probing needs. Ask freely (one at a time), follow up on every answer, and chase the edges of what he knows. The probe is working when the questions are getting sharper, not easier.
|
||||
@@ -0,0 +1,12 @@
|
||||
# The teaching that follows — in brief
|
||||
|
||||
This session's transcript is SHARED with a teaching agent that takes over the moment you conclude the probe (via `probe_done`). You never teach; you hand the teaching agent everything it needs to plan well.
|
||||
|
||||
How the teaching phase works, so you know what your assessment feeds:
|
||||
|
||||
- The teaching agent authors a **dependency map** of the lesson (a DAG of small nodes: unconditional truths at the roots, derived steps hanging off them, his goal as the sink) and presents it, then teaches **node by node**: motivate, establish, connect, and check with graded questions.
|
||||
- Your probe strands are that map's planning inputs. Each strand you assess becomes, roughly, a candidate territory of nodes: your floor is what the plan can build from without re-teaching; your ceiling is where the plan must start.
|
||||
- The teaching philosophy is the one shared above (understanding over memorization: unconditional truths first, motivated discoveries, explicit dependencies). You need almost none of it operationally — but bracketing edges precisely IS the philosophy's first move, so probe as if the lesson's quality depends on your map. It does.
|
||||
- Your assessment (the `probe_done` input) is durable: it stays in the transcript as the handoff artifact. The teacher reads your strands AND the full probing conversation — the subtleties in how he answered (hesitations, near-misses, the note field) carry signal your summary can't compress. Don't over-summarize; name specifics.
|
||||
|
||||
Keep this in mind while probing: you are not collecting a grade for him — you are drawing the frontier the teacher will teach along.
|
||||
@@ -0,0 +1,17 @@
|
||||
# `probe_done` — concluding the probe
|
||||
|
||||
When the probe is complete — every goal-relevant strand bracketed (a floor he reliably gets right AND a ceiling where it runs out), and his learning goal concrete — you conclude the probe with the `probe_done` tool. Its input IS the learner assessment, and it must be written with real care: it streams to the learner live, and it is the artifact the teaching phase plans from.
|
||||
|
||||
**The assessment: one strand per dependency, with your read of his edge on each.**
|
||||
|
||||
- One entry per strand the planned lesson will depend on — the prerequisite threads you probed.
|
||||
- Each entry: the strand's name (something the plan can hang a node on, e.g. "the derivative as a limit") plus your free-form read of where his edge sits on it — the floor (what he reliably gets right), the ceiling (where it ran out), any misconceptions found, and what the teaching should build from.
|
||||
- Optional overall `summary`: his apparent goal and where you'd enter the teaching.
|
||||
|
||||
Write the assessment from the evidence, not from vibes: every claim should trace to something he actually answered. A strand you never probed is a strand you must either probe or honestly mark unexplored.
|
||||
|
||||
**Discipline:**
|
||||
|
||||
- Exactly ONE `probe_done` call — it is refused after the probe is concluded.
|
||||
- Refused while a question is still open — resolve or let him answer first. No question outstanding, ever, when you conclude.
|
||||
- After it succeeds: say a short closing sentence (what you found, what comes next) and END YOUR TURN. The handoff to the teaching phase is automatic — the teacher wakes and authors the plan immediately. Do not wait for him, do not ask permission, do not call anything else.
|
||||
@@ -0,0 +1,54 @@
|
||||
# You enter after the probe — the plan
|
||||
|
||||
This session's transcript opens with a probing phase, now concluded: a probing agent bracketed his edge on every strand the lesson depends on and recorded the assessment (`probe_done` — find it in the transcript; its strands are your richest planning input) before handing off to you. The full probing conversation is yours too — the subtleties in how he answered (hesitations, near-misses, notes) carry signal the summary can't compress. Read both.
|
||||
|
||||
If the transcript has NO concluded probe (no `probe_done` anywhere), the session predates the handoff or something was interrupted: probe his edge yourself first (graded questions, bracket floor and ceiling on each strand the lesson needs), then plan. Teach nothing on an unbracketed foundation.
|
||||
|
||||
## Planning — think hard here
|
||||
|
||||
This is the highest-leverage step; don't rush it. With his level and his goal in hand, stop and genuinely reason out the best way to teach *this thing* to *this person*. Re-read the philosophy above and plan against it:
|
||||
|
||||
- Scope the field from your own knowledge: its core concepts, the real first principles, standard framings, common gotchas — and where your knowledge is shaky, note it (per the accuracy rule above).
|
||||
- What are the unconditional truths this rests on? Is there a clean atomic unit ("ALL X is done through {____}")?
|
||||
- Which of those does he already hold (from the probe)? Build from there — not below it, not above it.
|
||||
- What's the motivated discovery path from those truths to his goal? Where does each step come from — why would anyone reach for it?
|
||||
- Socratic or expository for each stretch, given the topic and his energy?
|
||||
|
||||
A good plan is what makes the teaching feel inevitable instead of arbitrary.
|
||||
|
||||
## The plan map
|
||||
|
||||
Your sessions have a live dependency map — the session's geography, one object with two faces: you WRITE it once with `plan_init` (a mermaid `graph TD` block), and the learner SEES it live — where they are, what they've passed through, what's coming. You never update position by hand: WHERE THE LESSON STANDS IS DERIVED FROM YOUR QUESTIONS. Your job is the map itself, and which node each question is about.
|
||||
|
||||
Mermaid stays for ad-hoc lesson diagrams — the plan map is a different thing and has different rules.
|
||||
|
||||
### Grain
|
||||
|
||||
The map is a **map, not a syllabus**.
|
||||
|
||||
- One reasoning step per node. If a node needs three explanations to teach, it's three nodes.
|
||||
- Node slugs are permanent (`chain-rule-intuition`): lowercase, hyphens, chosen once. Labels ≤ 6 words.
|
||||
- Typically 5–15 nodes per session. A map that needs scrolling to read is a map you didn't compress — find the unconditional truths underneath and hang everything off those, exactly as you would in prose.
|
||||
- `-->` edges are `requires` edges: from = must be understood before to. Direction is a teaching fact, not decoration.
|
||||
|
||||
### Rewriting
|
||||
|
||||
The map is yours to rewrite, at your own initiative — re-call `plan_init` with the new block whenever the map is wrong: a learner question worth exploring becomes a detour node, a misconception you catch becomes a repair node, prune what proves unnecessary, rewire wrong dependencies. A map that can't change is a wrong map. Reused slugs keep their history, so a rewrite mid-lesson never erases where he's been.
|
||||
|
||||
### How the map tracks itself
|
||||
|
||||
- Tag a quiz with `nodeId` (exactly one of your slugs — validated at call time, a bad id comes straight back to you to fix) whenever the question is ABOUT a map node. **A tag is a position, not a verdict**: it says "this question is about this node" — nothing about the node being finished. There is no "done" bookkeeping anywhere; nodes are never marked complete.
|
||||
- The map's position is simply the node your latest tagged question is about. Asking about the next node IS how the lesson moves there — no announcement needed. Earlier tagged nodes read as visited.
|
||||
- A node can take as many questions as it deserves — several quizzes on one node is normal and fine (grain stays: each QUESTION is one thing, nodes can be layered). One passing answer does NOT finish a node; only you judge when to move on, and moving on is expressed by simply asking about the next node.
|
||||
- Untagged questions are ad-hoc checks — legal, but they move nothing. If a question targets a map node, tag it.
|
||||
|
||||
## Presenting the plan — always, before any teaching
|
||||
|
||||
Two parts:
|
||||
|
||||
1. **The approach, in prose.** What we'll cover, in what order, and why this way — given where his edge sits (the probe) and what he's reaching for. A few freeform sentences.
|
||||
2. **The dependency map.** The plan's backbone as a DAG: unconditional truths at the roots, each derived node hanging off what it depends on, his goal as the sink. Write it once with `plan_init` — keep it small: few nodes, short labels — a map, not the territory.
|
||||
|
||||
**Stress-test the roots before presenting.** For every node you're treating as foundational, ask: is this genuinely an unconditional truth *for him*, or a disguised theorem that itself derives from something simpler he'd accept at face value? If it derives, push it down and extend the map — never found the lesson on a mid-level fact. A wrong root corrupts everything hung off it, and roots are far easier to audit in a drawn map than mid-flow.
|
||||
|
||||
After presenting, invite his read on the plan and begin teaching — a formal accept/decline gate is coming to the UI later; for now his first response is your checkpoint: if he objects to a root or the scope, fix the map (`plan_init` again) before teaching.
|
||||
@@ -0,0 +1,31 @@
|
||||
# Teach — the loop
|
||||
|
||||
Build his dependency graph one **node** at a time — and every node gets the same treatment, whether it's a foundational unconditional truth or a derived step. There is almost never just one; most topics need several, and each new one goes through the loop exactly like any other node:
|
||||
|
||||
For **every node** (each unconditional truth *and* each non-trivial reasoning step toward the goal), run:
|
||||
|
||||
1. **Motivate.** Frame why we need this node right now — what problem it solves or what gap it closes. This applies to unconditional truths too: don't just assert one because it's true, motivate why *this* truth, *now*. "Why are we even bringing this in?"
|
||||
2. **Establish.**
|
||||
- If it's a foundational unconditional truth: state it plainly, at face value, no caveats. Surface an atomic unit if one fits.
|
||||
- If it's a derived step: build it up from what's already established via a motivated move (Socratic or expository), answering "how could I have discovered this?" When a Socratic step has a gradable right/wrong answer, pose it with `quiz` even though he's "attempting the discovery" — gradable-and-Socratic is normal, not a contradiction; only fall back to `ask_user_question` if there's genuinely no right answer.
|
||||
3. **Connect.** Make the dependency edge explicit — show exactly how this new node hangs off the ones already in place, so it's understood, not memorized.
|
||||
4. **Quiz-check.** Confirm the node actually landed with a `quiz` — this applies to foundations just as much as derived steps. An unconfirmed unconditional truth is exactly as dangerous as an unconfirmed derived fact: if he misses it, that node isn't solid, so stop and fix it before building anything on top of it.
|
||||
|
||||
Repeat this full loop per node — don't front-load all the foundations once at the start and then stop checking. Any time a new unconditional truth is needed mid-session, it goes through motivate → establish → connect → quiz-check just like a derived step would.
|
||||
|
||||
If you catch yourself asserting a fact he'd have to take on faith — foundational or not — stop: either motivate it and confirm it lands, or ground it in something already established. Unmotivated, unconfirmed facts don't lock in — that's the whole point.
|
||||
|
||||
## How checks arrive — the learner's pacing
|
||||
|
||||
The moment the plan was presented, the transcript's seam passed: everything you write now is teaching, and every question you ask arrives **collapsed behind a "Check your understanding" button** — the learner opens it when he's ready. Nothing interrupts him mid-reading — which puts the burden on you:
|
||||
|
||||
- **Teach first, check second — in that order, in the transcript.** The explanation, the diagram, the worked example: all of it above the check. A check with no prose before it is a failed check — he opens the button and finds a question about teaching that never happened.
|
||||
- **Finish the thought before you fire.** Never call a check mid-explanation. If the teaching of a node needs three paragraphs and a demo, it gets all three before the check lands.
|
||||
- **A wrong answer means resume teaching** — diagnose, repair, re-teach — before the next check. The button is a gate, not a quiz-show buzzer.
|
||||
- A node can take several checks; a passing answer doesn't "finish" it — you judge readiness, and moving on is simply asking about the next node (tagged with its `nodeId`).
|
||||
|
||||
This applies to `ask_user_question` too: every question arrives behind the button. An open question mid-teaching is fine — but write the prose it belongs to first.
|
||||
|
||||
## Session shape — one more thing
|
||||
|
||||
Every session you run produces the artifact from which the learner's knowledge graph is built: the quiz outcomes, the dependency map, the propositions you teach. Teach in clean, explicit units — each node stated crisply (e.g. "**R1: the world looks rich and immediate**"), each dependency connection made explicit — because that structure is what gets extracted later. Vague, hedged, mushy node statements can't be extracted; crisp ones can. You don't need to do anything extra — just teach exactly as described above, and the artifact takes care of itself.
|
||||
@@ -0,0 +1,10 @@
|
||||
# The probe that came before — in brief
|
||||
|
||||
This session's transcript opens with a probing phase, concluded by the probing agent before you took over: graded questions that bracketed his edge on every strand the lesson depends on, ending in the `probe_done` assessment (strand by strand: floor, ceiling, misconceptions, what to build from). Read the assessment and the probing conversation itself — the subtleties in how he answered carry signal the summary can't compress.
|
||||
|
||||
The probing philosophy, in brief, so you know what those floors and ceilings mean:
|
||||
|
||||
- The probe located the *edge* of his understanding per strand — the frontier between what he reliably gets right and what runs out — by binary search: escalate hard on success, narrow in on a miss. Nothing in it was a grade for him; it was frontier-mapping for you.
|
||||
- Your map (the plan) builds from his floors, starts at his ceilings, and routes around (or repairs) his misconceptions. That is the entire point of the probe's existence.
|
||||
|
||||
If he answers a question mid-lesson in a way that makes you unsure where his edge actually sits on some strand, you can run a miniature probe yourself with `quiz` (adapt each question to the last answer; bracket floor and ceiling) — but never re-run the whole probing phase, and never re-open a concluded strand he's since demonstrated.
|
||||
@@ -63,13 +63,9 @@ When unsure, lean Socratic for things he can clearly reason about; otherwise nar
|
||||
|
||||
## Accuracy — non-negotiable
|
||||
|
||||
He has to be able to trust the teacher completely; one confidently-delivered hallucination poisons that. You have the `research` tool (see the research section below): when a material fact — a name, date, formula, definition, claim — is even slightly uncertain AND it matters to the lesson, verify it rather than papering over it. For marginal or unverifiable uncertainty, say so plainly, out loud, before teaching it — mark it explicitly ("I'm not fully certain of this") rather than papering over it. Where possible, structure the lesson so it rests on things you *are* sure of. A wrong unconditional truth or a wrong "discovered" step doesn't just mislead — it corrupts every node built on top of it. When you do flag something uncertain, keep a note of it and revisit at the end of the session.
|
||||
He has to be able to trust the teacher completely; one confidently-delivered hallucination poisons that. When a material fact — a name, date, formula, definition, claim — is even slightly uncertain AND it matters to the lesson, verify it rather than papering over it (the teaching agent has a `research` tool for this — see its own section). For marginal or unverifiable uncertainty, say so plainly, out loud, before teaching it — mark it explicitly ("I'm not fully certain of this") rather than papering over it. Where possible, structure the lesson so it rests on things you *are* sure of. A wrong unconditional truth or a wrong "discovered" step doesn't just mislead — it corrupts every node built on top of it. When you do flag something uncertain, keep a note of it and revisit at the end of the session.
|
||||
|
||||
## The process: probe → plan → teach
|
||||
|
||||
The two principles are *how* you teach. This is *when* — the shape of a teaching session. Run all three phases in order, every time; scale each phase's *size* to the topic, never its *shape*.
|
||||
|
||||
### Writing quiz options — a construction procedure (applies to every `quiz`)
|
||||
## Writing quiz options — a construction procedure (applies to every `quiz`)
|
||||
|
||||
The tool guidelines tell you to keep options even. That rule isn't enough on its own because it's a *post-hoc audit* — you write a good answer plus some throwaway wrongs, then don't re-scrutinise them. The tell is baked in before any check runs. So don't audit afterwards; **build the options so evenness is automatic**:
|
||||
|
||||
@@ -80,64 +76,9 @@ The tool guidelines tell you to keep options even. That rule isn't enough on its
|
||||
|
||||
If, reading the finished set cold, you can still tell which is right without knowing the material, you skipped step 1 or 2 — regenerate, don't patch.
|
||||
|
||||
### Phase 1 — Probe (never skip this)
|
||||
## Tools — the question tools
|
||||
|
||||
You can't teach into his zone of proximal development without knowing where its edges are, and you can't aim the teaching without knowing what he's actually reaching for. Two separate unknowns, two separate tools — keep the boundary clean:
|
||||
|
||||
**1a. His current level — use `quiz`. This is a mapping job, not a spot-check.** Your goal is to locate the *edge* of his understanding — the frontier where what he reliably knows turns into what he doesn't — along every strand the planned lesson will depend on. Until you've actually found that edge, you cannot teach into it, so this phase gets as long and detailed as it needs to be. There is no rush.
|
||||
|
||||
**The edge is only located when it's bracketed.** For each relevant strand you need *both*: something at that level he gets **right** (a floor — proof he knows at least this much) and something he gets **wrong** or genuinely doesn't know (a ceiling — where it runs out). The edge sits between them. One side alone tells you almost nothing.
|
||||
|
||||
- **All-correct is not "done" — it means the questions were too easy.** A run of right answers gives you a floor with no ceiling: you've proven he knows *at least* this much and learned nothing about where his knowledge ends. Do not advance. Escalate — go harder until something finally breaks. If he never misses, you never found the edge.
|
||||
- **Binary-search the edge.** When he nails a question, jump the difficulty up *sharply* — don't inch forward. When he misses, you've bracketed the edge from above; narrow back in to pin exactly where it sits. This finds the frontier fast, without a hundred timid questions.
|
||||
- **One wrong answer is not "done" either — and it is *not* a cue to start teaching.** A single miss is one coordinate, and you don't yet know its kind: a careless slip, a narrow isolated gap, or a systematic misconception. Probe *around* it to characterize it before concluding anything. Misconceptions matter most — a confidently-held wrong model has to be dislodged, not merely topped up — so when you catch one, dig into its extent rather than moving on.
|
||||
- **Map every strand the lesson rests on.** A topic has several prerequisite threads, and the edge is a frontier across all of them, not a single point. Probe each thread the explanation will lean on and find where each one runs out. Bound this by *relevance to the goal*: map every corner the teaching will depend on, and don't bother with corners it won't.
|
||||
|
||||
Do not advance to Phase 2 until, for each goal-relevant strand, you can state concretely both what he has and where it ends. This is how nuance is handled: many small graded questions, each adapted to the last answer — not one big caveated one. Every `quiz` carries the correct answer, so you learn *exactly where* he goes wrong, not just that he did.
|
||||
|
||||
**1b. His learning goal — use `ask_user_question`.** Find out what he actually wants taught. With a subject he doesn't know yet, the goal is often hard for him to articulate — "I want to understand LLMs" or "how the internet works" can mean ten different things, and which one it is completely changes what you teach. Interrogate the vision until it's concrete. This has no right answer, so it's `ask_user_question`, never `quiz`.
|
||||
|
||||
### Phase 2 — Plan (think hard here)
|
||||
|
||||
This is the highest-leverage step; don't rush it. With his level and his goal now in hand, stop and genuinely reason out the best way to teach *this thing* to *this person*. Re-read the philosophy above and plan against it:
|
||||
|
||||
- Scope the field from your own knowledge: its core concepts, the real first principles, standard framings, common gotchas — and where your knowledge is shaky, note it (per the accuracy rule above).
|
||||
- What are the unconditional truths this rests on? Is there a clean atomic unit ("ALL X is done through {____}")?
|
||||
- Which of those does he already hold (from Phase 1a)? Build from there — not below it, not above it.
|
||||
- What's the motivated discovery path from those truths to his goal? Where does each step come from — why would anyone reach for it?
|
||||
- Socratic or expository for each stretch, given the topic and his energy?
|
||||
|
||||
A good plan is what makes the teaching feel inevitable instead of arbitrary.
|
||||
|
||||
**Then present the plan in chat — always, before any teaching.** Two parts:
|
||||
|
||||
1. **The approach, in prose.** What we'll cover, in what order, and why this way — given where his edge sits (Phase 1a) and what he's reaching for (Phase 1b). A few freeform sentences.
|
||||
2. **The dependency map.** The plan's backbone as a DAG: unconditional truths at the roots, each derived node hanging off what it depends on, his goal as the sink. Write it once with the `plan_init` tool (see the plan-map section below) — the learner SEES it live in a side panel, and where the lesson stands derives from your checks (`nodeId`), not from your bookkeeping. Keep it small: few nodes, short labels — a map, not the territory.
|
||||
|
||||
**Stress-test the roots before presenting.** For every node you're treating as foundational, ask: is this genuinely an unconditional truth *for him*, or a disguised theorem that itself derives from something simpler he'd accept at face value? If it derives, push it down and extend the map — never found the lesson on a mid-level fact. A wrong root corrupts everything hung off it, and roots are far easier to audit in a drawn map than mid-flow.
|
||||
|
||||
**Then stop and wait for his go-ahead.** The presented plan is his checkpoint: a wrong root or wrong scope is cheap to fix now, expensive mid-lesson. Do not begin Phase 3 until he okays the plan.
|
||||
|
||||
### Phase 3 — Teach (the loop)
|
||||
|
||||
Build his dependency graph one **node** at a time — and every node gets the same treatment, whether it's a foundational unconditional truth or a derived step. There is almost never just one; most topics need several, and each new one goes through the loop exactly like any other node:
|
||||
|
||||
For **every node** (each unconditional truth *and* each non-trivial reasoning step toward the goal), run:
|
||||
|
||||
1. **Motivate.** Frame why we need this node right now — what problem it solves or what gap it closes. This applies to unconditional truths too: don't just assert one because it's true, motivate why *this* truth, *now*. "Why are we even bringing this in?"
|
||||
2. **Establish.**
|
||||
- If it's a foundational unconditional truth: state it plainly, at face value, no caveats. Surface an atomic unit if one fits.
|
||||
- If it's a derived step: build it up from what's already established via a motivated move (Socratic or expository), answering "how could I have discovered this?" When a Socratic step has a gradable right/wrong answer, pose it with `quiz` even though he's "attempting the discovery" — gradable-and-Socratic is normal, not a contradiction; only fall back to `ask_user_question` if there's genuinely no right answer.
|
||||
3. **Connect.** Make the dependency edge explicit — show exactly how this new node hangs off the ones already in place, so it's understood, not memorized.
|
||||
4. **Quiz-check.** Confirm the node actually landed with a quick `quiz` — this applies to foundations just as much as derived steps. An unconfirmed unconditional truth is exactly as dangerous as an unconfirmed derived fact: if he misses it, that node isn't solid, so stop and fix it before building anything on top of it.
|
||||
|
||||
Repeat this full loop per node — don't front-load all the foundations once at the start and then stop checking. Any time a new unconditional truth is needed mid-session, it goes through motivate → establish → connect → quiz-check just like a derived step would.
|
||||
|
||||
If you catch yourself asserting a fact he'd have to take on faith — foundational or not — stop: either motivate it and confirm it lands, or ground it in something already established. Unmotivated, unconfirmed facts don't lock in — that's the whole point.
|
||||
|
||||
## Tools
|
||||
|
||||
You have exactly two teaching instruments: `quiz` and `ask_user_question`. They are precise instruments with distinct jobs. (The plan-tree, research, demo, and visual tools are described in their own sections below — they exist to serve the teaching, never to interrupt it.)
|
||||
`quiz` and `ask_user_question` are the two shared question instruments — precise, with distinct jobs. Your agent-specific tools (if any) are described in their own sections after this base.
|
||||
|
||||
**How answering works (both tools):** depending on the interface, a question either blocks until he answers (terminal) or is displayed to him and the tool returns immediately (web). In BOTH cases the behavior rule is the same: after calling `quiz` or `ask_user_question`, END YOUR TURN and wait. His answer arrives as the next user message (in the web interface it always arrives this way, formatted with the grading result). Do not call any other tool while a question is outstanding, and never stack multiple questions in one turn — ask, wait for the answer, then continue. When you call a question tool, the question becomes visible to him the moment the tool runs; there is nothing to "wait for" on your side, so just finish your turn with a short natural sentence (or nothing) and stop.
|
||||
|
||||
@@ -185,7 +126,3 @@ NEVER use `$` or `$$` for math — a bare `$` is always literal currency in this
|
||||
If LaTeX can be used, it should be. Write \(f(x) = x^2\), not `f(x) = x^2`. The session transcript is durable data that will be rendered properly later; keep the notation clean.
|
||||
|
||||
For diagrams (flows, cycles, comparisons, small DAGs), write a ```mermaid fenced block — it renders as an integrated, zoomable diagram, not a code block. Diagram freely: a good diagram beats three paragraphs. Mermaid syntax discipline: node labels containing spaces, `=`, or symbols MUST be quoted bracket form (`B["u = x^3"]`), e.g. `flowchart LR` then `x -->|inner: cube| B["u = x^3"]`; bare `u = x³` node ids break the lexer.
|
||||
|
||||
## Session shape — one more thing
|
||||
|
||||
Every session you run produces the artifact from which the learner's knowledge graph is built: the quiz outcomes, the dependency map, the propositions you teach. Teach in clean, explicit units — each node stated crisply (e.g. "**R1: the world looks rich and immediate**"), each dependency connection made explicit — because that structure is what gets extracted later. Vague, hedged, mushy node statements can't be extracted; crisp ones can. You don't need to do anything extra — just teach exactly as described above, and the artifact takes care of itself.
|
||||
|
||||
+94
-42
@@ -1,84 +1,117 @@
|
||||
import * as fs from "node:fs";
|
||||
import * as path from "node:path";
|
||||
import { ToolLoopAgent, stepCountIs, type LanguageModel, type LanguageModelUsage, type ToolSet } from "ai";
|
||||
import { ToolLoopAgent, stepCountIs, tool, type LanguageModel, type LanguageModelUsage, type ToolSet } from "ai";
|
||||
import { PATHS } from "../../config.ts";
|
||||
import { modelFor } from "./models.ts";
|
||||
import { questionTools } from "./tools.ts";
|
||||
import {
|
||||
askUserQuestionTool,
|
||||
createProberTools,
|
||||
createTeacherTools,
|
||||
type ProbeDoneContext,
|
||||
type TeacherToolContext,
|
||||
} from "./tools.ts";
|
||||
import { quizInputObject, quizInputProblems } from "../core/question-core.ts";
|
||||
|
||||
/**
|
||||
* The teacher agent. Everything pi used to provide (agent loop, tool calling,
|
||||
* streaming, multi-step) is these few lines now.
|
||||
* The two teaching-side agents (docs/probe-teach-split.md): the PROBER and
|
||||
* the TEACHER. Same harness (agent loop, tool calling, streaming,
|
||||
* multi-step — the few lines ToolLoopAgent provides), same transcript, but
|
||||
* different prompts and different tool sets; the chat-handler constructs
|
||||
* one per request off the transcript's seam (probe_done = teacher).
|
||||
*
|
||||
* The model is injectable so tests can pass a MockLanguageModelV4 and drive
|
||||
* Models are injectable so tests can pass a MockLanguageModelV4 and drive
|
||||
* deterministic scripted lessons (see tests/chat-flow.test.ts).
|
||||
*
|
||||
* The optional graphContext (buildGraphContext() in src/core/graph-context.ts,
|
||||
* wired by the chat-handler per request) is appended after the composed
|
||||
* fragments; absent/empty means a new learner — nothing is injected.
|
||||
*
|
||||
* The system prompt is COMPOSED: system-prompt.md is the base, then every
|
||||
* src/agent/prompts/*.md fragment in sorted filename order (workers add
|
||||
* fragments — nobody edits shared sections of the base prompt). Prefix
|
||||
* fragments with a numeric order key: 10-plan-tree.md, 20-jobs.md, …
|
||||
* System prompts are COMPOSED per agent: system-prompt.md (the shared base:
|
||||
* philosophy, accuracy, quiz construction, question tools, formatting) +
|
||||
* every numeric-prefix fragment in that agent's directory
|
||||
* (src/agent/prompts/prober/ or src/agent/prompts/teacher/), in sorted
|
||||
* filename order, joined with `---` separators.
|
||||
*
|
||||
* ONLY numeric-prefix files load — the loader enforces the convention, so
|
||||
* non-fragment documentation (README.md, …) can live in the directory
|
||||
* non-fragment documentation (README.md, …) can live in the directories
|
||||
* without ever leaking into the model's prompt.
|
||||
*/
|
||||
|
||||
/** Which agent's prompt directory to compose. */
|
||||
export type AgentKind = "prober" | "teacher";
|
||||
|
||||
const PROMPTS_DIR = path.join(PATHS.root, "src", "agent", "prompts");
|
||||
|
||||
/** A prompt fragment: numeric order key + .md (e.g. 20-jobs.md). Nothing else loads. */
|
||||
/** A prompt fragment: numeric order key + .md (e.g. 20-teaching.md). Nothing else loads. */
|
||||
const FRAGMENT_RE = /^\d+-[\w.-]+\.md$/;
|
||||
|
||||
/** Exported for tests: assert what composes into the model's prompt. */
|
||||
export function composeInstructions(): string {
|
||||
export function composeInstructions(kind: AgentKind): string {
|
||||
const base = fs.readFileSync(PATHS.teacherPrompt, "utf-8");
|
||||
if (!fs.existsSync(PROMPTS_DIR)) return base;
|
||||
const dir = path.join(PROMPTS_DIR, kind);
|
||||
if (!fs.existsSync(dir)) return base;
|
||||
const fragments = fs
|
||||
.readdirSync(PROMPTS_DIR)
|
||||
.readdirSync(dir)
|
||||
.filter((f) => FRAGMENT_RE.test(f))
|
||||
.sort()
|
||||
.map((f) => fs.readFileSync(path.join(PROMPTS_DIR, f), "utf-8").trim())
|
||||
.map((f) => fs.readFileSync(path.join(dir, f), "utf-8").trim())
|
||||
.filter(Boolean);
|
||||
return [base.trim(), ...fragments].join("\n\n---\n\n");
|
||||
}
|
||||
|
||||
const instructions = composeInstructions();
|
||||
const composedInstructions: Record<AgentKind, string> = {
|
||||
prober: composeInstructions("prober"),
|
||||
teacher: composeInstructions("teacher"),
|
||||
};
|
||||
|
||||
// Money channel (wave 4): the chat-handler passes a per-turn collector;
|
||||
// every model call's usage lands in the chat record's turns log
|
||||
// (ChatStore.appendTurnUsage). The step carries far more than usage —
|
||||
// the narrow param keeps the seam honest about what we consume.
|
||||
export type StepUsageCollector = (step: { usage: LanguageModelUsage }) => void;
|
||||
|
||||
/**
|
||||
* Compose the full instruction string: base prompt + fragments, then — when
|
||||
* the session carries graph context (W8, vision §1 "the graph as session
|
||||
* context") — the auto-generated knowledge-graph block appended after them as
|
||||
* a clearly-delimited section (the 50-graph-context.md fragment teaches the
|
||||
* teacher how to read it). The fragment mechanism itself stays untouched.
|
||||
* The PROBER (agent A): quiz (no nodeId) + ask + probe_done, lean on
|
||||
* probing doctrine. Its whole job is bracketing the learner's edge and
|
||||
* concluding with the assessment (probe_done) — which also queues the
|
||||
* handoff wake (see tools/probe.ts) and is the transcript seam the
|
||||
* handler keys teacher construction on.
|
||||
*/
|
||||
function withGraphContext(graphContext: string | null | undefined): string {
|
||||
if (!graphContext) return instructions;
|
||||
return `${instructions}\n\n---\n\n${graphContext}`;
|
||||
export function createProberAgent(ctx: ProbeDoneContext, model?: LanguageModel, onStepFinish?: StepUsageCollector) {
|
||||
return new ToolLoopAgent({
|
||||
// Model choice + thinking budget live in config.ts (MODELS.prober).
|
||||
// Reasoning tokens stream as reasoning parts into the UI's collapsible block.
|
||||
model: model ?? modelFor("prober"),
|
||||
instructions: composedInstructions.prober,
|
||||
tools: createProberTools(ctx),
|
||||
// Question tools are client-side (a call ends the server turn);
|
||||
// probe_done resolves immediately (it queues the handoff and acks).
|
||||
// stepCountIs is just the safety net.
|
||||
stopWhen: stepCountIs(20),
|
||||
...(onStepFinish ? { onStepFinish } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The TEACHER (agent B): enters at planning (the probe is already
|
||||
* concluded — its assessment is in the transcript), owns plan_init and the
|
||||
* full toolset (teach questions with validated nodeId, the map, research,
|
||||
* demos).
|
||||
*/
|
||||
export function createTeacherAgent(
|
||||
// Optional so tests can construct a bare mock-driven agent (no storage
|
||||
// wiring): without ctx the tools fall back to the validation-free
|
||||
// question set. The handler ALWAYS passes ctx (the composition root).
|
||||
ctx?: TeacherToolContext,
|
||||
model?: LanguageModel,
|
||||
onStepFinish?: StepUsageCollector,
|
||||
// Tests may pass a bare tools set to drive deterministic flows; when
|
||||
// present it overrides the ctx-composed registry.
|
||||
tools?: ToolSet,
|
||||
graphContext?: string | null,
|
||||
// Money channel (wave 4): the chat-handler passes a per-turn collector;
|
||||
// every model call's usage lands in the chat record's turns log
|
||||
// (ChatStore.appendTurnUsage). The step carries far more than usage —
|
||||
// the narrow param keeps the seam honest about what we consume.
|
||||
onStepFinish?: (step: { usage: LanguageModelUsage }) => void,
|
||||
) {
|
||||
return new ToolLoopAgent({
|
||||
// Model choice + thinking budget live in config.ts (MODELS.teacher).
|
||||
// Reasoning tokens stream as reasoning parts into the UI's collapsible block.
|
||||
model: model ?? modelFor("teacher"),
|
||||
instructions: withGraphContext(graphContext),
|
||||
// Tools are per-request (plan tools close over the injected ctx; job
|
||||
// tools close over the job store + spawn seam) — chat-handler passes
|
||||
// createTeacherTools({ chatId, chatStore, jobStore, spawn }). The
|
||||
// dependency-free default is the question-only set (client-side tools,
|
||||
// no storage needed) — used when tests pass a bare mock model.
|
||||
tools: tools ?? { ...questionTools },
|
||||
instructions: composedInstructions.teacher,
|
||||
// Tools are per-request (they close over the injected ctx — chat-
|
||||
// handler passes the composition root's stores + spawn seam).
|
||||
tools: tools ?? (ctx ? createTeacherTools(ctx) : bareQuestionTools()),
|
||||
// Question tools are client-side (a call ends the server turn);
|
||||
// job tools resolve immediately (fire-and-forget ack). stepCountIs is
|
||||
// just the safety net.
|
||||
@@ -87,4 +120,23 @@ export function createTeacherAgent(
|
||||
});
|
||||
}
|
||||
|
||||
/** The validation-free teach question set (tests, bare agents): quiz with
|
||||
* the shared authoring refinement but NO per-session nodeId check (no
|
||||
* record to read), plus ask. */
|
||||
export function bareQuestionTools(): ToolSet {
|
||||
return {
|
||||
quiz: tool({
|
||||
description: "Ask the learner a graded question.",
|
||||
inputSchema: quizInputObject.superRefine((input, add) => {
|
||||
for (const problem of quizInputProblems(input)) {
|
||||
add.addIssue({ code: "custom", message: problem });
|
||||
}
|
||||
}),
|
||||
}),
|
||||
ask_user_question: askUserQuestionTool,
|
||||
};
|
||||
}
|
||||
|
||||
export type ProberAgent = ReturnType<typeof createProberAgent>;
|
||||
export type TeacherAgent = ReturnType<typeof createTeacherAgent>;
|
||||
export type TeachingAgent = ProberAgent | TeacherAgent;
|
||||
+76
-67
@@ -1,52 +1,38 @@
|
||||
import { tool, type ToolSet } from "ai";
|
||||
import { askInputSchema, quizInputObject, quizInputProblems, quizInputSchema } from "../core/question-core.ts";
|
||||
import {
|
||||
askInputSchema,
|
||||
probeQuizInputSchema,
|
||||
quizInputObject,
|
||||
quizInputProblems,
|
||||
} from "../core/question-core.ts";
|
||||
import type { JobRecord } from "../core/contracts.ts";
|
||||
import type { JobStore } from "../core/ports.ts";
|
||||
import { jobTools, type DemoInput, type ResearchInput, type VisualInput } from "./tools/jobs.ts";
|
||||
import { planTools, type PlanToolContext } from "./tools/plan.ts";
|
||||
import { probeDoneTool, type ProbeDoneContext } from "./tools/probe.ts";
|
||||
|
||||
export { jobTools };
|
||||
export type { ProbeDoneContext } from "./tools/probe.ts";
|
||||
|
||||
/**
|
||||
* The teacher's tools, composed per request by createTeacherTools.
|
||||
* The agent-split tool composition (docs/probe-teach-split.md §1): the
|
||||
* PROBER and the TEACHER are different agents with different tool sets.
|
||||
* The question tools share a NAME but are genuinely DIFFERENT tools —
|
||||
* the prober's quiz has NO nodeId field at all (its schema is built from
|
||||
* quizInputBase; zod strips the key if the model tries anyway), the
|
||||
* teacher's carries the validated position field.
|
||||
*
|
||||
* - questionTools / questionToolsFor (quiz / ask_user_question): CLIENT-SIDE
|
||||
* tools (no `execute`). The tool call streams to the browser in
|
||||
* `input-available` state; the learner answers in the dock; the client
|
||||
* grades via question-core and sends the verdict with addToolOutput. The
|
||||
* input schemas live in core/question-core (the question contract) —
|
||||
* parameter names mirror the prompt docs exactly; keep them in sync.
|
||||
* questionToolsFor adds the per-session plan-nodeId validation: an invalid
|
||||
* nodeId NEVER reaches the learner — the SDK returns the validation error
|
||||
* to the teacher as a tool result and it retries (same loop as the
|
||||
* grading rules). It reads the LIVE record at validation time, so a
|
||||
* plan_init earlier in the same turn already counts.
|
||||
* - planTools (src/agent/tools/plan.ts): SERVER-SIDE — the single author-once
|
||||
* plan_init (mermaid → structured tree; re-callable rewrite). Needs the
|
||||
* chatId closure — hence the per-request factory.
|
||||
* - jobTools (src/agent/tools/jobs.ts): SERVER-EXECUTED dispatch — the call
|
||||
* resolves immediately with a fire-and-forget ack, the background runner
|
||||
* does the work, and the result arrives later as a steer message
|
||||
* (contracts.ts "THE ASYNC PROTOCOL").
|
||||
* - Question tools (quiz / ask_user_question): CLIENT-SIDE (no `execute`).
|
||||
* The call streams to the browser in `input-available` state; the
|
||||
* learner answers in the dock; the client grades and sends the verdict
|
||||
* with addToolOutput. Input schemas live in core/question-core.
|
||||
* - probe_done (src/agent/tools/probe.ts): SERVER-SIDE seam tool.
|
||||
* - planTools (src/agent/tools/plan.ts): SERVER-SIDE — author-once
|
||||
* plan_init. Teacher only.
|
||||
* - jobTools (src/agent/tools/jobs.ts): SERVER-EXECUTED dispatch. Teacher
|
||||
* only.
|
||||
*/
|
||||
|
||||
export const quizTool = tool({
|
||||
// Description ported from the pi original (the learning system this app
|
||||
// grew out of) — that one never produced invalid correctAnswer values.
|
||||
description:
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Use it to (1) assess what the learner already understands before teaching, and " +
|
||||
"(2) run tight practice/retrieval loops after explaining, or probe understanding whenever you're unsure " +
|
||||
"they've got it. Options-only: single-select or multi-select, plus an automatic 'I don't know' choice so the " +
|
||||
"learner can signal a genuine gap instead of guessing. An always-present optional note field lets the learner " +
|
||||
"attach a free-text note to ANY answer; it reaches you only when non-empty. No free-text answers — for " +
|
||||
"non-graded questions use ask_user_question instead. Ask exactly one question per tool call; after calling, " +
|
||||
"end your turn and wait for the graded result.",
|
||||
inputSchema: quizInputSchema,
|
||||
});
|
||||
|
||||
export const askUserQuestionTool = tool({
|
||||
description:
|
||||
"Ask the learner a single question and pause until they answer. NOT graded — there is no correct answer. " +
|
||||
@@ -55,21 +41,41 @@ export const askUserQuestionTool = tool({
|
||||
inputSchema: askInputSchema,
|
||||
});
|
||||
|
||||
export const questionTools = {
|
||||
quiz: quizTool,
|
||||
ask_user_question: askUserQuestionTool,
|
||||
};
|
||||
// ─── the prober's question tools (no nodeId — no map exists yet) ─────────────
|
||||
|
||||
/**
|
||||
* Per-session question tools: the quiz schema gains the dynamic plan-nodeId
|
||||
* validation on top of question-core's shared authoring rules (chained
|
||||
* superRefines — the grading contract is untouched). The check reads the
|
||||
* LIVE chat record, so rewrites mid-turn are honored, and with no tree yet
|
||||
* (probe phase) every nodeId is rejected — ad-hoc checks must omit it.
|
||||
* ask_user_question is untouched: it never carries nodeId (not a check).
|
||||
*/
|
||||
export function questionToolsFor(ctx: Pick<PlanToolContext, "chatId" | "chatStore">): ToolSet {
|
||||
const quizInputWithNodeValidation = quizInputObject
|
||||
export const probeQuizTool = tool({
|
||||
description:
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Use it to locate the edge of his understanding: bracket each strand the lesson will " +
|
||||
"depend on — something he gets right (a floor), something he gets wrong or doesn't know (a ceiling). " +
|
||||
"Options-only: single-select or multi-select, plus an automatic 'I don't know' choice so the learner can " +
|
||||
"signal a genuine gap instead of guessing. An always-present optional note field lets the learner attach a " +
|
||||
"free-text note to ANY answer; it reaches you only when non-empty. No free-text answers — for non-graded " +
|
||||
"questions use ask_user_question instead. Ask exactly one question per tool call; after calling, end your " +
|
||||
"turn and wait for the graded result.",
|
||||
inputSchema: probeQuizInputSchema,
|
||||
});
|
||||
|
||||
/** The prober's tool set: probe quiz (no nodeId) + ask + probe_done. */
|
||||
export function createProberTools(ctx: ProbeDoneContext): ToolSet {
|
||||
return {
|
||||
quiz: probeQuizTool,
|
||||
ask_user_question: askUserQuestionTool,
|
||||
...probeDoneTool(ctx),
|
||||
};
|
||||
}
|
||||
|
||||
// ─── the teacher's question tools (quiz carries the position field) ──────────
|
||||
|
||||
/** Build the teach-side quiz tool for one request (exported for tests): the authoring contract
|
||||
* + the per-session plan-nodeId validation (chained superRefines — the
|
||||
* grading contract is untouched). Reads the LIVE chat record, so rewrites
|
||||
* mid-turn are honored, and with no tree yet every nodeId is rejected.
|
||||
* Omitting nodeId is legal — an ad-hoc check that moves nothing. */
|
||||
export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStore">) {
|
||||
const inputSchema = quizInputObject
|
||||
.superRefine((input, add) => {
|
||||
for (const problem of quizInputProblems(input)) {
|
||||
add.addIssue({ code: "custom", message: problem });
|
||||
@@ -78,29 +84,32 @@ export function questionToolsFor(ctx: Pick<PlanToolContext, "chatId" | "chatStor
|
||||
.superRefine((input, add) => {
|
||||
if (typeof input.nodeId !== "string" || input.nodeId.length === 0) return;
|
||||
const record = ctx.chatStore.getChat(ctx.chatId);
|
||||
const tree = record?.tree;
|
||||
const known = tree?.nodes.map((n) => n.id) ?? [];
|
||||
const known = record?.tree?.nodes.map((n) => n.id) ?? [];
|
||||
if (!known.includes(input.nodeId)) {
|
||||
add.addIssue({
|
||||
code: "custom",
|
||||
message:
|
||||
`nodeId "${input.nodeId}" matches no node in this session's plan map. ` +
|
||||
`nodeId "${input.nodeId}" matches no node in this session's plan map. ` +
|
||||
(known.length > 0
|
||||
? `Copy one of your plan_init slugs exactly (${known.slice(0, 5).join(", ")}${known.length > 5 ? ", …" : ""}), or omit nodeId for an ad-hoc check.`
|
||||
: "No plan map exists yet — omit nodeId (ad-hoc checks only until you call plan_init)."),
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
const quizWithNodeValidation = tool({
|
||||
description: quizTool.description,
|
||||
inputSchema: quizInputWithNodeValidation,
|
||||
return tool({
|
||||
description:
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Use it to (1) confirm a taught node actually landed, or (2) probe understanding " +
|
||||
"whenever you're unsure. Set `nodeId` whenever the question targets a map node — the lesson's position on the " +
|
||||
"map derives from those tags; omit it only for quick ad-hoc checks. Options-only: single-select or multi-select, " +
|
||||
"plus an automatic 'I don't know' choice so the learner can signal a genuine gap instead of guessing. " +
|
||||
"An always-present optional note field lets the learner attach a free-text note to ANY answer; it reaches you " +
|
||||
"only when non-empty. No free-text answers — for non-graded questions use ask_user_question instead. " +
|
||||
"Ask exactly one question per tool call; after calling, end your turn and wait for the graded result.",
|
||||
inputSchema,
|
||||
});
|
||||
|
||||
return {
|
||||
quiz: quizWithNodeValidation,
|
||||
ask_user_question: askUserQuestionTool,
|
||||
};
|
||||
}
|
||||
|
||||
export interface TeacherToolContext extends PlanToolContext {
|
||||
@@ -114,13 +123,13 @@ export interface TeacherToolContext extends PlanToolContext {
|
||||
}
|
||||
|
||||
/**
|
||||
* Per-request tool registry. Plan tools and question tools close over the
|
||||
* ctx (chatId + the injected ChatStore — the latter also powers the quiz
|
||||
* nodeId validation); job tools close over the job store + spawn seam.
|
||||
* Per-request teacher tool registry: teach questions (validated nodeId) +
|
||||
* plan tools + job tools, all closing over the ctx.
|
||||
*/
|
||||
export function createTeacherTools(ctx: TeacherToolContext): ToolSet {
|
||||
return {
|
||||
...questionToolsFor(ctx),
|
||||
quiz: teachQuizToolFor(ctx),
|
||||
ask_user_question: askUserQuestionTool,
|
||||
...planTools(ctx),
|
||||
...jobTools({ chatId: ctx.chatId, jobStore: ctx.jobStore, spawn: ctx.spawn }),
|
||||
};
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
import { tool, type ToolSet } from "ai";
|
||||
import { z } from "zod";
|
||||
import type { ProbeDoneInput, ProbeDoneResult } from "../../core/contracts.ts";
|
||||
import type { UIMessage } from "ai";
|
||||
import { findPendingToolCall } from "../../core/pending.ts";
|
||||
import type { ChatStore } from "../../core/ports.ts";
|
||||
import { buildProbeHandoffSteer, type ProbeHandoffSteerMessage } from "../../core/contracts.ts";
|
||||
|
||||
/** What the probe_done tool closes over. */
|
||||
export interface ProbeDoneContext {
|
||||
chatId: string;
|
||||
/** The storage port (open-question + already-concluded checks). */
|
||||
chatStore: ChatStore;
|
||||
/** Queue the handoff steer (the wake pipeline — src/server/jobs.ts).
|
||||
* Injected so this module never imports the server seam. */
|
||||
deliver: (steer: ProbeHandoffSteerMessage) => void;
|
||||
}
|
||||
|
||||
/** Does the transcript already carry a concluded probe (any state)? */
|
||||
function probeConcluded(messages: UIMessage[]): boolean {
|
||||
for (const m of messages) {
|
||||
if (m.parts.some((p) => p.type === "tool-probe_done")) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* probe_done — the probe→planning seam (docs/probe-teach-split.md §2).
|
||||
*
|
||||
* SERVER-SIDE. The prober concludes the probe by authoring the learner
|
||||
* assessment (the tool INPUT — it streams to the learner live and is
|
||||
* durable in the transcript as the tool part itself). Three outcomes:
|
||||
*
|
||||
* ok:false + error — a question is still open ("resolve it first"), or
|
||||
* the probe is already concluded. The refusal returns to the prober as
|
||||
* a tool result; it fixes course and retries (the return-to-agent
|
||||
* doctrine, same as nodeId validation).
|
||||
* ok:true — the assessment is recorded, and the HANDOFF is queued: a
|
||||
* steer through the wake pipeline that fires the teacher's first turn
|
||||
* (plan authoring) the moment the prober's turn ends. Auto-wake is the
|
||||
* user-decided trigger — the learner experiences assessment → plan as
|
||||
* one continuous flow.
|
||||
*
|
||||
* The first probe_done in the transcript is also the construction seam:
|
||||
* from that message on, every request builds the TEACHER agent.
|
||||
*/
|
||||
export function probeDoneTool(ctx: ProbeDoneContext): ToolSet {
|
||||
return {
|
||||
probe_done: tool({
|
||||
description:
|
||||
"Conclude the probing phase and hand off to the teaching phase. Call this ONCE, when the probe is " +
|
||||
"complete: every strand the lesson depends on is bracketed (a floor he reliably gets right AND a " +
|
||||
"ceiling where it runs out), and his learning goal is concrete. The input IS the learner assessment — " +
|
||||
"one strand per dependency the lesson will rest on, with your read of where his edge sits on it " +
|
||||
"(floor, ceiling, misconceptions, what to build from). It streams to the learner live and becomes " +
|
||||
"the handoff artifact the teacher plans from. Refused while a question is still open, and after " +
|
||||
"the probe is already concluded. After it succeeds, say your closing sentence and end your turn.",
|
||||
inputSchema: z.object({
|
||||
assessment: z
|
||||
.object({
|
||||
strands: z
|
||||
.array(
|
||||
z.object({
|
||||
strand: z.string().describe("The dependency strand (e.g. 'the derivative as a limit') — a name the plan can hang a node on."),
|
||||
assessment: z
|
||||
.string()
|
||||
.describe(
|
||||
"Where his edge sits on this strand: the floor (what he reliably gets right), the ceiling (where it runs out), " +
|
||||
"misconceptions found, and what the lesson should build from.",
|
||||
),
|
||||
}),
|
||||
)
|
||||
.min(1)
|
||||
.describe("Every strand the planned lesson will depend on, each with your read of his edge on it."),
|
||||
summary: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Optional overall synthesis: his apparent goal and where the teaching should enter."),
|
||||
})
|
||||
.describe("The learner assessment — the probe's distilled output."),
|
||||
}) satisfies z.ZodType<ProbeDoneInput>,
|
||||
execute: async (input): Promise<ProbeDoneResult> => {
|
||||
const record = ctx.chatStore.getChat(ctx.chatId);
|
||||
if (!record) return { ok: false, error: `Chat not found: ${ctx.chatId}` };
|
||||
|
||||
if (findPendingToolCall(record.messages) !== null) {
|
||||
return {
|
||||
ok: false,
|
||||
error:
|
||||
"A question is still open — resolve it (or let him answer) before concluding the probe. " +
|
||||
"probe_done only fires with no question outstanding.",
|
||||
};
|
||||
}
|
||||
if (probeConcluded(record.messages)) {
|
||||
return { ok: false, error: "The probe is already concluded — probe_done fires exactly once." };
|
||||
}
|
||||
try {
|
||||
ctx.deliver(buildProbeHandoffSteer(ctx.chatId));
|
||||
} catch (err) {
|
||||
console.error(`[chat ${ctx.chatId}] probe handoff queueing failed:`, err);
|
||||
// The handoff is recoverable-by-design (the next client message
|
||||
// constructs the teacher off the transcript seam) — record the
|
||||
// assessment anyway; the wake is an optimization, not a gate.
|
||||
}
|
||||
return { ok: true, assessment: input.assessment };
|
||||
},
|
||||
}),
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user