sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Pi-style ack-and-steer sub-agent protocol: dispatch mid-turn, steer via server wake turn

Browse Source 
The job tools (research/demo/visual) switch from client-side turn-breaking
calls to pi's model: execute resolves IMMEDIATELY with a fire-and-forget
ack, the teacher's turn continues, and the result arrives later as an
injected user-role steer message that wakes the teacher in a new
server-side turn.

- contracts.ts: THE ASYNC PROTOCOL rewritten (ack now, steer later); steer
  types (SubagentSteerMessage, metadata.subagentResult + data-subagent-
  artifact part), pure buildSubagentSteer/steerDelivered/isSubagentFailure
- tools/jobs.ts: jobTools(ctx) factory — server-side execute creates the
  job (deduped by toolCallId), fires spawn with the dispatch-time input
  (the part isn't persisted yet at execute — a transcript scan would race)
- runner.ts: runJob(jobId, input?) builds the steer and hands it to the
  injected deliver seam; pipeline owns the done/error flip (a polling
  client re-syncs once, at the end); failures still steer
- server/jobs.ts (new pipeline): per-chat serialized queues on globalThis,
  idle gates (turn liveness + pending question, 1.5s grace), 409 retry
  with backoff, idempotent per jobId, stale-job recovery sweep (re-deliver
  or re-spawn); wake turns run through the same handleChatRequest
  composition via setTurnRunner injection
- chat-handler.ts: wires runner+deliver+tools ctx; drops spawnTurnJobs
  from onFinish; teacher default registry = question tools only
- plan.ts: PlanToolContext split out (plan tools need no job wiring)
- visual-maker: edit_svg added (pi parity — exact-match single
  replacement, fuzzy context on a miss); researcher: pi's kept/dropped
  source accounting restored
- prompts 20-jobs/40-demos: async discipline rewritten (announce and keep
  teaching, no waiting/polling/fabricating, parallel dispatch, teach with
  the artifact when it arrives)
- tests: jobs.test.ts rewritten (dispatch ack mid-turn, runner steers,
  wake pipeline: deferral/409/idempotency/failure), demos.test.tsx runner
  tests ported to runJob + fake deliver; 222 green

FLAGGED surgical touches outside the stream's owned files, for master
review at merge: src/core/contracts.ts (protocol doc + steer types),
src/core/ui-message.ts (job tool + data-part types), src/server/
chat-handler.ts (wiring), src/server/turn-service.ts (spawnTurnJobs
retirement), src/agent/teacher-agent.ts (default tools), src/agent/tools/
plan.ts (context type split).

Verified live on PORT=3003 with real models: teacher dispatched research
mid-turn, researcher ran real searches/fetches, steer injected on quiz
answer, wake turn wove the brief in citing its sources, job flipped done;
a client/wake race correctly 409'd.
This commit is contained in:
Amos
2026-09-01 19:15:43 +03:00
parent cf3225fb23
commit 99fa3f0847
16 changed files with 1382 additions and 429 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/prompts/20-jobs.md
+12 -10
View File
@@ -9,19 +9,21 @@ outsource what you cannot reliably know yourself:
- **Never use it for:** anything you know cold, internal reasoning, or math. Do not offload
thinking — only facts you genuinely need verified.
**The call is asynchronous and your turn ends at it.** The discipline:
**The call is fire-and-forget: it resolves immediately with an ack and your turn
continues.** The discipline:
1. **Say your piece first.** Explain the concept, set up what you're doing and why, and tell
the learner roughly what you're checking — the research brief arrives later as the tool's
output, and the lesson resumes from there. A bare tool call with no surrounding text is a
failed turn.
1. **Announce it, then keep teaching.** Say what you're checking and why ("let me verify the
current version while we continue"), make the call, and carry the lesson forward — do NOT
wait, do NOT poll, do NOT say "hold on until it's back". Never fabricate the result.
2. **Call with a single, specific question.** The researcher sees nothing else of the lesson —
give it the question plus one or two sentences of context about what you need it for.
3. **Resume from the brief.** When the result returns, weave it into the lesson. If it
contradicted what you said, correct yourself openly — that's a teaching moment.
3. **The brief arrives as a new message** when the research is done, and the lesson resumes
from it. Weave it into the lesson then. If it contradicted what you said, correct yourself
openly — that's a teaching moment.
4. **Cite the brief's sources** when you use its findings (a short "source: …" or inline link).
Never present researched facts as your own certainty.
At most one research call per turn — don't fan out; the researcher searches multiple angles on
its own. If research fails or is unconfigured, you'll get a failure brief: say so plainly and
continue with what you can establish, flagging the uncertainty.
Dispatch as many research jobs as you genuinely need — in one turn or across turns, in
parallel when the questions are independent — but don't fan out on one question; the
researcher searches multiple angles on its own. If research fails, you'll get a failure
message: say so plainly and continue with what you can establish, flagging the uncertainty.
src/agent/prompts/40-demos.md
+16 -12
View File
@@ -3,8 +3,8 @@
If the concept is DYNAMIC — a parameter to sweep, a process to animate, a distribution to sample, a
system to simulate — build the learner the instrument rather than describing its behavior in prose.
A paragraph about how frequency response changes with damping teaches less than ten seconds of
dragging a slider. You have two artifact tools for this, both asynchronous (same discipline as
research):
dragging a slider. You have two artifact tools for this, both fire-and-forget (same async
discipline as research).
### `demo` — interactive teaching instruments
@@ -31,15 +31,19 @@ for geometry mermaid's auto-layout cannot express.
### The async discipline
The call is asynchronous and **your turn ends at it**, exactly like research:
Both calls resolve immediately with an ack and your turn continues:
1. **Say your piece first.** Set up the concept, explain what the learner is about to see and what to
watch for — the artifact arrives later as the tool's output and the lesson resumes from there. A
bare tool call with no surrounding text is a failed turn.
2. **At most one demo or visual per turn.**
3. **When the artifact arrives, TEACH WITH IT.** Reference its controls and elements by name: "drag
the damping slider all the way left and watch the overshoot", "notice the vector at t=2s". An
1. **Announce it, then keep teaching.** Set up the concept and tell the learner what's coming
("I'm having an interactive demo built for this — meanwhile, the idea itself…"), make the call,
and carry on — do NOT wait, do NOT poll, do NOT say "hold on until it's back". Never fabricate
the result. A bare tool call with no surrounding text is a failed turn.
2. **Dispatch freely.** Parallel jobs are encouraged when they're genuinely independent — a demo
and a figure for the same concept, several figures for a multi-step construction. Keep each
brief self-contained.
3. **The artifact arrives as a new message** when it's built, verified, and rendered in the
lesson — **TEACH WITH IT then.** Reference its controls and elements by name: "drag the
damping slider all the way left and watch the overshoot", "notice the vector at t=2s". An
artifact you never point at is decoration.
4. **Failures arrive as notes** ("FAILED: …") with no artifact — say so plainly, fall back to
explaining in prose, and if the brief was the problem, re-call with a tighter one. Re-calling a
tool produces a new version of the artifact; use that to fix a demo that misbehaves in the field.
4. **Failures arrive as messages** with no artifact — say so plainly, fall back to explaining
in prose, and if the brief was the problem, re-call with a tighter one. Re-calling a tool
produces a new version of the artifact; use that to fix a demo that misbehaves in the field.
src/agent/subagents/researcher.ts
+2 -1
View File
@@ -51,7 +51,8 @@ Numbered findings with inline source citations:
1. **Finding** — explanation. [Source](url)
## Sources
- Source Title (url) — why it's relevant
- Kept: Source Title (url) — why relevant
- Dropped: Source Title — why excluded
## Gaps
What couldn't be answered. Suggested next steps.`;
src/agent/subagents/runner.ts
+163 -163
View File
@@ -1,107 +1,168 @@
import { isToolUIPart, type LanguageModel, type UIMessage } from "ai";
import type { DemoResult, DemoToolInput, JobKind, ResearchResult, ResearchToolInput, VisualResult, VisualToolInput } from "../../core/contracts.ts";
import { JOB_TOOLS } from "../../core/pending.ts";
import {
buildSubagentSteer,
type DemoResult,
type DemoToolInput,
type JobKind,
type JobRecord,
type ResearchResult,
type ResearchToolInput,
type SubagentSteerMessage,
type VisualResult,
type VisualToolInput,
} from "../../core/contracts.ts";
import type { ChatStore, JobStore } from "../../core/ports.ts";
import { runDemoBuilder } from "./demo-builder.ts";
import { createResearcherAgent } from "./researcher.ts";
import { runVisualMaker } from "./visual-maker.ts";
/**
* The background job runner (contracts.ts "THE ASYNC PROTOCOL"), created via
* createJobRunner({ chatStore, jobStore }) — the composition root (chat-handler)
* injects the storage ports; this module never imports an adapter.
* The background job runner (contracts.ts "THE ASYNC PROTOCOL" §2–3), created
* via createJobRunner({ chatStore, jobStore, deliver }) — the composition root
* (chat-handler) injects the storage ports and the WAKE SEAM (the server-side
* steer delivery, src/server/jobs.ts); this module never imports an adapter.
*
* spawnPendingJobs — called from the turn service AFTER persist: finds
* input-available JOB_TOOLS parts in the last assistant message with no
* existing job and fires the kind's runner WITHOUT awaiting (the HTTP
* response is long gone). Kind dispatch (research/demo/visual) is a
* self-contained map here, so no caller changes for new job kinds —
* anything listed in JOB_TOOLS routes automatically.
* runJob(jobId) — runs the kind's sub-agent loop in the background
* (the job tools' execute fires this via the spawn seam, fire-and-forget),
* updating the job's activity line on every iteration. On completion the
* runner builds the STEER MESSAGE (buildSubagentSteer) and hands it to
* `deliver`; the wake pipeline fires the server-side teacher turn and
* flips the job to done/error when the lesson has actually resumed.
*
* runResearchJob / runDemoJob / runVisualJob — run the sub-agent loop in
* the background, updating the job's activity line on every iteration,
* then rewrite the tool part in the LATEST chat record to
* output-available (read-modify-write; no awaits between the read and
* the write). The part is fulfilled BEFORE the job status flips, so a
* client that polls the job and then re-syncs always sees the finished
* part.
*
* On failure the part is still fulfilled (DemoResult/VisualResult failure
* shape: empty html/svg + "FAILED: …" notes; research: failure brief) — the
* teacher must learn the outcome — and the job is marked error.
* On failure the result is the failure shape (empty html/svg + "FAILED: …"
* notes; research: failure brief) — and it STILL steers, because the
* teacher must learn the outcome. If delivery itself fails, the job is
* marked error (the learner's activity card shows it).
*/
/**
* Test seam: the spawn hook creates runners with no injection point, so
* tests swap in a MockLanguageModelV4 here to keep job tests off the
* network. Undefined in production.
* Test seam: the spawn wiring has no injection point at the tool layer, so
* tests swap in a MockLanguageModelV4 here to keep job tests off the network.
* Undefined in production. (The jobs spawn through the module-level KIND
* runners, which read these at call time.)
*/
export const researchModelOverride: { current?: LanguageModel } = {};
export const demoModelOverride: { current?: LanguageModel } = {};
export const visualModelOverride: { current?: LanguageModel } = {};
/** The dispatch-time tool input (from src/agent/tools/jobs.ts schemas). */
type JobToolInput = ResearchToolInput | DemoToolInput | VisualToolInput;
export interface JobRunnerDeps {
chatStore: ChatStore;
jobStore: JobStore;
}
export interface RunResearchJobOptions {
model?: LanguageModel;
}
export interface RunDemoJobOptions {
model?: LanguageModel;
}
export interface RunVisualJobOptions {
model?: LanguageModel;
/**
* Hand a finished job's steer to the wake pipeline — resolves when the
* steer is delivered (the wake turn completed and the job status
* flipped); rejects when delivery fails unrecoverably.
*/
deliver: (steer: SubagentSteerMessage) => Promise<void>;
}
export interface JobRunner {
spawnPendingJobs(chatId: string, messages: UIMessage[]): void;
runResearchJob(jobId: string, opts?: RunResearchJobOptions): Promise<void>;
runDemoJob(jobId: string, opts?: RunDemoJobOptions): Promise<void>;
runVisualJob(jobId: string, opts?: RunVisualJobOptions): Promise<void>;
/** Run one job's sub-agent loop to completion (background; never throws).
* The tool input rides along from dispatch (the spawn fires mid-stream,
* before the tool part is persisted — only crash-recovery re-spawns
* resolve the input from the transcript). */
runJob(jobId: string, input?: ResearchToolInput | DemoToolInput | VisualToolInput): Promise<void>;
}
export function createJobRunner(deps: JobRunnerDeps): JobRunner {
const { chatStore, jobStore } = deps;
const { chatStore, jobStore, deliver } = deps;
/** The kind→runner map: everything in JOB_TOOLS routes through here. */
const KIND_RUNNERS: Record<JobKind, (jobId: string) => Promise<void>> = {
research: (jobId) => runResearchJob(jobId, { model: researchModelOverride.current }),
demo: (jobId) => runDemoJob(jobId, { model: demoModelOverride.current }),
visual: (jobId) => runVisualJob(jobId, { model: visualModelOverride.current }),
/** The kind→runner map: one sub-agent loop per job kind. */
// Each kind's runner narrows the union input itself (dispatch-time inputs
// are kind-correct by construction; transcript fallbacks are cast after).
const KIND_RUNNERS: Record<JobKind, (jobId: string, input?: JobToolInput) => Promise<void>> = {
research: (jobId, input) => runResearchJob(jobId, researchModelOverride.current, input),
demo: (jobId, input) => runDemoJob(jobId, demoModelOverride.current, input),
visual: (jobId, input) => runVisualJob(jobId, visualModelOverride.current, input),
};
function spawnPendingJobs(chatId: string, messages: UIMessage[]): void {
const last = [...messages].reverse().find((m) => m.role === "assistant");
if (!last) return;
for (const part of last.parts) {
if (!isToolUIPart(part) || part.state !== "input-available") continue;
const tool = part.type.slice("tool-".length);
if (!(JOB_TOOLS as readonly string[]).includes(tool)) continue;
const runner = KIND_RUNNERS[tool as JobKind];
if (!runner) continue;
if (jobStore.findJobByToolCallId(chatId, part.toolCallId)) continue; // already has a job
const job = jobStore.createJob({ kind: tool as JobKind, chatId, toolCallId: part.toolCallId });
void runner(job.id).catch((err) => {
console.error(`[job ${job.id}] ${tool} runner crashed:`, err);
jobStore.updateJob(job.id, { status: "error", error: String(err), activity: "failed" });
});
async function runJob(jobId: string, input?: JobToolInput): Promise<void> {
const job = jobStore.getJob(jobId);
if (!job) {
console.error(`[job ${jobId}] not found — nothing to run`);
return;
}
if (job.status !== "running") return; // already finished / re-run after crash recovery
const run = KIND_RUNNERS[job.kind];
if (!run) {
console.error(`[job ${jobId}] unknown kind: ${job.kind}`);
jobStore.updateJob(jobId, { status: "error", error: `unknown kind ${job.kind}`, activity: "failed" });
return;
}
await run(jobId, input);
}
/** Resolve the job's tool input: dispatch passes it directly; only a
* crash-recovery re-spawn (no input) falls back to scanning the
* transcript (by then onFinish has long persisted the part). */
function resolveInput(
job: JobRecord,
input: JobToolInput | undefined,
): ResearchToolInput | DemoToolInput | VisualToolInput | undefined {
if (input) return input;
const chat = chatStore.getChat(job.chatId);
if (!chat) return undefined;
return (findToolPart(chat.messages, job.toolCallId)?.input ?? undefined) as
| ResearchToolInput
| DemoToolInput
| VisualToolInput
| undefined;
}
// ─── shared plumbing ────────────────────────────────────────────────────
function findToolPart(messages: UIMessage[], toolCallId: string) {
for (const m of messages) {
for (const part of m.parts) {
if (isToolUIPart(part) && part.toolCallId === toolCallId) return part;
}
}
return null;
}
/**
* The tail of every job: store the result, build the steer, hand it to
* the wake pipeline. The pipeline flips the status when the lesson has
* actually resumed — until then the job reads "running" so the client's
* activity card keeps polling and re-syncs once, at the end.
*/
async function finishJob(jobId: string, result: ResearchResult | DemoResult | VisualResult): Promise<void> {
const job = jobStore.getJob(jobId);
if (!job) return;
jobStore.updateJob(jobId, { result, activity: "delivering result" });
const steer = buildSubagentSteer(job, result);
try {
await deliver(steer);
} catch (err) {
const reason = err instanceof Error ? err.message : String(err);
console.error(`[job ${jobId}] steer delivery failed:`, err);
jobStore.updateJob(jobId, { status: "error", error: reason, activity: "delivery failed" });
}
}
async function runResearchJob(jobId: string, opts: RunResearchJobOptions = {}): Promise<void> {
/** Chat-record + input resolution shared by all three runners. */
function jobContext(jobId: string): { job: JobRecord; chatId: string } | null {
const job = jobStore.getJob(jobId);
if (!job || job.status !== "running") return;
if (!job || job.status !== "running") return null;
const chat = chatStore.getChat(job.chatId);
if (!chat) {
jobStore.updateJob(jobId, { status: "error", error: `Chat not found: ${job.chatId}`, activity: "failed" });
return;
return null;
}
const input = findToolPart(chat.messages, job.toolCallId)?.input as ResearchToolInput | undefined;
return { job, chatId: job.chatId };
}
// ─── research ───────────────────────────────────────────────────────────
async function runResearchJob(jobId: string, model?: LanguageModel, passedInput?: JobToolInput): Promise<void> {
const ctx = jobContext(jobId);
if (!ctx) return;
const { job } = ctx;
const input = (passedInput ?? resolveInput(job, passedInput)) as ResearchToolInput | undefined;
const question = typeof input?.question === "string" && input.question.trim() ? input.question : "(no question)";
const context = typeof input?.context === "string" && input.context.trim() ? input.context : undefined;
@@ -114,7 +175,7 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner {
.join("\n\n");
const fetchedUrls: string[] = [];
const agent = createResearcherAgent(opts.model);
const agent = createResearcherAgent(model);
try {
jobStore.updateJob(jobId, { activity: "researching" });
@@ -132,37 +193,22 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner {
const brief = result.text.trim();
if (!brief) throw new Error("the researcher produced no text");
const research: ResearchResult = { brief, sources: parseSources(brief, fetchedUrls) };
fulfillPart(job.chatId, job.toolCallId, research);
jobStore.updateJob(jobId, { status: "done", result: research }); // activity line kept: the last tool call is the record
await finishJob(jobId, research);
} catch (err) {
const reason = err instanceof Error ? err.message : String(err);
fulfillPart(job.chatId, job.toolCallId, { brief: `Research failed: ${reason}`, sources: [] });
jobStore.updateJob(jobId, { status: "error", activity: "failed", error: reason });
jobStore.updateJob(jobId, { error: reason });
await finishJob(jobId, { brief: `Research failed: ${reason}`, sources: [] });
}
}
// ─── demo + visual job wrappers ───────────────────────────────────────────
// ─── demo + visual ──────────────────────────────────────────────────────
/** Failure shape (W6 brief): html === "" with "FAILED: …" notes. */
function isDemoFailure(result: DemoResult): boolean {
return typeof result.html !== "string" || result.html.length === 0;
}
async function runDemoJob(jobId: string, model?: LanguageModel, passedInput?: JobToolInput): Promise<void> {
const ctx = jobContext(jobId);
if (!ctx) return;
const { job } = ctx;
function isVisualFailure(result: VisualResult): boolean {
return typeof result.svg !== "string" || result.svg.length === 0;
}
async function runDemoJob(jobId: string, opts: RunDemoJobOptions = {}): Promise<void> {
const job = jobStore.getJob(jobId);
if (!job || job.status !== "running") return;
const chat = chatStore.getChat(job.chatId);
if (!chat) {
jobStore.updateJob(jobId, { status: "error", error: `Chat not found: ${job.chatId}`, activity: "failed" });
return;
}
const input = findToolPart(chat.messages, job.toolCallId)?.input as DemoToolInput | undefined;
const input = (passedInput ?? resolveInput(job, passedInput)) as DemoToolInput | undefined;
const title = typeof input?.title === "string" && input.title.trim() ? input.title : "Interactive demo";
const brief = typeof input?.brief === "string" ? input.brief : "";
@@ -171,34 +217,25 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner {
const result = await runDemoBuilder(
{ title, brief },
{
model: opts.model,
model,
onActivity: (line) => jobStore.updateJob(jobId, { activity: line }),
},
);
fulfillPart(job.chatId, job.toolCallId, result);
if (isDemoFailure(result)) {
jobStore.updateJob(jobId, { status: "error", error: result.notes ?? "demo build failed", activity: "failed" });
} else {
jobStore.updateJob(jobId, { status: "done", result });
}
if (isDemoFailure(result)) jobStore.updateJob(jobId, { error: result.notes ?? "demo build failed" });
await finishJob(jobId, result);
} catch (err) {
const reason = err instanceof Error ? err.message : String(err);
const failure: DemoResult = { title, html: "", tested: true, notes: `FAILED: ${reason}` };
fulfillPart(job.chatId, job.toolCallId, failure);
jobStore.updateJob(jobId, { status: "error", error: reason, activity: "failed" });
jobStore.updateJob(jobId, { error: reason });
await finishJob(jobId, { title, html: "", tested: true, notes: `FAILED: ${reason}` });
}
}
async function runVisualJob(jobId: string, opts: RunVisualJobOptions = {}): Promise<void> {
const job = jobStore.getJob(jobId);
if (!job || job.status !== "running") return;
async function runVisualJob(jobId: string, model?: LanguageModel, passedInput?: JobToolInput): Promise<void> {
const ctx = jobContext(jobId);
if (!ctx) return;
const { job } = ctx;
const chat = chatStore.getChat(job.chatId);
if (!chat) {
jobStore.updateJob(jobId, { status: "error", error: `Chat not found: ${job.chatId}`, activity: "failed" });
return;
}
const input = findToolPart(chat.messages, job.toolCallId)?.input as VisualToolInput | undefined;
const input = (passedInput ?? resolveInput(job, passedInput)) as VisualToolInput | undefined;
const brief = typeof input?.brief === "string" ? input.brief : "";
try {
@@ -206,70 +243,33 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner {
const result = await runVisualMaker(
{ brief },
{
model: opts.model,
model,
onActivity: (line) => jobStore.updateJob(jobId, { activity: line }),
},
);
fulfillPart(job.chatId, job.toolCallId, result);
if (isVisualFailure(result)) {
jobStore.updateJob(jobId, { status: "error", error: result.notes ?? "visual build failed", activity: "failed" });
} else {
jobStore.updateJob(jobId, { status: "done", result });
}
if (isVisualFailure(result)) jobStore.updateJob(jobId, { error: result.notes ?? "visual build failed" });
await finishJob(jobId, result);
} catch (err) {
const reason = err instanceof Error ? err.message : String(err);
const failure: VisualResult = { svg: "", notes: `FAILED: ${reason}` };
fulfillPart(job.chatId, job.toolCallId, failure);
jobStore.updateJob(jobId, { status: "error", error: reason, activity: "failed" });
jobStore.updateJob(jobId, { error: reason });
await finishJob(jobId, { svg: "", notes: `FAILED: ${reason}` });
}
}
// ─── helpers ─────────────────────────────────────────────────────────────
function findToolPart(messages: UIMessage[], toolCallId: string) {
for (const m of messages) {
for (const part of m.parts) {
if (isToolUIPart(part) && part.toolCallId === toolCallId) return part;
}
}
return null;
}
/**
* Rewrite the tool part in the LATEST chat record to output-available.
* Synchronous read-modify-write (no awaits) — atomic within the event loop.
*/
function fulfillPart(chatId: string, toolCallId: string, output: ResearchResult | DemoResult | VisualResult): void {
const chat = chatStore.getChat(chatId);
if (!chat) return;
let found = false;
const messages = chat.messages.map((m) =>
m.role !== "assistant"
? m
: {
...m,
parts: m.parts.map((part) => {
if (isToolUIPart(part) && part.toolCallId === toolCallId && part.state === "input-available") {
found = true;
return { ...part, state: "output-available" as const, output };
}
return part;
}),
},
);
if (!found) return;
try {
chatStore.saveMessages(chatId, messages);
} catch (err) {
console.error(`[chat ${chatId}] fulfilling job part ${toolCallId} failed:`, err);
}
}
return { spawnPendingJobs, runResearchJob, runDemoJob, runVisualJob };
return { runJob };
}
// ─── module-level pure helpers ──────────────────────────────────────────────
/** Failure shape (W6 brief): html === "" with "FAILED: …" notes. */
function isDemoFailure(result: DemoResult): boolean {
return typeof result.html !== "string" || result.html.length === 0;
}
function isVisualFailure(result: VisualResult): boolean {
return typeof result.svg !== "string" || result.svg.length === 0;
}
/** The activity line shown in the UI: `web_search: "query"` / `web_fetch: host`. */
function activityLine(toolName: string, input: { query?: string; url?: string } | undefined): string {
if (toolName === "web_search") return `web_search: "${input?.query ?? ""}"`;
src/agent/subagents/visual-maker.ts
+70 -4
View File
@@ -49,8 +49,11 @@ composition, and above everything CORRECTNESS: the picture must not assert anyth
mark on the wrong corner, a vector pointing the wrong way, a point plotted at the wrong coordinate is a
failure even if it renders cleanly.
Your two tools:
Your three tools:
- write_svg({ svg }) — write the complete <svg>…</svg> source (overwrites the previous version).
- edit_svg({ old_text, new_text }) — a surgical exact-match replacement in the current SVG:
old_text must appear EXACTLY ONCE; it is replaced by new_text. Call write_svg first. For anything
beyond a one-spot fix (new elements, restructured layout), write_svg the whole document again.
- render_svg({}) — rasterize the current SVG in headless Chrome. The render arrives as an image in the
message right after the tool result.
@@ -64,8 +67,8 @@ says nothing about whether the geometry is right or the picture is readable. Whe
- Would a learner instantly read the intended idea from this picture alone?
Workflow: plan the coordinate space (viewBox + margins, nothing touching the edges) → write_svg →
render_svg → LOOK critically → fix (write_svg) and re-render until correct and clean. Never finish
without having looked at a render of your FINAL SVG.
render_svg → LOOK critically → fix (edit_svg for one-spot fixes, write_svg otherwise) and re-render
until correct and clean. Never finish without having looked at a render of your FINAL SVG.
SVG requirements:
- A complete <svg> with explicit width/height AND a matching viewBox.
@@ -86,6 +89,14 @@ export function createVisualTools() {
svg: z.string().describe('Complete <svg xmlns="…">…</svg> markup.'),
}),
}),
edit_svg: tool({
description:
"Replace one exact spot in the current SVG: old_text must appear EXACTLY ONCE and is replaced by new_text. Call write_svg first. For larger changes, write_svg the whole document.",
inputSchema: z.object({
old_text: z.string().describe("The exact text to replace — must match exactly once in the current SVG."),
new_text: z.string().describe("The replacement text."),
}),
}),
render_svg: tool({
description: "Render the current SVG to a PNG and show it to you for inspection. No arguments.",
inputSchema: z.object({}),
@@ -164,6 +175,36 @@ export async function runVisualMaker(input: VisualMakerInput, opts: VisualMakerO
currentSvg = svg;
opts.onActivity?.(`write_svg: ${(svg.length / 1024).toFixed(1)} KB`);
toolResults.push(toolText(call.toolCallId, call.toolName, `Saved (${svg.length} bytes). Call render_svg to look at it.`));
} else if (call.toolName === "edit_svg") {
const edit = call.input as { old_text?: unknown; new_text?: unknown };
if (!currentSvg) {
toolResults.push(toolText(call.toolCallId, call.toolName, "ERROR: nothing to edit — call write_svg first."));
continue;
}
if (typeof edit.old_text !== "string" || typeof edit.new_text !== "string" || !edit.old_text) {
toolResults.push(toolText(call.toolCallId, call.toolName, "ERROR: expected old_text (non-empty) and new_text strings."));
continue;
}
const occurrences = currentSvg.split(edit.old_text).length - 1;
if (occurrences === 0) {
toolResults.push(
toolText(
call.toolCallId,
call.toolName,
`ERROR: old_text not found. Closest line for reference:\n${snippetAround(currentSvg, edit.old_text)}`,
),
);
continue;
}
if (occurrences > 1) {
toolResults.push(
toolText(call.toolCallId, call.toolName, `ERROR: old_text appears ${occurrences} times — it must match EXACTLY ONCE. Include more surrounding context to make it unique.`),
);
continue;
}
currentSvg = currentSvg.replace(edit.old_text, edit.new_text);
opts.onActivity?.("edit_svg: applied");
toolResults.push(toolText(call.toolCallId, call.toolName, "OK — replaced 1 occurrence. Call render_svg to look at the edited SVG."));
} else if (call.toolName === "render_svg") {
if (!currentSvg) {
toolResults.push(toolText(call.toolCallId, call.toolName, "ERROR: nothing to render — call write_svg first."));
@@ -181,7 +222,7 @@ export async function runVisualMaker(input: VisualMakerInput, opts: VisualMakerO
toolResults.push(toolText(call.toolCallId, call.toolName, `RENDER ERROR (the SVG may be malformed): ${msg}. Fix it with write_svg and render again.`));
}
} else {
toolResults.push(toolText(call.toolCallId, call.toolName, `ERROR: unknown tool "${call.toolName}". Use write_svg or render_svg.`));
toolResults.push(toolText(call.toolCallId, call.toolName, `ERROR: unknown tool "${call.toolName}". Use write_svg, edit_svg, or render_svg.`));
}
}
// Message order: assistant tool-calls → tool results → render image.
@@ -215,6 +256,31 @@ function toolText(toolCallId: string, toolName: string, value: string): ToolResu
return { type: "tool-result", toolCallId, toolName, output: { type: "text", value } };
}
/** A 3-line numbered context window around the closest fuzzy match for a
* failed edit — enough for the model to see what went wrong and retry
* (the pi svg_tools snippetAround semantics). */
function snippetAround(svg: string, target: string): string {
const words = target.trim().split(/\s+/).filter(Boolean);
if (words.length === 0) return "(empty old_text)";
const lines = svg.split("\n");
let bestAt = -1;
let bestScore = -1;
for (let i = 0; i < lines.length; i++) {
const score = words.filter((w) => lines[i]!.includes(w)).length;
if (score > bestScore) {
bestScore = score;
bestAt = i;
}
}
if (bestAt < 0) return "(no similar line found)";
const from = Math.max(0, bestAt - 1);
const to = Math.min(lines.length, bestAt + 2);
return lines
.slice(from, to)
.map((line, idx) => `${from + idx + 1} | ${line}`)
.join("\n");
}
function finish(svg: string | null, finalText: string): VisualResult {
if (!svg) {
return {
src/agent/teacher-agent.ts
+8 -6
View File
@@ -2,8 +2,8 @@ import * as fs from "node:fs";
import * as path from "node:path";
import { ToolLoopAgent, stepCountIs, type LanguageModel, type ToolSet } from "ai";
import { PATHS } from "../../config.ts";
import { createTeacherTools, jobTools, questionTools } from "./tools.ts";
import { modelFor } from "./models.ts";
import { questionTools } from "./tools.ts";
/**
* The teacher agent. Everything pi used to provide (agent loop, tool calling,
@@ -56,13 +56,15 @@ export function createTeacherAgent(model?: LanguageModel, tools?: ToolSet, graph
// Reasoning tokens stream as reasoning parts into the UI's collapsible block.
model: model ?? modelFor("teacher"),
instructions: withGraphContext(graphContext),
// Tools are per-request (plan tools close over the injected ctx);
// chat-handler passes createTeacherTools({ chatId, chatStore }). The
// Tools are per-request (plan tools close over the injected ctx; job
// tools close over the job store + spawn seam) — chat-handler passes
// createTeacherTools({ chatId, chatStore, jobStore, spawn }). The
// dependency-free default is the question-only set (client-side tools,
// no storage needed) — used when tests pass a bare mock model.
tools: tools ?? { ...questionTools, ...jobTools() },
// Question and job tools are client-side — a call ends the server turn
// automatically. stepCountIs is just the safety net.
tools: tools ?? { ...questionTools },
// Question tools are client-side (a call ends the server turn);
// job tools resolve immediately (fire-and-forget ack). stepCountIs is
// just the safety net.
stopWhen: stepCountIs(20),
});
}
src/agent/tools.ts
+19 -14
View File
@@ -1,10 +1,11 @@
import { tool, type ToolSet } from "ai";
import { askInputSchema, quizInputSchema } from "../core/question-core.ts";
import type { ChatStore } from "../core/ports.ts";
import { jobTools } from "./tools/jobs.ts";
import type { JobRecord } from "../core/contracts.ts";
import type { JobStore } from "../core/ports.ts";
import { jobTools, type DemoInput, type ResearchInput, type VisualInput } from "./tools/jobs.ts";
import { planTools, type PlanToolContext } from "./tools/plan.ts";
export { jobTools };
import { planTools } from "./tools/plan.ts";
/**
* The teacher's tools, composed per request by createTeacherTools.
@@ -17,8 +18,10 @@ import { planTools } from "./tools/plan.ts";
* the prompt docs exactly; keep them in sync.
* - planTools (src/agent/tools/plan.ts): SERVER-SIDE, mutate the chat's plan
* tree (needs the chatId closure — hence the per-request factory).
* - jobTools (src/agent/tools/jobs.ts): CLIENT-SIDE semantics, server-side
* fulfillment by background sub-agents (see src/core/contracts.ts).
* - jobTools (src/agent/tools/jobs.ts): SERVER-EXECUTED dispatch — the call
* resolves immediately with a fire-and-forget ack, the background runner
* does the work, and the result arrives later as a steer message
* (contracts.ts "THE ASYNC PROTOCOL").
*/
export const quizTool = tool({
@@ -51,23 +54,25 @@ export const questionTools = {
ask_user_question: askUserQuestionTool,
};
export interface TeacherToolContext {
chatId: string;
/** The storage port plan tools persist through — the composition root
* (chat-handler) injects the wired adapter; the agent layer never
* imports one. */
chatStore: ChatStore;
export interface TeacherToolContext extends PlanToolContext {
/** Job-tool wiring (sub-agent dispatch): the job store + the runner's
* spawn seam. The composition root injects both. */
jobStore: JobStore;
/** Fire the background runner for a freshly created job. The tool input
* rides along — the spawn fires mid-stream, before the tool part is
* persisted to the transcript (a transcript scan would race it). */
spawn: (job: JobRecord, input: ResearchInput | DemoInput | VisualInput) => void;
}
/**
* Per-request tool registry. Plan tools close over the ctx (chatId + the
* injected ChatStore) for persistence; question and job tools are
* context-free.
* injected ChatStore) for persistence; question tools are context-free;
* job tools close over the job store + spawn seam.
*/
export function createTeacherTools(ctx: TeacherToolContext): ToolSet {
return {
...questionTools,
...planTools(ctx),
...jobTools(),
...jobTools({ chatId: ctx.chatId, jobStore: ctx.jobStore, spawn: ctx.spawn }),
};
}
src/agent/tools/jobs.ts
+83 -42
View File
@@ -1,16 +1,29 @@
import { tool, type ToolSet } from "ai";
import { z } from "zod";
import type { JobRecord } from "../../core/contracts.ts";
import type { JobStore } from "../../core/ports.ts";
/**
* Async sub-agent job tools (W2: research; W6 adds demo + visual).
* CLIENT-SIDE semantics (no execute) — exactly like quiz/ask: the tool call
* streams to the browser in `input-available` state and the server turn ends
* there. chat-handler's onFinish spawns a background runner that fulfills
* the part server-side; the client polls /api/jobs and the continuation
* resumes the teacher with the result as a normal tool output.
* See src/core/contracts.ts "THE ASYNC PROTOCOL".
* Async sub-agent job tools (vision §2.5; W2/W6 kinds + the pi-model
* protocol). SERVER-EXECUTED: each call creates the job record, fires the
* background runner (via the injected spawn seam — the agent layer never
* imports one), and resolves IMMEDIATELY with a fire-and-forget ack. The
* teacher's turn does NOT end at the call; the result arrives later as a
* steer message that wakes the teacher in a new turn. See contracts.ts
* "THE ASYNC PROTOCOL".
*/
/** What the job tools need at execute time (injected by the composition root). */
export interface JobToolContext {
chatId: string;
/** Creates the job record (deduped by toolCallId). */
jobStore: JobStore;
/** Fire the background runner for a fresh job — fire-and-forget. The
* input rides along because the spawn happens MID-STREAM, before
* onFinish persists the tool part — a transcript scan would race it. */
spawn: (job: JobRecord, input: ResearchInput | DemoInput | VisualInput) => void;
}
export const researchInputSchema = z.object({
question: z
.string()
@@ -21,15 +34,6 @@ export const researchInputSchema = z.object({
.describe("Why you need this / what the lesson needs it for — focuses the brief. One or two sentences."),
});
export const researchTool = tool({
description:
"Dispatch a background web-research agent: fact verification, current data, or niche specifics you are not certain about. " +
"The call is ASYNC — your turn ENDS here, so say everything you want to say first; the research brief arrives later as " +
"this tool's output and the lesson resumes from it. Cite the brief's sources when you use them. " +
"NEVER use it for things you know cold, internal reasoning, or math.",
inputSchema: researchInputSchema,
});
export const demoInputSchema = z.object({
title: z.string().describe("Short title for the demo card, e.g. 'Damped harmonic oscillator'."),
brief: z
@@ -40,16 +44,6 @@ export const demoInputSchema = z.object({
),
});
export const demoTool = tool({
description:
"Dispatch a background sub-agent that BUILDS and headlessly TESTS an interactive teaching instrument — a single-file HTML " +
"demo (parameter sweep, simulation, animation, sampling) rendered live in the lesson. Use it when the concept is DYNAMIC: " +
"something to sweep, animate, or sample is better experienced than described. " +
"The call is ASYNC — your turn ENDS here, so say everything you want to say first; the tested demo arrives later as this " +
"tool's output and the lesson resumes with it. Write a precise brief: concept, controls, interactions, style.",
inputSchema: demoInputSchema,
});
export const visualInputSchema = z.object({
brief: z
.string()
@@ -59,26 +53,73 @@ export const visualInputSchema = z.object({
),
});
export const visualTool = tool({
description:
"Dispatch a background sub-agent that hand-authors ONE precise SVG figure, renders it, LOOKS at the render, and iterates " +
"until correct. Use it for exact spatial pictures auto-layout diagrams can't do: coordinate geometry, number lines, vectors, " +
"function plots, physical layouts. For flows/relations/taxonomies write mermaid inline instead. " +
"The call is ASYNC — your turn ENDS here, so say everything you want to say first; the verified figure arrives later as this " +
"tool's output and the lesson resumes with it.",
inputSchema: visualInputSchema,
});
/**
* The per-call dispatch: create the job (idempotent by toolCallId), fire the
* runner, and return the ack. Shared by all three tools — only kind differs.
*/
function dispatch(
ctx: JobToolContext,
kind: JobRecord["kind"],
toolCallId: string,
ack: string,
input: ResearchInput | DemoInput | VisualInput,
): string {
const existing = ctx.jobStore.findJobByToolCallId(ctx.chatId, toolCallId);
const job = existing ?? ctx.jobStore.createJob({ kind, chatId: ctx.chatId, toolCallId });
if (!existing) ctx.spawn(job, input);
return `Dispatched (job ${job.id}). ${ack}`;
}
const RESEARCH_ACK =
"This call is fire-and-forget: continue your turn and keep teaching — do not wait, poll, or mention waiting. " +
"The researched brief arrives as a new message when ready and the lesson resumes from it; cite its sources when you use them.";
const DEMO_ACK =
"This call is fire-and-forget: continue your turn and keep teaching — do not wait, poll, or mention waiting. " +
"The tested demo arrives as a new message when ready and is rendered in the lesson; teach with it then.";
const VISUAL_ACK =
"This call is fire-and-forget: continue your turn and keep teaching — do not wait, poll, or mention waiting. " +
"The verified figure arrives as a new message when ready and is rendered in the lesson; reference it then.";
const RESEARCH_DESCRIPTION =
"Dispatch a background web-research agent: fact verification, current data, or niche specifics you are not certain about. " +
"The call is ASYNC and fire-and-forget — it resolves immediately and your turn continues; the research brief arrives later " +
"as a new message and the lesson resumes from it. Cite the brief's sources when you use them. " +
"NEVER use it for things you know cold, internal reasoning, or math.";
const DEMO_DESCRIPTION =
"Dispatch a background sub-agent that BUILDS and headlessly TESTS an interactive teaching instrument — a single-file HTML " +
"demo (parameter sweep, simulation, animation, sampling) rendered live in the lesson. Use it when the concept is DYNAMIC: " +
"something to sweep, animate, or sample is better experienced than described. " +
"The call is ASYNC and fire-and-forget — it resolves immediately and your turn continues; the tested demo arrives later " +
"as a new message. Write a precise brief: concept, controls, interactions, style.";
const VISUAL_DESCRIPTION =
"Dispatch a background sub-agent that hand-authors ONE precise SVG figure, renders it, LOOKS at the render, and iterates " +
"until correct. Use it for exact spatial pictures auto-layout diagrams can't do: coordinate geometry, number lines, vectors, " +
"function plots, physical layouts. For flows/relations/taxonomies write mermaid inline instead. " +
"The call is ASYNC and fire-and-forget — it resolves immediately and your turn continues; the verified figure arrives later " +
"as a new message.";
/**
* Per-request job tool registry. Context-free (the runner resolves
* everything from the chat record by toolCallId). Takes no dependencies —
* part of the dependency-free default registry in teacher-agent.ts.
* Per-request job tool registry: each tool closes over the composition root's
* context (chatId + jobStore + the spawn seam), so concurrent chats never
* share state. chat-handler wires it each turn; tests wire their own.
*/
export function jobTools(): ToolSet {
export function jobTools(ctx: JobToolContext): ToolSet {
return {
research: researchTool,
demo: demoTool,
visual: visualTool,
research: tool({
description: RESEARCH_DESCRIPTION,
inputSchema: researchInputSchema,
execute: async (input, { toolCallId }) => dispatch(ctx, "research", toolCallId, RESEARCH_ACK, input),
}),
demo: tool({
description: DEMO_DESCRIPTION,
inputSchema: demoInputSchema,
execute: async (input, { toolCallId }) => dispatch(ctx, "demo", toolCallId, DEMO_ACK, input),
}),
visual: tool({
description: VISUAL_DESCRIPTION,
inputSchema: visualInputSchema,
execute: async (input, { toolCallId }) => dispatch(ctx, "visual", toolCallId, VISUAL_ACK, input),
}),
};
}
src/agent/tools/plan.ts
+10 -3
View File
@@ -1,6 +1,7 @@
import { tool, type ToolSet } from "ai";
import { z } from "zod";
import type { PlanToolResult, PlanTree } from "../../core/contracts.ts";
import type { ChatStore } from "../../core/ports.ts";
import {
addEdge,
addNode,
@@ -13,7 +14,13 @@ import {
setCurrent,
updateNode,
} from "../../core/plan-tree.ts";
import type { TeacherToolContext } from "../tools.ts";
/** What the plan tools close over (the plan-tree half of the tool context). */
export interface PlanToolContext {
chatId: string;
/** The storage port plan tools persist through. */
chatStore: ChatStore;
}
/**
* Plan-tree tools — W1 workstream (docs/plan.md Wave 1; src/core/contracts.ts
@@ -48,7 +55,7 @@ const edgeSchema = z.object({
* empty) tree, so the model can see what it's working with and fix the call.
*/
async function runPlanOp(
ctx: TeacherToolContext,
ctx: PlanToolContext,
op: (tree: PlanTree) => PlanTree,
): Promise<PlanToolResult> {
const { chatId, chatStore } = ctx;
@@ -65,7 +72,7 @@ async function runPlanOp(
}
}
export function planTools(ctx: TeacherToolContext): ToolSet {
export function planTools(ctx: PlanToolContext): ToolSet {
return {
plan_init: tool({
description: