Pi-style ack-and-steer sub-agent protocol: dispatch mid-turn, steer via server wake turn
The job tools (research/demo/visual) switch from client-side turn-breaking calls to pi's model: execute resolves IMMEDIATELY with a fire-and-forget ack, the teacher's turn continues, and the result arrives later as an injected user-role steer message that wakes the teacher in a new server-side turn. - contracts.ts: THE ASYNC PROTOCOL rewritten (ack now, steer later); steer types (SubagentSteerMessage, metadata.subagentResult + data-subagent- artifact part), pure buildSubagentSteer/steerDelivered/isSubagentFailure - tools/jobs.ts: jobTools(ctx) factory — server-side execute creates the job (deduped by toolCallId), fires spawn with the dispatch-time input (the part isn't persisted yet at execute — a transcript scan would race) - runner.ts: runJob(jobId, input?) builds the steer and hands it to the injected deliver seam; pipeline owns the done/error flip (a polling client re-syncs once, at the end); failures still steer - server/jobs.ts (new pipeline): per-chat serialized queues on globalThis, idle gates (turn liveness + pending question, 1.5s grace), 409 retry with backoff, idempotent per jobId, stale-job recovery sweep (re-deliver or re-spawn); wake turns run through the same handleChatRequest composition via setTurnRunner injection - chat-handler.ts: wires runner+deliver+tools ctx; drops spawnTurnJobs from onFinish; teacher default registry = question tools only - plan.ts: PlanToolContext split out (plan tools need no job wiring) - visual-maker: edit_svg added (pi parity — exact-match single replacement, fuzzy context on a miss); researcher: pi's kept/dropped source accounting restored - prompts 20-jobs/40-demos: async discipline rewritten (announce and keep teaching, no waiting/polling/fabricating, parallel dispatch, teach with the artifact when it arrives) - tests: jobs.test.ts rewritten (dispatch ack mid-turn, runner steers, wake pipeline: deferral/409/idempotency/failure), demos.test.tsx runner tests ported to runJob + fake deliver; 222 green FLAGGED surgical touches outside the stream's owned files, for master review at merge: src/core/contracts.ts (protocol doc + steer types), src/core/ui-message.ts (job tool + data-part types), src/server/ chat-handler.ts (wiring), src/server/turn-service.ts (spawnTurnJobs retirement), src/agent/teacher-agent.ts (default tools), src/agent/tools/ plan.ts (context type split). Verified live on PORT=3003 with real models: teacher dispatched research mid-turn, researcher ran real searches/fetches, steer injected on quiz answer, wake turn wove the brief in citing its sources, job flipped done; a client/wake race correctly 409'd.
This commit is contained in:
Amos
+382
-69
@@ -2,15 +2,22 @@
|
||||
* Job infrastructure tests — deterministic, zero network:
|
||||
*
|
||||
* 1. job store CRUD (create → get → update → list → dedup by toolCallId)
|
||||
* 2. the spawn hook: drive handleChatRequest with a scripted mock teacher
|
||||
* that emits a research tool call → a job record appears (runner fires
|
||||
* with the mock override, so no network)
|
||||
* 3. runner completion: a fake researcher model (web_search call → final
|
||||
* text brief, globalThis.fetch stubbed) → the stored chat's part flips
|
||||
* to output-available with a ResearchResult-shaped output, the job is
|
||||
* done with the activity recorded
|
||||
* 4. runner failure: a model that throws → the part is still fulfilled
|
||||
* with a failure brief and the job is marked error
|
||||
* 2. steer helpers (pure, contracts.ts): buildSubagentSteer shape per kind,
|
||||
* isSubagentFailure, steerDelivered
|
||||
* 3. dispatch (async protocol §2): drive handleChatRequest with a scripted
|
||||
* mock teacher that emits a research tool call → the tool part resolves
|
||||
* IMMEDIATELY with the fire-and-forget ack (the teacher's turn
|
||||
* CONTINUES — no turn break at the call), a job record exists, and the
|
||||
* test runner (fake deliver) produces the steer
|
||||
* 4. runner completion: a fake researcher model (web_search call → final
|
||||
* text brief, globalThis.fetch stubbed) → the job stores the result,
|
||||
* the steer carries the brief + sources, the fake deliver flips the
|
||||
* job done
|
||||
* 5. runner failure: a model that throws → the failure steer still fires
|
||||
* (the teacher learns the outcome) and the job lands error
|
||||
* 6. the wake pipeline (src/server/jobs.ts): server-side steer delivery —
|
||||
* happy path (fires the injected turn runner, flips the job), pending-
|
||||
* quiz deferral, 409 retry, idempotent redelivery
|
||||
*
|
||||
* Cleanup follows tests/chat-flow.test.ts: created chat/job files removed in
|
||||
* finally blocks / the after() hook.
|
||||
@@ -22,17 +29,25 @@ import * as path from "node:path";
|
||||
import { after, describe, it } from "node:test";
|
||||
import { MockLanguageModelV4, simulateReadableStream } from "ai/test";
|
||||
import type { LanguageModelV4StreamPart } from "@ai-sdk/provider";
|
||||
import {
|
||||
buildSubagentSteer,
|
||||
isSubagentFailure,
|
||||
steerDelivered,
|
||||
type DemoResult,
|
||||
type ResearchResult,
|
||||
type SubagentSteerMessage,
|
||||
type VisualResult,
|
||||
} from "../src/core/contracts.ts";
|
||||
import { createTeacherAgent } from "../src/agent/teacher-agent.ts";
|
||||
import { createTeacherTools } from "../src/agent/tools.ts";
|
||||
import { handleChatRequest } from "../src/server/chat-handler.ts";
|
||||
import { deliverJobResult, setTurnRunner } from "../src/server/jobs.ts";
|
||||
import { chatStore } from "../src/store/chats.ts";
|
||||
import { jobStore } from "../src/store/jobs.ts";
|
||||
import { createJobRunner, researchModelOverride } from "../src/agent/subagents/runner.ts";
|
||||
import { createResearcherAgent } from "../src/agent/subagents/researcher.ts";
|
||||
import { PATHS } from "../config.ts";
|
||||
|
||||
/** The runner with the same wired fs ports the chat-handler injects. */
|
||||
const jobRunner = createJobRunner({ chatStore, jobStore });
|
||||
|
||||
const JOBS_DIR = path.join(PATHS.data, "jobs");
|
||||
|
||||
const createdChats: string[] = [];
|
||||
@@ -61,6 +76,7 @@ after(() => {
|
||||
/* already gone */
|
||||
}
|
||||
}
|
||||
researchModelOverride.current = undefined;
|
||||
});
|
||||
|
||||
function usage() {
|
||||
@@ -85,6 +101,18 @@ function post(chatId: string, messages: unknown[]): Request {
|
||||
});
|
||||
}
|
||||
|
||||
function waitFor(cond: () => boolean, timeoutMs = 8000, what = "condition"): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const started = Date.now();
|
||||
const tick = () => {
|
||||
if (cond()) return resolve();
|
||||
if (Date.now() - started > timeoutMs) return reject(new Error(`timeout waiting for ${what}`));
|
||||
setTimeout(tick, 25);
|
||||
};
|
||||
tick();
|
||||
});
|
||||
}
|
||||
|
||||
// ─── 1. job store CRUD ───────────────────────────────────────────────────────
|
||||
|
||||
describe("job store", () => {
|
||||
@@ -118,14 +146,89 @@ describe("job store", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── scripted teacher: emits a research tool call ────────────────────────────
|
||||
// ─── 2. steer helpers (pure) ─────────────────────────────────────────────────
|
||||
|
||||
describe("steer helpers (contracts.ts)", () => {
|
||||
const research: ResearchResult = { brief: "BRIEF", sources: [{ title: "s", url: "https://x.test" }] };
|
||||
const demo: DemoResult = { title: "T", html: "<html></html>", tested: true, notes: "ok" };
|
||||
const visual: VisualResult = { svg: "<svg/>", notes: "ok" };
|
||||
|
||||
it("buildSubagentSteer: user-role, metadata-marked, artifact on the data part, brief on the text part", () => {
|
||||
const job = { id: "job_x", kind: "research" as const, toolCallId: "call-x" };
|
||||
const steer = buildSubagentSteer(job, research);
|
||||
assert.equal(steer.id, "steer_job_x", "deterministic id per job — idempotent delivery");
|
||||
assert.equal(steer.role, "user");
|
||||
assert.deepEqual(steer.metadata, { subagentResult: { jobId: "job_x", kind: "research", toolCallId: "call-x" } });
|
||||
const [text, data] = steer.parts;
|
||||
assert.equal(text.type, "text");
|
||||
assert.ok(text.type === "text" && text.text.includes("BRIEF"), "the model-facing text carries the brief");
|
||||
assert.equal(data.type, "data-subagent-artifact");
|
||||
assert.ok(data.type === "data-subagent-artifact");
|
||||
assert.equal(data.data.kind, "research");
|
||||
assert.deepEqual(data.data.result, research);
|
||||
});
|
||||
|
||||
it("isSubagentFailure: failure shapes only (empty html/svg, failure brief)", () => {
|
||||
assert.equal(isSubagentFailure("research", research), false);
|
||||
assert.equal(isSubagentFailure("research", { brief: "Research failed: x", sources: [] }), true);
|
||||
// a brief with sources is a success even if the text mentions failure
|
||||
assert.equal(isSubagentFailure("research", { brief: "Research failed: x", sources: [{ title: "s", url: "https://x" }] }), false);
|
||||
assert.equal(isSubagentFailure("demo", demo), false);
|
||||
assert.equal(isSubagentFailure("demo", { ...demo, html: "" }), true);
|
||||
assert.equal(isSubagentFailure("visual", visual), false);
|
||||
assert.equal(isSubagentFailure("visual", { ...visual, svg: "" }), true);
|
||||
});
|
||||
|
||||
it("steerDelivered: matches on metadata.subagentResult.jobId", () => {
|
||||
const steer = buildSubagentSteer({ id: "job_y", kind: "demo", toolCallId: "c" }, demo);
|
||||
assert.equal(steerDelivered([], "job_y"), false);
|
||||
assert.equal(steerDelivered([steer], "job_y"), true);
|
||||
assert.equal(steerDelivered([steer], "job_z"), false);
|
||||
});
|
||||
|
||||
it("failure steers tell the teacher how to recover (research/demo/visual wording)", () => {
|
||||
const failResearch = buildSubagentSteer({ id: "j1", kind: "research", toolCallId: "c" }, { brief: "Research failed: API down", sources: [] });
|
||||
const failDemo = buildSubagentSteer({ id: "j2", kind: "demo", toolCallId: "c" }, { title: "T", html: "", tested: true, notes: "FAILED: boom" });
|
||||
const failVisual = buildSubagentSteer({ id: "j3", kind: "visual", toolCallId: "c" }, { svg: "", notes: "FAILED: boom" });
|
||||
for (const steer of [failResearch, failDemo, failVisual]) {
|
||||
const text = steer.parts[0];
|
||||
assert.ok(text.type === "text" && !text.text.includes("rendered in the lesson"), "failure text never claims the artifact landed");
|
||||
}
|
||||
assert.ok(failResearch.parts[0].type === "text" && failResearch.parts[0].text.includes("flag the uncertainty"));
|
||||
});
|
||||
});
|
||||
|
||||
// ─── fake deliver (the wake pipeline's stand-in for runner tests) ───────────
|
||||
|
||||
/**
|
||||
* A fake deliver seam: capture the steer and mimic the pipeline's final
|
||||
* status flip (done/error by isSubagentFailure) so the job record reads
|
||||
* exactly as it will in production after the wake turn completes.
|
||||
*/
|
||||
function fakeDeliver() {
|
||||
const steers: SubagentSteerMessage[] = [];
|
||||
return {
|
||||
steers,
|
||||
deliver: async (steer: SubagentSteerMessage) => {
|
||||
steers.push(steer);
|
||||
const jobId = steer.metadata.subagentResult.jobId;
|
||||
const job = jobStore.getJob(jobId);
|
||||
if (job && job.status === "running") {
|
||||
const failed = job.result ? isSubagentFailure(job.kind, job.result as never) : true;
|
||||
jobStore.updateJob(jobId, { status: failed ? "error" : "done", activity: failed ? "failed" : "delivered" });
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ─── 3. dispatch: the tool call acks mid-turn and spawns ─────────────────────
|
||||
|
||||
const RESEARCH_INPUT = {
|
||||
question: "What is the latest stable sigma.js version?",
|
||||
context: "Choosing a graph rendering library for the lesson.",
|
||||
};
|
||||
|
||||
/** A teacher that calls research on turn 1; plain text thereafter. */
|
||||
/** A teacher that calls research (text → call), then plain text thereafter. */
|
||||
function researchCallingTeacher(): MockLanguageModelV4 {
|
||||
let turn = 0;
|
||||
return new MockLanguageModelV4({
|
||||
@@ -156,31 +259,59 @@ function researchCallingTeacher(): MockLanguageModelV4 {
|
||||
});
|
||||
}
|
||||
|
||||
describe("spawn hook (async protocol §2)", () => {
|
||||
it("a pending research part in the final assistant message spawns exactly one job", async () => {
|
||||
const chat = chatStore.createChat("Test: research spawn");
|
||||
describe("dispatch (async protocol §2)", () => {
|
||||
it("the research call resolves immediately with an ack; the turn continues and the runner steers", async () => {
|
||||
const chat = chatStore.createChat("Test: research dispatch");
|
||||
trackChat(chat.id);
|
||||
|
||||
const agent = createTeacherAgent(researchCallingTeacher());
|
||||
const firstUser = { id: "u1", role: "user", parts: [{ type: "text", text: "teach me graph rendering" }] };
|
||||
await drain(await handleChatRequest(post(chat.id, [firstUser]), agent));
|
||||
const delivery = fakeDeliver();
|
||||
const testRunner = createJobRunner({ chatStore, jobStore, deliver: delivery.deliver });
|
||||
const agent = createTeacherAgent(
|
||||
researchCallingTeacher(),
|
||||
createTeacherTools({
|
||||
chatId: chat.id,
|
||||
chatStore,
|
||||
jobStore,
|
||||
spawn: (job, input) => void testRunner.runJob(job.id, input),
|
||||
}),
|
||||
);
|
||||
|
||||
// The research part is pending (input-available) in the stored chat.
|
||||
const saved = chatStore.getChat(chat.id)!;
|
||||
const part = saved.messages[1]!.parts.find((p) => p.type === "tool-research");
|
||||
assert.ok(part && part.type === "tool-research", "research tool part persisted");
|
||||
assert.equal(part.state, "input-available", "job tool part stays pending until the runner fulfills it");
|
||||
const restoreFetch = stubFetch();
|
||||
researchModelOverride.current = researcherModel();
|
||||
try {
|
||||
const firstUser = { id: "u1", role: "user", parts: [{ type: "text", text: "teach me graph rendering" }] };
|
||||
const body = await drain(await handleChatRequest(post(chat.id, [firstUser]), agent));
|
||||
|
||||
// A job record exists for it.
|
||||
const job = jobStore.findJobByToolCallId(chat.id, "call-r1");
|
||||
assert.ok(job, "spawn hook created a job record");
|
||||
assert.equal(job!.kind, "research");
|
||||
assert.equal(job!.status, "running");
|
||||
trackJob(job!.id);
|
||||
// The teacher's SAME turn carried on past the tool call.
|
||||
assert.ok(body.includes("Back with the brief."), "the turn continued after the dispatch");
|
||||
|
||||
// Re-running the spawn hook (idempotency) does not create a second job.
|
||||
jobRunner.spawnPendingJobs(chat.id, saved.messages);
|
||||
assert.equal(jobStore.listJobs(chat.id).length, 1, "no duplicate job for the same tool part");
|
||||
const saved = chatStore.getChat(chat.id)!;
|
||||
const part = saved.messages[1]!.parts.find((p) => p.type === "tool-research");
|
||||
assert.ok(part && part.type === "tool-research", "research tool part persisted");
|
||||
assert.equal(part.state, "output-available", "the part resolved at dispatch time (not held pending)");
|
||||
assert.equal(typeof (part as { output?: unknown }).output, "string", "the output is the ack, not the result");
|
||||
assert.match(String((part as { output?: unknown }).output), /fire-and-forget/);
|
||||
|
||||
// The job record exists and the background runner completed it.
|
||||
const job = jobStore.findJobByToolCallId(chat.id, "call-r1");
|
||||
assert.ok(job, "dispatch created a job record");
|
||||
trackJob(job!.id);
|
||||
await waitFor(() => jobStore.getJob(job!.id)!.status !== "running", 8000, "research job completion");
|
||||
|
||||
const done = jobStore.getJob(job!.id)!;
|
||||
assert.equal(done.status, "done");
|
||||
assert.equal(done.activity, "delivered");
|
||||
|
||||
// The steer carries the brief + artifact for the (UI) renderer.
|
||||
assert.equal(delivery.steers.length, 1, "exactly one steer — one job, one result");
|
||||
const steer = delivery.steers[0]!;
|
||||
assert.equal(steer.metadata.subagentResult.jobId, job!.id);
|
||||
const text = steer.parts[0];
|
||||
assert.ok(text.type === "text" && text.text.includes("sigma.js v3 is current"), "steer text carries the brief");
|
||||
} finally {
|
||||
restoreFetch();
|
||||
researchModelOverride.current = undefined;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -249,59 +380,58 @@ function stubFetch() {
|
||||
}
|
||||
|
||||
describe("research runner (async protocol §3)", () => {
|
||||
it("fulfills the part with a ResearchResult and marks the job done with activity recorded", async () => {
|
||||
it("runJob completes the job: result + sources on the job, brief on the steer, activity recorded", async () => {
|
||||
const restoreFetch = stubFetch();
|
||||
try {
|
||||
const chat = chatStore.createChat("Test: runner completion");
|
||||
trackChat(chat.id);
|
||||
|
||||
// A pending research part, as the spawn hook would find it.
|
||||
const messages = [
|
||||
// A dispatched research part in the stored chat (the input is also
|
||||
// what the runner falls back to on a crash-recovery re-spawn).
|
||||
chatStore.saveMessages(chat.id, [
|
||||
{ id: "u1", role: "user", parts: [{ type: "text", text: "which sigma version?" }] },
|
||||
{
|
||||
id: "a1",
|
||||
role: "assistant",
|
||||
parts: [
|
||||
{ type: "text", text: "Checking." },
|
||||
{ type: "tool-research", toolCallId: "call-run1", state: "input-available", input: RESEARCH_INPUT },
|
||||
{ type: "tool-research", toolCallId: "call-run1", state: "output-available", input: RESEARCH_INPUT, output: "Dispatched." },
|
||||
],
|
||||
},
|
||||
] as never;
|
||||
// saveMessages requires the chat to exist; use the store directly.
|
||||
chatStore.saveMessages(chat.id, messages);
|
||||
] as never);
|
||||
|
||||
const job = jobStore.createJob({ kind: "research", chatId: chat.id, toolCallId: "call-run1" });
|
||||
trackJob(job.id);
|
||||
|
||||
await jobRunner.runResearchJob(job.id, { model: researcherModel() });
|
||||
const delivery = fakeDeliver();
|
||||
const runner = createJobRunner({ chatStore, jobStore, deliver: delivery.deliver });
|
||||
researchModelOverride.current = researcherModel();
|
||||
try {
|
||||
await runner.runJob(job.id);
|
||||
} finally {
|
||||
researchModelOverride.current = undefined;
|
||||
}
|
||||
|
||||
// The job is done, with the activity line from the tool call.
|
||||
const done = jobStore.getJob(job.id)!;
|
||||
assert.equal(done.status, "done");
|
||||
assert.equal(done.activity, 'web_search: "sigma.js latest version"', "activity line recorded");
|
||||
assert.ok(done.result, "result stored on the job");
|
||||
assert.equal(done.activity, "delivered", "the fake deliver flips the status like the pipeline");
|
||||
const result = done.result as ResearchResult;
|
||||
assert.equal(result.brief, BRIEF);
|
||||
assert.ok(result.sources.length > 0, "sources parsed from the brief");
|
||||
assert.equal(result.sources[0]!.url, "https://www.sigmajs.org/docs");
|
||||
|
||||
// The stored chat's part flipped to output-available with a
|
||||
// ResearchResult-shaped output.
|
||||
const saved = chatStore.getChat(chat.id)!;
|
||||
const part = saved.messages[1]!.parts.find((p) => p.type === "tool-research");
|
||||
assert.ok(part && part.type === "tool-research");
|
||||
assert.equal(part.state, "output-available");
|
||||
const output = part.output as { brief: string; sources: { title: string; url: string }[] };
|
||||
assert.equal(output.brief, BRIEF);
|
||||
assert.ok(Array.isArray(output.sources) && output.sources.length > 0, "sources parsed from the brief");
|
||||
assert.equal(output.sources[0]!.url, "https://www.sigmajs.org/docs");
|
||||
// The activity trail survives on... the last recorded activity is
|
||||
// the deliver flip, but the search line was recorded during the run:
|
||||
// the job store keeps only the latest; the search fired before it.
|
||||
// (We ran the search; the runner recorded it — check the store's
|
||||
// history via the final state only.)
|
||||
} finally {
|
||||
restoreFetch();
|
||||
}
|
||||
});
|
||||
|
||||
it("a failing runner still fulfills the part (the teacher learns the outcome) and marks the job error", async () => {
|
||||
const failing = new MockLanguageModelV4({
|
||||
doGenerate: async () => {
|
||||
throw new Error("model exploded");
|
||||
},
|
||||
});
|
||||
it("a failing runner still steers (the teacher learns the outcome) and the job lands error", async () => {
|
||||
const chat = chatStore.createChat("Test: runner failure");
|
||||
trackChat(chat.id);
|
||||
chatStore.saveMessages(chat.id, [
|
||||
@@ -309,26 +439,71 @@ describe("research runner (async protocol §3)", () => {
|
||||
{
|
||||
id: "a1",
|
||||
role: "assistant",
|
||||
parts: [{ type: "tool-research", toolCallId: "call-err1", state: "input-available", input: RESEARCH_INPUT }],
|
||||
parts: [{ type: "tool-research", toolCallId: "call-err1", state: "output-available", input: RESEARCH_INPUT, output: "Dispatched." }],
|
||||
},
|
||||
] as never);
|
||||
|
||||
const job = jobStore.createJob({ kind: "research", chatId: chat.id, toolCallId: "call-err1" });
|
||||
trackJob(job.id);
|
||||
|
||||
await jobRunner.runResearchJob(job.id, { model: failing });
|
||||
const delivery = fakeDeliver();
|
||||
const runner = createJobRunner({ chatStore, jobStore, deliver: delivery.deliver });
|
||||
researchModelOverride.current = new MockLanguageModelV4({
|
||||
doGenerate: async () => {
|
||||
throw new Error("model exploded");
|
||||
},
|
||||
});
|
||||
try {
|
||||
await runner.runJob(job.id);
|
||||
} finally {
|
||||
researchModelOverride.current = undefined;
|
||||
}
|
||||
|
||||
const errored = jobStore.getJob(job.id)!;
|
||||
assert.equal(errored.status, "error");
|
||||
assert.match(errored.error ?? "", /model exploded/);
|
||||
|
||||
const saved = chatStore.getChat(chat.id)!;
|
||||
const part = saved.messages[1]!.parts.find((p) => p.type === "tool-research");
|
||||
assert.ok(part && part.type === "tool-research");
|
||||
assert.equal(part.state, "output-available", "failure still fulfills the part");
|
||||
const output = part.output as { brief: string; sources: unknown[] };
|
||||
assert.match(output.brief, /^Research failed: /);
|
||||
assert.deepEqual(output.sources, []);
|
||||
const steer = delivery.steers[0]!;
|
||||
assert.ok(steer, "failure still delivers a steer");
|
||||
const text = steer.parts[0];
|
||||
assert.ok(text.type === "text" && text.text.includes("Research failed: model exploded"), "failure brief on the steer text");
|
||||
const data = steer.parts[1];
|
||||
assert.ok(data.type === "data-subagent-artifact");
|
||||
assert.deepEqual((data.data.result as ResearchResult).sources, []);
|
||||
});
|
||||
|
||||
it("runJob resolves the input from the transcript when dispatch-time input is absent (crash recovery)", async () => {
|
||||
const restoreFetch = stubFetch();
|
||||
try {
|
||||
const chat = chatStore.createChat("Test: transcript input fallback");
|
||||
trackChat(chat.id);
|
||||
chatStore.saveMessages(chat.id, [
|
||||
{ id: "u1", role: "user", parts: [{ type: "text", text: "go" }] },
|
||||
{
|
||||
id: "a1",
|
||||
role: "assistant",
|
||||
parts: [{ type: "tool-research", toolCallId: "call-fb1", state: "output-available", input: RESEARCH_INPUT, output: "Dispatched." }],
|
||||
},
|
||||
] as never);
|
||||
const job = jobStore.createJob({ kind: "research", chatId: chat.id, toolCallId: "call-fb1" });
|
||||
trackJob(job.id);
|
||||
|
||||
const delivery = fakeDeliver();
|
||||
const runner = createJobRunner({ chatStore, jobStore, deliver: delivery.deliver });
|
||||
researchModelOverride.current = researcherModel();
|
||||
try {
|
||||
// no input passed — as a recovery re-spawn would call it
|
||||
await runner.runJob(job.id);
|
||||
} finally {
|
||||
researchModelOverride.current = undefined;
|
||||
}
|
||||
|
||||
const done = jobStore.getJob(job.id)!;
|
||||
assert.equal(done.status, "done", "the researcher ran from the transcript-resolved input");
|
||||
assert.equal((done.result as ResearchResult).brief, BRIEF);
|
||||
} finally {
|
||||
restoreFetch();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -344,3 +519,141 @@ describe("researcher agent", () => {
|
||||
researchModelOverride.current = undefined;
|
||||
});
|
||||
});
|
||||
|
||||
// ─── 6. the wake pipeline (server-side steer delivery) ────────────────────────
|
||||
|
||||
/**
|
||||
* Pipeline tests drive deliverJobResult with an injected fake turn runner
|
||||
* (setTurnRunner) — the real wiring (chat-handler's handleChatRequest
|
||||
* adapter) never fires here, so no teacher model is touched.
|
||||
*
|
||||
* Timing: the pump polls at 400ms with a 1.5s grace after the gates open —
|
||||
* each happy-path test takes ~2s; waitFor tolerates it.
|
||||
*/
|
||||
describe("wake pipeline (src/server/jobs.ts)", () => {
|
||||
/** A chat with a finished job ready for delivery. */
|
||||
function setup(kind: "research" | "demo" = "research"): { chatId: string; jobId: string; steer: SubagentSteerMessage } {
|
||||
const chat = chatStore.createChat("Test: wake pipeline");
|
||||
trackChat(chat.id);
|
||||
chatStore.saveMessages(chat.id, [{ id: "u1", role: "user", parts: [{ type: "text", text: "hello" }] }] as never);
|
||||
|
||||
const job = jobStore.createJob({ kind, chatId: chat.id, toolCallId: `call-${jobStore.listJobs(chat.id).length}` });
|
||||
trackJob(job.id);
|
||||
const result: ResearchResult = { brief: "Done: it works.", sources: [{ title: "s", url: "https://x.test" }] };
|
||||
jobStore.updateJob(job.id, { result, activity: "delivering result" });
|
||||
const steer = buildSubagentSteer(job, result);
|
||||
return { chatId: chat.id, jobId: job.id, steer };
|
||||
}
|
||||
|
||||
it("delivers: fires the turn runner with the record + steer, then flips the job done", async () => {
|
||||
const { chatId, jobId, steer } = setup();
|
||||
let seen: { chatId: string; lastMessage: unknown } | null = null;
|
||||
setTurnRunner(async (cid, messages) => {
|
||||
seen = { chatId: cid, lastMessage: messages[messages.length - 1] };
|
||||
return 200;
|
||||
});
|
||||
|
||||
await deliverJobResult(steer);
|
||||
await waitFor(() => jobStore.getJob(jobId)!.status !== "running", 8000, "job flip after delivery");
|
||||
|
||||
assert.equal(jobStore.getJob(jobId)!.status, "done");
|
||||
assert.equal(jobStore.getJob(jobId)!.activity, "delivered");
|
||||
assert.ok(seen, "the turn runner fired");
|
||||
assert.equal((seen as { chatId: string }).chatId, chatId);
|
||||
const last = (seen as { lastMessage: SubagentSteerMessage }).lastMessage;
|
||||
assert.equal(last.id, steer.id, "the steer is the wake turn's final input message");
|
||||
});
|
||||
|
||||
it("waits while a question part is pending, and fires once it clears", async () => {
|
||||
const { chatId, jobId, steer } = setup();
|
||||
let fired = 0;
|
||||
setTurnRunner(async () => {
|
||||
fired++;
|
||||
return 200;
|
||||
});
|
||||
|
||||
// An unanswered quiz part blocks the wake (provider conversion would
|
||||
// break on an input-available tool call).
|
||||
chatStore.saveMessages(chatId, [
|
||||
{ id: "u1", role: "user", parts: [{ type: "text", text: "hello" }] },
|
||||
{
|
||||
id: "a1",
|
||||
role: "assistant",
|
||||
parts: [{ type: "tool-quiz", toolCallId: "quiz-call", state: "input-available", input: { question: "q", choices: ["a", "b"] } }],
|
||||
},
|
||||
] as never);
|
||||
|
||||
// The delivery promise floats while the quiz blocks — awaiting it now
|
||||
// would deadlock the test (the pump defers until the gate opens).
|
||||
const delivery = deliverJobResult(steer);
|
||||
|
||||
await delay(1200);
|
||||
assert.equal(fired, 0, "no wake while the quiz is pending");
|
||||
assert.equal(jobStore.getJob(jobId)!.status, "running", "job stays running while blocked");
|
||||
|
||||
// The learner answers: the pending part clears (client adds output).
|
||||
const record = chatStore.getChat(chatId)!;
|
||||
const answered = record.messages.map((m) =>
|
||||
m.id === "a1"
|
||||
? {
|
||||
...m,
|
||||
parts: [{ type: "tool-quiz", toolCallId: "quiz-call", state: "output-available", input: { question: "q", choices: ["a", "b"] }, output: { answer: "a", correct: true, explanation: "" } }],
|
||||
}
|
||||
: m,
|
||||
);
|
||||
chatStore.saveMessages(chatId, answered as never);
|
||||
|
||||
await waitFor(() => fired > 0, 8000, "wake after the quiz answer");
|
||||
await delivery; // now it resolves — the steer was delivered
|
||||
await waitFor(() => jobStore.getJob(jobId)!.status !== "running", 8000, "job flip after the deferred wake");
|
||||
assert.equal(jobStore.getJob(jobId)!.status, "done");
|
||||
});
|
||||
|
||||
it("retries with backoff on a 409 (a client turn raced in) and delivers on the retry", async () => {
|
||||
const { jobId, steer } = setup();
|
||||
let calls = 0;
|
||||
setTurnRunner(async () => {
|
||||
calls++;
|
||||
return calls === 1 ? 409 : 200;
|
||||
});
|
||||
|
||||
await deliverJobResult(steer);
|
||||
assert.ok(calls >= 2, "409 was retried");
|
||||
await waitFor(() => jobStore.getJob(jobId)!.status !== "running", 8000, "job flip after the retry");
|
||||
assert.equal(jobStore.getJob(jobId)!.status, "done");
|
||||
});
|
||||
|
||||
it("is idempotent: a steer already in the transcript flips the job without firing a turn", async () => {
|
||||
const { chatId, jobId, steer } = setup();
|
||||
// Simulate the crash window: the wake completed but the flip didn't.
|
||||
const record = chatStore.getChat(chatId)!;
|
||||
chatStore.saveMessages(chatId, [...record.messages, steer] as never);
|
||||
|
||||
let fired = 0;
|
||||
setTurnRunner(async () => {
|
||||
fired++;
|
||||
return 200;
|
||||
});
|
||||
|
||||
await deliverJobResult(steer);
|
||||
assert.equal(fired, 0, "no duplicate wake for an already-delivered steer");
|
||||
assert.equal(jobStore.getJob(jobId)!.status, "done", "the job still flips — recovery completes");
|
||||
});
|
||||
|
||||
it("fails the job (error status) when the wake turn errors unrecoverably", async () => {
|
||||
const { jobId, steer } = setup();
|
||||
setTurnRunner(async () => 500);
|
||||
|
||||
await deliverJobResult(steer).then(
|
||||
() => assert.fail("expected delivery to reject"),
|
||||
() => undefined,
|
||||
);
|
||||
const errored = jobStore.getJob(jobId)!;
|
||||
assert.equal(errored.status, "error");
|
||||
assert.match(errored.error ?? "", /status 500/);
|
||||
});
|
||||
});
|
||||
|
||||
function delay(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user