sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

question tools: phase-visible names, hardened teach schemas, context-slim seams

Browse Source 
The tools_fails_2 diagnosis (glm-5.3 dropping trailing required fields):
- probe-phase quiz/explain renamed to probe/recall — a shared name carried
  the probe call shape into teaching (the first teach quiz dropped nodeId);
  display seams accept the old names forever
- teach quiz+explain schemas lead with the required pair (type, nodeId);
  explain gains the predict/check type and joins the reveal law (checks
  collapse, predictions arrive open; no-type records render check-style)
- plan_init/probe_done results go context-slim (owner, context diet):
  content = validated nodeIds + binding-id reminder / fixed ack; details
  keeps the tree for the popup and {ok:true} — serializer-verified that
  details never reaches the model; failure echoes dropped
- the cards carry the tool's own name (Probe/Recall) and the quiz type
  reads on the label (Prediction/Check) in composer and transcript
This commit is contained in:
amosblomqvist
2026-10-09 07:59:23 +03:00
parent c23eb1f15b
commit 1f41fb05a5
21 changed files with 411 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/tools/plan.ts
+34 -11
View File
@@ -3,9 +3,17 @@ import type { AgentTool } from "@earendil-works/pi-agent-core";
import { serverExecute } from "../tool-execute.ts";
import type { PlanToolResult, PlanTree } from "../../core/contracts.ts";
import type { ChatStore } from "../../core/ports.ts";
import { createTree, emptyTree, PlanTreeError } from "../../core/plan-tree.ts";
import { createTree, PlanTreeError } from "../../core/plan-tree.ts";
import { parseMermaidPlan } from "./mermaid-plan.ts";
/** The binding-id reminder riding plan_init's slim content (owner,
* 2026-10-09): plan_init is the exact moment the node ids become
* load-bearing — every later quiz/explain call must reference one. The
* field-drop incidents (tools_fails_2) showed the turn's tail is where
* discipline slips; the reminder lands right there. */
const PLAN_NODE_IDS_REMINDER =
"These are the plan's node ids — the only valid nodeId values for every quiz and explain call from here on. Reference them exactly.";
/** What the plan tools close over (the plan-tree half of the tool context). */
export interface PlanToolContext {
chatId: string;
@@ -33,10 +41,13 @@ export interface PlanToolContext {
*
* The former mutation tools (add/update/remove node/edge, set_current,
* mark_done) are gone by design: restructuring is a rewrite, tracking is
* derived. SERVER-SIDE with execute: parse → validate → persist → return
* the FULL canonicalized tree (PlanToolResult), so the tree panel renders the
* latest state straight from the transcript's tool outputs (the plan
* window and the plan blocks).
* derived. SERVER-SIDE with execute: parse → validate → persist. The
* result is CONTEXT-SLIM (owner, context diet 2026-10-09): the model
* WROTE the mermaid one message ago, so content never echoes the tree —
* it carries the validated nodeIds + the binding-nodeId reminder (the
* exact moment the ids become load-bearing); `details` keeps the full
* tree for the record and the plan popup (latestTree reads it — the
* agent-invisible channel, serializer-verified).
*/
const planInitInput = Type.Object({
mermaid: Type.String({
@@ -60,25 +71,37 @@ export function planTools(ctx: PlanToolContext): Record<string, AgentTool<any>>
"`-->` edges meaning 'from must be understood before to' (chains fine). 5–15 nodes: a map, not a syllabus — one reasoning step per node, compress. " +
"Every check you fire names the node it assesses via `nodeId`; where the lesson stands derives from those tags.",
parameters: planInitInput,
execute: serverExecute("plan_init", planInitInput, async ({ mermaid }): Promise<PlanToolResult> => {
execute: serverExecute("plan_init", planInitInput, async ({ mermaid }) => {
const { chatId, chatStore } = ctx;
const record = chatStore.getChat(chatId);
if (!record) return { ok: false, error: `Chat not found: ${chatId}`, tree: emptyTree() };
const current = record.tree ?? emptyTree();
if (!record) return { ok: false, error: `Chat not found: ${chatId}` };
const parsed = parseMermaidPlan(mermaid);
if (!parsed.ok || !parsed.plan) {
// The parser collects every problem; the teacher fixes and retries.
return { ok: false, error: parsed.problems.join("\n"), tree: current };
// No tree echo — the model needs the problems, not a restatement of
// the map it is rewriting (owner, context diet 2026-10-09).
return { ok: false, error: parsed.problems.join("\n") };
}
try {
// createTree validates the structure (duplicate ids, missing
// edge endpoints, self-loops, cycles → PlanTreeError).
const next = createTree(parsed.plan.nodes, parsed.plan.edges);
chatStore.saveTree(chatId, next);
return { ok: true, tree: next };
// Context diet: content = the validated nodeIds + the reminder (the
// ids just became binding for every quiz/explain call); details =
// the full tree for the record/UI — the plan popup's latestTree
// seam, agent-invisible (the serializer sends content only).
return {
text: JSON.stringify({
ok: true,
nodeIds: next.nodes.map((n) => n.id),
reminder: PLAN_NODE_IDS_REMINDER,
}),
details: { ok: true, tree: next },
};
} catch (err) {
if (err instanceof PlanTreeError) return { ok: false, error: err.message, tree: current };
if (err instanceof PlanTreeError) return { ok: false, error: err.message };
throw err;
}
}),
src/agent/tools/probe.ts
+14 -4
View File
@@ -1,6 +1,6 @@
import type { AgentTool } from "@earendil-works/pi-agent-core";
import { serverExecute } from "../tool-execute.ts";
import { probeDoneInputSchema, type ProbeDoneResult } from "../../core/contracts.ts";
import { probeDoneInputSchema, PROBE_DONE_ACK_TEXT } from "../../core/contracts.ts";
import { danglingQuestionOf, hasProbeDone, type LhMessage } from "../../core/native-messages.ts";
import type { ChatStore } from "../../core/ports.ts";
import { buildProbeHandoffSteer, type SteerMessage } from "../../core/contracts.ts";
@@ -37,7 +37,10 @@ function probeConcluded(messages: readonly LhMessage[]): boolean {
* steer through the wake pipeline that fires the teacher's first turn
* (plan authoring) the moment the prober's turn ends. Auto-wake is the
* user-decided trigger — the learner experiences assessment → plan as
* one continuous flow.
* one continuous flow. CONTEXT-SLIM (owner, context diet 2026-10-09):
* the call's arguments already carry the assessment (durable, streamed
* live), so the agent sees the fixed PROBE_DONE_ACK_TEXT — never an
* echo. details = {ok:true} only.
*
* The first probe_done in the transcript is also the construction seam:
* from that message on, every request builds the TEACHER agent.
@@ -58,7 +61,7 @@ export function probeDoneTool(ctx: ProbeDoneContext): Record<string, AgentTool<a
// The core-owned schema (contracts/probe.ts) — the SAME schema the UI
// parse seam (tool-parts.ts) runs, so boundary-valid ⇔ consumer-parseable.
parameters: probeDoneInputSchema,
execute: serverExecute("probe_done", probeDoneInputSchema, async (input): Promise<ProbeDoneResult> => {
execute: serverExecute("probe_done", probeDoneInputSchema, async (input) => {
const record = ctx.chatStore.getChat(ctx.chatId);
if (!record) return { ok: false, error: `Chat not found: ${ctx.chatId}` };
@@ -81,7 +84,14 @@ export function probeDoneTool(ctx: ProbeDoneContext): Record<string, AgentTool<a
// constructs the teacher off the transcript seam) — record the
// assessment anyway; the wake is an optimization, not a gate.
}
return { ok: true, assessment: input.assessment };
// Context diet (owner, 2026-10-09): the assessment lives ONCE — as
// this call's arguments. content is the fixed ack (the agent never
// sees an echo); details = {ok:true} for the record/UI — the
// probe-done card reads the call's input, which it always preferred.
return {
text: PROBE_DONE_ACK_TEXT,
details: { ok: true },
};
}),
},
};