sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

tools: the standard argument heal — every registry, both seams

Browse Source 
The whitehead incident (2026-10-09): glm-5.3 emitted quiz type with
quotes baked in ('"predict"') — rejected ten times, byte-identical
retries, the lesson never started — and correctAnswer with the JSON
key-value colon glued on ([':parmenides']). Third GLM wobble incident,
and each prior heal was wired only where its incident landed; this one
landed precisely where the practice heal never reached.

Layer 1 — healEncodedArguments moves to core/argument-heal (the display
seam needs it too — the tool-parts law) with withArgumentHeal +
healToolRecord (agent/tools/heal) wired at EVERY registry: prober,
teacher, practice, bareQuestionTools (it had no prepareArguments at
all), demo-builder, and submit-agent (composing after
healStringifiedArrays; the auditor's extraTools wrapped).

Layer 2 — stripModelValueDebris (core/value-debris): leading/trailing
colons, quotes, backticks, whitespace strip at the MEMBERSHIP seams —
coerceCorrectAnswer (boundary check + display + grading in one),
normalizeQuizType (persisted wobbled records keep their role),
resolveTargetId (copy-syntax on targets).

Layer 3 — schema description hygiene: the quiz type and practice
mode/outcome/rung/missClass values now read as bare words — the quoted
JSON-key style invited quote-copying.

Display mirror — parseQuizInput heals the raw record on read, so a
boundary-healed quiz whose record keeps raw args renders instead of
becoming an invisible pending quiz (learner lockout).

Pins: tests/tool-heal.test.ts (the incident shapes end-to-end through
pi-ai validation, the wiring, the strips, the mirror). The stringy-
options pins flip to boundary-parity per tool-parts' own law; the
chat-flow garbage pin keeps its honest-reject intent with an unhealable
shape. 1471 tests, typecheck, lint green.
This commit is contained in:
amosblomqvist
2026-10-09 04:54:34 +03:00
parent e4207e869f
commit c23eb1f15b
15 changed files with 446 additions and 136 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/tools/heal.ts
+48
View File
@@ -0,0 +1,48 @@
/**
* THE UNIVERSAL ARGUMENT HEAL — the agent-boundary wiring (owner,
* 2026-10-09 — generalize the 2026-10-04 practice heal).
*
* Every weak-model argument wobble incident produced a heal wired ONLY
* to the tool where it landed — parseOptionsIfString (questions,
* 2026-09-26), healStringifiedArrays (extraction),
* healEncodedArguments (practice, 2026-10-04) — and the whitehead death
* loop (2026-10-09: quiz `type: "\"predict\""` rejected ten times,
* byte-identical, in a session the lesson never reached) happened
* precisely because the practice heal never reached the quiz tools. The
* owner ruling: this is ONE standard pattern, applied at EVERY tool —
* heal what is mechanically recognizable (a JSON-encoded stand-in for
* the schema-expected shape; the membership-value debris strip lives in
* core/value-debris), let validation reject the rest honestly.
*
* The pure function lives in core/argument-heal (the display seam needs
* it too — the tool-parts law); this module is the per-tool/registry
* wiring at pi's prepareArguments seam — BEFORE TypeBox validation.
* Well-formed args pass through UNTOUCHED (same reference); heals are
* idempotent, so a per-tool heal composed under the record-level
* wrapper is a harmless second pass.
*/
import type { AgentTool } from "@earendil-works/pi-agent-core";
import { healEncodedArguments } from "../../core/argument-heal.ts";
export { healEncodedArguments };
/** Wrap ONE tool with the standard heal: the tool's own prepareArguments
* (options-unwrap, target normalization, …) runs first, then the heal
* against the tool's declared schema. Spread-safe (these are plain
* object literals) and idempotent. */
export function withArgumentHeal(tool: AgentTool<any>): AgentTool<any> {
const inner = tool.prepareArguments;
return {
...tool,
prepareArguments: (args: unknown) => healEncodedArguments(inner ? inner(args) : args, tool.parameters),
};
}
/** The registry-wide form (the owner's "standard pattern everywhere"):
* wrap every tool in a tool record. Applied at the registry-assembly
* points — createProberTools, createTeacherTools, practiceTools,
* bareQuestionTools, the subagent sets — so no tool can ship unwrapped. */
export function healToolRecord(tools: Record<string, AgentTool<any>>): Record<string, AgentTool<any>> {
return Object.fromEntries(Object.entries(tools).map(([name, tool]) => [name, withArgumentHeal(tool)]));
}
src/agent/tools/practice.ts
+11 -90
View File
@@ -26,91 +26,8 @@ import { proberGraphTools, type GraphToolContext } from "./graph.ts";
import { activeProblemOf } from "../../core/native-messages.ts";
import type { LhMessage } from "../../core/native-messages.ts";
import { concludeResult, invalidInputResult, problemResult } from "../tool-execute.ts";
import { healEncodedArguments, healToolRecord } from "./heal.ts";
// ─── the weak-model argument heal ─────────────────────────────────────────────────
/** One property schema of a TypeBox object (the shapes this fork emits:
* literal unions carry anyOf[{const}], scalars carry type). */
type PropSchema = {
type?: string;
const?: unknown;
anyOf?: PropSchema[];
};
/** The literal values a property accepts ([] when it is not a literal
* union / bare literal). */
function literalsOf(prop: PropSchema): unknown[] {
if (prop.anyOf) return prop.anyOf.flatMap((m) => literalsOf(m));
return "const" in prop ? [prop.const] : [];
}
/** Does the property accept null (Type.Null standalone or as a union
* member)? */
function acceptsNull(prop: PropSchema): boolean {
if (prop.type === "null") return true;
return (prop.anyOf ?? []).some((m) => m.type === "null");
}
/** JSON.parse that never throws (undefined for malformed input). */
function tryParse(raw: string): unknown {
try {
return JSON.parse(raw) as unknown;
} catch {
return undefined;
}
}
/** THE WEAK-MODEL ARGUMENT HEAL (owner, 2026-10-04 — the GLM incident:
* 19 consecutive rejected `problem` calls, every one carrying
* `"mode": "\"recall\""` — a JSON-string-encoded enum — while the
* TypeBox error text `must be equal to constant` never named the expected
* values, so the model re-emitted the identical payload each time). At
* pi's prepareArguments seam — BEFORE validation — the same discipline as
* healStringifiedArrays (extraction) and parseOptionsIfString (questions):
* a STRING value where the schema expects a literal union / boolean /
* null / number / array is JSON.parse-d and kept when the parsed value
* satisfies the expectation. Well-formed args pass through UNTOUCHED
* (same reference); unhealable values stay for validation to reject
* honestly. */
export function healEncodedArguments(
args: unknown,
schema: { properties?: Record<string, PropSchema> },
): unknown {
if (typeof args !== "object" || args === null || Array.isArray(args)) return args;
const properties = schema.properties;
if (!properties) return args;
let healed: Record<string, unknown> | null = null;
for (const [key, prop] of Object.entries(properties)) {
const raw = (args as Record<string, unknown>)[key];
if (typeof raw !== "string") continue;
const parsed = tryParse(raw);
let fixed: unknown;
const literals = literalsOf(prop);
if (literals.length > 0) {
if (literals.includes(raw)) continue; // already a valid member
if (literals.includes(parsed)) fixed = parsed;
}
// The literal branch does not cover null (Type.Null carries no
// const) — a union of literals ∪ null must also accept "null".
if (fixed === undefined && acceptsNull(prop) && parsed === null) fixed = null;
if (fixed === undefined && prop.type === "boolean" && typeof parsed === "boolean") {
fixed = parsed;
}
if (
fixed === undefined &&
(prop.type === "number" || prop.type === "integer") &&
typeof parsed === "number"
) {
fixed = parsed;
}
if (fixed === undefined && prop.type === "array" && Array.isArray(parsed)) fixed = parsed;
if (fixed !== undefined) {
healed ??= { ...(args as Record<string, unknown>) };
healed[key] = fixed;
}
}
return healed ?? args;
}
/** The record-reading context the gates need (the plan-tools idiom). */
export interface PracticeToolContext {
@@ -159,7 +76,7 @@ export const problemInputObject = Type.Object({
"solvable from the session's material. NEVER announces the target proposition.",
}),
mode: Type.Union([Type.Literal("recall"), Type.Literal("discrimination"), Type.Literal("composition")], {
description: 'REQUIRED — the demand mode: "recall" | "discrimination" | "composition".',
description: "REQUIRED — the demand mode: recall, discrimination, or composition (bare words, no quotes).",
}),
target: Type.Optional(
Type.Array(Type.String(), {
@@ -254,14 +171,14 @@ export const concludeInputObject = Type.Object({
Type.Literal("surrendered"),
Type.Literal("dismissed"),
],
{ description: 'REQUIRED — "solved" | "solvedWithHints" | "movedOn" | "surrendered" ("dismissed" is synthesized, never yours).' },
{ description: "REQUIRED — the outcome: solved, solvedWithHints, movedOn, or surrendered — bare words, no quotes (dismissed is synthesized, never yours)." },
),
verdict: Type.String({
description: "REQUIRED — the plain-language verdict: what was credited, where it broke. On a wrong answer: localize the break, do NOT give the correction.",
}),
rung: Type.Union(
[Type.Literal("R0"), Type.Literal("R1"), Type.Literal("R2"), Type.Literal("R3"), Type.Literal("R4"), Type.Literal("R5"), Type.Null()],
{ description: "REQUIRED — the hint-ladder rung the problem needed (null = no hint was given)." },
{ description: "REQUIRED — the hint-ladder rung the problem needed: R0 through R5, or null = no hint was given (bare, no quotes)." },
),
missClass: Type.Union(
[
@@ -272,7 +189,7 @@ export const concludeInputObject = Type.Object({
Type.Literal("structural"),
Type.Null(),
],
{ description: "REQUIRED — the miss classification (null = no miss)." },
{ description: "REQUIRED — the miss classification: cue, discrimination, execution, careless, or structural — or null = no miss (bare, no quotes)." },
),
retest: Type.Boolean({ description: "REQUIRED — true marks the problem for a fresh-surface retest before the session ends." }),
});
@@ -312,7 +229,11 @@ export function practiceTools(ctx: PracticeToolContext): Record<string, AgentToo
// toggle-filtered: problem/conclude are the session; these are
// exceptions the doctrine says to reach for rarely.
const jobs = ctx.jobs ? jobTools(ctx.jobs) : null;
return {
// healToolRecord (owner, 2026-10-09 — the whitehead generalization):
// the standard argument heal over the WHOLE registry. problem/conclude
// keep their own (target-normalizing) prepareArguments as the composed
// inner pass; the double heal is idempotent by construction.
return healToolRecord({
problem: problemTool(gate, ctx.graphNodes),
conclude: concludeTool(gate),
read: readTool,
@@ -322,5 +243,5 @@ export function practiceTools(ctx: PracticeToolContext): Record<string, AgentToo
...(jobs?.research ? { research: jobs.research } : {}),
...(jobs ? { pdf: jobs.pdf! } : {}),
...(ctx.graph ? proberGraphTools(ctx.graph) : {}),
};
});
}