sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

question tools: phase-visible names, hardened teach schemas, context-slim seams

Browse Source 
The tools_fails_2 diagnosis (glm-5.3 dropping trailing required fields):
- probe-phase quiz/explain renamed to probe/recall — a shared name carried
  the probe call shape into teaching (the first teach quiz dropped nodeId);
  display seams accept the old names forever
- teach quiz+explain schemas lead with the required pair (type, nodeId);
  explain gains the predict/check type and joins the reveal law (checks
  collapse, predictions arrive open; no-type records render check-style)
- plan_init/probe_done results go context-slim (owner, context diet):
  content = validated nodeIds + binding-id reminder / fixed ack; details
  keeps the tree for the popup and {ok:true} — serializer-verified that
  details never reaches the model; failure echoes dropped
- the cards carry the tool's own name (Probe/Recall) and the quiz type
  reads on the label (Prediction/Check) in composer and transcript
This commit is contained in:
amosblomqvist
2026-10-09 07:59:23 +03:00
parent c23eb1f15b
commit 1f41fb05a5
21 changed files with 411 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/core/contracts/plan.ts
+6 -1
View File
@@ -114,7 +114,12 @@ export function derivePlanStatuses(tree: PlanTree, messages: readonly PartsCarri
export interface PlanToolResult {
ok: boolean;
error?: string;
tree: PlanTree;
/** The saved tree — SUCCESS results only (owner, context diet
* 2026-10-09): this is the record/UI-facing `details` value. The
* agent-facing content never carries it — content is the validated
* nodeIds + reminder; failures carry just the error. The plan
* popup's latestTree seam reads this field. */
tree?: PlanTree;
}
src/core/contracts/probe.ts
+10 -1
View File
@@ -81,5 +81,14 @@ export interface ProbeDoneResult {
ok: boolean;
/** Refusal reason when ok === false (open question, already concluded). */
error?: string;
assessment?: ProbeAssessment;
}
/** The probe_done ack's model-facing text (owner, context diet
* 2026-10-09): the assessment is the CALL's arguments — durable in the
* transcript, streamed to the learner — so echoing it back in the
* result would duplicate the probe phase's largest payload in every
* future context. The agent sees this fixed line instead; `details`
* carries only {ok:true} for the record/UI. Same doctrine as
* CONCLUDE_ACK_TEXT. */
export const PROBE_DONE_ACK_TEXT =
"Assessment recorded — it is shown to the learner now and the teacher plans from it. Say your closing sentence and end your turn; the teaching phase takes over from here.";
src/core/pending.ts
+14 -4
View File
@@ -38,13 +38,23 @@ export interface PendingToolCall {
input: unknown;
}
/** Tools that render in the question dock and gate the composer. */
export const QUESTION_TOOLS = ["quiz", "ask_user_question", "explain"] as const;
/** Tools that render in the question dock and gate the composer. The
* probe-phase names (probe/recall, owner 2026-10-09) join their teach
* twins — old records keep the teach names; new probe-phase calls carry
* the phase names. */
export const QUESTION_TOOLS = [
"quiz",
"probe",
"ask_user_question",
"explain",
"recall",
] as const;
/** The tool→kind map (the dock's own fold, named once — the dismissal
* path and the dock must never disagree on what a tool is). */
* path and the dock must never disagree on what a tool is). The probe
* names fold onto their teach kinds — the dock is kind-driven. */
export function toolQuestionKind(tool: string): "quiz" | "ask" | "explain" {
return tool === "quiz" ? "quiz" : tool === "explain" ? "explain" : "ask";
return tool === "quiz" || tool === "probe" ? "quiz" : tool === "explain" || tool === "recall" ? "explain" : "ask";
}
/** Tools fulfilled by background sub-agent jobs (W2/W6). */
src/core/question-core.ts
+56 -18
View File
@@ -220,26 +220,41 @@ export function normalizeQuizType(raw: unknown): QuizQuestionType | undefined {
return v === "predict" ? "predict" : v === "check" ? "check" : undefined;
}
/** The reveal law as a pure predicate (owner 2026-10-03): a teach-phase
* CHECK collapses behind the "Check your understanding" reveal — a
* prediction NEVER does (it precedes the teaching and gates it;
* collapsing it would invert the cycle), and neither do probe/plan
* questions, ask, or explain. chat.tsx calls this for its pending
* question; the reading-thinking field mounts on the same truth. */
export function quizNeedsReveal(tool: string, rawInput: unknown, phase: string): boolean {
if (tool !== "quiz" || phase !== "teach") return false;
/** The reveal law as a pure predicate (owner 2026-10-03; explain joined
* 2026-10-09): a teach-phase CHECK collapses behind the "Check your
* understanding" reveal — a prediction NEVER does (it precedes the
* teaching and gates it; collapsing it would invert the cycle), and
* neither do probe/plan questions or ask. Both teach quizzing tools
* (quiz and explain) obey the same law. chat.tsx calls this for its
* pending question; the reading-thinking field mounts on the same
* truth. Records that predate the `type` field carry none — they render
* as checks, the standing default. */
export function questionNeedsReveal(tool: string, rawInput: unknown, phase: string): boolean {
if ((tool !== "quiz" && tool !== "explain") || phase !== "teach") return false;
return normalizeQuizType((rawInput as { type?: unknown } | null)?.type) !== "predict";
}
/** The teaching-cycle role field, shared VERBATIM by both teach quizzing
* tools (owner, 2026-10-09: explain gains the cycle's types — a
* free-recall check collapses behind the reveal exactly like a quiz
* check; a prediction never does). Bare words, never quoted (the
* quote-copy wobble — see the whitehead pins, tests/tool-heal.test.ts). */
const predictCheckType = Type.Union([Type.Literal("predict"), Type.Literal("check")], {
description:
"REQUIRED — the question's role in the teaching cycle, as a bare word (no quotes): predict = posed BEFORE " +
"you teach the step (its own turn; the learner answers, then your teaching resolves it — arrives open, never " +
"collapsed); check = posed AFTER the step's teaching is complete (arrives collapsed behind the Check your " +
"understanding reveal). ONE question per turn either way.",
});
/** The TEACH-side quiz object. The required pair leads the property
* order (owner, 2026-10-09): glm-5.3 dropped `type`/`nodeId` on long
* calls BECAUSE they trailed every content field — fields generated
* first get dropped least. The order is also the authoring order:
* commit to the node and the question's role, then write the question. */
export const quizInputObject = Type.Object({
...quizInputBase.properties,
type: Type.Union([Type.Literal("predict"), Type.Literal("check")], {
description:
"REQUIRED — the question's role in the teaching cycle, as a bare word (no quotes): predict = posed BEFORE " +
"you teach the step (its own turn; the learner answers, then your teaching resolves it — arrives open, never " +
"collapsed); check = posed AFTER the step's teaching is complete (arrives collapsed behind the Check your " +
"understanding reveal). ONE question per turn either way.",
}),
type: predictCheckType,
nodeId: Type.String({
description:
"REQUIRED plan-tree node id this question is about — must EXACTLY match one of this session's " +
@@ -249,6 +264,7 @@ export const quizInputObject = Type.Object({
"one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward. " +
"For off-map checks use ask_user_question.",
}),
...quizInputBase.properties,
});
/** The PROBE-side quiz declaration: the shared authoring contract, no
@@ -303,8 +319,12 @@ export const explainInputBase = Type.Object({
/** The TEACH-side explain object: base + the plan-position field (same
* semantics as the quiz's nodeId — the seam recognizes BOTH tools). */
/** The TEACH-side explain object: base + the teaching-cycle role field
* and the plan-position field (same semantics as the quiz's — the seam
* recognizes BOTH tools). The required pair leads the property order,
* exactly as the teach quiz's (owner, 2026-10-09). */
export const explainInputObject = Type.Object({
...explainInputBase.properties,
type: predictCheckType,
nodeId: Type.String({
description:
"REQUIRED plan-tree node id this question is about — must EXACTLY match one of this session's " +
@@ -314,6 +334,17 @@ export const explainInputObject = Type.Object({
"one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward. " +
"For off-map checks use ask_user_question.",
}),
...explainInputBase.properties,
});
/** The DISPLAY parse for persisted explain calls (tool-parts.ts): the
* base + `type` as an OPTIONAL plain string — exactly the quiz display
* pattern. Old records carry none; questionNeedsReveal treats them as
* checks (the standing default). nodeId stays stripped (unknown key).
* The PROBE recall parses against the bare base. */
export const explainDisplayInputSchema = Type.Object({
...explainInputBase.properties,
type: Type.Optional(Type.String()),
});
/** The explain authoring contract: the commitment must exist. Empty
@@ -404,6 +435,12 @@ export interface QuestionData {
* toolCallId restated — the toolResult message already carries it. */
export interface QuestionAnswerData {
kind: QuestionKind;
/** The persisted call's tool NAME (owner, 2026-10-09): probe-phase
* calls are `probe`/`recall`, teach calls `quiz`/`explain` — the
* cards label by it ("Probe"/"Recall"), so the phase boundary is
* visible in the transcript. Absent on pre-rename records: they
* keep the kind-based labels. */
tool?: string;
/** quiz only (teach phase): the role snapshot for rendering (a
* prediction tucks its explanation) and later calibration splits.
* Absent on old records and probe answers. */
@@ -644,9 +681,10 @@ function formatOptionRef(options: QuizOption[], index: number): string {
* Turn an AnswerRequest into the full answer record + the user message the
* agent will see. Quiz answers are graded here; ask answers are not.
*/
export function buildAnswer(qd: QuestionData, req: AnswerRequest): QuestionAnswerData {
export function buildAnswer(qd: QuestionData, req: AnswerRequest, tool?: string): QuestionAnswerData {
const base: QuestionAnswerData = {
kind: qd.kind,
...(tool ? { tool } : {}),
questionType: qd.questionType,
questionText: qd.question,
options: qd.options.map((o, i) => ({ index: i + 1, label: o.label })),
src/core/session-markdown.ts
+14 -2
View File
@@ -90,8 +90,20 @@ function toolResultBody(result: ToolResultMessage, ctx?: MarkdownContext): { hea
}>
| undefined;
if (result.toolName === "quiz" || result.toolName === "explain") {
const heading = result.toolName === "quiz" ? "Quiz" : "Recall check";
if (
result.toolName === "quiz" ||
result.toolName === "probe" ||
result.toolName === "explain" ||
result.toolName === "recall"
) {
const heading =
result.toolName === "quiz"
? "Quiz"
: result.toolName === "probe"
? "Probe"
: result.toolName === "recall"
? "Recall"
: "Recall check"; // explain — the teach free-recall name
const question = typeof out?.questionText === "string" && out.questionText ? out.questionText : "(no question text)";
const body = [`**Q:** ${question}`];
if (out?.dismissed === true) body.push("The learner skipped this check.");
src/core/tool-parts.ts
+8 -6
View File
@@ -29,7 +29,7 @@
import { Value } from "typebox/value";
import type { TSchema } from "typebox";
import { quizInputBase, quizDisplayInputSchema, askInputSchema, askOptionProblems, quizInputProblems, explainInputBase, explainInputProblems, type AskToolInput, type QuizToolInput, type ExplainToolInput } from "./question-core.ts";
import { quizDisplayInputSchema, askInputSchema, askOptionProblems, quizInputProblems, explainInputProblems, explainDisplayInputSchema, type AskToolInput, type QuizToolInput, type ExplainToolInput } from "./question-core.ts";
import { restoreMathEscapes } from "./math-delims.ts";
import { healEncodedArguments } from "./argument-heal.ts";
import { probeDoneInputSchema, type ProbeDoneInput } from "./contracts/probe.ts";
@@ -115,12 +115,14 @@ export function parseAskInput(input: unknown): AskToolInput | undefined {
return cleaned;
}
/** Explain parse for DISPLAY: the shared authoring base (the teacher's
* nodeId is an unknown key Value.Clean strips, same as the quiz seam) —
* both explain variants share the same fields, and the commitment
* refinement (explainInputProblems) mirrors the boundary chain. */
/** Explain parse for DISPLAY: the display schema (base + OPTIONAL
* lenient `type`, exactly the quiz pattern — teach explain calls carry
* the role since 2026-10-09; the commitment refinement
* (explainInputProblems) mirrors the boundary chain). Both explain
* variants share the same fields; the teacher's nodeId is an unknown
* key Value.Clean strips, same as the quiz seam. */
export function parseExplainInput(input: unknown): ExplainToolInput | undefined {
const cleaned = boundaryParse<ExplainToolInput>(explainInputBase, restoreMathEscapesDeep(input));
const cleaned = boundaryParse<ExplainToolInput>(explainDisplayInputSchema, restoreMathEscapesDeep(input));
if (!cleaned) return undefined;
if (explainInputProblems(cleaned).length > 0) return undefined;
return cleaned;