sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

quiz nodeId required (owner ruling 2026-09-26): no untagged teach quizzes — boundary rejects, model retries tagged; off-map checks route to ask_user_question; anti-rush doctrine in schema+prompt; docs+comments updated

Browse Source
This commit is contained in:
amosblomqvist
2026-09-27 02:05:51 +03:00
parent 8c0f0f7fbf
commit d6d7c09e2e
9 changed files with 72 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/prompts/teacher/10-planning.md
+1 -1
View File
@@ -26,7 +26,7 @@ You write the map with `plan_init` (a mermaid `graph TD` block). It is a **map,
The map is yours to rewrite at your own initiative: re-call `plan_init` whenever it's wrong — a detour node for a question worth exploring, a repair node for a caught misconception, prune what proves unnecessary. A map that can't change is a wrong map.
**Tag a quiz with `nodeId` whenever it's about a map node** — that's what moves the map: its live position is simply the latest tagged node, so asking about the next node IS how the lesson gets there. A tag is a position, not a verdict — nodes are never marked finished, a node takes as many questions as it deserves, and you move on by asking about the next one. Untagged questions are ad-hoc checks — legal, but they move nothing.
**Every quiz carries `nodeId`** (required — a quiz without it is rejected): that's what moves the map. Its live position is simply the latest tagged node, so asking about the next node IS how the lesson gets there. A tag is a position, not a verdict — nodes are never marked finished. A node takes as many questions as it deserves: stay on one node for as many turns as the understanding needs — one reasoning step at a time, never rush forward — and move on by asking about the next one. Off-map checks (no plan node fits) use ask_user_question.
## Presenting the plan — always, before any teaching
src/agent/tools.ts
+10 -9
View File
@@ -201,8 +201,8 @@ function nodeIdProblem(ctx: Pick<PlanToolContext, "chatId" | "chatStore">, nodeI
return (
`nodeId "${nodeId}" matches no node in this session's plan map. ` +
(known.length > 0
? `Copy one of your plan_init slugs exactly (${known.slice(0, 5).join(", ")}${known.length > 5 ? ", …" : ""}), or omit nodeId for an ad-hoc check.`
: "No plan map exists yet — omit nodeId (ad-hoc checks only until you call plan_init).")
? `Copy one of your plan_init slugs exactly (${known.slice(0, 5).join(", ")}${known.length > 5 ? ", …" : ""}).`
: "No plan map exists yet — call plan_init before posing quizzes.")
);
}
return null;
@@ -212,7 +212,9 @@ function nodeIdProblem(ctx: Pick<PlanToolContext, "chatId" | "chatStore">, nodeI
* + the per-session plan-nodeId validation (both run in execute — the
* grading contract is untouched). Reads the LIVE chat record, so rewrites
* mid-turn are honored, and with no tree yet every nodeId is rejected.
* Omitting nodeId is legal — an ad-hoc check that moves nothing. */
* nodeId is REQUIRED (owner ruling 2026-09-26): the untagged ad-hoc quiz is
* gone — off-map checks route through ask_user_question, and the teach
* seam (first tagged quiz) is structurally guaranteed. */
export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStore">): AgentTool<typeof quizInputObject> {
return {
name: "quiz",
@@ -222,8 +224,9 @@ export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStor
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
"can show an explanation. Use it to (1) confirm a taught node actually landed, or (2) probe understanding " +
"whenever you're unsure. Set `nodeId` whenever the question targets a map node — the lesson's position on the " +
"map derives from those tags; omit it only for quick ad-hoc checks. Options-only: single-select or multi-select, " +
"whenever you're unsure. Set `nodeId` (required) to the plan node this question is about — the lesson's " +
"position on the map derives from those tags; a node takes as many questions as it deserves, so don't rush " +
"forward. Off-map checks use ask_user_question. Options-only: single-select or multi-select, " +
"plus an automatic 'I don't know' choice so the learner can signal a genuine gap instead of guessing. " +
"An always-present optional note field lets the learner attach a free-text note to ANY answer; it reaches you " +
"only when non-empty. No free-text answers — for non-graded questions use ask_user_question instead. " +
@@ -233,10 +236,8 @@ export function teachQuizToolFor(ctx: Pick<PlanToolContext, "chatId" | "chatStor
executionMode: "sequential",
execute: async (_toolCallId, input) => {
const problems = [...quizInputProblems(input)];
if (typeof input.nodeId === "string" && input.nodeId.length > 0) {
const nodeIdProblemText = nodeIdProblem(ctx, input.nodeId);
if (nodeIdProblemText) problems.push(nodeIdProblemText);
}
const nodeIdProblemText = nodeIdProblem(ctx, input.nodeId);
if (nodeIdProblemText) problems.push(nodeIdProblemText);
if (problems.length > 0) throw new ToolInputError("quiz", problems);
return questionResult();
},