Match the pi learning system's tool definitions (quiz/ask)
Compared the port against the original pi setup
(vault/π/learn/.pi: quiz.ts, ask-user-question.ts, teach SKILL.md) —
the pi version never produced invalid correctAnswer values. Root cause
of the port's validation failures: the pi options schema's 'Give each
option a stable value; you reference the correct one by that value in
correctAnswer' sentence was dropped in the port. Without it the model
omits values and must copy a long natural-language label verbatim into
correctAnswer — and instead slugifies it ('cultural-diagnosis' for
'A cultural diagnosis') every turn.
Now matched to pi:
- quiz schema: pi's field order (question, details, options,
multiSelect, correctAnswer, explanation) and descriptions — options
carry the stable-value instruction, correctAnswer is phrased as a
value reference ('the value field of the option you intend'), not a
copy-exact-or-fail threat. nodeId (lh-only, graph) stays.
- ask schema: dropped the redundant 'mode' param (pi has none; the dock
already infers from options presence), pi's descriptions.
- quiz tool description: pi's full text (ask contrast, note field,
dontKnow semantics) minus terminal-only details, plus lh's turn
protocol sentence.
- system-prompt: fixed the stale 'exactly two tools' line (plan/research/
demo/visual tools have their own sections).
Not matched, deliberately: tool results stay JSON (pi sends curated
text; formatAnswerMessage exists if we want it later), shuffle param
(lh always shuffles seeded), pi's ask '(Recommended)' label hint.
This commit is contained in:
Amos
@@ -137,7 +137,7 @@ If you catch yourself asserting a fact he'd have to take on faith — foundation
|
||||
|
||||
## Tools
|
||||
|
||||
You have exactly two tools: `quiz` and `ask_user_question`. They are precise instruments with distinct jobs.
|
||||
You have exactly two teaching instruments: `quiz` and `ask_user_question`. They are precise instruments with distinct jobs. (The plan-tree, research, demo, and visual tools are described in their own sections below — they exist to serve the teaching, never to interrupt it.)
|
||||
|
||||
**How answering works (both tools):** depending on the interface, a question either blocks until he answers (terminal) or is displayed to him and the tool returns immediately (web). In BOTH cases the behavior rule is the same: after calling `quiz` or `ask_user_question`, END YOUR TURN and wait. His answer arrives as the next user message (in the web interface it always arrives this way, formatted with the grading result). Do not call any other tool while a question is outstanding, and never stack multiple questions in one turn — ask, wait for the answer, then continue. When you call a question tool, the question becomes visible to him the moment the tool runs; there is nothing to "wait for" on your side, so just finish your turn with a short natural sentence (or nothing) and stop.
|
||||
|
||||
|
||||
+11
-5
@@ -22,13 +22,19 @@ import { planTools } from "./tools/plan.ts";
|
||||
*/
|
||||
|
||||
export const quizTool = tool({
|
||||
// Description ported from the pi original (the learning system this app
|
||||
// grew out of) — that one never produced invalid correctAnswer values.
|
||||
description:
|
||||
"Ask the learner a GRADED question with a known correct answer, then instantly grade and give feedback. " +
|
||||
"Use it to (1) assess what the learner already understands before teaching, and (2) run tight practice/retrieval loops after explaining. " +
|
||||
"Options-only: single-select or multi-select, plus an automatic 'I don't know' choice so the learner can signal a genuine gap instead of guessing. " +
|
||||
"correctAnswer must be the EXACT value (or label) of one of the listed options — a correctAnswer that matches no option fails validation and the call is returned to you to fix. " +
|
||||
"No free-text answers — for non-graded questions use ask_user_question instead. " +
|
||||
"Ask exactly one question per tool call; after calling, end your turn and wait for the graded result.",
|
||||
"Unlike ask_user_question (which collects preferences/decisions with no right answer), quiz always has a " +
|
||||
"correct answer supplied by you, marks the learner's selection right/wrong, reveals the correct answer, and " +
|
||||
"can show an explanation. Use it to (1) assess what the learner already understands before teaching, and " +
|
||||
"(2) run tight practice/retrieval loops after explaining, or probe understanding whenever you're unsure " +
|
||||
"they've got it. Options-only: single-select or multi-select, plus an automatic 'I don't know' choice so the " +
|
||||
"learner can signal a genuine gap instead of guessing. An always-present optional note field lets the learner " +
|
||||
"attach a free-text note to ANY answer; it reaches you only when non-empty. No free-text answers — for " +
|
||||
"non-graded questions use ask_user_question instead. Ask exactly one question per tool call; after calling, " +
|
||||
"end your turn and wait for the graded result.",
|
||||
inputSchema: quizInputSchema,
|
||||
});
|
||||
|
||||
|
||||
+26
-18
@@ -109,25 +109,35 @@ export function quizInputProblems(input: {
|
||||
|
||||
export const quizInputSchema = z
|
||||
.object({
|
||||
question: z.string().describe("The single quiz question to ask"),
|
||||
question: z.string().describe("The single quiz question to ask. Ask exactly one question per tool call."),
|
||||
options: z
|
||||
.array(optionSchema)
|
||||
.min(2)
|
||||
.describe(
|
||||
"The answer options (2 or more). Options only — no free-text mode. Never add your own 'I don't know' option.",
|
||||
"The answer options (2 or more). Options only — there is no free-text mode. " +
|
||||
"Give each option a stable `value`; you reference the correct one by that value in correctAnswer. " +
|
||||
"Never add your own 'I don't know' option — that choice is added for you.",
|
||||
),
|
||||
multiSelect: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe("Set to true when more than one option is correct and the learner must select all of them."),
|
||||
correctAnswer: z
|
||||
.union([z.string(), z.array(z.string())])
|
||||
.describe(
|
||||
"The correct answer as the option value(s): copy the EXACT `value` — or `label` when `value` is omitted — " +
|
||||
"of the intended option(s), never a paraphrase or slug; a mismatch fails validation. " +
|
||||
"Single-select: one string. Multi-select: an array; the learner is only correct if their selection matches this set exactly.",
|
||||
"REQUIRED. The correct answer as the option value(s) — the `value` field of the option you intend. " +
|
||||
'Single-select: a single string (e.g. "mercury"). Multi-select: an array of strings (e.g. ["belize", "niue"]); ' +
|
||||
"the learner is only correct if their selection matches this set exactly. " +
|
||||
"Always pass the value, not a position number — this is self-checking and prevents miscounting. " +
|
||||
"A value that matches no option fails validation and the call is returned to you to fix.",
|
||||
),
|
||||
explanation: z
|
||||
.string()
|
||||
.describe("Explanation revealed AFTER the learner answers. Always say why the correct answer is correct."),
|
||||
multiSelect: z.boolean().optional().describe("Set true only when more than one option is correct"),
|
||||
details: z.string().optional().describe("Optional extra context or instructions shown under the question"),
|
||||
.describe(
|
||||
"REQUIRED. Explanation revealed AFTER the learner answers (shown whether they got it right or wrong). " +
|
||||
"Use it to reinforce why the correct answer is correct.",
|
||||
),
|
||||
details: z.string().optional().describe("Optional extra context or instructions shown under the question."),
|
||||
nodeId: z
|
||||
.string()
|
||||
.optional()
|
||||
@@ -150,22 +160,20 @@ export const quizInputSchema = z
|
||||
|
||||
export const askInputSchema = z
|
||||
.object({
|
||||
question: z.string().describe("The single question to ask"),
|
||||
mode: z
|
||||
.enum(["single-select", "multi-select", "text"])
|
||||
.optional()
|
||||
.describe(
|
||||
"Answer mode. Omit for single-select when options are given, or text when they are not.",
|
||||
),
|
||||
// No `mode` param — the pi original has none: the mode is inferred from
|
||||
// whether options are present (the dock does exactly this). One less
|
||||
// knob for the model to misuse.
|
||||
question: z.string().describe("The single question to ask the learner. Ask exactly one question per tool call."),
|
||||
details: z.string().optional().describe("Optional extra context or instructions shown under the question."),
|
||||
options: z
|
||||
.array(optionSchema)
|
||||
.min(2)
|
||||
.optional()
|
||||
.describe(
|
||||
"Optional multiple-choice options. Omit for free-form text input. The learner can always choose 'Other' and type a custom answer.",
|
||||
"Optional multiple-choice options. Omit for free-form text input. " +
|
||||
"The learner can always choose 'Other' and type a custom answer when options are provided.",
|
||||
),
|
||||
multiSelect: z.boolean().optional().describe("Set true to allow multiple answers to be selected"),
|
||||
details: z.string().optional().describe("Optional extra context or instructions shown under the question"),
|
||||
multiSelect: z.boolean().optional().describe("Set to true to allow multiple answers to the same question."),
|
||||
})
|
||||
// Same option sanity as the quiz (labels non-empty, values distinct — ask
|
||||
// is not graded, but duplicate values break the dock's value keying).
|
||||
|
||||
Reference in New Issue
Block a user