fix: quiet Retrying lines, quiz option descriptions nuked, dividers at the markers

Three owner rulings from the gradient-descent session
(chat_muhskq6p_e8c3600b — 7 failed quiz calls, answers leaked in
option descriptions, planning divider floating at the transcript top):

1. RETRYING LINES (general): any tool call that errors back to the
   agent (validation failure, malformed args — the retry loop) renders
   one quiet "Retrying…" SystemLine instead of silence. Question tools
   previously rendered null, so a flailing quiz loop read as a stall.
   Aborts are terminal ("Operation aborted") and stay silent. Verified
   live: exactly 7 lines render at the 7 failure sites of the session.

2. QUIZ OPTION DESCRIPTIONS: NUKED. The model used them to telegraph
   correctness in every quiz of the session ("that moves uphill — it
   climbs f instead of descending it") — answerable by elimination.
   The field is gone from BOTH quiz schemas (probe + teach); ask keeps
   it (ungraded — nothing to leak). The dock additionally suppresses
   descriptions for quiz kind, so pre-fix records stop leaking too.
   Verified live: the session's pending quiz renders bare labels.

   Plus the diagnosed failure-mode fixes: GLM sometimes emits `options`
   as a JSON-encoded string (6 of 7 failures). prepareArguments hooks
   on all three question tools coerce it BEFORE pi's validation (the
   call lands instead of bouncing); the client coerces again in
   normalizeOptions (pi persists raw call args). Schema descriptions
   hardened ("REQUIRED ... never a JSON-encoded string").

3. DIVIDERS AT THE MARKERS: the probe-handoff SystemLine
   ("Preparing your plan…") IS NOW the Planning divider; the accepted
   verdict IS NOW the Teaching divider ("✓ Plan accepted" line gone) —
   one signal per seam, not two. The fold-based divider block is
   DELETED: Wave D grouping merged the whole probe into one giant
   assistant group, so the inclusive-fold comparison floated the
   divider to the transcript top (reproduced against the real record:
   6 view messages, phase flipped at the giant group). The markers
   render in the right spot by record construction. A DECLINED verdict
   is not a seam — it keeps the "↻ Plan revision requested" line.
   Verified live: plan divider after the probe concluded, teach
   divider after the plan presentation, nothing at the top.

Tests: 1004/1004 — string-coercion pins (normalizeOptions + the
prepareArguments hooks on all three tools, identity fast path),
quiz-schema-has-no-description pin (ask keeps it).
This commit is contained in:
amosblomqvist
2026-09-26 19:08:39 +03:00
parent b84ab1cdc0
commit 9853075331
6 changed files with 186 additions and 44 deletions
+29
View File
@@ -8,6 +8,7 @@ import { Value } from "typebox/value";
import type { TSchema } from "typebox";
import {
askInputSchema,
quizInputBase,
askOptionProblems,
buildAnswer,
coerceCorrectAnswer,
@@ -184,6 +185,34 @@ describe("normalizeOptions", () => {
it("throws on duplicate option values", () => {
assert.throws(() => normalizeOptions([{ label: "a", value: "x" }, { label: "b", value: "x" }]), /duplicate/);
// GLM wobble guard (2026-09-26 gradient-descent session): the model
// sometimes emits `options` as a JSON-encoded string — 6 of 7 quiz
// validation failures in that session. The dock (and grading) coerce
// via the same parseOptionsIfString the server's prepareArguments
// runs pre-validation.
const asString = JSON.stringify([{ label: "Slope", value: "slope" }, { label: "Value", value: "value" }]);
const coerced = normalizeOptions(asString);
assert.deepEqual(coerced, [
{ label: "Slope", value: "slope", description: undefined },
{ label: "Value", value: "value", description: undefined },
], "a JSON-encoded string coerces to the same normalized options");
// Non-JSON strings pass through and behave like no options (the
// server's schema validation reports the honest error instead).
assert.deepEqual(normalizeOptions("not json"), []);
});
it("quiz options carry NO description field; ask keeps it (owner nuke, 2026-09-26)", () => {
// The model used quiz option descriptions to telegraph correctness
// ("that's not the gradient") in every quiz of the gradient-descent
// session — the field is gone from BOTH quiz schemas; ask (ungraded,
// nothing to leak) keeps it.
const quizOption = (quizInputBase.properties.options as { items?: { properties?: Record<string, unknown> } })
.items!.properties!;
assert.ok(!("description" in quizOption), "quiz option schema has no description");
const askOption = (askInputSchema.properties.options as { items?: { properties?: Record<string, unknown> } })
.items!.properties!;
assert.ok("description" in askOption, "ask option schema keeps description");
});
it("handles undefined input", () => {