sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Wave B: the native message record — input events, projection, adapter death

Browse Source 
The chat record is now pi-native agent history (LhMessage[]), the AI SDK's
UIMessage transcript contract is gone from the server, and every client turn
arrives as ONE explicit input event.

Core (new):
- src/core/native-messages.ts — the LhMessage taxonomy (user/assistant/
  toolResult + lhMarker customs: verdict, probe-handoff, subagent-steer,
  subagent-abort, plan-block), constructors, and toLlmMessages — the single
  model-facing seam (convertToLlm): strips ids, flattens markers to user
  text, drops UI-only plan-blocks, synthesizes INTERRUPTED results for
  dangling aborted calls, never throws.
- src/core/projection.ts — projectNative: the one-way native → UIMessage
  projection (consecutive assistant+toolResult runs group into one
  assistant view message; toolResults attach backward by toolCallId;
  markers project to data parts; plan-blocks splice after the last
  successful plan_init). Every existing consumer reads through it.
- src/core/input-events.ts — the ChatInput union (text/answer/verdict/
  continue), parseChatInput validation, and inputAppends (answer targets
  the dangling question call or 400s; verdict = optional note user
  message + native verdict marker).

Server:
- chat-handler.ts — the turn root takes {chatId, input}; write-ahead
  persists input appends + claimed steers before the turn; claimSteers
  coalesces the whole queue with the handler's steerDelivered filter as
  the dedupe seam; teacher-vs-prober branch on hasProbeDone.
- turn-service.ts — persistTurn stamps ids, drops clean question
  placeholder toolResults (the record ends at the unanswered call),
  emits plan-block markers idempotently (plan-blocks.ts rewrite), and
  persists abort partials.
- ui-stream.ts — PiTurnSpec takes the native base; the wire dialect is
  unchanged. jobs.ts steers are native markers (steer_<jobId> dedup ids);
  waitForIdle reads danglingQuestionOf. teacher-agent.ts wires
  convertToLlm: toLlmMessages. record-adapter.ts DELETED wholesale.
- GET /api/chats/[id] serves messages: projectNative(record.messages).

Client:
- chat-transport.ts — the 409 retry is IDENTITY (re-POST the same event
  body after the turn settles — the event was never persisted
  pre-acceptance; the merge logic is gone). The transport consumes the
  stashed input event via prepareSendMessagesRequest and throws loudly on
  a missing stash.
- chat.tsx — every send call site stashes exactly one ChatInput event
  (text, answer, verdict, continue) before invoking the SDK send.

Tests: full native migration (chat-flow rewritten to input events and
native record pins; jobs/extraction/demos/graph/usage/plan-blocks/
probe-done/reattach-transport converted; record-adapter.test.ts deleted;
chat-transport tests pin the identity retry). 982/982, typecheck, lint,
build all green.
This commit is contained in:
amosblomqvist
2026-09-26 00:37:46 +03:00
parent b87a6925af
commit 7ab47d4fcb
36 changed files with 2410 additions and 1447 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/server/turn-service.ts
+68 -57
View File
@@ -4,69 +4,85 @@
* implementing. Each takes its dependencies explicitly (storage ports, the
* job runner) — no module here imports an adapter.
*
* persistTurn — the canonical state write (client + agent messages)
* persistTurn — the canonical state write (Wave B: native history;
* id stamping, placeholder filtering, plan-block
* emission, abort-partial persistence)
* triggerExtraction — the graph write channel: fires the extraction
* pipeline (the sole graph writer) once a chat
* crosses the v1 quiz-answer threshold. The trigger
* mechanism is a v1 placeholder — this seam is the
* ONLY thing to swap when a better one lands.
* crosses the token-budget threshold.
*
* Logs are the minimal-observability baseline: one line per failure, tagged
* [chat <id>] — structured logging lands behind these seams later.
*/
import type { UIMessage } from "ai";
import { withPlanBlocks } from "../core/plan-blocks.ts";
import type { AgentMessage } from "@earendil-works/pi-agent-core";
import type { ToolResultMessage } from "@earendil-works/pi-ai";
import type { LhMessage } from "../core/native-messages.ts";
import { newMessageId } from "../core/native-messages.ts";
import { QUESTION_TOOLS } from "../core/pending.ts";
import { planBlocksFor } from "../core/plan-blocks.ts";
import { projectNative } from "../core/projection.ts";
import { coverageOf } from "../core/coverage.ts";
import type { ChatStore, GraphStore } from "../core/ports.ts";
import type { ExtractionPipeline } from "../agent/subagents/extraction.ts";
import { ModelsNotConfiguredError, ProviderKeyMissingError } from "../core/settings.ts";
/**
* The canonical state write: the client owns the message list between turns
* (useChat); this persists client messages + the agent response. Persists
* even when aborted — the partial transcript (including any pending
* question part) is what makes resume work.
* The canonical state write (Wave B, owner-approved 2026-09-26): append the
* turn's own messages — the agent loop's native history, provider/model/
* usage/thinkingSignature stamped by pi itself. Persists even when aborted
* — the partial transcript (including a pending question's unanswered
* call) is what makes resume work.
*
* - Id stamping: pi messages carry no id; the record's identity needs one
* per message (client keys, append-only adoption, the continuation
* seam). Minted here, at the single canonical write.
* - Placeholder filtering: a cleanly-posed question's synthetic
* "question posed" result is an in-memory loop device (Wave A's
* terminate) — NEVER persisted (fork 1: the record ends at the
* unanswered call; the learner's answer arrives as the real tool
* result). Errored question calls DO persist — the model's retry
* context.
* - Plan blocks: one frozen snapshot per successful plan_init call,
* idempotent (persist replays are safe).
*
* Returns the stamped new messages (usage provenance reads them).
*/
/**
* The SDK assigns assistant-message ids CLIENT-side (useChat); the server's
* onFinish synthesis leaves them EMPTY. A message persisted only server-side
* (the last turn's, or a wake turn's — nobody re-POSTs it with a client id)
* would otherwise land in the record with id "" — and two of those in one
* transcript break React keys (the chat map) and steer-dedup lookups. Fill any
* missing id here, at the single canonical write: client-authored ids are
* NEVER touched (the client's next POST still carries them verbatim).
*/
export function withMessageIds(messages: UIMessage[]): UIMessage[] {
const filled = messages.map((m) =>
m.id ? m : { ...m, id: `msg_${Date.now().toString(36)}_${crypto.randomUUID().slice(0, 8)}` },
);
// Same id twice = the SDK's continuation merge couldn't happen in place
// (continuation coalescing appends a user-role steer AFTER the assistant
// message being continued, so onFinish carries the input copy AND the
// extended response copy under one id). Keep the LAST occurrence — it is
// the SDK's final merge state (a superset of the earlier copy).
const seen = new Set<string>();
const deduped: UIMessage[] = [];
for (let i = filled.length - 1; i >= 0; i--) {
const m = filled[i]!;
if (seen.has(m.id)) continue;
seen.add(m.id);
deduped.unshift(m);
}
return deduped;
}
export function persistTurn(chatStore: ChatStore, chatId: string, messages: UIMessage[], isAborted: boolean): void {
export function persistTurn(
chatStore: ChatStore,
chatId: string,
baseCount: number,
turnMessages: readonly AgentMessage[],
isAborted: boolean,
): LhMessage[] {
try {
const deduped = withMessageIds(messages);
// Plan-block emission (wave 4): blocks are frozen transcript content —
// attached here, once, at the canonical state write. The tree was
// saved by the plan tool during the turn; snapshots derive now.
const tree = chatStore.getChat(chatId)?.tree;
chatStore.saveMessages(chatId, withPlanBlocks(deduped, tree));
const record = chatStore.getChat(chatId);
if (!record) return [];
const stamped: LhMessage[] = [];
for (const m of turnMessages.slice(baseCount)) {
const message = m as LhMessage;
// The placeholder question result: loop device, never a record fact.
if (
(message as { role?: string }).role === "toolResult" &&
(QUESTION_TOOLS as readonly string[]).includes((message as ToolResultMessage).toolName) &&
!(message as ToolResultMessage).isError
) {
continue;
}
stamped.push(message.id ? message : ({ ...message, id: newMessageId() } as LhMessage));
}
const tree = record.tree;
const blocks = planBlocksFor(
record.messages,
stamped,
tree,
projectNative([...record.messages, ...stamped]),
);
chatStore.saveMessages(chatId, [...record.messages, ...stamped, ...blocks]);
return stamped;
} catch (err) {
console.error(`[chat ${chatId}] persist failed:`, err, { isAborted });
return [];
}
}
@@ -74,15 +90,11 @@ export function persistTurn(chatStore: ChatStore, chatId: string, messages: UIMe
* The graph write channel (vision §1 construction): the extraction pipeline
* (extractor → auditor → merger — the sole graph writer) runs ASYNC once a
* chat's UNCOVERED transcript tokens reach the chunk budget
* (vision-extraction-coverage.md, 2026-09-15 — replacing the old
* every-3-quiz-answers count: content size is the honest signal, and
* quiz-less sessions extract too). The budget is the graph section's
* extractionChunkTokens setting (extraction-UX redesign, 2026-09-24) —
* the composition root passes the per-turn effective read (chat-handler
* resolves fresh per turn, same as every other gate). The pipeline
* itself extracts only the uncovered batches. Fire-and-forget — never blocks the turn; failures are
* logged by the pipeline and leave coverage untouched, so the next trigger
* (or the manual extract button) re-runs what's left.
* (vision-extraction-coverage.md, 2026-09-15). Coverage reads through the
* projection (the record is native history; the extraction view stays
* client-shaped until Wave D). Fire-and-forget — never blocks the turn;
* failures are logged by the pipeline and leave coverage untouched, so the
* next trigger (or the manual extract button) re-runs what's left.
*
* Deliberately isolated in this one seam — the pipeline is
* trigger-agnostic. Skipped under tests: scripted chats are fixtures, not
@@ -98,7 +110,7 @@ export function triggerExtraction(
try {
const record = deps.chatStore.getChat(chatId);
if (!record) return;
const covered = coverageOf(record.messages as UIMessage[], deps.graphStore.readCoverage(), chatId);
const covered = coverageOf(projectNative(record.messages), deps.graphStore.readCoverage(), chatId);
if (covered.uncoveredTokens < chunkTokens) return;
void extraction.runExtraction(chatId).catch((err) => {
// BYOK (owner, 2026-09-22) + key presence (provider-key interlude,
@@ -119,4 +131,3 @@ export function triggerExtraction(
console.error(`[chat ${chatId}] extraction trigger failed:`, err);
}
}