Extraction pipeline: extractor → auditor → merger — the sole graph writer (stage 5)
The knowledge graph's write channel is now the pivot's target
architecture (vision §1 construction):
- src/agent/subagents/extraction.ts — the pipeline factory
(createExtractionPipeline({ chatStore, graphStore, extractor?,
auditor? }); models env LH_EXTRACT_MODEL ?? LH_MODEL):
* transcriptView(): the compact view both LLM stages see — final plan
tree with node ids, conversation verbatim, QUIZ lines with outcomes,
mark_done lines; job tool parts are one-liners, plan ops fold into
the tree line.
* Extractor: transcript → SessionFindings (propositions incl.
misconceptions and prose-only supports edges, attribution for
UNTAGGED probes, all with verbatim evidence) via one zod-validated
submit_findings call.
* Auditor: transcript + findings → the verified subset + drops (the
faithfulness net) via submit_review.
- src/core/graph.ts — emitChat/foldStaleChats/replayFinalTree retire;
mergeFindings() replaces them as the pipeline's deterministic apply
stage: exact/alias identity resolution (one mapping line per (chat,
graph node)), edge dedupe, and event rebuild FROM THE TRANSCRIPT's tool
parts — kinds/timestamps/refs/notes deterministic, the LLM's only event
say being WHICH proposition an untagged probe tested (retroactive —
the vision's open question #3, resolved structurally: tagged events
resolve deterministically; attribution rescues untagged and
tagged-but-unmapped ones; unattributable land null + question text).
pendingQuizAnswers() is the trigger helper.
- Trigger (turn-service.ts triggerExtraction, replacing the old
foldTurnIntoGraph body): once a chat accumulates
EXTRACTION.everyNQuizAnswers (LH_EXTRACT_N, default 3) new answered
quizzes, fire the pipeline async — single-flight per chat, failure
checkpoints nothing and re-runs on the next trigger. Deliberately a v1
placeholder isolated in ONE seam, trigger-agnostic pipeline.
- GET /api/graph is now a pure read (foldStaleChats gone).
- NodeMapping.treeNodeId → sourceId (the mapping key is now the writer's
local proposition id — tree node when tree-derived); store strict-read
schema updated. Local ledger data regenerates through the pipeline.
- Tests: graph.test.ts rewritten around mergeFindings (identity,
attribution paths incl. ghost-rescue, idempotency, cross-session
provenance, trigger counting), graph-context.test.ts merge section
converted to findings-driven, NEW extraction.test.ts — transcriptView
units + the full pipeline with mock models (audit drops, failure
checkpoints nothing, single-flight). 195/195, tsc/eslint/build clean.
Probe attribution (vision open question): probes attach retroactively by
the extractor against the whole transcript; a pre-tree probe maps to a
proposition the tree only articulates later. Fuzzy/embedding merge
adjudication stays deferred (conservative exact/alias only).
This commit is contained in:
@@ -1,9 +1,10 @@
|
||||
import { createAgentUIStreamResponse, type UIMessage } from "ai";
|
||||
import { createTeacherAgent, type TeacherAgent } from "../agent/teacher-agent.ts";
|
||||
import { createJobRunner } from "../agent/subagents/runner.ts";
|
||||
import { createExtractionPipeline } from "../agent/subagents/extraction.ts";
|
||||
import { createTeacherTools } from "../agent/tools.ts";
|
||||
import { stores } from "./stores.ts";
|
||||
import { buildTeacherContext, foldTurnIntoGraph, persistTurn, spawnTurnJobs } from "./turn-service.ts";
|
||||
import { buildTeacherContext, persistTurn, spawnTurnJobs, triggerExtraction } from "./turn-service.ts";
|
||||
|
||||
/**
|
||||
* One chat turn: run the teacher agent over the client's message list and
|
||||
@@ -18,6 +19,7 @@ import { buildTeacherContext, foldTurnIntoGraph, persistTurn, spawnTurnJobs } fr
|
||||
*/
|
||||
|
||||
const jobRunner = createJobRunner(stores);
|
||||
const extraction = createExtractionPipeline(stores);
|
||||
|
||||
export async function handleChatRequest(req: Request, agent?: TeacherAgent): Promise<Response> {
|
||||
let body: { chatId?: unknown; messages?: unknown };
|
||||
@@ -46,7 +48,7 @@ export async function handleChatRequest(req: Request, agent?: TeacherAgent): Pro
|
||||
onFinish: ({ messages, isAborted }) => {
|
||||
persistTurn(stores.chatStore, chatId, messages as UIMessage[], isAborted);
|
||||
spawnTurnJobs(jobRunner, chatId, messages as UIMessage[]);
|
||||
foldTurnIntoGraph(stores, chatId);
|
||||
triggerExtraction(stores, extraction, chatId);
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user