sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Auditor submits links ONLY — the edges argument is gone (owner, 2026-09-28 evening)

Browse Source 
Hypothesis under test by the owner: coherent extractions turn to mush
after merging because the auditor minted too many bridging edges of its
own — inference, not identity, and out of the auditor's scope. The graph
now connects across sources solely through identified equivalences;
relations between propositions come from the extractor alone.

- auditSchema drops 'edges' (submit_review = links only); the submit-seam
  required-field doctrine reworded per tool
- AUDITOR_INSTRUCTIONS: the BRIDGING EDGES section dies; deliverable is
  the identity verdicts only
- AuditVerdicts drops edges; ExtractedEdge endpoints are extractor-local
  ids only; mergeFindings stops consuming verdict edges and its resolve()
  drops the raw-node-id branch (dead — the extractor never sees graph ids)
- extractor untouched: submit_findings keeps propositions + follows/
  supports edges, required as before
- tests: 17 submit fixtures + 9 typed verdict literals slimmed to links
- docs: architecture.md division-of-trust + pipeline diagram updated
This commit is contained in:
amosblomqvist
2026-09-29 01:16:32 +03:00
parent f72a3f3ada
commit 3d9d20794c
6 changed files with 65 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/subagents/extraction.ts
+18 -26
View File
@@ -45,9 +45,10 @@ import { auditorGraphTools } from "../tools/graph.ts";
* proposition, linkTo an existing graph node or create —
* decided by EXPLORING the graph through tools (find /
* list_sessions / list_propositions), not by an injected
* graph dump (context bloat, owner 2026-09-27). It may
* also draw BRIDGING edges from existing graph nodes to
* the batch's propositions.
* graph dump (context bloat, owner 2026-09-27). Links are
* its ONLY deliverable — it never mints edges (owner
* 2026-09-28: the graph connects across sources solely
* through identified equivalences).
* merger (core) mergeFindings() → deterministic ledger appends
*
* The events ledger is GONE (2026-09-27): quiz outcomes and understanding
@@ -159,9 +160,10 @@ const findingsProps = {
const findingsSchema = Type.Object(findingsProps);
// VERDICTS-ONLY (owner, 2026-09-28): the auditor never alters the
// extractor's findings, so it does not restate them — links + bridging
// edges are its entire deliverable. An order-of-magnitude smaller submit
// than the old full-findings echo.
// extractor's findings, so it does not restate them — links are its
// ENTIRE deliverable (owner, 2026-09-28 evening: the edges argument is
// GONE; the auditor never mints edges. The graph connects across sources
// solely through identified equivalences — a link, never an inference).
const auditSchema = Type.Object({
links: Type.Array(
Type.Object({
@@ -173,10 +175,6 @@ const auditSchema = Type.Object({
}),
{ description: "Identity verdicts — which findings propositions ARE existing graph nodes. A proposition absent from here = create a new node. Pass [] when none — never omit the field." },
),
edges: Type.Array(edgeSchema, {
description:
"BRIDGING edges ONLY — new edges welding this batch into the existing graph, with an EXISTING graph node id as one endpoint (or both) and/or a findings proposition id. NEVER restate the extractor's own edges. Pass [] when none — never omit the field.",
}),
});
// The notes front door's shapes are IDENTICAL to the session shapes now
@@ -213,8 +211,9 @@ function healStringifiedArrays(raw: unknown): unknown {
// ── the submit seam (owner, 2026-09-28) ──
//
// Both top-level fields on both submit tools are REQUIRED — a payload
// missing `edges`/`links` fails validation instead of silently merging
// Every top-level field on each submit tool is REQUIRED — a payload
// missing `edges` (findings) or `links` (review) fails validation instead
// of silently merging
// edge-less (the degraded-retry hole: the model's first truncated
// attempt failed validation, then the retry passed with propositions
// only — optional edges ??-filled [] — merging a pile of nodes with no
@@ -230,7 +229,6 @@ function fillFindings(raw: Static<typeof findingsSchema>): SessionFindings {
function fillAudit(raw: Static<typeof auditSchema>): AuditVerdicts {
return {
links: raw.links,
edges: raw.edges,
};
}
@@ -464,18 +462,12 @@ IDENTITY — links, per proposition:
propositions — different claims about the world — stay apart. You are NOT guarding against wrong merges; you are
hunting for right ones.
BRIDGING EDGES — the one place you create edges:
- You may add edges whose endpoints are an EXISTING graph node id and one of the batch's propositions (use the
existing node's real id as "from" or "to"). This is how the new material gets welded into the old graph: when the
existing graph's content makes a proposition follow, or grounds it, draw the edge.
- Keep edges typed like the extractor's: "follows" = derivable; "supports" = grounds without entailing.
- Never invent propositions. NEVER restate the extractor's own edges — they merge as-is.
Your deliverable is ONLY the identity verdicts: "links". NEVER restate the propositions or the extractor's edges —
the merger already has them. You NEVER create edges — relations between the new propositions come from the
extractor alone, and the batch connects to the existing graph ONLY through the equivalences you identify. Your
entire deliverable is ONE submit_review call.
Your deliverable is ONLY the verdicts: "links" + "edges" (bridging ONLY). NEVER restate the propositions or the
extractor's edges — the merger already has them. Your entire deliverable is ONE submit_review call.
Include EVERY top-level field of the call even when empty — "links": [] when nothing links, "edges": [] when there
are no bridges. NEVER omit a field because it is empty.`;
Include the "links" field even when empty — "links": [] when nothing links. NEVER omit it because it is empty.`;
/** The auditor — one shape for both front doors now (the notes schema was
* identical once attribution died). An agentic loop: graph-exploration
@@ -490,7 +482,7 @@ function runAuditor(
return runSubmitAgent({
role: "auditor",
toolName: "submit_review",
toolDescription: "Submit the verdicts ONLY: identity links + bridging edges — never the propositions themselves.",
toolDescription: "Submit the identity verdicts ONLY (links) — never the propositions themselves.",
parameters: auditSchema,
fill: fillAudit,
systemPrompt: AUDITOR_INSTRUCTIONS,
@@ -834,7 +826,7 @@ export function createExtractionPipeline(deps: ExtractionPipelineDeps): Extracti
);
console.info(
`[extraction ${jobKey}] audit: ${findings.propositions.length} propositions through, ` +
`${auditResult.links.length} linked to existing nodes, ${auditResult.edges.length} bridging edges`,
`${auditResult.links.length} linked to existing nodes`,
);
console.info(`[extraction ${jobKey}] auditor done in ${Math.round(performance.now() - t0)}ms`);
return auditResult;
src/core/contracts/graph.ts
+10 -15
View File
@@ -36,9 +36,7 @@ export interface EmissionStats {
/** One provenance point — message-granular (DECIDED 2026-09-02): nodes and
* edges record every (chat, message) where they came up, deep-linkable to
* the exact transcript point. Auditor-drawn bridging edges carry the chat
* but NO message anchor (nothing in the transcript asserts them — the
* auditor is the asserter); messageId is "" there. */
* the exact transcript point. */
export interface ProvenancePoint {
/** The chat this point anchors to — or a notes source id ("src_…"):
* imports carry no message anchors (provenance is the file itself,
@@ -211,14 +209,12 @@ export interface ExtractedProposition {
}
/** One proposed relation. Endpoints are extractor-local proposition ids
* ("p1", …); the AUDITOR may also emit bridging edges whose from/to is an
* EXISTING graph node id (connecting the new content to what the graph
* already holds) — the merger resolves both. */
* ("p1", …) — the extractor is the ONLY stage that ever proposes edges
* (the auditor's edges argument is gone, owner 2026-09-28 evening). */
export interface ExtractedEdge {
/** Proposition id (or, for auditor-drawn bridges, a graph node id)
* that is followed / supported. */
/** Proposition id that is followed / supported. */
from: string;
/** Proposition id (or graph node id) that follows / is supported. */
/** Proposition id that follows / is supported. */
to: string;
type: EdgeType;
}
@@ -244,14 +240,13 @@ export interface IdentityLink {
/** The auditor's output — VERDICTS ONLY (owner, 2026-09-28): the auditor
* never alters the extractor's findings, so it does not restate them;
* the deterministic merger joins findings + verdicts. `links` = the
* identity verdicts (local id ⇄ existing node); `edges` = BRIDGING
* edges the auditor draws from existing graph nodes to the batch's
* propositions (endpoints: existing node ids and/or local proposition
* ids). No drop channel — the auditor never rejects; the extractor is
* fully trusted (2026-09-27). */
* identity verdicts (local id ⇄ existing node) — the auditor's ENTIRE
* deliverable (owner, 2026-09-28 evening: the edges argument is GONE —
* the auditor never mints edges; the graph connects across sources
* solely through identified equivalences). No drop channel — the
* auditor never rejects; the extractor is fully trusted (2026-09-27). */
export interface AuditVerdicts {
links: IdentityLink[];
edges: ExtractedEdge[];
}
/** The GET /api/graph payload — everything the home view's PURE renderer needs. */
src/core/graph.ts
+9 -8
View File
@@ -11,10 +11,11 @@
* from the source ALONE (whole session as markdown,
* or raw notes). Fully trusted: no stage re-checks it.
* auditor (LLM) → AuditVerdicts: identity links (local id → an existing
* node, or create) via graph-exploration tools, plus
* any BRIDGING edges it draws between existing graph
* nodes and the new propositions. Never sees the
* source; never drops.
* node, or create) via graph-exploration tools —
* links ONLY (owner, 2026-09-28 evening: the auditor
* never mints edges; cross-source connection happens
* solely through identified equivalences). Never sees
* the source; never drops.
* merger (HERE) → mergeFindings(): deterministic ledger appends
*
* The merger is deliberately the ONLY stage that touches the ledgers, and it
@@ -199,8 +200,9 @@ export function mergeFindings(
if (line.provenance.some((pt) => pt.chatId === chatId)) assertedByChat.add(key);
}
const emitEdge = (edge: ExtractedEdge): void => {
const resolve = (endpoint: string): string | undefined =>
mapping.get(endpoint) ?? (existingIds.has(endpoint) ? endpoint : undefined);
// Extractor-local ids only — the auditor's bridging edges are gone
// (owner, 2026-09-28 evening), so raw node-id endpoints no longer occur.
const resolve = (endpoint: string): string | undefined => mapping.get(endpoint);
const from = resolve(edge.from);
const to = resolve(edge.to);
if (!from || !to || from === to) return; // unresolvable endpoint — skip
@@ -210,8 +212,7 @@ export function mergeFindings(
assertedByChat.add(key);
stats.edges++;
};
for (const edge of findings.edges) emitEdge(edge);
for (const edge of verdicts.edges) emitEdge(edge);
for (const edge of findings.edges) emitEdge(edge); // the extractor's edges — the ONLY edge source
// 3. Checkpoint — THE COVERAGE DISCIPLINE (vision-extraction-coverage.md):
// only now, after every append succeeded, is the batch covered. The