sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

researcher: PDFs from the web — the pdf-reader's text tools in the loop (design B)

Browse Source 
web_fetch_pdf downloads (25MB cap, probe-validated) into a per-job temp
dir and returns the triage summary — page count, size class (the pdf
doctrine's <=10/<=40 thresholds), flagged math/image-heavy pages. The
researcher's toolset then gains pdf_info / pdf_extract / pdf_search over
that dir, scoped by a resolve-and-startswith wall: the researcher's
filesystem ends at the temp dir. Render is deliberately absent — the
researcher slot has no vision; visual reading stays the pdf job's job.

web_fetch: a PDF URL now returns a POINTER to web_fetch_pdf instead of
dead-ending, and every output carries 'Source: <final url after
redirects> (HTTP <status>)' — the 'ill-formed request' lesson: a
200-served error body must be distinguishable from a real page.

The runner owns the temp lifecycle (mkdtemp before, recursive cleanup in
the job's finally — never enters the record) and web_fetch_pdf urls ride
the sources ledger. Doctrine step 5 added: fetch, triage, extract only
the pages that matter, cite as (p. N).

Tests build a REAL minimal PDF programmatically (correct xref offsets —
pdfjs parses it like any download): zero network, real parse.
This commit is contained in:
amosblomqvist
2026-10-07 22:36:11 +03:00
parent d450689a8b
commit 8dfe621748
5 changed files with 463 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/agent/subagents/runner.ts
+12 -2
View File
@@ -19,6 +19,9 @@ import {
type VisualToolInput,
} from "../../core/contracts.ts";
import { registerJobAbort, unregisterJobAbort } from "./aborts.ts";
import * as fs from "node:fs/promises";
import * as os from "node:os";
import * as path from "node:path";
import type { ModelRole } from "../../../config.ts";
import { ROLE_SLOT, modelsNotConfigured, type ModelSlotName, type ProviderKeyMissing } from "../../core/settings.ts";
import type { ChatStore, JobStore } from "../../core/ports.ts";
@@ -353,20 +356,25 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner {
.join("\n\n");
const fetchedUrls: string[] = [];
// Per-job PDF dir (researcher PDF wave, 2026-10-07): web_fetch_pdf
// downloads land here; every pdf tool is scoped to it; it dies with
// the job (never enters the record).
const pdfDir = await fs.mkdtemp(path.join(os.tmpdir(), "lh-research-"));
try {
jobStore.updateJob(jobId, { activity: "researching" });
const brief = await runResearcherAgent(prompt, {
engine: model,
search: deps.searchProvider,
pdfDir,
abortSignal: loop.abortSignal,
chatId: loop.chatId,
// Money channel: each agent step is one model call — fold its
// usage into the record as it completes (progress write).
onUsage: (usage) => loop.onUsage(piUsageToStep(usage)),
onToolStart: (toolName, args) => {
onToolStart: (toolName, args) => {
const input = args as { query?: string; url?: string } | undefined;
if (toolName === "web_fetch" && typeof input?.url === "string") {
if (typeof input?.url === "string" && (toolName === "web_fetch" || toolName === "web_fetch_pdf")) {
fetchedUrls.push(input.url);
}
jobStore.updateJob(jobId, { activity: activityLine(toolName, input) });
@@ -383,6 +391,8 @@ export function createJobRunner(deps: JobRunnerDeps): JobRunner {
const reason = err instanceof Error ? err.message : String(err);
if (stillRunning(jobId)) jobStore.updateJob(jobId, { error: reason });
await finishJob(jobId, { brief: `Research failed: ${reason}`, sources: [] });
} finally {
await fs.rm(pdfDir, { recursive: true, force: true });
}
}