amosblomqvist
543cf33789
The same extractor→auditor→merger pipeline now runs over raw personal
notes in arbitrary formats (vision §1, DECIDED 2026-09-13):
- NotesSource records (data/graph/sources.json, deterministic id from
path hash → idempotent re-imports) play the chatId role in mappings,
checkpoints, and observation events; usage lands on the source record
(appendSourceUsage, idempotent per startedAt, kept across upserts).
- mergeFindings is chat-store-free: callers pass MergeSource
{messages, foldedAt} — sessions pass the chat record, notes pass
messages:null (no quiz-event rebuild; a deterministic rebuild needs
structured quiz tool parts).
- Second extractor prompt (NOTES_EXTRACTOR_INSTRUCTIONS) for general-
purpose extraction; understanding judges the AUTHOR's grasp. Auditor
wording input-agnostic. extract/audit share one generic runSubmitAgent.
- graphSnapshot.fileSources (srcId→filePath); notes sources render as
static path entries in the detail card (FileTextIcon, never a link).
- One shared pipeline instance (exported from chat-handler): same
single-flight registry, live status chips (label = file path), usage
accounting, and true abort for both front doors.
- Transport: POST /api/graph/import (multipart, 3-wide worker pool) +
Import notes… button in the graph view; scripts/import-notes.ts for
batch runs (concurrency 4).
- real-data-extraction/ gitignored (raw import sources never committed).
355/355 tests; typecheck, lint, build green.
27 lines
1.1 KiB
TypeScript
27 lines
1.1 KiB
TypeScript
/**
|
|
* GET /api/graph/status — the in-flight background extraction runs
|
|
* (vision §1: the pipeline status must be visible on the home graph).
|
|
* Thin transport over the pipeline's module-level registry: phase
|
|
* (extracting / auditing / merging) per chat, oldest first. Titles joined
|
|
* from the chat store; a missing chat record still reports the run.
|
|
*/
|
|
import { chatStore, extractionStatuses } from "@/src/server";
|
|
|
|
export const dynamic = "force-dynamic";
|
|
|
|
export async function GET() {
|
|
const runs = extractionStatuses().map((run) => ({
|
|
chatId: run.chatId,
|
|
// The run's own label (the file path for notes imports) wins; chat
|
|
// runs fall back to the chat store's title.
|
|
title: run.label ?? chatStore.getChat(run.chatId)?.title ?? run.chatId,
|
|
phase: run.phase,
|
|
startedAt: run.startedAt,
|
|
// Live per-stage usage (the money channel): model + per-step CallUsage,
|
|
// accumulated as steps bill. Cost and context-window folds are the
|
|
// consumer's (core/prices.ts helpers — nothing stores dollars).
|
|
usage: run.usage,
|
|
}));
|
|
return Response.json({ runs });
|
|
}
|