feat: voice dictation (STT wave 1 — Deepgram cloud)
Two-band architecture from docs/research/stt.md: renderer captures mic audio in an AudioWorklet (linear16 16kHz mono, ~128ms frames, per-quantum RMS for the meter) and streams it directly to Deepgram over WebSocket — audio never loops through our server. The real key stays server-side; /api/stt/token mints a short-lived grant token (TTL 300s) per run. Patterns ported from the owner's pi dictate extension (tuned in real use): generation counter guarding every async handler, the latest-RMS-never-resets meter (-50..-10dB window, 60ms tick), mic-first stop + CloseStream + 3s safety net, endpointing=300 with the utterance_end_ms=1000 backstop. UI (owner placement): mic button in the stats bar's middle slot on all three surfaces (home graph-view, chat, draft); while recording the meter unfurls to fill the row's free width with the interim ghost at its flank. Esc ends dictation from the composer. Token mint comes before getUserMedia so a keyless app fails fast without a TCC prompt; one automatic reconnect on abnormal socket close, errors surface inline. Live-smoked the full error path in the dev shell (bar renders on all three pages, 403 grant failure shows inline, mic untouched). The full audio path needs a Deepgram key with Member permission to mint tokens.
This commit is contained in:
@@ -19,6 +19,7 @@ import { AnsweredCard } from "@/components/answered-card";
|
||||
import { artifactRenderers } from "@/components/artifacts";
|
||||
import { SubagentSteerCard } from "@/components/artifacts/steer-card";
|
||||
import { Composer } from "@/components/composer";
|
||||
import { DictationBar, useDictation } from "@/components/dictation";
|
||||
import { JobsContext } from "@/components/jobs-context";
|
||||
import { StatsBar } from "@/components/stats-bar";
|
||||
import { GraphContextMarker } from "@/components/graph-context-marker";
|
||||
@@ -246,6 +247,12 @@ export function Chat({
|
||||
(messages.at(-1)?.parts.some((p) => isToolUIPart(p) && p.state === "input-streaming") ?? false);
|
||||
|
||||
const [text, setText] = useState("");
|
||||
// Voice dictation (STT wave, 2026-09-17): finals append at the end of
|
||||
// the current text (the page owns the separator, dictate-flush style);
|
||||
// interims ghost in the stats bar's middle slot next to the meter.
|
||||
const dictation = useDictation({
|
||||
append: (t) => setText((v) => (v ? v.replace(/\s+$/, "") + " " + t : t)),
|
||||
});
|
||||
// Jobs never gate the composer (contracts.ts "THE ASYNC PROTOCOL" §4): job
|
||||
// tools resolve with an ack mid-turn, so the learner can interject while
|
||||
// sub-agents run — a feature, not a race: the wake pipeline queues steers
|
||||
@@ -864,6 +871,7 @@ export function Chat({
|
||||
phase={phases.at(-1) ?? "probe"}
|
||||
turns={turns}
|
||||
onReserve={setDockReserve}
|
||||
dictationSlot={<DictationBar dictation={dictation} />}
|
||||
composer={
|
||||
<Composer
|
||||
// Stop applies only to a LOCAL stream: a server-side turn has no
|
||||
@@ -874,6 +882,7 @@ export function Chat({
|
||||
onSend={onSend}
|
||||
onStop={() => stop()}
|
||||
canSend={canSend}
|
||||
onEscape={dictation.active ? dictation.stop : undefined}
|
||||
verdict={
|
||||
planVerdict
|
||||
? {
|
||||
@@ -940,6 +949,7 @@ function DockOverlay({
|
||||
onReserve,
|
||||
plan,
|
||||
onOpenPlan,
|
||||
dictationSlot,
|
||||
}: {
|
||||
pending: ReturnType<typeof findPendingToolCall>;
|
||||
/** Teach-phase pending question: arrives collapsed behind the reveal button (W5). */
|
||||
@@ -962,6 +972,9 @@ function DockOverlay({
|
||||
plan?: React.ReactNode;
|
||||
/** Opens the full-DAG popup (the marginalia + chip share it). */
|
||||
onOpenPlan: () => void;
|
||||
/** The dictation strip (STT wave): rendered in the stats bar's middle
|
||||
* slot — the mic button + live meter (components/dictation.tsx). */
|
||||
dictationSlot?: React.ReactNode;
|
||||
}) {
|
||||
const { isAtBottom, scrollToBottom, scrollRef } = useStickToBottomContext();
|
||||
// Teach-phase mount (session reload with a pending check): the dock starts
|
||||
@@ -1226,6 +1239,7 @@ function DockOverlay({
|
||||
chatId={chatId}
|
||||
settings={settings}
|
||||
onSettingsSaved={onSettingsSaved}
|
||||
dictationSlot={dictationSlot}
|
||||
/>
|
||||
</div>
|
||||
{/* The plan marginalia (owner, plan-placement round): the bare
|
||||
|
||||
Reference in New Issue
Block a user