feat: voice dictation (STT wave 1 — Deepgram cloud)

Two-band architecture from docs/research/stt.md: renderer captures mic
audio in an AudioWorklet (linear16 16kHz mono, ~128ms frames, per-quantum
RMS for the meter) and streams it directly to Deepgram over WebSocket —
audio never loops through our server. The real key stays server-side;
/api/stt/token mints a short-lived grant token (TTL 300s) per run.

Patterns ported from the owner's pi dictate extension (tuned in real
use): generation counter guarding every async handler, the
latest-RMS-never-resets meter (-50..-10dB window, 60ms tick), mic-first
stop + CloseStream + 3s safety net, endpointing=300 with the
utterance_end_ms=1000 backstop.

UI (owner placement): mic button in the stats bar's middle slot on all
three surfaces (home graph-view, chat, draft); while recording the meter
unfurls to fill the row's free width with the interim ghost at its
flank. Esc ends dictation from the composer. Token mint comes before
getUserMedia so a keyless app fails fast without a TCC prompt; one
automatic reconnect on abnormal socket close, errors surface inline.

Live-smoked the full error path in the dev shell (bar renders on all
three pages, 403 grant failure shows inline, mic untouched). The full
audio path needs a Deepgram key with Member permission to mint tokens.
This commit is contained in:
amosblomqvist
2026-09-17 19:31:25 +03:00
parent d46c3dae93
commit e3bdbc2f2c
8 changed files with 532 additions and 0 deletions
+8
View File
@@ -4,6 +4,7 @@ import { useCallback, useState } from "react";
import { useRouter } from "next/navigation";
import { Conversation, ConversationContent, ConversationEmptyState } from "@/components/ai-elements/conversation";
import { Composer } from "@/components/composer";
import { DictationBar, useDictation } from "@/components/dictation";
import { GraphContextControl, effectiveContextTokens } from "@/components/graph-context-control";
import { PhaseChip } from "@/components/phase-divider";
import { SidebarToggle } from "@/components/sidebar";
@@ -74,6 +75,11 @@ export function DraftShell({
onSettingsSaved: (s: ChatSettings) => void;
}) {
const canSend = text.trim().length > 0;
// Voice dictation (STT wave, 2026-09-17): finals append at the end of the
// draft text; the strip lives in the stats bar's middle slot.
const dictation = useDictation({
append: (t) => onChange(text ? text.replace(/\s+$/, "") + " " + t : t),
});
return (
<div className="flex h-dvh">
<div className="flex min-w-0 flex-1 flex-col">
@@ -110,9 +116,11 @@ export function DraftShell({
onStop={() => {}}
canSend={canSend}
autoFocus
onEscape={dictation.active ? dictation.stop : undefined}
/>
</div>
<StatsBar
dictationSlot={<DictationBar dictation={dictation} />}
turns={[]}
chatId=""
settings={settings}