feat: dictation round 1 — primary-input routing, no interims, optimistic meter, wide button
Owner's first live mic test (2026-09-17) drove four changes: - ONE primary transcription input at all times: while a quiz/ask question waits, finals land in its field (text mode → 'Your answer', select modes → 'Your thinking') via a module-level claim registry (useDictationTarget in dictation.tsx); the composer is the fallback. Esc in a question field also stops dictation (threaded through DockOverlay → QuestionDock). - No interim results: live words were distracting and shifted the meter. Params now match the pi dictate extension exactly (no interim_results, endpointing=300, no utterance_end_ms); the meter's width stays fixed. - Optimistic meter: the click enters the meter view immediately (Stop pill + meter within one frame, verified at 150ms in the dev shell); token mint and mic acquisition run concurrently behind it. - Wide idle button: the stats-bar middle is one full-width affordance — a quiet bordered 'Dictate' button when idle (failure text replaces the label), Stop pill + fixed meter while live. Live-verified in the Electron dev shell: wide button renders on home (167px) and chat (358px), meter + Stop appear at 150ms after click, flat 2px bars in silence, clean return to 'Dictate' on stop.
This commit is contained in:
@@ -872,6 +872,7 @@ export function Chat({
|
|||||||
turns={turns}
|
turns={turns}
|
||||||
onReserve={setDockReserve}
|
onReserve={setDockReserve}
|
||||||
dictationSlot={<DictationBar dictation={dictation} />}
|
dictationSlot={<DictationBar dictation={dictation} />}
|
||||||
|
dictationStop={dictation.stop}
|
||||||
composer={
|
composer={
|
||||||
<Composer
|
<Composer
|
||||||
// Stop applies only to a LOCAL stream: a server-side turn has no
|
// Stop applies only to a LOCAL stream: a server-side turn has no
|
||||||
@@ -950,6 +951,7 @@ function DockOverlay({
|
|||||||
plan,
|
plan,
|
||||||
onOpenPlan,
|
onOpenPlan,
|
||||||
dictationSlot,
|
dictationSlot,
|
||||||
|
dictationStop,
|
||||||
}: {
|
}: {
|
||||||
pending: ReturnType<typeof findPendingToolCall>;
|
pending: ReturnType<typeof findPendingToolCall>;
|
||||||
/** Teach-phase pending question: arrives collapsed behind the reveal button (W5). */
|
/** Teach-phase pending question: arrives collapsed behind the reveal button (W5). */
|
||||||
@@ -975,6 +977,9 @@ function DockOverlay({
|
|||||||
/** The dictation strip (STT wave): rendered in the stats bar's middle
|
/** The dictation strip (STT wave): rendered in the stats bar's middle
|
||||||
* slot — the mic button + live meter (components/dictation.tsx). */
|
* slot — the mic button + live meter (components/dictation.tsx). */
|
||||||
dictationSlot?: React.ReactNode;
|
dictationSlot?: React.ReactNode;
|
||||||
|
/** Esc in a pending question's field stops dictation (the strip is the
|
||||||
|
* stats bar's; the stop belongs to the page-owned hook, threaded down). */
|
||||||
|
dictationStop?: () => void;
|
||||||
}) {
|
}) {
|
||||||
const { isAtBottom, scrollToBottom, scrollRef } = useStickToBottomContext();
|
const { isAtBottom, scrollToBottom, scrollRef } = useStickToBottomContext();
|
||||||
// Teach-phase mount (session reload with a pending check): the dock starts
|
// Teach-phase mount (session reload with a pending check): the dock starts
|
||||||
@@ -1225,6 +1230,7 @@ function DockOverlay({
|
|||||||
phase={phase}
|
phase={phase}
|
||||||
onExpand={expandAndReveal}
|
onExpand={expandAndReveal}
|
||||||
onSubmit={onSubmitAnswer}
|
onSubmit={onSubmitAnswer}
|
||||||
|
onEscape={dictationStop}
|
||||||
/>
|
/>
|
||||||
) : (
|
) : (
|
||||||
composer
|
composer
|
||||||
|
|||||||
+129
-92
@@ -11,13 +11,13 @@ import { Loader2Icon, MicIcon } from "lucide-react";
|
|||||||
* opens the Deepgram streaming WebSocket DIRECTLY (audio never loops
|
* opens the Deepgram streaming WebSocket DIRECTLY (audio never loops
|
||||||
* through our server) with a short-lived token minted by /api/stt/token
|
* through our server) with a short-lived token minted by /api/stt/token
|
||||||
* (the real key stays server-side), and streams frames while it listens.
|
* (the real key stays server-side), and streams frames while it listens.
|
||||||
* Finals append into the composer; interims ghost beside the meter.
|
* Finals append into the PRIMARY INPUT (see below).
|
||||||
*
|
*
|
||||||
* Ported patterns from the owner's pi `dictate` extension (his words:
|
* Ported patterns from the owner's pi `dictate` extension (his words:
|
||||||
* "very well tuned") — see ~/.pi/agent/extensions/dictate:
|
* "very well tuned") — see ~/.pi/agent/extensions/dictate:
|
||||||
* - generation counter: every start/stop increments it; all async
|
* - generation counter: every start/stop increments it; all async
|
||||||
* handlers no-op when stale, so a dying session's socket can never
|
* handlers no-op when stale, so a dying session's socket can never
|
||||||
* tear down the current one (a real bug he hit in the terminal).
|
* tear down the current one.
|
||||||
* - meter: latest-RMS-never-resets — the level ref keeps the last
|
* - meter: latest-RMS-never-resets — the level ref keeps the last
|
||||||
* observed value, so a tick with no fresh audio re-renders the last
|
* observed value, so a tick with no fresh audio re-renders the last
|
||||||
* level instead of dropping to silence (60ms tick samples the ref).
|
* level instead of dropping to silence (60ms tick samples the ref).
|
||||||
@@ -25,6 +25,21 @@ import { Loader2Icon, MicIcon } from "lucide-react";
|
|||||||
* conversational speech as mid-height bars without pegging.
|
* conversational speech as mid-height bars without pegging.
|
||||||
* - stop: mic dies FIRST (no more audio enqueues), then CloseStream,
|
* - stop: mic dies FIRST (no more audio enqueues), then CloseStream,
|
||||||
* then a 3s safety net in case Deepgram never closes the socket.
|
* then a 3s safety net in case Deepgram never closes the socket.
|
||||||
|
* - interim_results=false, endpointing=300: his live-tuned params — no
|
||||||
|
* partial hypotheses, finals commit ~300ms after a pause (owner
|
||||||
|
* round 1: live words are distracting and shift the meter; the
|
||||||
|
* meter's width is FIXED while recording).
|
||||||
|
*
|
||||||
|
* Owner round 1 (2026-09-17, first live mic test):
|
||||||
|
* - ONE primary transcription input at all times: the pending
|
||||||
|
* quiz/ask question's field while one waits (text mode → the answer
|
||||||
|
* field, select modes → "Your thinking"), else the page's composer.
|
||||||
|
* Implemented as a module-level registry the question form claims on
|
||||||
|
* mount — see useDictationTarget below.
|
||||||
|
* - Click → meter IMMEDIATELY (optimistic): the meter strip renders
|
||||||
|
* from the click, connection + TCC happen behind it. The token mint
|
||||||
|
* and the mic acquisition run CONCURRENTLY (Promise.all) to shrink
|
||||||
|
* the real gap.
|
||||||
*
|
*
|
||||||
* Meter rendering lives in <DictationBar> (own 60ms timer over the level
|
* Meter rendering lives in <DictationBar> (own 60ms timer over the level
|
||||||
* ref) so the 16-renders/sec never touch the page component — only the
|
* ref) so the 16-renders/sec never touch the page component — only the
|
||||||
@@ -32,12 +47,9 @@ import { Loader2Icon, MicIcon } from "lucide-react";
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
/*
|
/*
|
||||||
* Deepgram streaming params (tuned in the owner's real-use dictate
|
* Deepgram streaming params (the owner's live-tuned dictate config): nova-3,
|
||||||
* extension + the research pass): nova-3, his endpointing=300 (faster
|
* endpointing=300, no interims. Silence streams too (the worklet never
|
||||||
* finals after a pause), utterance_end_ms=1000 as the noise-robust
|
* pauses), so the socket never idles — no KeepAlive needed.
|
||||||
* backstop, interim_results for the live ghost. Silence streams too
|
|
||||||
* (the worklet never pauses), so the 10s idle timeout can't bite — no
|
|
||||||
* KeepAlive needed.
|
|
||||||
*/
|
*/
|
||||||
const DG_URL =
|
const DG_URL =
|
||||||
"wss://api.deepgram.com/v1/listen" +
|
"wss://api.deepgram.com/v1/listen" +
|
||||||
@@ -45,11 +57,9 @@ const DG_URL =
|
|||||||
"&encoding=linear16" +
|
"&encoding=linear16" +
|
||||||
"&sample_rate=16000" +
|
"&sample_rate=16000" +
|
||||||
"&channels=1" +
|
"&channels=1" +
|
||||||
"&interim_results=true" +
|
|
||||||
"&smart_format=true" +
|
"&smart_format=true" +
|
||||||
"&punctuate=true" +
|
"&punctuate=true" +
|
||||||
"&endpointing=300" +
|
"&endpointing=300";
|
||||||
"&utterance_end_ms=1000";
|
|
||||||
|
|
||||||
/** Meter tuning — the owner's tested values, see the file header. */
|
/** Meter tuning — the owner's tested values, see the file header. */
|
||||||
const METER_FLOOR_DB = -50;
|
const METER_FLOOR_DB = -50;
|
||||||
@@ -66,33 +76,64 @@ export function rmsToMeter(rms: number): number {
|
|||||||
return Math.max(0, Math.min(1, t));
|
return Math.max(0, Math.min(1, t));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The primary-input registry: at most ONE non-composer input can claim the
|
||||||
|
* dictation target while mounted — the pending question's answer/thinking
|
||||||
|
* field (question-dock.tsx). The page's `append` (the composer) is the
|
||||||
|
* fallback when nobody claims. Module-level by design: the claimant is a
|
||||||
|
* sibling subtree far from the hook's owner, and there is exactly one
|
||||||
|
* window, one pending question, one target at a time.
|
||||||
|
*/
|
||||||
|
let questionTarget: ((text: string) => void) | null = null;
|
||||||
|
|
||||||
|
/** Join a dictated final onto the current text (shared separator rule). */
|
||||||
|
export function joinDictation(current: string, final: string): string {
|
||||||
|
return current ? current.replace(/\s+$/, "") + " " + final : final;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Claim the dictation target while mounted (and `claimed` is true). The
|
||||||
|
* unmount guard is identity-based so dev-mode double-invoked effects
|
||||||
|
* can't clear another claimant.
|
||||||
|
*/
|
||||||
|
export function useDictationTarget(append: (text: string) => void, claimed: boolean) {
|
||||||
|
useEffect(() => {
|
||||||
|
if (!claimed) return;
|
||||||
|
questionTarget = append;
|
||||||
|
return () => {
|
||||||
|
if (questionTarget === append) questionTarget = null;
|
||||||
|
};
|
||||||
|
}, [append, claimed]);
|
||||||
|
}
|
||||||
|
|
||||||
export type DictationPhase = "idle" | "connecting" | "recording" | "stopping";
|
export type DictationPhase = "idle" | "connecting" | "recording" | "stopping";
|
||||||
|
|
||||||
export type Dictation = {
|
export type Dictation = {
|
||||||
phase: DictationPhase;
|
phase: DictationPhase;
|
||||||
/** Terminal-state explanation surfaced next to the mic (cleared on the next start). */
|
/** Terminal-state explanation, shown inside the wide button (cleared on the next start). */
|
||||||
error: string | null;
|
error: string | null;
|
||||||
/** Live in-flight partial (the ghost); cleared when the final commits. */
|
|
||||||
interim: string;
|
|
||||||
/** Latest observed RMS, 0..1 — sampled by the meter's own tick, never reset. */
|
/** Latest observed RMS, 0..1 — sampled by the meter's own tick, never reset. */
|
||||||
levelRef: { current: number };
|
levelRef: { current: number };
|
||||||
/** Recording or finalizing — true from start to stop completion. */
|
/** Not idle — from the click until the stop completes (the optimistic window). */
|
||||||
active: boolean;
|
active: boolean;
|
||||||
toggle: () => void;
|
toggle: () => void;
|
||||||
stop: () => void;
|
stop: () => void;
|
||||||
};
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* One dictation instance per composer-bearing page. `append` receives each
|
* One dictation instance per composer-bearing page. `append` is the
|
||||||
* committed utterance (final) — the page joins it into its text state
|
* composer fallback; the mounted question form's claim wins while a
|
||||||
* (separator handling is the page's: it owns the current text).
|
* question waits.
|
||||||
*/
|
*/
|
||||||
export function useDictation({ append }: { append: (text: string) => void }): Dictation {
|
export function useDictation({ append }: { append: (text: string) => void }): Dictation {
|
||||||
const [phase, setPhase] = useState<DictationPhase>("idle");
|
const [phase, setPhase] = useState<DictationPhase>("idle");
|
||||||
const [error, setError] = useState<string | null>(null);
|
const [error, setError] = useState<string | null>(null);
|
||||||
const [interim, setInterim] = useState("");
|
|
||||||
/** Latest RMS from the worklet — the meter samples this (dictate pattern). */
|
/** Latest RMS from the worklet — the meter samples this (dictate pattern). */
|
||||||
const levelRef = useRef(0);
|
const levelRef = useRef(0);
|
||||||
|
/** The page's fallback append — kept fresh per render so a mid-recording
|
||||||
|
* retarget (composer↔question) is honored on the very next final. */
|
||||||
|
const appendRef = useRef(append);
|
||||||
|
appendRef.current = append;
|
||||||
|
|
||||||
// Live machinery (non-render values). AudioContext/MediaStream are
|
// Live machinery (non-render values). AudioContext/MediaStream are
|
||||||
// closed on stop; the socket is recreated per run.
|
// closed on stop; the socket is recreated per run.
|
||||||
@@ -121,12 +162,10 @@ export function useDictation({ append }: { append: (text: string) => void }): Di
|
|||||||
ws.close();
|
ws.close();
|
||||||
} catch {}
|
} catch {}
|
||||||
}
|
}
|
||||||
setInterim("");
|
|
||||||
setPhase("idle");
|
setPhase("idle");
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
/** Mint a short-lived token — the run's first step, BEFORE the mic: a
|
/** Mint a short-lived token (the route holds the real key server-side). */
|
||||||
* keyless app fails fast without ever triggering the macOS TCC prompt. */
|
|
||||||
const mintToken = useCallback(async () => {
|
const mintToken = useCallback(async () => {
|
||||||
const res = await fetch("/api/stt/token");
|
const res = await fetch("/api/stt/token");
|
||||||
const data = (await res.json()) as { token?: string; message?: string };
|
const data = (await res.json()) as { token?: string; message?: string };
|
||||||
@@ -157,16 +196,12 @@ export function useDictation({ append }: { append: (text: string) => void }): Di
|
|||||||
}
|
}
|
||||||
if (msg.type === "Results") {
|
if (msg.type === "Results") {
|
||||||
const text: string | undefined = msg.channel?.alternatives?.[0]?.transcript;
|
const text: string | undefined = msg.channel?.alternatives?.[0]?.transcript;
|
||||||
if (msg.is_final) {
|
// Only finals stream (interim_results=false): each Results
|
||||||
// The committed utterance: hand to the page, clear the ghost.
|
// message is a committed utterance — route it to the
|
||||||
if (text) append(text);
|
// primary input, question claim first, composer fallback.
|
||||||
setInterim("");
|
if (msg.is_final && text) {
|
||||||
} else if (text) {
|
(questionTarget ?? appendRef.current)(text);
|
||||||
setInterim(text);
|
|
||||||
}
|
}
|
||||||
} else if (msg.type === "UtteranceEnd") {
|
|
||||||
// Backstop: the endpointing VAD missed, the word-gap one didn't.
|
|
||||||
setInterim("");
|
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
ws.addEventListener("close", (ev) => {
|
ws.addEventListener("close", (ev) => {
|
||||||
@@ -197,7 +232,7 @@ export function useDictation({ append }: { append: (text: string) => void }): Di
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
},
|
},
|
||||||
[append, cleanup, mintToken],
|
[cleanup, mintToken],
|
||||||
);
|
);
|
||||||
|
|
||||||
const start = useCallback(async () => {
|
const start = useCallback(async () => {
|
||||||
@@ -206,25 +241,17 @@ export function useDictation({ append }: { append: (text: string) => void }): Di
|
|||||||
reconnectsRef.current = 0;
|
reconnectsRef.current = 0;
|
||||||
const gen = ++genRef.current;
|
const gen = ++genRef.current;
|
||||||
|
|
||||||
// 1. Token first — fail fast, no TCC prompt for a keyless app.
|
// Token + mic CONCURRENTLY (owner round 1: the sequential wait was the
|
||||||
|
// felt delay) — the optimistic meter already covers the gap. 127.0.0.1
|
||||||
|
// is a secure context, so getUserMedia works without TLS; macOS TCC
|
||||||
|
// prompts on first use.
|
||||||
|
const mic = navigator.mediaDevices.getUserMedia({
|
||||||
|
audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true, autoGainControl: true },
|
||||||
|
});
|
||||||
|
let token: string;
|
||||||
try {
|
try {
|
||||||
const token = await mintToken();
|
const [t, stream] = await Promise.all([mintToken(), mic]);
|
||||||
if (gen !== genRef.current) return;
|
token = t;
|
||||||
openSocket(gen, token);
|
|
||||||
} catch (e) {
|
|
||||||
if (gen !== genRef.current) return;
|
|
||||||
setError(e instanceof Error ? e.message : "token fetch failed");
|
|
||||||
setPhase("idle");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
// 2. Mic + capture graph. 127.0.0.1 is a secure context —
|
|
||||||
// getUserMedia works without TLS (research); macOS TCC prompts
|
|
||||||
// on first use.
|
|
||||||
const stream = await navigator.mediaDevices.getUserMedia({
|
|
||||||
audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true, autoGainControl: true },
|
|
||||||
});
|
|
||||||
if (gen !== genRef.current) {
|
if (gen !== genRef.current) {
|
||||||
for (const track of stream.getTracks()) track.stop();
|
for (const track of stream.getTracks()) track.stop();
|
||||||
return;
|
return;
|
||||||
@@ -259,7 +286,10 @@ export function useDictation({ append }: { append: (text: string) => void }): Di
|
|||||||
: "mic unavailable";
|
: "mic unavailable";
|
||||||
setError(msg);
|
setError(msg);
|
||||||
cleanup();
|
cleanup();
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
|
if (gen !== genRef.current) return;
|
||||||
|
openSocket(gen, token);
|
||||||
}, [openSocket, cleanup, mintToken]);
|
}, [openSocket, cleanup, mintToken]);
|
||||||
|
|
||||||
const stop = useCallback(() => {
|
const stop = useCallback(() => {
|
||||||
@@ -297,25 +327,30 @@ export function useDictation({ append }: { append: (text: string) => void }): Di
|
|||||||
// "stopping" presses are ignored — the flush is in flight.
|
// "stopping" presses are ignored — the flush is in flight.
|
||||||
}, [phase, start, stop]);
|
}, [phase, start, stop]);
|
||||||
|
|
||||||
return { phase, error, interim, levelRef, active: phase !== "idle", toggle, stop };
|
return { phase, error, levelRef, active: phase !== "idle", toggle, stop };
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The stats-bar dictation slot (owner placement, 2026-09-17): the mic
|
* The stats-bar dictation slot (owner placement, round 1): the row's middle
|
||||||
* button rides the middle of the stats row — idle it is just the quiet mic
|
* is one strip that is always the full free width —
|
||||||
* affordance; recording, the meter unfurls to fill the free middle space
|
*
|
||||||
* (width follows what the row has), with the live interim ghost at its
|
* idle a wide, quiet "Dictate" button (mic + label, grayed out — the
|
||||||
* flank. Red dot + red mic while live; spinner while Deepgram flushes.
|
* whole middle is the affordance, per the owner).
|
||||||
|
* active optimistic from the CLICK: red stop pill + the fixed-width
|
||||||
|
* meter (no interim text ever shifts it). Spinner while
|
||||||
|
* Deepgram flushes the last final.
|
||||||
|
* error the wide button again, label swapped for the failure reason
|
||||||
|
* (full text in the tooltip), cleared on the next click.
|
||||||
*/
|
*/
|
||||||
export function DictationBar({ dictation }: { dictation: Dictation }) {
|
export function DictationBar({ dictation }: { dictation: Dictation }) {
|
||||||
const { phase, error, interim, levelRef, toggle, active } = dictation;
|
const { phase, error, levelRef, toggle, active } = dictation;
|
||||||
const [levels, setLevels] = useState<number[]>(() => new Array(METER_CELLS).fill(0));
|
const [levels, setLevels] = useState<number[]>(() => new Array(METER_CELLS).fill(0));
|
||||||
const recording = phase === "recording" || phase === "stopping";
|
const recording = phase === "recording" || phase === "stopping";
|
||||||
|
|
||||||
// The meter's own heartbeat — 60ms samples of the level ref into a
|
// The meter's own heartbeat — 60ms samples of the level ref into a
|
||||||
// rolling ring. Only THIS strip re-renders; the page never does.
|
// rolling ring. Only THIS strip re-renders; the page never does.
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!recording) return;
|
if (!active) return;
|
||||||
setLevels(new Array(METER_CELLS).fill(0));
|
setLevels(new Array(METER_CELLS).fill(0));
|
||||||
const id = setInterval(() => {
|
const id = setInterval(() => {
|
||||||
setLevels((ring) => {
|
setLevels((ring) => {
|
||||||
@@ -325,52 +360,54 @@ export function DictationBar({ dictation }: { dictation: Dictation }) {
|
|||||||
});
|
});
|
||||||
}, METER_TICK_MS);
|
}, METER_TICK_MS);
|
||||||
return () => clearInterval(id);
|
return () => clearInterval(id);
|
||||||
}, [recording, levelRef]);
|
}, [active, levelRef]);
|
||||||
|
|
||||||
|
if (!active) {
|
||||||
|
return (
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={toggle}
|
||||||
|
title={error ?? "Dictate into the composer"}
|
||||||
|
aria-label={error ? `Dictation failed: ${error}` : "Start dictation"}
|
||||||
|
data-testid="dictation-toggle"
|
||||||
|
className="flex h-7 w-full items-center justify-center gap-1.5 rounded-md border border-border/50 text-xs text-muted-foreground/70 transition-colors hover:border-border hover:text-muted-foreground"
|
||||||
|
>
|
||||||
|
<MicIcon className="size-3.5" />
|
||||||
|
<span className="max-w-full truncate">{error ?? "Dictate"}</span>
|
||||||
|
</button>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="flex min-w-0 items-center gap-2" data-testid="dictation-bar">
|
<div className="flex w-full min-w-0 items-center gap-2" data-testid="dictation-bar">
|
||||||
<button
|
<button
|
||||||
type="button"
|
type="button"
|
||||||
onClick={toggle}
|
onClick={toggle}
|
||||||
disabled={phase === "stopping"}
|
disabled={phase === "stopping"}
|
||||||
title={active ? "Stop dictation (Esc)" : "Dictate"}
|
title="Stop dictation (Esc)"
|
||||||
aria-label={active ? "Stop dictation" : "Start dictation"}
|
aria-label="Stop dictation"
|
||||||
data-testid="dictation-toggle"
|
data-testid="dictation-toggle"
|
||||||
className="flex size-6 shrink-0 items-center justify-center rounded-md transition-colors hover:bg-accent hover:text-accent-foreground dark:hover:bg-accent/50"
|
className="flex h-7 shrink-0 items-center gap-1.5 rounded-md bg-destructive/10 px-2 text-xs font-medium text-destructive transition-colors hover:bg-destructive/20"
|
||||||
>
|
>
|
||||||
{phase === "stopping" ? (
|
{phase === "stopping" ? (
|
||||||
<Loader2Icon className="size-3.5 animate-spin text-muted-foreground" />
|
<Loader2Icon className="size-3.5 animate-spin" />
|
||||||
) : (
|
) : (
|
||||||
<MicIcon className={active ? "size-3.5 text-destructive" : "size-3.5 text-muted-foreground"} />
|
<MicIcon className="size-3.5" />
|
||||||
)}
|
)}
|
||||||
|
Stop
|
||||||
</button>
|
</button>
|
||||||
{error && (
|
<span className="size-1.5 shrink-0 animate-pulse rounded-full bg-destructive" aria-hidden />
|
||||||
<span className="max-w-64 truncate text-destructive" title={error}>
|
{/* Fixed width while recording — nothing ever appends text beside it
|
||||||
{error}
|
(no interims, owner round 1). Bars flex to the row's free space. */}
|
||||||
</span>
|
<div className="flex h-4 min-w-0 flex-1 items-center gap-[2px]" data-testid="dictation-meter">
|
||||||
)}
|
{levels.map((v, i) => (
|
||||||
{!error && recording && (
|
<span
|
||||||
<>
|
key={i}
|
||||||
<span className="size-1.5 shrink-0 animate-pulse rounded-full bg-destructive" aria-hidden />
|
className="min-w-[2px] flex-1 rounded-full bg-foreground/40"
|
||||||
{/* Bars flex to the row's free width — the owner's "fill up
|
style={{ height: `${2 + Math.round(v * 14)}px` }}
|
||||||
the whole middle section" ask: pitch is whatever space
|
/>
|
||||||
exists between session data and the model control. */}
|
))}
|
||||||
<div className="flex h-4 min-w-0 flex-1 items-center gap-[2px]" data-testid="dictation-meter">
|
</div>
|
||||||
{levels.map((v, i) => (
|
|
||||||
<span
|
|
||||||
key={i}
|
|
||||||
className="min-w-[2px] flex-1 rounded-full bg-foreground/40"
|
|
||||||
style={{ height: `${2 + Math.round(v * 14)}px` }}
|
|
||||||
/>
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
{interim && (
|
|
||||||
<span className="max-w-[45%] truncate text-muted-foreground/80" data-testid="dictation-interim">
|
|
||||||
{interim}
|
|
||||||
</span>
|
|
||||||
)}
|
|
||||||
</>
|
|
||||||
)}
|
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
"use client";
|
"use client";
|
||||||
|
|
||||||
import { useMemo, useState } from "react";
|
import { useCallback, useMemo, useState } from "react";
|
||||||
import { ArrowDownIcon, ChevronRightIcon, PlayIcon } from "lucide-react";
|
import { ArrowDownIcon, ChevronRightIcon, PlayIcon } from "lucide-react";
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
|
import { joinDictation, useDictationTarget } from "@/components/dictation";
|
||||||
import { MessageResponse } from "@/components/ai-elements/message";
|
import { MessageResponse } from "@/components/ai-elements/message";
|
||||||
import { cn } from "@/lib/utils";
|
import { cn } from "@/lib/utils";
|
||||||
import {
|
import {
|
||||||
@@ -63,6 +64,9 @@ export interface QuestionDockProps {
|
|||||||
reveal?: boolean;
|
reveal?: boolean;
|
||||||
onExpand: () => void;
|
onExpand: () => void;
|
||||||
onSubmit: (output: QuestionAnswerData) => void;
|
onSubmit: (output: QuestionAnswerData) => void;
|
||||||
|
/** Escape in a question field stops dictation (STT wave) — threaded
|
||||||
|
* from the page that owns the dictation hook. */
|
||||||
|
onEscape?: () => void;
|
||||||
/** Session phase at the question — scales the thinking field: probing
|
/** Session phase at the question — scales the thinking field: probing
|
||||||
* stays quick (one row), teaching invites dumping (three rows). */
|
* stays quick (one row), teaching invites dumping (three rows). */
|
||||||
phase?: Phase3;
|
phase?: Phase3;
|
||||||
@@ -76,6 +80,7 @@ export function QuestionDock({
|
|||||||
reveal = false,
|
reveal = false,
|
||||||
onExpand,
|
onExpand,
|
||||||
onSubmit,
|
onSubmit,
|
||||||
|
onEscape,
|
||||||
phase = "probe",
|
phase = "probe",
|
||||||
}: QuestionDockProps) {
|
}: QuestionDockProps) {
|
||||||
// Parse, don't cast (src/core/tool-parts.ts): the model authored this
|
// Parse, don't cast (src/core/tool-parts.ts): the model authored this
|
||||||
@@ -109,6 +114,7 @@ export function QuestionDock({
|
|||||||
input={parsed}
|
input={parsed}
|
||||||
phase={phase}
|
phase={phase}
|
||||||
onSubmit={onSubmit}
|
onSubmit={onSubmit}
|
||||||
|
onEscape={onEscape}
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
</>
|
</>
|
||||||
@@ -189,6 +195,7 @@ function QuestionDockForm({
|
|||||||
input,
|
input,
|
||||||
phase = "probe",
|
phase = "probe",
|
||||||
onSubmit,
|
onSubmit,
|
||||||
|
onEscape,
|
||||||
}: Omit<QuestionDockProps, "expanded" | "onExpand">) {
|
}: Omit<QuestionDockProps, "expanded" | "onExpand">) {
|
||||||
const kind = toolName === "quiz" ? ("quiz" as const) : ("ask" as const);
|
const kind = toolName === "quiz" ? ("quiz" as const) : ("ask" as const);
|
||||||
|
|
||||||
@@ -243,6 +250,22 @@ function QuestionDockForm({
|
|||||||
const [text, setText] = useState("");
|
const [text, setText] = useState("");
|
||||||
const selectMode: SelectMode = qd.mode === "multi-select" ? "multi-select" : "single-select";
|
const selectMode: SelectMode = qd.mode === "multi-select" ? "multi-select" : "single-select";
|
||||||
|
|
||||||
|
// The PRIMARY dictation input (owner, STT round 1): while a question
|
||||||
|
// waits, dictated finals land HERE, never in the hidden composer — text
|
||||||
|
// mode owns "Your answer", select modes own "Your thinking". The claim
|
||||||
|
// is effect-scoped, so the question unmounting releases it and
|
||||||
|
// dictation falls back to the composer automatically.
|
||||||
|
useDictationTarget(
|
||||||
|
useCallback(
|
||||||
|
(t: string) => {
|
||||||
|
if (qd.mode === "text") setText((v) => joinDictation(v, t));
|
||||||
|
else setThinking((v) => joinDictation(v, t));
|
||||||
|
},
|
||||||
|
[qd.mode],
|
||||||
|
),
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
|
||||||
const toggle = (idx: number) => setSel(toggleOption(selectMode, sel, idx));
|
const toggle = (idx: number) => setSel(toggleOption(selectMode, sel, idx));
|
||||||
|
|
||||||
const canSubmit =
|
const canSubmit =
|
||||||
@@ -420,6 +443,12 @@ function QuestionDockForm({
|
|||||||
value={text}
|
value={text}
|
||||||
onChange={(e) => setText(e.target.value)}
|
onChange={(e) => setText(e.target.value)}
|
||||||
onKeyDown={(e) => {
|
onKeyDown={(e) => {
|
||||||
|
// Esc ends dictation (the dictate decisions: click-again or Esc).
|
||||||
|
if (e.key === "Escape" && onEscape) {
|
||||||
|
e.preventDefault();
|
||||||
|
onEscape();
|
||||||
|
return;
|
||||||
|
}
|
||||||
// Chat convention: Enter submits, Shift+Enter is a newline;
|
// Chat convention: Enter submits, Shift+Enter is a newline;
|
||||||
// ⌘/Ctrl+Enter also submits (universal across the app).
|
// ⌘/Ctrl+Enter also submits (universal across the app).
|
||||||
if (e.key === "Enter" && (e.metaKey || e.ctrlKey || !e.shiftKey)) {
|
if (e.key === "Enter" && (e.metaKey || e.ctrlKey || !e.shiftKey)) {
|
||||||
@@ -436,6 +465,12 @@ function QuestionDockForm({
|
|||||||
value={thinking}
|
value={thinking}
|
||||||
onChange={(e) => setThinking(e.target.value)}
|
onChange={(e) => setThinking(e.target.value)}
|
||||||
onKeyDown={(e) => {
|
onKeyDown={(e) => {
|
||||||
|
// Esc ends dictation (the dictate decisions: click-again or Esc).
|
||||||
|
if (e.key === "Escape" && onEscape) {
|
||||||
|
e.preventDefault();
|
||||||
|
onEscape();
|
||||||
|
return;
|
||||||
|
}
|
||||||
// ⌘↵ on macOS, Ctrl↵ elsewhere — the Gmail convention; plain
|
// ⌘↵ on macOS, Ctrl↵ elsewhere — the Gmail convention; plain
|
||||||
// Enter stays a newline (this field is for dumping).
|
// Enter stays a newline (this field is for dumping).
|
||||||
if (e.key === "Enter" && (e.metaKey || e.ctrlKey)) {
|
if (e.key === "Enter" && (e.metaKey || e.ctrlKey)) {
|
||||||
|
|||||||
@@ -57,5 +57,13 @@ against our codebase. Decisions at the bottom.
|
|||||||
- **Activation: toggle mic** — click to start a dictation run, click again (or Esc) to end; no push-to-talk before the shortcuts milestone.
|
- **Activation: toggle mic** — click to start a dictation run, click again (or Esc) to end; no push-to-talk before the shortcuts milestone.
|
||||||
- **Wave scope: cloud first, local second** — capture component + Deepgram path ship and get live-smoked as one wave; whisper.cpp follows as a second wave.
|
- **Wave scope: cloud first, local second** — capture component + Deepgram path ship and get live-smoked as one wave; whisper.cpp follows as a second wave.
|
||||||
- Renderer never holds the real key (server mints short-lived Deepgram tokens); audio never loops through our server (renderer → wss direct).
|
- Renderer never holds the real key (server mints short-lived Deepgram tokens); audio never loops through our server (renderer → wss direct).
|
||||||
- Interim text ghosts in the composer; finals commit. Engine picker = small popover off the mic button (localStorage for now; settings milestone absorbs it).
|
- Engine picker = small popover off the mic button (localStorage for now; settings milestone absorbs it).
|
||||||
- Bands: 1) capture + mic states (no engine), 2) cloud end-to-end, 3) local end-to-end, 4) smoke + docs.
|
- Bands: 1) capture + mic states (no engine), 2) cloud end-to-end, 3) local end-to-end, 4) smoke + docs.
|
||||||
|
|
||||||
|
## Round-1 refinements (owner, 2026-09-17 — first live mic test)
|
||||||
|
|
||||||
|
- **No interims.** Live words beside the meter are distracting AND shift the meter around — the meter's width stays fixed while recording. Params now match the pi dictate extension exactly: `interim_results` omitted (false), `endpointing=300`, no `utterance_end_ms`.
|
||||||
|
- **One primary transcription input at all times**: while a quiz/ask question waits, finals land in ITS field (text mode → "Your answer", select modes → "Your thinking") — never the hidden composer. Implemented as a module-level claim registry (`useDictationTarget`); the composer is the fallback when nothing claims.
|
||||||
|
- **Optimistic meter**: the click enters the meter view immediately (Stop pill + meter within one frame); the token mint and mic acquisition run concurrently (Promise.all) behind it. Esc or click-again stops.
|
||||||
|
- **Wide idle button**: the stats-bar middle is one full-width affordance — a quiet bordered "Dictate" button when idle (error text replaces the label on failure), the Stop pill + fixed meter while live.
|
||||||
|
- Final-appending keeps the dictate-flush separator rule (`joinDictation`).
|
||||||
|
|||||||
Reference in New Issue
Block a user