Commit Graph
20 Commits
Author SHA1 Message Date
sorokin.oleg.edu 9727516926 Merge upstream v0.4.0 into Gitea fork 2026-10-31 13:45:06 +03:00
amosblomqvist c4e934c9dc feat: attach desktop file paths as editable learner chips 2026-10-28 18:54:57 +02:00
amosblomqvist fbafc5e53a Finish Deepgram language settings with shared searchable picker 2026-10-27 21:51:09 +02:00
sorokin.oleg.edu ecbd2b0bb9 feat: add OpenRouter dictation and speech recognition settings 2026-10-26 23:42:44 +03:00
sorokin.oleg.edu ce28d95bcf feat: add preferred STT languages and fix model settings layout 2026-10-26 02:14:27 +03:00
sorokin.oleg.edu bdd7d5e866 feat: add multilingual dictation and simplify chat STT picker 2026-10-26 00:58:25 +03:00
sorokin.oleg.edu a50ca4b867 feat: add Russian localization and independent STT languages
CI / Typecheck, tests, lint (push) Canceled after 0s
CI / Standalone (mac-arm64) (push) Canceled after 0s
CI / Standalone (linux-x64) (push) Canceled after 0s
CI / Standalone (windows-x64) (push) Canceled after 0s
2026-10-25 17:18:44 +03:00
Bartosz Chronowki 3bb28e5f29 feat(dictation): add persistent language selection in General settings 2026-10-25 14:33:49 +01:00
amosblomqvist 8460e96d6c dictation feel: finish-first submits, composer focus on run end
Two owner rulings, 2026-10-01 ('nothing gets lost' round):

1. SUBMIT FINISHES DICTATION FIRST. Every submit path — the dock's
   answer (Enter/Cmd+Enter/button) and the composer's send — stops any
   live run and lets Deepgram's last finals land in the target field
   BEFORE building the payload. Dictation.finish() resolves at cleanup,
   post-flush; a module finisher registry (same pattern as the question
   target) hands it to the dock without a prop bridge. Field values are
   read from REF MIRRORS (thinkingTextRef/answerTextRef, the composer's
   textRef): the flush lands via setState after an await, invisible to
   the closure. The quiz flow the owner described now holds: dictate
   reasoning → pick answer → pick confidence → Enter → dictation ends,
   reasoning saved into the thinking field, THEN the answer submits.
   The composer stays sendable while a run is live (its spoken text is
   deferred in the run's buffer) so Enter can trigger the finish-first
   send at all; nothing said → the send no-ops.

2. RUN END FOCUSES THE BARE COMPOSER. The falling edge of
   dictation.active focuses the composer's textarea — only when the
   composer is mounted: a pending question unmounts it (ref null, no
   focus), so the thinking field is deliberately left alone. Composer
   gains an optional inputRef for the page's focus handle.
2026-10-02 01:37:18 +03:00
amosblomqvist 07d2c84461 dictation start-latency mitigations A+B+C+D (owner 2026-09-27): queue-don't-drop (frames before socket-open buffer capped and flush in order), parallel mic-gated handshake, app-lifetime AudioContext + memoized worklet module, warm-token cache (mount + post-run pre-mint, TTL 3min, reconnect mints fresh); optimistic meter stays per owner 2026-09-27 04:36:00 +03:00
amosblomqvist b19e7e7eae Production tooltips: styled tooltips everywhere, shortcuts surfaced, dev-speak culled
Owner-approved full pass (tooltip wave, 2026-09-25). Mechanism: icon/pill
controls ride the styled Radix tooltip via a new Hint composition
(components/ui/hint.tsx) — the shared primitive adopted as-is, no
restyling; the shortcut appends as an em-dash suffix off the LIVE registry
(a rebind in Settings → Keyboard changes the suffix): "Dictate — ⌘D".
Surface-owned keys (Esc closes, Enter submits) ride a raw binding prop.
Native title= stays only for truncation-reveals (dock spans, the rename
field) and menu-interior abbreviations (MenuShell untouched). One
em-dash pattern; per-state truth on the send square (busy = "Stop the
reply — Esc", idle = "Send — ↵"), Enter on the plan verdict's accept,
"Stop dictation (Esc)" kept verbatim per the owner's flag.

Copy: the dev-speak cull — no pipeline/prober/tier/wave vocabulary
anywhere a user reads. New pins in tests/tooltip-mechanism.test.tsx
(hintText composition, asChild no-wrapper-DOM, the auto-update hint
constant); dock truncation titles pinned native; graph-chrome + settings
pins updated to the new copy.

Live-verified on :3001 — 12.6ms hover appearance (no native 1s lag);
glyphs render on every suffixed surface (⌘D, ⌘K, ⌘⇧F, ⌘,, ⌘B, F2, ↵,
Esc); per-page probes confirm Settings entry, Home logo, session search,
Start session, Close sidebar, model + thinking pills, stats summary,
rename title, plan chip, coverage ring, send square, cost button, node
search (⌘F), graph gear, settings close (Close — Esc), keyboard section
"Fixed — part of this screen", auto-update hint, light-mode "Coming
soon", the three slot rosters, graph-tools hint. Esc still closes the
settings dialog.
2026-09-25 01:52:20 +03:00
amosblomqvist 1f777f770c Zero-keys wave: provider-key gate renders honestly everywhere
Over Platform's 66dbea9 key-presence seam, the five owner-locked tasks:

1. Zero-keys pill — components/setup-cta.tsx: pure hasProviderKey
   (classifyKeys reuse — tool keys like Deepgram never count), useZeroKeys
   (GET /api/keys, re-read on the new lh:settings-saved 'keys' signal),
   ZeroKeysPill (testid setup-cta-keys) taking PRECEDENCE over the models
   CTA on chat/home/new-session. SettingsSection gains 'keys'; keys-section
   dispatches after save AND delete.

2. Setup stage 1 is self-contained: provider picker (MenuShell +
   ProviderMenuList) + key paste + Save via PUT /api/keys/:id — the dialog
   STAYS MOUNTED; its own save dispatches the keys signal, the fetch effect
   re-runs on it, and the stage flips to the pickers on its own.
   onOpenKeys + setup-open-keys die. SetupKeysBody exported for SSR pins.

3. parseProviderKeyRefusal (chat.tsx, envelope-verbatim twin of
   parseModelsRefusal) joins the revert-draft-and-open-setup interception;
   activity-dock reads 'needs a provider key'; dispatch-card shows
   'Add a key' — both deep-link setup, which lands on stage 1 by itself.

4. Dictation trio: the token route's only 503 (missing Deepgram key)
   becomes the client-authored DICTATION_KEYLESS short line; the errored
   idle click clears the error and deep-links Settings -> Keys ->
   DEEPGRAM_API_KEY via the new open(section, row) seam (dialog landOnRow
   -> SettingsDialogBody initialRow -> KeysSection openRow follow-effect).

Live-verified on :3001 under LH_IGNORE_ENV_KEYS=1 (the fresh-user switch):
pill on all three surfaces + precedence over the CTA, stage-1 save ->
auto-advance with dialog mounted, real 503 send refusal reverting the
draft and opening setup, dictation 503 -> short line -> Settings lands on
the unfolded Deepgram row with the error cleared, keys-section save and
delete both flipping the pill live, and a fabricated refused research job
rendering the 'Add a key' dispatch card. 926/926, tsc/lint/build green.
2026-09-24 23:20:59 +03:00
amosblomqvist 7a17230f26 Wave 7 close-out: the graphTools surface, wrapped cluster labels, the orphan sweep
Part 1 — the surface:
- Settings ▸ Agent ▸ Tools gains the prober GRAPH-TOOLS row as its FIRST
  row (owner ruling: in the Tools section, not the tool-toggles list —
  its subject is the map, its mechanism is agent capability). Writes
  agent.graphTools {enabled} (absent = on, the toggle discipline) via
  the section PUT; composeGraphTools pure seam. Its own key in the agent
  contract — never in the tools map.
- The row carries the search-row hint discipline: "Connects prior
  sessions' knowledge into new ones — the prober reads the knowledge
  map" (owner wording). The honest interplay composes both: when
  graph.enabled is off the row dims + disables with a "Graph is
  disabled" hint (the effective gate itself is server-side, per-turn
  fresh). AgentSection reads the graph section (record.graph, passed by
  the dialog) — read-only.
- Cluster labels wrap: wave 7 made labels session names and FILE PATHS,
  which the old single-line fillText could not bound. wrapRegionLabel
  (pure, exported): word-wrap first, char-split long paths (the
  node-label discipline), capped at 3 lines, overflow folded into an
  ellipsized last line (owner: wrap, cap 2–3, ellipsize). Pre-wrapped
  once per geometry build at the SCREEN-fixed size (viewer-relative
  labels — the wrap never goes stale mid-camera-move), drawn centered
  line-by-line at the centroid. Deleted-session clusters keep their
  hull-only honest silence (label null → no lines).
- The orphan sweep: StatsBar's leftSlot prop is dead (its one consumer,
  the knowledge-context slider, died in c690181) — removed, and pre-chat
  views now render NOTHING on the left (owner ruling this round: a
  "— · —" summary over an empty popover is pure noise) — the dictation
  middle just widens. Stale knowledge-context comments swept in
  chat.tsx (adoption-window + activity-nudge docs), activity-dock.tsx
  (useActivity ground rules), dictation.tsx (×2 adaptive-meter docs),
  home.tsx (pre-chat dock comment).

Part 2 — Teacher's c690181 review: clean. Imports, KIND_ICON/KIND_LABEL
entries, render branches and event seams all excised whole; the probe-
handoff branch re-indented intact and merge-adopted; tests updated in
lockstep (merge-adopted, record-adapter, render-new-session retired).
Platform's 4b60e0f roster-string fixes (labeler dropped from both
SLOT_META blocks) reviewed — consistent across agent-section +
setup-dialog.

Gates: tsc clean, 854/854 (8 new: wrapRegionLabel cases, graphTools
SSR rows ×3; stats-bar + ladder fixtures updated to the new rulings),
lint 0, build ok. Live on :3001: graphTools row first-in-group with
hint, off→{enabled:false}→on round-trip, graph-off dimming + hint +
disabled toggle, pre-chat rows summary-free, chat view keeps its
summary, long-title cluster label wrapped without console errors
(visual read unavailable this session — pure wrap is unit-pinned).
2026-09-23 15:57:59 +03:00
amosblomqvist ff3700a2d0 Polish: session names take 900 weight serif-off; floating controls match graph chrome
- .font-title now requests font-weight: 900 — with the serif toggle OFF,
  the prose font is a VARIABLE font and was rendering titles at 400, a
  whisper next to CMU's bold-only face; 900 lands directly on Source
  Sans 3 and CMU's single 700 face snaps to nearest, unchanged. Same
  rendering with serif on, real weight with it off.
- The model + thinking pills match the graph view's floating top-right
  chrome (import notes / gear / coverage): border bg-card/85 shadow-lg
  backdrop-blur — the graph no longer bleeds through the pills.
- The dictate button gets the same blurring at lower opacity
  (bg-card/50 backdrop-blur), staying quieter than the model pills.

Gates: tsc clean, 866/866, lint 0, build ok. Live-verified: pills/dictate
computed styles byte-identical to the reference chrome (bg .85/.5 alpha +
blur(8px) + same shadow stack); title requests 900 in both serif states.
2026-09-23 01:46:15 +03:00
amosblomqvist 8b158972db feat: keyboard waves A/B — global shortcut registry, question row machine, Esc-cancel dictation
Wave A (owner key map): registry in src/core/keyboard.ts + KeyboardShortcuts
provider; Cmd+D dictation, Cmd+K model picker, Shift+Tab thinking cycle,
Cmd+B sidebar, Cmd+Shift+F sessions search, Alt+arrows session cycle,
Alt+Enter open, Cmd+G graph, Cmd+N new, F2 rename, ? cheat sheet. Tab
fully suppressed; Esc chain popover > dictation > abort (local stream).
Esc now CANCELS dictation (discard) — flush stays on Cmd+D/Stop. Owner
bug reports fixed: focus outline removed, Alt+Enter setState-in-render,
Cmd+F graph node search (surface-scoped lh:kbd-find).

Wave B: question dock row machine — arrows walk options + dont-know/Other
with wrap + ring highlight, Space picks (mirrors click semantics), Enter
submits when canSubmit; typing surfaces and in-dock keys keep native
behavior. aria-checked={isSel} restored on option buttons.

docs/keyboard.md records the full map.
2026-09-17 21:18:56 +03:00
amosblomqvistandClaude Fable 4.5 plus-plus 682edde844 feat: keyboard shortcuts wave A — registry, provider, global bindings
The shortcut architecture (owner-approved plan, 2026-09-17):
- src/keyboard.ts: pure action registry (id, label, scope, default
  binding) + lh:keymap localStorage overrides + event normalization.
  The settings milestone renders this same registry as its editor.
- components/keyboard-shortcuts.tsx: the single document-level keydown
  listener (window level, bubble) owning the pieces no surface owns:
  - Tab is DEAD (owner decision): browser tab-focus navigation is
    abandoned; Tab is suppressed everywhere, Shift+Tab cycles the
    thinking level.
  - Esc is a layered dismiss: topmost popover first (popovers
    stopPropagation their own Esc — usePopoverDismiss + ModelControl),
    then dictation stop (lh:kbd-stop-dictation), then abort of an
    in-flight turn (lh:kbd-abort-turn; server-side wake turns are
    honestly non-stoppable — the seam only stops local streams).
  - Global actions dispatch lh:kbd-<action> window events (the
    lh:chat-renamed pattern); surfaces that aren't mounted simply
    don't listen.
- The input guard: inside a typing surface only mod/alt combos (and
  Tab) fire; bare keys pass through to the field.
- Cheat sheet overlay (?) renders from the registry, grouped by scope.

Bindings wired this wave: cmd+d dictation toggle, cmd+k model picker,
shift+tab thinking cycle, cmd+b sidebar, cmd+shift+f sessions search,
alt+up/down + alt+enter session cycling, cmd+g graph view, cmd+n new
session, f2 rename (chat), ? cheat sheet. Wave B adds the question
keyboard machine; wave C the per-surface bindings (find, import).

Verified live in the dev shell (CDP): all of the above via
Input.dispatchKeyEvent.

Co-Authored-By: Claude Fable 4.5 plus-plus <noreply@anthropic.com>
2026-09-17 20:47:23 +03:00
amosblomqvist 2d3f0e0412 feat: dictation round 2 — deferred transcript, adaptive meter
Owner's second live test:

- DEFERRED transcript: finals accumulate in a buffer during the run and
  flush into the primary input once, when dictation ends (Stop →
  CloseStream → Deepgram's last finals → socket close → one joined
  append; the 3s safety net covers a missing close). Nothing textual
  renders while recording — the meter is the only live feedback. Live
  output may return as a toggleable setting at the settings milestone;
  deferred is the default behavior.
- ADAPTIVE meter: cell count derives from the measured strip width (4px
  pitch, 8-48 cells, ResizeObserver; cells drop their minimum width) so
  narrow rows beside the knowledge-context slider fit — verified in the
  dev shell: draft 20 cells/83px, chat 48 cells/273px, no overlap.
2026-09-17 20:06:07 +03:00
amosblomqvist 4dbbfa6fe7 feat: dictation round 1 — primary-input routing, no interims, optimistic meter, wide button
Owner's first live mic test (2026-09-17) drove four changes:

- ONE primary transcription input at all times: while a quiz/ask
  question waits, finals land in its field (text mode → 'Your answer',
  select modes → 'Your thinking') via a module-level claim registry
  (useDictationTarget in dictation.tsx); the composer is the fallback.
  Esc in a question field also stops dictation (threaded through
  DockOverlay → QuestionDock).
- No interim results: live words were distracting and shifted the meter.
  Params now match the pi dictate extension exactly (no interim_results,
  endpointing=300, no utterance_end_ms); the meter's width stays fixed.
- Optimistic meter: the click enters the meter view immediately (Stop
  pill + meter within one frame, verified at 150ms in the dev shell);
  token mint and mic acquisition run concurrently behind it.
- Wide idle button: the stats-bar middle is one full-width affordance —
  a quiet bordered 'Dictate' button when idle (failure text replaces the
  label), Stop pill + fixed meter while live.

Live-verified in the Electron dev shell: wide button renders on home
(167px) and chat (358px), meter + Stop appear at 150ms after click,
flat 2px bars in silence, clean return to 'Dictate' on stop.
2026-09-17 19:59:14 +03:00
amosblomqvist 573e369db5 fix: dictation cloud path — bearer subprotocol, access_token grant field
Two live-verified Deepgram facts from the first real mic-test round:
- /v1/auth/grant returns { access_token }, not { token } — the route now
  parses the right field.
- Minted JWTs authenticate over the WebSocket via the "bearer"
  subprotocol; ["token", jwt] is rejected at the handshake (token is
  for raw API keys only, as in the pi dictate extension).

Also documents the env-precedence gotcha in docs/research/stt.md: a
DEEPGRAM_API_KEY exported in the shell (e.g. .zshrc) silently beats
.env.local in next dev and the desktop authEnv() seam.
2026-09-17 19:44:35 +03:00
amosblomqvist e3bdbc2f2c feat: voice dictation (STT wave 1 — Deepgram cloud)
Two-band architecture from docs/research/stt.md: renderer captures mic
audio in an AudioWorklet (linear16 16kHz mono, ~128ms frames, per-quantum
RMS for the meter) and streams it directly to Deepgram over WebSocket —
audio never loops through our server. The real key stays server-side;
/api/stt/token mints a short-lived grant token (TTL 300s) per run.

Patterns ported from the owner's pi dictate extension (tuned in real
use): generation counter guarding every async handler, the
latest-RMS-never-resets meter (-50..-10dB window, 60ms tick), mic-first
stop + CloseStream + 3s safety net, endpointing=300 with the
utterance_end_ms=1000 backstop.

UI (owner placement): mic button in the stats bar's middle slot on all
three surfaces (home graph-view, chat, draft); while recording the meter
unfurls to fill the row's free width with the interim ghost at its
flank. Esc ends dictation from the composer. Token mint comes before
getUserMedia so a keyless app fails fast without a TCC prompt; one
automatic reconnect on abnormal socket close, errors surface inline.

Live-smoked the full error path in the dev shell (bar renders on all
three pages, 403 grant failure shows inline, mic untouched). The full
audio path needs a Deepgram key with Member permission to mint tokens.
2026-09-17 19:31:25 +03:00