A-Mahla HF Staff commited on
Commit
1f28297
·
verified ·
1 Parent(s): 3019178

Deploy replica from source HEAD

Browse files
Files changed (22) hide show
  1. .dockerignore +12 -0
  2. .gitignore +16 -0
  3. CONTEXT.md +81 -0
  4. DESIGN.md +205 -0
  5. Dockerfile +14 -0
  6. README.md +182 -5
  7. auth.py +229 -0
  8. docs/adr/0001-docker-space-with-search-proxy.md +29 -0
  9. index.html +363 -0
  10. limiter.py +276 -0
  11. main.js +1245 -0
  12. requirements.txt +5 -0
  13. server.py +292 -0
  14. style.css +2509 -0
  15. ui/account.js +178 -0
  16. ui/chat.js +425 -0
  17. ui/dom.js +37 -0
  18. worklets/audio-playback.js +171 -0
  19. worklets/mic-capture.js +159 -0
  20. ws/codec.js +57 -0
  21. ws/orb-visualizer.js +98 -0
  22. ws/s2s-ws-client.js +899 -0
.dockerignore ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ .git
2
+ .gitignore
3
+ __pycache__/
4
+ *.pyc
5
+ .venv/
6
+ venv/
7
+ .vscode/
8
+ .idea/
9
+ .DS_Store
10
+ *.log
11
+ docs/
12
+ .claude/
.gitignore ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # macOS metadata
2
+ .DS_Store
3
+
4
+ # Editor scratch
5
+ .vscode/
6
+ .idea/
7
+ *.swp
8
+
9
+ # Static HTTP server logs (we don't ship them)
10
+ *.log
11
+
12
+ # Python
13
+ __pycache__/
14
+ *.pyc
15
+ .venv/
16
+ venv/
CONTEXT.md ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Context glossary
2
+
3
+ Canonical terms for this space. A glossary, not a spec — it defines what words mean,
4
+ not how anything is built. Keep design/implementation detail in `DESIGN.md` and the
5
+ code.
6
+
7
+ ## Speech-to-speech demo
8
+ The product: a voice conversation you have with a model by tapping the orb and
9
+ talking. "The demo" and "the space" refer to this same thing. It runs on Hugging
10
+ Face's open `speech-to-speech` backend.
11
+
12
+ ## The pipeline
13
+ The ordered path a turn travels, from your voice to the orb's reply. Order is
14
+ meaningful — each stage consumes the previous one's output:
15
+
16
+ `you speak → VAD → STT → VLM → TTS → orb replies`
17
+
18
+ - **VAD** — voice activity detection. Decides *when* you are speaking, so the system
19
+ knows a turn has started and ended. Model: silero-vad.
20
+ - **STT** — speech to text. Transcribes your speech into words. Model:
21
+ nvidia/parakeet-tdt-1.1b.
22
+ - **VLM** — the vision-language model that composes the reply. Served via Cerebras.
23
+ Model: google/gemma-4-31B-it.
24
+ - **TTS** — text to speech. Speaks the reply back in the chosen voice. Model:
25
+ Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice.
26
+
27
+ ## Builder
28
+ A Hugging Face user credited with making the space, shown by HF username. Current
29
+ builders: tfrere, A-Mahla and andito. Distinct from the *models'* authors (nvidia, google,
30
+ Qwen, snakers4), who are credited per pipeline stage.
31
+
32
+ ## Powered by
33
+ The infrastructure running the pipeline, named in the about panel: Hugging Face
34
+ Inference Endpoints (hosting) and Cerebras (LLM inference). Distinct from "built by"
35
+ (the people) and from the model authors (who trained each model).
36
+
37
+ ## Tool
38
+ A function the model can call mid-conversation to do something the pipeline
39
+ can't do on its own (look something up, look through the camera). Tools are
40
+ declared to the backend in the session config; the model decides when to call
41
+ one. Distinct from the *pipeline stages* (VAD/STT/VLM/TTS), which always run.
42
+
43
+ ## Tool executor
44
+ The client-side component that runs a tool when the model calls it and returns
45
+ the result to the backend, so the model can speak the answer. It is the missing
46
+ half of the round-trip: the backend already emits the call, the executor runs it
47
+ and replies. Distinct from the *tool* itself (the thing being run).
48
+
49
+ ## Web search tool
50
+ A tool that looks something up on the web for the model. The model calls it with
51
+ a query; the tool executor forwards the query to the search proxy and returns the
52
+ results as the tool result. Activates only when a search key is available.
53
+
54
+ ## Camera snapshot tool
55
+ A tool that lets the model look through the user's webcam. While enabled, a live
56
+ self-view is shown in the page (bottom-left); when the model calls the tool, the
57
+ executor captures a frame and sends it to the model as an image so the VLM can
58
+ see it. Distinct from the *preview* (what the user sees) and the *snapshot* (the
59
+ single frame sent to the model).
60
+
61
+ ## Search proxy
62
+ The same-origin server route (`/search`) that holds the search key and calls the
63
+ external search provider on the client's behalf, so the key never reaches the
64
+ browser. Lives in the same container as the page. Distinct from the *s2s backend*
65
+ (the separate load-balanced speech-to-speech service).
66
+
67
+ ## Tools panel
68
+ The dialog opened from the "Tools" button in the top-right, holding one switch per
69
+ tool (and the web-search key status). Turning a switch on/off declares or removes
70
+ that tool on the live session. Distinct from *Settings* (connection, voice,
71
+ instructions) and the *About panel* (project info).
72
+
73
+ ## Identity block
74
+ The top-left corner of the topbar (replacing the old wordmark): the demo name, a
75
+ one-line blurb, and the "powered by" / "built by" credits, shown directly rather
76
+ than hidden behind a click.
77
+
78
+ ## About panel
79
+ The popup opened from the (i) icon to the right of the identity block. Holds the
80
+ general introduction to the speech-to-speech project (with a repo link) and the
81
+ pipeline. Identity itself now lives in the corner, not here.
DESIGN.md ADDED
@@ -0,0 +1,205 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Design language
2
+
3
+ The reference for keeping this app visually coherent as it grows. Read it before
4
+ touching `style.css`, `index.html`, or any DOM-building code in `main.js`. Every
5
+ rule here is already live in the codebase — this file explains the *why* so changes
6
+ extend the system instead of drifting from it.
7
+
8
+ ---
9
+
10
+ ## The thesis: color belongs to the voice
11
+
12
+ This is a voice app. The one thing in the room that should have color is the thing
13
+ that is talking. So:
14
+
15
+ - **The orb** carries saturated color. It glows, and the glow's hue changes with
16
+ conversational state.
17
+ - **Everything else is monochrome** — a precise cool-grey dark canvas. Surfaces,
18
+ borders, buttons, panels, the transcript: all greyscale.
19
+ - The **only** exceptions are tiny *role echoes* (a one-word mono label, a small
20
+ icon) that borrow the orb's state hue so the transcript reads in the same color
21
+ language the orb speaks. They are accents the size of a word, never fills.
22
+ - **Brand logos keep their own color.** The Hugging Face mark (`#ffd21e`) and the
23
+ Cerebras mark (`#f15a29`) render in their brand colors in the identity credits and
24
+ the about panel — a deliberate, owner-approved exception. It applies to those two
25
+ logos only; do not generalize it to other chrome.
26
+
27
+ If you find yourself adding a tinted background, a colored border, or a bright
28
+ button anywhere outside the orb, stop — that color almost certainly belongs to the
29
+ orb instead, or shouldn't exist.
30
+
31
+ ---
32
+
33
+ ## Color tokens
34
+
35
+ All defined in `:root` in `style.css`. Use the variables, never raw hex in rules.
36
+
37
+ ### Canvas (the monochrome world)
38
+ | Token | Value | Use |
39
+ |---|---|---|
40
+ | `--bg` | `#0a0b10` | Page background (a cool near-black) |
41
+ | `--bg-elev` | `#13151c` | Raised surfaces: bubbles, panels, icon buttons |
42
+ | `--bg-elev-2` | `#1b1e29` | Surfaces on surfaces: history bodies, inputs |
43
+ | `--border` | `rgba(255,255,255,.08)` | Default hairline |
44
+ | `--border-strong` | `rgba(255,255,255,.16)` | Emphasised hairline |
45
+ | `--text` | `#f5f6fa` | Primary text; also the *primary button* fill |
46
+ | `--text-dim` | `rgba(245,246,250,.65)` | Secondary text |
47
+ | `--text-faint` | `rgba(245,246,250,.42)` | Captions, labels, footer |
48
+
49
+ ### Voice (the only saturated hues)
50
+ These are the orb's state colors. They appear on the orb, and as small role echoes
51
+ in the transcript — nowhere else.
52
+
53
+ | Token | Value | Meaning |
54
+ |---|---|---|
55
+ | `--accent` / `--speaking` | `#8b7dff` violet | Assistant speaking |
56
+ | `--accent-2` / `--listening` | `#22d3ee` cyan | You / listening |
57
+ | `--processing` | `#f59e0b` amber | Thinking / tool call |
58
+ | `--error` | `#ff6a75` | Error |
59
+ | `--success` | `#34d399` | Ready / connected |
60
+
61
+ ### Role echoes (semantic aliases — use these in chat code)
62
+ | Token | Maps to | Where it shows |
63
+ |---|---|---|
64
+ | `--voice-user` | cyan | `YOU` label + user bubble accents |
65
+ | `--voice-assistant` | violet | `ASSISTANT` label + assistant accents |
66
+ | `--voice-tool` | amber | `TOOL CALL` label, wrench icon |
67
+
68
+ **Why this mapping:** it mirrors the orb exactly — when *you* speak the orb is cyan
69
+ (`state-listening`), when the *assistant* speaks it's violet (`state-ai-speaking`),
70
+ when it's working it's amber (`state-processing`). The transcript is a quiet replay
71
+ of the orb's color story.
72
+
73
+ ### Orb state → glow (`.circle.state-*` → `--glow`)
74
+ | State | Glow |
75
+ |---|---|
76
+ | `signed-out` | violet `#8b7dff` |
77
+ | `authenticated` / `ready` | green `#34d399` |
78
+ | `connecting` / `connected` / `starting` | yellow `#facc15` |
79
+ | `listening` / `user-speaking` | cyan (`--listening`) |
80
+ | `processing` | amber (`--processing`) |
81
+ | `ai-speaking` | violet (`--speaking`) |
82
+ | `error` | red (`--error`) |
83
+
84
+ Adding a new state? Give it a `--glow`, and if it surfaces in the transcript, add a
85
+ matching `--voice-*` alias rather than a one-off color.
86
+
87
+ ---
88
+
89
+ ## Typography
90
+
91
+ Two faces, two jobs. Never reach for a third.
92
+
93
+ - **Inter** — body and UI. Wordmark, buttons, inputs, panel titles, prose, history
94
+ message bodies. The workhorse; it should feel neutral and get out of the way.
95
+ - **Geist Mono** (`--font-mono`) — the **machine voice**. Reserved for text the
96
+ *system* emits or identifiers it reports, never for human prose.
97
+
98
+ ### When mono is correct
99
+ Mono signals "this is the machine talking or naming itself." Use it for:
100
+ - the orb's status caption (`.circle-caption`)
101
+ - role eyebrows (`YOU` / `ASSISTANT` / `TOOL CALL`)
102
+ - tool-call names and argument JSON
103
+ - the `·WebSocket` transport tag, bitrate readouts, connection identifiers
104
+ - the empty-state label
105
+
106
+ Mono text is set uppercase with `letter-spacing: ~0.1–0.14em` and weight `500`, so it
107
+ reads as a typed status line, not a headline. Body copy, button labels, and
108
+ explanatory `small` text stay **Inter** — putting prose in mono breaks the metaphor.
109
+
110
+ The font is loaded in `index.html`; the stack falls back to system mono gracefully if
111
+ the CDN is blocked.
112
+
113
+ ---
114
+
115
+ ## Layout
116
+
117
+ - **One continuous canvas.** No dividers under the topbar or above the footer, no
118
+ panel chrome competing with content. The topbar and footer float over the stage.
119
+ Keep it that way — a new section earns a hairline (`--border`) only if it genuinely
120
+ needs separating.
121
+ - **The orb is the hero and the center of gravity.** It sits dead-center on the
122
+ stage. Controls flank it (mic / stop), captions sit beneath. Don't crowd it.
123
+ - **Hairlines, not boxes.** Separation comes from `1px` borders at 8–16% white and
124
+ from spacing, not from heavy fills or shadows. Shadows are soft and low
125
+ (`0 4px 18px rgba(0,0,0,.32)`), used only to lift floating elements (bubbles,
126
+ panels, modal).
127
+ - **Radii:** `--radius-sm: 8px` (buttons, inputs, chips), `--radius-md: 14px`
128
+ (bubbles, message bodies, modal), `--radius-lg: 22px` (reserved). Pick by element
129
+ size; don't invent new values.
130
+ - **Two reading surfaces for the transcript:** ephemeral bubbles top-right (desktop
131
+ only) that log and fade, and a slide-in history panel for review. On phones the
132
+ bubble stream is dropped and the panel goes full-screen — the panel is the single
133
+ source of truth there.
134
+
135
+ ---
136
+
137
+ ## Components
138
+
139
+ - **Buttons.** Default (`.btn`) is a neutral elevated surface. The *primary* button
140
+ is **near-white on dark** (`--text` fill, `--bg` text) — the highest-contrast thing
141
+ on the page that *isn't* the orb. There is no colored button; emphasis comes from
142
+ contrast, not hue.
143
+ - **Icon buttons** (`.icon-btn`) are `36px`, elevated surface, dim icon that brightens
144
+ on hover. Side controls (`.side-btn`) are circular, collapse to zero size until the
145
+ session is live (by width on desktop, by height in the mobile column).
146
+ - **Chat bubbles & history messages** share one neutral surface. They are
147
+ distinguished by **side** (you = left, assistant = right) plus the **mono role
148
+ label** in the role-echo hue — not by tinted fills. Tool entries use the wrench icon
149
+ + mono + amber, on the same neutral surface.
150
+ - **Badge** (new-message dot) is monochrome white — a signal, not a color accent.
151
+ - **Focus** is visible and neutral: inputs focus to `--text-dim`; the orb uses a
152
+ `--glow`-colored outline (it's the orb, so color is allowed).
153
+
154
+ ---
155
+
156
+ ## Motion
157
+
158
+ - **The orb is audio-reactive, not timer-driven.** Mic RMS (`--audio-level`) and the
159
+ assistant output level (`--ai-audio-level`) drive scale/opacity at display rate, so
160
+ every syllable moves it. This is the signature animation — keep new motion
161
+ subordinate to it.
162
+ - **Quiet by default.** Breathing/glow throbs are slow (1.4–2.4s) and low-contrast.
163
+ Resist adding scattered micro-animations; an orchestrated moment beats many small
164
+ ones, and excess motion reads as AI-generated.
165
+ - **First paint is frozen.** `body.booting` disables all transitions until the first
166
+ frame commits (stripped after one rAF in `main.js`). Anything new that would
167
+ otherwise animate-in on load must respect this.
168
+ - Honor `prefers-reduced-motion` for any motion you add.
169
+
170
+ ---
171
+
172
+ ## Writing / copy voice
173
+
174
+ - Sentence case, plain verbs, no filler. Tuned and quiet — match the minimal canvas.
175
+ - Name things by what the user controls, not by the system's internals. A button says
176
+ exactly what it does, and keeps the same word through the flow.
177
+ - **Empty states invite action** ("Tap the orb and start talking"), they don't just
178
+ set a mood.
179
+ - **Errors state what happened and how to recover**, in the interface's voice — they
180
+ don't apologize and are never vague.
181
+ - Mono labels are terse identifiers (`YOU`, `TOOL CALL`); prose stays in Inter.
182
+
183
+ ---
184
+
185
+ ## Responsive floor (non-negotiable)
186
+
187
+ Every change ships meeting these:
188
+ - Works down to a `360px`-wide phone. The `@media (max-width: 600px)` block already
189
+ handles the phone layout — extend it, don't fight it.
190
+ - Visible keyboard focus on every interactive element.
191
+ - `prefers-reduced-motion` respected.
192
+ - Tap targets ≥ `44px`; `touch-action: manipulation` on anything tappable.
193
+
194
+ ---
195
+
196
+ ## Before you ship — the mirror check
197
+
198
+ 1. Is every saturated color either on the orb or a word-sized role echo? If a fill or
199
+ border is colored, remove that accessory.
200
+ 2. Is mono used only for machine/system text, and Inter for everything human?
201
+ 3. Does any new state have both a `--glow` and (if it appears in chat) a `--voice-*`
202
+ alias?
203
+ 4. Are separations hairlines + spacing, not boxes and heavy shadows?
204
+ 5. Did you add motion? Is it quieter than the orb and reduced-motion-safe?
205
+ 6. Remove one accessory. The minimal look survives on precision, not addition.
Dockerfile ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Single container: serves the static front-end AND the /api/search proxy.
2
+ # HF Spaces (sdk: docker) routes traffic to $PORT, default 7860.
3
+ FROM python:3.11-slim
4
+
5
+ WORKDIR /app
6
+
7
+ COPY requirements.txt .
8
+ RUN pip install --no-cache-dir -r requirements.txt
9
+
10
+ COPY . .
11
+
12
+ EXPOSE 7860
13
+
14
+ CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "7860"]
README.md CHANGED
@@ -1,10 +1,187 @@
1
  ---
2
- title: Hf Realtime Voice
3
- emoji: 🐨
4
- colorFrom: yellow
5
- colorTo: gray
6
  sdk: docker
 
7
  pinned: false
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: HF Realtime Voice
3
+ emoji: 🎙️
4
+ colorFrom: indigo
5
+ colorTo: purple
6
  sdk: docker
7
+ app_port: 7860
8
  pinned: false
9
+ short_description: Voice chat over WebSocket against a HF speech-to-speech
10
+ hf_oauth: true
11
  ---
12
 
13
+ # Minimal Conversation App (S2S backend, **WebSocket** transport)
14
+
15
+ Drop-in alternative to [`amir-tfrere/minimal-conversation-app-s2s-backend`](https://huggingface.co/spaces/amir-tfrere/minimal-conversation-app-s2s-backend)
16
+ that uses the **WebSocket** route of the Hugging Face speech-to-speech
17
+ backend instead of the WebRTC SDP proxy. Same load balancer, same
18
+ `/session` handshake, same UI, same orb. Just a different wire.
19
+
20
+ ## How it works
21
+
22
+ 1. App POSTs `<lb_url>/session` (empty JSON body).
23
+ 2. The LB picks a ready compute (round-robin) and returns:
24
+ ```json
25
+ {
26
+ "session_id": "...",
27
+ "websocket_url": "wss://<compute>/v1/realtime",
28
+ "connect_url": "wss://<compute>/v1/realtime?session_token=<JWT>",
29
+ "session_token": "<JWT>",
30
+ "pending_timeout_s": 60
31
+ }
32
+ ```
33
+ 3. App opens a WebSocket **directly** on `connect_url` (no rewrite to
34
+ `https://`; unlike the WebRTC client which POSTs an SDP offer).
35
+ 4. Server pushes `session.created` on connect. Client replies with
36
+ `session.update` (OpenAI Realtime **GA** schema: `session.audio.input`,
37
+ `session.audio.output`, `session.output_modalities`).
38
+ 5. Client streams mic audio as PCM16 16 kHz mono base64 chunks
39
+ (`input_audio_buffer.append`, one frame every ~40 ms).
40
+ 6. Server pushes `response.output_audio.delta` (PCM16 24 kHz mono base64)
41
+ and transcript deltas.
42
+
43
+ The backend exposes one concurrent session per compute (same as WebRTC
44
+ mode); the LB pins the session via a signed `session_token`.
45
+
46
+ ## Why WebSocket instead of WebRTC
47
+
48
+ | | WebRTC (original) | WebSocket (this) |
49
+ |---|---|---|
50
+ | Transport | UDP + Opus 48 kHz + ICE/STUN | TCP + raw PCM16 |
51
+ | NAT traversal | needs STUN, can fail on corporate / cellular | none, works everywhere TCP is allowed |
52
+ | Audio quality | excellent (Opus, jitter buffer, FEC) | good (raw PCM, simple ring buffer) |
53
+ | Latency | lowest (~50-150 ms) | low (~150-300 ms typical) |
54
+ | Echo cancellation | browser AEC active on the WebRTC track | browser AEC active via `getUserMedia` constraints |
55
+ | Debuggability | needs `chrome://webrtc-internals` | `wscat` / DevTools network tab |
56
+ | Mobile data | sometimes blocked (UDP) | always works (HTTPS+WSS) |
57
+
58
+ ## Backend requirement
59
+
60
+ This app talks to the WebSocket route `@app.websocket("/v1/realtime")`
61
+ defined in
62
+ [`websocket_router.py`](https://github.com/huggingface/speech-to-speech/blob/feat/webrtc-transport/src/speech_to_speech/api/openai_realtime/websocket_router.py)
63
+ on the **`feat/webrtc-transport`** branch. The same compute serves both
64
+ the WebRTC POST and the WebSocket upgrade on the same path; no backend
65
+ change required.
66
+
67
+ Smoke-test from the shell:
68
+
69
+ ```bash
70
+ LB="https://kaa1l6rplzb1gg3y.us-east-1.aws.endpoints.huggingface.cloud"
71
+ curl -X POST "$LB/session" -H "Content-Type: application/json" -d '{}'
72
+ # -> { "connect_url": "wss://<compute>/v1/realtime?session_token=..." }
73
+ # Feed connect_url into a wscat / websocat and you should get a
74
+ # session.created event back immediately.
75
+ ```
76
+
77
+ ## Tools
78
+
79
+ The assistant can call two tools mid-conversation (toggle them from the **Tools**
80
+ button, top-right):
81
+
82
+ - **Web search** — Google results via Serper.dev, proxied server-side so the key
83
+ never reaches the browser. Set `SERPER_API_KEY` as a Space secret. Without it,
84
+ the tool is disabled unless the user pastes their own key in the Tools panel.
85
+ - **Camera** — while enabled, a live self-view shows bottom-left; when the model
86
+ calls the tool, the current frame is sent to the vision-language model so it can
87
+ see what you're showing it.
88
+
89
+ ## Connecting to a backend
90
+
91
+ The app connects **directly** to a speech-to-speech server's realtime WebSocket —
92
+ no load balancer, no `/session` step. Set the URL in two ways:
93
+
94
+ - **`LOAD_BALANCER_URL` env** (served via `/api/config`) provides the default URL
95
+ shown in Settings — handy for the deployed Space.
96
+ - **Settings → Speech-to-speech server URL** lets you override it: paste a full
97
+ `connect_url` (`wss://host/v1/realtime?...`) or a bare host like `localhost:8080`
98
+ (the app adds `/v1/realtime`).
99
+
100
+ **Settings → Restart** reconnects with the current voice, instructions and URL.
101
+
102
+ ## Usage limits
103
+
104
+ Conversation time is metered per UTC day by sign-in tier (see `limiter.py` /
105
+ `auth.py`), but **only on the deployed Space** — metering turns on only when BOTH
106
+ `LOAD_BALANCER_URL` and `SPACE_ID` (injected automatically by the HF Space
107
+ runtime) are present. Running locally — even with `LOAD_BALANCER_URL` exported —
108
+ leaves the app unmetered. Tunable via env:
109
+
110
+ | Env | Default | What |
111
+ |-----|---------|------|
112
+ | `LIMIT_ANON_SEC` | `300` | Daily seconds for anonymous visitors (5 min) |
113
+ | `LIMIT_FREE_SEC` | `600` | Daily seconds for signed-in non-PRO users (10 min) |
114
+ | `UNLIMITED_ORGS` | _(adds to defaults)_ | Extra HF org names whose members get **unlimited** usage, like PRO |
115
+ | `USAGE_HASH_SECRET` | _(random)_ | HMAC secret for hashing identity keys + signing the anon cookie |
116
+
117
+ PRO members are always unlimited. Members of `cerebras`, `HuggingFaceM4`,
118
+ `smolagents`, and `pollen-robotics` are unlimited out of the box (shown as
119
+ "Team", not "PRO"); set `UNLIMITED_ORGS=my-team` to add more. Matched
120
+ case-insensitively against the user's organisations from HF OAuth.
121
+
122
+ ## Run locally
123
+
124
+ The app is now a small FastAPI server (it serves the front-end *and* the search
125
+ proxy from one container).
126
+
127
+ ```bash
128
+ pip install -r requirements.txt
129
+ export SERPER_API_KEY=... # optional; web search is disabled without it
130
+ export LOAD_BALANCER_URL=... # optional; default s2s server URL (set it in Settings otherwise)
131
+ uvicorn server:app --reload --port 7860
132
+ # or, matching production: docker build -t s2s . && docker run -p 7860:7860 -e SERPER_API_KEY=... -e LOAD_BALANCER_URL=... s2s
133
+ ```
134
+
135
+ Then open <http://localhost:7860/>, click the orb, allow the mic, talk.
136
+
137
+ > Browsers require **HTTPS or `localhost`** for `getUserMedia()` (mic + camera).
138
+ > `127.0.0.1` and `localhost` both work; plain `http://192.168.x.y` does NOT.
139
+
140
+ ## Settings (stored in `localStorage`)
141
+
142
+ | Key | What |
143
+ |-----|------|
144
+ | Load balancer URL | Base URL of your S2S deployment. App POSTs `<lb>/session`. |
145
+ | Voice | Qwen3-TTS speaker name (Aiden, Ryan, Dylan, Eric, Ono_Anna, Serena, Sohee, Uncle_Fu, Vivian) |
146
+ | Instructions | System prompt sent in `session.update` once the WS opens |
147
+
148
+ LocalStorage keys are namespaced `s2s.ws.*` so this app's settings do
149
+ NOT collide with the WebRTC variant.
150
+
151
+ ## Files
152
+
153
+ | File | Role |
154
+ |------|------|
155
+ | `index.html` | Single page, orb + settings modal (identical UI to the WebRTC app) |
156
+ | `main.js` | State machine, settings, tools, camera, noise-gate UI wiring |
157
+ | `ui/chat.js` | `ChatView`: history panel, ephemeral bubbles, transcript/tool streaming |
158
+ | `ui/account.js` | `Account`: HF login chip + popover, daily-limit modal |
159
+ | `ui/dom.js` | Shared helpers: `$`, `escHtml`, `truncateError`, `DEBUG` |
160
+ | `auth.py` | HF OAuth + per-request identity (tier, hashed keys) |
161
+ | `limiter.py` | SQLite per-day talk-time budget (chunked server-clock reservation) |
162
+ | `ws/s2s-ws-client.js` | WebSocket handshake + OpenAI Realtime GA protocol |
163
+ | `ws/codec.js` | base64 <-> PCM helpers + transcript extraction (pure) |
164
+ | `ws/orb-visualizer.js` | `OrbVisualiser`: FFT bands -> orb CSS custom properties |
165
+ | `worklets/mic-capture.js` | AudioWorklet: 48 kHz Float32 -> 16 kHz Int16 PCM, posts ~40 ms chunks |
166
+ | `worklets/audio-playback.js` | AudioWorklet: 24 kHz Float32 ring buffer -> 48 kHz, linear interp, fade in/out |
167
+ | `style.css` | Orb animations, layout, dark theme (verbatim from the WebRTC app) |
168
+
169
+ ## Audio pipeline notes
170
+
171
+ - **Input**: `getUserMedia({ echoCancellation, noiseSuppression, autoGainControl })`
172
+ feeds the `mic-capture` worklet at the `AudioContext` rate. The worklet
173
+ resamples to 16 kHz (boxcar lowpass + decimation on the 48 -> 16 fast
174
+ path, linear interpolation fallback for odd rates) and packs Int16 LE.
175
+ - **Output**: `response.output_audio.delta` decodes to Int16 -> Float32
176
+ and is posted to the `audio-playback` worklet. The worklet maintains a
177
+ per-context ring buffer, linearly interpolates 24 -> 48, and applies
178
+ short 32-frame fades on entry/exit to suppress clicks.
179
+ - **Barge-in**: when the server VAD detects user speech mid-response
180
+ (`input_audio_buffer.speech_started` while `ai-speaking`), the client
181
+ posts `{ kind: "clear" }` to the playback worklet to wipe the queue
182
+ immediately. The server itself cancels the in-flight response.
183
+
184
+ ## Credits
185
+
186
+ - Backend: [huggingface/speech-to-speech](https://github.com/huggingface/speech-to-speech) on `feat/webrtc-transport`
187
+ - UI verbatim from `amir-tfrere/minimal-conversation-app-s2s-backend` (Pollen Robotics × Hugging Face)
auth.py ADDED
@@ -0,0 +1,229 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ HF OAuth + per-request identity for the duration limiter.
3
+
4
+ Login uses Hugging Face's native Spaces OAuth via `huggingface_hub`
5
+ (`attach_huggingface_oauth` / `parse_huggingface_oauth`). The OAuth env
6
+ (`OAUTH_CLIENT_ID`, ...) is injected by the platform when the Space README sets
7
+ `hf_oauth: true`, so this only activates on a deployed Space — locally and in
8
+ direct mode there's no OAuth and the limiter treats everyone as anonymous.
9
+
10
+ Identity:
11
+ - signed in -> tier 'pro' | 'free', keyed by hashed HF `sub`
12
+ - anonymous -> tier 'anon', keyed by BOTH hashed client IP and a hashed
13
+ signed-cookie id (OR-matched in the limiter)
14
+ """
15
+
16
+ import logging
17
+ import os
18
+ import secrets
19
+
20
+ import limiter
21
+
22
+ logger = logging.getLogger("s2s.auth")
23
+
24
+ # huggingface_hub adds these routes when OAuth is attached. Centralised so a
25
+ # version change is a one-line fix; the paths are handed to the client via
26
+ # /api/me rather than hardcoded there.
27
+ OAUTH_LOGIN_PATH = "/oauth/huggingface/login"
28
+ OAUTH_LOGOUT_PATH = "/oauth/huggingface/logout"
29
+
30
+ ANON_COOKIE = "s2s_anon"
31
+ _COOKIE_MAX_AGE = 60 * 60 * 24 * 30 # 30 days
32
+
33
+
34
+ # Members of these orgs get unlimited usage (like PRO) out of the box. The
35
+ # UNLIMITED_ORGS env adds to this set; it doesn't replace it.
36
+ _DEFAULT_UNLIMITED_ORGS = {"cerebras", "huggingfacem4", "smolagents", "pollen-robotics"}
37
+
38
+
39
+ def _unlimited_orgs() -> "set[str]":
40
+ """Org usernames whose members get unlimited usage (like PRO).
41
+
42
+ Defaults to {cerebras, HuggingFaceM4, smolagents}; the UNLIMITED_ORGS env
43
+ (comma/space-separated, e.g. `UNLIMITED_ORGS=my-team`) adds more. Matched
44
+ case-insensitively against the signed-in user's organisations."""
45
+ raw = os.environ.get("UNLIMITED_ORGS", "")
46
+ extra = {o.strip().lower() for o in raw.replace(",", " ").split() if o.strip()}
47
+ return _DEFAULT_UNLIMITED_ORGS | extra
48
+
49
+ try:
50
+ from huggingface_hub import attach_huggingface_oauth, parse_huggingface_oauth
51
+ _OAUTH_IMPORTABLE = True
52
+ except Exception as exc: # pragma: no cover - import guard
53
+ logger.info("huggingface_hub OAuth unavailable (%s); sign-in disabled.", exc)
54
+ _OAUTH_IMPORTABLE = False
55
+
56
+ # Set by attach(): True once OAuth is actually wired (importable + env present).
57
+ oauth_enabled = False
58
+
59
+
60
+ def attach(app) -> bool:
61
+ """Wire HF OAuth onto the app if it's importable and configured. Returns
62
+ whether sign-in is available."""
63
+ global oauth_enabled
64
+ if not _OAUTH_IMPORTABLE or not os.environ.get("OAUTH_CLIENT_ID"):
65
+ return False
66
+ try:
67
+ attach_huggingface_oauth(app)
68
+ oauth_enabled = True
69
+ logger.info("HF OAuth attached (sign-in enabled).")
70
+ except Exception as exc: # pragma: no cover - defensive
71
+ logger.warning("Failed to attach HF OAuth: %r", exc)
72
+ oauth_enabled = False
73
+ return oauth_enabled
74
+
75
+
76
+ def _field(obj, name, default=None):
77
+ """Read a field whether the user-info is an object or a dict."""
78
+ if obj is None:
79
+ return default
80
+ if isinstance(obj, dict):
81
+ return obj.get(name, default)
82
+ return getattr(obj, name, default)
83
+
84
+
85
+ # Surface what we detect (orgs, tier) in logs and on /api/me when set. Handy for
86
+ # verifying org gating on the live Space without guessing.
87
+ AUTH_DEBUG = bool(os.environ.get("AUTH_DEBUG"))
88
+
89
+ # whoami-v2 org lookups are cached for the process lifetime, keyed by token, so
90
+ # /api/me + /api/session don't each hit the Hub.
91
+ _orgs_cache: "dict[str, set[str]]" = {}
92
+
93
+
94
+ def current_oauth(request):
95
+ """The parsed HF OAuth info (user_info + access_token), or None."""
96
+ if not oauth_enabled:
97
+ return None
98
+ try:
99
+ return parse_huggingface_oauth(request)
100
+ except Exception:
101
+ return None
102
+
103
+
104
+ def current_user(request):
105
+ """The signed-in HF user-info, or None."""
106
+ return _field(current_oauth(request), "user_info")
107
+
108
+
109
+ def _user_org_names(user) -> "set[str]":
110
+ """The user's organisations from the OAuth userinfo, by username/name/id."""
111
+ names = set()
112
+ for org in _field(user, "orgs", []) or []:
113
+ for key in ("preferred_username", "name", "sub"):
114
+ val = _field(org, key)
115
+ if val:
116
+ names.add(str(val).lower())
117
+ return names
118
+
119
+
120
+ def _orgs_via_token(token: str) -> "set[str]":
121
+ """Fallback org lookup via the Hub `whoami-v2` API, using the user's OAuth
122
+ access token. Covers the case where the userinfo claim omits `orgs`."""
123
+ if not token:
124
+ return set()
125
+ if token in _orgs_cache:
126
+ return _orgs_cache[token]
127
+ names: "set[str]" = set()
128
+ try:
129
+ import httpx
130
+
131
+ resp = httpx.get(
132
+ "https://huggingface.co/api/whoami-v2",
133
+ headers={"Authorization": f"Bearer {token}"},
134
+ timeout=5.0,
135
+ )
136
+ resp.raise_for_status()
137
+ for org in resp.json().get("orgs", []) or []:
138
+ for key in ("name", "fullname"):
139
+ val = org.get(key)
140
+ if val:
141
+ names.add(str(val).lower())
142
+ except Exception as exc: # pragma: no cover - network/permission dependent
143
+ logger.info("whoami-v2 org lookup failed: %r", exc)
144
+ _orgs_cache[token] = names
145
+ return names
146
+
147
+
148
+ def _org_names(user, token=None, allow=None) -> "set[str]":
149
+ """The user's org usernames from the OAuth userinfo claim. If that doesn't
150
+ already satisfy `allow`, fall back to the Hub `whoami-v2` API (the claim is
151
+ often empty or partial), so membership is resolved either way."""
152
+ names = _user_org_names(user)
153
+ if token and (allow is None or not (allow & names)):
154
+ names = names | _orgs_via_token(token)
155
+ return names
156
+
157
+
158
+ def resolve_tier(user, token=None) -> str:
159
+ """Tier for a signed-in user: 'pro' (paying), 'org' (allow-listed org
160
+ member, unlimited), or 'free'. PRO wins over org if both apply."""
161
+ if bool(_field(user, "is_pro", False)):
162
+ return "pro"
163
+ allow = _unlimited_orgs()
164
+ names = _org_names(user, token, allow)
165
+ tier = "org" if (allow & names) else "free"
166
+ if AUTH_DEBUG:
167
+ logger.info("tier=%s orgs=%s allow=%s", tier, sorted(names), sorted(allow))
168
+ return tier
169
+
170
+
171
+ def user_view(request) -> dict:
172
+ """Public profile for /api/me."""
173
+ info = current_oauth(request)
174
+ user = _field(info, "user_info")
175
+ if not user:
176
+ return {"loggedIn": False, "tier": "anon"}
177
+ token = _field(info, "access_token")
178
+ out = {
179
+ "loggedIn": True,
180
+ "username": _field(user, "preferred_username") or _field(user, "name") or "you",
181
+ "avatar": _field(user, "picture"),
182
+ "tier": resolve_tier(user, token),
183
+ }
184
+ if AUTH_DEBUG:
185
+ out["orgs"] = sorted(_org_names(user, token))
186
+ return out
187
+
188
+
189
+ def _client_ip(request) -> str:
190
+ """Real client IP. On HF the app sits behind a proxy, so the user's address
191
+ is the first hop in X-Forwarded-For, not request.client.host."""
192
+ xff = request.headers.get("x-forwarded-for", "")
193
+ if xff:
194
+ return xff.split(",")[0].strip()
195
+ return request.client.host if request.client else "unknown"
196
+
197
+
198
+ def resolve_identity(request):
199
+ """Resolve (tier, keys, set_cookie) for this request.
200
+
201
+ `keys` are the limiter usage_daily keys to debit (one for signed-in, two for
202
+ anonymous). `set_cookie` is a signed value to Set-Cookie when we minted a new
203
+ anonymous id, else None.
204
+ """
205
+ info = current_oauth(request)
206
+ user = _field(info, "user_info")
207
+ if user:
208
+ sub = _field(user, "sub") or _field(user, "preferred_username")
209
+ token = _field(info, "access_token")
210
+ return resolve_tier(user, token), [limiter.hash_key(f"sub:{sub}")], None
211
+
212
+ # Anonymous: key by IP and a signed cookie id, minting the cookie if absent.
213
+ ip = _client_ip(request)
214
+ cookie_id = limiter.verify_cookie(request.cookies.get(ANON_COOKIE, ""))
215
+ set_cookie = None
216
+ if not cookie_id:
217
+ cookie_id = secrets.token_urlsafe(18)
218
+ set_cookie = limiter.sign_cookie(cookie_id)
219
+ keys = [limiter.hash_key(f"ip:{ip}"), limiter.hash_key(f"cookie:{cookie_id}")]
220
+ return "anon", keys, set_cookie
221
+
222
+
223
+ def set_anon_cookie(response, signed: str) -> None:
224
+ # The Space runs inside an iframe on huggingface.co, so the cookie lives in a
225
+ # cross-site context — it must be SameSite=None; Secure or the browser drops it.
226
+ response.set_cookie(
227
+ ANON_COOKIE, signed,
228
+ max_age=_COOKIE_MAX_AGE, httponly=True, samesite="none", secure=True,
229
+ )
docs/adr/0001-docker-space-with-search-proxy.md ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Convert the static Space into a Docker app with a search proxy
2
+
3
+ To give the model a web search tool, the executor (which runs in the browser of a
4
+ public Space) needs a search key. A `sdk: static` Space serves files as-is with no
5
+ runtime process, so it cannot hold a secret the browser uses without exposing it.
6
+ We convert the Space from `sdk: static` to `sdk: docker`: a single container runs a
7
+ small server (FastAPI + uvicorn) that both serves the existing front-end *unchanged*
8
+ and exposes a same-origin `/search` proxy holding `SERPER_API_KEY` server-side. The
9
+ whole app lives in that one container; the s2s speech-to-speech backend stays the
10
+ separate load-balanced service it already is.
11
+
12
+ ## Considered options
13
+
14
+ - **Stay static, user-supplied key only** — no owner key; search works only if each
15
+ user pastes their own. Rejected as the default because it leaves the deployed demo
16
+ with no working search.
17
+ - **Separate proxy service** — same secrecy, but splits the app across two deploys.
18
+ Rejected: the app must live in one container.
19
+ - **Key baked into client JS at build time** — would be readable in the served
20
+ bundle. Rejected: defeats the point.
21
+
22
+ ## Consequences
23
+
24
+ - Deployment is no longer static: there is a Dockerfile and a server process; the
25
+ README front-matter changes from `sdk: static` to `sdk: docker`.
26
+ - The client calls `/search` same-origin; the server reads the key from env. A user
27
+ may still supply their own key as a fallback, sent per-request to the proxy.
28
+ - The front-end, audio pipeline, and s2s handshake are untouched — only the hosting
29
+ shape and the new route are added.
index.html ADDED
@@ -0,0 +1,363 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="UTF-8" />
5
+ <meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
6
+ <title>Minimal Conversation · S2S backend (WebSocket)</title>
7
+ <link rel="preconnect" href="https://fonts.googleapis.com" />
8
+ <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
9
+ <link
10
+ href="https://fonts.googleapis.com/css2?family=Geist+Mono:wght@400;500;600&family=Inter:wght@400;500;600;700;800&display=swap"
11
+ rel="stylesheet"
12
+ />
13
+ <link rel="stylesheet" href="style.css" />
14
+ </head>
15
+ <!--
16
+ `booting` disables transitions/animations until the first paint commits.
17
+ Without this the orb visibly fades/scales in on the very first frame
18
+ because `state-idle` and the generic `.ind` defaults differ. The
19
+ bootstrap script at the end of main.js strips the class after one rAF.
20
+ -->
21
+ <body class="booting">
22
+ <div id="app">
23
+ <header class="topbar">
24
+ <div class="brand">
25
+ <div class="ident">
26
+ <div class="ident-head">
27
+ <a class="ident-title" href="https://github.com/huggingface/speech-to-speech" target="_blank" rel="noopener">Speech-to-speech demo</a>
28
+ <button id="about-btn" class="about-btn" title="About this space" aria-label="About this space">
29
+ <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="16" x2="12" y2="12"/><line x1="12" y1="8" x2="12.01" y2="8"/></svg>
30
+ </button>
31
+ </div>
32
+ <p class="ident-blurb">An open, real-time voice chat built on Hugging Face's speech-to-speech stack.</p>
33
+ <div class="ident-meta">
34
+ <span class="ident-row">
35
+ <span class="ident-label">Powered by</span>
36
+ <a href="https://huggingface.co/inference-endpoints" target="_blank" rel="noopener">Inference Endpoints</a>
37
+ <span class="sep" aria-hidden="true">·</span>
38
+ <a class="cerebras-credit" href="https://cerebras.ai" target="_blank" rel="noopener"><svg class="cerebras-mark" viewBox="0 0 50 50" aria-hidden="true"><path fill-rule="evenodd" clip-rule="evenodd" d="M29.4186 5.62713C24.2806 5.62713 19.353 7.6682 15.7199 11.3013C12.0868 14.9345 10.0457 19.862 10.0457 25C10.0457 30.1381 12.0868 35.0656 15.7199 38.6988C19.353 42.3319 24.2806 44.373 29.4186 44.373V47.2917C17.1061 47.2917 7.12695 37.3105 7.12695 24.998C7.12695 12.6855 17.104 2.7063 29.4165 2.7063V5.62505L29.4186 5.62713ZM39.3186 13.2875C37.7794 11.9809 35.9971 10.9915 34.0741 10.3761C32.1512 9.76067 30.1256 9.53147 28.1137 9.70166C26.1019 9.87185 24.1435 10.4381 22.3513 11.3677C20.559 12.2974 18.9683 13.5722 17.6704 15.1189C16.3726 16.6655 15.3933 18.4534 14.7888 20.3798C14.1844 22.3062 13.9667 24.3332 14.1483 26.344C14.33 28.3548 14.9073 30.3099 15.8472 32.0968C16.787 33.8838 18.0709 35.4673 19.6249 36.7563L17.7478 38.9938C15.9126 37.4545 14.3987 35.5687 13.2924 33.4442C12.1862 31.3197 11.5092 28.9981 11.3003 26.6119C11.0914 24.2258 11.3545 21.8219 12.0747 19.5374C12.7949 17.253 13.9581 15.1328 15.4978 13.298C17.0373 11.4628 18.9233 9.94877 21.0479 8.84247C23.1726 7.73618 25.4944 7.05923 27.8807 6.8503C30.267 6.64136 32.6711 6.90453 34.9557 7.62477C37.2403 8.34501 39.3607 9.50822 41.1957 11.048L39.3186 13.2875ZM34.6207 15.0459C31.9818 13.6829 28.9113 13.4172 26.0776 14.3068C23.2438 15.1964 20.876 17.1691 19.4895 19.7958C18.1029 22.4224 17.8099 25.4903 18.6741 28.332C19.5384 31.1736 21.4899 33.5589 24.104 34.9688L22.7374 37.5521C19.4721 35.7617 17.0418 32.7592 15.9707 29.1927C14.8996 25.6261 15.2737 21.7815 17.0123 18.4883C18.7509 15.1952 21.7146 12.7176 25.2637 11.5903C28.8129 10.463 32.663 10.7763 35.9832 12.4625L34.6207 15.0459ZM29.4165 17.7896C27.5042 17.7896 25.6702 18.5493 24.318 19.9015C22.9658 21.2537 22.2061 23.0877 22.2061 25C22.2061 26.9124 22.9658 28.7464 24.318 30.0986C25.6702 31.4508 27.5042 32.2105 29.4165 32.2105V35.1313C26.7296 35.1313 24.1526 34.0639 22.2527 32.1639C20.3527 30.2639 19.2853 27.687 19.2853 25C19.2853 22.3131 20.3527 19.7362 22.2527 17.8362C24.1526 15.9362 26.7296 14.8688 29.4165 14.8688V17.7896Z" fill="#F15A29"/><path d="M32.0981 22.5749C31.7875 22.2404 31.4295 21.9534 31.0356 21.7228C30.6925 21.519 30.3014 21.4097 29.9023 21.4062C29.371 21.4062 28.896 21.5041 28.4773 21.6978C28.0725 21.8802 27.7089 22.1425 27.4081 22.469C27.1074 22.7955 26.8758 23.1795 26.7273 23.5978C26.5731 24.0207 26.4981 24.4645 26.4981 24.9124C26.4981 25.3666 26.5731 25.8082 26.7273 26.227C26.877 26.6449 27.1091 27.0286 27.4096 27.3553C27.7102 27.682 28.0733 27.9451 28.4773 28.1291C28.8939 28.3228 29.3731 28.4207 29.9023 28.4207C30.3523 28.4207 30.771 28.3249 31.1564 28.1395C31.5481 27.952 31.8856 27.6707 32.146 27.3228L34.0794 29.4187C33.7877 29.7103 33.4544 29.9624 33.0752 30.1749C32.374 30.566 31.6108 30.8338 30.8189 30.9666C30.4648 31.0207 30.1585 31.0499 29.9023 31.0499C29.0617 31.0552 28.2271 30.9069 27.4398 30.6124C26.6954 30.3371 26.0144 29.914 25.4377 29.3687C24.8643 28.8208 24.4078 28.1624 24.096 27.4332C23.7572 26.6366 23.5883 25.778 23.6002 24.9124C23.6002 23.9874 23.7669 23.1478 24.096 22.3916C24.4085 21.6624 24.8627 21.0041 25.4356 20.4562C26.0148 19.9124 26.696 19.4895 27.4398 19.2145C28.2271 18.92 29.0617 18.7717 29.9023 18.777C30.6419 18.777 31.3856 18.9187 32.1356 19.202C32.8877 19.4895 33.5627 19.952 34.1023 20.5541L32.0981 22.5749Z" fill="#fff"/></svg>Cerebras</a>
39
+ </span>
40
+ <span class="ident-row">
41
+ <span class="ident-label">Built by</span>
42
+ <a class="hf-credit" href="https://huggingface.co" target="_blank" rel="noopener"><svg class="hf-mark" viewBox="0 0 24 24" fill="currentColor" aria-hidden="true"><path d="M12.025 1.13c-5.77 0-10.449 4.647-10.449 10.378 0 1.112.178 2.181.503 3.185.064-.222.203-.444.416-.577a.96.96 0 0 1 .524-.15c.293 0 .584.124.84.284.278.173.48.408.71.694.226.282.458.611.684.951v-.014c.017-.324.106-.622.264-.874s.403-.487.762-.543c.3-.047.596.06.787.203s.31.313.4.467c.15.257.212.468.233.542.01.026.653 1.552 1.657 2.54.616.605 1.01 1.223 1.082 1.912.055.537-.096 1.059-.38 1.572.637.121 1.294.187 1.967.187.657 0 1.298-.063 1.921-.178-.287-.517-.44-1.041-.384-1.581.07-.69.465-1.307 1.081-1.913 1.004-.987 1.647-2.513 1.657-2.539.021-.074.083-.285.233-.542.09-.154.208-.323.4-.467a1.08 1.08 0 0 1 .787-.203c.359.056.604.29.762.543s.247.55.265.874v.015c.225-.34.457-.67.683-.952.23-.286.432-.52.71-.694.257-.16.547-.284.84-.285a.97.97 0 0 1 .524.151c.228.143.373.388.43.625l.006.04a10.3 10.3 0 0 0 .534-3.273c0-5.731-4.678-10.378-10.449-10.378M8.327 6.583a1.5 1.5 0 0 1 .713.174 1.487 1.487 0 0 1 .617 2.013c-.183.343-.762-.214-1.102-.094-.38.134-.532.914-.917.71a1.487 1.487 0 0 1 .69-2.803m7.486 0a1.487 1.487 0 0 1 .689 2.803c-.385.204-.536-.576-.916-.71-.34-.12-.92.437-1.103.094a1.487 1.487 0 0 1 .617-2.013 1.5 1.5 0 0 1 .713-.174m-10.68 1.55a.96.96 0 1 1 0 1.921.96.96 0 0 1 0-1.92m13.838 0a.96.96 0 1 1 0 1.92.96.96 0 0 1 0-1.92M8.489 11.458c.588.01 1.965 1.157 3.572 1.164 1.607-.007 2.984-1.155 3.572-1.164.196-.003.305.12.305.454 0 .886-.424 2.328-1.563 3.202-.22-.756-1.396-1.366-1.63-1.32q-.011.001-.02.006l-.044.026-.01.008-.03.024q-.018.017-.035.036l-.032.04a1 1 0 0 0-.058.09l-.014.025q-.049.088-.11.19a1 1 0 0 1-.083.116 1.2 1.2 0 0 1-.173.18q-.035.029-.075.058a1.3 1.3 0 0 1-.251-.243 1 1 0 0 1-.076-.107c-.124-.193-.177-.363-.337-.444-.034-.016-.104-.008-.2.022q-.094.03-.216.087-.06.028-.125.063l-.13.074q-.067.04-.136.086a3 3 0 0 0-.135.096 3 3 0 0 0-.26.219 2 2 0 0 0-.12.121 2 2 0 0 0-.106.128l-.002.002a2 2 0 0 0-.09.132l-.001.001a1.2 1.2 0 0 0-.105.212q-.013.036-.024.073c-1.139-.875-1.563-2.317-1.563-3.203 0-.334.109-.457.305-.454m.836 10.354c.824-1.19.766-2.082-.365-3.194-1.13-1.112-1.789-2.738-1.789-2.738s-.246-.945-.806-.858-.97 1.499.202 2.362c1.173.864-.233 1.45-.685.64-.45-.812-1.683-2.896-2.322-3.295s-1.089-.175-.938.647 2.822 2.813 2.562 3.244-1.176-.506-1.176-.506-2.866-2.567-3.49-1.898.473 1.23 2.037 2.16c1.564.932 1.686 1.178 1.464 1.53s-3.675-2.511-4-1.297c-.323 1.214 3.524 1.567 3.287 2.405-.238.839-2.71-1.587-3.216-.642-.506.946 3.49 2.056 3.522 2.064 1.29.33 4.568 1.028 5.713-.624m5.349 0c-.824-1.19-.766-2.082.365-3.194 1.13-1.112 1.789-2.738 1.789-2.738s.246-.945.806-.858.97 1.499-.202 2.362c-1.173.864.233 1.45.685.64.451-.812 1.683-2.896 2.322-3.295s1.089-.175.938.647-2.822 2.813-2.562 3.244 1.176-.506 1.176-.506 2.866-2.567 3.49-1.898-.473 1.23-2.037 2.16c-1.564.932-1.686 1.178-1.464 1.53s3.675-2.511 4-1.297c.323 1.214-3.524 1.567-3.287 2.405.238.839 2.71-1.587 3.216-.642.506.946-3.49 2.056-3.522 2.064-1.29.33-4.568 1.028-5.713-.624"/></svg>Hugging Face</a>
43
+ <span class="sep" aria-hidden="true">·</span>
44
+ <a class="handle" href="https://huggingface.co/tfrere" target="_blank" rel="noopener">tfrere</a>
45
+ <span class="sep" aria-hidden="true">·</span>
46
+ <a class="handle" href="https://huggingface.co/A-Mahla" target="_blank" rel="noopener">A-Mahla</a>
47
+ <span class="sep" aria-hidden="true">·</span>
48
+ <a class="handle" href="https://huggingface.co/andito" target="_blank" rel="noopener">andito</a>
49
+ </span>
50
+ </div>
51
+ </div>
52
+ </div>
53
+ <div class="topbar-right">
54
+ <!-- HF login chip / sign-in pill. Populated by ui/account.js; only
55
+ shown when the deploy runs behind a load balancer with OAuth. -->
56
+ <div id="account" class="account" hidden></div>
57
+ <button id="about-btn-m" class="icon-btn about-btn-mobile" title="About this space" aria-label="About this space">
58
+ <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="16" x2="12" y2="12"/><line x1="12" y1="8" x2="12.01" y2="8"/></svg>
59
+ </button>
60
+ <button id="tools-btn" class="icon-btn" title="Tools" aria-label="Tools">
61
+ <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/></svg>
62
+ </button>
63
+ <button id="chat-btn" class="icon-btn" title="Conversation history" aria-label="Conversation history">
64
+ <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>
65
+ <span id="chat-badge" class="chat-badge" aria-hidden="true"></span>
66
+ </button>
67
+ <button id="settings-btn" class="icon-btn" title="Settings" aria-label="Settings">
68
+ <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="3"/><path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 1 1-2.83 2.83l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 1 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 1 1-2.83-2.83l.06-.06a1.65 1.65 0 0 0 .33-1.82 1.65 1.65 0 0 0-1.51-1H3a2 2 0 1 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 1 1 2.83-2.83l.06.06a1.65 1.65 0 0 0 1.82.33H9a1.65 1.65 0 0 0 1-1.51V3a2 2 0 1 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 1 1 2.83 2.83l-.06.06a1.65 1.65 0 0 0-.33 1.82V9c0 .66.39 1.25 1 1.51H21a2 2 0 1 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z"/></svg>
69
+ </button>
70
+ </div>
71
+ </header>
72
+
73
+ <main class="stage">
74
+ <div class="orb-wrap">
75
+ <div id="mic-gate" class="mic-gate">
76
+ <svg id="mic-gate-arc" class="mic-gate-arc" viewBox="0 0 100 100" aria-hidden="true">
77
+ <path id="mga-track" class="mga-track" fill="none" />
78
+ <path id="mga-fill" class="mga-fill" fill="none" />
79
+ <path id="mga-hit" class="mga-hit" fill="none" />
80
+ <circle id="mga-handle" class="mga-handle" r="3" />
81
+ </svg>
82
+ <button id="mic-btn" class="side-btn" type="button" aria-label="Mute" title="Mute" aria-hidden="true">
83
+ <svg class="mic-on" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="2" width="6" height="12" rx="3"/><path d="M5 10a7 7 0 0 0 14 0"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
84
+ <svg class="mic-off" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round"><line x1="2" y1="2" x2="22" y2="22"/><path d="M9 5a3 3 0 0 1 6 0v4"/><path d="M9 10v1a3 3 0 0 0 5.1 2.1"/><path d="M19 10a7 7 0 0 1-1.24 3.97"/><path d="M5 10a7 7 0 0 0 11 5.67"/><line x1="12" y1="19" x2="12" y2="22"/></svg>
85
+ </button>
86
+ </div>
87
+
88
+ <button
89
+ id="main-circle"
90
+ class="circle state-idle"
91
+ type="button"
92
+ aria-label="Start voice conversation"
93
+ >
94
+ <span class="circle-glow" aria-hidden="true"></span>
95
+ <span class="circle-ring" aria-hidden="true"></span>
96
+ <span class="circle-ring-outer" aria-hidden="true"></span>
97
+ <span class="circle-core">
98
+ <span class="circle-indicator" aria-hidden="true">
99
+ <svg class="ind ind-mic" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
100
+ <rect x="9" y="2" width="6" height="12" rx="3" fill="currentColor" stroke="none"/>
101
+ <path d="M5 10a7 7 0 0 0 14 0"/>
102
+ <line x1="12" y1="19" x2="12" y2="22"/>
103
+ <line x1="8" y1="22" x2="16" y2="22"/>
104
+ </svg>
105
+ <svg class="ind ind-error" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="9"/><line x1="12" y1="8" x2="12" y2="13"/><line x1="12" y1="16" x2="12" y2="16"/></svg>
106
+ <span class="ind ind-spinner"></span>
107
+ <span class="ind ind-thinking">
108
+ <span class="dot"></span>
109
+ <span class="dot"></span>
110
+ <span class="dot"></span>
111
+ </span>
112
+ <span class="ind ind-bars">
113
+ <span class="bar"></span>
114
+ <span class="bar"></span>
115
+ <span class="bar"></span>
116
+ <span class="bar"></span>
117
+ <span class="bar"></span>
118
+ </span>
119
+ <svg class="ind ind-voice" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">
120
+ <path d="M3 10v4a1 1 0 0 0 1 1h3l5 4V5L7 9H4a1 1 0 0 0-1 1z" fill="currentColor" stroke="none"/>
121
+ <path class="wave wave-1" d="M16 8a5 5 0 0 1 0 8"/>
122
+ <path class="wave wave-2" d="M19 5a9 9 0 0 1 0 14"/>
123
+ </svg>
124
+ </span>
125
+ </span>
126
+ </button>
127
+
128
+ <button id="stop-btn" class="side-btn" type="button" aria-label="End" title="End" aria-hidden="true">
129
+ <svg viewBox="0 0 24 24" fill="currentColor"><rect x="6" y="6" width="12" height="12" rx="2"/></svg>
130
+ </button>
131
+ </div>
132
+
133
+ <p id="circle-caption" class="circle-caption" role="status">Tap to start</p>
134
+ </main>
135
+
136
+ <footer class="footer">
137
+ <span>
138
+ Powered by
139
+ <a href="https://github.com/huggingface/speech-to-speech" target="_blank" rel="noopener">huggingface/speech-to-speech</a>
140
+ </span>
141
+ </footer>
142
+ </div>
143
+
144
+ <!-- Ephemeral chat bubbles anchored to the top-right -->
145
+ <div id="bubble-stack" class="bubble-stack" aria-live="polite" aria-atomic="false"></div>
146
+
147
+ <!-- Webcam self-view, shown bottom-left while the camera tool is enabled -->
148
+ <div id="cam-pip" class="cam-pip" aria-hidden="true">
149
+ <video id="cam-video" class="cam-video" autoplay playsinline muted></video>
150
+ <span class="cam-flash" aria-hidden="true"></span>
151
+ <span class="cam-label">camera</span>
152
+ </div>
153
+
154
+ <!-- Conversation history panel (overlay) -->
155
+ <div id="chat-panel" class="chat-panel">
156
+ <div id="chat-panel-backdrop" class="chat-panel-backdrop"></div>
157
+ <div class="chat-panel-inner">
158
+ <header class="chat-panel-header">
159
+ <h3>Conversation</h3>
160
+ <button id="chat-panel-close" class="icon-btn" aria-label="Close">
161
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
162
+ </button>
163
+ </header>
164
+ <div id="chat-history" class="chat-history"></div>
165
+ </div>
166
+ </div>
167
+
168
+ <dialog id="about-modal" class="modal about-modal">
169
+ <div class="modal-content">
170
+ <header class="modal-header">
171
+ <h2>About</h2>
172
+ <button id="about-close" class="icon-btn" aria-label="Close">
173
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
174
+ </button>
175
+ </header>
176
+
177
+ <!-- General intro to the speech-to-speech project -->
178
+ <div class="about-intro">
179
+ <p>Speech-to-speech is Hugging Face's open framework for real-time voice agents. Rather than one end-to-end model, it chains four open models from the Hub (speech detection, transcription, a vision-language model, and synthesis), so any stage can be swapped or run locally. This demo wires that pipeline to hosted inference.</p>
180
+ <a class="about-repo" href="https://github.com/huggingface/speech-to-speech" target="_blank" rel="noopener">View the project on GitHub<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
181
+ </div>
182
+
183
+ <!-- The backend, shown as the path a turn actually travels -->
184
+ <div class="about-pipeline">
185
+ <p class="pipeline-title">The pipeline</p>
186
+ <ol class="pipeline">
187
+ <li class="pipe-endpoint">You speak</li>
188
+ <li class="pipe-stage">
189
+ <span class="pipe-tag">VAD</span>
190
+ <span class="pipe-job">detects speech</span>
191
+ <a class="pipe-model" href="https://github.com/snakers4/silero-vad" target="_blank" rel="noopener">silero-vad<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
192
+ </li>
193
+ <li class="pipe-stage">
194
+ <span class="pipe-tag">STT</span>
195
+ <span class="pipe-job">transcribes it</span>
196
+ <a class="pipe-model" href="https://huggingface.co/nvidia/parakeet-tdt-1.1b" target="_blank" rel="noopener">nvidia/parakeet-tdt-1.1b<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
197
+ </li>
198
+ <li class="pipe-stage">
199
+ <span class="pipe-tag">VLM</span>
200
+ <span class="pipe-job">analyses and composes the reply <span class="pipe-note">· via <a class="cerebras-credit" href="https://cerebras.ai" target="_blank" rel="noopener"><svg class="cerebras-mark" viewBox="0 0 50 50" aria-hidden="true"><path fill-rule="evenodd" clip-rule="evenodd" d="M29.4186 5.62713C24.2806 5.62713 19.353 7.6682 15.7199 11.3013C12.0868 14.9345 10.0457 19.862 10.0457 25C10.0457 30.1381 12.0868 35.0656 15.7199 38.6988C19.353 42.3319 24.2806 44.373 29.4186 44.373V47.2917C17.1061 47.2917 7.12695 37.3105 7.12695 24.998C7.12695 12.6855 17.104 2.7063 29.4165 2.7063V5.62505L29.4186 5.62713ZM39.3186 13.2875C37.7794 11.9809 35.9971 10.9915 34.0741 10.3761C32.1512 9.76067 30.1256 9.53147 28.1137 9.70166C26.1019 9.87185 24.1435 10.4381 22.3513 11.3677C20.559 12.2974 18.9683 13.5722 17.6704 15.1189C16.3726 16.6655 15.3933 18.4534 14.7888 20.3798C14.1844 22.3062 13.9667 24.3332 14.1483 26.344C14.33 28.3548 14.9073 30.3099 15.8472 32.0968C16.787 33.8838 18.0709 35.4673 19.6249 36.7563L17.7478 38.9938C15.9126 37.4545 14.3987 35.5687 13.2924 33.4442C12.1862 31.3197 11.5092 28.9981 11.3003 26.6119C11.0914 24.2258 11.3545 21.8219 12.0747 19.5374C12.7949 17.253 13.9581 15.1328 15.4978 13.298C17.0373 11.4628 18.9233 9.94877 21.0479 8.84247C23.1726 7.73618 25.4944 7.05923 27.8807 6.8503C30.267 6.64136 32.6711 6.90453 34.9557 7.62477C37.2403 8.34501 39.3607 9.50822 41.1957 11.048L39.3186 13.2875ZM34.6207 15.0459C31.9818 13.6829 28.9113 13.4172 26.0776 14.3068C23.2438 15.1964 20.876 17.1691 19.4895 19.7958C18.1029 22.4224 17.8099 25.4903 18.6741 28.332C19.5384 31.1736 21.4899 33.5589 24.104 34.9688L22.7374 37.5521C19.4721 35.7617 17.0418 32.7592 15.9707 29.1927C14.8996 25.6261 15.2737 21.7815 17.0123 18.4883C18.7509 15.1952 21.7146 12.7176 25.2637 11.5903C28.8129 10.463 32.663 10.7763 35.9832 12.4625L34.6207 15.0459ZM29.4165 17.7896C27.5042 17.7896 25.6702 18.5493 24.318 19.9015C22.9658 21.2537 22.2061 23.0877 22.2061 25C22.2061 26.9124 22.9658 28.7464 24.318 30.0986C25.6702 31.4508 27.5042 32.2105 29.4165 32.2105V35.1313C26.7296 35.1313 24.1526 34.0639 22.2527 32.1639C20.3527 30.2639 19.2853 27.687 19.2853 25C19.2853 22.3131 20.3527 19.7362 22.2527 17.8362C24.1526 15.9362 26.7296 14.8688 29.4165 14.8688V17.7896Z" fill="#F15A29"/><path d="M32.0981 22.5749C31.7875 22.2404 31.4295 21.9534 31.0356 21.7228C30.6925 21.519 30.3014 21.4097 29.9023 21.4062C29.371 21.4062 28.896 21.5041 28.4773 21.6978C28.0725 21.8802 27.7089 22.1425 27.4081 22.469C27.1074 22.7955 26.8758 23.1795 26.7273 23.5978C26.5731 24.0207 26.4981 24.4645 26.4981 24.9124C26.4981 25.3666 26.5731 25.8082 26.7273 26.227C26.877 26.6449 27.1091 27.0286 27.4096 27.3553C27.7102 27.682 28.0733 27.9451 28.4773 28.1291C28.8939 28.3228 29.3731 28.4207 29.9023 28.4207C30.3523 28.4207 30.771 28.3249 31.1564 28.1395C31.5481 27.952 31.8856 27.6707 32.146 27.3228L34.0794 29.4187C33.7877 29.7103 33.4544 29.9624 33.0752 30.1749C32.374 30.566 31.6108 30.8338 30.8189 30.9666C30.4648 31.0207 30.1585 31.0499 29.9023 31.0499C29.0617 31.0552 28.2271 30.9069 27.4398 30.6124C26.6954 30.3371 26.0144 29.914 25.4377 29.3687C24.8643 28.8208 24.4078 28.1624 24.096 27.4332C23.7572 26.6366 23.5883 25.778 23.6002 24.9124C23.6002 23.9874 23.7669 23.1478 24.096 22.3916C24.4085 21.6624 24.8627 21.0041 25.4356 20.4562C26.0148 19.9124 26.696 19.4895 27.4398 19.2145C28.2271 18.92 29.0617 18.7717 29.9023 18.777C30.6419 18.777 31.3856 18.9187 32.1356 19.202C32.8877 19.4895 33.5627 19.952 34.1023 20.5541L32.0981 22.5749Z" fill="#fff"/></svg>Cerebras</a></span></span>
201
+ <a class="pipe-model" href="https://huggingface.co/google/gemma-4-31B-it" target="_blank" rel="noopener">google/gemma-4-31B-it<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
202
+ </li>
203
+ <li class="pipe-stage">
204
+ <span class="pipe-tag">TTS</span>
205
+ <span class="pipe-job">speaks it back</span>
206
+ <a class="pipe-model" href="https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice" target="_blank" rel="noopener">Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice<svg class="ext" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M8 7h9v9"/></svg></a>
207
+ </li>
208
+ <li class="pipe-endpoint">The orb replies</li>
209
+ </ol>
210
+ </div>
211
+ </div>
212
+ </dialog>
213
+
214
+ <dialog id="settings-modal" class="modal">
215
+ <form method="dialog" class="modal-content">
216
+ <header class="modal-header">
217
+ <h2>Settings</h2>
218
+ <button class="icon-btn" value="close" aria-label="Close">
219
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
220
+ </button>
221
+ </header>
222
+
223
+ <div class="tab-panels">
224
+ <section class="tab-panel active" role="tabpanel">
225
+ <label class="field" id="conn-field">
226
+ <span id="conn-label">Speech-to-speech server URL</span>
227
+ <input id="lb-url" type="text" autocomplete="off" spellcheck="false" placeholder="http://localhost:port" />
228
+ <small id="conn-hint">
229
+ URL of your speech-to-speech server, e.g.
230
+ <code>http://localhost:8080</code> (the app adds <code>/v1/realtime</code>).
231
+ </small>
232
+ </label>
233
+
234
+ <div class="field-row">
235
+ <label class="field">
236
+ <span>Voice</span>
237
+ <select id="voice">
238
+ <option value="Aiden" selected>Aiden</option>
239
+ <option value="Ryan">Ryan</option>
240
+ <option value="Dylan">Dylan</option>
241
+ <option value="Eric">Eric</option>
242
+ <option value="Ono_Anna">Ono_Anna</option>
243
+ <option value="Serena">Serena</option>
244
+ <option value="Sohee">Sohee</option>
245
+ <option value="Uncle_Fu">Uncle_Fu</option>
246
+ <option value="Vivian">Vivian</option>
247
+ </select>
248
+ </label>
249
+ </div>
250
+
251
+ <div class="field">
252
+ <span class="field-head">
253
+ Noise gate
254
+ <span id="gate-value" class="field-value">Off</span>
255
+ </span>
256
+ <div class="gate">
257
+ <div class="gate-track" aria-hidden="true">
258
+ <div id="gate-meter-fill" class="gate-meter-fill"></div>
259
+ <input
260
+ id="noise-gate"
261
+ type="range"
262
+ min="-66"
263
+ max="-3"
264
+ step="1"
265
+ value="-50"
266
+ aria-label="Noise gate threshold"
267
+ />
268
+ </div>
269
+ <div class="gate-ends">
270
+ <span>Off</span>
271
+ <span>−3 dB</span>
272
+ </div>
273
+ </div>
274
+ <small>
275
+ Mutes the mic below the handle so room noise isn't sent. Slide fully
276
+ left to turn it off; the bar shows your live input while in a call.
277
+ </small>
278
+ </div>
279
+
280
+ <label class="field">
281
+ <span>Instructions</span>
282
+ <textarea id="instructions" rows="5" placeholder="You are a friendly voice assistant..."></textarea>
283
+ </label>
284
+
285
+ <div class="field">
286
+ <button id="restart-conversation" type="button" class="btn primary wide" disabled>
287
+ Restart conversation with these settings
288
+ </button>
289
+ <small id="restart-hint">Connect first, then come back here to apply live changes.</small>
290
+ </div>
291
+ </section>
292
+ </div>
293
+
294
+ <footer class="modal-footer">
295
+ <button id="settings-save" type="submit" class="btn primary" value="save">Save</button>
296
+ </footer>
297
+ </form>
298
+ </dialog>
299
+
300
+ <dialog id="tools-modal" class="modal">
301
+ <div class="modal-content">
302
+ <header class="modal-header">
303
+ <h2>Tools</h2>
304
+ <button id="tools-close" class="icon-btn" aria-label="Close">
305
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
306
+ </button>
307
+ </header>
308
+
309
+ <p class="tools-intro">Let the assistant act during the conversation. Changes apply live.</p>
310
+
311
+ <div class="tool-list">
312
+ <div class="tool-row" id="tool-web-row">
313
+ <div class="tool-info">
314
+ <span class="tool-name">Web search</span>
315
+ <span class="tool-desc">Look things up on Google, via Serper.</span>
316
+ </div>
317
+ <label class="switch">
318
+ <input id="tool-web" type="checkbox" />
319
+ <span class="switch-track" aria-hidden="true"></span>
320
+ </label>
321
+ </div>
322
+ <label class="field tools-key">
323
+ <span>Search API key</span>
324
+ <input id="search-key" type="password" autocomplete="off" spellcheck="false" />
325
+ <small id="tool-web-hint"></small>
326
+ </label>
327
+
328
+ <div class="tool-row tool-row-sep" id="tool-cam-row">
329
+ <div class="tool-info">
330
+ <span class="tool-name">Camera</span>
331
+ <span class="tool-desc">Let the assistant see through your webcam.</span>
332
+ </div>
333
+ <label class="switch">
334
+ <input id="tool-cam" type="checkbox" />
335
+ <span class="switch-track" aria-hidden="true"></span>
336
+ </label>
337
+ </div>
338
+ <small id="tool-cam-hint" class="tool-hint">A live preview shows bottom-left while the camera is on.</small>
339
+ </div>
340
+ </div>
341
+ </dialog>
342
+
343
+ <!-- Shown when the per-day conversation budget is spent (mid-call or at
344
+ start). Title / message / CTA / note are filled per tier by
345
+ ui/account.js. The hero is the HF mark — a friendly smiling face. -->
346
+ <dialog id="limit-modal" class="modal limit-modal">
347
+ <div class="modal-content limit-card">
348
+ <button id="limit-close" class="icon-btn limit-close" aria-label="Close">
349
+ <svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
350
+ </button>
351
+ <div class="limit-badge" aria-hidden="true">
352
+ <svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>
353
+ </div>
354
+ <h2 id="limit-title" class="limit-title">That's a wrap for now</h2>
355
+ <p id="limit-msg" class="limit-msg"></p>
356
+ <a id="limit-cta" class="btn primary wide limit-cta" href="#"></a>
357
+ <p id="limit-note" class="limit-note"></p>
358
+ </div>
359
+ </dialog>
360
+
361
+ <script type="module" src="main.js"></script>
362
+ </body>
363
+ </html>
limiter.py ADDED
@@ -0,0 +1,276 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Per-day talk-time budget for the speech-to-speech demo.
3
+
4
+ Our server isn't in the audio path (the browser dials the compute WebSocket
5
+ directly), so it can't cut a live stream. What it *can* do is meter time with a
6
+ server-clock, chunked reservation:
7
+
8
+ - At grant we reserve the first chunk (CHUNK_SEC) and debit it from the day's
9
+ budget. A parallel grant therefore sees the budget already spent.
10
+ - The client heartbeats; each heartbeat extends the reservation one chunk at a
11
+ time until the daily budget runs out, then we report `expired` so the client
12
+ tears down.
13
+ - On a clean end (sendBeacon) we reconcile to the real elapsed time and refund
14
+ the unused chunk. A crash (no end, no heartbeats) is reaped by a sweep and
15
+ forfeits at most one chunk.
16
+
17
+ All time is the server's clock. Budgets are per UTC day; a new day is simply a
18
+ new row (no explicit reset). Logged-in users are keyed by a hashed HF `sub`;
19
+ anonymous users by BOTH a hashed IP and a hashed signed-cookie id, OR-matched
20
+ (spent = max of the two) so clearing one identifier doesn't reset the budget.
21
+
22
+ Storage is SQLite at $USAGE_DB_PATH, else /data (persistent Spaces storage),
23
+ else a /tmp fallback. On /tmp the budget is only per-uptime — flagged in logs.
24
+ """
25
+
26
+ import hashlib
27
+ import hmac
28
+ import logging
29
+ import math
30
+ import os
31
+ import sqlite3
32
+ import tempfile
33
+ import threading
34
+ import time
35
+ from datetime import datetime, timezone
36
+ from pathlib import Path
37
+
38
+ logger = logging.getLogger("s2s.limiter")
39
+
40
+ # ── Tunables (env-overridable) ───────────────────────────────────────────────
41
+ ANON_SEC = int(os.environ.get("LIMIT_ANON_SEC", "300")) # 5 min/day, not signed in
42
+ FREE_SEC = int(os.environ.get("LIMIT_FREE_SEC", "600")) # 10 min/day, signed in, no PRO
43
+ CHUNK_SEC = int(os.environ.get("RESERVE_CHUNK_SEC", "10")) # reservation granularity
44
+ HEARTBEAT_SEC = int(os.environ.get("HEARTBEAT_SEC", "5")) # advertised client cadence
45
+ REAP_AFTER_SEC = int(os.environ.get("SESSION_REAP_SEC", "15")) # silence before sweep
46
+
47
+ # Stable across restarts or the hashed keys (and signed cookies) rotate and the
48
+ # budget effectively resets. Set it as a Space secret. Falls back to a per-boot
49
+ # random value (keys then only hold within one uptime).
50
+ _HASH_SECRET = (os.environ.get("USAGE_HASH_SECRET", "").strip() or os.urandom(32).hex()).encode()
51
+
52
+ _lock = threading.Lock()
53
+ _db_path: "Path | None" = None
54
+
55
+
56
+ def budget_for(tier: str) -> "int | None":
57
+ """Daily second-budget for a tier, or None for unlimited.
58
+
59
+ Unlimited tiers: 'pro' (paying PRO members) and 'org' (members of an
60
+ allow-listed organisation, see UNLIMITED_ORGS in auth.py)."""
61
+ if tier in ("pro", "org"):
62
+ return None
63
+ if tier == "free":
64
+ return FREE_SEC
65
+ return ANON_SEC
66
+
67
+
68
+ def hash_key(raw: str) -> str:
69
+ """HMAC a raw identifier (sub / ip / cookie id) into an opaque storage key."""
70
+ digest = hmac.new(_HASH_SECRET, raw.encode("utf-8"), hashlib.sha256).hexdigest()
71
+ return f"k_{digest}"
72
+
73
+
74
+ def sign_cookie(value: str) -> str:
75
+ """`<id>.<sig>` so a forged anon-cookie id is rejected on read."""
76
+ sig = hmac.new(_HASH_SECRET, value.encode("utf-8"), hashlib.sha256).hexdigest()[:32]
77
+ return f"{value}.{sig}"
78
+
79
+
80
+ def verify_cookie(signed: str) -> "str | None":
81
+ """Return the id if the signature checks out, else None."""
82
+ if not signed or "." not in signed:
83
+ return None
84
+ value, _, sig = signed.rpartition(".")
85
+ want = hmac.new(_HASH_SECRET, value.encode("utf-8"), hashlib.sha256).hexdigest()[:32]
86
+ return value if hmac.compare_digest(sig, want) else None
87
+
88
+
89
+ def _today() -> str:
90
+ return datetime.now(timezone.utc).date().isoformat()
91
+
92
+
93
+ def _resolve_db_path() -> Path:
94
+ explicit = os.environ.get("USAGE_DB_PATH", "").strip()
95
+ if explicit:
96
+ return Path(explicit)
97
+ data = Path("/data")
98
+ if data.is_dir() and os.access(data, os.W_OK):
99
+ return data / "s2s-usage.sqlite3"
100
+ logger.warning("No persistent /data — usage budget falls back to /tmp (per-uptime only).")
101
+ return Path(tempfile.gettempdir()) / "s2s-usage.sqlite3"
102
+
103
+
104
+ def _connect() -> sqlite3.Connection:
105
+ con = sqlite3.connect(_db_path, timeout=5.0)
106
+ con.execute("PRAGMA journal_mode=WAL")
107
+ con.execute("PRAGMA busy_timeout=5000")
108
+ return con
109
+
110
+
111
+ def init() -> None:
112
+ """Create the schema. Call once at startup."""
113
+ global _db_path
114
+ _db_path = _resolve_db_path()
115
+ with _lock, _connect() as con:
116
+ con.execute(
117
+ """CREATE TABLE IF NOT EXISTS usage_daily (
118
+ user_key TEXT NOT NULL,
119
+ day TEXT NOT NULL,
120
+ spent_sec INTEGER NOT NULL DEFAULT 0,
121
+ updated_at INTEGER NOT NULL,
122
+ PRIMARY KEY (user_key, day)
123
+ )"""
124
+ )
125
+ con.execute(
126
+ """CREATE TABLE IF NOT EXISTS sessions (
127
+ session_id TEXT PRIMARY KEY,
128
+ keys TEXT NOT NULL, -- comma-joined usage_daily keys to debit
129
+ day TEXT NOT NULL,
130
+ tier TEXT NOT NULL,
131
+ grant_ts REAL NOT NULL,
132
+ last_seen_ts REAL NOT NULL,
133
+ reserved_sec INTEGER NOT NULL,
134
+ ended INTEGER NOT NULL DEFAULT 0
135
+ )"""
136
+ )
137
+ logger.info("Usage limiter ready at %s (anon=%ss free=%ss chunk=%ss)", _db_path, ANON_SEC, FREE_SEC, CHUNK_SEC)
138
+
139
+
140
+ # ── Internal helpers (call under _lock) ───────────────────────────────────────
141
+
142
+ def _spent(con, key: str, day: str) -> int:
143
+ row = con.execute(
144
+ "SELECT spent_sec FROM usage_daily WHERE user_key=? AND day=?", (key, day)
145
+ ).fetchone()
146
+ return int(row[0]) if row else 0
147
+
148
+
149
+ def _spent_max(con, keys, day: str) -> int:
150
+ """OR-match: the most-spent identifier governs."""
151
+ return max((_spent(con, k, day) for k in keys), default=0)
152
+
153
+
154
+ def _add(con, keys, day: str, delta: int) -> None:
155
+ now = int(time.time())
156
+ for k in keys:
157
+ cur = _spent(con, k, day)
158
+ nxt = max(0, cur + delta)
159
+ con.execute(
160
+ """INSERT INTO usage_daily (user_key, day, spent_sec, updated_at)
161
+ VALUES (?, ?, ?, ?)
162
+ ON CONFLICT(user_key, day) DO UPDATE SET
163
+ spent_sec = excluded.spent_sec, updated_at = excluded.updated_at""",
164
+ (k, day, nxt, now),
165
+ )
166
+
167
+
168
+ # ── Public API ────────────────────────────────────────────────────────────────
169
+
170
+ def remaining(keys, tier: str) -> "int | None":
171
+ """Seconds left today for these keys (None = unlimited). No mutation."""
172
+ budget = budget_for(tier)
173
+ if budget is None:
174
+ return None
175
+ with _lock, _connect() as con:
176
+ return max(0, budget - _spent_max(con, keys, _today()))
177
+
178
+
179
+ def begin(session_id: str, keys, tier: str) -> int:
180
+ """Reserve the first chunk for a new session and record it. Returns the
181
+ chunk reserved (0 if the budget is already exhausted — the first heartbeat
182
+ will then expire it). PRO (unlimited) is never tracked; don't call it here."""
183
+ day = _today()
184
+ budget = budget_for(tier)
185
+ now = time.time()
186
+ with _lock, _connect() as con:
187
+ avail = budget - _spent_max(con, keys, day) if budget is not None else CHUNK_SEC
188
+ chunk = max(0, min(CHUNK_SEC, avail))
189
+ if chunk:
190
+ _add(con, keys, day, chunk)
191
+ con.execute(
192
+ """INSERT OR REPLACE INTO sessions
193
+ (session_id, keys, day, tier, grant_ts, last_seen_ts, reserved_sec, ended)
194
+ VALUES (?, ?, ?, ?, ?, ?, ?, 0)""",
195
+ (session_id, ",".join(keys), day, tier, now, now, chunk),
196
+ )
197
+ return chunk
198
+
199
+
200
+ def heartbeat(session_id: str) -> bool:
201
+ """Keep a session alive: extend the reservation toward `elapsed + 1 chunk`,
202
+ debiting the budget chunk by chunk. Returns True while alive, False once the
203
+ budget is spent (caller should tear down) or the session is unknown/ended."""
204
+ now = time.time()
205
+ with _lock, _connect() as con:
206
+ row = con.execute(
207
+ "SELECT keys, day, tier, grant_ts, reserved_sec, ended FROM sessions WHERE session_id=?",
208
+ (session_id,),
209
+ ).fetchone()
210
+ if not row or row[5]:
211
+ return False
212
+ keys = row[0].split(",")
213
+ day, tier, grant_ts, reserved = row[1], row[2], row[3], int(row[4])
214
+ budget = budget_for(tier)
215
+ elapsed = now - grant_ts
216
+
217
+ # Grow the reservation one chunk at a time until it covers elapsed + a
218
+ # one-chunk lookahead, or the budget runs dry.
219
+ while reserved < elapsed + CHUNK_SEC:
220
+ if budget is not None and _spent_max(con, keys, day) >= budget:
221
+ break
222
+ _add(con, keys, day, CHUNK_SEC)
223
+ reserved += CHUNK_SEC
224
+
225
+ alive = reserved > elapsed # could we cover the time already elapsed?
226
+ con.execute(
227
+ "UPDATE sessions SET last_seen_ts=?, reserved_sec=?, ended=? WHERE session_id=?",
228
+ (now, reserved, 0 if alive else 1, session_id),
229
+ )
230
+ if not alive:
231
+ _reconcile(con, keys, day, grant_ts, reserved, end_ts=now)
232
+ return alive
233
+
234
+
235
+ def end(session_id: str) -> None:
236
+ """Clean teardown: reconcile to actual elapsed time and refund the unused
237
+ reservation. Idempotent."""
238
+ now = time.time()
239
+ with _lock, _connect() as con:
240
+ row = con.execute(
241
+ "SELECT keys, day, grant_ts, reserved_sec, ended FROM sessions WHERE session_id=?",
242
+ (session_id,),
243
+ ).fetchone()
244
+ if not row or row[4]:
245
+ return
246
+ keys, day, grant_ts, reserved = row[0].split(","), row[1], row[2], int(row[3])
247
+ _reconcile(con, keys, day, grant_ts, reserved, end_ts=now)
248
+ con.execute("UPDATE sessions SET ended=1, last_seen_ts=? WHERE session_id=?", (now, session_id))
249
+
250
+
251
+ def sweep() -> None:
252
+ """Reap sessions that went silent (crash / closed without a beacon): bill
253
+ their elapsed time, refund the rest, mark ended. Forfeits ≤ one chunk."""
254
+ now = time.time()
255
+ cutoff = now - REAP_AFTER_SEC
256
+ with _lock, _connect() as con:
257
+ stale = con.execute(
258
+ "SELECT session_id, keys, day, grant_ts, last_seen_ts, reserved_sec FROM sessions "
259
+ "WHERE ended=0 AND last_seen_ts < ?",
260
+ (cutoff,),
261
+ ).fetchall()
262
+ for session_id, keys_s, day, grant_ts, last_seen, reserved in stale:
263
+ _reconcile(con, keys_s.split(","), day, grant_ts, int(reserved), end_ts=last_seen)
264
+ con.execute("UPDATE sessions SET ended=1 WHERE session_id=?", (session_id,))
265
+ if stale:
266
+ logger.debug("swept %d stale session(s)", len(stale))
267
+
268
+
269
+ def _reconcile(con, keys, day: str, grant_ts: float, reserved: int, end_ts: float) -> None:
270
+ """Refund reserved-but-unused time. Bill elapsed rounded up to a chunk,
271
+ capped at what was reserved."""
272
+ elapsed = max(0.0, end_ts - grant_ts)
273
+ billed = min(reserved, int(math.ceil(elapsed / CHUNK_SEC) * CHUNK_SEC))
274
+ refund = reserved - billed
275
+ if refund > 0:
276
+ _add(con, keys, day, -refund)
main.js ADDED
@@ -0,0 +1,1245 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * Minimal voice conversation app, talking to a Hugging Face speech-to-speech
4
+ * backend over **WebSocket** (drop-in alternative to the WebRTC variant).
5
+ *
6
+ * Click the orb -> we ask for the mic, POST a session on the LB, open a
7
+ * WebSocket on the routed compute endpoint, push session.update + mic
8
+ * audio, play back the TTS audio. The orb visually reflects the live
9
+ * state (idle, connecting, listening, user-speaking, processing,
10
+ * ai-speaking).
11
+ *
12
+ * The only meaningful difference vs. the WebRTC main.js is that the
13
+ * client owns its own AudioContext (no `attachOutputTrack`), so we hand
14
+ * it the MediaStream directly.
15
+ *
16
+ * @typedef {"idle" | "connecting" | "listening" | "user-speaking" | "processing" | "ai-speaking" | "error"} AppState
17
+ */
18
+
19
+ import { S2sWsRealtimeClient } from "./ws/s2s-ws-client.js";
20
+ import { $, truncateError, DEBUG } from "./ui/dom.js";
21
+ import { ChatView } from "./ui/chat.js";
22
+ import { Account } from "./ui/account.js";
23
+
24
+ const DEFAULT_VOICE = "Aiden";
25
+ const DEFAULT_INSTRUCTIONS =
26
+ "You are a friendly voice assistant. " +
27
+ "Keep replies short, warm, and spoken. Avoid long monologues.";
28
+
29
+ // Appended to the user's instructions whenever at least one tool is enabled.
30
+ // Stops the model from announcing capabilities ("Yes, I can search") and then
31
+ // idling for the next turn — it should act immediately in the same response.
32
+ const TOOL_USE_HINT =
33
+ " When the user's request calls for one of your tools, do not describe your " +
34
+ "capabilities or say you can do it and wait for another turn. Instead, say " +
35
+ 'a brief acknowledgement like "Let me search for that..." and call the tool ' +
36
+ "right away in the same response.";
37
+
38
+ const STORAGE_KEYS = {
39
+ // Direct s2s server URL, used only when the deploy has no LOAD_BALANCER_URL
40
+ // (in LB mode the browser never learns the LB address — it POSTs /api/session).
41
+ directUrl: "s2s.ws.directUrl",
42
+ voice: "s2s.ws.voice",
43
+ instructions: "s2s.ws.instructions",
44
+ tools: "s2s.ws.tools",
45
+ searchKey: "s2s.ws.searchKey",
46
+ noiseGate: "s2s.ws.noiseGate",
47
+ };
48
+
49
+ // ── Noise gate ──────────────────────────────────────────────────────────────
50
+ // The Settings cursor sets the gate's open threshold in dBFS. Its leftmost
51
+ // position is an OFF detent (gate disabled, pure passthrough); the rest of the
52
+ // travel is the active threshold. The cursor shares the meter's dB axis, so the
53
+ // handle sits on the level bar — raise it until room noise stops lighting it up.
54
+ // The slider range IS the shared axis: the live meter fill and the threshold
55
+ // thumb both map across [GATE_OFF_DB, GATE_MAX_DB], so the thumb sits exactly
56
+ // where the gate cuts on the same scale as the level bar.
57
+ const GATE_OFF_DB = -66; // slider minimum = off / bottom of the meter axis
58
+ const GATE_MAX_DB = -3; // slider maximum = most aggressive / top of the meter axis
59
+ const GATE_DEFAULT_DB = -50; // first-run default: a gentle gate, enabled
60
+
61
+ /** @param {number} thresholdDb @returns {import("./ws/s2s-ws-client.js").NoiseGate} */
62
+ function gateParams(thresholdDb) {
63
+ return { enabled: thresholdDb > GATE_OFF_DB, thresholdDb };
64
+ }
65
+
66
+ // ── Tools ─────────────────────────────────────────────────────────────────
67
+ // Function tools we declare to the backend. The model decides when to call
68
+ // one; the executor below runs it and returns the result (see runTool).
69
+ /** @type {Record<string, import("./ws/s2s-ws-client.js").ToolDef>} */
70
+ const TOOL_DEFS = {
71
+ web_search: {
72
+ type: "function",
73
+ name: "web_search",
74
+ description:
75
+ "Search the web for current or factual information you don't already know " +
76
+ "(news, prices, facts, documentation). Returns the top results with titles, " +
77
+ "snippets and URLs.",
78
+ parameters: {
79
+ type: "object",
80
+ properties: { query: { type: "string", description: "The search query." } },
81
+ required: ["query"],
82
+ },
83
+ },
84
+ camera_snapshot: {
85
+ type: "function",
86
+ name: "camera_snapshot",
87
+ description:
88
+ "Capture the current frame from the user's webcam so you can see what they " +
89
+ "are showing you. Use it whenever the user refers to something visual or " +
90
+ "asks you to look.",
91
+ parameters: { type: "object", properties: {}, required: [] },
92
+ },
93
+ };
94
+
95
+ /** Longest edge of the snapshot sent to the VLM, in px (keeps payload sane). */
96
+ const SNAPSHOT_MAX_EDGE = 768;
97
+ const SNAPSHOT_QUALITY = 0.7;
98
+
99
+ function loadSettings() {
100
+ return {
101
+ directUrl: localStorage.getItem(STORAGE_KEYS.directUrl) || "",
102
+ voice: localStorage.getItem(STORAGE_KEYS.voice) || DEFAULT_VOICE,
103
+ instructions: localStorage.getItem(STORAGE_KEYS.instructions) || DEFAULT_INSTRUCTIONS,
104
+ noiseGate: loadGateThreshold(),
105
+ };
106
+ }
107
+
108
+ /** Stored gate threshold (dBFS), clamped to the slider range. Defaults to a
109
+ * gentle enabled gate (GATE_DEFAULT_DB) when the user hasn't set one yet. */
110
+ function loadGateThreshold() {
111
+ const stored = localStorage.getItem(STORAGE_KEYS.noiseGate);
112
+ // getItem returns null when unset, and Number(null) === 0 (finite!), so guard
113
+ // the missing/empty case explicitly before coercing — otherwise the default
114
+ // never fires and 0 clamps to the slider max.
115
+ if (stored === null || stored === "") return GATE_DEFAULT_DB;
116
+ const raw = Number(stored);
117
+ if (!Number.isFinite(raw)) return GATE_DEFAULT_DB;
118
+ return Math.min(GATE_MAX_DB, Math.max(GATE_OFF_DB, Math.round(raw)));
119
+ }
120
+
121
+ /** @param {ReturnType<typeof loadSettings>} s */
122
+ function saveSettings(s) {
123
+ localStorage.setItem(STORAGE_KEYS.directUrl, s.directUrl);
124
+ localStorage.setItem(STORAGE_KEYS.voice, s.voice);
125
+ localStorage.setItem(STORAGE_KEYS.instructions, s.instructions);
126
+ localStorage.setItem(STORAGE_KEYS.noiseGate, String(s.noiseGate));
127
+ }
128
+
129
+ /** @returns {{ web_search: boolean, camera_snapshot: boolean }} */
130
+ function loadTools() {
131
+ try {
132
+ const raw = JSON.parse(localStorage.getItem(STORAGE_KEYS.tools) || "{}");
133
+ // Both tools default ON (web search still only activates when a key exists).
134
+ // We never call getUserMedia on page load — the camera only actually starts
135
+ // on a user gesture (conversation start), so a default-on flag doesn't
136
+ // silently resume the webcam; an explicit saved `false` is respected.
137
+ return {
138
+ web_search: raw.web_search ?? true,
139
+ camera_snapshot: raw.camera_snapshot ?? true,
140
+ };
141
+ } catch {
142
+ return { web_search: true, camera_snapshot: true };
143
+ }
144
+ }
145
+
146
+ function saveTools() {
147
+ localStorage.setItem(STORAGE_KEYS.tools, JSON.stringify(toolsEnabled));
148
+ }
149
+
150
+ /** @type {Record<AppState, { caption: string; disabled: boolean }>} */
151
+ const STATE_VIEWS = {
152
+ idle: { caption: "Tap to start", disabled: false },
153
+ connecting: { caption: "Connecting", disabled: true },
154
+ listening: { caption: "", disabled: false },
155
+ "user-speaking": { caption: "", disabled: false },
156
+ processing: { caption: "", disabled: false },
157
+ "ai-speaking": { caption: "", disabled: false },
158
+ error: { caption: "Tap to retry", disabled: false },
159
+ };
160
+
161
+ /** @type {Record<AppState, string>} */
162
+ const STATE_CLASS = {
163
+ idle: "state-idle",
164
+ connecting: "state-connecting",
165
+ listening: "state-listening",
166
+ "user-speaking": "state-user-speaking",
167
+ processing: "state-processing",
168
+ "ai-speaking": "state-ai-speaking",
169
+ error: "state-error",
170
+ };
171
+
172
+ /** @type {ReadonlySet<AppState>} */
173
+ const LIVE_STATES = new Set(["listening", "user-speaking", "processing", "ai-speaking"]);
174
+
175
+ /** @type {HTMLButtonElement} */
176
+ const circleBtn = $("#main-circle");
177
+ /** @type {HTMLParagraphElement} */
178
+ const circleCaption = $("#circle-caption");
179
+ /** @type {HTMLElement} */
180
+ const orbWrap = $(".orb-wrap");
181
+ /** @type {HTMLButtonElement} */
182
+ const micBtn = $("#mic-btn");
183
+ /** @type {HTMLButtonElement} */
184
+ const stopBtn = $("#stop-btn");
185
+
186
+ /** @type {HTMLButtonElement} */
187
+ const settingsBtn = $("#settings-btn");
188
+ /** @type {HTMLDialogElement} */
189
+ const settingsModal = $("#settings-modal");
190
+
191
+ /** @type {HTMLButtonElement} */
192
+ const aboutBtn = $("#about-btn");
193
+ /** @type {HTMLDialogElement} */
194
+ const aboutModal = $("#about-modal");
195
+ /** @type {HTMLButtonElement} */
196
+ const aboutClose = $("#about-close");
197
+
198
+ /** @type {HTMLButtonElement} */
199
+ const toolsBtn = $("#tools-btn");
200
+ /** @type {HTMLDialogElement} */
201
+ const toolsModal = $("#tools-modal");
202
+ /** @type {HTMLButtonElement} */
203
+ const toolsClose = $("#tools-close");
204
+ /** @type {HTMLInputElement} */
205
+ const toolWebSwitch = $("#tool-web");
206
+ /** @type {HTMLInputElement} */
207
+ const toolCamSwitch = $("#tool-cam");
208
+ /** @type {HTMLElement} */
209
+ const toolWebRow = $("#tool-web-row");
210
+ /** @type {HTMLElement} */
211
+ const toolWebHint = $("#tool-web-hint");
212
+ /** @type {HTMLElement} */
213
+ const toolCamHint = $("#tool-cam-hint");
214
+ /** @type {HTMLInputElement} */
215
+ const searchKeyInput = $("#search-key");
216
+ /** @type {HTMLElement} */
217
+ const camPip = $("#cam-pip");
218
+ /** @type {HTMLVideoElement} */
219
+ const camVideo = $("#cam-video");
220
+
221
+ /** @type {HTMLInputElement} */
222
+ const inputLbUrl = $("#lb-url");
223
+ /** @type {HTMLElement} */
224
+ const connField = $("#conn-field");
225
+ /** @type {HTMLElement} */
226
+ const connHint = $("#conn-hint");
227
+ /** @type {HTMLSelectElement} */
228
+ const inputVoice = $("#voice");
229
+ /** @type {HTMLTextAreaElement} */
230
+ const inputInstructions = $("#instructions");
231
+ /** @type {HTMLInputElement} */
232
+ const inputNoiseGate = $("#noise-gate");
233
+ /** @type {HTMLElement} */
234
+ const gateValue = $("#gate-value");
235
+ /** @type {HTMLElement} */
236
+ const gateMeterFill = $("#gate-meter-fill");
237
+ /** @type {HTMLElement} */
238
+ const micGate = $("#mic-gate");
239
+ const mgaArc = /** @type {SVGSVGElement} */ (document.querySelector("#mic-gate-arc"));
240
+ const mgaTrack = /** @type {SVGPathElement} */ (document.querySelector("#mga-track"));
241
+ const mgaFill = /** @type {SVGPathElement} */ (document.querySelector("#mga-fill"));
242
+ const mgaHit = /** @type {SVGPathElement} */ (document.querySelector("#mga-hit"));
243
+ const mgaHandle = /** @type {SVGCircleElement} */ (document.querySelector("#mga-handle"));
244
+ /** @type {HTMLButtonElement} */
245
+ const restartBtn = $("#restart-conversation");
246
+ /** @type {HTMLElement} */
247
+ const restartHint = $("#restart-hint");
248
+ const settingsForm = /** @type {HTMLFormElement} */ (settingsModal.querySelector("form"));
249
+
250
+ /** @type {AppState} */
251
+ let currentState = "idle";
252
+ let settings = loadSettings();
253
+
254
+ // ── Connection target ────────────────────────────────────────────────────────
255
+ // Two modes, decided by the deploy via /api/config:
256
+ // • LOAD_BALANCER_URL set -> original flow: POST the same-origin /api/session
257
+ // proxy (the server forwards to the LB; the LB address is never sent here).
258
+ // • unset (allowDirect) -> the user sets a speech-to-speech server URL and
259
+ // the browser connects to it directly (no load balancer, no /session).
260
+ let lbMode = false;
261
+ // Fail open: direct entry is allowed unless /api/config reports an LB URL. This
262
+ // way a missing/unreachable config (e.g. static hosting) leaves the field
263
+ // usable rather than locked.
264
+ let allowDirect = true;
265
+
266
+ // ── Tool state ──────────────────────────────────────────────────────────────
267
+ let toolsEnabled = loadTools();
268
+ // Whether the server holds a Serper key (learned from /api/config on load).
269
+ let serverSearchKey = false;
270
+ // A user-supplied key (fallback when the deploy has none). localStorage only.
271
+ let userSearchKey = localStorage.getItem(STORAGE_KEYS.searchKey) || "";
272
+ /** @type {MediaStream | null} */
273
+ let cameraStream = null;
274
+
275
+ /** Search is usable if the server has a key or the user supplied one. */
276
+ function searchAvailable() {
277
+ return serverSearchKey || !!userSearchKey;
278
+ }
279
+
280
+ /** Tool definitions for the currently-enabled (and usable) tools. */
281
+ function activeToolDefs() {
282
+ const defs = [];
283
+ if (toolsEnabled.web_search && searchAvailable()) defs.push(TOOL_DEFS.web_search);
284
+ if (toolsEnabled.camera_snapshot) defs.push(TOOL_DEFS.camera_snapshot);
285
+ return defs;
286
+ }
287
+
288
+ /** Instructions plus the hidden tool-use hint when any tool is active. */
289
+ function effectiveInstructions() {
290
+ const base = settings.instructions;
291
+ return activeToolDefs().length ? base + TOOL_USE_HINT : base;
292
+ }
293
+
294
+ /** Push the active tool set to a live session so toggles apply mid-call. */
295
+ function pushToolsToSession() {
296
+ if (!client || !LIVE_STATES.has(currentState)) return;
297
+ client.setTools(activeToolDefs());
298
+ // The hidden tool-use hint depends on whether any tool is active, so refresh
299
+ // instructions alongside the tool set.
300
+ client.updateSession({ instructions: effectiveInstructions() });
301
+ }
302
+
303
+ // ── Chat view ───────────────────────────────────────────────────────────────
304
+ // Owns the history panel, the ephemeral bubbles, and all transcript/tool
305
+ // streaming state. The client's events are forwarded to its on* methods.
306
+ const chat = new ChatView();
307
+
308
+ // ── Account / limiter ─────────────────────────────────────────────────────
309
+ // Login chip + daily-limit modal (inert unless the deploy is in LB mode). The
310
+ // server meters conversation time; the client just heartbeats a live session
311
+ // and tears down when the server reports the budget is spent.
312
+ const account = new Account();
313
+ let limiterOn = false;
314
+ let heartbeatTimer = 0;
315
+ let trackedSessionId = "";
316
+ let trackedTier = "";
317
+
318
+ /** @type {S2sWsRealtimeClient | null} */
319
+ let client = null;
320
+ /** @type {MediaStream | null} */
321
+ let micStream = null;
322
+ let micMuted = false;
323
+
324
+ /** @param {AppState} next */
325
+ function setState(next) {
326
+ currentState = next;
327
+ const view = STATE_VIEWS[next];
328
+ circleBtn.disabled = view.disabled;
329
+ circleBtn.className = `circle ${STATE_CLASS[next]}`;
330
+ if (next !== "error") setCaption(view.caption);
331
+
332
+ const live = LIVE_STATES.has(next);
333
+ orbWrap.classList.toggle("live", live);
334
+ micBtn.setAttribute("aria-hidden", live ? "false" : "true");
335
+ stopBtn.setAttribute("aria-hidden", live ? "false" : "true");
336
+ micBtn.tabIndex = live ? 0 : -1;
337
+ stopBtn.tabIndex = live ? 0 : -1;
338
+
339
+ updateRestartAvailability();
340
+ }
341
+
342
+ function updateRestartAvailability() {
343
+ // Restart works from any settled state — it tears down a live call (if any)
344
+ // and reconnects with the current settings. Only block while mid-connect.
345
+ restartBtn.disabled = currentState === "connecting";
346
+ restartHint.hidden = false;
347
+ restartHint.textContent = LIVE_STATES.has(currentState)
348
+ ? "Reconnects now with the settings above."
349
+ : "Starts a conversation with the settings above.";
350
+ }
351
+
352
+ /**
353
+ * @param {string} text
354
+ * @param {"" | "error" | "muted"} [kind]
355
+ */
356
+ function setCaption(text, kind = "") {
357
+ const trimmed = text.trim();
358
+ circleCaption.textContent = trimmed;
359
+ circleCaption.className = `circle-caption${kind ? ` ${kind}` : ""}${trimmed ? "" : " empty"}`;
360
+ }
361
+
362
+ function openSettings() {
363
+ syncConnectionUi();
364
+ inputVoice.value = settings.voice;
365
+ inputInstructions.value = settings.instructions;
366
+ syncGateUi();
367
+ updateRestartAvailability();
368
+ settingsModal.showModal();
369
+ }
370
+
371
+ /** dB position (clamped to the slider axis) as a 0..1 fraction of the track.
372
+ * @param {number} db */
373
+ function dbToFraction(db) {
374
+ const clamped = Math.min(GATE_MAX_DB, Math.max(GATE_OFF_DB, db));
375
+ return (clamped - GATE_OFF_DB) / (GATE_MAX_DB - GATE_OFF_DB);
376
+ }
377
+
378
+ /** @param {number} f @returns {number} dB at a 0..1 position on the gate axis. */
379
+ function fractionToDb(f) {
380
+ const clamped = Math.min(1, Math.max(0, f));
381
+ return Math.round(GATE_OFF_DB + clamped * (GATE_MAX_DB - GATE_OFF_DB));
382
+ }
383
+
384
+ // ── Radial gate arc (around the mic button, live during a call) ─────────────
385
+ // A 270° arc with the gap facing the orb (right). Fraction 0 (=Off) sits at the
386
+ // bottom-ish start; 1 (=max) at the top-ish end. The level fill and the
387
+ // threshold handle ride this same axis, mirroring the Settings widget.
388
+ const ARC_R = 40;
389
+ // A ~200° arc centred on the left (180°) so the wide gap faces the orb (right).
390
+ const ARC_SPAN_DEG = 200;
391
+ const ARC_START_DEG = 180 - ARC_SPAN_DEG / 2; // lower-left start; Off end
392
+
393
+ /** Point at fraction f (0..1) and radius r, in the 0..100 viewBox.
394
+ * @param {number} f @param {number} [r] */
395
+ function arcPoint(f, r = ARC_R) {
396
+ const deg = ARC_START_DEG + f * ARC_SPAN_DEG;
397
+ const rad = (deg * Math.PI) / 180;
398
+ return { x: 50 + r * Math.cos(rad), y: 50 + r * Math.sin(rad) };
399
+ }
400
+
401
+ /** SVG path `d` for the full 0..1 arc (clockwise). */
402
+ function fullArcD() {
403
+ const a = arcPoint(0);
404
+ const b = arcPoint(1);
405
+ const largeArc = ARC_SPAN_DEG > 180 ? 1 : 0;
406
+ return `M ${a.x} ${a.y} A ${ARC_R} ${ARC_R} 0 ${largeArc} 1 ${b.x} ${b.y}`;
407
+ }
408
+
409
+ /** One-time geometry: track, fill (dash-revealed) and the transparent hit band. */
410
+ function initGateArc() {
411
+ const d = fullArcD();
412
+ mgaTrack.setAttribute("d", d);
413
+ mgaFill.setAttribute("d", d);
414
+ mgaHit.setAttribute("d", d);
415
+ // pathLength 100 lets us reveal the fill by fraction via dashoffset.
416
+ mgaFill.setAttribute("pathLength", "100");
417
+ mgaFill.style.strokeDasharray = "100 100";
418
+ mgaFill.style.strokeDashoffset = "100"; // empty until levels arrive
419
+ renderGateHandle();
420
+ }
421
+
422
+ /** Place the threshold bead on the arc at the stored threshold; flag off state. */
423
+ function renderGateHandle() {
424
+ const off = settings.noiseGate <= GATE_OFF_DB;
425
+ const p = arcPoint(dbToFraction(settings.noiseGate));
426
+ mgaHandle.setAttribute("cx", String(p.x));
427
+ mgaHandle.setAttribute("cy", String(p.y));
428
+ micGate.classList.toggle("gate-off", off);
429
+ }
430
+
431
+ /** Paint a 0..1 live level onto the arc fill (and the Settings meter if open).
432
+ * Brightens the tick when the level crosses the threshold — i.e. the gate is
433
+ * actually open — but only when gating is enabled.
434
+ * @param {number} rms */
435
+ function paintInputLevel(rms) {
436
+ const db = rms > 0 ? 20 * Math.log10(rms) : GATE_OFF_DB;
437
+ const f = dbToFraction(db);
438
+ mgaFill.style.strokeDashoffset = String(100 * (1 - f));
439
+ if (settingsModal.open) gateMeterFill.style.width = `${f * 100}%`;
440
+ const enabled = settings.noiseGate > GATE_OFF_DB;
441
+ micGate.classList.toggle("gate-open", enabled && f >= dbToFraction(settings.noiseGate));
442
+ }
443
+
444
+ /** The single place that commits a new gate threshold: updates both controls,
445
+ * persists, and applies live to the running session.
446
+ * @param {number} db */
447
+ function setGateThreshold(db) {
448
+ settings.noiseGate = Math.min(GATE_MAX_DB, Math.max(GATE_OFF_DB, Math.round(db)));
449
+ const off = settings.noiseGate <= GATE_OFF_DB;
450
+ inputNoiseGate.value = String(settings.noiseGate);
451
+ gateValue.textContent = off ? "Off" : `${settings.noiseGate} dB`;
452
+ renderGateHandle();
453
+ localStorage.setItem(STORAGE_KEYS.noiseGate, String(settings.noiseGate));
454
+ if (client && LIVE_STATES.has(currentState)) {
455
+ client.setNoiseGate(gateParams(settings.noiseGate));
456
+ }
457
+ }
458
+
459
+ /** Reflect the stored gate threshold into the slider, label and arc handle. */
460
+ function syncGateUi() {
461
+ inputNoiseGate.value = String(settings.noiseGate);
462
+ const off = settings.noiseGate <= GATE_OFF_DB;
463
+ gateValue.textContent = off ? "Off" : `${settings.noiseGate} dB`;
464
+ renderGateHandle();
465
+ }
466
+
467
+ // Drag along the arc band to set the threshold (a tap on the glyph still mutes).
468
+ let gateDragging = false;
469
+ /** @param {PointerEvent} e */
470
+ function gatePointerToDb(e) {
471
+ const rect = mgaArc.getBoundingClientRect();
472
+ const cx = rect.left + rect.width / 2;
473
+ const cy = rect.top + rect.height / 2;
474
+ let deg = (Math.atan2(e.clientY - cy, e.clientX - cx) * 180) / Math.PI;
475
+ if (deg < 0) deg += 360;
476
+ // Map the on-arc angle to a fraction; angles in the right-side gap fall
477
+ // outside [0,1] and fractionToDb clamps them to the nearest end (just-below
478
+ // start -> Off, just-past end -> max).
479
+ const f = (deg - ARC_START_DEG) / ARC_SPAN_DEG;
480
+ return fractionToDb(f);
481
+ }
482
+ mgaHit.addEventListener("pointerdown", (e) => {
483
+ gateDragging = true;
484
+ mgaHit.setPointerCapture(e.pointerId);
485
+ setGateThreshold(gatePointerToDb(e));
486
+ });
487
+ mgaHit.addEventListener("pointermove", (e) => {
488
+ if (gateDragging) setGateThreshold(gatePointerToDb(e));
489
+ });
490
+ const endGateDrag = (/** @type {PointerEvent} */ e) => {
491
+ if (!gateDragging) return;
492
+ gateDragging = false;
493
+ try { mgaHit.releasePointerCapture(e.pointerId); } catch {}
494
+ };
495
+ mgaHit.addEventListener("pointerup", endGateDrag);
496
+ mgaHit.addEventListener("pointercancel", endGateDrag);
497
+
498
+ settingsBtn.addEventListener("click", openSettings);
499
+
500
+ // About panel: native <dialog>, Esc closes for free; also close on the X and
501
+ // on a click in the backdrop (a click whose target is the dialog itself).
502
+ aboutBtn.addEventListener("click", () => aboutModal.showModal());
503
+ // Mobile twin of the (i), living in the right-hand control cluster.
504
+ $("#about-btn-m").addEventListener("click", () => aboutModal.showModal());
505
+ aboutClose.addEventListener("click", () => aboutModal.close());
506
+ aboutModal.addEventListener("click", (e) => {
507
+ if (e.target === aboutModal) aboutModal.close();
508
+ });
509
+
510
+ // ── Tools panel ───────────────────────────────────────────────────────────
511
+
512
+ /** Reflect the current tool state into the panel controls. */
513
+ function syncToolsUi() {
514
+ const avail = searchAvailable();
515
+ toolWebSwitch.checked = toolsEnabled.web_search && avail;
516
+ toolWebSwitch.disabled = !avail;
517
+ toolWebRow.classList.toggle("disabled", !avail);
518
+ toolCamSwitch.checked = toolsEnabled.camera_snapshot;
519
+
520
+ if (serverSearchKey) {
521
+ // Key lives server-side: show it as configured, never expose it.
522
+ searchKeyInput.value = "";
523
+ searchKeyInput.placeholder = "•••••••• · provided by the server";
524
+ searchKeyInput.disabled = true;
525
+ toolWebHint.textContent = "Ready. The search key is held server-side and never sent to your browser.";
526
+ } else {
527
+ searchKeyInput.disabled = false;
528
+ searchKeyInput.value = userSearchKey;
529
+ searchKeyInput.placeholder = "Paste a Serper key to enable web search";
530
+ toolWebHint.textContent = userSearchKey
531
+ ? "Using your key — stored in this browser only."
532
+ : "No server key configured. Add your own Serper key to enable web search.";
533
+ }
534
+ }
535
+
536
+ toolsBtn.addEventListener("click", () => { syncToolsUi(); toolsModal.showModal(); });
537
+ toolsClose.addEventListener("click", () => toolsModal.close());
538
+ toolsModal.addEventListener("click", (e) => {
539
+ if (e.target === toolsModal) toolsModal.close();
540
+ });
541
+
542
+ toolWebSwitch.addEventListener("change", () => {
543
+ if (toolWebSwitch.checked && !searchAvailable()) {
544
+ toolWebSwitch.checked = false; // guard: can't enable without a key
545
+ return;
546
+ }
547
+ toolsEnabled.web_search = toolWebSwitch.checked;
548
+ saveTools();
549
+ pushToolsToSession();
550
+ });
551
+
552
+ toolCamSwitch.addEventListener("change", async () => {
553
+ if (toolCamSwitch.checked) {
554
+ try {
555
+ // Flipping the switch always re-requests the camera, so a permission that
556
+ // was only dismissed earlier is asked again here.
557
+ await enableCamera();
558
+ } catch (err) {
559
+ toolCamSwitch.checked = false;
560
+ const denied = err instanceof Error && (err.name === "NotAllowedError" || err.name === "SecurityError");
561
+ toolCamHint.textContent = denied
562
+ ? "Camera blocked. Allow it from the camera icon in your browser's address bar — it switches on automatically."
563
+ : `Camera unavailable${err instanceof Error ? `: ${err.message}` : ""}`;
564
+ return;
565
+ }
566
+ toolsEnabled.camera_snapshot = true;
567
+ toolCamHint.textContent = "Camera on. The assistant can take a snapshot when it needs to see.";
568
+ } else {
569
+ disableCamera();
570
+ toolsEnabled.camera_snapshot = false;
571
+ toolCamHint.textContent = "Let the assistant see through your webcam.";
572
+ }
573
+ saveTools();
574
+ pushToolsToSession();
575
+ });
576
+
577
+ searchKeyInput.addEventListener("input", () => {
578
+ if (serverSearchKey) return;
579
+ userSearchKey = searchKeyInput.value.trim();
580
+ if (userSearchKey) localStorage.setItem(STORAGE_KEYS.searchKey, userSearchKey);
581
+ else localStorage.removeItem(STORAGE_KEYS.searchKey);
582
+
583
+ const avail = searchAvailable();
584
+ toolWebSwitch.disabled = !avail;
585
+ toolWebRow.classList.toggle("disabled", !avail);
586
+ // Losing the key disables a previously-enabled tool.
587
+ if (!avail && toolsEnabled.web_search) {
588
+ toolsEnabled.web_search = false;
589
+ toolWebSwitch.checked = false;
590
+ saveTools();
591
+ pushToolsToSession();
592
+ }
593
+ toolWebHint.textContent = userSearchKey
594
+ ? "Using your key — stored in this browser only."
595
+ : "No server key configured. Add your own Serper key to enable web search.";
596
+ });
597
+
598
+ // ── Camera ──────────────────────────────────────────────────────────────────
599
+
600
+ async function enableCamera() {
601
+ if (cameraStream) return;
602
+ cameraStream = await navigator.mediaDevices.getUserMedia({
603
+ video: { facingMode: "user" },
604
+ audio: false,
605
+ });
606
+ camVideo.srcObject = cameraStream;
607
+ try { await camVideo.play(); } catch { /* autoplay quirks; muted video is fine */ }
608
+ camPip.classList.add("visible");
609
+ camPip.setAttribute("aria-hidden", "false");
610
+ // Lets the footer reflow to the bottom-right (and hide on mobile) while the
611
+ // webcam preview occupies the bottom of the stage.
612
+ document.body.classList.add("cam-on");
613
+ }
614
+
615
+ function disableCamera() {
616
+ if (cameraStream) {
617
+ for (const t of cameraStream.getTracks()) t.stop();
618
+ cameraStream = null;
619
+ }
620
+ camVideo.srcObject = null;
621
+ camPip.classList.remove("visible");
622
+ camPip.setAttribute("aria-hidden", "true");
623
+ document.body.classList.remove("cam-on");
624
+ }
625
+
626
+ /** Auto-start the webcam on arrival (the camera tool is on by default). If the
627
+ * user declines the permission, switch the tool off and reflect it in the UI
628
+ * rather than nagging. */
629
+ async function autoStartCamera() {
630
+ if (!toolsEnabled.camera_snapshot || cameraStream) return;
631
+ try {
632
+ await enableCamera();
633
+ } catch (err) {
634
+ console.warn("[main] camera auto-start declined/failed:", err);
635
+ toolsEnabled.camera_snapshot = false;
636
+ saveTools();
637
+ syncToolsUi();
638
+ }
639
+ }
640
+
641
+ /** Track the browser's camera permission so a later re-grant (e.g. the user
642
+ * unblocks it from the address bar after a denial) turns the camera back on
643
+ * without another toggle, and a revoke turns it off. Best-effort: the
644
+ * Permissions API doesn't support "camera" everywhere (e.g. Safari). */
645
+ async function watchCameraPermission() {
646
+ try {
647
+ const status = await navigator.permissions?.query?.({ name: /** @type {any} */ ("camera") });
648
+ if (!status) return;
649
+ status.addEventListener("change", () => {
650
+ if (status.state === "granted") {
651
+ if (!toolsEnabled.camera_snapshot) { toolsEnabled.camera_snapshot = true; saveTools(); }
652
+ void autoStartCamera();
653
+ syncToolsUi();
654
+ } else if (status.state === "denied") {
655
+ disableCamera();
656
+ if (toolsEnabled.camera_snapshot) { toolsEnabled.camera_snapshot = false; saveTools(); }
657
+ syncToolsUi();
658
+ }
659
+ });
660
+ } catch {
661
+ // Permissions API unavailable for "camera" — the toggle still re-asks.
662
+ }
663
+ }
664
+
665
+ /**
666
+ * Grab the current webcam frame as a downscaled JPEG data URL. The preview is
667
+ * mirrored in CSS for a natural self-view, but we draw the raw (un-mirrored)
668
+ * video here so the model sees the scene in its true orientation.
669
+ * @returns {string | null}
670
+ */
671
+ function captureSnapshot() {
672
+ if (!cameraStream || !camVideo.videoWidth) return null;
673
+ const vw = camVideo.videoWidth;
674
+ const vh = camVideo.videoHeight;
675
+ const scale = Math.min(1, SNAPSHOT_MAX_EDGE / Math.max(vw, vh));
676
+ const w = Math.max(1, Math.round(vw * scale));
677
+ const h = Math.max(1, Math.round(vh * scale));
678
+ const canvas = document.createElement("canvas");
679
+ canvas.width = w;
680
+ canvas.height = h;
681
+ const ctx = canvas.getContext("2d");
682
+ if (!ctx) return null;
683
+ ctx.drawImage(camVideo, 0, 0, w, h);
684
+ return canvas.toDataURL("image/jpeg", SNAPSHOT_QUALITY);
685
+ }
686
+
687
+ /** Brief shutter flash on the preview so the user sees a snapshot was taken. */
688
+ function flashPreview() {
689
+ camPip.classList.remove("flash");
690
+ void camPip.offsetWidth; // reflow so the animation restarts
691
+ camPip.classList.add("flash");
692
+ }
693
+
694
+ // ── Tool executor ─────────────────────────────────────────────────────────
695
+ // Runs the function the model called, returns the result, and asks for a
696
+ // response so the model speaks it. Errors come back as the tool output too, so
697
+ // the model can recover gracefully instead of the turn stalling.
698
+
699
+ /**
700
+ * Run the function the model called, return its result to the backend, and ask
701
+ * for a follow-up response. We also hand the result back to the caller so it
702
+ * can be shown in the conversation once the tool has actually run.
703
+ * @param {string} name @param {string} argsJson @param {string} callId
704
+ * @returns {Promise<{ output: string, image?: string }>}
705
+ */
706
+ async function runTool(name, argsJson, callId) {
707
+ if (!client) return { output: "" };
708
+ let args = /** @type {Record<string, unknown>} */ ({});
709
+ try { args = JSON.parse(argsJson || "{}"); } catch { /* keep {} */ }
710
+
711
+ if (DEBUG) console.debug(`[tool] run name=${name} callId=${JSON.stringify(callId)} args=${argsJson}`);
712
+ if (!callId) console.warn("[tool] empty call_id — the backend didn't tag the call, can't return a function_call_output");
713
+
714
+ /** @type {{ output: string, image?: string }} */
715
+ let result = { output: "" };
716
+ try {
717
+ if (name === "web_search") {
718
+ const query = typeof args.query === "string" ? args.query : "";
719
+ result.output = await execWebSearch(query);
720
+ // Return the result and let the bare response.create (below) trigger the
721
+ // spoken answer.
722
+ client.sendToolOutput(callId, result.output);
723
+ } else if (name === "camera_snapshot") {
724
+ const dataUrl = captureSnapshot();
725
+ if (dataUrl) {
726
+ if (DEBUG) console.debug(`[tool] camera_snapshot captured frame (${dataUrl.length} chars), sending image + output`);
727
+ result = { output: "Snapshot captured from the webcam and attached as an image.", image: dataUrl };
728
+ // Return the tool output; the frame itself rides along with the
729
+ // response.create below (sent right before it), so the model sees the
730
+ // snapshot in the very response it's about to speak.
731
+ client.sendToolOutput(callId, result.output);
732
+ flashPreview();
733
+ } else {
734
+ console.warn("[tool] camera_snapshot: no frame — camera off or not ready");
735
+ result.output = "The camera is not available right now.";
736
+ client.sendToolOutput(callId, result.output);
737
+ }
738
+ } else {
739
+ result.output = `Unknown tool: ${name}`;
740
+ client.sendToolOutput(callId, result.output);
741
+ }
742
+ } catch (err) {
743
+ const msg = err instanceof Error ? err.message : String(err);
744
+ result.output = `Tool failed: ${msg}`;
745
+ client.sendToolOutput(callId, result.output);
746
+ }
747
+ if (DEBUG) console.debug(`[tool] requesting model response after ${name}`);
748
+ // Camera: the captured frame rides with the response.create (sent just before
749
+ // it) so it's in context for the reply. Other tools: a bare create.
750
+ client.requestResponse(result.image ? { image: result.image } : undefined);
751
+ return result;
752
+ }
753
+
754
+ /** @param {string} query @returns {Promise<string>} */
755
+ async function execWebSearch(query) {
756
+ if (!query) return "No query provided.";
757
+ /** @type {Record<string, string>} */
758
+ const body = { query };
759
+ // Only send a user key when there's no server key (server prefers its own).
760
+ if (!serverSearchKey && userSearchKey) body.key = userSearchKey;
761
+
762
+ const res = await fetch("api/search", {
763
+ method: "POST",
764
+ headers: { "Content-Type": "application/json" },
765
+ body: JSON.stringify(body),
766
+ });
767
+ if (!res.ok) {
768
+ let detail = String(res.status);
769
+ try { const j = await res.json(); if (j.detail) detail = j.detail; } catch {}
770
+ throw new Error(`search error (${detail})`);
771
+ }
772
+ const json = await res.json();
773
+ // Date-stamp the header so the model treats these as fresh realtime facts
774
+ // rather than its (older) training knowledge.
775
+ const today = new Date().toISOString().slice(0, 10);
776
+ /** @type {string[]} */
777
+ const lines = [`Google search result from ${today}:`];
778
+ if (json.answer) lines.push(`Answer: ${json.answer}`);
779
+ for (const r of json.results || []) {
780
+ lines.push(`- ${r.title}: ${r.snippet} (${r.url})`);
781
+ }
782
+ return lines.length > 1 ? lines.join("\n") : `${lines[0]}\nNo results found.`;
783
+ }
784
+
785
+ /** Learn server config (search key + connection target), then refresh the UI. */
786
+ async function fetchConfig() {
787
+ try {
788
+ const res = await fetch("api/config");
789
+ if (res.ok) {
790
+ const json = await res.json();
791
+ serverSearchKey = !!json.search;
792
+ lbMode = !!json.lb;
793
+ // Lock to LB mode only when the deploy reports a load balancer.
794
+ allowDirect = json.allowDirect ?? !lbMode;
795
+ // The conversation-time limiter rides on the LB being present.
796
+ limiterOn = lbMode;
797
+ }
798
+ // Non-OK response: leave the fail-open default (allowDirect = true).
799
+ } catch {
800
+ // Config endpoint unreachable (e.g. static hosting): keep direct entry.
801
+ }
802
+ if (DEBUG) console.debug(`[ui] config: allowDirect=${allowDirect} lbMode=${lbMode}`);
803
+ // Login chip + remaining-budget (no-op / hidden when the limiter is off).
804
+ void account.refresh();
805
+ syncToolsUi();
806
+ syncConnectionUi();
807
+ }
808
+
809
+ /**
810
+ * Resolve where to connect, per the deploy's mode:
811
+ * • LB mode -> `{ sessionUrl }`, the client POSTs the same-origin /api/session
812
+ * proxy and the server forwards to the LB (its address stays server-side).
813
+ * • direct -> `{ directUrl }`, connect straight to the s2s WebSocket.
814
+ * Throws a user-facing error if direct mode is on but no URL was entered.
815
+ * @returns {{ sessionUrl: string } | { directUrl: string }}
816
+ */
817
+ function connectionTarget() {
818
+ if (!allowDirect) {
819
+ return { sessionUrl: "api/session" };
820
+ }
821
+ const directUrl = buildDirectWsUrl(settings.directUrl);
822
+ if (!directUrl) {
823
+ throw new Error("Enter a speech-to-speech server URL in Settings.");
824
+ }
825
+ return { directUrl };
826
+ }
827
+
828
+ /**
829
+ * Normalise a user-typed server address into a realtime WebSocket URL.
830
+ * Accepts bare hosts (`localhost:8080`), http(s) URLs, or ws(s) URLs, and adds
831
+ * the `/v1/realtime` path when none is given. A full connect URL (with path
832
+ * and/or query) is preserved as-is.
833
+ * @param {string} raw @returns {string}
834
+ */
835
+ function buildDirectWsUrl(raw) {
836
+ let s = (raw || "").trim();
837
+ if (!s) return "";
838
+ if (!/^wss?:\/\//i.test(s)) {
839
+ if (/^https?:\/\//i.test(s)) {
840
+ s = s.replace(/^http/i, "ws"); // http→ws, https→wss
841
+ } else {
842
+ const isLocal = /^(localhost|127\.0\.0\.1|\[::1\])(:|\/|$)/i.test(s);
843
+ s = (isLocal ? "ws://" : "wss://") + s;
844
+ }
845
+ }
846
+ try {
847
+ const u = new URL(s);
848
+ if (u.pathname === "" || u.pathname === "/") u.pathname = "/v1/realtime";
849
+ return u.toString();
850
+ } catch {
851
+ return s;
852
+ }
853
+ }
854
+
855
+ /** Create + resume an AudioContext synchronously (must run inside the user
856
+ * gesture so iOS lets it start). Returns null if construction fails. */
857
+ function createResumedAudioContext() {
858
+ try {
859
+ const Ctx = window.AudioContext || /** @type {any} */ (window).webkitAudioContext;
860
+ const ctx = new Ctx({ latencyHint: "interactive" });
861
+ if (ctx.state === "suspended") void ctx.resume().catch(() => {});
862
+ return /** @type {AudioContext} */ (ctx);
863
+ } catch (err) {
864
+ console.warn("[main] AudioContext init failed:", err);
865
+ return null;
866
+ }
867
+ }
868
+
869
+ /** Read the editable settings out of the form. The URL field is only honoured
870
+ * in direct mode (in LB mode it's locked and server-owned). */
871
+ function readSettingsFromForm() {
872
+ return {
873
+ directUrl: allowDirect ? inputLbUrl.value.trim() : settings.directUrl,
874
+ voice: inputVoice.value || DEFAULT_VOICE,
875
+ instructions: inputInstructions.value.trim() || DEFAULT_INSTRUCTIONS,
876
+ noiseGate: readGateThreshold(),
877
+ };
878
+ }
879
+
880
+ /** Gate threshold (dBFS) currently shown on the slider, clamped to range. */
881
+ function readGateThreshold() {
882
+ const v = Math.round(Number(inputNoiseGate.value));
883
+ if (!Number.isFinite(v)) return GATE_OFF_DB;
884
+ return Math.min(GATE_MAX_DB, Math.max(GATE_OFF_DB, v));
885
+ }
886
+
887
+ /** Adapt the connection field to the mode learned from /api/config. */
888
+ function syncConnectionUi() {
889
+ if (allowDirect) {
890
+ // Direct mode: the user sets their own s2s server URL.
891
+ connField.hidden = false;
892
+ inputLbUrl.value = settings.directUrl;
893
+ inputLbUrl.placeholder = "http://localhost:port";
894
+ connHint.classList.remove("error");
895
+ connHint.textContent =
896
+ "URL of your speech-to-speech server, e.g. http://localhost:8080 (the app adds /v1/realtime).";
897
+ } else {
898
+ // LB mode: the load balancer URL is deployment-owned — hide it entirely so
899
+ // its address is never exposed in Settings.
900
+ connField.hidden = true;
901
+ }
902
+ }
903
+
904
+ /** True when the user must supply a server URL before connecting (direct mode
905
+ * with nothing set). */
906
+ function missingServerUrl() {
907
+ return allowDirect && !buildDirectWsUrl(settings.directUrl);
908
+ }
909
+
910
+ /** Open Settings and point the user at the empty server-URL field. */
911
+ function promptServerUrl() {
912
+ if (settingsModal.open) syncConnectionUi();
913
+ else openSettings();
914
+ connHint.textContent = "Set the speech-to-speech server URL to start.";
915
+ connHint.classList.add("error");
916
+ inputLbUrl.focus();
917
+ }
918
+
919
+ settingsForm.addEventListener("submit", (event) => {
920
+ const submitter = /** @type {HTMLButtonElement | null} */ ((/** @type {SubmitEvent} */ (event)).submitter);
921
+ if (submitter?.value !== "save") return;
922
+
923
+ settings = readSettingsFromForm();
924
+ saveSettings(settings);
925
+
926
+ // Voice + instructions can apply to a live session without reconnecting; a
927
+ // changed connection URL only takes effect on the next restart.
928
+ if (client && LIVE_STATES.has(currentState)) {
929
+ client.updateSession({ voice: settings.voice, instructions: effectiveInstructions() });
930
+ }
931
+ });
932
+
933
+ // The noise gate applies live (worklet param), so tune it without a restart:
934
+ // update the label/marker, persist, and push straight to the running client.
935
+ inputNoiseGate.addEventListener("input", () => {
936
+ setGateThreshold(readGateThreshold());
937
+ });
938
+
939
+ restartBtn.addEventListener("click", async () => {
940
+ if (currentState === "connecting") return; // a connect is already underway
941
+ settings = readSettingsFromForm();
942
+ saveSettings(settings);
943
+ if (missingServerUrl()) { promptServerUrl(); return; } // keep settings open
944
+ settingsModal.close();
945
+ // Grab the AudioContext NOW, inside the click gesture — teardown() awaits, and
946
+ // creating it afterwards would fall outside the gesture (silent on iOS).
947
+ const audioContext = createResumedAudioContext();
948
+ try {
949
+ if (client) await teardown();
950
+ await doStart(audioContext);
951
+ } catch (err) {
952
+ await handleStartError(err);
953
+ }
954
+ });
955
+
956
+ circleBtn.addEventListener("click", async () => {
957
+ try {
958
+ if (currentState === "idle" || currentState === "error") {
959
+ if (missingServerUrl()) { promptServerUrl(); return; }
960
+ await doStart();
961
+ }
962
+ } catch (err) {
963
+ await handleStartError(err);
964
+ }
965
+ });
966
+
967
+ /** A failed start is either the daily limit (show the modal, return to idle) or
968
+ * a real fault (surface it). doStart already closed any orphan AudioContext.
969
+ * @param {any} err */
970
+ async function handleStartError(err) {
971
+ if (err && err.code === "limit") {
972
+ await teardown();
973
+ account.showLimit(err.tier);
974
+ return;
975
+ }
976
+ onFatalError(err);
977
+ }
978
+
979
+ micBtn.addEventListener("click", () => {
980
+ if (!micStream || !client) return;
981
+ micMuted = !micMuted;
982
+ for (const track of micStream.getAudioTracks()) {
983
+ track.enabled = !micMuted;
984
+ }
985
+ client.setMuted(micMuted);
986
+ micBtn.classList.toggle("muted", micMuted);
987
+ micBtn.setAttribute("aria-label", micMuted ? "Unmute" : "Mute");
988
+ micBtn.title = micMuted ? "Unmute" : "Mute";
989
+ });
990
+
991
+ stopBtn.addEventListener("click", async () => {
992
+ await teardown();
993
+ });
994
+
995
+ /**
996
+ * Start a conversation. Pass a pre-created AudioContext when the caller already
997
+ * made one inside the tap/click gesture (required on iOS); otherwise one is
998
+ * created here, which is still inside the gesture for a direct orb tap.
999
+ * @param {AudioContext | null} [audioContext]
1000
+ */
1001
+ async function doStart(audioContext = null) {
1002
+ // Resolve the target before touching mic/audio so a misconfiguration (e.g.
1003
+ // direct mode with no URL) fails fast with a clear message.
1004
+ const target = connectionTarget();
1005
+
1006
+ chat.clear();
1007
+ chat.reset();
1008
+ setState("connecting");
1009
+ setCaption("Asking for mic…", "muted");
1010
+
1011
+ // Create + resume the AudioContext SYNCHRONOUSLY, still inside the gesture.
1012
+ // iOS Safari only starts an AudioContext from a user gesture; if we waited
1013
+ // until after the getUserMedia / session-creation awaits below, it would stay
1014
+ // suspended and the whole pipeline would be silent.
1015
+ if (!audioContext) audioContext = createResumedAudioContext();
1016
+
1017
+ try {
1018
+ micStream = await navigator.mediaDevices.getUserMedia({
1019
+ audio: {
1020
+ echoCancellation: true,
1021
+ noiseSuppression: true,
1022
+ autoGainControl: true,
1023
+ },
1024
+ });
1025
+ } catch (err) {
1026
+ if (audioContext) void audioContext.close().catch(() => {});
1027
+ throw new Error(
1028
+ `Microphone access denied${err instanceof Error ? `: ${err.message}` : ""}`,
1029
+ );
1030
+ }
1031
+
1032
+ // The webcam is started on arrival (autoStartCamera), so nothing to do here;
1033
+ // a still-pending grant just means the snapshot tool isn't ready yet.
1034
+
1035
+ const c = new S2sWsRealtimeClient({
1036
+ ...target,
1037
+ voice: settings.voice,
1038
+ instructions: effectiveInstructions(),
1039
+ micStream,
1040
+ tools: activeToolDefs(),
1041
+ noiseGate: gateParams(settings.noiseGate),
1042
+ ...(audioContext ? { audioContext } : {}),
1043
+ });
1044
+ client = c;
1045
+
1046
+ c.addEventListener("status", (e) => {
1047
+ const detail = /** @type {CustomEvent<{ status: string }>} */ (e).detail;
1048
+ onClientStatus(detail.status);
1049
+ });
1050
+ c.addEventListener("transcript", (e) => {
1051
+ const d = /** @type {CustomEvent<{ role: "user" | "assistant"; text: string; partial: boolean; itemId?: string; responseId?: string }>} */ (e).detail;
1052
+ chat.onTranscript(d);
1053
+ });
1054
+
1055
+ c.addEventListener("response-finished", (e) => {
1056
+ const detail = /** @type {CustomEvent<{ responseId: string; status: string; audible?: boolean; transcript?: string }>} */ (e).detail;
1057
+ chat.onResponseFinished(detail);
1058
+ });
1059
+
1060
+ c.addEventListener("toolcall", (e) => {
1061
+ const { name, arguments: args, callId } = /** @type {CustomEvent<{ name: string; arguments: string; callId: string }>} */ (e).detail;
1062
+ chat.onToolCall(name);
1063
+ // Execute the tool, then push it to the conversation once the result is in,
1064
+ // so the toggle shows both the call input and its output together.
1065
+ void runTool(name, args, callId).then(({ output, image }) => {
1066
+ chat.onToolResult(name, args, output, image);
1067
+ });
1068
+ });
1069
+ c.addEventListener("error", (e) => {
1070
+ const detail = /** @type {CustomEvent<{ error: unknown }>} */ (e).detail;
1071
+ onFatalError(detail.error);
1072
+ });
1073
+ c.addEventListener("server-error", (e) => {
1074
+ // Non-fatal: the backend reported an error mid-session. Log it, keep the
1075
+ // socket and the conversation alive (the model can recover on its own).
1076
+ const detail = /** @type {CustomEvent<{ error: unknown }>} */ (e).detail;
1077
+ const msg = detail.error instanceof Error ? detail.error.message : String(detail.error);
1078
+ console.warn("[main] server error (non-fatal):", msg);
1079
+ });
1080
+ c.addEventListener("session", (e) => {
1081
+ const info = /** @type {CustomEvent<{ info: import("./ws/s2s-ws-client.js").WsSessionInfo }>} */ (e).detail.info;
1082
+ console.log("[ws] session created:", info.sessionId);
1083
+ // A metered tier (anon / free): heartbeat so the server can extend the
1084
+ // reservation and tell us when the daily budget runs out. PRO isn't limited.
1085
+ if (info.limited && info.sessionId) {
1086
+ trackedSessionId = info.sessionId;
1087
+ trackedTier = info.tier || "anon";
1088
+ startHeartbeat(info.heartbeatSec || 5);
1089
+ }
1090
+ });
1091
+ c.addEventListener("input-level", (e) => {
1092
+ const { rms } = /** @type {CustomEvent<{ rms: number }>} */ (e).detail;
1093
+ paintInputLevel(rms);
1094
+ });
1095
+
1096
+ try {
1097
+ await c.connect();
1098
+ } catch (err) {
1099
+ // The grant can be refused (402 → limit) or the dial can fail. In LB mode
1100
+ // the AudioContext hasn't been adopted by the client yet (the session POST
1101
+ // runs first), so close the one we created here to avoid leaking it.
1102
+ if (audioContext) void audioContext.close().catch(() => {});
1103
+ throw err;
1104
+ }
1105
+ }
1106
+
1107
+ // ── Conversation-time heartbeat ─────────────────────────────────────────────
1108
+
1109
+ /** Ping the server every `sec` seconds so it can meter the live session; when
1110
+ * it reports the daily budget is spent, cut the call and show the limit modal.
1111
+ * @param {number} sec */
1112
+ function startHeartbeat(sec) {
1113
+ stopHeartbeat();
1114
+ heartbeatTimer = window.setInterval(async () => {
1115
+ if (!trackedSessionId) return;
1116
+ try {
1117
+ const res = await fetch("api/session/heartbeat", {
1118
+ method: "POST",
1119
+ headers: { "Content-Type": "application/json" },
1120
+ body: JSON.stringify({ sessionId: trackedSessionId }),
1121
+ keepalive: true,
1122
+ });
1123
+ const json = await res.json().catch(() => ({}));
1124
+ if (json.expired) await onLimitReached();
1125
+ } catch (err) {
1126
+ // A transient network blip shouldn't kill the call; the next tick retries.
1127
+ if (DEBUG) console.debug("[ui] heartbeat failed:", err);
1128
+ }
1129
+ }, Math.max(1, sec) * 1000);
1130
+ }
1131
+
1132
+ function stopHeartbeat() {
1133
+ if (heartbeatTimer) {
1134
+ clearInterval(heartbeatTimer);
1135
+ heartbeatTimer = 0;
1136
+ }
1137
+ }
1138
+
1139
+ /** The server cut the live session: tear down and explain why. */
1140
+ async function onLimitReached() {
1141
+ const tier = trackedTier;
1142
+ stopHeartbeat();
1143
+ await teardown();
1144
+ account.showLimit(tier);
1145
+ }
1146
+
1147
+ /** Tell the server a session ended so it reconciles + refunds the unused chunk.
1148
+ * Uses sendBeacon so it still fires when the tab is closing. */
1149
+ function endTrackedSession() {
1150
+ if (!trackedSessionId) return;
1151
+ const body = JSON.stringify({ sessionId: trackedSessionId });
1152
+ try {
1153
+ const blob = new Blob([body], { type: "application/json" });
1154
+ if (!navigator.sendBeacon("api/session/end", blob)) {
1155
+ void fetch("api/session/end", {
1156
+ method: "POST", headers: { "Content-Type": "application/json" }, body, keepalive: true,
1157
+ }).catch(() => {});
1158
+ }
1159
+ } catch {
1160
+ // Best-effort; the server sweep reaps the session anyway.
1161
+ }
1162
+ trackedSessionId = "";
1163
+ trackedTier = "";
1164
+ }
1165
+
1166
+ /** @param {string} status */
1167
+ function onClientStatus(status) {
1168
+ switch (status) {
1169
+ case "creating-session":
1170
+ case "connecting":
1171
+ setState("connecting");
1172
+ break;
1173
+ case "connected":
1174
+ setState("listening");
1175
+ break;
1176
+ case "user-speaking":
1177
+ setState("user-speaking");
1178
+ break;
1179
+ case "processing":
1180
+ setState("processing");
1181
+ break;
1182
+ case "ai-speaking":
1183
+ setState("ai-speaking");
1184
+ break;
1185
+ case "closed":
1186
+ // teardown() will move us to idle
1187
+ break;
1188
+ case "error":
1189
+ setState("error");
1190
+ break;
1191
+ }
1192
+ }
1193
+
1194
+ async function teardown() {
1195
+ stopHeartbeat();
1196
+ endTrackedSession();
1197
+ chat.reset({ dismiss: true });
1198
+ if (client) {
1199
+ try {
1200
+ await client.close();
1201
+ } catch (err) {
1202
+ console.warn("[main] error closing client:", err);
1203
+ }
1204
+ client = null;
1205
+ }
1206
+ if (micStream) {
1207
+ for (const track of micStream.getTracks()) track.stop();
1208
+ micStream = null;
1209
+ }
1210
+ // The webcam is independent of the call lifecycle (it runs while the user is
1211
+ // on the page), so we leave it on here — only the camera toggle stops it.
1212
+ micMuted = false;
1213
+ micBtn.classList.remove("muted");
1214
+ setState("idle");
1215
+ // Refresh the chip's remaining-today after the budget moved.
1216
+ if (limiterOn) void account.refresh();
1217
+ }
1218
+
1219
+ /** @param {unknown} err */
1220
+ function onFatalError(err) {
1221
+ console.error("[main] fatal:", err);
1222
+ setState("error");
1223
+ const message = err instanceof Error ? err.message : String(err);
1224
+ setCaption(truncateError(message), "error");
1225
+ void teardown().catch(() => {
1226
+ setState("error");
1227
+ setCaption(truncateError(message), "error");
1228
+ });
1229
+ }
1230
+
1231
+ setState("idle");
1232
+ chat.renderEmptyState();
1233
+ initGateArc();
1234
+ void fetchConfig();
1235
+ // Start the webcam as soon as the user lands (camera tool defaults on), and
1236
+ // react to later permission changes (re-grant after a denial re-enables it).
1237
+ void autoStartCamera();
1238
+ void watchCameraPermission();
1239
+
1240
+ // Reconcile a live session if the tab is closed/hidden mid-call (no teardown).
1241
+ window.addEventListener("pagehide", () => endTrackedSession());
1242
+
1243
+ requestAnimationFrame(() => {
1244
+ document.body.classList.remove("booting");
1245
+ });
requirements.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ fastapi==0.115.6
2
+ uvicorn[standard]==0.34.0
3
+ httpx==0.28.1
4
+ # [oauth] extra pulls authlib + itsdangerous for attach_huggingface_oauth.
5
+ huggingface_hub[oauth]>=0.30,<1.0
server.py ADDED
@@ -0,0 +1,292 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Tiny server for the speech-to-speech demo.
3
+
4
+ The demo used to ship as a `sdk: static` Space, but the web-search tool needs a
5
+ search key the browser must NOT see. A static Space has no runtime process, so it
6
+ can't hold a secret the front-end uses. This server fixes that: it serves the
7
+ unchanged front-end AND exposes a same-origin `/api/search` proxy that holds the
8
+ Serper key server-side (see docs/adr/0001).
9
+
10
+ Everything lives in one container; the speech-to-speech backend stays a separate,
11
+ load-balanced service the browser talks to over WebSocket as before. The load
12
+ balancer's address is a secret too (like the Serper key): the browser never sees
13
+ it. `/api/session` proxies the session handshake server-side so only the
14
+ per-session compute URL the LB hands back (which the browser must dial) is exposed.
15
+
16
+ On the deployed Space the server also meters conversation time by HF login tier
17
+ (anonymous / signed-in / PRO) — see `limiter.py` and `auth.py`. That whole feature
18
+ is off unless BOTH `LOAD_BALANCER_URL` and `SPACE_ID` are set, so it runs only on
19
+ the live Space, never locally (even with the LB exported for testing).
20
+
21
+ Endpoints:
22
+ GET /api/config -> { search, lb, allowDirect, auth }
23
+ GET /api/me -> login + tier + remaining budget (LB mode only)
24
+ POST /api/search -> { results, answer } Google via Serper.dev
25
+ POST /api/session -> proxies <LB>/session, reserves the first time chunk
26
+ POST /api/session/heartbeat-> extend the reservation; { expired }
27
+ POST /api/session/end -> reconcile + refund (sendBeacon on teardown)
28
+ /* -> static files (index.html, main.js, ...)
29
+ """
30
+
31
+ import asyncio
32
+ import logging
33
+ import os
34
+
35
+ import httpx
36
+ from fastapi import FastAPI, HTTPException, Request
37
+ from fastapi.responses import JSONResponse
38
+ from fastapi.staticfiles import StaticFiles
39
+ from pydantic import BaseModel
40
+
41
+ import auth
42
+ import limiter
43
+
44
+ logger = logging.getLogger("s2s.search")
45
+
46
+ SERPER_KEY = os.environ.get("SERPER_API_KEY", "").strip()
47
+ # Speech-to-speech load balancer URL. When set, the browser POSTs /api/session
48
+ # (which proxies <lb>/session here, server-side) and connects to the URL the LB
49
+ # returns (the original flow). The LB address itself is never sent to the browser.
50
+ # When empty, the user may instead set a direct s2s server URL in Settings and the
51
+ # browser connects to it straight (no load balancer).
52
+ LOAD_BALANCER_URL = os.environ.get("LOAD_BALANCER_URL", "").strip()
53
+ # HF injects SPACE_ID ("owner/space") into every Space runtime; it's absent
54
+ # locally and on a plain `docker run`. We meter conversation time ONLY on the
55
+ # deployed Space — i.e. when BOTH the LB is configured AND we're on a Space.
56
+ # Off-Space (local dev, even with the LB exported) the app still proxies the LB,
57
+ # but nothing is metered: no budget, no reservations, no sign-in gating.
58
+ SPACE_ID = os.environ.get("SPACE_ID", "").strip()
59
+ LIMITER_ENABLED = bool(LOAD_BALANCER_URL) and bool(SPACE_ID)
60
+ SERPER_URL = "https://google.serper.dev/search"
61
+ # Cap results so the tool output stays small enough to feed back to the model.
62
+ MAX_RESULTS = 5
63
+ HERE = os.path.dirname(os.path.abspath(__file__))
64
+
65
+ app = FastAPI(title="s2s-demo")
66
+
67
+ # Wire HF OAuth before the app serves (no-op unless the OAuth env is present).
68
+ # Sign-in only matters when we're metering (prod Space), so gate it on that.
69
+ AUTH_ENABLED = LIMITER_ENABLED and auth.attach(app)
70
+
71
+
72
+ @app.on_event("startup")
73
+ async def _startup():
74
+ """Stand up the usage DB and a periodic sweeper — metered (prod Space) only."""
75
+ if not LIMITER_ENABLED:
76
+ return
77
+ limiter.init()
78
+ asyncio.create_task(_sweeper())
79
+
80
+
81
+ async def _sweeper():
82
+ while True:
83
+ await asyncio.sleep(limiter.REAP_AFTER_SEC)
84
+ try:
85
+ await asyncio.to_thread(limiter.sweep)
86
+ except Exception as exc: # pragma: no cover - defensive
87
+ logger.warning("usage sweep failed: %r", exc)
88
+
89
+
90
+ class SearchRequest(BaseModel):
91
+ query: str
92
+ # Optional user-supplied key (fallback when the deploy has no server key).
93
+ # Used for this request only; never stored.
94
+ key: str | None = None
95
+
96
+
97
+ @app.get("/api/config")
98
+ def config():
99
+ """Client bootstrap: whether web search is available, whether the deploy runs
100
+ behind a load balancer (so the browser uses the /api/session proxy + limiter),
101
+ whether HF sign-in is available, and whether the user may instead set a direct
102
+ s2s server URL. The LB address itself is intentionally NOT included."""
103
+ return {
104
+ "search": bool(SERPER_KEY),
105
+ "lb": bool(LOAD_BALANCER_URL),
106
+ "allowDirect": not LOAD_BALANCER_URL,
107
+ "auth": AUTH_ENABLED,
108
+ }
109
+
110
+
111
+ @app.get("/api/me")
112
+ async def me(request: Request):
113
+ """Login state, tier, and remaining daily budget. Only meaningful in LB mode;
114
+ sets the anonymous tracking cookie when first seen."""
115
+ if not LIMITER_ENABLED:
116
+ return {"enabled": False}
117
+ view = auth.user_view(request)
118
+ tier, keys, set_cookie = auth.resolve_identity(request)
119
+ unlimited = limiter.budget_for(tier) is None
120
+ rem = None if unlimited else await asyncio.to_thread(limiter.remaining, keys, tier)
121
+ out = {
122
+ "enabled": True,
123
+ "auth": AUTH_ENABLED,
124
+ **view,
125
+ "remainingSec": rem,
126
+ "limitSec": limiter.budget_for(tier),
127
+ "loginUrl": auth.OAUTH_LOGIN_PATH if AUTH_ENABLED else None,
128
+ "logoutUrl": auth.OAUTH_LOGOUT_PATH if AUTH_ENABLED else None,
129
+ }
130
+ resp = JSONResponse(out)
131
+ if set_cookie:
132
+ auth.set_anon_cookie(resp, set_cookie)
133
+ return resp
134
+
135
+
136
+ @app.post("/api/search")
137
+ async def search(req: SearchRequest):
138
+ """Proxy a Google search via Serper.dev. The key stays on the server unless
139
+ the user brought their own (then theirs is used for this request only)."""
140
+ query = (req.query or "").strip()
141
+ if not query:
142
+ raise HTTPException(status_code=400, detail="Empty query.")
143
+
144
+ key = (req.key or "").strip() or SERPER_KEY
145
+ if not key:
146
+ # No server key and the user didn't supply one — search is unavailable.
147
+ raise HTTPException(status_code=503, detail="Search is not configured.")
148
+
149
+ headers = {"X-API-KEY": key, "Content-Type": "application/json"}
150
+ payload = {"q": query, "num": MAX_RESULTS}
151
+ try:
152
+ async with httpx.AsyncClient(timeout=12.0) as http:
153
+ resp = await http.post(SERPER_URL, headers=headers, json=payload)
154
+ except httpx.RequestError as exc:
155
+ logger.warning("Serper unreachable: %r", exc)
156
+ raise HTTPException(status_code=502, detail="Search provider unreachable.")
157
+
158
+ if resp.status_code != 200:
159
+ # Serper's error body carries the real reason (e.g. "Not enough
160
+ # credits") and contains no key, so it's safe to log and relay.
161
+ body = resp.text[:300]
162
+ logger.warning("Serper error %s: %s", resp.status_code, body)
163
+ msg = None
164
+ try:
165
+ msg = resp.json().get("message")
166
+ except Exception:
167
+ pass
168
+ detail = f"Search provider error ({resp.status_code})"
169
+ if msg:
170
+ detail += f": {msg}"
171
+ raise HTTPException(status_code=502, detail=detail)
172
+
173
+ data = resp.json()
174
+ results = []
175
+ for item in (data.get("organic") or [])[:MAX_RESULTS]:
176
+ results.append(
177
+ {
178
+ "title": item.get("title", ""),
179
+ "snippet": item.get("snippet", ""),
180
+ "url": item.get("link", ""),
181
+ }
182
+ )
183
+
184
+ # A direct answer when Google has one — saves the model a hop.
185
+ box = data.get("answerBox") or {}
186
+ answer = box.get("answer") or box.get("snippet") or None
187
+ if not answer:
188
+ kg = data.get("knowledgeGraph") or {}
189
+ answer = kg.get("description") or None
190
+
191
+ return JSONResponse({"query": query, "answer": answer, "results": results})
192
+
193
+
194
+ @app.post("/api/session")
195
+ async def session(request: Request):
196
+ """Proxy the session handshake to the load balancer, keeping its URL secret,
197
+ and meter conversation time by tier.
198
+
199
+ The browser POSTs here (same-origin); we resolve the caller's tier, refuse if
200
+ today's budget is already spent (402), otherwise POST <LOAD_BALANCER_URL>/session
201
+ and relay the JSON back. The LB body carries a per-session `connect_url`
202
+ (compute host + short-lived token) the browser must dial directly — that one
203
+ URL is unavoidably exposed, but the stable load-balancer address is not. On a
204
+ successful grant we reserve the first time chunk against the day's budget."""
205
+ if not LOAD_BALANCER_URL:
206
+ # No LB configured — this deploy is direct-mode only; the browser should
207
+ # never call this. 404 so it's indistinguishable from a missing route.
208
+ raise HTTPException(status_code=404, detail="Not found.")
209
+
210
+ tier, keys, set_cookie = auth.resolve_identity(request)
211
+ # Metering runs only on the deployed Space; off-Space the LB still proxies but
212
+ # nothing is tracked. Within metering, unlimited tiers (pro, org) aren't either.
213
+ tracked = LIMITER_ENABLED and limiter.budget_for(tier) is not None
214
+
215
+ # Refuse before troubling the LB if the day's budget is already gone.
216
+ if tracked:
217
+ rem = await asyncio.to_thread(limiter.remaining, keys, tier)
218
+ if rem is not None and rem <= 0:
219
+ resp = JSONResponse(
220
+ {"tier": tier, "reason": "limit", "remainingSec": 0}, status_code=402
221
+ )
222
+ if set_cookie:
223
+ auth.set_anon_cookie(resp, set_cookie)
224
+ return resp
225
+
226
+ url = f"{LOAD_BALANCER_URL.rstrip('/')}/session"
227
+ try:
228
+ async with httpx.AsyncClient(timeout=15.0) as http:
229
+ lb = await http.post(url, headers={"Content-Type": "application/json"}, content="{}")
230
+ except httpx.RequestError as exc:
231
+ logger.warning("Load balancer unreachable: %r", exc)
232
+ raise HTTPException(status_code=502, detail="Speech service unreachable.")
233
+
234
+ if lb.status_code != 200:
235
+ # The LB's error body may name the reason (e.g. capacity); it carries no
236
+ # secret, so relay a trimmed copy.
237
+ logger.warning("Session handshake failed %s: %s", lb.status_code, lb.text[:300])
238
+ raise HTTPException(status_code=502, detail=f"Session handshake failed ({lb.status_code}).")
239
+
240
+ data = lb.json()
241
+ remaining = None
242
+ if tracked and data.get("session_id"):
243
+ await asyncio.to_thread(limiter.begin, data["session_id"], keys, tier)
244
+ remaining = await asyncio.to_thread(limiter.remaining, keys, tier)
245
+
246
+ data.update({
247
+ "tier": tier,
248
+ "limited": tracked,
249
+ "remainingSec": remaining,
250
+ "heartbeatSec": limiter.HEARTBEAT_SEC,
251
+ })
252
+ resp = JSONResponse(data)
253
+ if set_cookie:
254
+ auth.set_anon_cookie(resp, set_cookie)
255
+ return resp
256
+
257
+
258
+ async def _session_id(request: Request) -> str:
259
+ """Pull `sessionId` from a JSON body, tolerating sendBeacon's blob posts."""
260
+ try:
261
+ data = await request.json()
262
+ except Exception:
263
+ return ""
264
+ return (data or {}).get("sessionId", "") if isinstance(data, dict) else ""
265
+
266
+
267
+ @app.post("/api/session/heartbeat")
268
+ async def session_heartbeat(request: Request):
269
+ """Extend the live reservation one chunk at a time. `expired` once the day's
270
+ budget is spent — the client then tears down."""
271
+ if not LIMITER_ENABLED:
272
+ raise HTTPException(status_code=404, detail="Not found.")
273
+ sid = await _session_id(request)
274
+ alive = bool(sid) and await asyncio.to_thread(limiter.heartbeat, sid)
275
+ return {"expired": not alive}
276
+
277
+
278
+ @app.post("/api/session/end")
279
+ async def session_end(request: Request):
280
+ """Clean teardown: reconcile to real elapsed time and refund the unused
281
+ chunk. Sent via navigator.sendBeacon, so it must succeed without a response."""
282
+ if not LIMITER_ENABLED:
283
+ raise HTTPException(status_code=404, detail="Not found.")
284
+ sid = await _session_id(request)
285
+ if sid:
286
+ await asyncio.to_thread(limiter.end, sid)
287
+ return {"ok": True}
288
+
289
+
290
+ # Static front-end. Registered last so the /api routes win. `html=True` serves
291
+ # index.html at "/". The repo is public anyway, so serving the dir is fine.
292
+ app.mount("/", StaticFiles(directory=HERE, html=True), name="static")
style.css ADDED
@@ -0,0 +1,2509 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ :root {
2
+ --bg: #0a0b10;
3
+ --bg-elev: #13151c;
4
+ --bg-elev-2: #1b1e29;
5
+ --border: rgba(255, 255, 255, 0.08);
6
+ --border-strong: rgba(255, 255, 255, 0.16);
7
+ --text: #f5f6fa;
8
+ --text-dim: rgba(245, 246, 250, 0.65);
9
+ --text-faint: rgba(245, 246, 250, 0.42);
10
+
11
+ --accent: #8b7dff;
12
+ --accent-2: #22d3ee;
13
+ --listening: #22d3ee;
14
+ --speaking: #8b7dff;
15
+ --processing: #f59e0b;
16
+ --error: #ff6a75;
17
+ --success: #34d399;
18
+
19
+ /* Mono is the "machine voice": reserved for system text the app emits —
20
+ * the orb's status caption, tool calls, the transport tag. Body/UI stays
21
+ * Inter. */
22
+ --font-mono: "Geist Mono", ui-monospace, SFMono-Regular, "SF Mono", Menlo, monospace;
23
+
24
+ /* Thesis: color belongs to the voice. The chrome is monochrome; saturated
25
+ * hue appears only on the orb and on these tiny role echoes, which mirror
26
+ * the orb's own state colors (you listening = cyan, assistant = violet,
27
+ * tool = amber) so the transcript reads in the same color language. */
28
+ --voice-user: var(--accent-2);
29
+ --voice-assistant: var(--accent);
30
+ --voice-tool: var(--processing);
31
+
32
+ /* Smoothed mic RMS in [0..1], updated every frame from JS while a session
33
+ * is active. Used by audio-reactive circle states. */
34
+ --audio-level: 0;
35
+ /* Five log-spaced frequency bands extracted from the mic analyser;
36
+ * drive each bar's height independently for a real "spectrum" feel. */
37
+ --bar0: 0;
38
+ --bar1: 0;
39
+ --bar2: 0;
40
+ --bar3: 0;
41
+ --bar4: 0;
42
+
43
+ --radius-sm: 8px;
44
+ --radius-md: 14px;
45
+ --radius-lg: 22px;
46
+
47
+ --shadow-soft: 0 10px 40px rgba(0, 0, 0, 0.35);
48
+
49
+ color-scheme: dark;
50
+ }
51
+
52
+ * {
53
+ box-sizing: border-box;
54
+ }
55
+
56
+ html,
57
+ body {
58
+ margin: 0;
59
+ padding: 0;
60
+ height: 100%;
61
+ }
62
+
63
+ body {
64
+ font-family: "Inter", system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
65
+ background: radial-gradient(ellipse at 50% 20%, #1a1c28 0%, var(--bg) 60%);
66
+ color: var(--text);
67
+ min-height: 100vh;
68
+ overflow-x: hidden;
69
+ -webkit-font-smoothing: antialiased;
70
+ }
71
+
72
+ button {
73
+ font-family: inherit;
74
+ }
75
+
76
+ a {
77
+ color: var(--text);
78
+ text-decoration: none;
79
+ border-bottom: 1px solid var(--border-strong);
80
+ }
81
+ a:hover {
82
+ border-bottom-color: var(--text);
83
+ }
84
+
85
+ #app {
86
+ display: grid;
87
+ grid-template-rows: auto 1fr auto;
88
+ min-height: 100vh;
89
+ }
90
+
91
+ .hidden {
92
+ display: none !important;
93
+ }
94
+
95
+ /* ─── Topbar ──────────────────────────────────────────────────────────── */
96
+
97
+ /* Topbar is just a floating row of controls over the stage - no
98
+ * separator line, no background. Same story for the footer. Keeps the
99
+ * app feeling like one continuous canvas. */
100
+ .topbar {
101
+ display: flex;
102
+ /* Top-align so the right control cluster sits up with the title instead of
103
+ * floating to the vertical middle of the tall identity block. */
104
+ align-items: flex-start;
105
+ justify-content: space-between;
106
+ padding: 18px 28px;
107
+ /* Default stacking (below the conversation panel, z 200) so the panel drawer
108
+ * covers the controls when it's open. */
109
+ }
110
+
111
+ .brand {
112
+ display: flex;
113
+ align-items: center;
114
+ gap: 10px;
115
+ font-weight: 600;
116
+ font-size: 14px;
117
+ letter-spacing: 0.01em;
118
+ color: var(--text-dim);
119
+ }
120
+
121
+ /* Transport tag rides the wordmark as a system label, not prose — mono,
122
+ * dimmed, the only mono in the topbar. */
123
+ .brand-tag {
124
+ font-family: var(--font-mono);
125
+ font-weight: 500;
126
+ font-size: 11px;
127
+ letter-spacing: 0.02em;
128
+ opacity: 0.55;
129
+ }
130
+
131
+ .brand .brand-logo {
132
+ width: 26px;
133
+ height: 26px;
134
+ object-fit: contain;
135
+ flex: none;
136
+ filter: drop-shadow(0 1px 2px rgba(0, 0, 0, 0.45));
137
+ transition: transform 0.2s ease;
138
+ }
139
+ .brand .brand-logo:hover {
140
+ transform: translateY(-1px) rotate(-2deg);
141
+ }
142
+
143
+ /* Narrow viewports (typically the mobile shell iframe and phone-sized
144
+ browser windows): the full product name pushes the topbar's right
145
+ cluster off-screen. Keep the logo as the brand cue and drop the
146
+ wordmark - the orb already gives enough context. */
147
+ @media (max-width: 600px) {
148
+ /* The full identity stack is too tall for a phone topbar. Show only the
149
+ * title + (i); the popup carries the blurb, credits and pipeline.
150
+ * `.brand` prefix raises specificity so this beats the later base
151
+ * `.ident-meta { display: flex }` rule regardless of source order. */
152
+ .brand .ident-blurb,
153
+ .brand .ident-meta {
154
+ display: none;
155
+ }
156
+ .ident-title {
157
+ font-size: 18px;
158
+ }
159
+ /* (i) moves into the right-hand control cluster on phones, leaving the
160
+ * title alone on the left. */
161
+ .brand .about-btn {
162
+ display: none;
163
+ }
164
+ .about-btn-mobile {
165
+ display: inline-flex;
166
+ }
167
+ /* Keep phone icons at the current compact size — the desktop bump above
168
+ * shouldn't leak into the mobile topbar. */
169
+ .topbar-right .icon-btn {
170
+ width: 36px;
171
+ height: 36px;
172
+ }
173
+ .topbar-right .icon-btn svg {
174
+ width: 18px;
175
+ height: 18px;
176
+ }
177
+ /* Keep the account control compact on phones: avatar-only chip, shorter pill. */
178
+ .account-chip {
179
+ height: 36px;
180
+ padding: 0 6px;
181
+ }
182
+ .account-handle {
183
+ display: none;
184
+ }
185
+ .signin-pill {
186
+ height: 36px;
187
+ padding: 0 12px;
188
+ }
189
+ }
190
+
191
+ .topbar-right {
192
+ display: flex;
193
+ align-items: center;
194
+ gap: 10px;
195
+ }
196
+
197
+ /* The HF user pill: compact avatar + handle. Rendered as a flex row so
198
+ * the avatar stays aligned with the text baseline even when the text
199
+ * wraps on narrow viewports. */
200
+ .hf-user {
201
+ display: inline-flex;
202
+ align-items: center;
203
+ gap: 8px;
204
+ font-size: 13px;
205
+ color: var(--text-dim);
206
+ padding: 4px 10px 4px 4px;
207
+ border-radius: 999px;
208
+ background: var(--bg-elev);
209
+ border: 1px solid var(--border);
210
+ line-height: 1;
211
+ }
212
+
213
+ .hf-avatar {
214
+ width: 24px;
215
+ height: 24px;
216
+ border-radius: 50%;
217
+ object-fit: cover;
218
+ background: color-mix(in srgb, var(--accent) 25%, var(--bg-elev-2));
219
+ /* Hidden until an actual URL loads (see `setHfAvatar` in main.ts).
220
+ * Keeps the initial login flash from showing a broken image icon. */
221
+ opacity: 0;
222
+ transition: opacity 0.25s ease;
223
+ flex: none;
224
+ }
225
+ .hf-avatar.loaded {
226
+ opacity: 1;
227
+ }
228
+
229
+ .hf-user-name {
230
+ white-space: nowrap;
231
+ }
232
+
233
+ /* ─── Transport pill ──────────────────────────────────────────────
234
+ * Surface the actual network path WebRTC picked for the robot peer
235
+ * connection (LAN, direct-through-NAT, or TURN-relayed). Lets the
236
+ * user spot at a glance when audio is going through the internet
237
+ * instead of staying on-prem.
238
+ */
239
+ .transport-pill {
240
+ display: inline-flex;
241
+ align-items: center;
242
+ gap: 6px;
243
+ font-size: 12px;
244
+ font-weight: 500;
245
+ letter-spacing: 0.02em;
246
+ padding: 5px 10px;
247
+ border-radius: 999px;
248
+ background: var(--bg-elev);
249
+ border: 1px solid var(--border);
250
+ color: var(--text-dim);
251
+ user-select: none;
252
+ transition: background 0.2s ease, border-color 0.2s ease, color 0.2s ease;
253
+ }
254
+
255
+ .transport-pill .transport-dot {
256
+ width: 7px;
257
+ height: 7px;
258
+ border-radius: 50%;
259
+ background: currentColor;
260
+ box-shadow: 0 0 0 3px color-mix(in srgb, currentColor 25%, transparent);
261
+ flex: none;
262
+ }
263
+
264
+ /* Bitrate readout sits after the kind label. Dimmer + monospace so the
265
+ * kind (LAN / Direct / Relayed) stays the primary info and the numbers
266
+ * don't wiggle the layout as digits change. */
267
+ .transport-pill .transport-bitrate {
268
+ font-family: var(--font-mono);
269
+ font-size: 11px;
270
+ font-weight: 500;
271
+ color: color-mix(in srgb, currentColor 75%, var(--text-dim));
272
+ opacity: 0.85;
273
+ padding-left: 6px;
274
+ border-left: 1px solid color-mix(in srgb, currentColor 25%, transparent);
275
+ }
276
+ .transport-pill .transport-bitrate:empty {
277
+ display: none;
278
+ }
279
+
280
+ .transport-pill.transport-checking {
281
+ color: var(--text-dim);
282
+ }
283
+ .transport-pill.transport-checking .transport-dot {
284
+ animation: transport-blink 1.2s ease-in-out infinite;
285
+ }
286
+
287
+ .transport-pill.transport-lan {
288
+ color: #4ade80;
289
+ border-color: color-mix(in srgb, #4ade80 35%, var(--border));
290
+ background: color-mix(in srgb, #4ade80 10%, var(--bg-elev));
291
+ }
292
+
293
+ .transport-pill.transport-direct {
294
+ color: #60a5fa;
295
+ border-color: color-mix(in srgb, #60a5fa 35%, var(--border));
296
+ background: color-mix(in srgb, #60a5fa 10%, var(--bg-elev));
297
+ }
298
+
299
+ .transport-pill.transport-relay {
300
+ color: #fbbf24;
301
+ border-color: color-mix(in srgb, #fbbf24 35%, var(--border));
302
+ background: color-mix(in srgb, #fbbf24 10%, var(--bg-elev));
303
+ }
304
+
305
+ @keyframes transport-blink {
306
+ 0%, 100% { opacity: 1; }
307
+ 50% { opacity: 0.35; }
308
+ }
309
+
310
+ .icon-btn {
311
+ background: var(--bg-elev);
312
+ border: 1px solid var(--border);
313
+ color: var(--text-dim);
314
+ width: 36px;
315
+ height: 36px;
316
+ border-radius: var(--radius-sm);
317
+ display: inline-flex;
318
+ align-items: center;
319
+ justify-content: center;
320
+ cursor: pointer;
321
+ touch-action: manipulation;
322
+ transition: background 0.15s, color 0.15s, border-color 0.15s;
323
+ }
324
+ .icon-btn:hover {
325
+ background: var(--bg-elev-2);
326
+ color: var(--text);
327
+ border-color: var(--border-strong);
328
+ }
329
+ /* Topbar control cluster reads a touch bigger on desktop. Phones keep the
330
+ * compact 36px (reset in the mobile media query). */
331
+ .topbar-right .icon-btn {
332
+ width: 40px;
333
+ height: 40px;
334
+ }
335
+ .topbar-right .icon-btn svg {
336
+ width: 20px;
337
+ height: 20px;
338
+ }
339
+
340
+ /* ─── Account (HF login chip + sign-in pill + popover) ─────────────────────── */
341
+ .account {
342
+ position: relative;
343
+ display: inline-flex;
344
+ align-items: center;
345
+ }
346
+ /* Signed-out: a compact pill that reads as the primary affordance. */
347
+ .signin-pill {
348
+ display: inline-flex;
349
+ align-items: center;
350
+ gap: 7px;
351
+ height: 40px;
352
+ padding: 0 14px;
353
+ border-radius: var(--radius-sm);
354
+ border: 1px solid var(--border-strong);
355
+ background: var(--bg-elev);
356
+ color: var(--text);
357
+ font-size: 13px;
358
+ font-weight: 600;
359
+ text-decoration: none;
360
+ transition: background 0.15s, border-color 0.15s;
361
+ }
362
+ .signin-pill svg {
363
+ width: 16px;
364
+ height: 16px;
365
+ flex: none;
366
+ }
367
+ .signin-pill:hover {
368
+ background: var(--bg-elev-2);
369
+ border-color: var(--text);
370
+ }
371
+ /* Narrow viewports: collapse the pill to an icon-only square. */
372
+ @media (max-width: 800px) {
373
+ .signin-pill {
374
+ gap: 0;
375
+ width: 40px;
376
+ padding: 0;
377
+ justify-content: center;
378
+ }
379
+ .signin-pill span {
380
+ display: none;
381
+ }
382
+ }
383
+ /* Signed-in: avatar + handle chip. */
384
+ .account-chip {
385
+ display: inline-flex;
386
+ align-items: center;
387
+ gap: 8px;
388
+ height: 40px;
389
+ padding: 0 10px 0 6px;
390
+ border-radius: var(--radius-sm);
391
+ border: 1px solid var(--border);
392
+ background: var(--bg-elev);
393
+ color: var(--text-dim);
394
+ cursor: pointer;
395
+ transition: background 0.15s, color 0.15s, border-color 0.15s;
396
+ }
397
+ .account-chip:hover {
398
+ background: var(--bg-elev-2);
399
+ color: var(--text);
400
+ border-color: var(--border-strong);
401
+ }
402
+ .account-avatar {
403
+ width: 26px;
404
+ height: 26px;
405
+ border-radius: 50%;
406
+ object-fit: cover;
407
+ flex: none;
408
+ }
409
+ .account-avatar-fallback {
410
+ display: inline-flex;
411
+ align-items: center;
412
+ justify-content: center;
413
+ background: var(--bg-elev-2);
414
+ color: var(--text);
415
+ font-size: 12px;
416
+ font-weight: 700;
417
+ }
418
+ .account-handle {
419
+ font-size: 13px;
420
+ font-weight: 600;
421
+ max-width: 12ch;
422
+ overflow: hidden;
423
+ text-overflow: ellipsis;
424
+ white-space: nowrap;
425
+ }
426
+ .account-pro {
427
+ font-size: 10px;
428
+ font-weight: 700;
429
+ letter-spacing: 0.04em;
430
+ padding: 2px 5px;
431
+ border-radius: 5px;
432
+ background: var(--text);
433
+ color: var(--bg);
434
+ }
435
+ /* Org "Team" badge: same pill as PRO, accent-coloured so it reads as
436
+ * unlimited without claiming a paid PRO subscription. */
437
+ .account-team {
438
+ background: var(--accent);
439
+ color: var(--bg);
440
+ }
441
+ .account-pop {
442
+ position: absolute;
443
+ top: calc(100% + 8px);
444
+ right: 0;
445
+ min-width: 200px;
446
+ padding: 6px;
447
+ border-radius: var(--radius-md);
448
+ border: 1px solid var(--border-strong);
449
+ background: var(--bg-elev);
450
+ box-shadow: var(--shadow-soft);
451
+ z-index: 50;
452
+ }
453
+ .account-pop[hidden] {
454
+ display: none;
455
+ }
456
+ .account-pop-row {
457
+ padding: 8px 10px;
458
+ }
459
+ .account-pop-name {
460
+ font-weight: 600;
461
+ font-size: 13px;
462
+ }
463
+ .account-pop-meta {
464
+ display: flex;
465
+ align-items: center;
466
+ justify-content: space-between;
467
+ gap: 10px;
468
+ padding-top: 0;
469
+ font-size: 12px;
470
+ color: var(--text-dim);
471
+ }
472
+ .account-tier {
473
+ font-weight: 600;
474
+ color: var(--text);
475
+ }
476
+ .account-pop-link {
477
+ display: block;
478
+ padding: 8px 10px;
479
+ border-radius: var(--radius-sm);
480
+ color: var(--text-dim);
481
+ font-size: 13px;
482
+ text-decoration: none;
483
+ transition: background 0.15s, color 0.15s;
484
+ }
485
+ .account-pop-link:hover {
486
+ background: var(--bg-elev-2);
487
+ color: var(--text);
488
+ }
489
+ .account-signout {
490
+ border-top: 1px solid var(--border);
491
+ margin-top: 4px;
492
+ padding-top: 10px;
493
+ border-radius: 0 0 var(--radius-sm) var(--radius-sm);
494
+ }
495
+
496
+ /* ─── Daily-limit modal ────────────────────────────────────────────────────── */
497
+ .limit-modal {
498
+ width: min(400px, 92vw);
499
+ }
500
+ .limit-card {
501
+ position: relative;
502
+ align-items: center;
503
+ text-align: center;
504
+ gap: 14px;
505
+ padding: 34px 28px 26px;
506
+ }
507
+ .limit-close {
508
+ position: absolute;
509
+ top: 12px;
510
+ right: 12px;
511
+ }
512
+ /* HF smiling face (brand yellow) on a neutral badge — friendly, not a stop sign.
513
+ * The logo is the single pop of colour, so the badge itself stays quiet. */
514
+ .limit-badge {
515
+ width: 66px;
516
+ height: 66px;
517
+ border-radius: 50%;
518
+ display: grid;
519
+ place-items: center;
520
+ margin: 2px auto 2px;
521
+ background: var(--bg-elev-2);
522
+ border: 1px solid var(--border-strong);
523
+ box-shadow: 0 0 0 6px rgba(255, 210, 30, 0.06);
524
+ }
525
+ .limit-badge .hf-logo {
526
+ width: 46px;
527
+ height: auto;
528
+ }
529
+ .limit-title {
530
+ margin: 0;
531
+ font-size: 20px;
532
+ font-weight: 700;
533
+ letter-spacing: 0;
534
+ }
535
+ .limit-msg {
536
+ color: var(--text-dim);
537
+ font-size: 14px;
538
+ line-height: 1.55;
539
+ margin: 0;
540
+ max-width: 30ch;
541
+ }
542
+ .limit-cta {
543
+ display: inline-flex;
544
+ align-items: center;
545
+ justify-content: center;
546
+ gap: 8px;
547
+ margin-top: 4px;
548
+ }
549
+ .limit-cta .hf-logo {
550
+ width: 20px;
551
+ height: auto;
552
+ flex: none;
553
+ }
554
+ .limit-note {
555
+ margin: 0;
556
+ font-size: 12px;
557
+ color: var(--text-faint);
558
+ }
559
+ #limit-cta[hidden] {
560
+ display: none;
561
+ }
562
+
563
+ /* ─── Stage ───────────────────────────────────────────────────────────── */
564
+
565
+ .stage {
566
+ display: flex;
567
+ flex-direction: column;
568
+ align-items: center;
569
+ justify-content: center;
570
+ padding: 24px 24px 32px;
571
+ gap: 20px;
572
+ }
573
+
574
+ /* ─── Central circle ──────────────────────────────────────────────────── */
575
+
576
+ /* Wraps the orb and its two side controls so they stay aligned on one row. */
577
+ .orb-wrap {
578
+ position: relative;
579
+ display: flex;
580
+ align-items: center;
581
+ justify-content: center;
582
+ gap: clamp(18px, 3vw, 32px);
583
+ }
584
+
585
+ .circle {
586
+ position: relative;
587
+ width: clamp(220px, 38vw, 320px);
588
+ aspect-ratio: 1 / 1;
589
+ border-radius: 50%;
590
+ border: none;
591
+ background: transparent;
592
+ padding: 0;
593
+ cursor: pointer;
594
+ outline: none;
595
+ display: grid;
596
+ place-items: center;
597
+ color: var(--glow, var(--accent));
598
+ transition: transform 0.18s ease, filter 0.18s ease;
599
+ -webkit-tap-highlight-color: transparent;
600
+ /* Treat taps as immediate clicks (no 300ms delay / double-tap-zoom). */
601
+ touch-action: manipulation;
602
+ }
603
+ .circle:hover {
604
+ filter: brightness(1.08);
605
+ }
606
+ .circle:active {
607
+ transform: scale(0.97);
608
+ filter: brightness(0.92);
609
+ }
610
+ .circle:focus-visible .circle-core {
611
+ outline: 2px solid var(--glow, var(--accent));
612
+ outline-offset: 6px;
613
+ }
614
+ .circle[disabled] {
615
+ cursor: default;
616
+ opacity: 0.75;
617
+ }
618
+
619
+ .circle-glow {
620
+ position: absolute;
621
+ inset: 0;
622
+ border-radius: 50%;
623
+ background: radial-gradient(circle at center, var(--glow, var(--accent)) 0%, transparent 65%);
624
+ filter: blur(28px);
625
+ opacity: 0.5;
626
+ transform: scale(1);
627
+ transition: opacity 0.25s, background 0.25s, transform 1.4s ease-in-out;
628
+ pointer-events: none;
629
+ }
630
+
631
+ /* Two nested circular rings: the inner tracks the core edge, the outer
632
+ * expands / fades to convey "audio radiating out" during speaking. */
633
+ .circle-ring,
634
+ .circle-ring-outer {
635
+ position: absolute;
636
+ top: 50%;
637
+ left: 50%;
638
+ border-radius: 50%;
639
+ transform: translate(-50%, -50%);
640
+ pointer-events: none;
641
+ transition: opacity 0.4s ease, border-color 0.4s ease, transform 0.3s ease;
642
+ }
643
+ .circle-ring {
644
+ width: 82%;
645
+ height: 82%;
646
+ border: 1.5px solid color-mix(in srgb, var(--glow, var(--accent)) 35%, transparent);
647
+ opacity: 0.35;
648
+ }
649
+ .circle-ring-outer {
650
+ width: 94%;
651
+ height: 94%;
652
+ border: 1px solid color-mix(in srgb, var(--glow, var(--accent)) 22%, transparent);
653
+ opacity: 0;
654
+ }
655
+
656
+ .circle-core {
657
+ position: relative;
658
+ width: 72%;
659
+ height: 72%;
660
+ border-radius: 50%;
661
+ background: radial-gradient(
662
+ circle at 35% 28%,
663
+ color-mix(in srgb, var(--glow, var(--accent)) 28%, transparent),
664
+ color-mix(in srgb, var(--glow, var(--accent)) 10%, transparent) 55%,
665
+ color-mix(in srgb, var(--glow, var(--accent)) 5%, transparent)
666
+ );
667
+ border: 2px solid color-mix(in srgb, var(--glow, var(--accent)) 40%, transparent);
668
+ display: grid;
669
+ place-items: center;
670
+ box-shadow:
671
+ 0 0 32px color-mix(in srgb, var(--glow, var(--accent)) 22%, transparent),
672
+ inset 0 0 28px color-mix(in srgb, var(--glow, var(--accent)) 18%, transparent);
673
+ transition: background 0.4s ease, border-color 0.4s ease, box-shadow 0.4s ease, transform 0.4s ease;
674
+ }
675
+
676
+ /* Indicator slot: a single SVG / spinner / bar group is visible at a time,
677
+ * driven by the state class on `.circle`. */
678
+ .circle-indicator {
679
+ position: relative;
680
+ width: 44%;
681
+ height: 44%;
682
+ display: grid;
683
+ place-items: center;
684
+ color: var(--glow, var(--accent));
685
+ }
686
+ .circle-indicator > .ind {
687
+ grid-area: 1 / 1;
688
+ opacity: 0;
689
+ transform: scale(0.85);
690
+ transition: opacity 0.25s ease, transform 0.25s ease;
691
+ pointer-events: none;
692
+ }
693
+ .circle-indicator > svg.ind {
694
+ width: 60%;
695
+ height: 60%;
696
+ }
697
+
698
+ /* Spinner: a rotating ring gap, CSS-only.
699
+ *
700
+ * Note: the base `.circle-indicator > .ind` rule forces `transform:
701
+ * scale(.85)` / `scale(1)` on every indicator to drive the show/hide
702
+ * transition. If we only rotate here, the browser has to interpolate
703
+ * between `scale(1)` and `rotate(360deg)` (two different transform
704
+ * functions), which produces a broken, barely-moving animation. So we
705
+ * include the scale explicitly in the keyframes and bump specificity
706
+ * with `!important` so the spinner always wins over the state rule. */
707
+ .ind-spinner {
708
+ width: 48%;
709
+ height: 48%;
710
+ border: 3px solid currentColor;
711
+ border-right-color: transparent;
712
+ border-radius: 50%;
713
+ opacity: 0;
714
+ animation: ind-spin 0.9s linear infinite;
715
+ }
716
+
717
+ /* Thinking dots: 3 soft pulsing dots while the model is composing a
718
+ * response. Apple-style cadence: each dot scales up and brightens in
719
+ * turn, staggered by ~160 ms. Per-dot animation is on the child, so
720
+ * the parent's scale(.85 → 1) show/hide transform composes cleanly. */
721
+ .ind-thinking {
722
+ display: flex;
723
+ align-items: center;
724
+ justify-content: center;
725
+ gap: 8px;
726
+ width: 60%;
727
+ height: 60%;
728
+ }
729
+ .ind-thinking .dot {
730
+ width: 10px;
731
+ height: 10px;
732
+ border-radius: 50%;
733
+ background: currentColor;
734
+ opacity: 0.3;
735
+ animation: thinking-dot 1.25s ease-in-out infinite;
736
+ }
737
+ .ind-thinking .dot:nth-child(1) { animation-delay: 0s; }
738
+ .ind-thinking .dot:nth-child(2) { animation-delay: 0.16s; }
739
+ .ind-thinking .dot:nth-child(3) { animation-delay: 0.32s; }
740
+
741
+ /* Bars: 5 vertical pills driven by --bar0..--bar4 CSS vars. */
742
+ .ind-bars {
743
+ display: flex;
744
+ align-items: center;
745
+ gap: 5px;
746
+ height: 42%;
747
+ }
748
+ .ind-bars .bar {
749
+ width: 4px;
750
+ min-height: 4px;
751
+ border-radius: 3px;
752
+ background: currentColor;
753
+ opacity: 0.7;
754
+ --h: var(--bar0, 0);
755
+ height: calc(4px + var(--h) * 36px);
756
+ transition: height 0.08s ease-out, opacity 0.08s ease-out;
757
+ }
758
+ .ind-bars .bar:nth-child(1) { --h: var(--bar0); }
759
+ .ind-bars .bar:nth-child(2) { --h: var(--bar1); }
760
+ .ind-bars .bar:nth-child(3) { --h: var(--bar2); }
761
+ .ind-bars .bar:nth-child(4) { --h: var(--bar3); }
762
+ .ind-bars .bar:nth-child(5) { --h: var(--bar4); }
763
+ .ind-bars .bar {
764
+ opacity: calc(0.55 + 0.45 * var(--h));
765
+ }
766
+
767
+ /* Active indicator per state.
768
+ *
769
+ * Note: `ai-speaking` deliberately has NO indicator here. The orb
770
+ * itself becomes the indicator by pulsing on Reachy's voice (see the
771
+ * `--ai-audio-level` rules further down), which is a lot clearer and
772
+ * less confusing than reusing the mic-bars (which viewers would read
773
+ * as "you are speaking"). */
774
+ .state-signed-out .ind-connect,
775
+ .state-authenticated .ind-mic,
776
+ .state-ready .ind-mic,
777
+ .state-connecting .ind-spinner,
778
+ .state-connected .ind-spinner,
779
+ .state-auto-selecting .ind-spinner,
780
+ .state-starting .ind-spinner,
781
+ .state-processing .ind-thinking,
782
+ .state-listening .ind-bars,
783
+ .state-user-speaking .ind-bars,
784
+ .state-ai-speaking .ind-voice,
785
+ .state-error .ind-error {
786
+ opacity: 1;
787
+ transform: scale(1);
788
+ }
789
+
790
+ /* AI speaking indicator: speaker + two sound waves.
791
+ *
792
+ * Each wave pulses outward (opacity + stroke grow) with a quarter-
793
+ * beat offset so it reads as sound radiating out. Kept as a pure
794
+ * CSS animation so the icon is always visually alive even between
795
+ * syllables when --ai-audio-level momentarily dips. */
796
+ .ind-voice .wave {
797
+ transform-origin: 50% 50%;
798
+ animation: voice-wave 1.35s ease-out infinite;
799
+ }
800
+ .ind-voice .wave-1 { animation-delay: 0s; }
801
+ .ind-voice .wave-2 { animation-delay: 0.35s; }
802
+
803
+ /* Idle indicators: a chain-link "connect" glyph for the signed-out
804
+ * step (invites the user to authenticate with HF) and a microphone
805
+ * once a session is ready. Both picked up by the generic
806
+ * `.circle-indicator > svg.ind` sizing rule (60% × 60% of the slot)
807
+ * and rely on per-state opacity transitions for show / hide. */
808
+ .ind-connect,
809
+ .ind-mic {
810
+ color: color-mix(in srgb, var(--glow, var(--accent)) 85%, white);
811
+ }
812
+
813
+ /* ─── Caption below the circle ────────────────────────────────────────── */
814
+
815
+ /* The caption under the orb is meant to whisper, not shout: micro-label
816
+ * vibe, uppercase, letter-spaced, muted. Only appears for actionable /
817
+ * transitional states (see STATE_VIEWS). During a live conversation the
818
+ * orb alone carries the state so we collapse this row entirely. */
819
+ .circle-caption {
820
+ margin: 0;
821
+ font-family: var(--font-mono);
822
+ font-size: 11px;
823
+ font-weight: 500;
824
+ letter-spacing: 0.14em;
825
+ text-transform: uppercase;
826
+ color: var(--text-faint);
827
+ min-height: 1.2em;
828
+ text-align: center;
829
+ opacity: 0.75;
830
+ transition: opacity 0.25s ease, color 0.25s ease, transform 0.25s ease,
831
+ min-height 0.25s ease;
832
+ }
833
+ .circle-caption.empty {
834
+ opacity: 0;
835
+ min-height: 0;
836
+ transform: translateY(-4px);
837
+ pointer-events: none;
838
+ }
839
+ .circle-caption.muted {
840
+ color: var(--text-faint);
841
+ opacity: 0.65;
842
+ }
843
+ .circle-caption.error {
844
+ color: var(--error);
845
+ opacity: 1;
846
+ letter-spacing: 0.08em;
847
+ }
848
+
849
+ /* ─── Tool-call toaster ───────────────────────────────────────────────── */
850
+
851
+ /* A small, non-interactive pill that appears below the circle when the
852
+ * model invokes a tool (move_head, play_move). Sits just under the
853
+ * state caption, collapses to zero height when no toast is active. */
854
+ .tool-toast {
855
+ display: inline-flex;
856
+ align-items: center;
857
+ gap: 8px;
858
+ margin-top: 10px;
859
+ padding: 6px 12px;
860
+ border-radius: 999px;
861
+ border: 1px solid var(--border-strong);
862
+ background: color-mix(in srgb, var(--bg-elev-2) 78%, transparent);
863
+ color: var(--text-dim);
864
+ font-size: 12px;
865
+ font-weight: 500;
866
+ letter-spacing: 0.01em;
867
+ line-height: 1;
868
+ white-space: nowrap;
869
+ max-width: 80vw;
870
+ overflow: hidden;
871
+ text-overflow: ellipsis;
872
+
873
+ opacity: 0;
874
+ transform: translateY(-4px) scale(0.96);
875
+ pointer-events: none;
876
+ transition: opacity 0.22s ease, transform 0.22s ease;
877
+ }
878
+ .tool-toast.visible {
879
+ opacity: 1;
880
+ transform: translateY(0) scale(1);
881
+ }
882
+ .tool-toast-icon {
883
+ width: 14px;
884
+ height: 14px;
885
+ flex: none;
886
+ color: color-mix(in srgb, var(--voice-tool) 80%, white);
887
+ animation: tool-toast-spin 3.2s linear infinite;
888
+ animation-play-state: paused;
889
+ }
890
+ .tool-toast.visible .tool-toast-icon {
891
+ animation-play-state: running;
892
+ }
893
+ .tool-toast-text {
894
+ display: inline-block;
895
+ overflow: hidden;
896
+ text-overflow: ellipsis;
897
+ }
898
+ @keyframes tool-toast-spin {
899
+ from { transform: rotate(0deg); }
900
+ to { transform: rotate(360deg); }
901
+ }
902
+
903
+ /* ─── Side controls (mic / stop) ──────────────────────────────────────── */
904
+
905
+ /* Mic button + its radial noise-gate arc. The wrapper centres the button; the
906
+ arc SVG is an absolute overlay larger than the button, revealed with the
907
+ live session (it has no layout footprint, so the idle collapse is unaffected). */
908
+ .mic-gate {
909
+ position: relative;
910
+ flex: none;
911
+ display: inline-flex;
912
+ align-items: center;
913
+ justify-content: center;
914
+ }
915
+ .mic-gate-arc {
916
+ position: absolute;
917
+ left: 50%;
918
+ top: 50%;
919
+ width: 80px;
920
+ height: 80px;
921
+ transform: translate(-50%, -50%);
922
+ pointer-events: none; /* only the hit-path below catches drags */
923
+ opacity: 0;
924
+ transition: opacity 0.25s ease;
925
+ }
926
+ .orb-wrap.live .mic-gate-arc { opacity: 1; }
927
+ .mga-track {
928
+ stroke: rgba(255, 255, 255, 0.1); /* hairline; recedes until needed */
929
+ stroke-width: 1.5;
930
+ stroke-linecap: round;
931
+ }
932
+ .mga-fill {
933
+ stroke: var(--accent-2);
934
+ stroke-width: 2;
935
+ stroke-linecap: round;
936
+ opacity: 0.85;
937
+ transition: stroke-dashoffset 0.06s linear;
938
+ }
939
+ /* Threshold setpoint: a bead riding the ring. White with a thin dark outline so
940
+ it stays legible over the moving fill; it recolors to cyan with a soft glow
941
+ the moment the live level crosses it (the gate opening). */
942
+ .mga-handle {
943
+ fill: var(--text);
944
+ stroke: none;
945
+ transition: fill 0.2s ease, filter 0.2s ease;
946
+ }
947
+ .mic-gate.gate-open .mga-handle {
948
+ fill: var(--accent-2);
949
+ filter: drop-shadow(0 0 3px var(--accent-2));
950
+ }
951
+ .mga-hit {
952
+ stroke: transparent;
953
+ stroke-width: 16;
954
+ pointer-events: none; /* enabled only while live (below) */
955
+ cursor: pointer;
956
+ touch-action: none;
957
+ }
958
+ /* Only catch drags during a live call; when idle the arc is hidden and must
959
+ not steal clicks near the collapsed mic button / orb. */
960
+ .orb-wrap.live .mga-hit { pointer-events: stroke; }
961
+
962
+ .side-btn {
963
+ flex: none;
964
+ width: 52px;
965
+ height: 52px;
966
+ border-radius: 50%;
967
+ border: 1px solid var(--border-strong);
968
+ /* Brighter background so the buttons actually stand out against the
969
+ * deep-blue stage gradient; previous var(--bg-elev) was too close to
970
+ * the page bg to be readable. */
971
+ background: color-mix(in srgb, var(--bg-elev-2) 80%, #2a2e3c);
972
+ color: var(--text);
973
+ display: inline-flex;
974
+ align-items: center;
975
+ justify-content: center;
976
+ cursor: pointer;
977
+ touch-action: manipulation;
978
+ box-shadow: 0 6px 16px rgba(0, 0, 0, 0.35),
979
+ inset 0 1px 0 rgba(255, 255, 255, 0.06);
980
+ /* Hidden by default: take up no space until the session is live. The
981
+ * `width: 0` collapse keeps the orb centered on the idle screen. */
982
+ opacity: 0;
983
+ transform: scale(0.55);
984
+ width: 0;
985
+ padding: 0;
986
+ pointer-events: none;
987
+ overflow: hidden;
988
+ transition: opacity 0.25s ease, transform 0.25s ease, width 0.25s ease,
989
+ background 0.15s, color 0.15s, border-color 0.15s;
990
+ }
991
+ .side-btn:hover {
992
+ background: color-mix(in srgb, var(--bg-elev-2) 60%, #323746);
993
+ border-color: var(--text-dim);
994
+ }
995
+ .side-btn svg {
996
+ width: 22px;
997
+ height: 22px;
998
+ flex: none;
999
+ }
1000
+ .side-btn .mic-off { display: none; }
1001
+ .side-btn.muted {
1002
+ color: #fff;
1003
+ border-color: var(--error);
1004
+ background: color-mix(in srgb, var(--error) 70%, #1a0e13);
1005
+ }
1006
+ .side-btn.muted .mic-on { display: none; }
1007
+ .side-btn.muted .mic-off { display: block; }
1008
+
1009
+ /* Stop button: subtle warm tint so "end" reads as destructive. */
1010
+ #stop-btn:hover {
1011
+ color: #fff;
1012
+ border-color: color-mix(in srgb, var(--error) 60%, var(--border-strong));
1013
+ background: color-mix(in srgb, var(--error) 25%, var(--bg-elev-2));
1014
+ }
1015
+
1016
+ /* Reveal when the session is live: buttons flank the orb on a flex row. */
1017
+ .orb-wrap.live .side-btn {
1018
+ opacity: 1;
1019
+ transform: scale(1);
1020
+ width: 52px;
1021
+ pointer-events: auto;
1022
+ }
1023
+
1024
+ /* Disable every transition / animation during the first paint so the
1025
+ * orb doesn't fade-and-scale in when the page loads. `main.ts` removes
1026
+ * the class after one animation frame. */
1027
+ body.booting,
1028
+ body.booting *,
1029
+ body.booting *::before,
1030
+ body.booting *::after {
1031
+ transition: none !important;
1032
+ animation-duration: 0s !important;
1033
+ animation-delay: 0s !important;
1034
+ }
1035
+
1036
+ /* ─── Circle animation keyframes ──────────────────────────────────────── */
1037
+
1038
+ /* Slow, subtle breathing for "warm idle" states. */
1039
+ @keyframes breathe {
1040
+ 0%, 100% { transform: translate(-50%, -50%) scale(1); opacity: 0.4; }
1041
+ 50% { transform: translate(-50%, -50%) scale(1.06); opacity: 0.15; }
1042
+ }
1043
+
1044
+ /* Outer ring expanding and fading - conveys "I am producing audio". */
1045
+ @keyframes ring-out {
1046
+ 0% { transform: translate(-50%, -50%) scale(1); opacity: 0.35; }
1047
+ 100% { transform: translate(-50%, -50%) scale(1.18); opacity: 0; }
1048
+ }
1049
+
1050
+ /* Soft inner scale for the core while talking. */
1051
+ @keyframes core-breathe {
1052
+ 0%, 100% { transform: scale(1); }
1053
+ 50% { transform: scale(1.04); }
1054
+ }
1055
+
1056
+ /* Subtle glow throb used for "thinking" — dimmer than speaking. */
1057
+ @keyframes thinking {
1058
+ 0%, 100% { transform: scale(1); opacity: 0.7; }
1059
+ 50% { transform: scale(0.96); opacity: 0.45; }
1060
+ }
1061
+
1062
+ /* Individual dot pulse for the 3-dot processing indicator. */
1063
+ @keyframes thinking-dot {
1064
+ 0%, 60%, 100% { transform: scale(0.7); opacity: 0.3; }
1065
+ 30% { transform: scale(1.15); opacity: 1; }
1066
+ }
1067
+
1068
+ /* Sound-wave pulse: arc fades in, scales up slightly, fades out.
1069
+ * The transform-origin is the speaker's center (roughly x=12), so
1070
+ * the waves feel like they're emanating from the cone. */
1071
+ @keyframes voice-wave {
1072
+ 0% { opacity: 0; transform: scale(0.7); }
1073
+ 30% { opacity: 1; transform: scale(1); }
1074
+ 70% { opacity: 0.2; transform: scale(1.12); }
1075
+ 100% { opacity: 0; transform: scale(0.7); }
1076
+ }
1077
+
1078
+ @keyframes ind-spin {
1079
+ /* Scale kept at 1 so we don't fight with the indicator's base
1080
+ * show/hide transform (see `.ind-spinner` for the rationale). */
1081
+ from { transform: scale(1) rotate(0deg); }
1082
+ to { transform: scale(1) rotate(360deg); }
1083
+ }
1084
+
1085
+ /* ─── State-specific colors ──────────────────────────────────────────── */
1086
+
1087
+ .circle.state-signed-out { --glow: #8b7dff; }
1088
+ .circle.state-authenticated,
1089
+ .circle.state-ready { --glow: #34d399; }
1090
+ .circle.state-connecting,
1091
+ .circle.state-connected,
1092
+ .circle.state-auto-selecting,
1093
+ .circle.state-starting { --glow: #facc15; }
1094
+ .circle.state-listening,
1095
+ .circle.state-user-speaking { --glow: var(--listening); }
1096
+ .circle.state-processing { --glow: var(--processing); }
1097
+ .circle.state-ai-speaking { --glow: var(--speaking); }
1098
+ .circle.state-error { --glow: var(--error); }
1099
+
1100
+ /* Idle / ready: gentle breathing of the inner ring. Kept out of the
1101
+ * `signed-out` state so the very first paint on page load stays quiet
1102
+ * (the orb now shows the Reachy head silhouette, no need to also pulse). */
1103
+ .circle.state-authenticated .circle-ring,
1104
+ .circle.state-ready .circle-ring {
1105
+ animation: breathe 2.4s ease-in-out infinite;
1106
+ }
1107
+
1108
+ /* Connecting flows: subtle glow throb so the orb feels thoughtful
1109
+ * while the session is being negotiated. Kept off `processing` on
1110
+ * purpose - the 3 thinking dots already pulse, and layering another
1111
+ * throb on top competes with them for attention. */
1112
+ .circle.state-connecting .circle-core,
1113
+ .circle.state-connected .circle-core,
1114
+ .circle.state-auto-selecting .circle-core,
1115
+ .circle.state-starting .circle-core {
1116
+ animation: thinking 1.4s ease-in-out infinite;
1117
+ }
1118
+
1119
+ /* User is speaking: the mic RMS drives scale + opacity via --audio-level. */
1120
+ .circle.state-user-speaking .circle-ring {
1121
+ animation: none;
1122
+ opacity: calc(0.25 + 0.55 * var(--audio-level));
1123
+ transform: translate(-50%, -50%) scale(calc(1 + 0.08 * var(--audio-level)));
1124
+ transition: transform 0.08s linear, opacity 0.08s linear;
1125
+ }
1126
+ .circle.state-user-speaking .circle-ring-outer {
1127
+ opacity: calc(0.1 + 0.35 * var(--audio-level));
1128
+ transform: translate(-50%, -50%) scale(calc(1 + 0.12 * var(--audio-level)));
1129
+ transition: transform 0.08s linear, opacity 0.08s linear;
1130
+ }
1131
+ .circle.state-listening .circle-ring {
1132
+ animation: breathe 2s ease-in-out infinite;
1133
+ }
1134
+
1135
+ /* Assistant is speaking: the whole orb breathes in sync with Reachy's
1136
+ * voice instead of running a fixed timer. `--ai-audio-level` (0-1) is
1137
+ * updated at display rate by AiLevelMonitor from the OpenAI output
1138
+ * track, so every syllable visibly moves the core + outer ring. This
1139
+ * reads instantly as "the orb is the voice" and completely avoids the
1140
+ * ambiguity of bars-vs-mic the user flagged. */
1141
+ .circle.state-ai-speaking .circle-core {
1142
+ animation: none;
1143
+ transform: scale(calc(1 + 0.09 * var(--ai-audio-level, 0)));
1144
+ transition: transform 0.08s linear;
1145
+ }
1146
+ .circle.state-ai-speaking .circle-ring {
1147
+ animation: none;
1148
+ opacity: calc(0.3 + 0.5 * var(--ai-audio-level, 0));
1149
+ transform: translate(-50%, -50%) scale(calc(1 + 0.05 * var(--ai-audio-level, 0)));
1150
+ transition: transform 0.08s linear, opacity 0.08s linear;
1151
+ }
1152
+ .circle.state-ai-speaking .circle-ring-outer {
1153
+ animation: none;
1154
+ opacity: calc(0.15 + 0.55 * var(--ai-audio-level, 0));
1155
+ transform: translate(-50%, -50%) scale(calc(1 + 0.18 * var(--ai-audio-level, 0)));
1156
+ transition: transform 0.08s linear, opacity 0.08s linear;
1157
+ }
1158
+ .circle.state-ai-speaking .circle-glow {
1159
+ opacity: calc(0.35 + 0.45 * var(--ai-audio-level, 0));
1160
+ transition: opacity 0.08s linear;
1161
+ }
1162
+
1163
+ /* ─── Robot picker ────────────────────────────────────────────────────── */
1164
+
1165
+ .robot-picker {
1166
+ width: min(420px, 92vw);
1167
+ background: var(--bg-elev);
1168
+ border: 1px solid var(--border);
1169
+ border-radius: var(--radius-md);
1170
+ padding: 14px 16px;
1171
+ box-shadow: var(--shadow-soft);
1172
+ }
1173
+
1174
+ .picker-title {
1175
+ margin: 0 0 10px;
1176
+ font-size: 11px;
1177
+ font-weight: 600;
1178
+ letter-spacing: 0.1em;
1179
+ text-transform: uppercase;
1180
+ color: var(--text-faint);
1181
+ }
1182
+
1183
+ .robot-list {
1184
+ display: flex;
1185
+ flex-direction: column;
1186
+ gap: 8px;
1187
+ }
1188
+
1189
+ .robot-card {
1190
+ display: flex;
1191
+ justify-content: space-between;
1192
+ align-items: center;
1193
+ padding: 12px 14px;
1194
+ border: 1px solid var(--border);
1195
+ border-radius: var(--radius-sm);
1196
+ background: var(--bg-elev-2);
1197
+ cursor: pointer;
1198
+ transition: border-color 0.15s, background 0.15s;
1199
+ }
1200
+ .robot-card:hover {
1201
+ border-color: var(--border-strong);
1202
+ }
1203
+ .robot-card.selected {
1204
+ border-color: var(--border-strong);
1205
+ background: var(--bg-elev);
1206
+ }
1207
+
1208
+ .robot-card .name {
1209
+ font-weight: 600;
1210
+ font-size: 14px;
1211
+ }
1212
+ .robot-card .id {
1213
+ font-family: var(--font-mono);
1214
+ font-size: 12px;
1215
+ color: var(--text-faint);
1216
+ }
1217
+
1218
+ .robot-empty {
1219
+ padding: 14px;
1220
+ text-align: center;
1221
+ color: var(--text-faint);
1222
+ font-size: 13px;
1223
+ }
1224
+
1225
+ /* ─── Footer ──────────────────────────────────────────────────────────── */
1226
+
1227
+ .footer {
1228
+ padding: 12px 28px 18px;
1229
+ font-size: 11px;
1230
+ letter-spacing: 0.02em;
1231
+ color: var(--text-faint);
1232
+ display: flex;
1233
+ justify-content: center;
1234
+ opacity: 0.55;
1235
+ transition: opacity 0.25s ease;
1236
+ }
1237
+ .footer:hover {
1238
+ opacity: 0.9;
1239
+ }
1240
+ .footer a {
1241
+ color: inherit;
1242
+ text-decoration: none;
1243
+ border-bottom: 1px dotted currentColor;
1244
+ }
1245
+ .footer a:hover {
1246
+ color: var(--text-dim);
1247
+ }
1248
+ /* While the webcam preview sits bottom-left, push the credit to the
1249
+ * bottom-right so the two don't collide. */
1250
+ body.cam-on .footer {
1251
+ justify-content: flex-end;
1252
+ }
1253
+
1254
+ /* ─── Modal ───────────────────────────────────────────────────────────── */
1255
+
1256
+ .modal {
1257
+ border: 1px solid var(--border);
1258
+ border-radius: var(--radius-md);
1259
+ padding: 0;
1260
+ background: var(--bg-elev);
1261
+ color: var(--text);
1262
+ width: min(440px, 92vw);
1263
+ box-shadow: var(--shadow-soft);
1264
+ }
1265
+ .modal::backdrop {
1266
+ background: rgba(8, 9, 13, 0.65);
1267
+ backdrop-filter: blur(4px);
1268
+ }
1269
+
1270
+ .modal-content {
1271
+ display: flex;
1272
+ flex-direction: column;
1273
+ gap: 16px;
1274
+ padding: 22px 24px 20px;
1275
+ }
1276
+
1277
+ .modal-header {
1278
+ display: flex;
1279
+ align-items: center;
1280
+ justify-content: space-between;
1281
+ margin-bottom: 4px;
1282
+ }
1283
+ .modal-header h2 {
1284
+ margin: 0;
1285
+ font-size: 16px;
1286
+ font-weight: 600;
1287
+ letter-spacing: 0.02em;
1288
+ }
1289
+
1290
+ .field[hidden] { display: none; }
1291
+ .field {
1292
+ display: flex;
1293
+ flex-direction: column;
1294
+ gap: 6px;
1295
+ font-size: 13px;
1296
+ font-weight: 500;
1297
+ color: var(--text-dim);
1298
+ }
1299
+ .field > span {
1300
+ color: var(--text);
1301
+ font-weight: 600;
1302
+ font-size: 12px;
1303
+ letter-spacing: 0.04em;
1304
+ text-transform: uppercase;
1305
+ }
1306
+ .field input,
1307
+ .field select,
1308
+ .field textarea {
1309
+ font-family: inherit;
1310
+ font-size: 14px;
1311
+ color: var(--text);
1312
+ background: var(--bg);
1313
+ border: 1px solid var(--border);
1314
+ border-radius: var(--radius-sm);
1315
+ padding: 10px 12px;
1316
+ outline: none;
1317
+ transition: border-color 0.15s;
1318
+ resize: vertical;
1319
+ }
1320
+ .field textarea {
1321
+ min-height: 84px;
1322
+ }
1323
+ .field input:focus,
1324
+ .field select:focus,
1325
+ .field textarea:focus {
1326
+ border-color: var(--text-dim);
1327
+ }
1328
+ .field small {
1329
+ color: var(--text-faint);
1330
+ font-size: 12px;
1331
+ line-height: 1.4;
1332
+ }
1333
+ .field small.error { color: var(--error); }
1334
+ .field small code {
1335
+ background: var(--bg);
1336
+ padding: 1px 5px;
1337
+ border-radius: 4px;
1338
+ border: 1px solid var(--border);
1339
+ }
1340
+
1341
+ /* Noise gate: a header with a live value, a level meter, and a range slider
1342
+ that shares the meter's horizontal (dB) axis. */
1343
+ .field-head {
1344
+ display: flex;
1345
+ align-items: baseline;
1346
+ justify-content: space-between;
1347
+ }
1348
+ .field-value {
1349
+ font-weight: 500;
1350
+ font-size: 12px;
1351
+ letter-spacing: 0;
1352
+ text-transform: none;
1353
+ color: var(--text-dim);
1354
+ }
1355
+ /* The slider and the level meter are one widget: the range input is overlaid
1356
+ on the meter track (its native track is transparent), so the live-level fill
1357
+ shows through behind the thumb and the thumb itself is the threshold. */
1358
+ .gate {
1359
+ display: flex;
1360
+ flex-direction: column;
1361
+ gap: 4px;
1362
+ }
1363
+ .gate-track {
1364
+ position: relative;
1365
+ height: 14px;
1366
+ display: flex;
1367
+ align-items: center;
1368
+ }
1369
+ .gate-track::before {
1370
+ /* the visible meter track */
1371
+ content: "";
1372
+ position: absolute;
1373
+ left: 0;
1374
+ right: 0;
1375
+ height: 8px;
1376
+ border-radius: 999px;
1377
+ background: var(--bg);
1378
+ border: 1px solid var(--border);
1379
+ }
1380
+ .gate-meter-fill {
1381
+ position: absolute;
1382
+ left: 1px;
1383
+ top: 50%;
1384
+ transform: translateY(-50%);
1385
+ height: 6px;
1386
+ width: 0;
1387
+ border-radius: 999px;
1388
+ background: var(--accent-2);
1389
+ transition: width 0.06s linear;
1390
+ pointer-events: none;
1391
+ }
1392
+ .gate-ends {
1393
+ display: flex;
1394
+ justify-content: space-between;
1395
+ font-size: 11px;
1396
+ color: var(--text-faint);
1397
+ letter-spacing: 0;
1398
+ text-transform: none;
1399
+ }
1400
+ /* The range input sits transparently on top of the meter track. */
1401
+ .gate-track input[type="range"] {
1402
+ position: relative;
1403
+ z-index: 1;
1404
+ width: 100%;
1405
+ margin: 0;
1406
+ -webkit-appearance: none;
1407
+ appearance: none;
1408
+ padding: 0;
1409
+ border: none;
1410
+ background: transparent;
1411
+ height: 14px;
1412
+ cursor: pointer;
1413
+ }
1414
+ .gate-track input[type="range"]::-webkit-slider-runnable-track {
1415
+ height: 14px;
1416
+ background: transparent;
1417
+ }
1418
+ .gate-track input[type="range"]::-moz-range-track {
1419
+ height: 14px;
1420
+ background: transparent;
1421
+ }
1422
+ .gate-track input[type="range"]::-webkit-slider-thumb {
1423
+ -webkit-appearance: none;
1424
+ appearance: none;
1425
+ width: 6px;
1426
+ height: 18px;
1427
+ border-radius: 3px;
1428
+ background: var(--text);
1429
+ border: 2px solid var(--bg-elev);
1430
+ box-shadow: 0 0 0 1px var(--border-strong);
1431
+ }
1432
+ .gate-track input[type="range"]::-moz-range-thumb {
1433
+ width: 6px;
1434
+ height: 18px;
1435
+ border-radius: 3px;
1436
+ background: var(--text);
1437
+ border: 2px solid var(--bg-elev);
1438
+ box-shadow: 0 0 0 1px var(--border-strong);
1439
+ }
1440
+ .gate-track input[type="range"]:focus { border: none; }
1441
+
1442
+ .modal-footer {
1443
+ display: flex;
1444
+ justify-content: space-between;
1445
+ gap: 12px;
1446
+ padding-top: 6px;
1447
+ }
1448
+
1449
+ .btn {
1450
+ padding: 10px 16px;
1451
+ border-radius: var(--radius-sm);
1452
+ border: 1px solid var(--border-strong);
1453
+ background: var(--bg-elev-2);
1454
+ color: var(--text);
1455
+ font-weight: 600;
1456
+ font-size: 13px;
1457
+ cursor: pointer;
1458
+ transition: background 0.15s, border-color 0.15s, transform 0.05s;
1459
+ }
1460
+ .btn:hover {
1461
+ border-color: var(--text);
1462
+ }
1463
+ .btn:active {
1464
+ transform: translateY(1px);
1465
+ }
1466
+ .btn.primary {
1467
+ background: var(--text);
1468
+ border-color: var(--text);
1469
+ color: var(--bg);
1470
+ }
1471
+ .btn.primary:hover {
1472
+ background: #fff;
1473
+ border-color: #fff;
1474
+ }
1475
+ .btn.ghost {
1476
+ background: transparent;
1477
+ border-color: var(--border);
1478
+ color: var(--text-dim);
1479
+ }
1480
+ .btn.wide {
1481
+ width: 100%;
1482
+ padding: 12px 16px;
1483
+ }
1484
+ .btn[disabled] {
1485
+ opacity: 0.45;
1486
+ cursor: not-allowed;
1487
+ }
1488
+ .btn[disabled]:hover {
1489
+ border-color: var(--border-strong);
1490
+ }
1491
+
1492
+ /* ─── Settings tabs ───────────────────────────────────────────────────── */
1493
+
1494
+ .tabs {
1495
+ display: flex;
1496
+ gap: 4px;
1497
+ padding: 4px;
1498
+ background: var(--bg);
1499
+ border: 1px solid var(--border);
1500
+ border-radius: var(--radius-sm);
1501
+ }
1502
+ .tab {
1503
+ flex: 1;
1504
+ padding: 8px 12px;
1505
+ border: none;
1506
+ background: transparent;
1507
+ color: var(--text-dim);
1508
+ font-family: inherit;
1509
+ font-size: 13px;
1510
+ font-weight: 600;
1511
+ letter-spacing: 0.02em;
1512
+ border-radius: calc(var(--radius-sm) - 3px);
1513
+ cursor: pointer;
1514
+ transition: background 0.15s, color 0.15s;
1515
+ }
1516
+ .tab:hover {
1517
+ color: var(--text);
1518
+ }
1519
+ .tab.active {
1520
+ background: var(--bg-elev-2);
1521
+ color: var(--text);
1522
+ box-shadow: 0 1px 2px rgba(0, 0, 0, 0.3);
1523
+ }
1524
+
1525
+ .tab-panels {
1526
+ display: flex;
1527
+ flex-direction: column;
1528
+ }
1529
+ .tab-panel {
1530
+ display: flex;
1531
+ flex-direction: column;
1532
+ gap: 16px;
1533
+ }
1534
+ .tab-panel[hidden] {
1535
+ display: none;
1536
+ }
1537
+
1538
+ /* Horizontal layout for Voice + Model. */
1539
+ .field-row {
1540
+ display: grid;
1541
+ grid-template-columns: 1fr 1fr;
1542
+ gap: 12px;
1543
+ }
1544
+
1545
+ /* ─── Chat button + badge ────────────────────────────────────────────── */
1546
+
1547
+ #chat-btn {
1548
+ position: relative;
1549
+ }
1550
+
1551
+ .chat-badge {
1552
+ position: absolute;
1553
+ top: 6px;
1554
+ right: 6px;
1555
+ width: 7px;
1556
+ height: 7px;
1557
+ border-radius: 50%;
1558
+ background: var(--text);
1559
+ border: 1.5px solid var(--bg-elev);
1560
+ opacity: 0;
1561
+ transform: scale(0);
1562
+ transition: opacity 0.2s ease, transform 0.2s ease;
1563
+ pointer-events: none;
1564
+ }
1565
+ .chat-badge.visible {
1566
+ opacity: 1;
1567
+ transform: scale(1);
1568
+ }
1569
+
1570
+ /* ─── Ephemeral bubble stack ─────────────────────────────────────────── */
1571
+
1572
+ .bubble-stack {
1573
+ position: fixed;
1574
+ /* Clear the topbar control row so the first bubble doesn't sit at the same
1575
+ * height as the buttons. */
1576
+ top: 96px;
1577
+ right: 28px;
1578
+ width: min(300px, calc(100vw - 56px));
1579
+ display: flex;
1580
+ flex-direction: column;
1581
+ gap: 8px;
1582
+ pointer-events: none;
1583
+ z-index: 100;
1584
+ }
1585
+
1586
+ .bubble {
1587
+ --bubble-dx: 10px;
1588
+ pointer-events: auto;
1589
+ padding: 10px 14px;
1590
+ border-radius: var(--radius-md);
1591
+ font-size: 13px;
1592
+ line-height: 1.5;
1593
+ border: 1px solid var(--border);
1594
+ background: var(--bg-elev);
1595
+ box-shadow: 0 4px 18px rgba(0, 0, 0, 0.32);
1596
+ max-width: 100%;
1597
+ word-break: break-word;
1598
+ opacity: 0;
1599
+ transform: translateX(var(--bubble-dx));
1600
+ transition: opacity 0.22s ease, transform 0.22s ease;
1601
+ }
1602
+ .bubble.in {
1603
+ opacity: 1;
1604
+ transform: translateX(0);
1605
+ }
1606
+ .bubble.out {
1607
+ opacity: 0;
1608
+ transform: translateX(var(--bubble-dx));
1609
+ pointer-events: none;
1610
+ transition: opacity 0.3s ease, transform 0.3s ease;
1611
+ }
1612
+
1613
+ /* Surfaces stay neutral; the side they sit on plus the mono role label
1614
+ * carry the distinction. No tinted fills — color is the orb's job. */
1615
+ .bubble.user {
1616
+ --bubble-dx: -10px;
1617
+ align-self: flex-start;
1618
+ }
1619
+ .bubble.assistant {
1620
+ --bubble-dx: 10px;
1621
+ align-self: flex-end;
1622
+ }
1623
+ .bubble.tool {
1624
+ --bubble-dx: -10px;
1625
+ align-self: flex-start;
1626
+ display: flex;
1627
+ align-items: center;
1628
+ gap: 8px;
1629
+ color: var(--text-dim);
1630
+ font-family: var(--font-mono);
1631
+ font-size: 12px;
1632
+ }
1633
+ .bubble.tool .bubble-tool-icon {
1634
+ width: 13px;
1635
+ height: 13px;
1636
+ flex: none;
1637
+ color: var(--voice-tool);
1638
+ }
1639
+
1640
+ .bubble-role {
1641
+ font-family: var(--font-mono);
1642
+ font-size: 10px;
1643
+ font-weight: 500;
1644
+ letter-spacing: 0.1em;
1645
+ text-transform: uppercase;
1646
+ margin-bottom: 4px;
1647
+ opacity: 0.7;
1648
+ }
1649
+ .bubble.user .bubble-role { color: var(--voice-user); }
1650
+ .bubble.assistant .bubble-role { color: var(--voice-assistant); }
1651
+
1652
+ /* ─── Conversation history panel ─────────────────────────────────────── */
1653
+
1654
+ .chat-panel {
1655
+ position: fixed;
1656
+ inset: 0;
1657
+ z-index: 200;
1658
+ pointer-events: none;
1659
+ }
1660
+ .chat-panel-backdrop {
1661
+ position: absolute;
1662
+ inset: 0;
1663
+ background: rgba(8, 9, 13, 0.45);
1664
+ backdrop-filter: blur(6px);
1665
+ -webkit-backdrop-filter: blur(6px);
1666
+ opacity: 0;
1667
+ transition: opacity 0.25s ease;
1668
+ }
1669
+ .chat-panel.open .chat-panel-backdrop {
1670
+ opacity: 1;
1671
+ pointer-events: auto;
1672
+ }
1673
+ .chat-panel-inner {
1674
+ position: absolute;
1675
+ top: 0;
1676
+ right: 0;
1677
+ bottom: 0;
1678
+ width: min(360px, 90vw);
1679
+ background: var(--bg-elev);
1680
+ border-left: 1px solid var(--border);
1681
+ display: flex;
1682
+ flex-direction: column;
1683
+ transform: translateX(100%);
1684
+ transition: transform 0.28s cubic-bezier(0.32, 0.72, 0, 1);
1685
+ pointer-events: auto;
1686
+ box-shadow: -8px 0 32px rgba(0, 0, 0, 0.35);
1687
+ }
1688
+ .chat-panel.open .chat-panel-inner {
1689
+ transform: translateX(0);
1690
+ }
1691
+ .chat-panel-header {
1692
+ display: flex;
1693
+ align-items: center;
1694
+ justify-content: space-between;
1695
+ padding: 18px 20px;
1696
+ border-bottom: 1px solid var(--border);
1697
+ flex: none;
1698
+ }
1699
+ .chat-panel-header h3 {
1700
+ margin: 0;
1701
+ font-size: 14px;
1702
+ font-weight: 600;
1703
+ letter-spacing: 0.01em;
1704
+ }
1705
+ .chat-history {
1706
+ flex: 1;
1707
+ overflow-y: auto;
1708
+ padding: 16px 20px;
1709
+ display: flex;
1710
+ flex-direction: column;
1711
+ gap: 10px;
1712
+ scroll-behavior: smooth;
1713
+ }
1714
+ .chat-history::-webkit-scrollbar { width: 3px; }
1715
+ .chat-history::-webkit-scrollbar-track { background: transparent; }
1716
+ .chat-history::-webkit-scrollbar-thumb { background: var(--border-strong); border-radius: 3px; }
1717
+
1718
+ .chat-empty {
1719
+ flex: 1;
1720
+ display: flex;
1721
+ flex-direction: column;
1722
+ align-items: center;
1723
+ justify-content: center;
1724
+ gap: 6px;
1725
+ color: var(--text-faint);
1726
+ padding: 48px 0;
1727
+ opacity: 0.65;
1728
+ }
1729
+ .chat-empty svg {
1730
+ margin-bottom: 6px;
1731
+ opacity: 0.8;
1732
+ }
1733
+ .chat-empty-title {
1734
+ font-family: var(--font-mono);
1735
+ font-size: 11px;
1736
+ font-weight: 500;
1737
+ letter-spacing: 0.14em;
1738
+ text-transform: uppercase;
1739
+ }
1740
+ .chat-empty-hint {
1741
+ font-size: 12px;
1742
+ color: var(--text-faint);
1743
+ opacity: 0.7;
1744
+ }
1745
+
1746
+ /* ─── History messages ───────────────────────────────────────────────── */
1747
+
1748
+ .hist-msg {
1749
+ display: flex;
1750
+ flex-direction: column;
1751
+ gap: 3px;
1752
+ max-width: 88%;
1753
+ }
1754
+ .hist-msg.user { align-self: flex-start; }
1755
+ .hist-msg.assistant { align-self: flex-end; }
1756
+ .hist-msg.tool { align-self: flex-start; max-width: 100%; }
1757
+
1758
+ .hist-role {
1759
+ font-family: var(--font-mono);
1760
+ font-size: 10px;
1761
+ font-weight: 500;
1762
+ letter-spacing: 0.1em;
1763
+ text-transform: uppercase;
1764
+ color: var(--text-faint);
1765
+ padding: 0 2px;
1766
+ }
1767
+ .hist-msg.user .hist-role { color: color-mix(in srgb, var(--voice-user) 75%, var(--text-faint)); }
1768
+ .hist-msg.assistant .hist-role { color: color-mix(in srgb, var(--voice-assistant) 75%, var(--text-faint)); }
1769
+ .hist-msg.tool .hist-role { color: color-mix(in srgb, var(--voice-tool) 75%, var(--text-faint)); }
1770
+
1771
+ .hist-body {
1772
+ padding: 9px 12px;
1773
+ border-radius: var(--radius-md);
1774
+ font-size: 13px;
1775
+ line-height: 1.5;
1776
+ border: 1px solid var(--border);
1777
+ background: var(--bg-elev-2);
1778
+ word-break: break-word;
1779
+ }
1780
+ /* Bodies share one neutral surface; alignment + the mono role label do the
1781
+ * distinguishing, so the panel reads as one quiet column. */
1782
+ .hist-msg.user .hist-body.partial { opacity: 0.65; }
1783
+
1784
+ /* ─── Tool call history item ─────────────────────────────────────────── */
1785
+
1786
+ .hist-tool-header {
1787
+ display: flex;
1788
+ align-items: center;
1789
+ gap: 8px;
1790
+ padding: 8px 12px;
1791
+ border-radius: var(--radius-md) var(--radius-md) 0 0;
1792
+ background: color-mix(in srgb, var(--processing) 10%, var(--bg-elev-2));
1793
+ border: 1px solid color-mix(in srgb, var(--processing) 22%, var(--border));
1794
+ border-bottom: 1px solid color-mix(in srgb, var(--processing) 15%, var(--border));
1795
+ cursor: pointer;
1796
+ color: var(--text);
1797
+ font-family: inherit;
1798
+ font-size: 13px;
1799
+ font-weight: 500;
1800
+ width: 100%;
1801
+ text-align: left;
1802
+ transition: background 0.15s;
1803
+ }
1804
+ .hist-tool-header:only-child {
1805
+ border-radius: var(--radius-md);
1806
+ border-bottom: 1px solid color-mix(in srgb, var(--processing) 22%, var(--border));
1807
+ }
1808
+ .hist-tool-header:hover {
1809
+ background: color-mix(in srgb, var(--processing) 16%, var(--bg-elev-2));
1810
+ }
1811
+ .hist-tool-icon {
1812
+ width: 13px;
1813
+ height: 13px;
1814
+ flex: none;
1815
+ color: var(--processing);
1816
+ }
1817
+ .hist-tool-name {
1818
+ font-family: var(--font-mono);
1819
+ font-size: 12px;
1820
+ color: var(--voice-tool);
1821
+ font-weight: 600;
1822
+ }
1823
+ .hist-tool-chevron {
1824
+ margin-left: auto;
1825
+ width: 13px;
1826
+ height: 13px;
1827
+ color: var(--text-faint);
1828
+ transition: transform 0.2s ease;
1829
+ flex: none;
1830
+ }
1831
+ .hist-tool-header[aria-expanded="true"] .hist-tool-chevron {
1832
+ transform: rotate(180deg);
1833
+ }
1834
+ .hist-tool-body {
1835
+ padding: 10px 12px 12px;
1836
+ border-radius: 0 0 var(--radius-md) var(--radius-md);
1837
+ background: var(--bg);
1838
+ border: 1px solid color-mix(in srgb, var(--processing) 22%, var(--border));
1839
+ border-top: none;
1840
+ display: none;
1841
+ }
1842
+ .hist-tool-body.open { display: block; }
1843
+ .hist-tool-label {
1844
+ font-family: var(--font-mono);
1845
+ font-size: 10px;
1846
+ letter-spacing: 0.08em;
1847
+ text-transform: uppercase;
1848
+ color: var(--text-faint);
1849
+ margin: 8px 0 4px;
1850
+ }
1851
+ .hist-tool-label:first-child { margin-top: 0; }
1852
+ .hist-tool-block {
1853
+ font-family: var(--font-mono);
1854
+ font-size: 11px;
1855
+ line-height: 1.65;
1856
+ color: var(--text-dim);
1857
+ white-space: pre-wrap;
1858
+ word-break: break-word;
1859
+ overflow-x: auto;
1860
+ }
1861
+ .hist-tool-output { color: var(--text); }
1862
+
1863
+ /* ─── Phone layout ────────────────────────────────────────────────────────
1864
+ * On phones the floating bubble stream overlaps the orb and there isn't room
1865
+ * for it, so we drop it entirely and rely on the conversation panel (opened
1866
+ * from the top-right chat button) as the single place to read the transcript.
1867
+ * The badge still pulses there when new messages arrive while it's closed.
1868
+ * We also stack the mic / stop controls above and below the orb (instead of
1869
+ * left/right) so the wide live row never overflows, and let the panel take
1870
+ * the full width. */
1871
+ @media (max-width: 600px) {
1872
+ .bubble-stack {
1873
+ display: none;
1874
+ }
1875
+
1876
+ .topbar {
1877
+ padding: 14px 16px;
1878
+ }
1879
+
1880
+ .stage {
1881
+ padding: 16px 12px 24px;
1882
+ }
1883
+
1884
+ /* Stack vertically: mic above the orb, stop below it (DOM order is
1885
+ * mic → circle → stop). */
1886
+ .orb-wrap {
1887
+ flex-direction: column;
1888
+ gap: 14px;
1889
+ }
1890
+
1891
+ .circle {
1892
+ width: clamp(170px, 56vw, 240px);
1893
+ }
1894
+
1895
+ /* In the column layout the side controls must collapse by HEIGHT, not
1896
+ * width, so they take no vertical space until the session is live. */
1897
+ .side-btn {
1898
+ width: 44px;
1899
+ height: 0;
1900
+ transition: opacity 0.25s ease, transform 0.25s ease, height 0.25s ease,
1901
+ background 0.15s, color 0.15s, border-color 0.15s;
1902
+ }
1903
+ .side-btn svg {
1904
+ width: 19px;
1905
+ height: 19px;
1906
+ }
1907
+ .orb-wrap.live .side-btn {
1908
+ width: 44px;
1909
+ height: 44px;
1910
+ }
1911
+ .mic-gate-arc { width: 70px; height: 70px; }
1912
+
1913
+ /* Full-screen conversation on phones — feels more deliberate than a
1914
+ * narrow slide-over. It already spans top-to-bottom (inset 0); make it
1915
+ * span edge-to-edge too and drop the now-pointless border/shadow. */
1916
+ .chat-panel-inner {
1917
+ width: 100vw;
1918
+ border-left: none;
1919
+ box-shadow: none;
1920
+ }
1921
+ }
1922
+
1923
+ /* ─── About panel ─────────────────────────────────────────────────────────
1924
+ * Opened from the (i) by the wordmark. Reuses the <dialog> .modal shell.
1925
+ * Strictly monochrome per DESIGN.md — the only "color" is the orb, never
1926
+ * here. Machine identifiers (model IDs, role tags, usernames) ride Geist
1927
+ * Mono; everything human stays Inter. */
1928
+ .about-modal {
1929
+ /* Roomier on tablet/desktop; the 92vw cap keeps phones full-width. */
1930
+ width: min(680px, 92vw);
1931
+ max-height: 88vh;
1932
+ }
1933
+ .about-modal .modal-content {
1934
+ max-height: 88vh;
1935
+ overflow-y: auto;
1936
+ gap: 20px;
1937
+ padding: 26px 30px 24px;
1938
+ }
1939
+ .about-modal .modal-content::-webkit-scrollbar { width: 3px; }
1940
+ .about-modal .modal-content::-webkit-scrollbar-track { background: transparent; }
1941
+ .about-modal .modal-content::-webkit-scrollbar-thumb { background: var(--border-strong); border-radius: 3px; }
1942
+
1943
+ /* Links here are quiet: no permanent underline (the global dotted rule is
1944
+ * too loud for a dense credit block), brighten + underline on hover only. */
1945
+ .about-modal a {
1946
+ color: var(--text);
1947
+ border-bottom: none;
1948
+ text-decoration: none;
1949
+ transition: color 0.15s ease;
1950
+ }
1951
+ .about-modal a:hover {
1952
+ color: #fff;
1953
+ }
1954
+ .about-intro a:hover {
1955
+ text-decoration: underline;
1956
+ text-underline-offset: 2px;
1957
+ }
1958
+ .about-modal .ext {
1959
+ width: 12px;
1960
+ height: 12px;
1961
+ flex: none;
1962
+ opacity: 0.5;
1963
+ }
1964
+
1965
+ /* Title row: the demo name and the (i) read as one unit. */
1966
+ .ident-head {
1967
+ display: flex;
1968
+ align-items: center;
1969
+ gap: 9px;
1970
+ }
1971
+ /* (i) trigger sits right after the title. A faint outline keeps it
1972
+ * catchable without turning into a card; it fills in on hover. */
1973
+ .about-btn {
1974
+ flex: none;
1975
+ width: 34px;
1976
+ height: 34px;
1977
+ border-radius: 50%;
1978
+ background: transparent;
1979
+ border: 1px solid var(--border-strong);
1980
+ color: var(--text-dim);
1981
+ display: inline-flex;
1982
+ align-items: center;
1983
+ justify-content: center;
1984
+ cursor: pointer;
1985
+ touch-action: manipulation;
1986
+ transition: background 0.15s ease, color 0.15s ease, border-color 0.15s ease;
1987
+ }
1988
+ .about-btn:hover {
1989
+ background: var(--bg-elev);
1990
+ border-color: var(--text-dim);
1991
+ color: var(--text);
1992
+ }
1993
+ .about-btn svg {
1994
+ width: 20px;
1995
+ height: 20px;
1996
+ }
1997
+
1998
+ /* The mobile twin of (i) lives in the right-hand control cluster and is
1999
+ * hidden on desktop (the in-title one shows there instead). */
2000
+ .about-btn-mobile {
2001
+ display: none;
2002
+ }
2003
+
2004
+ /* ── Popup intro: a plain paragraph on the project + a repo link ── */
2005
+ .about-intro p {
2006
+ margin: 0;
2007
+ font-size: 14px;
2008
+ line-height: 1.6;
2009
+ color: var(--text-dim);
2010
+ }
2011
+ .about-repo {
2012
+ display: inline-flex;
2013
+ align-items: center;
2014
+ gap: 5px;
2015
+ margin-top: 12px;
2016
+ font-size: 14px;
2017
+ font-weight: 500;
2018
+ }
2019
+
2020
+ /* ── Corner identity (replaces the wordmark) ──
2021
+ * A compact stack in the topbar: title, one-line blurb, two meta rows.
2022
+ * Monochrome; only the role/name identifiers ride the machine typeface. */
2023
+ .ident {
2024
+ display: flex;
2025
+ flex-direction: column;
2026
+ gap: 10px;
2027
+ min-width: 0;
2028
+ }
2029
+ /* Reset the global underlined-anchor styling inside the identity block. */
2030
+ .ident a {
2031
+ border-bottom: none;
2032
+ color: inherit;
2033
+ transition: color 0.15s ease;
2034
+ }
2035
+ .ident a:hover {
2036
+ color: var(--text);
2037
+ text-decoration: underline;
2038
+ text-underline-offset: 2px;
2039
+ }
2040
+ .ident-title {
2041
+ font-size: 22px;
2042
+ font-weight: 600;
2043
+ letter-spacing: 0.005em;
2044
+ line-height: 1.1;
2045
+ color: var(--text);
2046
+ }
2047
+ .ident-blurb {
2048
+ margin: 0;
2049
+ max-width: 48ch;
2050
+ font-size: 14.5px;
2051
+ line-height: 1.5;
2052
+ font-weight: 400;
2053
+ color: var(--text-dim);
2054
+ }
2055
+ .ident-meta {
2056
+ display: flex;
2057
+ flex-direction: column;
2058
+ gap: 7px;
2059
+ font-size: 14px;
2060
+ font-weight: 400;
2061
+ color: var(--text-dim);
2062
+ }
2063
+ .ident-row {
2064
+ display: flex;
2065
+ flex-wrap: wrap;
2066
+ align-items: center;
2067
+ gap: 4px 8px;
2068
+ }
2069
+ .ident-label {
2070
+ font-family: var(--font-mono);
2071
+ font-size: 11px;
2072
+ font-weight: 500;
2073
+ letter-spacing: 0.1em;
2074
+ text-transform: uppercase;
2075
+ color: var(--text-faint);
2076
+ }
2077
+
2078
+ /* Shared credit bits — now used in the corner identity block. */
2079
+ .sep {
2080
+ color: var(--text-faint);
2081
+ opacity: 0.6;
2082
+ }
2083
+ .hf-credit {
2084
+ display: inline-flex;
2085
+ align-items: center;
2086
+ gap: 5px;
2087
+ }
2088
+ /* Brand marks keep their own color — a deliberate exception to the
2089
+ * monochrome rule, for the HF and Cerebras logos only. */
2090
+ .hf-mark {
2091
+ width: 14px;
2092
+ height: 14px;
2093
+ flex: none;
2094
+ color: #ffd21e;
2095
+ }
2096
+ .cerebras-credit {
2097
+ display: inline-flex;
2098
+ align-items: center;
2099
+ gap: 5px;
2100
+ }
2101
+ .cerebras-mark {
2102
+ width: 14px;
2103
+ height: 14px;
2104
+ flex: none;
2105
+ }
2106
+ /* Usernames are identifiers, so they ride the machine typeface. */
2107
+ .handle {
2108
+ font-family: var(--font-mono);
2109
+ font-size: 13.5px;
2110
+ }
2111
+
2112
+ /* ── Pipeline (the signal flow) ── */
2113
+ .about-pipeline {
2114
+ border-top: 1px solid var(--border);
2115
+ padding-top: 16px;
2116
+ }
2117
+ .pipeline-title {
2118
+ margin: 0 0 14px;
2119
+ font-family: var(--font-mono);
2120
+ font-size: 11px;
2121
+ font-weight: 500;
2122
+ letter-spacing: 0.14em;
2123
+ text-transform: uppercase;
2124
+ color: var(--text-faint);
2125
+ }
2126
+ .pipeline {
2127
+ list-style: none;
2128
+ margin: 0;
2129
+ padding: 0;
2130
+ position: relative;
2131
+ }
2132
+ /* One continuous rail threading every node, dot centers at x=8.5px. */
2133
+ .pipeline::before {
2134
+ content: "";
2135
+ position: absolute;
2136
+ left: 8px;
2137
+ top: 7px;
2138
+ bottom: 7px;
2139
+ width: 1px;
2140
+ background: var(--border-strong);
2141
+ }
2142
+ .pipeline > li {
2143
+ position: relative;
2144
+ padding-left: 30px;
2145
+ }
2146
+ /* Stages: solid node. Endpoints (you / orb): hollow node. */
2147
+ .pipe-stage::before {
2148
+ content: "";
2149
+ position: absolute;
2150
+ left: 5px;
2151
+ top: 4px;
2152
+ width: 7px;
2153
+ height: 7px;
2154
+ border-radius: 50%;
2155
+ background: var(--text-dim);
2156
+ }
2157
+ .pipe-endpoint::before {
2158
+ content: "";
2159
+ position: absolute;
2160
+ left: 5px;
2161
+ top: 3px;
2162
+ width: 7px;
2163
+ height: 7px;
2164
+ border-radius: 50%;
2165
+ border: 1px solid var(--text-faint);
2166
+ background: var(--bg-elev);
2167
+ }
2168
+ .pipe-endpoint {
2169
+ font-family: var(--font-mono);
2170
+ font-size: 11px;
2171
+ font-weight: 500;
2172
+ letter-spacing: 0.12em;
2173
+ text-transform: uppercase;
2174
+ color: var(--text-faint);
2175
+ padding-bottom: 12px;
2176
+ }
2177
+ .pipeline > li.pipe-endpoint:last-child {
2178
+ padding-bottom: 0;
2179
+ }
2180
+ .pipe-stage {
2181
+ padding-bottom: 16px;
2182
+ }
2183
+ .pipe-tag {
2184
+ font-family: var(--font-mono);
2185
+ font-size: 12px;
2186
+ font-weight: 600;
2187
+ letter-spacing: 0.08em;
2188
+ color: var(--text);
2189
+ margin-right: 9px;
2190
+ }
2191
+ .pipe-job {
2192
+ font-size: 14px;
2193
+ color: var(--text-dim);
2194
+ }
2195
+ /* Middot between the job and its model link, matching the separators
2196
+ * used elsewhere. */
2197
+ .pipe-job::after {
2198
+ content: "·";
2199
+ margin: 0 7px;
2200
+ color: var(--text-faint);
2201
+ }
2202
+ .pipe-note {
2203
+ color: var(--text-faint);
2204
+ }
2205
+ /* The Cerebras link stays as quiet as the note; brightens + underlines on hover. */
2206
+ .pipe-note a {
2207
+ color: inherit;
2208
+ }
2209
+ .pipe-note a:hover {
2210
+ color: var(--text);
2211
+ text-decoration: underline;
2212
+ text-underline-offset: 2px;
2213
+ }
2214
+ .pipe-model {
2215
+ display: inline-flex;
2216
+ align-items: center;
2217
+ gap: 4px;
2218
+ margin-top: 4px;
2219
+ font-family: var(--font-mono);
2220
+ font-size: 12px;
2221
+ color: var(--text-dim);
2222
+ word-break: break-all;
2223
+ }
2224
+ .pipe-model:hover {
2225
+ color: #fff;
2226
+ text-decoration: underline;
2227
+ text-underline-offset: 2px;
2228
+ }
2229
+ .pipe-model .ext {
2230
+ width: 11px;
2231
+ height: 11px;
2232
+ }
2233
+
2234
+ /* "Interrupted" tag on an assistant reply the user barged in on. */
2235
+ .hist-note {
2236
+ font-family: var(--font-mono);
2237
+ font-size: 10px;
2238
+ font-weight: 500;
2239
+ letter-spacing: 0.1em;
2240
+ text-transform: uppercase;
2241
+ color: var(--text-faint);
2242
+ padding: 0 2px;
2243
+ }
2244
+ .hist-msg.assistant .hist-note {
2245
+ align-self: flex-end;
2246
+ }
2247
+ .hist-msg.interrupted .hist-body {
2248
+ opacity: 0.7;
2249
+ }
2250
+
2251
+ /* Captured webcam frame shown in the transcript (camera tool result). */
2252
+ .hist-image {
2253
+ display: block;
2254
+ width: 100%;
2255
+ max-width: 240px;
2256
+ border-radius: var(--radius-sm);
2257
+ border: 1px solid var(--border);
2258
+ margin-top: 2px;
2259
+ }
2260
+
2261
+ /* ─── Tools panel ──────────────────────────────────────────────────────────
2262
+ * Reuses the modal/field shell. Switches are monochrome (checked = near-white,
2263
+ * the same high-contrast treatment as the primary button): color belongs to
2264
+ * the voice, not the chrome. */
2265
+ .tools-intro {
2266
+ margin: 0;
2267
+ font-size: 13px;
2268
+ line-height: 1.5;
2269
+ color: var(--text-dim);
2270
+ }
2271
+ .tool-list {
2272
+ display: flex;
2273
+ flex-direction: column;
2274
+ }
2275
+ .tool-row {
2276
+ display: flex;
2277
+ align-items: center;
2278
+ justify-content: space-between;
2279
+ gap: 16px;
2280
+ padding: 12px 0;
2281
+ }
2282
+ .tool-row-sep {
2283
+ border-top: 1px solid var(--border);
2284
+ margin-top: 4px;
2285
+ }
2286
+ .tool-info {
2287
+ display: flex;
2288
+ flex-direction: column;
2289
+ gap: 3px;
2290
+ min-width: 0;
2291
+ }
2292
+ .tool-name {
2293
+ font-size: 14px;
2294
+ font-weight: 600;
2295
+ color: var(--text);
2296
+ }
2297
+ .tool-desc {
2298
+ font-size: 12.5px;
2299
+ color: var(--text-dim);
2300
+ }
2301
+ .tool-row.disabled .tool-name,
2302
+ .tool-row.disabled .tool-desc {
2303
+ opacity: 0.5;
2304
+ }
2305
+ .tool-hint {
2306
+ display: block;
2307
+ font-size: 12px;
2308
+ line-height: 1.4;
2309
+ color: var(--text-faint);
2310
+ }
2311
+ .tools-key {
2312
+ margin: 0 0 4px;
2313
+ }
2314
+
2315
+ /* Toggle switch */
2316
+ .switch {
2317
+ position: relative;
2318
+ display: inline-flex;
2319
+ flex: none;
2320
+ width: 40px;
2321
+ height: 24px;
2322
+ cursor: pointer;
2323
+ }
2324
+ .switch input {
2325
+ position: absolute;
2326
+ inset: 0;
2327
+ width: 100%;
2328
+ height: 100%;
2329
+ margin: 0;
2330
+ opacity: 0;
2331
+ cursor: pointer;
2332
+ }
2333
+ .switch-track {
2334
+ position: absolute;
2335
+ inset: 0;
2336
+ border-radius: 999px;
2337
+ background: var(--bg);
2338
+ border: 1px solid var(--border-strong);
2339
+ transition: background 0.15s, border-color 0.15s;
2340
+ }
2341
+ .switch-track::after {
2342
+ content: "";
2343
+ position: absolute;
2344
+ top: 50%;
2345
+ left: 3px;
2346
+ transform: translateY(-50%);
2347
+ width: 16px;
2348
+ height: 16px;
2349
+ border-radius: 50%;
2350
+ background: var(--text-dim);
2351
+ transition: transform 0.18s ease, background 0.15s;
2352
+ }
2353
+ .switch input:checked + .switch-track {
2354
+ background: var(--text);
2355
+ border-color: var(--text);
2356
+ }
2357
+ .switch input:checked + .switch-track::after {
2358
+ transform: translate(16px, -50%);
2359
+ background: var(--bg);
2360
+ }
2361
+ .switch input:focus-visible + .switch-track {
2362
+ outline: 2px solid var(--text-dim);
2363
+ outline-offset: 2px;
2364
+ }
2365
+ .switch input:disabled {
2366
+ cursor: not-allowed;
2367
+ }
2368
+ .switch input:disabled + .switch-track {
2369
+ opacity: 0.5;
2370
+ }
2371
+
2372
+ /* ─── Webcam preview (camera tool) ──────────────────────────────────────────
2373
+ * Floating self-view, bottom-left. Mirrored for the user; the frame sent to
2374
+ * the model is drawn un-mirrored (see captureSnapshot). */
2375
+ .cam-pip {
2376
+ position: fixed;
2377
+ left: 20px;
2378
+ bottom: 20px;
2379
+ width: 280px;
2380
+ aspect-ratio: 4 / 3;
2381
+ border-radius: var(--radius-md);
2382
+ overflow: hidden;
2383
+ border: 1px solid var(--border-strong);
2384
+ background: var(--bg-elev);
2385
+ box-shadow: var(--shadow-soft);
2386
+ z-index: 90;
2387
+ opacity: 0;
2388
+ transform: translateY(8px) scale(0.96);
2389
+ pointer-events: none;
2390
+ transition: opacity 0.22s ease, transform 0.22s ease;
2391
+ }
2392
+ .cam-pip.visible {
2393
+ opacity: 1;
2394
+ transform: none;
2395
+ }
2396
+ .cam-video {
2397
+ display: block;
2398
+ width: 100%;
2399
+ height: 100%;
2400
+ object-fit: cover;
2401
+ transform: scaleX(-1); /* mirror the self-view only */
2402
+ background: var(--bg);
2403
+ }
2404
+ .cam-label {
2405
+ position: absolute;
2406
+ left: 8px;
2407
+ bottom: 6px;
2408
+ font-family: var(--font-mono);
2409
+ font-size: 9.5px;
2410
+ letter-spacing: 0.14em;
2411
+ text-transform: uppercase;
2412
+ color: var(--text);
2413
+ opacity: 0.8;
2414
+ text-shadow: 0 1px 3px rgba(0, 0, 0, 0.7);
2415
+ }
2416
+ .cam-flash {
2417
+ position: absolute;
2418
+ inset: 0;
2419
+ background: #fff;
2420
+ opacity: 0;
2421
+ pointer-events: none;
2422
+ }
2423
+ .cam-pip.flash .cam-flash {
2424
+ animation: cam-flash 0.4s ease;
2425
+ }
2426
+ @keyframes cam-flash {
2427
+ 0% { opacity: 0; }
2428
+ 12% { opacity: 0.85; }
2429
+ 100% { opacity: 0; }
2430
+ }
2431
+ @media (max-width: 600px) {
2432
+ /* Bottom-centred on phones. Auto margins centre it without touching the
2433
+ * transform, so the slide/scale-in animation still works. */
2434
+ .cam-pip {
2435
+ left: 0;
2436
+ right: 0;
2437
+ margin-inline: auto;
2438
+ bottom: 16px;
2439
+ width: min(188px, 52vw);
2440
+ }
2441
+ /* The credit would sit under the centred preview, so drop it while the
2442
+ * camera is on. */
2443
+ body.cam-on .footer {
2444
+ display: none;
2445
+ }
2446
+ }
2447
+ @media (prefers-reduced-motion: reduce) {
2448
+ .cam-pip {
2449
+ transition: opacity 0.22s ease;
2450
+ transform: none;
2451
+ }
2452
+ .cam-pip.flash .cam-flash {
2453
+ animation: none;
2454
+ }
2455
+ }
2456
+
2457
+ /* ─── Desktop type scale ────────────────────────────────────────────────────
2458
+ * A uniform step up (~+1px, title +2) for all UI text on larger screens.
2459
+ * Scoped to min-width: 601px so it can't reach phones — the ≤600px layout
2460
+ * keeps every size exactly as it was. */
2461
+ @media (min-width: 601px) {
2462
+ .cam-label { font-size: 10.5px; }
2463
+
2464
+ .bubble-role,
2465
+ .hist-role { font-size: 11px; }
2466
+
2467
+ .circle-caption,
2468
+ .footer,
2469
+ .chat-empty-title,
2470
+ .hist-tool-block,
2471
+ .ident-label,
2472
+ .pipeline-title,
2473
+ .pipe-endpoint { font-size: 12px; }
2474
+
2475
+ .field > span,
2476
+ .field small,
2477
+ .bubble.tool,
2478
+ .chat-empty-hint,
2479
+ .hist-tool-name,
2480
+ .pipe-tag,
2481
+ .pipe-model,
2482
+ .tool-hint { font-size: 13px; }
2483
+
2484
+ .tool-desc { font-size: 13.5px; }
2485
+
2486
+ .btn,
2487
+ .bubble,
2488
+ .field,
2489
+ .hist-body,
2490
+ .hist-tool-header,
2491
+ .tools-intro { font-size: 14px; }
2492
+
2493
+ .handle { font-size: 14.5px; }
2494
+
2495
+ .about-intro p,
2496
+ .about-repo,
2497
+ .brand,
2498
+ .chat-panel-header h3,
2499
+ .field textarea,
2500
+ .ident-meta,
2501
+ .pipe-job,
2502
+ .tool-name { font-size: 15px; }
2503
+
2504
+ .ident-blurb { font-size: 15.5px; }
2505
+
2506
+ .modal-header h2 { font-size: 17px; }
2507
+
2508
+ .ident-title { font-size: 24px; }
2509
+ }
ui/account.js ADDED
@@ -0,0 +1,178 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * Account — the HF login chip and the daily-limit modal.
4
+ *
5
+ * Reads `/api/me` to learn the current tier (anonymous / signed-in / PRO) and
6
+ * remaining daily talk-time, renders a sign-in pill or a signed-in chip with a
7
+ * small popover (tier, remaining, sign out, upgrade), and shows the limit modal
8
+ * when a conversation is refused or cut. The time metering itself lives in
9
+ * main.js (heartbeat loop) + the server; this module is just the surface.
10
+ *
11
+ * Inert unless the deploy is in LB mode (`/api/me` → `{enabled:true}`).
12
+ */
13
+
14
+ import { $, escHtml } from "./dom.js";
15
+
16
+ const PRO_URL = "https://huggingface.co/subscribe/pro";
17
+
18
+ // Official multi-color Hugging Face logo, used in the badge + sign-in CTA.
19
+ const HF_MARK = `<svg class="hf-logo" viewBox="0 0 95 88" fill="none" aria-hidden="true"><path fill="#FFD21E" d="M47.21 76.5a34.75 34.75 0 1 0 0-69.5 34.75 34.75 0 0 0 0 69.5Z"/><path fill="#FF9D0B" d="M81.96 41.75a34.75 34.75 0 1 0-69.5 0 34.75 34.75 0 0 0 69.5 0Zm-73.5 0a38.75 38.75 0 1 1 77.5 0 38.75 38.75 0 0 1-77.5 0Z"/><path fill="#3A3B45" d="M58.5 32.3c1.28.44 1.78 3.06 3.07 2.38a5 5 0 1 0-6.76-2.07c.61 1.15 2.55-.72 3.7-.32ZM34.95 32.3c-1.28.44-1.79 3.06-3.07 2.38a5 5 0 1 1 6.76-2.07c-.61 1.15-2.56-.72-3.7-.32Z"/><path fill="#FF323D" d="M46.96 56.29c9.83 0 13-8.76 13-13.26 0-2.34-1.57-1.6-4.09-.36-2.33 1.15-5.46 2.74-8.9 2.74-7.19 0-13-6.88-13-2.38s3.16 13.26 13 13.26Z"/><path fill="#3A3B45" fill-rule="evenodd" d="M39.43 54a8.7 8.7 0 0 1 5.3-4.49c.4-.12.81.57 1.24 1.28.4.68.82 1.37 1.24 1.37.45 0 .9-.68 1.33-1.35.45-.7.89-1.38 1.32-1.25a8.61 8.61 0 0 1 5 4.17c3.73-2.94 5.1-7.74 5.1-10.7 0-2.34-1.57-1.6-4.09-.36l-.14.07c-2.31 1.15-5.39 2.67-8.77 2.67s-6.45-1.52-8.77-2.67c-2.6-1.29-4.23-2.1-4.23.29 0 3.05 1.46 8.06 5.47 10.97Z" clip-rule="evenodd"/><path fill="#FF9D0B" d="M70.71 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM24.21 37a3.25 3.25 0 1 0 0-6.5 3.25 3.25 0 0 0 0 6.5ZM17.52 48c-1.62 0-3.06.66-4.07 1.87a5.97 5.97 0 0 0-1.33 3.76 7.1 7.1 0 0 0-1.94-.3c-1.55 0-2.95.59-3.94 1.66a5.8 5.8 0 0 0-.8 7 5.3 5.3 0 0 0-1.79 2.82c-.24.9-.48 2.8.8 4.74a5.22 5.22 0 0 0-.37 5.02c1.02 2.32 3.57 4.14 8.52 6.1 3.07 1.22 5.89 2 5.91 2.01a44.33 44.33 0 0 0 10.93 1.6c5.86 0 10.05-1.8 12.46-5.34 3.88-5.69 3.33-10.9-1.7-15.92-2.77-2.78-4.62-6.87-5-7.77-.78-2.66-2.84-5.62-6.25-5.62a5.7 5.7 0 0 0-4.6 2.46c-1-1.26-1.98-2.25-2.86-2.82A7.4 7.4 0 0 0 17.52 48Zm0 4c.51 0 1.14.22 1.82.65 2.14 1.36 6.25 8.43 7.76 11.18.5.92 1.37 1.31 2.14 1.31 1.55 0 2.75-1.53.15-3.48-3.92-2.93-2.55-7.72-.68-8.01.08-.02.17-.02.24-.02 1.7 0 2.45 2.93 2.45 2.93s2.2 5.52 5.98 9.3c3.77 3.77 3.97 6.8 1.22 10.83-1.88 2.75-5.47 3.58-9.16 3.58-3.81 0-7.73-.9-9.92-1.46-.11-.03-13.45-3.8-11.76-7 .28-.54.75-.76 1.34-.76 2.38 0 6.7 3.54 8.57 3.54.41 0 .7-.17.83-.6.79-2.85-12.06-4.05-10.98-8.17.2-.73.71-1.02 1.44-1.02 3.14 0 10.2 5.53 11.68 5.53.11 0 .2-.03.24-.1.74-1.2.33-2.04-4.9-5.2-5.21-3.16-8.88-5.06-6.8-7.33.24-.26.58-.38 1-.38 3.17 0 10.66 6.82 10.66 6.82s2.02 2.1 3.25 2.1c.28 0 .52-.1.68-.38.86-1.46-8.06-8.22-8.56-11.01-.34-1.9.24-2.85 1.31-2.85Z"/><path fill="#FFD21E" d="M38.6 76.69c2.75-4.04 2.55-7.07-1.22-10.84-3.78-3.77-5.98-9.3-5.98-9.3s-.82-3.2-2.69-2.9c-1.87.3-3.24 5.08.68 8.01 3.91 2.93-.78 4.92-2.29 2.17-1.5-2.75-5.62-9.82-7.76-11.18-2.13-1.35-3.63-.6-3.13 2.2.5 2.79 9.43 9.55 8.56 11-.87 1.47-3.93-1.71-3.93-1.71s-9.57-8.71-11.66-6.44c-2.08 2.27 1.59 4.17 6.8 7.33 5.23 3.16 5.64 4 4.9 5.2-.75 1.2-12.28-8.53-13.36-4.4-1.08 4.11 11.77 5.3 10.98 8.15-.8 2.85-9.06-5.38-10.74-2.18-1.7 3.21 11.65 6.98 11.76 7.01 4.3 1.12 15.25 3.49 19.08-2.12Z"/><path fill="#FF9D0B" d="M77.4 48c1.62 0 3.07.66 4.07 1.87a5.97 5.97 0 0 1 1.33 3.76 7.1 7.1 0 0 1 1.95-.3c1.55 0 2.95.59 3.94 1.66a5.8 5.8 0 0 1 .8 7 5.3 5.3 0 0 1 1.78 2.82c.24.9.48 2.8-.8 4.74a5.22 5.22 0 0 1 .37 5.02c-1.02 2.32-3.57 4.14-8.51 6.1-3.08 1.22-5.9 2-5.92 2.01a44.33 44.33 0 0 1-10.93 1.6c-5.86 0-10.05-1.8-12.46-5.34-3.88-5.69-3.33-10.9 1.7-15.92 2.78-2.78 4.63-6.87 5.01-7.77.78-2.66 2.83-5.62 6.24-5.62a5.7 5.7 0 0 1 4.6 2.46c1-1.26 1.98-2.25 2.87-2.82A7.4 7.4 0 0 1 77.4 48Zm0 4c-.51 0-1.13.22-1.82.65-2.13 1.36-6.25 8.43-7.76 11.18a2.43 2.43 0 0 1-2.14 1.31c-1.54 0-2.75-1.53-.14-3.48 3.91-2.93 2.54-7.72.67-8.01a1.54 1.54 0 0 0-.24-.02c-1.7 0-2.45 2.93-2.45 2.93s-2.2 5.52-5.97 9.3c-3.78 3.77-3.98 6.8-1.22 10.83 1.87 2.75 5.47 3.58 9.15 3.58 3.82 0 7.73-.9 9.93-1.46.1-.03 13.45-3.8 11.76-7-.29-.54-.75-.76-1.34-.76-2.38 0-6.71 3.54-8.57 3.54-.42 0-.71-.17-.83-.6-.8-2.85 12.05-4.05 10.97-8.17-.19-.73-.7-1.02-1.44-1.02-3.14 0-10.2 5.53-11.68 5.53-.1 0-.19-.03-.23-.1-.74-1.2-.34-2.04 4.88-5.2 5.23-3.16 8.9-5.06 6.8-7.33-.23-.26-.57-.38-.98-.38-3.18 0-10.67 6.82-10.67 6.82s-2.02 2.1-3.24 2.1a.74.74 0 0 1-.68-.38c-.87-1.46 8.05-8.22 8.55-11.01.34-1.9-.24-2.85-1.31-2.85Z"/><path fill="#FFD21E" d="M56.33 76.69c-2.75-4.04-2.56-7.07 1.22-10.84 3.77-3.77 5.97-9.3 5.97-9.3s.82-3.2 2.7-2.9c1.86.3 3.23 5.08-.68 8.01-3.92 2.93.78 4.92 2.28 2.17 1.51-2.75 5.63-9.82 7.76-11.18 2.13-1.35 3.64-.6 3.13 2.2-.5 2.79-9.42 9.55-8.55 11 .86 1.47 3.92-1.71 3.92-1.71s9.58-8.71 11.66-6.44c2.08 2.27-1.58 4.17-6.8 7.33-5.23 3.16-5.63 4-4.9 5.2.75 1.2 12.28-8.53 13.36-4.4 1.08 4.11-11.76 5.3-10.97 8.15.8 2.85 9.05-5.38 10.74-2.18 1.69 3.21-11.65 6.98-11.76 7.01-4.31 1.12-15.26 3.49-19.08-2.12Z"/></svg>`;
20
+
21
+ /** @param {number} sec @returns {string} "m:ss" */
22
+ function fmt(sec) {
23
+ const s = Math.max(0, Math.round(sec));
24
+ return `${Math.floor(s / 60)}:${String(s % 60).padStart(2, "0")}`;
25
+ }
26
+
27
+ export class Account {
28
+ constructor() {
29
+ /** @type {HTMLElement} */
30
+ this._root = $("#account");
31
+ /** @type {HTMLDialogElement} */
32
+ this._modal = $("#limit-modal");
33
+ this._modalTitle = $("#limit-title");
34
+ this._modalMsg = $("#limit-msg");
35
+ this._modalNote = $("#limit-note");
36
+ /** @type {HTMLAnchorElement} */
37
+ this._modalCta = /** @type {any} */ ($("#limit-cta"));
38
+
39
+ /** @type {{enabled:boolean, auth?:boolean, loggedIn?:boolean, username?:string, avatar?:string, tier?:string, remainingSec?:number|null, limitSec?:number|null, loginUrl?:string|null, logoutUrl?:string|null}} */
40
+ this._me = { enabled: false };
41
+ this._popoverOpen = false;
42
+
43
+ $("#limit-close").addEventListener("click", () => this._modal.close());
44
+ this._modal.addEventListener("click", (e) => {
45
+ if (e.target === this._modal) this._modal.close();
46
+ });
47
+ // Close the popover on an outside click.
48
+ document.addEventListener("click", (e) => {
49
+ if (this._popoverOpen && !this._root.contains(/** @type {Node} */ (e.target))) {
50
+ this._closePopover();
51
+ }
52
+ });
53
+ }
54
+
55
+ get tier() {
56
+ return this._me.tier || "anon";
57
+ }
58
+
59
+ /** Fetch `/api/me` and (re)render the chip. Safe to call repeatedly (load,
60
+ * after the OAuth redirect, after a conversation ends). */
61
+ async refresh() {
62
+ try {
63
+ const res = await fetch("api/me");
64
+ this._me = res.ok ? await res.json() : { enabled: false };
65
+ } catch {
66
+ this._me = { enabled: false };
67
+ }
68
+ this._render();
69
+ }
70
+
71
+ _render() {
72
+ const me = this._me;
73
+ if (!me.enabled) {
74
+ this._root.hidden = true;
75
+ this._root.innerHTML = "";
76
+ return;
77
+ }
78
+ this._root.hidden = false;
79
+
80
+ if (!me.loggedIn) {
81
+ // Signed-out: a sign-in pill (only when OAuth is actually available).
82
+ if (me.auth && me.loginUrl) {
83
+ this._root.innerHTML = `<a class="signin-pill" href="${escHtml(me.loginUrl)}" title="Sign in for more time"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M15 3h4a2 2 0 0 1 2 2v14a2 2 0 0 1-2 2h-4"/><polyline points="10 17 15 12 10 7"/><line x1="15" y1="12" x2="3" y2="12"/></svg><span>Sign in</span></a>`;
84
+ } else {
85
+ this._root.innerHTML = "";
86
+ this._root.hidden = true;
87
+ }
88
+ return;
89
+ }
90
+
91
+ // Signed-in: avatar + handle chip that toggles a popover.
92
+ const isPro = me.tier === "pro";
93
+ // Org members get unlimited usage too, but aren't PRO — don't brand them so.
94
+ const isUnlimited = isPro || me.tier === "org";
95
+ const avatar = me.avatar
96
+ ? `<img class="account-avatar" src="${escHtml(me.avatar)}" alt="" />`
97
+ : `<span class="account-avatar account-avatar-fallback">${escHtml((me.username || "?")[0].toUpperCase())}</span>`;
98
+ const remaining =
99
+ isUnlimited || me.remainingSec == null
100
+ ? "Unlimited"
101
+ : `${fmt(me.remainingSec)} left today`;
102
+ const tierLabel = isPro ? "PRO" : isUnlimited ? "Team" : "Free";
103
+
104
+ this._root.innerHTML = `
105
+ <button id="account-chip" class="account-chip" aria-haspopup="true" aria-expanded="false">
106
+ ${avatar}
107
+ <span class="account-handle">${escHtml(me.username || "you")}</span>
108
+ ${isPro
109
+ ? '<span class="account-pro">PRO</span>'
110
+ : me.tier === "org"
111
+ ? '<span class="account-pro account-team">TEAM</span>'
112
+ : ""}
113
+ </button>
114
+ <div id="account-pop" class="account-pop" hidden>
115
+ <div class="account-pop-row account-pop-name">${escHtml(me.username || "you")}</div>
116
+ <div class="account-pop-row account-pop-meta">
117
+ <span class="account-tier">${tierLabel}</span>
118
+ <span class="account-remaining">${escHtml(remaining)}</span>
119
+ </div>
120
+ ${isUnlimited ? "" : `<a class="account-pop-link" href="${PRO_URL}" target="_blank" rel="noopener">Upgrade to PRO</a>`}
121
+ <a class="account-pop-link account-signout" href="${escHtml(me.logoutUrl || "#")}">Sign out</a>
122
+ </div>`;
123
+
124
+ const chip = $("#account-chip");
125
+ chip.addEventListener("click", (e) => {
126
+ e.stopPropagation();
127
+ this._popoverOpen ? this._closePopover() : this._openPopover();
128
+ });
129
+ }
130
+
131
+ _openPopover() {
132
+ const pop = document.getElementById("account-pop");
133
+ const chip = document.getElementById("account-chip");
134
+ if (!pop || !chip) return;
135
+ pop.hidden = false;
136
+ chip.setAttribute("aria-expanded", "true");
137
+ this._popoverOpen = true;
138
+ }
139
+
140
+ _closePopover() {
141
+ const pop = document.getElementById("account-pop");
142
+ const chip = document.getElementById("account-chip");
143
+ if (pop) pop.hidden = true;
144
+ if (chip) chip.setAttribute("aria-expanded", "false");
145
+ this._popoverOpen = false;
146
+ }
147
+
148
+ /**
149
+ * Show the limit modal for a tier — used both when a conversation is refused
150
+ * at start (402) and when a live one is cut (heartbeat `expired`).
151
+ * @param {string} [tier]
152
+ */
153
+ showLimit(tier = this.tier) {
154
+ const canSignIn = this._me.auth && this._me.loginUrl;
155
+ this._modalTitle.textContent = "Thanks for chatting!";
156
+ if (tier === "anon") {
157
+ this._modalMsg.textContent =
158
+ "Guest conversations run for 5 minutes. Sign in with Hugging Face to get 10 minutes a day for free, and PRO members chat with no limit at all.";
159
+ this._modalNote.textContent = "Your free minutes refresh tomorrow.";
160
+ if (canSignIn) {
161
+ this._modalCta.innerHTML = `${HF_MARK}<span>Sign in with Hugging Face</span>`;
162
+ this._modalCta.href = /** @type {string} */ (this._me.loginUrl);
163
+ this._modalCta.hidden = false;
164
+ } else {
165
+ this._modalCta.hidden = true;
166
+ }
167
+ } else {
168
+ // Signed-in, non-PRO.
169
+ this._modalMsg.textContent =
170
+ "You've enjoyed your 10 minutes for today. Go PRO for unlimited conversations and to support open source AI.";
171
+ this._modalNote.textContent = "Or come back tomorrow. Your minutes reset daily.";
172
+ this._modalCta.innerHTML = "<span>Upgrade to PRO</span>";
173
+ this._modalCta.href = PRO_URL;
174
+ this._modalCta.hidden = false;
175
+ }
176
+ if (!this._modal.open) this._modal.showModal();
177
+ }
178
+ }
ui/chat.js ADDED
@@ -0,0 +1,425 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * ChatView — owns the whole conversation surface: the slide-in history panel,
4
+ * the ephemeral on-orb bubbles, and all the transcript/tool/streaming
5
+ * bookkeeping. main.js wires the realtime client's events straight to the
6
+ * `on*` methods here and otherwise doesn't touch chat state.
7
+ *
8
+ * Two parallel surfaces share one shape (see `_buildMessageEl`):
9
+ * - ephemeral bubbles (`.bubble` / `.bubble-*`) fade on a timer
10
+ * - persistent history (`.hist-msg` / `.hist-*`) the durable panel log
11
+ *
12
+ * Keying:
13
+ * - user transcripts by the server's `item_id` — a speculative continuation
14
+ * REUSES it, so both segments land in one row/bubble; deltas are CUMULATIVE
15
+ * (each carries the full sentence so far), so we replace text wholesale.
16
+ * - assistant transcripts by `response_id`, so a cancelled speculative reply
17
+ * can be marked interrupted without erasing what was already shown.
18
+ */
19
+
20
+ import { $, escHtml, DEBUG } from "./dom.js";
21
+
22
+ const WRENCH_PATH = `<path d="M14.7 6.3a1 1 0 0 0 0 1.4l1.6 1.6a1 1 0 0 0 1.4 0l3.77-3.77a6 6 0 0 1-7.94 7.94l-6.91 6.91a2.12 2.12 0 0 1-3-3l6.91-6.91a6 6 0 0 1 7.94-7.94l-3.76 3.76z"/>`;
23
+ const CHAT_BUBBLE_SVG = `<svg width="28" height="28" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/></svg>`;
24
+ const EMPTY_STATE_HTML = `<div id="chat-empty" class="chat-empty">${CHAT_BUBBLE_SVG}<span class="chat-empty-title">No messages yet</span><span class="chat-empty-hint">Tap the orb and start talking</span></div>`;
25
+
26
+ export class ChatView {
27
+ constructor() {
28
+ /** @type {HTMLButtonElement} */
29
+ this._chatBtn = $("#chat-btn");
30
+ /** @type {HTMLSpanElement} */
31
+ this._chatBadge = $("#chat-badge");
32
+ /** @type {HTMLDivElement} */
33
+ this._chatPanel = $("#chat-panel");
34
+ /** @type {HTMLDivElement} */
35
+ this._chatPanelBackdrop = $("#chat-panel-backdrop");
36
+ /** @type {HTMLButtonElement} */
37
+ this._chatPanelClose = $("#chat-panel-close");
38
+ /** @type {HTMLDivElement} */
39
+ this._chatHistory = $("#chat-history");
40
+ /** @type {HTMLDivElement} */
41
+ this._bubbleStack = $("#bubble-stack");
42
+
43
+ this._panelOpen = false;
44
+ this._scrollQueued = false;
45
+
46
+ // ── User transcript state (keyed by item_id) ───────────────────────────
47
+ /** @type {Map<string, HTMLElement>} */
48
+ this._userHistByItem = new Map();
49
+ /** @type {HTMLElement | null} */
50
+ this._activeUserBubble = null;
51
+ this._activeUserItemId = "";
52
+ // Monotonic counter for synthesizing unique keys when the server omits an
53
+ // item_id / response_id, so id-less messages never collapse onto each other.
54
+ this._anonSeq = 0;
55
+
56
+ // ── Assistant transcript state (keyed by response_id) ──────────────────
57
+ /** @type {Map<string, { bubble: HTMLElement, hist: HTMLElement }>} */
58
+ this._asstByResp = new Map();
59
+
60
+ // ── Ephemeral bubble auto-dismiss ──────────────────────────────────────
61
+ // Per-element expiry (epoch ms). A bubble fades once its expiry passes —
62
+ // but only in stack order (see _reapBubbles). Refreshing the expiry keeps a
63
+ // bubble alive while it updates (e.g. the live user utterance).
64
+ /** @type {WeakMap<HTMLElement, number>} */
65
+ this._bubbleExpiry = new WeakMap();
66
+ // Single pending reaper handle: one timer for the whole stack (not one per
67
+ // bubble) so dismissal is strictly oldest-first regardless of per-bubble delays.
68
+ this._reaperHandle = 0;
69
+
70
+ this._chatBtn.addEventListener("click", () => (this._panelOpen ? this._closePanel() : this._openPanel()));
71
+ this._chatPanelClose.addEventListener("click", () => this._closePanel());
72
+ this._chatPanelBackdrop.addEventListener("click", () => this._closePanel());
73
+ document.addEventListener("keydown", (e) => {
74
+ if (e.key === "Escape" && this._panelOpen) this._closePanel();
75
+ });
76
+ }
77
+
78
+ // ── Panel ───────────────────────────────────────────────────────────────
79
+
80
+ _openPanel() {
81
+ this._panelOpen = true;
82
+ this._chatPanel.classList.add("open");
83
+ this._chatBadge.classList.remove("visible");
84
+ this._scrollToBottom();
85
+ }
86
+
87
+ _closePanel() {
88
+ this._panelOpen = false;
89
+ this._chatPanel.classList.remove("open");
90
+ }
91
+
92
+ // Coalesce scroll-to-bottom: a burst of cumulative transcript deltas would
93
+ // otherwise queue one rAF per delta, all writing the same scrollTop.
94
+ _scrollToBottom() {
95
+ if (!this._panelOpen || this._scrollQueued) return;
96
+ this._scrollQueued = true;
97
+ requestAnimationFrame(() => {
98
+ this._scrollQueued = false;
99
+ this._chatHistory.scrollTop = this._chatHistory.scrollHeight;
100
+ });
101
+ }
102
+
103
+ _markUnread() {
104
+ if (this._panelOpen) {
105
+ this._scrollToBottom();
106
+ return;
107
+ }
108
+ this._chatBadge.classList.add("visible");
109
+ }
110
+
111
+ // ── Shared rendering ──────────────────────────────────────────────────────
112
+
113
+ /**
114
+ * Build a role-labelled message element. Ephemeral bubbles and persistent
115
+ * history rows share the same shape and differ only in their class prefix
116
+ * (`bubble`/`bubble-*` vs `hist-msg`/`hist-*`).
117
+ * @param {{ container: string, prefix: string, role: "user"|"assistant", text: string, partial?: boolean }} o
118
+ * @returns {HTMLElement}
119
+ */
120
+ _buildMessageEl({ container, prefix, role, text, partial = false }) {
121
+ const el = document.createElement("div");
122
+ el.className = `${container} ${role}`;
123
+ const label = role === "user" ? "You" : "Assistant";
124
+ el.innerHTML = `<div class="${prefix}-role">${label}</div><div class="${prefix}-body${partial ? " partial" : ""}">${escHtml(text)}</div>`;
125
+ return el;
126
+ }
127
+
128
+ // ── Ephemeral bubbles ───────────────────────────────────────────────────
129
+
130
+ /** @param {"user"|"assistant"|"tool"} role @param {string} text @returns {HTMLElement} */
131
+ _spawnBubble(role, text) {
132
+ let el;
133
+ if (role === "tool") {
134
+ el = document.createElement("div");
135
+ el.className = "bubble tool";
136
+ el.innerHTML = `<svg class="bubble-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">${WRENCH_PATH}</svg><span class="bubble-tool-text">${escHtml(text)}</span>`;
137
+ } else {
138
+ el = this._buildMessageEl({ container: "bubble", prefix: "bubble", role, text });
139
+ }
140
+ this._bubbleStack.appendChild(el);
141
+ // Cap the stack at 3, but never evict the bubble the caller is still
142
+ // actively updating (the live user bubble) — drop the next-oldest instead.
143
+ const visible = /** @type {HTMLElement[]} */ ([...this._bubbleStack.querySelectorAll(".bubble:not(.out)")]);
144
+ if (visible.length > 3) {
145
+ this._dismissBubble(visible.find((b) => b !== this._activeUserBubble) ?? visible[0]);
146
+ }
147
+ requestAnimationFrame(() => el.classList.add("in"));
148
+ return el;
149
+ }
150
+
151
+ /** @param {HTMLElement} el @param {string} text */
152
+ _updateBubbleText(el, text) {
153
+ const t = el.querySelector(".bubble-body");
154
+ if (t) t.textContent = text;
155
+ }
156
+
157
+ /** @param {HTMLElement} el */
158
+ _dismissBubble(el) {
159
+ if (!el || el.classList.contains("out")) return; // idempotent
160
+ this._bubbleExpiry.delete(el);
161
+ el.classList.remove("in");
162
+ el.classList.add("out");
163
+ const remove = () => el.remove();
164
+ el.addEventListener("transitionend", remove, { once: true });
165
+ // Fallback: transitionend never fires if the bubble's visual state didn't
166
+ // change (dismissed pre-paint) or the tab is backgrounded. The transition
167
+ // is 0.3s, so force removal a little after.
168
+ setTimeout(remove, 400);
169
+ }
170
+
171
+ /**
172
+ * Fade bubbles whose expiry has passed — strictly oldest-first. We walk the
173
+ * stack top (oldest) to bottom (newest) and stop at the first bubble still
174
+ * alive: nothing newer may leave while an older bubble is still on screen. A
175
+ * bubble that keeps updating pushes its own expiry forward, so it (and
176
+ * everything behind it) stays put until it finally goes quiet.
177
+ */
178
+ _reapBubbles() {
179
+ this._reaperHandle = 0;
180
+ const now = Date.now();
181
+ const visible = /** @type {HTMLElement[]} */ ([...this._bubbleStack.querySelectorAll(".bubble:not(.out)")]);
182
+ let nextWake = Infinity;
183
+ for (const el of visible) {
184
+ const exp = this._bubbleExpiry.get(el) ?? now; // no expiry recorded → treat as due
185
+ if (exp <= now) {
186
+ this._dismissBubble(el);
187
+ } else {
188
+ // Oldest survivor isn't due yet; stop so nothing newer leaves before it.
189
+ nextWake = exp;
190
+ break;
191
+ }
192
+ }
193
+ if (nextWake !== Infinity) {
194
+ this._reaperHandle = setTimeout(() => this._reapBubbles(), Math.max(50, nextWake - Date.now()));
195
+ }
196
+ }
197
+
198
+ /**
199
+ * (Re)arm a bubble's auto-dismiss by pushing its expiry out by `delay`.
200
+ * Calling it again resets the countdown — so a bubble that keeps updating
201
+ * stays on screen and only fades once it goes quiet. Removal is ordered by the
202
+ * shared reaper, so the oldest bubble always disappears first.
203
+ * @param {HTMLElement} el @param {number} [delay]
204
+ */
205
+ _bumpDismiss(el, delay = 4000) {
206
+ this._bubbleExpiry.set(el, Date.now() + delay);
207
+ if (!this._reaperHandle) this._reaperHandle = setTimeout(() => this._reapBubbles(), delay);
208
+ }
209
+
210
+ // ── History ───────────────────────────────────────────────────────────────
211
+
212
+ /** Render the empty-state placeholder into the history panel. */
213
+ renderEmptyState() {
214
+ this._chatHistory.innerHTML = EMPTY_STATE_HTML;
215
+ }
216
+
217
+ /** Reset the panel to the empty state and clear the unread badge. */
218
+ clear() {
219
+ this.renderEmptyState();
220
+ this._chatBadge.classList.remove("visible");
221
+ }
222
+
223
+ /** @param {"user"|"assistant"} role @param {string} text @param {boolean} partial @returns {HTMLElement} */
224
+ _appendHistMsg(role, text, partial) {
225
+ const empty = this._chatHistory.querySelector(".chat-empty");
226
+ if (empty) empty.remove();
227
+ const el = this._buildMessageEl({ container: "hist-msg", prefix: "hist", role, text, partial });
228
+ this._chatHistory.appendChild(el);
229
+ this._scrollToBottom();
230
+ return el;
231
+ }
232
+
233
+ /** @param {HTMLElement | null} el @param {string} text @param {boolean} partial */
234
+ _updateHistMsg(el, text, partial) {
235
+ if (!el) return;
236
+ const body = /** @type {HTMLElement | null} */ (el.querySelector(".hist-body"));
237
+ if (!body) return;
238
+ body.textContent = text;
239
+ body.classList.toggle("partial", partial);
240
+ this._scrollToBottom();
241
+ }
242
+
243
+ /**
244
+ * Append a tool-call row to the conversation. We only add it once the tool
245
+ * has run, so the expandable toggle carries BOTH the call input and its result.
246
+ * @param {string} name @param {string} argsJson @param {string} output
247
+ */
248
+ _appendHistTool(name, argsJson, output) {
249
+ const empty = this._chatHistory.querySelector(".chat-empty");
250
+ if (empty) empty.remove();
251
+ let pretty = argsJson;
252
+ try { pretty = JSON.stringify(JSON.parse(argsJson), null, 2); } catch {}
253
+ const el = document.createElement("div");
254
+ el.className = "hist-msg tool";
255
+ el.innerHTML = `
256
+ <div class="hist-role">Tool call</div>
257
+ <button class="hist-tool-header" aria-expanded="false">
258
+ <svg class="hist-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">${WRENCH_PATH}</svg>
259
+ <span class="hist-tool-name">${escHtml(name)}</span>
260
+ <svg class="hist-tool-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="6 9 12 15 18 9"/></svg>
261
+ </button>
262
+ <div class="hist-tool-body">
263
+ <div class="hist-tool-label">Input</div>
264
+ <div class="hist-tool-block">${escHtml(pretty)}</div>
265
+ <div class="hist-tool-label">Output</div>
266
+ <div class="hist-tool-block hist-tool-output">${escHtml(output || "(no output)")}</div>
267
+ </div>
268
+ `;
269
+ const header = /** @type {HTMLButtonElement} */ (el.querySelector(".hist-tool-header"));
270
+ const body = /** @type {HTMLDivElement} */ (el.querySelector(".hist-tool-body"));
271
+ header.addEventListener("click", () => {
272
+ const expanded = header.getAttribute("aria-expanded") === "true";
273
+ header.setAttribute("aria-expanded", String(!expanded));
274
+ body.classList.toggle("open", !expanded);
275
+ });
276
+ this._chatHistory.appendChild(el);
277
+ this._scrollToBottom();
278
+ }
279
+
280
+ /** Tag an assistant history row as interrupted (user barged in mid-reply).
281
+ * @param {HTMLElement | null} hist */
282
+ _markHistInterrupted(hist) {
283
+ if (!hist || hist.querySelector(".hist-note")) return;
284
+ hist.classList.add("interrupted");
285
+ const note = document.createElement("div");
286
+ note.className = "hist-note";
287
+ note.textContent = "Interrupted";
288
+ hist.appendChild(note);
289
+ }
290
+
291
+ /** Render a captured webcam frame in the transcript (the camera tool result).
292
+ * @param {string} dataUrl */
293
+ _appendHistImage(dataUrl) {
294
+ const empty = this._chatHistory.querySelector(".chat-empty");
295
+ if (empty) empty.remove();
296
+ const el = document.createElement("div");
297
+ el.className = "hist-msg tool";
298
+ el.innerHTML = `<div class="hist-role">Snapshot</div><img class="hist-image" alt="Webcam snapshot sent to the model" />`;
299
+ const img = /** @type {HTMLImageElement} */ (el.querySelector("img"));
300
+ img.src = dataUrl;
301
+ this._chatHistory.appendChild(el);
302
+ this._scrollToBottom();
303
+ }
304
+
305
+ /**
306
+ * Reset all streaming bookkeeping for session start / teardown. Pass
307
+ * `dismiss` to also fade any bubbles still on screen.
308
+ * @param {{ dismiss?: boolean }} [opts]
309
+ */
310
+ reset(opts) {
311
+ if (opts?.dismiss) {
312
+ if (this._activeUserBubble) this._dismissBubble(this._activeUserBubble);
313
+ for (const { bubble } of this._asstByResp.values()) this._dismissBubble(bubble);
314
+ }
315
+ this._userHistByItem.clear();
316
+ this._activeUserBubble = null;
317
+ this._activeUserItemId = "";
318
+ this._asstByResp.clear();
319
+ }
320
+
321
+ // ── Client event handlers ─────────────────────────────────────────────────
322
+
323
+ /**
324
+ * A streamed transcript delta (user or assistant).
325
+ * @param {{ role: "user" | "assistant"; text: string; partial: boolean; itemId?: string; responseId?: string }} d
326
+ */
327
+ onTranscript(d) {
328
+ if (DEBUG) console.debug(`[ui] transcript role=${d.role} partial=${d.partial} item=${d.itemId} resp=${d.responseId} text=${JSON.stringify(d.text)}`);
329
+
330
+ if (d.role === "user") {
331
+ // Group by item_id: a speculative continuation reuses the same id, so it
332
+ // updates the same row/bubble. A missing id falls back to the active item
333
+ // (same utterance) or a fresh unique key, never a shared sentinel that
334
+ // would collapse distinct turns into one row.
335
+ const id = d.itemId || this._activeUserItemId || `_u${++this._anonSeq}`;
336
+ const text = d.text;
337
+
338
+ let hist = this._userHistByItem.get(id);
339
+ if (!hist) {
340
+ hist = this._appendHistMsg("user", text, d.partial);
341
+ this._userHistByItem.set(id, hist);
342
+ } else {
343
+ this._updateHistMsg(hist, text, d.partial);
344
+ }
345
+
346
+ // One ephemeral bubble per active item. Purely timer-based: the timer is
347
+ // refreshed on every delta, so it stays while the user keeps talking and
348
+ // fades a few seconds after they stop — no dependency on a response ever
349
+ // arriving, so it can never get stuck.
350
+ if (this._activeUserItemId !== id || !this._activeUserBubble) {
351
+ this._activeUserBubble = this._spawnBubble("user", text);
352
+ this._activeUserItemId = id;
353
+ } else {
354
+ this._updateBubbleText(this._activeUserBubble, text);
355
+ }
356
+ this._bumpDismiss(this._activeUserBubble, 6000);
357
+ this._markUnread();
358
+ } else if (d.role === "assistant") {
359
+ // Assistant transcript arrives once, as the full text, keyed by
360
+ // response_id so a cancelled speculative response can be removed later. A
361
+ // missing id gets a unique key so two id-less replies never collide.
362
+ const rid = d.responseId || `_a${++this._anonSeq}`;
363
+ const entry = this._asstByResp.get(rid);
364
+ if (!entry) {
365
+ const bubble = this._spawnBubble("assistant", d.text);
366
+ this._asstByResp.set(rid, { bubble, hist: this._appendHistMsg("assistant", d.text, false) });
367
+ this._bumpDismiss(bubble);
368
+ } else {
369
+ this._updateBubbleText(entry.bubble, d.text);
370
+ this._updateHistMsg(entry.hist, d.text, false);
371
+ this._bumpDismiss(entry.bubble);
372
+ }
373
+ this._markUnread();
374
+ }
375
+ }
376
+
377
+ /**
378
+ * A response closed (completed or cancelled).
379
+ * @param {{ responseId: string; status: string; audible?: boolean; transcript?: string }} detail
380
+ */
381
+ onResponseFinished(detail) {
382
+ const { responseId, status, audible, transcript } = detail;
383
+ if (DEBUG) console.debug(`[ui] response-finished resp=${responseId} status=${status} audible=${audible} known=${this._asstByResp.has(responseId)}`);
384
+ // Without an id we can't target a specific response; the bubble will
385
+ // auto-dismiss on its own timer regardless.
386
+ if (!responseId) return;
387
+ const entry = this._asstByResp.get(responseId);
388
+
389
+ if (status === "cancelled") {
390
+ // Keep every transcript that was received — mark it interrupted rather
391
+ // than erasing it. If the `*.transcript.done` never fired, build the row
392
+ // from the text carried in response.done.
393
+ let hist = entry?.hist ?? null;
394
+ if (!hist && transcript) {
395
+ hist = this._appendHistMsg("assistant", transcript, false);
396
+ } else if (hist && transcript) {
397
+ this._updateHistMsg(hist, transcript, false);
398
+ }
399
+ if (hist) this._markHistInterrupted(hist);
400
+ this._asstByResp.delete(responseId);
401
+ return;
402
+ }
403
+
404
+ // Any other terminal close (completed / failed / incomplete / …): just
405
+ // release the map entry. The bubble already auto-dismisses on its timer and
406
+ // the history row persists as the conversation log. Crucially we do NOT
407
+ // touch user state here — that lifecycle is fully independent.
408
+ this._asstByResp.delete(responseId);
409
+ }
410
+
411
+ /** The model called a tool — show an ephemeral "running" bubble.
412
+ * @param {string} name */
413
+ onToolCall(name) {
414
+ this._bumpDismiss(this._spawnBubble("tool", name));
415
+ this._markUnread();
416
+ }
417
+
418
+ /** The tool finished — append its call+result row (and any captured image).
419
+ * @param {string} name @param {string} argsJson @param {string} output @param {string} [image] */
420
+ onToolResult(name, argsJson, output, image) {
421
+ this._appendHistTool(name, argsJson, output);
422
+ if (image) this._appendHistImage(image); // show the captured frame below the call
423
+ this._markUnread();
424
+ }
425
+ }
ui/dom.js ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /** Small shared helpers used across the UI modules: a strict query selector,
3
+ * HTML escaping for text we drop into innerHTML, error-string trimming, and
4
+ * the opt-in debug flag. */
5
+
6
+ /** Opt-in tracing: `localStorage.setItem("s2s.debug", "1")` then reload. */
7
+ export const DEBUG = (() => {
8
+ try {
9
+ return localStorage.getItem("s2s.debug") === "1";
10
+ } catch {
11
+ return false;
12
+ }
13
+ })();
14
+
15
+ /**
16
+ * Query a single element, throwing if it's missing (so a broken selector fails
17
+ * loudly at startup rather than as a later null-deref).
18
+ * @template {HTMLElement} T
19
+ * @param {string} selector
20
+ * @returns {T}
21
+ */
22
+ export function $(selector) {
23
+ const el = document.querySelector(selector);
24
+ if (!el) throw new Error(`Missing element: ${selector}`);
25
+ return /** @type {T} */ (el);
26
+ }
27
+
28
+ /** @param {string} s @returns {string} */
29
+ export function escHtml(s) {
30
+ return s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
31
+ }
32
+
33
+ /** Trim a long error message to fit the orb caption. @param {string} text */
34
+ export function truncateError(text) {
35
+ if (text.length <= 90) return text;
36
+ return text.slice(0, 87) + "…";
37
+ }
worklets/audio-playback.js ADDED
@@ -0,0 +1,171 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * AudioWorkletProcessor that plays back Float32 mono samples received from
4
+ * the main thread, upsampling whatever incoming rate the server uses
5
+ * (typically 24 kHz PCM16) to the AudioContext rate (typically 48 kHz).
6
+ *
7
+ * Lifecycle / messaging:
8
+ *
9
+ * main -> worklet:
10
+ * { kind: "config", inputRate: 24000 } one-shot at startup
11
+ * { kind: "audio", samples: Float32Array } (transferable) per chunk
12
+ * { kind: "clear" } wipe queue (barge-in)
13
+ *
14
+ * worklet -> main:
15
+ * { kind: "stats", queuedMs, played } every ~250 ms
16
+ * { kind: "underrun" } every time the queue
17
+ * runs dry mid-playback
18
+ *
19
+ * Underrun strategy: output silence. We do NOT hold the last sample (that
20
+ * tends to produce audible clicks/buzzes when long gaps appear between
21
+ * TTS chunks). A short ramp-out + ramp-in at boundaries would be nicer but
22
+ * the server's 30 ms cadence makes underruns visible only at end of turn.
23
+ */
24
+
25
+ const STATS_INTERVAL_FRAMES = 12000;
26
+ const FADE_FRAMES = 32;
27
+
28
+ class AudioPlaybackProcessor extends AudioWorkletProcessor {
29
+ constructor() {
30
+ super();
31
+ this._inputRate = 24000;
32
+ this._stepRatio = this._inputRate / sampleRate;
33
+ this._queue = [];
34
+ this._readIdx = 0;
35
+ this._fracPos = 0;
36
+ this._playing = false;
37
+ this._framesSinceStats = 0;
38
+ this._totalPlayed = 0;
39
+ this._fadeIn = 0;
40
+ this._fadeOut = 0;
41
+ this._lastSample = 0;
42
+
43
+ this.port.onmessage = (e) => {
44
+ const data = e.data;
45
+ if (!data || typeof data !== "object") return;
46
+ switch (data.kind) {
47
+ case "config":
48
+ if (typeof data.inputRate === "number" && data.inputRate > 0) {
49
+ this._inputRate = data.inputRate;
50
+ this._stepRatio = this._inputRate / sampleRate;
51
+ }
52
+ break;
53
+ case "audio":
54
+ if (data.samples instanceof Float32Array && data.samples.length > 0) {
55
+ this._queue.push(data.samples);
56
+ if (!this._playing) {
57
+ this._playing = true;
58
+ this._fadeIn = FADE_FRAMES;
59
+ this._fadeOut = 0;
60
+ }
61
+ }
62
+ break;
63
+ case "clear":
64
+ this._queue.length = 0;
65
+ this._readIdx = 0;
66
+ this._fracPos = 0;
67
+ this._fadeOut = FADE_FRAMES;
68
+ break;
69
+ }
70
+ };
71
+ }
72
+
73
+ _queuedSamples() {
74
+ let total = -this._readIdx;
75
+ for (const buf of this._queue) total += buf.length;
76
+ return Math.max(0, total);
77
+ }
78
+
79
+ /** Linear-interp read at the current fractional position. */
80
+ _readInterpolated() {
81
+ if (this._queue.length === 0) return null;
82
+ const head = this._queue[0];
83
+ const idx = this._readIdx;
84
+ const frac = this._fracPos;
85
+
86
+ let a = head[idx];
87
+ let b;
88
+ if (idx + 1 < head.length) {
89
+ b = head[idx + 1];
90
+ } else if (this._queue.length > 1) {
91
+ b = this._queue[1][0];
92
+ } else {
93
+ b = a;
94
+ }
95
+ return a + (b - a) * frac;
96
+ }
97
+
98
+ /** Advance the read position by `stepRatio`; pop consumed buffers. */
99
+ _advance() {
100
+ this._fracPos += this._stepRatio;
101
+ while (this._fracPos >= 1) {
102
+ this._fracPos -= 1;
103
+ this._readIdx += 1;
104
+ }
105
+ while (this._queue.length > 0 && this._readIdx >= this._queue[0].length) {
106
+ this._readIdx -= this._queue[0].length;
107
+ this._queue.shift();
108
+ }
109
+ }
110
+
111
+ process(_, outputs) {
112
+ const channels = outputs[0];
113
+ if (!channels || channels.length === 0) return true;
114
+ const out = channels[0];
115
+ const stereo = channels.length > 1 ? channels[1] : null;
116
+
117
+ for (let i = 0; i < out.length; i++) {
118
+ let sample = 0;
119
+
120
+ if (this._playing) {
121
+ const v = this._readInterpolated();
122
+ if (v === null) {
123
+ // Underrun: try to ramp out cleanly to avoid clicks.
124
+ sample = this._lastSample * Math.max(0, 1 - 1 / FADE_FRAMES);
125
+ this._lastSample = sample;
126
+ if (Math.abs(sample) < 1e-4) {
127
+ this._playing = false;
128
+ this._lastSample = 0;
129
+ this.port.postMessage({ kind: "underrun" });
130
+ }
131
+ } else {
132
+ sample = v;
133
+ this._lastSample = v;
134
+ this._advance();
135
+ }
136
+
137
+ if (this._fadeIn > 0) {
138
+ const gain = 1 - this._fadeIn / FADE_FRAMES;
139
+ sample *= gain;
140
+ this._fadeIn -= 1;
141
+ }
142
+ if (this._fadeOut > 0) {
143
+ const gain = this._fadeOut / FADE_FRAMES;
144
+ sample *= gain;
145
+ this._fadeOut -= 1;
146
+ if (this._fadeOut === 0) {
147
+ this._playing = false;
148
+ this._lastSample = 0;
149
+ }
150
+ }
151
+
152
+ this._totalPlayed += 1;
153
+ }
154
+
155
+ out[i] = sample;
156
+ if (stereo) stereo[i] = sample;
157
+ }
158
+
159
+ this._framesSinceStats += out.length;
160
+ if (this._framesSinceStats >= STATS_INTERVAL_FRAMES) {
161
+ this._framesSinceStats = 0;
162
+ const queuedSamples = this._queuedSamples();
163
+ const queuedMs = (queuedSamples / this._inputRate) * 1000;
164
+ this.port.postMessage({ kind: "stats", queuedMs, played: this._totalPlayed });
165
+ }
166
+
167
+ return true;
168
+ }
169
+ }
170
+
171
+ registerProcessor("audio-playback", AudioPlaybackProcessor);
worklets/mic-capture.js ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * AudioWorkletProcessor that resamples the AudioContext rate (typically 48 kHz)
4
+ * down to 16 kHz, packs the result as little-endian Int16 PCM, and posts it
5
+ * back to the main thread in fixed-size chunks.
6
+ *
7
+ * The Hugging Face speech-to-speech WebSocket route expects the
8
+ * `input_audio_buffer.append` payload at 16 kHz PCM16 mono.
9
+ *
10
+ * Design notes:
11
+ * - 48 -> 16 is an exact 3:1 ratio so we use a 3-tap boxcar average as a
12
+ * cheap low-pass before decimating. Good enough for voice STT; we lose
13
+ * a tiny bit of >8 kHz content which the pipeline discards anyway.
14
+ * - Output frames are emitted at the cadence dictated by `chunkMs`
15
+ * (default 40 ms = 640 samples = 1280 bytes). The OpenAI Realtime
16
+ * server batches incoming audio so the cadence is flexible; 20-100 ms
17
+ * is the sweet spot.
18
+ * - Float -> Int16 saturates to [-1, 1] before scaling.
19
+ * - Optional noise gate: per-chunk RMS decides open/closed against a
20
+ * threshold; the gain ramps (fast attack, hold, slow release) so word
21
+ * onsets aren't clipped and quiet tails don't click. The gate only
22
+ * affects the audio we SEND; the main-thread visualiser taps the raw
23
+ * mic separately. We post the chunk RMS up every frame so the Settings
24
+ * mic meter can show the live level against the threshold.
25
+ */
26
+
27
+ const TARGET_RATE = 16000;
28
+ const DEFAULT_CHUNK_MS = 40;
29
+ // Gate envelope timing (fixed; only the threshold is user-tunable).
30
+ const GATE_ATTACK_MS = 5; // open almost instantly so word onsets survive
31
+ const GATE_HOLD_MS = 250; // stay open this long after the level drops back under
32
+ const GATE_RELEASE_MS = 80; // then fade closed over this long (no click)
33
+
34
+ class MicCaptureProcessor extends AudioWorkletProcessor {
35
+ constructor(options) {
36
+ super();
37
+ const chunkMs = options?.processorOptions?.chunkMs ?? DEFAULT_CHUNK_MS;
38
+ this._inputRate = sampleRate;
39
+ this._ratio = this._inputRate / TARGET_RATE;
40
+ this._chunkSamples16k = Math.round((TARGET_RATE * chunkMs) / 1000);
41
+ this._scratch = new Float32Array(0);
42
+ this._decimated = new Float32Array(this._chunkSamples16k);
43
+ this._enabled = true;
44
+
45
+ // Noise gate state. Disabled by default (pure passthrough).
46
+ this._gateEnabled = false;
47
+ this._thresholdLin = 0; // linear amplitude; signal RMS must exceed this to open
48
+ this._gateGain = 1; // smoothed gain currently applied
49
+ this._holdRemaining = 0; // samples left before the gate may start closing
50
+ this._attackCoef = Math.exp(-1 / ((GATE_ATTACK_MS / 1000) * TARGET_RATE));
51
+ this._releaseCoef = Math.exp(-1 / ((GATE_RELEASE_MS / 1000) * TARGET_RATE));
52
+ this._holdSamples = Math.round((GATE_HOLD_MS / 1000) * TARGET_RATE);
53
+
54
+ this.port.onmessage = (e) => {
55
+ const data = e.data;
56
+ if (data?.kind === "enable") this._enabled = !!data.value;
57
+ else if (data?.kind === "gate") {
58
+ this._gateEnabled = !!data.enabled;
59
+ // dB -> linear amplitude. When off, threshold 0 keeps the gate open.
60
+ this._thresholdLin = data.enabled ? Math.pow(10, data.thresholdDb / 20) : 0;
61
+ }
62
+ };
63
+ }
64
+
65
+ /**
66
+ * Append `incoming` to the internal scratch buffer, then emit as many
67
+ * full output chunks as we have material for.
68
+ * @param {Float32Array} incoming
69
+ */
70
+ _ingest(incoming) {
71
+ if (incoming.length === 0) return;
72
+ const next = new Float32Array(this._scratch.length + incoming.length);
73
+ next.set(this._scratch, 0);
74
+ next.set(incoming, this._scratch.length);
75
+ this._scratch = next;
76
+ this._maybeEmit();
77
+ }
78
+
79
+ _maybeEmit() {
80
+ const r = this._ratio;
81
+ const n = this._chunkSamples16k;
82
+ const needIn = Math.ceil(n * r);
83
+ const dec = this._decimated;
84
+ while (this._scratch.length >= needIn) {
85
+ // 1. Decimate to 16 kHz floats and accumulate energy for the gate/meter.
86
+ let sumSq = 0;
87
+ if (Math.abs(r - 3) < 1e-6) {
88
+ // 48 kHz -> 16 kHz fast path with boxcar lowpass.
89
+ for (let i = 0; i < n; i++) {
90
+ const idx = i * 3;
91
+ const s = (this._scratch[idx] + this._scratch[idx + 1] + this._scratch[idx + 2]) / 3;
92
+ dec[i] = s;
93
+ sumSq += s * s;
94
+ }
95
+ } else {
96
+ // Generic path: linear interpolation. Slower but works at any rate
97
+ // (e.g. some Windows boxes report sampleRate=44100).
98
+ for (let i = 0; i < n; i++) {
99
+ const srcPos = i * r;
100
+ const idx = Math.floor(srcPos);
101
+ const frac = srcPos - idx;
102
+ const a = this._scratch[idx];
103
+ const b = this._scratch[idx + 1] ?? a;
104
+ const s = a + (b - a) * frac;
105
+ dec[i] = s;
106
+ sumSq += s * s;
107
+ }
108
+ }
109
+ const rms = Math.sqrt(sumSq / n);
110
+
111
+ // 2. Decide the gate target for this chunk, then ramp sample-by-sample.
112
+ let target = 1;
113
+ if (this._gateEnabled) {
114
+ if (rms >= this._thresholdLin) {
115
+ this._holdRemaining = this._holdSamples; // re-arm the hold
116
+ } else if (this._holdRemaining > 0) {
117
+ this._holdRemaining -= n; // coasting through the hold window
118
+ } else {
119
+ target = 0;
120
+ }
121
+ }
122
+
123
+ // 3. Apply the (smoothed) gain and pack to Int16.
124
+ const out = new Int16Array(n);
125
+ let gain = this._gateGain;
126
+ for (let i = 0; i < n; i++) {
127
+ const coef = target > gain ? this._attackCoef : this._releaseCoef;
128
+ gain = target + (gain - target) * coef;
129
+ const s = dec[i] * gain;
130
+ const clamped = s < -1 ? -1 : s > 1 ? 1 : s;
131
+ out[i] = clamped < 0 ? clamped * 0x8000 : clamped * 0x7fff;
132
+ }
133
+ this._gateGain = gain;
134
+
135
+ // Shift the scratch buffer to keep only the trailing unused samples.
136
+ const consumed = Math.floor(n * r);
137
+ this._scratch = this._scratch.slice(consumed);
138
+
139
+ // Live input level for the Settings meter (raw RMS, pre-gate).
140
+ this.port.postMessage({ kind: "level", rms });
141
+
142
+ if (this._enabled) {
143
+ this.port.postMessage(out.buffer, [out.buffer]);
144
+ }
145
+ // When disabled (mic muted) we silently consume input so the worklet
146
+ // stays alive and the buffer never grows unbounded.
147
+ }
148
+ }
149
+
150
+ process(inputs) {
151
+ const input = inputs[0];
152
+ if (!input || input.length === 0 || !input[0]) return true;
153
+ const mono = input[0];
154
+ if (mono.length > 0) this._ingest(mono);
155
+ return true;
156
+ }
157
+ }
158
+
159
+ registerProcessor("mic-capture", MicCaptureProcessor);
ws/codec.js ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * Pure, stateless helpers for the WebSocket realtime client: base64 <-> PCM
4
+ * conversion for the audio frames on the wire, transcript extraction from a
5
+ * `response.done` payload, and a tiny URL helper. Kept separate from the client
6
+ * so the protocol/state logic stays readable.
7
+ */
8
+
9
+ /** @param {string} url */
10
+ export function trimTrailingSlash(url) {
11
+ return url.endsWith("/") ? url.slice(0, -1) : url;
12
+ }
13
+
14
+ /**
15
+ * Pull the assistant transcript out of a `response.done` payload. The text
16
+ * lives in `response.output[].content[].transcript` (audio) or `.text`. Used as
17
+ * the source of truth for interrupted replies, where the dedicated
18
+ * `*.transcript.done` event may never arrive.
19
+ * @param {any} response
20
+ * @returns {string}
21
+ */
22
+ export function extractResponseTranscript(response) {
23
+ const output = response?.output;
24
+ if (!Array.isArray(output)) return "";
25
+ /** @type {string[]} */
26
+ const parts = [];
27
+ for (const item of output) {
28
+ for (const part of item?.content ?? []) {
29
+ const text = part?.transcript ?? part?.text;
30
+ if (typeof text === "string" && text.trim()) parts.push(text.trim());
31
+ }
32
+ }
33
+ return parts.join(" ").trim();
34
+ }
35
+
36
+ /** @param {ArrayBuffer} buf */
37
+ export function base64FromArrayBuffer(buf) {
38
+ const bytes = new Uint8Array(buf);
39
+ // Chunked encoding so we don't blow up the call stack on long buffers.
40
+ let binary = "";
41
+ const chunk = 0x8000;
42
+ for (let i = 0; i < bytes.length; i += chunk) {
43
+ binary += String.fromCharCode.apply(null, /** @type {number[]} */ (
44
+ /** @type {unknown} */ (bytes.subarray(i, i + chunk))
45
+ ));
46
+ }
47
+ return btoa(binary);
48
+ }
49
+
50
+ /** @param {string} b64 */
51
+ export function base64ToBytes(b64) {
52
+ const binary = atob(b64);
53
+ const len = binary.length;
54
+ const out = new Uint8Array(len);
55
+ for (let i = 0; i < len; i++) out[i] = binary.charCodeAt(i);
56
+ return out;
57
+ }
ws/orb-visualizer.js ADDED
@@ -0,0 +1,98 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * Orb spectrum visualiser. Each animation frame it reads two AnalyserNodes (the
4
+ * mic input and the TTS output) and maps the low-frequency speech energy onto
5
+ * the orb's CSS custom properties:
6
+ * - `--bar0`..`--bar4` the 5-band level meter
7
+ * - `--ai-audio-level` the global "Reachy talks" glow / scale pulse
8
+ *
9
+ * The bottom of the FFT is where speech energy lives, so the band edges stay
10
+ * low — that keeps the bars dancing on voice rather than on noise. While the AI
11
+ * is speaking we source the bars from the OUTPUT analyser so the orb pulses with
12
+ * Reachy's voice instead of sitting dead while the user is silent.
13
+ */
14
+
15
+ // Exported so the client can size its AnalyserNodes to match our buffer.
16
+ export const VIS_FFT_SIZE = 256;
17
+ const VIS_BAND_COUNT = 5;
18
+ const VIS_BAND_EDGES = [2, 5, 9, 16, 28, 52];
19
+ const VIS_ATTACK = 0.6; // weight for new sample on upswing (snappy)
20
+ const VIS_RELEASE = 0.18; // weight for new sample on decay (gentle fade)
21
+
22
+ export class OrbVisualiser {
23
+ /**
24
+ * @param {AnalyserNode} micAnalyser
25
+ * @param {AnalyserNode} outAnalyser
26
+ * @param {() => boolean} isAiSpeaking Source the bars from the AI output when
27
+ * true, otherwise from the mic.
28
+ */
29
+ constructor(micAnalyser, outAnalyser, isAiSpeaking) {
30
+ this._mic = micAnalyser;
31
+ this._out = outAnalyser;
32
+ this._isAiSpeaking = isAiSpeaking;
33
+ this._buf = new Uint8Array(micAnalyser.frequencyBinCount);
34
+ this._bands = new Float32Array(VIS_BAND_COUNT);
35
+ this._aiLevel = 0;
36
+ /** @type {number | null} */
37
+ this._frame = null;
38
+ }
39
+
40
+ /** Begin the rAF loop (idempotent). */
41
+ start() {
42
+ if (this._frame !== null) return;
43
+ const root = document.documentElement;
44
+ const tick = () => {
45
+ this._frame = requestAnimationFrame(tick);
46
+ this._update(root);
47
+ };
48
+ this._frame = requestAnimationFrame(tick);
49
+ }
50
+
51
+ /** Stop the loop and clear the CSS vars so the orb returns to rest. */
52
+ stop() {
53
+ if (this._frame !== null) {
54
+ cancelAnimationFrame(this._frame);
55
+ this._frame = null;
56
+ }
57
+ const root = document.documentElement;
58
+ for (let i = 0; i < VIS_BAND_COUNT; i++) root.style.removeProperty(`--bar${i}`);
59
+ root.style.removeProperty("--ai-audio-level");
60
+ }
61
+
62
+ /** @param {HTMLElement} root */
63
+ _update(root) {
64
+ // Mic bars: split FFT into 5 log-ish bands, smooth, write CSS vars.
65
+ const source = this._isAiSpeaking() ? this._out : this._mic;
66
+ source.getByteFrequencyData(this._buf);
67
+
68
+ for (let b = 0; b < VIS_BAND_COUNT; b++) {
69
+ const lo = VIS_BAND_EDGES[b];
70
+ const hi = VIS_BAND_EDGES[b + 1];
71
+ let sum = 0;
72
+ let n = 0;
73
+ for (let i = lo; i < hi && i < this._buf.length; i++) {
74
+ sum += this._buf[i];
75
+ n += 1;
76
+ }
77
+ const target = n > 0 ? sum / (n * 255) : 0;
78
+ const prev = this._bands[b];
79
+ const k = target > prev ? VIS_ATTACK : VIS_RELEASE;
80
+ const next = prev + (target - prev) * k;
81
+ this._bands[b] = next;
82
+ root.style.setProperty(`--bar${b}`, next.toFixed(3));
83
+ }
84
+
85
+ // Global AI audio level: peak of the output analyser, used by the CSS to
86
+ // make the orb's glow / scale react to Reachy's voice.
87
+ this._out.getByteFrequencyData(this._buf);
88
+ let peak = 0;
89
+ const limit = Math.min(this._buf.length, VIS_BAND_EDGES[VIS_BAND_COUNT]);
90
+ for (let i = 0; i < limit; i++) {
91
+ if (this._buf[i] > peak) peak = this._buf[i];
92
+ }
93
+ const aiTarget = peak / 255;
94
+ const k = aiTarget > this._aiLevel ? VIS_ATTACK : VIS_RELEASE;
95
+ this._aiLevel = this._aiLevel + (aiTarget - this._aiLevel) * k;
96
+ root.style.setProperty("--ai-audio-level", this._aiLevel.toFixed(3));
97
+ }
98
+ }
ws/s2s-ws-client.js ADDED
@@ -0,0 +1,899 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // @ts-check
2
+ /**
3
+ * Minimal WebSocket client for the Hugging Face speech-to-speech load balancer.
4
+ *
5
+ * Two-step handshake (same /session route as the WebRTC client):
6
+ *
7
+ * 1. POST `<lb_url>/session` -> JSON `{ connect_url: wss://<compute>/v1/realtime?session_token=<JWT>, ... }`
8
+ * 2. Open a WebSocket directly on `connect_url` (no rewrite, unlike the WebRTC client).
9
+ *
10
+ * Once the socket is open we follow the OpenAI Realtime GA WebSocket
11
+ * protocol:
12
+ *
13
+ * - Server pushes `session.created` immediately after upgrade.
14
+ * - We send `session.update` (GA schema: `session.audio.{input,output}`,
15
+ * `session.output_modalities`, ...).
16
+ * - We stream mic audio as PCM16 16 kHz mono base64 chunks via
17
+ * `input_audio_buffer.append`.
18
+ * - The server pushes `response.output_audio.delta` (PCM16 24 kHz mono
19
+ * base64) and transcript deltas.
20
+ *
21
+ * Audio is handled internally via two AudioWorklet processors so the
22
+ * client owns the full mic-in / speaker-out pipeline. The main app only
23
+ * sees high-level lifecycle events (`status`, `transcript`, `error`,
24
+ * `session`), the same shape as the WebRTC client.
25
+ *
26
+ * @typedef {"idle" | "creating-session" | "connecting" | "connected" |
27
+ * "user-speaking" | "processing" | "ai-speaking" | "closed" | "error"
28
+ * } WsStatus
29
+ *
30
+ * @typedef {Object} WsSessionInfo
31
+ * @property {string} sessionId
32
+ * @property {string} connectUrl
33
+ * @property {string} websocketUrl
34
+ * @property {string} sessionToken
35
+ * @property {number} pendingTimeoutS
36
+ * @property {string} [tier] Login tier from the session proxy ("anon"|"free"|"pro").
37
+ * @property {boolean} [limited] Whether this session is metered (heartbeat needed).
38
+ * @property {number} [heartbeatSec] Suggested heartbeat cadence in seconds.
39
+ * @property {number} [remainingSec] Daily budget left after this grant (display).
40
+ *
41
+ * @typedef {Object} WsClientOptions
42
+ * @property {string} [sessionUrl] URL to POST for the session handshake (returns
43
+ * `{ connect_url, ... }`). Usually a same-origin proxy like `api/session` so the
44
+ * load-balancer address stays server-side. Provide this OR `directUrl`.
45
+ * @property {string} [loadBalancerUrl] Load-balancer base URL. Legacy/direct
46
+ * alternative to `sessionUrl`: the client POSTs `<lb>/session` itself. Prefer
47
+ * `sessionUrl` so the LB address isn't exposed to the browser.
48
+ * @property {string} [directUrl] Full WebSocket URL of an s2s realtime endpoint
49
+ * (e.g. `ws://localhost:8080/v1/realtime`). When set, the client skips the
50
+ * session POST and dials it directly — no load balancer in between.
51
+ * @property {string} voice
52
+ * @property {string} instructions
53
+ * @property {MediaStream} micStream
54
+ * @property {AudioContext} [audioContext] Pre-created (and resumed) context.
55
+ * iOS Safari only lets an AudioContext start from within a user gesture, so
56
+ * the caller creates/resumes it synchronously on the orb tap and hands it
57
+ * here; otherwise it stays suspended (silent) after the mic/session awaits.
58
+ * @property {ToolDef[]} [tools] Function tools declared to the backend in the
59
+ * initial `session.update`. The model decides when to call them; the caller
60
+ * executes and replies via `sendToolOutput` + `requestResponse`.
61
+ * @property {NoiseGate} [noiseGate] Client-side noise gate applied to the mic
62
+ * before it's sent. Tunable live via `setNoiseGate`.
63
+ *
64
+ * @typedef {Object} NoiseGate
65
+ * @property {boolean} enabled
66
+ * @property {number} thresholdDb Open threshold in dBFS (e.g. -45).
67
+ *
68
+ * @typedef {Object} ToolDef
69
+ * @property {"function"} type
70
+ * @property {string} name
71
+ * @property {string} description
72
+ * @property {object} parameters JSON Schema for the call arguments.
73
+ *
74
+ * @typedef {Object} TranscriptEvent
75
+ * @property {"user" | "assistant"} role
76
+ * @property {string} text
77
+ * @property {boolean} partial
78
+ */
79
+
80
+ import {
81
+ base64FromArrayBuffer,
82
+ base64ToBytes,
83
+ extractResponseTranscript,
84
+ trimTrailingSlash,
85
+ } from "./codec.js";
86
+ import { OrbVisualiser, VIS_FFT_SIZE } from "./orb-visualizer.js";
87
+
88
+ // The s2s pipeline runs internally at 16 kHz mono PCM. The WebRTC transport
89
+ // resamples to 48 kHz for Opus, but the WebSocket transport emits the
90
+ // native pipeline rate. We don't (can't) override it via `audio.output.format`
91
+ // because the server's pydantic validator rejects the whole `session.update`
92
+ // as soon as a sub-field shape it doesn't know about appears.
93
+ const OUTPUT_SAMPLE_RATE = 16000;
94
+ const MIC_CHUNK_MS = 40;
95
+
96
+ export class S2sWsRealtimeClient extends EventTarget {
97
+ /** @param {WsClientOptions} options */
98
+ constructor(options) {
99
+ super();
100
+ /** @type {WsClientOptions} */
101
+ this.options = options;
102
+ /** @type {ToolDef[]} Function tools declared to the backend. */
103
+ this._tools = options.tools ?? [];
104
+ /** @type {string} Direct realtime WS URL (set => skip the LB session POST). */
105
+ this._directUrl = options.directUrl ?? "";
106
+ /** @type {string} Where to POST for the session handshake. Prefer the
107
+ * explicit `sessionUrl`; fall back to `<loadBalancerUrl>/session` for callers
108
+ * that still pass the LB address directly. */
109
+ this._sessionUrl = options.sessionUrl
110
+ ? options.sessionUrl
111
+ : options.loadBalancerUrl
112
+ ? `${trimTrailingSlash(options.loadBalancerUrl)}/session`
113
+ : "";
114
+ /** @type {NoiseGate} Mic noise gate; off by default. */
115
+ this._noiseGate = options.noiseGate ?? { enabled: false, thresholdDb: -45 };
116
+ /** @type {WebSocket | null} */
117
+ this._ws = null;
118
+ /** @type {AudioContext | null} */
119
+ this._ctx = null;
120
+ /** @type {MediaStreamAudioSourceNode | null} */
121
+ this._micSrc = null;
122
+ /** @type {AudioWorkletNode | null} */
123
+ this._captureNode = null;
124
+ /** @type {AudioWorkletNode | null} */
125
+ this._playbackNode = null;
126
+ /** @type {GainNode | null} */
127
+ this._captureSink = null;
128
+ /** @type {AnalyserNode | null} */
129
+ this._micAnalyser = null;
130
+ /** @type {AnalyserNode | null} */
131
+ this._outAnalyser = null;
132
+ /** @type {OrbVisualiser | null} */
133
+ this._visualiser = null;
134
+ /** @type {WsStatus} */
135
+ this._status = "idle";
136
+ this._aiSpeaking = false;
137
+ /** @type {Set<string>} response_ids that have actually played audio, so the
138
+ * UI can tell a barge-in cut (keep it) from a never-heard speculative
139
+ * response (drop it). */
140
+ this._audibleResponses = new Set();
141
+ /** @type {Map<string, string>} The CURRENT assistant transcript segment per
142
+ * response, accumulated from streamed deltas (reset on each segment's done). */
143
+ this._asstTranscriptByResp = new Map();
144
+ /** @type {Map<string, string>} Completed assistant transcript segments per
145
+ * response, space-joined. A single response can emit several
146
+ * `*.transcript.done` events; we concatenate them until response.done. */
147
+ this._asstFullByResp = new Map();
148
+ this._muted = false;
149
+ // ── Response lock ────────────────────────────────────────────────────
150
+ // The backend allows only ONE response in flight: creating a second while
151
+ // one is active fails with `conversation_already_has_active_response`. So
152
+ // we serialize response.create. `_openResponses` counts responses the
153
+ // server has confirmed (response.created) but not yet finished
154
+ // (response.done) — it's cumulative, so every create maps to one done.
155
+ // `_createInFlight` covers the window after we send a create but before its
156
+ // response.created echo. Any requestResponse() made while locked is queued
157
+ // and replayed, one at a time, as each response.done frees the slot.
158
+ this._openResponses = 0;
159
+ this._createInFlight = false;
160
+ /** @type {{ image?: string }[]} Pending response.create payloads, one per
161
+ * queued requestResponse(). A payload may carry an image to send just
162
+ * before its create (so the frame travels with the create, not eagerly). */
163
+ this._createQueue = [];
164
+ /** @type {Promise<void> | null} */
165
+ this._readyPromise = null;
166
+ this._sessionConfigured = false;
167
+ this._debug = (() => { try { return localStorage.getItem("s2s.debug") === "1"; } catch { return false; } })();
168
+ }
169
+
170
+ get status() {
171
+ return this._status;
172
+ }
173
+
174
+ /** @param {WsStatus} status */
175
+ _setStatus(status) {
176
+ if (this._status === status) return;
177
+ this._status = status;
178
+ this.dispatchEvent(new CustomEvent("status", { detail: { status } }));
179
+ }
180
+
181
+ /** Full assistant transcript so far for a response: the completed segments
182
+ * plus the in-progress one, all space-joined.
183
+ * @param {string} rid @returns {string} */
184
+ _asstDisplay(rid) {
185
+ const full = this._asstFullByResp.get(rid) || "";
186
+ const seg = this._asstTranscriptByResp.get(rid) || "";
187
+ if (!seg) return full;
188
+ return full ? `${full} ${seg}` : seg;
189
+ }
190
+
191
+ _markAudible() {
192
+ if (this._status === "ai-speaking") return;
193
+ if (this._status === "closed" || this._status === "error") return;
194
+ this._setStatus("ai-speaking");
195
+ }
196
+
197
+ /**
198
+ * Full handshake. Resolves once the WS is open AND the audio pipeline is
199
+ * ready to send/receive samples.
200
+ * @returns {Promise<void>}
201
+ */
202
+ async connect() {
203
+ if (this._ws) throw new Error("Already connected");
204
+
205
+ let connectUrl;
206
+ if (this._directUrl) {
207
+ // Direct mode: no load balancer, no /session POST — dial the realtime
208
+ // endpoint straight away (e.g. a local s2s server).
209
+ connectUrl = this._directUrl;
210
+ this._setStatus("connecting");
211
+ } else {
212
+ if (!this._sessionUrl) {
213
+ throw new Error("No session endpoint or direct URL configured");
214
+ }
215
+ this._setStatus("creating-session");
216
+ const session = await this._createSession();
217
+ this.dispatchEvent(new CustomEvent("session", { detail: { info: session } }));
218
+ connectUrl = session.connectUrl;
219
+ this._setStatus("connecting");
220
+ }
221
+
222
+ // Spin up the AudioContext + worklets in parallel with the WS dial.
223
+ const audioReady = this._setupAudio();
224
+ const wsReady = this._openWebSocket(connectUrl);
225
+ await Promise.all([audioReady, wsReady]);
226
+ }
227
+
228
+ /**
229
+ * @returns {Promise<WsSessionInfo>}
230
+ */
231
+ async _createSession() {
232
+ const url = this._sessionUrl;
233
+ console.log("[ws] POST", url);
234
+ const response = await fetch(url, {
235
+ method: "POST",
236
+ headers: { "Content-Type": "application/json" },
237
+ body: "{}",
238
+ });
239
+ if (response.status === 402) {
240
+ // The session proxy refused: today's per-tier time budget is spent. Surface
241
+ // it as a typed error so the UI shows the limit modal, not a crash.
242
+ const body = await response.json().catch(() => ({}));
243
+ const err = /** @type {Error & { code?: string; tier?: string }} */ (
244
+ new Error("Daily conversation limit reached")
245
+ );
246
+ err.code = "limit";
247
+ err.tier = body?.tier;
248
+ throw err;
249
+ }
250
+ if (!response.ok) {
251
+ const text = await response.text().catch(() => "");
252
+ throw new Error(`/session failed (${response.status}): ${text}`);
253
+ }
254
+ const json = await response.json();
255
+ return {
256
+ sessionId: json.session_id,
257
+ connectUrl: json.connect_url,
258
+ websocketUrl: json.websocket_url,
259
+ sessionToken: json.session_token,
260
+ pendingTimeoutS: json.pending_timeout_s,
261
+ tier: json.tier,
262
+ limited: json.limited,
263
+ heartbeatSec: json.heartbeatSec,
264
+ remainingSec: json.remainingSec,
265
+ };
266
+ }
267
+
268
+ async _setupAudio() {
269
+ // Prefer a context the caller already created + resumed inside the tap
270
+ // gesture (required on iOS). Fall back to creating one here for callers
271
+ // that don't (desktop is lenient about the gesture timing).
272
+ // Most desktops give us 48 kHz, mobiles can give 44.1/24/16 kHz; the
273
+ // capture worklet handles any rate (linear interp fallback).
274
+ const ctx = this.options.audioContext ?? new AudioContext({ latencyHint: "interactive" });
275
+ this._ctx = ctx;
276
+
277
+ // Resume if still suspended. This is best-effort here — on iOS the resume
278
+ // that actually counts is the one the caller did synchronously on tap.
279
+ if (ctx.state === "suspended") {
280
+ try {
281
+ await ctx.resume();
282
+ } catch (err) {
283
+ console.warn("[ws] AudioContext resume failed:", err);
284
+ }
285
+ }
286
+
287
+ // The worklets live at the repo root, one level up from this module.
288
+ const base = new URL("../worklets/", import.meta.url);
289
+ await ctx.audioWorklet.addModule(new URL("mic-capture.js", base).href);
290
+ await ctx.audioWorklet.addModule(new URL("audio-playback.js", base).href);
291
+
292
+ const captureNode = new AudioWorkletNode(ctx, "mic-capture", {
293
+ numberOfInputs: 1,
294
+ numberOfOutputs: 0,
295
+ processorOptions: { chunkMs: MIC_CHUNK_MS },
296
+ });
297
+ captureNode.port.onmessage = (e) => {
298
+ const data = e.data;
299
+ if (data instanceof ArrayBuffer) {
300
+ this._onMicChunk(data);
301
+ } else if (data?.kind === "level") {
302
+ // Raw pre-gate mic RMS for the Settings meter.
303
+ this.dispatchEvent(new CustomEvent("input-level", { detail: { rms: data.rms } }));
304
+ }
305
+ };
306
+ // Push the initial gate config now that the worklet exists.
307
+ captureNode.port.postMessage({ kind: "gate", ...this._noiseGate });
308
+ this._captureNode = captureNode;
309
+
310
+ const micSrc = ctx.createMediaStreamSource(this.options.micStream);
311
+ micSrc.connect(captureNode);
312
+ this._micSrc = micSrc;
313
+
314
+ // Mic analyser: tap the mic in parallel with the worklet so we get the
315
+ // raw (un-resampled, un-clipped) signal for the visualiser.
316
+ const micAnalyser = ctx.createAnalyser();
317
+ micAnalyser.fftSize = VIS_FFT_SIZE;
318
+ micAnalyser.smoothingTimeConstant = 0;
319
+ micSrc.connect(micAnalyser);
320
+ this._micAnalyser = micAnalyser;
321
+
322
+ const playbackNode = new AudioWorkletNode(ctx, "audio-playback", {
323
+ numberOfInputs: 0,
324
+ numberOfOutputs: 1,
325
+ outputChannelCount: [1],
326
+ });
327
+ playbackNode.port.postMessage({ kind: "config", inputRate: OUTPUT_SAMPLE_RATE });
328
+ playbackNode.port.onmessage = (e) => this._onPlaybackMessage(e.data);
329
+
330
+ // Output analyser sits between the playback worklet and the speakers.
331
+ const outAnalyser = ctx.createAnalyser();
332
+ outAnalyser.fftSize = VIS_FFT_SIZE;
333
+ outAnalyser.smoothingTimeConstant = 0.3;
334
+ playbackNode.connect(outAnalyser);
335
+ outAnalyser.connect(ctx.destination);
336
+ this._outAnalyser = outAnalyser;
337
+ this._playbackNode = playbackNode;
338
+
339
+ this._visualiser = new OrbVisualiser(micAnalyser, outAnalyser, () => this._aiSpeaking);
340
+ this._visualiser.start();
341
+ }
342
+
343
+ /** @param {string} connectUrl */
344
+ _openWebSocket(connectUrl) {
345
+ return new Promise((resolve, reject) => {
346
+ const ws = new WebSocket(connectUrl);
347
+ ws.binaryType = "arraybuffer";
348
+ this._ws = ws;
349
+
350
+ const onceOpen = () => {
351
+ ws.removeEventListener("open", onceOpen);
352
+ ws.removeEventListener("error", onceErr);
353
+ resolve();
354
+ };
355
+ const onceErr = (e) => {
356
+ ws.removeEventListener("open", onceOpen);
357
+ ws.removeEventListener("error", onceErr);
358
+ reject(new Error(`WebSocket failed to open: ${e?.type ?? "error"}`));
359
+ };
360
+ ws.addEventListener("open", onceOpen);
361
+ ws.addEventListener("error", onceErr);
362
+
363
+ ws.addEventListener("message", (e) => this._onWsMessage(e.data));
364
+ ws.addEventListener("close", (e) => this._onWsClose(e));
365
+ ws.addEventListener("error", (e) => {
366
+ console.error("[ws] socket error", e);
367
+ });
368
+ });
369
+ }
370
+
371
+ /**
372
+ * @param {{ kind: string; queuedMs?: number; played?: number }} data
373
+ */
374
+ _onPlaybackMessage(data) {
375
+ if (data?.kind === "underrun") {
376
+ // Server stopped sending audio mid-response. Most likely the turn
377
+ // ended cleanly (a response.done usually arrives just before/after
378
+ // this). We let the state machine fall back to "connected" via the
379
+ // response.done event handler.
380
+ }
381
+ }
382
+
383
+ /**
384
+ * Mic worklet just sent us a ~40 ms PCM16 16 kHz mono chunk.
385
+ * Base64-encode and forward via the WS.
386
+ * @param {ArrayBuffer} pcm16Buffer
387
+ */
388
+ _onMicChunk(pcm16Buffer) {
389
+ if (!this._ws || this._ws.readyState !== WebSocket.OPEN) return;
390
+ if (!this._sessionConfigured) return; // Server rejects audio before session.update.
391
+ if (this._muted) return;
392
+ const b64 = base64FromArrayBuffer(pcm16Buffer);
393
+ this._send({ type: "input_audio_buffer.append", audio: b64 });
394
+ }
395
+
396
+ /**
397
+ * @param {string | ArrayBuffer | Blob} raw
398
+ */
399
+ async _onWsMessage(raw) {
400
+ let text;
401
+ if (typeof raw === "string") {
402
+ text = raw;
403
+ } else if (raw instanceof ArrayBuffer) {
404
+ text = new TextDecoder("utf-8").decode(raw);
405
+ } else if (raw instanceof Blob) {
406
+ text = await raw.text();
407
+ } else {
408
+ return;
409
+ }
410
+
411
+ let event;
412
+ try {
413
+ event = JSON.parse(text);
414
+ } catch {
415
+ return;
416
+ }
417
+
418
+ const type = event?.type;
419
+ if (typeof type !== "string") return;
420
+ // Opt-in event tracing for diagnosing turn/transcript issues. Enable with
421
+ // `localStorage.setItem("s2s.debug", "1")` in the browser console.
422
+ if (this._debug) {
423
+ const extra = type.startsWith("conversation.item.input_audio_transcription")
424
+ ? ` item=${event.item_id} ci=${event.content_index} ${event.delta ?? event.transcript ?? ""}`
425
+ : type.startsWith("response.")
426
+ ? ` resp=${event.response_id ?? event.response?.id ?? ""} status=${event.response?.status ?? ""} ${event.transcript ?? ""}`
427
+ : "";
428
+ console.debug(`[ws] ${type}${extra}`);
429
+ }
430
+
431
+ switch (type) {
432
+ case "session.created":
433
+ // Server-side defaults for the s2s pipeline are already what we
434
+ // want (server_vad, whisper-1 transcription, PCM16 16k in / 24k
435
+ // out). We only push the user-tunable bits: voice + instructions.
436
+ this._sendSessionUpdate();
437
+ this._sessionConfigured = true;
438
+ if (this._status === "connecting") this._setStatus("connected");
439
+ break;
440
+
441
+ case "session.updated":
442
+ // Acknowledged by server, nothing to do.
443
+ break;
444
+
445
+ case "input_audio_buffer.speech_started":
446
+ // User started speaking — stop any audio still playing OR queued, every
447
+ // time. We clear unconditionally (not just when `_aiSpeaking`): after a
448
+ // reply or a tool result the worklet's ring buffer can still be draining
449
+ // even though we already flipped `_aiSpeaking` off, and that tail would
450
+ // otherwise keep playing over the user's barge-in.
451
+ this._playbackNode?.port.postMessage({ kind: "clear" });
452
+ this._aiSpeaking = false;
453
+ this._setStatus("user-speaking");
454
+ break;
455
+
456
+ case "input_audio_buffer.speech_stopped":
457
+ if (this._status === "user-speaking") this._setStatus("processing");
458
+ break;
459
+
460
+ case "response.created":
461
+ // A response now owns the slot — count it and clear our create guard
462
+ // (this confirms either our create or a server-initiated one).
463
+ this._openResponses++;
464
+ this._createInFlight = false;
465
+ if (this._status === "connected" || this._status === "user-speaking") {
466
+ this._setStatus("processing");
467
+ }
468
+ break;
469
+
470
+ case "response.output_item.added":
471
+ if (this._status === "connected" || this._status === "user-speaking") {
472
+ this._setStatus("processing");
473
+ }
474
+ break;
475
+
476
+ case "response.audio.delta":
477
+ case "response.output_audio.delta": {
478
+ this._pushAudioDelta(event.delta);
479
+ const rid = event.response_id ?? event.response?.id;
480
+ if (rid) this._audibleResponses.add(rid);
481
+ if (!this._aiSpeaking) {
482
+ this._aiSpeaking = true;
483
+ this._markAudible();
484
+ }
485
+ break;
486
+ }
487
+
488
+ case "response.content_part.added": {
489
+ const part = event.part;
490
+ if (part?.type === "audio" || part?.type === "output_audio") {
491
+ this._markAudible();
492
+ }
493
+ break;
494
+ }
495
+
496
+ case "response.done": {
497
+ this._aiSpeaking = false;
498
+ // This response freed the slot (completion OR cancellation both arrive
499
+ // as response.done). Decrement and, if a create was waiting, replay it.
500
+ this._openResponses = Math.max(0, this._openResponses - 1);
501
+ if (this._status === "ai-speaking" || this._status === "processing") {
502
+ this._setStatus("connected");
503
+ }
504
+ // A response closes here for BOTH normal completion and cancellation
505
+ // (the s2s server signals a speculative-turn interrupt as
506
+ // `response.done` with status "cancelled" — there is no separate
507
+ // `response.cancelled` event). Surface the id + status so the UI can
508
+ // drop a cancelled response's transcript and commit a completed one.
509
+ const status = event.response?.status ?? "completed";
510
+ const responseId = event.response?.id ?? "";
511
+ // Did this response ever play audio? Distinguishes a barge-in cut (the
512
+ // user heard part of it) from a speculative response that never played.
513
+ const audible = responseId ? this._audibleResponses.has(responseId) : false;
514
+ this._audibleResponses.delete(responseId);
515
+ // Pull whatever transcript the response carries, falling back to the
516
+ // segments we concatenated from the `*.transcript.done` events (plus any
517
+ // in-progress delta). For an interrupted reply the response payload may
518
+ // be empty, so this is the last chance to capture the text.
519
+ const transcript =
520
+ extractResponseTranscript(event.response) ||
521
+ this._asstDisplay(responseId) ||
522
+ "";
523
+ // Response finished — clear both transcript accumulators for it.
524
+ this._asstTranscriptByResp.delete(responseId);
525
+ this._asstFullByResp.delete(responseId);
526
+ this.dispatchEvent(new CustomEvent("response-finished", {
527
+ detail: { responseId, status, audible, transcript },
528
+ }));
529
+ // The slot is free now — replay a queued create (e.g. a tool follow-up
530
+ // that arrived while this response was still running).
531
+ this._flushQueuedCreate();
532
+ break;
533
+ }
534
+
535
+ case "response.function_call_arguments.done": {
536
+ const name = typeof event.name === "string" ? event.name : "";
537
+ const args = typeof event.arguments === "string" ? event.arguments : "{}";
538
+ const callId = typeof event.call_id === "string" ? event.call_id : "";
539
+ if (name) {
540
+ this.dispatchEvent(new CustomEvent("toolcall", {
541
+ detail: { name, arguments: args, callId },
542
+ }));
543
+ } else {
544
+ // A nameless call can't be executed, so no function_call_output is
545
+ // ever sent and the model would wait forever for a result. The
546
+ // backend shouldn't emit these; warn loudly rather than stall silently.
547
+ console.warn(`[ws] function_call_arguments.done with no name (call_id=${callId}); cannot run tool — turn may stall`);
548
+ }
549
+ break;
550
+ }
551
+
552
+ case "conversation.item.input_audio_transcription.delta": {
553
+ const delta = typeof event.delta === "string" ? event.delta : "";
554
+ if (delta) {
555
+ // `itemId` is REUSED across a speculative continuation, so the UI
556
+ // groups both segments into one message. The delta carries the full
557
+ // cumulative transcript so far (not an increment).
558
+ this.dispatchEvent(
559
+ new CustomEvent("transcript", {
560
+ detail: {
561
+ role: "user",
562
+ text: delta,
563
+ partial: true,
564
+ itemId: typeof event.item_id === "string" ? event.item_id : "",
565
+ },
566
+ }),
567
+ );
568
+ }
569
+ break;
570
+ }
571
+
572
+ case "conversation.item.input_audio_transcription.completed": {
573
+ const transcript = typeof event.transcript === "string" ? event.transcript : "";
574
+ if (transcript) {
575
+ this.dispatchEvent(
576
+ new CustomEvent("transcript", {
577
+ detail: {
578
+ role: "user",
579
+ text: transcript,
580
+ partial: false,
581
+ itemId: typeof event.item_id === "string" ? event.item_id : "",
582
+ },
583
+ }),
584
+ );
585
+ }
586
+ break;
587
+ }
588
+
589
+ case "response.audio_transcript.delta":
590
+ case "response.output_audio_transcript.delta": {
591
+ // Stream the assistant transcript live: accumulate the incremental
592
+ // deltas and push the running text to the UI. Every transcribe event we
593
+ // receive reaches the conversation, so an interrupted reply already has
594
+ // its partial text even if the `.done` never fires.
595
+ this._markAudible();
596
+ const rid = typeof event.response_id === "string" ? event.response_id : "";
597
+ const delta = typeof event.delta === "string" ? event.delta : "";
598
+ if (delta) {
599
+ this._asstTranscriptByResp.set(rid, (this._asstTranscriptByResp.get(rid) || "") + delta);
600
+ // Show completed segments + the segment streaming in right now.
601
+ this.dispatchEvent(
602
+ new CustomEvent("transcript", {
603
+ detail: { role: "assistant", text: this._asstDisplay(rid), partial: true, responseId: rid },
604
+ }),
605
+ );
606
+ }
607
+ break;
608
+ }
609
+
610
+ case "response.audio_transcript.done":
611
+ case "response.output_audio_transcript.done": {
612
+ const rid = typeof event.response_id === "string" ? event.response_id : "";
613
+ // This is ONE completed segment. A response can emit several; concatenate
614
+ // them, space-separated, until response.done clears the accumulator.
615
+ const segment =
616
+ (typeof event.transcript === "string" && event.transcript) ||
617
+ this._asstTranscriptByResp.get(rid) ||
618
+ "";
619
+ this._asstTranscriptByResp.delete(rid); // segment finished; next one starts fresh
620
+ if (segment) {
621
+ const prev = this._asstFullByResp.get(rid) || "";
622
+ this._asstFullByResp.set(rid, prev ? `${prev} ${segment}` : segment);
623
+ }
624
+ const full = this._asstFullByResp.get(rid) || "";
625
+ if (full) {
626
+ this.dispatchEvent(
627
+ new CustomEvent("transcript", {
628
+ detail: { role: "assistant", text: full, partial: false, responseId: rid },
629
+ }),
630
+ );
631
+ }
632
+ break;
633
+ }
634
+
635
+ case "error": {
636
+ const err = event.error;
637
+ console.error("[ws] server error:", err);
638
+ // The "another response is already active" race: our optimistic create
639
+ // collided with a still-running response. Don't surface it — clear the
640
+ // in-flight guard and re-queue, so the create replays on the next
641
+ // response.done (never retried immediately, which would just collide
642
+ // again).
643
+ if (err?.type === "conversation_already_has_active_response" ||
644
+ err?.code === "conversation_already_has_active_response") {
645
+ if (this._createInFlight) {
646
+ this._createInFlight = false;
647
+ // Re-queue a BARE create: any image on the original payload was
648
+ // already sent before this (rejected) create, so don't resend it.
649
+ this._createQueue.push({});
650
+ }
651
+ break;
652
+ }
653
+ // Every other server error is non-fatal: surface it for logging but
654
+ // NEVER tear the socket down. Only transport failures (close / failed
655
+ // open) are fatal, and those come through their own paths.
656
+ this.dispatchEvent(
657
+ new CustomEvent("server-error", { detail: { error: new Error(err?.message ?? "Server error") } }),
658
+ );
659
+ break;
660
+ }
661
+ }
662
+ }
663
+
664
+ /** @param {string} b64 */
665
+ _pushAudioDelta(b64) {
666
+ if (!this._playbackNode) return;
667
+ if (!b64) return;
668
+ const bytes = base64ToBytes(b64);
669
+ const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
670
+ const samples = new Float32Array(bytes.byteLength / 2);
671
+ for (let i = 0; i < samples.length; i++) {
672
+ const s = view.getInt16(i * 2, true);
673
+ samples[i] = s < 0 ? s / 0x8000 : s / 0x7fff;
674
+ }
675
+ this._playbackNode.port.postMessage({ kind: "audio", samples }, [samples.buffer]);
676
+ }
677
+
678
+ /** @param {CloseEvent} ev */
679
+ _onWsClose(ev) {
680
+ console.log("[ws] socket closed:", ev.code, ev.reason);
681
+ if (this._status === "closed" || this._status === "error") return;
682
+ if (ev.code === 1000) {
683
+ this._setStatus("closed");
684
+ } else {
685
+ this.dispatchEvent(
686
+ new CustomEvent("error", {
687
+ detail: { error: new Error(`WebSocket closed (${ev.code}) ${ev.reason || ""}`.trim()) },
688
+ }),
689
+ );
690
+ this._setStatus("error");
691
+ }
692
+ }
693
+
694
+ _sendSessionUpdate() {
695
+ // Minimal payload: only the bits the user is allowed to configure.
696
+ // The s2s server already defaults to server_vad, whisper-1
697
+ // transcription, 16 kHz PCM input and 24 kHz PCM output, so we don't
698
+ // need (and must not send) `audio.input.format`, `audio.input.transcription`,
699
+ // `audio.input.turn_detection` or `audio.output.format`: the pydantic
700
+ // validator on the server rejects the whole event if any unknown or
701
+ // future-shaped sub-field shows up.
702
+ /** @type {Record<string, any>} */
703
+ const session = {
704
+ type: "realtime",
705
+ instructions: this.options.instructions,
706
+ audio: {
707
+ output: { voice: this.options.voice },
708
+ },
709
+ };
710
+ // Tools are declared here; the backend already accepts them in
711
+ // session.update and emits response.function_call_arguments.done when the
712
+ // model decides to call one. Only include the keys when we actually have
713
+ // tools — the server's pydantic validator is strict about shapes.
714
+ if (this._tools.length) {
715
+ session.tools = this._tools;
716
+ session.tool_choice = "auto";
717
+ }
718
+ this._send({ type: "session.update", session });
719
+ }
720
+
721
+ /** Update voice/instructions on a live session without tearing down. */
722
+ /** @param {{ voice?: string; instructions?: string }} patch */
723
+ updateSession(patch) {
724
+ /** @type {Record<string, any>} */
725
+ const session = { type: "realtime" };
726
+ if (patch.instructions) session.instructions = patch.instructions;
727
+ if (patch.voice) session.audio = { output: { voice: patch.voice } };
728
+ if (Object.keys(session).length > 1) {
729
+ this._send({ type: "session.update", session });
730
+ }
731
+ }
732
+
733
+ /**
734
+ * Replace the declared tool set on a live session (e.g. the user flipped a
735
+ * tool switch mid-conversation). Always sends `tools` — an empty array
736
+ * clears them — so toggling the last tool off actually removes it.
737
+ * @param {ToolDef[]} tools
738
+ */
739
+ setTools(tools) {
740
+ this._tools = tools;
741
+ this._send({
742
+ type: "session.update",
743
+ session: { type: "realtime", tools, tool_choice: tools.length ? "auto" : "none" },
744
+ });
745
+ }
746
+
747
+ /**
748
+ * Return a tool's result to the model. Pairs with the `toolcall` event's
749
+ * `callId`. Caller follows this with `requestResponse()` so the model speaks.
750
+ * @param {string} callId
751
+ * @param {string} output Plain text / JSON string the model will read.
752
+ */
753
+ sendToolOutput(callId, output) {
754
+ if (!callId) return; // Can't target a result without the call id.
755
+ this._send({
756
+ type: "conversation.item.create",
757
+ item: { type: "function_call_output", call_id: callId, output },
758
+ });
759
+ }
760
+
761
+ /**
762
+ * Add an image to the conversation as user content, so the vision-language
763
+ * model can see it (used by the camera tool). `dataUrl` is a
764
+ * `data:image/jpeg;base64,...` string.
765
+ * @param {string} dataUrl
766
+ */
767
+ sendUserImage(dataUrl) {
768
+ this._send({
769
+ type: "conversation.item.create",
770
+ item: {
771
+ type: "message",
772
+ role: "user",
773
+ content: [{ type: "input_image", image_url: dataUrl }],
774
+ },
775
+ });
776
+ }
777
+
778
+ /**
779
+ * Ask the model to generate a response now (after feeding tool results).
780
+ * Serialized: if a response is already in flight we queue this request and
781
+ * replay it once the active response finishes, so we never trip the
782
+ * backend's `conversation_already_has_active_response` guard.
783
+ *
784
+ * @param {{ image?: string }} [opts] Optional `image` (a data URL) sent as a
785
+ * user `input_image` immediately before this response.create — so the frame
786
+ * travels with the create (and is deferred together with it if queued),
787
+ * rather than being added to the conversation eagerly. Used by the camera
788
+ * tool so the model sees the snapshot in the response it's about to speak.
789
+ */
790
+ requestResponse(opts = {}) {
791
+ if (this._responseActive()) {
792
+ this._createQueue.push(opts);
793
+ if (this._debug) console.debug(`[ws] response.create queued (a response is active); pending=${this._createQueue.length}`);
794
+ return;
795
+ }
796
+ this._createResponseNow(opts);
797
+ }
798
+
799
+ /** True while a response occupies the single backend slot. */
800
+ _responseActive() {
801
+ return this._openResponses > 0 || this._createInFlight;
802
+ }
803
+
804
+ /** Send a response.create immediately and arm the in-flight guard. Any image
805
+ * on the payload is added as user content right before the create.
806
+ * @param {{ image?: string }} [opts] */
807
+ _createResponseNow(opts = {}) {
808
+ if (!this._ws || this._ws.readyState !== WebSocket.OPEN) return;
809
+ if (opts.image) this.sendUserImage(opts.image);
810
+ this._createInFlight = true;
811
+ this._send({ type: "response.create" });
812
+ }
813
+
814
+ /** Replay one queued response.create if the slot is now free. Called on every
815
+ * response.done, so queued creates drain one-per-completion. */
816
+ _flushQueuedCreate() {
817
+ if (this._createQueue.length > 0 && !this._responseActive()) {
818
+ const opts = this._createQueue.shift();
819
+ if (this._debug) console.debug(`[ws] replaying queued response.create; remaining=${this._createQueue.length}`);
820
+ this._createResponseNow(opts);
821
+ }
822
+ }
823
+
824
+ /** @param {boolean} muted */
825
+ setMuted(muted) {
826
+ this._muted = muted;
827
+ }
828
+
829
+ /**
830
+ * Update the mic noise gate live (the user moved the Settings cursor).
831
+ * @param {NoiseGate} gate
832
+ */
833
+ setNoiseGate(gate) {
834
+ this._noiseGate = gate;
835
+ this._captureNode?.port.postMessage({ kind: "gate", ...gate });
836
+ }
837
+
838
+ /** @param {Record<string, unknown>} event */
839
+ _send(event) {
840
+ if (!this._ws || this._ws.readyState !== WebSocket.OPEN) return;
841
+ this._ws.send(JSON.stringify(event));
842
+ }
843
+
844
+ async close() {
845
+ this._visualiser?.stop();
846
+ this._visualiser = null;
847
+ try {
848
+ if (this._ws && this._ws.readyState <= WebSocket.OPEN) {
849
+ this._ws.close(1000, "client closed");
850
+ }
851
+ } catch {
852
+ // ignored
853
+ }
854
+ this._ws = null;
855
+
856
+ try {
857
+ this._captureNode?.port.close?.();
858
+ } catch {
859
+ // ignored
860
+ }
861
+ try {
862
+ this._micSrc?.disconnect();
863
+ } catch {
864
+ // ignored
865
+ }
866
+ try {
867
+ this._captureNode?.disconnect();
868
+ } catch {
869
+ // ignored
870
+ }
871
+ try {
872
+ this._micAnalyser?.disconnect();
873
+ } catch {
874
+ // ignored
875
+ }
876
+ try {
877
+ this._outAnalyser?.disconnect();
878
+ } catch {
879
+ // ignored
880
+ }
881
+ try {
882
+ this._playbackNode?.disconnect();
883
+ } catch {
884
+ // ignored
885
+ }
886
+ try {
887
+ await this._ctx?.close();
888
+ } catch {
889
+ // ignored
890
+ }
891
+ this._ctx = null;
892
+ this._captureNode = null;
893
+ this._playbackNode = null;
894
+ this._micSrc = null;
895
+ this._micAnalyser = null;
896
+ this._outAnalyser = null;
897
+ this._setStatus("closed");
898
+ }
899
+ }