feat(clients): google cloud voice: generic client, TTS, STT, Gemini audio, live voice page
src/google: GoogleClient over molo::clients (key in x-goog-api-key, never in
URLs), Service base, TextToSpeech, SpeechToText, Generative (replies, audio
in, audio out), Recognizer/Synthesizer seams with cloud, gemini and vertex
backends, VoicePipeline with bounded history. Example: /voice live
conversation page, /voice/{say,hear,turn,reset}, /google/status. Keys load
from env, .env.json.local, .env.json. Paid routes are excluded from locust.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,90 @@
|
||||
# Google Cloud APIs - Specification
|
||||
|
||||
Owned directory: `src/google/`. Namespace `molo::google`. Depends on
|
||||
`molo::clients` (HTTP client, JSON DOM, transports) and the Base64 codec in
|
||||
`src/security/crypto`. No other part is included.
|
||||
|
||||
Purpose: one generic way to call any Google Cloud REST API, plus concrete
|
||||
services built on it. Voice (text-to-speech, speech-to-text, Gemini reply and audio,
|
||||
conversation pipeline) is the first set. Adding a service means one
|
||||
`ServiceEndpoint` constant and one `Service` subclass; no change to the
|
||||
client, credentials, errors, or caps.
|
||||
|
||||
## 1. Module map (one class or concern per file)
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings, never autotuned |
|
||||
| `errors.h/.cpp` | `GoogleError` (http status, Google status, message), `NotConfiguredError`, parser for Google's `{"error":{...}}` body |
|
||||
| `credentials.h/.cpp` | `Credentials`: API key and OAuth bearer providers, env loader |
|
||||
| `endpoints.h/.cpp` | `ServiceEndpoint` plus the catalog of known services |
|
||||
| `client.h/.cpp` | `GoogleClient`: auth, JSON in/out, error mapping, over any `clients::Transport` |
|
||||
| `service.h/.cpp` | `Service` base class: holds client plus endpoint, one `call()` |
|
||||
| `text_to_speech.h/.cpp` | `TextToSpeech::synthesize`, `voices` |
|
||||
| `speech_to_text.h/.cpp` | `SpeechToText::recognize` |
|
||||
| `generative.h/.cpp` | `Generative::generate` (Gemini `generateContent`) |
|
||||
| `voice_io.h/.cpp` | `Recognizer`, `Synthesizer` seams plus request and audio types |
|
||||
| `gemini_voice.h/.cpp` | Gemini-based `Recognizer` and `Synthesizer` |
|
||||
| `voice_backend.h/.cpp` | `cloud` / `gemini` / `vertex` selection and wiring |
|
||||
| `wav.h/.cpp` | PCM16 to WAV |
|
||||
| `voice.h/.cpp` | `Responder`, `GenerativeResponder`, `VoiceSessions`, `VoicePipeline` |
|
||||
|
||||
## 2. Static caps (see caps.h)
|
||||
|
||||
| Knob | Value |
|
||||
|---|---|
|
||||
| TTS text characters per request | 5000 |
|
||||
| STT audio bytes per request | 10 MiB (Google sync limit) |
|
||||
| Generation output tokens | 300 default, 1024 max |
|
||||
| Conversation history | 20 turns per conversation |
|
||||
| Conversations held in memory | 1000, oldest evicted |
|
||||
| Conversation id | `[A-Za-z0-9_-]`, 1 to 64 chars |
|
||||
| Message text per history entry | 4000 characters |
|
||||
| Request timeout (total) | 60 s (client default) |
|
||||
|
||||
## 3. Contracts
|
||||
|
||||
- Credentials: `GOOGLE_API_KEY` is read once by `GoogleClient::from_env()`
|
||||
from the process environment, else `.env.json.local` (git-ignored, put
|
||||
secrets here), else `.env.json`, in the working directory. All env knobs
|
||||
below follow the same order.
|
||||
The key travels in the `x-goog-api-key` header, never in the URL, so it
|
||||
never reaches a span, log line, or proxy URL. Bearer tokens (OAuth, for
|
||||
APIs that refuse keys) use the same seam: `Credentials::bearer`.
|
||||
- No key: `configured()` is false and every call throws `NotConfiguredError`
|
||||
(HTTP 503, status `NOT_CONFIGURED`) before any network traffic. Loud, not
|
||||
silent.
|
||||
- Upstream failure: any non-2xx becomes `GoogleError` carrying the HTTP
|
||||
status, Google's `status` string (`PERMISSION_DENIED`, `RESOURCE_EXHAUSTED`,
|
||||
...) and message. Transport failures become `GoogleError` with status
|
||||
`UNAVAILABLE`. The key is never part of an error message.
|
||||
- Invalid input (empty text, text over cap, empty or oversized audio, bad
|
||||
conversation id) throws `GoogleError` 400 `INVALID_ARGUMENT` before any
|
||||
network traffic.
|
||||
- Transports are swappable: production uses `HttpTransport` (HTTPS via
|
||||
wolfSSL), tests use `MockTransport`. Calling code does not change.
|
||||
- Voice replies disable model thinking (`thinkingBudget` 0): reasoning tokens
|
||||
count against the output cap and truncated spoken replies in practice.
|
||||
- Backends (`voice_backend.h`): `cloud` (classic TTS/STT), `gemini`
|
||||
(generativelanguage audio in/out), `vertex` (aiplatform, express keys
|
||||
starting `AQ.`). `GOOGLE_API_BACKEND` overrides the key-prefix guess.
|
||||
Gemini TTS returns raw PCM; the server wraps it as WAV.
|
||||
- Env knobs: `GOOGLE_API_KEY`, `GOOGLE_VOICE_LANG` (default `nl-NL`),
|
||||
`GOOGLE_VOICE_NAME` (default empty: Google picks the language default),
|
||||
`GOOGLE_GENAI_MODEL` (default `gemini-3.5-flash`).
|
||||
- `VoicePipeline::turn` = recognize, respond, synthesize. Silence (empty
|
||||
transcript) returns an empty result with no reply and no synthesis, so a
|
||||
quiet room costs one STT call and nothing else.
|
||||
- History is bounded per conversation and conversations are bounded in
|
||||
count; both evict oldest first.
|
||||
|
||||
## 4. Gaps
|
||||
|
||||
- Streaming recognition (gRPC/WebSocket) and the Gemini Live API are not
|
||||
implemented; turns are request/response over REST.
|
||||
- Service-account JWT exchange for OAuth tokens: callers pass a ready token
|
||||
to `Credentials::bearer`. Owner: this part, `credentials.cpp`.
|
||||
- Free-tier quota accounting is Google's; `RESOURCE_EXHAUSTED` is surfaced,
|
||||
not tracked.
|
||||
- Live behavior against Google needs a real `GOOGLE_API_KEY`; the suite
|
||||
proves request shape and error mapping with scripted responses only.
|
||||
Reference in New Issue
Block a user