feat(clients): live: Gemini Live API over WebSocket, RFC 6455 client, session, relay, discovery check
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,165 @@
|
||||
# Google Live API - Specification
|
||||
|
||||
Owned directory: `src/google/live/`. Namespace `molo::google::live`. Depends
|
||||
on `molo::google` (errors, `CloudContext`, `valid_segment`), `molo::clients`
|
||||
(JSON DOM, `ITlsSession`) and the Base64 codec. Does not use `src/realtime`:
|
||||
its WebSocket client is a byte-pump over a caller supplied transport with no
|
||||
socket, TLS or timeout handling, and `libgoogle` does not link the realtime
|
||||
part. An RFC 6455 client lives here instead.
|
||||
|
||||
Scope: the Gemini Live API (bidirectional text, audio, video) over a
|
||||
WebSocket, on Vertex AI and on the Gemini API (API key form). Typed setup,
|
||||
typed client messages, typed server events, bounded queues, clean close, and
|
||||
an HTTP relay for hosts without WebSocket.
|
||||
|
||||
## 1. Module map (one class per concern)
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings (`LiveCaps`) |
|
||||
| `live_spec.h` | The single table of every wire field name used (`kLiveSpec`), checked by `tools/check_discovery.py` |
|
||||
| `ws_error.h` | `WsError` (kind, HTTP status of a refused upgrade) |
|
||||
| `ws_url.h/.cpp` | `WsUrl`: ws/wss parsing, `display()` without the query |
|
||||
| `ws_frame.h/.cpp` | `encode_frame`, `FrameDecoder` (strict RFC 6455), `valid_utf8` |
|
||||
| `ws_handshake.h/.cpp` | Key, SHA-1 accept, request building, 101 validation |
|
||||
| `ws_stream.h/.cpp` | `IByteStream`, `TcpStream`, `TlsStream` (over `ITlsSession`), `open_stream` |
|
||||
| `ws_client.h/.cpp` | `WsClient`: masking, fragmentation, ping/pong, close, caps |
|
||||
| `live_endpoint.h/.cpp` | `LiveEndpoint`: URL, auth headers, model naming per mode |
|
||||
| `live_config.h/.cpp` | `LiveConfig`: typed setup message |
|
||||
| `live_requests.h/.cpp` | `LiveRequests`: text, turn, audio, video, audioStreamEnd, toolResponse |
|
||||
| `live_events.h/.cpp` | `LiveEvent`, `LiveParser`: typed server events |
|
||||
| `live_session.h/.cpp` | `LiveSession`: connect, send, bounded event queue, close |
|
||||
| `live_relay.h/.cpp` | `LiveRelay`: sessions by id, events by cursor, `event_to_json` |
|
||||
| `tools/check_discovery.py` | Discovery and endpoint check (python3 stdlib) |
|
||||
|
||||
## 2. Static caps (see caps.h)
|
||||
|
||||
| Knob | Value |
|
||||
|---|---|
|
||||
| Frame payload (inbound) | 4 MiB, checked from the header before buffering |
|
||||
| Message (inbound, reassembled) | 8 MiB |
|
||||
| Message (outbound) | 2 MiB |
|
||||
| Handshake response | 16 KiB |
|
||||
| Control frame payload | 125 (RFC), close reason 123 |
|
||||
| Connect / handshake / setup timeout | 10 s / 10 s / 15 s |
|
||||
| Socket I/O timeout, reader poll | 5 s, 200 ms |
|
||||
| Close grace | 1.5 s |
|
||||
| Session lifetime | 1800 s, then closed with a `Closed` event |
|
||||
| Queued events / bytes | 1024 / 8 MiB, oldest dropped and counted (`stats().dropped`) |
|
||||
| Text / audio chunk / video chunk | 32000 B / 256 KiB / 1 MiB |
|
||||
| System instruction / function declarations / tool responses | 8000 chars / 16 / 16 |
|
||||
| Relay: sessions / log / page | 8 / 512 events / 100 events |
|
||||
|
||||
## 3. Contracts
|
||||
|
||||
- Auth forms (`LiveEndpoint`):
|
||||
- `vertex(ctx, token)`: `wss://{location}-aiplatform.googleapis.com/ws/google.cloud.aiplatform.v1.LlmBidiService/BidiGenerateContent` (`global`: `aiplatform.googleapis.com`), header `Authorization: Bearer <token>`, model `projects/{p}/locations/{l}/publishers/google/models/{m}`.
|
||||
- `vertex_express(ctx, key)`: same URL, header `x-goog-api-key`. The key is never in the URL.
|
||||
- `api_key(key)`: `wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent?key=<key>`, model `models/{m}`. The documented form puts the key in the query, the one place the foundation rule "key never in a URL" cannot hold; `WsUrl::display()` and every error omit the query.
|
||||
- `url_override` replaces the URL (loopback tests, proxies).
|
||||
- No credential: `NotConfiguredError` (503 NOT_CONFIGURED) before any socket. Bad input (model id, voice, modality, caps, mime, header characters): `GoogleError` 400 INVALID_ARGUMENT before any socket or send.
|
||||
- Upstream failures map to `GoogleError` with service `live`: refused upgrade 400/401/403/404/429 keep their status, other upgrade failures and I/O are 502 UNAVAILABLE, timeouts 504 DEADLINE_EXCEEDED, a close frame before `setupComplete` is 409 FAILED_PRECONDITION carrying Google's close code and reason (e.g. 1008 "Publisher model ... was not found").
|
||||
- Framing: client frames always masked; server frames must be unmasked; RSV bits, unknown opcodes, fragmented or oversize control frames, non-minimal lengths, continuation without start, new message inside a fragmented one, invalid UTF-8 text all close with 1002 / 1007 and throw. Over-cap frame or message closes with 1009. Pings are answered with the same payload. A close frame is echoed; a dropped connection without close frame reports code 1006.
|
||||
- Server JSON arrives in text and in binary frames; both are parsed.
|
||||
- One server message can yield several events. Order: setupComplete; model text and inline data in part order; input/output transcription; interrupted; generationComplete; turnComplete last. Inline `audio/*` is `Audio` (decoded bytes), other inline data `Media`. Undecodable messages become one `Error` event, unknown ones `Unknown`, never silence.
|
||||
- Sending is thread safe; one reader thread receives. `close()` sends a close frame, waits up to the grace, joins.
|
||||
- Relay: the session resumption handle is not sent to browsers. Ids are `s` plus 12 hex chars.
|
||||
- TLS: `TlsStream` uses `ITlsSession` (wolfSSL) on a blocking fd with socket read and write timeouts. The backend reports timeout and dropped connection alike (zero read or a read error), so the stream tells them apart by elapsed time against the read timeout. It ignores SIGPIPE process wide (the backend writes without `MSG_NOSIGNAL`).
|
||||
|
||||
## 4. Example routes (`example/app/agent/LiveDemo.cpp`)
|
||||
|
||||
| Route | Behaviour |
|
||||
|---|---|
|
||||
| `GET /agent/live` | mobile-first page, script `example/assets/agent_live.js`; matrix row `agent-live` |
|
||||
| `GET /agent/live/status` | configured, auth mode, default model, caps |
|
||||
| `POST /agent/live/sessions` | create, 201 `{id}`; 503 `google_not_configured` without credential; 422 bad input; 429 at the session cap |
|
||||
| `POST /agent/live/sessions/{id}/send` | `{kind: text\|audio\|video\|audio_end, ...}`; 422, 404 unknown id |
|
||||
| `GET /agent/live/sessions/{id}/events?cursor=N&max=M` | events after the cursor, `next_cursor`, `open`, `dropped` |
|
||||
| `POST /agent/live/sessions/{id}/close` | 200, then 404 |
|
||||
|
||||
Credential resolved from the environment: `GOOGLE_ACCESS_TOKEN` plus
|
||||
`GOOGLE_CLOUD_PROJECT` (Vertex OAuth), else `GOOGLE_API_KEY` starting `AQ.`
|
||||
plus project (Vertex express), else any `GOOGLE_API_KEY` (Gemini API form).
|
||||
`GOOGLE_LIVE_URL` overrides the WebSocket URL.
|
||||
|
||||
## 5. Verified against discovery
|
||||
|
||||
`python3 -I src/google/live/tools/check_discovery.py`, run 2026-10-24: 69
|
||||
spec rows checked (2 Discovery documents, 1 documentation page), 0
|
||||
failures; 3 unauthenticated upgrade probes answered `101 Switching
|
||||
Protocols` with the correct `Sec-WebSocket-Accept`. A bogus service name
|
||||
answers 400, so the probe discriminates.
|
||||
|
||||
- `generativelanguage/v1beta` Discovery: the setup message
|
||||
`BidiGenerateContentSetup` and its nested types (generation config, speech
|
||||
config, content, tool, function declaration, session resumption, context
|
||||
window compression, blob, function call/response, usage metadata).
|
||||
- `aiplatform/v1` Discovery: only value types (`Transcription {text,
|
||||
finished}`, `Blob`); it has no Bidi setup or message schemas.
|
||||
- Documentation only (no Discovery document describes them): the message
|
||||
envelopes (`setup`, `clientContent`, `realtimeInput`, `toolResponse`;
|
||||
`setupComplete`, `serverContent`, `toolCall`, `toolCallCancellation`,
|
||||
`goAway`, `sessionResumptionUpdate`, `usageMetadata`) and their fields.
|
||||
Checked by name against https://ai.google.dev/api/live.
|
||||
- Endpoints: v1 and v1beta1 `LlmBidiService/BidiGenerateContent` on
|
||||
`us-central1-aiplatform.googleapis.com` and the Gemini API
|
||||
`GenerativeService.BidiGenerateContent` exist (101 on upgrade).
|
||||
|
||||
## 6. Verified live
|
||||
|
||||
All 2026-10-24, project from `GOOGLE_CLOUD_PROJECT`, model
|
||||
`gemini-live-2.5-flash-native-audio`, region `us-central1`, over wss with the
|
||||
wolfSSL session. Each run: upgrade, setup, `setupComplete`, one text turn
|
||||
("Say hello in one short sentence."), receive until `turnComplete`, close.
|
||||
|
||||
| Form | Result |
|
||||
|---|---|
|
||||
| Vertex, express key, header `x-goog-api-key` | works: `setupComplete`, output transcription "Hello there!", 4 audio chunks (`audio/pcm`, 11-15 KB each), `generationComplete`, `turnComplete`, clean close |
|
||||
| Vertex, OAuth bearer (service account token) | works, same event sequence |
|
||||
| Vertex, express key as `Authorization: Bearer` | refused: close 1008 "Request had invalid authentication credentials" |
|
||||
| Gemini API form with the express key (`?key=`) | upgrade ok, close 1008 "Gemini API ... has not been used in project ... or it is disabled" (the key's project has the API disabled; the form itself is accepted) |
|
||||
| `responseModalities: ["TEXT"]` on the native audio model | close 1007 "Text output is not supported for native audio output model." |
|
||||
| region `global`, or model `gemini-2.0-flash-live-001` | close 1008 "Publisher model ... was not found" |
|
||||
|
||||
Observed: server audio mime type is plain `audio/pcm` (no `rate=`); the
|
||||
docs state 24 kHz output and 16 kHz input (`audio/pcm;rate=16000`). The
|
||||
server closes TCP after its close frame without TLS close_notify.
|
||||
Opt-in tests: `MOLO_LIVE_NET=1` with `GOOGLE_CLOUD_PROJECT` and
|
||||
`GOOGLE_API_KEY` or `GOOGLE_ACCESS_TOKEN` (`live_net.*`, loud SKIP otherwise).
|
||||
|
||||
## 7. Verified locally (deterministic, no Google)
|
||||
|
||||
Framing, handshake failures, fragmentation with interleaved ping, protocol
|
||||
violation closes, caps, abrupt disconnect, the TLS seam (a passthrough
|
||||
`ITlsSession` with wolfSSL's zero-read contract) run against a loopback
|
||||
RFC 6455 server written in `tests/src/google/live/loopback.h`. Session and
|
||||
relay tests use the same server to carry hand-written JSON in the documented
|
||||
shapes; that proves the module's handling of those shapes, not Google's
|
||||
behavior. Request bodies are checked to use only names in `kLiveSpec`.
|
||||
|
||||
## Sources
|
||||
|
||||
- https://ai.google.dev/api/live (WebSocket API reference: messages, fields)
|
||||
- https://ai.google.dev/gemini-api/docs/live-api/get-started-websocket (URL with `?key=`, setup, `audio/pcm;rate=16000`)
|
||||
- https://generativelanguage.googleapis.com/$discovery/rest?version=v1beta
|
||||
- https://aiplatform.googleapis.com/$discovery/rest?version=v1 and `version=v1beta1`
|
||||
- https://gemini-ex.hexdocs.pm/live_api.md (community client: Vertex path `wss://{location}-aiplatform.googleapis.com/ws/google.cloud.aiplatform.v1.LlmBidiService/BidiGenerateContent`)
|
||||
- https://docs.litellm.ai/docs/providers/vertex_realtime (Vertex hosts per region, OAuth bearer)
|
||||
- RFC 6455
|
||||
|
||||
## Unverified
|
||||
|
||||
- The Vertex Live reference page (docs.cloud.google.com ... reference/models/live-api) could not be read in full (very large page, navigation only in the fetched part), so Vertex-specific message differences from the Gemini API reference are unread. The Vertex endpoint path and both auth forms are verified live instead.
|
||||
- `?key=` on the Vertex endpoint: only seen to be accepted at upgrade, no turn run.
|
||||
- Not exercised live: tool calls and tool responses, `toolCallCancellation`, `goAway`, session resumption (handle exchange, resume), context window compression, video input, `audioStreamEnd`, `clientContent` turns, input transcription, activity/VAD settings, interruption. They follow the documented shapes and are covered by local tests only.
|
||||
- Express key requirement of a project in the Vertex model name: the run used the project; not tried without.
|
||||
- `timeLeft` is parsed as a protobuf duration string ("30s"); the format is from the Duration type, not from a captured message.
|
||||
- Session limits (audio-only 15 min, audio+video 2 min) are from a community guide; the module enforces its own 1800 s cap.
|
||||
|
||||
## Gaps
|
||||
|
||||
- Outbound messages are single frames (no outbound fragmentation).
|
||||
- Proactivity, `historyConfig`, labels, VAD tuning, media resolution, affective dialog are not in `LiveConfig` (setup can be extended in `live_config.cpp`).
|
||||
- No automatic reconnect or resume on `goAway`; the handle is available from `LiveSession::resumption_handle()`.
|
||||
- Token refresh is the caller's job (foundation gap).
|
||||
- The relay polls; no push to the browser (the example server has no WebSocket).
|
||||
@@ -0,0 +1,46 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for the Live API module. Never autotuned.
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct LiveCaps {
|
||||
// WebSocket transport.
|
||||
static constexpr std::size_t kMaxUrlChars = 2048;
|
||||
static constexpr std::size_t kMaxHandshakeBytes = 16 * 1024;
|
||||
static constexpr std::size_t kMaxFramePayload = 4u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxInboundMessage = 8u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxOutboundMessage = 2u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxControlPayload = 125; // RFC 6455 5.5
|
||||
static constexpr std::size_t kMaxCloseReason = 123;
|
||||
static constexpr std::size_t kReadChunk = 16 * 1024;
|
||||
static constexpr int kConnectTimeoutMs = 10000;
|
||||
static constexpr int kHandshakeTimeoutMs = 10000;
|
||||
static constexpr int kIoTimeoutMs = 5000;
|
||||
static constexpr int kPollIntervalMs = 200;
|
||||
static constexpr int kCloseGraceMs = 1500;
|
||||
|
||||
// Session.
|
||||
static constexpr int kSetupTimeoutMs = 15000;
|
||||
static constexpr int kMaxSessionSeconds = 1800;
|
||||
static constexpr std::size_t kMaxQueuedEvents = 1024;
|
||||
static constexpr std::size_t kMaxQueuedBytes = 8u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxTextChars = 32000;
|
||||
static constexpr std::size_t kMaxAudioChunkBytes = 256u * 1024u;
|
||||
static constexpr std::size_t kMaxVideoChunkBytes = 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxSystemChars = 8000;
|
||||
static constexpr std::size_t kMaxFunctionDecls = 16;
|
||||
static constexpr std::size_t kMaxFunctionResponses = 16;
|
||||
static constexpr std::size_t kMaxDescriptionChars = 1000;
|
||||
static constexpr std::size_t kMaxMimeChars = 100;
|
||||
static constexpr std::size_t kMaxIdChars = 128;
|
||||
|
||||
// Relay (example HTTP bridge).
|
||||
static constexpr std::size_t kMaxRelaySessions = 8;
|
||||
static constexpr std::size_t kMaxRelayLog = 512;
|
||||
static constexpr std::size_t kMaxRelayPage = 100;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,137 @@
|
||||
#include "google/live/live_config.h"
|
||||
|
||||
#include <cctype>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/live/caps.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
bool valid_function_name(const std::string& s) {
|
||||
if (s.empty() || s.size() > 64) return false;
|
||||
if (!(std::isalpha((unsigned char)s[0]) || s[0] == '_')) return false;
|
||||
for (unsigned char c : s)
|
||||
if (!(std::isalnum(c) || c == '_' || c == '.' || c == '-')) return false;
|
||||
return true;
|
||||
}
|
||||
bool valid_handle(const std::string& s) {
|
||||
if (s.empty() || s.size() > 1024) return false;
|
||||
for (unsigned char c : s)
|
||||
if (c <= 0x20 || c == 0x7f || c == '"') return false;
|
||||
return true;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json LiveConfig::to_setup(const LiveEndpoint& endpoint) const {
|
||||
Json setup = Json::object();
|
||||
setup.set("model", endpoint.model_name(model));
|
||||
|
||||
if (response_modalities.empty() || response_modalities.size() > 2)
|
||||
throw GoogleError::invalid("response_modalities needs 1 or 2 entries");
|
||||
Json gen = Json::object();
|
||||
Json mods = Json::array();
|
||||
for (const auto& m : response_modalities) {
|
||||
if (m != "AUDIO" && m != "TEXT")
|
||||
throw GoogleError::invalid("response modality must be AUDIO or TEXT");
|
||||
mods.push(m);
|
||||
}
|
||||
gen.set("responseModalities", mods);
|
||||
if (temperature) {
|
||||
if (*temperature < 0.0 || *temperature > 2.0)
|
||||
throw GoogleError::invalid("temperature must be within 0..2");
|
||||
gen.set("temperature", *temperature);
|
||||
}
|
||||
if (max_output_tokens) {
|
||||
if (*max_output_tokens < 1 || *max_output_tokens > 65536)
|
||||
throw GoogleError::invalid("max_output_tokens must be within 1..65536");
|
||||
gen.set("maxOutputTokens", *max_output_tokens);
|
||||
}
|
||||
if (!voice_name.empty() || !language_code.empty()) {
|
||||
Json speech = Json::object();
|
||||
if (!voice_name.empty()) {
|
||||
if (voice_name.size() > 64) throw GoogleError::invalid("voice name too long");
|
||||
for (unsigned char c : voice_name)
|
||||
if (!(std::isalnum(c) || c == '_' || c == '-'))
|
||||
throw GoogleError::invalid("voice name has invalid characters");
|
||||
Json prebuilt = Json::object();
|
||||
prebuilt.set("voiceName", voice_name);
|
||||
Json voice = Json::object();
|
||||
voice.set("prebuiltVoiceConfig", prebuilt);
|
||||
speech.set("voiceConfig", voice);
|
||||
}
|
||||
if (!language_code.empty()) {
|
||||
if (language_code.size() > 16)
|
||||
throw GoogleError::invalid("language code too long");
|
||||
for (unsigned char c : language_code)
|
||||
if (!(std::isalnum(c) || c == '-'))
|
||||
throw GoogleError::invalid("language code has invalid characters");
|
||||
speech.set("languageCode", language_code);
|
||||
}
|
||||
gen.set("speechConfig", speech);
|
||||
}
|
||||
setup.set("generationConfig", gen);
|
||||
|
||||
if (!system_instruction.empty()) {
|
||||
if (system_instruction.size() > LiveCaps::kMaxSystemChars)
|
||||
throw GoogleError::invalid("system instruction over cap");
|
||||
Json part = Json::object();
|
||||
part.set("text", system_instruction);
|
||||
Json parts = Json::array();
|
||||
parts.push(part);
|
||||
Json content = Json::object();
|
||||
content.set("parts", parts);
|
||||
setup.set("systemInstruction", content);
|
||||
}
|
||||
|
||||
if (!functions.empty()) {
|
||||
if (functions.size() > LiveCaps::kMaxFunctionDecls)
|
||||
throw GoogleError::invalid("too many function declarations");
|
||||
Json decls = Json::array();
|
||||
for (const auto& f : functions) {
|
||||
if (!valid_function_name(f.name))
|
||||
throw GoogleError::invalid("function name is malformed");
|
||||
if (f.description.size() > LiveCaps::kMaxDescriptionChars)
|
||||
throw GoogleError::invalid("function description over cap");
|
||||
Json d = Json::object();
|
||||
d.set("name", f.name);
|
||||
if (!f.description.empty()) d.set("description", f.description);
|
||||
if (!f.parameters.is_null()) {
|
||||
if (!f.parameters.is_object())
|
||||
throw GoogleError::invalid("function parameters must be a JSON object");
|
||||
d.set("parameters", f.parameters);
|
||||
}
|
||||
decls.push(d);
|
||||
}
|
||||
Json tool = Json::object();
|
||||
tool.set("functionDeclarations", decls);
|
||||
Json tools = Json::array();
|
||||
tools.push(tool);
|
||||
setup.set("tools", tools);
|
||||
}
|
||||
|
||||
if (input_transcription) setup.set("inputAudioTranscription", Json::object());
|
||||
if (output_transcription) setup.set("outputAudioTranscription", Json::object());
|
||||
|
||||
if (session_resumption || !resumption_handle.empty()) {
|
||||
Json r = Json::object();
|
||||
if (!resumption_handle.empty()) {
|
||||
if (!valid_handle(resumption_handle))
|
||||
throw GoogleError::invalid("resumption handle is malformed");
|
||||
r.set("handle", resumption_handle);
|
||||
}
|
||||
setup.set("sessionResumption", r);
|
||||
}
|
||||
if (context_compression) {
|
||||
Json c = Json::object();
|
||||
c.set("slidingWindow", Json::object());
|
||||
setup.set("contextWindowCompression", c);
|
||||
}
|
||||
Json root = Json::object();
|
||||
root.set("setup", setup);
|
||||
return root;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,40 @@
|
||||
#pragma once
|
||||
|
||||
// Typed content of the setup message (the first frame of a session).
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "google/live/live_endpoint.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct LiveFunctionDeclaration {
|
||||
std::string name; // [A-Za-z_][A-Za-z0-9_.-]{0,63}
|
||||
std::string description;
|
||||
// OpenAPI-subset Schema object, passed through; null = no parameters.
|
||||
clients::body::Json parameters;
|
||||
};
|
||||
|
||||
struct LiveConfig {
|
||||
std::string model = "gemini-live-2.5-flash-native-audio";
|
||||
std::vector<std::string> response_modalities = {"AUDIO"}; // AUDIO or TEXT
|
||||
std::string system_instruction;
|
||||
std::string voice_name;
|
||||
std::string language_code;
|
||||
std::optional<double> temperature; // 0..2
|
||||
std::optional<int> max_output_tokens;
|
||||
bool input_transcription = false;
|
||||
bool output_transcription = false;
|
||||
bool session_resumption = false;
|
||||
std::string resumption_handle; // resume a previous session
|
||||
bool context_compression = false; // slidingWindow with server defaults
|
||||
std::vector<LiveFunctionDeclaration> functions;
|
||||
|
||||
// Validates and renders {"setup": {...}}. Throws GoogleError (invalid).
|
||||
clients::body::Json to_setup(const LiveEndpoint& endpoint) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,98 @@
|
||||
#include "google/live/live_endpoint.h"
|
||||
|
||||
#include <cctype>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/live/live_spec.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
namespace {
|
||||
bool clean_secret(const std::string& s) {
|
||||
if (s.empty() || s.size() > 4096) return false;
|
||||
for (unsigned char c : s)
|
||||
if (c <= 0x20 || c == 0x7f || c == '&' || c == '#' || c == '"') return false;
|
||||
return true;
|
||||
}
|
||||
bool valid_location(const std::string& s) {
|
||||
if (s.empty() || s.size() > 40) return false;
|
||||
for (unsigned char c : s)
|
||||
if (!(std::islower(c) || std::isdigit(c) || c == '-')) return false;
|
||||
return true;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
LiveEndpoint LiveEndpoint::vertex(const CloudContext& ctx, std::string token) {
|
||||
LiveEndpoint e;
|
||||
e.mode = Mode::Vertex;
|
||||
e.project = ctx.project;
|
||||
e.location = ctx.location;
|
||||
e.token = std::move(token);
|
||||
return e;
|
||||
}
|
||||
|
||||
LiveEndpoint LiveEndpoint::vertex_express(const CloudContext& ctx, std::string key) {
|
||||
LiveEndpoint e;
|
||||
e.mode = Mode::Vertex;
|
||||
e.project = ctx.project;
|
||||
e.location = ctx.location;
|
||||
e.key = std::move(key);
|
||||
return e;
|
||||
}
|
||||
|
||||
LiveEndpoint LiveEndpoint::api_key(std::string key) {
|
||||
LiveEndpoint e;
|
||||
e.mode = Mode::ApiKey;
|
||||
e.key = std::move(key);
|
||||
return e;
|
||||
}
|
||||
|
||||
bool LiveEndpoint::has_credential() const {
|
||||
return mode == Mode::Vertex ? !(token.empty() && key.empty()) : !key.empty();
|
||||
}
|
||||
|
||||
void LiveEndpoint::validate() const {
|
||||
if (!has_credential()) throw NotConfiguredError("live");
|
||||
if (mode == Mode::Vertex) {
|
||||
if (!token.empty() && !clean_secret(token))
|
||||
throw GoogleError::invalid("access token has forbidden characters");
|
||||
if (!key.empty() && !clean_secret(key))
|
||||
throw GoogleError::invalid("api key has forbidden characters");
|
||||
if (!valid_location(location)) throw GoogleError::invalid("location is malformed");
|
||||
CloudContext ctx;
|
||||
ctx.project = project;
|
||||
ctx.location = location;
|
||||
ctx.parent(); // validates the project
|
||||
} else if (!clean_secret(key)) {
|
||||
throw GoogleError::invalid("api key has forbidden characters");
|
||||
}
|
||||
}
|
||||
|
||||
std::string LiveEndpoint::url() const {
|
||||
if (!url_override.empty()) return url_override;
|
||||
if (mode == Mode::ApiKey) return std::string(kLiveApiKeyUrl) + "?key=" + key;
|
||||
std::string host = location == "global" ? "aiplatform.googleapis.com"
|
||||
: location + "-aiplatform.googleapis.com";
|
||||
return "wss://" + host + kLiveVertexPath;
|
||||
}
|
||||
|
||||
WsHeaders LiveEndpoint::headers() const {
|
||||
WsHeaders h;
|
||||
if (mode == Mode::Vertex) {
|
||||
// OAuth token wins; an express key travels in the header, not the URL.
|
||||
if (!token.empty()) h.emplace_back("Authorization", "Bearer " + token);
|
||||
else h.emplace_back("x-goog-api-key", key);
|
||||
}
|
||||
return h;
|
||||
}
|
||||
|
||||
std::string LiveEndpoint::model_name(const std::string& model_id) const {
|
||||
if (!valid_segment(model_id)) throw GoogleError::invalid("model id is malformed");
|
||||
if (mode == Mode::ApiKey) return "models/" + model_id;
|
||||
CloudContext ctx;
|
||||
ctx.project = project;
|
||||
ctx.location = location;
|
||||
return ctx.parent() + "/publishers/google/models/" + model_id;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
// Where and how a Live session connects. Two forms:
|
||||
// - Vertex AI: wss://{location}-aiplatform.googleapis.com/ws/<LlmBidiService>
|
||||
// with "Authorization: Bearer <OAuth token>"; model resource
|
||||
// projects/{p}/locations/{l}/publishers/google/models/{m}.
|
||||
// - API key (Gemini API): wss://generativelanguage.googleapis.com/ws/...
|
||||
// with the key in "?key=" (the documented form; the one place the key is
|
||||
// in a URL, so WsUrl::display() and every error omit the query);
|
||||
// model models/{m}.
|
||||
// url_override (tests, proxies) replaces the URL; headers and model naming
|
||||
// still follow the mode.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "google/live/ws_handshake.h"
|
||||
#include "google/resource.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct LiveEndpoint {
|
||||
enum class Mode { Vertex, ApiKey };
|
||||
Mode mode = Mode::ApiKey;
|
||||
std::string project;
|
||||
std::string location = "us-central1";
|
||||
std::string token; // Vertex: OAuth access token
|
||||
std::string key; // ApiKey mode, or Vertex express key when token is empty
|
||||
std::string url_override;
|
||||
|
||||
static LiveEndpoint vertex(const CloudContext& ctx, std::string token);
|
||||
// Vertex with an express-mode API key (sent as x-goog-api-key).
|
||||
static LiveEndpoint vertex_express(const CloudContext& ctx, std::string key);
|
||||
static LiveEndpoint api_key(std::string key);
|
||||
|
||||
bool has_credential() const;
|
||||
// Throws GoogleError (invalid) on a bad project, location or credential.
|
||||
void validate() const;
|
||||
std::string url() const; // full URL, may contain the key
|
||||
WsHeaders headers() const;
|
||||
// Model id -> wire name for this mode. Id validated like a path segment.
|
||||
std::string model_name(const std::string& model_id) const;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,192 @@
|
||||
#include "google/live/live_events.h"
|
||||
|
||||
#include <cstdlib>
|
||||
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using clients::body::Json;
|
||||
using molo::security::crypto::Base64;
|
||||
|
||||
const char* live_event_name(LiveEventKind k) {
|
||||
switch (k) {
|
||||
case LiveEventKind::SetupComplete: return "setup_complete";
|
||||
case LiveEventKind::Text: return "text";
|
||||
case LiveEventKind::Audio: return "audio";
|
||||
case LiveEventKind::Media: return "media";
|
||||
case LiveEventKind::InputTranscription: return "input_transcription";
|
||||
case LiveEventKind::OutputTranscription: return "output_transcription";
|
||||
case LiveEventKind::GenerationComplete: return "generation_complete";
|
||||
case LiveEventKind::TurnComplete: return "turn_complete";
|
||||
case LiveEventKind::Interrupted: return "interrupted";
|
||||
case LiveEventKind::ToolCall: return "tool_call";
|
||||
case LiveEventKind::ToolCallCancellation: return "tool_call_cancellation";
|
||||
case LiveEventKind::GoAway: return "go_away";
|
||||
case LiveEventKind::SessionResumption: return "session_resumption";
|
||||
case LiveEventKind::Usage: return "usage";
|
||||
case LiveEventKind::Unknown: return "unknown";
|
||||
case LiveEventKind::Closed: return "closed";
|
||||
case LiveEventKind::Error: return "error";
|
||||
}
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
double LiveParser::parse_duration(const std::string& s) {
|
||||
if (s.size() < 2 || s.back() != 's') return -1;
|
||||
std::string num = s.substr(0, s.size() - 1);
|
||||
char* end = nullptr;
|
||||
double v = std::strtod(num.c_str(), &end);
|
||||
if (end != num.c_str() + num.size() || v < 0) return -1;
|
||||
return v;
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
const std::string* str(const Json& obj, const char* key) {
|
||||
const Json* v = obj.find(key);
|
||||
return v && v->is_string() ? &v->as_string() : nullptr;
|
||||
}
|
||||
|
||||
LiveEvent error_event(const std::string& why) {
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::Error;
|
||||
e.text = why;
|
||||
return e;
|
||||
}
|
||||
|
||||
void parse_server_content(const Json& sc, std::vector<LiveEvent>& out) {
|
||||
if (const Json* turn = sc.find("modelTurn"); turn && turn->is_object()) {
|
||||
if (const Json* parts = turn->find("parts"); parts && parts->is_array()) {
|
||||
for (const Json& part : parts->as_array()) {
|
||||
if (!part.is_object()) continue;
|
||||
if (const std::string* t = str(part, "text")) {
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::Text;
|
||||
e.text = *t;
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (const Json* inl = part.find("inlineData"); inl && inl->is_object()) {
|
||||
const std::string* d = str(*inl, "data");
|
||||
const std::string* m = str(*inl, "mimeType");
|
||||
LiveEvent e;
|
||||
e.mime = m ? *m : "";
|
||||
if (!d || !Base64::decode(*d, e.data)) {
|
||||
out.push_back(error_event("inlineData is not valid base64"));
|
||||
continue;
|
||||
}
|
||||
e.kind = e.mime.rfind("audio/", 0) == 0 ? LiveEventKind::Audio
|
||||
: LiveEventKind::Media;
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const char* key : {"inputTranscription", "outputTranscription"}) {
|
||||
if (const Json* t = sc.find(key); t && t->is_object()) {
|
||||
LiveEvent e;
|
||||
e.kind = key[0] == 'i' ? LiveEventKind::InputTranscription
|
||||
: LiveEventKind::OutputTranscription;
|
||||
if (const std::string* s = str(*t, "text")) e.text = *s;
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
}
|
||||
auto flag = [&](const char* key, LiveEventKind kind) {
|
||||
if (const Json* f = sc.find(key); f && f->is_bool() && f->as_bool()) {
|
||||
LiveEvent e;
|
||||
e.kind = kind;
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
};
|
||||
flag("interrupted", LiveEventKind::Interrupted);
|
||||
flag("generationComplete", LiveEventKind::GenerationComplete);
|
||||
flag("turnComplete", LiveEventKind::TurnComplete);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::vector<LiveEvent> LiveParser::parse(const std::string& message) {
|
||||
std::vector<LiveEvent> out;
|
||||
Json root;
|
||||
try {
|
||||
root = Json::parse(message);
|
||||
} catch (const std::exception&) {
|
||||
out.push_back(error_event("server message is not valid JSON"));
|
||||
return out;
|
||||
}
|
||||
if (!root.is_object()) {
|
||||
out.push_back(error_event("server message is not a JSON object"));
|
||||
return out;
|
||||
}
|
||||
bool known = false;
|
||||
if (root.has("setupComplete")) {
|
||||
known = true;
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::SetupComplete;
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (const Json* sc = root.find("serverContent"); sc && sc->is_object()) {
|
||||
known = true;
|
||||
parse_server_content(*sc, out);
|
||||
}
|
||||
if (const Json* tc = root.find("toolCall"); tc && tc->is_object()) {
|
||||
known = true;
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::ToolCall;
|
||||
if (const Json* fc = tc->find("functionCalls"); fc && fc->is_array()) {
|
||||
for (const Json& c : fc->as_array()) {
|
||||
if (!c.is_object()) continue;
|
||||
LiveFunctionCall call;
|
||||
if (const std::string* s = str(c, "id")) call.id = *s;
|
||||
if (const std::string* s = str(c, "name")) call.name = *s;
|
||||
if (const Json* a = c.find("args")) call.args = *a;
|
||||
e.calls.push_back(std::move(call));
|
||||
}
|
||||
}
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (const Json* tc = root.find("toolCallCancellation"); tc && tc->is_object()) {
|
||||
known = true;
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::ToolCallCancellation;
|
||||
if (const Json* ids = tc->find("ids"); ids && ids->is_array())
|
||||
for (const Json& i : ids->as_array())
|
||||
if (i.is_string()) e.ids.push_back(i.as_string());
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (const Json* ga = root.find("goAway"); ga && ga->is_object()) {
|
||||
known = true;
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::GoAway;
|
||||
if (const std::string* t = str(*ga, "timeLeft")) {
|
||||
e.seconds = parse_duration(*t);
|
||||
e.text = *t;
|
||||
}
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (const Json* sr = root.find("sessionResumptionUpdate"); sr && sr->is_object()) {
|
||||
known = true;
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::SessionResumption;
|
||||
if (const std::string* h = str(*sr, "newHandle")) e.handle = *h;
|
||||
if (const Json* r = sr->find("resumable"); r && r->is_bool()) e.resumable = r->as_bool();
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (const Json* um = root.find("usageMetadata"); um && um->is_object()) {
|
||||
known = true;
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::Usage;
|
||||
if (const Json* t = um->find("totalTokenCount"); t && t->is_number())
|
||||
e.total_tokens = t->as_int();
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
if (!known) {
|
||||
LiveEvent e;
|
||||
e.kind = LiveEventKind::Unknown;
|
||||
e.text = message.substr(0, 200);
|
||||
out.push_back(std::move(e));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,67 @@
|
||||
#pragma once
|
||||
|
||||
// Typed events decoded from server messages. One server message can carry
|
||||
// several facets (model audio plus turnComplete); each becomes one event, in
|
||||
// a fixed order with turnComplete last.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
enum class LiveEventKind {
|
||||
SetupComplete,
|
||||
Text, // model text part
|
||||
Audio, // model inline audio (data = raw bytes, mime e.g. audio/pcm;rate=24000)
|
||||
Media, // other inline data
|
||||
InputTranscription,
|
||||
OutputTranscription,
|
||||
GenerationComplete,
|
||||
TurnComplete,
|
||||
Interrupted,
|
||||
ToolCall,
|
||||
ToolCallCancellation,
|
||||
GoAway,
|
||||
SessionResumption,
|
||||
Usage,
|
||||
Unknown, // a server message with no field this module knows
|
||||
Closed, // connection ended (code, text = reason)
|
||||
Error, // undecodable message or transport failure (text)
|
||||
};
|
||||
|
||||
const char* live_event_name(LiveEventKind kind);
|
||||
|
||||
struct LiveFunctionCall {
|
||||
std::string id;
|
||||
std::string name;
|
||||
clients::body::Json args;
|
||||
};
|
||||
|
||||
struct LiveEvent {
|
||||
LiveEventKind kind = LiveEventKind::Unknown;
|
||||
std::uint64_t seq = 0; // assigned by the session, strictly increasing
|
||||
std::string text; // Text, transcriptions, Error, Closed reason
|
||||
std::string data; // Audio and Media bytes (decoded)
|
||||
std::string mime;
|
||||
std::vector<LiveFunctionCall> calls; // ToolCall
|
||||
std::vector<std::string> ids; // ToolCallCancellation
|
||||
double seconds = 0; // GoAway timeLeft
|
||||
std::string handle; // SessionResumption newHandle
|
||||
bool resumable = false;
|
||||
long total_tokens = 0; // Usage
|
||||
int code = 0; // Closed
|
||||
|
||||
std::size_t weight() const { return text.size() + data.size() + 64; }
|
||||
};
|
||||
|
||||
class LiveParser {
|
||||
public:
|
||||
// Never throws: undecodable input becomes one Error event.
|
||||
static std::vector<LiveEvent> parse(const std::string& message);
|
||||
// "30s", "1.5s", "0.500s" -> seconds; -1 when malformed.
|
||||
static double parse_duration(const std::string& s);
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,162 @@
|
||||
#include "google/live/live_relay.h"
|
||||
|
||||
#include <sys/random.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
Json event_to_json(const LiveEvent& e) {
|
||||
Json o = Json::object();
|
||||
o.set("seq", (double)e.seq);
|
||||
o.set("type", live_event_name(e.kind));
|
||||
switch (e.kind) {
|
||||
case LiveEventKind::Text:
|
||||
case LiveEventKind::InputTranscription:
|
||||
case LiveEventKind::OutputTranscription:
|
||||
case LiveEventKind::Error:
|
||||
o.set("text", e.text);
|
||||
break;
|
||||
case LiveEventKind::Audio:
|
||||
case LiveEventKind::Media:
|
||||
o.set("mime", e.mime);
|
||||
o.set("data_b64", security::crypto::Base64::encode(e.data));
|
||||
break;
|
||||
case LiveEventKind::ToolCall: {
|
||||
Json calls = Json::array();
|
||||
for (const auto& c : e.calls) {
|
||||
Json j = Json::object();
|
||||
j.set("id", c.id);
|
||||
j.set("name", c.name);
|
||||
j.set("args", c.args);
|
||||
calls.push(j);
|
||||
}
|
||||
o.set("calls", calls);
|
||||
break;
|
||||
}
|
||||
case LiveEventKind::ToolCallCancellation: {
|
||||
Json ids = Json::array();
|
||||
for (const auto& i : e.ids) ids.push(i);
|
||||
o.set("ids", ids);
|
||||
break;
|
||||
}
|
||||
case LiveEventKind::GoAway:
|
||||
o.set("seconds", e.seconds);
|
||||
break;
|
||||
case LiveEventKind::SessionResumption:
|
||||
o.set("resumable", e.resumable);
|
||||
break; // the handle itself is not relayed to browsers
|
||||
case LiveEventKind::Usage:
|
||||
o.set("total_tokens", (double)e.total_tokens);
|
||||
break;
|
||||
case LiveEventKind::Closed:
|
||||
o.set("code", e.code);
|
||||
o.set("text", e.text);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
bool LiveRelay::valid_id(const std::string& id) {
|
||||
if (id.size() != 13 || id[0] != 's') return false;
|
||||
for (std::size_t i = 1; i < id.size(); ++i)
|
||||
if (!std::isxdigit((unsigned char)id[i])) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
std::shared_ptr<LiveRelay::Entry> LiveRelay::find(const std::string& id) const {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
auto it = entries_.find(id);
|
||||
if (!valid_id(id) || it == entries_.end())
|
||||
throw GoogleError(404, "NOT_FOUND", "no such live session", "live");
|
||||
return it->second;
|
||||
}
|
||||
|
||||
std::shared_ptr<LiveSession> LiveRelay::get(const std::string& id) const {
|
||||
return find(id)->session;
|
||||
}
|
||||
|
||||
std::string LiveRelay::create(LiveEndpoint endpoint, LiveConfig config,
|
||||
LiveOptions options) {
|
||||
{
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
for (auto it = entries_.begin(); it != entries_.end();) {
|
||||
if (!it->second->session->open() && it->second->session->pending() == 0)
|
||||
it = entries_.erase(it);
|
||||
else
|
||||
++it;
|
||||
}
|
||||
if (entries_.size() >= max_sessions_)
|
||||
throw GoogleError(429, "RESOURCE_EXHAUSTED", "too many live sessions", "live");
|
||||
}
|
||||
auto entry = std::make_shared<Entry>();
|
||||
entry->session = std::make_shared<LiveSession>(std::move(endpoint),
|
||||
std::move(config), std::move(options));
|
||||
entry->session->connect();
|
||||
unsigned char raw[6];
|
||||
if (::getrandom(raw, sizeof raw, 0) != (ssize_t)sizeof raw)
|
||||
throw GoogleError(500, "INTERNAL", "no entropy for session id", "live");
|
||||
static const char* hex = "0123456789abcdef";
|
||||
std::string id = "s";
|
||||
for (unsigned char b : raw) {
|
||||
id.push_back(hex[b >> 4]);
|
||||
id.push_back(hex[b & 15]);
|
||||
}
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
if (entries_.size() >= max_sessions_)
|
||||
throw GoogleError(429, "RESOURCE_EXHAUSTED", "too many live sessions", "live");
|
||||
entries_[id] = entry;
|
||||
return id;
|
||||
}
|
||||
|
||||
RelayPage LiveRelay::poll(const std::string& id, std::uint64_t cursor,
|
||||
std::size_t max) {
|
||||
auto entry = find(id);
|
||||
max = std::min(std::max<std::size_t>(max, 1), LiveCaps::kMaxRelayPage);
|
||||
std::lock_guard<std::mutex> lock(entry->mu);
|
||||
while (auto e = entry->session->next_event(0)) entry->log.push_back(std::move(*e));
|
||||
if (entry->log.size() > LiveCaps::kMaxRelayLog) {
|
||||
std::size_t drop = entry->log.size() - LiveCaps::kMaxRelayLog;
|
||||
entry->log.erase(entry->log.begin(), entry->log.begin() + (std::ptrdiff_t)drop);
|
||||
entry->dropped += drop;
|
||||
}
|
||||
RelayPage page;
|
||||
page.next_cursor = cursor;
|
||||
for (const auto& e : entry->log) {
|
||||
if (e.seq <= cursor) continue;
|
||||
if (page.events.size() >= max) break;
|
||||
page.events.push_back(e);
|
||||
page.next_cursor = e.seq;
|
||||
}
|
||||
page.open = entry->session->open();
|
||||
page.dropped = entry->dropped + entry->session->stats().dropped;
|
||||
return page;
|
||||
}
|
||||
|
||||
void LiveRelay::close(const std::string& id) {
|
||||
std::shared_ptr<Entry> entry;
|
||||
{
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
auto it = entries_.find(id);
|
||||
if (!valid_id(id) || it == entries_.end())
|
||||
throw GoogleError(404, "NOT_FOUND", "no such live session", "live");
|
||||
entry = it->second;
|
||||
entries_.erase(it);
|
||||
}
|
||||
entry->session->close();
|
||||
}
|
||||
|
||||
std::size_t LiveRelay::size() const {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
return entries_.size();
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,60 @@
|
||||
#pragma once
|
||||
|
||||
// HTTP-friendly view of live sessions for hosts without WebSocket (the
|
||||
// example server): sessions by id, events read by cursor from a bounded log.
|
||||
|
||||
#include <cstdint>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "google/live/live_session.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
// {"seq","type", ...}; audio and media bytes as "data_b64".
|
||||
clients::body::Json event_to_json(const LiveEvent& e);
|
||||
|
||||
struct RelayPage {
|
||||
std::vector<LiveEvent> events;
|
||||
std::uint64_t next_cursor = 0;
|
||||
bool open = false;
|
||||
std::uint64_t dropped = 0;
|
||||
};
|
||||
|
||||
class LiveRelay {
|
||||
public:
|
||||
explicit LiveRelay(std::size_t max_sessions = LiveCaps::kMaxRelaySessions)
|
||||
: max_sessions_(max_sessions) {}
|
||||
|
||||
static bool valid_id(const std::string& id);
|
||||
|
||||
// Connects (blocking) and registers. Throws GoogleError; 429
|
||||
// RESOURCE_EXHAUSTED when max_sessions open sessions exist.
|
||||
std::string create(LiveEndpoint endpoint, LiveConfig config,
|
||||
LiveOptions options = {});
|
||||
// Throws GoogleError 404 NOT_FOUND for unknown or malformed ids.
|
||||
std::shared_ptr<LiveSession> get(const std::string& id) const;
|
||||
// Events with seq > cursor, at most max (capped at kMaxRelayPage).
|
||||
RelayPage poll(const std::string& id, std::uint64_t cursor, std::size_t max);
|
||||
void close(const std::string& id);
|
||||
std::size_t size() const;
|
||||
|
||||
private:
|
||||
struct Entry {
|
||||
std::shared_ptr<LiveSession> session;
|
||||
std::mutex mu;
|
||||
std::vector<LiveEvent> log;
|
||||
std::uint64_t dropped = 0;
|
||||
};
|
||||
std::shared_ptr<Entry> find(const std::string& id) const;
|
||||
|
||||
std::size_t max_sessions_;
|
||||
mutable std::mutex mu_;
|
||||
std::map<std::string, std::shared_ptr<Entry>> entries_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,128 @@
|
||||
#include "google/live/live_requests.h"
|
||||
|
||||
#include <cctype>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/live/caps.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using clients::body::Json;
|
||||
using molo::security::crypto::Base64;
|
||||
|
||||
namespace {
|
||||
|
||||
void check_text(const std::string& t) {
|
||||
if (t.empty()) throw GoogleError::invalid("text is empty");
|
||||
if (t.size() > LiveCaps::kMaxTextChars)
|
||||
throw GoogleError::invalid("text over " + std::to_string(LiveCaps::kMaxTextChars) + " bytes");
|
||||
}
|
||||
|
||||
std::string blob_message(const char* field, const std::string& bytes,
|
||||
const std::string& mime, std::size_t cap) {
|
||||
if (bytes.empty()) throw GoogleError::invalid(std::string(field) + " chunk is empty");
|
||||
if (bytes.size() > cap)
|
||||
throw GoogleError::invalid(std::string(field) + " chunk over " + std::to_string(cap) + " bytes");
|
||||
Json blob = Json::object();
|
||||
blob.set("data", Base64::encode(bytes));
|
||||
blob.set("mimeType", mime);
|
||||
Json input = Json::object();
|
||||
input.set(field, blob);
|
||||
Json root = Json::object();
|
||||
root.set("realtimeInput", input);
|
||||
return root.dump();
|
||||
}
|
||||
|
||||
bool valid_id(const std::string& s) {
|
||||
if (s.empty() || s.size() > LiveCaps::kMaxIdChars) return false;
|
||||
for (unsigned char c : s)
|
||||
if (c <= 0x20 || c == 0x7f) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
void LiveRequests::check_mime(const std::string& mime, const char* a, const char* b) {
|
||||
if (mime.empty() || mime.size() > LiveCaps::kMaxMimeChars)
|
||||
throw GoogleError::invalid("mime type empty or too long");
|
||||
for (unsigned char c : mime)
|
||||
if (!(std::isalnum(c) || c == '/' || c == ';' || c == '=' || c == '-' ||
|
||||
c == '+' || c == '.' || c == '_'))
|
||||
throw GoogleError::invalid("mime type has invalid characters");
|
||||
bool ok = mime.rfind(a, 0) == 0 || (b && mime.rfind(b, 0) == 0);
|
||||
if (!ok) throw GoogleError::invalid("mime type must start with " + std::string(a) +
|
||||
(b ? std::string(" or ") + b : ""));
|
||||
}
|
||||
|
||||
std::string LiveRequests::text(const std::string& t) {
|
||||
check_text(t);
|
||||
Json input = Json::object();
|
||||
input.set("text", t);
|
||||
Json root = Json::object();
|
||||
root.set("realtimeInput", input);
|
||||
return root.dump();
|
||||
}
|
||||
|
||||
std::string LiveRequests::turn(const std::string& t, bool turn_complete) {
|
||||
check_text(t);
|
||||
Json part = Json::object();
|
||||
part.set("text", t);
|
||||
Json parts = Json::array();
|
||||
parts.push(part);
|
||||
Json turn = Json::object();
|
||||
turn.set("role", "user");
|
||||
turn.set("parts", parts);
|
||||
Json turns = Json::array();
|
||||
turns.push(turn);
|
||||
Json cc = Json::object();
|
||||
cc.set("turns", turns);
|
||||
cc.set("turnComplete", turn_complete);
|
||||
Json root = Json::object();
|
||||
root.set("clientContent", cc);
|
||||
return root.dump();
|
||||
}
|
||||
|
||||
std::string LiveRequests::audio(const std::string& bytes, const std::string& mime) {
|
||||
check_mime(mime, "audio/");
|
||||
return blob_message("audio", bytes, mime, LiveCaps::kMaxAudioChunkBytes);
|
||||
}
|
||||
|
||||
std::string LiveRequests::video(const std::string& bytes, const std::string& mime) {
|
||||
check_mime(mime, "image/", "video/");
|
||||
return blob_message("video", bytes, mime, LiveCaps::kMaxVideoChunkBytes);
|
||||
}
|
||||
|
||||
std::string LiveRequests::audio_stream_end() {
|
||||
Json input = Json::object();
|
||||
input.set("audioStreamEnd", true);
|
||||
Json root = Json::object();
|
||||
root.set("realtimeInput", input);
|
||||
return root.dump();
|
||||
}
|
||||
|
||||
std::string LiveRequests::tool_responses(const std::vector<LiveToolResponse>& rs) {
|
||||
if (rs.empty() || rs.size() > LiveCaps::kMaxFunctionResponses)
|
||||
throw GoogleError::invalid("tool responses need 1.." +
|
||||
std::to_string(LiveCaps::kMaxFunctionResponses) + " entries");
|
||||
Json arr = Json::array();
|
||||
for (const auto& r : rs) {
|
||||
if (!valid_id(r.id)) throw GoogleError::invalid("tool response id is malformed");
|
||||
if (r.name.empty() || r.name.size() > 64)
|
||||
throw GoogleError::invalid("tool response name is malformed");
|
||||
if (!r.response.is_object())
|
||||
throw GoogleError::invalid("tool response must be a JSON object");
|
||||
Json o = Json::object();
|
||||
o.set("id", r.id);
|
||||
o.set("name", r.name);
|
||||
o.set("response", r.response);
|
||||
arr.push(o);
|
||||
}
|
||||
Json tr = Json::object();
|
||||
tr.set("functionResponses", arr);
|
||||
Json root = Json::object();
|
||||
root.set("toolResponse", tr);
|
||||
return root.dump();
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,42 @@
|
||||
#pragma once
|
||||
|
||||
// Builders and validators for client messages after setup. Every builder
|
||||
// enforces the caps and returns the JSON text ready to send.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct LiveToolResponse {
|
||||
std::string id; // the functionCalls[].id being answered
|
||||
std::string name;
|
||||
clients::body::Json response; // JSON object
|
||||
};
|
||||
|
||||
class LiveRequests {
|
||||
public:
|
||||
static constexpr const char* kDefaultAudioMime = "audio/pcm;rate=16000";
|
||||
static constexpr const char* kDefaultVideoMime = "image/jpeg";
|
||||
|
||||
// {"realtimeInput":{"text":...}}
|
||||
static std::string text(const std::string& text);
|
||||
// {"clientContent":{"turns":[{"role":"user","parts":[{"text":...}]}],"turnComplete":b}}
|
||||
static std::string turn(const std::string& text, bool turn_complete);
|
||||
// {"realtimeInput":{"audio":{"data":b64,"mimeType":...}}}; mime audio/*.
|
||||
static std::string audio(const std::string& bytes, const std::string& mime);
|
||||
// {"realtimeInput":{"video":{...}}}; mime image/* or video/*.
|
||||
static std::string video(const std::string& bytes, const std::string& mime);
|
||||
// {"realtimeInput":{"audioStreamEnd":true}}
|
||||
static std::string audio_stream_end();
|
||||
// {"toolResponse":{"functionResponses":[...]}}
|
||||
static std::string tool_responses(const std::vector<LiveToolResponse>& r);
|
||||
|
||||
// Throws GoogleError (invalid) on a bad mime type.
|
||||
static void check_mime(const std::string& mime, const char* family_a,
|
||||
const char* family_b = nullptr);
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,224 @@
|
||||
#include "google/live/live_session.h"
|
||||
|
||||
#include <chrono>
|
||||
|
||||
#include "google/errors.h"
|
||||
#include "google/live/ws_error.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
namespace {
|
||||
|
||||
using Clock = std::chrono::steady_clock;
|
||||
|
||||
std::int64_t now_ms() {
|
||||
return std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
Clock::now().time_since_epoch())
|
||||
.count();
|
||||
}
|
||||
|
||||
GoogleError map_error(const WsError& e) {
|
||||
using K = WsError::Kind;
|
||||
switch (e.kind()) {
|
||||
case K::Invalid: return GoogleError(400, "INVALID_ARGUMENT", e.what(), "live");
|
||||
case K::Timeout: return GoogleError(504, "DEADLINE_EXCEEDED", e.what(), "live");
|
||||
case K::TooBig: return GoogleError(413, "RESOURCE_EXHAUSTED", e.what(), "live");
|
||||
case K::Handshake:
|
||||
switch (e.http_status()) {
|
||||
case 400: return GoogleError(400, "INVALID_ARGUMENT", e.what(), "live");
|
||||
case 401: return GoogleError(401, "UNAUTHENTICATED", e.what(), "live");
|
||||
case 403: return GoogleError(403, "PERMISSION_DENIED", e.what(), "live");
|
||||
case 404: return GoogleError(404, "NOT_FOUND", e.what(), "live");
|
||||
case 429: return GoogleError(429, "RESOURCE_EXHAUSTED", e.what(), "live");
|
||||
default: return GoogleError(502, "UNAVAILABLE", e.what(), "live");
|
||||
}
|
||||
case K::Closed: return GoogleError(409, "FAILED_PRECONDITION", e.what(), "live");
|
||||
default: return GoogleError(502, "UNAVAILABLE", e.what(), "live");
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
LiveSession::LiveSession(LiveEndpoint endpoint, LiveConfig config,
|
||||
LiveOptions options)
|
||||
: endpoint_(std::move(endpoint)),
|
||||
config_(std::move(config)),
|
||||
options_(std::move(options)) {}
|
||||
|
||||
LiveSession::~LiveSession() {
|
||||
try {
|
||||
close();
|
||||
} catch (...) {
|
||||
}
|
||||
}
|
||||
|
||||
void LiveSession::connect() {
|
||||
if (connected_.exchange(true)) throw GoogleError::invalid("session already connected");
|
||||
endpoint_.validate();
|
||||
std::string setup = config_.to_setup(endpoint_).dump();
|
||||
try {
|
||||
WsUrl url = WsUrl::parse(endpoint_.url());
|
||||
ws_ = std::make_unique<WsClient>(open_stream(
|
||||
url, options_.connect_timeout_ms, LiveCaps::kIoTimeoutMs, options_.tls));
|
||||
ws_->handshake(url, endpoint_.headers(), options_.handshake_timeout_ms);
|
||||
ws_->send_text(setup);
|
||||
ws_->set_read_timeout_ms(LiveCaps::kPollIntervalMs);
|
||||
auto deadline = now_ms() + options_.setup_timeout_ms;
|
||||
bool ready = false;
|
||||
while (!ready) {
|
||||
if (now_ms() > deadline) throw WsError(WsError::Kind::Timeout, "setupComplete not received in time");
|
||||
auto m = ws_->receive();
|
||||
if (!m) continue;
|
||||
if (m->kind == WsMessage::Kind::Close)
|
||||
throw WsError(WsError::Kind::Closed,
|
||||
"server closed before setup completed (code " +
|
||||
std::to_string(m->code) + "): " + m->data.substr(0, 500));
|
||||
for (auto& e : LiveParser::parse(m->data)) {
|
||||
if (e.kind == LiveEventKind::SetupComplete) ready = true;
|
||||
push(std::move(e));
|
||||
}
|
||||
}
|
||||
} catch (const WsError& e) {
|
||||
if (ws_) ws_->shutdown();
|
||||
ws_.reset();
|
||||
throw map_error(e);
|
||||
}
|
||||
open_ = true;
|
||||
reader_ = std::thread([this] { reader_loop(); });
|
||||
}
|
||||
|
||||
void LiveSession::send_raw(const std::string& json) {
|
||||
if (!open_ || !ws_) throw GoogleError(409, "FAILED_PRECONDITION", "session is not open", "live");
|
||||
try {
|
||||
ws_->send_text(json);
|
||||
} catch (const WsError& e) {
|
||||
throw map_error(e);
|
||||
}
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
++stats_.sent;
|
||||
}
|
||||
|
||||
void LiveSession::send_text(const std::string& t) { send_raw(LiveRequests::text(t)); }
|
||||
void LiveSession::send_turn(const std::string& t, bool c) { send_raw(LiveRequests::turn(t, c)); }
|
||||
void LiveSession::send_audio(const std::string& pcm, const std::string& mime) {
|
||||
send_raw(LiveRequests::audio(pcm, mime));
|
||||
}
|
||||
void LiveSession::send_video(const std::string& b, const std::string& mime) {
|
||||
send_raw(LiveRequests::video(b, mime));
|
||||
}
|
||||
void LiveSession::end_audio() { send_raw(LiveRequests::audio_stream_end()); }
|
||||
void LiveSession::send_tool_responses(const std::vector<LiveToolResponse>& r) {
|
||||
send_raw(LiveRequests::tool_responses(r));
|
||||
}
|
||||
|
||||
void LiveSession::push(LiveEvent e) {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
e.seq = ++seq_;
|
||||
if (e.kind == LiveEventKind::SessionResumption && e.resumable && !e.handle.empty())
|
||||
handle_ = e.handle;
|
||||
queued_bytes_ += e.weight();
|
||||
queue_.push_back(std::move(e));
|
||||
// Hard ceilings: drop oldest, count loudly in stats().dropped.
|
||||
while (queue_.size() > 1 && (queue_.size() > LiveCaps::kMaxQueuedEvents ||
|
||||
queued_bytes_ > LiveCaps::kMaxQueuedBytes)) {
|
||||
queued_bytes_ -= queue_.front().weight();
|
||||
queue_.pop_front();
|
||||
++stats_.dropped;
|
||||
}
|
||||
cv_.notify_all();
|
||||
}
|
||||
|
||||
void LiveSession::ingest(const WsMessage& m) {
|
||||
{
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
++stats_.received;
|
||||
}
|
||||
for (auto& e : LiveParser::parse(m.data)) {
|
||||
if (e.kind == LiveEventKind::Error) {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
++stats_.decode_errors;
|
||||
}
|
||||
push(std::move(e));
|
||||
}
|
||||
}
|
||||
|
||||
void LiveSession::reader_loop() {
|
||||
const auto started = now_ms();
|
||||
LiveEvent closed;
|
||||
closed.kind = LiveEventKind::Closed;
|
||||
try {
|
||||
for (;;) {
|
||||
if (now_ms() - started > (std::int64_t)options_.max_session_seconds * 1000) {
|
||||
closed.text = "session lifetime cap reached";
|
||||
closed.code = 1000;
|
||||
ws_->close(1000, "lifetime cap");
|
||||
break;
|
||||
}
|
||||
std::int64_t dl = close_deadline_ms_.load();
|
||||
if (dl != 0 && now_ms() > dl) {
|
||||
closed.text = "closed by client";
|
||||
closed.code = 1000;
|
||||
break;
|
||||
}
|
||||
auto m = ws_->receive();
|
||||
if (!m) continue;
|
||||
if (m->kind == WsMessage::Kind::Close) {
|
||||
closed.code = m->code;
|
||||
closed.text = m->data;
|
||||
break;
|
||||
}
|
||||
// Gemini sends JSON in text and in binary frames.
|
||||
ingest(*m);
|
||||
}
|
||||
} catch (const WsError& e) {
|
||||
closed.kind = LiveEventKind::Error;
|
||||
closed.text = e.what();
|
||||
push(std::move(closed));
|
||||
closed = LiveEvent{};
|
||||
closed.kind = LiveEventKind::Closed;
|
||||
closed.code = 1006;
|
||||
closed.text = "transport error";
|
||||
}
|
||||
push(std::move(closed));
|
||||
open_ = false;
|
||||
cv_.notify_all();
|
||||
}
|
||||
|
||||
std::optional<LiveEvent> LiveSession::next_event(int timeout_ms) {
|
||||
std::unique_lock<std::mutex> lock(mu_);
|
||||
cv_.wait_for(lock, std::chrono::milliseconds(timeout_ms),
|
||||
[this] { return !queue_.empty() || !open_; });
|
||||
if (queue_.empty()) return std::nullopt;
|
||||
LiveEvent e = std::move(queue_.front());
|
||||
queue_.pop_front();
|
||||
queued_bytes_ -= e.weight();
|
||||
return e;
|
||||
}
|
||||
|
||||
std::size_t LiveSession::pending() const {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
return queue_.size();
|
||||
}
|
||||
|
||||
void LiveSession::close() {
|
||||
if (!ws_) return;
|
||||
if (open_.exchange(false) || reader_.joinable()) {
|
||||
close_deadline_ms_ = now_ms() + LiveCaps::kCloseGraceMs;
|
||||
ws_->close(1000, "client closing");
|
||||
}
|
||||
if (reader_.joinable()) reader_.join();
|
||||
ws_->shutdown();
|
||||
cv_.notify_all();
|
||||
}
|
||||
|
||||
LiveSession::Stats LiveSession::stats() const {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
return stats_;
|
||||
}
|
||||
|
||||
std::string LiveSession::resumption_handle() const {
|
||||
std::lock_guard<std::mutex> lock(mu_);
|
||||
return handle_;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,96 @@
|
||||
#pragma once
|
||||
|
||||
// One Live API conversation: connect, setup, send text/audio/video/tool
|
||||
// responses, receive typed events through a bounded queue, close cleanly.
|
||||
// A reader thread owns receiving; sending is safe from any thread.
|
||||
|
||||
#include <atomic>
|
||||
#include <condition_variable>
|
||||
#include <cstdint>
|
||||
#include <deque>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
#include <vector>
|
||||
|
||||
#include "google/live/caps.h"
|
||||
#include "google/live/live_config.h"
|
||||
#include "google/live/live_endpoint.h"
|
||||
#include "google/live/live_events.h"
|
||||
#include "google/live/live_requests.h"
|
||||
#include "google/live/ws_client.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct LiveOptions {
|
||||
int connect_timeout_ms = LiveCaps::kConnectTimeoutMs;
|
||||
int handshake_timeout_ms = LiveCaps::kHandshakeTimeoutMs;
|
||||
int setup_timeout_ms = LiveCaps::kSetupTimeoutMs;
|
||||
int max_session_seconds = LiveCaps::kMaxSessionSeconds;
|
||||
TlsFactory tls; // null: wolfSSL via MakeTlsSession
|
||||
};
|
||||
|
||||
class LiveSession {
|
||||
public:
|
||||
LiveSession(LiveEndpoint endpoint, LiveConfig config, LiveOptions options = {});
|
||||
~LiveSession();
|
||||
LiveSession(const LiveSession&) = delete;
|
||||
LiveSession& operator=(const LiveSession&) = delete;
|
||||
|
||||
// Validates, connects, upgrades, sends setup and waits for setupComplete,
|
||||
// then starts the reader. Throws GoogleError (NotConfiguredError without a
|
||||
// credential, INVALID_ARGUMENT before any traffic, UNAVAILABLE and friends
|
||||
// for upstream failures). Call once.
|
||||
void connect();
|
||||
|
||||
void send_text(const std::string& text);
|
||||
void send_turn(const std::string& text, bool turn_complete = true);
|
||||
void send_audio(const std::string& pcm,
|
||||
const std::string& mime = LiveRequests::kDefaultAudioMime);
|
||||
void send_video(const std::string& bytes,
|
||||
const std::string& mime = LiveRequests::kDefaultVideoMime);
|
||||
void end_audio();
|
||||
void send_tool_responses(const std::vector<LiveToolResponse>& responses);
|
||||
|
||||
// Waits up to timeout_ms for the next event; nullopt on timeout or when
|
||||
// the session ended and the queue is drained.
|
||||
std::optional<LiveEvent> next_event(int timeout_ms);
|
||||
std::size_t pending() const;
|
||||
bool open() const { return open_.load(); }
|
||||
// Idempotent. Sends a close frame, waits up to kCloseGraceMs, joins.
|
||||
void close();
|
||||
|
||||
struct Stats {
|
||||
std::uint64_t sent = 0, received = 0, dropped = 0, decode_errors = 0;
|
||||
};
|
||||
Stats stats() const;
|
||||
std::string resumption_handle() const;
|
||||
|
||||
private:
|
||||
void send_raw(const std::string& json);
|
||||
void push(LiveEvent e);
|
||||
void reader_loop();
|
||||
void ingest(const WsMessage& m);
|
||||
|
||||
LiveEndpoint endpoint_;
|
||||
LiveConfig config_;
|
||||
LiveOptions options_;
|
||||
std::unique_ptr<WsClient> ws_;
|
||||
std::thread reader_;
|
||||
std::atomic<bool> open_{false};
|
||||
std::atomic<bool> stop_{false};
|
||||
std::atomic<bool> connected_{false};
|
||||
std::atomic<std::int64_t> close_deadline_ms_{0};
|
||||
|
||||
mutable std::mutex mu_;
|
||||
std::condition_variable cv_;
|
||||
std::deque<LiveEvent> queue_;
|
||||
std::size_t queued_bytes_ = 0;
|
||||
std::uint64_t seq_ = 0;
|
||||
Stats stats_;
|
||||
std::string handle_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,117 @@
|
||||
#pragma once
|
||||
|
||||
// The single declaration of every wire name this module sends or reads.
|
||||
// tools/check_discovery.py parses the rows below (one per line) and fails
|
||||
// when a row with a discovery source names a schema or field that the
|
||||
// published Discovery document does not have. Rows with a "docs:" source
|
||||
// are not in any Discovery document (the Live API is a WebSocket service:
|
||||
// Discovery lists the setup message and its nested types only) and are
|
||||
// checked against the documentation page text instead.
|
||||
//
|
||||
// Row format: {"Schema", "field", "source"},
|
||||
// source = "<service>/<version>" Discovery document (schemas.<Schema>.properties)
|
||||
// = "docs:<url>" official documentation only
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct LiveSpecRow {
|
||||
const char* schema;
|
||||
const char* field;
|
||||
const char* source;
|
||||
};
|
||||
|
||||
#define LIVE_GL "generativelanguage/v1beta"
|
||||
#define LIVE_AI "aiplatform/v1"
|
||||
#define LIVE_DOC "docs:https://ai.google.dev/api/live"
|
||||
|
||||
inline constexpr LiveSpecRow kLiveSpec[] = {
|
||||
// Setup message (Discovery: generativelanguage v1beta).
|
||||
{"BidiGenerateContentSetup", "model", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "generationConfig", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "systemInstruction", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "tools", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "realtimeInputConfig", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "sessionResumption", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "contextWindowCompression", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "inputAudioTranscription", LIVE_GL},
|
||||
{"BidiGenerateContentSetup", "outputAudioTranscription", LIVE_GL},
|
||||
{"GenerationConfig", "responseModalities", LIVE_GL},
|
||||
{"GenerationConfig", "speechConfig", LIVE_GL},
|
||||
{"GenerationConfig", "temperature", LIVE_GL},
|
||||
{"GenerationConfig", "maxOutputTokens", LIVE_GL},
|
||||
{"SpeechConfig", "voiceConfig", LIVE_GL},
|
||||
{"SpeechConfig", "languageCode", LIVE_GL},
|
||||
{"VoiceConfig", "prebuiltVoiceConfig", LIVE_GL},
|
||||
{"PrebuiltVoiceConfig", "voiceName", LIVE_GL},
|
||||
{"Content", "parts", LIVE_GL},
|
||||
{"Content", "role", LIVE_GL},
|
||||
{"Part", "text", LIVE_GL},
|
||||
{"Part", "inlineData", LIVE_GL},
|
||||
{"Blob", "mimeType", LIVE_GL},
|
||||
{"Blob", "data", LIVE_GL},
|
||||
{"Tool", "functionDeclarations", LIVE_GL},
|
||||
{"FunctionDeclaration", "name", LIVE_GL},
|
||||
{"FunctionDeclaration", "description", LIVE_GL},
|
||||
{"FunctionDeclaration", "parameters", LIVE_GL},
|
||||
{"SessionResumptionConfig", "handle", LIVE_GL},
|
||||
{"ContextWindowCompressionConfig", "slidingWindow", LIVE_GL},
|
||||
{"FunctionCall", "id", LIVE_GL},
|
||||
{"FunctionCall", "name", LIVE_GL},
|
||||
{"FunctionCall", "args", LIVE_GL},
|
||||
{"FunctionResponse", "id", LIVE_GL},
|
||||
{"FunctionResponse", "name", LIVE_GL},
|
||||
{"FunctionResponse", "response", LIVE_GL},
|
||||
{"UsageMetadata", "totalTokenCount", LIVE_GL},
|
||||
// Vertex Discovery has the transcription value type (no Bidi messages).
|
||||
{"GoogleCloudAiplatformV1Transcription", "text", LIVE_AI},
|
||||
{"GoogleCloudAiplatformV1Transcription", "finished", LIVE_AI},
|
||||
{"GoogleCloudAiplatformV1Blob", "mimeType", LIVE_AI},
|
||||
{"GoogleCloudAiplatformV1Blob", "data", LIVE_AI},
|
||||
// Message envelopes: documentation only.
|
||||
{"ClientMessage", "setup", LIVE_DOC},
|
||||
{"ClientMessage", "clientContent", LIVE_DOC},
|
||||
{"ClientMessage", "realtimeInput", LIVE_DOC},
|
||||
{"ClientMessage", "toolResponse", LIVE_DOC},
|
||||
{"ClientContent", "turns", LIVE_DOC},
|
||||
{"ClientContent", "turnComplete", LIVE_DOC},
|
||||
{"RealtimeInput", "audio", LIVE_DOC},
|
||||
{"RealtimeInput", "video", LIVE_DOC},
|
||||
{"RealtimeInput", "text", LIVE_DOC},
|
||||
{"RealtimeInput", "audioStreamEnd", LIVE_DOC},
|
||||
{"ToolResponse", "functionResponses", LIVE_DOC},
|
||||
{"ServerMessage", "setupComplete", LIVE_DOC},
|
||||
{"ServerMessage", "serverContent", LIVE_DOC},
|
||||
{"ServerMessage", "toolCall", LIVE_DOC},
|
||||
{"ServerMessage", "toolCallCancellation", LIVE_DOC},
|
||||
{"ServerMessage", "goAway", LIVE_DOC},
|
||||
{"ServerMessage", "sessionResumptionUpdate", LIVE_DOC},
|
||||
{"ServerMessage", "usageMetadata", LIVE_DOC},
|
||||
{"ServerContent", "modelTurn", LIVE_DOC},
|
||||
{"ServerContent", "generationComplete", LIVE_DOC},
|
||||
{"ServerContent", "turnComplete", LIVE_DOC},
|
||||
{"ServerContent", "interrupted", LIVE_DOC},
|
||||
{"ServerContent", "inputTranscription", LIVE_DOC},
|
||||
{"ServerContent", "outputTranscription", LIVE_DOC},
|
||||
{"ToolCallMessage", "functionCalls", LIVE_DOC},
|
||||
{"ToolCallCancellation", "ids", LIVE_DOC},
|
||||
{"GoAway", "timeLeft", LIVE_DOC},
|
||||
{"SessionResumptionUpdate", "newHandle", LIVE_DOC},
|
||||
{"SessionResumptionUpdate", "resumable", LIVE_DOC},
|
||||
};
|
||||
|
||||
inline constexpr std::size_t kLiveSpecRows = sizeof(kLiveSpec) / sizeof(kLiveSpec[0]);
|
||||
|
||||
// Endpoint templates (documentation / third-party only, see SPEC Unverified).
|
||||
inline constexpr const char* kLiveApiKeyUrl =
|
||||
"wss://generativelanguage.googleapis.com/ws/"
|
||||
"google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent";
|
||||
inline constexpr const char* kLiveVertexPath =
|
||||
"/ws/google.cloud.aiplatform.v1.LlmBidiService/BidiGenerateContent";
|
||||
|
||||
#undef LIVE_GL
|
||||
#undef LIVE_AI
|
||||
#undef LIVE_DOC
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Checks src/google/live/live_spec.h against Google's published machine-readable specs.
|
||||
|
||||
python3 stdlib only, no credentials.
|
||||
|
||||
1. Every row whose source is "<service>/<version>" must name a schema that
|
||||
exists in that service's Discovery document and a field that is one of
|
||||
its properties.
|
||||
2. Every row whose source is "docs:<url>" must have its field name present
|
||||
in the text of that documentation page (weaker: the Live API envelopes
|
||||
are WebSocket messages and appear in no Discovery document).
|
||||
3. The WebSocket endpoints are probed with an unauthenticated upgrade
|
||||
request: the server must answer 101 Switching Protocols with the correct
|
||||
Sec-WebSocket-Accept (it closes the socket right after, for lack of
|
||||
credentials). Pass --no-probe to skip.
|
||||
|
||||
Exit 0 when everything holds, 1 otherwise. Prints one line per failure and a
|
||||
summary.
|
||||
"""
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
import ssl
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
SPEC = os.path.join(HERE, "..", "live_spec.h")
|
||||
DISCOVERY = {
|
||||
"generativelanguage": "https://generativelanguage.googleapis.com/$discovery/rest?version=%s",
|
||||
"aiplatform": "https://aiplatform.googleapis.com/$discovery/rest?version=%s",
|
||||
}
|
||||
PROBES = [
|
||||
("wss://us-central1-aiplatform.googleapis.com/ws/google.cloud.aiplatform.v1.LlmBidiService/BidiGenerateContent"),
|
||||
("wss://us-central1-aiplatform.googleapis.com/ws/google.cloud.aiplatform.v1beta1.LlmBidiService/BidiGenerateContent"),
|
||||
("wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent"),
|
||||
]
|
||||
|
||||
|
||||
def fetch(url):
|
||||
with urllib.request.urlopen(url, timeout=60) as r:
|
||||
return r.read().decode("utf-8", "replace")
|
||||
|
||||
|
||||
def parse_spec(path):
|
||||
text = open(path, encoding="utf-8").read()
|
||||
text = "\n".join(l for l in text.splitlines() if not l.lstrip().startswith("//"))
|
||||
macros = dict(re.findall(r'#define\s+(LIVE_\w+)\s+"([^"]*)"', text))
|
||||
rows = []
|
||||
for m in re.finditer(r'\{"(\w+)",\s*"(\w+)",\s*(LIVE_\w+|"[^"]*")\}', text):
|
||||
schema, field, src = m.groups()
|
||||
src = macros[src] if src in macros else src.strip('"')
|
||||
rows.append((schema, field, src))
|
||||
return rows
|
||||
|
||||
|
||||
def probe(url):
|
||||
host = url.split("/")[2]
|
||||
path = url[len("wss://") + len(host):]
|
||||
key = base64.b64encode(os.urandom(16)).decode()
|
||||
ctx = ssl.create_default_context()
|
||||
ctx.set_alpn_protocols(["http/1.1"])
|
||||
with socket.create_connection((host, 443), timeout=20) as raw:
|
||||
with ctx.wrap_socket(raw, server_hostname=host) as s:
|
||||
s.settimeout(20)
|
||||
s.sendall(("GET %s HTTP/1.1\r\nHost: %s\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n"
|
||||
"Sec-WebSocket-Key: %s\r\nSec-WebSocket-Version: 13\r\n\r\n" % (path, host, key)).encode())
|
||||
buf = b""
|
||||
while b"\r\n\r\n" not in buf and len(buf) < 16384:
|
||||
chunk = s.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
head = buf.split(b"\r\n\r\n")[0].decode("latin-1")
|
||||
want = base64.b64encode(hashlib.sha1((key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").encode()).digest()).decode()
|
||||
ok = head.startswith("HTTP/1.1 101") and want in head
|
||||
return ok, head.split("\r\n")[0]
|
||||
|
||||
|
||||
def main():
|
||||
rows = parse_spec(SPEC)
|
||||
failures = []
|
||||
docs = {}
|
||||
disc = {}
|
||||
checked = 0
|
||||
for schema, field, src in rows:
|
||||
if src.startswith("docs:"):
|
||||
url = src[5:]
|
||||
if url not in docs:
|
||||
docs[url] = fetch(url)
|
||||
if field not in docs[url]:
|
||||
failures.append("docs: %s.%s not found on %s" % (schema, field, url))
|
||||
checked += 1
|
||||
continue
|
||||
svc, ver = src.split("/")
|
||||
if src not in disc:
|
||||
disc[src] = json.loads(fetch(DISCOVERY[svc] % ver))["schemas"]
|
||||
s = disc[src]
|
||||
if schema not in s:
|
||||
failures.append("discovery %s: schema %s missing" % (src, schema))
|
||||
elif field not in s[schema].get("properties", {}):
|
||||
failures.append("discovery %s: %s.%s missing" % (src, schema, field))
|
||||
checked += 1
|
||||
for url in ([] if "--no-probe" in sys.argv else PROBES):
|
||||
try:
|
||||
ok, line = probe(url)
|
||||
except Exception as e: # network trouble is a failure, loudly
|
||||
ok, line = False, "error: %s" % e
|
||||
print("%s upgrade probe %s -> %s" % ("ok " if ok else "FAIL", url.split("/ws/")[1], line))
|
||||
if not ok:
|
||||
failures.append("probe failed: " + url)
|
||||
for f in failures:
|
||||
print("FAIL", f)
|
||||
print("%d spec rows checked (%d discovery docs, %d doc pages), %d failures"
|
||||
% (checked, len(disc), len(docs), len(failures)))
|
||||
return 1 if failures else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,185 @@
|
||||
#include "google/live/ws_client.h"
|
||||
|
||||
#include <sys/random.h>
|
||||
|
||||
#include <chrono>
|
||||
|
||||
#include "google/live/caps.h"
|
||||
#include "google/live/ws_error.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using K = WsError::Kind;
|
||||
|
||||
WsClient::WsClient(std::unique_ptr<IByteStream> stream)
|
||||
: stream_(std::move(stream)), decoder_(LiveCaps::kMaxFramePayload, false) {}
|
||||
|
||||
void WsClient::handshake(const WsUrl& url, const WsHeaders& headers,
|
||||
int timeout_ms) {
|
||||
std::string key = WsHandshake::make_key();
|
||||
std::string req = WsHandshake::build_request(url, key, headers);
|
||||
stream_->write_all(req.data(), req.size());
|
||||
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
|
||||
stream_->set_read_timeout_ms(1000);
|
||||
std::string head;
|
||||
char buf[LiveCaps::kReadChunk];
|
||||
std::size_t end;
|
||||
while ((end = head.find("\r\n\r\n")) == std::string::npos) {
|
||||
if (head.size() > LiveCaps::kMaxHandshakeBytes)
|
||||
throw WsError(K::Handshake, "handshake response over cap");
|
||||
if (std::chrono::steady_clock::now() > deadline)
|
||||
throw WsError(K::Timeout, "handshake timed out");
|
||||
ReadResult r = stream_->read(buf, sizeof buf);
|
||||
if (r.status == ReadStatus::Eof)
|
||||
throw WsError(K::Handshake, "peer closed during handshake");
|
||||
if (r.status == ReadStatus::Data) head.append(buf, r.n);
|
||||
}
|
||||
std::string rest = head.substr(end + 4);
|
||||
head.resize(end + 4);
|
||||
WsHandshake::check_response(head, key);
|
||||
if (!rest.empty()) decoder_.feed(rest.data(), rest.size());
|
||||
}
|
||||
|
||||
void WsClient::send_frame(WsOpcode op, const std::string& payload) {
|
||||
std::uint8_t mask[4];
|
||||
if (::getrandom(mask, sizeof mask, 0) != (ssize_t)sizeof mask)
|
||||
throw WsError(K::Io, "no entropy for frame mask");
|
||||
std::string wire = encode_frame(true, op, payload, mask);
|
||||
std::lock_guard<std::mutex> lock(write_mu_);
|
||||
stream_->write_all(wire.data(), wire.size());
|
||||
std::lock_guard<std::mutex> s(stats_mu_);
|
||||
++stats_.frames_out;
|
||||
}
|
||||
|
||||
void WsClient::send_text(const std::string& text) {
|
||||
if (closed_ || close_sent_) throw WsError(K::Closed, "connection closed");
|
||||
if (text.size() > LiveCaps::kMaxOutboundMessage)
|
||||
throw WsError(K::TooBig, "outbound message over cap");
|
||||
send_frame(WsOpcode::Text, text);
|
||||
}
|
||||
|
||||
void WsClient::send_binary(const std::string& data) {
|
||||
if (closed_ || close_sent_) throw WsError(K::Closed, "connection closed");
|
||||
if (data.size() > LiveCaps::kMaxOutboundMessage)
|
||||
throw WsError(K::TooBig, "outbound message over cap");
|
||||
send_frame(WsOpcode::Binary, data);
|
||||
}
|
||||
|
||||
void WsClient::send_ping(const std::string& payload) {
|
||||
if (payload.size() > LiveCaps::kMaxControlPayload)
|
||||
throw WsError(K::Invalid, "ping payload over 125 bytes");
|
||||
if (closed_ || close_sent_) throw WsError(K::Closed, "connection closed");
|
||||
send_frame(WsOpcode::Ping, payload);
|
||||
}
|
||||
|
||||
void WsClient::close(std::uint16_t code, const std::string& reason) {
|
||||
if (close_sent_.exchange(true)) return;
|
||||
std::string payload;
|
||||
payload.push_back((char)(code >> 8));
|
||||
payload.push_back((char)(code & 0xFF));
|
||||
payload += reason.substr(0, LiveCaps::kMaxCloseReason);
|
||||
try {
|
||||
send_frame(WsOpcode::Close, payload);
|
||||
} catch (const WsError&) {
|
||||
}
|
||||
}
|
||||
|
||||
void WsClient::fail(std::uint16_t code, WsError::Kind kind, const std::string& why) {
|
||||
close(code, "");
|
||||
closed_ = true;
|
||||
throw WsError(kind, why);
|
||||
}
|
||||
|
||||
std::optional<WsMessage> WsClient::receive() {
|
||||
char buf[LiveCaps::kReadChunk];
|
||||
for (;;) {
|
||||
std::optional<WsFrame> frame;
|
||||
try {
|
||||
frame = decoder_.next();
|
||||
} catch (const WsError& e) {
|
||||
fail(e.kind() == K::TooBig ? 1009 : 1002, e.kind(), e.what());
|
||||
}
|
||||
if (!frame) {
|
||||
if (closed_) {
|
||||
WsMessage m;
|
||||
m.kind = WsMessage::Kind::Close;
|
||||
m.code = 1006;
|
||||
return m;
|
||||
}
|
||||
ReadResult r = stream_->read(buf, sizeof buf);
|
||||
if (r.status == ReadStatus::Timeout) return std::nullopt;
|
||||
if (r.status == ReadStatus::Eof) {
|
||||
closed_ = true;
|
||||
WsMessage m;
|
||||
m.kind = WsMessage::Kind::Close;
|
||||
m.code = 1006;
|
||||
m.data = "connection lost without close frame";
|
||||
return m;
|
||||
}
|
||||
decoder_.feed(buf, r.n);
|
||||
continue;
|
||||
}
|
||||
{
|
||||
std::lock_guard<std::mutex> s(stats_mu_);
|
||||
++stats_.frames_in;
|
||||
}
|
||||
auto op = (WsOpcode)frame->opcode;
|
||||
if (op == WsOpcode::Ping) {
|
||||
{
|
||||
std::lock_guard<std::mutex> s(stats_mu_);
|
||||
++stats_.pings_in;
|
||||
++stats_.pongs_out;
|
||||
}
|
||||
if (!close_sent_) send_frame(WsOpcode::Pong, frame->payload);
|
||||
continue;
|
||||
}
|
||||
if (op == WsOpcode::Pong) continue;
|
||||
if (op == WsOpcode::Close) {
|
||||
WsMessage m;
|
||||
m.kind = WsMessage::Kind::Close;
|
||||
m.code = 1005;
|
||||
const std::string& p = frame->payload;
|
||||
if (p.size() == 1) fail(1002, K::Protocol, "close payload of 1 byte");
|
||||
if (p.size() >= 2) {
|
||||
m.code = (std::uint16_t)(((std::uint8_t)p[0] << 8) | (std::uint8_t)p[1]);
|
||||
m.data = p.substr(2);
|
||||
if (!valid_utf8(m.data)) fail(1007, K::Protocol, "close reason is not UTF-8");
|
||||
}
|
||||
close(m.code == 1005 ? 1000 : m.code, ""); // echo
|
||||
closed_ = true;
|
||||
return m;
|
||||
}
|
||||
// Data frames.
|
||||
if (op == WsOpcode::Continuation) {
|
||||
if (!assembling_) fail(1002, K::Protocol, "continuation without a message");
|
||||
} else {
|
||||
if (assembling_) fail(1002, K::Protocol, "new message inside a fragmented one");
|
||||
assembling_ = true;
|
||||
assembling_text_ = op == WsOpcode::Text;
|
||||
assembly_.clear();
|
||||
}
|
||||
if (assembly_.size() + frame->payload.size() > LiveCaps::kMaxInboundMessage)
|
||||
fail(1009, K::TooBig, "message over cap");
|
||||
assembly_ += frame->payload;
|
||||
if (!frame->fin) continue;
|
||||
assembling_ = false;
|
||||
WsMessage m;
|
||||
m.kind = assembling_text_ ? WsMessage::Kind::Text : WsMessage::Kind::Binary;
|
||||
if (assembling_text_ && !valid_utf8(assembly_))
|
||||
fail(1007, K::Protocol, "text message is not UTF-8");
|
||||
m.data = std::move(assembly_);
|
||||
assembly_.clear();
|
||||
{
|
||||
std::lock_guard<std::mutex> s(stats_mu_);
|
||||
++stats_.messages_in;
|
||||
}
|
||||
return m;
|
||||
}
|
||||
}
|
||||
|
||||
WsClient::Stats WsClient::stats() const {
|
||||
std::lock_guard<std::mutex> s(stats_mu_);
|
||||
return stats_;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,74 @@
|
||||
#pragma once
|
||||
|
||||
// RFC 6455 client over an IByteStream. Masks every outbound frame,
|
||||
// reassembles fragmented messages (control frames may interleave), answers
|
||||
// pings, performs the closing handshake, and enforces frame and message
|
||||
// caps. One thread calls receive(); any thread may send (writes are
|
||||
// serialised).
|
||||
|
||||
#include <atomic>
|
||||
#include <cstdint>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
#include "google/live/ws_error.h"
|
||||
#include "google/live/ws_frame.h"
|
||||
#include "google/live/ws_handshake.h"
|
||||
#include "google/live/ws_stream.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct WsMessage {
|
||||
enum class Kind { Text, Binary, Close };
|
||||
Kind kind = Kind::Text;
|
||||
std::string data; // payload, or close reason
|
||||
std::uint16_t code = 0; // close code (1005 when none, 1006 abnormal)
|
||||
};
|
||||
|
||||
class WsClient {
|
||||
public:
|
||||
explicit WsClient(std::unique_ptr<IByteStream> stream);
|
||||
|
||||
// Runs the opening handshake on the stream. Throws WsError.
|
||||
void handshake(const WsUrl& url, const WsHeaders& headers, int timeout_ms);
|
||||
|
||||
void send_text(const std::string& text);
|
||||
void send_binary(const std::string& data);
|
||||
void send_ping(const std::string& payload = "");
|
||||
// Sends a close frame once; later calls are no-ops.
|
||||
void close(std::uint16_t code = 1000, const std::string& reason = "");
|
||||
|
||||
// Next complete data message or Close; nullopt when the read timed out.
|
||||
// Throws WsError on a protocol violation (after a best-effort 1002/1007/
|
||||
// 1009 close).
|
||||
std::optional<WsMessage> receive();
|
||||
|
||||
void set_read_timeout_ms(int ms) { stream_->set_read_timeout_ms(ms); }
|
||||
void shutdown() { stream_->shutdown(); }
|
||||
bool closed() const { return closed_.load(); }
|
||||
|
||||
struct Stats {
|
||||
std::uint64_t frames_in = 0, frames_out = 0, pings_in = 0, pongs_out = 0;
|
||||
std::uint64_t messages_in = 0;
|
||||
};
|
||||
Stats stats() const;
|
||||
|
||||
private:
|
||||
void send_frame(WsOpcode op, const std::string& payload);
|
||||
void fail(std::uint16_t code, WsError::Kind kind, const std::string& why);
|
||||
|
||||
std::unique_ptr<IByteStream> stream_;
|
||||
FrameDecoder decoder_;
|
||||
std::mutex write_mu_;
|
||||
std::atomic<bool> closed_{false};
|
||||
std::atomic<bool> close_sent_{false};
|
||||
bool assembling_ = false;
|
||||
bool assembling_text_ = false;
|
||||
std::string assembly_;
|
||||
mutable std::mutex stats_mu_;
|
||||
Stats stats_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,29 @@
|
||||
#pragma once
|
||||
|
||||
// Transport-level WebSocket failure. Messages never carry URLs with query
|
||||
// strings or credentials.
|
||||
|
||||
#include <stdexcept>
|
||||
#include <string>
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
class WsError : public std::runtime_error {
|
||||
public:
|
||||
enum class Kind { Invalid, Io, Timeout, Handshake, Protocol, TooBig, Closed };
|
||||
|
||||
WsError(Kind kind, const std::string& message, int http_status = 0)
|
||||
: std::runtime_error("molo::google::live::ws: " + message),
|
||||
kind_(kind),
|
||||
http_status_(http_status) {}
|
||||
|
||||
Kind kind() const noexcept { return kind_; }
|
||||
// HTTP status of a refused upgrade, else 0.
|
||||
int http_status() const noexcept { return http_status_; }
|
||||
|
||||
private:
|
||||
Kind kind_;
|
||||
int http_status_;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,132 @@
|
||||
#include "google/live/ws_frame.h"
|
||||
|
||||
#include "google/live/caps.h"
|
||||
#include "google/live/ws_error.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using K = WsError::Kind;
|
||||
|
||||
std::string encode_frame(bool fin, WsOpcode opcode, const std::string& payload,
|
||||
const std::uint8_t* mask_key) {
|
||||
std::string out;
|
||||
out.push_back((char)((fin ? 0x80 : 0x00) | (std::uint8_t)opcode));
|
||||
std::uint8_t mask_bit = mask_key ? 0x80 : 0x00;
|
||||
std::size_t n = payload.size();
|
||||
if (n < 126) {
|
||||
out.push_back((char)(mask_bit | n));
|
||||
} else if (n <= 0xFFFF) {
|
||||
out.push_back((char)(mask_bit | 126));
|
||||
out.push_back((char)(n >> 8));
|
||||
out.push_back((char)(n & 0xFF));
|
||||
} else {
|
||||
out.push_back((char)(mask_bit | 127));
|
||||
for (int shift = 56; shift >= 0; shift -= 8)
|
||||
out.push_back((char)(((std::uint64_t)n >> shift) & 0xFF));
|
||||
}
|
||||
if (mask_key) {
|
||||
out.append(reinterpret_cast<const char*>(mask_key), 4);
|
||||
for (std::size_t i = 0; i < n; ++i)
|
||||
out.push_back((char)((std::uint8_t)payload[i] ^ mask_key[i & 3]));
|
||||
} else {
|
||||
out += payload;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
bool valid_utf8(const std::string& s) {
|
||||
std::size_t i = 0, n = s.size();
|
||||
while (i < n) {
|
||||
std::uint8_t c = (std::uint8_t)s[i];
|
||||
std::size_t len;
|
||||
std::uint32_t cp;
|
||||
if (c < 0x80) {
|
||||
++i;
|
||||
continue;
|
||||
} else if (c >= 0xC2 && c <= 0xDF) {
|
||||
len = 2;
|
||||
cp = c & 0x1F;
|
||||
} else if (c >= 0xE0 && c <= 0xEF) {
|
||||
len = 3;
|
||||
cp = c & 0x0F;
|
||||
} else if (c >= 0xF0 && c <= 0xF4) {
|
||||
len = 4;
|
||||
cp = c & 0x07;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
if (i + len > n) return false;
|
||||
for (std::size_t k = 1; k < len; ++k) {
|
||||
std::uint8_t cc = (std::uint8_t)s[i + k];
|
||||
if ((cc & 0xC0) != 0x80) return false;
|
||||
cp = (cp << 6) | (cc & 0x3F);
|
||||
}
|
||||
if (len == 3 && (cp < 0x800 || (cp >= 0xD800 && cp <= 0xDFFF))) return false;
|
||||
if (len == 4 && (cp < 0x10000 || cp > 0x10FFFF)) return false;
|
||||
i += len;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
void FrameDecoder::feed(const char* data, std::size_t size) {
|
||||
if (pos_ > 0 && pos_ == buf_.size()) {
|
||||
buf_.clear();
|
||||
pos_ = 0;
|
||||
} else if (pos_ > LiveCaps::kReadChunk) {
|
||||
buf_.erase(0, pos_);
|
||||
pos_ = 0;
|
||||
}
|
||||
buf_.append(data, size);
|
||||
}
|
||||
|
||||
std::optional<WsFrame> FrameDecoder::next() {
|
||||
const std::size_t avail = buf_.size() - pos_;
|
||||
if (avail < 2) return std::nullopt;
|
||||
const auto* p = reinterpret_cast<const std::uint8_t*>(buf_.data()) + pos_;
|
||||
bool fin = p[0] & 0x80;
|
||||
if (p[0] & 0x70) throw WsError(K::Protocol, "reserved bits set");
|
||||
std::uint8_t op = p[0] & 0x0F;
|
||||
bool known = op == 0 || op == 1 || op == 2 || (op >= 8 && op <= 10);
|
||||
if (!known) throw WsError(K::Protocol, "unknown opcode");
|
||||
bool masked = p[1] & 0x80;
|
||||
if (masked != expect_masked_)
|
||||
throw WsError(K::Protocol, expect_masked_ ? "unmasked client frame"
|
||||
: "masked server frame");
|
||||
std::uint64_t len = p[1] & 0x7F;
|
||||
std::size_t header = 2;
|
||||
if (len == 126) {
|
||||
if (avail < 4) return std::nullopt;
|
||||
len = ((std::uint64_t)p[2] << 8) | p[3];
|
||||
header = 4;
|
||||
if (len < 126) throw WsError(K::Protocol, "non-minimal length");
|
||||
} else if (len == 127) {
|
||||
if (avail < 10) return std::nullopt;
|
||||
len = 0;
|
||||
for (int i = 0; i < 8; ++i) len = (len << 8) | p[2 + i];
|
||||
header = 10;
|
||||
if (len >> 63) throw WsError(K::Protocol, "length high bit set");
|
||||
if (len <= 0xFFFF) throw WsError(K::Protocol, "non-minimal length");
|
||||
}
|
||||
bool control = op >= 8;
|
||||
if (control) {
|
||||
if (!fin) throw WsError(K::Protocol, "fragmented control frame");
|
||||
if (len > LiveCaps::kMaxControlPayload)
|
||||
throw WsError(K::Protocol, "control payload over 125 bytes");
|
||||
}
|
||||
if (len > max_payload_) throw WsError(K::TooBig, "frame payload over cap");
|
||||
std::size_t mask_len = masked ? 4 : 0;
|
||||
if (avail < header + mask_len + len) return std::nullopt;
|
||||
WsFrame f;
|
||||
f.fin = fin;
|
||||
f.opcode = op;
|
||||
const std::uint8_t* key = p + header;
|
||||
const std::uint8_t* body = p + header + mask_len;
|
||||
f.payload.assign(reinterpret_cast<const char*>(body), (std::size_t)len);
|
||||
if (masked)
|
||||
for (std::size_t i = 0; i < f.payload.size(); ++i)
|
||||
f.payload[i] = (char)((std::uint8_t)f.payload[i] ^ key[i & 3]);
|
||||
pos_ += header + mask_len + (std::size_t)len;
|
||||
return f;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,54 @@
|
||||
#pragma once
|
||||
|
||||
// RFC 6455 framing: encoder (optionally masked) and an incremental decoder
|
||||
// that enforces the protocol rules and the payload cap before buffering.
|
||||
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
enum class WsOpcode : std::uint8_t {
|
||||
Continuation = 0x0,
|
||||
Text = 0x1,
|
||||
Binary = 0x2,
|
||||
Close = 0x8,
|
||||
Ping = 0x9,
|
||||
Pong = 0xA,
|
||||
};
|
||||
|
||||
struct WsFrame {
|
||||
bool fin = true;
|
||||
std::uint8_t opcode = 0;
|
||||
std::string payload; // unmasked
|
||||
};
|
||||
|
||||
// mask_key null: unmasked (server direction). Non-null: 4 bytes, masked
|
||||
// (client direction, mandatory per RFC 6455 5.3).
|
||||
std::string encode_frame(bool fin, WsOpcode opcode, const std::string& payload,
|
||||
const std::uint8_t* mask_key);
|
||||
|
||||
// Strict UTF-8 (RFC 3629): no overlongs, no surrogates, max U+10FFFF.
|
||||
bool valid_utf8(const std::string& s);
|
||||
|
||||
class FrameDecoder {
|
||||
public:
|
||||
// expect_masked false: frames come from a server and must be unmasked.
|
||||
// true: frames come from a client and must be masked (test servers).
|
||||
explicit FrameDecoder(std::size_t max_payload, bool expect_masked = false)
|
||||
: max_payload_(max_payload), expect_masked_(expect_masked) {}
|
||||
|
||||
void feed(const char* data, std::size_t size);
|
||||
// Next complete frame or nullopt. Throws WsError(Protocol or TooBig).
|
||||
std::optional<WsFrame> next();
|
||||
std::size_t buffered() const { return buf_.size() - pos_; }
|
||||
|
||||
private:
|
||||
std::size_t max_payload_;
|
||||
bool expect_masked_;
|
||||
std::string buf_;
|
||||
std::size_t pos_ = 0;
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,155 @@
|
||||
#include "google/live/ws_handshake.h"
|
||||
|
||||
#include <sys/random.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <cctype>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
|
||||
#include "google/live/ws_error.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using K = WsError::Kind;
|
||||
using molo::security::crypto::Base64;
|
||||
|
||||
namespace {
|
||||
|
||||
std::array<std::uint8_t, 20> sha1(const std::string& msg) {
|
||||
std::uint32_t h[5] = {0x67452301, 0xEFCDAB89, 0x98BADCFE, 0x10325476, 0xC3D2E1F0};
|
||||
std::string data = msg;
|
||||
data.push_back((char)0x80);
|
||||
while (data.size() % 64 != 56) data.push_back(0);
|
||||
std::uint64_t bits = (std::uint64_t)msg.size() * 8;
|
||||
for (int i = 7; i >= 0; --i) data.push_back((char)((bits >> (i * 8)) & 0xFF));
|
||||
auto rol = [](std::uint32_t v, int s) { return (v << s) | (v >> (32 - s)); };
|
||||
for (std::size_t off = 0; off < data.size(); off += 64) {
|
||||
std::uint32_t w[80];
|
||||
for (int i = 0; i < 16; ++i)
|
||||
w[i] = ((std::uint32_t)(std::uint8_t)data[off + i * 4] << 24) |
|
||||
((std::uint32_t)(std::uint8_t)data[off + i * 4 + 1] << 16) |
|
||||
((std::uint32_t)(std::uint8_t)data[off + i * 4 + 2] << 8) |
|
||||
(std::uint32_t)(std::uint8_t)data[off + i * 4 + 3];
|
||||
for (int i = 16; i < 80; ++i)
|
||||
w[i] = rol(w[i - 3] ^ w[i - 8] ^ w[i - 14] ^ w[i - 16], 1);
|
||||
std::uint32_t a = h[0], b = h[1], c = h[2], d = h[3], e = h[4];
|
||||
for (int i = 0; i < 80; ++i) {
|
||||
std::uint32_t f, k;
|
||||
if (i < 20) {
|
||||
f = (b & c) | (~b & d);
|
||||
k = 0x5A827999;
|
||||
} else if (i < 40) {
|
||||
f = b ^ c ^ d;
|
||||
k = 0x6ED9EBA1;
|
||||
} else if (i < 60) {
|
||||
f = (b & c) | (b & d) | (c & d);
|
||||
k = 0x8F1BBCDC;
|
||||
} else {
|
||||
f = b ^ c ^ d;
|
||||
k = 0xCA62C1D6;
|
||||
}
|
||||
std::uint32_t t = rol(a, 5) + f + e + k + w[i];
|
||||
e = d;
|
||||
d = c;
|
||||
c = rol(b, 30);
|
||||
b = a;
|
||||
a = t;
|
||||
}
|
||||
h[0] += a;
|
||||
h[1] += b;
|
||||
h[2] += c;
|
||||
h[3] += d;
|
||||
h[4] += e;
|
||||
}
|
||||
std::array<std::uint8_t, 20> out{};
|
||||
for (int i = 0; i < 5; ++i)
|
||||
for (int j = 0; j < 4; ++j) out[i * 4 + j] = (std::uint8_t)(h[i] >> (24 - j * 8));
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string lower(std::string s) {
|
||||
for (auto& c : s) c = (char)std::tolower((unsigned char)c);
|
||||
return s;
|
||||
}
|
||||
|
||||
std::string trim(const std::string& s) {
|
||||
std::size_t a = s.find_first_not_of(" \t");
|
||||
if (a == std::string::npos) return "";
|
||||
std::size_t b = s.find_last_not_of(" \t\r");
|
||||
return s.substr(a, b - a + 1);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string WsHandshake::make_key() {
|
||||
std::uint8_t raw[16];
|
||||
std::size_t got = 0;
|
||||
while (got < sizeof raw) {
|
||||
ssize_t n = ::getrandom(raw + got, sizeof raw - got, 0);
|
||||
if (n <= 0) throw WsError(K::Io, "no entropy for handshake key");
|
||||
got += (std::size_t)n;
|
||||
}
|
||||
return Base64::encode(raw, sizeof raw);
|
||||
}
|
||||
|
||||
std::string WsHandshake::accept_for(const std::string& key) {
|
||||
auto d = sha1(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11");
|
||||
return Base64::encode(d.data(), d.size());
|
||||
}
|
||||
|
||||
std::string WsHandshake::build_request(const WsUrl& url, const std::string& key,
|
||||
const WsHeaders& headers) {
|
||||
std::string host = url.host;
|
||||
if (url.port != (url.secure ? 443 : 80)) host += ":" + std::to_string(url.port);
|
||||
std::string out = "GET " + url.target + " HTTP/1.1\r\nHost: " + host +
|
||||
"\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n"
|
||||
"Sec-WebSocket-Key: " + key + "\r\nSec-WebSocket-Version: 13\r\n";
|
||||
for (const auto& [name, value] : headers) {
|
||||
if (name.empty() || name.find_first_of("\r\n:") != std::string::npos ||
|
||||
value.find_first_of("\r\n") != std::string::npos)
|
||||
throw WsError(K::Invalid, "header with forbidden characters");
|
||||
out += name + ": " + value + "\r\n";
|
||||
}
|
||||
return out + "\r\n";
|
||||
}
|
||||
|
||||
void WsHandshake::check_response(const std::string& head, const std::string& key) {
|
||||
std::size_t eol = head.find("\r\n");
|
||||
if (eol == std::string::npos) throw WsError(K::Handshake, "malformed response");
|
||||
std::string status_line = head.substr(0, eol);
|
||||
int status = 0;
|
||||
if (status_line.rfind("HTTP/1.1 ", 0) != 0 || status_line.size() < 12)
|
||||
throw WsError(K::Handshake, "malformed status line");
|
||||
for (int i = 9; i < 12; ++i) {
|
||||
if (!std::isdigit((unsigned char)status_line[i]))
|
||||
throw WsError(K::Handshake, "malformed status line");
|
||||
status = status * 10 + (status_line[i] - '0');
|
||||
}
|
||||
if (status != 101)
|
||||
throw WsError(K::Handshake, "upgrade refused with HTTP " + std::to_string(status), status);
|
||||
std::string upgrade, connection, accept;
|
||||
std::size_t pos = eol + 2;
|
||||
while (pos < head.size()) {
|
||||
std::size_t e = head.find("\r\n", pos);
|
||||
if (e == std::string::npos || e == pos) break;
|
||||
std::string line = head.substr(pos, e - pos);
|
||||
pos = e + 2;
|
||||
std::size_t c = line.find(':');
|
||||
if (c == std::string::npos) continue;
|
||||
std::string name = lower(trim(line.substr(0, c)));
|
||||
std::string value = trim(line.substr(c + 1));
|
||||
if (name == "upgrade") upgrade = lower(value);
|
||||
else if (name == "connection") connection = lower(value);
|
||||
else if (name == "sec-websocket-accept") accept = value;
|
||||
}
|
||||
if (upgrade != "websocket") throw WsError(K::Handshake, "missing Upgrade: websocket", 101);
|
||||
if (connection.find("upgrade") == std::string::npos)
|
||||
throw WsError(K::Handshake, "missing Connection: Upgrade", 101);
|
||||
if (accept != accept_for(key))
|
||||
throw WsError(K::Handshake, "Sec-WebSocket-Accept mismatch", 101);
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,30 @@
|
||||
#pragma once
|
||||
|
||||
// Opening handshake (RFC 6455 4.1): request building and strict validation
|
||||
// of the 101 answer, including Sec-WebSocket-Accept.
|
||||
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include "google/live/ws_url.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using WsHeaders = std::vector<std::pair<std::string, std::string>>;
|
||||
|
||||
class WsHandshake {
|
||||
public:
|
||||
// 16 random bytes, base64.
|
||||
static std::string make_key();
|
||||
// base64(sha1(key + GUID)).
|
||||
static std::string accept_for(const std::string& key);
|
||||
// Headers must not contain CR or LF; throws WsError(Invalid).
|
||||
static std::string build_request(const WsUrl& url, const std::string& key,
|
||||
const WsHeaders& headers);
|
||||
// head = response bytes up to and including the blank line. Throws
|
||||
// WsError(Handshake) carrying the HTTP status when not an accepted 101.
|
||||
static void check_response(const std::string& head, const std::string& key);
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,191 @@
|
||||
#include "google/live/ws_stream.h"
|
||||
|
||||
#include <fcntl.h>
|
||||
#include <netdb.h>
|
||||
#include <poll.h>
|
||||
#include <sys/socket.h>
|
||||
#include <sys/time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <cerrno>
|
||||
#include <chrono>
|
||||
#include <csignal>
|
||||
#include <cstring>
|
||||
#include <mutex>
|
||||
|
||||
#include "clients/transports/tls_wolfssl.h"
|
||||
#include "google/live/ws_error.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
using K = WsError::Kind;
|
||||
|
||||
namespace {
|
||||
void set_timeout(int fd, int opt, int ms) {
|
||||
timeval tv{};
|
||||
tv.tv_sec = ms / 1000;
|
||||
tv.tv_usec = (ms % 1000) * 1000;
|
||||
::setsockopt(fd, SOL_SOCKET, opt, &tv, sizeof tv);
|
||||
}
|
||||
} // namespace
|
||||
|
||||
TcpStream::TcpStream(int fd, int io_timeout_ms) : fd_(fd) {
|
||||
set_timeout(fd_, SO_RCVTIMEO, io_timeout_ms);
|
||||
set_timeout(fd_, SO_SNDTIMEO, io_timeout_ms);
|
||||
}
|
||||
|
||||
TcpStream::~TcpStream() {
|
||||
if (fd_ >= 0) ::close(fd_);
|
||||
}
|
||||
|
||||
void TcpStream::set_read_timeout_ms(int ms) {
|
||||
set_timeout(fd_, SO_RCVTIMEO, ms);
|
||||
}
|
||||
|
||||
ReadResult TcpStream::read(char* out, std::size_t max) {
|
||||
for (;;) {
|
||||
ssize_t n = ::recv(fd_, out, max, 0);
|
||||
if (n > 0) return {ReadStatus::Data, (std::size_t)n};
|
||||
if (n == 0) return {ReadStatus::Eof, 0};
|
||||
if (errno == EINTR) continue;
|
||||
if (errno == EAGAIN || errno == EWOULDBLOCK)
|
||||
return {ReadStatus::Timeout, 0};
|
||||
if (errno == ECONNRESET) return {ReadStatus::Eof, 0};
|
||||
throw WsError(K::Io, std::string("recv failed: ") + std::strerror(errno));
|
||||
}
|
||||
}
|
||||
|
||||
void TcpStream::write_all(const char* data, std::size_t size) {
|
||||
std::size_t sent = 0;
|
||||
while (sent < size) {
|
||||
ssize_t n = ::send(fd_, data + sent, size - sent, MSG_NOSIGNAL);
|
||||
if (n > 0) {
|
||||
sent += (std::size_t)n;
|
||||
continue;
|
||||
}
|
||||
if (n < 0 && errno == EINTR) continue;
|
||||
if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK))
|
||||
throw WsError(K::Timeout, "send timed out");
|
||||
throw WsError(K::Io, std::string("send failed: ") +
|
||||
std::strerror(n < 0 ? errno : EPIPE));
|
||||
}
|
||||
}
|
||||
|
||||
void TcpStream::shutdown() {
|
||||
if (fd_ >= 0) ::shutdown(fd_, SHUT_RDWR);
|
||||
}
|
||||
|
||||
TlsStream::TlsStream(int fd,
|
||||
std::shared_ptr<clients::transports::ITlsSession> session,
|
||||
const std::string& hostname, int io_timeout_ms)
|
||||
: fd_(fd), read_timeout_ms_(io_timeout_ms), session_(std::move(session)) {
|
||||
set_timeout(fd_, SO_RCVTIMEO, io_timeout_ms);
|
||||
set_timeout(fd_, SO_SNDTIMEO, io_timeout_ms);
|
||||
// The TLS backend writes without MSG_NOSIGNAL; a peer that vanished would
|
||||
// otherwise kill the process. Process-wide, once.
|
||||
static std::once_flag sigpipe_once;
|
||||
std::call_once(sigpipe_once, [] { std::signal(SIGPIPE, SIG_IGN); });
|
||||
try {
|
||||
session_->handshake(fd_, hostname, false);
|
||||
} catch (const clients::transports::TlsError& e) {
|
||||
::close(fd_);
|
||||
fd_ = -1;
|
||||
throw WsError(K::Io, e.what());
|
||||
}
|
||||
}
|
||||
|
||||
TlsStream::~TlsStream() {
|
||||
try {
|
||||
if (session_) session_->shutdown();
|
||||
} catch (...) {
|
||||
}
|
||||
if (fd_ >= 0) ::close(fd_);
|
||||
}
|
||||
|
||||
void TlsStream::set_read_timeout_ms(int ms) {
|
||||
read_timeout_ms_ = ms;
|
||||
set_timeout(fd_, SO_RCVTIMEO, ms);
|
||||
}
|
||||
|
||||
// ITlsSession::recv_some returns 0 for both "timed out" and "peer closed".
|
||||
// A return well before the socket timeout is a close; one that waited out
|
||||
// at least half the timeout is a timeout.
|
||||
ReadResult TlsStream::read(char* out, std::size_t max) {
|
||||
auto t0 = std::chrono::steady_clock::now();
|
||||
std::size_t n = 0;
|
||||
// The backend reports a socket timeout and a dropped connection (Google
|
||||
// closes TCP after its close frame, without close_notify) both as a read
|
||||
// error or a zero read. Elapsed time tells them apart.
|
||||
try {
|
||||
n = session_->recv_some(out, max);
|
||||
} catch (const clients::transports::TlsError&) {
|
||||
n = 0;
|
||||
}
|
||||
if (n > 0) return {ReadStatus::Data, n};
|
||||
auto ms = std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now() - t0)
|
||||
.count();
|
||||
if (ms * 2 >= read_timeout_ms_) return {ReadStatus::Timeout, 0};
|
||||
return {ReadStatus::Eof, 0};
|
||||
}
|
||||
|
||||
void TlsStream::write_all(const char* data, std::size_t size) {
|
||||
try {
|
||||
session_->send_all(data, size);
|
||||
} catch (const clients::transports::TlsError& e) {
|
||||
throw WsError(K::Io, e.what());
|
||||
}
|
||||
}
|
||||
|
||||
void TlsStream::shutdown() {
|
||||
if (fd_ >= 0) ::shutdown(fd_, SHUT_RDWR);
|
||||
}
|
||||
|
||||
std::unique_ptr<IByteStream> open_stream(const WsUrl& url,
|
||||
int connect_timeout_ms,
|
||||
int io_timeout_ms, TlsFactory tls) {
|
||||
addrinfo hints{};
|
||||
hints.ai_family = AF_UNSPEC;
|
||||
hints.ai_socktype = SOCK_STREAM;
|
||||
addrinfo* res = nullptr;
|
||||
if (::getaddrinfo(url.host.c_str(), std::to_string(url.port).c_str(), &hints,
|
||||
&res) != 0 ||
|
||||
!res)
|
||||
throw WsError(K::Io, "cannot resolve " + url.host);
|
||||
int fd = -1;
|
||||
std::string last = "connect failed";
|
||||
for (addrinfo* ai = res; ai; ai = ai->ai_next) {
|
||||
fd = ::socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol);
|
||||
if (fd < 0) continue;
|
||||
int flags = ::fcntl(fd, F_GETFL, 0);
|
||||
::fcntl(fd, F_SETFL, flags | O_NONBLOCK);
|
||||
int rc = ::connect(fd, ai->ai_addr, ai->ai_addrlen);
|
||||
bool ok = rc == 0;
|
||||
if (!ok && errno == EINPROGRESS) {
|
||||
pollfd p{fd, POLLOUT, 0};
|
||||
if (::poll(&p, 1, connect_timeout_ms) == 1) {
|
||||
int err = 0;
|
||||
socklen_t len = sizeof err;
|
||||
::getsockopt(fd, SOL_SOCKET, SO_ERROR, &err, &len);
|
||||
ok = err == 0;
|
||||
if (!ok) last = std::string("connect failed: ") + std::strerror(err);
|
||||
} else {
|
||||
last = "connect timed out";
|
||||
}
|
||||
}
|
||||
if (ok) {
|
||||
::fcntl(fd, F_SETFL, flags);
|
||||
break;
|
||||
}
|
||||
::close(fd);
|
||||
fd = -1;
|
||||
}
|
||||
::freeaddrinfo(res);
|
||||
if (fd < 0) throw WsError(K::Io, last + " to " + url.display());
|
||||
if (!url.secure) return std::make_unique<TcpStream>(fd, io_timeout_ms);
|
||||
auto session = tls ? tls() : clients::transports::MakeTlsSession();
|
||||
return std::make_unique<TlsStream>(fd, std::move(session), url.host,
|
||||
io_timeout_ms);
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
// Blocking byte streams under the WebSocket client: plain TCP and TLS via
|
||||
// the existing ITlsSession (wolfSSL). Read and write timeouts are socket
|
||||
// level; a read that times out reports Timeout, a closed peer reports Eof.
|
||||
|
||||
#include <cstddef>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "clients/transports/tls.h"
|
||||
#include "google/live/ws_url.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
enum class ReadStatus { Data, Timeout, Eof };
|
||||
struct ReadResult {
|
||||
ReadStatus status = ReadStatus::Timeout;
|
||||
std::size_t n = 0;
|
||||
};
|
||||
|
||||
class IByteStream {
|
||||
public:
|
||||
virtual ~IByteStream() = default;
|
||||
virtual ReadResult read(char* out, std::size_t max) = 0;
|
||||
virtual void write_all(const char* data, std::size_t size) = 0;
|
||||
virtual void set_read_timeout_ms(int ms) = 0;
|
||||
virtual void shutdown() = 0;
|
||||
};
|
||||
|
||||
class TcpStream : public IByteStream {
|
||||
public:
|
||||
TcpStream(int fd, int io_timeout_ms); // takes ownership of fd
|
||||
~TcpStream() override;
|
||||
ReadResult read(char* out, std::size_t max) override;
|
||||
void write_all(const char* data, std::size_t size) override;
|
||||
void set_read_timeout_ms(int ms) override;
|
||||
void shutdown() override;
|
||||
|
||||
private:
|
||||
int fd_;
|
||||
};
|
||||
|
||||
class TlsStream : public IByteStream {
|
||||
public:
|
||||
// Handshakes immediately; the fd is owned by this stream.
|
||||
TlsStream(int fd, std::shared_ptr<clients::transports::ITlsSession> session,
|
||||
const std::string& hostname, int io_timeout_ms);
|
||||
~TlsStream() override;
|
||||
ReadResult read(char* out, std::size_t max) override;
|
||||
void write_all(const char* data, std::size_t size) override;
|
||||
void set_read_timeout_ms(int ms) override;
|
||||
void shutdown() override;
|
||||
|
||||
private:
|
||||
int fd_;
|
||||
int read_timeout_ms_;
|
||||
std::shared_ptr<clients::transports::ITlsSession> session_;
|
||||
};
|
||||
|
||||
using TlsFactory =
|
||||
std::function<std::shared_ptr<clients::transports::ITlsSession>()>;
|
||||
|
||||
// Resolves, connects (bounded by connect_timeout_ms) and, for wss, runs the
|
||||
// TLS handshake. A null factory uses clients::transports::MakeTlsSession.
|
||||
std::unique_ptr<IByteStream> open_stream(const WsUrl& url,
|
||||
int connect_timeout_ms,
|
||||
int io_timeout_ms,
|
||||
TlsFactory tls = nullptr);
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,68 @@
|
||||
#include "google/live/ws_url.h"
|
||||
|
||||
#include <cctype>
|
||||
|
||||
#include "google/live/caps.h"
|
||||
#include "google/live/ws_error.h"
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
WsUrl WsUrl::parse(const std::string& url) {
|
||||
using K = WsError::Kind;
|
||||
if (url.empty() || url.size() > LiveCaps::kMaxUrlChars)
|
||||
throw WsError(K::Invalid, "url empty or too long");
|
||||
for (unsigned char c : url)
|
||||
if (c <= 0x20 || c == 0x7f)
|
||||
throw WsError(K::Invalid, "url has control or space characters");
|
||||
WsUrl out;
|
||||
std::size_t pos;
|
||||
if (url.rfind("wss://", 0) == 0) {
|
||||
out.secure = true;
|
||||
out.port = 443;
|
||||
pos = 6;
|
||||
} else if (url.rfind("ws://", 0) == 0) {
|
||||
pos = 5;
|
||||
} else {
|
||||
throw WsError(K::Invalid, "url scheme must be ws or wss");
|
||||
}
|
||||
std::size_t end = url.find_first_of("/?#", pos);
|
||||
std::string authority =
|
||||
url.substr(pos, end == std::string::npos ? std::string::npos : end - pos);
|
||||
if (authority.find('@') != std::string::npos)
|
||||
throw WsError(K::Invalid, "url must not carry userinfo");
|
||||
std::size_t colon = authority.rfind(':');
|
||||
if (colon != std::string::npos) {
|
||||
std::string p = authority.substr(colon + 1);
|
||||
authority.resize(colon);
|
||||
if (p.empty() || p.size() > 5) throw WsError(K::Invalid, "bad port");
|
||||
int v = 0;
|
||||
for (char c : p) {
|
||||
if (!std::isdigit((unsigned char)c)) throw WsError(K::Invalid, "bad port");
|
||||
v = v * 10 + (c - '0');
|
||||
}
|
||||
if (v < 1 || v > 65535) throw WsError(K::Invalid, "port out of range");
|
||||
out.port = v;
|
||||
}
|
||||
if (authority.empty() || authority.size() > 253)
|
||||
throw WsError(K::Invalid, "bad host");
|
||||
for (unsigned char c : authority)
|
||||
if (!(std::isalnum(c) || c == '.' || c == '-' || c == '_'))
|
||||
throw WsError(K::Invalid, "bad host character");
|
||||
out.host = authority;
|
||||
if (end != std::string::npos) {
|
||||
std::string rest = url.substr(end);
|
||||
std::size_t hash = rest.find('#');
|
||||
if (hash != std::string::npos) rest.resize(hash);
|
||||
if (rest.empty() || rest[0] == '?') rest = "/" + rest;
|
||||
out.target = rest;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string WsUrl::display() const {
|
||||
std::string path = target.substr(0, target.find('?'));
|
||||
return std::string(secure ? "wss://" : "ws://") + host + ":" +
|
||||
std::to_string(port) + path;
|
||||
}
|
||||
|
||||
} // namespace molo::google::live
|
||||
@@ -0,0 +1,21 @@
|
||||
#pragma once
|
||||
|
||||
// ws:// and wss:// URL, parsed and validated. display() omits the query so
|
||||
// an API key in "?key=" never reaches an error message or a log line.
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace molo::google::live {
|
||||
|
||||
struct WsUrl {
|
||||
bool secure = false;
|
||||
std::string host;
|
||||
int port = 80;
|
||||
std::string target = "/"; // path plus query, as sent on the request line
|
||||
|
||||
// Throws WsError(Invalid).
|
||||
static WsUrl parse(const std::string& url);
|
||||
std::string display() const; // scheme://host:port/path, no query
|
||||
};
|
||||
|
||||
} // namespace molo::google::live
|
||||
Reference in New Issue
Block a user