feat(clients): google cloud voice: generic client, TTS, STT, Gemini audio, live voice page

src/google: GoogleClient over molo::clients (key in x-goog-api-key, never in
URLs), Service base, TextToSpeech, SpeechToText, Generative (replies, audio
in, audio out), Recognizer/Synthesizer seams with cloud, gemini and vertex
backends, VoicePipeline with bounded history. Example: /voice live
conversation page, /voice/{say,hear,turn,reset}, /google/status. Keys load
from env, .env.json.local, .env.json. Paid routes are excluded from locust.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 04:05:41 +02:00
co-authored by Claude Sonnet 5.5
parent dd773d3015
commit 837de58136
54 changed files with 3086 additions and 21 deletions
+8 -2
View File
@@ -14,7 +14,7 @@ Open `http://127.0.0.1:8080/` and click through the matrix; `/stats` has the
charts (Chart.js is embedded in the binary, no CDN). Narrow the
window to 360px: the top nav collapses into a dropdown menu.
Run the suite (105 route-level integration tests, real stack, no mocks):
Run the suite (111 route-level integration tests, real stack, no mocks):
```sh
make -f example/example.mk example-test
@@ -114,6 +114,12 @@ on a missing row.
| statistics | `/stats` | `tests/example/stats/page_test.cpp` | `plans/02-observability.md` |
| statistics data | `/stats/data` | `tests/example/stats/data_test.cpp` | `plans/02-observability.md` |
| google status | `/google/status` | `tests/example/google/status_test.cpp` | `src/google/SPEC.md` |
| voice conversation page | `/voice` | `tests/example/voice/page_test.cpp` | `src/google/SPEC.md` |
| voice text to speech | `/voice/say` | `tests/example/voice/say_test.cpp` | `src/google/SPEC.md` |
| voice speech to text | `/voice/hear` | `tests/example/voice/hear_test.cpp` | `src/google/SPEC.md` |
| voice turn | `/voice/turn` | `tests/example/voice/turn_test.cpp` | `src/google/SPEC.md` |
| voice reset | `/voice/reset` | `tests/example/voice/reset_test.cpp` | `src/google/SPEC.md` |
### Extended features
@@ -203,7 +209,7 @@ example/
fw/ # skeletal framework public API (namespace molo::example::fw)
app/ # demo routes (namespace molo::example::app)
static/ # disk static assets (responsive CSS, tiny JS)
assets/ # embedded at build time (Chart.js, stats.js) via src/cli/embed
assets/ # embedded at build time (Chart.js, stats.js, voice.js) via src/cli/embed
tools/ # build helper wrapping the embedder
tests/example/ # route-level integration tests mirroring URLs
```
+26 -5
View File
@@ -45,6 +45,20 @@ owning sibling plan and the file that will close it.
builder, singleflight, ttl store). Each is a skeleton of the real `src/`
module; per-module gaps are in the header of its `fw/` file.
16. Voice (`src/google`, the real framework part, not a skeleton):
`GET /voice` is a hands-free conversation page. The browser detects
speech from the microphone level, records one utterance, posts it to
`POST /voice/turn`, plays the spoken answer, and listens again;
speaking over the answer interrupts it. The server runs Google
speech-to-text, a Gemini reply and Google text-to-speech per turn, with
bounded per-conversation history. `POST /voice/say` and `/voice/hear`
expose the two speech services alone; `GET /google/status` shows
credential state and known services (never the key). No
`GOOGLE_API_KEY`: 503 `google_not_configured`. Upstream failure: 502
`google_error` with Google's status. Bad input: 422 or 415 before any
network call. Routes and tests use a scripted transport; the key lives
only in the `x-goog-api-key` header.
## Caps
All ceilings live as literals in `example/fw/Caps.hpp`: 1 MiB request body,
@@ -76,11 +90,13 @@ auth, 403 bad credentials, 404 unknown, 405 wrong method, 406 no format,
## Isolation
May depend on: C++23 stdlib, POSIX sockets, `example/fw/` only. Build-time
only: the embedder sources in `src/cli/embed` (asset embedding). May NOT
depend on: any `src/` tree (does not exist yet in this worktree), wolfSSL
(stub behind `ITlsContext`), real Postgres/SQLite (in-memory doubles),
real network except loopback. Namespaces mirror directories:
May depend on: C++23 stdlib, POSIX sockets, `example/fw/`, and the real
framework parts it links on purpose: `src/google` over `src/clients` (HTTPS
via wolfSSL) and `src/operability`. Build-time only: the embedder sources in
`src/cli/embed` (asset embedding). May NOT
depend on: other `src/` trees, real Postgres/SQLite (in-memory doubles),
real network except Google through `src/google` (tests script that
transport; only the opt-in `MOLO_LIVE_NET=1` tests touch the network). Namespaces mirror directories:
`molo::example::fw` and `molo::example::app`.
## Gaps
@@ -98,6 +114,11 @@ real network except loopback. Namespaces mirror directories:
- Full validation coercions, template sandbox, DI container, flash
messages: owner 05-request-routing.md, files `example/fw/Validate.*`,
`example/fw/Templates.*`, `example/app/App.*`.
- Voice turns are request/response over REST (about 2 to 4 s per turn).
The loopback server accepts sequentially, so one in-flight turn blocks
other requests meanwhile. Streaming recognition and the Gemini Live API:
owner `src/google/SPEC.md`, gaps section. Owner of concurrent accept:
01-runtime.md, `example/fw/Server.*`.
- Server accepts sequentially and stops after 512 connections (cap above);
the stats page shows connections left. Owner: 01-runtime.md,
`example/fw/Server.*`.
+4
View File
@@ -67,6 +67,10 @@ std::shared_ptr<fw::Application> build_application(Services& services) {
register_resilience_demo(resilience, services);
app->include_router("", resilience);
fw::Router voice;
register_voice_demo(voice, services);
app->include_router("", voice);
register_chain_demo(app, services);
return app;
}
+11
View File
@@ -5,6 +5,7 @@
// Gap: real molo::di::Container and plugin registry in 05-request-routing.md.
#include <map>
#include <optional>
#include <memory>
#include <string>
#include <utility>
@@ -18,6 +19,9 @@
#include "../fw/Observ.hpp"
#include "../fw/Stats.hpp"
#include "../fw/Templates.hpp"
#include "google/client.h"
#include "google/voice.h"
#include "google/voice_backend.h"
namespace molo::example::app {
@@ -49,6 +53,12 @@ struct Services {
std::shared_ptr<fw::OpenApi> openapi = std::make_shared<fw::OpenApi>();
std::shared_ptr<fw::Environment> templates =
std::make_shared<fw::Environment>();
// Google Cloud client (null: built from GOOGLE_API_KEY on first use) and
// the voice pipeline wired on it. Tests inject a scripted client.
std::shared_ptr<google::GoogleClient> google;
std::shared_ptr<google::VoicePipeline> voice;
// Unset: chosen from GOOGLE_API_BACKEND / the key prefix (see src/google).
std::optional<google::VoiceBackend> voice_backend;
// Feature matrix rows (feature, route) shown on the index page. Each
// register_*_demo adds its own rows with add_feature().
std::vector<std::pair<std::string, std::string>> features;
@@ -76,6 +86,7 @@ void register_security_extra_demo(fw::Router& router, Services& services);
void register_platform_demo(fw::Router& router, Services& services);
void register_enterprise_demo(fw::Router& router, Services& services);
void register_resilience_demo(fw::Router& router, Services& services);
void register_voice_demo(fw::Router& router, Services& services);
void load_templates(Services& services);
void load_embedded(Services& services);
+34
View File
@@ -22,6 +22,7 @@ const char kBase[] = R"BASE(<!DOCTYPE html>
<a href="/templates/page">Templates</a>
<a href="/forms/contact">Forms</a>
<a href="/stats">Stats</a>
<a href="/voice">Voice</a>
<a href="/docs">Docs</a>
<a href="/health">Health</a>
</nav>
@@ -32,6 +33,7 @@ const char kBase[] = R"BASE(<!DOCTYPE html>
<a href="/templates/page">Templates</a>
<a href="/forms/contact">Forms</a>
<a href="/stats">Stats</a>
<a href="/voice">Voice</a>
<a href="/docs">Docs</a>
<a href="/health">Health</a>
</nav>
@@ -168,6 +170,37 @@ after its connection cap, so auto refresh is off by default.</p>
{% endblock %}
)STATS";
const char kVoice[] = R"VOICE({% extends "base.html" %}
{% block content %}
<h1>Voice conversation</h1>
<p class="hint">Talk, pause, and the assistant answers out loud. Google
speech-to-text, Gemini, and Google text-to-speech run on the server.
Speak over the answer to interrupt it.</p>
<p id="v-config" class="hint">Checking Google credentials...</p>
<section class="voice">
<div class="voice-controls">
<button id="v-toggle" type="button" class="primary" disabled>Start conversation</button>
<label class="inline">Language
<select id="v-lang">
<option value="nl-NL">Nederlands</option>
<option value="en-US">English</option>
<option value="de-DE">Deutsch</option>
</select></label>
</div>
<div class="voice-state"><span id="v-state" class="pill">idle</span>
<div class="meter"><div id="v-level"></div></div></div>
<p id="v-error" class="error" role="alert"></p>
<ol id="v-log" class="voice-log"></ol>
<form id="v-form" class="voice-form">
<input id="v-text" type="text" maxlength="500" placeholder="Or type a message" autocomplete="off">
<button type="submit">Send</button>
<button id="v-reset" type="button">Reset</button>
</form>
</section>
<script src="/embedded/voice.js"></script>
{% endblock %}
)VOICE";
const char kDocs[] = R"DOCS({% extends "base.html" %}
{% block content %}
<h1>API docs</h1>
@@ -198,6 +231,7 @@ void load_templates(Services& services) {
services.templates->add_template("page.html", kPage);
services.templates->add_template("form_page.html", kFormPage);
services.templates->add_template("docs.html", kDocs);
services.templates->add_template("voice.html", kVoice);
services.templates->add_template("stats.html", kStats);
services.templates->add_template("error.html", kError);
services.templates->add_template("nav_hint.html", kNavHint);
+208
View File
@@ -0,0 +1,208 @@
#include "App.hpp"
#include <cctype>
#include <cstdlib>
#include "../fw/Router.hpp"
#include "../fw/Templates.hpp"
#include "../fw/Validate.hpp"
#include "google/credentials.h"
#include "google/endpoints.h"
#include "security/crypto/Base64.h"
namespace molo::example::app {
// Google Cloud voice over the real src/google part: text-to-speech,
// speech-to-text, a Gemini reply, and the live conversation page.
// Gap: turns are request/response over REST; streaming recognition and the
// Gemini Live API are listed in src/google/SPEC.md.
namespace {
std::string env_or(const char* name, const char* fallback) {
std::string v = google::Credentials::env(name);
return v.empty() ? std::string(fallback) : v;
}
// "nl-NL", "en-US": two or three lowercase letters, dash, two uppercase.
bool valid_language(const std::string& s) {
std::size_t dash = s.find('-');
if (dash < 2 || dash > 3 || s.size() != dash + 3) return false;
for (std::size_t i = 0; i < dash; ++i)
if (!std::islower((unsigned char)s[i])) return false;
return std::isupper((unsigned char)s[dash + 1]) &&
std::isupper((unsigned char)s[dash + 2]);
}
std::string language_of(const std::string& value) {
std::string lang = value.empty() ? env_or("GOOGLE_VOICE_LANG", "nl-NL") : value;
if (!valid_language(lang))
throw fw::HttpError(422, "validation_error", "lang must look like nl-NL");
return lang;
}
bool valid_voice(const std::string& s) {
if (s.size() > 64) return false;
for (char c : s)
if (!std::isalnum((unsigned char)c) && c != '-' && c != '_') return false;
return true;
}
std::string voice_of(const std::string& value) {
std::string voice = value.empty() ? google::Credentials::env("GOOGLE_VOICE_NAME") : value;
if (!valid_voice(voice))
throw fw::HttpError(422, "validation_error", "voice has invalid characters");
return voice;
}
// Maps Google part errors onto the example's JSON error format.
[[noreturn]] void rethrow(const google::GoogleError& e) {
if (e.http_status() == 503 && e.status() == "NOT_CONFIGURED")
throw fw::HttpError(503, "google_not_configured", e.message());
// Local input validation carries no service name; upstream errors do.
if (e.service().empty() && e.status() == "INVALID_ARGUMENT")
throw fw::HttpError(422, "validation_error", e.message());
throw fw::HttpError(502, "google_error",
e.status() + ": " + e.message());
}
template <typename F>
auto guarded(F&& f) -> decltype(f()) {
try {
return f();
} catch (const google::GoogleError& e) {
rethrow(e);
}
}
std::string quote(const std::string& s) {
return "\"" + fw::json_escape(s) + "\"";
}
fw::Response no_store(fw::Response res) {
res.headers.set("Cache-Control", "no-store");
return res;
}
} // namespace
void register_voice_demo(fw::Router& router, Services& services) {
if (!services.google) services.google = google::GoogleClient::from_env();
auto gclient = services.google;
google::VoiceBackend backend =
services.voice_backend.value_or(google::voice_backend_from_env());
google::VoiceParts parts = google::make_voice_parts(gclient, backend);
auto tts = parts.synthesizer;
auto stt = parts.recognizer;
services.voice = std::make_shared<google::VoicePipeline>(
stt, parts.responder, tts, std::make_shared<google::VoiceSessions>());
auto pipeline = services.voice;
fw::Environment* env = services.templates.get();
services.add_feature("google", "/google/status");
services.add_feature("voice", "/voice");
router.get("/google/status", "google-status", [gclient, backend](fw::Request req) {
(void)req;
std::string out = std::string("{\"configured\":") +
(gclient->configured() ? "true" : "false") +
",\"credential\":\"x-goog-api-key\",\"backend\":" +
quote(google::voice_backend_name(backend)) + ",\"model\":" +
quote(google::Generative::default_model()) +
",\"language\":" + quote(env_or("GOOGLE_VOICE_LANG", "nl-NL")) +
",\"services\":[";
bool first = true;
for (const auto& e : google::endpoints::catalog()) {
out += std::string(first ? "" : ",") + "{\"name\":" + quote(e.name) +
",\"base_url\":" + quote(e.base_url) + "}";
first = false;
}
return no_store(fw::JsonResponse(out + "]}", 200));
});
router.get("/voice", "voice-page", [env](fw::Request req) {
(void)req;
fw::TemplateCtx ctx;
ctx["title"] = std::string("Voice conversation");
return fw::Response(200, env->render("voice.html", ctx),
"text/html; charset=utf-8");
});
// Text in, spoken audio (mp3) out.
router.post("/voice/say", "voice-say", [tts](fw::Request req) {
auto in = fw::parse_json_object_flat(req.body);
google::SynthesisRequest speak;
speak.text = in.count("text") ? in["text"] : "";
speak.language = language_of(in.count("lang") ? in["lang"] : "");
speak.voice = voice_of(in.count("voice") ? in["voice"] : "");
return guarded([&] {
google::Audio audio = tts->synthesize(speak);
return no_store(fw::Response(200, audio.bytes, audio.mime));
});
});
// Audio bytes in (Content-Type names the encoding), transcript out.
router.post("/voice/hear", "voice-hear", [stt](fw::Request req) {
google::RecognitionRequest heard;
heard.audio = req.body;
if (!google::speech_encoding_from_mime(req.headers.get("Content-Type"),
heard.encoding))
throw fw::HttpError(415, "unsupported_media_type",
"send audio/webm, audio/ogg, audio/flac, audio/wav or audio/mpeg");
heard.language = language_of(req.query_or("lang", ""));
return guarded([&] {
google::Transcript t = stt->recognize(heard);
return no_store(fw::JsonResponse(
"{\"text\":" + quote(t.text) +
",\"confidence\":" + std::to_string(t.confidence) + "}",
200));
});
});
// One conversation turn. Audio body = spoken turn; JSON body {"text"} =
// typed turn. Both answer {transcript, reply, audio_b64, mime, language};
// silence answers an empty transcript and no audio.
router.post("/voice/turn", "voice-turn", [pipeline](fw::Request req) {
std::string cid = req.query_or("cid", "");
if (!google::VoiceSessions::valid_id(cid))
throw fw::HttpError(422, "validation_error",
"cid must be 1-64 chars of A-Z a-z 0-9 - _");
std::string lang = language_of(req.query_or("lang", ""));
std::string voice = voice_of(req.query_or("voice", ""));
std::string ctype = req.headers.get("Content-Type");
return guarded([&] {
google::VoiceTurn turn;
if (ctype.find("json") != std::string::npos) {
auto in = fw::parse_json_object_flat(req.body);
turn = pipeline->turn_text(cid, in.count("text") ? in["text"] : "",
lang, voice);
} else {
google::RecognitionRequest heard;
heard.audio = req.body;
heard.language = lang;
if (!google::speech_encoding_from_mime(ctype, heard.encoding))
throw fw::HttpError(415, "unsupported_media_type",
"send audio/webm, audio/ogg, audio/flac, audio/wav or audio/mpeg");
turn = pipeline->turn(cid, heard, voice);
}
return no_store(fw::JsonResponse(
"{\"transcript\":" + quote(turn.transcript) +
",\"reply\":" + quote(turn.reply) + ",\"language\":" +
quote(turn.language) + ",\"mime\":" + quote(turn.audio.mime) +
",\"audio_b64\":\"" +
security::crypto::Base64::encode(turn.audio.bytes) + "\"}",
200));
});
});
router.post("/voice/reset", "voice-reset", [pipeline](fw::Request req) {
std::string cid = req.query_or("cid", "");
if (!google::VoiceSessions::valid_id(cid))
throw fw::HttpError(422, "validation_error",
"cid must be 1-64 chars of A-Z a-z 0-9 - _");
pipeline->sessions().reset(cid);
return fw::JsonResponse("{\"reset\":true}", 200);
});
}
} // namespace molo::example::app
+244
View File
@@ -0,0 +1,244 @@
// Voice conversation page: hands-free loop in the browser.
// listen -> detect speech (mic level) -> record one utterance -> POST
// /voice/turn -> play the spoken answer -> listen again. Speaking over the
// answer interrupts it. All limits are constants below.
(function () {
"use strict";
var LEVEL_START = 0.04; // RMS that counts as speech
var LEVEL_BARGE = 0.09; // RMS that interrupts the assistant's voice
var ONSET_MS = 80; // speech must last this long to start recording
var BARGE_MS = 300; // ... and this long to interrupt playback
var SILENCE_MS = 900; // quiet this long ends the utterance
var MIN_SPEECH_MS = 300; // shorter utterances are noise, discarded
var MAX_UTTERANCE_MS = 30000;
var TICK_MS = 40;
var $ = function (id) { return document.getElementById(id); };
var toggle = $("v-toggle"), langSel = $("v-lang"), stateEl = $("v-state");
var levelEl = $("v-level"), errorEl = $("v-error"), logEl = $("v-log");
var form = $("v-form"), textEl = $("v-text"), resetBtn = $("v-reset");
var cid = newId();
var stream = null, ctx = null, analyser = null, buf = null, timer = null;
var recorder = null, chunks = [], recStart = 0, voicedMs = 0, quietMs = 0;
var onsetMs = 0, bargeMs = 0;
var player = null;
var state = "idle"; // idle | listening | hearing | thinking | speaking
function newId() {
var a = new Uint8Array(12);
crypto.getRandomValues(a);
return Array.prototype.map.call(a, function (b) {
return ("0" + b.toString(16)).slice(-2);
}).join("");
}
function setState(s) {
state = s;
stateEl.textContent = s;
stateEl.className = "pill " + s;
}
function showError(msg) { errorEl.textContent = msg || ""; }
function addLog(who, text) {
var li = document.createElement("li");
li.className = who;
var b = document.createElement("b");
b.textContent = who === "you" ? "You" : "Assistant";
li.appendChild(b);
li.appendChild(document.createTextNode(": " + text));
logEl.appendChild(li);
li.scrollIntoView({ block: "nearest" });
}
function errorText(res, body) {
try {
var j = JSON.parse(body);
return j.message || j.error || ("HTTP " + res.status);
} catch (e) { return "HTTP " + res.status; }
}
function rms() {
analyser.getByteTimeDomainData(buf);
var sum = 0;
for (var i = 0; i < buf.length; i++) {
var v = (buf[i] - 128) / 128;
sum += v * v;
}
return Math.sqrt(sum / buf.length);
}
function pickMime() {
var list = ["audio/webm;codecs=opus", "audio/ogg;codecs=opus", "audio/webm"];
for (var i = 0; i < list.length; i++)
if (window.MediaRecorder && MediaRecorder.isTypeSupported(list[i])) return list[i];
return "";
}
function startRecording() {
chunks = [];
var mime = pickMime();
recorder = mime ? new MediaRecorder(stream, { mimeType: mime }) : new MediaRecorder(stream);
recorder.ondataavailable = function (e) { if (e.data.size) chunks.push(e.data); };
recorder.start(250);
recStart = Date.now();
voicedMs = 0; quietMs = 0;
setState("hearing");
}
function stopRecording(send) {
var r = recorder, voiced = voicedMs;
recorder = null;
r.onstop = function () {
var blob = new Blob(chunks, { type: r.mimeType || "audio/webm" });
chunks = [];
if (send && voiced >= MIN_SPEECH_MS && state !== "idle") sendAudio(blob);
else if (state !== "idle") setState("listening");
};
r.stop();
}
function turnUrl() {
return "/voice/turn?cid=" + encodeURIComponent(cid) +
"&lang=" + encodeURIComponent(langSel.value);
}
function sendAudio(blob) {
setState("thinking");
post(turnUrl(), blob, blob.type || "audio/webm");
}
function post(url, body, type) {
showError("");
fetch(url, { method: "POST", headers: { "Content-Type": type }, body: body })
.then(function (res) { return res.text().then(function (t) { return [res, t]; }); })
.then(function (pair) {
var res = pair[0], text = pair[1];
if (!res.ok) {
showError(errorText(res, text));
if (res.status === 503) { stop(); return; }
if (state !== "idle") setState(stream ? "listening" : "idle");
return;
}
handleTurn(JSON.parse(text));
})
.catch(function (err) {
showError("Network error: " + err.message);
if (state !== "idle") setState(stream ? "listening" : "idle");
});
}
function handleTurn(turn) {
if (!turn.transcript) { // nothing intelligible was heard
if (state !== "idle") setState(stream ? "listening" : "idle");
return;
}
addLog("you", turn.transcript);
addLog("bot", turn.reply);
if (state === "idle" && !stream) return;
play(turn);
}
function play(turn) {
stopPlayback();
player = new Audio("data:" + turn.mime + ";base64," + turn.audio_b64);
var mine = player;
mine.onended = function () {
if (player === mine) { player = null; if (stream) setState("listening"); else setState("idle"); }
};
mine.onerror = function () {
if (player === mine) { player = null; showError("Could not play audio"); if (stream) setState("listening"); }
};
setState(stream ? "speaking" : "idle");
mine.play().catch(function (e) { showError("Playback blocked: " + e.message); if (stream) setState("listening"); });
}
function stopPlayback() {
if (player) { var p = player; player = null; p.onended = null; p.pause(); }
}
function tick() {
var level = rms();
levelEl.style.width = Math.min(100, Math.round(level * 400)) + "%";
if (state === "listening") {
onsetMs = level > LEVEL_START ? onsetMs + TICK_MS : 0;
if (onsetMs >= ONSET_MS) { onsetMs = 0; startRecording(); }
} else if (state === "speaking") {
bargeMs = level > LEVEL_BARGE ? bargeMs + TICK_MS : 0;
if (bargeMs >= BARGE_MS) { bargeMs = 0; stopPlayback(); startRecording(); }
} else if (state === "hearing") {
if (level > LEVEL_START) { voicedMs += TICK_MS; quietMs = 0; }
else quietMs += TICK_MS;
if (quietMs >= SILENCE_MS || Date.now() - recStart >= MAX_UTTERANCE_MS)
stopRecording(true);
}
}
function start() {
showError("");
if (!navigator.mediaDevices || !window.MediaRecorder) {
showError("This browser has no microphone recording support.");
return;
}
navigator.mediaDevices.getUserMedia({
audio: { echoCancellation: true, noiseSuppression: true, autoGainControl: true }
}).then(function (s) {
stream = s;
ctx = new (window.AudioContext || window.webkitAudioContext)();
var src = ctx.createMediaStreamSource(stream);
analyser = ctx.createAnalyser();
analyser.fftSize = 1024;
buf = new Uint8Array(analyser.fftSize);
src.connect(analyser);
onsetMs = 0; bargeMs = 0;
timer = setInterval(tick, TICK_MS);
toggle.textContent = "Stop conversation";
toggle.classList.add("live");
setState("listening");
}).catch(function (err) {
showError("Microphone unavailable: " + err.message);
});
}
function stop() {
if (timer) { clearInterval(timer); timer = null; }
if (recorder) { var r = recorder; recorder = null; r.onstop = null; try { r.stop(); } catch (e) {} }
stopPlayback();
if (stream) { stream.getTracks().forEach(function (t) { t.stop(); }); stream = null; }
if (ctx) { ctx.close(); ctx = null; }
levelEl.style.width = "0";
toggle.textContent = "Start conversation";
toggle.classList.remove("live");
setState("idle");
}
toggle.addEventListener("click", function () { if (stream) stop(); else start(); });
form.addEventListener("submit", function (ev) {
ev.preventDefault();
var text = textEl.value.trim();
if (!text || state === "thinking") return;
textEl.value = "";
stopPlayback();
setState("thinking");
post(turnUrl(), JSON.stringify({ text: text }), "application/json");
});
resetBtn.addEventListener("click", function () {
fetch("/voice/reset?cid=" + encodeURIComponent(cid), { method: "POST" });
cid = newId();
logEl.innerHTML = "";
showError("");
});
fetch("/google/status").then(function (r) { return r.json(); }).then(function (j) {
var cfg = $("v-config");
if (j.configured) {
cfg.textContent = "Google credentials found. Model: " + j.model + ".";
cfg.className = "ok";
toggle.disabled = false;
} else {
cfg.textContent = "No Google API key. Start the server with GOOGLE_API_KEY set.";
cfg.className = "error";
}
}).catch(function () { $("v-config").textContent = "Status unavailable."; });
setState("idle");
})();
+12 -2
View File
@@ -12,6 +12,10 @@ HERE = Path(__file__).resolve().parent
APP = HERE / "app"
LOCUST = HERE / "locustfile.py"
# Routes that call paid Google APIs: never load tested. The check fails if
# locust references one, and does not require them to be covered.
EXCLUDED = {"/voice/say", "/voice/hear", "/voice/turn", "/voice/reset"}
FILLS = {"name": "locust", "id": "1", "status": "404", "tenant": "acme"}
@@ -55,7 +59,7 @@ def main() -> int:
routes = collect_routes()
urls, methods = collect_locust_urls()
missing: list[str] = []
for r in sorted(routes):
for r in sorted(routes - EXCLUDED):
c = fill(r)
ok = (
c in urls
@@ -68,6 +72,9 @@ def main() -> int:
)
if not ok:
missing.append(f"{r} -> {c}")
leaked = sorted(EXCLUDED & urls)
for r in leaked:
print("FORBIDDEN (costs money):", r)
required_methods = {"get", "post", "put", "options"}
missing_methods = required_methods - methods
print(f"routes={len(routes)} urls={len(urls)} methods={sorted(methods)}")
@@ -75,10 +82,13 @@ def main() -> int:
print("MISSING METHODS:", sorted(missing_methods))
for m in missing:
print("MISS", m)
if leaked:
print(f"FAIL: {len(leaked)} paid Google routes in locust")
return 1
if missing or missing_methods:
print(f"FAIL: {len(missing)} missing routes, {len(missing_methods)} missing methods")
return 1
print(f"OK: {len(routes)}/{len(routes)} routes covered, methods {sorted(methods)}")
print(f"OK: {len(routes) - len(EXCLUDED)} routes covered, {len(EXCLUDED)} paid Google routes excluded, methods {sorted(methods)}")
return 0
+19 -8
View File
@@ -52,7 +52,8 @@ EXAMPLE_SRCS = \
$(EXAMPLE_DIR)/app/SecurityExtraDemo.cpp \
$(EXAMPLE_DIR)/app/PlatformDemo.cpp \
$(EXAMPLE_DIR)/app/EnterpriseDemo.cpp \
$(EXAMPLE_DIR)/app/ResilienceDemo.cpp
$(EXAMPLE_DIR)/app/ResilienceDemo.cpp \
$(EXAMPLE_DIR)/app/VoiceDemo.cpp
# Tests are discovered: every tests/example/**/*_test.cpp is one route test.
EXAMPLE_TEST_JOBS ?= 8
@@ -68,10 +69,15 @@ EXAMPLE_ASSETS = \
$(EXAMPLE_DIR)/assets/app.css \
$(EXAMPLE_DIR)/assets/chart.umd.min.js \
$(EXAMPLE_DIR)/assets/hello.txt \
$(EXAMPLE_DIR)/assets/stats.js
EXAMPLE_INC = -I$(EXAMPLE_DIR) -I$(EXAMPLE_BUILD)
$(EXAMPLE_DIR)/assets/stats.js \
$(EXAMPLE_DIR)/assets/voice.js
EXAMPLE_INC = -I$(EXAMPLE_DIR) -I$(EXAMPLE_BUILD) -Isrc
# Real framework parts the example links: Google Cloud (src/google) over the
# HTTP client (src/clients, wolfSSL TLS) and the operability breaker.
EXAMPLE_REAL_LIBS = build/google/libgoogle.a build/clients/libclients.a \
build/liboperability.a
.PHONY: example example-assets example-test example-clean example-locust
.PHONY: example example-real-libs example-assets example-test example-clean example-locust
# Asset embedding is part of the build: example/assets (Chart.js included)
# becomes generated C++ under build/example/gen via the MoloApp embedder.
@@ -86,7 +92,12 @@ $(EXAMPLE_GEN)/assets.cpp: $(EXAMPLE_ASSETS) $(EXAMPLE_DIR)/tools/embed_assets.c
$(EXAMPLE_BUILD)/embed_assets $(EXAMPLE_DIR)/assets \
$(EXAMPLE_GEN)/assets.cpp $(EXAMPLE_GEN)/assets.hpp molo::example::gen
example: example-assets
example-real-libs:
$(MAKE) -f src/operability.mk build/liboperability.a
$(MAKE) -f src/clients/clients.mk clients
$(MAKE) -f src/google/google.mk google
example: example-assets example-real-libs
mkdir -p $(EXAMPLE_BUILD)/obj
for s in $(EXAMPLE_SRCS); do \
o=$(EXAMPLE_BUILD)/obj/`echo $$s | tr '/' '_'`.o; \
@@ -98,7 +109,7 @@ example: example-assets
$(MAKE) -f src/supervisor/supervisor.mk build/supervisor/libsupervisor.a
$(CXX) $(EXAMPLE_CXXFLAGS) -I$(EXAMPLE_DIR) -Isrc $(EXAMPLE_DIR)/main.cpp \
$(EXAMPLE_BUILD)/libexample.a build/supervisor/libsupervisor.a \
-o $(EXAMPLE_BUILD)/living-server
$(EXAMPLE_REAL_LIBS) -lwolfssl -o $(EXAMPLE_BUILD)/living-server
example-test: example
rm -rf $(EXAMPLE_BUILD)/tests
@@ -107,8 +118,8 @@ example-test: example
xargs -P $(EXAMPLE_TEST_JOBS) -I{} sh -c ' \
t={}; n=`echo $$t | sed "s|$(EXAMPLE_TESTDIR)/||; s|/|_|g; s|\\.cpp$$||"`; \
b=$(EXAMPLE_BUILD)/tests/$$n; \
if $(CXX) $(EXAMPLE_CXXFLAGS) -I$(EXAMPLE_DIR) -I$(EXAMPLE_TESTDIR) \
$$t $(EXAMPLE_BUILD)/libexample.a -o $$b > $$b.log 2>&1; \
if $(CXX) $(EXAMPLE_CXXFLAGS) -I$(EXAMPLE_DIR) -I$(EXAMPLE_TESTDIR) -Isrc \
$$t $(EXAMPLE_BUILD)/libexample.a $(EXAMPLE_REAL_LIBS) -lwolfssl -o $$b > $$b.log 2>&1; \
then $$b >> $$b.log 2>&1; else echo "COMPILE FAILED: $$t" >> $$b.log; false; fi; \
echo $$? > $$b.rc' < $(EXAMPLE_BUILD)/tests/list.txt
pass=0; fail=0; \
+3
View File
@@ -39,6 +39,7 @@ GETS = [
"/csp/page", "/csp/report",
"/di/services", "/di/resolve/locust",
"/embedded/chart.umd.min.js", "/embedded/hello.txt", "/embedded/stats.js",
"/embedded/voice.js", "/google/status", "/voice",
"/events/fire", "/expose/record", "/expose/stats",
"/files/", "/files/welcome.txt",
"/flash", "/flash/set",
@@ -87,6 +88,8 @@ GETS = [
"/webdav/files/test.txt",
]
# NOT load tested on purpose: /voice/say, /voice/hear, /voice/turn and
# /voice/reset call paid Google APIs (check_locust_coverage.py enforces this).
POSTS = [
("/audit/log", {"event": "locust"}),
("/audit/retention/hold", {"hold": True}),
+26
View File
@@ -76,3 +76,29 @@ label.inline input { padding: 0; }
.nav-wide { display: flex; }
.nav-narrow { display: none; }
}
/* Voice conversation page */
.voice { display: flex; flex-direction: column; gap: 0.75rem; }
.voice-controls { display: flex; flex-wrap: wrap; align-items: center; gap: 0.75rem; }
.voice-controls button.primary { flex: 1 1 12rem; padding: 0.8rem 1rem; font-weight: 600;
border: 0; border-radius: 6px; background: var(--series-1); color: #fff; }
.voice-controls button.primary.live { background: var(--series-8); }
.voice-controls button:disabled { opacity: 0.5; }
.voice-controls select { font-size: 1rem; padding: 0.4rem; }
.voice-state { display: flex; align-items: center; gap: 0.75rem; }
.pill { min-width: 6rem; text-align: center; padding: 0.2rem 0.6rem; border-radius: 1rem;
border: 1px solid var(--border); color: var(--text-secondary); }
.pill.listening { border-color: var(--series-3); color: var(--series-3); }
.pill.hearing { border-color: var(--series-2); color: var(--series-2); }
.pill.thinking { border-color: var(--series-4); color: var(--series-4); }
.pill.speaking { border-color: var(--series-1); color: var(--series-1); }
.meter { flex: 1; height: 0.6rem; border: 1px solid var(--border); border-radius: 0.3rem; overflow: hidden; }
.meter > div { height: 100%; width: 0; background: var(--series-3); }
.voice-log { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column;
gap: 0.4rem; max-height: 22rem; overflow-y: auto; }
.voice-log li { padding: 0.4rem 0.6rem; border: 1px solid var(--border); border-radius: 6px;
overflow-wrap: anywhere; }
.voice-log li.you { border-left: 4px solid var(--series-2); }
.voice-log li.bot { border-left: 4px solid var(--series-1); }
.voice-form { flex-direction: row; flex-wrap: wrap; max-width: none; }
.voice-form input[type="text"] { flex: 1 1 12rem; min-width: 0; }