diff --git a/plans/16-agent-platform.md b/plans/16-agent-platform.md index 1ff31d8..3cdca21 100644 --- a/plans/16-agent-platform.md +++ b/plans/16-agent-platform.md @@ -30,25 +30,31 @@ Foundation (already in `src/google/`, read it first): `GoogleClient` 1. Read `AGENTS.md`, `src/google/SPEC.md`, the foundation headers, one finished service (`generative.*`, `voice*.*`) and its tests. -2. Research first. Use WebFetch/WebSearch on Google's official docs - (`docs.cloud.google.com/gemini-enterprise-agent-platform`, Vertex AI - REST references, `cloud.google.com/*/docs`). Record exact resource - paths, HTTP methods, request and response fields. +2. Research first, from machine-readable sources. For every Google API + fetch its Discovery document (`https://.googleapis.com/$discovery/ + rest?version=`, no auth) and derive paths, methods, parameters and + fields from it; read the official docs for semantics. Protocols without + Discovery (WebSocket, MCP, A2A) are read from their primary spec. 3. The module SPEC (`src/google//SPEC.md`) has: scope, module map - (one class per concern), static caps table, contracts, `## Sources` - (URLs read), `## Unverified` (every shape or path inferred rather than - read from docs), `## Gaps`. Never invent an endpoint silently. If a - product has no public API, say so and implement only what exists, with - any local abstraction named as local. + (one class per concern), static caps table, contracts, `## Sources`, + `## Verified against discovery` (date, counts), `## Verified live` + (call, status, date), `## Unverified` (every shape inferred rather than + read or seen), `## Gaps`. Never invent an endpoint silently. If a product + has no public API, say so and name any local abstraction as local. 4. Object oriented, one concern per file, namespace `molo::google::`, services derive from `Service`. Hard caps for every count, size, interval and wait; validate all input and every id (`valid_segment`) before any network traffic; list calls cap pages. Destructive calls (delete, purge) are explicit methods, never defaults. -5. Tests: `tests/src/google//test_*.cpp` with `MockTransport` - scripted from the documented shapes: request shape, happy path, error - mapping, validation, caps, pagination and long-running paths. Use the - existing harness (`TEST(suite, name)`, `framework.h`). +5. NO verification with scripted Google responses. Each module declares its + calls in one table and ships `tools/check_discovery.py` (python3 stdlib) + that fails when a declared call or request field is absent from + Discovery. Tests under `tests/src/google//` are deterministic and + local (validation, caps, ids, request bodies against the table, local + logic with a fake `LanguageModel`, loopback servers for non-Google + protocols) plus opt-in live tests (`MOLO_LIVE_NET=1`, loud SKIP + otherwise). Live checks are read-only or a few tokens: no resource + creation, update or delete. 6. Example: fill `example/app/agent/Demo.cpp`. Routes live under `/agent/`. Contract: GET page and status routes answer 2xx with no credential (the matrix crawler hits them); every route that would @@ -60,9 +66,9 @@ Foundation (already in `src/google/`, read it first): `GoogleClient` for the GET page only. State is created inside the register function; do not edit `Services`. 7. Example tests: `tests/example/agent//*_test.cpp` (Arrange / Act / - Assert as in `tests/README.md`) using `support/GoogleRig.hpp`: happy - path, no credential, validation, 405, one limit case. Set - `GOOGLE_CLOUD_PROJECT` with `setenv` inside the test. + Assert as in `tests/README.md`) with `support/GoogleRig.hpp` only to + inject an unconfigured or dummy client: GET pages, 405, 422, 503 + `google_not_configured`, limit cases. Never script a Google reply. 8. All of `/agent/*`, `/google/*`, `/voice/*` are excluded from locust (`example/check_locust_coverage.py`). Do not add them to `example/locustfile.py`. @@ -74,7 +80,9 @@ Foundation (already in `src/google/`, read it first): `GoogleClient` modules' files, or `src/google/*.{h,cpp}` foundation files. If a foundation gap blocks you, work around it inside your module and list it in your report. Build globs pick up new files automatically. -10. No live network, no keys. The suite is scripted. Never print or commit +10. Live checks use `GOOGLE_API_KEY`/`GOOGLE_CLOUD_PROJECT` from the + environment and `export GOOGLE_ACCESS_TOKEN=$(cd /home/retoor/projects/ + moloapp && ./build/google/sa-token)` for OAuth; never print, log or commit a credential. No em dashes anywhere (code, docs, comments, commits). 11. Build and test: `make google-test` (framework tests) and `make -f example/example.mk example-test` (example suite). Do not run diff --git a/src/google/credentials.cpp b/src/google/credentials.cpp index 06f8ba4..8be9ae3 100644 --- a/src/google/credentials.cpp +++ b/src/google/credentials.cpp @@ -2,14 +2,31 @@ #include +#include "google/errors.h" #include "google/service_account.h" #include +#include #include #include "clients/body/json.h" namespace molo::google { +FileTokenAuth::FileTokenAuth(std::string path) : path_(std::move(path)) {} + +void FileTokenAuth::apply(clients::ClientRequest& request) const { + std::ifstream in(path_, std::ios::binary); + std::string token((std::istreambuf_iterator(in)), std::istreambuf_iterator()); + if (token.size() > 8 * 1024) token.clear(); + while (!token.empty() && (token.back() == '\n' || token.back() == '\r' || + token.back() == ' ')) + token.pop_back(); + if (token.empty()) + throw GoogleError(401, "UNAUTHENTICATED", + "access token file is missing, empty or over the size cap"); + request.bearer(token); +} + std::shared_ptr Credentials::api_key(std::string key) { return std::make_shared(std::move(key), "x-goog-api-key"); } @@ -62,6 +79,8 @@ std::shared_ptr Credentials::api_key_from_env(const char* name) { std::shared_ptr Credentials::oauth_from_env() { std::string token = env("GOOGLE_ACCESS_TOKEN"); if (!token.empty()) return bearer(std::move(token)); + std::string file = env("GOOGLE_ACCESS_TOKEN_FILE"); + if (!file.empty()) return std::make_shared(std::move(file)); return ServiceAccountAuth::from_env(); } diff --git a/src/google/credentials.h b/src/google/credentials.h index 257c726..b6d62b7 100644 --- a/src/google/credentials.h +++ b/src/google/credentials.h @@ -11,6 +11,19 @@ namespace molo::google { +// Bearer token read from a file on every request, so an external refresher +// (cron, `gcloud auth print-access-token > file`, the sa-token tool) can +// rotate it without restarting the process. File capped at 8 KiB. +class FileTokenAuth : public clients::Auth { + public: + explicit FileTokenAuth(std::string path); + void apply(clients::ClientRequest& request) const override; + const char* scheme() const override { return "bearer-file"; } + + private: + std::string path_; +}; + class Credentials { public: static std::shared_ptr api_key(std::string key); @@ -22,7 +35,8 @@ class Credentials { // API key only (GOOGLE_API_KEY), or nullptr. static std::shared_ptr api_key_from_env( const char* name = "GOOGLE_API_KEY"); - // OAuth only: GOOGLE_ACCESS_TOKEN as a bearer, else the service account in + // OAuth only: GOOGLE_ACCESS_TOKEN as a bearer, else GOOGLE_ACCESS_TOKEN_FILE + // (re-read per request), else the service account in // GOOGLE_APPLICATION_CREDENTIALS (self-refreshing), else nullptr. static std::shared_ptr oauth_from_env(); // Legacy single credential: token, else key, else nullptr. diff --git a/src/google/gemini_types.cpp b/src/google/gemini_types.cpp index eafcb8c..31a394f 100644 --- a/src/google/gemini_types.cpp +++ b/src/google/gemini_types.cpp @@ -72,6 +72,7 @@ Json Part::to_json() const { out.set("text", Json(text)); } if (thought) out.set("thought", Json(true)); + if (!thought_signature.empty()) out.set("thoughtSignature", Json(thought_signature)); return out; } @@ -79,6 +80,7 @@ Part Part::from_json(const Json& doc) { Part p; p.text = str_of(doc, "text"); if (const Json* t = doc.find("thought"); t && t->is_bool()) p.thought = t->as_bool(); + p.thought_signature = str_of(doc, "thoughtSignature"); if (const Json* d = doc.find("inlineData"); d && d->is_object()) { std::string raw; Base64::decode(str_of(*d, "data"), raw); diff --git a/src/google/gemini_types.h b/src/google/gemini_types.h index a4c3671..179d37b 100644 --- a/src/google/gemini_types.h +++ b/src/google/gemini_types.h @@ -39,6 +39,9 @@ struct FunctionResponse { struct Part { std::string text; bool thought = false; + // Opaque, base64 as received. Gemini 3 rejects a replayed functionCall part + // that lacks the signature it was given; keep it and send it back. + std::string thought_signature; std::optional inline_data; std::optional file_data; std::optional function_call; diff --git a/tests/src/google/test_foundation.cpp b/tests/src/google/test_foundation.cpp index 2e15047..948502c 100644 --- a/tests/src/google/test_foundation.cpp +++ b/tests/src/google/test_foundation.cpp @@ -1,6 +1,9 @@ // Foundation shared by every agent-platform module: all-verb client, // streaming decoder, typed Gemini content, resource names, operations, // pagination. +#include + +#include #include #include "framework.h" @@ -261,3 +264,34 @@ TEST(foundation, access_token_beats_key) { ASSERT_EQ(std::string(Credentials::from_env()->scheme()), "apikey"); unsetenv("GOOGLE_API_KEY"); } + +TEST(foundation, thought_signature_roundtrips) { + Part p = Part::of_call(FunctionCall{"f", Json::object(), ""}); + p.thought_signature = "c2lnbmF0dXJl"; + Part back = Part::from_json(Json::parse(p.to_json().dump())); + ASSERT_EQ(back.thought_signature, "c2lnbmF0dXJl"); + ASSERT_TRUE(back.function_call.has_value()); + Part plain = Part::of_text("x"); + ASSERT_TRUE(plain.to_json().find("thoughtSignature") == nullptr); +} + +TEST(foundation, file_token_is_reread_per_request) { + char path[] = "/tmp/molo_tokenXXXXXX"; + int fd = mkstemp(path); + ASSERT_TRUE(fd >= 0); + close(fd); + auto write = [&](const std::string& t) { std::ofstream(path, std::ios::trunc) << t; }; + FileTokenAuth auth(path); + molo::clients::ClientRequest r1 = molo::clients::ClientRequest::get("https://x.googleapis.com/"); + write("tok-one\n"); + auth.apply(r1); + ASSERT_EQ(r1.headers().get_or("Authorization", ""), "Bearer tok-one"); + molo::clients::ClientRequest r2 = molo::clients::ClientRequest::get("https://x.googleapis.com/"); + write("tok-two"); + auth.apply(r2); + ASSERT_EQ(r2.headers().get_or("Authorization", ""), "Bearer tok-two"); + write(""); + molo::clients::ClientRequest r3 = molo::clients::ClientRequest::get("https://x.googleapis.com/"); + ASSERT_THROW(auth.apply(r3), GoogleError); + unlink(path); +}