feat(clients): sessions+registry+optimize+armor: close gaps, loud list truncation, shared Gemini adapter

sessions: operations get/list/cancel/delete/wait for engine, session, memory and
serving profile; servingProfiles CRUD. registry: ListResult with truncated flag,
v1alpha twins, gateway/policy operations, authz IAM, deny policy update, boundary
patch/bindings. optimize: make_gemini_model over models::VertexModel (DirectModel
removed), generateSyntheticData, online evaluator operations, trace/log writers.
armor: SecurityAdmin for all SCC v2 methods (attack exposure excluded), documented
agent threat categories, floor settings fixed to location global after live run.
check_discovery.py of all four enforce coverage.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 05:43:13 +02:00
co-authored by Claude Sonnet 5.5
parent 29d775a923
commit a3396bfc68
83 changed files with 5020 additions and 378 deletions
+54 -12
View File
@@ -34,7 +34,9 @@ optimizer; those are local abstractions and are named local in code and here.
| `observability.h/.cpp` | `TraceReader`, `LogReader`, `Trace`, `Span`, `LogEntry`, `ConversationBuilder`, `summarize` |
| `conversation.h/.cpp` | `Conversation`, `Turn`, transcript parse and render |
| `model_json.h/.cpp` | `JsonModelCall`: one JSON object from a `LanguageModel`, one retry |
| `direct_model.h/.cpp` | `DirectModel : LanguageModel` over `GoogleClient` (private to this module) |
| `model_factory.h/.cpp` | `make_gemini_model`: the one Gemini adapter, `models::VertexModel` from `src/google/models`, target picked from the credentials (the former private `DirectModel` is gone) |
| `operations_api.h/.cpp` | `OnlineEvaluatorOperations`: get, list (loud truncation flag), cancel, delete, server-side wait for online evaluator operations |
| `telemetry_writer.h/.cpp` | `TraceWriter` (patchTraces, batchWrite, createSpan) and `LogWriter` (entries:write with dryRun, entries:copy) |
| `autorater.h/.cpp` | `Autorater`, `Rubric`, `Score` |
| `scenario.h/.cpp` | `Scenario` (mirrors Google's `UserScenario`) |
| `agent.h/.cpp` | `Agent` seam, `ModelAgent` |
@@ -97,23 +99,37 @@ optimizer; those are local abstractions and are named local in code and here.
- Simulator loops are bounded by turns, tool rounds and tool calls; the
stop reason is reported (`user_done`, `max_turns`, `tool_round_cap`,
`tool_call_cap`).
- Credentials: this module never reads or prints them. `DirectModel::from_env`
picks the endpoint: OAuth token plus project gives the project-scoped
Vertex path (global location), an express key the Vertex publisher path,
otherwise Generative Language.
- Credentials: this module never reads or prints them. `make_gemini_model`
builds a `models::VertexModel` (so there is one Gemini adapter in the
platform): an OAuth token (`GOOGLE_ACCESS_TOKEN`) with a valid
`GOOGLE_CLOUD_PROJECT` gives the project scoped Vertex path in location
`global`, anything else Vertex express mode (publisher path). Behavior
change from `DirectModel`: the Generative Language endpoint (non Vertex API
keys) is no longer used; such a key now fails at Google on the Vertex path.
Streaming is real (`?alt=sse`) instead of one chunk.
- Write calls are explicit classes (`TraceWriter`, `LogWriter`), bodies are raw
Json checked against the table and capped (1 MiB, 1000 log entries, 2000
spans). `LogWriter::write` supports `dryRun`; the demo route is a dry run
unless `confirm` is `write`. `entries.tail` is excluded: an endless server
stream has no cap. `OnlineEvaluatorOperations::list` stops at 20 pages /
1000 items and says `truncated` with the next page token.
## 5. Example routes (`/agent/optimize`)
GET `/agent/optimize` (page, matrix row), GET `/agent/optimize/status`.
POST (503 `google_not_configured` without credentials or a valid
`GOOGLE_CLOUD_PROJECT`, 422 on bad input, 502 `google_error` upstream):
`evaluate`, `scenarios`, `runs`, `traces`, `logs` (Google APIs) and `score`,
`evaluate`, `scenarios`, `runs`, `traces`, `logs`, `synthetic`, `operations/list`, `operations/get`, `operations/wait`, `operations/cancel`, `logs/write` (Google APIs; DELETE `operations?name=` removes an operation record) and `score`,
`cluster`, `suggest`, `simulate` (local logic on a Gemini model).
## 6. Verified against discovery
`python3 -I src/google/optimize/tools/check_discovery.py`, run 2026-10-24:
`OK: 38 calls and 64 schemas match discovery`. It fetches
`python3 -I src/google/optimize/tools/check_discovery.py`, run 2026-10-24
(gap-closing round): `OK: 46 calls and 64 schemas match discovery`. It also
enforces coverage: every aiplatform v1 `evaluat*`, `generate*` and
`onlineEvaluators*` method, every Cloud Trace v1 and v2 method and every
`logging.entries*` method must be a table row or be listed in `EXCLUDED`
(only `logging.entries.tail`, endless stream). It fetches
`https://{aiplatform,cloudtrace,logging}.googleapis.com/$discovery/rest`
(aiplatform v1, cloudtrace v1 and v2, logging v2) and fails on any declared
call absent or different in verb, path, request fields or query parameters,
@@ -141,7 +157,21 @@ One run of the OAuth live suite showed 1 failed test of 5 (name not captured);
the immediate rerun and a later run passed 5 of 5. Treat as an unexplained
transient until reproduced.
Not run live (would create or change resources): create/patch/delete/import
Gap-closing round, 2026-10-24, service-account token and express key,
read-only (`optimize_live_gaps.*`, `optimize_live.*`):
| Call | Result |
|---|---|
| `GET onlineEvaluators/1/operations?pageSize=1` (absent evaluator) | 200, no operations |
| `GET onlineEvaluators/probe/operations` | 400 INVALID_ARGUMENT: the LRO layer wants a numeric evaluator id ("must be a Long") |
| `make_gemini_model` (`models::VertexModel`) `generate`, OAuth token, project path, location global | 200, a reply |
| `Autorater` through `make_gemini_model` with the express key (no token) | verdicts parsed, score 1 |
| `Autorater` through `make_gemini_model` with the token | verdicts parsed, score 1 |
| `evaluate_instances` exactMatch / trajectory (key), five resource lists (token) | unchanged from above: 200 |
Not run live (would create or change resources): `generateSyntheticData`, the
online evaluator operation cancel, delete and wait, `patchTraces`, `batchWrite`,
`createSpan`, `entries:write`, `entries:copy`, create/patch/delete/import
of sets, items, metrics, runs, online evaluators, `evaluateDataset`,
`generateInstanceRubrics`, `generateUserScenarios`, `generateLossClusters`.
These are verified by discovery only.
@@ -164,12 +194,24 @@ These are verified by discovery only.
- The default autorater model for LLM-judged metrics is unavailable to the
test project (see 7); `AutoraterOptions::model` is the workaround, untested.
- v1beta1 differences are not checked.
- `generateSyntheticData`: request and reply shapes are raw Json; the reply type
is not typed here and was never seen.
- Online evaluator operation names: the id of an evaluator in an operation
name must be numeric at Google's LRO layer (400 seen); evaluator ids Google
assigns were never observed because none exists.
- `entries:copy` destination string format and `entries:write` resource labels
are passed through, only top-level fields are checked.
- The shared adapter is Vertex only: a non Vertex API key (Generative Language
`AIza` key) is not routed to the Generative Language endpoint any more.
## 9. Gaps
- Cloud Trace v2 `batchWrite` and `createSpan` and Cloud Logging
`entries.write/tail` are not wrapped: agents write through OTel exporters.
The v2 rows are not in the table because nothing calls them.
- `logging.entries.tail` (endless server stream) is not wrapped; the write
calls (`TraceWriter`, `LogWriter`) exist for tools, agents normally write
through OTel exporters.
- Trace and log lists (`TraceReader::list`, `LogReader::entries`) still stop at
their caps without a `truncated` flag; only the new operations list reports
truncation. Owner: `observability.cpp`.
- Dashboards: only numbers (`DashboardData`); no Cloud Monitoring reads.
- `evaluateDataset`, `generateInstanceRubrics`, `evaluationMetrics` and item
bodies are passed as raw `Json` (checked at the top level against