feat(clients): sessions+registry+optimize+armor: close gaps, loud list truncation, shared Gemini adapter
sessions: operations get/list/cancel/delete/wait for engine, session, memory and serving profile; servingProfiles CRUD. registry: ListResult with truncated flag, v1alpha twins, gateway/policy operations, authz IAM, deny policy update, boundary patch/bindings. optimize: make_gemini_model over models::VertexModel (DirectModel removed), generateSyntheticData, online evaluator operations, trace/log writers. armor: SecurityAdmin for all SCC v2 methods (attack exposure excluded), documented agent threat categories, floor settings fixed to location global after live run. check_discovery.py of all four enforce coverage. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
+54
-12
@@ -34,7 +34,9 @@ optimizer; those are local abstractions and are named local in code and here.
|
||||
| `observability.h/.cpp` | `TraceReader`, `LogReader`, `Trace`, `Span`, `LogEntry`, `ConversationBuilder`, `summarize` |
|
||||
| `conversation.h/.cpp` | `Conversation`, `Turn`, transcript parse and render |
|
||||
| `model_json.h/.cpp` | `JsonModelCall`: one JSON object from a `LanguageModel`, one retry |
|
||||
| `direct_model.h/.cpp` | `DirectModel : LanguageModel` over `GoogleClient` (private to this module) |
|
||||
| `model_factory.h/.cpp` | `make_gemini_model`: the one Gemini adapter, `models::VertexModel` from `src/google/models`, target picked from the credentials (the former private `DirectModel` is gone) |
|
||||
| `operations_api.h/.cpp` | `OnlineEvaluatorOperations`: get, list (loud truncation flag), cancel, delete, server-side wait for online evaluator operations |
|
||||
| `telemetry_writer.h/.cpp` | `TraceWriter` (patchTraces, batchWrite, createSpan) and `LogWriter` (entries:write with dryRun, entries:copy) |
|
||||
| `autorater.h/.cpp` | `Autorater`, `Rubric`, `Score` |
|
||||
| `scenario.h/.cpp` | `Scenario` (mirrors Google's `UserScenario`) |
|
||||
| `agent.h/.cpp` | `Agent` seam, `ModelAgent` |
|
||||
@@ -97,23 +99,37 @@ optimizer; those are local abstractions and are named local in code and here.
|
||||
- Simulator loops are bounded by turns, tool rounds and tool calls; the
|
||||
stop reason is reported (`user_done`, `max_turns`, `tool_round_cap`,
|
||||
`tool_call_cap`).
|
||||
- Credentials: this module never reads or prints them. `DirectModel::from_env`
|
||||
picks the endpoint: OAuth token plus project gives the project-scoped
|
||||
Vertex path (global location), an express key the Vertex publisher path,
|
||||
otherwise Generative Language.
|
||||
- Credentials: this module never reads or prints them. `make_gemini_model`
|
||||
builds a `models::VertexModel` (so there is one Gemini adapter in the
|
||||
platform): an OAuth token (`GOOGLE_ACCESS_TOKEN`) with a valid
|
||||
`GOOGLE_CLOUD_PROJECT` gives the project scoped Vertex path in location
|
||||
`global`, anything else Vertex express mode (publisher path). Behavior
|
||||
change from `DirectModel`: the Generative Language endpoint (non Vertex API
|
||||
keys) is no longer used; such a key now fails at Google on the Vertex path.
|
||||
Streaming is real (`?alt=sse`) instead of one chunk.
|
||||
- Write calls are explicit classes (`TraceWriter`, `LogWriter`), bodies are raw
|
||||
Json checked against the table and capped (1 MiB, 1000 log entries, 2000
|
||||
spans). `LogWriter::write` supports `dryRun`; the demo route is a dry run
|
||||
unless `confirm` is `write`. `entries.tail` is excluded: an endless server
|
||||
stream has no cap. `OnlineEvaluatorOperations::list` stops at 20 pages /
|
||||
1000 items and says `truncated` with the next page token.
|
||||
|
||||
## 5. Example routes (`/agent/optimize`)
|
||||
|
||||
GET `/agent/optimize` (page, matrix row), GET `/agent/optimize/status`.
|
||||
POST (503 `google_not_configured` without credentials or a valid
|
||||
`GOOGLE_CLOUD_PROJECT`, 422 on bad input, 502 `google_error` upstream):
|
||||
`evaluate`, `scenarios`, `runs`, `traces`, `logs` (Google APIs) and `score`,
|
||||
`evaluate`, `scenarios`, `runs`, `traces`, `logs`, `synthetic`, `operations/list`, `operations/get`, `operations/wait`, `operations/cancel`, `logs/write` (Google APIs; DELETE `operations?name=` removes an operation record) and `score`,
|
||||
`cluster`, `suggest`, `simulate` (local logic on a Gemini model).
|
||||
|
||||
## 6. Verified against discovery
|
||||
|
||||
`python3 -I src/google/optimize/tools/check_discovery.py`, run 2026-10-24:
|
||||
`OK: 38 calls and 64 schemas match discovery`. It fetches
|
||||
`python3 -I src/google/optimize/tools/check_discovery.py`, run 2026-10-24
|
||||
(gap-closing round): `OK: 46 calls and 64 schemas match discovery`. It also
|
||||
enforces coverage: every aiplatform v1 `evaluat*`, `generate*` and
|
||||
`onlineEvaluators*` method, every Cloud Trace v1 and v2 method and every
|
||||
`logging.entries*` method must be a table row or be listed in `EXCLUDED`
|
||||
(only `logging.entries.tail`, endless stream). It fetches
|
||||
`https://{aiplatform,cloudtrace,logging}.googleapis.com/$discovery/rest`
|
||||
(aiplatform v1, cloudtrace v1 and v2, logging v2) and fails on any declared
|
||||
call absent or different in verb, path, request fields or query parameters,
|
||||
@@ -141,7 +157,21 @@ One run of the OAuth live suite showed 1 failed test of 5 (name not captured);
|
||||
the immediate rerun and a later run passed 5 of 5. Treat as an unexplained
|
||||
transient until reproduced.
|
||||
|
||||
Not run live (would create or change resources): create/patch/delete/import
|
||||
Gap-closing round, 2026-10-24, service-account token and express key,
|
||||
read-only (`optimize_live_gaps.*`, `optimize_live.*`):
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| `GET onlineEvaluators/1/operations?pageSize=1` (absent evaluator) | 200, no operations |
|
||||
| `GET onlineEvaluators/probe/operations` | 400 INVALID_ARGUMENT: the LRO layer wants a numeric evaluator id ("must be a Long") |
|
||||
| `make_gemini_model` (`models::VertexModel`) `generate`, OAuth token, project path, location global | 200, a reply |
|
||||
| `Autorater` through `make_gemini_model` with the express key (no token) | verdicts parsed, score 1 |
|
||||
| `Autorater` through `make_gemini_model` with the token | verdicts parsed, score 1 |
|
||||
| `evaluate_instances` exactMatch / trajectory (key), five resource lists (token) | unchanged from above: 200 |
|
||||
|
||||
Not run live (would create or change resources): `generateSyntheticData`, the
|
||||
online evaluator operation cancel, delete and wait, `patchTraces`, `batchWrite`,
|
||||
`createSpan`, `entries:write`, `entries:copy`, create/patch/delete/import
|
||||
of sets, items, metrics, runs, online evaluators, `evaluateDataset`,
|
||||
`generateInstanceRubrics`, `generateUserScenarios`, `generateLossClusters`.
|
||||
These are verified by discovery only.
|
||||
@@ -164,12 +194,24 @@ These are verified by discovery only.
|
||||
- The default autorater model for LLM-judged metrics is unavailable to the
|
||||
test project (see 7); `AutoraterOptions::model` is the workaround, untested.
|
||||
- v1beta1 differences are not checked.
|
||||
- `generateSyntheticData`: request and reply shapes are raw Json; the reply type
|
||||
is not typed here and was never seen.
|
||||
- Online evaluator operation names: the id of an evaluator in an operation
|
||||
name must be numeric at Google's LRO layer (400 seen); evaluator ids Google
|
||||
assigns were never observed because none exists.
|
||||
- `entries:copy` destination string format and `entries:write` resource labels
|
||||
are passed through, only top-level fields are checked.
|
||||
- The shared adapter is Vertex only: a non Vertex API key (Generative Language
|
||||
`AIza` key) is not routed to the Generative Language endpoint any more.
|
||||
|
||||
## 9. Gaps
|
||||
|
||||
- Cloud Trace v2 `batchWrite` and `createSpan` and Cloud Logging
|
||||
`entries.write/tail` are not wrapped: agents write through OTel exporters.
|
||||
The v2 rows are not in the table because nothing calls them.
|
||||
- `logging.entries.tail` (endless server stream) is not wrapped; the write
|
||||
calls (`TraceWriter`, `LogWriter`) exist for tools, agents normally write
|
||||
through OTel exporters.
|
||||
- Trace and log lists (`TraceReader::list`, `LogReader::entries`) still stop at
|
||||
their caps without a `truncated` flag; only the new operations list reports
|
||||
truncation. Owner: `observability.cpp`.
|
||||
- Dashboards: only numbers (`DashboardData`); no Cloud Monitoring reads.
|
||||
- `evaluateDataset`, `generateInstanceRubrics`, `evaluationMetrics` and item
|
||||
bodies are passed as raw `Json` (checked at the top level against
|
||||
|
||||
Reference in New Issue
Block a user