feat(clients): models+runtime: close gaps (all Discovery methods wrapped, live re-verification)

models: EndpointAdmin and PublisherAdmin wrap every remaining method of the
owned resources (84 call table rows, 132 Discovery methods covered, v1beta1
twins, IAM, operations, responses, export, config, enableModel); planner
rejects undeclared body keys; VertexModel::TurnBuilder replays model turns
with every thoughtSignature; Lyria WAV mime from bytes; veo-3.1 default.
runtime: v1beta1 feedbackEntries, A2A gateway (40 rpcs), a2aTasks, task store
records, scoped operations, beta twin flag (103 rows, 214 methods covered);
checker verifies twins, a2aEndpoint patterns and coverage.
example: admin routes for both parts. SPECs: live results moved from
Unverified to Verified live (Veo, Lyria, thought signatures, v1beta1 routes).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 05:47:34 +02:00
co-authored by Claude Sonnet 5.5
parent 29d775a923
commit 8e807945a2
38 changed files with 4437 additions and 72 deletions
+145 -23
View File
@@ -30,7 +30,10 @@ Two addressing modes, one class (`VertexTarget`):
| `vertex_service.h/.cpp` | `VertexService : Service`: runs planned calls |
| `json_read.h` | tolerant response readers |
| `gemini.h/.cpp` | `Gemini`: generate, stream (`?alt=sse`), count/compute tokens, embedContent, predict embeddings |
| `vertex_model.h/.cpp` | `VertexModel : LanguageModel` |
| `vertex_model.h/.cpp` | `VertexModel : LanguageModel`, `TurnBuilder` (replayable model turn with every `thoughtSignature`) |
| `admin_checks.h` | Input rules shared by the admin classes (object, size, mask, confirm) |
| `endpoint_admin.h/.cpp` | `EndpointAdmin`: endpoint lifecycle, remaining inference surfaces, endpoint operations, v1beta1 IAM and list |
| `publisher_admin.h/.cpp` | `PublisherAdmin`: publisher `v1/responses`, serverStreamingPredict, export, config, enableModel, IAM, v1beta1 forms |
| `model_garden.h/.cpp` | `ModelGarden`: list, get, fetch config, EULA check/accept, deploy |
| `claude_models.h/.cpp` | `ClaudeModels`: rawPredict, streamRawPredict, `text_delta` |
| `open_models.h/.cpp` | `OpenModels`: endpoints list/get, predict, rawPredict, streamRawPredict, serverStreamingPredict, generate/stream/count on endpoints, predictLongRunning, fetchPredictOperation, publisher invoke |
@@ -49,7 +52,9 @@ dimensions; list page size 1..100, 20 pages, 2000 models returned; Claude
200 messages, 1,000,000 chars, max_tokens 1..32768; endpoint instances 16,
100,000 chars, Gemma max_tokens 8192; media prompt 4000 chars, images 4,
music samples 4, video samples 4, video 8 seconds, 32 MiB and 8 items per
result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
result, operation name 512 chars; video polls 120 at 5 s (10 minutes). Admin surface: request body 262144
chars serialized, updateMask 512 chars, operations.wait 1..600 s, operation
list pages 1..5, 64 permissions, 5000 streamed events.
## 4. Contracts
@@ -68,11 +73,36 @@ result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
(fetchPredictOperation, a POST), `wait_video` bounded by the caps.
`wait_operation` (GET) is not used: Discovery has no GET on these
operations for publisher models.
- Admin bodies are Json whose top-level keys must be declared in the call
table: `CallPlanner::plan` rejects a stray key (`INVALID_ARGUMENT`) before
any traffic; rows with `-` (raw HTTP bodies) take any object.
- `EndpointAdmin(beta=true)` sends the v1beta1 twin of every `endpoint.` row
(`CallPlanner::plan(..., beta=true)`); `tools/check_discovery.py` verifies
each twin (same method, path, rpc id, fields). `endpoint.beta_*` rows exist
in v1beta1 only (list with `gdcZone`, IAM).
- Gated admin calls refuse without `confirm=true`: endpoint delete, deployModel,
undeployModel, mutateDeployedModel, IAM set, stored response delete,
operation record delete, publisher export, setPublisherModelConfig,
enableModel. The example routes answer a dry run instead.
- `VertexModel::TurnBuilder` builds the model turn to replay from stream
chunks: text parts merge only while they carry no `thoughtSignature`, a
signature that arrives on an empty part attaches to the part before it,
calls and data parts are kept as received. `VertexModel::model_turn` and
`replay` do the same for a whole reply. `Gemini::validate` accepts a
signature-only part (it is not "empty").
- Example (`/agent/models`): GET page, `/status`, `/calls` need no
credential; every Google call is a POST that answers 503
`google_not_configured` without a credential (checked first), 422 on bad
input. Routes: generate, count, embed, claude, list, endpoint, image,
music, video, video/status.
music, video, video/status, and the admin routes `endpoints` (create,
patch, update, delete, deploy, undeploy, mutate, list_beta, get_iam,
set_iam, test_iam), `endpoint/call` (compute_tokens, direct_predict,
direct_raw_predict, explain, invoke, deployed_invoke, chat, openapi,
science, response_get, response_delete), `endpoint/operations` (get, list,
cancel, wait, delete), `publisher` (stream_predict, responses_*, export,
set_config, enable, iam, beta_get, beta_generate, beta_count, beta_post,
beta_invoke). The video default model is `veo-3.1-generate-001`
(`veo-3.0-generate-001` answers 404 for the project).
## 5. Verified against discovery
@@ -80,7 +110,18 @@ result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
`https://aiplatform.googleapis.com/$discovery/rest?version=v1` and `v1beta1`
and fails when any row of `calls.h` has a missing method or path, a wrong
rpc id, an absent request body field (dotted names follow the schema) or an
absent query parameter. Result on 2026-10-24: `29 calls checked, 0 problems`.
absent query parameter. Result on 2026-10-24: `84 calls checked, 132 owned
discovery methods covered, 0 problems`.
Coverage is part of the check: every Discovery method whose rpc id starts with
`aiplatform.publishers.models.`, `aiplatform.projects.locations.publishers.`,
`aiplatform.projects.publishers.models.`, `aiplatform.projects.locations.endpoints.`,
`aiplatform.projects.modelGardenEula.` or `aiplatform.projects.locations.deployPublisherModel`
(v1 and v1beta1, 132 methods) must be a row, the express form of a row, or the
v1beta1 twin of an `endpoint.` row (verified field by field); the check fails
on any other method until it is wrapped or excluded on purpose (`EXCLUDED`,
empty today). So a Discovery change that adds a method to these resources is
loud.
The same pass shows what Discovery says about express forms:
generateContent, streamGenerateContent, countTokens, computeTokens, predict,
predictLongRunning, fetchPredictOperation exist without a project;
@@ -89,8 +130,9 @@ embedContent, rawPredict, streamRawPredict, publisher invoke do not
only in v1beta1; `publishers.models.get` in v1 and v1beta1.
Request construction is compared to the table in
`tests/src/google/models/test_models.cpp` (every body key must be a declared
field).
`tests/src/google/models/test_models.cpp` and `test_models_admin.cpp` (every
body key must be a declared field; the planner also rejects a stray key at
run time; every row plans to a concrete URL, express and project scoped).
## 6. Verified live
@@ -124,9 +166,68 @@ With the service account OAuth token (project `lisa-448004`):
- Generation calls may answer 429 on the shared project; the live test notes
that instead of failing.
Not live-verified (mutating or billable): deploy, EULA accept, image, music
and video generation, endpoint predict family (no endpoint exists), Gemma
shapes. These are verified by Discovery only.
Run 2026-10-24 (closing the gaps), service account token, project
`lisa-448004`, through the new classes (`models_live_admin.*`) or direct
requests:
- `EndpointAdmin` read-only: v1beta1 endpoints list (`us-central1`): 200 `{}`
(an empty list has no `endpoints` key); v1 and v1beta1 operations list of a
missing endpoint: 200 `{}`; v1beta1 `getIamPolicy`, `operations.get` on a
missing endpoint: 404 NOT_FOUND; stored responses
(`endpoints/{id}/responses/{id}` get): 501 UNIMPLEMENTED for the project.
- `PublisherAdmin`: v1beta1 `publishers/google/models/gemini-2.5-flash` get: 200;
project scoped v1beta1 `getIamPolicy` on a publisher model: 400
INVALID_ARGUMENT (nothing to read there).
- Thought signatures: `gemini-3.5-flash` at `global` with a function
declaration, streamed: the `functionCall` part carries a `thoughtSignature`
(`TurnBuilder` keeps 1 signed part). The stream ends with an empty text
part, which `TurnBuilder` drops. The turn replayed with a `functionResponse`
via `VertexModel::replay` is accepted (200, "The weather in Paris is 18°C.").
Whether Google rejects a replay that lacks the signature was not tested.
- Veo: `veo-3.1-generate-001` exists for the project (us-central1 and
`global`); `veo-3.0-generate-001`, `veo-3.0-fast-generate-001`,
`veo-2.0-generate-001` and every Imagen model tried
(`imagen-3.0-generate-001/002`, `imagen-3.0-fast-generate-001`,
`imagen-4.0-generate-001`, `imagen-4.0-fast-generate-001`,
`imagen-4.0-ultra-generate-001`) answer 404 "not found or your project
does not have access". `predictLongRunning` accepts `instances[{prompt}]`
(an instance with `text` instead: 400 "No inputs provided") and answers
`{"name": "projects/.../publishers/google/models/veo-3.1-generate-001/operations/<uuid>"}`.
Parameters are validated inside the operation, which finishes
`done:true` with `error{code:3,message}` on `fetchPredictOperation`:
`sampleCount` 99 -> "Invalid sample count 99. The sample count should be in
the range of [1, 4]" (name and range match our cap 1..4); `durationSeconds`
99 -> "Unsupported output video duration 99 seconds, supported durations are
[8,4,6] for feature text_to_video" (name confirmed; the model accepts only
4, 6, 8, our cap 1..8 is wider); `storageUri` "bogus" -> "Unsupported output
storage uri: bogus" (name confirmed); `aspectRatio` "bogus" is rejected
synchronously with 400 "Invalid aspect ratio: bogus".
- A finished Veo operation, read with `fetchPredictOperation`: `done:true`,
`response` = `{"@type":"type.googleapis.com/cloud.ai.large_models.vision.GenerateVideoResponse",
"raiMediaFilteredCount":0, "videos":[{"bytesBase64Encoded":"...","mimeType":"video/mp4"}]}`
(one 8 s clip, about 13.8 MB decoded, under the 32 MiB cap). This is the
real shape of the operation `response` Any for video and is what
`wait_video` reads.
- Lyria: `lyria-002` `predict` (project scoped, us-central1 and `global`):
`instances[{prompt}]` -> 200 `{"predictions":[{"bytesBase64Encoded":"UklGR..."}]}`
with no `mimeType`; the bytes are a RIFF/WAVE file, so `items_from` names
`audio/wav` when the mime is missing. An empty prompt or an instance
without `prompt` -> 400 "Request contains an invalid argument".
- Claude `rawPredict` and `streamRawPredict` at `global`: still 429
RESOURCE_EXHAUSTED on quota `global_online_prediction_requests_per_base_model`
(`anthropic-claude-sonnet-4-6`). The streaming call reports the error as a JSON
array `[{"error":{...}}]`, not as one object.
Cost note: the Veo and Lyria probes were meant to be rejected requests. Google
accepted several, so the run started 5 Veo operations (4 failed validation
inside the operation and cost nothing, one `generateAudio:"str"` probe rendered
one 8 s clip) and generated 3 Lyria clips. The probe scripts are not part of
the repo; the live tests in the repo never call a generation method.
Not live-verified (mutating or billable): deploy, EULA accept, image
generation (no Imagen model is visible to the project), endpoint create, patch,
update, delete, deployModel, undeployModel, mutateDeployedModel, IAM set,
publisher export, config and enableModel, the endpoint inference family (no
endpoint exists). These are verified by Discovery and by local request tests.
## 7. Sources
@@ -136,28 +237,49 @@ shapes. These are verified by Discovery only.
## 8. Unverified (neither Discovery nor docs read, nor live)
Settled on 2026-10-24 and moved to section 6: Veo `prompt`, `sampleCount`,
`durationSeconds`, `aspectRatio`, `storageUri` and the finished operation
`response`; Lyria `prompt` and the `predictions[].bytesBase64Encoded` WAV
response; the empty endpoint list shape; thought signatures on function calls.
- Imagen `predict` instance/parameter names (`prompt`; `sampleCount`,
`aspectRatio`, `negativePrompt`) and the `predictions[].bytesBase64Encoded`
/ `mimeType` response: Discovery types them as free-form `Value`; the doc
pages could not be read.
- Lyria shape (`prompt`, `negative_prompt`, `seed`; `sample_count`).
- Veo shape (`sampleCount`, `durationSeconds`, `aspectRatio`, `storageUri`;
final `response.videos[]` with `gcsUri` / `bytesBase64Encoded` / `mimeType`).
`fetchPredictOperation` itself is Discovery-verified and reachable live.
/ `mimeType` response: no Imagen model is visible to project `lisa-448004`
(404 for every id tried), so even a rejected request is not informative.
- Lyria `negative_prompt`, `seed` and `sample_count`: a live call accepted
them, but it also accepted junk values (`seed:"abc"`, `sample_count:99`), so
acceptance does not prove the names. Whether a clip count above 1 is
honored was not checked.
- Veo `generateAudio` and the `gcsUri` result form with a real `storageUri`
bucket (no bucket was used). A missing `mimeType` on Veo output was not seen.
- Gemma/vLLM instance shape in `gemma_instance` (`prompt`, `max_tokens`,
`temperature`): the serving container's, not Google's API.
`temperature`): the serving container's, not Google's API; no endpoint
exists to ask.
- Anthropic streaming event names (`content_block_delta` / `text_delta`)
come from Anthropic's format; Claude completions were not obtained live.
- Embedding predict response `embeddings.values` / `statistics.token_count`
was observed live for text-embedding-005, not documented-read.
come from Anthropic's format; Claude completions were not obtained live
(429 quota on every attempt).
- `deployPublisherModel` field semantics beyond the schema names.
- Response shapes of the admin methods that need a real endpoint or a
mutation: deployModel, directPredict, directRawPredict, explain, chat
completions, openapi, invoke, stored responses (501 for this project),
export, setPublisherModelConfig, enableModel. They are returned as raw Json
(no field is read), so a different shape cannot break the client.
- Whether Google rejects a Gemini 3 replay that omits the `thoughtSignature`
(the accepted replay with it is verified).
## 9. Gaps
- Discovery methods deliberately not wrapped (owner: this module): publisher
`export`, `setPublisherModelConfig`, `enableModel`, IAM policy, `v1/responses`
on publishers, publisher-level `serverStreamingPredict` and v1beta1
`publishers.models.*` project-less duplicates.
- Every Discovery method of the owned resources is in the call table (see
section 5); the only intentional difference is that project-less v1 forms
are the `express` forms of rows and the v1beta1 forms of endpoint rows are
twins selected with `EndpointAdmin(beta=true)`.
- The admin request bodies are validated by top-level key against the call
table and by caps; nested fields (a `DeployedModel`, a `PublisherModelConfig`)
are Json the caller owns. Typed structs exist where the module reads fields
(`EndpointInfo`, `Operation`, `PublisherModel`).
- Endpoint `generateContent` family is wired but only Discovery-verified.
- `VertexModel` has no retry; quota 429 surfaces as `GoogleError`.
- A streaming error body shaped as a JSON array (`[{"error":...}]`, seen on
Claude `streamRawPredict`) goes through the foundation `StreamDecoder`, which
is outside this module; not verified that it maps to `GoogleError`.
- Token refresh is the caller's job (foundation gap).