feat(clients): models+runtime: close gaps (all Discovery methods wrapped, live re-verification)
models: EndpointAdmin and PublisherAdmin wrap every remaining method of the owned resources (84 call table rows, 132 Discovery methods covered, v1beta1 twins, IAM, operations, responses, export, config, enableModel); planner rejects undeclared body keys; VertexModel::TurnBuilder replays model turns with every thoughtSignature; Lyria WAV mime from bytes; veo-3.1 default. runtime: v1beta1 feedbackEntries, A2A gateway (40 rpcs), a2aTasks, task store records, scoped operations, beta twin flag (103 rows, 214 methods covered); checker verifies twins, a2aEndpoint patterns and coverage. example: admin routes for both parts. SPECs: live results moved from Unverified to Verified live (Veo, Lyria, thought signatures, v1beta1 routes). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
+145
-23
@@ -30,7 +30,10 @@ Two addressing modes, one class (`VertexTarget`):
|
||||
| `vertex_service.h/.cpp` | `VertexService : Service`: runs planned calls |
|
||||
| `json_read.h` | tolerant response readers |
|
||||
| `gemini.h/.cpp` | `Gemini`: generate, stream (`?alt=sse`), count/compute tokens, embedContent, predict embeddings |
|
||||
| `vertex_model.h/.cpp` | `VertexModel : LanguageModel` |
|
||||
| `vertex_model.h/.cpp` | `VertexModel : LanguageModel`, `TurnBuilder` (replayable model turn with every `thoughtSignature`) |
|
||||
| `admin_checks.h` | Input rules shared by the admin classes (object, size, mask, confirm) |
|
||||
| `endpoint_admin.h/.cpp` | `EndpointAdmin`: endpoint lifecycle, remaining inference surfaces, endpoint operations, v1beta1 IAM and list |
|
||||
| `publisher_admin.h/.cpp` | `PublisherAdmin`: publisher `v1/responses`, serverStreamingPredict, export, config, enableModel, IAM, v1beta1 forms |
|
||||
| `model_garden.h/.cpp` | `ModelGarden`: list, get, fetch config, EULA check/accept, deploy |
|
||||
| `claude_models.h/.cpp` | `ClaudeModels`: rawPredict, streamRawPredict, `text_delta` |
|
||||
| `open_models.h/.cpp` | `OpenModels`: endpoints list/get, predict, rawPredict, streamRawPredict, serverStreamingPredict, generate/stream/count on endpoints, predictLongRunning, fetchPredictOperation, publisher invoke |
|
||||
@@ -49,7 +52,9 @@ dimensions; list page size 1..100, 20 pages, 2000 models returned; Claude
|
||||
200 messages, 1,000,000 chars, max_tokens 1..32768; endpoint instances 16,
|
||||
100,000 chars, Gemma max_tokens 8192; media prompt 4000 chars, images 4,
|
||||
music samples 4, video samples 4, video 8 seconds, 32 MiB and 8 items per
|
||||
result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
|
||||
result, operation name 512 chars; video polls 120 at 5 s (10 minutes). Admin surface: request body 262144
|
||||
chars serialized, updateMask 512 chars, operations.wait 1..600 s, operation
|
||||
list pages 1..5, 64 permissions, 5000 streamed events.
|
||||
|
||||
## 4. Contracts
|
||||
|
||||
@@ -68,11 +73,36 @@ result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
|
||||
(fetchPredictOperation, a POST), `wait_video` bounded by the caps.
|
||||
`wait_operation` (GET) is not used: Discovery has no GET on these
|
||||
operations for publisher models.
|
||||
- Admin bodies are Json whose top-level keys must be declared in the call
|
||||
table: `CallPlanner::plan` rejects a stray key (`INVALID_ARGUMENT`) before
|
||||
any traffic; rows with `-` (raw HTTP bodies) take any object.
|
||||
- `EndpointAdmin(beta=true)` sends the v1beta1 twin of every `endpoint.` row
|
||||
(`CallPlanner::plan(..., beta=true)`); `tools/check_discovery.py` verifies
|
||||
each twin (same method, path, rpc id, fields). `endpoint.beta_*` rows exist
|
||||
in v1beta1 only (list with `gdcZone`, IAM).
|
||||
- Gated admin calls refuse without `confirm=true`: endpoint delete, deployModel,
|
||||
undeployModel, mutateDeployedModel, IAM set, stored response delete,
|
||||
operation record delete, publisher export, setPublisherModelConfig,
|
||||
enableModel. The example routes answer a dry run instead.
|
||||
- `VertexModel::TurnBuilder` builds the model turn to replay from stream
|
||||
chunks: text parts merge only while they carry no `thoughtSignature`, a
|
||||
signature that arrives on an empty part attaches to the part before it,
|
||||
calls and data parts are kept as received. `VertexModel::model_turn` and
|
||||
`replay` do the same for a whole reply. `Gemini::validate` accepts a
|
||||
signature-only part (it is not "empty").
|
||||
- Example (`/agent/models`): GET page, `/status`, `/calls` need no
|
||||
credential; every Google call is a POST that answers 503
|
||||
`google_not_configured` without a credential (checked first), 422 on bad
|
||||
input. Routes: generate, count, embed, claude, list, endpoint, image,
|
||||
music, video, video/status.
|
||||
music, video, video/status, and the admin routes `endpoints` (create,
|
||||
patch, update, delete, deploy, undeploy, mutate, list_beta, get_iam,
|
||||
set_iam, test_iam), `endpoint/call` (compute_tokens, direct_predict,
|
||||
direct_raw_predict, explain, invoke, deployed_invoke, chat, openapi,
|
||||
science, response_get, response_delete), `endpoint/operations` (get, list,
|
||||
cancel, wait, delete), `publisher` (stream_predict, responses_*, export,
|
||||
set_config, enable, iam, beta_get, beta_generate, beta_count, beta_post,
|
||||
beta_invoke). The video default model is `veo-3.1-generate-001`
|
||||
(`veo-3.0-generate-001` answers 404 for the project).
|
||||
|
||||
## 5. Verified against discovery
|
||||
|
||||
@@ -80,7 +110,18 @@ result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
|
||||
`https://aiplatform.googleapis.com/$discovery/rest?version=v1` and `v1beta1`
|
||||
and fails when any row of `calls.h` has a missing method or path, a wrong
|
||||
rpc id, an absent request body field (dotted names follow the schema) or an
|
||||
absent query parameter. Result on 2026-10-24: `29 calls checked, 0 problems`.
|
||||
absent query parameter. Result on 2026-10-24: `84 calls checked, 132 owned
|
||||
discovery methods covered, 0 problems`.
|
||||
|
||||
Coverage is part of the check: every Discovery method whose rpc id starts with
|
||||
`aiplatform.publishers.models.`, `aiplatform.projects.locations.publishers.`,
|
||||
`aiplatform.projects.publishers.models.`, `aiplatform.projects.locations.endpoints.`,
|
||||
`aiplatform.projects.modelGardenEula.` or `aiplatform.projects.locations.deployPublisherModel`
|
||||
(v1 and v1beta1, 132 methods) must be a row, the express form of a row, or the
|
||||
v1beta1 twin of an `endpoint.` row (verified field by field); the check fails
|
||||
on any other method until it is wrapped or excluded on purpose (`EXCLUDED`,
|
||||
empty today). So a Discovery change that adds a method to these resources is
|
||||
loud.
|
||||
The same pass shows what Discovery says about express forms:
|
||||
generateContent, streamGenerateContent, countTokens, computeTokens, predict,
|
||||
predictLongRunning, fetchPredictOperation exist without a project;
|
||||
@@ -89,8 +130,9 @@ embedContent, rawPredict, streamRawPredict, publisher invoke do not
|
||||
only in v1beta1; `publishers.models.get` in v1 and v1beta1.
|
||||
|
||||
Request construction is compared to the table in
|
||||
`tests/src/google/models/test_models.cpp` (every body key must be a declared
|
||||
field).
|
||||
`tests/src/google/models/test_models.cpp` and `test_models_admin.cpp` (every
|
||||
body key must be a declared field; the planner also rejects a stray key at
|
||||
run time; every row plans to a concrete URL, express and project scoped).
|
||||
|
||||
## 6. Verified live
|
||||
|
||||
@@ -124,9 +166,68 @@ With the service account OAuth token (project `lisa-448004`):
|
||||
- Generation calls may answer 429 on the shared project; the live test notes
|
||||
that instead of failing.
|
||||
|
||||
Not live-verified (mutating or billable): deploy, EULA accept, image, music
|
||||
and video generation, endpoint predict family (no endpoint exists), Gemma
|
||||
shapes. These are verified by Discovery only.
|
||||
Run 2026-10-24 (closing the gaps), service account token, project
|
||||
`lisa-448004`, through the new classes (`models_live_admin.*`) or direct
|
||||
requests:
|
||||
- `EndpointAdmin` read-only: v1beta1 endpoints list (`us-central1`): 200 `{}`
|
||||
(an empty list has no `endpoints` key); v1 and v1beta1 operations list of a
|
||||
missing endpoint: 200 `{}`; v1beta1 `getIamPolicy`, `operations.get` on a
|
||||
missing endpoint: 404 NOT_FOUND; stored responses
|
||||
(`endpoints/{id}/responses/{id}` get): 501 UNIMPLEMENTED for the project.
|
||||
- `PublisherAdmin`: v1beta1 `publishers/google/models/gemini-2.5-flash` get: 200;
|
||||
project scoped v1beta1 `getIamPolicy` on a publisher model: 400
|
||||
INVALID_ARGUMENT (nothing to read there).
|
||||
- Thought signatures: `gemini-3.5-flash` at `global` with a function
|
||||
declaration, streamed: the `functionCall` part carries a `thoughtSignature`
|
||||
(`TurnBuilder` keeps 1 signed part). The stream ends with an empty text
|
||||
part, which `TurnBuilder` drops. The turn replayed with a `functionResponse`
|
||||
via `VertexModel::replay` is accepted (200, "The weather in Paris is 18°C.").
|
||||
Whether Google rejects a replay that lacks the signature was not tested.
|
||||
- Veo: `veo-3.1-generate-001` exists for the project (us-central1 and
|
||||
`global`); `veo-3.0-generate-001`, `veo-3.0-fast-generate-001`,
|
||||
`veo-2.0-generate-001` and every Imagen model tried
|
||||
(`imagen-3.0-generate-001/002`, `imagen-3.0-fast-generate-001`,
|
||||
`imagen-4.0-generate-001`, `imagen-4.0-fast-generate-001`,
|
||||
`imagen-4.0-ultra-generate-001`) answer 404 "not found or your project
|
||||
does not have access". `predictLongRunning` accepts `instances[{prompt}]`
|
||||
(an instance with `text` instead: 400 "No inputs provided") and answers
|
||||
`{"name": "projects/.../publishers/google/models/veo-3.1-generate-001/operations/<uuid>"}`.
|
||||
Parameters are validated inside the operation, which finishes
|
||||
`done:true` with `error{code:3,message}` on `fetchPredictOperation`:
|
||||
`sampleCount` 99 -> "Invalid sample count 99. The sample count should be in
|
||||
the range of [1, 4]" (name and range match our cap 1..4); `durationSeconds`
|
||||
99 -> "Unsupported output video duration 99 seconds, supported durations are
|
||||
[8,4,6] for feature text_to_video" (name confirmed; the model accepts only
|
||||
4, 6, 8, our cap 1..8 is wider); `storageUri` "bogus" -> "Unsupported output
|
||||
storage uri: bogus" (name confirmed); `aspectRatio` "bogus" is rejected
|
||||
synchronously with 400 "Invalid aspect ratio: bogus".
|
||||
- A finished Veo operation, read with `fetchPredictOperation`: `done:true`,
|
||||
`response` = `{"@type":"type.googleapis.com/cloud.ai.large_models.vision.GenerateVideoResponse",
|
||||
"raiMediaFilteredCount":0, "videos":[{"bytesBase64Encoded":"...","mimeType":"video/mp4"}]}`
|
||||
(one 8 s clip, about 13.8 MB decoded, under the 32 MiB cap). This is the
|
||||
real shape of the operation `response` Any for video and is what
|
||||
`wait_video` reads.
|
||||
- Lyria: `lyria-002` `predict` (project scoped, us-central1 and `global`):
|
||||
`instances[{prompt}]` -> 200 `{"predictions":[{"bytesBase64Encoded":"UklGR..."}]}`
|
||||
with no `mimeType`; the bytes are a RIFF/WAVE file, so `items_from` names
|
||||
`audio/wav` when the mime is missing. An empty prompt or an instance
|
||||
without `prompt` -> 400 "Request contains an invalid argument".
|
||||
- Claude `rawPredict` and `streamRawPredict` at `global`: still 429
|
||||
RESOURCE_EXHAUSTED on quota `global_online_prediction_requests_per_base_model`
|
||||
(`anthropic-claude-sonnet-4-6`). The streaming call reports the error as a JSON
|
||||
array `[{"error":{...}}]`, not as one object.
|
||||
|
||||
Cost note: the Veo and Lyria probes were meant to be rejected requests. Google
|
||||
accepted several, so the run started 5 Veo operations (4 failed validation
|
||||
inside the operation and cost nothing, one `generateAudio:"str"` probe rendered
|
||||
one 8 s clip) and generated 3 Lyria clips. The probe scripts are not part of
|
||||
the repo; the live tests in the repo never call a generation method.
|
||||
|
||||
Not live-verified (mutating or billable): deploy, EULA accept, image
|
||||
generation (no Imagen model is visible to the project), endpoint create, patch,
|
||||
update, delete, deployModel, undeployModel, mutateDeployedModel, IAM set,
|
||||
publisher export, config and enableModel, the endpoint inference family (no
|
||||
endpoint exists). These are verified by Discovery and by local request tests.
|
||||
|
||||
## 7. Sources
|
||||
|
||||
@@ -136,28 +237,49 @@ shapes. These are verified by Discovery only.
|
||||
|
||||
## 8. Unverified (neither Discovery nor docs read, nor live)
|
||||
|
||||
Settled on 2026-10-24 and moved to section 6: Veo `prompt`, `sampleCount`,
|
||||
`durationSeconds`, `aspectRatio`, `storageUri` and the finished operation
|
||||
`response`; Lyria `prompt` and the `predictions[].bytesBase64Encoded` WAV
|
||||
response; the empty endpoint list shape; thought signatures on function calls.
|
||||
|
||||
- Imagen `predict` instance/parameter names (`prompt`; `sampleCount`,
|
||||
`aspectRatio`, `negativePrompt`) and the `predictions[].bytesBase64Encoded`
|
||||
/ `mimeType` response: Discovery types them as free-form `Value`; the doc
|
||||
pages could not be read.
|
||||
- Lyria shape (`prompt`, `negative_prompt`, `seed`; `sample_count`).
|
||||
- Veo shape (`sampleCount`, `durationSeconds`, `aspectRatio`, `storageUri`;
|
||||
final `response.videos[]` with `gcsUri` / `bytesBase64Encoded` / `mimeType`).
|
||||
`fetchPredictOperation` itself is Discovery-verified and reachable live.
|
||||
/ `mimeType` response: no Imagen model is visible to project `lisa-448004`
|
||||
(404 for every id tried), so even a rejected request is not informative.
|
||||
- Lyria `negative_prompt`, `seed` and `sample_count`: a live call accepted
|
||||
them, but it also accepted junk values (`seed:"abc"`, `sample_count:99`), so
|
||||
acceptance does not prove the names. Whether a clip count above 1 is
|
||||
honored was not checked.
|
||||
- Veo `generateAudio` and the `gcsUri` result form with a real `storageUri`
|
||||
bucket (no bucket was used). A missing `mimeType` on Veo output was not seen.
|
||||
- Gemma/vLLM instance shape in `gemma_instance` (`prompt`, `max_tokens`,
|
||||
`temperature`): the serving container's, not Google's API.
|
||||
`temperature`): the serving container's, not Google's API; no endpoint
|
||||
exists to ask.
|
||||
- Anthropic streaming event names (`content_block_delta` / `text_delta`)
|
||||
come from Anthropic's format; Claude completions were not obtained live.
|
||||
- Embedding predict response `embeddings.values` / `statistics.token_count`
|
||||
was observed live for text-embedding-005, not documented-read.
|
||||
come from Anthropic's format; Claude completions were not obtained live
|
||||
(429 quota on every attempt).
|
||||
- `deployPublisherModel` field semantics beyond the schema names.
|
||||
- Response shapes of the admin methods that need a real endpoint or a
|
||||
mutation: deployModel, directPredict, directRawPredict, explain, chat
|
||||
completions, openapi, invoke, stored responses (501 for this project),
|
||||
export, setPublisherModelConfig, enableModel. They are returned as raw Json
|
||||
(no field is read), so a different shape cannot break the client.
|
||||
- Whether Google rejects a Gemini 3 replay that omits the `thoughtSignature`
|
||||
(the accepted replay with it is verified).
|
||||
|
||||
## 9. Gaps
|
||||
|
||||
- Discovery methods deliberately not wrapped (owner: this module): publisher
|
||||
`export`, `setPublisherModelConfig`, `enableModel`, IAM policy, `v1/responses`
|
||||
on publishers, publisher-level `serverStreamingPredict` and v1beta1
|
||||
`publishers.models.*` project-less duplicates.
|
||||
- Every Discovery method of the owned resources is in the call table (see
|
||||
section 5); the only intentional difference is that project-less v1 forms
|
||||
are the `express` forms of rows and the v1beta1 forms of endpoint rows are
|
||||
twins selected with `EndpointAdmin(beta=true)`.
|
||||
- The admin request bodies are validated by top-level key against the call
|
||||
table and by caps; nested fields (a `DeployedModel`, a `PublisherModelConfig`)
|
||||
are Json the caller owns. Typed structs exist where the module reads fields
|
||||
(`EndpointInfo`, `Operation`, `PublisherModel`).
|
||||
- Endpoint `generateContent` family is wired but only Discovery-verified.
|
||||
- `VertexModel` has no retry; quota 429 surfaces as `GoogleError`.
|
||||
- A streaming error body shaped as a JSON array (`[{"error":...}]`, seen on
|
||||
Claude `streamRawPredict`) goes through the foundation `StreamDecoder`, which
|
||||
is outside this module; not verified that it maps to `GoogleError`.
|
||||
- Token refresh is the caller's job (foundation gap).
|
||||
|
||||
Reference in New Issue
Block a user