One call table verified against Discovery (29 calls), live-verified read-only with an express key and a service account token, /agent/models example routes. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
9.1 KiB
Models (Gemini Enterprise Agent Platform) - Specification
Owned directory: src/google/models/. Namespace molo::google::models.
Depends on the foundation in src/google/ (GoogleClient, Service,
GoogleError, CloudContext, Operation, gemini_types.h). Voice and
Generative are untouched.
1. Scope
Gemini on Vertex AI (generate, stream, count tokens, compute tokens,
embeddings), Model Garden (list, get, publisher model config, EULA, deploy),
partner models (Claude rawPredict and streamRawPredict), endpoint models
(Gemma and any deployed model: list, get, predict family), media generation
(image, music, video via predict and predictLongRunning), and
VertexModel : LanguageModel.
Two addressing modes, one class (VertexTarget):
- express (API key, no project):
https://aiplatform.googleapis.com/v1/publishers/google/models/{m}:generateContent - project scoped:
https://{loc}-aiplatform.googleapis.com/v1/projects/{p}/locations/{loc}/publishers/{pub}/models/{m}:...(globaluses the host without a prefix).
2. Module map (one concern per file)
| File | Concern |
|---|---|
caps.h |
Static ceilings |
calls.h |
THE call table: every call, its Discovery rpc id and request fields |
target.h/.cpp |
VertexTarget: express vs project, id validation |
planner.h/.cpp |
CallPlanner: table row + ids -> exact request (pure, no I/O) |
vertex_service.h/.cpp |
VertexService : Service: runs planned calls |
json_read.h |
tolerant response readers |
gemini.h/.cpp |
Gemini: generate, stream (?alt=sse), count/compute tokens, embedContent, predict embeddings |
vertex_model.h/.cpp |
VertexModel : LanguageModel |
model_garden.h/.cpp |
ModelGarden: list, get, fetch config, EULA check/accept, deploy |
claude_models.h/.cpp |
ClaudeModels: rawPredict, streamRawPredict, text_delta |
open_models.h/.cpp |
OpenModels: endpoints list/get, predict, rawPredict, streamRawPredict, serverStreamingPredict, generate/stream/count on endpoints, predictLongRunning, fetchPredictOperation, publisher invoke |
media.h/.cpp |
MediaGenerator: image, music, video start/status/wait |
tools/check_discovery.py |
checks calls.h against Google's Discovery documents |
Every call has a plan_* twin returning a PlannedCall (method, endpoint,
path, body) without sending anything; tests and check_discovery.py verify
requests that way.
3. Static caps (caps.h)
Contents per request 200; request text 1,000,000 chars; output tokens 1..65536; candidates 1..8; embed batch 100, 100,000 chars per text, 3072 dimensions; list page size 1..100, 20 pages, 2000 models returned; Claude 200 messages, 1,000,000 chars, max_tokens 1..32768; endpoint instances 16, 100,000 chars, Gemma max_tokens 8192; media prompt 4000 chars, images 4, music samples 4, video samples 4, video 8 seconds, 32 MiB and 8 items per result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
4. Contracts
- Every id (publisher, model, endpoint, invoke id) passes
valid_segmentbefore a path is built; operation names andpublisherModelstrings have their own character check. Bad input isGoogleError::invalid(400INVALID_ARGUMENT) before any network traffic. - A call whose table row has
express=falsethrows invalid in express mode, naming the missing project. No credential:NotConfiguredError(503). - Costly or consent calls are explicit and gated:
ModelGarden::deployneedsDeployOptions.confirm,accept_eulaneedsconfirm=true; both refuse otherwise. Nothing in this module deletes. - Streaming uses
GoogleClient::stream_json(SSE via?alt=ssefor Gemini, Anthropic events for Claude); the sink returns false to stop. - Video:
start_video(predictLongRunning),video_status(fetchPredictOperation, a POST),wait_videobounded by the caps.wait_operation(GET) is not used: Discovery has no GET on these operations for publisher models. - Example (
/agent/models): GET page,/status,/callsneed no credential; every Google call is a POST that answers 503google_not_configuredwithout a credential (checked first), 422 on bad input. Routes: generate, count, embed, claude, list, endpoint, image, music, video, video/status.
5. Verified against discovery
python3 -I src/google/models/tools/check_discovery.py fetches
https://aiplatform.googleapis.com/$discovery/rest?version=v1 and v1beta1
and fails when any row of calls.h has a missing method or path, a wrong
rpc id, an absent request body field (dotted names follow the schema) or an
absent query parameter. Result on 2026-10-24: 29 calls checked, 0 problems.
The same pass shows what Discovery says about express forms:
generateContent, streamGenerateContent, countTokens, computeTokens, predict,
predictLongRunning, fetchPredictOperation exist without a project;
embedContent, rawPredict, streamRawPredict, publisher invoke do not
(embedContent express answers 404 live). publishers.models.list exists
only in v1beta1; publishers.models.get in v1 and v1beta1.
Request construction is compared to the table in
tests/src/google/models/test_models.cpp (every body key must be a declared
field).
6. Verified live
Run 2026-10-24 through this module's C++ client
(MOLO_LIVE_NET=1 build/google_test, tests models_live.*), read-only or
tiny (20 tokens). Credentials never printed or stored.
With an express API key (AQ.):
Gemini::generate,stream(SSE),count_tokens,compute_tokens,embed(text-embedding-005 predict) onaiplatform.googleapis.com/v1/publishers/google/models/gemini-3.5-flash: 200.- Project scoped
countTokensat locationglobal: 200. - Refused with 401 "API keys are not supported": Model Garden list/get,
endpoints list, rawPredict, project embedContent. Express
embedContent: 404 (matches Discovery).
With the service account OAuth token (project lisa-448004):
- v1beta1
publishers/google/modelslist (google, anthropic), v1publishers/google/models/{gemini-2.5-flash,gemini-3.5-flash}get incl.view=PUBLISHER_MODEL_VIEW_FULL: 200. - Endpoints list (
us-central1): 200, empty. embedContentproject scopedus-central1gemini-embedding-001: 200 (global and text-embedding-005 not available there: 404/400).- Project scoped
generateContentandcomputeTokensatglobal: 200 (us-central1+ gemini-2.5-flash: 404 "not found or no access"). - Claude
rawPredictatglobal(claude-sonnet-4-6): reached, 429 RESOURCE_EXHAUSTED (project quota);us-east5: 404 no access. Route, auth and body accepted; no completion was obtained. fetchPublisherModelConfig: 404 "PublisherModelConfig not found" (route exists, nothing configured). EULA check forpublishers/google/models/gemma3: 404 model not found (route exists).- Generation calls may answer 429 on the shared project; the live test notes that instead of failing.
Not live-verified (mutating or billable): deploy, EULA accept, image, music and video generation, endpoint predict family (no endpoint exists), Gemma shapes. These are verified by Discovery only.
7. Sources
- Discovery documents:
https://aiplatform.googleapis.com/$discovery/rest?version=v1and?version=v1beta1(fetched 2026-10-24). - Claude on Vertex:
https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/partner-models/claude/use-claude(rawPredict/streamRawPredict paths,anthropic_version: vertex-2023-10-16, messages/max_tokens/stream/thinking, response shape). - Reference pages read for semantics:
.../reference/rest/v1/projects.locations.publishers.models/{countTokens,embedContent},.../reference/rest/v1/publishers.models/get(view enum, languageCode).
8. Unverified (neither Discovery nor docs read, nor live)
- Imagen
predictinstance/parameter names (prompt;sampleCount,aspectRatio,negativePrompt) and thepredictions[].bytesBase64Encoded/mimeTyperesponse: Discovery types them as free-formValue; the doc pages could not be read. - Lyria shape (
prompt,negative_prompt,seed;sample_count). - Veo shape (
sampleCount,durationSeconds,aspectRatio,storageUri; finalresponse.videos[]withgcsUri/bytesBase64Encoded/mimeType).fetchPredictOperationitself is Discovery-verified and reachable live. - Gemma/vLLM instance shape in
gemma_instance(prompt,max_tokens,temperature): the serving container's, not Google's API. - Anthropic streaming event names (
content_block_delta/text_delta) come from Anthropic's format; Claude completions were not obtained live. - Embedding predict response
embeddings.values/statistics.token_countwas observed live for text-embedding-005, not documented-read. deployPublisherModelfield semantics beyond the schema names.
9. Gaps
- Discovery methods deliberately not wrapped (owner: this module): publisher
export,setPublisherModelConfig,enableModel, IAM policy,v1/responseson publishers, publisher-levelserverStreamingPredictand v1beta1publishers.models.*project-less duplicates. - Endpoint
generateContentfamily is wired but only Discovery-verified. VertexModelhas no retry; quota 429 surfaces asGoogleError.- Token refresh is the caller's job (foundation gap).