Files
moloapp/src/google/models/SPEC.md
T

164 lines
9.1 KiB
Markdown
Raw Normal View History

# Models (Gemini Enterprise Agent Platform) - Specification
Owned directory: `src/google/models/`. Namespace `molo::google::models`.
Depends on the foundation in `src/google/` (`GoogleClient`, `Service`,
`GoogleError`, `CloudContext`, `Operation`, `gemini_types.h`). Voice and
`Generative` are untouched.
## 1. Scope
Gemini on Vertex AI (generate, stream, count tokens, compute tokens,
embeddings), Model Garden (list, get, publisher model config, EULA, deploy),
partner models (Claude rawPredict and streamRawPredict), endpoint models
(Gemma and any deployed model: list, get, predict family), media generation
(image, music, video via predict and predictLongRunning), and
`VertexModel : LanguageModel`.
Two addressing modes, one class (`VertexTarget`):
- express (API key, no project): `https://aiplatform.googleapis.com/v1/publishers/google/models/{m}:generateContent`
- project scoped: `https://{loc}-aiplatform.googleapis.com/v1/projects/{p}/locations/{loc}/publishers/{pub}/models/{m}:...`
(`global` uses the host without a prefix).
## 2. Module map (one concern per file)
| File | Concern |
|---|---|
| `caps.h` | Static ceilings |
| `calls.h` | THE call table: every call, its Discovery rpc id and request fields |
| `target.h/.cpp` | `VertexTarget`: express vs project, id validation |
| `planner.h/.cpp` | `CallPlanner`: table row + ids -> exact request (pure, no I/O) |
| `vertex_service.h/.cpp` | `VertexService : Service`: runs planned calls |
| `json_read.h` | tolerant response readers |
| `gemini.h/.cpp` | `Gemini`: generate, stream (`?alt=sse`), count/compute tokens, embedContent, predict embeddings |
| `vertex_model.h/.cpp` | `VertexModel : LanguageModel` |
| `model_garden.h/.cpp` | `ModelGarden`: list, get, fetch config, EULA check/accept, deploy |
| `claude_models.h/.cpp` | `ClaudeModels`: rawPredict, streamRawPredict, `text_delta` |
| `open_models.h/.cpp` | `OpenModels`: endpoints list/get, predict, rawPredict, streamRawPredict, serverStreamingPredict, generate/stream/count on endpoints, predictLongRunning, fetchPredictOperation, publisher invoke |
| `media.h/.cpp` | `MediaGenerator`: image, music, video start/status/wait |
| `tools/check_discovery.py` | checks `calls.h` against Google's Discovery documents |
Every call has a `plan_*` twin returning a `PlannedCall` (method, endpoint,
path, body) without sending anything; tests and `check_discovery.py` verify
requests that way.
## 3. Static caps (`caps.h`)
Contents per request 200; request text 1,000,000 chars; output tokens
1..65536; candidates 1..8; embed batch 100, 100,000 chars per text, 3072
dimensions; list page size 1..100, 20 pages, 2000 models returned; Claude
200 messages, 1,000,000 chars, max_tokens 1..32768; endpoint instances 16,
100,000 chars, Gemma max_tokens 8192; media prompt 4000 chars, images 4,
music samples 4, video samples 4, video 8 seconds, 32 MiB and 8 items per
result, operation name 512 chars; video polls 120 at 5 s (10 minutes).
## 4. Contracts
- Every id (publisher, model, endpoint, invoke id) passes `valid_segment`
before a path is built; operation names and `publisherModel` strings have
their own character check. Bad input is `GoogleError::invalid` (400
`INVALID_ARGUMENT`) before any network traffic.
- A call whose table row has `express=false` throws invalid in express mode,
naming the missing project. No credential: `NotConfiguredError` (503).
- Costly or consent calls are explicit and gated: `ModelGarden::deploy`
needs `DeployOptions.confirm`, `accept_eula` needs `confirm=true`; both
refuse otherwise. Nothing in this module deletes.
- Streaming uses `GoogleClient::stream_json` (SSE via `?alt=sse` for Gemini,
Anthropic events for Claude); the sink returns false to stop.
- Video: `start_video` (predictLongRunning), `video_status`
(fetchPredictOperation, a POST), `wait_video` bounded by the caps.
`wait_operation` (GET) is not used: Discovery has no GET on these
operations for publisher models.
- Example (`/agent/models`): GET page, `/status`, `/calls` need no
credential; every Google call is a POST that answers 503
`google_not_configured` without a credential (checked first), 422 on bad
input. Routes: generate, count, embed, claude, list, endpoint, image,
music, video, video/status.
## 5. Verified against discovery
`python3 -I src/google/models/tools/check_discovery.py` fetches
`https://aiplatform.googleapis.com/$discovery/rest?version=v1` and `v1beta1`
and fails when any row of `calls.h` has a missing method or path, a wrong
rpc id, an absent request body field (dotted names follow the schema) or an
absent query parameter. Result on 2026-10-24: `29 calls checked, 0 problems`.
The same pass shows what Discovery says about express forms:
generateContent, streamGenerateContent, countTokens, computeTokens, predict,
predictLongRunning, fetchPredictOperation exist without a project;
embedContent, rawPredict, streamRawPredict, publisher invoke do not
(`embedContent` express answers 404 live). `publishers.models.list` exists
only in v1beta1; `publishers.models.get` in v1 and v1beta1.
Request construction is compared to the table in
`tests/src/google/models/test_models.cpp` (every body key must be a declared
field).
## 6. Verified live
Run 2026-10-24 through this module's C++ client
(`MOLO_LIVE_NET=1 build/google_test`, tests `models_live.*`), read-only or
tiny (20 tokens). Credentials never printed or stored.
With an express API key (`AQ.`):
- `Gemini::generate`, `stream` (SSE), `count_tokens`, `compute_tokens`,
`embed` (text-embedding-005 predict) on `aiplatform.googleapis.com/v1/publishers/google/models/gemini-3.5-flash`: 200.
- Project scoped `countTokens` at location `global`: 200.
- Refused with 401 "API keys are not supported": Model Garden list/get,
endpoints list, rawPredict, project embedContent. Express `embedContent`:
404 (matches Discovery).
With the service account OAuth token (project `lisa-448004`):
- v1beta1 `publishers/google/models` list (google, anthropic), v1
`publishers/google/models/{gemini-2.5-flash,gemini-3.5-flash}` get incl.
`view=PUBLISHER_MODEL_VIEW_FULL`: 200.
- Endpoints list (`us-central1`): 200, empty.
- `embedContent` project scoped `us-central1` `gemini-embedding-001`: 200
(global and text-embedding-005 not available there: 404/400).
- Project scoped `generateContent` and `computeTokens` at `global`: 200
(`us-central1` + gemini-2.5-flash: 404 "not found or no access").
- Claude `rawPredict` at `global` (claude-sonnet-4-6): reached, 429
RESOURCE_EXHAUSTED (project quota); `us-east5`: 404 no access. Route, auth
and body accepted; no completion was obtained.
- `fetchPublisherModelConfig`: 404 "PublisherModelConfig not found" (route
exists, nothing configured). EULA check for `publishers/google/models/gemma3`:
404 model not found (route exists).
- Generation calls may answer 429 on the shared project; the live test notes
that instead of failing.
Not live-verified (mutating or billable): deploy, EULA accept, image, music
and video generation, endpoint predict family (no endpoint exists), Gemma
shapes. These are verified by Discovery only.
## 7. Sources
- Discovery documents: `https://aiplatform.googleapis.com/$discovery/rest?version=v1` and `?version=v1beta1` (fetched 2026-10-24).
- Claude on Vertex: `https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/partner-models/claude/use-claude` (rawPredict/streamRawPredict paths, `anthropic_version: vertex-2023-10-16`, messages/max_tokens/stream/thinking, response shape).
- Reference pages read for semantics: `.../reference/rest/v1/projects.locations.publishers.models/{countTokens,embedContent}`, `.../reference/rest/v1/publishers.models/get` (view enum, languageCode).
## 8. Unverified (neither Discovery nor docs read, nor live)
- Imagen `predict` instance/parameter names (`prompt`; `sampleCount`,
`aspectRatio`, `negativePrompt`) and the `predictions[].bytesBase64Encoded`
/ `mimeType` response: Discovery types them as free-form `Value`; the doc
pages could not be read.
- Lyria shape (`prompt`, `negative_prompt`, `seed`; `sample_count`).
- Veo shape (`sampleCount`, `durationSeconds`, `aspectRatio`, `storageUri`;
final `response.videos[]` with `gcsUri` / `bytesBase64Encoded` / `mimeType`).
`fetchPredictOperation` itself is Discovery-verified and reachable live.
- Gemma/vLLM instance shape in `gemma_instance` (`prompt`, `max_tokens`,
`temperature`): the serving container's, not Google's API.
- Anthropic streaming event names (`content_block_delta` / `text_delta`)
come from Anthropic's format; Claude completions were not obtained live.
- Embedding predict response `embeddings.values` / `statistics.token_count`
was observed live for text-embedding-005, not documented-read.
- `deployPublisherModel` field semantics beyond the schema names.
## 9. Gaps
- Discovery methods deliberately not wrapped (owner: this module): publisher
`export`, `setPublisherModelConfig`, `enableModel`, IAM policy, `v1/responses`
on publishers, publisher-level `serverStreamingPredict` and v1beta1
`publishers.models.*` project-less duplicates.
- Endpoint `generateContent` family is wired but only Discovery-verified.
- `VertexModel` has no retry; quota 429 surfaces as `GoogleError`.
- Token refresh is the caller's job (foundation gap).