docs(clients): vendor platform integration: llm spec, readmes, example spec, plan gaps, counts

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 08:21:14 +02:00
co-authored by Claude Sonnet 5.5
parent f1ca245f1d
commit 38687de58f
6 changed files with 122 additions and 7 deletions
+1 -1
View File
@@ -31,7 +31,7 @@ make test # whole repo
make -f example/example.mk example-test # this part only, standalone
```
Expected: `example suite: 163 passed, 0 failed`. Failure action: reproduce
Expected: `example suite: 205 passed, 0 failed`. Failure action: reproduce
through the real stack first (failing route-level test), then fix; the
reproduction stays in the suite. Conventions: `tests/README.md`.
+22 -3
View File
@@ -35,13 +35,13 @@ is not defined in the Makefile yet.
## The living example
A route-level demo of every feature, with 163 integration tests on the real
A route-level demo of every feature, with 205 integration tests on the real
stack (no mocks). Standalone, no other part needed.
```sh
make -f example/example.mk example # build
./build/example/living-server 8080 # run
make -f example/example.mk example-test # expect: 163 passed, 0 failed
make -f example/example.mk example-test # expect: 205 passed, 0 failed
```
Open `http://127.0.0.1:8080/` and click through the matrix. Narrow the
@@ -78,7 +78,22 @@ Credentials (environment, then `.env.json.local`, then `.env.json`):
`GOOGLE_APPLICATION_CREDENTIALS` (git-ignored `.secrets/`). `make
google-token && ./build/google/sa-token` prints an access token.
The Agent Platform demo pages are under `/agent/<module>` and, like all
`/voice`, `/google` and `/agent` routes, are excluded from locust.
`/voice`, `/google`, `/agent` and `/ai` routes, are excluded from locust.
Other vendors (plan `plans/17-llm-vendor-platforms.md`, shared layer
`src/llm/SPEC.md`): pages `/ai/xai`, `/ai/xai/platform`, `/ai/anthropic`,
`/ai/anthropic/admin`, `/ai/meta`, `/ai/deepseek`, `/ai/openai`,
`/ai/openai/platform`, `/ai/openai/admin`, `/ai/openrouter`. Keys go in
`.env.json.local` or the environment: `XAI_API_KEY`, `XAI_MANAGEMENT_KEY`,
`ANTHROPIC_API_KEY`, `ANTHROPIC_ADMIN_KEY`, `META_API_KEY` (or
`MODEL_API_KEY`), `DEEPSEEK_API_KEY`, `OPENAI_API_KEY`, `OPENAI_ADMIN_KEY`,
`OPENROUTER_API_KEY`, `OPENROUTER_MANAGEMENT_KEY`. Without a key a page and
its status route still load and every calling route answers 503
`not_configured`; paid or destructive routes are dry runs unless the body has
`"confirm":true`. `MOLO_LIVE_NET=1 make ai-test` adds opt-in live checks;
anything that costs money also needs `MOLO_LIVE_PAID=1`. No vendor key
exists in this repo's environment yet, so every vendor package is verified
against its published spec and unauthenticated probes only.
Test conventions (one test file per URL, Arrange/Act/Assert, integration
only) are in `tests/README.md`.
@@ -208,6 +223,10 @@ src/app application, routers, views, forms, validation, templates
src/realtime WebSocket, SSE, Socket.IO, streaming, WebDAV, expose tunnels
src/supervisor process supervisor, tasks, replication
src/clients HTTP/REST client, OpenAPI, test double
src/llm shared layer for the vendor platforms below (client, SSE, errors, ChatModel)
src/xai src/anthropic src/meta src/deepseek src/openai src/openrouter
whole public API platforms of Grok, Claude, Meta Model API (Muse Spark),
DeepSeek, OpenAI and OpenRouter, one dir per work package
src/google Google Cloud: generic client, OAuth + service accounts, voice, and the
Gemini Enterprise Agent Platform (models, runtime, sessions, sandbox,
registry, armor, optimize, adk, live), one dir per module
+11 -1
View File
@@ -14,7 +14,7 @@ Open `http://127.0.0.1:8080/` and click through the matrix; `/stats` has the
charts (Chart.js is embedded in the binary, no CDN). Narrow the
window to 360px: the top nav collapses into a dropdown menu.
Run the suite (163 route-level integration tests, real stack, no mocks):
Run the suite (205 route-level integration tests, real stack, no mocks):
```sh
make -f example/example.mk example-test
@@ -129,6 +129,16 @@ on a missing row.
| agent evaluation and optimize | `/agent/optimize` | `tests/example/agent/optimize/page_test.cpp` | `plans/16-agent-platform.md` |
| agent development kit | `/agent/adk` | `tests/example/agent/adk/page_test.cpp` | `plans/16-agent-platform.md` |
| agent live streaming | `/agent/live` | `tests/example/agent/live/page_test.cpp` | `plans/16-agent-platform.md` |
| xai grok chat and tools | `/ai/xai` | `tests/example/ai/xai/core/` | `plans/17-llm-vendor-platforms.md` |
| xai grok voice, collections, management | `/ai/xai/platform` | `tests/example/ai/xai/platform/` | `plans/17-llm-vendor-platforms.md` |
| anthropic messages and tools | `/ai/anthropic` | `tests/example/ai/anthropic/core/` | `plans/17-llm-vendor-platforms.md` |
| anthropic admin and agents | `/ai/anthropic/admin` | `tests/example/ai/anthropic/admin/` | `plans/17-llm-vendor-platforms.md` |
| meta model api (muse spark) | `/ai/meta` | `tests/example/ai/meta/` | `plans/17-llm-vendor-platforms.md` |
| deepseek | `/ai/deepseek` | `tests/example/ai/deepseek/` | `plans/17-llm-vendor-platforms.md` |
| openai responses and media | `/ai/openai` | `tests/example/ai/openai/core/` | `plans/17-llm-vendor-platforms.md` |
| openai platform services | `/ai/openai/platform` | `tests/example/ai/openai/platform/` | `plans/17-llm-vendor-platforms.md` |
| openai organization admin | `/ai/openai/admin` | `tests/example/ai/openai/admin/` | `plans/17-llm-vendor-platforms.md` |
| openrouter | `/ai/openrouter` | `tests/example/ai/openrouter/` | `plans/17-llm-vendor-platforms.md` |
### Extended features
+8
View File
@@ -66,6 +66,14 @@ owning sibling plan and the file that will close it.
routes need an explicit confirmation. No route test scripts a Google
reply. All `/agent`, `/voice` and `/google` routes are excluded from
locust; `check_locust_coverage.py` fails if one is referenced.
18. LLM vendor platforms (`src/<vendor>`, shared layer `src/llm`): ten demo
pages at `/ai/<vendor>[/<part>]` for xAI, Anthropic, Meta, DeepSeek,
OpenAI and OpenRouter. Same contract as item 17: GET pages and `/status`
answer 2xx with no credential, every calling route is POST or DELETE and
answers 503 `not_configured` without one and 422 on bad input; paid or
destructive routes are dry runs unless the body has `"confirm":true`.
No route test scripts a vendor reply. All `/ai` routes are excluded from
locust.
## Caps
+19 -2
View File
@@ -1,6 +1,6 @@
# 17 LLM vendor platforms: xAI, Anthropic, Meta, DeepSeek, OpenAI, OpenRouter
Status: plan, 2026-10-24. Same method as plan 16 (Google): one module per
Status: implemented and merged, 2026-10-24 (all ten packages; see each package SPEC for what is verified against a published spec, probed live, or unverified). Same method as plan 16 (Google): one module per
product area, derived from authoritative machine-readable sources, one call
table per module, a checker script, local deterministic tests, opt-in live
tests, a demo page. Scope is each vendor's whole public API platform, not
@@ -123,4 +123,21 @@ Merge each branch; write the vendor-level `SPEC.md` overview, READMEs and
## 4. Gaps
- No vendor keys yet: every package is verified against its published spec
and unauthenticated probes only until keys are provided.
(or, where the vendor publishes none, its official SDK source plus the
rendered reference pages) and unauthenticated probes only. No success
response body was seen from any vendor. Provide keys in `.env.json.local`
and run `MOLO_LIVE_NET=1 make ai-test` (plus `MOLO_LIVE_PAID=1` for paid
calls) to close this.
- Anthropic and DeepSeek publish no OpenAPI document; Meta publishes prose
docs only. Their checkers compare the call tables with SDK source and
rendered pages, which is weaker than a schema check.
- `src/llm` gaps (GET streaming, response headers on streams, per-call
timeouts, opaque reasoning fields, comment lines in SSE, error `code`
handling) are listed in `src/llm/SPEC.md` section 4.
- Anthropic core and admin both wrap user profiles; unify when a key allows
checking which package should own them.
- Meta's Muse Code protocol client never ran against a real `muse` binary.
- OpenAI Assistants/Threads answered 404 unauthenticated (announced sunset,
not confirmed with a key); `GET /v1/videos` and `GET /v1/moderations`
answered 404 unexplained.
- xAI `GET /v1/skills` is in the OpenAPI document but answered 404 live.
+61
View File
@@ -0,0 +1,61 @@
# LLM vendor foundation - Specification
Owned directory: `src/llm/`. Namespace `molo::llm`. Depends on `molo::clients`
only. Used by the vendor modules `src/xai`, `src/anthropic`, `src/meta`,
`src/deepseek`, `src/openai`, `src/openrouter` (plan `plans/17-llm-vendor-platforms.md`).
## 1. Module map (one concern per file)
| File | Concern |
|---|---|
| `caps.h` | Static ceilings, never autotuned |
| `error.h/.cpp` | `ApiError` (vendor, status, type, message, request id), `NotConfiguredError`, parser for the common error envelopes |
| `config.h/.cpp` | `Config::get`: environment, then `.env.json.local`, then `.env.json`; `MOLO_ENV_FILES=off` for tests |
| `sse.h/.cpp` | `SseDecoder`: named events, `[DONE]`, comments, JSON error bodies in place of a stream |
| `api_client.h/.cpp` | `ApiClient` (bearer or header-key auth, default headers, JSON / raw / multipart / SSE POST) and the `Service` base |
| `chat_types.h/.cpp` | Vendor-neutral messages, tools, request, response, chunk, and the `ChatModel` seam |
## 2. Static caps (see caps.h)
| Knob | Value |
|---|---|
| Request body | 32 MiB |
| Response body | 64 MiB |
| Streamed bytes | 64 MiB |
| One SSE event | 4 MiB |
| Upload | 512 MiB |
| Messages / tools per request | 4096 / 256 |
| Page walk | 100 pages |
| Config file | 64 KiB |
## 3. Contracts
- No key: `configured()` is false and every call throws `NotConfiguredError`
(503, `NOT_CONFIGURED`) before any network traffic.
- The key travels in a header only, never in a URL, never in an error.
- Every failure is an `ApiError`; HTTP errors keep the vendor's own type or
code and the request id header when present.
- Paths must start with `/` and contain no `..`; ids go through
`Service::valid_id` before they are placed in a path.
- A JSON body where a stream was expected is the vendor's error document and
throws.
- Test runs set `MOLO_ENV_FILES=off`; `MOLO_LIVE_NET=1` re-enables config
files for opt-in live checks; `MOLO_LIVE_PAID=1` additionally allows calls
that cost money.
## 4. Gaps
Reported by the vendor modules, owner `src/llm`:
- `ApiClient::stream` is POST-only: no GET stream (resource event replay),
and it exposes no response headers (rate-limit headers on streams).
- `ApiClient` cannot send an unauthenticated request, and has no per-call
timeout; `post_multipart` only parses JSON replies.
- `ApiError` drops the error `code`/`param` when the type is taken from
elsewhere, drops `Retry-After`, and `error_from_response` loses the
top-level `code` when `error` is a plain string (xAI).
- `SseDecoder` drops comment lines, so keep-alives cannot be counted.
- `ContentPart`/`ChatChunk` have no field for opaque reasoning payloads
(Anthropic redacted thinking, Meta encrypted reasoning, OpenRouter
`reasoning_details`); vendor models carry them on the side.
- `Url` percent-encodes `[` and `]` in query names.