Files
moloapp/plans/17-llm-vendor-platforms.md
T

144 lines
12 KiB
Markdown

# 17 LLM vendor platforms: xAI, Anthropic, Meta, DeepSeek, OpenAI, OpenRouter
Status: implemented and merged, 2026-10-24 (all ten packages; see each package SPEC for what is verified against a published spec, probed live, or unverified). Same method as plan 16 (Google): one module per
product area, derived from authoritative machine-readable sources, one call
table per module, a checker script, local deterministic tests, opt-in live
tests, a demo page. Scope is each vendor's whole public API platform, not
just chat. "Meta" means the Meta Model API (docs: dev.meta.ai/docs, base URL
https://api.meta.ai/v1) that serves Muse Spark and the other Muse / Meta
models. It does NOT mean the Llama API.
Foundation (already in `src/llm/`, read it first): `ApiClient` (bearer or
header-key auth, default headers, JSON / raw / multipart / SSE streaming,
`ApiError`), `Service` base, `SseDecoder` (named events, `[DONE]`),
`Config::get` (environment, then `.env.json.local`, then `.env.json`),
neutral chat types and the `ChatModel` seam (`chat_types.h`).
`src/google/` is the reference implementation of this method: read one
finished module (for example `src/google/runtime`: `calls.cpp` table,
`tools/check_discovery.py`, SPEC layout, demo, tests).
## 1. Work packages (one worktree each)
| Package | Dirs | Covers |
|---|---|---|
| xai core | `src/xai/core` | Chat completions, Responses API (stored responses get/delete), tools (function calling, web search, X search, code execution, collections search, remote MCP), structured outputs, reasoning, image generation and editing, video generation, embeddings if public, models, tokenize, deferred completions, batch API, the Anthropic-compatible messages endpoint, `XaiChatModel : ChatModel` |
| xai platform | `src/xai/platform` | Voice (realtime WebSocket agent API, TTS, STT), collections and documents, files, management API (API keys, billing, usage, team), API-key info; reuse `src/google/live` `WsClient` for WebSocket |
| anthropic core | `src/anthropic/core` | Messages (create, streaming with every event type, thinking, tools: client tools and server tools such as web search, web fetch, code execution, computer use, bash, text editor, memory, tool search, MCP connector), vision / PDF / documents / citations / search results, prompt caching, structured outputs, Message Batches, token counting, Models, Files, Skills, beta header handling, `AnthropicChatModel : ChatModel` |
| anthropic admin | `src/anthropic/admin` | Admin API (organization, users, invites, workspaces and members, API keys), usage and cost reports (messages, Claude Code), and any public managed-agents / sessions / environments REST surface; admin key is a separate credential |
| meta | `src/meta` | Meta Model API (dev.meta.ai): Muse Spark (Responses-style and any chat/messages endpoints the docs list), tool calling, structured output, reasoning, vision input, Muse Wonder, Muse Image, Muse Voice Transcribe, Segment Anything, Muse Spark Open, Muse Glimmer, models, files / uploads if public, usage and pricing fields, the Muse Code agent surface if it has an API, `MetaChatModel : ChatModel` |
| openai core | `src/openai/core` | Responses API (create, retrieve, delete, cancel, input items, streaming events), Chat Completions (stored completions get/list/update/delete), legacy Completions, tools (function calling, web search, file search, code interpreter, computer use, image generation tool, remote MCP, custom tools), structured outputs, reasoning, Images (generate, edit, variations), Audio (speech, transcription, translation, streaming), Videos (Sora), Embeddings, Moderations, Models, `OpenaiChatModel : ChatModel` |
| openai platform | `src/openai/platform` | Realtime (WebSocket and client secrets / ephemeral keys, SIP calls; reuse `src/google/live` `WsClient`), Files, Uploads (multipart parts), Batch, Fine-tuning (jobs, events, checkpoints, graders), Vector stores (files, file batches, search), Evals (runs, output items), Conversations (items), Containers (files), Assistants / Threads / Runs (legacy beta), ChatKit and agent builder surfaces if public |
| openai admin | `src/openai/admin` | Organization Admin API (admin keys, users, invites, projects, project users, service accounts, API keys, rate limits, certificates, groups / roles if public), Usage and Costs endpoints, Audit logs, Webhooks (events, signature verification), data residency fields; separate admin key |
| openrouter | `src/openrouter` | Chat completions, Responses (beta) and the Anthropic-compatible messages endpoint if public, models (list, per-model endpoints, user models, filters), providers, generation stats, credits, activity, API keys management (create / list / update / delete with the management key), presets, routing parameters (provider preferences, fallbacks, `models` array, `:online`, variants like `:nitro`), plugins (web search, file parser, response healing), embeddings, auth (OAuth PKCE key exchange), BYOK and guardrails if public, `OpenrouterChatModel : ChatModel` |
| deepseek | `src/deepseek` | Chat completions (chat and reasoner, `reasoning_content`, tool calls, JSON output), beta: prefix completion and FIM, context caching usage, models, user balance, the Anthropic-compatible endpoint, errors and rate-limit semantics, `DeepseekChatModel : ChatModel` |
## 2. Rules for every package
1. Read `AGENTS.md`, this plan, `src/llm/*.h`, and one finished Google
module with its tests and demo.
2. Research first, from machine-readable sources, deeply. Order of
preference: the vendor's OpenAPI document (many are published, or
referenced by the official SDK repos, for example a `.stats.yml` or
`openapi.json` link); the official SDK source (Python / TypeScript /
Go: types, endpoints, headers, beta flags, error classes); the official
API reference and guides. Cross-check at least two sources for every
path and field. Record everything under `## Sources` in the SPEC.
3. Each package SPEC (`src/<vendor>/<part>/SPEC.md`): scope, module map (one
class per concern), static caps table, contracts, `## Sources`,
`## Verified against spec` (which document, revision, counts),
`## Verified live`, `## Unverified` (every shape inferred rather than
read from a spec or seen), `## Gaps`. Never invent an endpoint
silently; a product with no public API is said so.
4. Object oriented, one concern per file, namespace
`molo::<vendor>::<part>` (for example `molo::anthropic::admin`), services
derive from `llm::Service`, hard caps for every count, size, interval,
wait and page walk, ids validated with `Service::valid_id` before any
path is built. List calls report truncation loudly (never silent).
Destructive calls (delete, revoke, purge, cancel) are explicit methods
with no default.
5. Every call lives in one table constant in the package (HTTP method, path
template, spec operation id, request and response schema names). A
committed `tools/check_spec.py` (python3 stdlib only) fetches the
vendor spec (or reads the pinned SDK source file you cite) and fails if
a declared call or request field is absent, and fails on any spec
operation that is neither in the table nor in an explicit exclusion
list. Run it and record the result and date in the SPEC.
6. NO verification with scripted vendor responses. Tests under
`tests/src/<vendor>/` are deterministic and local: validation, caps,
ids, request bodies against the table and schemas, parsing of the
documented examples as fixtures taken verbatim from the vendor docs
and labelled as such, SSE decoding of a documented stream transcript,
local logic with a fake `ChatModel`, loopback servers for protocols the
vendor does not own (WebSocket). Plus opt-in live tests (see 7).
7. Live checks. No vendor key exists in this environment yet. What you CAN
do: unauthenticated probes (the host resolves, TLS works, a documented
path answers 401 rather than 404, the documented error body shape).
Record them under `## Verified live`. Build opt-in live tests gated on
`MOLO_LIVE_NET=1` plus the vendor key (`ANTHROPIC_API_KEY`,
`ANTHROPIC_ADMIN_KEY`, `XAI_API_KEY`, `XAI_MANAGEMENT_KEY`,
`DEEPSEEK_API_KEY`, `META_API_KEY` (the docs' own example uses `MODEL_API_KEY`; accept both), `OPENAI_API_KEY`, `OPENAI_ADMIN_KEY`, `OPENROUTER_API_KEY`, `OPENROUTER_MANAGEMENT_KEY`, loaded via `llm::Config::get`), with
a loud SKIP otherwise. Live tests are read-only (models list, balance,
retrieve) plus at most one smallest-possible completion. Anything that
costs real money (media generation, batches, fine-tuning, long
contexts, repeated calls) additionally requires `MOLO_LIVE_PAID=1`.
Never create, update or delete vendor resources in a live test. Never
print, log or commit a credential. Do not read `.env.json.local`.
8. Example demo: fill your stub(s) in `example/app/ai/` (one per package).
Routes under `/ai/<vendor>[/<part>]`. GET page and `/status` answer 2xx
with no credential (the matrix crawler hits them); every route that
calls the vendor is POST or DELETE and answers 503 `not_configured`
without a credential, 422 on bad input; paid or destructive routes are
dry runs unless the body has `"confirm":true`. Pages are mobile-first
(360px): register one template inside the register function plus an
optional script `example/assets/ai_<vendor>_<part>.js`. Add matrix rows
with `services.add_feature(name, page)` for the GET page only. State
lives inside the register function; do not edit `Services`.
9. Example tests: `tests/example/ai/<vendor>/<part>/*_test.cpp` (Arrange /
Act / Assert, see `tests/README.md`): GET pages, 405, 422, 503, limit
cases. Never script a vendor reply. Use `tests/example/support/Assert.hpp`
(`Fixture`).
10. Locust: everything under `/ai` is excluded (`check_locust_coverage.py`).
Do not touch `example/locustfile.py`.
11. Do not edit shared files: `Makefile`, `src/ai.mk`, `src/google/**`,
`example/example.mk`, `example/app/App.*`, `example/app/AiDemos.hpp`,
`example/app/ai/AiDemos.cpp`, `example/app/Templates.cpp`,
`example/locustfile.py`, `example/check_locust_coverage.py`,
`tests/example/support/*`, any README, `example/SPEC.md`, `src/llm/*`
(if the foundation blocks you, work around it inside your package and
report it), other packages' files. Build globs pick new files up
automatically. The `src/clients` tree is not yours either.
12. Build and test: `make ai-test` (all vendor and shared tests) and
`make -f example/example.mk example-test`. Do not run `make lint`. No
em dashes anywhere. Commit subjects at most 100 characters,
`feat(clients): <package>: <summary>`, ending with the Co-Authored-By
line from your system reminder. Final report under 250 words: files,
routes, test counts, what was verified how, Unverified list, foundation
gaps.
## 3. Integration (after the worktrees finish)
Merge each branch; write the vendor-level `SPEC.md` overview, READMEs and
`example/SPEC.md` rows; run the whole suite and lint; collect gaps below.
## 4. Gaps
- No vendor keys yet: every package is verified against its published spec
(or, where the vendor publishes none, its official SDK source plus the
rendered reference pages) and unauthenticated probes only. No success
response body was seen from any vendor. Provide keys in `.env.json.local`
and run `MOLO_LIVE_NET=1 make ai-test` (plus `MOLO_LIVE_PAID=1` for paid
calls) to close this.
- Anthropic and DeepSeek publish no OpenAPI document; Meta publishes prose
docs only. Their checkers compare the call tables with SDK source and
rendered pages, which is weaker than a schema check.
- `src/llm` gaps (GET streaming, response headers on streams, per-call
timeouts, opaque reasoning fields, comment lines in SSE, error `code`
handling) are listed in `src/llm/SPEC.md` section 4.
- Anthropic core and admin both wrap user profiles; unify when a key allows
checking which package should own them.
- Meta's Muse Code protocol client never ran against a real `muse` binary.
- OpenAI Assistants/Threads answered 404 unauthenticated (announced sunset,
not confirmed with a key); `GET /v1/videos` and `GET /v1/moderations`
answered 404 unexplained.
- xAI `GET /v1/skills` is in the OpenAPI document but answered 404 live.