docs(clients): plan 17 for the xAI, Anthropic, Meta and DeepSeek platforms
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,121 @@
|
||||
# 17 LLM vendor platforms: xAI, Anthropic, Meta, DeepSeek
|
||||
|
||||
Status: plan, 2026-10-24. Same method as plan 16 (Google): one module per
|
||||
product area, derived from authoritative machine-readable sources, one call
|
||||
table per module, a checker script, local deterministic tests, opt-in live
|
||||
tests, a demo page. Scope is each vendor's whole public API platform, not
|
||||
just chat. "Meta" means the Llama API (llama.developer.meta.com); if a
|
||||
module owner finds that wrong or incomplete they say so in the SPEC.
|
||||
|
||||
Foundation (already in `src/llm/`, read it first): `ApiClient` (bearer or
|
||||
header-key auth, default headers, JSON / raw / multipart / SSE streaming,
|
||||
`ApiError`), `Service` base, `SseDecoder` (named events, `[DONE]`),
|
||||
`Config::get` (environment, then `.env.json.local`, then `.env.json`),
|
||||
neutral chat types and the `ChatModel` seam (`chat_types.h`).
|
||||
`src/google/` is the reference implementation of this method: read one
|
||||
finished module (for example `src/google/runtime`: `calls.cpp` table,
|
||||
`tools/check_discovery.py`, SPEC layout, demo, tests).
|
||||
|
||||
## 1. Work packages (one worktree each)
|
||||
|
||||
| Package | Dirs | Covers |
|
||||
|---|---|---|
|
||||
| xai core | `src/xai/core` | Chat completions, Responses API (stored responses get/delete), tools (function calling, web search, X search, code execution, collections search, remote MCP), structured outputs, reasoning, image generation and editing, video generation, embeddings if public, models, tokenize, deferred completions, batch API, the Anthropic-compatible messages endpoint, `XaiChatModel : ChatModel` |
|
||||
| xai platform | `src/xai/platform` | Voice (realtime WebSocket agent API, TTS, STT), collections and documents, files, management API (API keys, billing, usage, team), API-key info; reuse `src/google/live` `WsClient` for WebSocket |
|
||||
| anthropic core | `src/anthropic/core` | Messages (create, streaming with every event type, thinking, tools: client tools and server tools such as web search, web fetch, code execution, computer use, bash, text editor, memory, tool search, MCP connector), vision / PDF / documents / citations / search results, prompt caching, structured outputs, Message Batches, token counting, Models, Files, Skills, beta header handling, `AnthropicChatModel : ChatModel` |
|
||||
| anthropic admin | `src/anthropic/admin` | Admin API (organization, users, invites, workspaces and members, API keys), usage and cost reports (messages, Claude Code), and any public managed-agents / sessions / environments REST surface; admin key is a separate credential |
|
||||
| meta | `src/meta` | Llama API: chat completions (native and OpenAI-compatible), models, moderations, tool calling, structured output, image understanding, files / uploads if public, speech if public, `LlamaChatModel : ChatModel` |
|
||||
| deepseek | `src/deepseek` | Chat completions (chat and reasoner, `reasoning_content`, tool calls, JSON output), beta: prefix completion and FIM, context caching usage, models, user balance, the Anthropic-compatible endpoint, errors and rate-limit semantics, `DeepseekChatModel : ChatModel` |
|
||||
|
||||
## 2. Rules for every package
|
||||
|
||||
1. Read `AGENTS.md`, this plan, `src/llm/*.h`, and one finished Google
|
||||
module with its tests and demo.
|
||||
2. Research first, from machine-readable sources, deeply. Order of
|
||||
preference: the vendor's OpenAPI document (many are published, or
|
||||
referenced by the official SDK repos, for example a `.stats.yml` or
|
||||
`openapi.json` link); the official SDK source (Python / TypeScript /
|
||||
Go: types, endpoints, headers, beta flags, error classes); the official
|
||||
API reference and guides. Cross-check at least two sources for every
|
||||
path and field. Record everything under `## Sources` in the SPEC.
|
||||
3. Each package SPEC (`src/<vendor>/<part>/SPEC.md`): scope, module map (one
|
||||
class per concern), static caps table, contracts, `## Sources`,
|
||||
`## Verified against spec` (which document, revision, counts),
|
||||
`## Verified live`, `## Unverified` (every shape inferred rather than
|
||||
read from a spec or seen), `## Gaps`. Never invent an endpoint
|
||||
silently; a product with no public API is said so.
|
||||
4. Object oriented, one concern per file, namespace
|
||||
`molo::<vendor>::<part>` (for example `molo::anthropic::admin`), services
|
||||
derive from `llm::Service`, hard caps for every count, size, interval,
|
||||
wait and page walk, ids validated with `Service::valid_id` before any
|
||||
path is built. List calls report truncation loudly (never silent).
|
||||
Destructive calls (delete, revoke, purge, cancel) are explicit methods
|
||||
with no default.
|
||||
5. Every call lives in one table constant in the package (HTTP method, path
|
||||
template, spec operation id, request and response schema names). A
|
||||
committed `tools/check_spec.py` (python3 stdlib only) fetches the
|
||||
vendor spec (or reads the pinned SDK source file you cite) and fails if
|
||||
a declared call or request field is absent, and fails on any spec
|
||||
operation that is neither in the table nor in an explicit exclusion
|
||||
list. Run it and record the result and date in the SPEC.
|
||||
6. NO verification with scripted vendor responses. Tests under
|
||||
`tests/src/<vendor>/` are deterministic and local: validation, caps,
|
||||
ids, request bodies against the table and schemas, parsing of the
|
||||
documented examples as fixtures taken verbatim from the vendor docs
|
||||
and labelled as such, SSE decoding of a documented stream transcript,
|
||||
local logic with a fake `ChatModel`, loopback servers for protocols the
|
||||
vendor does not own (WebSocket). Plus opt-in live tests (see 7).
|
||||
7. Live checks. No vendor key exists in this environment yet. What you CAN
|
||||
do: unauthenticated probes (the host resolves, TLS works, a documented
|
||||
path answers 401 rather than 404, the documented error body shape).
|
||||
Record them under `## Verified live`. Build opt-in live tests gated on
|
||||
`MOLO_LIVE_NET=1` plus the vendor key (`ANTHROPIC_API_KEY`,
|
||||
`ANTHROPIC_ADMIN_KEY`, `XAI_API_KEY`, `XAI_MANAGEMENT_KEY`,
|
||||
`DEEPSEEK_API_KEY`, `LLAMA_API_KEY`, loaded via `llm::Config::get`), with
|
||||
a loud SKIP otherwise. Live tests are read-only (models list, balance,
|
||||
retrieve) plus at most one smallest-possible completion. Anything that
|
||||
costs real money (media generation, batches, fine-tuning, long
|
||||
contexts, repeated calls) additionally requires `MOLO_LIVE_PAID=1`.
|
||||
Never create, update or delete vendor resources in a live test. Never
|
||||
print, log or commit a credential. Do not read `.env.json.local`.
|
||||
8. Example demo: fill your stub(s) in `example/app/ai/` (one per package).
|
||||
Routes under `/ai/<vendor>[/<part>]`. GET page and `/status` answer 2xx
|
||||
with no credential (the matrix crawler hits them); every route that
|
||||
calls the vendor is POST or DELETE and answers 503 `not_configured`
|
||||
without a credential, 422 on bad input; paid or destructive routes are
|
||||
dry runs unless the body has `"confirm":true`. Pages are mobile-first
|
||||
(360px): register one template inside the register function plus an
|
||||
optional script `example/assets/ai_<vendor>_<part>.js`. Add matrix rows
|
||||
with `services.add_feature(name, page)` for the GET page only. State
|
||||
lives inside the register function; do not edit `Services`.
|
||||
9. Example tests: `tests/example/ai/<vendor>/<part>/*_test.cpp` (Arrange /
|
||||
Act / Assert, see `tests/README.md`): GET pages, 405, 422, 503, limit
|
||||
cases. Never script a vendor reply. Use `tests/example/support/Assert.hpp`
|
||||
(`Fixture`).
|
||||
10. Locust: everything under `/ai` is excluded (`check_locust_coverage.py`).
|
||||
Do not touch `example/locustfile.py`.
|
||||
11. Do not edit shared files: `Makefile`, `src/ai.mk`, `src/google/**`,
|
||||
`example/example.mk`, `example/app/App.*`, `example/app/AiDemos.hpp`,
|
||||
`example/app/ai/AiDemos.cpp`, `example/app/Templates.cpp`,
|
||||
`example/locustfile.py`, `example/check_locust_coverage.py`,
|
||||
`tests/example/support/*`, any README, `example/SPEC.md`, `src/llm/*`
|
||||
(if the foundation blocks you, work around it inside your package and
|
||||
report it), other packages' files. Build globs pick new files up
|
||||
automatically. The `src/clients` tree is not yours either.
|
||||
12. Build and test: `make ai-test` (all vendor and shared tests) and
|
||||
`make -f example/example.mk example-test`. Do not run `make lint`. No
|
||||
em dashes anywhere. Commit subjects at most 100 characters,
|
||||
`feat(clients): <package>: <summary>`, ending with the Co-Authored-By
|
||||
line from your system reminder. Final report under 250 words: files,
|
||||
routes, test counts, what was verified how, Unverified list, foundation
|
||||
gaps.
|
||||
|
||||
## 3. Integration (after the worktrees finish)
|
||||
|
||||
Merge each branch; write the vendor-level `SPEC.md` overview, READMEs and
|
||||
`example/SPEC.md` rows; run the whole suite and lint; collect gaps below.
|
||||
|
||||
## 4. Gaps
|
||||
|
||||
- No vendor keys yet: every package is verified against its published spec
|
||||
and unauthenticated probes only until keys are provided.
|
||||
Reference in New Issue
Block a user