From 2674e62c391081e756b583337d551b5d3453c87e Mon Sep 17 00:00:00 2001 From: retoor Date: Sat, 24 Oct 2026 06:08:45 +0200 Subject: [PATCH] docs(clients): plan 17 for the xAI, Anthropic, Meta and DeepSeek platforms Co-Authored-By: Claude Sonnet 5.5 --- plans/17-llm-vendor-platforms.md | 121 +++++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 plans/17-llm-vendor-platforms.md diff --git a/plans/17-llm-vendor-platforms.md b/plans/17-llm-vendor-platforms.md new file mode 100644 index 0000000..5c0ff52 --- /dev/null +++ b/plans/17-llm-vendor-platforms.md @@ -0,0 +1,121 @@ +# 17 LLM vendor platforms: xAI, Anthropic, Meta, DeepSeek + +Status: plan, 2026-10-24. Same method as plan 16 (Google): one module per +product area, derived from authoritative machine-readable sources, one call +table per module, a checker script, local deterministic tests, opt-in live +tests, a demo page. Scope is each vendor's whole public API platform, not +just chat. "Meta" means the Llama API (llama.developer.meta.com); if a +module owner finds that wrong or incomplete they say so in the SPEC. + +Foundation (already in `src/llm/`, read it first): `ApiClient` (bearer or +header-key auth, default headers, JSON / raw / multipart / SSE streaming, +`ApiError`), `Service` base, `SseDecoder` (named events, `[DONE]`), +`Config::get` (environment, then `.env.json.local`, then `.env.json`), +neutral chat types and the `ChatModel` seam (`chat_types.h`). +`src/google/` is the reference implementation of this method: read one +finished module (for example `src/google/runtime`: `calls.cpp` table, +`tools/check_discovery.py`, SPEC layout, demo, tests). + +## 1. Work packages (one worktree each) + +| Package | Dirs | Covers | +|---|---|---| +| xai core | `src/xai/core` | Chat completions, Responses API (stored responses get/delete), tools (function calling, web search, X search, code execution, collections search, remote MCP), structured outputs, reasoning, image generation and editing, video generation, embeddings if public, models, tokenize, deferred completions, batch API, the Anthropic-compatible messages endpoint, `XaiChatModel : ChatModel` | +| xai platform | `src/xai/platform` | Voice (realtime WebSocket agent API, TTS, STT), collections and documents, files, management API (API keys, billing, usage, team), API-key info; reuse `src/google/live` `WsClient` for WebSocket | +| anthropic core | `src/anthropic/core` | Messages (create, streaming with every event type, thinking, tools: client tools and server tools such as web search, web fetch, code execution, computer use, bash, text editor, memory, tool search, MCP connector), vision / PDF / documents / citations / search results, prompt caching, structured outputs, Message Batches, token counting, Models, Files, Skills, beta header handling, `AnthropicChatModel : ChatModel` | +| anthropic admin | `src/anthropic/admin` | Admin API (organization, users, invites, workspaces and members, API keys), usage and cost reports (messages, Claude Code), and any public managed-agents / sessions / environments REST surface; admin key is a separate credential | +| meta | `src/meta` | Llama API: chat completions (native and OpenAI-compatible), models, moderations, tool calling, structured output, image understanding, files / uploads if public, speech if public, `LlamaChatModel : ChatModel` | +| deepseek | `src/deepseek` | Chat completions (chat and reasoner, `reasoning_content`, tool calls, JSON output), beta: prefix completion and FIM, context caching usage, models, user balance, the Anthropic-compatible endpoint, errors and rate-limit semantics, `DeepseekChatModel : ChatModel` | + +## 2. Rules for every package + +1. Read `AGENTS.md`, this plan, `src/llm/*.h`, and one finished Google + module with its tests and demo. +2. Research first, from machine-readable sources, deeply. Order of + preference: the vendor's OpenAPI document (many are published, or + referenced by the official SDK repos, for example a `.stats.yml` or + `openapi.json` link); the official SDK source (Python / TypeScript / + Go: types, endpoints, headers, beta flags, error classes); the official + API reference and guides. Cross-check at least two sources for every + path and field. Record everything under `## Sources` in the SPEC. +3. Each package SPEC (`src///SPEC.md`): scope, module map (one + class per concern), static caps table, contracts, `## Sources`, + `## Verified against spec` (which document, revision, counts), + `## Verified live`, `## Unverified` (every shape inferred rather than + read from a spec or seen), `## Gaps`. Never invent an endpoint + silently; a product with no public API is said so. +4. Object oriented, one concern per file, namespace + `molo::::` (for example `molo::anthropic::admin`), services + derive from `llm::Service`, hard caps for every count, size, interval, + wait and page walk, ids validated with `Service::valid_id` before any + path is built. List calls report truncation loudly (never silent). + Destructive calls (delete, revoke, purge, cancel) are explicit methods + with no default. +5. Every call lives in one table constant in the package (HTTP method, path + template, spec operation id, request and response schema names). A + committed `tools/check_spec.py` (python3 stdlib only) fetches the + vendor spec (or reads the pinned SDK source file you cite) and fails if + a declared call or request field is absent, and fails on any spec + operation that is neither in the table nor in an explicit exclusion + list. Run it and record the result and date in the SPEC. +6. NO verification with scripted vendor responses. Tests under + `tests/src//` are deterministic and local: validation, caps, + ids, request bodies against the table and schemas, parsing of the + documented examples as fixtures taken verbatim from the vendor docs + and labelled as such, SSE decoding of a documented stream transcript, + local logic with a fake `ChatModel`, loopback servers for protocols the + vendor does not own (WebSocket). Plus opt-in live tests (see 7). +7. Live checks. No vendor key exists in this environment yet. What you CAN + do: unauthenticated probes (the host resolves, TLS works, a documented + path answers 401 rather than 404, the documented error body shape). + Record them under `## Verified live`. Build opt-in live tests gated on + `MOLO_LIVE_NET=1` plus the vendor key (`ANTHROPIC_API_KEY`, + `ANTHROPIC_ADMIN_KEY`, `XAI_API_KEY`, `XAI_MANAGEMENT_KEY`, + `DEEPSEEK_API_KEY`, `LLAMA_API_KEY`, loaded via `llm::Config::get`), with + a loud SKIP otherwise. Live tests are read-only (models list, balance, + retrieve) plus at most one smallest-possible completion. Anything that + costs real money (media generation, batches, fine-tuning, long + contexts, repeated calls) additionally requires `MOLO_LIVE_PAID=1`. + Never create, update or delete vendor resources in a live test. Never + print, log or commit a credential. Do not read `.env.json.local`. +8. Example demo: fill your stub(s) in `example/app/ai/` (one per package). + Routes under `/ai/[/]`. GET page and `/status` answer 2xx + with no credential (the matrix crawler hits them); every route that + calls the vendor is POST or DELETE and answers 503 `not_configured` + without a credential, 422 on bad input; paid or destructive routes are + dry runs unless the body has `"confirm":true`. Pages are mobile-first + (360px): register one template inside the register function plus an + optional script `example/assets/ai__.js`. Add matrix rows + with `services.add_feature(name, page)` for the GET page only. State + lives inside the register function; do not edit `Services`. +9. Example tests: `tests/example/ai///*_test.cpp` (Arrange / + Act / Assert, see `tests/README.md`): GET pages, 405, 422, 503, limit + cases. Never script a vendor reply. Use `tests/example/support/Assert.hpp` + (`Fixture`). +10. Locust: everything under `/ai` is excluded (`check_locust_coverage.py`). + Do not touch `example/locustfile.py`. +11. Do not edit shared files: `Makefile`, `src/ai.mk`, `src/google/**`, + `example/example.mk`, `example/app/App.*`, `example/app/AiDemos.hpp`, + `example/app/ai/AiDemos.cpp`, `example/app/Templates.cpp`, + `example/locustfile.py`, `example/check_locust_coverage.py`, + `tests/example/support/*`, any README, `example/SPEC.md`, `src/llm/*` + (if the foundation blocks you, work around it inside your package and + report it), other packages' files. Build globs pick new files up + automatically. The `src/clients` tree is not yours either. +12. Build and test: `make ai-test` (all vendor and shared tests) and + `make -f example/example.mk example-test`. Do not run `make lint`. No + em dashes anywhere. Commit subjects at most 100 characters, + `feat(clients): : `, ending with the Co-Authored-By + line from your system reminder. Final report under 250 words: files, + routes, test counts, what was verified how, Unverified list, foundation + gaps. + +## 3. Integration (after the worktrees finish) + +Merge each branch; write the vendor-level `SPEC.md` overview, READMEs and +`example/SPEC.md` rows; run the whole suite and lint; collect gaps below. + +## 4. Gaps + +- No vendor keys yet: every package is verified against its published spec + and unauthenticated probes only until keys are provided.