docs(clients): vendor platform integration: llm spec, readmes, example spec, plan gaps, counts

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 08:21:14 +02:00
co-authored by Claude Sonnet 5.5
parent f1ca245f1d
commit 38687de58f
6 changed files with 122 additions and 7 deletions
+61
View File
@@ -0,0 +1,61 @@
# LLM vendor foundation - Specification
Owned directory: `src/llm/`. Namespace `molo::llm`. Depends on `molo::clients`
only. Used by the vendor modules `src/xai`, `src/anthropic`, `src/meta`,
`src/deepseek`, `src/openai`, `src/openrouter` (plan `plans/17-llm-vendor-platforms.md`).
## 1. Module map (one concern per file)
| File | Concern |
|---|---|
| `caps.h` | Static ceilings, never autotuned |
| `error.h/.cpp` | `ApiError` (vendor, status, type, message, request id), `NotConfiguredError`, parser for the common error envelopes |
| `config.h/.cpp` | `Config::get`: environment, then `.env.json.local`, then `.env.json`; `MOLO_ENV_FILES=off` for tests |
| `sse.h/.cpp` | `SseDecoder`: named events, `[DONE]`, comments, JSON error bodies in place of a stream |
| `api_client.h/.cpp` | `ApiClient` (bearer or header-key auth, default headers, JSON / raw / multipart / SSE POST) and the `Service` base |
| `chat_types.h/.cpp` | Vendor-neutral messages, tools, request, response, chunk, and the `ChatModel` seam |
## 2. Static caps (see caps.h)
| Knob | Value |
|---|---|
| Request body | 32 MiB |
| Response body | 64 MiB |
| Streamed bytes | 64 MiB |
| One SSE event | 4 MiB |
| Upload | 512 MiB |
| Messages / tools per request | 4096 / 256 |
| Page walk | 100 pages |
| Config file | 64 KiB |
## 3. Contracts
- No key: `configured()` is false and every call throws `NotConfiguredError`
(503, `NOT_CONFIGURED`) before any network traffic.
- The key travels in a header only, never in a URL, never in an error.
- Every failure is an `ApiError`; HTTP errors keep the vendor's own type or
code and the request id header when present.
- Paths must start with `/` and contain no `..`; ids go through
`Service::valid_id` before they are placed in a path.
- A JSON body where a stream was expected is the vendor's error document and
throws.
- Test runs set `MOLO_ENV_FILES=off`; `MOLO_LIVE_NET=1` re-enables config
files for opt-in live checks; `MOLO_LIVE_PAID=1` additionally allows calls
that cost money.
## 4. Gaps
Reported by the vendor modules, owner `src/llm`:
- `ApiClient::stream` is POST-only: no GET stream (resource event replay),
and it exposes no response headers (rate-limit headers on streams).
- `ApiClient` cannot send an unauthenticated request, and has no per-call
timeout; `post_multipart` only parses JSON replies.
- `ApiError` drops the error `code`/`param` when the type is taken from
elsewhere, drops `Retry-After`, and `error_from_response` loses the
top-level `code` when `error` is a plain string (xAI).
- `SseDecoder` drops comment lines, so keep-alives cannot be counted.
- `ContentPart`/`ChatChunk` have no field for opaque reasoning payloads
(Anthropic redacted thinking, Meta encrypted reasoning, OpenRouter
`reasoning_details`); vendor models carry them on the side.
- `Url` percent-encodes `[` and `]` in query names.