Files
moloapp/src/openrouter/SPEC.md
T

148 lines
8.8 KiB
Markdown
Raw Normal View History

# OpenRouter (src/openrouter)
Namespace `molo::openrouter`. Whole public OpenRouter API
(`https://openrouter.ai/api/v1`) over the shared `llm::ApiClient` foundation.
Status 2026-10-24: implemented and locally tested; no OpenRouter key exists in
this environment, so only unauthenticated routes were verified live.
## Scope
Chat completions (streaming incl. SSE heartbeats and mid-stream errors, tools,
structured outputs, reasoning, usage and cost), legacy completions, the
Responses API (beta), the Anthropic-compatible messages endpoint, routing
(`provider` preferences, `models` fallbacks, `route`, model variants, plugins,
server tools), models (list with every filter, one model, per-model endpoints,
count, user models, ZDR endpoints, embedding / image / video model lists),
providers, generation stats / content / feedback, key info, credits, activity
and analytics, API key management, embeddings, rerank, images, video, audio
(speech, transcription), batches, files and containers, presets, server tool
registry and public datasets, guardrails, BYOK, workspaces and budgets,
observability destinations, private endpoints, external users, organization
settings, SCIM, vault secrets, OAuth (PKCE code exchange, authorization code
creation, JWKS, workload token exchange), attribution headers, error semantics,
`OpenrouterChatModel : llm::ChatModel`.
## Module map (one class per concern)
| File | Class | Concern |
|---|---|---|
| `calls.h` | `kCalls` | the ONE call table: 140 rows (139 spec operations + `POST /completions`) |
| `call_plan` | `CallPlanner`, `Query`, `PlannedCall` | table-checked request planning; no network |
| `gateway` | `Gateway`, `AnonymousClient`, `Attribution`, `Endpoint` | two keyed `ApiClient`s + keyless client, attribution headers |
| `errors` | `ErrorBody`, `ErrorInfo` | envelope, typed `error_type`, 200-with-error shapes |
| `model_ref` | `ModelRef` | `author/slug` + catalog (`:free :batch :thinking :extended`) and routing (`:nitro :floor :exacto :online`) variants |
| `provider_preferences` | `ProviderPreferences` | order, only, ignore, allow_fallbacks, require_parameters, data_collection, zdr, quantizations, sort(+partition), max_price, preferred_max_latency / min_throughput, options |
| `reasoning` | `ReasoningConfig` | `reasoning` object |
| `plugins` | `Plugin` family | web, file-parser, response-healing, moderation, context-compression, auto-router, web-fetch, `RawPlugin` for the rest |
| `server_tools` | `ServerTools` | function tools and `openrouter:*` server tools |
| `chat_params` | `ChatParams`, `ResponseFormats` | validated `/chat/completions` body |
| `chat_result`, `chat_stream` | `ChatResult`, `ChatStreamParser` | response and SSE folding |
| `chat_service` | `ChatService` | chat + legacy completions, blocking and streaming |
| `chat_model` | `OpenrouterChatModel` | neutral `ChatModel` |
| `inference_services`, `stream_summary` | `ResponsesService`, `MessagesService`, stream parsers | Responses (beta) and Messages skins |
| `models` | `ModelsService`, `ProvidersService`, `ModelFilter` | catalog |
| `account` | `AccountService`, `GenerationService`, `KeysService` | key, credits, activity, generation stats, key management |
| `admin` | guardrails, BYOK, workspaces, observability, private endpoints, directory, vault | management surface |
| `platform` | embeddings, media, batches, files, presets, catalog, auth | the rest |
| `pkce.cpp` | `PkcePair` | RFC 7636 S256 |
## Credentials
Two separate keys, two `ApiClient`s, never a fallback between them:
`OPENROUTER_API_KEY` (inference and reads) and `OPENROUTER_MANAGEMENT_KEY`
(rows with key `mgmt`). Public rows (key `none`: models, providers, counts,
jwks, ...) go through a keyless client. Optional: `OPENROUTER_APP_URL`
(`HTTP-Referer`, required for app attribution), `OPENROUTER_APP_TITLE`
(`X-OpenRouter-Title` and legacy `X-Title`), `OPENROUTER_APP_CATEGORIES`
(`X-OpenRouter-Categories`, at most 2), `OPENROUTER_BASE_URL` (https or
loopback http only). Router metadata opt-in is `Attribution::router_metadata`
(`X-OpenRouter-Metadata: enabled`).
## Contracts
- Every path, query parameter and body field goes through `CallPlanner` and
must be declared in `calls.h`; undeclared input is rejected before any I/O.
- Ids are `Service::valid_id` checked (author may carry the alias marker `~`);
partner text (`{user}`) is 1..512 bytes, URL-encoded, `.` / `..` refused.
- Caps live in `caps.h`; list walks (`ModelsService::list_all`) stop at
`kMaxModelPages` and report `truncated`.
- Destructive calls are separate methods without defaults (`remove`,
`delete_*`, `unassign_*`); the demo adds a dry-run gate.
- A top-level `error` on a 200 body and a choice with `finish_reason: "error"`
are thrown as `llm::ApiError` (type = `error_type` when sent, else the code).
- Streaming: SSE comments (`: OPENROUTER PROCESSING`) are dropped by
`SseDecoder`; a mid-stream error is thrown after the chunks before it.
- `Usage` maps prompt/completion/total, `prompt_tokens_details.cached_tokens`,
`completion_tokens_details.reasoning_tokens`; cost (USD) is on `ChatResult`.
## Sources
1. `https://openrouter.ai/openapi.json` (OpenAPI 3.1, info version 1.0.0,
fetched 2026-10-24): 139 operations, 919 schemas. Primary source.
Saved copy used for exploration only; the checker fetches it live.
2. `https://openrouter.ai/docs/llms-full.txt` (docs plus generated SDK
references, 4.6 MB, 2026-10-24): errors guide (typed error codes, mid-stream
errors, 200-with-error), streaming guide (heartbeat comments), model
variants, plugins, app attribution, router metadata, reasoning.
3. Live unauthenticated probes (below).
Cross-checks: paths and fields come from (1) and the SDK reference pages in
(2) (they are generated from the same document, so they are not independent
for field lists); behavior (errors, SSE, variants, headers) comes from (2) and
was checked against live responses where a key is not needed.
## Verified against spec
`python3 -I src/openrouter/tools/check_spec.py --probe` on 2026-10-24: 140
rows checked (1 documented-only), 139 spec operations, 0 problems. The checker
verifies method, path, operationId, public-vs-keyed (401 present), body
fields (both directions), query parameters (both directions), multipart / form
formats, and fails on any spec operation missing from the table.
## Verified live (2026-10-24, no key)
- `GET /models` 200, 469 models, `total_count` and `links.next` present;
pricing values are decimal strings; ids include `~alias/...` authors and
`:free` / `:batch` variants. `list_all` walk read the whole catalog and every
id parsed as a `ModelRef` (`MOLO_LIVE_NET=1 ./build/ai_test`).
- `GET /providers` 200 (114 providers); `GET /models/count` 200;
`GET /models/openai/gpt-4o/endpoints` 200; `GET /model/openai/gpt-4o` 200;
`GET /embeddings/models`, `GET /endpoints/zdr` 200 without a key.
- `GET /key`, `GET /credits`, `POST /chat/completions`, `POST /messages`,
`POST /completions` answer 401 `{"error":{"message":"No cookie auth
credentials found","code":401}}` (numeric code); `POST /auth/keys` with `{}`
answers 400 (public).
## Unverified
- Every authenticated response shape (chat, responses, messages, generation,
keys, admin lists) is taken from spec examples and docs, not seen.
- `POST /completions` exists (401 live, docs list it under router metadata)
but is absent from the OpenAPI document; its field list in `calls.h` is the
OpenAI legacy set plus OpenRouter routing fields, inferred.
- `reasoning.exclude` is documented but not in the spec schema; kept as sent.
- Server tool `type` names beyond the spec's ten are accepted by form only.
- The `grant_type` literal for `/oauth/token` (RFC 8693 URN) is the standard
value; the spec lists the field without an enum.
- `models` fallback count: no vendor limit found, local ceiling 16.
- Streaming of `/images` (`stream` field) is not parsed (raw body only).
- Rows keyed `mgmt` follow the spec descriptions ("Management key required");
vault rows are keyed `api` per their securitySchemes.
## Gaps
- `llm::ApiError` carries no `error.metadata`, `Retry-After` or response
headers (foundation, `src/llm/error.h`); `describe()` works from status and
type only. Mid-stream and 200-with-error shapes keep full metadata via
`ErrorBody`.
- `SseDecoder` drops comment lines, so heartbeats cannot be counted or
surfaced as progress (`src/llm/sse.cpp`).
- `ai.mk` does not link `src/security` beyond Base64, so `pkce.cpp` carries
its own SHA-256 and reads `/dev/urandom`.
- `reasoning_details` cannot round-trip through the neutral `llm::Message`;
tool loops with reasoning models need `ChatService` with raw messages.
- Files upload takes the whole file in memory (`Caps` upload ceiling of the
foundation); no resumable upload.
- Typed result classes exist for chat, models, endpoints, providers, key info,
credits, generation; other responses are returned as `Json`.