148 lines
8.8 KiB
Markdown
148 lines
8.8 KiB
Markdown
# OpenRouter (src/openrouter)
|
|||
|
|
|
||
|
|
Namespace `molo::openrouter`. Whole public OpenRouter API
|
||
|
|
(`https://openrouter.ai/api/v1`) over the shared `llm::ApiClient` foundation.
|
||
|
|
Status 2026-10-24: implemented and locally tested; no OpenRouter key exists in
|
||
|
|
this environment, so only unauthenticated routes were verified live.
|
||
|
|
|
||
|
|
## Scope
|
||
|
|
|
||
|
|
Chat completions (streaming incl. SSE heartbeats and mid-stream errors, tools,
|
||
|
|
structured outputs, reasoning, usage and cost), legacy completions, the
|
||
|
|
Responses API (beta), the Anthropic-compatible messages endpoint, routing
|
||
|
|
(`provider` preferences, `models` fallbacks, `route`, model variants, plugins,
|
||
|
|
server tools), models (list with every filter, one model, per-model endpoints,
|
||
|
|
count, user models, ZDR endpoints, embedding / image / video model lists),
|
||
|
|
providers, generation stats / content / feedback, key info, credits, activity
|
||
|
|
and analytics, API key management, embeddings, rerank, images, video, audio
|
||
|
|
(speech, transcription), batches, files and containers, presets, server tool
|
||
|
|
registry and public datasets, guardrails, BYOK, workspaces and budgets,
|
||
|
|
observability destinations, private endpoints, external users, organization
|
||
|
|
settings, SCIM, vault secrets, OAuth (PKCE code exchange, authorization code
|
||
|
|
creation, JWKS, workload token exchange), attribution headers, error semantics,
|
||
|
|
`OpenrouterChatModel : llm::ChatModel`.
|
||
|
|
|
||
|
|
## Module map (one class per concern)
|
||
|
|
|
||
|
|
| File | Class | Concern |
|
||
|
|
|---|---|---|
|
||
|
|
| `calls.h` | `kCalls` | the ONE call table: 140 rows (139 spec operations + `POST /completions`) |
|
||
|
|
| `call_plan` | `CallPlanner`, `Query`, `PlannedCall` | table-checked request planning; no network |
|
||
|
|
| `gateway` | `Gateway`, `AnonymousClient`, `Attribution`, `Endpoint` | two keyed `ApiClient`s + keyless client, attribution headers |
|
||
|
|
| `errors` | `ErrorBody`, `ErrorInfo` | envelope, typed `error_type`, 200-with-error shapes |
|
||
|
|
| `model_ref` | `ModelRef` | `author/slug` + catalog (`:free :batch :thinking :extended`) and routing (`:nitro :floor :exacto :online`) variants |
|
||
|
|
| `provider_preferences` | `ProviderPreferences` | order, only, ignore, allow_fallbacks, require_parameters, data_collection, zdr, quantizations, sort(+partition), max_price, preferred_max_latency / min_throughput, options |
|
||
|
|
| `reasoning` | `ReasoningConfig` | `reasoning` object |
|
||
|
|
| `plugins` | `Plugin` family | web, file-parser, response-healing, moderation, context-compression, auto-router, web-fetch, `RawPlugin` for the rest |
|
||
|
|
| `server_tools` | `ServerTools` | function tools and `openrouter:*` server tools |
|
||
|
|
| `chat_params` | `ChatParams`, `ResponseFormats` | validated `/chat/completions` body |
|
||
|
|
| `chat_result`, `chat_stream` | `ChatResult`, `ChatStreamParser` | response and SSE folding |
|
||
|
|
| `chat_service` | `ChatService` | chat + legacy completions, blocking and streaming |
|
||
|
|
| `chat_model` | `OpenrouterChatModel` | neutral `ChatModel` |
|
||
|
|
| `inference_services`, `stream_summary` | `ResponsesService`, `MessagesService`, stream parsers | Responses (beta) and Messages skins |
|
||
|
|
| `models` | `ModelsService`, `ProvidersService`, `ModelFilter` | catalog |
|
||
|
|
| `account` | `AccountService`, `GenerationService`, `KeysService` | key, credits, activity, generation stats, key management |
|
||
|
|
| `admin` | guardrails, BYOK, workspaces, observability, private endpoints, directory, vault | management surface |
|
||
|
|
| `platform` | embeddings, media, batches, files, presets, catalog, auth | the rest |
|
||
|
|
| `pkce.cpp` | `PkcePair` | RFC 7636 S256 |
|
||
|
|
|
||
|
|
## Credentials
|
||
|
|
|
||
|
|
Two separate keys, two `ApiClient`s, never a fallback between them:
|
||
|
|
`OPENROUTER_API_KEY` (inference and reads) and `OPENROUTER_MANAGEMENT_KEY`
|
||
|
|
(rows with key `mgmt`). Public rows (key `none`: models, providers, counts,
|
||
|
|
jwks, ...) go through a keyless client. Optional: `OPENROUTER_APP_URL`
|
||
|
|
(`HTTP-Referer`, required for app attribution), `OPENROUTER_APP_TITLE`
|
||
|
|
(`X-OpenRouter-Title` and legacy `X-Title`), `OPENROUTER_APP_CATEGORIES`
|
||
|
|
(`X-OpenRouter-Categories`, at most 2), `OPENROUTER_BASE_URL` (https or
|
||
|
|
loopback http only). Router metadata opt-in is `Attribution::router_metadata`
|
||
|
|
(`X-OpenRouter-Metadata: enabled`).
|
||
|
|
|
||
|
|
## Contracts
|
||
|
|
|
||
|
|
- Every path, query parameter and body field goes through `CallPlanner` and
|
||
|
|
must be declared in `calls.h`; undeclared input is rejected before any I/O.
|
||
|
|
- Ids are `Service::valid_id` checked (author may carry the alias marker `~`);
|
||
|
|
partner text (`{user}`) is 1..512 bytes, URL-encoded, `.` / `..` refused.
|
||
|
|
- Caps live in `caps.h`; list walks (`ModelsService::list_all`) stop at
|
||
|
|
`kMaxModelPages` and report `truncated`.
|
||
|
|
- Destructive calls are separate methods without defaults (`remove`,
|
||
|
|
`delete_*`, `unassign_*`); the demo adds a dry-run gate.
|
||
|
|
- A top-level `error` on a 200 body and a choice with `finish_reason: "error"`
|
||
|
|
are thrown as `llm::ApiError` (type = `error_type` when sent, else the code).
|
||
|
|
- Streaming: SSE comments (`: OPENROUTER PROCESSING`) are dropped by
|
||
|
|
`SseDecoder`; a mid-stream error is thrown after the chunks before it.
|
||
|
|
- `Usage` maps prompt/completion/total, `prompt_tokens_details.cached_tokens`,
|
||
|
|
`completion_tokens_details.reasoning_tokens`; cost (USD) is on `ChatResult`.
|
||
|
|
|
||
|
|
## Sources
|
||
|
|
|
||
|
|
1. `https://openrouter.ai/openapi.json` (OpenAPI 3.1, info version 1.0.0,
|
||
|
|
fetched 2026-10-24): 139 operations, 919 schemas. Primary source.
|
||
|
|
Saved copy used for exploration only; the checker fetches it live.
|
||
|
|
2. `https://openrouter.ai/docs/llms-full.txt` (docs plus generated SDK
|
||
|
|
references, 4.6 MB, 2026-10-24): errors guide (typed error codes, mid-stream
|
||
|
|
errors, 200-with-error), streaming guide (heartbeat comments), model
|
||
|
|
variants, plugins, app attribution, router metadata, reasoning.
|
||
|
|
3. Live unauthenticated probes (below).
|
||
|
|
|
||
|
|
Cross-checks: paths and fields come from (1) and the SDK reference pages in
|
||
|
|
(2) (they are generated from the same document, so they are not independent
|
||
|
|
for field lists); behavior (errors, SSE, variants, headers) comes from (2) and
|
||
|
|
was checked against live responses where a key is not needed.
|
||
|
|
|
||
|
|
## Verified against spec
|
||
|
|
|
||
|
|
`python3 -I src/openrouter/tools/check_spec.py --probe` on 2026-10-24: 140
|
||
|
|
rows checked (1 documented-only), 139 spec operations, 0 problems. The checker
|
||
|
|
verifies method, path, operationId, public-vs-keyed (401 present), body
|
||
|
|
fields (both directions), query parameters (both directions), multipart / form
|
||
|
|
formats, and fails on any spec operation missing from the table.
|
||
|
|
|
||
|
|
## Verified live (2026-10-24, no key)
|
||
|
|
|
||
|
|
- `GET /models` 200, 469 models, `total_count` and `links.next` present;
|
||
|
|
pricing values are decimal strings; ids include `~alias/...` authors and
|
||
|
|
`:free` / `:batch` variants. `list_all` walk read the whole catalog and every
|
||
|
|
id parsed as a `ModelRef` (`MOLO_LIVE_NET=1 ./build/ai_test`).
|
||
|
|
- `GET /providers` 200 (114 providers); `GET /models/count` 200;
|
||
|
|
`GET /models/openai/gpt-4o/endpoints` 200; `GET /model/openai/gpt-4o` 200;
|
||
|
|
`GET /embeddings/models`, `GET /endpoints/zdr` 200 without a key.
|
||
|
|
- `GET /key`, `GET /credits`, `POST /chat/completions`, `POST /messages`,
|
||
|
|
`POST /completions` answer 401 `{"error":{"message":"No cookie auth
|
||
|
|
credentials found","code":401}}` (numeric code); `POST /auth/keys` with `{}`
|
||
|
|
answers 400 (public).
|
||
|
|
|
||
|
|
## Unverified
|
||
|
|
|
||
|
|
- Every authenticated response shape (chat, responses, messages, generation,
|
||
|
|
keys, admin lists) is taken from spec examples and docs, not seen.
|
||
|
|
- `POST /completions` exists (401 live, docs list it under router metadata)
|
||
|
|
but is absent from the OpenAPI document; its field list in `calls.h` is the
|
||
|
|
OpenAI legacy set plus OpenRouter routing fields, inferred.
|
||
|
|
- `reasoning.exclude` is documented but not in the spec schema; kept as sent.
|
||
|
|
- Server tool `type` names beyond the spec's ten are accepted by form only.
|
||
|
|
- The `grant_type` literal for `/oauth/token` (RFC 8693 URN) is the standard
|
||
|
|
value; the spec lists the field without an enum.
|
||
|
|
- `models` fallback count: no vendor limit found, local ceiling 16.
|
||
|
|
- Streaming of `/images` (`stream` field) is not parsed (raw body only).
|
||
|
|
- Rows keyed `mgmt` follow the spec descriptions ("Management key required");
|
||
|
|
vault rows are keyed `api` per their securitySchemes.
|
||
|
|
|
||
|
|
## Gaps
|
||
|
|
|
||
|
|
- `llm::ApiError` carries no `error.metadata`, `Retry-After` or response
|
||
|
|
headers (foundation, `src/llm/error.h`); `describe()` works from status and
|
||
|
|
type only. Mid-stream and 200-with-error shapes keep full metadata via
|
||
|
|
`ErrorBody`.
|
||
|
|
- `SseDecoder` drops comment lines, so heartbeats cannot be counted or
|
||
|
|
surfaced as progress (`src/llm/sse.cpp`).
|
||
|
|
- `ai.mk` does not link `src/security` beyond Base64, so `pkce.cpp` carries
|
||
|
|
its own SHA-256 and reads `/dev/urandom`.
|
||
|
|
- `reasoning_details` cannot round-trip through the neutral `llm::Message`;
|
||
|
|
tool loops with reasoning models need `ChatService` with raw messages.
|
||
|
|
- Files upload takes the whole file in memory (`Caps` upload ceiling of the
|
||
|
|
foundation); no resumable upload.
|
||
|
|
- Typed result classes exist for chat, models, endpoints, providers, key info,
|
||
|
|
credits, generation; other responses are returned as `Json`.
|