feat(clients): openrouter: whole public API, spec-checked call table, chat model, demo

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 07:46:42 +02:00
co-authored by Claude Sonnet 5.5
parent 2e0270c043
commit 9a24bedc7c
54 changed files with 7109 additions and 3 deletions
+147
View File
@@ -0,0 +1,147 @@
# OpenRouter (src/openrouter)
Namespace `molo::openrouter`. Whole public OpenRouter API
(`https://openrouter.ai/api/v1`) over the shared `llm::ApiClient` foundation.
Status 2026-10-24: implemented and locally tested; no OpenRouter key exists in
this environment, so only unauthenticated routes were verified live.
## Scope
Chat completions (streaming incl. SSE heartbeats and mid-stream errors, tools,
structured outputs, reasoning, usage and cost), legacy completions, the
Responses API (beta), the Anthropic-compatible messages endpoint, routing
(`provider` preferences, `models` fallbacks, `route`, model variants, plugins,
server tools), models (list with every filter, one model, per-model endpoints,
count, user models, ZDR endpoints, embedding / image / video model lists),
providers, generation stats / content / feedback, key info, credits, activity
and analytics, API key management, embeddings, rerank, images, video, audio
(speech, transcription), batches, files and containers, presets, server tool
registry and public datasets, guardrails, BYOK, workspaces and budgets,
observability destinations, private endpoints, external users, organization
settings, SCIM, vault secrets, OAuth (PKCE code exchange, authorization code
creation, JWKS, workload token exchange), attribution headers, error semantics,
`OpenrouterChatModel : llm::ChatModel`.
## Module map (one class per concern)
| File | Class | Concern |
|---|---|---|
| `calls.h` | `kCalls` | the ONE call table: 140 rows (139 spec operations + `POST /completions`) |
| `call_plan` | `CallPlanner`, `Query`, `PlannedCall` | table-checked request planning; no network |
| `gateway` | `Gateway`, `AnonymousClient`, `Attribution`, `Endpoint` | two keyed `ApiClient`s + keyless client, attribution headers |
| `errors` | `ErrorBody`, `ErrorInfo` | envelope, typed `error_type`, 200-with-error shapes |
| `model_ref` | `ModelRef` | `author/slug` + catalog (`:free :batch :thinking :extended`) and routing (`:nitro :floor :exacto :online`) variants |
| `provider_preferences` | `ProviderPreferences` | order, only, ignore, allow_fallbacks, require_parameters, data_collection, zdr, quantizations, sort(+partition), max_price, preferred_max_latency / min_throughput, options |
| `reasoning` | `ReasoningConfig` | `reasoning` object |
| `plugins` | `Plugin` family | web, file-parser, response-healing, moderation, context-compression, auto-router, web-fetch, `RawPlugin` for the rest |
| `server_tools` | `ServerTools` | function tools and `openrouter:*` server tools |
| `chat_params` | `ChatParams`, `ResponseFormats` | validated `/chat/completions` body |
| `chat_result`, `chat_stream` | `ChatResult`, `ChatStreamParser` | response and SSE folding |
| `chat_service` | `ChatService` | chat + legacy completions, blocking and streaming |
| `chat_model` | `OpenrouterChatModel` | neutral `ChatModel` |
| `inference_services`, `stream_summary` | `ResponsesService`, `MessagesService`, stream parsers | Responses (beta) and Messages skins |
| `models` | `ModelsService`, `ProvidersService`, `ModelFilter` | catalog |
| `account` | `AccountService`, `GenerationService`, `KeysService` | key, credits, activity, generation stats, key management |
| `admin` | guardrails, BYOK, workspaces, observability, private endpoints, directory, vault | management surface |
| `platform` | embeddings, media, batches, files, presets, catalog, auth | the rest |
| `pkce.cpp` | `PkcePair` | RFC 7636 S256 |
## Credentials
Two separate keys, two `ApiClient`s, never a fallback between them:
`OPENROUTER_API_KEY` (inference and reads) and `OPENROUTER_MANAGEMENT_KEY`
(rows with key `mgmt`). Public rows (key `none`: models, providers, counts,
jwks, ...) go through a keyless client. Optional: `OPENROUTER_APP_URL`
(`HTTP-Referer`, required for app attribution), `OPENROUTER_APP_TITLE`
(`X-OpenRouter-Title` and legacy `X-Title`), `OPENROUTER_APP_CATEGORIES`
(`X-OpenRouter-Categories`, at most 2), `OPENROUTER_BASE_URL` (https or
loopback http only). Router metadata opt-in is `Attribution::router_metadata`
(`X-OpenRouter-Metadata: enabled`).
## Contracts
- Every path, query parameter and body field goes through `CallPlanner` and
must be declared in `calls.h`; undeclared input is rejected before any I/O.
- Ids are `Service::valid_id` checked (author may carry the alias marker `~`);
partner text (`{user}`) is 1..512 bytes, URL-encoded, `.` / `..` refused.
- Caps live in `caps.h`; list walks (`ModelsService::list_all`) stop at
`kMaxModelPages` and report `truncated`.
- Destructive calls are separate methods without defaults (`remove`,
`delete_*`, `unassign_*`); the demo adds a dry-run gate.
- A top-level `error` on a 200 body and a choice with `finish_reason: "error"`
are thrown as `llm::ApiError` (type = `error_type` when sent, else the code).
- Streaming: SSE comments (`: OPENROUTER PROCESSING`) are dropped by
`SseDecoder`; a mid-stream error is thrown after the chunks before it.
- `Usage` maps prompt/completion/total, `prompt_tokens_details.cached_tokens`,
`completion_tokens_details.reasoning_tokens`; cost (USD) is on `ChatResult`.
## Sources
1. `https://openrouter.ai/openapi.json` (OpenAPI 3.1, info version 1.0.0,
fetched 2026-10-24): 139 operations, 919 schemas. Primary source.
Saved copy used for exploration only; the checker fetches it live.
2. `https://openrouter.ai/docs/llms-full.txt` (docs plus generated SDK
references, 4.6 MB, 2026-10-24): errors guide (typed error codes, mid-stream
errors, 200-with-error), streaming guide (heartbeat comments), model
variants, plugins, app attribution, router metadata, reasoning.
3. Live unauthenticated probes (below).
Cross-checks: paths and fields come from (1) and the SDK reference pages in
(2) (they are generated from the same document, so they are not independent
for field lists); behavior (errors, SSE, variants, headers) comes from (2) and
was checked against live responses where a key is not needed.
## Verified against spec
`python3 -I src/openrouter/tools/check_spec.py --probe` on 2026-10-24: 140
rows checked (1 documented-only), 139 spec operations, 0 problems. The checker
verifies method, path, operationId, public-vs-keyed (401 present), body
fields (both directions), query parameters (both directions), multipart / form
formats, and fails on any spec operation missing from the table.
## Verified live (2026-10-24, no key)
- `GET /models` 200, 469 models, `total_count` and `links.next` present;
pricing values are decimal strings; ids include `~alias/...` authors and
`:free` / `:batch` variants. `list_all` walk read the whole catalog and every
id parsed as a `ModelRef` (`MOLO_LIVE_NET=1 ./build/ai_test`).
- `GET /providers` 200 (114 providers); `GET /models/count` 200;
`GET /models/openai/gpt-4o/endpoints` 200; `GET /model/openai/gpt-4o` 200;
`GET /embeddings/models`, `GET /endpoints/zdr` 200 without a key.
- `GET /key`, `GET /credits`, `POST /chat/completions`, `POST /messages`,
`POST /completions` answer 401 `{"error":{"message":"No cookie auth
credentials found","code":401}}` (numeric code); `POST /auth/keys` with `{}`
answers 400 (public).
## Unverified
- Every authenticated response shape (chat, responses, messages, generation,
keys, admin lists) is taken from spec examples and docs, not seen.
- `POST /completions` exists (401 live, docs list it under router metadata)
but is absent from the OpenAPI document; its field list in `calls.h` is the
OpenAI legacy set plus OpenRouter routing fields, inferred.
- `reasoning.exclude` is documented but not in the spec schema; kept as sent.
- Server tool `type` names beyond the spec's ten are accepted by form only.
- The `grant_type` literal for `/oauth/token` (RFC 8693 URN) is the standard
value; the spec lists the field without an enum.
- `models` fallback count: no vendor limit found, local ceiling 16.
- Streaming of `/images` (`stream` field) is not parsed (raw body only).
- Rows keyed `mgmt` follow the spec descriptions ("Management key required");
vault rows are keyed `api` per their securitySchemes.
## Gaps
- `llm::ApiError` carries no `error.metadata`, `Retry-After` or response
headers (foundation, `src/llm/error.h`); `describe()` works from status and
type only. Mid-stream and 200-with-error shapes keep full metadata via
`ErrorBody`.
- `SseDecoder` drops comment lines, so heartbeats cannot be counted or
surfaced as progress (`src/llm/sse.cpp`).
- `ai.mk` does not link `src/security` beyond Base64, so `pkce.cpp` carries
its own SHA-256 and reads `/dev/urandom`.
- `reasoning_details` cannot round-trip through the neutral `llm::Message`;
tool loops with reasoning models need `ChatService` with raw messages.
- Files upload takes the whole file in memory (`Caps` upload ceiling of the
foundation); no resumable upload.
- Typed result classes exist for chat, models, endpoints, providers, key info,
credits, generation; other responses are returned as `Json`.
+287
View File
@@ -0,0 +1,287 @@
#include "openrouter/account.h"
#include <cmath>
#include "llm/error.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
const Json& data_of(const Json& doc) {
const Json* d = doc.is_object() ? doc.find("data") : nullptr;
if (!d || !d->is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "response has no data object");
return *d;
}
std::string str(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
double num(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
return v && v->is_number() ? v->as_number() : 0;
}
long integer(const Json& o, const char* k) { return static_cast<long>(num(o, k)); }
bool flag(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
return v && v->is_bool() && v->as_bool();
}
std::optional<double> opt_num(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
if (v && v->is_number()) return v->as_number();
return std::nullopt;
}
bool is_hex(char c) {
return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F');
}
void check_reset(const std::string& r) {
if (!r.empty() && r != "daily" && r != "weekly" && r != "monthly")
bad("limit_reset must be daily, weekly or monthly");
}
void check_limit_value(const std::optional<double>& l) {
if (l && (!std::isfinite(*l) || *l < 0)) bad("limit must be a non-negative number");
}
} // namespace
bool is_key_hash(const std::string& s) {
if (s.size() != 64) return false;
for (char c : s)
if (!is_hex(c)) return false;
return true;
}
bool is_uuid(const std::string& s) {
if (s.size() != 36) return false;
for (std::size_t i = 0; i < s.size(); ++i) {
if (i == 8 || i == 13 || i == 18 || i == 23) {
if (s[i] != '-') return false;
} else if (!is_hex(s[i])) {
return false;
}
}
return true;
}
bool is_iso_utc_seconds(const std::string& s) {
// 2027-12-31T23:59:59Z or 2027-12-31T23:59:59.123Z
if (s.size() < 20 || s.back() != 'Z') return false;
auto digit = [&](std::size_t i) { return s[i] >= '0' && s[i] <= '9'; };
for (std::size_t i : {0, 1, 2, 3, 5, 6, 8, 9, 11, 12, 14, 15, 17, 18})
if (!digit(i)) return false;
if (s[4] != '-' || s[7] != '-' || s[10] != 'T' || s[13] != ':' || s[16] != ':') return false;
if (s.size() == 20) return true;
if (s[19] != '.' || s.size() < 22) return false;
for (std::size_t i = 20; i + 1 < s.size(); ++i)
if (!digit(i)) return false;
return true;
}
KeyInfo KeyInfo::parse(const Json& doc) {
const Json& d = data_of(doc);
KeyInfo k;
k.raw = d;
k.label = str(d, "label");
k.limit = opt_num(d, "limit");
k.limit_remaining = opt_num(d, "limit_remaining");
k.limit_reset = str(d, "limit_reset");
k.usage = num(d, "usage");
k.usage_daily = num(d, "usage_daily");
k.usage_weekly = num(d, "usage_weekly");
k.usage_monthly = num(d, "usage_monthly");
k.byok_usage = num(d, "byok_usage");
k.is_free_tier = flag(d, "is_free_tier");
k.is_management_key = flag(d, "is_management_key");
k.include_byok_in_limit = flag(d, "include_byok_in_limit");
k.expires_at = str(d, "expires_at");
k.creator_user_id = str(d, "creator_user_id");
k.workspace_id = str(d, "workspace_id");
if (const Json* v = d.find("free_model_daily_requests")) k.free_model_daily_requests = *v;
if (const Json* v = d.find("effective_budget")) k.effective_budget = *v;
if (const Json* v = d.find("budgets")) k.budgets = *v;
return k;
}
Credits Credits::parse(const Json& doc) {
const Json& d = data_of(doc);
Credits c;
c.total_credits = num(d, "total_credits");
c.total_usage = num(d, "total_usage");
return c;
}
void ActivityFilter::validate() const {
if (!date.empty()) {
bool ok = date.size() == 10 && date[4] == '-' && date[7] == '-';
for (std::size_t i = 0; ok && i < date.size(); ++i)
if (i != 4 && i != 7 && !(date[i] >= '0' && date[i] <= '9')) ok = false;
if (!ok) bad("date must be YYYY-MM-DD");
}
if (!api_key_hash.empty() && !is_key_hash(api_key_hash)) bad("api_key_hash must be a 64 character hex string");
if (user_id.size() > 128) bad("user_id too long");
if (!group_by.empty() && group_by != "workspace") bad("group_by must be workspace");
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
}
Query ActivityFilter::to_query() const {
validate();
Query q;
q.add_if("date", date);
q.add_if("api_key_hash", api_key_hash);
q.add_if("user_id", user_id);
q.add_if("group_by", group_by);
q.add_if("workspace_id", workspace_id);
return q;
}
KeyInfo AccountService::key_info() const { return KeyInfo::parse(run("getCurrentKey")); }
Credits AccountService::credits() const { return Credits::parse(run("getCredits")); }
Json AccountService::activity(const ActivityFilter& f) const { return run("getUserActivity", {}, f.to_query()); }
Json AccountService::analytics_meta() const { return run("getAnalyticsMeta"); }
Json AccountService::analytics_query(const Json& body) const { return run("queryAnalytics", {}, {}, body); }
GenerationInfo GenerationInfo::parse(const Json& doc) {
const Json& d = data_of(doc);
GenerationInfo g;
g.raw = d;
g.id = str(d, "id");
g.model = str(d, "model");
g.provider_name = str(d, "provider_name");
g.created_at = str(d, "created_at");
g.finish_reason = str(d, "finish_reason");
g.native_finish_reason = str(d, "native_finish_reason");
g.api_type = str(d, "api_type");
g.origin = str(d, "origin");
g.total_cost = num(d, "total_cost");
g.usage = num(d, "usage");
g.cache_discount = num(d, "cache_discount");
g.latency_ms = num(d, "latency");
g.generation_time_ms = num(d, "generation_time");
g.tokens_prompt = integer(d, "tokens_prompt");
g.tokens_completion = integer(d, "tokens_completion");
g.native_tokens_prompt = integer(d, "native_tokens_prompt");
g.native_tokens_completion = integer(d, "native_tokens_completion");
g.native_tokens_reasoning = integer(d, "native_tokens_reasoning");
g.native_tokens_cached = integer(d, "native_tokens_cached");
g.streamed = flag(d, "streamed");
g.cancelled = flag(d, "cancelled");
g.is_byok = flag(d, "is_byok");
if (const Json* v = d.find("provider_responses")) g.provider_responses = *v;
return g;
}
GenerationInfo GenerationService::get(const std::string& id) const {
check_id("generation", id);
return GenerationInfo::parse(run("getGeneration", {}, Query().add("id", id)));
}
Json GenerationService::content(const std::string& id) const {
check_id("generation", id);
return run("listGenerationContent", {}, Query().add("id", id));
}
Json GenerationService::feedback(const std::string& id, const std::string& category,
const std::string& comment) const {
check_id("generation", id);
static const char* const kCats[] = {"latency", "incoherence", "incorrect_response", "formatting",
"billing", "api_error", "other"};
bool ok = false;
for (const char* c : kCats)
if (category == c) ok = true;
if (!ok) bad("feedback category must be latency incoherence incorrect_response formatting billing api_error or other");
if (comment.size() > 1000) bad("feedback comment is at most 1000 characters");
Json b = Json::object();
b.set("generation_id", Json(id));
b.set("category", Json(category));
if (!comment.empty()) b.set("comment", Json(comment));
return run("submitGenerationFeedback", {}, {}, b);
}
void KeyDraft::validate() const {
if (name.empty() || name.size() > 256) bad("key name must be 1..256 characters");
check_limit_value(limit);
check_reset(limit_reset);
if (!expires_at.empty() && !is_iso_utc_seconds(expires_at)) bad("expires_at must be YYYY-MM-DDTHH:MM:SSZ");
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
if (external_user.size() > 512) bad("external user too long");
}
Json KeyDraft::to_json() const {
validate();
Json b = Json::object();
b.set("name", Json(name));
if (limit) b.set("limit", Json(*limit));
if (!limit_reset.empty()) b.set("limit_reset", Json(limit_reset));
if (include_byok_in_limit) b.set("include_byok_in_limit", Json(*include_byok_in_limit));
if (!expires_at.empty()) b.set("expires_at", Json(expires_at));
if (!workspace_id.empty()) b.set("workspace_id", Json(workspace_id));
if (!creator_user_id.empty()) b.set("creator_user_id", Json(creator_user_id));
if (!external_user.empty()) {
Json e = Json::object();
e.set("user", Json(external_user));
b.set("external", e);
}
return b;
}
void KeyUpdate::validate() const {
if (name && (name->empty() || name->size() > 256)) bad("key name must be 1..256 characters");
check_limit_value(limit);
check_reset(limit_reset);
if (clear_limit_reset && !limit_reset.empty()) bad("limit_reset and clear_limit_reset together");
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
}
Json KeyUpdate::to_json() const {
validate();
Json b = Json::object();
if (name) b.set("name", Json(*name));
if (disabled) b.set("disabled", Json(*disabled));
if (limit) b.set("limit", Json(*limit));
if (!limit_reset.empty()) b.set("limit_reset", Json(limit_reset));
if (clear_limit_reset) b.set("limit_reset", Json());
if (include_byok_in_limit) b.set("include_byok_in_limit", Json(*include_byok_in_limit));
if (!workspace_id.empty()) b.set("workspace_id", Json(workspace_id));
if (b.as_object().empty()) bad("nothing to update");
return b;
}
Json KeysService::list(std::optional<int> offset, std::optional<int> limit, bool include_disabled,
bool include_expired, const std::string& workspace_id) const {
if (offset && *offset < 0) bad("offset must be >= 0");
if (limit && (*limit < 1 || *limit > Caps::kMaxKeysLimit)) bad("limit must be 1..100");
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
Query q;
if (offset) q.add("offset", *offset);
if (limit) q.add("limit", *limit);
if (include_disabled) q.add("include_disabled", true);
if (include_expired) q.add("include_expired", true);
q.add_if("workspace_id", workspace_id);
return run("listKeys", {}, q);
}
Json KeysService::get(const std::string& h) const {
if (!is_key_hash(h)) bad("key hash must be a 64 character hex string");
return run("getKey", {{"hash", h}});
}
Json KeysService::create(const KeyDraft& draft) const { return run("createKeys", {}, {}, draft.to_json()); }
Json KeysService::update(const std::string& h, const KeyUpdate& u) const {
if (!is_key_hash(h)) bad("key hash must be a 64 character hex string");
return run("updateKeys", {{"hash", h}}, {}, u.to_json());
}
Json KeysService::remove(const std::string& h) const {
if (!is_key_hash(h)) bad("key hash must be a 64 character hex string");
return run("deleteKeys", {{"hash", h}});
}
} // namespace molo::openrouter
+153
View File
@@ -0,0 +1,153 @@
#pragma once
// Account and usage: GET /key (the calling key), GET /credits (management
// key), GET /activity and the analytics routes (management key), generation
// stats (GET /generation?id=, content, feedback), and API key management
// (GET/POST /keys, GET/PATCH/DELETE /keys/{hash}; management key only).
#include <optional>
#include <string>
#include <vector>
#include "openrouter/gateway.h"
namespace molo::openrouter {
struct KeyInfo {
std::string label;
std::optional<double> limit; // USD, null: no limit
std::optional<double> limit_remaining;
std::string limit_reset; // daily weekly monthly or ""
double usage = 0, usage_daily = 0, usage_weekly = 0, usage_monthly = 0; // USD
double byok_usage = 0;
bool is_free_tier = false;
bool is_management_key = false;
bool include_byok_in_limit = false;
std::string expires_at;
std::string creator_user_id;
std::string workspace_id;
Json free_model_daily_requests; // {limit, remaining, used}
Json effective_budget; // the budget that blocks first, or null
Json budgets; // every budget layer, enforcement order
Json raw;
static KeyInfo parse(const Json& doc); // doc is the response; reads `data`
};
struct Credits {
double total_credits = 0;
double total_usage = 0;
double remaining() const { return total_credits - total_usage; }
static Credits parse(const Json& doc);
};
// Query for GET /activity (last 30 days, grouped by endpoint).
struct ActivityFilter {
std::string date; // YYYY-MM-DD, within the last 30 days
std::string api_key_hash; // SHA-256 hex
std::string user_id; // org member
std::string group_by; // "", workspace
std::string workspace_id; // uuid
void validate() const;
Query to_query() const;
};
class AccountService : public Endpoint {
public:
explicit AccountService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
KeyInfo key_info() const; // OPENROUTER_API_KEY
Credits credits() const; // OPENROUTER_MANAGEMENT_KEY
Json activity(const ActivityFilter& filter = {}) const; // management key
Json analytics_meta() const; // management key
Json analytics_query(const Json& body) const; // management key
};
struct GenerationInfo {
std::string id;
std::string model;
std::string provider_name;
std::string created_at;
std::string finish_reason;
std::string native_finish_reason;
std::string api_type;
std::string origin;
double total_cost = 0; // USD
double usage = 0;
double cache_discount = 0;
double latency_ms = 0;
double generation_time_ms = 0;
long tokens_prompt = 0, tokens_completion = 0;
long native_tokens_prompt = 0, native_tokens_completion = 0;
long native_tokens_reasoning = 0, native_tokens_cached = 0;
bool streamed = false;
bool cancelled = false;
bool is_byok = false;
Json provider_responses;
Json raw;
static GenerationInfo parse(const Json& doc); // reads `data`
};
class GenerationService : public Endpoint {
public:
explicit GenerationService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
// id is the "gen-..." id of a completion (ChatResult::id). Stats can lag the
// completion by a moment: a 404 right after the call is not a failure.
GenerationInfo get(const std::string& generation_id) const;
// Stored prompt, completion and error content; needs input/output logging.
Json content(const std::string& generation_id) const;
// categories: latency incoherence incorrect_response formatting billing api_error other.
// Uses the management key (per the spec text).
Json feedback(const std::string& generation_id, const std::string& category,
const std::string& comment = "") const;
};
// Body of POST /keys.
struct KeyDraft {
std::string name; // required
std::optional<double> limit; // USD
std::string limit_reset; // "", daily, weekly, monthly
std::optional<bool> include_byok_in_limit;
std::string expires_at; // YYYY-MM-DDTHH:MM:SSZ (seconds required)
std::string workspace_id; // uuid
std::string creator_user_id;
std::string external_user; // partner attribution
void validate() const;
Json to_json() const;
};
// Body of PATCH /keys/{hash}; only the set fields are sent.
struct KeyUpdate {
std::optional<std::string> name;
std::optional<bool> disabled;
std::optional<double> limit;
std::string limit_reset; // "", daily, weekly, monthly
bool clear_limit_reset = false; // send limit_reset:null
std::optional<bool> include_byok_in_limit;
std::string workspace_id;
void validate() const;
Json to_json() const;
};
class KeysService : public Endpoint {
public:
// Keys are managed with OPENROUTER_MANAGEMENT_KEY, never the inference key.
explicit KeysService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
// limit 1..100 (vendor), offset >= 0.
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
bool include_disabled = false, bool include_expired = false,
const std::string& workspace_id = "") const;
Json get(const std::string& key_hash) const;
// The response carries the new secret `key` exactly once. Callers must not log it.
Json create(const KeyDraft& draft) const;
Json update(const std::string& key_hash, const KeyUpdate& update) const;
// Irreversible. No default; the caller decides.
Json remove(const std::string& key_hash) const;
};
// True for a 64-character lowercase or uppercase hex string (SHA-256).
bool is_key_hash(const std::string& s);
// "YYYY-MM-DDTHH:MM:SS[.fff]Z" shape check (seconds required).
bool is_iso_utc_seconds(const std::string& s);
// UUID 8-4-4-4-12 hex shape check.
bool is_uuid(const std::string& s);
} // namespace molo::openrouter
+332
View File
@@ -0,0 +1,332 @@
#include "openrouter/admin.h"
#include "llm/error.h"
#include "openrouter/account.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
Json string_list_body(const char* key, const std::vector<std::string>& values, bool hashes = false) {
if (values.empty()) bad(std::string(key) + " must not be empty");
if (values.size() > Caps::kMaxBulkIds) bad(std::string(key) + " has too many entries");
Json a = Json::array();
for (const std::string& v : values) {
if (hashes ? !is_key_hash(v) : (v.empty() || v.size() > 256)) bad(std::string(key) + " has a malformed entry");
a.push(Json(v));
}
Json b = Json::object();
b.set(key, a);
return b;
}
void check_body(const Json& body, const char* what) {
if (!body.is_object() || body.as_object().empty()) bad(std::string(what) + " needs a non-empty JSON object");
}
Json secret_body(const std::string& value, const std::vector<std::string>& hosts) {
if (value.empty() || value.size() > 65536) bad("secret value must be 1..65536 characters");
if (hosts.empty() || hosts.size() > 100) bad("secret hosts must hold 1..100 hostnames");
Json h = Json::array();
for (const std::string& host : hosts) {
if (host.empty() || host.size() > 254) bad("secret host must be 1..254 characters");
for (char c : host)
if (c == '/' || c == ':' || c == '*' || c == ' ') bad("secret host must be a bare hostname");
h.push(Json(host));
}
Json b = Json::object();
b.set("value", Json(value));
b.set("hosts", h);
return b;
}
} // namespace
Query page_query(std::optional<int> offset, std::optional<int> limit, int max_limit) {
if (offset && *offset < 0) bad("offset must be >= 0");
if (limit && (*limit < 1 || *limit > max_limit)) bad("limit must be 1.." + std::to_string(max_limit));
Query q;
if (offset) q.add("offset", *offset);
if (limit) q.add("limit", *limit);
return q;
}
// --- guardrails ------------------------------------------------------------
Json GuardrailsService::list(std::optional<int> o, std::optional<int> l, const std::string& ws) const {
Query q = page_query(o, l);
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
q.add_if("workspace_id", ws);
return run("listGuardrails", {}, q);
}
Json GuardrailsService::get(const std::string& id) const { return run("getGuardrail", {{"id", id}}); }
Json GuardrailsService::create(const Json& body) const {
check_body(body, "guardrail");
if (!body.has("name")) bad("guardrail needs a name");
return run("createGuardrail", {}, {}, body);
}
Json GuardrailsService::update(const std::string& id, const Json& body) const {
check_body(body, "guardrail update");
return run("updateGuardrail", {{"id", id}}, {}, body);
}
Json GuardrailsService::remove(const std::string& id) const { return run("deleteGuardrail", {{"id", id}}); }
Json GuardrailsService::key_assignments(const std::string& id, std::optional<int> o, std::optional<int> l) const {
return run("listGuardrailKeyAssignments", {{"id", id}}, page_query(o, l));
}
Json GuardrailsService::assign_keys(const std::string& id, const std::vector<std::string>& h) const {
return run("bulkAssignKeysToGuardrail", {{"id", id}}, {}, string_list_body("key_hashes", h, true));
}
Json GuardrailsService::unassign_keys(const std::string& id, const std::vector<std::string>& h) const {
return run("bulkUnassignKeysFromGuardrail", {{"id", id}}, {}, string_list_body("key_hashes", h, true));
}
Json GuardrailsService::member_assignments(const std::string& id, std::optional<int> o, std::optional<int> l) const {
return run("listGuardrailMemberAssignments", {{"id", id}}, page_query(o, l));
}
Json GuardrailsService::assign_members(const std::string& id, const std::vector<std::string>& u) const {
return run("bulkAssignMembersToGuardrail", {{"id", id}}, {}, string_list_body("member_user_ids", u));
}
Json GuardrailsService::unassign_members(const std::string& id, const std::vector<std::string>& u) const {
return run("bulkUnassignMembersFromGuardrail", {{"id", id}}, {}, string_list_body("member_user_ids", u));
}
Json GuardrailsService::all_key_assignments(std::optional<int> o, std::optional<int> l) const {
return run("listKeyAssignments", {}, page_query(o, l));
}
Json GuardrailsService::all_member_assignments(std::optional<int> o, std::optional<int> l) const {
return run("listMemberAssignments", {}, page_query(o, l));
}
// --- BYOK ------------------------------------------------------------------
Json ByokService::list(std::optional<int> o, std::optional<int> l, const std::string& ws,
const std::string& provider) const {
Query q = page_query(o, l);
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
q.add_if("workspace_id", ws);
q.add_if("provider", provider);
return run("listBYOKKeys", {}, q);
}
Json ByokService::get(const std::string& id) const { return run("getBYOKKey", {{"id", id}}); }
Json ByokService::create(const Json& body) const {
check_body(body, "BYOK credential");
if (!body.has("provider") || !body.has("key")) bad("BYOK credential needs provider and key");
return run("createBYOKKey", {}, {}, body);
}
Json ByokService::update(const std::string& id, const Json& body) const {
check_body(body, "BYOK update");
return run("updateBYOKKey", {{"id", id}}, {}, body);
}
Json ByokService::remove(const std::string& id) const { return run("deleteBYOKKey", {{"id", id}}); }
// --- workspaces ------------------------------------------------------------
bool WorkspacesService::valid_interval(const std::string& i) {
return i == "daily" || i == "weekly" || i == "monthly" || i == "lifetime";
}
Json WorkspacesService::list(std::optional<int> o, std::optional<int> l) const {
return run("listWorkspaces", {}, page_query(o, l));
}
Json WorkspacesService::get(const std::string& id) const { return run("getWorkspace", {{"id", id}}); }
Json WorkspacesService::create(const Json& body) const {
check_body(body, "workspace");
if (!body.has("name") || !body.has("slug")) bad("workspace needs name and slug");
return run("createWorkspace", {}, {}, body);
}
Json WorkspacesService::update(const std::string& id, const Json& body) const {
check_body(body, "workspace update");
return run("updateWorkspace", {{"id", id}}, {}, body);
}
Json WorkspacesService::remove(const std::string& id, bool confirm_default) const {
Query q;
if (confirm_default) q.add("confirm_default_workspace_deletion", true);
return run("deleteWorkspace", {{"id", id}}, q);
}
Json WorkspacesService::members(const std::string& id, std::optional<int> o, std::optional<int> l) const {
return run("listWorkspaceMembers", {{"id", id}}, page_query(o, l));
}
Json WorkspacesService::add_members(const std::string& id, const std::vector<std::string>& u) const {
return run("bulkAddWorkspaceMembers", {{"id", id}}, {}, string_list_body("user_ids", u));
}
Json WorkspacesService::remove_members(const std::string& id, const std::vector<std::string>& u) const {
return run("bulkRemoveWorkspaceMembers", {{"id", id}}, {}, string_list_body("user_ids", u));
}
Json WorkspacesService::budgets(const std::string& ref) const {
return run("listWorkspaceBudgets", {{"workspace_ref", ref}});
}
Json WorkspacesService::budget(const std::string& ref, const std::string& interval) const {
if (!valid_interval(interval)) bad("interval must be daily weekly monthly or lifetime");
return run("getWorkspaceBudget", {{"workspace_ref", ref}, {"interval", interval}});
}
Json WorkspacesService::set_budget(const std::string& ref, const std::string& interval, double limit_usd,
std::optional<bool> include_byok) const {
if (!valid_interval(interval)) bad("interval must be daily weekly monthly or lifetime");
if (!(limit_usd > 0)) bad("limit_usd must be greater than 0");
Json b = Json::object();
b.set("limit_usd", Json(limit_usd));
if (include_byok) b.set("include_byok_in_budgets", Json(*include_byok));
return run("upsertWorkspaceBudget", {{"workspace_ref", ref}, {"interval", interval}}, {}, b);
}
Json WorkspacesService::remove_budget(const std::string& ref, const std::string& interval) const {
if (!valid_interval(interval)) bad("interval must be daily weekly monthly or lifetime");
return run("deleteWorkspaceBudget", {{"workspace_ref", ref}, {"interval", interval}});
}
// --- observability -----------------------------------------------------------
Json ObservabilityService::list(std::optional<int> o, std::optional<int> l, const std::string& ws) const {
Query q = page_query(o, l);
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
q.add_if("workspace_id", ws);
return run("listObservabilityDestinations", {}, q);
}
Json ObservabilityService::get(const std::string& id) const {
return run("getObservabilityDestination", {{"id", id}});
}
Json ObservabilityService::create(const Json& body) const {
check_body(body, "observability destination");
if (!body.has("type") || !body.has("name") || !body.has("config"))
bad("observability destination needs type, name and config");
return run("createObservabilityDestination", {}, {}, body);
}
Json ObservabilityService::update(const std::string& id, const Json& body) const {
check_body(body, "observability update");
return run("updateObservabilityDestination", {{"id", id}}, {}, body);
}
Json ObservabilityService::remove(const std::string& id) const {
return run("deleteObservabilityDestination", {{"id", id}});
}
// --- private endpoints ---------------------------------------------------------
Json PrivateEndpointsService::list() const { return run("listPrivateEndpoints"); }
Json PrivateEndpointsService::get(const std::string& id) const { return run("getPrivateEndpoint", {{"id", id}}); }
Json PrivateEndpointsService::create(const Json& body) const {
check_body(body, "private endpoint");
return run("createPrivateEndpoint", {}, {}, body);
}
Json PrivateEndpointsService::update(const std::string& id, const Json& body) const {
check_body(body, "private endpoint update");
return run("updatePrivateEndpoint", {{"id", id}}, {}, body);
}
Json PrivateEndpointsService::remove(const std::string& id, bool draft_only) const {
Query q;
if (draft_only) q.add("draft_only", true);
return run("deletePrivateEndpoint", {{"id", id}}, q);
}
Json PrivateEndpointsService::activate(const std::string& id) const { return run("activatePrivateEndpoint", {{"id", id}}); }
Json PrivateEndpointsService::enable(const std::string& id) const { return run("enablePrivateEndpoint", {{"id", id}}); }
Json PrivateEndpointsService::disable(const std::string& id) const { return run("disablePrivateEndpoint", {{"id", id}}); }
Json PrivateEndpointsService::set_pricing(const std::string& id, const Json& pricing) const {
check_body(pricing, "pricing");
Json b = Json::object();
b.set("pricing", pricing);
return run("updatePrivateEndpointPricing", {{"id", id}}, {}, b);
}
Json PrivateEndpointsService::validate(const std::string& id, const std::string& ws) const {
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
if (ws.empty()) return run("validatePrivateEndpoint", {{"id", id}}, {}, Json::object());
Json b = Json::object();
b.set("workspace_id", Json(ws));
return run("validatePrivateEndpoint", {{"id", id}}, {}, b);
}
// --- directory ---------------------------------------------------------------------
Json DirectoryService::external_users(std::optional<int> o, std::optional<int> l, const std::string& user,
bool include_inactive) const {
Query q = page_query(o, l);
q.add_if("user", user);
if (include_inactive) q.add("include_inactive", true);
return run("listExternalUsers", {}, q);
}
Json DirectoryService::external_user(const std::string& user) const { return run("getExternalUser", {{"user", user}}); }
Json DirectoryService::register_external_user(const std::string& user) const {
if (user.empty() || user.size() > 512 || user.find('\0') != std::string::npos)
bad("external user must be 1..512 characters without NUL");
Json b = Json::object();
b.set("user", Json(user));
return run("createExternalUser", {}, {}, b);
}
Json DirectoryService::set_external_user_active(const std::string& user, bool active) const {
Json b = Json::object();
b.set("is_active", Json(active));
return run("updateExternalUser", {{"user", user}}, {}, b);
}
Json DirectoryService::deactivate_external_user(const std::string& user) const {
return run("deleteExternalUser", {{"user", user}});
}
Json DirectoryService::organization_members(std::optional<int> o, std::optional<int> l) const {
return run("listOrganizationMembers", {}, page_query(o, l));
}
Json DirectoryService::organization_settings() const { return run("getOrganizationSettings"); }
Json DirectoryService::set_filtered_model_catalog(bool enabled) const {
Json b = Json::object();
b.set("is_filtered_model_catalog_enabled", Json(enabled));
return run("updateOrganizationSettings", {}, {}, b);
}
Json DirectoryService::scim_group_mappings(std::optional<int> o, std::optional<int> l) const {
return run("listScimGroupMappings", {}, page_query(o, l));
}
Json DirectoryService::scim_group_mapping(const std::string& id) const {
return run("getScimGroupMapping", {{"id", id}});
}
Json DirectoryService::create_scim_group_mapping(const Json& body) const {
check_body(body, "SCIM group mapping");
if (!body.has("scim_group_id") || !body.has("role")) bad("SCIM group mapping needs scim_group_id and role");
return run("createScimGroupMapping", {}, {}, body);
}
Json DirectoryService::update_scim_group_mapping(const std::string& id, const std::string& role) const {
if (role.empty()) bad("role must not be empty");
Json b = Json::object();
b.set("role", Json(role));
return run("updateScimGroupMapping", {{"id", id}}, {}, b);
}
Json DirectoryService::remove_scim_group_mapping(const std::string& id, bool keep_members) const {
Query q;
if (keep_members) q.add("keep_members", true);
return run("deleteScimGroupMapping", {{"id", id}}, q);
}
Json DirectoryService::scim_groups(std::optional<int> o, std::optional<int> l, const std::string& display_name,
const std::string& external_id) const {
Query q = page_query(o, l);
q.add_if("display_name", display_name);
q.add_if("external_id", external_id);
return run("listScimGroups", {}, q);
}
Json DirectoryService::start_scim_sync() const { return run("createScimSyncJob"); }
Json DirectoryService::scim_sync_job(const std::string& id) const { return run("getScimSyncJob", {{"id", id}}); }
// --- vault -----------------------------------------------------------------------------
Json VaultService::secrets(std::optional<int> o, std::optional<int> l) const {
return run("listVaultSecrets", {}, page_query(o, l));
}
Json VaultService::put_secret(const std::string& name, const std::string& value,
const std::vector<std::string>& hosts) const {
return run("storeVaultSecret", {{"name", name}}, {}, secret_body(value, hosts));
}
Json VaultService::delete_secret(const std::string& name) const { return run("deleteVaultSecret", {{"name", name}}); }
Json VaultService::intern_secrets(const std::string& intern, std::optional<int> o, std::optional<int> l) const {
return run("listInternVaultSecrets", {{"internId", intern}}, page_query(o, l));
}
Json VaultService::intern_effective_secrets(const std::string& intern, std::optional<int> o,
std::optional<int> l) const {
return run("listInternEffectiveVaultSecrets", {{"internId", intern}}, page_query(o, l));
}
Json VaultService::intern_connections(const std::string& intern) const {
return run("listInternConnections", {{"internId", intern}});
}
Json VaultService::put_intern_secret(const std::string& intern, const std::string& name,
const std::string& value, const std::vector<std::string>& hosts) const {
return run("storeInternVaultSecret", {{"internId", intern}, {"name", name}}, {}, secret_body(value, hosts));
}
Json VaultService::delete_intern_secret(const std::string& intern, const std::string& name) const {
return run("deleteInternVaultSecret", {{"internId", intern}, {"name", name}});
}
Json VaultService::copy_secrets_to_intern(const std::string& intern, const std::vector<std::string>& names) const {
return run("copyVaultSecretsToIntern", {{"internId", intern}}, {}, string_list_body("names", names));
}
} // namespace molo::openrouter
+155
View File
@@ -0,0 +1,155 @@
#pragma once
// Management surface. All classes here use OPENROUTER_MANAGEMENT_KEY (the
// separate management / provisioning key), except VaultService, which the
// spec secures with the normal API key. Request bodies are JSON objects whose
// top level keys are checked against the call table (calls.h); the large
// nested shapes (guardrail content filters, observability destination
// configs, private endpoint pricing) are passed through as given.
//
// Every class that can change or delete something exposes those calls as
// separate explicit methods (create, update, remove, ...), never with a
// default. The demo layer adds the dry-run gate.
#include <optional>
#include <string>
#include <vector>
#include "openrouter/gateway.h"
namespace molo::openrouter {
// offset >= 0, limit 1..max_limit; throws llm::ApiError::invalid otherwise.
Query page_query(std::optional<int> offset, std::optional<int> limit, int max_limit = 1000);
class GuardrailsService : public Endpoint {
public:
explicit GuardrailsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& workspace_id = "") const;
Json get(const std::string& id) const;
Json create(const Json& body) const; // name is required by the spec
Json update(const std::string& id, const Json& body) const;
Json remove(const std::string& id) const;
Json key_assignments(const std::string& id, std::optional<int> offset = std::nullopt,
std::optional<int> limit = std::nullopt) const;
Json assign_keys(const std::string& id, const std::vector<std::string>& key_hashes) const;
Json unassign_keys(const std::string& id, const std::vector<std::string>& key_hashes) const;
Json member_assignments(const std::string& id, std::optional<int> offset = std::nullopt,
std::optional<int> limit = std::nullopt) const;
Json assign_members(const std::string& id, const std::vector<std::string>& user_ids) const;
Json unassign_members(const std::string& id, const std::vector<std::string>& user_ids) const;
Json all_key_assignments(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
Json all_member_assignments(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
};
// BYOK: provider credentials you bring. create() sends a provider secret in
// `key`; it is never echoed back and must never be logged.
class ByokService : public Endpoint {
public:
explicit ByokService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& workspace_id = "", const std::string& provider = "") const;
Json get(const std::string& id) const;
Json create(const Json& body) const;
Json update(const std::string& id, const Json& body) const;
Json remove(const std::string& id) const;
};
class WorkspacesService : public Endpoint {
public:
explicit WorkspacesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
Json get(const std::string& id) const;
Json create(const Json& body) const;
Json update(const std::string& id, const Json& body) const;
// The default workspace is only deleted when confirm_default is true.
Json remove(const std::string& id, bool confirm_default) const;
Json members(const std::string& id, std::optional<int> offset = std::nullopt,
std::optional<int> limit = std::nullopt) const;
Json add_members(const std::string& id, const std::vector<std::string>& user_ids) const;
Json remove_members(const std::string& id, const std::vector<std::string>& user_ids) const;
Json budgets(const std::string& workspace_ref) const;
Json budget(const std::string& workspace_ref, const std::string& interval) const;
// interval: daily weekly monthly lifetime. limit_usd > 0.
Json set_budget(const std::string& workspace_ref, const std::string& interval, double limit_usd,
std::optional<bool> include_byok_in_budgets = std::nullopt) const;
Json remove_budget(const std::string& workspace_ref, const std::string& interval) const;
static bool valid_interval(const std::string& interval);
};
class ObservabilityService : public Endpoint {
public:
explicit ObservabilityService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& workspace_id = "") const;
Json get(const std::string& id) const;
Json create(const Json& body) const;
Json update(const std::string& id, const Json& body) const;
Json remove(const std::string& id) const;
};
class PrivateEndpointsService : public Endpoint {
public:
explicit PrivateEndpointsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
Json list() const;
Json get(const std::string& id) const;
Json create(const Json& body) const;
Json update(const std::string& id, const Json& body) const;
Json remove(const std::string& id, bool draft_only) const;
Json activate(const std::string& id) const;
Json enable(const std::string& id) const;
Json disable(const std::string& id) const;
Json set_pricing(const std::string& id, const Json& pricing) const;
Json validate(const std::string& id, const std::string& workspace_id = "") const;
};
// End users (external users), organization and SCIM.
class DirectoryService : public Endpoint {
public:
explicit DirectoryService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
// External users.
Json external_users(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& user = "", bool include_inactive = false) const;
Json external_user(const std::string& user) const;
Json register_external_user(const std::string& user) const;
Json set_external_user_active(const std::string& user, bool active) const;
Json deactivate_external_user(const std::string& user) const;
// Organization.
Json organization_members(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
Json organization_settings() const;
Json set_filtered_model_catalog(bool enabled) const;
// SCIM.
Json scim_group_mappings(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
Json scim_group_mapping(const std::string& id) const;
Json create_scim_group_mapping(const Json& body) const;
Json update_scim_group_mapping(const std::string& id, const std::string& role) const;
Json remove_scim_group_mapping(const std::string& id, bool keep_members) const;
Json scim_groups(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& display_name = "", const std::string& external_id = "") const;
Json start_scim_sync() const;
Json scim_sync_job(const std::string& id) const;
};
// Secrets vault for workspaces and interns. Secured with OPENROUTER_API_KEY.
class VaultService : public Endpoint {
public:
explicit VaultService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
Json secrets(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
// value: 1..65536 characters, hosts: 1..100 exact hostnames. The value is
// never returned and must never be logged.
Json put_secret(const std::string& name, const std::string& value,
const std::vector<std::string>& hosts) const;
Json delete_secret(const std::string& name) const;
Json intern_secrets(const std::string& intern_id, std::optional<int> offset = std::nullopt,
std::optional<int> limit = std::nullopt) const;
Json intern_effective_secrets(const std::string& intern_id, std::optional<int> offset = std::nullopt,
std::optional<int> limit = std::nullopt) const;
Json intern_connections(const std::string& intern_id) const;
Json put_intern_secret(const std::string& intern_id, const std::string& name,
const std::string& value, const std::vector<std::string>& hosts) const;
Json delete_intern_secret(const std::string& intern_id, const std::string& name) const;
Json copy_secrets_to_intern(const std::string& intern_id, const std::vector<std::string>& names) const;
};
} // namespace molo::openrouter
+178
View File
@@ -0,0 +1,178 @@
#include "openrouter/call_plan.h"
#include <cstdlib>
#include <cstring>
#include <format>
#include <sstream>
#include "clients/url.h"
#include "llm/api_client.h"
#include "llm/error.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) {
throw llm::ApiError::invalid(kVendor, why);
}
std::set<std::string> split_fields(const char* fields, bool want_query) {
std::set<std::string> out;
std::istringstream in(fields);
for (std::string f; in >> f;) {
if (f == "-") continue;
bool is_query = f.front() == '?';
if (is_query == want_query) out.insert(is_query ? f.substr(1) : f);
}
return out;
}
bool free_text_arg(const std::string& placeholder) {
// External user ids are partner-defined text; everything else is a slug,
// hash or uuid and goes through Service::valid_id.
return placeholder == "user";
}
} // namespace
Query& Query::add(const std::string& name, const std::string& value) {
if (items_.size() >= Caps::kMaxQueryParams) bad("too many query parameters");
if (value.size() > Caps::kMaxQueryValueBytes) bad("query value too long: " + name);
items_.emplace_back(name, value);
return *this;
}
Query& Query::add(const std::string& name, long value) { return add(name, std::to_string(value)); }
Query& Query::add(const std::string& name, double value) { return add(name, std::format("{}", value)); }
Query& Query::add(const std::string& name, bool value) { return add(name, std::string(value ? "true" : "false")); }
Query& Query::add_if(const std::string& name, const std::string& value) {
if (!value.empty()) add(name, value);
return *this;
}
const CallSpec* CallPlanner::find(std::string_view name) {
for (const CallSpec& s : kCalls)
if (name == s.name) return &s;
return nullptr;
}
const CallSpec& CallPlanner::spec(std::string_view name) {
const CallSpec* s = find(name);
if (!s) bad("unknown call: " + std::string(name));
return *s;
}
KeyKind CallPlanner::key_kind(const CallSpec& spec) {
if (std::strcmp(spec.key, "mgmt") == 0) return KeyKind::kManagement;
if (std::strcmp(spec.key, "none") == 0) return KeyKind::kNone;
return KeyKind::kApi;
}
clients::Method CallPlanner::method_of(const CallSpec& spec) {
if (!std::strcmp(spec.http, "GET")) return clients::Method::kGet;
if (!std::strcmp(spec.http, "POST")) return clients::Method::kPost;
if (!std::strcmp(spec.http, "PUT")) return clients::Method::kPut;
if (!std::strcmp(spec.http, "PATCH")) return clients::Method::kPatch;
if (!std::strcmp(spec.http, "DELETE")) return clients::Method::kDelete;
bad(std::string("bad method in table for ") + spec.name);
}
std::set<std::string> CallPlanner::body_fields(const CallSpec& spec) {
return split_fields(spec.fields, false);
}
std::set<std::string> CallPlanner::query_fields(const CallSpec& spec) {
return split_fields(spec.fields, true);
}
std::vector<std::string> CallPlanner::placeholders(const std::string& tmpl) {
std::vector<std::string> out;
std::size_t pos = 0;
while ((pos = tmpl.find('{', pos)) != std::string::npos) {
std::size_t end = tmpl.find('}', pos);
if (end == std::string::npos) break;
out.push_back(tmpl.substr(pos + 1, end - pos - 1));
pos = end + 1;
}
return out;
}
bool CallPlanner::accepts_field(std::string_view name, const std::string& field) {
const CallSpec* s = find(name);
return s && body_fields(*s).count(field) > 0;
}
PlannedCall CallPlanner::plan(std::string_view name, const PathArgs& args,
const Query& query, const Json* body) {
const CallSpec& s = spec(name);
PlannedCall out;
out.spec = &s;
out.method = method_of(s);
out.key = key_kind(s);
std::string path = s.path;
std::set<std::string> used;
for (const std::string& ph : placeholders(s.path)) {
auto it = args.find(ph);
if (it == args.end()) bad("missing path argument: " + ph);
const std::string& value = it->second;
if (free_text_arg(ph)) {
// Partner text of 1..512 bytes; "." and ".." cannot be addressed by path.
if (value.empty() || value.size() > 512 || value == "." || value == ".." ||
value.find('\0') != std::string::npos)
bad("bad " + ph);
} else if (!llm::Service::valid_id(ph == "author" && value.rfind("~", 0) == 0
? value.substr(1)
: value)) {
bad("bad " + ph + " id"); // author may carry the alias marker "~"
}
std::string token = "{" + ph + "}";
// valid_id characters are all path-safe; only free text is encoded.
path.replace(path.find(token), token.size(),
free_text_arg(ph) ? clients::Url::encode_component(value) : value);
used.insert(ph);
}
for (const auto& [k, v] : args) {
(void)v;
if (!used.count(k)) bad("unexpected path argument: " + k);
}
out.path = path;
std::set<std::string> qf = query_fields(s);
std::string qs;
for (const auto& [k, v] : query.items()) {
if (!qf.count(k)) bad("query parameter not declared for " + std::string(name) + ": " + k);
if (v.empty()) bad("empty query value: " + k);
qs += (qs.empty() ? "?" : "&") + clients::Url::encode_component(k) + "=" +
clients::Url::encode_component(v);
}
out.query = qs;
std::set<std::string> bf = body_fields(s);
if (body) {
if (bf.empty()) bad(std::string(name) + " takes no request body");
if (!body->is_object()) bad("request body must be a JSON object");
for (const auto& [k, v] : body->as_object()) {
(void)v;
if (!bf.count(k)) bad("body field not declared for " + std::string(name) + ": " + k);
}
out.has_body = true;
out.body = *body;
}
return out;
}
std::pair<std::string, std::string> split_model_id(const std::string& model_id) {
std::size_t slash = model_id.find('/');
if (slash == std::string::npos || slash == 0 || slash + 1 >= model_id.size() ||
model_id.find('/', slash + 1) != std::string::npos)
bad("model id must look like author/slug");
std::string author = model_id.substr(0, slash);
std::string slug = model_id.substr(slash + 1);
if (!llm::Service::valid_id(author.rfind("~", 0) == 0 ? author.substr(1) : author) ||
!llm::Service::valid_id(slug))
bad("bad model id");
return {author, slug};
}
} // namespace molo::openrouter
+87
View File
@@ -0,0 +1,87 @@
#pragma once
// CallPlanner: turns a call name from the table (calls.h) plus path
// arguments, query arguments and a JSON body into a PlannedCall, and rejects
// anything the table does not declare (unknown call, missing or malformed
// path argument, undeclared query parameter, undeclared body field, body on a
// call without one). Nothing here touches the network, so every request the
// module can send is checkable against the spec-verified table.
#include <map>
#include <set>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
#include "clients/body/json.h"
#include "clients/request.h"
#include "openrouter/calls.h"
namespace molo::openrouter {
using Json = clients::body::Json;
inline constexpr const char* kVendor = "openrouter";
using PathArgs = std::map<std::string, std::string>;
// Ordered query parameters (order is preserved on the wire).
class Query {
public:
Query& add(const std::string& name, const std::string& value);
Query& add(const std::string& name, const char* value) { return add(name, std::string(value)); }
Query& add(const std::string& name, long value);
Query& add(const std::string& name, int value) { return add(name, static_cast<long>(value)); }
Query& add(const std::string& name, double value);
Query& add(const std::string& name, bool value);
// Adds only when the value is non-empty.
Query& add_if(const std::string& name, const std::string& value);
const std::vector<std::pair<std::string, std::string>>& items() const { return items_; }
bool empty() const { return items_.empty(); }
private:
std::vector<std::pair<std::string, std::string>> items_;
};
enum class KeyKind { kApi, kManagement, kNone };
struct PlannedCall {
const CallSpec* spec = nullptr;
clients::Method method = clients::Method::kGet;
std::string path; // placeholders substituted, segments URL-encoded
std::string query; // "" or "?a=b&c=d"
bool has_body = false;
Json body;
KeyKind key = KeyKind::kApi;
std::string name() const { return spec ? spec->name : ""; }
std::string fmt() const { return spec ? spec->fmt : "json"; }
std::string target() const { return path + query; }
};
class CallPlanner {
public:
static const CallSpec* find(std::string_view name);
// Throws llm::ApiError::invalid for an unknown name.
static const CallSpec& spec(std::string_view name);
static KeyKind key_kind(const CallSpec& spec);
static clients::Method method_of(const CallSpec& spec);
static std::set<std::string> body_fields(const CallSpec& spec);
static std::set<std::string> query_fields(const CallSpec& spec);
static std::vector<std::string> placeholders(const std::string& path_template);
// Validates and builds. `body` may be null. Throws llm::ApiError::invalid.
static PlannedCall plan(std::string_view name, const PathArgs& args = {},
const Query& query = {}, const Json* body = nullptr);
// True when `field` may be sent in the body of `name`.
static bool accepts_field(std::string_view name, const std::string& field);
};
// Splits "author/slug[:variant]" into its two path segments. Throws
// llm::ApiError::invalid when it is not exactly two valid segments.
std::pair<std::string, std::string> split_model_id(const std::string& model_id);
} // namespace molo::openrouter
+181
View File
@@ -0,0 +1,181 @@
#pragma once
// The ONE table of every OpenRouter call this module makes. Each row is
// checked by tools/check_spec.py against OpenRouter's OpenAPI document
// (https://openrouter.ai/openapi.json, 139 operations when this table was
// written, plus one documented-only row, see SPEC.md "Verified against
// spec"). One row per line, in exactly this shape (the checker parses it):
// {"name", "HTTP", "/path/{param}", "operationId", "key", "fmt", "fields"},
//
// operationId the spec's operationId, or "doc:<slug>" for a documented
// route the OpenAPI document does not list (checked live only).
// key "api" : OPENROUTER_API_KEY (bearer)
// "mgmt" : OPENROUTER_MANAGEMENT_KEY (bearer, separate key)
// "none" : public route, works without any key
// fmt "json" | "multipart" | "form" | "raw" (response is bytes)
// fields space separated: top level request body field names,
// "?name" for a query parameter, "-" for none.
#include <cstddef>
#include <string_view>
namespace molo::openrouter {
struct CallSpec {
const char* name;
const char* http;
const char* path;
const char* op;
const char* key;
const char* fmt;
const char* fields;
};
// clang-format off
inline constexpr CallSpec kCalls[] = {
{"getUserActivity", "GET", "/activity", "getUserActivity", "mgmt", "json", "?date ?api_key_hash ?user_id ?group_by ?workspace_id"},
{"getAnalyticsMeta", "GET", "/analytics/meta", "getAnalyticsMeta", "mgmt", "json", "-"},
{"queryAnalytics", "POST", "/analytics/query", "queryAnalytics", "mgmt", "json", "classifier_dimensions classifier_filters dimensions filters granularity group_limit limit metrics order_by time_range"},
{"createApiAlphaDecisions", "POST", "/api/alpha/decisions", "createApiAlphaDecisions", "api", "json", "model provider questions session_id state trace user"},
{"createAudioSpeech", "POST", "/audio/speech", "createAudioSpeech", "api", "raw", "input input_references instructions model provider response_format session_id speed trace user voice"},
{"createAudioTranscriptions", "POST", "/audio/transcriptions", "createAudioTranscriptions", "api", "json", "diarize input_audio keyterms language model provider response_format session_id temperature timestamp_granularities trace user"},
{"exchangeAuthCodeForAPIKey", "POST", "/auth/keys", "exchangeAuthCodeForAPIKey", "none", "json", "code code_challenge_method code_verifier"},
{"createAuthKeysCode", "POST", "/auth/keys/code", "createAuthKeysCode", "api", "json", "callback_url code_challenge code_challenge_method expires_at key_label limit spawn_agent spawn_cloud usage_limit_type workspace_id"},
{"listBatches", "GET", "/batches", "listBatches", "api", "json", "?limit ?after ?status ?created_after ?created_before"},
{"createBatches", "POST", "/batches", "createBatches", "api", "json", "callback_url completion_window endpoint model provider requests"},
{"deleteBatch", "DELETE", "/batches/{id}", "deleteBatch", "api", "json", "-"},
{"getBatches", "GET", "/batches/{id}", "getBatches", "api", "json", "-"},
{"getBenchmarks", "GET", "/benchmarks", "getBenchmarks", "api", "json", "?source ?task_type ?benchmark_type ?include_run_config ?search_engine ?search_surface ?arena ?category ?max_results"},
{"listBYOKKeys", "GET", "/byok", "listBYOKKeys", "mgmt", "json", "?offset ?limit ?workspace_id ?provider"},
{"createBYOKKey", "POST", "/byok", "createBYOKKey", "mgmt", "json", "allowed_api_key_hashes allowed_models allowed_user_ids declared_region declared_zdr disabled is_byok_only is_fallback is_required key name provider workspace_id"},
{"deleteBYOKKey", "DELETE", "/byok/{id}", "deleteBYOKKey", "mgmt", "json", "-"},
{"getBYOKKey", "GET", "/byok/{id}", "getBYOKKey", "mgmt", "json", "-"},
{"updateBYOKKey", "PATCH", "/byok/{id}", "updateBYOKKey", "mgmt", "json", "allowed_api_key_hashes allowed_models allowed_user_ids declared_region declared_zdr disabled is_byok_only is_fallback is_required key name"},
{"sendChatCompletionRequest", "POST", "/chat/completions", "sendChatCompletionRequest", "api", "json", "cache_control debug deferred_tools frequency_penalty image_config logit_bias logprobs max_completion_tokens max_tokens messages metadata min_p modalities model models parallel_tool_calls plugins prediction presence_penalty prompt_cache_key prompt_cache_options provider reasoning reasoning_effort repetition_penalty response_format route seed service_tier session_id stop stop_server_tools_when stream stream_options temperature tool_choice tools top_a top_k top_logprobs top_p trace user"},
{"getTaskClassifications", "GET", "/classifications/task", "getTaskClassifications", "api", "json", "?window"},
{"createCompletions", "POST", "/completions", "doc:completions", "api", "json", "model models prompt suffix max_tokens temperature top_p top_k stop stream seed frequency_penalty presence_penalty repetition_penalty logit_bias logprobs top_logprobs min_p top_a n echo best_of user provider route plugins session_id response_format stream_options reasoning transforms"},
{"listContainerFiles", "GET", "/containers/{container_id}/files", "listContainerFiles", "api", "json", "?limit ?after"},
{"getContainerFile", "GET", "/containers/{container_id}/files/{file_id}", "getContainerFile", "api", "json", "-"},
{"downloadContainerFileContent", "GET", "/containers/{container_id}/files/{file_id}/content", "downloadContainerFileContent", "api", "raw", "-"},
{"promoteContainerFile", "POST", "/containers/{container_id}/files/{file_id}/promote", "promoteContainerFile", "api", "json", "-"},
{"getCredits", "GET", "/credits", "getCredits", "mgmt", "json", "-"},
{"createCoinbaseCharge", "POST", "/credits/coinbase", "createCoinbaseCharge", "none", "json", "-"},
{"getAppRankings", "GET", "/datasets/app-rankings", "getAppRankings", "api", "json", "?category ?subcategory ?sort ?start_date ?end_date ?limit ?offset"},
{"getRankingsDaily", "GET", "/datasets/rankings-daily", "getRankingsDaily", "api", "json", "?start_date ?end_date ?period ?modality ?context_bucket ?category ?language_type"},
{"getSessionCost", "GET", "/datasets/session-cost", "getSessionCost", "api", "json", "?app_slug ?model ?turn_range ?limit ?offset"},
{"createEmbeddings", "POST", "/embeddings", "createEmbeddings", "api", "json", "dimensions encoding_format input input_type model provider session_id trace user"},
{"listEmbeddingsModels", "GET", "/embeddings/models", "listEmbeddingsModels", "none", "json", "?offset ?limit"},
{"listEndpointsZdr", "GET", "/endpoints/zdr", "listEndpointsZdr", "none", "json", "-"},
{"listExternalUsers", "GET", "/external-users", "listExternalUsers", "mgmt", "json", "?offset ?limit ?user ?include_inactive"},
{"createExternalUser", "POST", "/external-users", "createExternalUser", "mgmt", "json", "user"},
{"deleteExternalUser", "DELETE", "/external-users/{user}", "deleteExternalUser", "mgmt", "json", "-"},
{"getExternalUser", "GET", "/external-users/{user}", "getExternalUser", "mgmt", "json", "-"},
{"updateExternalUser", "PATCH", "/external-users/{user}", "updateExternalUser", "mgmt", "json", "is_active"},
{"listFiles", "GET", "/files", "listFiles", "api", "json", "?limit ?cursor ?workspace_id ?provider ?after ?after_id ?before_id ?order"},
{"uploadFile", "POST", "/files", "uploadFile", "api", "multipart", "file ?workspace_id ?provider"},
{"deleteFile", "DELETE", "/files/{file_id}", "deleteFile", "api", "json", "?workspace_id ?provider"},
{"getFileMetadata", "GET", "/files/{file_id}", "getFileMetadata", "api", "json", "?workspace_id ?provider"},
{"downloadFileContent", "GET", "/files/{file_id}/content", "downloadFileContent", "api", "raw", "?workspace_id ?provider"},
{"getGeneration", "GET", "/generation", "getGeneration", "api", "json", "?id"},
{"listGenerationContent", "GET", "/generation/content", "listGenerationContent", "api", "json", "?id"},
{"submitGenerationFeedback", "POST", "/generation/feedback", "submitGenerationFeedback", "mgmt", "json", "category comment generation_id"},
{"listGuardrails", "GET", "/guardrails", "listGuardrails", "mgmt", "json", "?offset ?limit ?workspace_id"},
{"createGuardrail", "POST", "/guardrails", "createGuardrail", "mgmt", "json", "allow_safety_retention_google allow_safety_retention_other allowed_data_regions allowed_models allowed_providers content_filter_builtins content_filters description enable_free_model_publication enable_free_model_training enable_paid_model_training enforce_zdr enforce_zdr_anthropic enforce_zdr_google enforce_zdr_openai enforce_zdr_other enforce_zdr_xai ignored_models ignored_providers include_byok_in_budgets limit_usd name reset_interval workspace_id"},
{"listKeyAssignments", "GET", "/guardrails/assignments/keys", "listKeyAssignments", "mgmt", "json", "?offset ?limit"},
{"listMemberAssignments", "GET", "/guardrails/assignments/members", "listMemberAssignments", "mgmt", "json", "?offset ?limit"},
{"deleteGuardrail", "DELETE", "/guardrails/{id}", "deleteGuardrail", "mgmt", "json", "-"},
{"getGuardrail", "GET", "/guardrails/{id}", "getGuardrail", "mgmt", "json", "-"},
{"updateGuardrail", "PATCH", "/guardrails/{id}", "updateGuardrail", "mgmt", "json", "allow_safety_retention_google allow_safety_retention_other allowed_data_regions allowed_models allowed_providers content_filter_builtins content_filters description enable_free_model_publication enable_free_model_training enable_paid_model_training enforce_zdr enforce_zdr_anthropic enforce_zdr_google enforce_zdr_openai enforce_zdr_other enforce_zdr_xai ignored_models ignored_providers include_byok_in_budgets limit_usd name reset_interval"},
{"listGuardrailKeyAssignments", "GET", "/guardrails/{id}/assignments/keys", "listGuardrailKeyAssignments", "mgmt", "json", "?offset ?limit"},
{"bulkAssignKeysToGuardrail", "POST", "/guardrails/{id}/assignments/keys", "bulkAssignKeysToGuardrail", "mgmt", "json", "key_hashes"},
{"bulkUnassignKeysFromGuardrail", "POST", "/guardrails/{id}/assignments/keys/remove", "bulkUnassignKeysFromGuardrail", "mgmt", "json", "key_hashes"},
{"listGuardrailMemberAssignments", "GET", "/guardrails/{id}/assignments/members", "listGuardrailMemberAssignments", "mgmt", "json", "?offset ?limit"},
{"bulkAssignMembersToGuardrail", "POST", "/guardrails/{id}/assignments/members", "bulkAssignMembersToGuardrail", "mgmt", "json", "member_user_ids"},
{"bulkUnassignMembersFromGuardrail", "POST", "/guardrails/{id}/assignments/members/remove", "bulkUnassignMembersFromGuardrail", "mgmt", "json", "member_user_ids"},
{"createImages", "POST", "/images", "createImages", "api", "json", "aspect_ratio background input_references model n output_compression output_format prompt provider quality resolution seed session_id size stream trace user"},
{"listImageModels", "GET", "/images/models", "listImageModels", "none", "json", "-"},
{"listImageModelEndpoints", "GET", "/images/models/{author}/{slug}/endpoints", "listImageModelEndpoints", "none", "json", "-"},
{"getCurrentKey", "GET", "/key", "getCurrentKey", "api", "json", "-"},
{"listKeys", "GET", "/keys", "list", "mgmt", "json", "?offset ?limit ?include_disabled ?include_expired ?workspace_id"},
{"createKeys", "POST", "/keys", "createKeys", "mgmt", "json", "creator_user_id expires_at external include_byok_in_limit limit limit_reset name workspace_id"},
{"deleteKeys", "DELETE", "/keys/{hash}", "deleteKeys", "mgmt", "json", "-"},
{"getKey", "GET", "/keys/{hash}", "getKey", "mgmt", "json", "-"},
{"updateKeys", "PATCH", "/keys/{hash}", "updateKeys", "mgmt", "json", "disabled include_byok_in_limit limit limit_reset name workspace_id"},
{"createMessages", "POST", "/messages", "createMessages", "api", "json", "cache_control context_management deferred_tools fallbacks max_tokens messages metadata model models output_config plugins provider route safeguards service_tier session_id speed stop_sequences stop_server_tools_when stream system temperature thinking tool_choice tools top_k top_p trace user"},
{"getModel", "GET", "/model/{author}/{slug}", "getModel", "none", "json", "-"},
{"getModels", "GET", "/models", "getModels", "none", "json", "?offset ?limit ?category ?supported_parameters ?output_modalities ?sort ?q ?input_modalities ?context ?min_price ?max_price ?arch ?model_authors ?providers ?distillable ?zdr ?region ?min_output_price ?max_output_price ?min_age_days ?max_age_days ?min_intelligence_index ?max_intelligence_index ?min_coding_index ?max_coding_index ?min_agentic_index ?max_agentic_index ?min_tool_success_rate ?max_tool_success_rate"},
{"listModelsCount", "GET", "/models/count", "listModelsCount", "none", "json", "?output_modalities"},
{"listModelsUser", "GET", "/models/user", "listModelsUser", "api", "json", "?offset ?limit ?output_modalities ?sort"},
{"listEndpoints", "GET", "/models/{author}/{slug}/endpoints", "listEndpoints", "none", "json", "-"},
{"listOauthJwks", "GET", "/oauth/jwks", "listOauthJwks", "none", "json", "-"},
{"createOauthToken", "POST", "/oauth/token", "createOauthToken", "none", "form", "federation_policy_id grant_type requested_token_type scope subject_token subject_token_type"},
{"listObservabilityDestinations", "GET", "/observability/destinations", "listObservabilityDestinations", "mgmt", "json", "?offset ?limit ?workspace_id"},
{"createObservabilityDestination", "POST", "/observability/destinations", "createObservabilityDestination", "mgmt", "json", "api_key_hashes broadcast_generation_cost broadcast_generation_cost_allocation broadcast_generation_identity broadcast_generation_request_context config enabled filter_rules name privacy_mode regions sampling_rate type workspace_id"},
{"deleteObservabilityDestination", "DELETE", "/observability/destinations/{id}", "deleteObservabilityDestination", "mgmt", "json", "-"},
{"getObservabilityDestination", "GET", "/observability/destinations/{id}", "getObservabilityDestination", "mgmt", "json", "-"},
{"updateObservabilityDestination", "PATCH", "/observability/destinations/{id}", "updateObservabilityDestination", "mgmt", "json", "api_key_hashes broadcast_generation_cost broadcast_generation_cost_allocation broadcast_generation_identity broadcast_generation_request_context config enabled filter_rules name privacy_mode regions sampling_rate"},
{"listOrganizationMembers", "GET", "/organization/members", "listOrganizationMembers", "mgmt", "json", "?offset ?limit"},
{"getOrganizationSettings", "GET", "/organization/settings", "getOrganizationSettings", "mgmt", "json", "-"},
{"updateOrganizationSettings", "PATCH", "/organization/settings", "updateOrganizationSettings", "mgmt", "json", "is_filtered_model_catalog_enabled"},
{"listPresets", "GET", "/presets", "listPresets", "api", "json", "?offset ?limit"},
{"getPreset", "GET", "/presets/{slug}", "getPreset", "api", "json", "-"},
{"createPresetsChatCompletions", "POST", "/presets/{slug}/chat/completions", "createPresetsChatCompletions", "api", "json", "cache_control debug deferred_tools frequency_penalty image_config logit_bias logprobs max_completion_tokens max_tokens messages metadata min_p modalities model models parallel_tool_calls plugins prediction presence_penalty prompt_cache_key prompt_cache_options provider reasoning reasoning_effort repetition_penalty response_format route seed service_tier session_id stop stop_server_tools_when stream stream_options temperature tool_choice tools top_a top_k top_logprobs top_p trace user"},
{"createPresetsMessages", "POST", "/presets/{slug}/messages", "createPresetsMessages", "api", "json", "cache_control context_management deferred_tools fallbacks max_tokens messages metadata model models output_config plugins provider route safeguards service_tier session_id speed stop_sequences stop_server_tools_when stream system temperature thinking tool_choice tools top_k top_p trace user"},
{"createPresetsResponses", "POST", "/presets/{slug}/responses", "createPresetsResponses", "api", "json", "background cache_control debug deferred_tools frequency_penalty image_config include input instructions max_output_tokens max_tool_calls metadata modalities model models parallel_tool_calls plugins presence_penalty previous_response_id prompt prompt_cache_key prompt_cache_options provider reasoning route safety_identifier service_tier session_id stop stop_server_tools_when store stream temperature text tool_choice tools top_k top_logprobs top_p trace truncation user"},
{"listPresetVersions", "GET", "/presets/{slug}/versions", "listPresetVersions", "api", "json", "?offset ?limit"},
{"getPresetVersion", "GET", "/presets/{slug}/versions/{version}", "getPresetVersion", "api", "json", "-"},
{"listPrivateEndpoints", "GET", "/private-endpoints", "listPrivateEndpoints", "mgmt", "json", "-"},
{"createPrivateEndpoint", "POST", "/private-endpoints", "createPrivateEndpoint", "mgmt", "json", "activate base_url declared_region declared_zdr model_permaslug pricing provider_slug upstream_model_id"},
{"deletePrivateEndpoint", "DELETE", "/private-endpoints/{id}", "deletePrivateEndpoint", "mgmt", "json", "?draft_only"},
{"getPrivateEndpoint", "GET", "/private-endpoints/{id}", "getPrivateEndpoint", "mgmt", "json", "-"},
{"updatePrivateEndpoint", "PATCH", "/private-endpoints/{id}", "updatePrivateEndpoint", "mgmt", "json", "base_url declared_region declared_zdr upstream_model_id"},
{"activatePrivateEndpoint", "POST", "/private-endpoints/{id}/activate", "activatePrivateEndpoint", "mgmt", "json", "-"},
{"disablePrivateEndpoint", "POST", "/private-endpoints/{id}/disable", "disablePrivateEndpoint", "mgmt", "json", "-"},
{"enablePrivateEndpoint", "POST", "/private-endpoints/{id}/enable", "enablePrivateEndpoint", "mgmt", "json", "-"},
{"updatePrivateEndpointPricing", "PUT", "/private-endpoints/{id}/pricing", "updatePrivateEndpointPricing", "mgmt", "json", "pricing"},
{"validatePrivateEndpoint", "POST", "/private-endpoints/{id}/validate", "validatePrivateEndpoint", "mgmt", "json", "workspace_id"},
{"listProviders", "GET", "/providers", "listProviders", "none", "json", "-"},
{"createRerank", "POST", "/rerank", "createRerank", "api", "json", "documents model provider query session_id top_n trace user"},
{"createResponses", "POST", "/responses", "createResponses", "api", "json", "background cache_control debug deferred_tools frequency_penalty image_config include input instructions max_output_tokens max_tool_calls metadata modalities model models parallel_tool_calls plugins presence_penalty previous_response_id prompt prompt_cache_key prompt_cache_options provider reasoning route safety_identifier service_tier session_id stop stop_server_tools_when store stream temperature text tool_choice tools top_k top_logprobs top_p trace truncation user"},
{"listScimGroupMappings", "GET", "/scim/group-mappings", "listScimGroupMappings", "mgmt", "json", "?offset ?limit"},
{"createScimGroupMapping", "POST", "/scim/group-mappings", "createScimGroupMapping", "mgmt", "json", "role scim_group_id workspace_id"},
{"deleteScimGroupMapping", "DELETE", "/scim/group-mappings/{id}", "deleteScimGroupMapping", "mgmt", "json", "?keep_members"},
{"getScimGroupMapping", "GET", "/scim/group-mappings/{id}", "getScimGroupMapping", "mgmt", "json", "-"},
{"updateScimGroupMapping", "PATCH", "/scim/group-mappings/{id}", "updateScimGroupMapping", "mgmt", "json", "role"},
{"listScimGroups", "GET", "/scim/groups", "listScimGroups", "mgmt", "json", "?offset ?limit ?display_name ?external_id"},
{"createScimSyncJob", "POST", "/scim/sync-jobs", "createScimSyncJob", "mgmt", "json", "-"},
{"getScimSyncJob", "GET", "/scim/sync-jobs/{id}", "getScimSyncJob", "mgmt", "json", "-"},
{"createSystemone", "POST", "/systemone", "createSystemone", "api", "json", "model provider questions session_id state trace user"},
{"listTools", "GET", "/tools", "listTools", "api", "json", "?offset ?limit ?api_format"},
{"getTool", "GET", "/tools/{name}", "getTool", "api", "json", "-"},
{"listInternConnections", "GET", "/vault/interns/{internId}/connections", "listInternConnections", "api", "json", "-"},
{"listInternEffectiveVaultSecrets", "GET", "/vault/interns/{internId}/effective-secrets", "listInternEffectiveVaultSecrets", "api", "json", "?limit ?offset"},
{"listInternVaultSecrets", "GET", "/vault/interns/{internId}/secrets", "listInternVaultSecrets", "api", "json", "?limit ?offset"},
{"copyVaultSecretsToIntern", "POST", "/vault/interns/{internId}/secrets/copy", "copyVaultSecretsToIntern", "api", "json", "names"},
{"deleteInternVaultSecret", "DELETE", "/vault/interns/{internId}/secrets/{name}", "deleteInternVaultSecret", "api", "json", "-"},
{"storeInternVaultSecret", "PUT", "/vault/interns/{internId}/secrets/{name}", "storeInternVaultSecret", "api", "json", "hosts value"},
{"listVaultSecrets", "GET", "/vault/secrets", "listVaultSecrets", "api", "json", "?limit ?offset"},
{"deleteVaultSecret", "DELETE", "/vault/secrets/{name}", "deleteVaultSecret", "api", "json", "-"},
{"storeVaultSecret", "PUT", "/vault/secrets/{name}", "storeVaultSecret", "api", "json", "hosts value"},
{"createVideos", "POST", "/videos", "createVideos", "api", "json", "aspect_ratio callback_url creativity duration frame_images generate_audio input_references model previous_job_id prompt provider resolution seed session_id size trace upscale_factor user"},
{"listVideosModels", "GET", "/videos/models", "listVideosModels", "none", "json", "-"},
{"getVideos", "GET", "/videos/{jobId}", "getVideos", "api", "json", "-"},
{"listVideosContent", "GET", "/videos/{jobId}/content", "listVideosContent", "api", "raw", "?index"},
{"listWorkspaces", "GET", "/workspaces", "listWorkspaces", "mgmt", "json", "?offset ?limit"},
{"createWorkspace", "POST", "/workspaces", "createWorkspace", "mgmt", "json", "default_image_model default_provider_sort default_text_model description disabled_server_tools io_logging_api_key_ids io_logging_sampling_rate is_data_discount_logging_enabled is_observability_broadcast_enabled is_observability_io_logging_enabled name slug"},
{"deleteWorkspace", "DELETE", "/workspaces/{id}", "deleteWorkspace", "mgmt", "json", "?confirm_default_workspace_deletion"},
{"getWorkspace", "GET", "/workspaces/{id}", "getWorkspace", "mgmt", "json", "-"},
{"updateWorkspace", "PATCH", "/workspaces/{id}", "updateWorkspace", "mgmt", "json", "default_image_model default_provider_sort default_text_model description disabled_server_tools io_logging_api_key_ids io_logging_sampling_rate is_data_discount_logging_enabled is_observability_broadcast_enabled is_observability_io_logging_enabled name slug"},
{"listWorkspaceMembers", "GET", "/workspaces/{id}/members", "listWorkspaceMembers", "mgmt", "json", "?offset ?limit"},
{"bulkAddWorkspaceMembers", "POST", "/workspaces/{id}/members/add", "bulkAddWorkspaceMembers", "mgmt", "json", "user_ids"},
{"bulkRemoveWorkspaceMembers", "POST", "/workspaces/{id}/members/remove", "bulkRemoveWorkspaceMembers", "mgmt", "json", "user_ids"},
{"listWorkspaceBudgets", "GET", "/workspaces/{workspace_ref}/budgets", "listWorkspaceBudgets", "mgmt", "json", "-"},
{"deleteWorkspaceBudget", "DELETE", "/workspaces/{workspace_ref}/budgets/{interval}", "deleteWorkspaceBudget", "mgmt", "json", "-"},
{"getWorkspaceBudget", "GET", "/workspaces/{workspace_ref}/budgets/{interval}", "getWorkspaceBudget", "mgmt", "json", "-"},
{"upsertWorkspaceBudget", "PUT", "/workspaces/{workspace_ref}/budgets/{interval}", "upsertWorkspaceBudget", "mgmt", "json", "include_byok_in_budgets limit_usd"},
};
// clang-format on
inline constexpr std::size_t kCallCount = sizeof(kCalls) / sizeof(kCalls[0]);
} // namespace molo::openrouter
+54
View File
@@ -0,0 +1,54 @@
#pragma once
// Static ceilings for the OpenRouter module. Never autotuned. Values marked
// "vendor" mirror a limit the OpenRouter spec states; the rest are local
// ceilings chosen by this module (a request over one is rejected before any
// network traffic, a response over one is reported as truncated).
#include <cstddef>
namespace molo::openrouter {
struct Caps {
// Request shaping.
static constexpr std::size_t kMaxFallbackModels = 16; // local: `models` array
static constexpr std::size_t kMaxPlugins = 16; // local
static constexpr std::size_t kMaxProviderList = 64; // local: order / only / ignore
static constexpr std::size_t kMaxQuantizations = 16; // local
static constexpr std::size_t kMaxStopSequences = 4; // vendor: "up to 4"
static constexpr std::size_t kMaxQueryParams = 64; // local
static constexpr std::size_t kMaxQueryValueBytes = 2048; // local
static constexpr std::size_t kMaxPathArgBytes = 256; // matches Service::valid_id
static constexpr std::size_t kMaxBulkIds = 1000; // local: bulk assign / remove lists
static constexpr std::size_t kMaxEmbeddingInputs = 2048; // local
static constexpr std::size_t kMaxRerankDocuments = 1000; // local
static constexpr std::size_t kMaxBatchRequests = 10000; // local
static constexpr std::size_t kMaxMetadataPairs = 16; // vendor: chat `metadata`
static constexpr std::size_t kMaxMetadataKeyBytes = 64; // vendor
static constexpr std::size_t kMaxMetadataValueBytes = 512; // vendor
static constexpr int kMaxTopLogprobs = 20; // vendor: 0..20
static constexpr std::size_t kMaxAppTitleBytes = 120; // local
static constexpr std::size_t kMaxAppUrlBytes = 512; // local
static constexpr std::size_t kMaxCategories = 2; // vendor: "up to 2 categories per request"
static constexpr std::size_t kMaxCategoryBytes = 30; // vendor
// Listing. /models allows limit up to 1000 (vendor); a full walk stops at
// kMaxModelPages pages and says so (never silent).
static constexpr int kModelPageSize = 500;
static constexpr int kMaxListLimit = 1000;
static constexpr int kMaxKeysLimit = 100; // vendor: /keys limit max 100
static constexpr std::size_t kMaxModelPages = 20;
// Streaming and parsing.
static constexpr std::size_t kMaxStreamEvents = 200000;
static constexpr std::size_t kMaxStreamTextBytes = 16u * 1024u * 1024u;
static constexpr std::size_t kMaxToolCalls = 128;
static constexpr std::size_t kMaxReasoningDetails = 4096;
static constexpr std::size_t kMaxChoices = 16;
// PKCE (RFC 7636): verifier 43..128 characters.
static constexpr std::size_t kMinPkceVerifier = 43;
static constexpr std::size_t kMaxPkceVerifier = 128;
};
} // namespace molo::openrouter
+196
View File
@@ -0,0 +1,196 @@
#include "openrouter/chat_model.h"
#include "llm/error.h"
#include "openrouter/server_tools.h"
#include "security/crypto/Base64.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
[[noreturn]] void bad(const std::string& why) {
throw llm::ApiError::invalid(kVendor, why);
}
std::string image_url(const llm::ImageData& image) {
if (!image.url.empty()) return image.url;
if (image.bytes.empty() || image.mime.empty()) bad("image needs a url or bytes plus mime");
return "data:" + image.mime + ";base64," + security::crypto::Base64::encode(image.bytes);
}
Json text_part(const std::string& text) {
Json p = Json::object();
p.set("type", Json("text"));
p.set("text", Json(text));
return p;
}
Json user_content(const llm::Message& m) {
bool has_image = false;
for (const llm::ContentPart& p : m.parts)
if (p.image) has_image = true;
if (!has_image) return Json(m.text());
Json arr = Json::array();
for (const llm::ContentPart& p : m.parts) {
if (!p.text.empty()) arr.push(text_part(p.text));
if (p.image) {
Json img = Json::object();
img.set("url", Json(image_url(*p.image)));
Json part = Json::object();
part.set("type", Json("image_url"));
part.set("image_url", img);
arr.push(part);
}
}
return arr;
}
void add_messages(const llm::ChatRequest& req, Json& out) {
if (!req.system.empty()) {
Json m = Json::object();
m.set("role", Json("system"));
m.set("content", Json(req.system));
out.push(m);
}
for (const llm::Message& msg : req.messages) {
switch (msg.role) {
case llm::Role::kSystem: {
Json m = Json::object();
m.set("role", Json("system"));
m.set("content", Json(msg.text()));
if (!msg.name.empty()) m.set("name", Json(msg.name));
out.push(m);
break;
}
case llm::Role::kUser: {
Json m = Json::object();
m.set("role", Json("user"));
m.set("content", user_content(msg));
if (!msg.name.empty()) m.set("name", Json(msg.name));
out.push(m);
break;
}
case llm::Role::kAssistant: {
Json m = Json::object();
m.set("role", Json("assistant"));
std::string text = msg.text();
Json calls = Json::array();
for (const llm::ContentPart& p : msg.parts) {
if (!p.tool_call) continue;
Json fn = Json::object();
fn.set("name", Json(p.tool_call->name));
fn.set("arguments", Json(p.tool_call->arguments.dump()));
Json call = Json::object();
call.set("id", Json(p.tool_call->id));
call.set("type", Json("function"));
call.set("function", fn);
calls.push(call);
}
if (!text.empty() || calls.as_array().empty()) m.set("content", Json(text));
else m.set("content", Json());
if (!calls.as_array().empty()) m.set("tool_calls", calls);
if (!msg.name.empty()) m.set("name", Json(msg.name));
out.push(m);
break;
}
case llm::Role::kTool: {
for (const llm::ContentPart& p : msg.parts) {
if (!p.tool_result) continue;
Json m = Json::object();
m.set("role", Json("tool"));
m.set("tool_call_id", Json(p.tool_result->call_id));
m.set("content", Json(p.tool_result->content));
out.push(m);
}
break;
}
}
}
}
} // namespace
OpenrouterChatModel::OpenrouterChatModel(std::shared_ptr<Gateway> gateway,
std::string default_model, ChatModelDefaults defaults)
: service_(std::move(gateway)),
default_model_(std::move(default_model)),
defaults_(std::move(defaults)) {}
Json OpenrouterChatModel::to_body(const llm::ChatRequest& req) const {
ChatParams p;
p.model = req.model.empty() ? default_model_ : req.model;
p.models = defaults_.fallback_models;
add_messages(req, p.messages);
for (const llm::ToolSpec& t : req.tools) p.tools.push(ServerTools::function(t));
if (!req.tool_choice.empty()) {
if (req.tool_choice == "auto" || req.tool_choice == "none" || req.tool_choice == "required") {
p.tool_choice = Json(req.tool_choice);
} else {
Json fn = Json::object();
fn.set("name", Json(req.tool_choice));
Json tc = Json::object();
tc.set("type", Json("function"));
tc.set("function", fn);
p.tool_choice = tc;
}
}
if (req.max_tokens) p.max_tokens = *req.max_tokens;
p.temperature = req.temperature;
p.top_p = req.top_p;
p.stop = req.stop;
if (req.response_schema.is_object()) {
p.response_format = ResponseFormats::json_schema("response", req.response_schema, true);
} else if (req.json_mode) {
p.response_format = ResponseFormats::json_object();
}
p.provider = defaults_.provider;
p.plugins = defaults_.plugins;
p.route = defaults_.route;
p.reasoning = defaults_.reasoning;
if (!req.reasoning_effort.empty()) p.reasoning = ReasoningConfig::with_effort(req.reasoning_effort);
Json body = p.to_json(false);
if (!req.extra.is_null()) {
if (!req.extra.is_object()) bad("ChatRequest::extra must be an object");
for (const auto& [k, v] : req.extra.as_object()) {
if (!CallPlanner::accepts_field("sendChatCompletionRequest", k))
bad("extra field not in the spec: " + k);
body.set(k, v);
}
}
return body;
}
llm::ChatResponse OpenrouterChatModel::to_response(const ChatResult& r) {
llm::ChatResponse out;
out.id = r.id;
out.model = r.model;
out.raw = r.raw;
out.message.role = llm::Role::kAssistant;
if (!r.choices.empty()) {
const ChatChoiceResult& c = r.first();
out.finish_reason = c.finish_reason;
if (!c.content.empty() || !c.reasoning.empty()) {
llm::ContentPart part = llm::ContentPart::of_text(c.content);
part.reasoning = c.reasoning;
out.message.parts.push_back(std::move(part));
}
for (const llm::ToolCall& call : c.tool_calls)
out.message.parts.push_back(llm::ContentPart::of_call(call));
}
if (r.usage) out.usage = *r.usage;
return out;
}
llm::ChatResponse OpenrouterChatModel::generate(const llm::ChatRequest& request) {
return to_response(ChatResult::parse(service_.create_raw(to_body(request))));
}
void OpenrouterChatModel::stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
service_.stream_body(to_body(request), on_chunk);
}
} // namespace molo::openrouter
+63
View File
@@ -0,0 +1,63 @@
#pragma once
// OpenrouterChatModel: llm::ChatModel over POST /chat/completions. Maps the
// neutral request (messages, tools, structured output, reasoning effort) to
// the OpenRouter wire format and the response back, and carries the routing
// defaults (provider preferences, fallback models, plugins, reasoning) that
// every request of this model object should use.
//
// Mapping notes:
// - `ChatRequest::extra` is merged last and may only hold fields the table
// declares for chat completions; it overrides typed values.
// - `ChatRequest::reasoning_effort` becomes the `reasoning` object
// {"effort": ...} (overriding the default reasoning); the model's
// reasoning text lands in ContentPart::reasoning.
// - `reasoning_details` (needed to continue a tool loop with reasoning
// models) are kept in ChatResponse::raw and cannot be replayed through the
// neutral types; use ChatService with raw messages for that.
// - A provider error mid-stream is thrown after the chunks before it.
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "llm/chat_types.h"
#include "openrouter/chat_service.h"
namespace molo::openrouter {
struct ChatModelDefaults {
std::optional<ProviderPreferences> provider;
std::vector<std::string> fallback_models; // sent as `models`
std::vector<PluginPtr> plugins;
std::optional<ReasoningConfig> reasoning;
std::string route; // "", "fallback", "sort"
};
class OpenrouterChatModel : public llm::ChatModel {
public:
OpenrouterChatModel(std::shared_ptr<Gateway> gateway, std::string default_model,
ChatModelDefaults defaults = {});
std::string vendor() const override { return "openrouter"; }
llm::ChatResponse generate(const llm::ChatRequest& request) override;
void stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
// The request body for a neutral request (validated; stream flag not set).
clients::body::Json to_body(const llm::ChatRequest& request) const;
// Neutral view of a parsed result.
static llm::ChatResponse to_response(const ChatResult& result);
const std::string& default_model() const { return default_model_; }
const ChatService& service() const { return service_; }
private:
ChatService service_;
std::string default_model_;
ChatModelDefaults defaults_;
};
} // namespace molo::openrouter
+218
View File
@@ -0,0 +1,218 @@
#include "openrouter/chat_params.h"
#include <algorithm>
#include <cmath>
#include "llm/caps.h"
#include "llm/error.h"
#include "openrouter/call_plan.h"
#include "openrouter/caps.h"
#include "openrouter/model_ref.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
constexpr const char* kChatCall = "sendChatCompletionRequest";
[[noreturn]] void bad(const std::string& why) {
throw llm::ApiError::invalid(kVendor, "chat: " + why);
}
bool one_of(const std::string& v, std::initializer_list<const char*> all) {
return std::any_of(all.begin(), all.end(), [&](const char* s) { return v == s; });
}
void range(const char* what, const std::optional<double>& v, double lo, double hi) {
if (!v) return;
if (!std::isfinite(*v) || *v < lo || *v > hi)
bad(std::string(what) + " must be between " + std::to_string(lo) + " and " + std::to_string(hi));
}
Json strings(const std::vector<std::string>& v) {
Json a = Json::array();
for (const std::string& s : v) a.push(Json(s));
return a;
}
void check_metadata(const Json& m) {
if (m.is_null()) return;
if (!m.is_object()) bad("metadata must be an object");
if (m.as_object().size() > Caps::kMaxMetadataPairs) bad("metadata: at most 16 pairs");
for (const auto& [k, v] : m.as_object()) {
if (k.empty() || k.size() > Caps::kMaxMetadataKeyBytes) bad("metadata key must be 1..64 bytes");
if (!v.is_string() || v.as_string().size() > Caps::kMaxMetadataValueBytes)
bad("metadata values must be strings of at most 512 bytes");
}
}
} // namespace
Json ResponseFormats::text() {
Json o = Json::object();
o.set("type", Json("text"));
return o;
}
Json ResponseFormats::json_object() {
Json o = Json::object();
o.set("type", Json("json_object"));
return o;
}
Json ResponseFormats::json_schema(const std::string& name, const Json& schema, bool strict,
const std::string& description) {
if (name.empty() || name.size() > 64) bad("json_schema name must be 1..64 characters");
for (char c : name)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '_' || c == '-'))
bad("json_schema name may only use letters, digits, _ and -");
if (!schema.is_object()) bad("json_schema schema must be a JSON Schema object");
Json js = Json::object();
js.set("name", Json(name));
js.set("schema", schema);
js.set("strict", Json(strict));
if (!description.empty()) js.set("description", Json(description));
Json o = Json::object();
o.set("type", Json("json_schema"));
o.set("json_schema", js);
return o;
}
Json ResponseFormats::grammar(const std::string& grammar) {
if (grammar.empty()) bad("grammar must not be empty");
Json o = Json::object();
o.set("type", Json("grammar"));
o.set("grammar", Json(grammar));
return o;
}
void ChatParams::validate() const {
if (!model.empty()) ModelRef::parse(model);
if (models.size() > Caps::kMaxFallbackModels) bad("too many fallback models");
for (const std::string& m : models) ModelRef::parse(m);
if (!messages.is_array() || messages.as_array().empty()) bad("messages must be a non-empty array");
if (messages.as_array().size() > llm::Caps::kMaxMessages) bad("too many messages");
for (const Json& m : messages.as_array())
if (!m.is_object() || !m.has("role")) bad("every message needs a role");
range("temperature", temperature, 0, 2);
range("top_p", top_p, 0, 1);
range("frequency_penalty", frequency_penalty, -2, 2);
range("presence_penalty", presence_penalty, -2, 2);
range("min_p", min_p, 0, 1);
range("top_a", top_a, 0, 1);
if (repetition_penalty && (!std::isfinite(*repetition_penalty) || *repetition_penalty < 0))
bad("repetition_penalty must be >= 0");
if (top_k && *top_k < 0) bad("top_k must be >= 0");
if (max_tokens && *max_tokens < 1) bad("max_tokens must be >= 1");
if (max_completion_tokens && *max_completion_tokens < 1) bad("max_completion_tokens must be >= 1");
if (top_logprobs && (*top_logprobs < 0 || *top_logprobs > Caps::kMaxTopLogprobs))
bad("top_logprobs must be 0..20");
if (stop.size() > Caps::kMaxStopSequences) bad("at most 4 stop sequences");
if (!logit_bias.is_null() && !logit_bias.is_object()) bad("logit_bias must be an object");
if (!response_format.is_null()) {
const Json* t = response_format.is_object() ? response_format.find("type") : nullptr;
if (!t || !t->is_string() ||
!one_of(t->as_string(), {"text", "json_object", "json_schema", "grammar", "python"}))
bad("response_format.type must be text json_object json_schema grammar or python");
}
if (!tools.is_array() || tools.as_array().size() > llm::Caps::kMaxTools) bad("tools must be an array of at most 256");
for (const Json& t : tools.as_array())
if (!t.is_object() || !t.has("type")) bad("every tool needs a type");
if (!tool_choice.is_null() && !tool_choice.is_string() && !tool_choice.is_object())
bad("tool_choice must be a string or an object");
if (tool_choice.is_string() && !one_of(tool_choice.as_string(), {"none", "auto", "required"}))
bad("tool_choice string must be none, auto or required");
if (provider) provider->validate();
if (reasoning) reasoning->validate();
if (reasoning && !reasoning_effort.empty()) bad("send reasoning or reasoning_effort, not both");
if (!reasoning_effort.empty() && !ReasoningConfig::valid_effort(reasoning_effort))
bad("reasoning_effort must be one of max xhigh high medium low minimal none");
plugins_to_json(plugins);
if (!route.empty() && route != "fallback" && route != "sort") bad("route must be fallback or sort");
if (modalities.size() > 3) bad("modalities: at most text, image, audio");
for (const std::string& m : modalities)
if (!one_of(m, {"text", "image", "audio"})) bad("modalities must be text, image or audio");
if (!service_tier.empty() &&
!one_of(service_tier, {"auto", "default", "fast", "flex", "priority", "scale", "ultrafast"}))
bad("unknown service_tier");
check_metadata(metadata);
if (!trace.is_null() && !trace.is_object()) bad("trace must be an object");
if (!debug.is_null() && !debug.is_object()) bad("debug must be an object");
if (!extra.is_object()) bad("extra must be an object");
}
Json ChatParams::to_json(bool stream) const {
validate();
Json o = Json::object();
o.set("messages", messages);
if (!model.empty()) o.set("model", Json(model));
if (!models.empty()) o.set("models", strings(models));
auto num = [&](const char* k, const std::optional<double>& v) { if (v) o.set(k, Json(*v)); };
auto inum = [&](const char* k, const std::optional<int>& v) { if (v) o.set(k, Json(*v)); };
num("temperature", temperature);
num("top_p", top_p);
num("frequency_penalty", frequency_penalty);
num("presence_penalty", presence_penalty);
num("repetition_penalty", repetition_penalty);
num("min_p", min_p);
num("top_a", top_a);
inum("top_k", top_k);
inum("max_tokens", max_tokens);
inum("max_completion_tokens", max_completion_tokens);
inum("seed", seed);
inum("top_logprobs", top_logprobs);
if (logprobs) o.set("logprobs", Json(*logprobs));
if (!stop.empty()) o.set("stop", strings(stop));
if (logit_bias.is_object()) o.set("logit_bias", logit_bias);
if (response_format.is_object()) o.set("response_format", response_format);
if (!tools.as_array().empty()) o.set("tools", tools);
if (!tool_choice.is_null()) o.set("tool_choice", tool_choice);
if (parallel_tool_calls) o.set("parallel_tool_calls", Json(*parallel_tool_calls));
if (provider && !provider->empty()) o.set("provider", provider->to_json());
if (reasoning && !reasoning->empty()) o.set("reasoning", reasoning->to_json());
if (!reasoning_effort.empty()) o.set("reasoning_effort", Json(reasoning_effort));
if (!plugins.empty()) o.set("plugins", plugins_to_json(plugins));
if (!route.empty()) o.set("route", Json(route));
if (!modalities.empty()) o.set("modalities", strings(modalities));
if (!service_tier.empty()) o.set("service_tier", Json(service_tier));
if (!user.empty()) o.set("user", Json(user));
if (!session_id.empty()) o.set("session_id", Json(session_id));
if (!prompt_cache_key.empty()) o.set("prompt_cache_key", Json(prompt_cache_key));
if (metadata.is_object()) o.set("metadata", metadata);
if (trace.is_object()) o.set("trace", trace);
if (prediction.is_object()) o.set("prediction", prediction);
if (debug.is_object()) o.set("debug", debug);
for (const auto& [k, v] : extra.as_object()) {
if (!CallPlanner::accepts_field(kChatCall, k)) bad("extra field not in the spec: " + k);
if (o.has(k)) bad("extra field already set by a typed member: " + k);
o.set(k, v);
}
if (stream) o.set("stream", Json(true));
return o;
}
ChatParams& ChatParams::add_message(const std::string& role, const std::string& text) {
if (!one_of(role, {"system", "developer", "user", "assistant"}))
bad("role must be system, developer, user or assistant (tool messages carry a tool_call_id: add them to messages directly)");
Json m = Json::object();
m.set("role", Json(role));
m.set("content", Json(text));
messages.push(m);
return *this;
}
ChatParams ChatParams::simple(const std::string& model, const std::string& user_text) {
ChatParams p;
p.model = model;
p.add_message("user", user_text);
return p;
}
} // namespace molo::openrouter
+82
View File
@@ -0,0 +1,82 @@
#pragma once
// ChatParams: a typed, validated builder for the body of POST
// /chat/completions (spec schema ChatRequest). Every field the table
// (calls.h) declares for sendChatCompletionRequest is either a typed member
// here or can go through `extra`; `extra` keys are checked against the table,
// so nothing the spec does not know can be sent.
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "openrouter/plugins.h"
#include "openrouter/provider_preferences.h"
#include "openrouter/reasoning.h"
namespace molo::openrouter {
// Values for the `response_format` field.
struct ResponseFormats {
static clients::body::Json text();
static clients::body::Json json_object();
// name: a-z A-Z 0-9 _ -, at most 64 characters. schema: a JSON Schema object.
static clients::body::Json json_schema(const std::string& name,
const clients::body::Json& schema, bool strict = true,
const std::string& description = "");
static clients::body::Json grammar(const std::string& grammar);
};
struct ChatParams {
std::string model; // "" : the account default
std::vector<std::string> models; // fallback models, tried in order
clients::body::Json messages = clients::body::Json::array(); // OpenAI-format messages
std::optional<double> temperature; // 0..2
std::optional<double> top_p; // 0..1
std::optional<double> frequency_penalty; // -2..2
std::optional<double> presence_penalty; // -2..2
std::optional<double> repetition_penalty;
std::optional<double> min_p; // 0..1
std::optional<double> top_a; // 0..1
std::optional<int> top_k;
std::optional<int> max_tokens;
std::optional<int> max_completion_tokens;
std::optional<int> seed;
std::optional<bool> logprobs;
std::optional<int> top_logprobs; // 0..20
std::vector<std::string> stop; // at most 4
clients::body::Json logit_bias; // null or object token -> bias
clients::body::Json response_format; // null or ResponseFormats::*
clients::body::Json tools = clients::body::Json::array(); // ServerTools::*
clients::body::Json tool_choice; // null, "none" "auto" "required", or an object
std::optional<bool> parallel_tool_calls;
std::optional<ProviderPreferences> provider;
std::optional<ReasoningConfig> reasoning;
std::string reasoning_effort; // shorthand, not together with `reasoning`
std::vector<PluginPtr> plugins;
std::string route; // "", "fallback", "sort" (deprecated spec field)
std::vector<std::string> modalities; // text image audio
std::string service_tier; // auto default fast flex priority scale ultrafast
std::string user;
std::string session_id;
std::string prompt_cache_key;
clients::body::Json metadata; // null or object of string pairs (16 pairs, 64/512 bytes)
clients::body::Json trace; // null or {trace_id, trace_name, span_name, generation_name, parent_span_id}
clients::body::Json prediction;
clients::body::Json debug; // {"echo_upstream_body": true} (streaming only)
clients::body::Json extra = clients::body::Json::object(); // other declared fields
// Validates and builds the request body. `stream` sets "stream": true.
clients::body::Json to_json(bool stream = false) const;
void validate() const;
ChatParams& add_message(const std::string& role, const std::string& text);
static ChatParams simple(const std::string& model, const std::string& user_text);
};
} // namespace molo::openrouter
+127
View File
@@ -0,0 +1,127 @@
#include "openrouter/chat_result.h"
#include "llm/error.h"
#include "openrouter/call_plan.h"
#include "openrouter/caps.h"
#include "openrouter/errors.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
std::string str(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
int integer(const Json* o, const char* k) {
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
return v && v->is_number() ? static_cast<int>(v->as_int()) : 0;
}
[[noreturn]] void bad_response(const std::string& why) {
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", why);
}
} // namespace
llm::Usage parse_usage(const Json& u) {
llm::Usage out;
if (!u.is_object()) return out;
out.input_tokens = integer(&u, "prompt_tokens");
out.output_tokens = integer(&u, "completion_tokens");
out.total_tokens = integer(&u, "total_tokens");
if (out.total_tokens == 0) out.total_tokens = out.input_tokens + out.output_tokens;
out.cached_tokens = integer(u.find("prompt_tokens_details"), "cached_tokens");
out.reasoning_tokens = integer(u.find("completion_tokens_details"), "reasoning_tokens");
return out;
}
Json parse_tool_arguments(const std::string& text) {
if (text.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
try {
Json j = Json::parse(text);
if (j.is_object()) return j;
} catch (const std::exception&) {
}
Json bad = Json::object();
bad.set("_invalid_arguments", Json(text));
return bad;
}
ChatResult ChatResult::parse(const Json& doc) {
if (!doc.is_object()) bad_response("chat response is not a JSON object");
throw_if_error_body(doc);
ChatResult r;
r.raw = doc;
r.id = str(doc, "id");
r.model = str(doc, "model");
r.provider = str(doc, "provider");
r.service_tier = str(doc, "service_tier");
if (const Json* c = doc.find("created"); c && c->is_number()) r.created = c->as_int();
if (const Json* m = doc.find("openrouter_metadata"); m && m->is_object()) r.openrouter_metadata = *m;
if (const Json* u = doc.find("usage"); u && u->is_object()) {
r.usage = parse_usage(*u);
if (const Json* c = u->find("cost"); c && c->is_number()) r.cost = c->as_number();
if (const Json* b = u->find("is_byok"); b && b->is_bool()) r.is_byok = b->as_bool();
if (const Json* d = u->find("cost_details"); d && d->is_object()) r.cost_details = *d;
}
const Json* choices = doc.find("choices");
if (!choices || !choices->is_array()) bad_response("chat response has no choices array");
if (choices->as_array().size() > Caps::kMaxChoices) bad_response("too many choices");
for (const Json& c : choices->as_array()) {
if (!c.is_object()) bad_response("choice is not an object");
ChatChoiceResult cr;
cr.index = integer(&c, "index");
cr.finish_reason = str(c, "finish_reason");
cr.native_finish_reason = str(c, "native_finish_reason");
if (const Json* e = c.find("error"); e && e->is_object()) throw ErrorBody::parse(*e).to_api_error();
if (cr.finish_reason == "error")
throw llm::ApiError(kVendor, 502, "provider_unavailable", "generation ended with finish_reason error");
if (const Json* m = c.find("message"); m && m->is_object()) {
cr.message = *m;
if (const Json* content = m->find("content")) {
if (content->is_string()) {
cr.content = content->as_string();
} else if (content->is_array()) {
for (const Json& part : content->as_array())
if (part.is_object() && str(part, "type") == "text") cr.content += str(part, "text");
}
}
cr.reasoning = str(*m, "reasoning");
cr.refusal = str(*m, "refusal");
if (const Json* rd = m->find("reasoning_details"); rd && rd->is_array()) {
if (rd->as_array().size() > Caps::kMaxReasoningDetails) bad_response("too many reasoning details");
cr.reasoning_details = *rd;
}
if (const Json* tcs = m->find("tool_calls"); tcs && tcs->is_array()) {
if (tcs->as_array().size() > Caps::kMaxToolCalls) bad_response("too many tool calls");
for (const Json& tc : tcs->as_array()) {
const Json* fn = tc.is_object() ? tc.find("function") : nullptr;
if (!fn || !fn->is_object()) bad_response("tool call without function");
llm::ToolCall call;
call.id = str(tc, "id");
call.name = str(*fn, "name");
call.arguments = parse_tool_arguments(str(*fn, "arguments"));
cr.tool_calls.push_back(std::move(call));
}
}
}
r.choices.push_back(std::move(cr));
}
return r;
}
const ChatChoiceResult& ChatResult::first() const {
if (choices.empty()) bad_response("chat response has no choice");
return choices.front();
}
std::string ChatResult::text() const { return choices.empty() ? "" : choices.front().content; }
std::string ChatResult::finish_reason() const {
return choices.empty() ? "" : choices.front().finish_reason;
}
} // namespace molo::openrouter
+57
View File
@@ -0,0 +1,57 @@
#pragma once
// ChatResult: typed view of a chat completion (spec schema ChatResult) and of
// the document assembled from a stream. The raw document is kept for fields
// the typed view omits (annotations, images, audio, logprobs, ...).
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
namespace molo::openrouter {
struct ChatChoiceResult {
int index = 0;
std::string finish_reason; // tool_calls stop length content_filter error
std::string native_finish_reason; // the provider's own value, when sent
std::string content;
std::string reasoning; // message.reasoning
std::string refusal;
clients::body::Json reasoning_details = clients::body::Json::array(); // replay with tool calls
std::vector<llm::ToolCall> tool_calls;
clients::body::Json message; // the raw message object
};
struct ChatResult {
std::string id; // "gen-..." : the generation id for GET /generation
std::string model;
std::string provider; // top level `provider` when sent
std::string service_tier;
long created = 0;
std::vector<ChatChoiceResult> choices;
std::optional<llm::Usage> usage;
std::optional<double> cost; // usage.cost in USD (credits)
bool is_byok = false;
clients::body::Json cost_details;
clients::body::Json openrouter_metadata; // present when router metadata is opted in
clients::body::Json raw;
// Parses a non-streaming response body. A top-level `error`, or a choice
// with finish_reason "error", throws llm::ApiError (the 200-with-error cases).
static ChatResult parse(const clients::body::Json& doc);
const ChatChoiceResult& first() const; // throws when there is no choice
std::string text() const;
std::string finish_reason() const;
};
// usage object -> neutral Usage (prompt, completion, total, cached, reasoning).
llm::Usage parse_usage(const clients::body::Json& usage);
// Parses function-call arguments (a JSON string). Invalid JSON is kept as
// {"_invalid_arguments": "<text>"} so the caller can see and report it.
clients::body::Json parse_tool_arguments(const std::string& text);
} // namespace molo::openrouter
+67
View File
@@ -0,0 +1,67 @@
#include "openrouter/chat_service.h"
#include "openrouter/errors.h"
namespace molo::openrouter {
namespace {
constexpr const char* kChat = "sendChatCompletionRequest";
constexpr const char* kLegacy = "createCompletions";
Json with_stream(const Json& body) {
Json b = body;
b.set("stream", Json(true));
return b;
}
} // namespace
Json ChatService::create_raw(const Json& body) const {
Json b = body;
if (const Json* s = b.find("stream"); s && s->is_bool() && s->as_bool())
throw llm::ApiError::invalid(kVendor, "use stream() for stream:true");
Json doc = run(kChat, {}, {}, b);
throw_if_error_body(doc);
return doc;
}
ChatResult ChatService::create(const ChatParams& params) const {
return ChatResult::parse(create_raw(params.to_json(false)));
}
void ChatService::stream_events(const Json& body, const llm::SseDecoder::Sink& sink) const {
Json b = with_stream(body);
gateway_->stream(CallPlanner::plan(kChat, {}, {}, &b), sink);
}
ChatResult ChatService::stream(const ChatParams& params,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const {
return stream_body(params.to_json(true), on_chunk);
}
ChatResult ChatService::stream_body(const Json& raw_body,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const {
Json body = with_stream(raw_body);
PlannedCall call = CallPlanner::plan(kChat, {}, {}, &body);
ChatStreamParser parser;
gateway_->stream(call, [&](const llm::SseEvent& ev) {
for (const llm::ChatChunk& chunk : parser.feed(ev)) {
if (on_chunk && !on_chunk(chunk)) return false;
}
return !parser.errored();
});
parser.throw_if_errored();
return parser.result();
}
Json ChatService::create_completion(const Json& body) const {
Json doc = run(kLegacy, {}, {}, body);
throw_if_error_body(doc);
return doc;
}
void ChatService::stream_completion(const Json& body, const llm::SseDecoder::Sink& sink) const {
Json b = with_stream(body);
gateway_->stream(CallPlanner::plan(kLegacy, {}, {}, &b), sink);
}
} // namespace molo::openrouter
+45
View File
@@ -0,0 +1,45 @@
#pragma once
// ChatService: POST /chat/completions (and the legacy POST /completions),
// non-streaming and streaming, over the OpenRouter-native types.
#include <functional>
#include "openrouter/chat_params.h"
#include "openrouter/chat_result.h"
#include "openrouter/chat_stream.h"
#include "openrouter/gateway.h"
namespace molo::openrouter {
class ChatService : public Endpoint {
public:
explicit ChatService(std::shared_ptr<Gateway> gateway) : Endpoint(std::move(gateway)) {}
// Non-streaming. Throws llm::ApiError, including for the 200-with-error shapes.
ChatResult create(const ChatParams& params) const;
// The raw document (still checked for a top-level error).
Json create_raw(const Json& body) const;
// Streaming. `on_chunk` returns false to stop reading (the connection is
// abandoned, no error). Returns the assembled result. A provider error that
// arrives mid-stream is thrown as llm::ApiError AFTER the chunks before it
// were delivered; the assembled partial result is lost to the caller, so a
// caller that wants the partial text keeps what it received in the callback.
ChatResult stream(const ChatParams& params,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const;
// Same, from an already built body (stream:true is set here). The body is
// checked against the call table.
ChatResult stream_body(const Json& body,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const;
// Raw events (decoded data chunks), for callers that parse themselves.
void stream_events(const Json& body, const llm::SseDecoder::Sink& sink) const;
// Legacy text completions (POST /completions). `body` keys are checked
// against the table row createCompletions (documented, not in the OpenAPI
// document; see SPEC "Unverified").
Json create_completion(const Json& body) const;
void stream_completion(const Json& body, const llm::SseDecoder::Sink& sink) const;
};
} // namespace molo::openrouter
+166
View File
@@ -0,0 +1,166 @@
#include "openrouter/chat_stream.h"
#include "llm/error.h"
#include "openrouter/call_plan.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
std::string str(const Json* o, const char* k) {
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
[[noreturn]] void over_cap(const std::string& why) {
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", why);
}
} // namespace
std::vector<llm::ChatChunk> ChatStreamParser::feed(const llm::SseEvent& event) {
std::vector<llm::ChatChunk> out;
if (++events_ > Caps::kMaxStreamEvents) over_cap("stream over the event cap");
const Json& doc = event.data;
if (!doc.is_object()) return out;
if (id_.empty()) id_ = str(&doc, "id");
if (model_.empty()) model_ = str(&doc, "model");
if (provider_.empty()) provider_ = str(&doc, "provider");
if (service_tier_.empty()) service_tier_ = str(&doc, "service_tier");
if (const Json* c = doc.find("created"); c && c->is_number() && created_ == 0) created_ = c->as_int();
if (const Json* m = doc.find("openrouter_metadata"); m && m->is_object()) openrouter_metadata_ = *m;
if (const Json* u = doc.find("usage"); u && u->is_object()) {
usage_ = parse_usage(*u);
if (const Json* c = u->find("cost"); c && c->is_number()) cost_ = c->as_number();
if (const Json* b = u->find("is_byok"); b && b->is_bool()) is_byok_ = b->as_bool();
if (const Json* d = u->find("cost_details"); d && d->is_object()) cost_details_ = *d;
llm::ChatChunk uc;
uc.usage = usage_;
out.push_back(std::move(uc));
}
if (const Json* e = doc.find("error"); e && e->is_object() && !error_) {
error_ = ErrorBody::parse(*e);
llm::ChatChunk ec;
ec.finish_reason = "error";
out.push_back(std::move(ec));
}
const Json* choices = doc.find("choices");
if (!choices || !choices->is_array()) return out;
if (choices->as_array().size() > Caps::kMaxChoices) over_cap("too many choices in a chunk");
for (const Json& choice : choices->as_array()) {
if (!choice.is_object()) continue;
const Json* idx = choice.find("index");
if (idx && idx->is_number() && idx->as_int() != 0) continue; // OpenRouter streams one choice
if (const Json* delta = choice.find("delta"); delta && delta->is_object()) {
llm::ChatChunk ch;
std::string content = str(delta, "content");
if (!content.empty()) {
if (text_.size() + content.size() > Caps::kMaxStreamTextBytes) over_cap("stream text over the byte cap");
text_ += content;
ch.text_delta = content;
}
std::string reasoning = str(delta, "reasoning");
if (!reasoning.empty()) {
if (reasoning_.size() + reasoning.size() > Caps::kMaxStreamTextBytes)
over_cap("stream reasoning over the byte cap");
reasoning_ += reasoning;
ch.reasoning_delta = reasoning;
}
refusal_ += str(delta, "refusal");
if (const Json* rd = delta->find("reasoning_details"); rd && rd->is_array())
for (const Json& d : rd->as_array()) {
if (reasoning_details_.as_array().size() >= Caps::kMaxReasoningDetails)
over_cap("too many reasoning details");
reasoning_details_.push(d);
}
if (const Json* tcs = delta->find("tool_calls"); tcs && tcs->is_array()) {
for (const Json& tc : tcs->as_array()) {
if (!tc.is_object()) continue;
const Json* ti = tc.find("index");
int index = ti && ti->is_number() ? static_cast<int>(ti->as_int()) : 0;
if (index < 0 || static_cast<std::size_t>(index) >= Caps::kMaxToolCalls) over_cap("tool call index over the cap");
ToolBuild& tb = tools_[index];
std::string id = str(&tc, "id");
const Json* fn = tc.find("function");
std::string name = str(fn, "name");
std::string args = str(fn, "arguments");
if (!id.empty()) tb.id = id;
if (!name.empty()) tb.name = name;
if (tb.arguments.size() + args.size() > Caps::kMaxStreamTextBytes) over_cap("tool arguments over the byte cap");
tb.arguments += args;
llm::ChatChunk tcc;
tcc.tool_call_id = tb.id;
tcc.tool_call_name = name; // only the fragment that carries the name
tcc.tool_args_delta = args;
out.push_back(std::move(tcc));
}
}
if (!ch.text_delta.empty() || !ch.reasoning_delta.empty()) out.push_back(std::move(ch));
}
std::string fr = str(&choice, "finish_reason");
if (!fr.empty()) {
finish_reason_ = fr;
native_finish_reason_ = str(&choice, "native_finish_reason");
if (fr != "error" || !error_) {
llm::ChatChunk fc;
fc.finish_reason = fr;
out.push_back(std::move(fc));
}
}
}
return out;
}
void ChatStreamParser::throw_if_errored() const {
if (error_) throw error_->to_api_error();
}
ChatResult ChatStreamParser::result() const {
ChatResult r;
r.id = id_;
r.model = model_;
r.provider = provider_;
r.service_tier = service_tier_;
r.created = created_;
r.usage = usage_;
r.cost = cost_;
r.is_byok = is_byok_;
r.cost_details = cost_details_;
r.openrouter_metadata = openrouter_metadata_;
ChatChoiceResult c;
c.index = 0;
c.finish_reason = finish_reason_;
c.native_finish_reason = native_finish_reason_;
c.content = text_;
c.reasoning = reasoning_;
c.refusal = refusal_;
c.reasoning_details = reasoning_details_;
Json message = Json::object();
message.set("role", Json("assistant"));
message.set("content", Json(text_));
for (const auto& [index, tb] : tools_) {
(void)index;
llm::ToolCall call;
call.id = tb.id;
call.name = tb.name;
call.arguments = parse_tool_arguments(tb.arguments);
c.tool_calls.push_back(std::move(call));
}
c.message = message;
r.choices.push_back(std::move(c));
Json raw = Json::object();
raw.set("id", Json(id_));
raw.set("model", Json(model_));
raw.set("assembled_from_stream", Json(true));
r.raw = raw;
return r;
}
} // namespace molo::openrouter
+70
View File
@@ -0,0 +1,70 @@
#pragma once
// ChatStreamParser: turns the SSE events of a streamed chat completion
// (spec schema ChatStreamChunk) into neutral ChatChunks and assembles the
// final ChatResult. Pure logic over decoded events, so it is testable with a
// documented transcript and no network.
//
// Wire facts handled (https://openrouter.ai/docs/api-reference/streaming and
// /errors):
// - SSE comment lines such as ": OPENROUTER PROCESSING" are heartbeats and
// are dropped by llm::SseDecoder before they reach this parser.
// - "data: [DONE]" ends the stream (consumed by the decoder).
// - the final data chunk carries `usage` (often with an empty `choices`).
// - a mid-stream error is a normal chunk with a top-level `error` object
// and a choice with finish_reason "error"; the HTTP status stays 200.
// - tool calls arrive as fragments keyed by `index`; the first fragment has
// id and name, later ones only argument text.
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
#include "llm/sse.h"
#include "openrouter/chat_result.h"
#include "openrouter/errors.h"
namespace molo::openrouter {
class ChatStreamParser {
public:
// Returns the neutral chunks this event produced (possibly none). Never
// throws for a provider error: it is recorded (see error()) and a chunk
// with finish_reason "error" is returned. Throws llm::ApiError when a
// static cap (events, text bytes, tool calls) is exceeded.
std::vector<llm::ChatChunk> feed(const llm::SseEvent& event);
bool errored() const { return error_.has_value(); }
const std::optional<ErrorBody>& error() const { return error_; }
// Throws the recorded error as llm::ApiError, if any.
void throw_if_errored() const;
std::size_t event_count() const { return events_; }
// The assembled result (valid after any number of events).
ChatResult result() const;
private:
struct ToolBuild {
std::string id;
std::string name;
std::string arguments;
};
std::string id_, model_, provider_, service_tier_;
long created_ = 0;
std::string text_, reasoning_, refusal_, finish_reason_, native_finish_reason_;
clients::body::Json reasoning_details_ = clients::body::Json::array();
std::map<int, ToolBuild> tools_;
std::optional<llm::Usage> usage_;
std::optional<double> cost_;
bool is_byok_ = false;
clients::body::Json cost_details_;
clients::body::Json openrouter_metadata_;
std::optional<ErrorBody> error_;
std::size_t events_ = 0;
};
} // namespace molo::openrouter
+163
View File
@@ -0,0 +1,163 @@
#include "openrouter/errors.h"
#include <cstdlib>
#include "openrouter/call_plan.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
struct TypedError {
const char* type;
int status;
};
// metadata.error_type values and their HTTP statuses, from the "Typed error
// codes" tables of the errors guide.
constexpr TypedError kTyped[] = {
{"context_length_exceeded", 400}, {"max_tokens_exceeded", 400},
{"token_limit_exceeded", 400}, {"string_too_long", 400},
{"authentication", 401}, {"permission_denied", 403},
{"payment_required", 402}, {"rate_limit_exceeded", 429},
{"provider_overloaded", 503}, {"provider_unavailable", 502},
{"invalid_request", 400}, {"invalid_prompt", 400},
{"not_found", 404}, {"precondition_failed", 412},
{"payload_too_large", 413}, {"unprocessable", 422},
{"content_policy_violation", 403}, {"refusal", 403},
{"invalid_image", 400}, {"image_too_large", 400},
{"image_too_small", 400}, {"unsupported_image_format", 400},
{"image_not_found", 404}, {"image_download_failed", 400},
{"server", 500}, {"timeout", 504},
{"unmapped", 500},
};
std::string string_of(const Json* v) {
return v && v->is_string() ? v->as_string() : "";
}
} // namespace
const char* failure_kind_name(FailureKind kind) {
switch (kind) {
case FailureKind::kInvalidRequest: return "invalid_request";
case FailureKind::kUnauthorized: return "unauthorized";
case FailureKind::kCredits: return "insufficient_credits";
case FailureKind::kForbidden: return "forbidden";
case FailureKind::kNotFound: return "not_found";
case FailureKind::kTimeout: return "timeout";
case FailureKind::kRateLimited: return "rate_limited";
case FailureKind::kProviderFailed: return "provider_failed";
case FailureKind::kNoProvider: return "no_provider";
case FailureKind::kOverloaded: return "overloaded";
case FailureKind::kServer: return "server";
case FailureKind::kOther: return "other";
}
return "other";
}
FailureKind classify_status(int s) {
switch (s) {
case 400: case 413: case 422: return FailureKind::kInvalidRequest;
case 401: return FailureKind::kUnauthorized;
case 402: return FailureKind::kCredits;
case 403: return FailureKind::kForbidden;
case 404: return FailureKind::kNotFound;
case 408: case 504: case 524: return FailureKind::kTimeout;
case 429: return FailureKind::kRateLimited;
case 502: return FailureKind::kProviderFailed;
case 503: return FailureKind::kNoProvider;
case 529: return FailureKind::kOverloaded;
default: break;
}
return s >= 500 ? FailureKind::kServer : FailureKind::kOther;
}
bool failure_retryable(FailureKind k) {
return k == FailureKind::kTimeout || k == FailureKind::kRateLimited ||
k == FailureKind::kProviderFailed || k == FailureKind::kNoProvider ||
k == FailureKind::kOverloaded;
}
ErrorInfo describe(const llm::ApiError& e) {
ErrorInfo info;
info.kind = classify_status(e.http_status());
// A typed error_type refines the status (a 403 refusal vs a budget block).
if (e.type() == "rate_limit_exceeded") info.kind = FailureKind::kRateLimited;
if (e.type() == "payment_required") info.kind = FailureKind::kCredits;
info.retryable = failure_retryable(info.kind);
switch (info.kind) {
case FailureKind::kCredits:
info.hint = "Add credits or raise the key limit, then retry (402 is not a wait-and-retry case).";
break;
case FailureKind::kRateLimited:
info.hint = "Rate limited: wait (Retry-After) and retry; free models have daily limits.";
break;
case FailureKind::kProviderFailed:
info.hint = "The chosen model or provider failed: retry, or add fallbacks with `models` and provider.allow_fallbacks.";
break;
case FailureKind::kNoProvider:
info.hint = "No provider meets the routing requirements: relax provider.order/only/require_parameters/max_price.";
break;
case FailureKind::kForbidden:
info.hint = "Blocked by a guardrail, moderation or a budget: see the error message.";
break;
case FailureKind::kUnauthorized:
info.hint = "The key is missing, invalid, disabled or expired.";
break;
case FailureKind::kInvalidRequest:
info.hint = "The request was rejected as malformed.";
break;
default:
break;
}
return info;
}
bool is_known_error_type(const std::string& type) { return status_for_error_type(type) != 0; }
int status_for_error_type(const std::string& type) {
for (const TypedError& t : kTyped)
if (type == t.type) return t.status;
return 0;
}
ErrorBody ErrorBody::parse(const Json& err) {
ErrorBody b;
if (!err.is_object()) {
b.message = err.is_string() ? err.as_string() : "unknown error";
return b;
}
if (const Json* c = err.find("code")) {
if (c->is_number()) b.code = static_cast<int>(c->as_int());
else if (c->is_string()) b.code = std::atoi(c->as_string().c_str());
}
b.message = string_of(err.find("message"));
if (const Json* m = err.find("metadata"); m && m->is_object()) {
b.metadata = *m;
b.error_type = string_of(m->find("error_type"));
b.provider_code = string_of(m->find("provider_code"));
b.provider_name = string_of(m->find("provider_name"));
}
// Anthropic skin: error_type sits on the error object itself.
if (b.error_type.empty()) b.error_type = string_of(err.find("error_type"));
return b;
}
llm::ApiError ErrorBody::to_api_error() const {
int status = code;
if (status < 400 || status > 599) status = status_for_error_type(error_type);
if (status < 400 || status > 599) status = 502;
std::string type = !error_type.empty() ? error_type : std::to_string(code);
return llm::ApiError(kVendor, status, type, message.empty() ? "provider error" : message);
}
void throw_if_error_body(const Json& doc) {
if (!doc.is_object()) return;
const Json* err = doc.find("error");
if (err && err->is_object()) throw ErrorBody::parse(*err).to_api_error();
}
} // namespace molo::openrouter
+72
View File
@@ -0,0 +1,72 @@
#pragma once
// OpenRouter error semantics (https://openrouter.ai/docs/api-reference/errors).
// Envelope: {"error":{"code":402,"message":"...","metadata":{...}}} with a
// NUMERIC code. Provider errors add metadata.error_type (a stable typed
// vocabulary), provider_code, raw, provider_name. A 200 response can still
// carry an error: a mid-stream error arrives as an SSE chunk with a top-level
// `error` and finish_reason "error"; a non-streaming provider failure arrives
// as a 200 JSON body holding only `error`. Both are mapped to llm::ApiError
// here so callers have one failure path.
#include <string>
#include "clients/body/json.h"
#include "llm/error.h"
namespace molo::openrouter {
enum class FailureKind {
kInvalidRequest, // 400, 413, 422
kUnauthorized, // 401
kCredits, // 402 insufficient credits (or an in-flight budget)
kForbidden, // 403 guardrail, moderation, budget, permission
kNotFound, // 404
kTimeout, // 408, 504, 524
kRateLimited, // 429
kProviderFailed, // 502 model down or invalid provider response
kNoProvider, // 503 no provider meets the routing requirements
kOverloaded, // 529
kServer, // other 5xx
kOther,
};
const char* failure_kind_name(FailureKind kind);
FailureKind classify_status(int http_status);
// True when waiting and retrying can help (408, 429, 502, 503, 504, 524, 529).
bool failure_retryable(FailureKind kind);
struct ErrorInfo {
FailureKind kind = FailureKind::kOther;
bool retryable = false;
std::string hint; // One line for people reading logs.
};
// Never throws. Based on the status and the vendor type string only (the
// foundation ApiError does not carry the error metadata or Retry-After).
ErrorInfo describe(const llm::ApiError& error);
// The typed error_type vocabulary from the docs (metadata.error_type).
bool is_known_error_type(const std::string& type);
// HTTP status the docs assign to an error_type, 0 when unknown.
int status_for_error_type(const std::string& type);
// The `error` object of a chunk or body.
struct ErrorBody {
int code = 0;
std::string message;
std::string error_type; // metadata.error_type
std::string provider_code; // metadata.provider_code
std::string provider_name; // metadata.provider_name
clients::body::Json metadata;
static ErrorBody parse(const clients::body::Json& error_object);
// vendor "openrouter"; type is error_type when present, else the code.
llm::ApiError to_api_error() const;
};
// Throws when `doc` is a JSON object with a top-level `error` object (the
// 200-with-error shapes). Does nothing otherwise.
void throw_if_error_body(const clients::body::Json& doc);
} // namespace molo::openrouter
+260
View File
@@ -0,0 +1,260 @@
#include "openrouter/gateway.h"
#include <cctype>
#include "clients/body/form.h"
#include "clients/transports/http_transport.h"
#include "llm/caps.h"
#include "llm/config.h"
#include "llm/error.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
bool safe_header_text(const std::string& s) {
for (unsigned char c : s)
if (c < 0x20 || c == 0x7f) return false;
return true;
}
bool valid_category(const std::string& c) {
if (c.empty() || c.size() > Caps::kMaxCategoryBytes) return false;
for (char ch : c)
if (!((ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '-')) return false;
return true;
}
bool base_url_ok(const std::string& url) {
if (url.rfind("https://", 0) == 0) return true;
return url.rfind("http://127.0.0.1", 0) == 0 || url.rfind("http://localhost", 0) == 0;
}
std::string base_url_from_env() {
std::string url = llm::Config::get("OPENROUTER_BASE_URL");
if (url.empty()) return kDefaultBaseUrl;
while (!url.empty() && url.back() == '/') url.pop_back();
if (!base_url_ok(url)) bad("OPENROUTER_BASE_URL must be https (or loopback http)");
return url;
}
clients::ClientConfig anon_config() {
clients::ClientConfig c;
c.persist_cookies = false;
return c;
}
llm::ApiClientConfig client_config(const std::string& env, const std::string& base,
const llm::Headers& headers) {
llm::ApiClientConfig c;
c.vendor = kVendor;
c.base_url = base;
c.key_env = env;
c.auth = llm::AuthStyle::kBearer;
c.default_headers = headers;
return c;
}
} // namespace
// ---------------------------------------------------------------------------
// Attribution
Attribution Attribution::from_env() {
Attribution a;
a.referer = llm::Config::get("OPENROUTER_APP_URL");
a.title = llm::Config::get("OPENROUTER_APP_TITLE");
a.categories = llm::Config::get("OPENROUTER_APP_CATEGORIES");
return a;
}
void Attribution::validate() const {
if (referer.size() > Caps::kMaxAppUrlBytes || !safe_header_text(referer))
bad("app url is too long or has control characters");
if (title.size() > Caps::kMaxAppTitleBytes || !safe_header_text(title))
bad("app title is too long or has control characters");
if (!categories.empty()) {
std::size_t count = 0, pos = 0;
while (pos <= categories.size()) {
std::size_t comma = categories.find(',', pos);
std::string one = categories.substr(pos, comma == std::string::npos ? comma : comma - pos);
if (!valid_category(one)) bad("category must be lowercase a-z 0-9 hyphen, at most 30 chars");
++count;
if (comma == std::string::npos) break;
pos = comma + 1;
}
if (count > Caps::kMaxCategories) bad("at most 2 categories per request");
}
}
llm::Headers Attribution::headers() const {
validate();
llm::Headers h;
if (!referer.empty()) h["HTTP-Referer"] = referer;
if (!title.empty()) {
h["X-OpenRouter-Title"] = title;
h["X-Title"] = title; // legacy name, still accepted by OpenRouter
}
if (!categories.empty()) h["X-OpenRouter-Categories"] = categories;
if (hidden) h["X-OpenRouter-App-Visibility"] = "hidden";
if (router_metadata) h["X-OpenRouter-Metadata"] = "enabled";
return h;
}
// ---------------------------------------------------------------------------
// AnonymousClient
AnonymousClient::AnonymousClient(std::unique_ptr<clients::Transport> transport,
std::string base_url, llm::Headers default_headers)
: base_url_(std::move(base_url)),
headers_(std::move(default_headers)),
client_(std::move(transport), anon_config()) {}
llm::RawResponse AnonymousClient::send(clients::Method method, const std::string& target,
const std::string* body,
const std::string& content_type,
const llm::Headers& extra) {
if (target.empty() || target.front() != '/' || target.find("..") != std::string::npos)
bad("bad request path");
try {
clients::ClientRequest req(method, clients::Url::parse(base_url_ + target));
for (const auto& [k, v] : headers_) req.header(k, v);
for (const auto& [k, v] : extra) req.header(k, v);
if (body) {
if (body->size() > llm::Caps::kMaxRequestBytes) bad("request body over the byte cap");
req.bytes(*body, content_type);
}
clients::ClientResponse res = client_.send_blocking(std::move(req));
if (!res.is_success()) throw llm::error_from_response(kVendor, res);
if (res.body().size() > llm::Caps::kMaxResponseBytes)
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "response over the byte cap");
llm::RawResponse out;
out.status = res.status();
out.body = res.body();
out.content_type = res.content_type();
out.headers = res.headers();
return out;
} catch (const llm::ApiError&) {
throw;
} catch (const std::exception& e) {
throw llm::ApiError(kVendor, 502, "UNAVAILABLE", e.what());
}
}
// ---------------------------------------------------------------------------
// Gateway
Gateway::Gateway(std::shared_ptr<llm::ApiClient> api,
std::shared_ptr<llm::ApiClient> management,
std::shared_ptr<AnonymousClient> anonymous)
: api_(std::move(api)), management_(std::move(management)), anonymous_(std::move(anonymous)) {
if (!api_ || !management_ || !anonymous_) bad("gateway needs all three clients");
}
std::shared_ptr<Gateway> Gateway::from_env(const Attribution& attribution) {
llm::Headers headers = attribution.headers();
std::string base = base_url_from_env();
auto api = llm::ApiClient::from_env(client_config(kApiKeyEnv, base, headers));
auto mgmt = llm::ApiClient::from_env(client_config(kManagementKeyEnv, base, headers));
auto anon = std::make_shared<AnonymousClient>(
std::make_unique<clients::transports::HttpTransport>(), base, headers);
return std::make_shared<Gateway>(std::move(api), std::move(mgmt), std::move(anon));
}
bool Gateway::ready(KeyKind kind) const {
switch (kind) {
case KeyKind::kApi: return api_->configured();
case KeyKind::kManagement: return management_->configured();
case KeyKind::kNone: return true;
}
return false;
}
llm::ApiClient& Gateway::client_for(KeyKind kind) const {
return kind == KeyKind::kManagement ? *management_ : *api_;
}
Json Gateway::call(const PlannedCall& c, const llm::Headers& extra) const {
const std::string fmt = c.fmt();
if (fmt == "multipart" || fmt == "raw") bad(c.name() + " is a " + fmt + " call");
const std::string target = c.target();
if (fmt == "form") {
clients::body::UrlEncodedForm form;
if (c.has_body)
for (const auto& [k, v] : c.body.as_object()) {
if (!v.is_string()) bad("form field must be a string: " + k);
form.add(k, v.as_string());
}
std::string encoded = form.encode();
llm::RawResponse raw;
if (c.key == KeyKind::kNone) {
raw = anonymous_->send(c.method, target, &encoded,
clients::body::UrlEncodedForm::content_type(), extra);
} else {
raw = client_for(c.key).request_raw(c.method, target, &encoded,
clients::body::UrlEncodedForm::content_type(), extra);
}
if (raw.body.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
try {
return Json::parse(raw.body);
} catch (const std::exception& e) {
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", e.what());
}
}
if (c.key == KeyKind::kNone) {
std::string text = c.has_body ? c.body.dump() : "";
llm::RawResponse raw = anonymous_->send(c.method, target, c.has_body ? &text : nullptr,
"application/json", extra);
if (raw.body.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
try {
return Json::parse(raw.body);
} catch (const std::exception& e) {
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", e.what());
}
}
return client_for(c.key).request(c.method, target, c.has_body ? &c.body : nullptr, extra);
}
llm::RawResponse Gateway::call_raw(const PlannedCall& c, const llm::Headers& extra) const {
if (c.fmt() != "raw") bad(c.name() + " is not a raw call");
std::string text = c.has_body ? c.body.dump() : "";
const std::string* body = c.has_body ? &text : nullptr;
const char* type = c.has_body ? "application/json" : "";
if (c.key == KeyKind::kNone) return anonymous_->send(c.method, c.target(), body, type, extra);
return client_for(c.key).request_raw(c.method, c.target(), body, type, extra);
}
Json Gateway::call_multipart(const PlannedCall& c, const clients::body::MultipartBody& form,
const llm::Headers& extra) const {
if (c.fmt() != "multipart") bad(c.name() + " is not a multipart call");
if (c.key == KeyKind::kNone) bad("multipart calls need a key");
return client_for(c.key).post_multipart(c.target(), form, extra);
}
void Gateway::stream(const PlannedCall& c, const llm::SseDecoder::Sink& sink,
const llm::Headers& extra) const {
if (!c.has_body || c.method != clients::Method::kPost) bad(c.name() + " cannot stream");
if (c.key == KeyKind::kNone) bad("streaming calls need a key");
client_for(c.key).stream(c.target(), c.body, sink, extra);
}
// ---------------------------------------------------------------------------
// Endpoint
Endpoint::Endpoint(std::shared_ptr<Gateway> gateway, KeyKind primary)
: llm::Service(gateway->api()), gateway_(std::move(gateway)), primary_(primary) {}
Json Endpoint::run(std::string_view name, const PathArgs& args, const Query& query,
const Json* body, const llm::Headers& extra) const {
return gateway_->call(CallPlanner::plan(name, args, query, body), extra);
}
llm::RawResponse Endpoint::run_raw(std::string_view name, const PathArgs& args,
const Query& query) const {
return gateway_->call_raw(CallPlanner::plan(name, args, query, nullptr));
}
} // namespace molo::openrouter
+114
View File
@@ -0,0 +1,114 @@
#pragma once
// Gateway: the one place OpenRouter traffic leaves the module. Owns the two
// credentialed ApiClients (inference key and management key are different
// credentials with different powers) and an anonymous client for the public
// routes that need no key (models, providers, counts, jwks, ...). Applies the
// attribution headers to all of them.
#include <memory>
#include <string>
#include "clients/body/json.h"
#include "clients/transport.h"
#include "llm/api_client.h"
#include "openrouter/call_plan.h"
namespace molo::openrouter {
inline constexpr const char* kDefaultBaseUrl = "https://openrouter.ai/api/v1";
inline constexpr const char* kApiKeyEnv = "OPENROUTER_API_KEY";
inline constexpr const char* kManagementKeyEnv = "OPENROUTER_MANAGEMENT_KEY";
// App attribution (https://openrouter.ai/docs/app-attribution). HTTP-Referer
// is what identifies the app for rankings; the title and categories refine it.
struct Attribution {
std::string referer; // HTTP-Referer
std::string title; // X-OpenRouter-Title (and the legacy X-Title)
std::string categories; // X-OpenRouter-Categories: "cli-agent,cloud-agent", at most 2
bool hidden = false; // X-OpenRouter-App-Visibility: hidden (new apps only)
bool router_metadata = false; // X-OpenRouter-Metadata: enabled
// OPENROUTER_APP_URL, OPENROUTER_APP_TITLE, OPENROUTER_APP_CATEGORIES via llm::Config.
static Attribution from_env();
// Throws llm::ApiError::invalid on over-long or malformed values.
void validate() const;
llm::Headers headers() const;
};
// Keyless client for the public routes. Same error mapping as ApiClient.
class AnonymousClient {
public:
AnonymousClient(std::unique_ptr<clients::Transport> transport, std::string base_url,
llm::Headers default_headers);
llm::RawResponse send(clients::Method method, const std::string& target,
const std::string* body, const std::string& content_type,
const llm::Headers& extra = {});
clients::ClientStatsSnapshot stats() const { return client_.stats(); }
private:
std::string base_url_;
llm::Headers headers_;
clients::Client client_;
};
class Gateway {
public:
Gateway(std::shared_ptr<llm::ApiClient> api, std::shared_ptr<llm::ApiClient> management,
std::shared_ptr<AnonymousClient> anonymous);
// Real HTTPS clients, keys from llm::Config, base URL from OPENROUTER_BASE_URL
// when set (https, or http on loopback only), attribution as given.
static std::shared_ptr<Gateway> from_env(const Attribution& attribution = Attribution::from_env());
bool api_configured() const { return api_->configured(); }
bool management_configured() const { return management_->configured(); }
// True when a call of this key kind can run (public calls always can).
bool ready(KeyKind kind) const;
const std::shared_ptr<llm::ApiClient>& api() const { return api_; }
const std::shared_ptr<llm::ApiClient>& management() const { return management_; }
// JSON in, JSON out ("json" and "form" calls; form bodies are url-encoded).
Json call(const PlannedCall& call, const llm::Headers& extra = {}) const;
// Bytes out ("raw" calls such as file and video content).
llm::RawResponse call_raw(const PlannedCall& call, const llm::Headers& extra = {}) const;
// "multipart" calls (file upload).
Json call_multipart(const PlannedCall& call, const clients::body::MultipartBody& form,
const llm::Headers& extra = {}) const;
// POST with SSE streaming. The planned call must carry a body.
void stream(const PlannedCall& call, const llm::SseDecoder::Sink& sink,
const llm::Headers& extra = {}) const;
private:
llm::ApiClient& client_for(KeyKind kind) const;
std::shared_ptr<llm::ApiClient> api_;
std::shared_ptr<llm::ApiClient> management_;
std::shared_ptr<AnonymousClient> anonymous_;
};
// Base for the service classes. Holds the gateway; the llm::Service base
// points at the inference client (use ready() to ask the right key).
class Endpoint : public llm::Service {
public:
explicit Endpoint(std::shared_ptr<Gateway> gateway, KeyKind primary = KeyKind::kApi);
const std::shared_ptr<Gateway>& gateway() const { return gateway_; }
// True when this service's primary key kind is configured.
bool ready() const { return gateway_->ready(primary_); }
KeyKind primary_key() const { return primary_; }
protected:
Json run(std::string_view name, const PathArgs& args = {}, const Query& query = {},
const Json* body = nullptr, const llm::Headers& extra = {}) const;
Json run(std::string_view name, const PathArgs& args, const Query& query,
const Json& body, const llm::Headers& extra = {}) const {
return run(name, args, query, &body, extra);
}
llm::RawResponse run_raw(std::string_view name, const PathArgs& args = {},
const Query& query = {}) const;
std::shared_ptr<Gateway> gateway_;
KeyKind primary_;
};
} // namespace molo::openrouter
+84
View File
@@ -0,0 +1,84 @@
#include "openrouter/inference_services.h"
#include "llm/error.h"
namespace molo::openrouter {
namespace {
void reject_stream_flag(const Json& body) {
if (const Json* s = body.find("stream"); s && s->is_bool() && s->as_bool())
throw llm::ApiError::invalid(kVendor, "use stream() for stream:true");
}
Json streaming(const Json& body) {
Json b = body;
b.set("stream", Json(true));
return b;
}
} // namespace
Json ResponsesService::create(const Json& body) const {
reject_stream_flag(body);
Json doc = run("createResponses", {}, {}, body);
throw_if_error_body(doc);
if (const Json* st = doc.find("status"); st && st->is_string() && st->as_string() == "failed") {
ErrorBody e;
if (const Json* er = doc.find("error"); er && er->is_object()) {
if (const Json* m = er->find("message"); m && m->is_string()) e.message = m->as_string();
if (const Json* c = er->find("code"); c && c->is_string()) e.provider_code = c->as_string();
}
if (const Json* et = doc.find("error_type"); et && et->is_string()) e.error_type = et->as_string();
e.code = status_for_error_type(e.error_type);
throw e.to_api_error();
}
return doc;
}
StreamSummary ResponsesService::stream(
const Json& body,
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const {
Json b = streaming(body);
PlannedCall call = CallPlanner::plan("createResponses", {}, {}, &b);
ResponsesStreamParser parser;
gateway_->stream(call, [&](const llm::SseEvent& ev) {
std::string delta = parser.feed(ev);
if (on_text && !on_text(delta, ev)) return false;
return !parser.summary().error;
});
parser.throw_if_errored();
return parser.summary();
}
Json MessagesService::create(const Json& body) const {
reject_stream_flag(body);
if (!body.has("model") || !body.has("messages"))
throw llm::ApiError::invalid(kVendor, "messages: model and messages are required");
Json doc = run("createMessages", {}, {}, body);
throw_if_error_body(doc);
if (const Json* t = doc.find("type"); t && t->is_string() && t->as_string() == "error") {
const Json* er = doc.find("error");
throw ErrorBody::parse(er ? *er : Json::object()).to_api_error();
}
return doc;
}
StreamSummary MessagesService::stream(
const Json& body,
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const {
if (!body.has("model") || !body.has("messages"))
throw llm::ApiError::invalid(kVendor, "messages: model and messages are required");
Json b = streaming(body);
PlannedCall call = CallPlanner::plan("createMessages", {}, {}, &b);
MessagesStreamParser parser;
gateway_->stream(call, [&](const llm::SseEvent& ev) {
std::string delta = parser.feed(ev);
if (on_text && !on_text(delta, ev)) return false;
return !parser.summary().error;
});
parser.throw_if_errored();
return parser.summary();
}
} // namespace molo::openrouter
+40
View File
@@ -0,0 +1,40 @@
#pragma once
// ResponsesService (POST /responses, beta) and MessagesService (POST
// /messages, the Anthropic-compatible skin). Bodies are JSON objects checked
// against the call table (calls.h); the request shapes of these two skins are
// large and owned by their respective protocols, so they are not re-modelled
// field by field. Routing fields (provider, models, route, plugins, trace,
// session_id) are the same objects ChatParams builds: use ProviderPreferences,
// plugins_to_json and friends to fill them.
#include <functional>
#include "openrouter/gateway.h"
#include "openrouter/stream_summary.h"
namespace molo::openrouter {
class ResponsesService : public Endpoint {
public:
explicit ResponsesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
// Non-streaming. Throws on HTTP errors, on a top-level error object and on
// a response whose status is "failed".
Json create(const Json& body) const;
// Streaming. `on_text` gets each text delta and the raw event; return false
// to stop. The summary holds text, reasoning, usage and status. A failed
// response or `error` event is thrown after delivery.
StreamSummary stream(const Json& body,
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const;
};
class MessagesService : public Endpoint {
public:
explicit MessagesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
// `model` and `messages` are required by the spec.
Json create(const Json& body) const;
StreamSummary stream(const Json& body,
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const;
};
} // namespace molo::openrouter
+108
View File
@@ -0,0 +1,108 @@
#include "openrouter/model_ref.h"
#include <algorithm>
#include "llm/api_client.h"
#include "llm/error.h"
#include "openrouter/call_plan.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
bool segment_ok(const std::string& s, bool allow_alias) {
std::string t = allow_alias && s.rfind("~", 0) == 0 ? s.substr(1) : s;
if (t.empty() || t.size() > 128) return false;
for (char c : t)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '-' || c == '_' || c == '.'))
return false;
return true;
}
} // namespace
bool ModelRef::is_catalog_variant(const std::string& s) {
return s == "free" || s == "batch" || s == "thinking" || s == "extended";
}
bool ModelRef::is_routing_variant(const std::string& s) {
return s == "nitro" || s == "floor" || s == "exacto" || s == "online";
}
bool ModelRef::is_deprecated_variant(const std::string& s) {
return s == "thinking" || s == "extended" || s == "online";
}
ModelRef ModelRef::parse(const std::string& id) {
std::size_t slash = id.find('/');
if (slash == std::string::npos || id.find('/', slash + 1) != std::string::npos)
bad("model id must look like author/slug");
ModelRef out;
out.author_ = id.substr(0, slash);
std::string rest = id.substr(slash + 1);
if (!segment_ok(out.author_, true)) bad("bad model author");
std::size_t pos = 0;
bool first = true;
while (true) {
std::size_t colon = rest.find(':', pos);
std::string part = rest.substr(pos, colon == std::string::npos ? colon : colon - pos);
if (first) {
if (!segment_ok(part, false)) bad("bad model slug");
out.slug_ = part;
first = false;
} else if (is_catalog_variant(part)) {
if (!out.catalog_.empty()) bad("a model id carries at most one catalog variant");
out.catalog_ = part;
} else if (is_routing_variant(part)) {
if (std::find(out.routing_.begin(), out.routing_.end(), part) == out.routing_.end())
out.routing_.push_back(part);
} else {
bad("unknown model variant: " + part);
}
if (colon == std::string::npos) break;
pos = colon + 1;
}
return out;
}
std::string ModelRef::str() const {
std::string s = base();
if (!catalog_.empty()) s += ":" + catalog_;
for (const std::string& r : routing_) s += ":" + r;
return s;
}
std::string ModelRef::catalog_id() const {
return catalog_.empty() ? base() : base() + ":" + catalog_;
}
bool ModelRef::uses_deprecated_variant() const {
if (is_deprecated_variant(catalog_)) return true;
for (const std::string& r : routing_)
if (is_deprecated_variant(r)) return true;
return false;
}
ModelRef ModelRef::with_variant(const std::string& suffix) const {
ModelRef out = *this;
if (is_catalog_variant(suffix)) {
if (!out.catalog_.empty() && out.catalog_ != suffix)
bad("a model id carries at most one catalog variant");
out.catalog_ = suffix;
} else if (is_routing_variant(suffix)) {
if (std::find(out.routing_.begin(), out.routing_.end(), suffix) == out.routing_.end())
out.routing_.push_back(suffix);
} else {
bad("unknown model variant: " + suffix);
}
return out;
}
ModelRef ModelRef::without_routing_variants() const {
ModelRef out = *this;
out.routing_.clear();
return out;
}
} // namespace molo::openrouter
+57
View File
@@ -0,0 +1,57 @@
#pragma once
// ModelRef: an OpenRouter model id with its variant suffixes, per the model
// variants guide (https://openrouter.ai/docs/guides/routing/model-variants).
//
// author/slug[:catalog][:routing]...
//
// catalog variants (own entry in GET /models; at most one):
// free, batch, thinking (deprecated), extended (deprecated)
// routing variants (accepted on any model, never listed; any number, the
// last sorting one wins): nitro, floor, exacto, online (deprecated)
//
// Any other suffix is rejected: the docs say not to rely on other suffixes.
// An author may carry the alias marker "~" (for example ~openai/gpt-latest).
#include <string>
#include <vector>
namespace molo::openrouter {
class ModelRef {
public:
// Throws llm::ApiError::invalid on a malformed id or an unknown suffix.
static ModelRef parse(const std::string& id);
static bool is_catalog_variant(const std::string& suffix);
static bool is_routing_variant(const std::string& suffix);
static bool is_deprecated_variant(const std::string& suffix);
const std::string& author() const { return author_; }
const std::string& slug() const { return slug_; }
const std::string& catalog_variant() const { return catalog_; }
const std::vector<std::string>& routing_variants() const { return routing_; }
// "author/slug".
std::string base() const { return author_ + "/" + slug_; }
// The id as sent in a request `model` field.
std::string str() const;
// The id of the catalog entry that describes this model: base plus the
// catalog variant, routing variants dropped (":free" is NOT dropped).
std::string catalog_id() const;
// True when any suffix on this id is deprecated.
bool uses_deprecated_variant() const;
// Returns a copy with the variant added. A second catalog variant or an
// unknown suffix throws; a repeated routing variant is kept once.
ModelRef with_variant(const std::string& suffix) const;
ModelRef without_routing_variants() const;
private:
std::string author_;
std::string slug_;
std::string catalog_;
std::vector<std::string> routing_;
};
} // namespace molo::openrouter
+374
View File
@@ -0,0 +1,374 @@
#include "openrouter/models.h"
#include <algorithm>
#include <cmath>
#include <cstdlib>
#include "llm/error.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, "models: " + why); }
bool in(const std::string& v, std::initializer_list<const char*> all) {
return std::any_of(all.begin(), all.end(), [&](const char* s) { return v == s; });
}
std::string str(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
long integer(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
return v && v->is_number() ? v->as_int() : 0;
}
std::optional<double> real(const Json& o, const char* k) {
const Json* v = o.is_object() ? o.find(k) : nullptr;
if (v && v->is_number()) return v->as_number();
return std::nullopt;
}
std::vector<std::string> strings(const Json& o, const char* k) {
std::vector<std::string> out;
const Json* v = o.is_object() ? o.find(k) : nullptr;
if (v && v->is_array())
for (const Json& s : v->as_array())
if (s.is_string()) out.push_back(s.as_string());
return out;
}
std::string join(const std::vector<std::string>& v) {
std::string out;
for (const std::string& s : v) out += (out.empty() ? "" : ",") + s;
return out;
}
void check_csv(const char* what, const std::vector<std::string>& v) {
if (v.size() > Caps::kMaxProviderList) bad(std::string(what) + " has too many entries");
for (const std::string& s : v) {
if (s.empty() || s.size() > 128 || s.find(',') != std::string::npos)
bad(std::string(what) + " entries must be non-empty, without commas");
}
}
void check_pair(const char* what, const std::optional<double>& lo, const std::optional<double>& hi,
double min_allowed, double max_allowed) {
for (const auto* v : {&lo, &hi})
if (*v && (!std::isfinite(**v) || **v < min_allowed || **v > max_allowed))
bad(std::string(what) + " out of range");
if (lo && hi && *lo > *hi) bad(std::string(what) + ": min is above max");
}
void check_limit(const std::optional<int>& offset, const std::optional<int>& limit) {
if (offset && *offset < 0) bad("offset must be >= 0");
if (limit && (*limit < 1 || *limit > Caps::kMaxListLimit)) bad("limit must be 1..1000");
}
ModelPage parse_page(const Json& doc) {
ModelPage p;
const Json* data = doc.is_object() ? doc.find("data") : nullptr;
if (!data || !data->is_array()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "models response has no data array");
for (const Json& m : data->as_array()) p.models.push_back(ModelInfo::parse(m));
p.total_count = integer(doc, "total_count");
if (p.total_count == 0) p.total_count = static_cast<long>(p.models.size());
if (const Json* links = doc.find("links"); links && links->is_object()) p.next = str(*links, "next");
return p;
}
} // namespace
bool ModelFilter::valid_sort(const std::string& s) {
return in(s, {"most-popular", "newest", "top-weekly", "pricing-low-to-high", "pricing-high-to-low",
"context-high-to-low", "throughput-high-to-low", "latency-low-to-high",
"intelligence-high-to-low", "coding-high-to-low", "agentic-high-to-low",
"design-arena-elo-high-to-low"});
}
bool ModelFilter::valid_category(const std::string& s) {
return in(s, {"programming", "roleplay", "marketing", "marketing/seo", "technology", "science",
"translation", "legal", "finance", "health", "trivia", "academia"});
}
bool ModelFilter::valid_output_modality(const std::string& s) {
return in(s, {"text", "image", "embeddings", "audio", "video", "rerank", "decisions", "speech",
"transcription", "all"});
}
bool ModelFilter::valid_input_modality(const std::string& s) {
return in(s, {"text", "image", "audio", "file"});
}
void ModelFilter::validate() const {
check_limit(offset, limit);
if (!category.empty() && !valid_category(category)) bad("unknown category");
if (!sort.empty() && !valid_sort(sort)) bad("unknown sort");
check_csv("supported_parameters", supported_parameters);
check_csv("model_authors", model_authors);
check_csv("providers", providers);
for (const std::string& m : output_modalities)
if (!valid_output_modality(m)) bad("unknown output modality: " + m);
for (const std::string& m : input_modalities)
if (!valid_input_modality(m)) bad("unknown input modality: " + m);
if (!region.empty() && region != "eu" && region != "us") bad("region must be eu or us");
if (context && *context < 1) bad("context must be >= 1");
if (q.size() > 256 || arch.size() > 128) bad("q or arch too long");
constexpr double kBig = 1e9;
check_pair("price", min_price, max_price, 0, kBig);
check_pair("output price", min_output_price, max_output_price, 0, kBig);
check_pair("intelligence index", min_intelligence_index, max_intelligence_index, 0, kBig);
check_pair("coding index", min_coding_index, max_coding_index, 0, kBig);
check_pair("agentic index", min_agentic_index, max_agentic_index, 0, kBig);
check_pair("tool success rate", min_tool_success_rate, max_tool_success_rate, 0, 1);
if ((min_age_days && *min_age_days < 0) || (max_age_days && *max_age_days < 0)) bad("age days must be >= 0");
if (min_age_days && max_age_days && *min_age_days > *max_age_days) bad("age days: min is above max");
}
Query ModelFilter::to_query() const {
validate();
Query q_;
if (offset) q_.add("offset", *offset);
if (limit) q_.add("limit", *limit);
q_.add_if("category", category);
q_.add_if("supported_parameters", join(supported_parameters));
q_.add_if("output_modalities", join(output_modalities));
q_.add_if("input_modalities", join(input_modalities));
q_.add_if("sort", sort);
q_.add_if("q", q);
if (context) q_.add("context", *context);
auto d = [&](const char* k, const std::optional<double>& v) { if (v) q_.add(k, *v); };
d("min_price", min_price);
d("max_price", max_price);
q_.add_if("arch", arch);
q_.add_if("model_authors", join(model_authors));
q_.add_if("providers", join(providers));
if (distillable) q_.add("distillable", *distillable ? "true" : "false");
if (zdr) q_.add("zdr", "true");
q_.add_if("region", region);
d("min_output_price", min_output_price);
d("max_output_price", max_output_price);
if (min_age_days) q_.add("min_age_days", *min_age_days);
if (max_age_days) q_.add("max_age_days", *max_age_days);
d("min_intelligence_index", min_intelligence_index);
d("max_intelligence_index", max_intelligence_index);
d("min_coding_index", min_coding_index);
d("max_coding_index", max_coding_index);
d("min_agentic_index", min_agentic_index);
d("max_agentic_index", max_agentic_index);
d("min_tool_success_rate", min_tool_success_rate);
d("max_tool_success_rate", max_tool_success_rate);
return q_;
}
ModelInfo ModelInfo::parse(const Json& doc) {
if (!doc.is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "model is not an object");
ModelInfo m;
m.raw = doc;
m.id = str(doc, "id");
m.canonical_slug = str(doc, "canonical_slug");
m.name = str(doc, "name");
m.description = str(doc, "description");
m.created = integer(doc, "created");
m.context_length = integer(doc, "context_length");
if (const Json* a = doc.find("architecture"); a && a->is_object()) {
m.modality = str(*a, "modality");
m.input_modalities = strings(*a, "input_modalities");
m.output_modalities = strings(*a, "output_modalities");
m.tokenizer = str(*a, "tokenizer");
}
m.supported_parameters = strings(doc, "supported_parameters");
if (const Json* p = doc.find("pricing")) m.pricing = *p;
if (const Json* t = doc.find("top_provider")) m.top_provider = *t;
if (const Json* d = doc.find("default_parameters")) m.default_parameters = *d;
if (const Json* r = doc.find("reasoning")) m.reasoning = *r;
m.expiration_date = str(doc, "expiration_date");
if (const Json* at = doc.find("alias_target")) m.alias_target = at->is_string() ? at->as_string() : at->dump();
return m;
}
bool ModelInfo::supports(const std::string& parameter) const {
return std::find(supported_parameters.begin(), supported_parameters.end(), parameter) !=
supported_parameters.end();
}
std::optional<double> ModelInfo::price_per_million(const std::string& key) const {
const Json* v = pricing.is_object() ? pricing.find(key) : nullptr;
if (!v) return std::nullopt;
double per_token;
if (v->is_number()) {
per_token = v->as_number();
} else if (v->is_string()) {
char* end = nullptr;
per_token = std::strtod(v->as_string().c_str(), &end);
if (end == v->as_string().c_str()) return std::nullopt;
} else {
return std::nullopt;
}
return per_token * 1e6;
}
bool ModelInfo::is_free() const {
if (id.size() > 5 && id.compare(id.size() - 5, 5, ":free") == 0) return true;
auto p = price_per_million("prompt");
auto c = price_per_million("completion");
return p && c && *p == 0 && *c == 0;
}
EndpointInfo EndpointInfo::parse(const Json& d) {
if (!d.is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "endpoint is not an object");
EndpointInfo e;
e.raw = d;
e.name = str(d, "name");
e.model_id = str(d, "model_id");
e.model_name = str(d, "model_name");
e.provider_name = str(d, "provider_name");
e.tag = str(d, "tag");
e.quantization = str(d, "quantization");
e.service_tier = str(d, "service_tier");
e.context_length = integer(d, "context_length");
if (const Json* v = d.find("max_completion_tokens"); v && v->is_number()) e.max_completion_tokens = v->as_int();
if (const Json* v = d.find("max_prompt_tokens"); v && v->is_number()) e.max_prompt_tokens = v->as_int();
e.status = static_cast<int>(integer(d, "status"));
e.uptime_last_5m = real(d, "uptime_last_5m");
e.uptime_last_30m = real(d, "uptime_last_30m");
e.uptime_last_1d = real(d, "uptime_last_1d");
if (const Json* v = d.find("latency_last_30m")) e.latency_last_30m = *v;
if (const Json* v = d.find("throughput_last_30m")) e.throughput_last_30m = *v;
if (const Json* v = d.find("pricing")) e.pricing = *v;
if (const Json* v = d.find("supports_tool_choice")) e.supports_tool_choice = *v;
e.supported_parameters = strings(d, "supported_parameters");
if (const Json* v = d.find("supports_implicit_caching"); v && v->is_bool()) e.supports_implicit_caching = v->as_bool();
return e;
}
ProviderInfo ProviderInfo::parse(const Json& d) {
if (!d.is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "provider is not an object");
ProviderInfo p;
p.name = str(d, "name");
p.slug = str(d, "slug");
p.privacy_policy_url = str(d, "privacy_policy_url");
p.terms_of_service_url = str(d, "terms_of_service_url");
p.status_page_url = str(d, "status_page_url");
p.headquarters = str(d, "headquarters");
p.datacenters = strings(d, "datacenters");
return p;
}
ModelPage ModelsService::page(const ModelFilter& filter) const {
return parse_page(run("getModels", {}, filter.to_query()));
}
ModelList ModelsService::list_all(const ModelFilter& filter, std::size_t max_models) const {
if (max_models == 0) bad("max_models must be >= 1");
ModelList out;
ModelFilter f = filter;
f.limit = Caps::kModelPageSize;
int offset = filter.offset.value_or(0);
while (true) {
if (out.pages >= Caps::kMaxModelPages || out.models.size() >= max_models) {
out.truncated = true;
break;
}
f.offset = offset;
ModelPage p = page(f);
++out.pages;
out.total_count = p.total_count;
for (ModelInfo& m : p.models) {
if (out.models.size() >= max_models) {
out.truncated = true;
return out;
}
out.models.push_back(std::move(m));
}
if (p.next.empty() || p.models.empty()) break;
offset += static_cast<int>(p.models.size());
if (static_cast<long>(offset) >= p.total_count) break;
}
return out;
}
ModelInfo ModelsService::get(const std::string& model_id) const {
auto [a, s] = split_model_id(model_id);
Json doc = run("getModel", {{"author", a}, {"slug", s}});
const Json* d = doc.find("data");
return ModelInfo::parse(d ? *d : doc);
}
ModelEndpoints ModelsService::endpoints(const std::string& model_id) const {
auto [a, s] = split_model_id(model_id);
Json doc = run("listEndpoints", {{"author", a}, {"slug", s}});
const Json* d = doc.find("data");
if (!d || !d->is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "endpoints response has no data");
ModelEndpoints out;
out.raw = doc;
out.id = str(*d, "id");
out.name = str(*d, "name");
out.created = integer(*d, "created");
out.description = str(*d, "description");
if (const Json* a2 = d->find("architecture")) out.architecture = *a2;
if (const Json* eps = d->find("endpoints"); eps && eps->is_array())
for (const Json& e : eps->as_array()) out.endpoints.push_back(EndpointInfo::parse(e));
return out;
}
long ModelsService::count(const std::string& output_modalities) const {
Query q;
if (!output_modalities.empty()) {
std::size_t pos = 0;
while (pos <= output_modalities.size()) {
std::size_t comma = output_modalities.find(',', pos);
std::string one = output_modalities.substr(pos, comma == std::string::npos ? comma : comma - pos);
if (!ModelFilter::valid_output_modality(one)) bad("unknown output modality: " + one);
if (comma == std::string::npos) break;
pos = comma + 1;
}
q.add("output_modalities", output_modalities);
}
Json doc = run("listModelsCount", {}, q);
const Json* d = doc.find("data");
return d ? integer(*d, "count") : 0;
}
Json ModelsService::zdr_endpoints() const { return run("listEndpointsZdr"); }
Json ModelsService::embeddings_models(std::optional<int> offset, std::optional<int> limit) const {
check_limit(offset, limit);
Query q;
if (offset) q.add("offset", *offset);
if (limit) q.add("limit", *limit);
return run("listEmbeddingsModels", {}, q);
}
Json ModelsService::image_models() const { return run("listImageModels"); }
Json ModelsService::video_models() const { return run("listVideosModels"); }
Json ModelsService::image_model_endpoints(const std::string& model_id) const {
auto [a, s] = split_model_id(model_id);
return run("listImageModelEndpoints", {{"author", a}, {"slug", s}});
}
ModelPage ModelsService::user_models(std::optional<int> offset, std::optional<int> limit,
const std::string& output_modalities,
const std::string& sort) const {
check_limit(offset, limit);
if (!sort.empty() && !ModelFilter::valid_sort(sort)) bad("unknown sort");
Query q;
if (offset) q.add("offset", *offset);
if (limit) q.add("limit", *limit);
q.add_if("output_modalities", output_modalities);
q.add_if("sort", sort);
if (!gateway_->api_configured()) throw llm::NotConfiguredError(kVendor, kApiKeyEnv);
return parse_page(run("listModelsUser", {}, q));
}
std::vector<ProviderInfo> ProvidersService::list() const {
Json doc = run("listProviders");
const Json* d = doc.find("data");
if (!d || !d->is_array()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "providers response has no data array");
std::vector<ProviderInfo> out;
for (const Json& p : d->as_array()) out.push_back(ProviderInfo::parse(p));
return out;
}
} // namespace molo::openrouter
+165
View File
@@ -0,0 +1,165 @@
#pragma once
// Model catalog: GET /models (with every documented filter), /model/{a}/{s},
// /models/{a}/{s}/endpoints (provider, pricing, quantization, latency,
// uptime), /models/count, /models/user, /endpoints/zdr, the embeddings,
// image and video model lists, and GET /providers. The listing routes are
// public: they work without any key. /models/user needs the API key.
#include <optional>
#include <string>
#include <vector>
#include "openrouter/gateway.h"
namespace molo::openrouter {
// Query for GET /models. Every field is optional; validate() is strict.
struct ModelFilter {
std::optional<int> offset;
std::optional<int> limit; // 1..1000; both omitted: the API returns the full list
std::string category; // programming roleplay marketing marketing/seo technology science translation legal finance health trivia academia
std::vector<std::string> supported_parameters; // joined with commas
std::vector<std::string> output_modalities; // text image embeddings audio video rerank decisions speech transcription, or "all"
std::vector<std::string> input_modalities; // text image audio file
std::string sort; // most-popular newest top-weekly pricing-low-to-high pricing-high-to-low context-high-to-low throughput-high-to-low latency-low-to-high intelligence-high-to-low coding-high-to-low agentic-high-to-low design-arena-elo-high-to-low
std::string q; // free text on name or slug
std::optional<int> context; // minimum context length
std::optional<double> min_price, max_price; // prompt, USD per million tokens
std::optional<double> min_output_price, max_output_price; // completion
std::string arch; // GPT, Claude, Gemini, Llama, ...
std::vector<std::string> model_authors; // author slugs
std::vector<std::string> providers; // provider names
std::optional<bool> distillable;
bool zdr = false; // only models with zero data retention endpoints
std::string region; // "", eu, us
std::optional<int> min_age_days, max_age_days;
std::optional<double> min_intelligence_index, max_intelligence_index;
std::optional<double> min_coding_index, max_coding_index;
std::optional<double> min_agentic_index, max_agentic_index;
std::optional<double> min_tool_success_rate, max_tool_success_rate; // 0..1
void validate() const;
Query to_query() const; // validates
static bool valid_sort(const std::string& s);
static bool valid_category(const std::string& s);
static bool valid_output_modality(const std::string& s);
static bool valid_input_modality(const std::string& s);
};
struct ModelInfo {
std::string id;
std::string canonical_slug;
std::string name;
std::string description;
long created = 0;
long context_length = 0;
std::string modality; // "text+image->text"
std::vector<std::string> input_modalities;
std::vector<std::string> output_modalities;
std::string tokenizer;
std::vector<std::string> supported_parameters;
Json pricing; // values are decimal strings (USD per token) and sometimes numbers
Json top_provider;
Json default_parameters;
Json reasoning; // {mandatory, default_enabled, supported_efforts, default_effort}
std::string expiration_date;
std::string alias_target;
Json raw;
static ModelInfo parse(const Json& doc);
bool supports(const std::string& parameter) const;
bool is_free() const; // id ends with ":free" or both prompt and completion price are zero
// USD per million tokens for a pricing key ("prompt", "completion", ...), if present.
std::optional<double> price_per_million(const std::string& key) const;
};
struct ModelPage {
std::vector<ModelInfo> models;
long total_count = 0;
std::string next; // link to the next page, "" on the last
};
struct ModelList {
std::vector<ModelInfo> models;
long total_count = 0;
bool truncated = false; // stopped at a cap before reading everything
std::size_t pages = 0;
};
struct EndpointInfo {
std::string name;
std::string model_id;
std::string model_name;
std::string provider_name;
std::string tag; // routing slug such as "azure" or "mancer/fp8"
std::string quantization; // int4 ... fp32 or unknown
std::string service_tier; // "", flex, priority, ultrafast
long context_length = 0;
std::optional<long> max_completion_tokens;
std::optional<long> max_prompt_tokens;
int status = 0; // 0 ok; negative values are degraded states
std::optional<double> uptime_last_5m, uptime_last_30m, uptime_last_1d; // percent
Json latency_last_30m; // {p50,p75,p90,p99} seconds, or null
Json throughput_last_30m; // {p50,p75,p90,p99} tokens/second, or null
Json pricing;
Json supports_tool_choice;
std::vector<std::string> supported_parameters;
bool supports_implicit_caching = false;
Json raw;
static EndpointInfo parse(const Json& doc);
};
struct ModelEndpoints {
std::string id;
std::string name;
long created = 0;
std::string description;
Json architecture;
std::vector<EndpointInfo> endpoints;
Json raw;
};
struct ProviderInfo {
std::string name;
std::string slug;
std::string privacy_policy_url;
std::string terms_of_service_url;
std::string status_page_url;
std::string headquarters;
std::vector<std::string> datacenters;
static ProviderInfo parse(const Json& doc);
};
class ModelsService : public Endpoint {
public:
explicit ModelsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kNone) {}
// One request. With neither offset nor limit the API answers the whole list.
ModelPage page(const ModelFilter& filter = {}) const;
// Walks offset/limit pages of Caps::kModelPageSize until total_count is read
// or `max_models` / Caps::kMaxModelPages is hit; then truncated is true.
ModelList list_all(const ModelFilter& filter = {}, std::size_t max_models = 5000) const;
ModelInfo get(const std::string& model_id) const;
ModelEndpoints endpoints(const std::string& model_id) const;
long count(const std::string& output_modalities = "") const;
Json zdr_endpoints() const;
Json embeddings_models(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
Json image_models() const;
Json image_model_endpoints(const std::string& model_id) const;
Json video_models() const;
// Models filtered by the account's provider preferences, privacy settings
// and guardrails. Needs OPENROUTER_API_KEY.
ModelPage user_models(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& output_modalities = "", const std::string& sort = "") const;
};
class ProvidersService : public Endpoint {
public:
explicit ProvidersService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kNone) {}
std::vector<ProviderInfo> list() const;
};
} // namespace molo::openrouter
+93
View File
@@ -0,0 +1,93 @@
// PKCE helpers. SHA-256 and the random source live here because the ai build
// does not link src/security (see SPEC "Gaps").
#include <array>
#include <cstdint>
#include <cstdio>
#include "llm/error.h"
#include "openrouter/caps.h"
#include "openrouter/platform.h"
#include "security/crypto/Base64.h"
namespace molo::openrouter {
namespace {
constexpr std::uint32_t kK[64] = {
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2};
std::uint32_t rotr(std::uint32_t x, int n) { return (x >> n) | (x << (32 - n)); }
std::array<std::uint8_t, 32> sha256(const std::string& msg) {
std::uint32_t h[8] = {0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19};
std::string data = msg;
data.push_back(static_cast<char>(0x80));
while (data.size() % 64 != 56) data.push_back('\0');
std::uint64_t bits = static_cast<std::uint64_t>(msg.size()) * 8;
for (int i = 7; i >= 0; --i) data.push_back(static_cast<char>((bits >> (i * 8)) & 0xff));
for (std::size_t off = 0; off < data.size(); off += 64) {
std::uint32_t w[64];
for (int i = 0; i < 16; ++i) {
const unsigned char* p = reinterpret_cast<const unsigned char*>(data.data()) + off + i * 4;
w[i] = (std::uint32_t(p[0]) << 24) | (std::uint32_t(p[1]) << 16) | (std::uint32_t(p[2]) << 8) | p[3];
}
for (int i = 16; i < 64; ++i) {
std::uint32_t s0 = rotr(w[i - 15], 7) ^ rotr(w[i - 15], 18) ^ (w[i - 15] >> 3);
std::uint32_t s1 = rotr(w[i - 2], 17) ^ rotr(w[i - 2], 19) ^ (w[i - 2] >> 10);
w[i] = w[i - 16] + s0 + w[i - 7] + s1;
}
std::uint32_t a = h[0], b = h[1], c = h[2], d = h[3], e = h[4], f = h[5], g = h[6], hh = h[7];
for (int i = 0; i < 64; ++i) {
std::uint32_t t1 = hh + (rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25)) + ((e & f) ^ (~e & g)) + kK[i] + w[i];
std::uint32_t t2 = (rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22)) + ((a & b) ^ (a & c) ^ (b & c));
hh = g; g = f; f = e; e = d + t1; d = c; c = b; b = a; a = t1 + t2;
}
h[0] += a; h[1] += b; h[2] += c; h[3] += d; h[4] += e; h[5] += f; h[6] += g; h[7] += hh;
}
std::array<std::uint8_t, 32> out{};
for (int i = 0; i < 8; ++i)
for (int j = 0; j < 4; ++j) out[i * 4 + j] = static_cast<std::uint8_t>(h[i] >> (24 - j * 8));
return out;
}
bool verifier_ok(const std::string& v) {
if (v.size() < Caps::kMinPkceVerifier || v.size() > Caps::kMaxPkceVerifier) return false;
for (char c : v)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
c == '.' || c == '_' || c == '~'))
return false;
return true;
}
} // namespace
PkcePair PkcePair::from_verifier(const std::string& verifier) {
if (!verifier_ok(verifier))
throw llm::ApiError::invalid(kVendor, "PKCE verifier must be 43..128 characters of A-Za-z0-9-._~");
PkcePair p;
p.verifier = verifier;
auto d = sha256(verifier);
p.challenge = security::crypto::Base64::encode_url(d.data(), d.size());
return p;
}
PkcePair PkcePair::generate() {
unsigned char raw[48];
std::FILE* f = std::fopen("/dev/urandom", "rb");
if (!f || std::fread(raw, 1, sizeof raw, f) != sizeof raw) {
if (f) std::fclose(f);
throw llm::ApiError(kVendor, 500, "NO_ENTROPY", "no system random source for the PKCE verifier");
}
std::fclose(f);
return from_verifier(security::crypto::Base64::encode_url(raw, sizeof raw)); // 64 chars
}
} // namespace molo::openrouter
+238
View File
@@ -0,0 +1,238 @@
#include "openrouter/platform.h"
#include "clients/body/form.h"
#include "clients/url.h"
#include "llm/error.h"
#include "openrouter/admin.h"
#include "openrouter/caps.h"
#include "openrouter/errors.h"
namespace molo::openrouter {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
void need(const Json& body, std::initializer_list<const char*> keys, const char* what) {
if (!body.is_object()) bad(std::string(what) + " needs a JSON object");
for (const char* k : keys)
if (!body.has(k)) bad(std::string(what) + " needs " + k);
}
std::string pick_after(const std::string& after) {
if (after.size() > 256) bad("cursor too long");
return after;
}
} // namespace
Json EmbeddingsService::create(const Json& body) const {
need(body, {"model", "input"}, "embeddings");
const Json& in = *body.find("input");
if (in.is_string() ? in.as_string().empty()
: (!in.is_array() || in.as_array().empty() || in.as_array().size() > Caps::kMaxEmbeddingInputs))
bad("input must be a non-empty string or an array of at most 2048 entries");
if (const Json* e = body.find("encoding_format"); e && e->is_string() && e->as_string() != "float" && e->as_string() != "base64")
bad("encoding_format must be float or base64");
if (const Json* d = body.find("dimensions"); d && (!d->is_number() || d->as_int() < 1)) bad("dimensions must be >= 1");
Json doc = run("createEmbeddings", {}, {}, body);
throw_if_error_body(doc);
return doc;
}
Json EmbeddingsService::rerank(const std::string& model, const std::string& query,
const std::vector<std::string>& documents, std::optional<int> top_n) const {
if (model.empty() || query.empty()) bad("rerank needs model and query");
if (documents.empty() || documents.size() > Caps::kMaxRerankDocuments) bad("rerank needs 1..1000 documents");
if (top_n && *top_n < 1) bad("top_n must be >= 1");
Json docs = Json::array();
for (const std::string& d : documents) docs.push(Json(d));
Json b = Json::object();
b.set("model", Json(model));
b.set("query", Json(query));
b.set("documents", docs);
if (top_n) b.set("top_n", Json(*top_n));
Json doc = run("createRerank", {}, {}, b);
throw_if_error_body(doc);
return doc;
}
std::vector<std::vector<double>> EmbeddingsService::vectors(const Json& response) {
std::vector<std::vector<double>> out;
const Json* data = response.is_object() ? response.find("data") : nullptr;
if (!data || !data->is_array()) bad("embeddings response has no data array");
for (const Json& item : data->as_array()) {
const Json* e = item.is_object() ? item.find("embedding") : nullptr;
if (!e || !e->is_array()) bad("embedding is not a float array (use encoding_format float)");
std::vector<double> v;
for (const Json& x : e->as_array()) v.push_back(x.is_number() ? x.as_number() : 0);
out.push_back(std::move(v));
}
return out;
}
Json MediaService::generate_image(const Json& body) const {
need(body, {"model", "prompt"}, "image generation");
return run("createImages", {}, {}, body);
}
Json MediaService::submit_video(const Json& body) const {
need(body, {"model", "prompt"}, "video generation");
return run("createVideos", {}, {}, body);
}
Json MediaService::video_status(const std::string& id) const { return run("getVideos", {{"jobId", id}}); }
llm::RawResponse MediaService::video_content(const std::string& id, std::optional<int> index) const {
Query q;
if (index) {
if (*index < 0) bad("index must be >= 0");
q.add("index", *index);
}
return run_raw("listVideosContent", {{"jobId", id}}, q);
}
llm::RawResponse MediaService::speech(const Json& body) const {
need(body, {"model", "input", "voice"}, "speech");
return gateway_->call_raw(CallPlanner::plan("createAudioSpeech", {}, {}, &body));
}
Json MediaService::transcribe(const Json& body) const {
need(body, {"model", "input_audio"}, "transcription");
return run("createAudioTranscriptions", {}, {}, body);
}
Json MediaService::decisions(const Json& body) const {
need(body, {"questions"}, "decisions");
return run("createApiAlphaDecisions", {}, {}, body);
}
Json MediaService::system_one(const Json& body) const {
need(body, {"questions"}, "systemone");
return run("createSystemone", {}, {}, body);
}
Json BatchesService::list(std::optional<int> limit, const std::string& after, const std::string& status) const {
Query q;
if (limit) {
if (*limit < 1 || *limit > 1000) bad("limit must be 1..1000");
q.add("limit", *limit);
}
q.add_if("after", pick_after(after));
q.add_if("status", status);
return run("listBatches", {}, q);
}
Json BatchesService::get(const std::string& id) const { return run("getBatches", {{"id", id}}); }
Json BatchesService::create(const Json& body) const {
need(body, {"endpoint", "model", "requests"}, "batch");
const Json& r = *body.find("requests");
if (!r.is_array() || r.as_array().empty() || r.as_array().size() > Caps::kMaxBatchRequests)
bad("requests must be an array of 1..10000");
return run("createBatches", {}, {}, body);
}
Json BatchesService::remove(const std::string& id) const { return run("deleteBatch", {{"id", id}}); }
Json FilesService::list(std::optional<int> limit, const std::string& cursor) const {
Query q;
if (limit) {
if (*limit < 1 || *limit > 1000) bad("limit must be 1..1000");
q.add("limit", *limit);
}
q.add_if("cursor", pick_after(cursor));
return run("listFiles", {}, q);
}
Json FilesService::metadata(const std::string& id) const { return run("getFileMetadata", {{"file_id", id}}); }
llm::RawResponse FilesService::content(const std::string& id) const {
return run_raw("downloadFileContent", {{"file_id", id}});
}
Json FilesService::upload(const std::string& filename, const std::string& content_type,
const std::string& data) const {
if (filename.empty() || filename.size() > 255 || filename.find_first_of("/\\\"\r\n") != std::string::npos)
bad("bad file name");
if (data.empty()) bad("file is empty");
clients::body::MultipartBody form;
form.add_file("file", filename, content_type.empty() ? "application/octet-stream" : content_type, data);
return gateway_->call_multipart(CallPlanner::plan("uploadFile"), form);
}
Json FilesService::remove(const std::string& id) const { return run("deleteFile", {{"file_id", id}}); }
Json FilesService::container_files(const std::string& c, std::optional<int> limit, const std::string& after) const {
Query q;
if (limit) {
if (*limit < 1 || *limit > 1000) bad("limit must be 1..1000");
q.add("limit", *limit);
}
q.add_if("after", pick_after(after));
return run("listContainerFiles", {{"container_id", c}}, q);
}
Json FilesService::container_file(const std::string& c, const std::string& f) const {
return run("getContainerFile", {{"container_id", c}, {"file_id", f}});
}
llm::RawResponse FilesService::container_file_content(const std::string& c, const std::string& f) const {
return run_raw("downloadContainerFileContent", {{"container_id", c}, {"file_id", f}});
}
Json FilesService::promote_container_file(const std::string& c, const std::string& f) const {
return run("promoteContainerFile", {{"container_id", c}, {"file_id", f}});
}
Json PresetsService::list(std::optional<int> o, std::optional<int> l) const {
return run("listPresets", {}, page_query(o, l));
}
Json PresetsService::get(const std::string& slug) const { return run("getPreset", {{"slug", slug}}); }
Json PresetsService::versions(const std::string& slug, std::optional<int> o, std::optional<int> l) const {
return run("listPresetVersions", {{"slug", slug}}, page_query(o, l));
}
Json PresetsService::version(const std::string& slug, const std::string& v) const {
return run("getPresetVersion", {{"slug", slug}, {"version", v}});
}
Json PresetsService::create_from_chat(const std::string& slug, const Json& b) const {
return run("createPresetsChatCompletions", {{"slug", slug}}, {}, b);
}
Json PresetsService::create_from_messages(const std::string& slug, const Json& b) const {
return run("createPresetsMessages", {{"slug", slug}}, {}, b);
}
Json PresetsService::create_from_responses(const std::string& slug, const Json& b) const {
return run("createPresetsResponses", {{"slug", slug}}, {}, b);
}
Json CatalogService::tools(std::optional<int> o, std::optional<int> l, const std::string& fmt) const {
Query q = page_query(o, l);
q.add_if("api_format", fmt);
return run("listTools", {}, q);
}
Json CatalogService::tool(const std::string& name) const { return run("getTool", {{"name", name}}); }
Json CatalogService::benchmarks(const Query& q) const { return run("getBenchmarks", {}, q); }
Json CatalogService::task_classifications(const std::string& w) const {
return run("getTaskClassifications", {}, Query().add_if("window", w));
}
Json CatalogService::app_rankings(const Query& q) const { return run("getAppRankings", {}, q); }
Json CatalogService::rankings_daily(const Query& q) const { return run("getRankingsDaily", {}, q); }
Json CatalogService::session_cost(const Query& q) const { return run("getSessionCost", {}, q); }
std::string AuthService::authorization_url(const std::string& callback_url, const PkcePair& pkce) {
if (callback_url.rfind("https://", 0) != 0 && callback_url.rfind("http://localhost", 0) != 0 &&
callback_url.rfind("http://127.0.0.1", 0) != 0)
bad("callback_url must be https, or localhost / 127.0.0.1");
if (pkce.challenge.empty()) bad("PKCE challenge is empty");
return "https://openrouter.ai/auth?callback_url=" + clients::Url::encode_component(callback_url) +
"&code_challenge=" + clients::Url::encode_component(pkce.challenge) + "&code_challenge_method=S256";
}
Json AuthService::exchange_code(const std::string& code, const std::string& verifier) const {
if (code.empty() || code.size() > 512) bad("code must be 1..512 characters");
PkcePair::from_verifier(verifier);
Json b = Json::object();
b.set("code", Json(code));
b.set("code_verifier", Json(verifier));
b.set("code_challenge_method", Json("S256"));
return run("exchangeAuthCodeForAPIKey", {}, {}, b);
}
Json AuthService::create_code(const Json& body) const {
need(body, {"callback_url", "code_challenge"}, "auth code");
return run("createAuthKeysCode", {}, {}, body);
}
Json AuthService::jwks() const { return run("listOauthJwks"); }
Json AuthService::exchange_token(const std::string& policy, const std::string& subject_token,
const std::string& subject_token_type) const {
if (policy.empty() || subject_token.empty() || subject_token_type.empty()) bad("token exchange needs all three values");
Json b = Json::object();
b.set("grant_type", Json("urn:ietf:params:oauth:grant-type:token-exchange"));
b.set("federation_policy_id", Json(policy));
b.set("subject_token", Json(subject_token));
b.set("subject_token_type", Json(subject_token_type));
return run("createOauthToken", {}, {}, b);
}
} // namespace molo::openrouter
+118
View File
@@ -0,0 +1,118 @@
#pragma once
// The remaining public routes: embeddings, rerank, images, video, audio,
// batches, files, containers, presets, server tools and public datasets,
// decisions, and the auth / OAuth routes (PKCE code exchange). JSON bodies
// are checked against the call table (calls.h).
#include <optional>
#include <string>
#include <vector>
#include "openrouter/gateway.h"
namespace molo::openrouter {
class EmbeddingsService : public Endpoint {
public:
explicit EmbeddingsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
// model + input (string, or array of up to Caps::kMaxEmbeddingInputs).
Json create(const Json& body) const;
Json rerank(const std::string& model, const std::string& query, const std::vector<std::string>& documents,
std::optional<int> top_n = std::nullopt) const;
// The float vectors of an embeddings response (encoding_format float).
static std::vector<std::vector<double>> vectors(const Json& response);
};
class MediaService : public Endpoint {
public:
explicit MediaService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
Json generate_image(const Json& body) const; // model + prompt
Json submit_video(const Json& body) const; // model + prompt; async job (202)
Json video_status(const std::string& job_id) const;
llm::RawResponse video_content(const std::string& job_id, std::optional<int> index = std::nullopt) const;
llm::RawResponse speech(const Json& body) const; // model + input + voice; audio bytes
Json transcribe(const Json& body) const; // model + input_audio
Json decisions(const Json& body) const; // POST /api/alpha/decisions
Json system_one(const Json& body) const; // POST /systemone
};
class BatchesService : public Endpoint {
public:
explicit BatchesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
Json list(std::optional<int> limit = std::nullopt, const std::string& after = "",
const std::string& status = "") const;
Json get(const std::string& id) const;
// endpoint, model and requests (at most Caps::kMaxBatchRequests). Costs money.
Json create(const Json& body) const;
Json remove(const std::string& id) const;
};
class FilesService : public Endpoint {
public:
explicit FilesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
Json list(std::optional<int> limit = std::nullopt, const std::string& cursor = "") const;
Json metadata(const std::string& file_id) const;
llm::RawResponse content(const std::string& file_id) const;
Json upload(const std::string& filename, const std::string& content_type, const std::string& data) const;
Json remove(const std::string& file_id) const;
Json container_files(const std::string& container_id, std::optional<int> limit = std::nullopt,
const std::string& after = "") const;
Json container_file(const std::string& container_id, const std::string& file_id) const;
llm::RawResponse container_file_content(const std::string& container_id, const std::string& file_id) const;
Json promote_container_file(const std::string& container_id, const std::string& file_id) const;
};
class PresetsService : public Endpoint {
public:
explicit PresetsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
Json get(const std::string& slug) const;
Json versions(const std::string& slug, std::optional<int> offset = std::nullopt,
std::optional<int> limit = std::nullopt) const;
Json version(const std::string& slug, const std::string& version) const;
// These three CREATE a preset from a request body of that skin (they do not run inference).
Json create_from_chat(const std::string& slug, const Json& chat_body) const;
Json create_from_messages(const std::string& slug, const Json& messages_body) const;
Json create_from_responses(const std::string& slug, const Json& responses_body) const;
};
// Server tool registry and public datasets.
class CatalogService : public Endpoint {
public:
explicit CatalogService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
Json tools(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
const std::string& api_format = "") const;
Json tool(const std::string& name) const;
Json benchmarks(const Query& query = {}) const;
Json task_classifications(const std::string& window = "") const;
Json app_rankings(const Query& query = {}) const;
Json rankings_daily(const Query& query = {}) const;
Json session_cost(const Query& query = {}) const;
};
// PKCE (RFC 7636) helpers and the OAuth routes.
struct PkcePair {
std::string verifier; // 43..128 chars of [A-Za-z0-9-._~]
std::string challenge; // base64url(SHA-256(verifier)), no padding
static PkcePair from_verifier(const std::string& verifier);
static PkcePair generate(); // verifier from the system CSPRNG
};
class AuthService : public Endpoint {
public:
explicit AuthService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kNone) {}
// The browser URL the user visits: https://openrouter.ai/auth?callback_url=...&code_challenge=...&code_challenge_method=S256
static std::string authorization_url(const std::string& callback_url, const PkcePair& pkce);
// POST /auth/keys: exchanges the code from the redirect for a user key. The
// response `key` is a secret: never log it.
Json exchange_code(const std::string& code, const std::string& verifier) const;
// POST /auth/keys/code (needs the API key): creates an authorization code.
Json create_code(const Json& body) const;
Json jwks() const;
// POST /oauth/token: workload identity token exchange (form encoded).
Json exchange_token(const std::string& federation_policy_id, const std::string& subject_token,
const std::string& subject_token_type) const;
};
} // namespace molo::openrouter
+151
View File
@@ -0,0 +1,151 @@
#include "openrouter/plugins.h"
#include <algorithm>
#include <set>
#include "llm/error.h"
#include "openrouter/call_plan.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
[[noreturn]] void bad(const std::string& id, const std::string& why) {
throw llm::ApiError::invalid(kVendor, "plugin " + id + ": " + why);
}
bool one_of(const std::string& v, std::initializer_list<const char*> all) {
return std::any_of(all.begin(), all.end(), [&](const char* s) { return v == s; });
}
Json strings(const std::vector<std::string>& v) {
Json a = Json::array();
for (const std::string& s : v) a.push(Json(s));
return a;
}
void check_strings(const std::string& id, const char* what, const std::vector<std::string>& v) {
if (v.size() > Caps::kMaxProviderList) bad(id, std::string(what) + " has too many entries");
for (const std::string& s : v)
if (s.empty() || s.size() > 256) bad(id, std::string(what) + " has an empty or over-long entry");
}
} // namespace
const std::vector<std::string>& Plugin::known_ids() {
// Spec: ChatRequest.plugins oneOf (OpenRouter OpenAPI, 14 entries).
static const std::vector<std::string> kIds = {
"auto-router", "auto-beta-router", "moderation", "web", "web-fetch", "file-parser",
"response-healing", "context-compression", "pareto-router", "fusion",
"switchyard-router", "jev-router", "alignment", "armor"};
return kIds;
}
bool Plugin::is_known(const std::string& id) {
const auto& all = known_ids();
return std::find(all.begin(), all.end(), id) != all.end();
}
Json Plugin::to_json() const {
validate();
Json o = Json::object();
o.set("id", Json(id()));
if (enabled) o.set("enabled", Json(*enabled));
fill(o);
return o;
}
void WebSearchPlugin::validate() const {
if (!engine.empty() && !one_of(engine, {"native", "exa", "parallel", "firecrawl", "perplexity", "auto"}))
bad(id(), "engine must be native exa parallel firecrawl perplexity or auto");
if (!mode.empty() && !one_of(mode, {"instant", "fast", "auto", "deep-lite", "deep", "deep-reasoning", "turbo", "basic", "advanced"}))
bad(id(), "unknown mode");
if (max_results && *max_results < 1) bad(id(), "max_results must be >= 1");
if (max_uses && *max_uses < 1) bad(id(), "max_uses must be >= 1");
check_strings(id(), "include_domains", include_domains);
check_strings(id(), "exclude_domains", exclude_domains);
}
void WebSearchPlugin::fill(Json& o) const {
if (!engine.empty()) o.set("engine", Json(engine));
if (max_results) o.set("max_results", Json(*max_results));
if (max_uses) o.set("max_uses", Json(*max_uses));
if (!search_prompt.empty()) o.set("search_prompt", Json(search_prompt));
if (!mode.empty()) o.set("mode", Json(mode));
if (!include_domains.empty()) o.set("include_domains", strings(include_domains));
if (!exclude_domains.empty()) o.set("exclude_domains", strings(exclude_domains));
}
void FileParserPlugin::validate() const {
if (!pdf_engine.empty() && !one_of(pdf_engine, {"mistral-ocr", "native", "cloudflare-ai", "pdf-text"}))
bad(id(), "pdf engine must be mistral-ocr native cloudflare-ai or pdf-text");
}
void FileParserPlugin::fill(Json& o) const {
if (pdf_engine.empty()) return;
Json pdf = Json::object();
pdf.set("engine", Json(pdf_engine));
o.set("pdf", pdf);
}
void ContextCompressionPlugin::fill(Json& o) const { o.set("engine", Json("middle-out")); }
void AutoRouterPlugin::validate() const {
if (!cost_tier.empty() && !one_of(cost_tier, {"low", "medium", "high", "xhigh", "max"}))
bad(id(), "cost_tier must be low medium high xhigh or max");
check_strings(id(), "allowed_models", allowed_models);
check_strings(id(), "excluded_models", excluded_models);
}
void AutoRouterPlugin::fill(Json& o) const {
if (!allowed_models.empty()) o.set("allowed_models", strings(allowed_models));
if (!excluded_models.empty()) o.set("excluded_models", strings(excluded_models));
if (!cost_tier.empty()) o.set("cost_tier", Json(cost_tier));
if (pin_model) o.set("pin_model", Json(*pin_model));
}
void WebFetchPlugin::validate() const {
if (max_content_tokens && *max_content_tokens < 1) bad(id(), "max_content_tokens must be >= 1");
if (max_uses && *max_uses < 1) bad(id(), "max_uses must be >= 1");
check_strings(id(), "allowed_domains", allowed_domains);
check_strings(id(), "blocked_domains", blocked_domains);
}
void WebFetchPlugin::fill(Json& o) const {
if (!allowed_domains.empty()) o.set("allowed_domains", strings(allowed_domains));
if (!blocked_domains.empty()) o.set("blocked_domains", strings(blocked_domains));
if (max_content_tokens) o.set("max_content_tokens", Json(*max_content_tokens));
if (max_uses) o.set("max_uses", Json(*max_uses));
}
RawPlugin::RawPlugin(std::string id, Json fields) : id_(std::move(id)), fields_(std::move(fields)) {}
void RawPlugin::validate() const {
if (!is_known(id_)) bad(id_, "not a plugin id in the spec");
if (!fields_.is_null() && !fields_.is_object()) bad(id_, "fields must be an object");
if (fields_.is_object() && (fields_.has("id") || fields_.has("enabled")))
bad(id_, "set id/enabled on the plugin, not in fields");
}
void RawPlugin::fill(Json& o) const {
if (!fields_.is_object()) return;
for (const auto& [k, v] : fields_.as_object()) o.set(k, v);
}
Json plugins_to_json(const std::vector<PluginPtr>& plugins) {
if (plugins.size() > Caps::kMaxPlugins) throw llm::ApiError::invalid(kVendor, "too many plugins");
std::set<std::string> seen;
Json a = Json::array();
for (const PluginPtr& p : plugins) {
if (!p) throw llm::ApiError::invalid(kVendor, "null plugin");
if (!seen.insert(p->id()).second)
throw llm::ApiError::invalid(kVendor, "plugin listed twice: " + p->id());
a.push(p->to_json());
}
return a;
}
} // namespace molo::openrouter
+146
View File
@@ -0,0 +1,146 @@
#pragma once
// Request plugins (`plugins` array on chat, responses and messages requests;
// https://openrouter.ai/docs/guides/features/plugins). One class per plugin
// the spec schemas define fields for; the remaining spec plugins
// (pareto-router, fusion, switchyard-router, jev-router, alignment, armor,
// auto-beta-router) go through RawPlugin, which checks the id against the
// spec's list and passes the object through.
//
// Notes from the docs: the `transforms` request parameter is gone, the
// ContextCompressionPlugin (middle-out) replaces it; the ":online" model
// variant and the web plugin are deprecated in favour of the
// `openrouter:web_search` server tool (see server_tools.h).
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::openrouter {
class Plugin {
public:
virtual ~Plugin() = default;
virtual std::string id() const = 0;
// Throws llm::ApiError::invalid.
virtual void validate() const {}
// The object as sent: {"id": ..., ...}. Calls validate().
clients::body::Json to_json() const;
// false disables the plugin for this request even when the account enables it.
std::optional<bool> enabled;
static const std::vector<std::string>& known_ids();
static bool is_known(const std::string& id);
protected:
virtual void fill(clients::body::Json& object) const = 0;
};
using PluginPtr = std::shared_ptr<const Plugin>;
// "web": engine native exa parallel firecrawl perplexity auto.
class WebSearchPlugin : public Plugin {
public:
std::string engine;
std::optional<int> max_results;
std::optional<int> max_uses;
std::string search_prompt;
std::string mode; // instant fast auto deep-lite deep deep-reasoning turbo basic advanced
std::vector<std::string> include_domains;
std::vector<std::string> exclude_domains;
std::string id() const override { return "web"; }
void validate() const override;
protected:
void fill(clients::body::Json& object) const override;
};
// "file-parser": pdf engine mistral-ocr native cloudflare-ai pdf-text.
class FileParserPlugin : public Plugin {
public:
std::string pdf_engine;
std::string id() const override { return "file-parser"; }
void validate() const override;
protected:
void fill(clients::body::Json& object) const override;
};
// "response-healing": repairs malformed JSON from structured-output requests.
class ResponseHealingPlugin : public Plugin {
public:
std::string id() const override { return "response-healing"; }
protected:
void fill(clients::body::Json&) const override {}
};
class ModerationPlugin : public Plugin {
public:
std::string id() const override { return "moderation"; }
protected:
void fill(clients::body::Json&) const override {}
};
// "context-compression": middle-out truncation of prompts over the context.
class ContextCompressionPlugin : public Plugin {
public:
std::string id() const override { return "context-compression"; }
protected:
void fill(clients::body::Json& object) const override;
};
// "auto-router": model patterns may use wildcards ("anthropic/*").
class AutoRouterPlugin : public Plugin {
public:
std::vector<std::string> allowed_models;
std::vector<std::string> excluded_models;
std::string cost_tier; // low medium high xhigh max
std::optional<bool> pin_model;
std::string id() const override { return "auto-router"; }
void validate() const override;
protected:
void fill(clients::body::Json& object) const override;
};
// "web-fetch".
class WebFetchPlugin : public Plugin {
public:
std::vector<std::string> allowed_domains;
std::vector<std::string> blocked_domains;
std::optional<int> max_content_tokens;
std::optional<int> max_uses;
std::string id() const override { return "web-fetch"; }
void validate() const override;
protected:
void fill(clients::body::Json& object) const override;
};
// A spec plugin without a typed class: id plus a free-form object.
class RawPlugin : public Plugin {
public:
RawPlugin(std::string id, clients::body::Json fields);
std::string id() const override { return id_; }
void validate() const override;
protected:
void fill(clients::body::Json& object) const override;
private:
std::string id_;
clients::body::Json fields_;
};
// The `plugins` array. Throws when over Caps::kMaxPlugins, on a null entry
// or on a repeated plugin id.
clients::body::Json plugins_to_json(const std::vector<PluginPtr>& plugins);
} // namespace molo::openrouter
+257
View File
@@ -0,0 +1,257 @@
#include "openrouter/provider_preferences.h"
#include <algorithm>
#include <cmath>
#include <cstdlib>
#include "llm/error.h"
#include "openrouter/call_plan.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
[[noreturn]] void bad(const std::string& why) {
throw llm::ApiError::invalid(kVendor, "provider: " + why);
}
bool slug_ok(const std::string& s) {
// Provider slugs look like "openai", "amazon-bedrock", "anthropic/2".
if (s.empty() || s.size() > 64) return false;
for (char c : s)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '-' || c == '_' || c == '.' || c == '/'))
return false;
return true;
}
void check_list(const char* what, const std::vector<std::string>& v) {
if (v.size() > Caps::kMaxProviderList) bad(std::string(what) + " has too many entries");
for (const std::string& s : v)
if (!slug_ok(s)) bad(std::string(what) + " has a malformed provider slug");
}
Json string_array(const std::vector<std::string>& v) {
Json a = Json::array();
for (const std::string& s : v) a.push(Json(s));
return a;
}
bool decimal_ok(const std::string& s) {
if (s.empty() || s.size() > 32) return false;
char* end = nullptr;
double d = std::strtod(s.c_str(), &end);
return end == s.c_str() + s.size() && std::isfinite(d) && d >= 0;
}
void check_percentiles(const char* what, const std::map<std::string, double>& m) {
for (const auto& [k, v] : m) {
if (k != "p50" && k != "p75" && k != "p90" && k != "p99") bad(std::string(what) + " key must be p50 p75 p90 or p99");
if (!(v >= 0) || !std::isfinite(v)) bad(std::string(what) + " must be a non-negative number");
}
}
Json percentile_or_number(const std::optional<double>& one, const std::map<std::string, double>& m) {
if (!m.empty()) {
Json o = Json::object();
for (const auto& [k, v] : m) o.set(k, Json(v));
return o;
}
return Json(*one);
}
} // namespace
bool ProviderPreferences::valid_quantization(const std::string& q) {
static const char* const kAll[] = {"int4", "int8", "fp4", "mxfp4", "nvfp4", "fp6",
"fp8", "mxfp8", "fp16", "bf16", "fp32", "unknown"};
return std::any_of(std::begin(kAll), std::end(kAll), [&](const char* s) { return q == s; });
}
bool ProviderPreferences::valid_sort(const std::string& s) {
return s == "price" || s == "throughput" || s == "latency" || s == "exacto";
}
bool ProviderPreferences::empty() const {
return order.empty() && only.empty() && ignore.empty() && !allow_fallbacks &&
!require_parameters && data_collection.empty() && !zdr && !enforce_distillable_text &&
quantizations.empty() && sort.empty() && max_price.empty() &&
!preferred_max_latency && preferred_max_latency_percentiles.empty() &&
!preferred_min_throughput && preferred_min_throughput_percentiles.empty() &&
!(options.is_object() && !options.as_object().empty());
}
void ProviderPreferences::validate() const {
check_list("order", order);
check_list("only", only);
check_list("ignore", ignore);
if (!data_collection.empty() && data_collection != "allow" && data_collection != "deny")
bad("data_collection must be allow or deny");
if (quantizations.size() > Caps::kMaxQuantizations) bad("too many quantizations");
for (const std::string& q : quantizations)
if (!valid_quantization(q)) bad("unknown quantization: " + q);
if (!sort.empty() && !valid_sort(sort)) bad("sort must be price, throughput, latency or exacto");
if (!sort_partition.empty()) {
if (sort.empty()) bad("sort_partition needs sort");
if (sort_partition != "model" && sort_partition != "none") bad("sort_partition must be model or none");
}
static const char* const kPrice[] = {"prompt", "completion", "image", "audio", "request"};
for (const auto& [k, v] : max_price) {
if (std::none_of(std::begin(kPrice), std::end(kPrice), [&](const char* p) { return k == p; }))
bad("max_price key must be prompt, completion, image, audio or request");
if (!decimal_ok(v)) bad("max_price." + k + " must be a non-negative decimal string");
}
if (preferred_max_latency && !preferred_max_latency_percentiles.empty())
bad("preferred_max_latency: send one number or percentiles, not both");
if (preferred_min_throughput && !preferred_min_throughput_percentiles.empty())
bad("preferred_min_throughput: send one number or percentiles, not both");
if (preferred_max_latency && !(*preferred_max_latency >= 0)) bad("preferred_max_latency must be >= 0");
if (preferred_min_throughput && !(*preferred_min_throughput >= 0)) bad("preferred_min_throughput must be >= 0");
check_percentiles("preferred_max_latency", preferred_max_latency_percentiles);
check_percentiles("preferred_min_throughput", preferred_min_throughput_percentiles);
if (!options.is_null() && !options.is_object()) bad("options must be an object");
}
Json ProviderPreferences::to_json() const {
validate();
Json o = Json::object();
if (!order.empty()) o.set("order", string_array(order));
if (!only.empty()) o.set("only", string_array(only));
if (!ignore.empty()) o.set("ignore", string_array(ignore));
if (allow_fallbacks) o.set("allow_fallbacks", Json(*allow_fallbacks));
if (require_parameters) o.set("require_parameters", Json(*require_parameters));
if (!data_collection.empty()) o.set("data_collection", Json(data_collection));
if (zdr) o.set("zdr", Json(*zdr));
if (enforce_distillable_text) o.set("enforce_distillable_text", Json(*enforce_distillable_text));
if (!quantizations.empty()) o.set("quantizations", string_array(quantizations));
if (!sort.empty()) {
if (sort_partition.empty()) {
o.set("sort", Json(sort));
} else {
Json s = Json::object();
s.set("by", Json(sort));
s.set("partition", Json(sort_partition));
o.set("sort", s);
}
}
if (!max_price.empty()) {
Json p = Json::object();
for (const auto& [k, v] : max_price) p.set(k, Json(v));
o.set("max_price", p);
}
if (preferred_max_latency || !preferred_max_latency_percentiles.empty())
o.set("preferred_max_latency",
percentile_or_number(preferred_max_latency, preferred_max_latency_percentiles));
if (preferred_min_throughput || !preferred_min_throughput_percentiles.empty())
o.set("preferred_min_throughput",
percentile_or_number(preferred_min_throughput, preferred_min_throughput_percentiles));
if (options.is_object() && !options.as_object().empty()) o.set("options", options);
return o;
}
ProviderPreferences ProviderPreferences::from_json(const Json& doc) {
if (!doc.is_object()) bad("must be a JSON object");
ProviderPreferences p;
auto strings = [&](const char* key, std::vector<std::string>& out) {
const Json* v = doc.find(key);
if (!v || v->is_null()) return;
if (!v->is_array()) bad(std::string(key) + " must be an array");
for (const Json& s : v->as_array()) {
if (!s.is_string()) bad(std::string(key) + " must hold strings");
out.push_back(s.as_string());
}
};
auto flag = [&](const char* key, std::optional<bool>& out) {
const Json* v = doc.find(key);
if (!v || v->is_null()) return;
if (!v->is_bool()) bad(std::string(key) + " must be true or false");
out = v->as_bool();
};
auto cutoff = [&](const char* key, std::optional<double>& one, std::map<std::string, double>& many) {
const Json* v = doc.find(key);
if (!v || v->is_null()) return;
if (v->is_number()) {
one = v->as_number();
} else if (v->is_object()) {
for (const auto& [k, n] : v->as_object()) {
if (n.is_null()) continue;
if (!n.is_number()) bad(std::string(key) + " percentiles must be numbers");
many[k] = n.as_number();
}
} else {
bad(std::string(key) + " must be a number or an object");
}
};
strings("order", p.order);
strings("only", p.only);
strings("ignore", p.ignore);
strings("quantizations", p.quantizations);
flag("allow_fallbacks", p.allow_fallbacks);
flag("require_parameters", p.require_parameters);
flag("zdr", p.zdr);
flag("enforce_distillable_text", p.enforce_distillable_text);
if (const Json* v = doc.find("data_collection"); v && v->is_string()) p.data_collection = v->as_string();
if (const Json* v = doc.find("sort"); v && !v->is_null()) {
if (v->is_string()) {
p.sort = v->as_string();
} else if (v->is_object()) {
if (const Json* by = v->find("by"); by && by->is_string()) p.sort = by->as_string();
if (const Json* part = v->find("partition"); part && part->is_string()) p.sort_partition = part->as_string();
} else {
bad("sort must be a string or an object");
}
}
if (const Json* v = doc.find("max_price"); v && v->is_object())
for (const auto& [k, n] : v->as_object()) {
if (!n.is_string()) bad("max_price values must be strings");
p.max_price[k] = n.as_string();
}
cutoff("preferred_max_latency", p.preferred_max_latency, p.preferred_max_latency_percentiles);
cutoff("preferred_min_throughput", p.preferred_min_throughput, p.preferred_min_throughput_percentiles);
if (const Json* v = doc.find("options"); v && v->is_object()) p.options = *v;
p.validate();
return p;
}
ProviderPreferences ProviderPreferences::only_providers(std::vector<std::string> slugs) {
ProviderPreferences p;
p.only = std::move(slugs);
return p;
}
ProviderPreferences ProviderPreferences::prefer(std::vector<std::string> slugs, bool fallbacks) {
ProviderPreferences p;
p.order = std::move(slugs);
p.allow_fallbacks = fallbacks;
return p;
}
ProviderPreferences ProviderPreferences::cheapest() {
ProviderPreferences p;
p.sort = "price";
return p;
}
ProviderPreferences ProviderPreferences::fastest() {
ProviderPreferences p;
p.sort = "throughput";
return p;
}
ProviderPreferences ProviderPreferences::lowest_latency() {
ProviderPreferences p;
p.sort = "latency";
return p;
}
ProviderPreferences ProviderPreferences::no_data_collection(bool zdr_only) {
ProviderPreferences p;
p.data_collection = "deny";
if (zdr_only) p.zdr = true;
return p;
}
} // namespace molo::openrouter
+61
View File
@@ -0,0 +1,61 @@
#pragma once
// ProviderPreferences: the `provider` object of a request (provider routing,
// https://openrouter.ai/docs/guides/routing/provider-selection). Field names
// and enums follow the spec schema ProviderPreferences. Validation is local
// and loud; to_json() emits only what was set.
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::openrouter {
class ProviderPreferences {
public:
// Provider slugs, in the order to try them. Empty: not sent.
std::vector<std::string> order;
std::vector<std::string> only; // allow list
std::vector<std::string> ignore; // deny list
std::optional<bool> allow_fallbacks;
std::optional<bool> require_parameters;
std::string data_collection; // "", "allow", "deny"
std::optional<bool> zdr; // zero data retention endpoints only
std::optional<bool> enforce_distillable_text;
std::vector<std::string> quantizations; // int4 int8 fp4 mxfp4 nvfp4 fp6 fp8 mxfp8 fp16 bf16 fp32 unknown
std::string sort; // "", price, throughput, latency, exacto
std::string sort_partition; // "", model, none (only with sort; sends {by, partition})
// USD per million tokens (prompt, completion), per image, audio unit,
// request. Values are decimal strings as the spec types them.
std::map<std::string, std::string> max_price;
// Latency ceilings in seconds / throughput floors in tokens per second:
// either one number (applies to p50) or percentile keys p50 p75 p90 p99.
std::optional<double> preferred_max_latency;
std::map<std::string, double> preferred_max_latency_percentiles;
std::optional<double> preferred_min_throughput;
std::map<std::string, double> preferred_min_throughput_percentiles;
// Per-provider passthrough options: provider slug -> object.
clients::body::Json options; // null or object
static bool valid_quantization(const std::string& q);
static bool valid_sort(const std::string& s);
bool empty() const;
// Throws llm::ApiError::invalid.
void validate() const;
clients::body::Json to_json() const;
static ProviderPreferences from_json(const clients::body::Json& doc);
// Convenience builders for the common intents.
static ProviderPreferences only_providers(std::vector<std::string> slugs);
static ProviderPreferences prefer(std::vector<std::string> slugs, bool allow_fallbacks = true);
static ProviderPreferences cheapest();
static ProviderPreferences fastest(); // sort by throughput
static ProviderPreferences lowest_latency();
static ProviderPreferences no_data_collection(bool zdr_only = false);
};
} // namespace molo::openrouter
+73
View File
@@ -0,0 +1,73 @@
#include "openrouter/reasoning.h"
#include <algorithm>
#include "llm/error.h"
#include "openrouter/call_plan.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
[[noreturn]] void bad(const std::string& why) {
throw llm::ApiError::invalid(kVendor, "reasoning: " + why);
}
} // namespace
bool ReasoningConfig::valid_effort(const std::string& e) {
static const char* const kAll[] = {"max", "xhigh", "high", "medium", "low", "minimal", "none"};
return std::any_of(std::begin(kAll), std::end(kAll), [&](const char* s) { return e == s; });
}
bool ReasoningConfig::empty() const {
return effort.empty() && summary.empty() && !max_tokens && !enabled && !exclude &&
mode.empty() && context.empty();
}
void ReasoningConfig::validate() const {
if (!effort.empty() && !valid_effort(effort))
bad("effort must be one of max xhigh high medium low minimal none");
if (!summary.empty() && summary != "auto" && summary != "concise" && summary != "detailed")
bad("summary must be auto, concise or detailed");
if (max_tokens && *max_tokens < 1) bad("max_tokens must be >= 1");
if (!effort.empty() && max_tokens) bad("send effort or max_tokens, not both");
if (!mode.empty() && mode != "standard" && mode != "pro") bad("mode must be standard or pro");
if (!context.empty() && context != "auto" && context != "all_turns" && context != "current_turn")
bad("context must be auto, all_turns or current_turn");
}
Json ReasoningConfig::to_json() const {
validate();
Json o = Json::object();
if (!effort.empty()) o.set("effort", Json(effort));
if (!summary.empty()) o.set("summary", Json(summary));
if (max_tokens) o.set("max_tokens", Json(*max_tokens));
if (enabled) o.set("enabled", Json(*enabled));
if (exclude) o.set("exclude", Json(*exclude));
if (!mode.empty()) o.set("mode", Json(mode));
if (!context.empty()) o.set("context", Json(context));
return o;
}
ReasoningConfig ReasoningConfig::with_effort(const std::string& effort) {
ReasoningConfig r;
r.effort = effort;
r.validate();
return r;
}
ReasoningConfig ReasoningConfig::with_budget(int max_tokens) {
ReasoningConfig r;
r.max_tokens = max_tokens;
r.validate();
return r;
}
ReasoningConfig ReasoningConfig::hidden(const std::string& effort) {
ReasoningConfig r = with_effort(effort);
r.exclude = true;
return r;
}
} // namespace molo::openrouter
+38
View File
@@ -0,0 +1,38 @@
#pragma once
// ReasoningConfig: the unified `reasoning` request parameter
// (https://openrouter.ai/docs/guides/best-practices/reasoning-tokens).
// effort and max_tokens are mutually exclusive on the wire (the docs say the
// unified parameter accepts either, not both). `reasoning_effort` (the flat
// shorthand) is a separate request field and cannot be combined with this
// object; ChatParams enforces that.
#include <optional>
#include <string>
#include "clients/body/json.h"
namespace molo::openrouter {
struct ReasoningConfig {
std::string effort; // "", max xhigh high medium low minimal none
std::string summary; // "", auto concise detailed
std::optional<int> max_tokens; // reasoning token budget
std::optional<bool> enabled;
std::optional<bool> exclude; // reason, but do not return the text (still billed)
std::string mode; // "", standard, pro (OpenAI GPT-5.6 and newer)
std::string context; // "", auto, all_turns, current_turn
static bool valid_effort(const std::string& e);
bool empty() const;
// Throws llm::ApiError::invalid.
void validate() const;
clients::body::Json to_json() const;
static ReasoningConfig with_effort(const std::string& effort);
static ReasoningConfig with_budget(int max_tokens);
static ReasoningConfig hidden(const std::string& effort);
};
} // namespace molo::openrouter
+68
View File
@@ -0,0 +1,68 @@
#include "openrouter/server_tools.h"
#include "llm/error.h"
#include "openrouter/call_plan.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, "tool: " + why); }
} // namespace
Json ServerTools::function(const std::string& name, const std::string& description,
const Json& parameters, bool strict) {
if (name.empty() || name.size() > 64) bad("function name must be 1..64 characters");
for (char c : name)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '_' || c == '-'))
bad("function name may only use letters, digits, _ and -");
if (!parameters.is_object()) bad("parameters must be a JSON Schema object");
Json fn = Json::object();
fn.set("name", Json(name));
if (!description.empty()) fn.set("description", Json(description));
fn.set("parameters", parameters);
if (strict) fn.set("strict", Json(true));
Json t = Json::object();
t.set("type", Json("function"));
t.set("function", fn);
return t;
}
Json ServerTools::function(const llm::ToolSpec& spec) {
return function(spec.name, spec.description, spec.parameters);
}
Json ServerTools::server(const std::string& type, const Json& parameters) {
static const std::string kPrefix = "openrouter:";
if (type.rfind(kPrefix, 0) != 0 || type.size() == kPrefix.size() || type.size() > 96)
bad("server tool type must look like openrouter:<name>");
for (std::size_t i = kPrefix.size(); i < type.size(); ++i) {
char c = type[i];
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '_' || c == ':'))
bad("server tool type may only use a-z 0-9 _ :");
}
Json t = Json::object();
t.set("type", Json(type));
if (!parameters.is_null()) {
if (!parameters.is_object()) bad("parameters must be an object");
t.set("parameters", parameters);
}
return t;
}
Json ServerTools::web_search(const Json& p) { return server("openrouter:web_search", p); }
Json ServerTools::web_fetch(const Json& p) { return server("openrouter:web_fetch", p); }
Json ServerTools::datetime(const Json& p) { return server("openrouter:datetime", p); }
const std::vector<std::string>& ServerTools::known_types() {
static const std::vector<std::string> kTypes = {
"openrouter:advisor", "openrouter:bash", "openrouter:datetime",
"openrouter:files", "openrouter:fusion", "openrouter:image_generation",
"openrouter:experimental__search_models", "openrouter:subagent",
"openrouter:web_fetch", "openrouter:web_search"};
return kTypes;
}
} // namespace molo::openrouter
+39
View File
@@ -0,0 +1,39 @@
#pragma once
// Builders for entries of the request `tools` array: ordinary function tools
// and OpenRouter server tools ("openrouter:*", run by OpenRouter itself, see
// https://openrouter.ai/docs/guides/features/server-tools). The tool type
// names come from the spec schema ChatFunctionTool; a generic envelope
// ChatDynamicServerTool accepts any registered `openrouter:*` type, so
// server() takes any well-formed name.
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
namespace molo::openrouter {
class ServerTools {
public:
// {"type":"function","function":{"name","description","parameters"[,"strict"]}}
static clients::body::Json function(const std::string& name, const std::string& description,
const clients::body::Json& parameters,
bool strict = false);
static clients::body::Json function(const llm::ToolSpec& spec);
// {"type":"openrouter:<name>"[,"parameters":{...}]}. `type` must start with
// "openrouter:" and use [a-z0-9_:] after it. `parameters` null: omitted.
static clients::body::Json server(const std::string& type,
const clients::body::Json& parameters = clients::body::Json());
static clients::body::Json web_search(const clients::body::Json& parameters = clients::body::Json());
static clients::body::Json web_fetch(const clients::body::Json& parameters = clients::body::Json());
static clients::body::Json datetime(const clients::body::Json& parameters = clients::body::Json());
// Server tool types named by the spec (not exhaustive: see server()).
static const std::vector<std::string>& known_types();
};
} // namespace molo::openrouter
+148
View File
@@ -0,0 +1,148 @@
#include "openrouter/stream_summary.h"
#include "llm/error.h"
#include "openrouter/call_plan.h"
#include "openrouter/caps.h"
namespace molo::openrouter {
using clients::body::Json;
namespace {
std::string str(const Json* o, const char* k) {
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
return v && v->is_string() ? v->as_string() : "";
}
int num(const Json* o, const char* k) {
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
return v && v->is_number() ? static_cast<int>(v->as_int()) : 0;
}
std::string type_of(const llm::SseEvent& ev) {
std::string t = str(&ev.data, "type");
return t.empty() ? ev.event : t;
}
void count(StreamSummary& s, const std::string& type) {
if (++s.events > Caps::kMaxStreamEvents)
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "stream over the event cap");
if (s.event_counts.size() < 256 || s.event_counts.count(type)) ++s.event_counts[type];
}
void append(std::string& dst, const std::string& add) {
if (dst.size() + add.size() > Caps::kMaxStreamTextBytes)
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "stream text over the byte cap");
dst += add;
}
} // namespace
std::string ResponsesStreamParser::feed(const llm::SseEvent& ev) {
std::string type = type_of(ev);
count(summary_, type);
const Json& d = ev.data;
if (type == "response.output_text.delta") {
std::string delta = str(&d, "delta");
append(summary_.text, delta);
return delta;
}
if (type == "response.reasoning_text.delta" || type == "response.reasoning_summary_text.delta") {
append(summary_.reasoning, str(&d, "delta"));
return "";
}
if (type == "response.created" || type == "response.in_progress" ||
type == "response.completed" || type == "response.incomplete" || type == "response.failed") {
const Json* r = d.is_object() ? d.find("response") : nullptr;
if (r && r->is_object()) {
if (summary_.id.empty()) summary_.id = str(r, "id");
if (summary_.model.empty()) summary_.model = str(r, "model");
std::string status = str(r, "status");
if (!status.empty() && type != "response.created" && type != "response.in_progress")
summary_.status = status;
if (const Json* u = r->find("usage"); u && u->is_object()) {
llm::Usage usage;
usage.input_tokens = num(u, "input_tokens");
usage.output_tokens = num(u, "output_tokens");
usage.total_tokens = num(u, "total_tokens");
usage.cached_tokens = num(u->find("input_tokens_details"), "cached_tokens");
usage.reasoning_tokens = num(u->find("output_tokens_details"), "reasoning_tokens");
summary_.usage = usage;
}
if (type == "response.failed" && !summary_.error) {
ErrorBody e;
if (const Json* er = r->find("error"); er && er->is_object()) {
e.message = str(er, "message");
e.provider_code = str(er, "code");
}
e.error_type = str(r, "error_type"); // top level on the Responses skin
e.code = status_for_error_type(e.error_type);
summary_.error = e;
}
}
return "";
}
if (type == "error" && !summary_.error) {
ErrorBody e;
e.message = str(&d, "message");
e.provider_code = str(&d, "code");
e.error_type = str(&d, "error_type");
e.code = status_for_error_type(e.error_type);
summary_.error = e;
}
return "";
}
void ResponsesStreamParser::throw_if_errored() const {
if (summary_.error) throw summary_.error->to_api_error();
}
std::string MessagesStreamParser::feed(const llm::SseEvent& ev) {
std::string type = type_of(ev);
count(summary_, type);
const Json& d = ev.data;
if (type == "message_start") {
const Json* m = d.is_object() ? d.find("message") : nullptr;
if (m) {
summary_.id = str(m, "id");
summary_.model = str(m, "model");
}
} else if (type == "content_block_delta") {
const Json* delta = d.is_object() ? d.find("delta") : nullptr;
std::string dt = str(delta, "type");
if (dt == "text_delta") {
std::string t = str(delta, "text");
append(summary_.text, t);
return t;
}
if (dt == "thinking_delta") append(summary_.reasoning, str(delta, "thinking"));
} else if (type == "message_delta") {
const Json* delta = d.is_object() ? d.find("delta") : nullptr;
std::string sr = str(delta, "stop_reason");
if (!sr.empty()) summary_.status = sr;
if (const Json* u = d.is_object() ? d.find("usage") : nullptr; u && u->is_object()) {
llm::Usage usage = summary_.usage.value_or(llm::Usage{});
if (int in = num(u, "input_tokens")) usage.input_tokens = in;
if (int out = num(u, "output_tokens")) usage.output_tokens = out;
if (int cr = num(u, "cache_read_input_tokens")) usage.cached_tokens = cr;
usage.total_tokens = usage.input_tokens + usage.output_tokens;
summary_.usage = usage;
}
} else if (type == "error" && !summary_.error) {
const Json* er = d.is_object() ? d.find("error") : nullptr;
ErrorBody e;
e.message = str(er, "message");
e.error_type = str(er, "error_type");
e.provider_code = str(er, "type");
e.code = status_for_error_type(e.error_type);
summary_.error = e;
}
return "";
}
void MessagesStreamParser::throw_if_errored() const {
if (summary_.error) throw summary_.error->to_api_error();
}
} // namespace molo::openrouter
+54
View File
@@ -0,0 +1,54 @@
#pragma once
// Stream parsers for the two non-chat inference skins: the Responses API
// (POST /responses, "beta") and the Anthropic-compatible Messages API
// (POST /messages). Event type names come from the spec schemas
// StreamEvents and MessagesStreamEvents. Each parser folds the events into a
// StreamSummary (text, reasoning, usage, terminal status or error) without
// touching the network.
#include <map>
#include <optional>
#include <string>
#include "clients/body/json.h"
#include "llm/chat_types.h"
#include "llm/sse.h"
#include "openrouter/errors.h"
namespace molo::openrouter {
struct StreamSummary {
std::string id;
std::string model;
std::string text;
std::string reasoning;
std::string status; // responses: completed incomplete failed; messages: stop_reason
std::optional<llm::Usage> usage;
std::optional<ErrorBody> error;
std::map<std::string, int> event_counts; // by event type
std::size_t events = 0;
};
class ResponsesStreamParser {
public:
// Returns the text delta this event carried ("" when none).
std::string feed(const llm::SseEvent& event);
const StreamSummary& summary() const { return summary_; }
void throw_if_errored() const;
private:
StreamSummary summary_;
};
class MessagesStreamParser {
public:
std::string feed(const llm::SseEvent& event);
const StreamSummary& summary() const { return summary_; }
void throw_if_errored() const;
private:
StreamSummary summary_;
};
} // namespace molo::openrouter
+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env python3
"""Checks calls.h (the module's one call table) against OpenRouter's OpenAPI
document. Python 3 stdlib only, no credentials.
python3 -I src/openrouter/tools/check_spec.py [--spec FILE] [--probe]
For every row it requires: the HTTP method and path exist in the spec, the
operationId matches, the declared key kind agrees with the operation's
responses (a public row must have no 401), every body field is a top level
property of the request schema (allOf followed), every "?name" is a query
parameter. Coverage: every spec operation must be a row, or listed in EXCLUDED
with a reason. Rows whose operationId starts with "doc:" are documented-only
routes absent from the spec; with --probe each one is POSTed with an empty body
and must answer 401 or 400, never 404 (no credential is sent).
Exit 0: all rows verified. Exit 1: at least one mismatch (printed).
"""
import json
import urllib.error
import os
import re
import sys
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
TABLE = os.path.join(HERE, "..", "calls.h")
URL = "https://openrouter.ai/openapi.json"
BASE = "https://openrouter.ai/api/v1"
ROW = re.compile(
r'^\s*\{"([^"]+)",\s*"([A-Z]+)",\s*"([^"]+)",\s*"([^"]+)",\s*"([^"]+)",\s*"([^"]+)",\s*"([^"]*)"\},?\s*$')
# operationId -> reason it is deliberately not wrapped (none today).
EXCLUDED = {}
def load(spec):
if spec:
return json.load(open(spec))
req = urllib.request.Request(URL, headers={"User-Agent": "molo-check-spec/1"})
with urllib.request.urlopen(req, timeout=120) as r:
return json.loads(r.read())
def resolve(doc, node):
seen = 0
while isinstance(node, dict) and "$ref" in node and seen < 20:
cur = doc
for p in node["$ref"].lstrip("#/").split("/"):
cur = cur[p]
node = cur
seen += 1
return node
def props(doc, schema):
schema = resolve(doc, schema)
if not isinstance(schema, dict):
return {}
out = dict(schema.get("properties", {}))
for sub in schema.get("allOf", []):
out.update(props(doc, sub))
return out
def main():
spec = None
if "--spec" in sys.argv:
spec = sys.argv[sys.argv.index("--spec") + 1]
doc = load(spec)
rows = [m.groups() for m in map(ROW.match, open(TABLE)) if m]
if not rows:
print("no rows parsed from calls.h")
return 1
bad = []
covered = set()
ops = {}
for path, item in doc["paths"].items():
for method, op in item.items():
if isinstance(op, dict) and method in ("get", "post", "put", "patch", "delete"):
ops[(method.upper(), path)] = op
names = set()
doc_rows = []
for name, http, path, opid, key, fmt, fields in rows:
if name in names:
bad.append("duplicate row name %s" % name)
names.add(name)
if opid.startswith("doc:"):
doc_rows.append((name, http, path))
continue
op = ops.get((http, path))
if op is None:
bad.append("%s: %s %s not in spec" % (name, http, path))
continue
covered.add((http, path))
if op.get("operationId") != opid:
bad.append("%s: operationId %s, table says %s" % (name, op.get("operationId"), opid))
has401 = "401" in op["responses"]
if key == "none" and has401:
bad.append("%s: table says public but the spec lists 401" % name)
if key != "none" and not has401:
bad.append("%s: table says %s but the spec lists no 401" % (name, key))
params = []
for p in op.get("parameters", []):
params.append(resolve(doc, p))
qnames = {p["name"] for p in params if p["in"] == "query"}
body_props = {}
rb = op.get("requestBody")
if rb:
rb = resolve(doc, rb)
content = rb.get("content", {})
ct = next(iter(content), None)
if ct:
body_props = props(doc, content[ct].get("schema"))
if "multipart" in ct and fmt != "multipart":
bad.append("%s: multipart body but fmt %s" % (name, fmt))
if "x-www-form" in ct and fmt != "form":
bad.append("%s: form body but fmt %s" % (name, fmt))
for f in fields.split():
if f == "-":
continue
if f.startswith("?"):
if f[1:] not in qnames:
bad.append("%s: query parameter %s absent" % (name, f[1:]))
elif f not in body_props:
bad.append("%s: body field %s absent" % (name, f))
# Reverse: every spec query parameter and body property is declared.
declared = set(fields.split())
for q in qnames:
if "?" + q not in declared:
bad.append("%s: spec query parameter %s not in the table" % (name, q))
for b in body_props:
if b not in declared:
bad.append("%s: spec body field %s not in the table" % (name, b))
for key, op in ops.items():
if key not in covered and op.get("operationId") not in EXCLUDED:
bad.append("unwrapped spec operation %s %s (%s)" % (key[0], key[1], op.get("operationId")))
if "--probe" in sys.argv:
for name, http, path in doc_rows:
req = urllib.request.Request(BASE + path, data=b"{}", method=http,
headers={"Content-Type": "application/json", "User-Agent": "molo-check-spec/1"})
try:
urllib.request.urlopen(req, timeout=60)
status = 200
except urllib.error.HTTPError as e:
status = e.code
print("probe %s %s -> %d" % (http, path, status))
if status == 404:
bad.append("%s: documented-only route answers 404" % name)
for b in bad:
print("FAIL " + b)
print("%d rows checked (%d documented-only), %d spec operations, %d problems"
% (len(rows), len(doc_rows), len(ops), len(bad)))
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main())