Merge openrouter package
This commit is contained in:
@@ -0,0 +1,147 @@
|
||||
# OpenRouter (src/openrouter)
|
||||
|
||||
Namespace `molo::openrouter`. Whole public OpenRouter API
|
||||
(`https://openrouter.ai/api/v1`) over the shared `llm::ApiClient` foundation.
|
||||
Status 2026-10-24: implemented and locally tested; no OpenRouter key exists in
|
||||
this environment, so only unauthenticated routes were verified live.
|
||||
|
||||
## Scope
|
||||
|
||||
Chat completions (streaming incl. SSE heartbeats and mid-stream errors, tools,
|
||||
structured outputs, reasoning, usage and cost), legacy completions, the
|
||||
Responses API (beta), the Anthropic-compatible messages endpoint, routing
|
||||
(`provider` preferences, `models` fallbacks, `route`, model variants, plugins,
|
||||
server tools), models (list with every filter, one model, per-model endpoints,
|
||||
count, user models, ZDR endpoints, embedding / image / video model lists),
|
||||
providers, generation stats / content / feedback, key info, credits, activity
|
||||
and analytics, API key management, embeddings, rerank, images, video, audio
|
||||
(speech, transcription), batches, files and containers, presets, server tool
|
||||
registry and public datasets, guardrails, BYOK, workspaces and budgets,
|
||||
observability destinations, private endpoints, external users, organization
|
||||
settings, SCIM, vault secrets, OAuth (PKCE code exchange, authorization code
|
||||
creation, JWKS, workload token exchange), attribution headers, error semantics,
|
||||
`OpenrouterChatModel : llm::ChatModel`.
|
||||
|
||||
## Module map (one class per concern)
|
||||
|
||||
| File | Class | Concern |
|
||||
|---|---|---|
|
||||
| `calls.h` | `kCalls` | the ONE call table: 140 rows (139 spec operations + `POST /completions`) |
|
||||
| `call_plan` | `CallPlanner`, `Query`, `PlannedCall` | table-checked request planning; no network |
|
||||
| `gateway` | `Gateway`, `AnonymousClient`, `Attribution`, `Endpoint` | two keyed `ApiClient`s + keyless client, attribution headers |
|
||||
| `errors` | `ErrorBody`, `ErrorInfo` | envelope, typed `error_type`, 200-with-error shapes |
|
||||
| `model_ref` | `ModelRef` | `author/slug` + catalog (`:free :batch :thinking :extended`) and routing (`:nitro :floor :exacto :online`) variants |
|
||||
| `provider_preferences` | `ProviderPreferences` | order, only, ignore, allow_fallbacks, require_parameters, data_collection, zdr, quantizations, sort(+partition), max_price, preferred_max_latency / min_throughput, options |
|
||||
| `reasoning` | `ReasoningConfig` | `reasoning` object |
|
||||
| `plugins` | `Plugin` family | web, file-parser, response-healing, moderation, context-compression, auto-router, web-fetch, `RawPlugin` for the rest |
|
||||
| `server_tools` | `ServerTools` | function tools and `openrouter:*` server tools |
|
||||
| `chat_params` | `ChatParams`, `ResponseFormats` | validated `/chat/completions` body |
|
||||
| `chat_result`, `chat_stream` | `ChatResult`, `ChatStreamParser` | response and SSE folding |
|
||||
| `chat_service` | `ChatService` | chat + legacy completions, blocking and streaming |
|
||||
| `chat_model` | `OpenrouterChatModel` | neutral `ChatModel` |
|
||||
| `inference_services`, `stream_summary` | `ResponsesService`, `MessagesService`, stream parsers | Responses (beta) and Messages skins |
|
||||
| `models` | `ModelsService`, `ProvidersService`, `ModelFilter` | catalog |
|
||||
| `account` | `AccountService`, `GenerationService`, `KeysService` | key, credits, activity, generation stats, key management |
|
||||
| `admin` | guardrails, BYOK, workspaces, observability, private endpoints, directory, vault | management surface |
|
||||
| `platform` | embeddings, media, batches, files, presets, catalog, auth | the rest |
|
||||
| `pkce.cpp` | `PkcePair` | RFC 7636 S256 |
|
||||
|
||||
## Credentials
|
||||
|
||||
Two separate keys, two `ApiClient`s, never a fallback between them:
|
||||
`OPENROUTER_API_KEY` (inference and reads) and `OPENROUTER_MANAGEMENT_KEY`
|
||||
(rows with key `mgmt`). Public rows (key `none`: models, providers, counts,
|
||||
jwks, ...) go through a keyless client. Optional: `OPENROUTER_APP_URL`
|
||||
(`HTTP-Referer`, required for app attribution), `OPENROUTER_APP_TITLE`
|
||||
(`X-OpenRouter-Title` and legacy `X-Title`), `OPENROUTER_APP_CATEGORIES`
|
||||
(`X-OpenRouter-Categories`, at most 2), `OPENROUTER_BASE_URL` (https or
|
||||
loopback http only). Router metadata opt-in is `Attribution::router_metadata`
|
||||
(`X-OpenRouter-Metadata: enabled`).
|
||||
|
||||
## Contracts
|
||||
|
||||
- Every path, query parameter and body field goes through `CallPlanner` and
|
||||
must be declared in `calls.h`; undeclared input is rejected before any I/O.
|
||||
- Ids are `Service::valid_id` checked (author may carry the alias marker `~`);
|
||||
partner text (`{user}`) is 1..512 bytes, URL-encoded, `.` / `..` refused.
|
||||
- Caps live in `caps.h`; list walks (`ModelsService::list_all`) stop at
|
||||
`kMaxModelPages` and report `truncated`.
|
||||
- Destructive calls are separate methods without defaults (`remove`,
|
||||
`delete_*`, `unassign_*`); the demo adds a dry-run gate.
|
||||
- A top-level `error` on a 200 body and a choice with `finish_reason: "error"`
|
||||
are thrown as `llm::ApiError` (type = `error_type` when sent, else the code).
|
||||
- Streaming: SSE comments (`: OPENROUTER PROCESSING`) are dropped by
|
||||
`SseDecoder`; a mid-stream error is thrown after the chunks before it.
|
||||
- `Usage` maps prompt/completion/total, `prompt_tokens_details.cached_tokens`,
|
||||
`completion_tokens_details.reasoning_tokens`; cost (USD) is on `ChatResult`.
|
||||
|
||||
## Sources
|
||||
|
||||
1. `https://openrouter.ai/openapi.json` (OpenAPI 3.1, info version 1.0.0,
|
||||
fetched 2026-10-24): 139 operations, 919 schemas. Primary source.
|
||||
Saved copy used for exploration only; the checker fetches it live.
|
||||
2. `https://openrouter.ai/docs/llms-full.txt` (docs plus generated SDK
|
||||
references, 4.6 MB, 2026-10-24): errors guide (typed error codes, mid-stream
|
||||
errors, 200-with-error), streaming guide (heartbeat comments), model
|
||||
variants, plugins, app attribution, router metadata, reasoning.
|
||||
3. Live unauthenticated probes (below).
|
||||
|
||||
Cross-checks: paths and fields come from (1) and the SDK reference pages in
|
||||
(2) (they are generated from the same document, so they are not independent
|
||||
for field lists); behavior (errors, SSE, variants, headers) comes from (2) and
|
||||
was checked against live responses where a key is not needed.
|
||||
|
||||
## Verified against spec
|
||||
|
||||
`python3 -I src/openrouter/tools/check_spec.py --probe` on 2026-10-24: 140
|
||||
rows checked (1 documented-only), 139 spec operations, 0 problems. The checker
|
||||
verifies method, path, operationId, public-vs-keyed (401 present), body
|
||||
fields (both directions), query parameters (both directions), multipart / form
|
||||
formats, and fails on any spec operation missing from the table.
|
||||
|
||||
## Verified live (2026-10-24, no key)
|
||||
|
||||
- `GET /models` 200, 469 models, `total_count` and `links.next` present;
|
||||
pricing values are decimal strings; ids include `~alias/...` authors and
|
||||
`:free` / `:batch` variants. `list_all` walk read the whole catalog and every
|
||||
id parsed as a `ModelRef` (`MOLO_LIVE_NET=1 ./build/ai_test`).
|
||||
- `GET /providers` 200 (114 providers); `GET /models/count` 200;
|
||||
`GET /models/openai/gpt-4o/endpoints` 200; `GET /model/openai/gpt-4o` 200;
|
||||
`GET /embeddings/models`, `GET /endpoints/zdr` 200 without a key.
|
||||
- `GET /key`, `GET /credits`, `POST /chat/completions`, `POST /messages`,
|
||||
`POST /completions` answer 401 `{"error":{"message":"No cookie auth
|
||||
credentials found","code":401}}` (numeric code); `POST /auth/keys` with `{}`
|
||||
answers 400 (public).
|
||||
|
||||
## Unverified
|
||||
|
||||
- Every authenticated response shape (chat, responses, messages, generation,
|
||||
keys, admin lists) is taken from spec examples and docs, not seen.
|
||||
- `POST /completions` exists (401 live, docs list it under router metadata)
|
||||
but is absent from the OpenAPI document; its field list in `calls.h` is the
|
||||
OpenAI legacy set plus OpenRouter routing fields, inferred.
|
||||
- `reasoning.exclude` is documented but not in the spec schema; kept as sent.
|
||||
- Server tool `type` names beyond the spec's ten are accepted by form only.
|
||||
- The `grant_type` literal for `/oauth/token` (RFC 8693 URN) is the standard
|
||||
value; the spec lists the field without an enum.
|
||||
- `models` fallback count: no vendor limit found, local ceiling 16.
|
||||
- Streaming of `/images` (`stream` field) is not parsed (raw body only).
|
||||
- Rows keyed `mgmt` follow the spec descriptions ("Management key required");
|
||||
vault rows are keyed `api` per their securitySchemes.
|
||||
|
||||
## Gaps
|
||||
|
||||
- `llm::ApiError` carries no `error.metadata`, `Retry-After` or response
|
||||
headers (foundation, `src/llm/error.h`); `describe()` works from status and
|
||||
type only. Mid-stream and 200-with-error shapes keep full metadata via
|
||||
`ErrorBody`.
|
||||
- `SseDecoder` drops comment lines, so heartbeats cannot be counted or
|
||||
surfaced as progress (`src/llm/sse.cpp`).
|
||||
- `ai.mk` does not link `src/security` beyond Base64, so `pkce.cpp` carries
|
||||
its own SHA-256 and reads `/dev/urandom`.
|
||||
- `reasoning_details` cannot round-trip through the neutral `llm::Message`;
|
||||
tool loops with reasoning models need `ChatService` with raw messages.
|
||||
- Files upload takes the whole file in memory (`Caps` upload ceiling of the
|
||||
foundation); no resumable upload.
|
||||
- Typed result classes exist for chat, models, endpoints, providers, key info,
|
||||
credits, generation; other responses are returned as `Json`.
|
||||
@@ -0,0 +1,287 @@
|
||||
#include "openrouter/account.h"
|
||||
|
||||
#include <cmath>
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
|
||||
|
||||
const Json& data_of(const Json& doc) {
|
||||
const Json* d = doc.is_object() ? doc.find("data") : nullptr;
|
||||
if (!d || !d->is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "response has no data object");
|
||||
return *d;
|
||||
}
|
||||
|
||||
std::string str(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
double num(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
return v && v->is_number() ? v->as_number() : 0;
|
||||
}
|
||||
long integer(const Json& o, const char* k) { return static_cast<long>(num(o, k)); }
|
||||
bool flag(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
return v && v->is_bool() && v->as_bool();
|
||||
}
|
||||
std::optional<double> opt_num(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
if (v && v->is_number()) return v->as_number();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
bool is_hex(char c) {
|
||||
return (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F');
|
||||
}
|
||||
|
||||
void check_reset(const std::string& r) {
|
||||
if (!r.empty() && r != "daily" && r != "weekly" && r != "monthly")
|
||||
bad("limit_reset must be daily, weekly or monthly");
|
||||
}
|
||||
|
||||
void check_limit_value(const std::optional<double>& l) {
|
||||
if (l && (!std::isfinite(*l) || *l < 0)) bad("limit must be a non-negative number");
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool is_key_hash(const std::string& s) {
|
||||
if (s.size() != 64) return false;
|
||||
for (char c : s)
|
||||
if (!is_hex(c)) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool is_uuid(const std::string& s) {
|
||||
if (s.size() != 36) return false;
|
||||
for (std::size_t i = 0; i < s.size(); ++i) {
|
||||
if (i == 8 || i == 13 || i == 18 || i == 23) {
|
||||
if (s[i] != '-') return false;
|
||||
} else if (!is_hex(s[i])) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool is_iso_utc_seconds(const std::string& s) {
|
||||
// 2027-12-31T23:59:59Z or 2027-12-31T23:59:59.123Z
|
||||
if (s.size() < 20 || s.back() != 'Z') return false;
|
||||
auto digit = [&](std::size_t i) { return s[i] >= '0' && s[i] <= '9'; };
|
||||
for (std::size_t i : {0, 1, 2, 3, 5, 6, 8, 9, 11, 12, 14, 15, 17, 18})
|
||||
if (!digit(i)) return false;
|
||||
if (s[4] != '-' || s[7] != '-' || s[10] != 'T' || s[13] != ':' || s[16] != ':') return false;
|
||||
if (s.size() == 20) return true;
|
||||
if (s[19] != '.' || s.size() < 22) return false;
|
||||
for (std::size_t i = 20; i + 1 < s.size(); ++i)
|
||||
if (!digit(i)) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
KeyInfo KeyInfo::parse(const Json& doc) {
|
||||
const Json& d = data_of(doc);
|
||||
KeyInfo k;
|
||||
k.raw = d;
|
||||
k.label = str(d, "label");
|
||||
k.limit = opt_num(d, "limit");
|
||||
k.limit_remaining = opt_num(d, "limit_remaining");
|
||||
k.limit_reset = str(d, "limit_reset");
|
||||
k.usage = num(d, "usage");
|
||||
k.usage_daily = num(d, "usage_daily");
|
||||
k.usage_weekly = num(d, "usage_weekly");
|
||||
k.usage_monthly = num(d, "usage_monthly");
|
||||
k.byok_usage = num(d, "byok_usage");
|
||||
k.is_free_tier = flag(d, "is_free_tier");
|
||||
k.is_management_key = flag(d, "is_management_key");
|
||||
k.include_byok_in_limit = flag(d, "include_byok_in_limit");
|
||||
k.expires_at = str(d, "expires_at");
|
||||
k.creator_user_id = str(d, "creator_user_id");
|
||||
k.workspace_id = str(d, "workspace_id");
|
||||
if (const Json* v = d.find("free_model_daily_requests")) k.free_model_daily_requests = *v;
|
||||
if (const Json* v = d.find("effective_budget")) k.effective_budget = *v;
|
||||
if (const Json* v = d.find("budgets")) k.budgets = *v;
|
||||
return k;
|
||||
}
|
||||
|
||||
Credits Credits::parse(const Json& doc) {
|
||||
const Json& d = data_of(doc);
|
||||
Credits c;
|
||||
c.total_credits = num(d, "total_credits");
|
||||
c.total_usage = num(d, "total_usage");
|
||||
return c;
|
||||
}
|
||||
|
||||
void ActivityFilter::validate() const {
|
||||
if (!date.empty()) {
|
||||
bool ok = date.size() == 10 && date[4] == '-' && date[7] == '-';
|
||||
for (std::size_t i = 0; ok && i < date.size(); ++i)
|
||||
if (i != 4 && i != 7 && !(date[i] >= '0' && date[i] <= '9')) ok = false;
|
||||
if (!ok) bad("date must be YYYY-MM-DD");
|
||||
}
|
||||
if (!api_key_hash.empty() && !is_key_hash(api_key_hash)) bad("api_key_hash must be a 64 character hex string");
|
||||
if (user_id.size() > 128) bad("user_id too long");
|
||||
if (!group_by.empty() && group_by != "workspace") bad("group_by must be workspace");
|
||||
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
|
||||
}
|
||||
|
||||
Query ActivityFilter::to_query() const {
|
||||
validate();
|
||||
Query q;
|
||||
q.add_if("date", date);
|
||||
q.add_if("api_key_hash", api_key_hash);
|
||||
q.add_if("user_id", user_id);
|
||||
q.add_if("group_by", group_by);
|
||||
q.add_if("workspace_id", workspace_id);
|
||||
return q;
|
||||
}
|
||||
|
||||
KeyInfo AccountService::key_info() const { return KeyInfo::parse(run("getCurrentKey")); }
|
||||
Credits AccountService::credits() const { return Credits::parse(run("getCredits")); }
|
||||
Json AccountService::activity(const ActivityFilter& f) const { return run("getUserActivity", {}, f.to_query()); }
|
||||
Json AccountService::analytics_meta() const { return run("getAnalyticsMeta"); }
|
||||
Json AccountService::analytics_query(const Json& body) const { return run("queryAnalytics", {}, {}, body); }
|
||||
|
||||
GenerationInfo GenerationInfo::parse(const Json& doc) {
|
||||
const Json& d = data_of(doc);
|
||||
GenerationInfo g;
|
||||
g.raw = d;
|
||||
g.id = str(d, "id");
|
||||
g.model = str(d, "model");
|
||||
g.provider_name = str(d, "provider_name");
|
||||
g.created_at = str(d, "created_at");
|
||||
g.finish_reason = str(d, "finish_reason");
|
||||
g.native_finish_reason = str(d, "native_finish_reason");
|
||||
g.api_type = str(d, "api_type");
|
||||
g.origin = str(d, "origin");
|
||||
g.total_cost = num(d, "total_cost");
|
||||
g.usage = num(d, "usage");
|
||||
g.cache_discount = num(d, "cache_discount");
|
||||
g.latency_ms = num(d, "latency");
|
||||
g.generation_time_ms = num(d, "generation_time");
|
||||
g.tokens_prompt = integer(d, "tokens_prompt");
|
||||
g.tokens_completion = integer(d, "tokens_completion");
|
||||
g.native_tokens_prompt = integer(d, "native_tokens_prompt");
|
||||
g.native_tokens_completion = integer(d, "native_tokens_completion");
|
||||
g.native_tokens_reasoning = integer(d, "native_tokens_reasoning");
|
||||
g.native_tokens_cached = integer(d, "native_tokens_cached");
|
||||
g.streamed = flag(d, "streamed");
|
||||
g.cancelled = flag(d, "cancelled");
|
||||
g.is_byok = flag(d, "is_byok");
|
||||
if (const Json* v = d.find("provider_responses")) g.provider_responses = *v;
|
||||
return g;
|
||||
}
|
||||
|
||||
GenerationInfo GenerationService::get(const std::string& id) const {
|
||||
check_id("generation", id);
|
||||
return GenerationInfo::parse(run("getGeneration", {}, Query().add("id", id)));
|
||||
}
|
||||
|
||||
Json GenerationService::content(const std::string& id) const {
|
||||
check_id("generation", id);
|
||||
return run("listGenerationContent", {}, Query().add("id", id));
|
||||
}
|
||||
|
||||
Json GenerationService::feedback(const std::string& id, const std::string& category,
|
||||
const std::string& comment) const {
|
||||
check_id("generation", id);
|
||||
static const char* const kCats[] = {"latency", "incoherence", "incorrect_response", "formatting",
|
||||
"billing", "api_error", "other"};
|
||||
bool ok = false;
|
||||
for (const char* c : kCats)
|
||||
if (category == c) ok = true;
|
||||
if (!ok) bad("feedback category must be latency incoherence incorrect_response formatting billing api_error or other");
|
||||
if (comment.size() > 1000) bad("feedback comment is at most 1000 characters");
|
||||
Json b = Json::object();
|
||||
b.set("generation_id", Json(id));
|
||||
b.set("category", Json(category));
|
||||
if (!comment.empty()) b.set("comment", Json(comment));
|
||||
return run("submitGenerationFeedback", {}, {}, b);
|
||||
}
|
||||
|
||||
void KeyDraft::validate() const {
|
||||
if (name.empty() || name.size() > 256) bad("key name must be 1..256 characters");
|
||||
check_limit_value(limit);
|
||||
check_reset(limit_reset);
|
||||
if (!expires_at.empty() && !is_iso_utc_seconds(expires_at)) bad("expires_at must be YYYY-MM-DDTHH:MM:SSZ");
|
||||
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
|
||||
if (external_user.size() > 512) bad("external user too long");
|
||||
}
|
||||
|
||||
Json KeyDraft::to_json() const {
|
||||
validate();
|
||||
Json b = Json::object();
|
||||
b.set("name", Json(name));
|
||||
if (limit) b.set("limit", Json(*limit));
|
||||
if (!limit_reset.empty()) b.set("limit_reset", Json(limit_reset));
|
||||
if (include_byok_in_limit) b.set("include_byok_in_limit", Json(*include_byok_in_limit));
|
||||
if (!expires_at.empty()) b.set("expires_at", Json(expires_at));
|
||||
if (!workspace_id.empty()) b.set("workspace_id", Json(workspace_id));
|
||||
if (!creator_user_id.empty()) b.set("creator_user_id", Json(creator_user_id));
|
||||
if (!external_user.empty()) {
|
||||
Json e = Json::object();
|
||||
e.set("user", Json(external_user));
|
||||
b.set("external", e);
|
||||
}
|
||||
return b;
|
||||
}
|
||||
|
||||
void KeyUpdate::validate() const {
|
||||
if (name && (name->empty() || name->size() > 256)) bad("key name must be 1..256 characters");
|
||||
check_limit_value(limit);
|
||||
check_reset(limit_reset);
|
||||
if (clear_limit_reset && !limit_reset.empty()) bad("limit_reset and clear_limit_reset together");
|
||||
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
|
||||
}
|
||||
|
||||
Json KeyUpdate::to_json() const {
|
||||
validate();
|
||||
Json b = Json::object();
|
||||
if (name) b.set("name", Json(*name));
|
||||
if (disabled) b.set("disabled", Json(*disabled));
|
||||
if (limit) b.set("limit", Json(*limit));
|
||||
if (!limit_reset.empty()) b.set("limit_reset", Json(limit_reset));
|
||||
if (clear_limit_reset) b.set("limit_reset", Json());
|
||||
if (include_byok_in_limit) b.set("include_byok_in_limit", Json(*include_byok_in_limit));
|
||||
if (!workspace_id.empty()) b.set("workspace_id", Json(workspace_id));
|
||||
if (b.as_object().empty()) bad("nothing to update");
|
||||
return b;
|
||||
}
|
||||
|
||||
Json KeysService::list(std::optional<int> offset, std::optional<int> limit, bool include_disabled,
|
||||
bool include_expired, const std::string& workspace_id) const {
|
||||
if (offset && *offset < 0) bad("offset must be >= 0");
|
||||
if (limit && (*limit < 1 || *limit > Caps::kMaxKeysLimit)) bad("limit must be 1..100");
|
||||
if (!workspace_id.empty() && !is_uuid(workspace_id)) bad("workspace_id must be a uuid");
|
||||
Query q;
|
||||
if (offset) q.add("offset", *offset);
|
||||
if (limit) q.add("limit", *limit);
|
||||
if (include_disabled) q.add("include_disabled", true);
|
||||
if (include_expired) q.add("include_expired", true);
|
||||
q.add_if("workspace_id", workspace_id);
|
||||
return run("listKeys", {}, q);
|
||||
}
|
||||
|
||||
Json KeysService::get(const std::string& h) const {
|
||||
if (!is_key_hash(h)) bad("key hash must be a 64 character hex string");
|
||||
return run("getKey", {{"hash", h}});
|
||||
}
|
||||
|
||||
Json KeysService::create(const KeyDraft& draft) const { return run("createKeys", {}, {}, draft.to_json()); }
|
||||
|
||||
Json KeysService::update(const std::string& h, const KeyUpdate& u) const {
|
||||
if (!is_key_hash(h)) bad("key hash must be a 64 character hex string");
|
||||
return run("updateKeys", {{"hash", h}}, {}, u.to_json());
|
||||
}
|
||||
|
||||
Json KeysService::remove(const std::string& h) const {
|
||||
if (!is_key_hash(h)) bad("key hash must be a 64 character hex string");
|
||||
return run("deleteKeys", {{"hash", h}});
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,153 @@
|
||||
#pragma once
|
||||
|
||||
// Account and usage: GET /key (the calling key), GET /credits (management
|
||||
// key), GET /activity and the analytics routes (management key), generation
|
||||
// stats (GET /generation?id=, content, feedback), and API key management
|
||||
// (GET/POST /keys, GET/PATCH/DELETE /keys/{hash}; management key only).
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "openrouter/gateway.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct KeyInfo {
|
||||
std::string label;
|
||||
std::optional<double> limit; // USD, null: no limit
|
||||
std::optional<double> limit_remaining;
|
||||
std::string limit_reset; // daily weekly monthly or ""
|
||||
double usage = 0, usage_daily = 0, usage_weekly = 0, usage_monthly = 0; // USD
|
||||
double byok_usage = 0;
|
||||
bool is_free_tier = false;
|
||||
bool is_management_key = false;
|
||||
bool include_byok_in_limit = false;
|
||||
std::string expires_at;
|
||||
std::string creator_user_id;
|
||||
std::string workspace_id;
|
||||
Json free_model_daily_requests; // {limit, remaining, used}
|
||||
Json effective_budget; // the budget that blocks first, or null
|
||||
Json budgets; // every budget layer, enforcement order
|
||||
Json raw;
|
||||
static KeyInfo parse(const Json& doc); // doc is the response; reads `data`
|
||||
};
|
||||
|
||||
struct Credits {
|
||||
double total_credits = 0;
|
||||
double total_usage = 0;
|
||||
double remaining() const { return total_credits - total_usage; }
|
||||
static Credits parse(const Json& doc);
|
||||
};
|
||||
|
||||
// Query for GET /activity (last 30 days, grouped by endpoint).
|
||||
struct ActivityFilter {
|
||||
std::string date; // YYYY-MM-DD, within the last 30 days
|
||||
std::string api_key_hash; // SHA-256 hex
|
||||
std::string user_id; // org member
|
||||
std::string group_by; // "", workspace
|
||||
std::string workspace_id; // uuid
|
||||
void validate() const;
|
||||
Query to_query() const;
|
||||
};
|
||||
|
||||
class AccountService : public Endpoint {
|
||||
public:
|
||||
explicit AccountService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
KeyInfo key_info() const; // OPENROUTER_API_KEY
|
||||
Credits credits() const; // OPENROUTER_MANAGEMENT_KEY
|
||||
Json activity(const ActivityFilter& filter = {}) const; // management key
|
||||
Json analytics_meta() const; // management key
|
||||
Json analytics_query(const Json& body) const; // management key
|
||||
};
|
||||
|
||||
struct GenerationInfo {
|
||||
std::string id;
|
||||
std::string model;
|
||||
std::string provider_name;
|
||||
std::string created_at;
|
||||
std::string finish_reason;
|
||||
std::string native_finish_reason;
|
||||
std::string api_type;
|
||||
std::string origin;
|
||||
double total_cost = 0; // USD
|
||||
double usage = 0;
|
||||
double cache_discount = 0;
|
||||
double latency_ms = 0;
|
||||
double generation_time_ms = 0;
|
||||
long tokens_prompt = 0, tokens_completion = 0;
|
||||
long native_tokens_prompt = 0, native_tokens_completion = 0;
|
||||
long native_tokens_reasoning = 0, native_tokens_cached = 0;
|
||||
bool streamed = false;
|
||||
bool cancelled = false;
|
||||
bool is_byok = false;
|
||||
Json provider_responses;
|
||||
Json raw;
|
||||
static GenerationInfo parse(const Json& doc); // reads `data`
|
||||
};
|
||||
|
||||
class GenerationService : public Endpoint {
|
||||
public:
|
||||
explicit GenerationService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
// id is the "gen-..." id of a completion (ChatResult::id). Stats can lag the
|
||||
// completion by a moment: a 404 right after the call is not a failure.
|
||||
GenerationInfo get(const std::string& generation_id) const;
|
||||
// Stored prompt, completion and error content; needs input/output logging.
|
||||
Json content(const std::string& generation_id) const;
|
||||
// categories: latency incoherence incorrect_response formatting billing api_error other.
|
||||
// Uses the management key (per the spec text).
|
||||
Json feedback(const std::string& generation_id, const std::string& category,
|
||||
const std::string& comment = "") const;
|
||||
};
|
||||
|
||||
// Body of POST /keys.
|
||||
struct KeyDraft {
|
||||
std::string name; // required
|
||||
std::optional<double> limit; // USD
|
||||
std::string limit_reset; // "", daily, weekly, monthly
|
||||
std::optional<bool> include_byok_in_limit;
|
||||
std::string expires_at; // YYYY-MM-DDTHH:MM:SSZ (seconds required)
|
||||
std::string workspace_id; // uuid
|
||||
std::string creator_user_id;
|
||||
std::string external_user; // partner attribution
|
||||
void validate() const;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
// Body of PATCH /keys/{hash}; only the set fields are sent.
|
||||
struct KeyUpdate {
|
||||
std::optional<std::string> name;
|
||||
std::optional<bool> disabled;
|
||||
std::optional<double> limit;
|
||||
std::string limit_reset; // "", daily, weekly, monthly
|
||||
bool clear_limit_reset = false; // send limit_reset:null
|
||||
std::optional<bool> include_byok_in_limit;
|
||||
std::string workspace_id;
|
||||
void validate() const;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
class KeysService : public Endpoint {
|
||||
public:
|
||||
// Keys are managed with OPENROUTER_MANAGEMENT_KEY, never the inference key.
|
||||
explicit KeysService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
// limit 1..100 (vendor), offset >= 0.
|
||||
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
bool include_disabled = false, bool include_expired = false,
|
||||
const std::string& workspace_id = "") const;
|
||||
Json get(const std::string& key_hash) const;
|
||||
// The response carries the new secret `key` exactly once. Callers must not log it.
|
||||
Json create(const KeyDraft& draft) const;
|
||||
Json update(const std::string& key_hash, const KeyUpdate& update) const;
|
||||
// Irreversible. No default; the caller decides.
|
||||
Json remove(const std::string& key_hash) const;
|
||||
};
|
||||
|
||||
// True for a 64-character lowercase or uppercase hex string (SHA-256).
|
||||
bool is_key_hash(const std::string& s);
|
||||
// "YYYY-MM-DDTHH:MM:SS[.fff]Z" shape check (seconds required).
|
||||
bool is_iso_utc_seconds(const std::string& s);
|
||||
// UUID 8-4-4-4-12 hex shape check.
|
||||
bool is_uuid(const std::string& s);
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,332 @@
|
||||
#include "openrouter/admin.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/account.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
|
||||
|
||||
Json string_list_body(const char* key, const std::vector<std::string>& values, bool hashes = false) {
|
||||
if (values.empty()) bad(std::string(key) + " must not be empty");
|
||||
if (values.size() > Caps::kMaxBulkIds) bad(std::string(key) + " has too many entries");
|
||||
Json a = Json::array();
|
||||
for (const std::string& v : values) {
|
||||
if (hashes ? !is_key_hash(v) : (v.empty() || v.size() > 256)) bad(std::string(key) + " has a malformed entry");
|
||||
a.push(Json(v));
|
||||
}
|
||||
Json b = Json::object();
|
||||
b.set(key, a);
|
||||
return b;
|
||||
}
|
||||
|
||||
void check_body(const Json& body, const char* what) {
|
||||
if (!body.is_object() || body.as_object().empty()) bad(std::string(what) + " needs a non-empty JSON object");
|
||||
}
|
||||
|
||||
Json secret_body(const std::string& value, const std::vector<std::string>& hosts) {
|
||||
if (value.empty() || value.size() > 65536) bad("secret value must be 1..65536 characters");
|
||||
if (hosts.empty() || hosts.size() > 100) bad("secret hosts must hold 1..100 hostnames");
|
||||
Json h = Json::array();
|
||||
for (const std::string& host : hosts) {
|
||||
if (host.empty() || host.size() > 254) bad("secret host must be 1..254 characters");
|
||||
for (char c : host)
|
||||
if (c == '/' || c == ':' || c == '*' || c == ' ') bad("secret host must be a bare hostname");
|
||||
h.push(Json(host));
|
||||
}
|
||||
Json b = Json::object();
|
||||
b.set("value", Json(value));
|
||||
b.set("hosts", h);
|
||||
return b;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Query page_query(std::optional<int> offset, std::optional<int> limit, int max_limit) {
|
||||
if (offset && *offset < 0) bad("offset must be >= 0");
|
||||
if (limit && (*limit < 1 || *limit > max_limit)) bad("limit must be 1.." + std::to_string(max_limit));
|
||||
Query q;
|
||||
if (offset) q.add("offset", *offset);
|
||||
if (limit) q.add("limit", *limit);
|
||||
return q;
|
||||
}
|
||||
|
||||
// --- guardrails ------------------------------------------------------------
|
||||
|
||||
Json GuardrailsService::list(std::optional<int> o, std::optional<int> l, const std::string& ws) const {
|
||||
Query q = page_query(o, l);
|
||||
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
|
||||
q.add_if("workspace_id", ws);
|
||||
return run("listGuardrails", {}, q);
|
||||
}
|
||||
Json GuardrailsService::get(const std::string& id) const { return run("getGuardrail", {{"id", id}}); }
|
||||
Json GuardrailsService::create(const Json& body) const {
|
||||
check_body(body, "guardrail");
|
||||
if (!body.has("name")) bad("guardrail needs a name");
|
||||
return run("createGuardrail", {}, {}, body);
|
||||
}
|
||||
Json GuardrailsService::update(const std::string& id, const Json& body) const {
|
||||
check_body(body, "guardrail update");
|
||||
return run("updateGuardrail", {{"id", id}}, {}, body);
|
||||
}
|
||||
Json GuardrailsService::remove(const std::string& id) const { return run("deleteGuardrail", {{"id", id}}); }
|
||||
Json GuardrailsService::key_assignments(const std::string& id, std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listGuardrailKeyAssignments", {{"id", id}}, page_query(o, l));
|
||||
}
|
||||
Json GuardrailsService::assign_keys(const std::string& id, const std::vector<std::string>& h) const {
|
||||
return run("bulkAssignKeysToGuardrail", {{"id", id}}, {}, string_list_body("key_hashes", h, true));
|
||||
}
|
||||
Json GuardrailsService::unassign_keys(const std::string& id, const std::vector<std::string>& h) const {
|
||||
return run("bulkUnassignKeysFromGuardrail", {{"id", id}}, {}, string_list_body("key_hashes", h, true));
|
||||
}
|
||||
Json GuardrailsService::member_assignments(const std::string& id, std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listGuardrailMemberAssignments", {{"id", id}}, page_query(o, l));
|
||||
}
|
||||
Json GuardrailsService::assign_members(const std::string& id, const std::vector<std::string>& u) const {
|
||||
return run("bulkAssignMembersToGuardrail", {{"id", id}}, {}, string_list_body("member_user_ids", u));
|
||||
}
|
||||
Json GuardrailsService::unassign_members(const std::string& id, const std::vector<std::string>& u) const {
|
||||
return run("bulkUnassignMembersFromGuardrail", {{"id", id}}, {}, string_list_body("member_user_ids", u));
|
||||
}
|
||||
Json GuardrailsService::all_key_assignments(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listKeyAssignments", {}, page_query(o, l));
|
||||
}
|
||||
Json GuardrailsService::all_member_assignments(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listMemberAssignments", {}, page_query(o, l));
|
||||
}
|
||||
|
||||
// --- BYOK ------------------------------------------------------------------
|
||||
|
||||
Json ByokService::list(std::optional<int> o, std::optional<int> l, const std::string& ws,
|
||||
const std::string& provider) const {
|
||||
Query q = page_query(o, l);
|
||||
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
|
||||
q.add_if("workspace_id", ws);
|
||||
q.add_if("provider", provider);
|
||||
return run("listBYOKKeys", {}, q);
|
||||
}
|
||||
Json ByokService::get(const std::string& id) const { return run("getBYOKKey", {{"id", id}}); }
|
||||
Json ByokService::create(const Json& body) const {
|
||||
check_body(body, "BYOK credential");
|
||||
if (!body.has("provider") || !body.has("key")) bad("BYOK credential needs provider and key");
|
||||
return run("createBYOKKey", {}, {}, body);
|
||||
}
|
||||
Json ByokService::update(const std::string& id, const Json& body) const {
|
||||
check_body(body, "BYOK update");
|
||||
return run("updateBYOKKey", {{"id", id}}, {}, body);
|
||||
}
|
||||
Json ByokService::remove(const std::string& id) const { return run("deleteBYOKKey", {{"id", id}}); }
|
||||
|
||||
// --- workspaces ------------------------------------------------------------
|
||||
|
||||
bool WorkspacesService::valid_interval(const std::string& i) {
|
||||
return i == "daily" || i == "weekly" || i == "monthly" || i == "lifetime";
|
||||
}
|
||||
Json WorkspacesService::list(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listWorkspaces", {}, page_query(o, l));
|
||||
}
|
||||
Json WorkspacesService::get(const std::string& id) const { return run("getWorkspace", {{"id", id}}); }
|
||||
Json WorkspacesService::create(const Json& body) const {
|
||||
check_body(body, "workspace");
|
||||
if (!body.has("name") || !body.has("slug")) bad("workspace needs name and slug");
|
||||
return run("createWorkspace", {}, {}, body);
|
||||
}
|
||||
Json WorkspacesService::update(const std::string& id, const Json& body) const {
|
||||
check_body(body, "workspace update");
|
||||
return run("updateWorkspace", {{"id", id}}, {}, body);
|
||||
}
|
||||
Json WorkspacesService::remove(const std::string& id, bool confirm_default) const {
|
||||
Query q;
|
||||
if (confirm_default) q.add("confirm_default_workspace_deletion", true);
|
||||
return run("deleteWorkspace", {{"id", id}}, q);
|
||||
}
|
||||
Json WorkspacesService::members(const std::string& id, std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listWorkspaceMembers", {{"id", id}}, page_query(o, l));
|
||||
}
|
||||
Json WorkspacesService::add_members(const std::string& id, const std::vector<std::string>& u) const {
|
||||
return run("bulkAddWorkspaceMembers", {{"id", id}}, {}, string_list_body("user_ids", u));
|
||||
}
|
||||
Json WorkspacesService::remove_members(const std::string& id, const std::vector<std::string>& u) const {
|
||||
return run("bulkRemoveWorkspaceMembers", {{"id", id}}, {}, string_list_body("user_ids", u));
|
||||
}
|
||||
Json WorkspacesService::budgets(const std::string& ref) const {
|
||||
return run("listWorkspaceBudgets", {{"workspace_ref", ref}});
|
||||
}
|
||||
Json WorkspacesService::budget(const std::string& ref, const std::string& interval) const {
|
||||
if (!valid_interval(interval)) bad("interval must be daily weekly monthly or lifetime");
|
||||
return run("getWorkspaceBudget", {{"workspace_ref", ref}, {"interval", interval}});
|
||||
}
|
||||
Json WorkspacesService::set_budget(const std::string& ref, const std::string& interval, double limit_usd,
|
||||
std::optional<bool> include_byok) const {
|
||||
if (!valid_interval(interval)) bad("interval must be daily weekly monthly or lifetime");
|
||||
if (!(limit_usd > 0)) bad("limit_usd must be greater than 0");
|
||||
Json b = Json::object();
|
||||
b.set("limit_usd", Json(limit_usd));
|
||||
if (include_byok) b.set("include_byok_in_budgets", Json(*include_byok));
|
||||
return run("upsertWorkspaceBudget", {{"workspace_ref", ref}, {"interval", interval}}, {}, b);
|
||||
}
|
||||
Json WorkspacesService::remove_budget(const std::string& ref, const std::string& interval) const {
|
||||
if (!valid_interval(interval)) bad("interval must be daily weekly monthly or lifetime");
|
||||
return run("deleteWorkspaceBudget", {{"workspace_ref", ref}, {"interval", interval}});
|
||||
}
|
||||
|
||||
// --- observability -----------------------------------------------------------
|
||||
|
||||
Json ObservabilityService::list(std::optional<int> o, std::optional<int> l, const std::string& ws) const {
|
||||
Query q = page_query(o, l);
|
||||
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
|
||||
q.add_if("workspace_id", ws);
|
||||
return run("listObservabilityDestinations", {}, q);
|
||||
}
|
||||
Json ObservabilityService::get(const std::string& id) const {
|
||||
return run("getObservabilityDestination", {{"id", id}});
|
||||
}
|
||||
Json ObservabilityService::create(const Json& body) const {
|
||||
check_body(body, "observability destination");
|
||||
if (!body.has("type") || !body.has("name") || !body.has("config"))
|
||||
bad("observability destination needs type, name and config");
|
||||
return run("createObservabilityDestination", {}, {}, body);
|
||||
}
|
||||
Json ObservabilityService::update(const std::string& id, const Json& body) const {
|
||||
check_body(body, "observability update");
|
||||
return run("updateObservabilityDestination", {{"id", id}}, {}, body);
|
||||
}
|
||||
Json ObservabilityService::remove(const std::string& id) const {
|
||||
return run("deleteObservabilityDestination", {{"id", id}});
|
||||
}
|
||||
|
||||
// --- private endpoints ---------------------------------------------------------
|
||||
|
||||
Json PrivateEndpointsService::list() const { return run("listPrivateEndpoints"); }
|
||||
Json PrivateEndpointsService::get(const std::string& id) const { return run("getPrivateEndpoint", {{"id", id}}); }
|
||||
Json PrivateEndpointsService::create(const Json& body) const {
|
||||
check_body(body, "private endpoint");
|
||||
return run("createPrivateEndpoint", {}, {}, body);
|
||||
}
|
||||
Json PrivateEndpointsService::update(const std::string& id, const Json& body) const {
|
||||
check_body(body, "private endpoint update");
|
||||
return run("updatePrivateEndpoint", {{"id", id}}, {}, body);
|
||||
}
|
||||
Json PrivateEndpointsService::remove(const std::string& id, bool draft_only) const {
|
||||
Query q;
|
||||
if (draft_only) q.add("draft_only", true);
|
||||
return run("deletePrivateEndpoint", {{"id", id}}, q);
|
||||
}
|
||||
Json PrivateEndpointsService::activate(const std::string& id) const { return run("activatePrivateEndpoint", {{"id", id}}); }
|
||||
Json PrivateEndpointsService::enable(const std::string& id) const { return run("enablePrivateEndpoint", {{"id", id}}); }
|
||||
Json PrivateEndpointsService::disable(const std::string& id) const { return run("disablePrivateEndpoint", {{"id", id}}); }
|
||||
Json PrivateEndpointsService::set_pricing(const std::string& id, const Json& pricing) const {
|
||||
check_body(pricing, "pricing");
|
||||
Json b = Json::object();
|
||||
b.set("pricing", pricing);
|
||||
return run("updatePrivateEndpointPricing", {{"id", id}}, {}, b);
|
||||
}
|
||||
Json PrivateEndpointsService::validate(const std::string& id, const std::string& ws) const {
|
||||
if (!ws.empty() && !is_uuid(ws)) bad("workspace_id must be a uuid");
|
||||
if (ws.empty()) return run("validatePrivateEndpoint", {{"id", id}}, {}, Json::object());
|
||||
Json b = Json::object();
|
||||
b.set("workspace_id", Json(ws));
|
||||
return run("validatePrivateEndpoint", {{"id", id}}, {}, b);
|
||||
}
|
||||
|
||||
// --- directory ---------------------------------------------------------------------
|
||||
|
||||
Json DirectoryService::external_users(std::optional<int> o, std::optional<int> l, const std::string& user,
|
||||
bool include_inactive) const {
|
||||
Query q = page_query(o, l);
|
||||
q.add_if("user", user);
|
||||
if (include_inactive) q.add("include_inactive", true);
|
||||
return run("listExternalUsers", {}, q);
|
||||
}
|
||||
Json DirectoryService::external_user(const std::string& user) const { return run("getExternalUser", {{"user", user}}); }
|
||||
Json DirectoryService::register_external_user(const std::string& user) const {
|
||||
if (user.empty() || user.size() > 512 || user.find('\0') != std::string::npos)
|
||||
bad("external user must be 1..512 characters without NUL");
|
||||
Json b = Json::object();
|
||||
b.set("user", Json(user));
|
||||
return run("createExternalUser", {}, {}, b);
|
||||
}
|
||||
Json DirectoryService::set_external_user_active(const std::string& user, bool active) const {
|
||||
Json b = Json::object();
|
||||
b.set("is_active", Json(active));
|
||||
return run("updateExternalUser", {{"user", user}}, {}, b);
|
||||
}
|
||||
Json DirectoryService::deactivate_external_user(const std::string& user) const {
|
||||
return run("deleteExternalUser", {{"user", user}});
|
||||
}
|
||||
Json DirectoryService::organization_members(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listOrganizationMembers", {}, page_query(o, l));
|
||||
}
|
||||
Json DirectoryService::organization_settings() const { return run("getOrganizationSettings"); }
|
||||
Json DirectoryService::set_filtered_model_catalog(bool enabled) const {
|
||||
Json b = Json::object();
|
||||
b.set("is_filtered_model_catalog_enabled", Json(enabled));
|
||||
return run("updateOrganizationSettings", {}, {}, b);
|
||||
}
|
||||
Json DirectoryService::scim_group_mappings(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listScimGroupMappings", {}, page_query(o, l));
|
||||
}
|
||||
Json DirectoryService::scim_group_mapping(const std::string& id) const {
|
||||
return run("getScimGroupMapping", {{"id", id}});
|
||||
}
|
||||
Json DirectoryService::create_scim_group_mapping(const Json& body) const {
|
||||
check_body(body, "SCIM group mapping");
|
||||
if (!body.has("scim_group_id") || !body.has("role")) bad("SCIM group mapping needs scim_group_id and role");
|
||||
return run("createScimGroupMapping", {}, {}, body);
|
||||
}
|
||||
Json DirectoryService::update_scim_group_mapping(const std::string& id, const std::string& role) const {
|
||||
if (role.empty()) bad("role must not be empty");
|
||||
Json b = Json::object();
|
||||
b.set("role", Json(role));
|
||||
return run("updateScimGroupMapping", {{"id", id}}, {}, b);
|
||||
}
|
||||
Json DirectoryService::remove_scim_group_mapping(const std::string& id, bool keep_members) const {
|
||||
Query q;
|
||||
if (keep_members) q.add("keep_members", true);
|
||||
return run("deleteScimGroupMapping", {{"id", id}}, q);
|
||||
}
|
||||
Json DirectoryService::scim_groups(std::optional<int> o, std::optional<int> l, const std::string& display_name,
|
||||
const std::string& external_id) const {
|
||||
Query q = page_query(o, l);
|
||||
q.add_if("display_name", display_name);
|
||||
q.add_if("external_id", external_id);
|
||||
return run("listScimGroups", {}, q);
|
||||
}
|
||||
Json DirectoryService::start_scim_sync() const { return run("createScimSyncJob"); }
|
||||
Json DirectoryService::scim_sync_job(const std::string& id) const { return run("getScimSyncJob", {{"id", id}}); }
|
||||
|
||||
// --- vault -----------------------------------------------------------------------------
|
||||
|
||||
Json VaultService::secrets(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listVaultSecrets", {}, page_query(o, l));
|
||||
}
|
||||
Json VaultService::put_secret(const std::string& name, const std::string& value,
|
||||
const std::vector<std::string>& hosts) const {
|
||||
return run("storeVaultSecret", {{"name", name}}, {}, secret_body(value, hosts));
|
||||
}
|
||||
Json VaultService::delete_secret(const std::string& name) const { return run("deleteVaultSecret", {{"name", name}}); }
|
||||
Json VaultService::intern_secrets(const std::string& intern, std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listInternVaultSecrets", {{"internId", intern}}, page_query(o, l));
|
||||
}
|
||||
Json VaultService::intern_effective_secrets(const std::string& intern, std::optional<int> o,
|
||||
std::optional<int> l) const {
|
||||
return run("listInternEffectiveVaultSecrets", {{"internId", intern}}, page_query(o, l));
|
||||
}
|
||||
Json VaultService::intern_connections(const std::string& intern) const {
|
||||
return run("listInternConnections", {{"internId", intern}});
|
||||
}
|
||||
Json VaultService::put_intern_secret(const std::string& intern, const std::string& name,
|
||||
const std::string& value, const std::vector<std::string>& hosts) const {
|
||||
return run("storeInternVaultSecret", {{"internId", intern}, {"name", name}}, {}, secret_body(value, hosts));
|
||||
}
|
||||
Json VaultService::delete_intern_secret(const std::string& intern, const std::string& name) const {
|
||||
return run("deleteInternVaultSecret", {{"internId", intern}, {"name", name}});
|
||||
}
|
||||
Json VaultService::copy_secrets_to_intern(const std::string& intern, const std::vector<std::string>& names) const {
|
||||
return run("copyVaultSecretsToIntern", {{"internId", intern}}, {}, string_list_body("names", names));
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,155 @@
|
||||
#pragma once
|
||||
|
||||
// Management surface. All classes here use OPENROUTER_MANAGEMENT_KEY (the
|
||||
// separate management / provisioning key), except VaultService, which the
|
||||
// spec secures with the normal API key. Request bodies are JSON objects whose
|
||||
// top level keys are checked against the call table (calls.h); the large
|
||||
// nested shapes (guardrail content filters, observability destination
|
||||
// configs, private endpoint pricing) are passed through as given.
|
||||
//
|
||||
// Every class that can change or delete something exposes those calls as
|
||||
// separate explicit methods (create, update, remove, ...), never with a
|
||||
// default. The demo layer adds the dry-run gate.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "openrouter/gateway.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
// offset >= 0, limit 1..max_limit; throws llm::ApiError::invalid otherwise.
|
||||
Query page_query(std::optional<int> offset, std::optional<int> limit, int max_limit = 1000);
|
||||
|
||||
class GuardrailsService : public Endpoint {
|
||||
public:
|
||||
explicit GuardrailsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& workspace_id = "") const;
|
||||
Json get(const std::string& id) const;
|
||||
Json create(const Json& body) const; // name is required by the spec
|
||||
Json update(const std::string& id, const Json& body) const;
|
||||
Json remove(const std::string& id) const;
|
||||
Json key_assignments(const std::string& id, std::optional<int> offset = std::nullopt,
|
||||
std::optional<int> limit = std::nullopt) const;
|
||||
Json assign_keys(const std::string& id, const std::vector<std::string>& key_hashes) const;
|
||||
Json unassign_keys(const std::string& id, const std::vector<std::string>& key_hashes) const;
|
||||
Json member_assignments(const std::string& id, std::optional<int> offset = std::nullopt,
|
||||
std::optional<int> limit = std::nullopt) const;
|
||||
Json assign_members(const std::string& id, const std::vector<std::string>& user_ids) const;
|
||||
Json unassign_members(const std::string& id, const std::vector<std::string>& user_ids) const;
|
||||
Json all_key_assignments(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
Json all_member_assignments(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
};
|
||||
|
||||
// BYOK: provider credentials you bring. create() sends a provider secret in
|
||||
// `key`; it is never echoed back and must never be logged.
|
||||
class ByokService : public Endpoint {
|
||||
public:
|
||||
explicit ByokService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& workspace_id = "", const std::string& provider = "") const;
|
||||
Json get(const std::string& id) const;
|
||||
Json create(const Json& body) const;
|
||||
Json update(const std::string& id, const Json& body) const;
|
||||
Json remove(const std::string& id) const;
|
||||
};
|
||||
|
||||
class WorkspacesService : public Endpoint {
|
||||
public:
|
||||
explicit WorkspacesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
Json get(const std::string& id) const;
|
||||
Json create(const Json& body) const;
|
||||
Json update(const std::string& id, const Json& body) const;
|
||||
// The default workspace is only deleted when confirm_default is true.
|
||||
Json remove(const std::string& id, bool confirm_default) const;
|
||||
Json members(const std::string& id, std::optional<int> offset = std::nullopt,
|
||||
std::optional<int> limit = std::nullopt) const;
|
||||
Json add_members(const std::string& id, const std::vector<std::string>& user_ids) const;
|
||||
Json remove_members(const std::string& id, const std::vector<std::string>& user_ids) const;
|
||||
Json budgets(const std::string& workspace_ref) const;
|
||||
Json budget(const std::string& workspace_ref, const std::string& interval) const;
|
||||
// interval: daily weekly monthly lifetime. limit_usd > 0.
|
||||
Json set_budget(const std::string& workspace_ref, const std::string& interval, double limit_usd,
|
||||
std::optional<bool> include_byok_in_budgets = std::nullopt) const;
|
||||
Json remove_budget(const std::string& workspace_ref, const std::string& interval) const;
|
||||
static bool valid_interval(const std::string& interval);
|
||||
};
|
||||
|
||||
class ObservabilityService : public Endpoint {
|
||||
public:
|
||||
explicit ObservabilityService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& workspace_id = "") const;
|
||||
Json get(const std::string& id) const;
|
||||
Json create(const Json& body) const;
|
||||
Json update(const std::string& id, const Json& body) const;
|
||||
Json remove(const std::string& id) const;
|
||||
};
|
||||
|
||||
class PrivateEndpointsService : public Endpoint {
|
||||
public:
|
||||
explicit PrivateEndpointsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
Json list() const;
|
||||
Json get(const std::string& id) const;
|
||||
Json create(const Json& body) const;
|
||||
Json update(const std::string& id, const Json& body) const;
|
||||
Json remove(const std::string& id, bool draft_only) const;
|
||||
Json activate(const std::string& id) const;
|
||||
Json enable(const std::string& id) const;
|
||||
Json disable(const std::string& id) const;
|
||||
Json set_pricing(const std::string& id, const Json& pricing) const;
|
||||
Json validate(const std::string& id, const std::string& workspace_id = "") const;
|
||||
};
|
||||
|
||||
// End users (external users), organization and SCIM.
|
||||
class DirectoryService : public Endpoint {
|
||||
public:
|
||||
explicit DirectoryService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kManagement) {}
|
||||
// External users.
|
||||
Json external_users(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& user = "", bool include_inactive = false) const;
|
||||
Json external_user(const std::string& user) const;
|
||||
Json register_external_user(const std::string& user) const;
|
||||
Json set_external_user_active(const std::string& user, bool active) const;
|
||||
Json deactivate_external_user(const std::string& user) const;
|
||||
// Organization.
|
||||
Json organization_members(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
Json organization_settings() const;
|
||||
Json set_filtered_model_catalog(bool enabled) const;
|
||||
// SCIM.
|
||||
Json scim_group_mappings(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
Json scim_group_mapping(const std::string& id) const;
|
||||
Json create_scim_group_mapping(const Json& body) const;
|
||||
Json update_scim_group_mapping(const std::string& id, const std::string& role) const;
|
||||
Json remove_scim_group_mapping(const std::string& id, bool keep_members) const;
|
||||
Json scim_groups(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& display_name = "", const std::string& external_id = "") const;
|
||||
Json start_scim_sync() const;
|
||||
Json scim_sync_job(const std::string& id) const;
|
||||
};
|
||||
|
||||
// Secrets vault for workspaces and interns. Secured with OPENROUTER_API_KEY.
|
||||
class VaultService : public Endpoint {
|
||||
public:
|
||||
explicit VaultService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
Json secrets(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
// value: 1..65536 characters, hosts: 1..100 exact hostnames. The value is
|
||||
// never returned and must never be logged.
|
||||
Json put_secret(const std::string& name, const std::string& value,
|
||||
const std::vector<std::string>& hosts) const;
|
||||
Json delete_secret(const std::string& name) const;
|
||||
Json intern_secrets(const std::string& intern_id, std::optional<int> offset = std::nullopt,
|
||||
std::optional<int> limit = std::nullopt) const;
|
||||
Json intern_effective_secrets(const std::string& intern_id, std::optional<int> offset = std::nullopt,
|
||||
std::optional<int> limit = std::nullopt) const;
|
||||
Json intern_connections(const std::string& intern_id) const;
|
||||
Json put_intern_secret(const std::string& intern_id, const std::string& name,
|
||||
const std::string& value, const std::vector<std::string>& hosts) const;
|
||||
Json delete_intern_secret(const std::string& intern_id, const std::string& name) const;
|
||||
Json copy_secrets_to_intern(const std::string& intern_id, const std::vector<std::string>& names) const;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,178 @@
|
||||
#include "openrouter/call_plan.h"
|
||||
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <format>
|
||||
#include <sstream>
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "llm/api_client.h"
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) {
|
||||
throw llm::ApiError::invalid(kVendor, why);
|
||||
}
|
||||
|
||||
std::set<std::string> split_fields(const char* fields, bool want_query) {
|
||||
std::set<std::string> out;
|
||||
std::istringstream in(fields);
|
||||
for (std::string f; in >> f;) {
|
||||
if (f == "-") continue;
|
||||
bool is_query = f.front() == '?';
|
||||
if (is_query == want_query) out.insert(is_query ? f.substr(1) : f);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
bool free_text_arg(const std::string& placeholder) {
|
||||
// External user ids are partner-defined text; everything else is a slug,
|
||||
// hash or uuid and goes through Service::valid_id.
|
||||
return placeholder == "user";
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Query& Query::add(const std::string& name, const std::string& value) {
|
||||
if (items_.size() >= Caps::kMaxQueryParams) bad("too many query parameters");
|
||||
if (value.size() > Caps::kMaxQueryValueBytes) bad("query value too long: " + name);
|
||||
items_.emplace_back(name, value);
|
||||
return *this;
|
||||
}
|
||||
Query& Query::add(const std::string& name, long value) { return add(name, std::to_string(value)); }
|
||||
Query& Query::add(const std::string& name, double value) { return add(name, std::format("{}", value)); }
|
||||
Query& Query::add(const std::string& name, bool value) { return add(name, std::string(value ? "true" : "false")); }
|
||||
Query& Query::add_if(const std::string& name, const std::string& value) {
|
||||
if (!value.empty()) add(name, value);
|
||||
return *this;
|
||||
}
|
||||
|
||||
const CallSpec* CallPlanner::find(std::string_view name) {
|
||||
for (const CallSpec& s : kCalls)
|
||||
if (name == s.name) return &s;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
const CallSpec& CallPlanner::spec(std::string_view name) {
|
||||
const CallSpec* s = find(name);
|
||||
if (!s) bad("unknown call: " + std::string(name));
|
||||
return *s;
|
||||
}
|
||||
|
||||
KeyKind CallPlanner::key_kind(const CallSpec& spec) {
|
||||
if (std::strcmp(spec.key, "mgmt") == 0) return KeyKind::kManagement;
|
||||
if (std::strcmp(spec.key, "none") == 0) return KeyKind::kNone;
|
||||
return KeyKind::kApi;
|
||||
}
|
||||
|
||||
clients::Method CallPlanner::method_of(const CallSpec& spec) {
|
||||
if (!std::strcmp(spec.http, "GET")) return clients::Method::kGet;
|
||||
if (!std::strcmp(spec.http, "POST")) return clients::Method::kPost;
|
||||
if (!std::strcmp(spec.http, "PUT")) return clients::Method::kPut;
|
||||
if (!std::strcmp(spec.http, "PATCH")) return clients::Method::kPatch;
|
||||
if (!std::strcmp(spec.http, "DELETE")) return clients::Method::kDelete;
|
||||
bad(std::string("bad method in table for ") + spec.name);
|
||||
}
|
||||
|
||||
std::set<std::string> CallPlanner::body_fields(const CallSpec& spec) {
|
||||
return split_fields(spec.fields, false);
|
||||
}
|
||||
std::set<std::string> CallPlanner::query_fields(const CallSpec& spec) {
|
||||
return split_fields(spec.fields, true);
|
||||
}
|
||||
|
||||
std::vector<std::string> CallPlanner::placeholders(const std::string& tmpl) {
|
||||
std::vector<std::string> out;
|
||||
std::size_t pos = 0;
|
||||
while ((pos = tmpl.find('{', pos)) != std::string::npos) {
|
||||
std::size_t end = tmpl.find('}', pos);
|
||||
if (end == std::string::npos) break;
|
||||
out.push_back(tmpl.substr(pos + 1, end - pos - 1));
|
||||
pos = end + 1;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
bool CallPlanner::accepts_field(std::string_view name, const std::string& field) {
|
||||
const CallSpec* s = find(name);
|
||||
return s && body_fields(*s).count(field) > 0;
|
||||
}
|
||||
|
||||
PlannedCall CallPlanner::plan(std::string_view name, const PathArgs& args,
|
||||
const Query& query, const Json* body) {
|
||||
const CallSpec& s = spec(name);
|
||||
PlannedCall out;
|
||||
out.spec = &s;
|
||||
out.method = method_of(s);
|
||||
out.key = key_kind(s);
|
||||
|
||||
std::string path = s.path;
|
||||
std::set<std::string> used;
|
||||
for (const std::string& ph : placeholders(s.path)) {
|
||||
auto it = args.find(ph);
|
||||
if (it == args.end()) bad("missing path argument: " + ph);
|
||||
const std::string& value = it->second;
|
||||
if (free_text_arg(ph)) {
|
||||
// Partner text of 1..512 bytes; "." and ".." cannot be addressed by path.
|
||||
if (value.empty() || value.size() > 512 || value == "." || value == ".." ||
|
||||
value.find('\0') != std::string::npos)
|
||||
bad("bad " + ph);
|
||||
} else if (!llm::Service::valid_id(ph == "author" && value.rfind("~", 0) == 0
|
||||
? value.substr(1)
|
||||
: value)) {
|
||||
bad("bad " + ph + " id"); // author may carry the alias marker "~"
|
||||
}
|
||||
std::string token = "{" + ph + "}";
|
||||
// valid_id characters are all path-safe; only free text is encoded.
|
||||
path.replace(path.find(token), token.size(),
|
||||
free_text_arg(ph) ? clients::Url::encode_component(value) : value);
|
||||
used.insert(ph);
|
||||
}
|
||||
for (const auto& [k, v] : args) {
|
||||
(void)v;
|
||||
if (!used.count(k)) bad("unexpected path argument: " + k);
|
||||
}
|
||||
out.path = path;
|
||||
|
||||
std::set<std::string> qf = query_fields(s);
|
||||
std::string qs;
|
||||
for (const auto& [k, v] : query.items()) {
|
||||
if (!qf.count(k)) bad("query parameter not declared for " + std::string(name) + ": " + k);
|
||||
if (v.empty()) bad("empty query value: " + k);
|
||||
qs += (qs.empty() ? "?" : "&") + clients::Url::encode_component(k) + "=" +
|
||||
clients::Url::encode_component(v);
|
||||
}
|
||||
out.query = qs;
|
||||
|
||||
std::set<std::string> bf = body_fields(s);
|
||||
if (body) {
|
||||
if (bf.empty()) bad(std::string(name) + " takes no request body");
|
||||
if (!body->is_object()) bad("request body must be a JSON object");
|
||||
for (const auto& [k, v] : body->as_object()) {
|
||||
(void)v;
|
||||
if (!bf.count(k)) bad("body field not declared for " + std::string(name) + ": " + k);
|
||||
}
|
||||
out.has_body = true;
|
||||
out.body = *body;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::pair<std::string, std::string> split_model_id(const std::string& model_id) {
|
||||
std::size_t slash = model_id.find('/');
|
||||
if (slash == std::string::npos || slash == 0 || slash + 1 >= model_id.size() ||
|
||||
model_id.find('/', slash + 1) != std::string::npos)
|
||||
bad("model id must look like author/slug");
|
||||
std::string author = model_id.substr(0, slash);
|
||||
std::string slug = model_id.substr(slash + 1);
|
||||
if (!llm::Service::valid_id(author.rfind("~", 0) == 0 ? author.substr(1) : author) ||
|
||||
!llm::Service::valid_id(slug))
|
||||
bad("bad model id");
|
||||
return {author, slug};
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,87 @@
|
||||
#pragma once
|
||||
|
||||
// CallPlanner: turns a call name from the table (calls.h) plus path
|
||||
// arguments, query arguments and a JSON body into a PlannedCall, and rejects
|
||||
// anything the table does not declare (unknown call, missing or malformed
|
||||
// path argument, undeclared query parameter, undeclared body field, body on a
|
||||
// call without one). Nothing here touches the network, so every request the
|
||||
// module can send is checkable against the spec-verified table.
|
||||
|
||||
#include <map>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "clients/request.h"
|
||||
#include "openrouter/calls.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using Json = clients::body::Json;
|
||||
|
||||
inline constexpr const char* kVendor = "openrouter";
|
||||
|
||||
using PathArgs = std::map<std::string, std::string>;
|
||||
|
||||
// Ordered query parameters (order is preserved on the wire).
|
||||
class Query {
|
||||
public:
|
||||
Query& add(const std::string& name, const std::string& value);
|
||||
Query& add(const std::string& name, const char* value) { return add(name, std::string(value)); }
|
||||
Query& add(const std::string& name, long value);
|
||||
Query& add(const std::string& name, int value) { return add(name, static_cast<long>(value)); }
|
||||
Query& add(const std::string& name, double value);
|
||||
Query& add(const std::string& name, bool value);
|
||||
// Adds only when the value is non-empty.
|
||||
Query& add_if(const std::string& name, const std::string& value);
|
||||
const std::vector<std::pair<std::string, std::string>>& items() const { return items_; }
|
||||
bool empty() const { return items_.empty(); }
|
||||
|
||||
private:
|
||||
std::vector<std::pair<std::string, std::string>> items_;
|
||||
};
|
||||
|
||||
enum class KeyKind { kApi, kManagement, kNone };
|
||||
|
||||
struct PlannedCall {
|
||||
const CallSpec* spec = nullptr;
|
||||
clients::Method method = clients::Method::kGet;
|
||||
std::string path; // placeholders substituted, segments URL-encoded
|
||||
std::string query; // "" or "?a=b&c=d"
|
||||
bool has_body = false;
|
||||
Json body;
|
||||
KeyKind key = KeyKind::kApi;
|
||||
|
||||
std::string name() const { return spec ? spec->name : ""; }
|
||||
std::string fmt() const { return spec ? spec->fmt : "json"; }
|
||||
std::string target() const { return path + query; }
|
||||
};
|
||||
|
||||
class CallPlanner {
|
||||
public:
|
||||
static const CallSpec* find(std::string_view name);
|
||||
// Throws llm::ApiError::invalid for an unknown name.
|
||||
static const CallSpec& spec(std::string_view name);
|
||||
|
||||
static KeyKind key_kind(const CallSpec& spec);
|
||||
static clients::Method method_of(const CallSpec& spec);
|
||||
static std::set<std::string> body_fields(const CallSpec& spec);
|
||||
static std::set<std::string> query_fields(const CallSpec& spec);
|
||||
static std::vector<std::string> placeholders(const std::string& path_template);
|
||||
|
||||
// Validates and builds. `body` may be null. Throws llm::ApiError::invalid.
|
||||
static PlannedCall plan(std::string_view name, const PathArgs& args = {},
|
||||
const Query& query = {}, const Json* body = nullptr);
|
||||
|
||||
// True when `field` may be sent in the body of `name`.
|
||||
static bool accepts_field(std::string_view name, const std::string& field);
|
||||
};
|
||||
|
||||
// Splits "author/slug[:variant]" into its two path segments. Throws
|
||||
// llm::ApiError::invalid when it is not exactly two valid segments.
|
||||
std::pair<std::string, std::string> split_model_id(const std::string& model_id);
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,181 @@
|
||||
#pragma once
|
||||
|
||||
// The ONE table of every OpenRouter call this module makes. Each row is
|
||||
// checked by tools/check_spec.py against OpenRouter's OpenAPI document
|
||||
// (https://openrouter.ai/openapi.json, 139 operations when this table was
|
||||
// written, plus one documented-only row, see SPEC.md "Verified against
|
||||
// spec"). One row per line, in exactly this shape (the checker parses it):
|
||||
// {"name", "HTTP", "/path/{param}", "operationId", "key", "fmt", "fields"},
|
||||
//
|
||||
// operationId the spec's operationId, or "doc:<slug>" for a documented
|
||||
// route the OpenAPI document does not list (checked live only).
|
||||
// key "api" : OPENROUTER_API_KEY (bearer)
|
||||
// "mgmt" : OPENROUTER_MANAGEMENT_KEY (bearer, separate key)
|
||||
// "none" : public route, works without any key
|
||||
// fmt "json" | "multipart" | "form" | "raw" (response is bytes)
|
||||
// fields space separated: top level request body field names,
|
||||
// "?name" for a query parameter, "-" for none.
|
||||
|
||||
#include <cstddef>
|
||||
#include <string_view>
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct CallSpec {
|
||||
const char* name;
|
||||
const char* http;
|
||||
const char* path;
|
||||
const char* op;
|
||||
const char* key;
|
||||
const char* fmt;
|
||||
const char* fields;
|
||||
};
|
||||
|
||||
// clang-format off
|
||||
inline constexpr CallSpec kCalls[] = {
|
||||
{"getUserActivity", "GET", "/activity", "getUserActivity", "mgmt", "json", "?date ?api_key_hash ?user_id ?group_by ?workspace_id"},
|
||||
{"getAnalyticsMeta", "GET", "/analytics/meta", "getAnalyticsMeta", "mgmt", "json", "-"},
|
||||
{"queryAnalytics", "POST", "/analytics/query", "queryAnalytics", "mgmt", "json", "classifier_dimensions classifier_filters dimensions filters granularity group_limit limit metrics order_by time_range"},
|
||||
{"createApiAlphaDecisions", "POST", "/api/alpha/decisions", "createApiAlphaDecisions", "api", "json", "model provider questions session_id state trace user"},
|
||||
{"createAudioSpeech", "POST", "/audio/speech", "createAudioSpeech", "api", "raw", "input input_references instructions model provider response_format session_id speed trace user voice"},
|
||||
{"createAudioTranscriptions", "POST", "/audio/transcriptions", "createAudioTranscriptions", "api", "json", "diarize input_audio keyterms language model provider response_format session_id temperature timestamp_granularities trace user"},
|
||||
{"exchangeAuthCodeForAPIKey", "POST", "/auth/keys", "exchangeAuthCodeForAPIKey", "none", "json", "code code_challenge_method code_verifier"},
|
||||
{"createAuthKeysCode", "POST", "/auth/keys/code", "createAuthKeysCode", "api", "json", "callback_url code_challenge code_challenge_method expires_at key_label limit spawn_agent spawn_cloud usage_limit_type workspace_id"},
|
||||
{"listBatches", "GET", "/batches", "listBatches", "api", "json", "?limit ?after ?status ?created_after ?created_before"},
|
||||
{"createBatches", "POST", "/batches", "createBatches", "api", "json", "callback_url completion_window endpoint model provider requests"},
|
||||
{"deleteBatch", "DELETE", "/batches/{id}", "deleteBatch", "api", "json", "-"},
|
||||
{"getBatches", "GET", "/batches/{id}", "getBatches", "api", "json", "-"},
|
||||
{"getBenchmarks", "GET", "/benchmarks", "getBenchmarks", "api", "json", "?source ?task_type ?benchmark_type ?include_run_config ?search_engine ?search_surface ?arena ?category ?max_results"},
|
||||
{"listBYOKKeys", "GET", "/byok", "listBYOKKeys", "mgmt", "json", "?offset ?limit ?workspace_id ?provider"},
|
||||
{"createBYOKKey", "POST", "/byok", "createBYOKKey", "mgmt", "json", "allowed_api_key_hashes allowed_models allowed_user_ids declared_region declared_zdr disabled is_byok_only is_fallback is_required key name provider workspace_id"},
|
||||
{"deleteBYOKKey", "DELETE", "/byok/{id}", "deleteBYOKKey", "mgmt", "json", "-"},
|
||||
{"getBYOKKey", "GET", "/byok/{id}", "getBYOKKey", "mgmt", "json", "-"},
|
||||
{"updateBYOKKey", "PATCH", "/byok/{id}", "updateBYOKKey", "mgmt", "json", "allowed_api_key_hashes allowed_models allowed_user_ids declared_region declared_zdr disabled is_byok_only is_fallback is_required key name"},
|
||||
{"sendChatCompletionRequest", "POST", "/chat/completions", "sendChatCompletionRequest", "api", "json", "cache_control debug deferred_tools frequency_penalty image_config logit_bias logprobs max_completion_tokens max_tokens messages metadata min_p modalities model models parallel_tool_calls plugins prediction presence_penalty prompt_cache_key prompt_cache_options provider reasoning reasoning_effort repetition_penalty response_format route seed service_tier session_id stop stop_server_tools_when stream stream_options temperature tool_choice tools top_a top_k top_logprobs top_p trace user"},
|
||||
{"getTaskClassifications", "GET", "/classifications/task", "getTaskClassifications", "api", "json", "?window"},
|
||||
{"createCompletions", "POST", "/completions", "doc:completions", "api", "json", "model models prompt suffix max_tokens temperature top_p top_k stop stream seed frequency_penalty presence_penalty repetition_penalty logit_bias logprobs top_logprobs min_p top_a n echo best_of user provider route plugins session_id response_format stream_options reasoning transforms"},
|
||||
{"listContainerFiles", "GET", "/containers/{container_id}/files", "listContainerFiles", "api", "json", "?limit ?after"},
|
||||
{"getContainerFile", "GET", "/containers/{container_id}/files/{file_id}", "getContainerFile", "api", "json", "-"},
|
||||
{"downloadContainerFileContent", "GET", "/containers/{container_id}/files/{file_id}/content", "downloadContainerFileContent", "api", "raw", "-"},
|
||||
{"promoteContainerFile", "POST", "/containers/{container_id}/files/{file_id}/promote", "promoteContainerFile", "api", "json", "-"},
|
||||
{"getCredits", "GET", "/credits", "getCredits", "mgmt", "json", "-"},
|
||||
{"createCoinbaseCharge", "POST", "/credits/coinbase", "createCoinbaseCharge", "none", "json", "-"},
|
||||
{"getAppRankings", "GET", "/datasets/app-rankings", "getAppRankings", "api", "json", "?category ?subcategory ?sort ?start_date ?end_date ?limit ?offset"},
|
||||
{"getRankingsDaily", "GET", "/datasets/rankings-daily", "getRankingsDaily", "api", "json", "?start_date ?end_date ?period ?modality ?context_bucket ?category ?language_type"},
|
||||
{"getSessionCost", "GET", "/datasets/session-cost", "getSessionCost", "api", "json", "?app_slug ?model ?turn_range ?limit ?offset"},
|
||||
{"createEmbeddings", "POST", "/embeddings", "createEmbeddings", "api", "json", "dimensions encoding_format input input_type model provider session_id trace user"},
|
||||
{"listEmbeddingsModels", "GET", "/embeddings/models", "listEmbeddingsModels", "none", "json", "?offset ?limit"},
|
||||
{"listEndpointsZdr", "GET", "/endpoints/zdr", "listEndpointsZdr", "none", "json", "-"},
|
||||
{"listExternalUsers", "GET", "/external-users", "listExternalUsers", "mgmt", "json", "?offset ?limit ?user ?include_inactive"},
|
||||
{"createExternalUser", "POST", "/external-users", "createExternalUser", "mgmt", "json", "user"},
|
||||
{"deleteExternalUser", "DELETE", "/external-users/{user}", "deleteExternalUser", "mgmt", "json", "-"},
|
||||
{"getExternalUser", "GET", "/external-users/{user}", "getExternalUser", "mgmt", "json", "-"},
|
||||
{"updateExternalUser", "PATCH", "/external-users/{user}", "updateExternalUser", "mgmt", "json", "is_active"},
|
||||
{"listFiles", "GET", "/files", "listFiles", "api", "json", "?limit ?cursor ?workspace_id ?provider ?after ?after_id ?before_id ?order"},
|
||||
{"uploadFile", "POST", "/files", "uploadFile", "api", "multipart", "file ?workspace_id ?provider"},
|
||||
{"deleteFile", "DELETE", "/files/{file_id}", "deleteFile", "api", "json", "?workspace_id ?provider"},
|
||||
{"getFileMetadata", "GET", "/files/{file_id}", "getFileMetadata", "api", "json", "?workspace_id ?provider"},
|
||||
{"downloadFileContent", "GET", "/files/{file_id}/content", "downloadFileContent", "api", "raw", "?workspace_id ?provider"},
|
||||
{"getGeneration", "GET", "/generation", "getGeneration", "api", "json", "?id"},
|
||||
{"listGenerationContent", "GET", "/generation/content", "listGenerationContent", "api", "json", "?id"},
|
||||
{"submitGenerationFeedback", "POST", "/generation/feedback", "submitGenerationFeedback", "mgmt", "json", "category comment generation_id"},
|
||||
{"listGuardrails", "GET", "/guardrails", "listGuardrails", "mgmt", "json", "?offset ?limit ?workspace_id"},
|
||||
{"createGuardrail", "POST", "/guardrails", "createGuardrail", "mgmt", "json", "allow_safety_retention_google allow_safety_retention_other allowed_data_regions allowed_models allowed_providers content_filter_builtins content_filters description enable_free_model_publication enable_free_model_training enable_paid_model_training enforce_zdr enforce_zdr_anthropic enforce_zdr_google enforce_zdr_openai enforce_zdr_other enforce_zdr_xai ignored_models ignored_providers include_byok_in_budgets limit_usd name reset_interval workspace_id"},
|
||||
{"listKeyAssignments", "GET", "/guardrails/assignments/keys", "listKeyAssignments", "mgmt", "json", "?offset ?limit"},
|
||||
{"listMemberAssignments", "GET", "/guardrails/assignments/members", "listMemberAssignments", "mgmt", "json", "?offset ?limit"},
|
||||
{"deleteGuardrail", "DELETE", "/guardrails/{id}", "deleteGuardrail", "mgmt", "json", "-"},
|
||||
{"getGuardrail", "GET", "/guardrails/{id}", "getGuardrail", "mgmt", "json", "-"},
|
||||
{"updateGuardrail", "PATCH", "/guardrails/{id}", "updateGuardrail", "mgmt", "json", "allow_safety_retention_google allow_safety_retention_other allowed_data_regions allowed_models allowed_providers content_filter_builtins content_filters description enable_free_model_publication enable_free_model_training enable_paid_model_training enforce_zdr enforce_zdr_anthropic enforce_zdr_google enforce_zdr_openai enforce_zdr_other enforce_zdr_xai ignored_models ignored_providers include_byok_in_budgets limit_usd name reset_interval"},
|
||||
{"listGuardrailKeyAssignments", "GET", "/guardrails/{id}/assignments/keys", "listGuardrailKeyAssignments", "mgmt", "json", "?offset ?limit"},
|
||||
{"bulkAssignKeysToGuardrail", "POST", "/guardrails/{id}/assignments/keys", "bulkAssignKeysToGuardrail", "mgmt", "json", "key_hashes"},
|
||||
{"bulkUnassignKeysFromGuardrail", "POST", "/guardrails/{id}/assignments/keys/remove", "bulkUnassignKeysFromGuardrail", "mgmt", "json", "key_hashes"},
|
||||
{"listGuardrailMemberAssignments", "GET", "/guardrails/{id}/assignments/members", "listGuardrailMemberAssignments", "mgmt", "json", "?offset ?limit"},
|
||||
{"bulkAssignMembersToGuardrail", "POST", "/guardrails/{id}/assignments/members", "bulkAssignMembersToGuardrail", "mgmt", "json", "member_user_ids"},
|
||||
{"bulkUnassignMembersFromGuardrail", "POST", "/guardrails/{id}/assignments/members/remove", "bulkUnassignMembersFromGuardrail", "mgmt", "json", "member_user_ids"},
|
||||
{"createImages", "POST", "/images", "createImages", "api", "json", "aspect_ratio background input_references model n output_compression output_format prompt provider quality resolution seed session_id size stream trace user"},
|
||||
{"listImageModels", "GET", "/images/models", "listImageModels", "none", "json", "-"},
|
||||
{"listImageModelEndpoints", "GET", "/images/models/{author}/{slug}/endpoints", "listImageModelEndpoints", "none", "json", "-"},
|
||||
{"getCurrentKey", "GET", "/key", "getCurrentKey", "api", "json", "-"},
|
||||
{"listKeys", "GET", "/keys", "list", "mgmt", "json", "?offset ?limit ?include_disabled ?include_expired ?workspace_id"},
|
||||
{"createKeys", "POST", "/keys", "createKeys", "mgmt", "json", "creator_user_id expires_at external include_byok_in_limit limit limit_reset name workspace_id"},
|
||||
{"deleteKeys", "DELETE", "/keys/{hash}", "deleteKeys", "mgmt", "json", "-"},
|
||||
{"getKey", "GET", "/keys/{hash}", "getKey", "mgmt", "json", "-"},
|
||||
{"updateKeys", "PATCH", "/keys/{hash}", "updateKeys", "mgmt", "json", "disabled include_byok_in_limit limit limit_reset name workspace_id"},
|
||||
{"createMessages", "POST", "/messages", "createMessages", "api", "json", "cache_control context_management deferred_tools fallbacks max_tokens messages metadata model models output_config plugins provider route safeguards service_tier session_id speed stop_sequences stop_server_tools_when stream system temperature thinking tool_choice tools top_k top_p trace user"},
|
||||
{"getModel", "GET", "/model/{author}/{slug}", "getModel", "none", "json", "-"},
|
||||
{"getModels", "GET", "/models", "getModels", "none", "json", "?offset ?limit ?category ?supported_parameters ?output_modalities ?sort ?q ?input_modalities ?context ?min_price ?max_price ?arch ?model_authors ?providers ?distillable ?zdr ?region ?min_output_price ?max_output_price ?min_age_days ?max_age_days ?min_intelligence_index ?max_intelligence_index ?min_coding_index ?max_coding_index ?min_agentic_index ?max_agentic_index ?min_tool_success_rate ?max_tool_success_rate"},
|
||||
{"listModelsCount", "GET", "/models/count", "listModelsCount", "none", "json", "?output_modalities"},
|
||||
{"listModelsUser", "GET", "/models/user", "listModelsUser", "api", "json", "?offset ?limit ?output_modalities ?sort"},
|
||||
{"listEndpoints", "GET", "/models/{author}/{slug}/endpoints", "listEndpoints", "none", "json", "-"},
|
||||
{"listOauthJwks", "GET", "/oauth/jwks", "listOauthJwks", "none", "json", "-"},
|
||||
{"createOauthToken", "POST", "/oauth/token", "createOauthToken", "none", "form", "federation_policy_id grant_type requested_token_type scope subject_token subject_token_type"},
|
||||
{"listObservabilityDestinations", "GET", "/observability/destinations", "listObservabilityDestinations", "mgmt", "json", "?offset ?limit ?workspace_id"},
|
||||
{"createObservabilityDestination", "POST", "/observability/destinations", "createObservabilityDestination", "mgmt", "json", "api_key_hashes broadcast_generation_cost broadcast_generation_cost_allocation broadcast_generation_identity broadcast_generation_request_context config enabled filter_rules name privacy_mode regions sampling_rate type workspace_id"},
|
||||
{"deleteObservabilityDestination", "DELETE", "/observability/destinations/{id}", "deleteObservabilityDestination", "mgmt", "json", "-"},
|
||||
{"getObservabilityDestination", "GET", "/observability/destinations/{id}", "getObservabilityDestination", "mgmt", "json", "-"},
|
||||
{"updateObservabilityDestination", "PATCH", "/observability/destinations/{id}", "updateObservabilityDestination", "mgmt", "json", "api_key_hashes broadcast_generation_cost broadcast_generation_cost_allocation broadcast_generation_identity broadcast_generation_request_context config enabled filter_rules name privacy_mode regions sampling_rate"},
|
||||
{"listOrganizationMembers", "GET", "/organization/members", "listOrganizationMembers", "mgmt", "json", "?offset ?limit"},
|
||||
{"getOrganizationSettings", "GET", "/organization/settings", "getOrganizationSettings", "mgmt", "json", "-"},
|
||||
{"updateOrganizationSettings", "PATCH", "/organization/settings", "updateOrganizationSettings", "mgmt", "json", "is_filtered_model_catalog_enabled"},
|
||||
{"listPresets", "GET", "/presets", "listPresets", "api", "json", "?offset ?limit"},
|
||||
{"getPreset", "GET", "/presets/{slug}", "getPreset", "api", "json", "-"},
|
||||
{"createPresetsChatCompletions", "POST", "/presets/{slug}/chat/completions", "createPresetsChatCompletions", "api", "json", "cache_control debug deferred_tools frequency_penalty image_config logit_bias logprobs max_completion_tokens max_tokens messages metadata min_p modalities model models parallel_tool_calls plugins prediction presence_penalty prompt_cache_key prompt_cache_options provider reasoning reasoning_effort repetition_penalty response_format route seed service_tier session_id stop stop_server_tools_when stream stream_options temperature tool_choice tools top_a top_k top_logprobs top_p trace user"},
|
||||
{"createPresetsMessages", "POST", "/presets/{slug}/messages", "createPresetsMessages", "api", "json", "cache_control context_management deferred_tools fallbacks max_tokens messages metadata model models output_config plugins provider route safeguards service_tier session_id speed stop_sequences stop_server_tools_when stream system temperature thinking tool_choice tools top_k top_p trace user"},
|
||||
{"createPresetsResponses", "POST", "/presets/{slug}/responses", "createPresetsResponses", "api", "json", "background cache_control debug deferred_tools frequency_penalty image_config include input instructions max_output_tokens max_tool_calls metadata modalities model models parallel_tool_calls plugins presence_penalty previous_response_id prompt prompt_cache_key prompt_cache_options provider reasoning route safety_identifier service_tier session_id stop stop_server_tools_when store stream temperature text tool_choice tools top_k top_logprobs top_p trace truncation user"},
|
||||
{"listPresetVersions", "GET", "/presets/{slug}/versions", "listPresetVersions", "api", "json", "?offset ?limit"},
|
||||
{"getPresetVersion", "GET", "/presets/{slug}/versions/{version}", "getPresetVersion", "api", "json", "-"},
|
||||
{"listPrivateEndpoints", "GET", "/private-endpoints", "listPrivateEndpoints", "mgmt", "json", "-"},
|
||||
{"createPrivateEndpoint", "POST", "/private-endpoints", "createPrivateEndpoint", "mgmt", "json", "activate base_url declared_region declared_zdr model_permaslug pricing provider_slug upstream_model_id"},
|
||||
{"deletePrivateEndpoint", "DELETE", "/private-endpoints/{id}", "deletePrivateEndpoint", "mgmt", "json", "?draft_only"},
|
||||
{"getPrivateEndpoint", "GET", "/private-endpoints/{id}", "getPrivateEndpoint", "mgmt", "json", "-"},
|
||||
{"updatePrivateEndpoint", "PATCH", "/private-endpoints/{id}", "updatePrivateEndpoint", "mgmt", "json", "base_url declared_region declared_zdr upstream_model_id"},
|
||||
{"activatePrivateEndpoint", "POST", "/private-endpoints/{id}/activate", "activatePrivateEndpoint", "mgmt", "json", "-"},
|
||||
{"disablePrivateEndpoint", "POST", "/private-endpoints/{id}/disable", "disablePrivateEndpoint", "mgmt", "json", "-"},
|
||||
{"enablePrivateEndpoint", "POST", "/private-endpoints/{id}/enable", "enablePrivateEndpoint", "mgmt", "json", "-"},
|
||||
{"updatePrivateEndpointPricing", "PUT", "/private-endpoints/{id}/pricing", "updatePrivateEndpointPricing", "mgmt", "json", "pricing"},
|
||||
{"validatePrivateEndpoint", "POST", "/private-endpoints/{id}/validate", "validatePrivateEndpoint", "mgmt", "json", "workspace_id"},
|
||||
{"listProviders", "GET", "/providers", "listProviders", "none", "json", "-"},
|
||||
{"createRerank", "POST", "/rerank", "createRerank", "api", "json", "documents model provider query session_id top_n trace user"},
|
||||
{"createResponses", "POST", "/responses", "createResponses", "api", "json", "background cache_control debug deferred_tools frequency_penalty image_config include input instructions max_output_tokens max_tool_calls metadata modalities model models parallel_tool_calls plugins presence_penalty previous_response_id prompt prompt_cache_key prompt_cache_options provider reasoning route safety_identifier service_tier session_id stop stop_server_tools_when store stream temperature text tool_choice tools top_k top_logprobs top_p trace truncation user"},
|
||||
{"listScimGroupMappings", "GET", "/scim/group-mappings", "listScimGroupMappings", "mgmt", "json", "?offset ?limit"},
|
||||
{"createScimGroupMapping", "POST", "/scim/group-mappings", "createScimGroupMapping", "mgmt", "json", "role scim_group_id workspace_id"},
|
||||
{"deleteScimGroupMapping", "DELETE", "/scim/group-mappings/{id}", "deleteScimGroupMapping", "mgmt", "json", "?keep_members"},
|
||||
{"getScimGroupMapping", "GET", "/scim/group-mappings/{id}", "getScimGroupMapping", "mgmt", "json", "-"},
|
||||
{"updateScimGroupMapping", "PATCH", "/scim/group-mappings/{id}", "updateScimGroupMapping", "mgmt", "json", "role"},
|
||||
{"listScimGroups", "GET", "/scim/groups", "listScimGroups", "mgmt", "json", "?offset ?limit ?display_name ?external_id"},
|
||||
{"createScimSyncJob", "POST", "/scim/sync-jobs", "createScimSyncJob", "mgmt", "json", "-"},
|
||||
{"getScimSyncJob", "GET", "/scim/sync-jobs/{id}", "getScimSyncJob", "mgmt", "json", "-"},
|
||||
{"createSystemone", "POST", "/systemone", "createSystemone", "api", "json", "model provider questions session_id state trace user"},
|
||||
{"listTools", "GET", "/tools", "listTools", "api", "json", "?offset ?limit ?api_format"},
|
||||
{"getTool", "GET", "/tools/{name}", "getTool", "api", "json", "-"},
|
||||
{"listInternConnections", "GET", "/vault/interns/{internId}/connections", "listInternConnections", "api", "json", "-"},
|
||||
{"listInternEffectiveVaultSecrets", "GET", "/vault/interns/{internId}/effective-secrets", "listInternEffectiveVaultSecrets", "api", "json", "?limit ?offset"},
|
||||
{"listInternVaultSecrets", "GET", "/vault/interns/{internId}/secrets", "listInternVaultSecrets", "api", "json", "?limit ?offset"},
|
||||
{"copyVaultSecretsToIntern", "POST", "/vault/interns/{internId}/secrets/copy", "copyVaultSecretsToIntern", "api", "json", "names"},
|
||||
{"deleteInternVaultSecret", "DELETE", "/vault/interns/{internId}/secrets/{name}", "deleteInternVaultSecret", "api", "json", "-"},
|
||||
{"storeInternVaultSecret", "PUT", "/vault/interns/{internId}/secrets/{name}", "storeInternVaultSecret", "api", "json", "hosts value"},
|
||||
{"listVaultSecrets", "GET", "/vault/secrets", "listVaultSecrets", "api", "json", "?limit ?offset"},
|
||||
{"deleteVaultSecret", "DELETE", "/vault/secrets/{name}", "deleteVaultSecret", "api", "json", "-"},
|
||||
{"storeVaultSecret", "PUT", "/vault/secrets/{name}", "storeVaultSecret", "api", "json", "hosts value"},
|
||||
{"createVideos", "POST", "/videos", "createVideos", "api", "json", "aspect_ratio callback_url creativity duration frame_images generate_audio input_references model previous_job_id prompt provider resolution seed session_id size trace upscale_factor user"},
|
||||
{"listVideosModels", "GET", "/videos/models", "listVideosModels", "none", "json", "-"},
|
||||
{"getVideos", "GET", "/videos/{jobId}", "getVideos", "api", "json", "-"},
|
||||
{"listVideosContent", "GET", "/videos/{jobId}/content", "listVideosContent", "api", "raw", "?index"},
|
||||
{"listWorkspaces", "GET", "/workspaces", "listWorkspaces", "mgmt", "json", "?offset ?limit"},
|
||||
{"createWorkspace", "POST", "/workspaces", "createWorkspace", "mgmt", "json", "default_image_model default_provider_sort default_text_model description disabled_server_tools io_logging_api_key_ids io_logging_sampling_rate is_data_discount_logging_enabled is_observability_broadcast_enabled is_observability_io_logging_enabled name slug"},
|
||||
{"deleteWorkspace", "DELETE", "/workspaces/{id}", "deleteWorkspace", "mgmt", "json", "?confirm_default_workspace_deletion"},
|
||||
{"getWorkspace", "GET", "/workspaces/{id}", "getWorkspace", "mgmt", "json", "-"},
|
||||
{"updateWorkspace", "PATCH", "/workspaces/{id}", "updateWorkspace", "mgmt", "json", "default_image_model default_provider_sort default_text_model description disabled_server_tools io_logging_api_key_ids io_logging_sampling_rate is_data_discount_logging_enabled is_observability_broadcast_enabled is_observability_io_logging_enabled name slug"},
|
||||
{"listWorkspaceMembers", "GET", "/workspaces/{id}/members", "listWorkspaceMembers", "mgmt", "json", "?offset ?limit"},
|
||||
{"bulkAddWorkspaceMembers", "POST", "/workspaces/{id}/members/add", "bulkAddWorkspaceMembers", "mgmt", "json", "user_ids"},
|
||||
{"bulkRemoveWorkspaceMembers", "POST", "/workspaces/{id}/members/remove", "bulkRemoveWorkspaceMembers", "mgmt", "json", "user_ids"},
|
||||
{"listWorkspaceBudgets", "GET", "/workspaces/{workspace_ref}/budgets", "listWorkspaceBudgets", "mgmt", "json", "-"},
|
||||
{"deleteWorkspaceBudget", "DELETE", "/workspaces/{workspace_ref}/budgets/{interval}", "deleteWorkspaceBudget", "mgmt", "json", "-"},
|
||||
{"getWorkspaceBudget", "GET", "/workspaces/{workspace_ref}/budgets/{interval}", "getWorkspaceBudget", "mgmt", "json", "-"},
|
||||
{"upsertWorkspaceBudget", "PUT", "/workspaces/{workspace_ref}/budgets/{interval}", "upsertWorkspaceBudget", "mgmt", "json", "include_byok_in_budgets limit_usd"},
|
||||
};
|
||||
// clang-format on
|
||||
|
||||
inline constexpr std::size_t kCallCount = sizeof(kCalls) / sizeof(kCalls[0]);
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,54 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for the OpenRouter module. Never autotuned. Values marked
|
||||
// "vendor" mirror a limit the OpenRouter spec states; the rest are local
|
||||
// ceilings chosen by this module (a request over one is rejected before any
|
||||
// network traffic, a response over one is reported as truncated).
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct Caps {
|
||||
// Request shaping.
|
||||
static constexpr std::size_t kMaxFallbackModels = 16; // local: `models` array
|
||||
static constexpr std::size_t kMaxPlugins = 16; // local
|
||||
static constexpr std::size_t kMaxProviderList = 64; // local: order / only / ignore
|
||||
static constexpr std::size_t kMaxQuantizations = 16; // local
|
||||
static constexpr std::size_t kMaxStopSequences = 4; // vendor: "up to 4"
|
||||
static constexpr std::size_t kMaxQueryParams = 64; // local
|
||||
static constexpr std::size_t kMaxQueryValueBytes = 2048; // local
|
||||
static constexpr std::size_t kMaxPathArgBytes = 256; // matches Service::valid_id
|
||||
static constexpr std::size_t kMaxBulkIds = 1000; // local: bulk assign / remove lists
|
||||
static constexpr std::size_t kMaxEmbeddingInputs = 2048; // local
|
||||
static constexpr std::size_t kMaxRerankDocuments = 1000; // local
|
||||
static constexpr std::size_t kMaxBatchRequests = 10000; // local
|
||||
static constexpr std::size_t kMaxMetadataPairs = 16; // vendor: chat `metadata`
|
||||
static constexpr std::size_t kMaxMetadataKeyBytes = 64; // vendor
|
||||
static constexpr std::size_t kMaxMetadataValueBytes = 512; // vendor
|
||||
static constexpr int kMaxTopLogprobs = 20; // vendor: 0..20
|
||||
static constexpr std::size_t kMaxAppTitleBytes = 120; // local
|
||||
static constexpr std::size_t kMaxAppUrlBytes = 512; // local
|
||||
static constexpr std::size_t kMaxCategories = 2; // vendor: "up to 2 categories per request"
|
||||
static constexpr std::size_t kMaxCategoryBytes = 30; // vendor
|
||||
|
||||
// Listing. /models allows limit up to 1000 (vendor); a full walk stops at
|
||||
// kMaxModelPages pages and says so (never silent).
|
||||
static constexpr int kModelPageSize = 500;
|
||||
static constexpr int kMaxListLimit = 1000;
|
||||
static constexpr int kMaxKeysLimit = 100; // vendor: /keys limit max 100
|
||||
static constexpr std::size_t kMaxModelPages = 20;
|
||||
|
||||
// Streaming and parsing.
|
||||
static constexpr std::size_t kMaxStreamEvents = 200000;
|
||||
static constexpr std::size_t kMaxStreamTextBytes = 16u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxToolCalls = 128;
|
||||
static constexpr std::size_t kMaxReasoningDetails = 4096;
|
||||
static constexpr std::size_t kMaxChoices = 16;
|
||||
|
||||
// PKCE (RFC 7636): verifier 43..128 characters.
|
||||
static constexpr std::size_t kMinPkceVerifier = 43;
|
||||
static constexpr std::size_t kMaxPkceVerifier = 128;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,196 @@
|
||||
#include "openrouter/chat_model.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/server_tools.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) {
|
||||
throw llm::ApiError::invalid(kVendor, why);
|
||||
}
|
||||
|
||||
std::string image_url(const llm::ImageData& image) {
|
||||
if (!image.url.empty()) return image.url;
|
||||
if (image.bytes.empty() || image.mime.empty()) bad("image needs a url or bytes plus mime");
|
||||
return "data:" + image.mime + ";base64," + security::crypto::Base64::encode(image.bytes);
|
||||
}
|
||||
|
||||
Json text_part(const std::string& text) {
|
||||
Json p = Json::object();
|
||||
p.set("type", Json("text"));
|
||||
p.set("text", Json(text));
|
||||
return p;
|
||||
}
|
||||
|
||||
Json user_content(const llm::Message& m) {
|
||||
bool has_image = false;
|
||||
for (const llm::ContentPart& p : m.parts)
|
||||
if (p.image) has_image = true;
|
||||
if (!has_image) return Json(m.text());
|
||||
Json arr = Json::array();
|
||||
for (const llm::ContentPart& p : m.parts) {
|
||||
if (!p.text.empty()) arr.push(text_part(p.text));
|
||||
if (p.image) {
|
||||
Json img = Json::object();
|
||||
img.set("url", Json(image_url(*p.image)));
|
||||
Json part = Json::object();
|
||||
part.set("type", Json("image_url"));
|
||||
part.set("image_url", img);
|
||||
arr.push(part);
|
||||
}
|
||||
}
|
||||
return arr;
|
||||
}
|
||||
|
||||
void add_messages(const llm::ChatRequest& req, Json& out) {
|
||||
if (!req.system.empty()) {
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("system"));
|
||||
m.set("content", Json(req.system));
|
||||
out.push(m);
|
||||
}
|
||||
for (const llm::Message& msg : req.messages) {
|
||||
switch (msg.role) {
|
||||
case llm::Role::kSystem: {
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("system"));
|
||||
m.set("content", Json(msg.text()));
|
||||
if (!msg.name.empty()) m.set("name", Json(msg.name));
|
||||
out.push(m);
|
||||
break;
|
||||
}
|
||||
case llm::Role::kUser: {
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("user"));
|
||||
m.set("content", user_content(msg));
|
||||
if (!msg.name.empty()) m.set("name", Json(msg.name));
|
||||
out.push(m);
|
||||
break;
|
||||
}
|
||||
case llm::Role::kAssistant: {
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("assistant"));
|
||||
std::string text = msg.text();
|
||||
Json calls = Json::array();
|
||||
for (const llm::ContentPart& p : msg.parts) {
|
||||
if (!p.tool_call) continue;
|
||||
Json fn = Json::object();
|
||||
fn.set("name", Json(p.tool_call->name));
|
||||
fn.set("arguments", Json(p.tool_call->arguments.dump()));
|
||||
Json call = Json::object();
|
||||
call.set("id", Json(p.tool_call->id));
|
||||
call.set("type", Json("function"));
|
||||
call.set("function", fn);
|
||||
calls.push(call);
|
||||
}
|
||||
if (!text.empty() || calls.as_array().empty()) m.set("content", Json(text));
|
||||
else m.set("content", Json());
|
||||
if (!calls.as_array().empty()) m.set("tool_calls", calls);
|
||||
if (!msg.name.empty()) m.set("name", Json(msg.name));
|
||||
out.push(m);
|
||||
break;
|
||||
}
|
||||
case llm::Role::kTool: {
|
||||
for (const llm::ContentPart& p : msg.parts) {
|
||||
if (!p.tool_result) continue;
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("tool"));
|
||||
m.set("tool_call_id", Json(p.tool_result->call_id));
|
||||
m.set("content", Json(p.tool_result->content));
|
||||
out.push(m);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
OpenrouterChatModel::OpenrouterChatModel(std::shared_ptr<Gateway> gateway,
|
||||
std::string default_model, ChatModelDefaults defaults)
|
||||
: service_(std::move(gateway)),
|
||||
default_model_(std::move(default_model)),
|
||||
defaults_(std::move(defaults)) {}
|
||||
|
||||
Json OpenrouterChatModel::to_body(const llm::ChatRequest& req) const {
|
||||
ChatParams p;
|
||||
p.model = req.model.empty() ? default_model_ : req.model;
|
||||
p.models = defaults_.fallback_models;
|
||||
add_messages(req, p.messages);
|
||||
for (const llm::ToolSpec& t : req.tools) p.tools.push(ServerTools::function(t));
|
||||
if (!req.tool_choice.empty()) {
|
||||
if (req.tool_choice == "auto" || req.tool_choice == "none" || req.tool_choice == "required") {
|
||||
p.tool_choice = Json(req.tool_choice);
|
||||
} else {
|
||||
Json fn = Json::object();
|
||||
fn.set("name", Json(req.tool_choice));
|
||||
Json tc = Json::object();
|
||||
tc.set("type", Json("function"));
|
||||
tc.set("function", fn);
|
||||
p.tool_choice = tc;
|
||||
}
|
||||
}
|
||||
if (req.max_tokens) p.max_tokens = *req.max_tokens;
|
||||
p.temperature = req.temperature;
|
||||
p.top_p = req.top_p;
|
||||
p.stop = req.stop;
|
||||
if (req.response_schema.is_object()) {
|
||||
p.response_format = ResponseFormats::json_schema("response", req.response_schema, true);
|
||||
} else if (req.json_mode) {
|
||||
p.response_format = ResponseFormats::json_object();
|
||||
}
|
||||
p.provider = defaults_.provider;
|
||||
p.plugins = defaults_.plugins;
|
||||
p.route = defaults_.route;
|
||||
p.reasoning = defaults_.reasoning;
|
||||
if (!req.reasoning_effort.empty()) p.reasoning = ReasoningConfig::with_effort(req.reasoning_effort);
|
||||
|
||||
Json body = p.to_json(false);
|
||||
if (!req.extra.is_null()) {
|
||||
if (!req.extra.is_object()) bad("ChatRequest::extra must be an object");
|
||||
for (const auto& [k, v] : req.extra.as_object()) {
|
||||
if (!CallPlanner::accepts_field("sendChatCompletionRequest", k))
|
||||
bad("extra field not in the spec: " + k);
|
||||
body.set(k, v);
|
||||
}
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
llm::ChatResponse OpenrouterChatModel::to_response(const ChatResult& r) {
|
||||
llm::ChatResponse out;
|
||||
out.id = r.id;
|
||||
out.model = r.model;
|
||||
out.raw = r.raw;
|
||||
out.message.role = llm::Role::kAssistant;
|
||||
if (!r.choices.empty()) {
|
||||
const ChatChoiceResult& c = r.first();
|
||||
out.finish_reason = c.finish_reason;
|
||||
if (!c.content.empty() || !c.reasoning.empty()) {
|
||||
llm::ContentPart part = llm::ContentPart::of_text(c.content);
|
||||
part.reasoning = c.reasoning;
|
||||
out.message.parts.push_back(std::move(part));
|
||||
}
|
||||
for (const llm::ToolCall& call : c.tool_calls)
|
||||
out.message.parts.push_back(llm::ContentPart::of_call(call));
|
||||
}
|
||||
if (r.usage) out.usage = *r.usage;
|
||||
return out;
|
||||
}
|
||||
|
||||
llm::ChatResponse OpenrouterChatModel::generate(const llm::ChatRequest& request) {
|
||||
return to_response(ChatResult::parse(service_.create_raw(to_body(request))));
|
||||
}
|
||||
|
||||
void OpenrouterChatModel::stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
|
||||
service_.stream_body(to_body(request), on_chunk);
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,63 @@
|
||||
#pragma once
|
||||
|
||||
// OpenrouterChatModel: llm::ChatModel over POST /chat/completions. Maps the
|
||||
// neutral request (messages, tools, structured output, reasoning effort) to
|
||||
// the OpenRouter wire format and the response back, and carries the routing
|
||||
// defaults (provider preferences, fallback models, plugins, reasoning) that
|
||||
// every request of this model object should use.
|
||||
//
|
||||
// Mapping notes:
|
||||
// - `ChatRequest::extra` is merged last and may only hold fields the table
|
||||
// declares for chat completions; it overrides typed values.
|
||||
// - `ChatRequest::reasoning_effort` becomes the `reasoning` object
|
||||
// {"effort": ...} (overriding the default reasoning); the model's
|
||||
// reasoning text lands in ContentPart::reasoning.
|
||||
// - `reasoning_details` (needed to continue a tool loop with reasoning
|
||||
// models) are kept in ChatResponse::raw and cannot be replayed through the
|
||||
// neutral types; use ChatService with raw messages for that.
|
||||
// - A provider error mid-stream is thrown after the chunks before it.
|
||||
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "llm/chat_types.h"
|
||||
#include "openrouter/chat_service.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct ChatModelDefaults {
|
||||
std::optional<ProviderPreferences> provider;
|
||||
std::vector<std::string> fallback_models; // sent as `models`
|
||||
std::vector<PluginPtr> plugins;
|
||||
std::optional<ReasoningConfig> reasoning;
|
||||
std::string route; // "", "fallback", "sort"
|
||||
};
|
||||
|
||||
class OpenrouterChatModel : public llm::ChatModel {
|
||||
public:
|
||||
OpenrouterChatModel(std::shared_ptr<Gateway> gateway, std::string default_model,
|
||||
ChatModelDefaults defaults = {});
|
||||
|
||||
std::string vendor() const override { return "openrouter"; }
|
||||
llm::ChatResponse generate(const llm::ChatRequest& request) override;
|
||||
void stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
|
||||
|
||||
// The request body for a neutral request (validated; stream flag not set).
|
||||
clients::body::Json to_body(const llm::ChatRequest& request) const;
|
||||
// Neutral view of a parsed result.
|
||||
static llm::ChatResponse to_response(const ChatResult& result);
|
||||
|
||||
const std::string& default_model() const { return default_model_; }
|
||||
const ChatService& service() const { return service_; }
|
||||
|
||||
private:
|
||||
ChatService service_;
|
||||
std::string default_model_;
|
||||
ChatModelDefaults defaults_;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,218 @@
|
||||
#include "openrouter/chat_params.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
#include "openrouter/caps.h"
|
||||
#include "openrouter/model_ref.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
constexpr const char* kChatCall = "sendChatCompletionRequest";
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) {
|
||||
throw llm::ApiError::invalid(kVendor, "chat: " + why);
|
||||
}
|
||||
|
||||
bool one_of(const std::string& v, std::initializer_list<const char*> all) {
|
||||
return std::any_of(all.begin(), all.end(), [&](const char* s) { return v == s; });
|
||||
}
|
||||
|
||||
void range(const char* what, const std::optional<double>& v, double lo, double hi) {
|
||||
if (!v) return;
|
||||
if (!std::isfinite(*v) || *v < lo || *v > hi)
|
||||
bad(std::string(what) + " must be between " + std::to_string(lo) + " and " + std::to_string(hi));
|
||||
}
|
||||
|
||||
Json strings(const std::vector<std::string>& v) {
|
||||
Json a = Json::array();
|
||||
for (const std::string& s : v) a.push(Json(s));
|
||||
return a;
|
||||
}
|
||||
|
||||
void check_metadata(const Json& m) {
|
||||
if (m.is_null()) return;
|
||||
if (!m.is_object()) bad("metadata must be an object");
|
||||
if (m.as_object().size() > Caps::kMaxMetadataPairs) bad("metadata: at most 16 pairs");
|
||||
for (const auto& [k, v] : m.as_object()) {
|
||||
if (k.empty() || k.size() > Caps::kMaxMetadataKeyBytes) bad("metadata key must be 1..64 bytes");
|
||||
if (!v.is_string() || v.as_string().size() > Caps::kMaxMetadataValueBytes)
|
||||
bad("metadata values must be strings of at most 512 bytes");
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Json ResponseFormats::text() {
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("text"));
|
||||
return o;
|
||||
}
|
||||
|
||||
Json ResponseFormats::json_object() {
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("json_object"));
|
||||
return o;
|
||||
}
|
||||
|
||||
Json ResponseFormats::json_schema(const std::string& name, const Json& schema, bool strict,
|
||||
const std::string& description) {
|
||||
if (name.empty() || name.size() > 64) bad("json_schema name must be 1..64 characters");
|
||||
for (char c : name)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '_' || c == '-'))
|
||||
bad("json_schema name may only use letters, digits, _ and -");
|
||||
if (!schema.is_object()) bad("json_schema schema must be a JSON Schema object");
|
||||
Json js = Json::object();
|
||||
js.set("name", Json(name));
|
||||
js.set("schema", schema);
|
||||
js.set("strict", Json(strict));
|
||||
if (!description.empty()) js.set("description", Json(description));
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("json_schema"));
|
||||
o.set("json_schema", js);
|
||||
return o;
|
||||
}
|
||||
|
||||
Json ResponseFormats::grammar(const std::string& grammar) {
|
||||
if (grammar.empty()) bad("grammar must not be empty");
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("grammar"));
|
||||
o.set("grammar", Json(grammar));
|
||||
return o;
|
||||
}
|
||||
|
||||
void ChatParams::validate() const {
|
||||
if (!model.empty()) ModelRef::parse(model);
|
||||
if (models.size() > Caps::kMaxFallbackModels) bad("too many fallback models");
|
||||
for (const std::string& m : models) ModelRef::parse(m);
|
||||
if (!messages.is_array() || messages.as_array().empty()) bad("messages must be a non-empty array");
|
||||
if (messages.as_array().size() > llm::Caps::kMaxMessages) bad("too many messages");
|
||||
for (const Json& m : messages.as_array())
|
||||
if (!m.is_object() || !m.has("role")) bad("every message needs a role");
|
||||
|
||||
range("temperature", temperature, 0, 2);
|
||||
range("top_p", top_p, 0, 1);
|
||||
range("frequency_penalty", frequency_penalty, -2, 2);
|
||||
range("presence_penalty", presence_penalty, -2, 2);
|
||||
range("min_p", min_p, 0, 1);
|
||||
range("top_a", top_a, 0, 1);
|
||||
if (repetition_penalty && (!std::isfinite(*repetition_penalty) || *repetition_penalty < 0))
|
||||
bad("repetition_penalty must be >= 0");
|
||||
if (top_k && *top_k < 0) bad("top_k must be >= 0");
|
||||
if (max_tokens && *max_tokens < 1) bad("max_tokens must be >= 1");
|
||||
if (max_completion_tokens && *max_completion_tokens < 1) bad("max_completion_tokens must be >= 1");
|
||||
if (top_logprobs && (*top_logprobs < 0 || *top_logprobs > Caps::kMaxTopLogprobs))
|
||||
bad("top_logprobs must be 0..20");
|
||||
if (stop.size() > Caps::kMaxStopSequences) bad("at most 4 stop sequences");
|
||||
if (!logit_bias.is_null() && !logit_bias.is_object()) bad("logit_bias must be an object");
|
||||
|
||||
if (!response_format.is_null()) {
|
||||
const Json* t = response_format.is_object() ? response_format.find("type") : nullptr;
|
||||
if (!t || !t->is_string() ||
|
||||
!one_of(t->as_string(), {"text", "json_object", "json_schema", "grammar", "python"}))
|
||||
bad("response_format.type must be text json_object json_schema grammar or python");
|
||||
}
|
||||
if (!tools.is_array() || tools.as_array().size() > llm::Caps::kMaxTools) bad("tools must be an array of at most 256");
|
||||
for (const Json& t : tools.as_array())
|
||||
if (!t.is_object() || !t.has("type")) bad("every tool needs a type");
|
||||
if (!tool_choice.is_null() && !tool_choice.is_string() && !tool_choice.is_object())
|
||||
bad("tool_choice must be a string or an object");
|
||||
if (tool_choice.is_string() && !one_of(tool_choice.as_string(), {"none", "auto", "required"}))
|
||||
bad("tool_choice string must be none, auto or required");
|
||||
|
||||
if (provider) provider->validate();
|
||||
if (reasoning) reasoning->validate();
|
||||
if (reasoning && !reasoning_effort.empty()) bad("send reasoning or reasoning_effort, not both");
|
||||
if (!reasoning_effort.empty() && !ReasoningConfig::valid_effort(reasoning_effort))
|
||||
bad("reasoning_effort must be one of max xhigh high medium low minimal none");
|
||||
plugins_to_json(plugins);
|
||||
if (!route.empty() && route != "fallback" && route != "sort") bad("route must be fallback or sort");
|
||||
|
||||
if (modalities.size() > 3) bad("modalities: at most text, image, audio");
|
||||
for (const std::string& m : modalities)
|
||||
if (!one_of(m, {"text", "image", "audio"})) bad("modalities must be text, image or audio");
|
||||
if (!service_tier.empty() &&
|
||||
!one_of(service_tier, {"auto", "default", "fast", "flex", "priority", "scale", "ultrafast"}))
|
||||
bad("unknown service_tier");
|
||||
check_metadata(metadata);
|
||||
if (!trace.is_null() && !trace.is_object()) bad("trace must be an object");
|
||||
if (!debug.is_null() && !debug.is_object()) bad("debug must be an object");
|
||||
if (!extra.is_object()) bad("extra must be an object");
|
||||
}
|
||||
|
||||
Json ChatParams::to_json(bool stream) const {
|
||||
validate();
|
||||
Json o = Json::object();
|
||||
o.set("messages", messages);
|
||||
if (!model.empty()) o.set("model", Json(model));
|
||||
if (!models.empty()) o.set("models", strings(models));
|
||||
auto num = [&](const char* k, const std::optional<double>& v) { if (v) o.set(k, Json(*v)); };
|
||||
auto inum = [&](const char* k, const std::optional<int>& v) { if (v) o.set(k, Json(*v)); };
|
||||
num("temperature", temperature);
|
||||
num("top_p", top_p);
|
||||
num("frequency_penalty", frequency_penalty);
|
||||
num("presence_penalty", presence_penalty);
|
||||
num("repetition_penalty", repetition_penalty);
|
||||
num("min_p", min_p);
|
||||
num("top_a", top_a);
|
||||
inum("top_k", top_k);
|
||||
inum("max_tokens", max_tokens);
|
||||
inum("max_completion_tokens", max_completion_tokens);
|
||||
inum("seed", seed);
|
||||
inum("top_logprobs", top_logprobs);
|
||||
if (logprobs) o.set("logprobs", Json(*logprobs));
|
||||
if (!stop.empty()) o.set("stop", strings(stop));
|
||||
if (logit_bias.is_object()) o.set("logit_bias", logit_bias);
|
||||
if (response_format.is_object()) o.set("response_format", response_format);
|
||||
if (!tools.as_array().empty()) o.set("tools", tools);
|
||||
if (!tool_choice.is_null()) o.set("tool_choice", tool_choice);
|
||||
if (parallel_tool_calls) o.set("parallel_tool_calls", Json(*parallel_tool_calls));
|
||||
if (provider && !provider->empty()) o.set("provider", provider->to_json());
|
||||
if (reasoning && !reasoning->empty()) o.set("reasoning", reasoning->to_json());
|
||||
if (!reasoning_effort.empty()) o.set("reasoning_effort", Json(reasoning_effort));
|
||||
if (!plugins.empty()) o.set("plugins", plugins_to_json(plugins));
|
||||
if (!route.empty()) o.set("route", Json(route));
|
||||
if (!modalities.empty()) o.set("modalities", strings(modalities));
|
||||
if (!service_tier.empty()) o.set("service_tier", Json(service_tier));
|
||||
if (!user.empty()) o.set("user", Json(user));
|
||||
if (!session_id.empty()) o.set("session_id", Json(session_id));
|
||||
if (!prompt_cache_key.empty()) o.set("prompt_cache_key", Json(prompt_cache_key));
|
||||
if (metadata.is_object()) o.set("metadata", metadata);
|
||||
if (trace.is_object()) o.set("trace", trace);
|
||||
if (prediction.is_object()) o.set("prediction", prediction);
|
||||
if (debug.is_object()) o.set("debug", debug);
|
||||
for (const auto& [k, v] : extra.as_object()) {
|
||||
if (!CallPlanner::accepts_field(kChatCall, k)) bad("extra field not in the spec: " + k);
|
||||
if (o.has(k)) bad("extra field already set by a typed member: " + k);
|
||||
o.set(k, v);
|
||||
}
|
||||
if (stream) o.set("stream", Json(true));
|
||||
return o;
|
||||
}
|
||||
|
||||
ChatParams& ChatParams::add_message(const std::string& role, const std::string& text) {
|
||||
if (!one_of(role, {"system", "developer", "user", "assistant"}))
|
||||
bad("role must be system, developer, user or assistant (tool messages carry a tool_call_id: add them to messages directly)");
|
||||
Json m = Json::object();
|
||||
m.set("role", Json(role));
|
||||
m.set("content", Json(text));
|
||||
messages.push(m);
|
||||
return *this;
|
||||
}
|
||||
|
||||
ChatParams ChatParams::simple(const std::string& model, const std::string& user_text) {
|
||||
ChatParams p;
|
||||
p.model = model;
|
||||
p.add_message("user", user_text);
|
||||
return p;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,82 @@
|
||||
#pragma once
|
||||
|
||||
// ChatParams: a typed, validated builder for the body of POST
|
||||
// /chat/completions (spec schema ChatRequest). Every field the table
|
||||
// (calls.h) declares for sendChatCompletionRequest is either a typed member
|
||||
// here or can go through `extra`; `extra` keys are checked against the table,
|
||||
// so nothing the spec does not know can be sent.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "openrouter/plugins.h"
|
||||
#include "openrouter/provider_preferences.h"
|
||||
#include "openrouter/reasoning.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
// Values for the `response_format` field.
|
||||
struct ResponseFormats {
|
||||
static clients::body::Json text();
|
||||
static clients::body::Json json_object();
|
||||
// name: a-z A-Z 0-9 _ -, at most 64 characters. schema: a JSON Schema object.
|
||||
static clients::body::Json json_schema(const std::string& name,
|
||||
const clients::body::Json& schema, bool strict = true,
|
||||
const std::string& description = "");
|
||||
static clients::body::Json grammar(const std::string& grammar);
|
||||
};
|
||||
|
||||
struct ChatParams {
|
||||
std::string model; // "" : the account default
|
||||
std::vector<std::string> models; // fallback models, tried in order
|
||||
clients::body::Json messages = clients::body::Json::array(); // OpenAI-format messages
|
||||
|
||||
std::optional<double> temperature; // 0..2
|
||||
std::optional<double> top_p; // 0..1
|
||||
std::optional<double> frequency_penalty; // -2..2
|
||||
std::optional<double> presence_penalty; // -2..2
|
||||
std::optional<double> repetition_penalty;
|
||||
std::optional<double> min_p; // 0..1
|
||||
std::optional<double> top_a; // 0..1
|
||||
std::optional<int> top_k;
|
||||
std::optional<int> max_tokens;
|
||||
std::optional<int> max_completion_tokens;
|
||||
std::optional<int> seed;
|
||||
std::optional<bool> logprobs;
|
||||
std::optional<int> top_logprobs; // 0..20
|
||||
std::vector<std::string> stop; // at most 4
|
||||
clients::body::Json logit_bias; // null or object token -> bias
|
||||
|
||||
clients::body::Json response_format; // null or ResponseFormats::*
|
||||
clients::body::Json tools = clients::body::Json::array(); // ServerTools::*
|
||||
clients::body::Json tool_choice; // null, "none" "auto" "required", or an object
|
||||
std::optional<bool> parallel_tool_calls;
|
||||
|
||||
std::optional<ProviderPreferences> provider;
|
||||
std::optional<ReasoningConfig> reasoning;
|
||||
std::string reasoning_effort; // shorthand, not together with `reasoning`
|
||||
std::vector<PluginPtr> plugins;
|
||||
std::string route; // "", "fallback", "sort" (deprecated spec field)
|
||||
|
||||
std::vector<std::string> modalities; // text image audio
|
||||
std::string service_tier; // auto default fast flex priority scale ultrafast
|
||||
std::string user;
|
||||
std::string session_id;
|
||||
std::string prompt_cache_key;
|
||||
clients::body::Json metadata; // null or object of string pairs (16 pairs, 64/512 bytes)
|
||||
clients::body::Json trace; // null or {trace_id, trace_name, span_name, generation_name, parent_span_id}
|
||||
clients::body::Json prediction;
|
||||
clients::body::Json debug; // {"echo_upstream_body": true} (streaming only)
|
||||
clients::body::Json extra = clients::body::Json::object(); // other declared fields
|
||||
|
||||
// Validates and builds the request body. `stream` sets "stream": true.
|
||||
clients::body::Json to_json(bool stream = false) const;
|
||||
void validate() const;
|
||||
|
||||
ChatParams& add_message(const std::string& role, const std::string& text);
|
||||
static ChatParams simple(const std::string& model, const std::string& user_text);
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,127 @@
|
||||
#include "openrouter/chat_result.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
#include "openrouter/caps.h"
|
||||
#include "openrouter/errors.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
std::string str(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
int integer(const Json* o, const char* k) {
|
||||
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
|
||||
return v && v->is_number() ? static_cast<int>(v->as_int()) : 0;
|
||||
}
|
||||
|
||||
[[noreturn]] void bad_response(const std::string& why) {
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", why);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
llm::Usage parse_usage(const Json& u) {
|
||||
llm::Usage out;
|
||||
if (!u.is_object()) return out;
|
||||
out.input_tokens = integer(&u, "prompt_tokens");
|
||||
out.output_tokens = integer(&u, "completion_tokens");
|
||||
out.total_tokens = integer(&u, "total_tokens");
|
||||
if (out.total_tokens == 0) out.total_tokens = out.input_tokens + out.output_tokens;
|
||||
out.cached_tokens = integer(u.find("prompt_tokens_details"), "cached_tokens");
|
||||
out.reasoning_tokens = integer(u.find("completion_tokens_details"), "reasoning_tokens");
|
||||
return out;
|
||||
}
|
||||
|
||||
Json parse_tool_arguments(const std::string& text) {
|
||||
if (text.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
|
||||
try {
|
||||
Json j = Json::parse(text);
|
||||
if (j.is_object()) return j;
|
||||
} catch (const std::exception&) {
|
||||
}
|
||||
Json bad = Json::object();
|
||||
bad.set("_invalid_arguments", Json(text));
|
||||
return bad;
|
||||
}
|
||||
|
||||
ChatResult ChatResult::parse(const Json& doc) {
|
||||
if (!doc.is_object()) bad_response("chat response is not a JSON object");
|
||||
throw_if_error_body(doc);
|
||||
ChatResult r;
|
||||
r.raw = doc;
|
||||
r.id = str(doc, "id");
|
||||
r.model = str(doc, "model");
|
||||
r.provider = str(doc, "provider");
|
||||
r.service_tier = str(doc, "service_tier");
|
||||
if (const Json* c = doc.find("created"); c && c->is_number()) r.created = c->as_int();
|
||||
if (const Json* m = doc.find("openrouter_metadata"); m && m->is_object()) r.openrouter_metadata = *m;
|
||||
if (const Json* u = doc.find("usage"); u && u->is_object()) {
|
||||
r.usage = parse_usage(*u);
|
||||
if (const Json* c = u->find("cost"); c && c->is_number()) r.cost = c->as_number();
|
||||
if (const Json* b = u->find("is_byok"); b && b->is_bool()) r.is_byok = b->as_bool();
|
||||
if (const Json* d = u->find("cost_details"); d && d->is_object()) r.cost_details = *d;
|
||||
}
|
||||
const Json* choices = doc.find("choices");
|
||||
if (!choices || !choices->is_array()) bad_response("chat response has no choices array");
|
||||
if (choices->as_array().size() > Caps::kMaxChoices) bad_response("too many choices");
|
||||
for (const Json& c : choices->as_array()) {
|
||||
if (!c.is_object()) bad_response("choice is not an object");
|
||||
ChatChoiceResult cr;
|
||||
cr.index = integer(&c, "index");
|
||||
cr.finish_reason = str(c, "finish_reason");
|
||||
cr.native_finish_reason = str(c, "native_finish_reason");
|
||||
if (const Json* e = c.find("error"); e && e->is_object()) throw ErrorBody::parse(*e).to_api_error();
|
||||
if (cr.finish_reason == "error")
|
||||
throw llm::ApiError(kVendor, 502, "provider_unavailable", "generation ended with finish_reason error");
|
||||
if (const Json* m = c.find("message"); m && m->is_object()) {
|
||||
cr.message = *m;
|
||||
if (const Json* content = m->find("content")) {
|
||||
if (content->is_string()) {
|
||||
cr.content = content->as_string();
|
||||
} else if (content->is_array()) {
|
||||
for (const Json& part : content->as_array())
|
||||
if (part.is_object() && str(part, "type") == "text") cr.content += str(part, "text");
|
||||
}
|
||||
}
|
||||
cr.reasoning = str(*m, "reasoning");
|
||||
cr.refusal = str(*m, "refusal");
|
||||
if (const Json* rd = m->find("reasoning_details"); rd && rd->is_array()) {
|
||||
if (rd->as_array().size() > Caps::kMaxReasoningDetails) bad_response("too many reasoning details");
|
||||
cr.reasoning_details = *rd;
|
||||
}
|
||||
if (const Json* tcs = m->find("tool_calls"); tcs && tcs->is_array()) {
|
||||
if (tcs->as_array().size() > Caps::kMaxToolCalls) bad_response("too many tool calls");
|
||||
for (const Json& tc : tcs->as_array()) {
|
||||
const Json* fn = tc.is_object() ? tc.find("function") : nullptr;
|
||||
if (!fn || !fn->is_object()) bad_response("tool call without function");
|
||||
llm::ToolCall call;
|
||||
call.id = str(tc, "id");
|
||||
call.name = str(*fn, "name");
|
||||
call.arguments = parse_tool_arguments(str(*fn, "arguments"));
|
||||
cr.tool_calls.push_back(std::move(call));
|
||||
}
|
||||
}
|
||||
}
|
||||
r.choices.push_back(std::move(cr));
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
const ChatChoiceResult& ChatResult::first() const {
|
||||
if (choices.empty()) bad_response("chat response has no choice");
|
||||
return choices.front();
|
||||
}
|
||||
|
||||
std::string ChatResult::text() const { return choices.empty() ? "" : choices.front().content; }
|
||||
std::string ChatResult::finish_reason() const {
|
||||
return choices.empty() ? "" : choices.front().finish_reason;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
// ChatResult: typed view of a chat completion (spec schema ChatResult) and of
|
||||
// the document assembled from a stream. The raw document is kept for fields
|
||||
// the typed view omits (annotations, images, audio, logprobs, ...).
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/chat_types.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct ChatChoiceResult {
|
||||
int index = 0;
|
||||
std::string finish_reason; // tool_calls stop length content_filter error
|
||||
std::string native_finish_reason; // the provider's own value, when sent
|
||||
std::string content;
|
||||
std::string reasoning; // message.reasoning
|
||||
std::string refusal;
|
||||
clients::body::Json reasoning_details = clients::body::Json::array(); // replay with tool calls
|
||||
std::vector<llm::ToolCall> tool_calls;
|
||||
clients::body::Json message; // the raw message object
|
||||
};
|
||||
|
||||
struct ChatResult {
|
||||
std::string id; // "gen-..." : the generation id for GET /generation
|
||||
std::string model;
|
||||
std::string provider; // top level `provider` when sent
|
||||
std::string service_tier;
|
||||
long created = 0;
|
||||
std::vector<ChatChoiceResult> choices;
|
||||
std::optional<llm::Usage> usage;
|
||||
std::optional<double> cost; // usage.cost in USD (credits)
|
||||
bool is_byok = false;
|
||||
clients::body::Json cost_details;
|
||||
clients::body::Json openrouter_metadata; // present when router metadata is opted in
|
||||
clients::body::Json raw;
|
||||
|
||||
// Parses a non-streaming response body. A top-level `error`, or a choice
|
||||
// with finish_reason "error", throws llm::ApiError (the 200-with-error cases).
|
||||
static ChatResult parse(const clients::body::Json& doc);
|
||||
|
||||
const ChatChoiceResult& first() const; // throws when there is no choice
|
||||
std::string text() const;
|
||||
std::string finish_reason() const;
|
||||
};
|
||||
|
||||
// usage object -> neutral Usage (prompt, completion, total, cached, reasoning).
|
||||
llm::Usage parse_usage(const clients::body::Json& usage);
|
||||
// Parses function-call arguments (a JSON string). Invalid JSON is kept as
|
||||
// {"_invalid_arguments": "<text>"} so the caller can see and report it.
|
||||
clients::body::Json parse_tool_arguments(const std::string& text);
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,67 @@
|
||||
#include "openrouter/chat_service.h"
|
||||
|
||||
#include "openrouter/errors.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
constexpr const char* kChat = "sendChatCompletionRequest";
|
||||
constexpr const char* kLegacy = "createCompletions";
|
||||
|
||||
Json with_stream(const Json& body) {
|
||||
Json b = body;
|
||||
b.set("stream", Json(true));
|
||||
return b;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json ChatService::create_raw(const Json& body) const {
|
||||
Json b = body;
|
||||
if (const Json* s = b.find("stream"); s && s->is_bool() && s->as_bool())
|
||||
throw llm::ApiError::invalid(kVendor, "use stream() for stream:true");
|
||||
Json doc = run(kChat, {}, {}, b);
|
||||
throw_if_error_body(doc);
|
||||
return doc;
|
||||
}
|
||||
|
||||
ChatResult ChatService::create(const ChatParams& params) const {
|
||||
return ChatResult::parse(create_raw(params.to_json(false)));
|
||||
}
|
||||
|
||||
void ChatService::stream_events(const Json& body, const llm::SseDecoder::Sink& sink) const {
|
||||
Json b = with_stream(body);
|
||||
gateway_->stream(CallPlanner::plan(kChat, {}, {}, &b), sink);
|
||||
}
|
||||
|
||||
ChatResult ChatService::stream(const ChatParams& params,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const {
|
||||
return stream_body(params.to_json(true), on_chunk);
|
||||
}
|
||||
|
||||
ChatResult ChatService::stream_body(const Json& raw_body,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const {
|
||||
Json body = with_stream(raw_body);
|
||||
PlannedCall call = CallPlanner::plan(kChat, {}, {}, &body);
|
||||
ChatStreamParser parser;
|
||||
gateway_->stream(call, [&](const llm::SseEvent& ev) {
|
||||
for (const llm::ChatChunk& chunk : parser.feed(ev)) {
|
||||
if (on_chunk && !on_chunk(chunk)) return false;
|
||||
}
|
||||
return !parser.errored();
|
||||
});
|
||||
parser.throw_if_errored();
|
||||
return parser.result();
|
||||
}
|
||||
|
||||
Json ChatService::create_completion(const Json& body) const {
|
||||
Json doc = run(kLegacy, {}, {}, body);
|
||||
throw_if_error_body(doc);
|
||||
return doc;
|
||||
}
|
||||
|
||||
void ChatService::stream_completion(const Json& body, const llm::SseDecoder::Sink& sink) const {
|
||||
Json b = with_stream(body);
|
||||
gateway_->stream(CallPlanner::plan(kLegacy, {}, {}, &b), sink);
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,45 @@
|
||||
#pragma once
|
||||
|
||||
// ChatService: POST /chat/completions (and the legacy POST /completions),
|
||||
// non-streaming and streaming, over the OpenRouter-native types.
|
||||
|
||||
#include <functional>
|
||||
|
||||
#include "openrouter/chat_params.h"
|
||||
#include "openrouter/chat_result.h"
|
||||
#include "openrouter/chat_stream.h"
|
||||
#include "openrouter/gateway.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class ChatService : public Endpoint {
|
||||
public:
|
||||
explicit ChatService(std::shared_ptr<Gateway> gateway) : Endpoint(std::move(gateway)) {}
|
||||
|
||||
// Non-streaming. Throws llm::ApiError, including for the 200-with-error shapes.
|
||||
ChatResult create(const ChatParams& params) const;
|
||||
// The raw document (still checked for a top-level error).
|
||||
Json create_raw(const Json& body) const;
|
||||
|
||||
// Streaming. `on_chunk` returns false to stop reading (the connection is
|
||||
// abandoned, no error). Returns the assembled result. A provider error that
|
||||
// arrives mid-stream is thrown as llm::ApiError AFTER the chunks before it
|
||||
// were delivered; the assembled partial result is lost to the caller, so a
|
||||
// caller that wants the partial text keeps what it received in the callback.
|
||||
ChatResult stream(const ChatParams& params,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const;
|
||||
// Same, from an already built body (stream:true is set here). The body is
|
||||
// checked against the call table.
|
||||
ChatResult stream_body(const Json& body,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) const;
|
||||
// Raw events (decoded data chunks), for callers that parse themselves.
|
||||
void stream_events(const Json& body, const llm::SseDecoder::Sink& sink) const;
|
||||
|
||||
// Legacy text completions (POST /completions). `body` keys are checked
|
||||
// against the table row createCompletions (documented, not in the OpenAPI
|
||||
// document; see SPEC "Unverified").
|
||||
Json create_completion(const Json& body) const;
|
||||
void stream_completion(const Json& body, const llm::SseDecoder::Sink& sink) const;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,166 @@
|
||||
#include "openrouter/chat_stream.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
std::string str(const Json* o, const char* k) {
|
||||
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
[[noreturn]] void over_cap(const std::string& why) {
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", why);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::vector<llm::ChatChunk> ChatStreamParser::feed(const llm::SseEvent& event) {
|
||||
std::vector<llm::ChatChunk> out;
|
||||
if (++events_ > Caps::kMaxStreamEvents) over_cap("stream over the event cap");
|
||||
const Json& doc = event.data;
|
||||
if (!doc.is_object()) return out;
|
||||
|
||||
if (id_.empty()) id_ = str(&doc, "id");
|
||||
if (model_.empty()) model_ = str(&doc, "model");
|
||||
if (provider_.empty()) provider_ = str(&doc, "provider");
|
||||
if (service_tier_.empty()) service_tier_ = str(&doc, "service_tier");
|
||||
if (const Json* c = doc.find("created"); c && c->is_number() && created_ == 0) created_ = c->as_int();
|
||||
if (const Json* m = doc.find("openrouter_metadata"); m && m->is_object()) openrouter_metadata_ = *m;
|
||||
|
||||
if (const Json* u = doc.find("usage"); u && u->is_object()) {
|
||||
usage_ = parse_usage(*u);
|
||||
if (const Json* c = u->find("cost"); c && c->is_number()) cost_ = c->as_number();
|
||||
if (const Json* b = u->find("is_byok"); b && b->is_bool()) is_byok_ = b->as_bool();
|
||||
if (const Json* d = u->find("cost_details"); d && d->is_object()) cost_details_ = *d;
|
||||
llm::ChatChunk uc;
|
||||
uc.usage = usage_;
|
||||
out.push_back(std::move(uc));
|
||||
}
|
||||
|
||||
if (const Json* e = doc.find("error"); e && e->is_object() && !error_) {
|
||||
error_ = ErrorBody::parse(*e);
|
||||
llm::ChatChunk ec;
|
||||
ec.finish_reason = "error";
|
||||
out.push_back(std::move(ec));
|
||||
}
|
||||
|
||||
const Json* choices = doc.find("choices");
|
||||
if (!choices || !choices->is_array()) return out;
|
||||
if (choices->as_array().size() > Caps::kMaxChoices) over_cap("too many choices in a chunk");
|
||||
for (const Json& choice : choices->as_array()) {
|
||||
if (!choice.is_object()) continue;
|
||||
const Json* idx = choice.find("index");
|
||||
if (idx && idx->is_number() && idx->as_int() != 0) continue; // OpenRouter streams one choice
|
||||
if (const Json* delta = choice.find("delta"); delta && delta->is_object()) {
|
||||
llm::ChatChunk ch;
|
||||
std::string content = str(delta, "content");
|
||||
if (!content.empty()) {
|
||||
if (text_.size() + content.size() > Caps::kMaxStreamTextBytes) over_cap("stream text over the byte cap");
|
||||
text_ += content;
|
||||
ch.text_delta = content;
|
||||
}
|
||||
std::string reasoning = str(delta, "reasoning");
|
||||
if (!reasoning.empty()) {
|
||||
if (reasoning_.size() + reasoning.size() > Caps::kMaxStreamTextBytes)
|
||||
over_cap("stream reasoning over the byte cap");
|
||||
reasoning_ += reasoning;
|
||||
ch.reasoning_delta = reasoning;
|
||||
}
|
||||
refusal_ += str(delta, "refusal");
|
||||
if (const Json* rd = delta->find("reasoning_details"); rd && rd->is_array())
|
||||
for (const Json& d : rd->as_array()) {
|
||||
if (reasoning_details_.as_array().size() >= Caps::kMaxReasoningDetails)
|
||||
over_cap("too many reasoning details");
|
||||
reasoning_details_.push(d);
|
||||
}
|
||||
if (const Json* tcs = delta->find("tool_calls"); tcs && tcs->is_array()) {
|
||||
for (const Json& tc : tcs->as_array()) {
|
||||
if (!tc.is_object()) continue;
|
||||
const Json* ti = tc.find("index");
|
||||
int index = ti && ti->is_number() ? static_cast<int>(ti->as_int()) : 0;
|
||||
if (index < 0 || static_cast<std::size_t>(index) >= Caps::kMaxToolCalls) over_cap("tool call index over the cap");
|
||||
ToolBuild& tb = tools_[index];
|
||||
std::string id = str(&tc, "id");
|
||||
const Json* fn = tc.find("function");
|
||||
std::string name = str(fn, "name");
|
||||
std::string args = str(fn, "arguments");
|
||||
if (!id.empty()) tb.id = id;
|
||||
if (!name.empty()) tb.name = name;
|
||||
if (tb.arguments.size() + args.size() > Caps::kMaxStreamTextBytes) over_cap("tool arguments over the byte cap");
|
||||
tb.arguments += args;
|
||||
llm::ChatChunk tcc;
|
||||
tcc.tool_call_id = tb.id;
|
||||
tcc.tool_call_name = name; // only the fragment that carries the name
|
||||
tcc.tool_args_delta = args;
|
||||
out.push_back(std::move(tcc));
|
||||
}
|
||||
}
|
||||
if (!ch.text_delta.empty() || !ch.reasoning_delta.empty()) out.push_back(std::move(ch));
|
||||
}
|
||||
std::string fr = str(&choice, "finish_reason");
|
||||
if (!fr.empty()) {
|
||||
finish_reason_ = fr;
|
||||
native_finish_reason_ = str(&choice, "native_finish_reason");
|
||||
if (fr != "error" || !error_) {
|
||||
llm::ChatChunk fc;
|
||||
fc.finish_reason = fr;
|
||||
out.push_back(std::move(fc));
|
||||
}
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
void ChatStreamParser::throw_if_errored() const {
|
||||
if (error_) throw error_->to_api_error();
|
||||
}
|
||||
|
||||
ChatResult ChatStreamParser::result() const {
|
||||
ChatResult r;
|
||||
r.id = id_;
|
||||
r.model = model_;
|
||||
r.provider = provider_;
|
||||
r.service_tier = service_tier_;
|
||||
r.created = created_;
|
||||
r.usage = usage_;
|
||||
r.cost = cost_;
|
||||
r.is_byok = is_byok_;
|
||||
r.cost_details = cost_details_;
|
||||
r.openrouter_metadata = openrouter_metadata_;
|
||||
ChatChoiceResult c;
|
||||
c.index = 0;
|
||||
c.finish_reason = finish_reason_;
|
||||
c.native_finish_reason = native_finish_reason_;
|
||||
c.content = text_;
|
||||
c.reasoning = reasoning_;
|
||||
c.refusal = refusal_;
|
||||
c.reasoning_details = reasoning_details_;
|
||||
Json message = Json::object();
|
||||
message.set("role", Json("assistant"));
|
||||
message.set("content", Json(text_));
|
||||
for (const auto& [index, tb] : tools_) {
|
||||
(void)index;
|
||||
llm::ToolCall call;
|
||||
call.id = tb.id;
|
||||
call.name = tb.name;
|
||||
call.arguments = parse_tool_arguments(tb.arguments);
|
||||
c.tool_calls.push_back(std::move(call));
|
||||
}
|
||||
c.message = message;
|
||||
r.choices.push_back(std::move(c));
|
||||
Json raw = Json::object();
|
||||
raw.set("id", Json(id_));
|
||||
raw.set("model", Json(model_));
|
||||
raw.set("assembled_from_stream", Json(true));
|
||||
r.raw = raw;
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,70 @@
|
||||
#pragma once
|
||||
|
||||
// ChatStreamParser: turns the SSE events of a streamed chat completion
|
||||
// (spec schema ChatStreamChunk) into neutral ChatChunks and assembles the
|
||||
// final ChatResult. Pure logic over decoded events, so it is testable with a
|
||||
// documented transcript and no network.
|
||||
//
|
||||
// Wire facts handled (https://openrouter.ai/docs/api-reference/streaming and
|
||||
// /errors):
|
||||
// - SSE comment lines such as ": OPENROUTER PROCESSING" are heartbeats and
|
||||
// are dropped by llm::SseDecoder before they reach this parser.
|
||||
// - "data: [DONE]" ends the stream (consumed by the decoder).
|
||||
// - the final data chunk carries `usage` (often with an empty `choices`).
|
||||
// - a mid-stream error is a normal chunk with a top-level `error` object
|
||||
// and a choice with finish_reason "error"; the HTTP status stays 200.
|
||||
// - tool calls arrive as fragments keyed by `index`; the first fragment has
|
||||
// id and name, later ones only argument text.
|
||||
|
||||
#include <map>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/chat_types.h"
|
||||
#include "llm/sse.h"
|
||||
#include "openrouter/chat_result.h"
|
||||
#include "openrouter/errors.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class ChatStreamParser {
|
||||
public:
|
||||
// Returns the neutral chunks this event produced (possibly none). Never
|
||||
// throws for a provider error: it is recorded (see error()) and a chunk
|
||||
// with finish_reason "error" is returned. Throws llm::ApiError when a
|
||||
// static cap (events, text bytes, tool calls) is exceeded.
|
||||
std::vector<llm::ChatChunk> feed(const llm::SseEvent& event);
|
||||
|
||||
bool errored() const { return error_.has_value(); }
|
||||
const std::optional<ErrorBody>& error() const { return error_; }
|
||||
// Throws the recorded error as llm::ApiError, if any.
|
||||
void throw_if_errored() const;
|
||||
|
||||
std::size_t event_count() const { return events_; }
|
||||
// The assembled result (valid after any number of events).
|
||||
ChatResult result() const;
|
||||
|
||||
private:
|
||||
struct ToolBuild {
|
||||
std::string id;
|
||||
std::string name;
|
||||
std::string arguments;
|
||||
};
|
||||
|
||||
std::string id_, model_, provider_, service_tier_;
|
||||
long created_ = 0;
|
||||
std::string text_, reasoning_, refusal_, finish_reason_, native_finish_reason_;
|
||||
clients::body::Json reasoning_details_ = clients::body::Json::array();
|
||||
std::map<int, ToolBuild> tools_;
|
||||
std::optional<llm::Usage> usage_;
|
||||
std::optional<double> cost_;
|
||||
bool is_byok_ = false;
|
||||
clients::body::Json cost_details_;
|
||||
clients::body::Json openrouter_metadata_;
|
||||
std::optional<ErrorBody> error_;
|
||||
std::size_t events_ = 0;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,163 @@
|
||||
#include "openrouter/errors.h"
|
||||
|
||||
#include <cstdlib>
|
||||
|
||||
#include "openrouter/call_plan.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
struct TypedError {
|
||||
const char* type;
|
||||
int status;
|
||||
};
|
||||
|
||||
// metadata.error_type values and their HTTP statuses, from the "Typed error
|
||||
// codes" tables of the errors guide.
|
||||
constexpr TypedError kTyped[] = {
|
||||
{"context_length_exceeded", 400}, {"max_tokens_exceeded", 400},
|
||||
{"token_limit_exceeded", 400}, {"string_too_long", 400},
|
||||
{"authentication", 401}, {"permission_denied", 403},
|
||||
{"payment_required", 402}, {"rate_limit_exceeded", 429},
|
||||
{"provider_overloaded", 503}, {"provider_unavailable", 502},
|
||||
{"invalid_request", 400}, {"invalid_prompt", 400},
|
||||
{"not_found", 404}, {"precondition_failed", 412},
|
||||
{"payload_too_large", 413}, {"unprocessable", 422},
|
||||
{"content_policy_violation", 403}, {"refusal", 403},
|
||||
{"invalid_image", 400}, {"image_too_large", 400},
|
||||
{"image_too_small", 400}, {"unsupported_image_format", 400},
|
||||
{"image_not_found", 404}, {"image_download_failed", 400},
|
||||
{"server", 500}, {"timeout", 504},
|
||||
{"unmapped", 500},
|
||||
};
|
||||
|
||||
std::string string_of(const Json* v) {
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
const char* failure_kind_name(FailureKind kind) {
|
||||
switch (kind) {
|
||||
case FailureKind::kInvalidRequest: return "invalid_request";
|
||||
case FailureKind::kUnauthorized: return "unauthorized";
|
||||
case FailureKind::kCredits: return "insufficient_credits";
|
||||
case FailureKind::kForbidden: return "forbidden";
|
||||
case FailureKind::kNotFound: return "not_found";
|
||||
case FailureKind::kTimeout: return "timeout";
|
||||
case FailureKind::kRateLimited: return "rate_limited";
|
||||
case FailureKind::kProviderFailed: return "provider_failed";
|
||||
case FailureKind::kNoProvider: return "no_provider";
|
||||
case FailureKind::kOverloaded: return "overloaded";
|
||||
case FailureKind::kServer: return "server";
|
||||
case FailureKind::kOther: return "other";
|
||||
}
|
||||
return "other";
|
||||
}
|
||||
|
||||
FailureKind classify_status(int s) {
|
||||
switch (s) {
|
||||
case 400: case 413: case 422: return FailureKind::kInvalidRequest;
|
||||
case 401: return FailureKind::kUnauthorized;
|
||||
case 402: return FailureKind::kCredits;
|
||||
case 403: return FailureKind::kForbidden;
|
||||
case 404: return FailureKind::kNotFound;
|
||||
case 408: case 504: case 524: return FailureKind::kTimeout;
|
||||
case 429: return FailureKind::kRateLimited;
|
||||
case 502: return FailureKind::kProviderFailed;
|
||||
case 503: return FailureKind::kNoProvider;
|
||||
case 529: return FailureKind::kOverloaded;
|
||||
default: break;
|
||||
}
|
||||
return s >= 500 ? FailureKind::kServer : FailureKind::kOther;
|
||||
}
|
||||
|
||||
bool failure_retryable(FailureKind k) {
|
||||
return k == FailureKind::kTimeout || k == FailureKind::kRateLimited ||
|
||||
k == FailureKind::kProviderFailed || k == FailureKind::kNoProvider ||
|
||||
k == FailureKind::kOverloaded;
|
||||
}
|
||||
|
||||
ErrorInfo describe(const llm::ApiError& e) {
|
||||
ErrorInfo info;
|
||||
info.kind = classify_status(e.http_status());
|
||||
// A typed error_type refines the status (a 403 refusal vs a budget block).
|
||||
if (e.type() == "rate_limit_exceeded") info.kind = FailureKind::kRateLimited;
|
||||
if (e.type() == "payment_required") info.kind = FailureKind::kCredits;
|
||||
info.retryable = failure_retryable(info.kind);
|
||||
switch (info.kind) {
|
||||
case FailureKind::kCredits:
|
||||
info.hint = "Add credits or raise the key limit, then retry (402 is not a wait-and-retry case).";
|
||||
break;
|
||||
case FailureKind::kRateLimited:
|
||||
info.hint = "Rate limited: wait (Retry-After) and retry; free models have daily limits.";
|
||||
break;
|
||||
case FailureKind::kProviderFailed:
|
||||
info.hint = "The chosen model or provider failed: retry, or add fallbacks with `models` and provider.allow_fallbacks.";
|
||||
break;
|
||||
case FailureKind::kNoProvider:
|
||||
info.hint = "No provider meets the routing requirements: relax provider.order/only/require_parameters/max_price.";
|
||||
break;
|
||||
case FailureKind::kForbidden:
|
||||
info.hint = "Blocked by a guardrail, moderation or a budget: see the error message.";
|
||||
break;
|
||||
case FailureKind::kUnauthorized:
|
||||
info.hint = "The key is missing, invalid, disabled or expired.";
|
||||
break;
|
||||
case FailureKind::kInvalidRequest:
|
||||
info.hint = "The request was rejected as malformed.";
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return info;
|
||||
}
|
||||
|
||||
bool is_known_error_type(const std::string& type) { return status_for_error_type(type) != 0; }
|
||||
|
||||
int status_for_error_type(const std::string& type) {
|
||||
for (const TypedError& t : kTyped)
|
||||
if (type == t.type) return t.status;
|
||||
return 0;
|
||||
}
|
||||
|
||||
ErrorBody ErrorBody::parse(const Json& err) {
|
||||
ErrorBody b;
|
||||
if (!err.is_object()) {
|
||||
b.message = err.is_string() ? err.as_string() : "unknown error";
|
||||
return b;
|
||||
}
|
||||
if (const Json* c = err.find("code")) {
|
||||
if (c->is_number()) b.code = static_cast<int>(c->as_int());
|
||||
else if (c->is_string()) b.code = std::atoi(c->as_string().c_str());
|
||||
}
|
||||
b.message = string_of(err.find("message"));
|
||||
if (const Json* m = err.find("metadata"); m && m->is_object()) {
|
||||
b.metadata = *m;
|
||||
b.error_type = string_of(m->find("error_type"));
|
||||
b.provider_code = string_of(m->find("provider_code"));
|
||||
b.provider_name = string_of(m->find("provider_name"));
|
||||
}
|
||||
// Anthropic skin: error_type sits on the error object itself.
|
||||
if (b.error_type.empty()) b.error_type = string_of(err.find("error_type"));
|
||||
return b;
|
||||
}
|
||||
|
||||
llm::ApiError ErrorBody::to_api_error() const {
|
||||
int status = code;
|
||||
if (status < 400 || status > 599) status = status_for_error_type(error_type);
|
||||
if (status < 400 || status > 599) status = 502;
|
||||
std::string type = !error_type.empty() ? error_type : std::to_string(code);
|
||||
return llm::ApiError(kVendor, status, type, message.empty() ? "provider error" : message);
|
||||
}
|
||||
|
||||
void throw_if_error_body(const Json& doc) {
|
||||
if (!doc.is_object()) return;
|
||||
const Json* err = doc.find("error");
|
||||
if (err && err->is_object()) throw ErrorBody::parse(*err).to_api_error();
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
// OpenRouter error semantics (https://openrouter.ai/docs/api-reference/errors).
|
||||
// Envelope: {"error":{"code":402,"message":"...","metadata":{...}}} with a
|
||||
// NUMERIC code. Provider errors add metadata.error_type (a stable typed
|
||||
// vocabulary), provider_code, raw, provider_name. A 200 response can still
|
||||
// carry an error: a mid-stream error arrives as an SSE chunk with a top-level
|
||||
// `error` and finish_reason "error"; a non-streaming provider failure arrives
|
||||
// as a 200 JSON body holding only `error`. Both are mapped to llm::ApiError
|
||||
// here so callers have one failure path.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
enum class FailureKind {
|
||||
kInvalidRequest, // 400, 413, 422
|
||||
kUnauthorized, // 401
|
||||
kCredits, // 402 insufficient credits (or an in-flight budget)
|
||||
kForbidden, // 403 guardrail, moderation, budget, permission
|
||||
kNotFound, // 404
|
||||
kTimeout, // 408, 504, 524
|
||||
kRateLimited, // 429
|
||||
kProviderFailed, // 502 model down or invalid provider response
|
||||
kNoProvider, // 503 no provider meets the routing requirements
|
||||
kOverloaded, // 529
|
||||
kServer, // other 5xx
|
||||
kOther,
|
||||
};
|
||||
|
||||
const char* failure_kind_name(FailureKind kind);
|
||||
FailureKind classify_status(int http_status);
|
||||
// True when waiting and retrying can help (408, 429, 502, 503, 504, 524, 529).
|
||||
bool failure_retryable(FailureKind kind);
|
||||
|
||||
struct ErrorInfo {
|
||||
FailureKind kind = FailureKind::kOther;
|
||||
bool retryable = false;
|
||||
std::string hint; // One line for people reading logs.
|
||||
};
|
||||
|
||||
// Never throws. Based on the status and the vendor type string only (the
|
||||
// foundation ApiError does not carry the error metadata or Retry-After).
|
||||
ErrorInfo describe(const llm::ApiError& error);
|
||||
|
||||
// The typed error_type vocabulary from the docs (metadata.error_type).
|
||||
bool is_known_error_type(const std::string& type);
|
||||
// HTTP status the docs assign to an error_type, 0 when unknown.
|
||||
int status_for_error_type(const std::string& type);
|
||||
|
||||
// The `error` object of a chunk or body.
|
||||
struct ErrorBody {
|
||||
int code = 0;
|
||||
std::string message;
|
||||
std::string error_type; // metadata.error_type
|
||||
std::string provider_code; // metadata.provider_code
|
||||
std::string provider_name; // metadata.provider_name
|
||||
clients::body::Json metadata;
|
||||
|
||||
static ErrorBody parse(const clients::body::Json& error_object);
|
||||
// vendor "openrouter"; type is error_type when present, else the code.
|
||||
llm::ApiError to_api_error() const;
|
||||
};
|
||||
|
||||
// Throws when `doc` is a JSON object with a top-level `error` object (the
|
||||
// 200-with-error shapes). Does nothing otherwise.
|
||||
void throw_if_error_body(const clients::body::Json& doc);
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,260 @@
|
||||
#include "openrouter/gateway.h"
|
||||
|
||||
#include <cctype>
|
||||
|
||||
#include "clients/body/form.h"
|
||||
#include "clients/transports/http_transport.h"
|
||||
#include "llm/caps.h"
|
||||
#include "llm/config.h"
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
|
||||
|
||||
bool safe_header_text(const std::string& s) {
|
||||
for (unsigned char c : s)
|
||||
if (c < 0x20 || c == 0x7f) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool valid_category(const std::string& c) {
|
||||
if (c.empty() || c.size() > Caps::kMaxCategoryBytes) return false;
|
||||
for (char ch : c)
|
||||
if (!((ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '-')) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool base_url_ok(const std::string& url) {
|
||||
if (url.rfind("https://", 0) == 0) return true;
|
||||
return url.rfind("http://127.0.0.1", 0) == 0 || url.rfind("http://localhost", 0) == 0;
|
||||
}
|
||||
|
||||
std::string base_url_from_env() {
|
||||
std::string url = llm::Config::get("OPENROUTER_BASE_URL");
|
||||
if (url.empty()) return kDefaultBaseUrl;
|
||||
while (!url.empty() && url.back() == '/') url.pop_back();
|
||||
if (!base_url_ok(url)) bad("OPENROUTER_BASE_URL must be https (or loopback http)");
|
||||
return url;
|
||||
}
|
||||
|
||||
clients::ClientConfig anon_config() {
|
||||
clients::ClientConfig c;
|
||||
c.persist_cookies = false;
|
||||
return c;
|
||||
}
|
||||
|
||||
llm::ApiClientConfig client_config(const std::string& env, const std::string& base,
|
||||
const llm::Headers& headers) {
|
||||
llm::ApiClientConfig c;
|
||||
c.vendor = kVendor;
|
||||
c.base_url = base;
|
||||
c.key_env = env;
|
||||
c.auth = llm::AuthStyle::kBearer;
|
||||
c.default_headers = headers;
|
||||
return c;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Attribution
|
||||
|
||||
Attribution Attribution::from_env() {
|
||||
Attribution a;
|
||||
a.referer = llm::Config::get("OPENROUTER_APP_URL");
|
||||
a.title = llm::Config::get("OPENROUTER_APP_TITLE");
|
||||
a.categories = llm::Config::get("OPENROUTER_APP_CATEGORIES");
|
||||
return a;
|
||||
}
|
||||
|
||||
void Attribution::validate() const {
|
||||
if (referer.size() > Caps::kMaxAppUrlBytes || !safe_header_text(referer))
|
||||
bad("app url is too long or has control characters");
|
||||
if (title.size() > Caps::kMaxAppTitleBytes || !safe_header_text(title))
|
||||
bad("app title is too long or has control characters");
|
||||
if (!categories.empty()) {
|
||||
std::size_t count = 0, pos = 0;
|
||||
while (pos <= categories.size()) {
|
||||
std::size_t comma = categories.find(',', pos);
|
||||
std::string one = categories.substr(pos, comma == std::string::npos ? comma : comma - pos);
|
||||
if (!valid_category(one)) bad("category must be lowercase a-z 0-9 hyphen, at most 30 chars");
|
||||
++count;
|
||||
if (comma == std::string::npos) break;
|
||||
pos = comma + 1;
|
||||
}
|
||||
if (count > Caps::kMaxCategories) bad("at most 2 categories per request");
|
||||
}
|
||||
}
|
||||
|
||||
llm::Headers Attribution::headers() const {
|
||||
validate();
|
||||
llm::Headers h;
|
||||
if (!referer.empty()) h["HTTP-Referer"] = referer;
|
||||
if (!title.empty()) {
|
||||
h["X-OpenRouter-Title"] = title;
|
||||
h["X-Title"] = title; // legacy name, still accepted by OpenRouter
|
||||
}
|
||||
if (!categories.empty()) h["X-OpenRouter-Categories"] = categories;
|
||||
if (hidden) h["X-OpenRouter-App-Visibility"] = "hidden";
|
||||
if (router_metadata) h["X-OpenRouter-Metadata"] = "enabled";
|
||||
return h;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// AnonymousClient
|
||||
|
||||
AnonymousClient::AnonymousClient(std::unique_ptr<clients::Transport> transport,
|
||||
std::string base_url, llm::Headers default_headers)
|
||||
: base_url_(std::move(base_url)),
|
||||
headers_(std::move(default_headers)),
|
||||
client_(std::move(transport), anon_config()) {}
|
||||
|
||||
llm::RawResponse AnonymousClient::send(clients::Method method, const std::string& target,
|
||||
const std::string* body,
|
||||
const std::string& content_type,
|
||||
const llm::Headers& extra) {
|
||||
if (target.empty() || target.front() != '/' || target.find("..") != std::string::npos)
|
||||
bad("bad request path");
|
||||
try {
|
||||
clients::ClientRequest req(method, clients::Url::parse(base_url_ + target));
|
||||
for (const auto& [k, v] : headers_) req.header(k, v);
|
||||
for (const auto& [k, v] : extra) req.header(k, v);
|
||||
if (body) {
|
||||
if (body->size() > llm::Caps::kMaxRequestBytes) bad("request body over the byte cap");
|
||||
req.bytes(*body, content_type);
|
||||
}
|
||||
clients::ClientResponse res = client_.send_blocking(std::move(req));
|
||||
if (!res.is_success()) throw llm::error_from_response(kVendor, res);
|
||||
if (res.body().size() > llm::Caps::kMaxResponseBytes)
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "response over the byte cap");
|
||||
llm::RawResponse out;
|
||||
out.status = res.status();
|
||||
out.body = res.body();
|
||||
out.content_type = res.content_type();
|
||||
out.headers = res.headers();
|
||||
return out;
|
||||
} catch (const llm::ApiError&) {
|
||||
throw;
|
||||
} catch (const std::exception& e) {
|
||||
throw llm::ApiError(kVendor, 502, "UNAVAILABLE", e.what());
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Gateway
|
||||
|
||||
Gateway::Gateway(std::shared_ptr<llm::ApiClient> api,
|
||||
std::shared_ptr<llm::ApiClient> management,
|
||||
std::shared_ptr<AnonymousClient> anonymous)
|
||||
: api_(std::move(api)), management_(std::move(management)), anonymous_(std::move(anonymous)) {
|
||||
if (!api_ || !management_ || !anonymous_) bad("gateway needs all three clients");
|
||||
}
|
||||
|
||||
std::shared_ptr<Gateway> Gateway::from_env(const Attribution& attribution) {
|
||||
llm::Headers headers = attribution.headers();
|
||||
std::string base = base_url_from_env();
|
||||
auto api = llm::ApiClient::from_env(client_config(kApiKeyEnv, base, headers));
|
||||
auto mgmt = llm::ApiClient::from_env(client_config(kManagementKeyEnv, base, headers));
|
||||
auto anon = std::make_shared<AnonymousClient>(
|
||||
std::make_unique<clients::transports::HttpTransport>(), base, headers);
|
||||
return std::make_shared<Gateway>(std::move(api), std::move(mgmt), std::move(anon));
|
||||
}
|
||||
|
||||
bool Gateway::ready(KeyKind kind) const {
|
||||
switch (kind) {
|
||||
case KeyKind::kApi: return api_->configured();
|
||||
case KeyKind::kManagement: return management_->configured();
|
||||
case KeyKind::kNone: return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
llm::ApiClient& Gateway::client_for(KeyKind kind) const {
|
||||
return kind == KeyKind::kManagement ? *management_ : *api_;
|
||||
}
|
||||
|
||||
Json Gateway::call(const PlannedCall& c, const llm::Headers& extra) const {
|
||||
const std::string fmt = c.fmt();
|
||||
if (fmt == "multipart" || fmt == "raw") bad(c.name() + " is a " + fmt + " call");
|
||||
const std::string target = c.target();
|
||||
if (fmt == "form") {
|
||||
clients::body::UrlEncodedForm form;
|
||||
if (c.has_body)
|
||||
for (const auto& [k, v] : c.body.as_object()) {
|
||||
if (!v.is_string()) bad("form field must be a string: " + k);
|
||||
form.add(k, v.as_string());
|
||||
}
|
||||
std::string encoded = form.encode();
|
||||
llm::RawResponse raw;
|
||||
if (c.key == KeyKind::kNone) {
|
||||
raw = anonymous_->send(c.method, target, &encoded,
|
||||
clients::body::UrlEncodedForm::content_type(), extra);
|
||||
} else {
|
||||
raw = client_for(c.key).request_raw(c.method, target, &encoded,
|
||||
clients::body::UrlEncodedForm::content_type(), extra);
|
||||
}
|
||||
if (raw.body.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
|
||||
try {
|
||||
return Json::parse(raw.body);
|
||||
} catch (const std::exception& e) {
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", e.what());
|
||||
}
|
||||
}
|
||||
if (c.key == KeyKind::kNone) {
|
||||
std::string text = c.has_body ? c.body.dump() : "";
|
||||
llm::RawResponse raw = anonymous_->send(c.method, target, c.has_body ? &text : nullptr,
|
||||
"application/json", extra);
|
||||
if (raw.body.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
|
||||
try {
|
||||
return Json::parse(raw.body);
|
||||
} catch (const std::exception& e) {
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", e.what());
|
||||
}
|
||||
}
|
||||
return client_for(c.key).request(c.method, target, c.has_body ? &c.body : nullptr, extra);
|
||||
}
|
||||
|
||||
llm::RawResponse Gateway::call_raw(const PlannedCall& c, const llm::Headers& extra) const {
|
||||
if (c.fmt() != "raw") bad(c.name() + " is not a raw call");
|
||||
std::string text = c.has_body ? c.body.dump() : "";
|
||||
const std::string* body = c.has_body ? &text : nullptr;
|
||||
const char* type = c.has_body ? "application/json" : "";
|
||||
if (c.key == KeyKind::kNone) return anonymous_->send(c.method, c.target(), body, type, extra);
|
||||
return client_for(c.key).request_raw(c.method, c.target(), body, type, extra);
|
||||
}
|
||||
|
||||
Json Gateway::call_multipart(const PlannedCall& c, const clients::body::MultipartBody& form,
|
||||
const llm::Headers& extra) const {
|
||||
if (c.fmt() != "multipart") bad(c.name() + " is not a multipart call");
|
||||
if (c.key == KeyKind::kNone) bad("multipart calls need a key");
|
||||
return client_for(c.key).post_multipart(c.target(), form, extra);
|
||||
}
|
||||
|
||||
void Gateway::stream(const PlannedCall& c, const llm::SseDecoder::Sink& sink,
|
||||
const llm::Headers& extra) const {
|
||||
if (!c.has_body || c.method != clients::Method::kPost) bad(c.name() + " cannot stream");
|
||||
if (c.key == KeyKind::kNone) bad("streaming calls need a key");
|
||||
client_for(c.key).stream(c.target(), c.body, sink, extra);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Endpoint
|
||||
|
||||
Endpoint::Endpoint(std::shared_ptr<Gateway> gateway, KeyKind primary)
|
||||
: llm::Service(gateway->api()), gateway_(std::move(gateway)), primary_(primary) {}
|
||||
|
||||
Json Endpoint::run(std::string_view name, const PathArgs& args, const Query& query,
|
||||
const Json* body, const llm::Headers& extra) const {
|
||||
return gateway_->call(CallPlanner::plan(name, args, query, body), extra);
|
||||
}
|
||||
|
||||
llm::RawResponse Endpoint::run_raw(std::string_view name, const PathArgs& args,
|
||||
const Query& query) const {
|
||||
return gateway_->call_raw(CallPlanner::plan(name, args, query, nullptr));
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,114 @@
|
||||
#pragma once
|
||||
|
||||
// Gateway: the one place OpenRouter traffic leaves the module. Owns the two
|
||||
// credentialed ApiClients (inference key and management key are different
|
||||
// credentials with different powers) and an anonymous client for the public
|
||||
// routes that need no key (models, providers, counts, jwks, ...). Applies the
|
||||
// attribution headers to all of them.
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "clients/transport.h"
|
||||
#include "llm/api_client.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
inline constexpr const char* kDefaultBaseUrl = "https://openrouter.ai/api/v1";
|
||||
inline constexpr const char* kApiKeyEnv = "OPENROUTER_API_KEY";
|
||||
inline constexpr const char* kManagementKeyEnv = "OPENROUTER_MANAGEMENT_KEY";
|
||||
|
||||
// App attribution (https://openrouter.ai/docs/app-attribution). HTTP-Referer
|
||||
// is what identifies the app for rankings; the title and categories refine it.
|
||||
struct Attribution {
|
||||
std::string referer; // HTTP-Referer
|
||||
std::string title; // X-OpenRouter-Title (and the legacy X-Title)
|
||||
std::string categories; // X-OpenRouter-Categories: "cli-agent,cloud-agent", at most 2
|
||||
bool hidden = false; // X-OpenRouter-App-Visibility: hidden (new apps only)
|
||||
bool router_metadata = false; // X-OpenRouter-Metadata: enabled
|
||||
|
||||
// OPENROUTER_APP_URL, OPENROUTER_APP_TITLE, OPENROUTER_APP_CATEGORIES via llm::Config.
|
||||
static Attribution from_env();
|
||||
// Throws llm::ApiError::invalid on over-long or malformed values.
|
||||
void validate() const;
|
||||
llm::Headers headers() const;
|
||||
};
|
||||
|
||||
// Keyless client for the public routes. Same error mapping as ApiClient.
|
||||
class AnonymousClient {
|
||||
public:
|
||||
AnonymousClient(std::unique_ptr<clients::Transport> transport, std::string base_url,
|
||||
llm::Headers default_headers);
|
||||
llm::RawResponse send(clients::Method method, const std::string& target,
|
||||
const std::string* body, const std::string& content_type,
|
||||
const llm::Headers& extra = {});
|
||||
clients::ClientStatsSnapshot stats() const { return client_.stats(); }
|
||||
|
||||
private:
|
||||
std::string base_url_;
|
||||
llm::Headers headers_;
|
||||
clients::Client client_;
|
||||
};
|
||||
|
||||
class Gateway {
|
||||
public:
|
||||
Gateway(std::shared_ptr<llm::ApiClient> api, std::shared_ptr<llm::ApiClient> management,
|
||||
std::shared_ptr<AnonymousClient> anonymous);
|
||||
|
||||
// Real HTTPS clients, keys from llm::Config, base URL from OPENROUTER_BASE_URL
|
||||
// when set (https, or http on loopback only), attribution as given.
|
||||
static std::shared_ptr<Gateway> from_env(const Attribution& attribution = Attribution::from_env());
|
||||
|
||||
bool api_configured() const { return api_->configured(); }
|
||||
bool management_configured() const { return management_->configured(); }
|
||||
// True when a call of this key kind can run (public calls always can).
|
||||
bool ready(KeyKind kind) const;
|
||||
const std::shared_ptr<llm::ApiClient>& api() const { return api_; }
|
||||
const std::shared_ptr<llm::ApiClient>& management() const { return management_; }
|
||||
|
||||
// JSON in, JSON out ("json" and "form" calls; form bodies are url-encoded).
|
||||
Json call(const PlannedCall& call, const llm::Headers& extra = {}) const;
|
||||
// Bytes out ("raw" calls such as file and video content).
|
||||
llm::RawResponse call_raw(const PlannedCall& call, const llm::Headers& extra = {}) const;
|
||||
// "multipart" calls (file upload).
|
||||
Json call_multipart(const PlannedCall& call, const clients::body::MultipartBody& form,
|
||||
const llm::Headers& extra = {}) const;
|
||||
// POST with SSE streaming. The planned call must carry a body.
|
||||
void stream(const PlannedCall& call, const llm::SseDecoder::Sink& sink,
|
||||
const llm::Headers& extra = {}) const;
|
||||
|
||||
private:
|
||||
llm::ApiClient& client_for(KeyKind kind) const;
|
||||
|
||||
std::shared_ptr<llm::ApiClient> api_;
|
||||
std::shared_ptr<llm::ApiClient> management_;
|
||||
std::shared_ptr<AnonymousClient> anonymous_;
|
||||
};
|
||||
|
||||
// Base for the service classes. Holds the gateway; the llm::Service base
|
||||
// points at the inference client (use ready() to ask the right key).
|
||||
class Endpoint : public llm::Service {
|
||||
public:
|
||||
explicit Endpoint(std::shared_ptr<Gateway> gateway, KeyKind primary = KeyKind::kApi);
|
||||
const std::shared_ptr<Gateway>& gateway() const { return gateway_; }
|
||||
// True when this service's primary key kind is configured.
|
||||
bool ready() const { return gateway_->ready(primary_); }
|
||||
KeyKind primary_key() const { return primary_; }
|
||||
|
||||
protected:
|
||||
Json run(std::string_view name, const PathArgs& args = {}, const Query& query = {},
|
||||
const Json* body = nullptr, const llm::Headers& extra = {}) const;
|
||||
Json run(std::string_view name, const PathArgs& args, const Query& query,
|
||||
const Json& body, const llm::Headers& extra = {}) const {
|
||||
return run(name, args, query, &body, extra);
|
||||
}
|
||||
llm::RawResponse run_raw(std::string_view name, const PathArgs& args = {},
|
||||
const Query& query = {}) const;
|
||||
|
||||
std::shared_ptr<Gateway> gateway_;
|
||||
KeyKind primary_;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,84 @@
|
||||
#include "openrouter/inference_services.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
void reject_stream_flag(const Json& body) {
|
||||
if (const Json* s = body.find("stream"); s && s->is_bool() && s->as_bool())
|
||||
throw llm::ApiError::invalid(kVendor, "use stream() for stream:true");
|
||||
}
|
||||
|
||||
Json streaming(const Json& body) {
|
||||
Json b = body;
|
||||
b.set("stream", Json(true));
|
||||
return b;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Json ResponsesService::create(const Json& body) const {
|
||||
reject_stream_flag(body);
|
||||
Json doc = run("createResponses", {}, {}, body);
|
||||
throw_if_error_body(doc);
|
||||
if (const Json* st = doc.find("status"); st && st->is_string() && st->as_string() == "failed") {
|
||||
ErrorBody e;
|
||||
if (const Json* er = doc.find("error"); er && er->is_object()) {
|
||||
if (const Json* m = er->find("message"); m && m->is_string()) e.message = m->as_string();
|
||||
if (const Json* c = er->find("code"); c && c->is_string()) e.provider_code = c->as_string();
|
||||
}
|
||||
if (const Json* et = doc.find("error_type"); et && et->is_string()) e.error_type = et->as_string();
|
||||
e.code = status_for_error_type(e.error_type);
|
||||
throw e.to_api_error();
|
||||
}
|
||||
return doc;
|
||||
}
|
||||
|
||||
StreamSummary ResponsesService::stream(
|
||||
const Json& body,
|
||||
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const {
|
||||
Json b = streaming(body);
|
||||
PlannedCall call = CallPlanner::plan("createResponses", {}, {}, &b);
|
||||
ResponsesStreamParser parser;
|
||||
gateway_->stream(call, [&](const llm::SseEvent& ev) {
|
||||
std::string delta = parser.feed(ev);
|
||||
if (on_text && !on_text(delta, ev)) return false;
|
||||
return !parser.summary().error;
|
||||
});
|
||||
parser.throw_if_errored();
|
||||
return parser.summary();
|
||||
}
|
||||
|
||||
Json MessagesService::create(const Json& body) const {
|
||||
reject_stream_flag(body);
|
||||
if (!body.has("model") || !body.has("messages"))
|
||||
throw llm::ApiError::invalid(kVendor, "messages: model and messages are required");
|
||||
Json doc = run("createMessages", {}, {}, body);
|
||||
throw_if_error_body(doc);
|
||||
if (const Json* t = doc.find("type"); t && t->is_string() && t->as_string() == "error") {
|
||||
const Json* er = doc.find("error");
|
||||
throw ErrorBody::parse(er ? *er : Json::object()).to_api_error();
|
||||
}
|
||||
return doc;
|
||||
}
|
||||
|
||||
StreamSummary MessagesService::stream(
|
||||
const Json& body,
|
||||
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const {
|
||||
if (!body.has("model") || !body.has("messages"))
|
||||
throw llm::ApiError::invalid(kVendor, "messages: model and messages are required");
|
||||
Json b = streaming(body);
|
||||
PlannedCall call = CallPlanner::plan("createMessages", {}, {}, &b);
|
||||
MessagesStreamParser parser;
|
||||
gateway_->stream(call, [&](const llm::SseEvent& ev) {
|
||||
std::string delta = parser.feed(ev);
|
||||
if (on_text && !on_text(delta, ev)) return false;
|
||||
return !parser.summary().error;
|
||||
});
|
||||
parser.throw_if_errored();
|
||||
return parser.summary();
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,40 @@
|
||||
#pragma once
|
||||
|
||||
// ResponsesService (POST /responses, beta) and MessagesService (POST
|
||||
// /messages, the Anthropic-compatible skin). Bodies are JSON objects checked
|
||||
// against the call table (calls.h); the request shapes of these two skins are
|
||||
// large and owned by their respective protocols, so they are not re-modelled
|
||||
// field by field. Routing fields (provider, models, route, plugins, trace,
|
||||
// session_id) are the same objects ChatParams builds: use ProviderPreferences,
|
||||
// plugins_to_json and friends to fill them.
|
||||
|
||||
#include <functional>
|
||||
|
||||
#include "openrouter/gateway.h"
|
||||
#include "openrouter/stream_summary.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class ResponsesService : public Endpoint {
|
||||
public:
|
||||
explicit ResponsesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
// Non-streaming. Throws on HTTP errors, on a top-level error object and on
|
||||
// a response whose status is "failed".
|
||||
Json create(const Json& body) const;
|
||||
// Streaming. `on_text` gets each text delta and the raw event; return false
|
||||
// to stop. The summary holds text, reasoning, usage and status. A failed
|
||||
// response or `error` event is thrown after delivery.
|
||||
StreamSummary stream(const Json& body,
|
||||
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const;
|
||||
};
|
||||
|
||||
class MessagesService : public Endpoint {
|
||||
public:
|
||||
explicit MessagesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
// `model` and `messages` are required by the spec.
|
||||
Json create(const Json& body) const;
|
||||
StreamSummary stream(const Json& body,
|
||||
const std::function<bool(const std::string&, const llm::SseEvent&)>& on_text) const;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,108 @@
|
||||
#include "openrouter/model_ref.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
#include "llm/api_client.h"
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
|
||||
|
||||
bool segment_ok(const std::string& s, bool allow_alias) {
|
||||
std::string t = allow_alias && s.rfind("~", 0) == 0 ? s.substr(1) : s;
|
||||
if (t.empty() || t.size() > 128) return false;
|
||||
for (char c : t)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '-' || c == '_' || c == '.'))
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool ModelRef::is_catalog_variant(const std::string& s) {
|
||||
return s == "free" || s == "batch" || s == "thinking" || s == "extended";
|
||||
}
|
||||
bool ModelRef::is_routing_variant(const std::string& s) {
|
||||
return s == "nitro" || s == "floor" || s == "exacto" || s == "online";
|
||||
}
|
||||
bool ModelRef::is_deprecated_variant(const std::string& s) {
|
||||
return s == "thinking" || s == "extended" || s == "online";
|
||||
}
|
||||
|
||||
ModelRef ModelRef::parse(const std::string& id) {
|
||||
std::size_t slash = id.find('/');
|
||||
if (slash == std::string::npos || id.find('/', slash + 1) != std::string::npos)
|
||||
bad("model id must look like author/slug");
|
||||
ModelRef out;
|
||||
out.author_ = id.substr(0, slash);
|
||||
std::string rest = id.substr(slash + 1);
|
||||
if (!segment_ok(out.author_, true)) bad("bad model author");
|
||||
std::size_t pos = 0;
|
||||
bool first = true;
|
||||
while (true) {
|
||||
std::size_t colon = rest.find(':', pos);
|
||||
std::string part = rest.substr(pos, colon == std::string::npos ? colon : colon - pos);
|
||||
if (first) {
|
||||
if (!segment_ok(part, false)) bad("bad model slug");
|
||||
out.slug_ = part;
|
||||
first = false;
|
||||
} else if (is_catalog_variant(part)) {
|
||||
if (!out.catalog_.empty()) bad("a model id carries at most one catalog variant");
|
||||
out.catalog_ = part;
|
||||
} else if (is_routing_variant(part)) {
|
||||
if (std::find(out.routing_.begin(), out.routing_.end(), part) == out.routing_.end())
|
||||
out.routing_.push_back(part);
|
||||
} else {
|
||||
bad("unknown model variant: " + part);
|
||||
}
|
||||
if (colon == std::string::npos) break;
|
||||
pos = colon + 1;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string ModelRef::str() const {
|
||||
std::string s = base();
|
||||
if (!catalog_.empty()) s += ":" + catalog_;
|
||||
for (const std::string& r : routing_) s += ":" + r;
|
||||
return s;
|
||||
}
|
||||
|
||||
std::string ModelRef::catalog_id() const {
|
||||
return catalog_.empty() ? base() : base() + ":" + catalog_;
|
||||
}
|
||||
|
||||
bool ModelRef::uses_deprecated_variant() const {
|
||||
if (is_deprecated_variant(catalog_)) return true;
|
||||
for (const std::string& r : routing_)
|
||||
if (is_deprecated_variant(r)) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
ModelRef ModelRef::with_variant(const std::string& suffix) const {
|
||||
ModelRef out = *this;
|
||||
if (is_catalog_variant(suffix)) {
|
||||
if (!out.catalog_.empty() && out.catalog_ != suffix)
|
||||
bad("a model id carries at most one catalog variant");
|
||||
out.catalog_ = suffix;
|
||||
} else if (is_routing_variant(suffix)) {
|
||||
if (std::find(out.routing_.begin(), out.routing_.end(), suffix) == out.routing_.end())
|
||||
out.routing_.push_back(suffix);
|
||||
} else {
|
||||
bad("unknown model variant: " + suffix);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
ModelRef ModelRef::without_routing_variants() const {
|
||||
ModelRef out = *this;
|
||||
out.routing_.clear();
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
// ModelRef: an OpenRouter model id with its variant suffixes, per the model
|
||||
// variants guide (https://openrouter.ai/docs/guides/routing/model-variants).
|
||||
//
|
||||
// author/slug[:catalog][:routing]...
|
||||
//
|
||||
// catalog variants (own entry in GET /models; at most one):
|
||||
// free, batch, thinking (deprecated), extended (deprecated)
|
||||
// routing variants (accepted on any model, never listed; any number, the
|
||||
// last sorting one wins): nitro, floor, exacto, online (deprecated)
|
||||
//
|
||||
// Any other suffix is rejected: the docs say not to rely on other suffixes.
|
||||
// An author may carry the alias marker "~" (for example ~openai/gpt-latest).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class ModelRef {
|
||||
public:
|
||||
// Throws llm::ApiError::invalid on a malformed id or an unknown suffix.
|
||||
static ModelRef parse(const std::string& id);
|
||||
|
||||
static bool is_catalog_variant(const std::string& suffix);
|
||||
static bool is_routing_variant(const std::string& suffix);
|
||||
static bool is_deprecated_variant(const std::string& suffix);
|
||||
|
||||
const std::string& author() const { return author_; }
|
||||
const std::string& slug() const { return slug_; }
|
||||
const std::string& catalog_variant() const { return catalog_; }
|
||||
const std::vector<std::string>& routing_variants() const { return routing_; }
|
||||
|
||||
// "author/slug".
|
||||
std::string base() const { return author_ + "/" + slug_; }
|
||||
// The id as sent in a request `model` field.
|
||||
std::string str() const;
|
||||
// The id of the catalog entry that describes this model: base plus the
|
||||
// catalog variant, routing variants dropped (":free" is NOT dropped).
|
||||
std::string catalog_id() const;
|
||||
// True when any suffix on this id is deprecated.
|
||||
bool uses_deprecated_variant() const;
|
||||
|
||||
// Returns a copy with the variant added. A second catalog variant or an
|
||||
// unknown suffix throws; a repeated routing variant is kept once.
|
||||
ModelRef with_variant(const std::string& suffix) const;
|
||||
ModelRef without_routing_variants() const;
|
||||
|
||||
private:
|
||||
std::string author_;
|
||||
std::string slug_;
|
||||
std::string catalog_;
|
||||
std::vector<std::string> routing_;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,374 @@
|
||||
#include "openrouter/models.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <cstdlib>
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, "models: " + why); }
|
||||
|
||||
bool in(const std::string& v, std::initializer_list<const char*> all) {
|
||||
return std::any_of(all.begin(), all.end(), [&](const char* s) { return v == s; });
|
||||
}
|
||||
|
||||
std::string str(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
long integer(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
return v && v->is_number() ? v->as_int() : 0;
|
||||
}
|
||||
|
||||
std::optional<double> real(const Json& o, const char* k) {
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
if (v && v->is_number()) return v->as_number();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
std::vector<std::string> strings(const Json& o, const char* k) {
|
||||
std::vector<std::string> out;
|
||||
const Json* v = o.is_object() ? o.find(k) : nullptr;
|
||||
if (v && v->is_array())
|
||||
for (const Json& s : v->as_array())
|
||||
if (s.is_string()) out.push_back(s.as_string());
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string join(const std::vector<std::string>& v) {
|
||||
std::string out;
|
||||
for (const std::string& s : v) out += (out.empty() ? "" : ",") + s;
|
||||
return out;
|
||||
}
|
||||
|
||||
void check_csv(const char* what, const std::vector<std::string>& v) {
|
||||
if (v.size() > Caps::kMaxProviderList) bad(std::string(what) + " has too many entries");
|
||||
for (const std::string& s : v) {
|
||||
if (s.empty() || s.size() > 128 || s.find(',') != std::string::npos)
|
||||
bad(std::string(what) + " entries must be non-empty, without commas");
|
||||
}
|
||||
}
|
||||
|
||||
void check_pair(const char* what, const std::optional<double>& lo, const std::optional<double>& hi,
|
||||
double min_allowed, double max_allowed) {
|
||||
for (const auto* v : {&lo, &hi})
|
||||
if (*v && (!std::isfinite(**v) || **v < min_allowed || **v > max_allowed))
|
||||
bad(std::string(what) + " out of range");
|
||||
if (lo && hi && *lo > *hi) bad(std::string(what) + ": min is above max");
|
||||
}
|
||||
|
||||
void check_limit(const std::optional<int>& offset, const std::optional<int>& limit) {
|
||||
if (offset && *offset < 0) bad("offset must be >= 0");
|
||||
if (limit && (*limit < 1 || *limit > Caps::kMaxListLimit)) bad("limit must be 1..1000");
|
||||
}
|
||||
|
||||
ModelPage parse_page(const Json& doc) {
|
||||
ModelPage p;
|
||||
const Json* data = doc.is_object() ? doc.find("data") : nullptr;
|
||||
if (!data || !data->is_array()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "models response has no data array");
|
||||
for (const Json& m : data->as_array()) p.models.push_back(ModelInfo::parse(m));
|
||||
p.total_count = integer(doc, "total_count");
|
||||
if (p.total_count == 0) p.total_count = static_cast<long>(p.models.size());
|
||||
if (const Json* links = doc.find("links"); links && links->is_object()) p.next = str(*links, "next");
|
||||
return p;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool ModelFilter::valid_sort(const std::string& s) {
|
||||
return in(s, {"most-popular", "newest", "top-weekly", "pricing-low-to-high", "pricing-high-to-low",
|
||||
"context-high-to-low", "throughput-high-to-low", "latency-low-to-high",
|
||||
"intelligence-high-to-low", "coding-high-to-low", "agentic-high-to-low",
|
||||
"design-arena-elo-high-to-low"});
|
||||
}
|
||||
bool ModelFilter::valid_category(const std::string& s) {
|
||||
return in(s, {"programming", "roleplay", "marketing", "marketing/seo", "technology", "science",
|
||||
"translation", "legal", "finance", "health", "trivia", "academia"});
|
||||
}
|
||||
bool ModelFilter::valid_output_modality(const std::string& s) {
|
||||
return in(s, {"text", "image", "embeddings", "audio", "video", "rerank", "decisions", "speech",
|
||||
"transcription", "all"});
|
||||
}
|
||||
bool ModelFilter::valid_input_modality(const std::string& s) {
|
||||
return in(s, {"text", "image", "audio", "file"});
|
||||
}
|
||||
|
||||
void ModelFilter::validate() const {
|
||||
check_limit(offset, limit);
|
||||
if (!category.empty() && !valid_category(category)) bad("unknown category");
|
||||
if (!sort.empty() && !valid_sort(sort)) bad("unknown sort");
|
||||
check_csv("supported_parameters", supported_parameters);
|
||||
check_csv("model_authors", model_authors);
|
||||
check_csv("providers", providers);
|
||||
for (const std::string& m : output_modalities)
|
||||
if (!valid_output_modality(m)) bad("unknown output modality: " + m);
|
||||
for (const std::string& m : input_modalities)
|
||||
if (!valid_input_modality(m)) bad("unknown input modality: " + m);
|
||||
if (!region.empty() && region != "eu" && region != "us") bad("region must be eu or us");
|
||||
if (context && *context < 1) bad("context must be >= 1");
|
||||
if (q.size() > 256 || arch.size() > 128) bad("q or arch too long");
|
||||
constexpr double kBig = 1e9;
|
||||
check_pair("price", min_price, max_price, 0, kBig);
|
||||
check_pair("output price", min_output_price, max_output_price, 0, kBig);
|
||||
check_pair("intelligence index", min_intelligence_index, max_intelligence_index, 0, kBig);
|
||||
check_pair("coding index", min_coding_index, max_coding_index, 0, kBig);
|
||||
check_pair("agentic index", min_agentic_index, max_agentic_index, 0, kBig);
|
||||
check_pair("tool success rate", min_tool_success_rate, max_tool_success_rate, 0, 1);
|
||||
if ((min_age_days && *min_age_days < 0) || (max_age_days && *max_age_days < 0)) bad("age days must be >= 0");
|
||||
if (min_age_days && max_age_days && *min_age_days > *max_age_days) bad("age days: min is above max");
|
||||
}
|
||||
|
||||
Query ModelFilter::to_query() const {
|
||||
validate();
|
||||
Query q_;
|
||||
if (offset) q_.add("offset", *offset);
|
||||
if (limit) q_.add("limit", *limit);
|
||||
q_.add_if("category", category);
|
||||
q_.add_if("supported_parameters", join(supported_parameters));
|
||||
q_.add_if("output_modalities", join(output_modalities));
|
||||
q_.add_if("input_modalities", join(input_modalities));
|
||||
q_.add_if("sort", sort);
|
||||
q_.add_if("q", q);
|
||||
if (context) q_.add("context", *context);
|
||||
auto d = [&](const char* k, const std::optional<double>& v) { if (v) q_.add(k, *v); };
|
||||
d("min_price", min_price);
|
||||
d("max_price", max_price);
|
||||
q_.add_if("arch", arch);
|
||||
q_.add_if("model_authors", join(model_authors));
|
||||
q_.add_if("providers", join(providers));
|
||||
if (distillable) q_.add("distillable", *distillable ? "true" : "false");
|
||||
if (zdr) q_.add("zdr", "true");
|
||||
q_.add_if("region", region);
|
||||
d("min_output_price", min_output_price);
|
||||
d("max_output_price", max_output_price);
|
||||
if (min_age_days) q_.add("min_age_days", *min_age_days);
|
||||
if (max_age_days) q_.add("max_age_days", *max_age_days);
|
||||
d("min_intelligence_index", min_intelligence_index);
|
||||
d("max_intelligence_index", max_intelligence_index);
|
||||
d("min_coding_index", min_coding_index);
|
||||
d("max_coding_index", max_coding_index);
|
||||
d("min_agentic_index", min_agentic_index);
|
||||
d("max_agentic_index", max_agentic_index);
|
||||
d("min_tool_success_rate", min_tool_success_rate);
|
||||
d("max_tool_success_rate", max_tool_success_rate);
|
||||
return q_;
|
||||
}
|
||||
|
||||
ModelInfo ModelInfo::parse(const Json& doc) {
|
||||
if (!doc.is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "model is not an object");
|
||||
ModelInfo m;
|
||||
m.raw = doc;
|
||||
m.id = str(doc, "id");
|
||||
m.canonical_slug = str(doc, "canonical_slug");
|
||||
m.name = str(doc, "name");
|
||||
m.description = str(doc, "description");
|
||||
m.created = integer(doc, "created");
|
||||
m.context_length = integer(doc, "context_length");
|
||||
if (const Json* a = doc.find("architecture"); a && a->is_object()) {
|
||||
m.modality = str(*a, "modality");
|
||||
m.input_modalities = strings(*a, "input_modalities");
|
||||
m.output_modalities = strings(*a, "output_modalities");
|
||||
m.tokenizer = str(*a, "tokenizer");
|
||||
}
|
||||
m.supported_parameters = strings(doc, "supported_parameters");
|
||||
if (const Json* p = doc.find("pricing")) m.pricing = *p;
|
||||
if (const Json* t = doc.find("top_provider")) m.top_provider = *t;
|
||||
if (const Json* d = doc.find("default_parameters")) m.default_parameters = *d;
|
||||
if (const Json* r = doc.find("reasoning")) m.reasoning = *r;
|
||||
m.expiration_date = str(doc, "expiration_date");
|
||||
if (const Json* at = doc.find("alias_target")) m.alias_target = at->is_string() ? at->as_string() : at->dump();
|
||||
return m;
|
||||
}
|
||||
|
||||
bool ModelInfo::supports(const std::string& parameter) const {
|
||||
return std::find(supported_parameters.begin(), supported_parameters.end(), parameter) !=
|
||||
supported_parameters.end();
|
||||
}
|
||||
|
||||
std::optional<double> ModelInfo::price_per_million(const std::string& key) const {
|
||||
const Json* v = pricing.is_object() ? pricing.find(key) : nullptr;
|
||||
if (!v) return std::nullopt;
|
||||
double per_token;
|
||||
if (v->is_number()) {
|
||||
per_token = v->as_number();
|
||||
} else if (v->is_string()) {
|
||||
char* end = nullptr;
|
||||
per_token = std::strtod(v->as_string().c_str(), &end);
|
||||
if (end == v->as_string().c_str()) return std::nullopt;
|
||||
} else {
|
||||
return std::nullopt;
|
||||
}
|
||||
return per_token * 1e6;
|
||||
}
|
||||
|
||||
bool ModelInfo::is_free() const {
|
||||
if (id.size() > 5 && id.compare(id.size() - 5, 5, ":free") == 0) return true;
|
||||
auto p = price_per_million("prompt");
|
||||
auto c = price_per_million("completion");
|
||||
return p && c && *p == 0 && *c == 0;
|
||||
}
|
||||
|
||||
EndpointInfo EndpointInfo::parse(const Json& d) {
|
||||
if (!d.is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "endpoint is not an object");
|
||||
EndpointInfo e;
|
||||
e.raw = d;
|
||||
e.name = str(d, "name");
|
||||
e.model_id = str(d, "model_id");
|
||||
e.model_name = str(d, "model_name");
|
||||
e.provider_name = str(d, "provider_name");
|
||||
e.tag = str(d, "tag");
|
||||
e.quantization = str(d, "quantization");
|
||||
e.service_tier = str(d, "service_tier");
|
||||
e.context_length = integer(d, "context_length");
|
||||
if (const Json* v = d.find("max_completion_tokens"); v && v->is_number()) e.max_completion_tokens = v->as_int();
|
||||
if (const Json* v = d.find("max_prompt_tokens"); v && v->is_number()) e.max_prompt_tokens = v->as_int();
|
||||
e.status = static_cast<int>(integer(d, "status"));
|
||||
e.uptime_last_5m = real(d, "uptime_last_5m");
|
||||
e.uptime_last_30m = real(d, "uptime_last_30m");
|
||||
e.uptime_last_1d = real(d, "uptime_last_1d");
|
||||
if (const Json* v = d.find("latency_last_30m")) e.latency_last_30m = *v;
|
||||
if (const Json* v = d.find("throughput_last_30m")) e.throughput_last_30m = *v;
|
||||
if (const Json* v = d.find("pricing")) e.pricing = *v;
|
||||
if (const Json* v = d.find("supports_tool_choice")) e.supports_tool_choice = *v;
|
||||
e.supported_parameters = strings(d, "supported_parameters");
|
||||
if (const Json* v = d.find("supports_implicit_caching"); v && v->is_bool()) e.supports_implicit_caching = v->as_bool();
|
||||
return e;
|
||||
}
|
||||
|
||||
ProviderInfo ProviderInfo::parse(const Json& d) {
|
||||
if (!d.is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "provider is not an object");
|
||||
ProviderInfo p;
|
||||
p.name = str(d, "name");
|
||||
p.slug = str(d, "slug");
|
||||
p.privacy_policy_url = str(d, "privacy_policy_url");
|
||||
p.terms_of_service_url = str(d, "terms_of_service_url");
|
||||
p.status_page_url = str(d, "status_page_url");
|
||||
p.headquarters = str(d, "headquarters");
|
||||
p.datacenters = strings(d, "datacenters");
|
||||
return p;
|
||||
}
|
||||
|
||||
ModelPage ModelsService::page(const ModelFilter& filter) const {
|
||||
return parse_page(run("getModels", {}, filter.to_query()));
|
||||
}
|
||||
|
||||
ModelList ModelsService::list_all(const ModelFilter& filter, std::size_t max_models) const {
|
||||
if (max_models == 0) bad("max_models must be >= 1");
|
||||
ModelList out;
|
||||
ModelFilter f = filter;
|
||||
f.limit = Caps::kModelPageSize;
|
||||
int offset = filter.offset.value_or(0);
|
||||
while (true) {
|
||||
if (out.pages >= Caps::kMaxModelPages || out.models.size() >= max_models) {
|
||||
out.truncated = true;
|
||||
break;
|
||||
}
|
||||
f.offset = offset;
|
||||
ModelPage p = page(f);
|
||||
++out.pages;
|
||||
out.total_count = p.total_count;
|
||||
for (ModelInfo& m : p.models) {
|
||||
if (out.models.size() >= max_models) {
|
||||
out.truncated = true;
|
||||
return out;
|
||||
}
|
||||
out.models.push_back(std::move(m));
|
||||
}
|
||||
if (p.next.empty() || p.models.empty()) break;
|
||||
offset += static_cast<int>(p.models.size());
|
||||
if (static_cast<long>(offset) >= p.total_count) break;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
ModelInfo ModelsService::get(const std::string& model_id) const {
|
||||
auto [a, s] = split_model_id(model_id);
|
||||
Json doc = run("getModel", {{"author", a}, {"slug", s}});
|
||||
const Json* d = doc.find("data");
|
||||
return ModelInfo::parse(d ? *d : doc);
|
||||
}
|
||||
|
||||
ModelEndpoints ModelsService::endpoints(const std::string& model_id) const {
|
||||
auto [a, s] = split_model_id(model_id);
|
||||
Json doc = run("listEndpoints", {{"author", a}, {"slug", s}});
|
||||
const Json* d = doc.find("data");
|
||||
if (!d || !d->is_object()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "endpoints response has no data");
|
||||
ModelEndpoints out;
|
||||
out.raw = doc;
|
||||
out.id = str(*d, "id");
|
||||
out.name = str(*d, "name");
|
||||
out.created = integer(*d, "created");
|
||||
out.description = str(*d, "description");
|
||||
if (const Json* a2 = d->find("architecture")) out.architecture = *a2;
|
||||
if (const Json* eps = d->find("endpoints"); eps && eps->is_array())
|
||||
for (const Json& e : eps->as_array()) out.endpoints.push_back(EndpointInfo::parse(e));
|
||||
return out;
|
||||
}
|
||||
|
||||
long ModelsService::count(const std::string& output_modalities) const {
|
||||
Query q;
|
||||
if (!output_modalities.empty()) {
|
||||
std::size_t pos = 0;
|
||||
while (pos <= output_modalities.size()) {
|
||||
std::size_t comma = output_modalities.find(',', pos);
|
||||
std::string one = output_modalities.substr(pos, comma == std::string::npos ? comma : comma - pos);
|
||||
if (!ModelFilter::valid_output_modality(one)) bad("unknown output modality: " + one);
|
||||
if (comma == std::string::npos) break;
|
||||
pos = comma + 1;
|
||||
}
|
||||
q.add("output_modalities", output_modalities);
|
||||
}
|
||||
Json doc = run("listModelsCount", {}, q);
|
||||
const Json* d = doc.find("data");
|
||||
return d ? integer(*d, "count") : 0;
|
||||
}
|
||||
|
||||
Json ModelsService::zdr_endpoints() const { return run("listEndpointsZdr"); }
|
||||
|
||||
Json ModelsService::embeddings_models(std::optional<int> offset, std::optional<int> limit) const {
|
||||
check_limit(offset, limit);
|
||||
Query q;
|
||||
if (offset) q.add("offset", *offset);
|
||||
if (limit) q.add("limit", *limit);
|
||||
return run("listEmbeddingsModels", {}, q);
|
||||
}
|
||||
|
||||
Json ModelsService::image_models() const { return run("listImageModels"); }
|
||||
Json ModelsService::video_models() const { return run("listVideosModels"); }
|
||||
|
||||
Json ModelsService::image_model_endpoints(const std::string& model_id) const {
|
||||
auto [a, s] = split_model_id(model_id);
|
||||
return run("listImageModelEndpoints", {{"author", a}, {"slug", s}});
|
||||
}
|
||||
|
||||
ModelPage ModelsService::user_models(std::optional<int> offset, std::optional<int> limit,
|
||||
const std::string& output_modalities,
|
||||
const std::string& sort) const {
|
||||
check_limit(offset, limit);
|
||||
if (!sort.empty() && !ModelFilter::valid_sort(sort)) bad("unknown sort");
|
||||
Query q;
|
||||
if (offset) q.add("offset", *offset);
|
||||
if (limit) q.add("limit", *limit);
|
||||
q.add_if("output_modalities", output_modalities);
|
||||
q.add_if("sort", sort);
|
||||
if (!gateway_->api_configured()) throw llm::NotConfiguredError(kVendor, kApiKeyEnv);
|
||||
return parse_page(run("listModelsUser", {}, q));
|
||||
}
|
||||
|
||||
std::vector<ProviderInfo> ProvidersService::list() const {
|
||||
Json doc = run("listProviders");
|
||||
const Json* d = doc.find("data");
|
||||
if (!d || !d->is_array()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "providers response has no data array");
|
||||
std::vector<ProviderInfo> out;
|
||||
for (const Json& p : d->as_array()) out.push_back(ProviderInfo::parse(p));
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,165 @@
|
||||
#pragma once
|
||||
|
||||
// Model catalog: GET /models (with every documented filter), /model/{a}/{s},
|
||||
// /models/{a}/{s}/endpoints (provider, pricing, quantization, latency,
|
||||
// uptime), /models/count, /models/user, /endpoints/zdr, the embeddings,
|
||||
// image and video model lists, and GET /providers. The listing routes are
|
||||
// public: they work without any key. /models/user needs the API key.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "openrouter/gateway.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
// Query for GET /models. Every field is optional; validate() is strict.
|
||||
struct ModelFilter {
|
||||
std::optional<int> offset;
|
||||
std::optional<int> limit; // 1..1000; both omitted: the API returns the full list
|
||||
std::string category; // programming roleplay marketing marketing/seo technology science translation legal finance health trivia academia
|
||||
std::vector<std::string> supported_parameters; // joined with commas
|
||||
std::vector<std::string> output_modalities; // text image embeddings audio video rerank decisions speech transcription, or "all"
|
||||
std::vector<std::string> input_modalities; // text image audio file
|
||||
std::string sort; // most-popular newest top-weekly pricing-low-to-high pricing-high-to-low context-high-to-low throughput-high-to-low latency-low-to-high intelligence-high-to-low coding-high-to-low agentic-high-to-low design-arena-elo-high-to-low
|
||||
std::string q; // free text on name or slug
|
||||
std::optional<int> context; // minimum context length
|
||||
std::optional<double> min_price, max_price; // prompt, USD per million tokens
|
||||
std::optional<double> min_output_price, max_output_price; // completion
|
||||
std::string arch; // GPT, Claude, Gemini, Llama, ...
|
||||
std::vector<std::string> model_authors; // author slugs
|
||||
std::vector<std::string> providers; // provider names
|
||||
std::optional<bool> distillable;
|
||||
bool zdr = false; // only models with zero data retention endpoints
|
||||
std::string region; // "", eu, us
|
||||
std::optional<int> min_age_days, max_age_days;
|
||||
std::optional<double> min_intelligence_index, max_intelligence_index;
|
||||
std::optional<double> min_coding_index, max_coding_index;
|
||||
std::optional<double> min_agentic_index, max_agentic_index;
|
||||
std::optional<double> min_tool_success_rate, max_tool_success_rate; // 0..1
|
||||
|
||||
void validate() const;
|
||||
Query to_query() const; // validates
|
||||
|
||||
static bool valid_sort(const std::string& s);
|
||||
static bool valid_category(const std::string& s);
|
||||
static bool valid_output_modality(const std::string& s);
|
||||
static bool valid_input_modality(const std::string& s);
|
||||
};
|
||||
|
||||
struct ModelInfo {
|
||||
std::string id;
|
||||
std::string canonical_slug;
|
||||
std::string name;
|
||||
std::string description;
|
||||
long created = 0;
|
||||
long context_length = 0;
|
||||
std::string modality; // "text+image->text"
|
||||
std::vector<std::string> input_modalities;
|
||||
std::vector<std::string> output_modalities;
|
||||
std::string tokenizer;
|
||||
std::vector<std::string> supported_parameters;
|
||||
Json pricing; // values are decimal strings (USD per token) and sometimes numbers
|
||||
Json top_provider;
|
||||
Json default_parameters;
|
||||
Json reasoning; // {mandatory, default_enabled, supported_efforts, default_effort}
|
||||
std::string expiration_date;
|
||||
std::string alias_target;
|
||||
Json raw;
|
||||
|
||||
static ModelInfo parse(const Json& doc);
|
||||
bool supports(const std::string& parameter) const;
|
||||
bool is_free() const; // id ends with ":free" or both prompt and completion price are zero
|
||||
// USD per million tokens for a pricing key ("prompt", "completion", ...), if present.
|
||||
std::optional<double> price_per_million(const std::string& key) const;
|
||||
};
|
||||
|
||||
struct ModelPage {
|
||||
std::vector<ModelInfo> models;
|
||||
long total_count = 0;
|
||||
std::string next; // link to the next page, "" on the last
|
||||
};
|
||||
|
||||
struct ModelList {
|
||||
std::vector<ModelInfo> models;
|
||||
long total_count = 0;
|
||||
bool truncated = false; // stopped at a cap before reading everything
|
||||
std::size_t pages = 0;
|
||||
};
|
||||
|
||||
struct EndpointInfo {
|
||||
std::string name;
|
||||
std::string model_id;
|
||||
std::string model_name;
|
||||
std::string provider_name;
|
||||
std::string tag; // routing slug such as "azure" or "mancer/fp8"
|
||||
std::string quantization; // int4 ... fp32 or unknown
|
||||
std::string service_tier; // "", flex, priority, ultrafast
|
||||
long context_length = 0;
|
||||
std::optional<long> max_completion_tokens;
|
||||
std::optional<long> max_prompt_tokens;
|
||||
int status = 0; // 0 ok; negative values are degraded states
|
||||
std::optional<double> uptime_last_5m, uptime_last_30m, uptime_last_1d; // percent
|
||||
Json latency_last_30m; // {p50,p75,p90,p99} seconds, or null
|
||||
Json throughput_last_30m; // {p50,p75,p90,p99} tokens/second, or null
|
||||
Json pricing;
|
||||
Json supports_tool_choice;
|
||||
std::vector<std::string> supported_parameters;
|
||||
bool supports_implicit_caching = false;
|
||||
Json raw;
|
||||
|
||||
static EndpointInfo parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct ModelEndpoints {
|
||||
std::string id;
|
||||
std::string name;
|
||||
long created = 0;
|
||||
std::string description;
|
||||
Json architecture;
|
||||
std::vector<EndpointInfo> endpoints;
|
||||
Json raw;
|
||||
};
|
||||
|
||||
struct ProviderInfo {
|
||||
std::string name;
|
||||
std::string slug;
|
||||
std::string privacy_policy_url;
|
||||
std::string terms_of_service_url;
|
||||
std::string status_page_url;
|
||||
std::string headquarters;
|
||||
std::vector<std::string> datacenters;
|
||||
static ProviderInfo parse(const Json& doc);
|
||||
};
|
||||
|
||||
class ModelsService : public Endpoint {
|
||||
public:
|
||||
explicit ModelsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kNone) {}
|
||||
|
||||
// One request. With neither offset nor limit the API answers the whole list.
|
||||
ModelPage page(const ModelFilter& filter = {}) const;
|
||||
// Walks offset/limit pages of Caps::kModelPageSize until total_count is read
|
||||
// or `max_models` / Caps::kMaxModelPages is hit; then truncated is true.
|
||||
ModelList list_all(const ModelFilter& filter = {}, std::size_t max_models = 5000) const;
|
||||
ModelInfo get(const std::string& model_id) const;
|
||||
ModelEndpoints endpoints(const std::string& model_id) const;
|
||||
long count(const std::string& output_modalities = "") const;
|
||||
Json zdr_endpoints() const;
|
||||
Json embeddings_models(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
Json image_models() const;
|
||||
Json image_model_endpoints(const std::string& model_id) const;
|
||||
Json video_models() const;
|
||||
// Models filtered by the account's provider preferences, privacy settings
|
||||
// and guardrails. Needs OPENROUTER_API_KEY.
|
||||
ModelPage user_models(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& output_modalities = "", const std::string& sort = "") const;
|
||||
};
|
||||
|
||||
class ProvidersService : public Endpoint {
|
||||
public:
|
||||
explicit ProvidersService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kNone) {}
|
||||
std::vector<ProviderInfo> list() const;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,93 @@
|
||||
// PKCE helpers. SHA-256 and the random source live here because the ai build
|
||||
// does not link src/security (see SPEC "Gaps").
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/caps.h"
|
||||
#include "openrouter/platform.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
constexpr std::uint32_t kK[64] = {
|
||||
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
|
||||
0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
|
||||
0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
|
||||
0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
|
||||
0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
|
||||
0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
|
||||
0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
|
||||
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2};
|
||||
|
||||
std::uint32_t rotr(std::uint32_t x, int n) { return (x >> n) | (x << (32 - n)); }
|
||||
|
||||
std::array<std::uint8_t, 32> sha256(const std::string& msg) {
|
||||
std::uint32_t h[8] = {0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a,
|
||||
0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19};
|
||||
std::string data = msg;
|
||||
data.push_back(static_cast<char>(0x80));
|
||||
while (data.size() % 64 != 56) data.push_back('\0');
|
||||
std::uint64_t bits = static_cast<std::uint64_t>(msg.size()) * 8;
|
||||
for (int i = 7; i >= 0; --i) data.push_back(static_cast<char>((bits >> (i * 8)) & 0xff));
|
||||
for (std::size_t off = 0; off < data.size(); off += 64) {
|
||||
std::uint32_t w[64];
|
||||
for (int i = 0; i < 16; ++i) {
|
||||
const unsigned char* p = reinterpret_cast<const unsigned char*>(data.data()) + off + i * 4;
|
||||
w[i] = (std::uint32_t(p[0]) << 24) | (std::uint32_t(p[1]) << 16) | (std::uint32_t(p[2]) << 8) | p[3];
|
||||
}
|
||||
for (int i = 16; i < 64; ++i) {
|
||||
std::uint32_t s0 = rotr(w[i - 15], 7) ^ rotr(w[i - 15], 18) ^ (w[i - 15] >> 3);
|
||||
std::uint32_t s1 = rotr(w[i - 2], 17) ^ rotr(w[i - 2], 19) ^ (w[i - 2] >> 10);
|
||||
w[i] = w[i - 16] + s0 + w[i - 7] + s1;
|
||||
}
|
||||
std::uint32_t a = h[0], b = h[1], c = h[2], d = h[3], e = h[4], f = h[5], g = h[6], hh = h[7];
|
||||
for (int i = 0; i < 64; ++i) {
|
||||
std::uint32_t t1 = hh + (rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25)) + ((e & f) ^ (~e & g)) + kK[i] + w[i];
|
||||
std::uint32_t t2 = (rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22)) + ((a & b) ^ (a & c) ^ (b & c));
|
||||
hh = g; g = f; f = e; e = d + t1; d = c; c = b; b = a; a = t1 + t2;
|
||||
}
|
||||
h[0] += a; h[1] += b; h[2] += c; h[3] += d; h[4] += e; h[5] += f; h[6] += g; h[7] += hh;
|
||||
}
|
||||
std::array<std::uint8_t, 32> out{};
|
||||
for (int i = 0; i < 8; ++i)
|
||||
for (int j = 0; j < 4; ++j) out[i * 4 + j] = static_cast<std::uint8_t>(h[i] >> (24 - j * 8));
|
||||
return out;
|
||||
}
|
||||
|
||||
bool verifier_ok(const std::string& v) {
|
||||
if (v.size() < Caps::kMinPkceVerifier || v.size() > Caps::kMaxPkceVerifier) return false;
|
||||
for (char c : v)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
|
||||
c == '.' || c == '_' || c == '~'))
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
PkcePair PkcePair::from_verifier(const std::string& verifier) {
|
||||
if (!verifier_ok(verifier))
|
||||
throw llm::ApiError::invalid(kVendor, "PKCE verifier must be 43..128 characters of A-Za-z0-9-._~");
|
||||
PkcePair p;
|
||||
p.verifier = verifier;
|
||||
auto d = sha256(verifier);
|
||||
p.challenge = security::crypto::Base64::encode_url(d.data(), d.size());
|
||||
return p;
|
||||
}
|
||||
|
||||
PkcePair PkcePair::generate() {
|
||||
unsigned char raw[48];
|
||||
std::FILE* f = std::fopen("/dev/urandom", "rb");
|
||||
if (!f || std::fread(raw, 1, sizeof raw, f) != sizeof raw) {
|
||||
if (f) std::fclose(f);
|
||||
throw llm::ApiError(kVendor, 500, "NO_ENTROPY", "no system random source for the PKCE verifier");
|
||||
}
|
||||
std::fclose(f);
|
||||
return from_verifier(security::crypto::Base64::encode_url(raw, sizeof raw)); // 64 chars
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,238 @@
|
||||
#include "openrouter/platform.h"
|
||||
|
||||
#include "clients/body/form.h"
|
||||
#include "clients/url.h"
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/admin.h"
|
||||
#include "openrouter/caps.h"
|
||||
#include "openrouter/errors.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, why); }
|
||||
|
||||
void need(const Json& body, std::initializer_list<const char*> keys, const char* what) {
|
||||
if (!body.is_object()) bad(std::string(what) + " needs a JSON object");
|
||||
for (const char* k : keys)
|
||||
if (!body.has(k)) bad(std::string(what) + " needs " + k);
|
||||
}
|
||||
|
||||
std::string pick_after(const std::string& after) {
|
||||
if (after.size() > 256) bad("cursor too long");
|
||||
return after;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
Json EmbeddingsService::create(const Json& body) const {
|
||||
need(body, {"model", "input"}, "embeddings");
|
||||
const Json& in = *body.find("input");
|
||||
if (in.is_string() ? in.as_string().empty()
|
||||
: (!in.is_array() || in.as_array().empty() || in.as_array().size() > Caps::kMaxEmbeddingInputs))
|
||||
bad("input must be a non-empty string or an array of at most 2048 entries");
|
||||
if (const Json* e = body.find("encoding_format"); e && e->is_string() && e->as_string() != "float" && e->as_string() != "base64")
|
||||
bad("encoding_format must be float or base64");
|
||||
if (const Json* d = body.find("dimensions"); d && (!d->is_number() || d->as_int() < 1)) bad("dimensions must be >= 1");
|
||||
Json doc = run("createEmbeddings", {}, {}, body);
|
||||
throw_if_error_body(doc);
|
||||
return doc;
|
||||
}
|
||||
|
||||
Json EmbeddingsService::rerank(const std::string& model, const std::string& query,
|
||||
const std::vector<std::string>& documents, std::optional<int> top_n) const {
|
||||
if (model.empty() || query.empty()) bad("rerank needs model and query");
|
||||
if (documents.empty() || documents.size() > Caps::kMaxRerankDocuments) bad("rerank needs 1..1000 documents");
|
||||
if (top_n && *top_n < 1) bad("top_n must be >= 1");
|
||||
Json docs = Json::array();
|
||||
for (const std::string& d : documents) docs.push(Json(d));
|
||||
Json b = Json::object();
|
||||
b.set("model", Json(model));
|
||||
b.set("query", Json(query));
|
||||
b.set("documents", docs);
|
||||
if (top_n) b.set("top_n", Json(*top_n));
|
||||
Json doc = run("createRerank", {}, {}, b);
|
||||
throw_if_error_body(doc);
|
||||
return doc;
|
||||
}
|
||||
|
||||
std::vector<std::vector<double>> EmbeddingsService::vectors(const Json& response) {
|
||||
std::vector<std::vector<double>> out;
|
||||
const Json* data = response.is_object() ? response.find("data") : nullptr;
|
||||
if (!data || !data->is_array()) bad("embeddings response has no data array");
|
||||
for (const Json& item : data->as_array()) {
|
||||
const Json* e = item.is_object() ? item.find("embedding") : nullptr;
|
||||
if (!e || !e->is_array()) bad("embedding is not a float array (use encoding_format float)");
|
||||
std::vector<double> v;
|
||||
for (const Json& x : e->as_array()) v.push_back(x.is_number() ? x.as_number() : 0);
|
||||
out.push_back(std::move(v));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
Json MediaService::generate_image(const Json& body) const {
|
||||
need(body, {"model", "prompt"}, "image generation");
|
||||
return run("createImages", {}, {}, body);
|
||||
}
|
||||
Json MediaService::submit_video(const Json& body) const {
|
||||
need(body, {"model", "prompt"}, "video generation");
|
||||
return run("createVideos", {}, {}, body);
|
||||
}
|
||||
Json MediaService::video_status(const std::string& id) const { return run("getVideos", {{"jobId", id}}); }
|
||||
llm::RawResponse MediaService::video_content(const std::string& id, std::optional<int> index) const {
|
||||
Query q;
|
||||
if (index) {
|
||||
if (*index < 0) bad("index must be >= 0");
|
||||
q.add("index", *index);
|
||||
}
|
||||
return run_raw("listVideosContent", {{"jobId", id}}, q);
|
||||
}
|
||||
llm::RawResponse MediaService::speech(const Json& body) const {
|
||||
need(body, {"model", "input", "voice"}, "speech");
|
||||
return gateway_->call_raw(CallPlanner::plan("createAudioSpeech", {}, {}, &body));
|
||||
}
|
||||
Json MediaService::transcribe(const Json& body) const {
|
||||
need(body, {"model", "input_audio"}, "transcription");
|
||||
return run("createAudioTranscriptions", {}, {}, body);
|
||||
}
|
||||
Json MediaService::decisions(const Json& body) const {
|
||||
need(body, {"questions"}, "decisions");
|
||||
return run("createApiAlphaDecisions", {}, {}, body);
|
||||
}
|
||||
Json MediaService::system_one(const Json& body) const {
|
||||
need(body, {"questions"}, "systemone");
|
||||
return run("createSystemone", {}, {}, body);
|
||||
}
|
||||
|
||||
Json BatchesService::list(std::optional<int> limit, const std::string& after, const std::string& status) const {
|
||||
Query q;
|
||||
if (limit) {
|
||||
if (*limit < 1 || *limit > 1000) bad("limit must be 1..1000");
|
||||
q.add("limit", *limit);
|
||||
}
|
||||
q.add_if("after", pick_after(after));
|
||||
q.add_if("status", status);
|
||||
return run("listBatches", {}, q);
|
||||
}
|
||||
Json BatchesService::get(const std::string& id) const { return run("getBatches", {{"id", id}}); }
|
||||
Json BatchesService::create(const Json& body) const {
|
||||
need(body, {"endpoint", "model", "requests"}, "batch");
|
||||
const Json& r = *body.find("requests");
|
||||
if (!r.is_array() || r.as_array().empty() || r.as_array().size() > Caps::kMaxBatchRequests)
|
||||
bad("requests must be an array of 1..10000");
|
||||
return run("createBatches", {}, {}, body);
|
||||
}
|
||||
Json BatchesService::remove(const std::string& id) const { return run("deleteBatch", {{"id", id}}); }
|
||||
|
||||
Json FilesService::list(std::optional<int> limit, const std::string& cursor) const {
|
||||
Query q;
|
||||
if (limit) {
|
||||
if (*limit < 1 || *limit > 1000) bad("limit must be 1..1000");
|
||||
q.add("limit", *limit);
|
||||
}
|
||||
q.add_if("cursor", pick_after(cursor));
|
||||
return run("listFiles", {}, q);
|
||||
}
|
||||
Json FilesService::metadata(const std::string& id) const { return run("getFileMetadata", {{"file_id", id}}); }
|
||||
llm::RawResponse FilesService::content(const std::string& id) const {
|
||||
return run_raw("downloadFileContent", {{"file_id", id}});
|
||||
}
|
||||
Json FilesService::upload(const std::string& filename, const std::string& content_type,
|
||||
const std::string& data) const {
|
||||
if (filename.empty() || filename.size() > 255 || filename.find_first_of("/\\\"\r\n") != std::string::npos)
|
||||
bad("bad file name");
|
||||
if (data.empty()) bad("file is empty");
|
||||
clients::body::MultipartBody form;
|
||||
form.add_file("file", filename, content_type.empty() ? "application/octet-stream" : content_type, data);
|
||||
return gateway_->call_multipart(CallPlanner::plan("uploadFile"), form);
|
||||
}
|
||||
Json FilesService::remove(const std::string& id) const { return run("deleteFile", {{"file_id", id}}); }
|
||||
Json FilesService::container_files(const std::string& c, std::optional<int> limit, const std::string& after) const {
|
||||
Query q;
|
||||
if (limit) {
|
||||
if (*limit < 1 || *limit > 1000) bad("limit must be 1..1000");
|
||||
q.add("limit", *limit);
|
||||
}
|
||||
q.add_if("after", pick_after(after));
|
||||
return run("listContainerFiles", {{"container_id", c}}, q);
|
||||
}
|
||||
Json FilesService::container_file(const std::string& c, const std::string& f) const {
|
||||
return run("getContainerFile", {{"container_id", c}, {"file_id", f}});
|
||||
}
|
||||
llm::RawResponse FilesService::container_file_content(const std::string& c, const std::string& f) const {
|
||||
return run_raw("downloadContainerFileContent", {{"container_id", c}, {"file_id", f}});
|
||||
}
|
||||
Json FilesService::promote_container_file(const std::string& c, const std::string& f) const {
|
||||
return run("promoteContainerFile", {{"container_id", c}, {"file_id", f}});
|
||||
}
|
||||
|
||||
Json PresetsService::list(std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listPresets", {}, page_query(o, l));
|
||||
}
|
||||
Json PresetsService::get(const std::string& slug) const { return run("getPreset", {{"slug", slug}}); }
|
||||
Json PresetsService::versions(const std::string& slug, std::optional<int> o, std::optional<int> l) const {
|
||||
return run("listPresetVersions", {{"slug", slug}}, page_query(o, l));
|
||||
}
|
||||
Json PresetsService::version(const std::string& slug, const std::string& v) const {
|
||||
return run("getPresetVersion", {{"slug", slug}, {"version", v}});
|
||||
}
|
||||
Json PresetsService::create_from_chat(const std::string& slug, const Json& b) const {
|
||||
return run("createPresetsChatCompletions", {{"slug", slug}}, {}, b);
|
||||
}
|
||||
Json PresetsService::create_from_messages(const std::string& slug, const Json& b) const {
|
||||
return run("createPresetsMessages", {{"slug", slug}}, {}, b);
|
||||
}
|
||||
Json PresetsService::create_from_responses(const std::string& slug, const Json& b) const {
|
||||
return run("createPresetsResponses", {{"slug", slug}}, {}, b);
|
||||
}
|
||||
|
||||
Json CatalogService::tools(std::optional<int> o, std::optional<int> l, const std::string& fmt) const {
|
||||
Query q = page_query(o, l);
|
||||
q.add_if("api_format", fmt);
|
||||
return run("listTools", {}, q);
|
||||
}
|
||||
Json CatalogService::tool(const std::string& name) const { return run("getTool", {{"name", name}}); }
|
||||
Json CatalogService::benchmarks(const Query& q) const { return run("getBenchmarks", {}, q); }
|
||||
Json CatalogService::task_classifications(const std::string& w) const {
|
||||
return run("getTaskClassifications", {}, Query().add_if("window", w));
|
||||
}
|
||||
Json CatalogService::app_rankings(const Query& q) const { return run("getAppRankings", {}, q); }
|
||||
Json CatalogService::rankings_daily(const Query& q) const { return run("getRankingsDaily", {}, q); }
|
||||
Json CatalogService::session_cost(const Query& q) const { return run("getSessionCost", {}, q); }
|
||||
|
||||
std::string AuthService::authorization_url(const std::string& callback_url, const PkcePair& pkce) {
|
||||
if (callback_url.rfind("https://", 0) != 0 && callback_url.rfind("http://localhost", 0) != 0 &&
|
||||
callback_url.rfind("http://127.0.0.1", 0) != 0)
|
||||
bad("callback_url must be https, or localhost / 127.0.0.1");
|
||||
if (pkce.challenge.empty()) bad("PKCE challenge is empty");
|
||||
return "https://openrouter.ai/auth?callback_url=" + clients::Url::encode_component(callback_url) +
|
||||
"&code_challenge=" + clients::Url::encode_component(pkce.challenge) + "&code_challenge_method=S256";
|
||||
}
|
||||
|
||||
Json AuthService::exchange_code(const std::string& code, const std::string& verifier) const {
|
||||
if (code.empty() || code.size() > 512) bad("code must be 1..512 characters");
|
||||
PkcePair::from_verifier(verifier);
|
||||
Json b = Json::object();
|
||||
b.set("code", Json(code));
|
||||
b.set("code_verifier", Json(verifier));
|
||||
b.set("code_challenge_method", Json("S256"));
|
||||
return run("exchangeAuthCodeForAPIKey", {}, {}, b);
|
||||
}
|
||||
Json AuthService::create_code(const Json& body) const {
|
||||
need(body, {"callback_url", "code_challenge"}, "auth code");
|
||||
return run("createAuthKeysCode", {}, {}, body);
|
||||
}
|
||||
Json AuthService::jwks() const { return run("listOauthJwks"); }
|
||||
Json AuthService::exchange_token(const std::string& policy, const std::string& subject_token,
|
||||
const std::string& subject_token_type) const {
|
||||
if (policy.empty() || subject_token.empty() || subject_token_type.empty()) bad("token exchange needs all three values");
|
||||
Json b = Json::object();
|
||||
b.set("grant_type", Json("urn:ietf:params:oauth:grant-type:token-exchange"));
|
||||
b.set("federation_policy_id", Json(policy));
|
||||
b.set("subject_token", Json(subject_token));
|
||||
b.set("subject_token_type", Json(subject_token_type));
|
||||
return run("createOauthToken", {}, {}, b);
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,118 @@
|
||||
#pragma once
|
||||
|
||||
// The remaining public routes: embeddings, rerank, images, video, audio,
|
||||
// batches, files, containers, presets, server tools and public datasets,
|
||||
// decisions, and the auth / OAuth routes (PKCE code exchange). JSON bodies
|
||||
// are checked against the call table (calls.h).
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "openrouter/gateway.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class EmbeddingsService : public Endpoint {
|
||||
public:
|
||||
explicit EmbeddingsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
// model + input (string, or array of up to Caps::kMaxEmbeddingInputs).
|
||||
Json create(const Json& body) const;
|
||||
Json rerank(const std::string& model, const std::string& query, const std::vector<std::string>& documents,
|
||||
std::optional<int> top_n = std::nullopt) const;
|
||||
// The float vectors of an embeddings response (encoding_format float).
|
||||
static std::vector<std::vector<double>> vectors(const Json& response);
|
||||
};
|
||||
|
||||
class MediaService : public Endpoint {
|
||||
public:
|
||||
explicit MediaService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
Json generate_image(const Json& body) const; // model + prompt
|
||||
Json submit_video(const Json& body) const; // model + prompt; async job (202)
|
||||
Json video_status(const std::string& job_id) const;
|
||||
llm::RawResponse video_content(const std::string& job_id, std::optional<int> index = std::nullopt) const;
|
||||
llm::RawResponse speech(const Json& body) const; // model + input + voice; audio bytes
|
||||
Json transcribe(const Json& body) const; // model + input_audio
|
||||
Json decisions(const Json& body) const; // POST /api/alpha/decisions
|
||||
Json system_one(const Json& body) const; // POST /systemone
|
||||
};
|
||||
|
||||
class BatchesService : public Endpoint {
|
||||
public:
|
||||
explicit BatchesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
Json list(std::optional<int> limit = std::nullopt, const std::string& after = "",
|
||||
const std::string& status = "") const;
|
||||
Json get(const std::string& id) const;
|
||||
// endpoint, model and requests (at most Caps::kMaxBatchRequests). Costs money.
|
||||
Json create(const Json& body) const;
|
||||
Json remove(const std::string& id) const;
|
||||
};
|
||||
|
||||
class FilesService : public Endpoint {
|
||||
public:
|
||||
explicit FilesService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
Json list(std::optional<int> limit = std::nullopt, const std::string& cursor = "") const;
|
||||
Json metadata(const std::string& file_id) const;
|
||||
llm::RawResponse content(const std::string& file_id) const;
|
||||
Json upload(const std::string& filename, const std::string& content_type, const std::string& data) const;
|
||||
Json remove(const std::string& file_id) const;
|
||||
Json container_files(const std::string& container_id, std::optional<int> limit = std::nullopt,
|
||||
const std::string& after = "") const;
|
||||
Json container_file(const std::string& container_id, const std::string& file_id) const;
|
||||
llm::RawResponse container_file_content(const std::string& container_id, const std::string& file_id) const;
|
||||
Json promote_container_file(const std::string& container_id, const std::string& file_id) const;
|
||||
};
|
||||
|
||||
class PresetsService : public Endpoint {
|
||||
public:
|
||||
explicit PresetsService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
Json list(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt) const;
|
||||
Json get(const std::string& slug) const;
|
||||
Json versions(const std::string& slug, std::optional<int> offset = std::nullopt,
|
||||
std::optional<int> limit = std::nullopt) const;
|
||||
Json version(const std::string& slug, const std::string& version) const;
|
||||
// These three CREATE a preset from a request body of that skin (they do not run inference).
|
||||
Json create_from_chat(const std::string& slug, const Json& chat_body) const;
|
||||
Json create_from_messages(const std::string& slug, const Json& messages_body) const;
|
||||
Json create_from_responses(const std::string& slug, const Json& responses_body) const;
|
||||
};
|
||||
|
||||
// Server tool registry and public datasets.
|
||||
class CatalogService : public Endpoint {
|
||||
public:
|
||||
explicit CatalogService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g)) {}
|
||||
Json tools(std::optional<int> offset = std::nullopt, std::optional<int> limit = std::nullopt,
|
||||
const std::string& api_format = "") const;
|
||||
Json tool(const std::string& name) const;
|
||||
Json benchmarks(const Query& query = {}) const;
|
||||
Json task_classifications(const std::string& window = "") const;
|
||||
Json app_rankings(const Query& query = {}) const;
|
||||
Json rankings_daily(const Query& query = {}) const;
|
||||
Json session_cost(const Query& query = {}) const;
|
||||
};
|
||||
|
||||
// PKCE (RFC 7636) helpers and the OAuth routes.
|
||||
struct PkcePair {
|
||||
std::string verifier; // 43..128 chars of [A-Za-z0-9-._~]
|
||||
std::string challenge; // base64url(SHA-256(verifier)), no padding
|
||||
static PkcePair from_verifier(const std::string& verifier);
|
||||
static PkcePair generate(); // verifier from the system CSPRNG
|
||||
};
|
||||
|
||||
class AuthService : public Endpoint {
|
||||
public:
|
||||
explicit AuthService(std::shared_ptr<Gateway> g) : Endpoint(std::move(g), KeyKind::kNone) {}
|
||||
// The browser URL the user visits: https://openrouter.ai/auth?callback_url=...&code_challenge=...&code_challenge_method=S256
|
||||
static std::string authorization_url(const std::string& callback_url, const PkcePair& pkce);
|
||||
// POST /auth/keys: exchanges the code from the redirect for a user key. The
|
||||
// response `key` is a secret: never log it.
|
||||
Json exchange_code(const std::string& code, const std::string& verifier) const;
|
||||
// POST /auth/keys/code (needs the API key): creates an authorization code.
|
||||
Json create_code(const Json& body) const;
|
||||
Json jwks() const;
|
||||
// POST /oauth/token: workload identity token exchange (form encoded).
|
||||
Json exchange_token(const std::string& federation_policy_id, const std::string& subject_token,
|
||||
const std::string& subject_token_type) const;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,151 @@
|
||||
#include "openrouter/plugins.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <set>
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& id, const std::string& why) {
|
||||
throw llm::ApiError::invalid(kVendor, "plugin " + id + ": " + why);
|
||||
}
|
||||
|
||||
bool one_of(const std::string& v, std::initializer_list<const char*> all) {
|
||||
return std::any_of(all.begin(), all.end(), [&](const char* s) { return v == s; });
|
||||
}
|
||||
|
||||
Json strings(const std::vector<std::string>& v) {
|
||||
Json a = Json::array();
|
||||
for (const std::string& s : v) a.push(Json(s));
|
||||
return a;
|
||||
}
|
||||
|
||||
void check_strings(const std::string& id, const char* what, const std::vector<std::string>& v) {
|
||||
if (v.size() > Caps::kMaxProviderList) bad(id, std::string(what) + " has too many entries");
|
||||
for (const std::string& s : v)
|
||||
if (s.empty() || s.size() > 256) bad(id, std::string(what) + " has an empty or over-long entry");
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
const std::vector<std::string>& Plugin::known_ids() {
|
||||
// Spec: ChatRequest.plugins oneOf (OpenRouter OpenAPI, 14 entries).
|
||||
static const std::vector<std::string> kIds = {
|
||||
"auto-router", "auto-beta-router", "moderation", "web", "web-fetch", "file-parser",
|
||||
"response-healing", "context-compression", "pareto-router", "fusion",
|
||||
"switchyard-router", "jev-router", "alignment", "armor"};
|
||||
return kIds;
|
||||
}
|
||||
|
||||
bool Plugin::is_known(const std::string& id) {
|
||||
const auto& all = known_ids();
|
||||
return std::find(all.begin(), all.end(), id) != all.end();
|
||||
}
|
||||
|
||||
Json Plugin::to_json() const {
|
||||
validate();
|
||||
Json o = Json::object();
|
||||
o.set("id", Json(id()));
|
||||
if (enabled) o.set("enabled", Json(*enabled));
|
||||
fill(o);
|
||||
return o;
|
||||
}
|
||||
|
||||
void WebSearchPlugin::validate() const {
|
||||
if (!engine.empty() && !one_of(engine, {"native", "exa", "parallel", "firecrawl", "perplexity", "auto"}))
|
||||
bad(id(), "engine must be native exa parallel firecrawl perplexity or auto");
|
||||
if (!mode.empty() && !one_of(mode, {"instant", "fast", "auto", "deep-lite", "deep", "deep-reasoning", "turbo", "basic", "advanced"}))
|
||||
bad(id(), "unknown mode");
|
||||
if (max_results && *max_results < 1) bad(id(), "max_results must be >= 1");
|
||||
if (max_uses && *max_uses < 1) bad(id(), "max_uses must be >= 1");
|
||||
check_strings(id(), "include_domains", include_domains);
|
||||
check_strings(id(), "exclude_domains", exclude_domains);
|
||||
}
|
||||
|
||||
void WebSearchPlugin::fill(Json& o) const {
|
||||
if (!engine.empty()) o.set("engine", Json(engine));
|
||||
if (max_results) o.set("max_results", Json(*max_results));
|
||||
if (max_uses) o.set("max_uses", Json(*max_uses));
|
||||
if (!search_prompt.empty()) o.set("search_prompt", Json(search_prompt));
|
||||
if (!mode.empty()) o.set("mode", Json(mode));
|
||||
if (!include_domains.empty()) o.set("include_domains", strings(include_domains));
|
||||
if (!exclude_domains.empty()) o.set("exclude_domains", strings(exclude_domains));
|
||||
}
|
||||
|
||||
void FileParserPlugin::validate() const {
|
||||
if (!pdf_engine.empty() && !one_of(pdf_engine, {"mistral-ocr", "native", "cloudflare-ai", "pdf-text"}))
|
||||
bad(id(), "pdf engine must be mistral-ocr native cloudflare-ai or pdf-text");
|
||||
}
|
||||
|
||||
void FileParserPlugin::fill(Json& o) const {
|
||||
if (pdf_engine.empty()) return;
|
||||
Json pdf = Json::object();
|
||||
pdf.set("engine", Json(pdf_engine));
|
||||
o.set("pdf", pdf);
|
||||
}
|
||||
|
||||
void ContextCompressionPlugin::fill(Json& o) const { o.set("engine", Json("middle-out")); }
|
||||
|
||||
void AutoRouterPlugin::validate() const {
|
||||
if (!cost_tier.empty() && !one_of(cost_tier, {"low", "medium", "high", "xhigh", "max"}))
|
||||
bad(id(), "cost_tier must be low medium high xhigh or max");
|
||||
check_strings(id(), "allowed_models", allowed_models);
|
||||
check_strings(id(), "excluded_models", excluded_models);
|
||||
}
|
||||
|
||||
void AutoRouterPlugin::fill(Json& o) const {
|
||||
if (!allowed_models.empty()) o.set("allowed_models", strings(allowed_models));
|
||||
if (!excluded_models.empty()) o.set("excluded_models", strings(excluded_models));
|
||||
if (!cost_tier.empty()) o.set("cost_tier", Json(cost_tier));
|
||||
if (pin_model) o.set("pin_model", Json(*pin_model));
|
||||
}
|
||||
|
||||
void WebFetchPlugin::validate() const {
|
||||
if (max_content_tokens && *max_content_tokens < 1) bad(id(), "max_content_tokens must be >= 1");
|
||||
if (max_uses && *max_uses < 1) bad(id(), "max_uses must be >= 1");
|
||||
check_strings(id(), "allowed_domains", allowed_domains);
|
||||
check_strings(id(), "blocked_domains", blocked_domains);
|
||||
}
|
||||
|
||||
void WebFetchPlugin::fill(Json& o) const {
|
||||
if (!allowed_domains.empty()) o.set("allowed_domains", strings(allowed_domains));
|
||||
if (!blocked_domains.empty()) o.set("blocked_domains", strings(blocked_domains));
|
||||
if (max_content_tokens) o.set("max_content_tokens", Json(*max_content_tokens));
|
||||
if (max_uses) o.set("max_uses", Json(*max_uses));
|
||||
}
|
||||
|
||||
RawPlugin::RawPlugin(std::string id, Json fields) : id_(std::move(id)), fields_(std::move(fields)) {}
|
||||
|
||||
void RawPlugin::validate() const {
|
||||
if (!is_known(id_)) bad(id_, "not a plugin id in the spec");
|
||||
if (!fields_.is_null() && !fields_.is_object()) bad(id_, "fields must be an object");
|
||||
if (fields_.is_object() && (fields_.has("id") || fields_.has("enabled")))
|
||||
bad(id_, "set id/enabled on the plugin, not in fields");
|
||||
}
|
||||
|
||||
void RawPlugin::fill(Json& o) const {
|
||||
if (!fields_.is_object()) return;
|
||||
for (const auto& [k, v] : fields_.as_object()) o.set(k, v);
|
||||
}
|
||||
|
||||
Json plugins_to_json(const std::vector<PluginPtr>& plugins) {
|
||||
if (plugins.size() > Caps::kMaxPlugins) throw llm::ApiError::invalid(kVendor, "too many plugins");
|
||||
std::set<std::string> seen;
|
||||
Json a = Json::array();
|
||||
for (const PluginPtr& p : plugins) {
|
||||
if (!p) throw llm::ApiError::invalid(kVendor, "null plugin");
|
||||
if (!seen.insert(p->id()).second)
|
||||
throw llm::ApiError::invalid(kVendor, "plugin listed twice: " + p->id());
|
||||
a.push(p->to_json());
|
||||
}
|
||||
return a;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,146 @@
|
||||
#pragma once
|
||||
|
||||
// Request plugins (`plugins` array on chat, responses and messages requests;
|
||||
// https://openrouter.ai/docs/guides/features/plugins). One class per plugin
|
||||
// the spec schemas define fields for; the remaining spec plugins
|
||||
// (pareto-router, fusion, switchyard-router, jev-router, alignment, armor,
|
||||
// auto-beta-router) go through RawPlugin, which checks the id against the
|
||||
// spec's list and passes the object through.
|
||||
//
|
||||
// Notes from the docs: the `transforms` request parameter is gone, the
|
||||
// ContextCompressionPlugin (middle-out) replaces it; the ":online" model
|
||||
// variant and the web plugin are deprecated in favour of the
|
||||
// `openrouter:web_search` server tool (see server_tools.h).
|
||||
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class Plugin {
|
||||
public:
|
||||
virtual ~Plugin() = default;
|
||||
virtual std::string id() const = 0;
|
||||
// Throws llm::ApiError::invalid.
|
||||
virtual void validate() const {}
|
||||
// The object as sent: {"id": ..., ...}. Calls validate().
|
||||
clients::body::Json to_json() const;
|
||||
|
||||
// false disables the plugin for this request even when the account enables it.
|
||||
std::optional<bool> enabled;
|
||||
|
||||
static const std::vector<std::string>& known_ids();
|
||||
static bool is_known(const std::string& id);
|
||||
|
||||
protected:
|
||||
virtual void fill(clients::body::Json& object) const = 0;
|
||||
};
|
||||
|
||||
using PluginPtr = std::shared_ptr<const Plugin>;
|
||||
|
||||
// "web": engine native exa parallel firecrawl perplexity auto.
|
||||
class WebSearchPlugin : public Plugin {
|
||||
public:
|
||||
std::string engine;
|
||||
std::optional<int> max_results;
|
||||
std::optional<int> max_uses;
|
||||
std::string search_prompt;
|
||||
std::string mode; // instant fast auto deep-lite deep deep-reasoning turbo basic advanced
|
||||
std::vector<std::string> include_domains;
|
||||
std::vector<std::string> exclude_domains;
|
||||
std::string id() const override { return "web"; }
|
||||
void validate() const override;
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json& object) const override;
|
||||
};
|
||||
|
||||
// "file-parser": pdf engine mistral-ocr native cloudflare-ai pdf-text.
|
||||
class FileParserPlugin : public Plugin {
|
||||
public:
|
||||
std::string pdf_engine;
|
||||
std::string id() const override { return "file-parser"; }
|
||||
void validate() const override;
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json& object) const override;
|
||||
};
|
||||
|
||||
// "response-healing": repairs malformed JSON from structured-output requests.
|
||||
class ResponseHealingPlugin : public Plugin {
|
||||
public:
|
||||
std::string id() const override { return "response-healing"; }
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json&) const override {}
|
||||
};
|
||||
|
||||
class ModerationPlugin : public Plugin {
|
||||
public:
|
||||
std::string id() const override { return "moderation"; }
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json&) const override {}
|
||||
};
|
||||
|
||||
// "context-compression": middle-out truncation of prompts over the context.
|
||||
class ContextCompressionPlugin : public Plugin {
|
||||
public:
|
||||
std::string id() const override { return "context-compression"; }
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json& object) const override;
|
||||
};
|
||||
|
||||
// "auto-router": model patterns may use wildcards ("anthropic/*").
|
||||
class AutoRouterPlugin : public Plugin {
|
||||
public:
|
||||
std::vector<std::string> allowed_models;
|
||||
std::vector<std::string> excluded_models;
|
||||
std::string cost_tier; // low medium high xhigh max
|
||||
std::optional<bool> pin_model;
|
||||
std::string id() const override { return "auto-router"; }
|
||||
void validate() const override;
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json& object) const override;
|
||||
};
|
||||
|
||||
// "web-fetch".
|
||||
class WebFetchPlugin : public Plugin {
|
||||
public:
|
||||
std::vector<std::string> allowed_domains;
|
||||
std::vector<std::string> blocked_domains;
|
||||
std::optional<int> max_content_tokens;
|
||||
std::optional<int> max_uses;
|
||||
std::string id() const override { return "web-fetch"; }
|
||||
void validate() const override;
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json& object) const override;
|
||||
};
|
||||
|
||||
// A spec plugin without a typed class: id plus a free-form object.
|
||||
class RawPlugin : public Plugin {
|
||||
public:
|
||||
RawPlugin(std::string id, clients::body::Json fields);
|
||||
std::string id() const override { return id_; }
|
||||
void validate() const override;
|
||||
|
||||
protected:
|
||||
void fill(clients::body::Json& object) const override;
|
||||
|
||||
private:
|
||||
std::string id_;
|
||||
clients::body::Json fields_;
|
||||
};
|
||||
|
||||
// The `plugins` array. Throws when over Caps::kMaxPlugins, on a null entry
|
||||
// or on a repeated plugin id.
|
||||
clients::body::Json plugins_to_json(const std::vector<PluginPtr>& plugins);
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,257 @@
|
||||
#include "openrouter/provider_preferences.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <cstdlib>
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) {
|
||||
throw llm::ApiError::invalid(kVendor, "provider: " + why);
|
||||
}
|
||||
|
||||
bool slug_ok(const std::string& s) {
|
||||
// Provider slugs look like "openai", "amazon-bedrock", "anthropic/2".
|
||||
if (s.empty() || s.size() > 64) return false;
|
||||
for (char c : s)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '-' || c == '_' || c == '.' || c == '/'))
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
void check_list(const char* what, const std::vector<std::string>& v) {
|
||||
if (v.size() > Caps::kMaxProviderList) bad(std::string(what) + " has too many entries");
|
||||
for (const std::string& s : v)
|
||||
if (!slug_ok(s)) bad(std::string(what) + " has a malformed provider slug");
|
||||
}
|
||||
|
||||
Json string_array(const std::vector<std::string>& v) {
|
||||
Json a = Json::array();
|
||||
for (const std::string& s : v) a.push(Json(s));
|
||||
return a;
|
||||
}
|
||||
|
||||
bool decimal_ok(const std::string& s) {
|
||||
if (s.empty() || s.size() > 32) return false;
|
||||
char* end = nullptr;
|
||||
double d = std::strtod(s.c_str(), &end);
|
||||
return end == s.c_str() + s.size() && std::isfinite(d) && d >= 0;
|
||||
}
|
||||
|
||||
void check_percentiles(const char* what, const std::map<std::string, double>& m) {
|
||||
for (const auto& [k, v] : m) {
|
||||
if (k != "p50" && k != "p75" && k != "p90" && k != "p99") bad(std::string(what) + " key must be p50 p75 p90 or p99");
|
||||
if (!(v >= 0) || !std::isfinite(v)) bad(std::string(what) + " must be a non-negative number");
|
||||
}
|
||||
}
|
||||
|
||||
Json percentile_or_number(const std::optional<double>& one, const std::map<std::string, double>& m) {
|
||||
if (!m.empty()) {
|
||||
Json o = Json::object();
|
||||
for (const auto& [k, v] : m) o.set(k, Json(v));
|
||||
return o;
|
||||
}
|
||||
return Json(*one);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool ProviderPreferences::valid_quantization(const std::string& q) {
|
||||
static const char* const kAll[] = {"int4", "int8", "fp4", "mxfp4", "nvfp4", "fp6",
|
||||
"fp8", "mxfp8", "fp16", "bf16", "fp32", "unknown"};
|
||||
return std::any_of(std::begin(kAll), std::end(kAll), [&](const char* s) { return q == s; });
|
||||
}
|
||||
|
||||
bool ProviderPreferences::valid_sort(const std::string& s) {
|
||||
return s == "price" || s == "throughput" || s == "latency" || s == "exacto";
|
||||
}
|
||||
|
||||
bool ProviderPreferences::empty() const {
|
||||
return order.empty() && only.empty() && ignore.empty() && !allow_fallbacks &&
|
||||
!require_parameters && data_collection.empty() && !zdr && !enforce_distillable_text &&
|
||||
quantizations.empty() && sort.empty() && max_price.empty() &&
|
||||
!preferred_max_latency && preferred_max_latency_percentiles.empty() &&
|
||||
!preferred_min_throughput && preferred_min_throughput_percentiles.empty() &&
|
||||
!(options.is_object() && !options.as_object().empty());
|
||||
}
|
||||
|
||||
void ProviderPreferences::validate() const {
|
||||
check_list("order", order);
|
||||
check_list("only", only);
|
||||
check_list("ignore", ignore);
|
||||
if (!data_collection.empty() && data_collection != "allow" && data_collection != "deny")
|
||||
bad("data_collection must be allow or deny");
|
||||
if (quantizations.size() > Caps::kMaxQuantizations) bad("too many quantizations");
|
||||
for (const std::string& q : quantizations)
|
||||
if (!valid_quantization(q)) bad("unknown quantization: " + q);
|
||||
if (!sort.empty() && !valid_sort(sort)) bad("sort must be price, throughput, latency or exacto");
|
||||
if (!sort_partition.empty()) {
|
||||
if (sort.empty()) bad("sort_partition needs sort");
|
||||
if (sort_partition != "model" && sort_partition != "none") bad("sort_partition must be model or none");
|
||||
}
|
||||
static const char* const kPrice[] = {"prompt", "completion", "image", "audio", "request"};
|
||||
for (const auto& [k, v] : max_price) {
|
||||
if (std::none_of(std::begin(kPrice), std::end(kPrice), [&](const char* p) { return k == p; }))
|
||||
bad("max_price key must be prompt, completion, image, audio or request");
|
||||
if (!decimal_ok(v)) bad("max_price." + k + " must be a non-negative decimal string");
|
||||
}
|
||||
if (preferred_max_latency && !preferred_max_latency_percentiles.empty())
|
||||
bad("preferred_max_latency: send one number or percentiles, not both");
|
||||
if (preferred_min_throughput && !preferred_min_throughput_percentiles.empty())
|
||||
bad("preferred_min_throughput: send one number or percentiles, not both");
|
||||
if (preferred_max_latency && !(*preferred_max_latency >= 0)) bad("preferred_max_latency must be >= 0");
|
||||
if (preferred_min_throughput && !(*preferred_min_throughput >= 0)) bad("preferred_min_throughput must be >= 0");
|
||||
check_percentiles("preferred_max_latency", preferred_max_latency_percentiles);
|
||||
check_percentiles("preferred_min_throughput", preferred_min_throughput_percentiles);
|
||||
if (!options.is_null() && !options.is_object()) bad("options must be an object");
|
||||
}
|
||||
|
||||
Json ProviderPreferences::to_json() const {
|
||||
validate();
|
||||
Json o = Json::object();
|
||||
if (!order.empty()) o.set("order", string_array(order));
|
||||
if (!only.empty()) o.set("only", string_array(only));
|
||||
if (!ignore.empty()) o.set("ignore", string_array(ignore));
|
||||
if (allow_fallbacks) o.set("allow_fallbacks", Json(*allow_fallbacks));
|
||||
if (require_parameters) o.set("require_parameters", Json(*require_parameters));
|
||||
if (!data_collection.empty()) o.set("data_collection", Json(data_collection));
|
||||
if (zdr) o.set("zdr", Json(*zdr));
|
||||
if (enforce_distillable_text) o.set("enforce_distillable_text", Json(*enforce_distillable_text));
|
||||
if (!quantizations.empty()) o.set("quantizations", string_array(quantizations));
|
||||
if (!sort.empty()) {
|
||||
if (sort_partition.empty()) {
|
||||
o.set("sort", Json(sort));
|
||||
} else {
|
||||
Json s = Json::object();
|
||||
s.set("by", Json(sort));
|
||||
s.set("partition", Json(sort_partition));
|
||||
o.set("sort", s);
|
||||
}
|
||||
}
|
||||
if (!max_price.empty()) {
|
||||
Json p = Json::object();
|
||||
for (const auto& [k, v] : max_price) p.set(k, Json(v));
|
||||
o.set("max_price", p);
|
||||
}
|
||||
if (preferred_max_latency || !preferred_max_latency_percentiles.empty())
|
||||
o.set("preferred_max_latency",
|
||||
percentile_or_number(preferred_max_latency, preferred_max_latency_percentiles));
|
||||
if (preferred_min_throughput || !preferred_min_throughput_percentiles.empty())
|
||||
o.set("preferred_min_throughput",
|
||||
percentile_or_number(preferred_min_throughput, preferred_min_throughput_percentiles));
|
||||
if (options.is_object() && !options.as_object().empty()) o.set("options", options);
|
||||
return o;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::from_json(const Json& doc) {
|
||||
if (!doc.is_object()) bad("must be a JSON object");
|
||||
ProviderPreferences p;
|
||||
auto strings = [&](const char* key, std::vector<std::string>& out) {
|
||||
const Json* v = doc.find(key);
|
||||
if (!v || v->is_null()) return;
|
||||
if (!v->is_array()) bad(std::string(key) + " must be an array");
|
||||
for (const Json& s : v->as_array()) {
|
||||
if (!s.is_string()) bad(std::string(key) + " must hold strings");
|
||||
out.push_back(s.as_string());
|
||||
}
|
||||
};
|
||||
auto flag = [&](const char* key, std::optional<bool>& out) {
|
||||
const Json* v = doc.find(key);
|
||||
if (!v || v->is_null()) return;
|
||||
if (!v->is_bool()) bad(std::string(key) + " must be true or false");
|
||||
out = v->as_bool();
|
||||
};
|
||||
auto cutoff = [&](const char* key, std::optional<double>& one, std::map<std::string, double>& many) {
|
||||
const Json* v = doc.find(key);
|
||||
if (!v || v->is_null()) return;
|
||||
if (v->is_number()) {
|
||||
one = v->as_number();
|
||||
} else if (v->is_object()) {
|
||||
for (const auto& [k, n] : v->as_object()) {
|
||||
if (n.is_null()) continue;
|
||||
if (!n.is_number()) bad(std::string(key) + " percentiles must be numbers");
|
||||
many[k] = n.as_number();
|
||||
}
|
||||
} else {
|
||||
bad(std::string(key) + " must be a number or an object");
|
||||
}
|
||||
};
|
||||
strings("order", p.order);
|
||||
strings("only", p.only);
|
||||
strings("ignore", p.ignore);
|
||||
strings("quantizations", p.quantizations);
|
||||
flag("allow_fallbacks", p.allow_fallbacks);
|
||||
flag("require_parameters", p.require_parameters);
|
||||
flag("zdr", p.zdr);
|
||||
flag("enforce_distillable_text", p.enforce_distillable_text);
|
||||
if (const Json* v = doc.find("data_collection"); v && v->is_string()) p.data_collection = v->as_string();
|
||||
if (const Json* v = doc.find("sort"); v && !v->is_null()) {
|
||||
if (v->is_string()) {
|
||||
p.sort = v->as_string();
|
||||
} else if (v->is_object()) {
|
||||
if (const Json* by = v->find("by"); by && by->is_string()) p.sort = by->as_string();
|
||||
if (const Json* part = v->find("partition"); part && part->is_string()) p.sort_partition = part->as_string();
|
||||
} else {
|
||||
bad("sort must be a string or an object");
|
||||
}
|
||||
}
|
||||
if (const Json* v = doc.find("max_price"); v && v->is_object())
|
||||
for (const auto& [k, n] : v->as_object()) {
|
||||
if (!n.is_string()) bad("max_price values must be strings");
|
||||
p.max_price[k] = n.as_string();
|
||||
}
|
||||
cutoff("preferred_max_latency", p.preferred_max_latency, p.preferred_max_latency_percentiles);
|
||||
cutoff("preferred_min_throughput", p.preferred_min_throughput, p.preferred_min_throughput_percentiles);
|
||||
if (const Json* v = doc.find("options"); v && v->is_object()) p.options = *v;
|
||||
p.validate();
|
||||
return p;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::only_providers(std::vector<std::string> slugs) {
|
||||
ProviderPreferences p;
|
||||
p.only = std::move(slugs);
|
||||
return p;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::prefer(std::vector<std::string> slugs, bool fallbacks) {
|
||||
ProviderPreferences p;
|
||||
p.order = std::move(slugs);
|
||||
p.allow_fallbacks = fallbacks;
|
||||
return p;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::cheapest() {
|
||||
ProviderPreferences p;
|
||||
p.sort = "price";
|
||||
return p;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::fastest() {
|
||||
ProviderPreferences p;
|
||||
p.sort = "throughput";
|
||||
return p;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::lowest_latency() {
|
||||
ProviderPreferences p;
|
||||
p.sort = "latency";
|
||||
return p;
|
||||
}
|
||||
|
||||
ProviderPreferences ProviderPreferences::no_data_collection(bool zdr_only) {
|
||||
ProviderPreferences p;
|
||||
p.data_collection = "deny";
|
||||
if (zdr_only) p.zdr = true;
|
||||
return p;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,61 @@
|
||||
#pragma once
|
||||
|
||||
// ProviderPreferences: the `provider` object of a request (provider routing,
|
||||
// https://openrouter.ai/docs/guides/routing/provider-selection). Field names
|
||||
// and enums follow the spec schema ProviderPreferences. Validation is local
|
||||
// and loud; to_json() emits only what was set.
|
||||
|
||||
#include <map>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class ProviderPreferences {
|
||||
public:
|
||||
// Provider slugs, in the order to try them. Empty: not sent.
|
||||
std::vector<std::string> order;
|
||||
std::vector<std::string> only; // allow list
|
||||
std::vector<std::string> ignore; // deny list
|
||||
std::optional<bool> allow_fallbacks;
|
||||
std::optional<bool> require_parameters;
|
||||
std::string data_collection; // "", "allow", "deny"
|
||||
std::optional<bool> zdr; // zero data retention endpoints only
|
||||
std::optional<bool> enforce_distillable_text;
|
||||
std::vector<std::string> quantizations; // int4 int8 fp4 mxfp4 nvfp4 fp6 fp8 mxfp8 fp16 bf16 fp32 unknown
|
||||
std::string sort; // "", price, throughput, latency, exacto
|
||||
std::string sort_partition; // "", model, none (only with sort; sends {by, partition})
|
||||
// USD per million tokens (prompt, completion), per image, audio unit,
|
||||
// request. Values are decimal strings as the spec types them.
|
||||
std::map<std::string, std::string> max_price;
|
||||
// Latency ceilings in seconds / throughput floors in tokens per second:
|
||||
// either one number (applies to p50) or percentile keys p50 p75 p90 p99.
|
||||
std::optional<double> preferred_max_latency;
|
||||
std::map<std::string, double> preferred_max_latency_percentiles;
|
||||
std::optional<double> preferred_min_throughput;
|
||||
std::map<std::string, double> preferred_min_throughput_percentiles;
|
||||
// Per-provider passthrough options: provider slug -> object.
|
||||
clients::body::Json options; // null or object
|
||||
|
||||
static bool valid_quantization(const std::string& q);
|
||||
static bool valid_sort(const std::string& s);
|
||||
|
||||
bool empty() const;
|
||||
// Throws llm::ApiError::invalid.
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
static ProviderPreferences from_json(const clients::body::Json& doc);
|
||||
|
||||
// Convenience builders for the common intents.
|
||||
static ProviderPreferences only_providers(std::vector<std::string> slugs);
|
||||
static ProviderPreferences prefer(std::vector<std::string> slugs, bool allow_fallbacks = true);
|
||||
static ProviderPreferences cheapest();
|
||||
static ProviderPreferences fastest(); // sort by throughput
|
||||
static ProviderPreferences lowest_latency();
|
||||
static ProviderPreferences no_data_collection(bool zdr_only = false);
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,73 @@
|
||||
#include "openrouter/reasoning.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
[[noreturn]] void bad(const std::string& why) {
|
||||
throw llm::ApiError::invalid(kVendor, "reasoning: " + why);
|
||||
}
|
||||
} // namespace
|
||||
|
||||
bool ReasoningConfig::valid_effort(const std::string& e) {
|
||||
static const char* const kAll[] = {"max", "xhigh", "high", "medium", "low", "minimal", "none"};
|
||||
return std::any_of(std::begin(kAll), std::end(kAll), [&](const char* s) { return e == s; });
|
||||
}
|
||||
|
||||
bool ReasoningConfig::empty() const {
|
||||
return effort.empty() && summary.empty() && !max_tokens && !enabled && !exclude &&
|
||||
mode.empty() && context.empty();
|
||||
}
|
||||
|
||||
void ReasoningConfig::validate() const {
|
||||
if (!effort.empty() && !valid_effort(effort))
|
||||
bad("effort must be one of max xhigh high medium low minimal none");
|
||||
if (!summary.empty() && summary != "auto" && summary != "concise" && summary != "detailed")
|
||||
bad("summary must be auto, concise or detailed");
|
||||
if (max_tokens && *max_tokens < 1) bad("max_tokens must be >= 1");
|
||||
if (!effort.empty() && max_tokens) bad("send effort or max_tokens, not both");
|
||||
if (!mode.empty() && mode != "standard" && mode != "pro") bad("mode must be standard or pro");
|
||||
if (!context.empty() && context != "auto" && context != "all_turns" && context != "current_turn")
|
||||
bad("context must be auto, all_turns or current_turn");
|
||||
}
|
||||
|
||||
Json ReasoningConfig::to_json() const {
|
||||
validate();
|
||||
Json o = Json::object();
|
||||
if (!effort.empty()) o.set("effort", Json(effort));
|
||||
if (!summary.empty()) o.set("summary", Json(summary));
|
||||
if (max_tokens) o.set("max_tokens", Json(*max_tokens));
|
||||
if (enabled) o.set("enabled", Json(*enabled));
|
||||
if (exclude) o.set("exclude", Json(*exclude));
|
||||
if (!mode.empty()) o.set("mode", Json(mode));
|
||||
if (!context.empty()) o.set("context", Json(context));
|
||||
return o;
|
||||
}
|
||||
|
||||
ReasoningConfig ReasoningConfig::with_effort(const std::string& effort) {
|
||||
ReasoningConfig r;
|
||||
r.effort = effort;
|
||||
r.validate();
|
||||
return r;
|
||||
}
|
||||
|
||||
ReasoningConfig ReasoningConfig::with_budget(int max_tokens) {
|
||||
ReasoningConfig r;
|
||||
r.max_tokens = max_tokens;
|
||||
r.validate();
|
||||
return r;
|
||||
}
|
||||
|
||||
ReasoningConfig ReasoningConfig::hidden(const std::string& effort) {
|
||||
ReasoningConfig r = with_effort(effort);
|
||||
r.exclude = true;
|
||||
return r;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,38 @@
|
||||
#pragma once
|
||||
|
||||
// ReasoningConfig: the unified `reasoning` request parameter
|
||||
// (https://openrouter.ai/docs/guides/best-practices/reasoning-tokens).
|
||||
// effort and max_tokens are mutually exclusive on the wire (the docs say the
|
||||
// unified parameter accepts either, not both). `reasoning_effort` (the flat
|
||||
// shorthand) is a separate request field and cannot be combined with this
|
||||
// object; ChatParams enforces that.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct ReasoningConfig {
|
||||
std::string effort; // "", max xhigh high medium low minimal none
|
||||
std::string summary; // "", auto concise detailed
|
||||
std::optional<int> max_tokens; // reasoning token budget
|
||||
std::optional<bool> enabled;
|
||||
std::optional<bool> exclude; // reason, but do not return the text (still billed)
|
||||
std::string mode; // "", standard, pro (OpenAI GPT-5.6 and newer)
|
||||
std::string context; // "", auto, all_turns, current_turn
|
||||
|
||||
static bool valid_effort(const std::string& e);
|
||||
|
||||
bool empty() const;
|
||||
// Throws llm::ApiError::invalid.
|
||||
void validate() const;
|
||||
clients::body::Json to_json() const;
|
||||
|
||||
static ReasoningConfig with_effort(const std::string& effort);
|
||||
static ReasoningConfig with_budget(int max_tokens);
|
||||
static ReasoningConfig hidden(const std::string& effort);
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,68 @@
|
||||
#include "openrouter/server_tools.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid(kVendor, "tool: " + why); }
|
||||
} // namespace
|
||||
|
||||
Json ServerTools::function(const std::string& name, const std::string& description,
|
||||
const Json& parameters, bool strict) {
|
||||
if (name.empty() || name.size() > 64) bad("function name must be 1..64 characters");
|
||||
for (char c : name)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '_' || c == '-'))
|
||||
bad("function name may only use letters, digits, _ and -");
|
||||
if (!parameters.is_object()) bad("parameters must be a JSON Schema object");
|
||||
Json fn = Json::object();
|
||||
fn.set("name", Json(name));
|
||||
if (!description.empty()) fn.set("description", Json(description));
|
||||
fn.set("parameters", parameters);
|
||||
if (strict) fn.set("strict", Json(true));
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("function"));
|
||||
t.set("function", fn);
|
||||
return t;
|
||||
}
|
||||
|
||||
Json ServerTools::function(const llm::ToolSpec& spec) {
|
||||
return function(spec.name, spec.description, spec.parameters);
|
||||
}
|
||||
|
||||
Json ServerTools::server(const std::string& type, const Json& parameters) {
|
||||
static const std::string kPrefix = "openrouter:";
|
||||
if (type.rfind(kPrefix, 0) != 0 || type.size() == kPrefix.size() || type.size() > 96)
|
||||
bad("server tool type must look like openrouter:<name>");
|
||||
for (std::size_t i = kPrefix.size(); i < type.size(); ++i) {
|
||||
char c = type[i];
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '_' || c == ':'))
|
||||
bad("server tool type may only use a-z 0-9 _ :");
|
||||
}
|
||||
Json t = Json::object();
|
||||
t.set("type", Json(type));
|
||||
if (!parameters.is_null()) {
|
||||
if (!parameters.is_object()) bad("parameters must be an object");
|
||||
t.set("parameters", parameters);
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
Json ServerTools::web_search(const Json& p) { return server("openrouter:web_search", p); }
|
||||
Json ServerTools::web_fetch(const Json& p) { return server("openrouter:web_fetch", p); }
|
||||
Json ServerTools::datetime(const Json& p) { return server("openrouter:datetime", p); }
|
||||
|
||||
const std::vector<std::string>& ServerTools::known_types() {
|
||||
static const std::vector<std::string> kTypes = {
|
||||
"openrouter:advisor", "openrouter:bash", "openrouter:datetime",
|
||||
"openrouter:files", "openrouter:fusion", "openrouter:image_generation",
|
||||
"openrouter:experimental__search_models", "openrouter:subagent",
|
||||
"openrouter:web_fetch", "openrouter:web_search"};
|
||||
return kTypes;
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,39 @@
|
||||
#pragma once
|
||||
|
||||
// Builders for entries of the request `tools` array: ordinary function tools
|
||||
// and OpenRouter server tools ("openrouter:*", run by OpenRouter itself, see
|
||||
// https://openrouter.ai/docs/guides/features/server-tools). The tool type
|
||||
// names come from the spec schema ChatFunctionTool; a generic envelope
|
||||
// ChatDynamicServerTool accepts any registered `openrouter:*` type, so
|
||||
// server() takes any well-formed name.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/chat_types.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
class ServerTools {
|
||||
public:
|
||||
// {"type":"function","function":{"name","description","parameters"[,"strict"]}}
|
||||
static clients::body::Json function(const std::string& name, const std::string& description,
|
||||
const clients::body::Json& parameters,
|
||||
bool strict = false);
|
||||
static clients::body::Json function(const llm::ToolSpec& spec);
|
||||
|
||||
// {"type":"openrouter:<name>"[,"parameters":{...}]}. `type` must start with
|
||||
// "openrouter:" and use [a-z0-9_:] after it. `parameters` null: omitted.
|
||||
static clients::body::Json server(const std::string& type,
|
||||
const clients::body::Json& parameters = clients::body::Json());
|
||||
|
||||
static clients::body::Json web_search(const clients::body::Json& parameters = clients::body::Json());
|
||||
static clients::body::Json web_fetch(const clients::body::Json& parameters = clients::body::Json());
|
||||
static clients::body::Json datetime(const clients::body::Json& parameters = clients::body::Json());
|
||||
|
||||
// Server tool types named by the spec (not exhaustive: see server()).
|
||||
static const std::vector<std::string>& known_types();
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,148 @@
|
||||
#include "openrouter/stream_summary.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
#include "openrouter/call_plan.h"
|
||||
#include "openrouter/caps.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
|
||||
std::string str(const Json* o, const char* k) {
|
||||
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
|
||||
return v && v->is_string() ? v->as_string() : "";
|
||||
}
|
||||
|
||||
int num(const Json* o, const char* k) {
|
||||
const Json* v = o && o->is_object() ? o->find(k) : nullptr;
|
||||
return v && v->is_number() ? static_cast<int>(v->as_int()) : 0;
|
||||
}
|
||||
|
||||
std::string type_of(const llm::SseEvent& ev) {
|
||||
std::string t = str(&ev.data, "type");
|
||||
return t.empty() ? ev.event : t;
|
||||
}
|
||||
|
||||
void count(StreamSummary& s, const std::string& type) {
|
||||
if (++s.events > Caps::kMaxStreamEvents)
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "stream over the event cap");
|
||||
if (s.event_counts.size() < 256 || s.event_counts.count(type)) ++s.event_counts[type];
|
||||
}
|
||||
|
||||
void append(std::string& dst, const std::string& add) {
|
||||
if (dst.size() + add.size() > Caps::kMaxStreamTextBytes)
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "stream text over the byte cap");
|
||||
dst += add;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string ResponsesStreamParser::feed(const llm::SseEvent& ev) {
|
||||
std::string type = type_of(ev);
|
||||
count(summary_, type);
|
||||
const Json& d = ev.data;
|
||||
if (type == "response.output_text.delta") {
|
||||
std::string delta = str(&d, "delta");
|
||||
append(summary_.text, delta);
|
||||
return delta;
|
||||
}
|
||||
if (type == "response.reasoning_text.delta" || type == "response.reasoning_summary_text.delta") {
|
||||
append(summary_.reasoning, str(&d, "delta"));
|
||||
return "";
|
||||
}
|
||||
if (type == "response.created" || type == "response.in_progress" ||
|
||||
type == "response.completed" || type == "response.incomplete" || type == "response.failed") {
|
||||
const Json* r = d.is_object() ? d.find("response") : nullptr;
|
||||
if (r && r->is_object()) {
|
||||
if (summary_.id.empty()) summary_.id = str(r, "id");
|
||||
if (summary_.model.empty()) summary_.model = str(r, "model");
|
||||
std::string status = str(r, "status");
|
||||
if (!status.empty() && type != "response.created" && type != "response.in_progress")
|
||||
summary_.status = status;
|
||||
if (const Json* u = r->find("usage"); u && u->is_object()) {
|
||||
llm::Usage usage;
|
||||
usage.input_tokens = num(u, "input_tokens");
|
||||
usage.output_tokens = num(u, "output_tokens");
|
||||
usage.total_tokens = num(u, "total_tokens");
|
||||
usage.cached_tokens = num(u->find("input_tokens_details"), "cached_tokens");
|
||||
usage.reasoning_tokens = num(u->find("output_tokens_details"), "reasoning_tokens");
|
||||
summary_.usage = usage;
|
||||
}
|
||||
if (type == "response.failed" && !summary_.error) {
|
||||
ErrorBody e;
|
||||
if (const Json* er = r->find("error"); er && er->is_object()) {
|
||||
e.message = str(er, "message");
|
||||
e.provider_code = str(er, "code");
|
||||
}
|
||||
e.error_type = str(r, "error_type"); // top level on the Responses skin
|
||||
e.code = status_for_error_type(e.error_type);
|
||||
summary_.error = e;
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
if (type == "error" && !summary_.error) {
|
||||
ErrorBody e;
|
||||
e.message = str(&d, "message");
|
||||
e.provider_code = str(&d, "code");
|
||||
e.error_type = str(&d, "error_type");
|
||||
e.code = status_for_error_type(e.error_type);
|
||||
summary_.error = e;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
void ResponsesStreamParser::throw_if_errored() const {
|
||||
if (summary_.error) throw summary_.error->to_api_error();
|
||||
}
|
||||
|
||||
std::string MessagesStreamParser::feed(const llm::SseEvent& ev) {
|
||||
std::string type = type_of(ev);
|
||||
count(summary_, type);
|
||||
const Json& d = ev.data;
|
||||
if (type == "message_start") {
|
||||
const Json* m = d.is_object() ? d.find("message") : nullptr;
|
||||
if (m) {
|
||||
summary_.id = str(m, "id");
|
||||
summary_.model = str(m, "model");
|
||||
}
|
||||
} else if (type == "content_block_delta") {
|
||||
const Json* delta = d.is_object() ? d.find("delta") : nullptr;
|
||||
std::string dt = str(delta, "type");
|
||||
if (dt == "text_delta") {
|
||||
std::string t = str(delta, "text");
|
||||
append(summary_.text, t);
|
||||
return t;
|
||||
}
|
||||
if (dt == "thinking_delta") append(summary_.reasoning, str(delta, "thinking"));
|
||||
} else if (type == "message_delta") {
|
||||
const Json* delta = d.is_object() ? d.find("delta") : nullptr;
|
||||
std::string sr = str(delta, "stop_reason");
|
||||
if (!sr.empty()) summary_.status = sr;
|
||||
if (const Json* u = d.is_object() ? d.find("usage") : nullptr; u && u->is_object()) {
|
||||
llm::Usage usage = summary_.usage.value_or(llm::Usage{});
|
||||
if (int in = num(u, "input_tokens")) usage.input_tokens = in;
|
||||
if (int out = num(u, "output_tokens")) usage.output_tokens = out;
|
||||
if (int cr = num(u, "cache_read_input_tokens")) usage.cached_tokens = cr;
|
||||
usage.total_tokens = usage.input_tokens + usage.output_tokens;
|
||||
summary_.usage = usage;
|
||||
}
|
||||
} else if (type == "error" && !summary_.error) {
|
||||
const Json* er = d.is_object() ? d.find("error") : nullptr;
|
||||
ErrorBody e;
|
||||
e.message = str(er, "message");
|
||||
e.error_type = str(er, "error_type");
|
||||
e.provider_code = str(er, "type");
|
||||
e.code = status_for_error_type(e.error_type);
|
||||
summary_.error = e;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
void MessagesStreamParser::throw_if_errored() const {
|
||||
if (summary_.error) throw summary_.error->to_api_error();
|
||||
}
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,54 @@
|
||||
#pragma once
|
||||
|
||||
// Stream parsers for the two non-chat inference skins: the Responses API
|
||||
// (POST /responses, "beta") and the Anthropic-compatible Messages API
|
||||
// (POST /messages). Event type names come from the spec schemas
|
||||
// StreamEvents and MessagesStreamEvents. Each parser folds the events into a
|
||||
// StreamSummary (text, reasoning, usage, terminal status or error) without
|
||||
// touching the network.
|
||||
|
||||
#include <map>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/chat_types.h"
|
||||
#include "llm/sse.h"
|
||||
#include "openrouter/errors.h"
|
||||
|
||||
namespace molo::openrouter {
|
||||
|
||||
struct StreamSummary {
|
||||
std::string id;
|
||||
std::string model;
|
||||
std::string text;
|
||||
std::string reasoning;
|
||||
std::string status; // responses: completed incomplete failed; messages: stop_reason
|
||||
std::optional<llm::Usage> usage;
|
||||
std::optional<ErrorBody> error;
|
||||
std::map<std::string, int> event_counts; // by event type
|
||||
std::size_t events = 0;
|
||||
};
|
||||
|
||||
class ResponsesStreamParser {
|
||||
public:
|
||||
// Returns the text delta this event carried ("" when none).
|
||||
std::string feed(const llm::SseEvent& event);
|
||||
const StreamSummary& summary() const { return summary_; }
|
||||
void throw_if_errored() const;
|
||||
|
||||
private:
|
||||
StreamSummary summary_;
|
||||
};
|
||||
|
||||
class MessagesStreamParser {
|
||||
public:
|
||||
std::string feed(const llm::SseEvent& event);
|
||||
const StreamSummary& summary() const { return summary_; }
|
||||
void throw_if_errored() const;
|
||||
|
||||
private:
|
||||
StreamSummary summary_;
|
||||
};
|
||||
|
||||
} // namespace molo::openrouter
|
||||
@@ -0,0 +1,155 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Checks calls.h (the module's one call table) against OpenRouter's OpenAPI
|
||||
document. Python 3 stdlib only, no credentials.
|
||||
|
||||
python3 -I src/openrouter/tools/check_spec.py [--spec FILE] [--probe]
|
||||
|
||||
For every row it requires: the HTTP method and path exist in the spec, the
|
||||
operationId matches, the declared key kind agrees with the operation's
|
||||
responses (a public row must have no 401), every body field is a top level
|
||||
property of the request schema (allOf followed), every "?name" is a query
|
||||
parameter. Coverage: every spec operation must be a row, or listed in EXCLUDED
|
||||
with a reason. Rows whose operationId starts with "doc:" are documented-only
|
||||
routes absent from the spec; with --probe each one is POSTed with an empty body
|
||||
and must answer 401 or 400, never 404 (no credential is sent).
|
||||
Exit 0: all rows verified. Exit 1: at least one mismatch (printed).
|
||||
"""
|
||||
import json
|
||||
import urllib.error
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
TABLE = os.path.join(HERE, "..", "calls.h")
|
||||
URL = "https://openrouter.ai/openapi.json"
|
||||
BASE = "https://openrouter.ai/api/v1"
|
||||
ROW = re.compile(
|
||||
r'^\s*\{"([^"]+)",\s*"([A-Z]+)",\s*"([^"]+)",\s*"([^"]+)",\s*"([^"]+)",\s*"([^"]+)",\s*"([^"]*)"\},?\s*$')
|
||||
# operationId -> reason it is deliberately not wrapped (none today).
|
||||
EXCLUDED = {}
|
||||
|
||||
|
||||
def load(spec):
|
||||
if spec:
|
||||
return json.load(open(spec))
|
||||
req = urllib.request.Request(URL, headers={"User-Agent": "molo-check-spec/1"})
|
||||
with urllib.request.urlopen(req, timeout=120) as r:
|
||||
return json.loads(r.read())
|
||||
|
||||
|
||||
def resolve(doc, node):
|
||||
seen = 0
|
||||
while isinstance(node, dict) and "$ref" in node and seen < 20:
|
||||
cur = doc
|
||||
for p in node["$ref"].lstrip("#/").split("/"):
|
||||
cur = cur[p]
|
||||
node = cur
|
||||
seen += 1
|
||||
return node
|
||||
|
||||
|
||||
def props(doc, schema):
|
||||
schema = resolve(doc, schema)
|
||||
if not isinstance(schema, dict):
|
||||
return {}
|
||||
out = dict(schema.get("properties", {}))
|
||||
for sub in schema.get("allOf", []):
|
||||
out.update(props(doc, sub))
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
spec = None
|
||||
if "--spec" in sys.argv:
|
||||
spec = sys.argv[sys.argv.index("--spec") + 1]
|
||||
doc = load(spec)
|
||||
rows = [m.groups() for m in map(ROW.match, open(TABLE)) if m]
|
||||
if not rows:
|
||||
print("no rows parsed from calls.h")
|
||||
return 1
|
||||
bad = []
|
||||
covered = set()
|
||||
ops = {}
|
||||
for path, item in doc["paths"].items():
|
||||
for method, op in item.items():
|
||||
if isinstance(op, dict) and method in ("get", "post", "put", "patch", "delete"):
|
||||
ops[(method.upper(), path)] = op
|
||||
names = set()
|
||||
doc_rows = []
|
||||
for name, http, path, opid, key, fmt, fields in rows:
|
||||
if name in names:
|
||||
bad.append("duplicate row name %s" % name)
|
||||
names.add(name)
|
||||
if opid.startswith("doc:"):
|
||||
doc_rows.append((name, http, path))
|
||||
continue
|
||||
op = ops.get((http, path))
|
||||
if op is None:
|
||||
bad.append("%s: %s %s not in spec" % (name, http, path))
|
||||
continue
|
||||
covered.add((http, path))
|
||||
if op.get("operationId") != opid:
|
||||
bad.append("%s: operationId %s, table says %s" % (name, op.get("operationId"), opid))
|
||||
has401 = "401" in op["responses"]
|
||||
if key == "none" and has401:
|
||||
bad.append("%s: table says public but the spec lists 401" % name)
|
||||
if key != "none" and not has401:
|
||||
bad.append("%s: table says %s but the spec lists no 401" % (name, key))
|
||||
params = []
|
||||
for p in op.get("parameters", []):
|
||||
params.append(resolve(doc, p))
|
||||
qnames = {p["name"] for p in params if p["in"] == "query"}
|
||||
body_props = {}
|
||||
rb = op.get("requestBody")
|
||||
if rb:
|
||||
rb = resolve(doc, rb)
|
||||
content = rb.get("content", {})
|
||||
ct = next(iter(content), None)
|
||||
if ct:
|
||||
body_props = props(doc, content[ct].get("schema"))
|
||||
if "multipart" in ct and fmt != "multipart":
|
||||
bad.append("%s: multipart body but fmt %s" % (name, fmt))
|
||||
if "x-www-form" in ct and fmt != "form":
|
||||
bad.append("%s: form body but fmt %s" % (name, fmt))
|
||||
for f in fields.split():
|
||||
if f == "-":
|
||||
continue
|
||||
if f.startswith("?"):
|
||||
if f[1:] not in qnames:
|
||||
bad.append("%s: query parameter %s absent" % (name, f[1:]))
|
||||
elif f not in body_props:
|
||||
bad.append("%s: body field %s absent" % (name, f))
|
||||
# Reverse: every spec query parameter and body property is declared.
|
||||
declared = set(fields.split())
|
||||
for q in qnames:
|
||||
if "?" + q not in declared:
|
||||
bad.append("%s: spec query parameter %s not in the table" % (name, q))
|
||||
for b in body_props:
|
||||
if b not in declared:
|
||||
bad.append("%s: spec body field %s not in the table" % (name, b))
|
||||
for key, op in ops.items():
|
||||
if key not in covered and op.get("operationId") not in EXCLUDED:
|
||||
bad.append("unwrapped spec operation %s %s (%s)" % (key[0], key[1], op.get("operationId")))
|
||||
if "--probe" in sys.argv:
|
||||
for name, http, path in doc_rows:
|
||||
req = urllib.request.Request(BASE + path, data=b"{}", method=http,
|
||||
headers={"Content-Type": "application/json", "User-Agent": "molo-check-spec/1"})
|
||||
try:
|
||||
urllib.request.urlopen(req, timeout=60)
|
||||
status = 200
|
||||
except urllib.error.HTTPError as e:
|
||||
status = e.code
|
||||
print("probe %s %s -> %d" % (http, path, status))
|
||||
if status == 404:
|
||||
bad.append("%s: documented-only route answers 404" % name)
|
||||
for b in bad:
|
||||
print("FAIL " + b)
|
||||
print("%d rows checked (%d documented-only), %d spec operations, %d problems"
|
||||
% (len(rows), len(doc_rows), len(ops), len(bad)))
|
||||
return 1 if bad else 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user