feat(clients): xai core: Responses, chat, tools, media, batch, models, ChatModel

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-24 08:04:30 +02:00
co-authored by Claude Sonnet 5.5
parent 2e0270c043
commit a39826aaea
55 changed files with 8148 additions and 3 deletions
+164
View File
@@ -0,0 +1,164 @@
# xAI core (`molo::xai::core`)
Status 2026-10-24. Grok API platform inference host `https://api.x.ai`,
credential `XAI_API_KEY` (bearer) read through `llm::Config`. Owned by plan
17, package "xai core". Voice, collections, documents search, files and the
management API belong to `src/xai/platform` and are not implemented here.
## Scope
Chat completions, the Responses API (create, stream, retrieve, delete, input
items, compact), tools (function calling, web search, X search, code
execution, collections search, remote MCP, image generation, shell, tool
search) with their stream events, structured outputs, reasoning controls,
Live Search parameters and guidelines, image generation and editing, video
generation / editing / extension (async), embeddings, models (generic,
language, image, video, embedding, with pricing and capability fields),
tokenize-text, deferred completions, the Batch API, API key info and `/v1/me`,
hosted Skills, the Anthropic-compatible messages endpoint, the legacy text
completion endpoints, and `XaiChatModel : llm::ChatModel`.
## Module map
| File | Concern |
|---|---|
| `caps.h` | every static ceiling, marked (spec), (docs) or ours |
| `params.*` | `Check` validators and `jx` tolerant JSON readers |
| `calls.*` | the ONE call table (43 rows), `render`, `Query`, tool and schema parity tables |
| `service.*` | `XaiService`: dispatch from the table, `Plan` (testable without network), body field check |
| `usage.*` | `TokenUsage` (chat and Responses shapes), `ToolUsage`, `MediaUsage`, cost ticks |
| `tools.*` | one class per tool, `ToolChoice` |
| `response_format.*` | `ResponseFormat` (text, json_object, json_schema) and the schema lint |
| `search_parameters.*` | Live Search `search_parameters`, `Guideline` |
| `input.*` | Responses input items and parts |
| `response.*` | `ModelResponse`, `OutputItem`, citations, server tool classification, replay items |
| `stream_events.*` | `ResponseEvent` decoder, `ResponseAccumulator` |
| `responses.*` | `ResponseRequest`, `ResponsesService` |
| `chat.*` | chat completions request, response, chunks, accumulator, `ChatService` (deferred) |
| `messages.*` | Anthropic-compatible `/v1/messages`, legacy `/v1/complete` and `/v1/completions` |
| `media.*`, `images.*`, `videos.*` | media sources, storage options, image and video services |
| `embeddings.*` | embeddings, base64 float decoding, tokenize |
| `models.*` | `ModelInfo`, `ModelsService` |
| `account.*` | `/v1/api-key`, `/v1/me`, Skills |
| `batches.*` | Batch API |
| `poll.*` | bounded polling (`WaitPolicy`, injectable sleeper) |
| `chat_model.*` | `XaiChatModel` (Responses wire by default, chat completions optional) |
| `client.*` | `XaiClient`: one `ApiClient`, every service |
| `tools/check_spec.py` | spec checker |
## Contracts
- All validation (ids, ranges, counts, enums, URLs, dates, schema lint) runs
before the credential check, so bad input is 422 in the demo with or
without a key and never reaches xAI. Without a key every call throws
`NotConfiguredError` before network traffic.
- `XaiService::plan` rejects any top-level body field that the call table does
not declare. `llm::ChatRequest::extra` is merged and then checked the same
way: an unknown field is refused, never silently sent.
- Destructive calls are explicit methods: `ResponsesService::remove`,
`SkillsService::remove`, `BatchService::cancel`. Image, video and batch
submission cost money; the demo makes them dry runs unless `confirm:true`.
- Lists report truncation (`truncated`) when a cap stops them; page walks are
bounded by `llm::Caps::kMaxPages` or `Caps::kMaxBatchPagesWalked`.
- Waits (`wait_deferred`, `ImageService::wait`, `VideoService::wait`) take a
`WaitPolicy` (interval 1..60 s, total at most 30 min) and fail with 504
`WAIT_TIMEOUT`; a failed job is 502 `GENERATION_FAILED`.
- Deferred chat results are fetchable once within 24 h (docs); `poll_deferred`
returns `nullopt` on 202.
- `XaiChatModel` sends `store=false` on the Responses wire, refuses `stop`
there (not in the Responses schema), and maps reasoning, tool calls, images,
JSON mode and `response_schema`.
- Cost is `usage.cost_in_usd_ticks`; 1e10 ticks = 1 USD.
## Excluded fields
Sent by no builder because the spec marks them (Unsupported) or "only
maintained for compatibility": Responses `background`, `metadata`,
`truncation`, `context_management`; chat `logit_bias`, `web_search_options`;
legacy completions `best_of`, `frequency_penalty`, `logit_bias`, `suffix`;
messages `top_k`; embeddings `preview`; web search tool `external_web_access`
and `search_context_size` (the request is rejected when set); MCP
`require_approval` and `connector_id` (docs: not supported).
## Sources
1. OpenAPI 3.1 document `https://api.x.ai/api-docs/openapi.json` (identical
bytes at `https://docs.x.ai/openapi.json`): 39 paths, 44 operations, 203
schemas, sha256 prefix `f34f5c20564fefd1`.
2. REST reference markdown pages `docs.x.ai/developers/rest-api-reference/inference/{responses,chat-completions,images,videos,batches,models,other,legacy}.md`.
3. Guides: `advanced-api-usage/{batch-api,deferred-chat-completions}`, `model-capabilities/{text/*,images/*,video/*}`, `tools/*`, `models`, `pricing`; `docs.x.ai/llms.txt` index.
4. Unauthenticated probes of `api.x.ai` (below).
5. Not read: the xai-sdk-python and OpenAI-compatible SDK sources (not fetched in this round; the OpenAPI document plus two doc sources cover every row).
## Verified against spec
`python3 -I src/xai/core/tools/check_spec.py` on 2026-10-24:
`spec ok: 43 calls (36 openapi, 7 docs), 66 schemas, 9 tool types, 8 platform operations excluded; spec xAI's REST API v1.0.0 sha256:f34f5c20564fefd1, 44 operations`.
It checks every OpenAPI row (method, path, operationId, query parameters,
request body fields, request and response schema names), the schema fields
this module reads, the tool fields the builders write against the `ModelTool`
variants, the 7 Batch rows and documented-only fields against the docs pages,
and fails on any spec operation that is neither in the table nor in the
exclusion list (7 files operations and `documents/search`, owned by
`src/xai/platform`).
## Verified live (unauthenticated, 2026-10-24, no key exists)
- `api.x.ai` resolves, TLS verifies, HTTP/2, behind Cloudflare. `GET /` answers 421 with a welcome text.
- Missing credential: 401 `{"code":"unauthenticated:no-credentials","error":"No credentials presented."}` on `/v1/models`, `/v1/api-key`, `/v1/me`, `/v1/responses/{id}` (GET, DELETE), `/v1/responses/{id}/input_items`, `/v1/images/{id}`, `/v1/videos/{id}`, `/v1/chat/deferred-completion/{id}`, `/v1/language-models[/{id}]`, `/v1/embedding-models`, `/v1/batches`, `/v1/batches/{id}`, `/v1/batches/{id}/requests`, `/v1/batches/{id}/results`, `POST /v1/batches/{id}:cancel`.
- `POST` routes without `Content-Type: application/json` answer 415 (`/v1/embeddings`, `/v1/images/generations`, `/v1/videos/generations`, `/v1/videos/extensions`, `/v1/tokenize-text`, `/v1/responses/compact`, `/v1/batches/{id}/requests`): the routes exist.
- `POST /v1/responses` with `{}` and no key answers 422 `{"error":"Failed to deserialize the JSON body into the target type: missing field `input`"}` (validation runs before authentication). `POST /v1/chat/completions` with a bogus key and `{}` answers 400 `{"code":"400","error":"Bad data: Messages cannot be empty"}`.
- Wrong key: 400 `{"code":"invalid-argument","error":"Incorrect API key provided. ..."}` (400, not 401).
- Absent routes answer 404: `POST /v1/batches/{id}/cancel` (the real route is `:cancel`), `POST /v1/messages/count_tokens`, `POST /v1/responses/{id}/cancel`.
- `GET /v1/skills` answers 404 although the OpenAPI document lists it (see Unverified).
- Error body shape: `{"code": ..., "error": "text"}` (error is a string).
Opt-in live tests (`tests/src/xai/core/test_xai_core.cpp`, suite `xai_core_live`):
`MOLO_LIVE_NET=1` plus `XAI_API_KEY` runs a read-only pass (key info, every
model list, batch list) and one smallest completion; `MOLO_LIVE_PAID=1`
additionally runs one image generation. They SKIP loudly otherwise. Not run:
no key exists.
## Unverified
- Every response shape was read from the OpenAPI document and the docs, never
seen from a live call with a key.
- Responses stream event names beyond those the guides name (see
`stream_events.h`): the remaining kinds follow the OpenAI Responses event
vocabulary xAI declares compatibility with. The `error` event payload is
assumed `{type:"error", message}`.
- Anthropic stream events of `/v1/messages` (`message_start`,
`content_block_*`, `message_delta`, `message_stop`) are Anthropic's; xAI's
reference only says the stream ends with `data: [DONE]`. xAI marks the whole
Anthropic compatibility "fully deprecated".
- `GET /v1/skills` is listed in the OpenAPI document but answered 404 live;
the Skills service follows the document.
- Batch routes are not in the OpenAPI document. Request entry keys differ
between sources: the reference shows `chat_get_completion`, the guide shows
`responses`, `image_generation`, `image_edit`, `video_generation`,
`video_extension`. `BatchEntry` offers both; a video edit is sent under
`video_generation` as in the guide. `input_file_id` on batch create (JSONL
file batches) is guide-only and not exposed.
- Documented but absent from the OpenAPI document: image `quality`
(`low|medium|auto`), video `last_frame`. They are in the table's doc-only
column and checked against the guides.
- `ModelToolChoice`: the schema says `{type:"function", name}`, its
description says `{type:"function", function:{name}}`. The schema is used on
the Responses wire; the chat wire uses the OpenAI nesting.
- Code execution `container` accepts "an id or an object" per the spec, shown
by no guide; only the id form is offered.
- Video job statuses beyond `pending|done|failed` (an expired status) are not
documented; unknown statuses read as pending.
- Rate limits and per-model batch support are model specific; not modelled.
- `reasoning_effort` accepted values are model specific (`ModelInfo::accepts_effort`).
## Gaps
- No streaming for the legacy `/v1/completions` and `/v1/complete` calls.
- No WebSocket mode for the Responses API (docs: advanced-api-usage/websocket-mode); owned by the platform package if wanted.
- File-backed batches (`input_file_id`) need the Files API in `src/xai/platform`.
- `llm::error_from_response` drops the top-level `code` when `error` is a
string, so xAI errors surface with type `UNKNOWN` and the message intact
(foundation gap in `src/llm/error.cpp`, not edited here).
- `llm::ApiClient` has no per-request timeout setting; reasoning requests can
run long.
+109
View File
@@ -0,0 +1,109 @@
#include "xai/core/account.h"
#include "llm/caps.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
ApiKeyInfo ApiKeyInfo::parse(const Json& d) {
ApiKeyInfo k;
k.api_key_id = jx::str(d, "api_key_id");
k.name = jx::str(d, "name");
k.redacted_api_key = jx::str(d, "redacted_api_key");
k.team_id = jx::str(d, "team_id");
k.user_id = jx::str(d, "user_id");
k.create_time = jx::str(d, "create_time");
k.modify_time = jx::str(d, "modify_time");
k.modified_by = jx::str(d, "modified_by");
k.acls = jx::strings(d, "acls");
k.api_key_blocked = jx::flag(d, "api_key_blocked");
k.api_key_disabled = jx::flag(d, "api_key_disabled");
k.team_blocked = jx::flag(d, "team_blocked");
return k;
}
MeInfo MeInfo::parse(const Json& d) {
MeInfo m;
m.team_id = jx::str(d, "team_id");
m.user_id = jx::str(d, "user_id");
m.zdr_status = jx::str(d, "zdr_status");
m.team_blocked = jx::flag(d, "team_blocked");
if (const Json* k = jx::get(d, "api_key"); k && k->is_object()) {
m.has_api_key = true;
m.api_key_id = jx::str(*k, "api_key_id");
m.redacted_api_key = jx::str(*k, "redacted_api_key");
m.api_key_blocked = jx::flag(*k, "blocked");
m.api_key_disabled = jx::flag(*k, "disabled");
}
if (const Json* o = jx::get(d, "oauth"); o && o->is_object()) {
m.has_oauth = true;
m.oauth_client_id = jx::str(*o, "client_id");
}
return m;
}
SkillInfo SkillInfo::parse(const Json& d) {
SkillInfo s;
s.id = jx::str(d, "id");
s.name = jx::str(d, "name");
s.description = jx::str(d, "description");
s.default_version = jx::str(d, "default_version");
s.latest_version = jx::str(d, "latest_version");
s.object = jx::str(d, "object");
s.created_at = jx::num(d, "created_at");
return s;
}
ApiKeyInfo AccountService::api_key() { return ApiKeyInfo::parse(call_json(Call::kApiKeyInfo)); }
MeInfo AccountService::me() { return MeInfo::parse(call_json(Call::kMe)); }
SkillList SkillsService::list(std::optional<int> limit, const std::string& after, const std::string& order) {
Query q;
if (limit) {
Check::range("limit", *limit, 1, Caps::kMaxListLimit);
q.add(Call::kSkillsList, "limit", static_cast<long>(*limit));
}
if (!after.empty()) {
Check::id("after", after);
q.add(Call::kSkillsList, "after", after);
}
if (!order.empty()) {
Check::one_of("order", order, {"asc", "desc"});
q.add(Call::kSkillsList, "order", order);
}
Json doc = call_json(Call::kSkillsList, {}, nullptr, q.str());
SkillList out;
const Json* data = jx::get(doc, "data");
if (data && data->is_array())
for (const Json& e : data->as_array()) {
if (out.data.size() >= Caps::kMaxAccumulatedItems) break;
out.data.push_back(SkillInfo::parse(e));
}
out.first_id = jx::str(doc, "first_id");
out.last_id = jx::str(doc, "last_id");
out.has_more = jx::flag(doc, "has_more");
return out;
}
SkillInfo SkillsService::get(const std::string& skill_id) {
return SkillInfo::parse(call_json(Call::kSkillsGet, {skill_id}));
}
SkillInfo SkillsService::upload_zip(const std::string& filename, const std::string& zip_bytes) {
Check::text("filename", filename, 255);
Check::text("zip", zip_bytes, llm::Caps::kMaxUploadBytes);
plan(Call::kSkillsUpload); // table check
clients::body::MultipartBody form;
form.add_file("files", filename, "application/zip", zip_bytes);
return SkillInfo::parse(client_->post_multipart(render(Call::kSkillsUpload), form));
}
bool SkillsService::remove(const std::string& skill_id) {
return jx::flag(call_json(Call::kSkillsDelete, {skill_id}), "deleted");
}
std::string SkillsService::download(const std::string& skill_id) {
return call_raw(Call::kSkillsContent, {skill_id}).body;
}
} // namespace molo::xai::core
+66
View File
@@ -0,0 +1,66 @@
#pragma once
// Account calls on the inference host: GET /v1/api-key (key metadata, never
// the key itself: xAI returns a redacted form) and GET /v1/me (caller
// identity, works for API keys and OAuth tokens), plus hosted Skills
// (/v1/skills). Management, billing, collections and files belong to
// src/xai/platform.
#include <string>
#include <vector>
#include "xai/core/service.h"
namespace molo::xai::core {
struct ApiKeyInfo {
std::string api_key_id, name, redacted_api_key, team_id, user_id, create_time, modify_time, modified_by;
std::vector<std::string> acls;
bool api_key_blocked = false, api_key_disabled = false, team_blocked = false;
static ApiKeyInfo parse(const Json& doc);
bool usable() const { return !api_key_blocked && !api_key_disabled && !team_blocked; }
};
struct MeInfo {
std::string team_id, user_id, zdr_status; // no_zdr | zdr | pii_scrubbing
bool team_blocked = false;
bool has_api_key = false, has_oauth = false;
std::string api_key_id, redacted_api_key, oauth_client_id;
bool api_key_blocked = false, api_key_disabled = false;
static MeInfo parse(const Json& doc);
};
struct SkillInfo {
std::string id, name, description, default_version, latest_version, object;
long created_at = 0;
static SkillInfo parse(const Json& doc);
};
struct SkillList {
std::vector<SkillInfo> data;
std::string first_id, last_id;
bool has_more = false;
};
class AccountService : public XaiService {
public:
explicit AccountService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
ApiKeyInfo api_key();
MeInfo me();
};
class SkillsService : public XaiService {
public:
explicit SkillsService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
SkillList list(std::optional<int> limit = std::nullopt, const std::string& after = "",
const std::string& order = "");
SkillInfo get(const std::string& skill_id);
// Upload a zip (a skill directory zipped; SKILL.md inside). Bounded by llm::Caps::kMaxUploadBytes.
SkillInfo upload_zip(const std::string& filename, const std::string& zip_bytes);
// Destructive, explicit.
bool remove(const std::string& skill_id);
// The skill as a zip.
std::string download(const std::string& skill_id);
};
} // namespace molo::xai::core
+183
View File
@@ -0,0 +1,183 @@
#include "xai/core/batches.h"
#include <set>
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
void check_batch_id(const std::string& id) {
Check::id("batch", id);
if (id.find(':') != std::string::npos) Check::fail("bad batch id");
}
Query page_query(Call call, std::optional<int> limit, const std::string& token) {
Query q;
if (limit) {
Check::range("limit", *limit, 1, Caps::kMaxBatchPageLimit);
q.add(call, "limit", static_cast<long>(*limit));
}
if (!token.empty()) {
Check::text("pagination_token", token, Caps::kMaxQueryValueLen);
q.add(call, "pagination_token", token);
}
return q;
}
} // namespace
BatchEntry BatchEntry::make(const std::string& id, const char* kind, Json body) {
Check::id("batch request", id);
Json inner = Json::object();
inner.set(kind, std::move(body));
Json o = Json::object();
o.set("batch_request_id", Json(id));
o.set("batch_request", inner);
BatchEntry e(std::move(o));
if (e.bytes() > Caps::kMaxBatchRequestBytes) Check::fail("batch request over 25 MB");
return e;
}
BatchEntry BatchEntry::chat(const std::string& id, const ChatRequest& r) { return make(id, "chat_get_completion", r.to_json()); }
BatchEntry BatchEntry::responses(const std::string& id, const ResponseRequest& r) { return make(id, "responses", r.to_json()); }
BatchEntry BatchEntry::image_generation(const std::string& id, const ImageGenerationRequest& r) { return make(id, "image_generation", r.to_json()); }
BatchEntry BatchEntry::image_edit(const std::string& id, const ImageEditRequest& r) { return make(id, "image_edit", r.to_json()); }
BatchEntry BatchEntry::video_generation(const std::string& id, const VideoGenerationRequest& r) { return make(id, "video_generation", r.to_json()); }
BatchEntry BatchEntry::video_edit(const std::string& id, const VideoEditRequest& r) { return make(id, "video_generation", r.to_json()); }
BatchEntry BatchEntry::video_extension(const std::string& id, const VideoExtendRequest& r) { return make(id, "video_extension", r.to_json()); }
Batch Batch::parse(const Json& d) {
Batch b;
b.raw = d;
b.batch_id = jx::str(d, "batch_id");
b.name = jx::str(d, "name");
b.create_time = jx::str(d, "create_time");
b.expire_time = jx::str(d, "expire_time");
b.cancel_time = jx::str(d, "cancel_time");
b.cancel_by_xai_message = jx::str(d, "cancel_by_xai_message");
b.create_api_key_id = jx::str(d, "create_api_key_id");
Json s = jx::object(d, "state");
b.state = {jx::num(s, "num_requests"), jx::num(s, "num_pending"), jx::num(s, "num_success"),
jx::num(s, "num_error"), jx::num(s, "num_cancelled")};
return b;
}
Batch BatchService::create(const std::string& name) {
Check::text("name", name, Caps::kMaxNameLen);
Json body = Json::object();
body.set("name", Json(name));
return Batch::parse(call_json(Call::kBatchCreate, {}, &body));
}
Batch BatchService::add_requests(const std::string& batch_id, const std::vector<BatchEntry>& entries) {
check_batch_id(batch_id);
Check::count("batch requests", entries.size(), 1, Caps::kMaxBatchRequestsPerAdd);
std::set<std::string> seen;
Json arr = Json::array();
for (const auto& e : entries) {
if (!seen.insert(jx::str(e.json(), "batch_request_id")).second)
Check::fail("batch_request_id must be unique within the call");
arr.push(e.json());
}
Json body = Json::object();
body.set("batch_requests", arr);
call_json(Call::kBatchRequestsAdd, {batch_id}, &body);
return get(batch_id);
}
Batch BatchService::get(const std::string& batch_id) {
check_batch_id(batch_id);
return Batch::parse(call_json(Call::kBatchGet, {batch_id}));
}
BatchPage BatchService::list(std::optional<int> limit, const std::string& token) {
Json doc = call_json(Call::kBatchList, {}, nullptr, page_query(Call::kBatchList, limit, token).str());
BatchPage p;
const Json* arr = jx::get(doc, "batches");
if (arr && arr->is_array())
for (const Json& b : arr->as_array()) {
if (p.batches.size() >= static_cast<std::size_t>(Caps::kMaxBatchPageLimit)) break;
p.batches.push_back(Batch::parse(b));
}
p.pagination_token = jx::str(doc, "pagination_token");
return p;
}
BatchRequestPage BatchService::list_requests(const std::string& batch_id, std::optional<int> limit,
const std::string& token) {
check_batch_id(batch_id);
Json doc = call_json(Call::kBatchRequestsList, {batch_id}, nullptr,
page_query(Call::kBatchRequestsList, limit, token).str());
BatchRequestPage p;
const Json* arr = jx::get(doc, "batch_request_metadata");
if (arr && arr->is_array())
for (const Json& m : arr->as_array()) {
if (p.requests.size() >= static_cast<std::size_t>(Caps::kMaxBatchPageLimit)) break;
p.requests.push_back({jx::str(m, "batch_request_id"), jx::str(m, "endpoint"), jx::str(m, "model"),
jx::str(m, "state"), jx::str(m, "create_time"), jx::str(m, "finish_time")});
}
p.pagination_token = jx::str(doc, "pagination_token");
return p;
}
BatchResultPage BatchService::results(const std::string& batch_id, std::optional<int> limit,
const std::string& token) {
check_batch_id(batch_id);
Json doc = call_json(Call::kBatchResults, {batch_id}, nullptr, page_query(Call::kBatchResults, limit, token).str());
BatchResultPage p;
const Json* arr = jx::get(doc, "results");
if (arr && arr->is_array())
for (const Json& r : arr->as_array()) {
if (p.results.size() >= static_cast<std::size_t>(Caps::kMaxBatchPageLimit)) break;
BatchResult o;
o.batch_request_id = jx::str(r, "batch_request_id");
Json result = jx::object(r, "batch_result");
Json response = jx::object(result, "response");
if (response.is_object() && !response.as_object().empty()) {
// The response object carries exactly one typed member.
const auto& members = response.as_object();
auto it = members.begin();
o.kind = it->first;
o.response = it->second;
o.succeeded = it->second.is_object();
if (o.kind == "chat_get_completion")
o.cost_usd_ticks = jx::num(jx::object(o.response, "usage"), "cost_in_usd_ticks");
}
std::string err = jx::str(result, "error");
if (err.empty()) err = jx::str(r, "error_message");
if (!err.empty()) {
o.succeeded = false;
o.error = err;
}
p.results.push_back(std::move(o));
}
p.pagination_token = jx::str(doc, "pagination_token");
return p;
}
BatchResultSet BatchService::all_results(const std::string& batch_id, std::size_t max_items) {
Check::range("max_items", static_cast<double>(max_items), 1, 100000);
BatchResultSet out;
std::string token;
for (std::size_t page = 0; page < Caps::kMaxBatchPagesWalked; ++page) {
BatchResultPage p = results(batch_id, 1000, token);
for (auto& r : p.results) {
if (out.results.size() >= max_items) {
out.truncated = true;
return out;
}
out.results.push_back(std::move(r));
}
if (p.pagination_token.empty()) return out;
token = p.pagination_token;
}
out.truncated = true;
return out;
}
Batch BatchService::cancel(const std::string& batch_id) {
check_batch_id(batch_id);
return Batch::parse(call_json(Call::kBatchCancel, {batch_id}));
}
} // namespace molo::xai::core
+106
View File
@@ -0,0 +1,106 @@
#pragma once
// Batch API (/v1/batches...): a batch is a container; requests are added to
// it, processed in the background at reduced price, and results are paged.
// The OpenAPI document does not list these routes; they come from the REST
// reference and the Batch API guide (see SPEC). The reference names the chat
// entry `chat_get_completion`, the guide names `responses`, `image_generation`,
// `image_edit`, `video_generation`, `video_extension`; both are offered and the
// difference is recorded under Unverified.
#include <optional>
#include <string>
#include <vector>
#include "xai/core/chat.h"
#include "xai/core/images.h"
#include "xai/core/responses.h"
#include "xai/core/service.h"
#include "xai/core/videos.h"
namespace molo::xai::core {
class BatchEntry {
public:
static BatchEntry chat(const std::string& batch_request_id, const ChatRequest& request);
static BatchEntry responses(const std::string& batch_request_id, const ResponseRequest& request);
static BatchEntry image_generation(const std::string& batch_request_id, const ImageGenerationRequest& request);
static BatchEntry image_edit(const std::string& batch_request_id, const ImageEditRequest& request);
static BatchEntry video_generation(const std::string& batch_request_id, const VideoGenerationRequest& request);
static BatchEntry video_edit(const std::string& batch_request_id, const VideoEditRequest& request);
static BatchEntry video_extension(const std::string& batch_request_id, const VideoExtendRequest& request);
const Json& json() const { return json_; }
std::size_t bytes() const { return json_.dump().size(); }
private:
explicit BatchEntry(Json j) : json_(std::move(j)) {}
static BatchEntry make(const std::string& id, const char* kind, Json body);
Json json_;
};
struct BatchState {
long num_requests = 0, num_pending = 0, num_success = 0, num_error = 0, num_cancelled = 0;
};
struct Batch {
std::string batch_id, name, create_time, expire_time, cancel_time, cancel_by_xai_message, create_api_key_id;
BatchState state;
Json raw;
static Batch parse(const Json& doc);
// True when nothing is pending and at least one request was added.
bool finished() const { return state.num_requests > 0 && state.num_pending == 0; }
};
struct BatchPage {
std::vector<Batch> batches;
std::string pagination_token; // empty on the last page
};
struct BatchRequestMeta {
std::string batch_request_id, endpoint, model, state, create_time, finish_time;
};
struct BatchRequestPage {
std::vector<BatchRequestMeta> requests;
std::string pagination_token;
};
struct BatchResult {
std::string batch_request_id;
bool succeeded = false;
std::string error; // when failed
std::string kind; // chat_get_completion | image_generation | video_generation | responses ...
Json response; // the typed response document for `kind`
long cost_usd_ticks = 0; // chat results: usage.cost_in_usd_ticks
};
struct BatchResultPage {
std::vector<BatchResult> results;
std::string pagination_token;
};
struct BatchResultSet {
std::vector<BatchResult> results;
bool truncated = false; // a page or item cap stopped the walk
};
class BatchService : public XaiService {
public:
explicit BatchService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
Batch create(const std::string& name);
// Adds 1..Caps::kMaxBatchRequestsPerAdd entries; each is capped at 25 MB, ids unique.
Batch add_requests(const std::string& batch_id, const std::vector<BatchEntry>& entries);
Batch get(const std::string& batch_id);
BatchPage list(std::optional<int> limit = std::nullopt, const std::string& pagination_token = "");
BatchRequestPage list_requests(const std::string& batch_id, std::optional<int> limit = std::nullopt,
const std::string& pagination_token = "");
BatchResultPage results(const std::string& batch_id, std::optional<int> limit = std::nullopt,
const std::string& pagination_token = "");
// Walks result pages up to Caps::kMaxBatchPagesWalked and max_items; truncated is loud.
BatchResultSet all_results(const std::string& batch_id, std::size_t max_items = 10000);
// Destructive: pending requests are not processed; finished results stay.
Batch cancel(const std::string& batch_id);
};
} // namespace molo::xai::core
+255
View File
@@ -0,0 +1,255 @@
#include "xai/core/calls.h"
#include <cstring>
#include "xai/core/caps.h"
#include "xai/core/params.h"
namespace molo::xai::core {
// Row: call, method, path, operationId (or "docs"), body fields, query
// params, doc-only body fields ("field@docs page"), request schema, response
// schema, source. Fields the spec marks (Unsupported) or "only maintained for
// compatibility" are not sent and not listed (see SPEC "Excluded fields").
//
// CALLS-BEGIN (parsed by tools/check_spec.py; one row per line)
static const CallSpec kCalls[] = {
{Call::kChatCreate, "POST", "/v1/chat/completions", "handle_generic_completion_request", "deferred,frequency_penalty,guidelines,logprobs,max_completion_tokens,max_tokens,messages,model,n,parallel_tool_calls,presence_penalty,prompt_cache_key,reasoning_effort,response_format,safety_identifier,search_parameters,seed,service_tier,stop,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user", "", "", "ChatRequest", "ChatResponse", "openapi"},
{Call::kChatDeferredGet, "GET", "/v1/chat/deferred-completion/{request_id}", "handle_get_deferred_completion_request", "", "", "", "", "ChatResponse", "openapi"},
{Call::kCompletionsLegacy, "POST", "/v1/completions", "handle_sample_request", "echo,logprobs,max_tokens,model,n,presence_penalty,prompt,seed,stop,temperature,top_p,user", "", "", "SampleRequest", "SampleResponse", "openapi"},
{Call::kCompleteLegacy, "POST", "/v1/complete", "handle_generic_complete_request", "max_tokens_to_sample,metadata,model,prompt,stop_sequences,temperature,top_p", "", "", "CompleteRequest", "CompleteResponse", "openapi"},
{Call::kMessagesCreate, "POST", "/v1/messages", "handle_generic_messages_request", "max_tokens,messages,metadata,model,stop_sequences,stream,system,temperature,tool_choice,tools,top_p", "", "", "MessageRequest", "MessageResponse", "openapi"},
{Call::kResponsesCreate, "POST", "/v1/responses", "handle_generic_model_request", "guidelines,include,input,instructions,logprobs,max_output_tokens,max_turns,min_p,model,parallel_tool_calls,previous_response_id,prompt_cache_key,reasoning,reasoning_effort,safety_identifier,search_parameters,service_tier,store,stream,temperature,text,tool_choice,tools,top_k,top_logprobs,top_p,user", "", "", "ModelRequest", "ModelResponse", "openapi"},
{Call::kResponsesGet, "GET", "/v1/responses/{response_id}", "handle_get_stored_completion_request", "", "", "", "", "ModelResponse", "openapi"},
{Call::kResponsesDelete, "DELETE", "/v1/responses/{response_id}", "handle_delete_stored_completion_request", "", "", "", "", "DeleteStoredCompletionResponse", "openapi"},
{Call::kResponsesInputItems, "GET", "/v1/responses/{response_id}/input_items", "handle_list_input_items", "", "limit,order,after", "", "", "ListInputItemsResponse", "openapi"},
{Call::kResponsesCompact, "POST", "/v1/responses/compact", "handle_compact_request", "input,model", "", "", "CompactRequest", "CompactResponse", "openapi"},
{Call::kImagesGenerate, "POST", "/v1/images/generations", "handle_generate_image_request", "aspect_ratio,deferred,model,n,output,prompt,resolution,response_format,storage_options,user", "", "quality@model-capabilities/images/generation", "GenerateImageRequest", "GeneratedImageResponse", "openapi"},
{Call::kImagesEdit, "POST", "/v1/images/edits", "handle_edit_image_request", "aspect_ratio,deferred,image,images,model,n,output,prompt,resolution,response_format,storage_options,user", "", "", "EditImageRequest", "GeneratedImageResponse", "openapi"},
{Call::kImagesGet, "GET", "/v1/images/{request_id}", "handle_get_deferred_image_request", "", "", "", "", "GetDeferredImageResponse", "openapi"},
{Call::kVideosGenerate, "POST", "/v1/videos/generations", "handle_generate_video_request", "aspect_ratio,duration,image,keyframes,model,output,prompt,reference_audios,reference_images,resolution,storage_options,user", "", "last_frame@model-capabilities/video/reference-to-video", "GenerateVideoRequest", "StartDeferredResponse", "openapi"},
{Call::kVideosEdit, "POST", "/v1/videos/edits", "handle_edit_video_request", "model,output,prompt,storage_options,user,video", "", "", "EditVideoRequest", "StartDeferredResponse", "openapi"},
{Call::kVideosExtend, "POST", "/v1/videos/extensions", "handle_extend_video_request", "duration,model,output,prompt,storage_options,video", "", "", "ExtendVideoRequest", "StartDeferredResponse", "openapi"},
{Call::kVideosGet, "GET", "/v1/videos/{request_id}", "handle_get_deferred_video_request", "", "", "", "", "VideoResponse", "openapi"},
{Call::kEmbeddingsCreate, "POST", "/v1/embeddings", "handle_embedding_request", "dimensions,encoding_format,input,model,user", "", "", "EmbeddingRequest", "EmbeddingResponse", "openapi"},
{Call::kTokenizeText, "POST", "/v1/tokenize-text", "handle_tokenize_text_request", "model,text,user", "", "", "TokenizeRequest", "TokenizeResponse", "openapi"},
{Call::kModelsList, "GET", "/v1/models", "handle_models_list_request", "", "", "", "", "ListModelsResponse", "openapi"},
{Call::kModelsGet, "GET", "/v1/models/{model_id}", "handle_model_get_request", "", "", "", "", "Model", "openapi"},
{Call::kLanguageModelsList, "GET", "/v1/language-models", "handle_language_models_list_request", "", "", "", "", "ListLanguageModelsResponse", "openapi"},
{Call::kLanguageModelsGet, "GET", "/v1/language-models/{model_id}", "handle_language_model_get_request", "", "", "", "", "LanguageModel", "openapi"},
{Call::kImageModelsList, "GET", "/v1/image-generation-models", "handle_image_generation_models_list_request", "", "", "", "", "ListImageGenerationModelsResponse", "openapi"},
{Call::kImageModelsGet, "GET", "/v1/image-generation-models/{model_id}", "handle_image_generation_model_get_request", "", "", "", "", "ImageGenerationModel", "openapi"},
{Call::kVideoModelsList, "GET", "/v1/video-generation-models", "handle_video_generation_models_list_request", "", "", "", "", "ListVideoGenerationModelsResponse", "openapi"},
{Call::kVideoModelsGet, "GET", "/v1/video-generation-models/{model_id}", "handle_video_generation_model_get_request", "", "", "", "", "VideoGenerationModel", "openapi"},
{Call::kEmbeddingModelsList, "GET", "/v1/embedding-models", "handle_embedding_models_list_request", "", "", "", "", "ListEmbeddingModelsResponse", "openapi"},
{Call::kEmbeddingModelsGet, "GET", "/v1/embedding-models/{model_id}", "handle_embedding_model_get_request", "", "", "", "", "EmbeddingModel", "openapi"},
{Call::kApiKeyInfo, "GET", "/v1/api-key", "handle_get_api_key_info_request", "", "", "", "", "ApiKey", "openapi"},
{Call::kMe, "GET", "/v1/me", "handle_get_me_request", "", "", "", "", "GetMeResponse", "openapi"},
{Call::kSkillsList, "GET", "/v1/skills", "handle_list_skills_request", "", "limit,after,order", "", "", "SkillList", "openapi"},
{Call::kSkillsUpload, "POST", "/v1/skills", "handle_upload_skill_request", "files", "", "", "UploadSkillMultipartRequest", "Skill", "openapi"},
{Call::kSkillsGet, "GET", "/v1/skills/{skill_id}", "handle_retrieve_skill_request", "", "", "", "", "Skill", "openapi"},
{Call::kSkillsDelete, "DELETE", "/v1/skills/{skill_id}", "handle_delete_skill_request", "", "", "", "", "DeletedSkill", "openapi"},
{Call::kSkillsContent, "GET", "/v1/skills/{skill_id}/content", "handle_download_skill_content_request", "", "", "", "", "string", "openapi"},
{Call::kBatchCreate, "POST", "/v1/batches", "POST /v1/batches", "name", "", "input_file_id@advanced-api-usage/batch-api", "", "", "docs"},
{Call::kBatchList, "GET", "/v1/batches", "GET /v1/batches", "", "limit,pagination_token", "", "", "", "docs"},
{Call::kBatchGet, "GET", "/v1/batches/{batch_id}", "GET /v1/batches/{batch_id}", "", "", "", "", "", "docs"},
{Call::kBatchRequestsAdd, "POST", "/v1/batches/{batch_id}/requests", "POST /v1/batches/{batch_id}/requests", "batch_requests", "", "", "", "", "docs"},
{Call::kBatchRequestsList, "GET", "/v1/batches/{batch_id}/requests", "GET /v1/batches/{batch_id}/requests", "", "limit,pagination_token", "", "", "", "docs"},
{Call::kBatchResults, "GET", "/v1/batches/{batch_id}/results", "GET /v1/batches/{batch_id}/results", "", "limit,pagination_token", "", "", "", "docs"},
{Call::kBatchCancel, "POST", "/v1/batches/{batch_id}:cancel", "POST /v1/batches/{batch_id}:cancel", "", "", "", "", "", "docs"},
};
// CALLS-END
// Response schema fields this module reads (checked against the OpenAPI
// component schemas; a renamed or removed field fails the checker).
// SCHEMAS-BEGIN
static const SchemaFields kSchemas[] = {
{"ChatResponse", "choices,citations,created,id,model,object,output_files,service_tier,system_fingerprint,usage"},
{"Choice", "finish_reason,index,logprobs,message"},
{"ChoiceMessage", "content,reasoning_content,refusal,role,tool_calls"},
{"ChatResponseChunk", "choices,citations,created,id,model,object,output_files,service_tier,system_fingerprint,usage"},
{"ChoiceChunk", "delta,finish_reason,index,logprobs"},
{"Delta", "content,images,reasoning_content,role,tool_calls"},
{"ToolCall", "function,id,index,type"},
{"Function", "arguments,name"},
{"Usage", "completion_tokens,completion_tokens_details,cost_in_usd_ticks,num_sources_used,prompt_tokens,prompt_tokens_details,server_side_tool_usage_details,total_tokens"},
{"CompletionUsageDetail", "accepted_prediction_tokens,audio_tokens,reasoning_tokens,rejected_prediction_tokens"},
{"PromptUsageDetail", "audio_tokens,cached_tokens,image_tokens,text_tokens"},
{"ServerSideToolUsageDetails", "code_interpreter_calls,document_search_calls,file_search_calls,image_generation_calls,mcp_calls,web_search_calls,x_posts_fetched,x_search_calls,x_users_fetched"},
{"ModelResponse", "background,completed_at,created_at,error,id,incomplete_details,instructions,max_output_tokens,max_tool_calls,model,object,output,parallel_tool_calls,previous_response_id,reasoning,service_tier,status,store,temperature,text,tool_choice,tools,top_p,usage,user"},
{"ModelUsage", "context_details,cost_in_nano_usd,cost_in_usd_ticks,input_tokens,input_tokens_details,num_server_side_tools_used,num_sources_used,output_tokens,output_tokens_details,server_side_tool_usage_details,total_tokens"},
{"OutputMessage", "content,id,role,status,type"},
{"Annotation", "end_index,start_index,title,type,url"},
{"FunctionToolCall", "arguments,call_id,id,name,status,type"},
{"Reasoning", "content,encrypted_content,id,status,summary,type"},
{"WebSearchCall", "action,id,status,type"},
{"FileSearchCall", "id,queries,results,status,type"},
{"CodeInterpreterCall", "code,id,outputs,status,type"},
{"McpCall", "arguments,error,id,name,output,server_label,status,type"},
{"CustomToolCall", "call_id,id,input,name,status,type"},
{"ShellCall", "action,call_id,environment,id,status,type"},
{"ImageGenerationCall", "id,prompt,result,status,type"},
{"ToolSearchCall", "arguments,call_id,execution,id,status,type"},
{"ToolSearchOutput", "call_id,execution,id,status,tools,type"},
{"DeleteStoredCompletionResponse", "deleted,id,object"},
{"ListInputItemsResponse", "data,first_id,has_more,last_id,object"},
{"CompactResponse", "created_at,id,model,object,output,usage"},
{"CompactUsage", "dropped_message_count,input_tokens,output_tokens,total_tokens"},
{"GeneratedImageResponse", "data,usage"},
{"GeneratedImage", "b64_json,file_output,mime_type,storage_error,url"},
{"FileOutput", "expires_at,file_id,filename,public_url,public_url_error,public_url_expires_at"},
{"GetDeferredImageResponse", "data,error,request_id,status,usage"},
{"ImageError", "code,message"},
{"MediaUsage", "cost_in_usd_ticks,input_tokens,input_tokens_details,output_tokens,output_tokens_details,total_tokens"},
{"StartDeferredResponse", "request_id"},
{"StartDeferredChatResponse", "request_id"},
{"VideoResponse", "error,model,progress,status,usage,video"},
{"GeneratedVideo", "duration,file_output,respect_moderation,storage_error,url"},
{"VideoError", "code,message"},
{"EmbeddingResponse", "data,model,object,usage"},
{"Embedding", "embedding,index,object"},
{"EmbeddingUsage", "prompt_tokens,total_tokens"},
{"TokenizeResponse", "token_ids"},
{"TokenizeResponseToken", "string_token,token_bytes,token_id"},
{"Model", "aliases,cached_prompt_text_token_price,cached_prompt_text_token_price_long_context,capabilities,completion_text_token_price,completion_text_token_price_long_context,context_length,created,id,image_price,long_context_threshold,object,owned_by,pricing,prompt_image_token_price,prompt_text_token_price,prompt_text_token_price_long_context"},
{"ListModelsResponse", "data,object"},
{"LanguageModel", "aliases,cached_prompt_text_token_price,cached_prompt_text_token_price_long_context,capabilities,completion_text_token_price,completion_text_token_price_long_context,created,fingerprint,id,input_modalities,long_context_threshold,object,output_modalities,owned_by,prompt_image_token_price,prompt_text_token_price,prompt_text_token_price_long_context,search_price,version"},
{"ModelCapabilities", "default_reasoning_effort,reasoning_effort"},
{"ImageGenerationModel", "aliases,created,fingerprint,id,image_price,input_modalities,max_prompt_length,object,output_modalities,owned_by,pricing,version"},
{"ImagePricingTier", "price_per_image,quality,resolution"},
{"VideoGenerationModel", "aliases,created,fingerprint,id,input_modalities,object,output_modalities,owned_by,version"},
{"EmbeddingModel", "aliases,created,fingerprint,id,input_modalities,object,output_modalities,owned_by,prompt_image_token_price,prompt_text_token_price,version"},
{"ApiKey", "acls,api_key_blocked,api_key_disabled,api_key_id,create_time,modified_by,modify_time,name,redacted_api_key,team_blocked,team_id,user_id"},
{"GetMeResponse", "api_key,oauth,team_blocked,team_id,user_id,zdr_status"},
{"MeApiKeyInfo", "api_key_id,blocked,disabled,redacted_api_key"},
{"Skill", "created_at,default_version,description,id,latest_version,name,object"},
{"SkillList", "data,first_id,has_more,last_id,object"},
{"DeletedSkill", "deleted,id,object"},
{"MessageResponse", "content,id,model,role,stop_reason,stop_sequence,type,usage"},
{"MessageUsage", "cache_creation_input_tokens,cache_read_input_tokens,input_tokens,output_tokens"},
{"CompleteResponse", "completion,id,model,stop_reason,type"},
{"SampleResponse", "choices,created,id,model,object,system_fingerprint,usage"},
{"SampleChoice", "finish_reason,index,text"},
};
// SCHEMAS-END
// Server-side and client tool types the builders write, with the fields each
// builder sets (checked against the ModelTool variants).
// TOOLS-BEGIN
static const ToolFields kTools[] = {
{"function", "name,description,parameters,strict,defer_loading"},
{"web_search", "allowed_domains,excluded_domains,enable_image_search,enable_image_understanding,user_location"},
{"x_search", "allowed_x_handles,excluded_x_handles,from_date,to_date,enable_image_understanding,enable_video_understanding"},
{"image_generation", "action"},
{"file_search", "vector_store_ids,max_num_results"},
{"code_interpreter", "container"},
{"mcp", "server_label,server_url,server_description,allowed_tools,authorization,headers,defer_loading"},
{"shell", "environment"},
{"tool_search", "execution"},
};
// TOOLS-END
const CallSpec& spec_at(std::size_t index) { return kCalls[index]; }
std::size_t call_count() { return sizeof(kCalls) / sizeof(kCalls[0]); }
std::size_t tool_field_count() { return sizeof(kTools) / sizeof(kTools[0]); }
const ToolFields& tool_fields_at(std::size_t index) { return kTools[index]; }
const CallSpec& spec_of(Call call) {
for (const CallSpec& s : kCalls)
if (s.call == call) return s;
Check::fail("call is not in the table");
}
clients::Method method_of(Call call) {
const char* m = spec_of(call).http_method;
if (!std::strcmp(m, "GET")) return clients::Method::kGet;
if (!std::strcmp(m, "DELETE")) return clients::Method::kDelete;
return clients::Method::kPost;
}
namespace {
bool in_list(const char* list, const std::string& item) {
std::string l(list);
std::size_t pos = 0;
while (pos <= l.size()) {
std::size_t comma = l.find(',', pos);
std::string part = l.substr(pos, comma == std::string::npos ? std::string::npos : comma - pos);
std::size_t at = part.find('@');
if (at != std::string::npos) part = part.substr(0, at);
if (part == item) return true;
if (comma == std::string::npos) break;
pos = comma + 1;
}
return false;
}
std::string percent_encode(const std::string& value) {
static const char* hex = "0123456789ABCDEF";
std::string out;
for (unsigned char c : value) {
bool keep = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '-' || c == '_' || c == '.' || c == '~';
if (keep) {
out += static_cast<char>(c);
} else {
out += '%';
out += hex[c >> 4];
out += hex[c & 15];
}
}
return out;
}
} // namespace
bool declares_body_field(Call call, const std::string& field) {
const CallSpec& s = spec_of(call);
return in_list(s.body_fields, field) || in_list(s.doc_fields, field);
}
bool declares_query_param(Call call, const std::string& name) {
return in_list(spec_of(call).query_params, name);
}
std::string render(Call call, const std::vector<std::string>& ids) {
std::string tmpl = spec_of(call).path;
std::string out;
std::size_t used = 0;
for (std::size_t i = 0; i < tmpl.size(); ++i) {
if (tmpl[i] != '{') {
out += tmpl[i];
continue;
}
std::size_t close = tmpl.find('}', i);
if (close == std::string::npos) Check::fail("bad path template");
std::string name = tmpl.substr(i + 1, close - i - 1);
if (used >= ids.size()) Check::fail("missing " + name);
std::string label = name.size() > 3 && name.compare(name.size() - 3, 3, "_id") == 0
? name.substr(0, name.size() - 3)
: name;
Check::id(label, ids[used]);
out += ids[used++];
i = close;
}
if (used != ids.size()) Check::fail("surplus path ids");
return out;
}
Query& Query::add(Call call, const std::string& name, const std::string& value) {
if (!declares_query_param(call, name)) Check::fail("query parameter " + name + " is not in the call table");
if (pairs_.size() >= Caps::kMaxQueryParams) Check::fail("too many query parameters");
if (value.empty() || value.size() > Caps::kMaxQueryValueLen) Check::fail("bad value for " + name);
pairs_.emplace_back(name, value);
return *this;
}
Query& Query::add(Call call, const std::string& name, long value) {
return add(call, name, std::to_string(value));
}
std::string Query::str() const {
std::string out;
for (const auto& [k, v] : pairs_) out += (out.empty() ? "?" : "&") + k + "=" + percent_encode(v);
return out;
}
} // namespace molo::xai::core
+90
View File
@@ -0,0 +1,90 @@
#pragma once
// The ONE table of REST calls this module makes. Every row is checked
// against xAI's OpenAPI document (https://api.x.ai/api-docs/openapi.json) or,
// for the batch API that the document does not list, against the REST
// reference page, by tools/check_spec.py. Code builds every URL from this
// table, never from a literal.
#include <string>
#include <vector>
#include "clients/request.h"
namespace molo::xai::core {
enum class Call {
// Text
kChatCreate, kChatDeferredGet, kCompletionsLegacy, kCompleteLegacy, kMessagesCreate,
kResponsesCreate, kResponsesGet, kResponsesDelete, kResponsesInputItems, kResponsesCompact,
// Media
kImagesGenerate, kImagesEdit, kImagesGet,
kVideosGenerate, kVideosEdit, kVideosExtend, kVideosGet,
// Embeddings, tokenizer
kEmbeddingsCreate, kTokenizeText,
// Models
kModelsList, kModelsGet, kLanguageModelsList, kLanguageModelsGet, kImageModelsList,
kImageModelsGet, kVideoModelsList, kVideoModelsGet, kEmbeddingModelsList, kEmbeddingModelsGet,
// Account
kApiKeyInfo, kMe,
// Skills
kSkillsList, kSkillsUpload, kSkillsGet, kSkillsDelete, kSkillsContent,
// Batch (documented in the REST reference, absent from the OpenAPI document)
kBatchCreate, kBatchList, kBatchGet, kBatchRequestsAdd, kBatchRequestsList, kBatchResults,
kBatchCancel,
};
struct CallSpec {
Call call;
const char* http_method; // GET POST DELETE
const char* path; // template: /v1/responses/{response_id}
const char* op_id; // OpenAPI operationId, or "docs" for documentation-only rows
const char* body_fields; // comma list: request body fields we send (in the spec)
const char* query_params; // comma list: query parameters we send
const char* doc_fields; // comma list: fields only the guides document (see SPEC Unverified)
const char* request_schema; // OpenAPI schema name of the request body, "" for none
const char* response_schema; // OpenAPI schema name of the response body
const char* source; // "openapi" or "docs"
};
struct SchemaFields {
const char* schema; // OpenAPI schema name
const char* fields; // comma list: fields this module reads
};
struct ToolFields {
const char* type; // tool `type` discriminator
const char* fields; // comma list: fields the builder writes
};
const CallSpec& spec_of(Call call);
clients::Method method_of(Call call);
std::size_t call_count();
const CallSpec& spec_at(std::size_t index);
// Path for the call. `ids` fill the {placeholders} in order; each is
// validated with Service::valid_id. Throws ApiError::invalid on a missing,
// surplus or bad id.
std::string render(Call call, const std::vector<std::string>& ids = {});
// True when `field` is a body field or doc-only field the row declares.
bool declares_body_field(Call call, const std::string& field);
bool declares_query_param(Call call, const std::string& name);
// Query string builder: percent-encodes, validates names against the row.
class Query {
public:
Query& add(Call call, const std::string& name, const std::string& value);
Query& add(Call call, const std::string& name, long value);
bool empty() const { return pairs_.empty(); }
std::string str() const; // "" or "?a=b&c=d"
private:
std::vector<std::pair<std::string, std::string>> pairs_;
};
// Parity data for the checker and tests.
std::size_t tool_field_count();
const ToolFields& tool_fields_at(std::size_t index);
} // namespace molo::xai::core
+81
View File
@@ -0,0 +1,81 @@
#pragma once
// Static ceilings for the xAI core module. Never autotuned. Ceilings that
// come from the vendor are marked (spec) or (docs); the others are ours and
// exist so that no count, size, wait or page walk is unbounded.
#include <cstddef>
namespace molo::xai::core {
struct Caps {
// Text and structure
static constexpr std::size_t kMaxModelLen = 128;
static constexpr std::size_t kMaxNameLen = 256;
static constexpr std::size_t kMaxTextBytes = 4u * 1024u * 1024u;
static constexpr std::size_t kMaxPromptBytes = 64u * 1024u; // image / video prompts
static constexpr std::size_t kMaxUrlLen = 8u * 1024u * 1024u; // data: URLs carry images
static constexpr std::size_t kMaxTokenLen = 128; // enum-like strings
static constexpr std::size_t kMaxQueryValueLen = 2048;
static constexpr std::size_t kMaxQueryParams = 16;
// Conversation
static constexpr std::size_t kMaxInputItems = 4096;
static constexpr std::size_t kMaxContentParts = 256;
static constexpr std::size_t kMaxChatTools = 128; // (docs) chat completions
static constexpr std::size_t kMaxResponsesTools = 1000; // (spec) responses
static constexpr std::size_t kMaxStop = 4; // (spec)
static constexpr std::size_t kMaxIncludes = 16;
static constexpr std::size_t kMaxChoices = 16; // chat `n`, ours
static constexpr int kMaxTopLogprobs = 8; // (spec)
static constexpr std::size_t kMaxGuidelines = 3; // (spec) one per category
static constexpr int kMaxTurns = 1000; // responses max_turns, ours
// Tools
static constexpr std::size_t kMaxDomains = 5; // (spec) web search allow / exclude
static constexpr std::size_t kMaxXHandles = 10; // (docs) x search
static constexpr std::size_t kMaxCollections = 10; // ours, file_search vector_store_ids
static constexpr std::size_t kMaxMcpHeaders = 32;
static constexpr std::size_t kMaxAllowedTools = 256;
// Media
static constexpr int kMaxImages = 10; // (docs) n = 1..10
static constexpr int kMinVideoSeconds = 1; // (spec)
static constexpr int kMaxVideoSeconds = 15; // (spec)
static constexpr int kMinExtensionSeconds = 2; // (spec)
static constexpr int kMaxExtensionSeconds = 10; // (spec)
static constexpr std::size_t kMaxReferenceImages = 14; // (docs) grok-imagine-video-1.5
static constexpr std::size_t kMaxReferenceAudios = 3; // (docs)
static constexpr std::size_t kMaxKeyframes = 4; // (docs)
static constexpr std::size_t kMaxEditImages = 16; // ours, multi-image edits
static constexpr long kMaxStorageExpirySeconds = 2592000; // (spec) 30 days
static constexpr long kMinPublicUrlSeconds = 3600; // (spec)
// Embeddings and tokenize
static constexpr std::size_t kMaxEmbeddingInputs = 1024;
static constexpr int kMaxEmbeddingDimensions = 65536;
static constexpr std::size_t kMaxTokenizeBytes = 1u * 1024u * 1024u;
// Batch
static constexpr std::size_t kMaxBatchRequestsPerAdd = 1000; // ours
static constexpr int kMaxBatchPageLimit = 1000; // (docs)
static constexpr std::size_t kMaxBatchPagesWalked = 100;
static constexpr std::size_t kMaxBatchRequestBytes = 25u * 1024u * 1024u; // (docs)
// Lists
static constexpr int kMaxListLimit = 1000;
// Waits (deferred results, video)
static constexpr int kMinPollIntervalMs = 1000;
static constexpr int kMaxPollIntervalMs = 60000;
static constexpr int kMaxWaitMs = 30 * 60 * 1000;
static constexpr int kDefaultWaitMs = 15 * 60 * 1000;
static constexpr int kDefaultPollIntervalMs = 5000;
// Streaming
static constexpr std::size_t kMaxStreamEvents = 200000;
static constexpr std::size_t kMaxStreamTextBytes = 16u * 1024u * 1024u;
static constexpr std::size_t kMaxAccumulatedItems = 4096;
};
} // namespace molo::xai::core
+428
View File
@@ -0,0 +1,428 @@
#include "xai/core/chat.h"
#include "llm/caps.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
Json obj(const char* type) {
Json o = Json::object();
o.set("type", Json(type));
return o;
}
ChatToolCall parse_tool_call(const Json& tc) {
ChatToolCall c;
c.id = jx::str(tc, "id");
c.type = jx::str(tc, "type");
Json fn = jx::object(tc, "function");
c.name = jx::str(fn, "name");
const Json* args = jx::get(fn, "arguments");
if (args && args->is_string()) c.arguments = args->as_string();
else if (args && args->is_object()) c.arguments = args->dump();
c.index = jx::num(tc, "index", -1);
return c;
}
std::vector<ChatToolCall> parse_tool_calls(const Json& doc, const char* key) {
std::vector<ChatToolCall> out;
const Json* arr = jx::get(doc, key);
if (arr && arr->is_array())
for (const Json& tc : arr->as_array()) {
if (out.size() >= Caps::kMaxAccumulatedItems) break;
out.push_back(parse_tool_call(tc));
}
return out;
}
std::vector<OutputFile> parse_files(const Json& doc) {
std::vector<OutputFile> out;
const Json* arr = jx::get(doc, "output_files");
if (arr && arr->is_array())
for (const Json& f : arr->as_array()) {
if (out.size() >= Caps::kMaxAccumulatedItems) break;
out.push_back({jx::str(f, "file_id"), jx::str(f, "name")});
}
return out;
}
Json content_value(const std::vector<ChatPart>& parts) {
Check::count("content parts", parts.size(), 1, Caps::kMaxContentParts);
Json arr = Json::array();
for (const auto& p : parts) arr.push(p.json());
return arr;
}
} // namespace
ChatPart ChatPart::text(std::string text) {
Check::text("text", text, Caps::kMaxTextBytes, true);
Json o = obj("text");
o.set("text", Json(std::move(text)));
return ChatPart(std::move(o));
}
ChatPart ChatPart::image(std::string url) {
Check::url("image url", url);
Json u = Json::object();
u.set("url", Json(std::move(url)));
Json o = obj("image_url");
o.set("image_url", u);
return ChatPart(std::move(o));
}
ChatPart ChatPart::image_file(std::string file_id) {
Check::id("file", file_id);
Json u = Json::object();
u.set("file_id", Json(std::move(file_id)));
Json o = obj("image_url");
o.set("image_url", u);
return ChatPart(std::move(o));
}
ChatPart ChatPart::file(std::string file_id) {
Check::id("file", file_id);
Json f = Json::object();
f.set("file_id", Json(std::move(file_id)));
Json o = obj("file");
o.set("file", f);
return ChatPart(std::move(o));
}
ChatPart ChatPart::file_url(std::string url) {
Check::http_url("file url", url);
Json f = Json::object();
f.set("url", Json(std::move(url)));
Json o = obj("file");
o.set("file", f);
return ChatPart(std::move(o));
}
Json ChatToolCall::arguments_json() const {
if (arguments.empty()) return Json::object();
try {
Json j = Json::parse(arguments);
return j.is_object() ? j : Json::object();
} catch (const std::exception&) {
return Json::object();
}
}
namespace {
Json text_message(const char* role, std::string text, bool allow_empty) {
Check::text("message", text, Caps::kMaxTextBytes, allow_empty);
Json o = Json::object();
o.set("role", Json(role));
o.set("content", Json(std::move(text)));
return o;
}
} // namespace
ChatMessage ChatMessage::system(std::string text) {
return ChatMessage(text_message("system", std::move(text), false));
}
ChatMessage ChatMessage::developer(std::string text) {
return ChatMessage(text_message("developer", std::move(text), false));
}
ChatMessage ChatMessage::user(std::string text) {
return ChatMessage(text_message("user", std::move(text), false));
}
ChatMessage ChatMessage::user(const std::vector<ChatPart>& parts) {
Json o = Json::object();
o.set("role", Json("user"));
o.set("content", content_value(parts));
return ChatMessage(std::move(o));
}
ChatMessage ChatMessage::assistant(std::string text) {
return ChatMessage(text_message("assistant", std::move(text), true));
}
ChatMessage ChatMessage::assistant_calls(std::string text, const std::vector<ChatToolCall>& calls,
std::string reasoning_content) {
Check::text("message", text, Caps::kMaxTextBytes, true);
Check::count("tool calls", calls.size(), 1, Caps::kMaxChatTools);
Json o = Json::object();
o.set("role", Json("assistant"));
if (!text.empty()) o.set("content", Json(std::move(text)));
if (!reasoning_content.empty()) o.set("reasoning_content", Json(std::move(reasoning_content)));
Json arr = Json::array();
for (const auto& c : calls) {
Check::id("tool call", c.id);
Check::id("function", c.name);
Json fn = Json::object();
fn.set("name", Json(c.name));
fn.set("arguments", Json(c.arguments.empty() ? std::string("{}") : c.arguments));
Json tc = Json::object();
tc.set("id", Json(c.id));
tc.set("type", Json("function"));
tc.set("function", fn);
arr.push(tc);
}
o.set("tool_calls", arr);
return ChatMessage(std::move(o));
}
ChatMessage ChatMessage::tool_result(const std::string& tool_call_id, std::string content) {
Check::id("tool call", tool_call_id);
Check::text("tool result", content, Caps::kMaxTextBytes, true);
Json o = Json::object();
o.set("role", Json("tool"));
o.set("tool_call_id", Json(tool_call_id));
o.set("content", Json(std::move(content)));
return ChatMessage(std::move(o));
}
ChatMessage& ChatMessage::name(std::string n) {
if (!n.empty()) {
Check::text("name", n, 256);
json_.set("name", Json(std::move(n)));
}
return *this;
}
std::string ChatMessage::role() const { return jx::str(json_, "role"); }
ChatRequest& ChatRequest::add(ChatMessage m) {
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
messages_.push_back(std::move(m));
return *this;
}
ChatRequest& ChatRequest::add_tool(const FunctionTool& tool) {
Check::upto("tools", tools_.size(), Caps::kMaxChatTools - 1);
tools_.push_back(tool.to_chat_json());
return *this;
}
Json ChatRequest::to_json(bool stream, bool deferred) const {
Check::model(model_);
Check::count("messages", messages_.size(), 1, llm::Caps::kMaxMessages);
if (stream && deferred) Check::fail("a request cannot be both streamed and deferred");
Json body = Json::object();
body.set("model", Json(model_));
Json msgs = Json::array();
for (const auto& m : messages_) msgs.push(m.json());
body.set("messages", msgs);
if (stream) {
body.set("stream", Json(true));
if (include_usage_) {
Json so = Json::object();
so.set("include_usage", Json(*include_usage_));
body.set("stream_options", so);
}
} else if (include_usage_) {
Check::fail("include_usage only applies to streamed requests");
}
if (deferred) body.set("deferred", Json(true));
if (max_completion_ && max_tokens_) Check::fail("max_tokens is deprecated; set only max_completion_tokens");
if (max_completion_) {
Check::range("max_completion_tokens", *max_completion_, 1, 10'000'000);
body.set("max_completion_tokens", Json(*max_completion_));
}
if (max_tokens_) {
Check::range("max_tokens", *max_tokens_, 1, 10'000'000);
body.set("max_tokens", Json(*max_tokens_));
}
if (temperature_) { Check::range("temperature", *temperature_, 0, 2); body.set("temperature", Json(*temperature_)); }
if (top_p_) { Check::range("top_p", *top_p_, 0, 1); body.set("top_p", Json(*top_p_)); }
if (frequency_) { Check::range("frequency_penalty", *frequency_, -2, 2); body.set("frequency_penalty", Json(*frequency_)); }
if (presence_) { Check::range("presence_penalty", *presence_, -2, 2); body.set("presence_penalty", Json(*presence_)); }
if (n_) { Check::range("n", *n_, 1, static_cast<double>(Caps::kMaxChoices)); body.set("n", Json(*n_)); }
jx::put(body, "seed", seed_);
Check::strings("stop", stop_, Caps::kMaxStop, 256);
jx::put_strings(body, "stop", stop_);
jx::put(body, "logprobs", logprobs_);
if (top_logprobs_) {
Check::range("top_logprobs", *top_logprobs_, 0, Caps::kMaxTopLogprobs);
if (!logprobs_ || !*logprobs_) Check::fail("top_logprobs needs logprobs=true");
body.set("top_logprobs", Json(*top_logprobs_));
}
if (!effort_.empty()) { Check::token("reasoning_effort", effort_); body.set("reasoning_effort", Json(effort_)); }
if (format_) body.set("response_format", format_->to_chat());
if (!tools_.empty()) {
Json arr = Json::array();
for (const Json& t : tools_) arr.push(t);
body.set("tools", arr);
}
if (tool_choice_) body.set("tool_choice", tool_choice_->to_chat());
jx::put(body, "parallel_tool_calls", parallel_);
if (search_) body.set("search_parameters", search_->to_json());
if (!guidelines_.empty()) body.set("guidelines", guidelines_json(guidelines_));
if (tier_) { Check::one_of("service_tier", *tier_, {"default", "priority", "fast"}); body.set("service_tier", Json(*tier_)); }
if (user_) { Check::text("user", *user_, 256); body.set("user", Json(*user_)); }
if (safety_) { Check::text("safety_identifier", *safety_, 256); body.set("safety_identifier", Json(*safety_)); }
if (cache_key_) { Check::text("prompt_cache_key", *cache_key_, 256); body.set("prompt_cache_key", Json(*cache_key_)); }
return body;
}
ChatCompletion ChatCompletion::parse(const Json& doc) {
ChatCompletion c;
c.raw = doc;
c.id = jx::str(doc, "id");
c.model = jx::str(doc, "model");
c.object = jx::str(doc, "object");
c.created = jx::num(doc, "created");
c.system_fingerprint = jx::str(doc, "system_fingerprint");
c.service_tier = jx::str(doc, "service_tier");
const Json* choices = jx::get(doc, "choices");
if (choices && choices->is_array())
for (const Json& ch : choices->as_array()) {
if (c.choices.size() >= Caps::kMaxChoices) break;
ChatChoice o;
o.index = jx::num(ch, "index");
o.finish_reason = jx::str(ch, "finish_reason");
if (const Json* lp = jx::get(ch, "logprobs"); lp && lp->is_object()) o.logprobs = *lp;
Json msg = jx::object(ch, "message");
o.role = jx::str(msg, "role");
o.content = jx::str(msg, "content");
o.reasoning_content = jx::str(msg, "reasoning_content");
o.refusal = jx::str(msg, "refusal");
o.tool_calls = parse_tool_calls(msg, "tool_calls");
c.choices.push_back(std::move(o));
}
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) c.usage = TokenUsage::from_chat(*u);
c.citations = jx::strings(doc, "citations");
c.output_files = parse_files(doc);
return c;
}
std::vector<ChatToolCall> ChatCompletion::function_calls() const {
std::vector<ChatToolCall> out;
if (!first()) return out;
for (const auto& c : first()->tool_calls)
if (c.type.empty() || c.type == "function") out.push_back(c);
return out;
}
CompletionChunk CompletionChunk::parse(const Json& doc) {
CompletionChunk c;
c.id = jx::str(doc, "id");
c.model = jx::str(doc, "model");
c.created = jx::num(doc, "created");
const Json* choices = jx::get(doc, "choices");
if (choices && choices->is_array())
for (const Json& ch : choices->as_array()) {
if (c.choices.size() >= Caps::kMaxChoices) break;
ChatDelta d;
d.index = jx::num(ch, "index");
d.finish_reason = jx::str(ch, "finish_reason");
Json delta = jx::object(ch, "delta");
d.role = jx::str(delta, "role");
d.content = jx::str(delta, "content");
d.reasoning_content = jx::str(delta, "reasoning_content");
d.images = jx::strings(delta, "images");
d.tool_calls = parse_tool_calls(delta, "tool_calls");
c.choices.push_back(std::move(d));
}
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) c.usage = TokenUsage::from_chat(*u);
c.citations = jx::strings(doc, "citations");
c.output_files = parse_files(doc);
return c;
}
bool ChatStreamAccumulator::feed(const CompletionChunk& chunk) {
if (stopped_) return false;
if (++chunks_ > static_cast<long>(Caps::kMaxStreamEvents)) {
stopped_ = true;
return false;
}
if (!chunk.id.empty()) id_ = chunk.id;
if (!chunk.model.empty()) model_ = chunk.model;
if (chunk.created) created_ = chunk.created;
if (chunk.usage) usage_ = *chunk.usage;
if (!chunk.citations.empty()) citations_ = chunk.citations;
if (!chunk.output_files.empty()) files_ = chunk.output_files;
for (const ChatDelta& d : chunk.choices) {
if (d.index < 0 || d.index >= static_cast<long>(Caps::kMaxChoices)) continue;
if (choices_.size() <= static_cast<std::size_t>(d.index)) choices_.resize(d.index + 1);
ChatChoice& c = choices_[d.index];
c.index = d.index;
if (!d.role.empty()) c.role = d.role;
if (c.content.size() + d.content.size() + c.reasoning_content.size() + d.reasoning_content.size() >
Caps::kMaxStreamTextBytes) {
stopped_ = true;
return false;
}
c.content += d.content;
c.reasoning_content += d.reasoning_content;
if (!d.finish_reason.empty()) c.finish_reason = d.finish_reason;
for (const ChatToolCall& frag : d.tool_calls) {
// Fragments carry an index; the first one has id and name, later ones only arguments.
long slot = frag.index >= 0 ? frag.index : static_cast<long>(c.tool_calls.size()) - 1;
if (frag.index < 0 && !frag.id.empty()) slot = static_cast<long>(c.tool_calls.size());
if (slot < 0) slot = 0;
if (slot >= static_cast<long>(Caps::kMaxChatTools)) {
stopped_ = true;
return false;
}
if (c.tool_calls.size() <= static_cast<std::size_t>(slot)) c.tool_calls.resize(slot + 1);
ChatToolCall& t = c.tool_calls[slot];
t.index = slot;
if (!frag.id.empty()) t.id = frag.id;
if (!frag.type.empty()) t.type = frag.type;
if (!frag.name.empty()) t.name = frag.name;
t.arguments += frag.arguments;
}
}
return true;
}
ChatCompletion ChatStreamAccumulator::result() const {
ChatCompletion c;
c.id = id_;
c.model = model_;
c.created = created_;
c.object = "chat.completion";
c.choices = choices_;
c.usage = usage_;
c.citations = citations_;
c.output_files = files_;
return c;
}
ChatCompletion ChatService::create(const ChatRequest& request) {
Json body = request.to_json(false, false);
return ChatCompletion::parse(call_json(Call::kChatCreate, {}, &body));
}
ChatCompletion ChatService::stream(const ChatRequest& request, const ChunkSink& sink) {
Json body = request.to_json(true, false);
ChatStreamAccumulator acc;
call_stream(Call::kChatCreate, body, [&](const llm::SseEvent& ev) {
CompletionChunk chunk = CompletionChunk::parse(ev.data);
bool keep = acc.feed(chunk);
bool user_keep = sink ? sink(chunk) : true;
return keep && user_keep;
});
if (acc.stopped()) throw llm::ApiError(kVendor, 502, "STREAM_CAP", "chat stream hit a cap");
return acc.result();
}
std::string ChatService::start_deferred(const ChatRequest& request) {
Json body = request.to_json(false, true);
Json reply = call_json(Call::kChatCreate, {}, &body);
std::string id = jx::str(reply, "request_id");
if (id.empty()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "deferred reply has no request_id");
return id;
}
std::optional<ChatCompletion> ChatService::poll_deferred(const std::string& request_id) {
llm::RawResponse raw = call_raw(Call::kChatDeferredGet, {request_id});
if (raw.status == 202) return std::nullopt;
ChatCompletion c = ChatCompletion::parse(parse_body(raw));
if (c.pending()) return std::nullopt;
return c;
}
ChatCompletion ChatService::wait_deferred(const std::string& request_id, const WaitPolicy& policy,
const Sleeper& sleeper) {
Check::id("request", request_id);
std::function<std::optional<ChatCompletion>()> step = [&]() { return poll_deferred(request_id); };
std::optional<ChatCompletion> r = poll_until<ChatCompletion>(policy, step, sleeper);
if (!r) throw llm::ApiError(kVendor, 504, "WAIT_TIMEOUT", "deferred completion not ready within the wait budget");
return *r;
}
} // namespace molo::xai::core
+204
View File
@@ -0,0 +1,204 @@
#pragma once
// Chat completions (POST /v1/chat/completions), the legacy wire xAI keeps for
// OpenAI-SDK compatibility: request builder, typed response and chunk
// parsing, stream accumulation, and the deferred-completion calls.
#include <functional>
#include <optional>
#include <string>
#include <vector>
#include "xai/core/poll.h"
#include "xai/core/response_format.h"
#include "xai/core/search_parameters.h"
#include "xai/core/service.h"
#include "xai/core/tools.h"
#include "xai/core/usage.h"
namespace molo::xai::core {
class ChatPart {
public:
static ChatPart text(std::string text);
static ChatPart image(std::string url); // public or data: URL
static ChatPart image_file(std::string file_id);
static ChatPart file(std::string file_id);
static ChatPart file_url(std::string url);
const Json& json() const { return json_; }
private:
explicit ChatPart(Json j) : json_(std::move(j)) {}
Json json_;
};
struct ChatToolCall {
std::string id;
std::string type; // "function" or a server-side tool type
std::string name;
std::string arguments; // JSON string as sent by the model
long index = -1;
Json arguments_json() const; // parsed; {} when empty or invalid
};
class ChatMessage {
public:
static ChatMessage system(std::string text);
static ChatMessage developer(std::string text);
static ChatMessage user(std::string text);
static ChatMessage user(const std::vector<ChatPart>& parts);
static ChatMessage assistant(std::string text);
// An assistant turn that called tools (content may be empty).
static ChatMessage assistant_calls(std::string text, const std::vector<ChatToolCall>& calls,
std::string reasoning_content = "");
static ChatMessage tool_result(const std::string& tool_call_id, std::string content);
ChatMessage& name(std::string n);
const Json& json() const { return json_; }
std::string role() const;
private:
explicit ChatMessage(Json j) : json_(std::move(j)) {}
Json json_;
};
class ChatRequest {
public:
explicit ChatRequest(std::string model) : model_(std::move(model)) {}
ChatRequest& add(ChatMessage m);
ChatRequest& max_completion_tokens(int n) { max_completion_ = n; return *this; }
ChatRequest& max_tokens_deprecated(int n) { max_tokens_ = n; return *this; }
ChatRequest& temperature(double v) { temperature_ = v; return *this; }
ChatRequest& top_p(double v) { top_p_ = v; return *this; }
ChatRequest& n(int v) { n_ = v; return *this; }
ChatRequest& seed(long v) { seed_ = v; return *this; }
ChatRequest& stop(std::vector<std::string> s) { stop_ = std::move(s); return *this; }
ChatRequest& frequency_penalty(double v) { frequency_ = v; return *this; }
ChatRequest& presence_penalty(double v) { presence_ = v; return *this; }
ChatRequest& logprobs(bool v) { logprobs_ = v; return *this; }
ChatRequest& top_logprobs(int v) { top_logprobs_ = v; return *this; }
ChatRequest& reasoning_effort(std::string e) { effort_ = std::move(e); return *this; }
ChatRequest& response_format(ResponseFormat f) { format_ = std::move(f); return *this; }
ChatRequest& add_tool(const FunctionTool& tool);
ChatRequest& tool_choice(ToolChoice c) { tool_choice_ = std::move(c); return *this; }
ChatRequest& parallel_tool_calls(bool v) { parallel_ = v; return *this; }
ChatRequest& search_parameters(SearchParameters p) { search_ = std::move(p); return *this; }
ChatRequest& guideline(Guideline g) { guidelines_.push_back(std::move(g)); return *this; }
ChatRequest& service_tier(std::string t) { tier_ = std::move(t); return *this; }
ChatRequest& user(std::string u) { user_ = std::move(u); return *this; }
ChatRequest& safety_identifier(std::string s) { safety_ = std::move(s); return *this; }
ChatRequest& prompt_cache_key(std::string k) { cache_key_ = std::move(k); return *this; }
ChatRequest& include_usage(bool v) { include_usage_ = v; return *this; } // stream_options
const std::string& model() const { return model_; }
// Validates and returns the wire body. `stream` sets stream:true (and
// stream_options.include_usage when requested); `deferred` sets deferred:true.
Json to_json(bool stream = false, bool deferred = false) const;
private:
std::string model_;
std::vector<ChatMessage> messages_;
std::optional<int> max_completion_, max_tokens_, n_, top_logprobs_;
std::optional<double> temperature_, top_p_, frequency_, presence_;
std::optional<long> seed_;
std::optional<bool> logprobs_, parallel_, include_usage_;
std::vector<std::string> stop_;
std::string effort_;
std::optional<ResponseFormat> format_;
std::vector<Json> tools_;
std::optional<ToolChoice> tool_choice_;
std::optional<SearchParameters> search_;
std::vector<Guideline> guidelines_;
std::optional<std::string> tier_, user_, safety_, cache_key_;
};
struct ChatChoice {
long index = 0;
std::string finish_reason;
std::string role;
std::string content;
std::string reasoning_content;
std::string refusal;
std::vector<ChatToolCall> tool_calls;
Json logprobs;
};
struct OutputFile {
std::string file_id;
std::string name;
};
struct ChatCompletion {
std::string id, model, object, system_fingerprint, service_tier;
long created = 0;
std::vector<ChatChoice> choices;
TokenUsage usage;
std::vector<std::string> citations;
std::vector<OutputFile> output_files;
Json raw;
static ChatCompletion parse(const Json& doc);
const ChatChoice* first() const { return choices.empty() ? nullptr : &choices.front(); }
std::string text() const { return first() ? first()->content : ""; }
std::string reasoning() const { return first() ? first()->reasoning_content : ""; }
std::string finish_reason() const { return first() ? first()->finish_reason : ""; }
std::vector<ChatToolCall> function_calls() const; // type function (or empty type)
bool pending() const { return id.empty() && choices.empty(); }
};
// One streamed chunk (`chat.completion.chunk`).
struct ChatDelta {
long index = 0;
std::string role, content, reasoning_content, finish_reason;
std::vector<std::string> images;
std::vector<ChatToolCall> tool_calls; // fragments: id/name once, arguments in pieces
};
struct CompletionChunk {
std::string id, model;
long created = 0;
std::vector<ChatDelta> choices;
std::optional<TokenUsage> usage;
std::vector<std::string> citations;
std::vector<OutputFile> output_files;
static CompletionChunk parse(const Json& doc);
};
// Folds chunks into a ChatCompletion (choice 0 and further `n` choices).
class ChatStreamAccumulator {
public:
bool feed(const CompletionChunk& chunk); // false when a cap stopped the fold
ChatCompletion result() const;
bool stopped() const { return stopped_; }
long chunks() const { return chunks_; }
private:
std::string id_, model_;
long created_ = 0, chunks_ = 0;
std::vector<ChatChoice> choices_;
TokenUsage usage_;
std::vector<std::string> citations_;
std::vector<OutputFile> files_;
bool stopped_ = false;
};
class ChatService : public XaiService {
public:
using ChunkSink = std::function<bool(const CompletionChunk&)>; // false stops
explicit ChatService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
ChatCompletion create(const ChatRequest& request);
ChatCompletion stream(const ChatRequest& request, const ChunkSink& sink = {});
// Deferred completions: the result is fetchable exactly once within 24 hours.
std::string start_deferred(const ChatRequest& request);
// nullopt while the API answers 202 (not ready).
std::optional<ChatCompletion> poll_deferred(const std::string& request_id);
// Polls within the policy; throws ApiError 504 WAIT_TIMEOUT when the budget is spent.
ChatCompletion wait_deferred(const std::string& request_id, const WaitPolicy& policy = {},
const Sleeper& sleeper = real_sleeper());
};
} // namespace molo::xai::core
+280
View File
@@ -0,0 +1,280 @@
#include "xai/core/chat_model.h"
#include <map>
#include "llm/caps.h"
#include "security/crypto/Base64.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
std::string image_url_of(const llm::ImageData& img) {
if (!img.url.empty()) return img.url;
if (img.bytes.empty()) Check::fail("image has neither url nor bytes");
std::string mime = img.mime.empty() ? "image/png" : img.mime;
return "data:" + mime + ";base64," + security::crypto::Base64::encode(img.bytes);
}
std::string tool_args_text(const llm::ToolCall& c) {
return c.arguments.is_object() ? c.arguments.dump() : std::string("{}");
}
std::string finish_of(const std::string& vendor_reason, bool has_calls) {
if (has_calls) return "tool_calls";
if (vendor_reason == "length" || vendor_reason == "max_output_tokens") return "length";
if (vendor_reason == "content_filter") return "content_filter";
return "stop";
}
std::optional<ResponseFormat> format_of(const llm::ChatRequest& r) {
if (r.response_schema.is_object()) return ResponseFormat::json_schema("response", r.response_schema);
if (r.json_mode) return ResponseFormat::json_object();
return std::nullopt;
}
llm::ChatResponse from_responses(const ModelResponse& r) {
llm::ChatResponse out;
out.id = r.id();
out.model = r.model();
out.message.role = llm::Role::kAssistant;
std::string reasoning = r.reasoning_text();
if (reasoning.empty()) reasoning = r.summary_text();
if (!reasoning.empty()) {
llm::ContentPart p;
p.reasoning = reasoning;
out.message.parts.push_back(std::move(p));
}
std::string text = r.text();
if (!text.empty()) out.message.parts.push_back(llm::ContentPart::of_text(text));
bool calls = false;
for (const OutputItem& c : r.function_calls()) {
llm::ToolCall tc;
tc.id = c.call_id();
tc.name = c.name();
tc.arguments = c.arguments();
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
calls = true;
}
out.finish_reason = finish_of(r.incomplete_reason(), calls);
out.usage = r.usage().to_neutral();
out.raw = r.raw();
return out;
}
llm::ChatResponse from_chat(const ChatCompletion& c) {
llm::ChatResponse out;
out.id = c.id;
out.model = c.model;
out.message.role = llm::Role::kAssistant;
if (const ChatChoice* ch = c.first()) {
if (!ch->reasoning_content.empty()) {
llm::ContentPart p;
p.reasoning = ch->reasoning_content;
out.message.parts.push_back(std::move(p));
}
if (!ch->content.empty()) out.message.parts.push_back(llm::ContentPart::of_text(ch->content));
for (const ChatToolCall& t : c.function_calls()) {
llm::ToolCall tc;
tc.id = t.id;
tc.name = t.name;
tc.arguments = t.arguments_json();
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
}
out.finish_reason = finish_of(ch->finish_reason, !c.function_calls().empty());
}
out.usage = c.usage.to_neutral();
out.raw = c.raw;
return out;
}
} // namespace
XaiChatModel::XaiChatModel(std::shared_ptr<llm::ApiClient> client, std::string default_model, Wire wire)
: client_(std::move(client)), default_model_(std::move(default_model)), wire_(wire) {}
std::string XaiChatModel::model_of(const llm::ChatRequest& r) const {
return r.model.empty() ? default_model_ : r.model;
}
Json XaiChatModel::responses_body(const llm::ChatRequest& r, bool stream) const {
if (!r.stop.empty())
Check::fail("stop sequences are not supported by the Responses API; use Wire::kChatCompletions");
ResponseRequest req(model_of(r));
req.store(false);
if (!r.system.empty()) req.instructions(r.system);
Check::count("messages", r.messages.size(), 1, llm::Caps::kMaxMessages);
for (const llm::Message& m : r.messages) {
if (m.role == llm::Role::kTool) {
for (const auto& p : m.parts)
if (p.tool_result) req.add_input(InputItem::function_output(p.tool_result->call_id, p.tool_result->content));
continue;
}
std::vector<InputPart> parts;
std::string role = llm::role_name(m.role);
bool assistant = m.role == llm::Role::kAssistant;
for (const auto& p : m.parts) {
if (!p.text.empty()) parts.push_back(assistant ? InputPart::assistant_text(p.text) : InputPart::text(p.text));
if (p.image) parts.push_back(InputPart::image(image_url_of(*p.image)));
}
if (!parts.empty()) req.add_input(InputItem::message(role, parts));
for (const auto& p : m.parts)
if (p.tool_call) {
Json fc = Json::object();
fc.set("type", Json("function_call"));
fc.set("call_id", Json(p.tool_call->id));
fc.set("name", Json(p.tool_call->name));
fc.set("arguments", Json(tool_args_text(*p.tool_call)));
req.add_input(InputItem::replay(fc));
}
}
for (const llm::ToolSpec& t : r.tools) req.add_tool(FunctionTool(t.name, t.description, t.parameters));
if (auto tc = ToolChoice::from_neutral(r.tool_choice)) req.tool_choice(*tc);
if (r.max_tokens) req.max_output_tokens(*r.max_tokens);
if (r.temperature) req.temperature(*r.temperature);
if (r.top_p) req.top_p(*r.top_p);
if (auto f = format_of(r)) req.response_format(*f);
if (!r.reasoning_effort.empty()) req.reasoning_effort(r.reasoning_effort);
Json body = req.to_json(stream);
jx::merge(body, r.extra);
return body;
}
Json XaiChatModel::chat_body(const llm::ChatRequest& r, bool stream) const {
ChatRequest req(model_of(r));
if (!r.system.empty()) req.add(ChatMessage::system(r.system));
for (const llm::Message& m : r.messages) {
if (m.role == llm::Role::kTool) {
for (const auto& p : m.parts)
if (p.tool_result) req.add(ChatMessage::tool_result(p.tool_result->call_id, p.tool_result->content));
continue;
}
std::string text;
std::vector<ChatPart> parts;
std::vector<ChatToolCall> calls;
std::string reasoning;
bool has_image = false;
for (const auto& p : m.parts) {
if (!p.text.empty()) { text += p.text; parts.push_back(ChatPart::text(p.text)); }
if (p.image) { has_image = true; parts.push_back(ChatPart::image(image_url_of(*p.image))); }
if (p.tool_call) calls.push_back({p.tool_call->id, "function", p.tool_call->name, tool_args_text(*p.tool_call), -1});
reasoning += p.reasoning;
}
switch (m.role) {
case llm::Role::kSystem: req.add(ChatMessage::system(text)); break;
case llm::Role::kUser:
req.add(has_image ? ChatMessage::user(parts) : ChatMessage::user(text));
break;
case llm::Role::kAssistant:
req.add(calls.empty() ? ChatMessage::assistant(text)
: ChatMessage::assistant_calls(text, calls, reasoning));
break;
default: break;
}
}
for (const llm::ToolSpec& t : r.tools) req.add_tool(FunctionTool(t.name, t.description, t.parameters));
if (auto tc = ToolChoice::from_neutral(r.tool_choice)) req.tool_choice(*tc);
if (r.max_tokens) req.max_completion_tokens(*r.max_tokens);
if (r.temperature) req.temperature(*r.temperature);
if (r.top_p) req.top_p(*r.top_p);
if (!r.stop.empty()) req.stop(r.stop);
if (auto f = format_of(r)) req.response_format(*f);
if (!r.reasoning_effort.empty()) req.reasoning_effort(r.reasoning_effort);
if (stream) req.include_usage(true);
Json body = req.to_json(stream, false);
jx::merge(body, r.extra);
return body;
}
llm::ChatResponse XaiChatModel::generate(const llm::ChatRequest& request) {
if (wire_ == Wire::kResponses) {
Json body = responses_body(request, false);
return from_responses(ModelResponse::parse(
client_->request(clients::Method::kPost, XaiService::plan(Call::kResponsesCreate, {}, &body).path, &body)));
}
Json body = chat_body(request, false);
return from_chat(ChatCompletion::parse(
client_->request(clients::Method::kPost, XaiService::plan(Call::kChatCreate, {}, &body).path, &body)));
}
void XaiChatModel::stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
if (wire_ == Wire::kChatCompletions) {
Json body = chat_body(request, true);
std::string path = XaiService::plan(Call::kChatCreate, {}, &body).path;
ChatStreamAccumulator acc;
std::map<long, std::string> ids;
client_->stream(path, body, [&](const llm::SseEvent& ev) {
CompletionChunk chunk = CompletionChunk::parse(ev.data);
if (!acc.feed(chunk)) return false;
bool cont = true;
for (const ChatDelta& d : chunk.choices) {
llm::ChatChunk out;
out.text_delta = d.content;
out.reasoning_delta = d.reasoning_content;
out.finish_reason = d.finish_reason.empty() ? "" : finish_of(d.finish_reason, d.finish_reason == "tool_calls");
for (const ChatToolCall& t : d.tool_calls) {
llm::ChatChunk tc;
tc.tool_call_id = t.id;
tc.tool_call_name = t.name;
tc.tool_args_delta = t.arguments;
cont = on_chunk(tc) && cont;
}
if (!out.text_delta.empty() || !out.reasoning_delta.empty() || !out.finish_reason.empty())
cont = on_chunk(out) && cont;
}
if (chunk.usage) {
llm::ChatChunk u;
u.usage = chunk.usage->to_neutral();
cont = on_chunk(u) && cont;
}
return cont;
});
return;
}
Json body = responses_body(request, true);
std::string path = XaiService::plan(Call::kResponsesCreate, {}, &body).path;
ResponseAccumulator acc;
std::map<std::string, std::string> call_of_item;
client_->stream(path, body, [&](const llm::SseEvent& ev) {
ResponseEvent e = ResponseEvent::parse(ev);
bool keep = acc.feed(e);
llm::ChatChunk out;
bool emit = false;
switch (e.kind) {
case EventKind::kOutputTextDelta: out.text_delta = e.delta; emit = true; break;
case EventKind::kReasoningTextDelta:
case EventKind::kReasoningSummaryTextDelta: out.reasoning_delta = e.delta; emit = true; break;
case EventKind::kOutputItemAdded:
if (jx::str(e.item, "type") == "function_call") {
call_of_item[e.item_id] = jx::str(e.item, "call_id");
out.tool_call_id = jx::str(e.item, "call_id");
out.tool_call_name = jx::str(e.item, "name");
emit = true;
}
break;
case EventKind::kFunctionCallArgumentsDelta:
out.tool_call_id = call_of_item.count(e.item_id) ? call_of_item[e.item_id] : e.item_id;
out.tool_args_delta = e.delta;
emit = true;
break;
case EventKind::kCompleted:
case EventKind::kIncomplete: {
ModelResponse r = ModelResponse::parse(e.response);
out.finish_reason = finish_of(r.incomplete_reason(), !r.function_calls().empty());
out.usage = r.usage().to_neutral();
emit = true;
break;
}
case EventKind::kFailed:
case EventKind::kError:
throw llm::ApiError(kVendor, 502, "STREAM_FAILED",
acc.error_message().empty() ? "the response stream failed" : acc.error_message());
default: break;
}
bool cont = emit ? on_chunk(out) : true;
return keep && cont;
});
if (acc.stopped()) throw llm::ApiError(kVendor, 502, "STREAM_CAP", acc.stop_reason());
}
} // namespace molo::xai::core
+42
View File
@@ -0,0 +1,42 @@
#pragma once
// XaiChatModel: llm::ChatModel over xAI. Default wire is the Responses API
// (the recommended one, sent with store=false so nothing is retained); the
// chat-completions wire is selectable. The vendor-neutral `extra` member of
// llm::ChatRequest is merged into the body and checked against the call
// table: an undeclared field is rejected, never silently sent.
#include <string>
#include "llm/chat_types.h"
#include "xai/core/chat.h"
#include "xai/core/responses.h"
namespace molo::xai::core {
class XaiChatModel : public llm::ChatModel {
public:
enum class Wire { kResponses, kChatCompletions };
XaiChatModel(std::shared_ptr<llm::ApiClient> client, std::string default_model,
Wire wire = Wire::kResponses);
std::string vendor() const override { return "xai"; }
llm::ChatResponse generate(const llm::ChatRequest& request) override;
void stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
// The wire bodies, exposed so callers and tests can inspect the mapping.
Json responses_body(const llm::ChatRequest& request, bool stream) const;
Json chat_body(const llm::ChatRequest& request, bool stream) const;
Wire wire() const { return wire_; }
private:
std::string model_of(const llm::ChatRequest& request) const;
std::shared_ptr<llm::ApiClient> client_;
std::string default_model_;
Wire wire_;
};
} // namespace molo::xai::core
+19
View File
@@ -0,0 +1,19 @@
#include "xai/core/client.h"
namespace molo::xai::core {
XaiClient::XaiClient(std::shared_ptr<llm::ApiClient> api)
: responses(api), chat(api), legacy(api), messages(api), images(api), videos(api),
embeddings(api), tokenizer(api), models(api), account(api), skills(api), batches(api),
api_(std::move(api)) {}
std::shared_ptr<XaiClient> XaiClient::from_env() {
return std::make_shared<XaiClient>(XaiService::client_from_env());
}
std::shared_ptr<XaiChatModel> XaiClient::chat_model(const std::string& default_model,
XaiChatModel::Wire wire) const {
return std::make_shared<XaiChatModel>(api_, default_model, wire);
}
} // namespace molo::xai::core
+51
View File
@@ -0,0 +1,51 @@
#pragma once
// XaiClient: one shared ApiClient (XAI_API_KEY via llm::Config) and every core
// service built on it.
#include <memory>
#include "xai/core/account.h"
#include "xai/core/batches.h"
#include "xai/core/chat.h"
#include "xai/core/chat_model.h"
#include "xai/core/embeddings.h"
#include "xai/core/images.h"
#include "xai/core/messages.h"
#include "xai/core/models.h"
#include "xai/core/responses.h"
#include "xai/core/videos.h"
namespace molo::xai::core {
class XaiClient {
public:
explicit XaiClient(std::shared_ptr<llm::ApiClient> api);
// Key from XAI_API_KEY (environment, then the env files); a missing key is
// fine: every call then throws NotConfiguredError before any traffic.
static std::shared_ptr<XaiClient> from_env();
bool configured() const { return api_->configured(); }
std::shared_ptr<llm::ApiClient> api() const { return api_; }
ResponsesService responses;
ChatService chat;
LegacyService legacy;
MessagesService messages;
ImageService images;
VideoService videos;
EmbeddingService embeddings;
TokenizeService tokenizer;
ModelsService models;
AccountService account;
SkillsService skills;
BatchService batches;
std::shared_ptr<XaiChatModel> chat_model(const std::string& default_model,
XaiChatModel::Wire wire = XaiChatModel::Wire::kResponses) const;
private:
std::shared_ptr<llm::ApiClient> api_;
};
} // namespace molo::xai::core
+125
View File
@@ -0,0 +1,125 @@
#include "xai/core/embeddings.h"
#include <cstring>
#include "security/crypto/Base64.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
Json EmbeddingRequest::to_json() const {
Check::model(model);
if (!texts.empty() && !token_arrays.empty()) Check::fail("give texts or token_arrays, not both");
Json body = Json::object();
body.set("model", Json(model));
if (!token_arrays.empty()) {
Check::upto("token arrays", token_arrays.size(), Caps::kMaxEmbeddingInputs);
Json outer = Json::array();
for (const auto& arr : token_arrays) {
Check::count("tokens", arr.size(), 1, 1u << 20);
Json inner = Json::array();
for (int t : arr) {
if (t < 0) Check::fail("token ids are non-negative");
inner.push(Json(t));
}
outer.push(inner);
}
body.set("input", token_arrays.size() == 1 ? outer.as_array().front() : outer);
} else {
Check::count("texts", texts.size(), 1, Caps::kMaxEmbeddingInputs);
for (const auto& t : texts) Check::text("text", t, Caps::kMaxTextBytes);
body.set("input", texts.size() == 1 ? Json(texts.front()) : jx::string_array(texts));
}
if (dimensions) {
Check::range("dimensions", *dimensions, 1, Caps::kMaxEmbeddingDimensions);
body.set("dimensions", Json(*dimensions));
}
if (!encoding_format.empty()) {
Check::one_of("encoding_format", encoding_format, {"float", "base64"});
body.set("encoding_format", Json(encoding_format));
}
if (!user.empty()) {
Check::text("user", user, 256);
body.set("user", Json(user));
}
return body;
}
std::vector<float> floats_from_base64(const std::string& text) {
std::string raw;
std::vector<float> out;
if (text.empty() || !security::crypto::Base64::decode(text, raw) || raw.size() % 4 != 0) return out;
out.resize(raw.size() / 4);
std::memcpy(out.data(), raw.data(), raw.size()); // little-endian hosts
return out;
}
EmbeddingResult EmbeddingResult::parse(const Json& doc) {
EmbeddingResult r;
r.raw = doc;
r.model = jx::str(doc, "model");
r.object = jx::str(doc, "object");
const Json* data = jx::get(doc, "data");
if (data && data->is_array())
for (const Json& d : data->as_array()) {
if (r.data.size() >= Caps::kMaxEmbeddingInputs) break;
EmbeddingVector v;
v.index = jx::num(d, "index");
const Json* e = jx::get(d, "embedding");
if (e && e->is_array()) {
for (const Json& x : e->as_array())
if (x.is_number()) v.values.push_back(static_cast<float>(x.as_number()));
} else if (e && e->is_string()) {
v.base64 = e->as_string();
v.values = floats_from_base64(v.base64);
}
r.data.push_back(std::move(v));
}
Json usage = jx::object(doc, "usage");
r.prompt_tokens = jx::num(usage, "prompt_tokens");
r.total_tokens = jx::num(usage, "total_tokens");
return r;
}
EmbeddingResult EmbeddingService::embed(const EmbeddingRequest& request) {
Json body = request.to_json();
return EmbeddingResult::parse(call_json(Call::kEmbeddingsCreate, {}, &body));
}
EmbeddingResult EmbeddingService::embed(const std::string& model, const std::vector<std::string>& texts) {
EmbeddingRequest r;
r.model = model;
r.texts = texts;
return embed(r);
}
std::vector<TokenizeToken> TokenizeService::tokenize(const std::string& model, const std::string& text,
const std::string& user) {
Check::model(model);
Check::text("text", text, Caps::kMaxTokenizeBytes);
Json body = Json::object();
body.set("model", Json(model));
body.set("text", Json(text));
if (!user.empty()) {
Check::text("user", user, 256);
body.set("user", Json(user));
}
Json reply = call_json(Call::kTokenizeText, {}, &body);
std::vector<TokenizeToken> out;
const Json* ids = jx::get(reply, "token_ids");
if (ids && ids->is_array())
for (const Json& t : ids->as_array()) {
if (out.size() >= Caps::kMaxTokenizeBytes) break;
TokenizeToken tok;
tok.string_token = jx::str(t, "string_token");
tok.token_id = jx::num(t, "token_id");
const Json* bytes = jx::get(t, "token_bytes");
if (bytes && bytes->is_array())
for (const Json& b : bytes->as_array())
if (b.is_number()) tok.token_bytes.push_back(static_cast<int>(b.as_int()));
out.push_back(std::move(tok));
}
return out;
}
} // namespace molo::xai::core
+64
View File
@@ -0,0 +1,64 @@
#pragma once
// Embeddings (POST /v1/embeddings) and tokenize-text (POST /v1/tokenize-text).
// The xAI guidance for retrieval: prepend "query: " to query text and
// "passage: " to document text (spec: EmbeddingInput.String).
#include <optional>
#include <string>
#include <vector>
#include "xai/core/service.h"
#include "xai/core/usage.h"
namespace molo::xai::core {
struct EmbeddingRequest {
std::string model;
std::vector<std::string> texts; // strings to embed ...
std::vector<std::vector<int>> token_arrays; // ... or token id arrays (exclusive)
std::optional<int> dimensions;
std::string encoding_format; // "float" (default) | "base64"
std::string user;
Json to_json() const;
};
struct EmbeddingVector {
long index = 0;
std::vector<float> values; // filled for float and for decoded base64 output
std::string base64; // as received when encoding_format is base64
};
struct EmbeddingResult {
std::string model, object;
std::vector<EmbeddingVector> data;
long prompt_tokens = 0, total_tokens = 0;
Json raw;
static EmbeddingResult parse(const Json& doc);
};
// Little-endian float32 array in base64 -> floats; empty on invalid input.
std::vector<float> floats_from_base64(const std::string& text);
struct TokenizeToken {
std::string string_token;
std::vector<int> token_bytes;
long token_id = 0;
};
class EmbeddingService : public XaiService {
public:
explicit EmbeddingService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
EmbeddingResult embed(const EmbeddingRequest& request);
// Convenience: one string per input, default encoding.
EmbeddingResult embed(const std::string& model, const std::vector<std::string>& texts);
};
class TokenizeService : public XaiService {
public:
explicit TokenizeService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
std::vector<TokenizeToken> tokenize(const std::string& model, const std::string& text,
const std::string& user = "");
};
} // namespace molo::xai::core
+174
View File
@@ -0,0 +1,174 @@
#include "xai/core/images.h"
#include "security/crypto/Base64.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
void check_ratio(const std::string& r) {
Check::one_of("aspect_ratio", r,
{"1:1", "3:4", "4:3", "9:16", "16:9", "2:3", "3:2", "9:19.5", "19.5:9", "9:20",
"20:9", "1:2", "2:1", "21:9", "5:2", "auto"});
}
} // namespace
void ImageOptions::write(Json& body, bool deferred) const {
if (!model.empty()) {
Check::model(model);
body.set("model", Json(model));
}
Check::range("n", n, 1, Caps::kMaxImages);
if (n != 1) body.set("n", Json(n));
if (!aspect_ratio.empty()) {
check_ratio(aspect_ratio);
body.set("aspect_ratio", Json(aspect_ratio));
}
if (!resolution.empty()) {
Check::one_of("resolution", resolution, {"1k", "1.5k", "2k"});
body.set("resolution", Json(resolution));
}
if (!response_format.empty()) {
Check::one_of("response_format", response_format, {"url", "b64_json"});
if (deferred && response_format != "url") Check::fail("deferred image requests support response_format url only");
body.set("response_format", Json(response_format));
}
if (!quality.empty()) {
Check::one_of("quality", quality, {"low", "medium", "auto"});
body.set("quality", Json(quality));
}
if (storage) body.set("storage_options", storage->to_json());
if (!upload_urls.empty()) {
if (upload_urls.size() != static_cast<std::size_t>(n))
Check::fail("output.upload_urls must hold exactly n URLs");
for (const auto& u : upload_urls) Check::http_url("upload url", u);
Json out = Json::object();
out.set("upload_urls", jx::string_array(upload_urls));
body.set("output", out);
}
if (!user.empty()) {
Check::text("user", user, 256);
body.set("user", Json(user));
}
if (deferred) body.set("deferred", Json(true));
}
Json ImageGenerationRequest::to_json(bool deferred) const {
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
Json body = Json::object();
body.set("prompt", Json(prompt));
options.write(body, deferred);
return body;
}
Json ImageEditRequest::to_json(bool deferred) const {
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
if (image && !images.empty()) Check::fail("image and images are mutually exclusive");
if (!image && images.empty()) Check::fail("an edit needs image or images");
Json body = Json::object();
body.set("prompt", Json(prompt));
if (image) body.set("image", image->to_json());
if (!images.empty()) {
Check::upto("images", images.size(), Caps::kMaxEditImages);
Json arr = Json::array();
for (const auto& i : images) arr.push(i.to_json());
body.set("images", arr);
}
options.write(body, deferred);
return body;
}
std::string GeneratedImage::bytes() const {
std::string out;
if (b64_json.empty()) return out;
if (!security::crypto::Base64::decode(b64_json, out)) return "";
return out;
}
namespace {
GeneratedImage parse_image(const Json& d) {
GeneratedImage g;
g.url = jx::str(d, "url");
g.b64_json = jx::str(d, "b64_json");
g.mime_type = jx::str(d, "mime_type");
g.storage_error = jx::str(d, "storage_error");
if (const Json* f = jx::get(d, "file_output"); f && f->is_object()) g.file = StoredFile::parse(*f);
return g;
}
} // namespace
ImageResult ImageResult::parse(const Json& doc) {
ImageResult r;
r.raw = doc;
const Json* data = jx::get(doc, "data");
if (data && data->is_array())
for (const Json& d : data->as_array()) {
if (r.data.size() >= static_cast<std::size_t>(Caps::kMaxImages)) break;
r.data.push_back(parse_image(d));
}
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) r.usage = MediaUsage::from_json(*u);
return r;
}
DeferredImage DeferredImage::parse(const Json& doc, const std::string& request_id, int http_status) {
DeferredImage d;
d.request_id = request_id;
if (http_status == 202) {
d.status = "pending";
return d;
}
d.status = jx::str(doc, "status");
if (d.status.empty()) d.status = jx::get(doc, "data") ? "done" : (jx::get(doc, "error") ? "failed" : "pending");
d.error = MediaError::parse(doc, "error");
if (d.status == "done") d.result = ImageResult::parse(doc);
return d;
}
ImageResult ImageService::generate(const ImageGenerationRequest& request) {
Json body = request.to_json(false);
return ImageResult::parse(call_json(Call::kImagesGenerate, {}, &body));
}
ImageResult ImageService::edit(const ImageEditRequest& request) {
Json body = request.to_json(false);
return ImageResult::parse(call_json(Call::kImagesEdit, {}, &body));
}
static std::string request_id_of(const Json& reply) {
std::string id = jx::str(reply, "request_id");
if (id.empty()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "deferred reply has no request_id");
return id;
}
std::string ImageService::start_generate(const ImageGenerationRequest& request) {
Json body = request.to_json(true);
return request_id_of(call_json(Call::kImagesGenerate, {}, &body));
}
std::string ImageService::start_edit(const ImageEditRequest& request) {
Json body = request.to_json(true);
return request_id_of(call_json(Call::kImagesEdit, {}, &body));
}
DeferredImage ImageService::poll(const std::string& request_id) {
llm::RawResponse raw = call_raw(Call::kImagesGet, {request_id});
return DeferredImage::parse(parse_body(raw), request_id, raw.status);
}
ImageResult ImageService::wait(const std::string& request_id, const WaitPolicy& policy,
const Sleeper& sleeper) {
Check::id("request", request_id);
std::function<std::optional<DeferredImage>()> step = [&]() -> std::optional<DeferredImage> {
DeferredImage d = poll(request_id);
if (d.pending()) return std::nullopt;
return d;
};
std::optional<DeferredImage> d = poll_until<DeferredImage>(policy, step, sleeper);
if (!d) throw llm::ApiError(kVendor, 504, "WAIT_TIMEOUT", "image not ready within the wait budget");
if (d->failed())
throw llm::ApiError(kVendor, 502, "GENERATION_FAILED",
d->error ? d->error->code + ": " + d->error->message : "image generation failed");
return *d->result;
}
} // namespace molo::xai::core
+88
View File
@@ -0,0 +1,88 @@
#pragma once
// Image generation and editing (POST /v1/images/generations, /v1/images/edits)
// and the deferred poll (GET /v1/images/{request_id}). Generation costs money
// (see MediaUsage::cost_usd_ticks); nothing here retries.
#include <optional>
#include <string>
#include <vector>
#include "xai/core/media.h"
#include "xai/core/poll.h"
#include "xai/core/service.h"
namespace molo::xai::core {
// Shared by generation and editing.
struct ImageOptions {
std::string model; // optional; the server picks a default
int n = 1; // 1..10
std::string aspect_ratio; // "1:1", "16:9", ..., "auto" (grok-imagine models)
std::string resolution; // "1k" | "1.5k" | "2k" (grok-imagine models)
std::string response_format; // "url" (default) | "b64_json"
std::string quality; // "low" | "medium" | "auto"; documented in the guides, not in the OpenAPI document
std::optional<StorageOptions> storage;
std::vector<std::string> upload_urls; // signed PUT URLs, exactly n of them (output.upload_urls)
std::string user;
void write(Json& body, bool deferred) const;
};
struct ImageGenerationRequest {
std::string prompt;
ImageOptions options;
Json to_json(bool deferred = false) const;
};
struct ImageEditRequest {
std::string prompt;
std::optional<MediaSource> image; // single input
std::vector<MediaSource> images; // multi-reference input; exclusive with `image`
ImageOptions options;
Json to_json(bool deferred = false) const;
};
struct GeneratedImage {
std::string url, b64_json, mime_type, storage_error;
std::optional<StoredFile> file;
// Decoded bytes of b64_json; empty when absent or invalid.
std::string bytes() const;
};
struct ImageResult {
std::vector<GeneratedImage> data;
MediaUsage usage;
Json raw;
static ImageResult parse(const Json& doc);
};
struct DeferredImage {
std::string request_id;
std::string status; // pending | done | failed
std::optional<ImageResult> result;
std::optional<MediaError> error;
bool done() const { return status == "done"; }
bool failed() const { return status == "failed"; }
bool pending() const { return !done() && !failed(); }
static DeferredImage parse(const Json& doc, const std::string& request_id, int http_status);
};
class ImageService : public XaiService {
public:
explicit ImageService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
ImageResult generate(const ImageGenerationRequest& request);
ImageResult edit(const ImageEditRequest& request);
// Deferred: returns the request id; only response_format url is supported.
std::string start_generate(const ImageGenerationRequest& request);
std::string start_edit(const ImageEditRequest& request);
// 202 while pending; 200 carries data (done) or error (failed).
DeferredImage poll(const std::string& request_id);
// Polls within the policy; throws ApiError 504 WAIT_TIMEOUT when the budget is spent
// and 502 GENERATION_FAILED when the job failed.
ImageResult wait(const std::string& request_id, const WaitPolicy& policy = {},
const Sleeper& sleeper = real_sleeper());
};
} // namespace molo::xai::core
+176
View File
@@ -0,0 +1,176 @@
#include "xai/core/input.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
Json part_object(const char* type) {
Json o = Json::object();
o.set("type", Json(type));
return o;
}
Json parts_array(const std::vector<InputPart>& parts) {
Check::count("content parts", parts.size(), 1, Caps::kMaxContentParts);
Json arr = Json::array();
for (const auto& p : parts) arr.push(p.json());
return arr;
}
void check_role(const std::string& role) {
Check::one_of("role", role, {"user", "assistant", "system", "developer"});
}
Json shell_result(std::string out, std::string err, Json outcome) {
Check::text("stdout", out, Caps::kMaxTextBytes, true);
Check::text("stderr", err, Caps::kMaxTextBytes, true);
Json r = Json::object();
r.set("stdout", Json(std::move(out)));
r.set("stderr", Json(std::move(err)));
r.set("outcome", std::move(outcome));
return r;
}
} // namespace
InputPart InputPart::text(std::string text) {
Check::text("text", text, Caps::kMaxTextBytes, true);
Json o = part_object("input_text");
o.set("text", Json(std::move(text)));
return InputPart(std::move(o));
}
InputPart InputPart::assistant_text(std::string text) {
Check::text("text", text, Caps::kMaxTextBytes, true);
Json o = part_object("output_text");
o.set("text", Json(std::move(text)));
return InputPart(std::move(o));
}
InputPart InputPart::image(std::string url) {
Check::url("image_url", url);
Json o = part_object("input_image");
o.set("image_url", Json(std::move(url)));
return InputPart(std::move(o));
}
InputPart InputPart::file_id(std::string id) {
Check::id("file", id);
Json o = part_object("input_file");
o.set("file_id", Json(std::move(id)));
return InputPart(std::move(o));
}
InputPart InputPart::file_url(std::string url) {
Check::http_url("file_url", url);
Json o = part_object("input_file");
o.set("file_url", Json(std::move(url)));
return InputPart(std::move(o));
}
InputPart InputPart::file_data(std::string filename, std::string base64, std::string mime_type) {
Check::text("filename", filename, 255);
Check::text("file_data", base64, Caps::kMaxUrlLen);
Json o = part_object("input_file");
o.set("filename", Json(std::move(filename)));
o.set("file_data", Json(std::move(base64)));
if (!mime_type.empty()) {
Check::token("mime_type", mime_type);
o.set("mime_type", Json(std::move(mime_type)));
}
return InputPart(std::move(o));
}
InputItem InputItem::message(const std::string& role, std::string text) {
check_role(role);
Check::text("message", text, Caps::kMaxTextBytes, true);
Json o = Json::object();
o.set("role", Json(role));
o.set("content", Json(std::move(text)));
return InputItem(std::move(o));
}
InputItem InputItem::message(const std::string& role, const std::vector<InputPart>& parts) {
check_role(role);
Json o = Json::object();
o.set("role", Json(role));
o.set("content", parts_array(parts));
return InputItem(std::move(o));
}
InputItem InputItem::function_output(const std::string& call_id, std::string output) {
Check::id("call", call_id);
Check::text("output", output, Caps::kMaxTextBytes, true);
Json o = part_object("function_call_output");
o.set("call_id", Json(call_id));
o.set("output", Json(std::move(output)));
return InputItem(std::move(o));
}
InputItem InputItem::function_output(const std::string& call_id, const std::vector<InputPart>& parts) {
Check::id("call", call_id);
Json o = part_object("function_call_output");
o.set("call_id", Json(call_id));
o.set("output", parts_array(parts));
return InputItem(std::move(o));
}
InputItem InputItem::shell_output(const std::string& call_id, std::string out, std::string err,
int exit_code) {
Check::id("call", call_id);
Json outcome = part_object("exit");
outcome.set("exit_code", Json(exit_code));
Json results = Json::array();
results.push(shell_result(std::move(out), std::move(err), std::move(outcome)));
Json o = part_object("shell_call_output");
o.set("call_id", Json(call_id));
o.set("output", results);
return InputItem(std::move(o));
}
InputItem InputItem::shell_timeout(const std::string& call_id, std::string out, std::string err) {
Check::id("call", call_id);
Json results = Json::array();
results.push(shell_result(std::move(out), std::move(err), part_object("timeout")));
Json o = part_object("shell_call_output");
o.set("call_id", Json(call_id));
o.set("output", results);
return InputItem(std::move(o));
}
InputItem InputItem::replay(const Json& item) {
Check::object("replayed item", item);
std::string type = jx::str(item, "type");
Check::one_of("replayed item type", type,
{"message", "reasoning", "function_call", "web_search_call", "file_search_call",
"code_interpreter_call", "mcp_call", "custom_tool_call", "shell_call",
"image_generation_call", "tool_search_call", "tool_search_output", "compaction",
"function_call_output", "shell_call_output"});
if (item.dump().size() > Caps::kMaxTextBytes) Check::fail("replayed item is over the size cap");
return InputItem(item);
}
std::string InputItem::type() const {
std::string t = jx::str(json_, "type");
return t.empty() ? "message" : t;
}
Input& Input::add(InputItem item) {
if (has_text_) Check::fail("input is a plain string; cannot add items");
Check::upto("input items", items_.size(), Caps::kMaxInputItems - 1);
items_.push_back(std::move(item));
return *this;
}
Json Input::to_json() const {
if (has_text_) {
Check::text("input", text_, Caps::kMaxTextBytes);
return Json(text_);
}
Check::count("input items", items_.size(), 1, Caps::kMaxInputItems);
Json arr = Json::array();
for (const auto& i : items_) arr.push(i.json());
return arr;
}
} // namespace molo::xai::core
+72
View File
@@ -0,0 +1,72 @@
#pragma once
// Input items of the Responses API (`ModelInputPart`): messages with text,
// image and file parts, function / shell tool outputs, and verbatim replay of
// earlier output items (encrypted reasoning, compaction, tool calls).
#include <string>
#include <vector>
#include "xai/core/params.h"
namespace molo::xai::core {
// One piece of message content: input_text, output_text, input_image, input_file.
class InputPart {
public:
static InputPart text(std::string text);
static InputPart assistant_text(std::string text); // output_text, for replayed assistant turns
// Public https URL or base64 data: URL (JPEG, PNG, WebP; max 20 MiB per the models guide).
static InputPart image(std::string url);
// Exactly one source: a Files API id, a public URL, or inline bytes.
static InputPart file_id(std::string id);
static InputPart file_url(std::string url);
static InputPart file_data(std::string filename, std::string base64, std::string mime_type = "");
const Json& json() const { return json_; }
private:
explicit InputPart(Json j) : json_(std::move(j)) {}
Json json_;
};
class InputItem {
public:
// role: user, assistant, system or developer.
static InputItem message(const std::string& role, std::string text);
static InputItem message(const std::string& role, const std::vector<InputPart>& parts);
static InputItem function_output(const std::string& call_id, std::string output);
static InputItem function_output(const std::string& call_id, const std::vector<InputPart>& parts);
// Result of a shell_call: outcome is an exit code, or timeout.
static InputItem shell_output(const std::string& call_id, std::string stdout_text,
std::string stderr_text, int exit_code);
static InputItem shell_timeout(const std::string& call_id, std::string stdout_text,
std::string stderr_text);
// Replays an item the API returned (unchanged). Only known item types.
static InputItem replay(const Json& item);
const Json& json() const { return json_; }
std::string type() const;
private:
explicit InputItem(Json j) : json_(std::move(j)) {}
Json json_;
};
// The `input` field: a bare string or an array of items.
class Input {
public:
Input() = default;
explicit Input(std::string text) : text_(std::move(text)), has_text_(true) {}
Input& add(InputItem item);
bool empty() const { return !has_text_ && items_.empty(); }
std::size_t size() const { return has_text_ ? 1 : items_.size(); }
Json to_json() const;
private:
std::string text_;
bool has_text_ = false;
std::vector<InputItem> items_;
};
} // namespace molo::xai::core
+89
View File
@@ -0,0 +1,89 @@
#include "xai/core/media.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
Json with(const char* key, const std::string& value) {
Json o = Json::object();
o.set(key, Json(value));
return o;
}
} // namespace
MediaSource MediaSource::image_url(std::string url) {
Check::url("image url", url);
return MediaSource(with("url", url));
}
MediaSource MediaSource::image_file(std::string file_id) {
Check::id("file", file_id);
return MediaSource(with("file_id", file_id));
}
MediaSource MediaSource::video_url(std::string url) {
Check::url("video url", url);
if (url.rfind("data:", 0) != 0) {
std::string path = url.substr(0, url.find_first_of("?#"));
if (path.size() < 4 || path.compare(path.size() - 4, 4, ".mp4") != 0)
Check::fail("video url must end in .mp4");
}
return MediaSource(with("url", url));
}
MediaSource MediaSource::video_file(std::string file_id) {
Check::id("file", file_id);
return MediaSource(with("file_id", file_id));
}
MediaSource MediaSource::audio_url(std::string url) {
Check::url("audio url", url);
return MediaSource(with("url", url));
}
MediaSource MediaSource::audio_voice(std::string voice_id) {
Check::id("voice", voice_id);
return MediaSource(with("voice_id", voice_id));
}
Json StorageOptions::to_json() const {
Check::text("filename", filename, 255);
if (filename.find('/') != std::string::npos || filename.find('\\') != std::string::npos)
Check::fail("filename must not contain a path");
Json o = Json::object();
o.set("filename", Json(filename));
if (expires_after) {
Check::range("expires_after", static_cast<double>(*expires_after), 1, Caps::kMaxStorageExpirySeconds);
o.set("expires_after", Json(*expires_after));
}
if (public_url_expires_after) {
Check::range("public url expires_after", static_cast<double>(*public_url_expires_after),
static_cast<double>(Caps::kMinPublicUrlSeconds), Caps::kMaxStorageExpirySeconds);
Json p = Json::object();
p.set("expires_after", Json(*public_url_expires_after));
o.set("public_url", p);
} else if (public_url) {
o.set("public_url", Json(true));
}
return o;
}
StoredFile StoredFile::parse(const Json& doc) {
StoredFile f;
f.file_id = jx::str(doc, "file_id");
f.filename = jx::str(doc, "filename");
f.public_url = jx::str(doc, "public_url");
f.public_url_error = jx::str(doc, "public_url_error");
if (const Json* v = jx::get(doc, "expires_at"); v && v->is_number()) f.expires_at = v->as_int();
if (const Json* v = jx::get(doc, "public_url_expires_at"); v && v->is_number()) f.public_url_expires_at = v->as_int();
return f;
}
std::optional<MediaError> MediaError::parse(const Json& doc, const char* key) {
const Json* e = jx::get(doc, key);
if (!e || !e->is_object()) return std::nullopt;
return MediaError{jx::str(*e, "code"), jx::str(*e, "message")};
}
} // namespace molo::xai::core
+65
View File
@@ -0,0 +1,65 @@
#pragma once
// Pieces shared by image and video generation: media sources (URL, data URL
// or Files API id), file storage options, uploaded output, stored-file
// references and generation errors.
#include <optional>
#include <string>
#include <vector>
#include "xai/core/params.h"
#include "xai/core/usage.h"
namespace molo::xai::core {
// `ImageUrl` / `VideoUrl`: exactly one of a URL (public or base64 data URL)
// or a Files API id.
class MediaSource {
public:
MediaSource() : json_(Json::object()) {} // empty: rejected wherever a source is required
bool empty() const { return json_.as_object().empty(); }
static MediaSource image_url(std::string url);
static MediaSource image_file(std::string file_id);
// Video sources must be .mp4 (spec), 2..15 s for extensions (docs).
static MediaSource video_url(std::string url);
static MediaSource video_file(std::string file_id);
static MediaSource audio_url(std::string url); // reference audio: data: URL or http(s)
static MediaSource audio_voice(std::string voice_id); // preset voice id (same as TTS voices)
Json to_json() const { return json_; }
private:
explicit MediaSource(Json j) : json_(std::move(j)) {}
Json json_;
};
// `storage_options`: keep the generated media as a Files API file.
struct StorageOptions {
std::string filename; // required
std::optional<long> expires_after; // seconds, <= 2592000
bool public_url = false; // create a public URL
std::optional<long> public_url_expires_after; // seconds, 3600..2592000
Json to_json() const;
};
// `output`: have the API upload the media to signed URLs instead of
// hosting it (one URL per image, a single one for a video).
struct UploadOutput {
std::vector<std::string> urls;
};
// `FileOutput` in a result: the stored file.
struct StoredFile {
std::string file_id, filename, public_url, public_url_error;
std::optional<long> expires_at, public_url_expires_at;
static StoredFile parse(const Json& doc);
};
struct MediaError {
std::string code; // invalid_argument | permission_denied | failed_precondition | service_unavailable | internal_error
std::string message;
static std::optional<MediaError> parse(const Json& doc, const char* key);
};
} // namespace molo::xai::core
+393
View File
@@ -0,0 +1,393 @@
#include "xai/core/messages.h"
#include "llm/caps.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
Json block(const char* type) {
Json o = Json::object();
o.set("type", Json(type));
return o;
}
} // namespace
MessageBlock MessageBlock::text(std::string text) {
Check::text("text", text, Caps::kMaxTextBytes, true);
Json o = block("text");
o.set("text", Json(std::move(text)));
return MessageBlock(std::move(o));
}
MessageBlock MessageBlock::image_base64(std::string media_type, std::string data) {
Check::one_of("media_type", media_type, {"image/jpeg", "image/png", "image/webp"});
Check::text("image data", data, Caps::kMaxUrlLen);
Json src = block("base64");
src.set("media_type", Json(std::move(media_type)));
src.set("data", Json(std::move(data)));
Json o = block("image");
o.set("source", src);
return MessageBlock(std::move(o));
}
MessageBlock MessageBlock::image_url(std::string url) {
Check::http_url("image url", url);
Json src = block("url");
src.set("url", Json(std::move(url)));
Json o = block("image");
o.set("source", src);
return MessageBlock(std::move(o));
}
MessageBlock MessageBlock::tool_use(std::string id, std::string name, Json input) {
Check::id("tool use", id);
Check::id("tool", name);
Check::object("tool input", input);
Json o = block("tool_use");
o.set("id", Json(std::move(id)));
o.set("name", Json(std::move(name)));
o.set("input", std::move(input));
return MessageBlock(std::move(o));
}
MessageBlock MessageBlock::tool_result(std::string tool_use_id, std::string content, bool is_error) {
Check::id("tool use", tool_use_id);
Check::text("tool result", content, Caps::kMaxTextBytes, true);
Json o = block("tool_result");
o.set("tool_use_id", Json(std::move(tool_use_id)));
o.set("content", Json(std::move(content)));
if (is_error) o.set("is_error", Json(true));
return MessageBlock(std::move(o));
}
MessageBlock MessageBlock::thinking(std::string thinking, std::string signature) {
Check::text("thinking", thinking, Caps::kMaxTextBytes, true);
Json o = block("thinking");
o.set("thinking", Json(std::move(thinking)));
if (!signature.empty()) o.set("signature", Json(std::move(signature)));
return MessageBlock(std::move(o));
}
MessageBlock MessageBlock::redacted_thinking(std::string data) {
Check::text("redacted data", data, Caps::kMaxTextBytes);
Json o = block("redacted_thinking");
o.set("data", Json(std::move(data)));
return MessageBlock(std::move(o));
}
MessagesRequest& MessagesRequest::user(std::string text) {
Check::text("message", text, Caps::kMaxTextBytes);
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
Json m = Json::object();
m.set("role", Json("user"));
m.set("content", Json(std::move(text)));
messages_.push_back(std::move(m));
return *this;
}
MessagesRequest& MessagesRequest::assistant(std::string text) {
Check::text("message", text, Caps::kMaxTextBytes);
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
Json m = Json::object();
m.set("role", Json("assistant"));
m.set("content", Json(std::move(text)));
messages_.push_back(std::move(m));
return *this;
}
MessagesRequest& MessagesRequest::add(const std::string& role, const std::vector<MessageBlock>& blocks) {
Check::one_of("role", role, {"user", "assistant"});
Check::count("content blocks", blocks.size(), 1, Caps::kMaxContentParts);
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
Json arr = Json::array();
for (const auto& b : blocks) arr.push(b.json());
Json m = Json::object();
m.set("role", Json(role));
m.set("content", arr);
messages_.push_back(std::move(m));
return *this;
}
MessagesRequest& MessagesRequest::add_tool(std::string name, std::string description, Json input_schema) {
Check::id("tool", name);
Check::text("tool description", description, 16u * 1024u);
Check::json_schema("input_schema", input_schema);
Check::upto("tools", tools_.size(), Caps::kMaxResponsesTools - 1);
Json t = Json::object();
t.set("name", Json(std::move(name)));
t.set("description", Json(std::move(description)));
t.set("input_schema", std::move(input_schema));
tools_.push_back(std::move(t));
return *this;
}
Json MessagesRequest::to_json(bool stream) const {
Check::model(model_);
Check::range("max_tokens", max_tokens_, 1, 10'000'000);
Check::count("messages", messages_.size(), 1, llm::Caps::kMaxMessages);
Json body = Json::object();
body.set("model", Json(model_));
body.set("max_tokens", Json(max_tokens_));
Json msgs = Json::array();
for (const Json& m : messages_) msgs.push(m);
body.set("messages", msgs);
if (!system_.empty()) {
Check::text("system", system_, Caps::kMaxTextBytes);
body.set("system", Json(system_));
}
if (stream) body.set("stream", Json(true));
if (temperature_) { Check::range("temperature", *temperature_, 0, 2); body.set("temperature", Json(*temperature_)); }
if (top_p_) { Check::range("top_p", *top_p_, 0, 1); body.set("top_p", Json(*top_p_)); }
Check::strings("stop_sequences", stop_, Caps::kMaxStop, 256);
jx::put_strings(body, "stop_sequences", stop_);
if (user_id_) {
Check::text("user_id", *user_id_, 256);
Json meta = Json::object();
meta.set("user_id", Json(*user_id_));
body.set("metadata", meta);
}
if (!tools_.empty()) {
Json arr = Json::array();
for (const Json& t : tools_) arr.push(t);
body.set("tools", arr);
}
if (!choice_.empty()) {
Json c = Json::object();
c.set("type", Json(choice_));
if (choice_ == "tool") {
Check::id("tool", choice_name_);
c.set("name", Json(choice_name_));
}
body.set("tool_choice", c);
}
return body;
}
MessagesResponse MessagesResponse::parse(const Json& doc) {
MessagesResponse r;
r.raw = doc;
r.id = jx::str(doc, "id");
r.model = jx::str(doc, "model");
r.role = jx::str(doc, "role");
r.type = jx::str(doc, "type");
r.stop_reason = jx::str(doc, "stop_reason");
r.stop_sequence = jx::str(doc, "stop_sequence");
const Json* content = jx::get(doc, "content");
if (content && content->is_array())
for (const Json& c : content->as_array()) {
if (r.content.size() >= Caps::kMaxAccumulatedItems) break;
MessageContent m;
m.type = jx::str(c, "type");
m.text = m.type == "thinking" ? jx::str(c, "thinking") : jx::str(c, "text");
m.signature = jx::str(c, "signature");
m.data = jx::str(c, "data");
m.id = jx::str(c, "id");
m.name = jx::str(c, "name");
if (const Json* in = jx::get(c, "input"); in && in->is_object()) m.input = *in;
r.content.push_back(std::move(m));
}
Json u = jx::object(doc, "usage");
r.input_tokens = jx::num(u, "input_tokens");
r.output_tokens = jx::num(u, "output_tokens");
r.cache_read_input_tokens = jx::num(u, "cache_read_input_tokens");
r.cache_creation_input_tokens = jx::num(u, "cache_creation_input_tokens");
return r;
}
std::string MessagesResponse::text() const {
std::string out;
for (const auto& c : content)
if (c.type == "text") out += c.text;
return out;
}
std::string MessagesResponse::thinking() const {
std::string out;
for (const auto& c : content)
if (c.type == "thinking") out += c.text;
return out;
}
std::vector<MessageContent> MessagesResponse::tool_uses() const {
std::vector<MessageContent> out;
for (const auto& c : content)
if (c.type == "tool_use") out.push_back(c);
return out;
}
bool MessagesStreamAccumulator::feed(const std::string& event_name, const Json& data) {
if (stopped_ || done_) return false;
if (++events_ > static_cast<long>(Caps::kMaxStreamEvents)) {
stopped_ = true;
return false;
}
std::string type = jx::str(data, "type");
if (type.empty()) type = event_name;
if (type == "message_start") {
Json msg = jx::object(data, "message");
MessagesResponse head = MessagesResponse::parse(msg);
response_.id = head.id;
response_.model = head.model;
response_.role = head.role;
response_.type = head.type;
response_.input_tokens = head.input_tokens;
response_.output_tokens = head.output_tokens;
response_.cache_read_input_tokens = head.cache_read_input_tokens;
} else if (type == "content_block_start") {
long index = jx::num(data, "index");
if (index < 0 || index >= static_cast<long>(Caps::kMaxAccumulatedItems)) { stopped_ = true; return false; }
if (response_.content.size() <= static_cast<std::size_t>(index)) {
response_.content.resize(index + 1);
input_json_.resize(index + 1);
}
Json cb = jx::object(data, "content_block");
MessageContent& m = response_.content[index];
m.type = jx::str(cb, "type");
m.text = jx::str(cb, m.type == "thinking" ? "thinking" : "text");
m.id = jx::str(cb, "id");
m.name = jx::str(cb, "name");
m.data = jx::str(cb, "data");
} else if (type == "content_block_delta") {
long index = jx::num(data, "index");
if (index < 0 || index >= static_cast<long>(response_.content.size())) return true;
Json d = jx::object(data, "delta");
MessageContent& m = response_.content[index];
std::string dt = jx::str(d, "type");
std::size_t add = 0;
if (dt == "text_delta") { m.text += jx::str(d, "text"); add = jx::str(d, "text").size(); }
else if (dt == "thinking_delta") { m.text += jx::str(d, "thinking"); add = jx::str(d, "thinking").size(); }
else if (dt == "signature_delta") m.signature += jx::str(d, "signature");
else if (dt == "input_json_delta") { input_json_[index] += jx::str(d, "partial_json"); add = jx::str(d, "partial_json").size(); }
if (add > Caps::kMaxStreamTextBytes || m.text.size() > Caps::kMaxStreamTextBytes) { stopped_ = true; return false; }
} else if (type == "content_block_stop") {
long index = jx::num(data, "index");
if (index >= 0 && index < static_cast<long>(input_json_.size()) && !input_json_[index].empty()) {
try {
Json parsed = Json::parse(input_json_[index]);
if (parsed.is_object()) response_.content[index].input = parsed;
} catch (const std::exception&) {
// Invalid tool input JSON stays empty; the caller sees an empty input object.
}
}
} else if (type == "message_delta") {
Json d = jx::object(data, "delta");
if (jx::has_string(d, "stop_reason")) response_.stop_reason = jx::str(d, "stop_reason");
if (jx::has_string(d, "stop_sequence")) response_.stop_sequence = jx::str(d, "stop_sequence");
Json u = jx::object(data, "usage");
if (jx::get(u, "output_tokens")) response_.output_tokens = jx::num(u, "output_tokens");
if (jx::get(u, "input_tokens")) response_.input_tokens = jx::num(u, "input_tokens");
} else if (type == "message_stop") {
done_ = true;
} else if (type == "error") {
failed_ = true;
error_ = jx::str(jx::object(data, "error"), "message");
if (error_.empty()) error_ = jx::str(data, "message");
done_ = true;
}
return !done_;
}
MessagesResponse MessagesStreamAccumulator::result() const { return response_; }
MessagesResponse MessagesService::create(const MessagesRequest& request) {
Json body = request.to_json(false);
return MessagesResponse::parse(call_json(Call::kMessagesCreate, {}, &body));
}
MessagesResponse MessagesService::stream(const MessagesRequest& request, const EventSink& sink) {
Json body = request.to_json(true);
MessagesStreamAccumulator acc;
call_stream(Call::kMessagesCreate, body, [&](const llm::SseEvent& ev) {
bool keep = acc.feed(ev.event, ev.data);
bool user_keep = sink ? sink(ev.event, ev.data) : true;
return keep && user_keep;
});
if (acc.failed()) throw llm::ApiError(kVendor, 502, "STREAM_FAILED", acc.error_message());
if (acc.stopped()) throw llm::ApiError(kVendor, 502, "STREAM_CAP", "messages stream hit a cap");
return acc.result();
}
Json TextCompletionRequest::to_json() const {
Check::model(model);
Check::count("prompt", prompt.size(), 1, 64);
for (const auto& p : prompt) Check::text("prompt", p, Caps::kMaxTextBytes, true);
Json body = Json::object();
body.set("model", Json(model));
if (prompt.size() == 1) body.set("prompt", Json(prompt.front()));
else body.set("prompt", jx::string_array(prompt));
if (max_tokens) { Check::range("max_tokens", *max_tokens, 1, 10'000'000); body.set("max_tokens", Json(*max_tokens)); }
if (n) { Check::range("n", *n, 1, static_cast<double>(Caps::kMaxChoices)); body.set("n", Json(*n)); }
if (temperature) { Check::range("temperature", *temperature, 0, 2); body.set("temperature", Json(*temperature)); }
if (top_p) { Check::range("top_p", *top_p, 0, 1); body.set("top_p", Json(*top_p)); }
if (presence_penalty) { Check::range("presence_penalty", *presence_penalty, -2, 2); body.set("presence_penalty", Json(*presence_penalty)); }
jx::put(body, "seed", seed);
Check::strings("stop", stop, Caps::kMaxStop, 256);
jx::put_strings(body, "stop", stop);
jx::put(body, "echo", echo);
jx::put(body, "logprobs", logprobs);
if (!user.empty()) { Check::text("user", user, 256); body.set("user", Json(user)); }
return body;
}
TextCompletion TextCompletion::parse(const Json& doc) {
TextCompletion t;
t.raw = doc;
t.id = jx::str(doc, "id");
t.model = jx::str(doc, "model");
t.object = jx::str(doc, "object");
t.system_fingerprint = jx::str(doc, "system_fingerprint");
t.created = jx::num(doc, "created");
const Json* choices = jx::get(doc, "choices");
if (choices && choices->is_array())
for (const Json& c : choices->as_array()) {
if (t.choices.size() >= Caps::kMaxChoices) break;
t.choices.push_back({jx::num(c, "index"), jx::str(c, "text"), jx::str(c, "finish_reason")});
}
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) t.usage = TokenUsage::from_chat(*u);
return t;
}
Json AnthropicCompleteRequest::to_json() const {
Check::model(model);
Check::text("prompt", prompt, Caps::kMaxTextBytes);
Check::range("max_tokens_to_sample", max_tokens_to_sample, 1, 10'000'000);
Json body = Json::object();
body.set("model", Json(model));
body.set("prompt", Json(prompt));
body.set("max_tokens_to_sample", Json(max_tokens_to_sample));
if (temperature) { Check::range("temperature", *temperature, 0, 2); body.set("temperature", Json(*temperature)); }
if (top_p) { Check::range("top_p", *top_p, 0, 1); body.set("top_p", Json(*top_p)); }
Check::strings("stop_sequences", stop_sequences, Caps::kMaxStop, 256);
jx::put_strings(body, "stop_sequences", stop_sequences);
if (!user_id.empty()) {
Check::text("user_id", user_id, 256);
Json meta = Json::object();
meta.set("user_id", Json(user_id));
body.set("metadata", meta);
}
return body;
}
AnthropicComplete AnthropicComplete::parse(const Json& doc) {
AnthropicComplete c;
c.raw = doc;
c.id = jx::str(doc, "id");
c.model = jx::str(doc, "model");
c.type = jx::str(doc, "type");
c.completion = jx::str(doc, "completion");
c.stop_reason = jx::str(doc, "stop_reason");
return c;
}
TextCompletion LegacyService::completions(const TextCompletionRequest& request) {
Json body = request.to_json();
return TextCompletion::parse(call_json(Call::kCompletionsLegacy, {}, &body));
}
AnthropicComplete LegacyService::complete(const AnthropicCompleteRequest& request) {
Json body = request.to_json();
return AnthropicComplete::parse(call_json(Call::kCompleteLegacy, {}, &body));
}
} // namespace molo::xai::core
+167
View File
@@ -0,0 +1,167 @@
#pragma once
// Anthropic-compatible Messages endpoint (POST /v1/messages) and the legacy
// text completion endpoints (/v1/complete Anthropic style, /v1/completions
// OpenAI style). xAI marks the Anthropic SDK compatibility "fully deprecated"
// in favour of the Responses API; it is still in the OpenAPI document, so it
// is covered here. The stream event vocabulary is Anthropic's and is listed
// under Unverified in the SPEC.
#include <functional>
#include <optional>
#include <string>
#include <vector>
#include "xai/core/service.h"
#include "xai/core/usage.h"
namespace molo::xai::core {
class MessageBlock {
public:
static MessageBlock text(std::string text);
static MessageBlock image_base64(std::string media_type, std::string data); // jpeg, png, webp
static MessageBlock image_url(std::string url);
static MessageBlock tool_use(std::string id, std::string name, Json input);
static MessageBlock tool_result(std::string tool_use_id, std::string content, bool is_error = false);
static MessageBlock thinking(std::string thinking, std::string signature = "");
static MessageBlock redacted_thinking(std::string data);
const Json& json() const { return json_; }
private:
explicit MessageBlock(Json j) : json_(std::move(j)) {}
Json json_;
};
class MessagesRequest {
public:
// max_tokens is required by the endpoint.
MessagesRequest(std::string model, int max_tokens) : model_(std::move(model)), max_tokens_(max_tokens) {}
MessagesRequest& system(std::string text) { system_ = std::move(text); return *this; }
MessagesRequest& user(std::string text);
MessagesRequest& assistant(std::string text);
MessagesRequest& add(const std::string& role, const std::vector<MessageBlock>& blocks);
MessagesRequest& temperature(double v) { temperature_ = v; return *this; }
MessagesRequest& top_p(double v) { top_p_ = v; return *this; }
MessagesRequest& stop_sequences(std::vector<std::string> s) { stop_ = std::move(s); return *this; }
MessagesRequest& user_id(std::string id) { user_id_ = std::move(id); return *this; }
MessagesRequest& add_tool(std::string name, std::string description, Json input_schema);
// auto | any | tool (named).
MessagesRequest& tool_choice_auto() { choice_ = "auto"; return *this; }
MessagesRequest& tool_choice_any() { choice_ = "any"; return *this; }
MessagesRequest& tool_choice_tool(std::string name) { choice_ = "tool"; choice_name_ = std::move(name); return *this; }
Json to_json(bool stream = false) const;
private:
std::string model_;
int max_tokens_;
std::string system_, choice_, choice_name_;
std::vector<Json> messages_;
std::vector<Json> tools_;
std::optional<double> temperature_, top_p_;
std::vector<std::string> stop_;
std::optional<std::string> user_id_;
};
struct MessageContent {
std::string type; // text | thinking | redacted_thinking | tool_use
std::string text; // text, or thinking text
std::string signature; // thinking
std::string data; // redacted_thinking
std::string id, name; // tool_use
Json input; // tool_use
};
struct MessagesResponse {
std::string id, model, role, type, stop_reason, stop_sequence;
std::vector<MessageContent> content;
long input_tokens = 0, output_tokens = 0, cache_read_input_tokens = 0, cache_creation_input_tokens = 0;
Json raw;
static MessagesResponse parse(const Json& doc);
std::string text() const;
std::string thinking() const;
std::vector<MessageContent> tool_uses() const;
};
// Folds Anthropic-style stream events (message_start, content_block_start,
// content_block_delta with text_delta / thinking_delta / signature_delta /
// input_json_delta, content_block_stop, message_delta, message_stop, ping,
// error) into a MessagesResponse.
class MessagesStreamAccumulator {
public:
bool feed(const std::string& event_name, const Json& data); // false when a cap stopped it
MessagesResponse result() const;
bool stopped() const { return stopped_; }
bool failed() const { return failed_; }
const std::string& error_message() const { return error_; }
bool done() const { return done_; }
private:
MessagesResponse response_;
std::vector<std::string> input_json_; // per block index
std::string error_;
long events_ = 0;
bool stopped_ = false, failed_ = false, done_ = false;
};
class MessagesService : public XaiService {
public:
// false stops; delta text is "" for non-text events.
using EventSink = std::function<bool(const std::string& event, const Json& data)>;
explicit MessagesService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
MessagesResponse create(const MessagesRequest& request);
MessagesResponse stream(const MessagesRequest& request, const EventSink& sink = {});
};
// ---- Legacy text completions --------------------------------------------------
struct TextCompletionRequest { // POST /v1/completions
std::string model;
std::vector<std::string> prompt; // one string, or several
std::optional<int> max_tokens, n;
std::optional<double> temperature, top_p, presence_penalty;
std::optional<long> seed;
std::vector<std::string> stop;
std::optional<bool> echo, logprobs;
std::string user;
Json to_json() const;
};
struct TextCompletion {
std::string id, model, object, system_fingerprint;
long created = 0;
struct Choice { long index = 0; std::string text, finish_reason; };
std::vector<Choice> choices;
TokenUsage usage;
Json raw;
static TextCompletion parse(const Json& doc);
std::string text() const { return choices.empty() ? "" : choices.front().text; }
};
struct AnthropicCompleteRequest { // POST /v1/complete
std::string model, prompt;
int max_tokens_to_sample = 0;
std::optional<double> temperature, top_p;
std::vector<std::string> stop_sequences;
std::string user_id;
Json to_json() const;
};
struct AnthropicComplete {
std::string id, model, type, completion, stop_reason;
Json raw;
static AnthropicComplete parse(const Json& doc);
};
class LegacyService : public XaiService {
public:
explicit LegacyService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
TextCompletion completions(const TextCompletionRequest& request);
AnthropicComplete complete(const AnthropicCompleteRequest& request);
};
} // namespace molo::xai::core
+94
View File
@@ -0,0 +1,94 @@
#include "xai/core/models.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
const char* const kPriceKeys[] = {
"prompt_text_token_price", "cached_prompt_text_token_price", "completion_text_token_price",
"prompt_text_token_price_long_context", "cached_prompt_text_token_price_long_context",
"completion_text_token_price_long_context", "prompt_image_token_price", "search_price",
"image_price"};
Call list_call(ModelKind k) {
switch (k) {
case ModelKind::kLanguage: return Call::kLanguageModelsList;
case ModelKind::kImage: return Call::kImageModelsList;
case ModelKind::kVideo: return Call::kVideoModelsList;
case ModelKind::kEmbedding: return Call::kEmbeddingModelsList;
default: return Call::kModelsList;
}
}
Call get_call(ModelKind k) {
switch (k) {
case ModelKind::kLanguage: return Call::kLanguageModelsGet;
case ModelKind::kImage: return Call::kImageModelsGet;
case ModelKind::kVideo: return Call::kVideoModelsGet;
case ModelKind::kEmbedding: return Call::kEmbeddingModelsGet;
default: return Call::kModelsGet;
}
}
} // namespace
ModelInfo ModelInfo::parse(ModelKind kind, const Json& d) {
ModelInfo m;
m.kind = kind;
m.raw = d;
m.id = jx::str(d, "id");
m.object = jx::str(d, "object");
m.owned_by = jx::str(d, "owned_by");
m.version = jx::str(d, "version");
m.fingerprint = jx::str(d, "fingerprint");
m.created = jx::num(d, "created");
m.aliases = jx::strings(d, "aliases");
m.input_modalities = jx::strings(d, "input_modalities");
m.output_modalities = jx::strings(d, "output_modalities");
auto opt = [&](const char* k) -> std::optional<long> {
const Json* v = jx::get(d, k);
if (v && v->is_number()) return v->as_int();
return std::nullopt;
};
m.context_length = opt("context_length");
m.long_context_threshold = opt("long_context_threshold");
m.max_prompt_length = opt("max_prompt_length");
for (const char* k : kPriceKeys)
if (auto v = opt(k)) m.prices[k] = *v;
const Json* pricing = jx::get(d, "pricing");
if (pricing && pricing->is_array())
for (const Json& t : pricing->as_array()) {
if (m.image_pricing.size() >= 64) break;
m.image_pricing.push_back({jx::str(t, "quality"), jx::str(t, "resolution"), jx::num(t, "price_per_image")});
}
Json caps = jx::object(d, "capabilities");
m.reasoning_efforts = jx::strings(caps, "reasoning_effort");
m.default_reasoning_effort = jx::str(caps, "default_reasoning_effort");
return m;
}
bool ModelInfo::accepts_effort(const std::string& effort) const {
for (const auto& e : reasoning_efforts)
if (e == effort) return true;
return false;
}
ModelList ModelsService::list(ModelKind kind) {
Json doc = call_json(list_call(kind));
ModelList out;
const Json* arr = jx::get(doc, kind == ModelKind::kGeneric ? "data" : "models");
if (arr && arr->is_array())
for (const Json& e : arr->as_array()) {
if (out.models.size() >= Caps::kMaxAccumulatedItems) {
out.truncated = true;
break;
}
out.models.push_back(ModelInfo::parse(kind, e));
}
return out;
}
ModelInfo ModelsService::get(ModelKind kind, const std::string& model_id) {
return ModelInfo::parse(kind, call_json(get_call(kind), {model_id}));
}
} // namespace molo::xai::core
+58
View File
@@ -0,0 +1,58 @@
#pragma once
// Models API: /v1/models (generic, OpenAI-compatible), /v1/language-models,
// /v1/image-generation-models, /v1/video-generation-models,
// /v1/embedding-models. One ModelInfo covers all five shapes; fields a shape
// does not carry stay at their defaults. Prices are as xAI reports them
// (USD cents per 100 million tokens unless stated; embedding prices per
// million tokens; image pricing tiers in 1/100,000,000 USD cent).
#include <map>
#include <string>
#include <vector>
#include "xai/core/service.h"
namespace molo::xai::core {
enum class ModelKind { kGeneric, kLanguage, kImage, kVideo, kEmbedding };
struct ImagePriceTier {
std::string quality, resolution;
long price_per_image = 0;
};
struct ModelInfo {
ModelKind kind = ModelKind::kGeneric;
std::string id, object, owned_by, version, fingerprint;
long created = 0;
std::vector<std::string> aliases, input_modalities, output_modalities;
std::optional<long> context_length; // generic /v1/models only
std::optional<long> long_context_threshold;
// Named prices: prompt_text_token_price, cached_prompt_text_token_price,
// completion_text_token_price, *_long_context, prompt_image_token_price,
// search_price, image_price.
std::map<std::string, long> prices;
std::vector<ImagePriceTier> image_pricing;
std::optional<long> max_prompt_length; // image models
std::vector<std::string> reasoning_efforts; // capabilities.reasoning_effort
std::string default_reasoning_effort;
Json raw;
static ModelInfo parse(ModelKind kind, const Json& doc);
bool accepts_effort(const std::string& effort) const;
};
struct ModelList {
std::vector<ModelInfo> models;
bool truncated = false; // more entries than Caps::kMaxAccumulatedItems
};
class ModelsService : public XaiService {
public:
explicit ModelsService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
ModelList list(ModelKind kind = ModelKind::kGeneric);
ModelInfo get(ModelKind kind, const std::string& model_id);
};
} // namespace molo::xai::core
+197
View File
@@ -0,0 +1,197 @@
#include "xai/core/params.h"
#include <cmath>
#include "llm/api_client.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
void Check::fail(const std::string& message) {
throw llm::ApiError::invalid(kVendor, message);
}
void Check::model(const std::string& model) {
if (model.empty() || model.size() > Caps::kMaxModelLen) fail("model must be 1.." + std::to_string(Caps::kMaxModelLen) + " characters");
for (unsigned char c : model)
if (c <= 0x20 || c >= 0x7f) fail("model has an invalid character");
}
void Check::id(const std::string& what, const std::string& id) {
if (!llm::Service::valid_id(id)) fail("bad " + what + " id");
}
void Check::non_empty(const char* name, const std::string& value) {
if (value.empty()) fail(std::string(name) + " is required");
}
void Check::text(const char* name, const std::string& value, std::size_t max_bytes,
bool allow_empty) {
if (value.empty() && !allow_empty) fail(std::string(name) + " is required");
if (value.size() > max_bytes)
fail(std::string(name) + " is over " + std::to_string(max_bytes) + " bytes");
}
void Check::token(const char* name, const std::string& value) {
if (value.empty() || value.size() > Caps::kMaxTokenLen)
fail(std::string(name) + " must be 1.." + std::to_string(Caps::kMaxTokenLen) + " characters");
for (char c : value) {
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '.' || c == '_' || c == ':' || c == '/' || c == '+' || c == '-';
if (!ok) fail(std::string(name) + " has an invalid character");
}
}
void Check::one_of(const char* name, const std::string& value,
std::initializer_list<const char*> allowed) {
for (const char* a : allowed)
if (value == a) return;
std::string list;
for (const char* a : allowed) list += (list.empty() ? "" : ", ") + std::string(a);
fail(std::string(name) + " must be one of " + list);
}
void Check::range(const char* name, double value, double lo, double hi) {
if (std::isnan(value) || value < lo || value > hi)
fail(std::string(name) + " must be between " + std::to_string(lo) + " and " + std::to_string(hi));
}
void Check::count(const char* name, std::size_t n, std::size_t lo, std::size_t hi) {
if (n < lo || n > hi)
fail(std::string(name) + " must hold " + std::to_string(lo) + ".." + std::to_string(hi) + " items");
}
void Check::upto(const char* name, std::size_t n, std::size_t hi) {
if (n > hi) fail(std::string(name) + " holds more than " + std::to_string(hi) + " items");
}
void Check::url(const char* name, const std::string& value) {
non_empty(name, value);
if (value.size() > Caps::kMaxUrlLen) fail(std::string(name) + " is over the size cap");
bool ok = value.rfind("https://", 0) == 0 || value.rfind("http://", 0) == 0 ||
value.rfind("data:", 0) == 0;
if (!ok) fail(std::string(name) + " must be an http(s) or data: URL");
}
void Check::http_url(const char* name, const std::string& value) {
non_empty(name, value);
if (value.size() > 4096) fail(std::string(name) + " is over 4096 characters");
if (value.rfind("https://", 0) != 0 && value.rfind("http://", 0) != 0)
fail(std::string(name) + " must be an http(s) URL");
for (unsigned char c : value)
if (c <= 0x20 || c >= 0x7f) fail(std::string(name) + " has an invalid character");
}
void Check::date(const char* name, const std::string& value) {
bool ok = value.size() == 10 && value[4] == '-' && value[7] == '-';
for (std::size_t i = 0; ok && i < value.size(); ++i)
if (i != 4 && i != 7 && (value[i] < '0' || value[i] > '9')) ok = false;
if (ok) {
int month = std::stoi(value.substr(5, 2));
int day = std::stoi(value.substr(8, 2));
ok = month >= 1 && month <= 12 && day >= 1 && day <= 31;
}
if (!ok) fail(std::string(name) + " must be an ISO-8601 date YYYY-MM-DD");
}
void Check::strings(const char* name, const std::vector<std::string>& v, std::size_t max,
std::size_t each_max) {
upto(name, v.size(), max);
for (const auto& s : v) {
if (s.empty()) fail(std::string(name) + " has an empty entry");
if (s.size() > each_max) fail(std::string(name) + " has an entry over " + std::to_string(each_max) + " bytes");
}
}
void Check::object(const char* name, const Json& value) {
if (!value.is_object()) fail(std::string(name) + " must be a JSON object");
}
void Check::json_schema(const char* name, const Json& value) {
object(name, value);
if (value.dump().size() > Caps::kMaxTextBytes) fail(std::string(name) + " is over the size cap");
}
namespace jx {
const Json* get(const Json& doc, const char* key) { return doc.is_object() ? doc.find(key) : nullptr; }
std::string str(const Json& doc, const char* key, const std::string& fallback) {
const Json* v = get(doc, key);
return v && v->is_string() ? v->as_string() : fallback;
}
long num(const Json& doc, const char* key, long fallback) {
const Json* v = get(doc, key);
return v && v->is_number() ? v->as_int() : fallback;
}
double dbl(const Json& doc, const char* key, double fallback) {
const Json* v = get(doc, key);
return v && v->is_number() ? v->as_number() : fallback;
}
bool flag(const Json& doc, const char* key, bool fallback) {
const Json* v = get(doc, key);
return v && v->is_bool() ? v->as_bool() : fallback;
}
std::vector<std::string> strings(const Json& doc, const char* key) {
std::vector<std::string> out;
const Json* v = get(doc, key);
if (v && v->is_array())
for (const Json& e : v->as_array())
if (e.is_string()) out.push_back(e.as_string());
return out;
}
Json object(const Json& doc, const char* key) {
const Json* v = get(doc, key);
return v && v->is_object() ? *v : Json::object();
}
Json array(const Json& doc, const char* key) {
const Json* v = get(doc, key);
return v && v->is_array() ? *v : Json::array();
}
bool has_string(const Json& doc, const char* key) {
const Json* v = get(doc, key);
return v && v->is_string();
}
void put(Json& doc, const char* key, const std::optional<std::string>& v) {
if (v) doc.set(key, Json(*v));
}
void put(Json& doc, const char* key, const std::optional<int>& v) {
if (v) doc.set(key, Json(*v));
}
void put(Json& doc, const char* key, const std::optional<long>& v) {
if (v) doc.set(key, Json(*v));
}
void put(Json& doc, const char* key, const std::optional<double>& v) {
if (v) doc.set(key, Json(*v));
}
void put(Json& doc, const char* key, const std::optional<bool>& v) {
if (v) doc.set(key, Json(*v));
}
Json string_array(const std::vector<std::string>& v) {
Json arr = Json::array();
for (const auto& s : v) arr.push(Json(s));
return arr;
}
void put_strings(Json& doc, const char* key, const std::vector<std::string>& v) {
if (!v.empty()) doc.set(key, string_array(v));
}
void merge(Json& doc, const Json& extra) {
if (!extra.is_object()) return;
if (extra.as_object().size() > 64) Check::fail("extra holds more than 64 members");
for (const auto& [k, v] : extra.as_object()) doc.set(k, v);
}
} // namespace jx
} // namespace molo::xai::core
+75
View File
@@ -0,0 +1,75 @@
#pragma once
// Input validation (Check) and tolerant JSON readers (jx) shared by every
// xAI core class. Validation throws llm::ApiError::invalid (HTTP 400,
// INVALID_ARGUMENT) before any network traffic; readers never throw on a
// missing field, they return the stated default.
#include <initializer_list>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/error.h"
namespace molo::xai::core {
using Json = clients::body::Json;
inline constexpr const char* kVendor = "xai";
class Check {
public:
[[noreturn]] static void fail(const std::string& message);
// 1..kMaxModelLen printable ASCII without spaces.
static void model(const std::string& model);
// llm::Service::valid_id, reported as "bad <what> id".
static void id(const std::string& what, const std::string& id);
static void non_empty(const char* name, const std::string& value);
static void text(const char* name, const std::string& value, std::size_t max_bytes,
bool allow_empty = false);
// Enum-like token: 1..kMaxTokenLen of [A-Za-z0-9._:/+-].
static void token(const char* name, const std::string& value);
static void one_of(const char* name, const std::string& value,
std::initializer_list<const char*> allowed);
static void range(const char* name, double value, double lo, double hi);
static void count(const char* name, std::size_t n, std::size_t lo, std::size_t hi);
static void upto(const char* name, std::size_t n, std::size_t hi);
// http(s):// or data: URL, bounded.
static void url(const char* name, const std::string& value);
static void http_url(const char* name, const std::string& value);
// YYYY-MM-DD with a plausible month and day.
static void date(const char* name, const std::string& value);
static void strings(const char* name, const std::vector<std::string>& v, std::size_t max,
std::size_t each_max);
// A JSON object (not null, not array).
static void object(const char* name, const Json& value);
static void json_schema(const char* name, const Json& value);
};
// Tolerant readers.
namespace jx {
const Json* get(const Json& doc, const char* key);
std::string str(const Json& doc, const char* key, const std::string& fallback = "");
long num(const Json& doc, const char* key, long fallback = 0);
double dbl(const Json& doc, const char* key, double fallback = 0);
bool flag(const Json& doc, const char* key, bool fallback = false);
std::vector<std::string> strings(const Json& doc, const char* key);
Json object(const Json& doc, const char* key); // {} when absent
Json array(const Json& doc, const char* key); // [] when absent
bool has_string(const Json& doc, const char* key);
// Sets only when the value is present.
void put(Json& doc, const char* key, const std::optional<std::string>& v);
void put(Json& doc, const char* key, const std::optional<int>& v);
void put(Json& doc, const char* key, const std::optional<long>& v);
void put(Json& doc, const char* key, const std::optional<double>& v);
void put(Json& doc, const char* key, const std::optional<bool>& v);
void put_strings(Json& doc, const char* key, const std::vector<std::string>& v);
Json string_array(const std::vector<std::string>& v);
// Merge `extra` object members into `doc` (extra wins) with a hard cap on members.
void merge(Json& doc, const Json& extra);
} // namespace jx
} // namespace molo::xai::core
+12
View File
@@ -0,0 +1,12 @@
#include "xai/core/poll.h"
#include <chrono>
#include <thread>
namespace molo::xai::core {
Sleeper real_sleeper() {
return [](int ms) { std::this_thread::sleep_for(std::chrono::milliseconds(ms)); };
}
} // namespace molo::xai::core
+44
View File
@@ -0,0 +1,44 @@
#pragma once
// Bounded polling for deferred results (chat, image, video). Elapsed time is
// the sum of the sleeps, so the bound is deterministic and testable with a
// fake sleeper. Nothing here waits unbounded.
#include <functional>
#include <optional>
#include "xai/core/caps.h"
#include "xai/core/params.h"
namespace molo::xai::core {
struct WaitPolicy {
int interval_ms = Caps::kDefaultPollIntervalMs;
int timeout_ms = Caps::kDefaultWaitMs;
void validate() const {
Check::range("poll interval ms", interval_ms, Caps::kMinPollIntervalMs, Caps::kMaxPollIntervalMs);
Check::range("wait timeout ms", timeout_ms, Caps::kMinPollIntervalMs, Caps::kMaxWaitMs);
}
// Upper bound of step() calls: one immediately, then one per interval.
int max_polls() const { return timeout_ms / interval_ms + 1; }
};
using Sleeper = std::function<void(int ms)>;
Sleeper real_sleeper(); // std::this_thread::sleep_for
// Calls `step` until it yields a value. Returns nullopt when the policy's
// budget is spent (the caller reports the timeout loudly).
template <typename T>
std::optional<T> poll_until(const WaitPolicy& policy, const std::function<std::optional<T>()>& step,
const Sleeper& sleeper) {
policy.validate();
const int polls = policy.max_polls();
for (int i = 0; i < polls; ++i) {
if (std::optional<T> v = step()) return v;
if (i + 1 < polls) sleeper(policy.interval_ms);
}
return std::nullopt;
}
} // namespace molo::xai::core
+287
View File
@@ -0,0 +1,287 @@
#include "xai/core/response.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
ItemKind item_kind_of(const std::string& type) {
if (type == "message") return ItemKind::kMessage;
if (type == "function_call") return ItemKind::kFunctionCall;
if (type == "reasoning") return ItemKind::kReasoning;
if (type == "web_search_call") return ItemKind::kWebSearchCall;
if (type == "file_search_call") return ItemKind::kFileSearchCall;
if (type == "code_interpreter_call") return ItemKind::kCodeInterpreterCall;
if (type == "mcp_call") return ItemKind::kMcpCall;
if (type == "custom_tool_call") return ItemKind::kCustomToolCall;
if (type == "shell_call") return ItemKind::kShellCall;
if (type == "image_generation_call") return ItemKind::kImageGenerationCall;
if (type == "tool_search_call") return ItemKind::kToolSearchCall;
if (type == "tool_search_output") return ItemKind::kToolSearchOutput;
return ItemKind::kUnknown;
}
OutputItem OutputItem::parse(const Json& item) {
OutputItem o;
o.raw_ = item;
o.type_ = jx::str(item, "type");
// Input-style messages omit `type`; role marks them.
if (o.type_.empty() && jx::has_string(item, "role")) o.type_ = "message";
o.kind_ = item_kind_of(o.type_);
return o;
}
std::string OutputItem::text() const {
std::string out;
if (kind_ != ItemKind::kMessage) return out;
const Json* content = jx::get(raw_, "content");
if (content && content->is_string()) return content->as_string();
if (content && content->is_array())
for (const Json& part : content->as_array())
if (jx::str(part, "type") == "output_text" || jx::str(part, "type") == "input_text")
out += jx::str(part, "text");
return out;
}
std::string OutputItem::refusal() const {
std::string out;
const Json* content = jx::get(raw_, "content");
if (content && content->is_array())
for (const Json& part : content->as_array())
if (jx::str(part, "type") == "refusal") out += jx::str(part, "refusal");
return out;
}
std::vector<Citation> OutputItem::citations() const {
std::vector<Citation> out;
const Json* content = jx::get(raw_, "content");
if (!content || !content->is_array()) return out;
for (const Json& part : content->as_array()) {
const Json* ann = jx::get(part, "annotations");
if (!ann || !ann->is_array()) continue;
for (const Json& a : ann->as_array()) {
if (out.size() >= Caps::kMaxAccumulatedItems) return out;
Citation c;
c.url = jx::str(a, "url");
c.title = jx::str(a, "title");
if (const Json* s = jx::get(a, "start_index"); s && s->is_number()) c.start_index = s->as_int();
if (const Json* e = jx::get(a, "end_index"); e && e->is_number()) c.end_index = e->as_int();
out.push_back(std::move(c));
}
}
return out;
}
std::string OutputItem::arguments_text() const {
const Json* a = jx::get(raw_, "arguments");
if (!a) a = jx::get(raw_, "input");
if (a && a->is_string()) return a->as_string();
if (a && a->is_object()) return a->dump();
return "";
}
bool OutputItem::arguments_valid() const {
std::string t = arguments_text();
if (t.empty()) return true;
try {
return Json::parse(t).is_object();
} catch (const std::exception&) {
return false;
}
}
Json OutputItem::arguments() const {
std::string t = arguments_text();
if (t.empty()) return Json::object();
try {
Json j = Json::parse(t);
return j.is_object() ? j : Json::object();
} catch (const std::exception&) {
return Json::object();
}
}
std::string OutputItem::reasoning_text() const {
std::string out;
const Json* c = jx::get(raw_, "content");
if (c && c->is_array())
for (const Json& e : c->as_array()) out += jx::str(e, "text");
return out;
}
std::string OutputItem::summary_text() const {
std::string out;
const Json* c = jx::get(raw_, "summary");
if (c && c->is_array())
for (const Json& e : c->as_array()) out += jx::str(e, "text");
return out;
}
ServerTool OutputItem::server_tool() const {
switch (kind_) {
case ItemKind::kWebSearchCall: return ServerTool::kWebSearch;
case ItemKind::kFileSearchCall: return ServerTool::kCollectionsSearch;
case ItemKind::kCodeInterpreterCall: return ServerTool::kCodeExecution;
case ItemKind::kMcpCall: return ServerTool::kMcp;
case ItemKind::kImageGenerationCall: return ServerTool::kImageGeneration;
case ItemKind::kToolSearchCall:
case ItemKind::kToolSearchOutput: return ServerTool::kToolSearch;
case ItemKind::kCustomToolCall: {
std::string id = call_id();
if (id.rfind("xs_", 0) == 0) return ServerTool::kXSearch;
if (id.rfind("as_", 0) == 0) return ServerTool::kAttachmentSearch;
return ServerTool::kNone;
}
default: return ServerTool::kNone;
}
}
std::string OutputItem::web_search_query() const {
return jx::str(jx::object(raw_, "action"), "query");
}
std::vector<std::string> OutputItem::web_search_sources() const {
std::vector<std::string> out;
Json action = jx::object(raw_, "action");
const Json* s = jx::get(action, "sources");
if (s && s->is_array())
for (const Json& e : s->as_array())
if (jx::has_string(e, "url")) out.push_back(jx::str(e, "url"));
return out;
}
ModelResponse ModelResponse::parse(const Json& doc) {
ModelResponse r;
r.raw_ = doc;
r.id_ = jx::str(doc, "id");
r.model_ = jx::str(doc, "model");
r.status_ = jx::str(doc, "status");
r.created_at_ = jx::num(doc, "created_at");
r.completed_at_ = jx::num(doc, "completed_at");
r.previous_ = jx::str(doc, "previous_response_id");
r.service_tier_ = jx::str(doc, "service_tier");
r.store_ = jx::flag(doc, "store");
r.incomplete_reason_ = jx::str(jx::object(doc, "incomplete_details"), "reason");
const Json* out = jx::get(doc, "output");
if (out && out->is_array())
for (const Json& item : out->as_array()) {
if (r.output_.size() >= Caps::kMaxAccumulatedItems) break;
r.output_.push_back(OutputItem::parse(item));
}
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) r.usage_ = TokenUsage::from_responses(*u);
if (const Json* e = jx::get(doc, "error"); e && e->is_object()) {
ResponseError err;
err.code = jx::str(*e, "code");
err.message = jx::str(*e, "message");
r.error_ = err;
} else if (e && e->is_string()) {
r.error_ = ResponseError{"", e->as_string()};
}
return r;
}
std::string ModelResponse::text() const {
std::string out;
for (const auto& i : output_)
if (i.kind() == ItemKind::kMessage) out += i.text();
return out;
}
std::string ModelResponse::reasoning_text() const {
std::string out;
for (const auto& i : output_)
if (i.kind() == ItemKind::kReasoning) out += i.reasoning_text();
return out;
}
std::string ModelResponse::summary_text() const {
std::string out;
for (const auto& i : output_)
if (i.kind() == ItemKind::kReasoning) out += i.summary_text();
return out;
}
std::vector<Citation> ModelResponse::citations() const {
std::vector<Citation> out;
for (const auto& i : output_)
if (i.kind() == ItemKind::kMessage)
for (auto& c : i.citations()) out.push_back(std::move(c));
return out;
}
std::vector<OutputItem> ModelResponse::function_calls() const {
std::vector<OutputItem> out;
for (const auto& i : output_)
if (i.is_client_call()) out.push_back(i);
return out;
}
std::vector<OutputItem> ModelResponse::server_tool_calls() const {
std::vector<OutputItem> out;
for (const auto& i : output_)
if (i.server_tool() != ServerTool::kNone) out.push_back(i);
return out;
}
std::vector<Json> ModelResponse::replay_items() const {
std::vector<Json> out;
for (const auto& i : output_) {
switch (i.kind()) {
case ItemKind::kReasoning:
if (!i.encrypted_content().empty()) out.push_back(i.raw());
break;
case ItemKind::kFunctionCall:
case ItemKind::kMessage:
out.push_back(i.raw());
break;
default:
if (i.server_tool() != ServerTool::kNone) out.push_back(i.raw());
}
}
return out;
}
DeletedResponse DeletedResponse::parse(const Json& doc) {
DeletedResponse d;
d.id = jx::str(doc, "id");
d.deleted = jx::flag(doc, "deleted");
d.object = jx::str(doc, "object");
return d;
}
InputItemList InputItemList::parse(const Json& doc) {
InputItemList l;
const Json* data = jx::get(doc, "data");
if (data && data->is_array())
for (const Json& e : data->as_array()) {
if (l.data.size() >= Caps::kMaxAccumulatedItems) { l.truncated = true; break; }
l.data.push_back(e);
}
l.first_id = jx::str(doc, "first_id");
l.last_id = jx::str(doc, "last_id");
l.object = jx::str(doc, "object");
l.has_more = jx::flag(doc, "has_more");
return l;
}
CompactResult CompactResult::parse(const Json& doc) {
CompactResult c;
c.raw = doc;
c.id = jx::str(doc, "id");
c.model = jx::str(doc, "model");
c.object = jx::str(doc, "object");
c.created_at = jx::num(doc, "created_at");
const Json* out = jx::get(doc, "output");
if (out && out->is_array())
for (const Json& e : out->as_array()) {
if (c.output.size() >= Caps::kMaxAccumulatedItems) break;
c.output.push_back(e);
}
Json u = jx::object(doc, "usage");
c.input_tokens = jx::num(u, "input_tokens");
c.output_tokens = jx::num(u, "output_tokens");
c.total_tokens = jx::num(u, "total_tokens");
c.dropped_message_count = jx::num(u, "dropped_message_count");
return c;
}
} // namespace molo::xai::core
+151
View File
@@ -0,0 +1,151 @@
#pragma once
// Typed view of a Responses API document (`ModelResponse`): output items
// (message, function_call, reasoning, the server-side tool calls), usage,
// status and error. Parsing is tolerant: unknown item types stay reachable
// through raw().
#include <optional>
#include <string>
#include <vector>
#include "xai/core/params.h"
#include "xai/core/usage.h"
namespace molo::xai::core {
enum class ItemKind {
kMessage, kFunctionCall, kReasoning, kWebSearchCall, kFileSearchCall, kCodeInterpreterCall,
kMcpCall, kCustomToolCall, kShellCall, kImageGenerationCall, kToolSearchCall,
kToolSearchOutput, kUnknown
};
ItemKind item_kind_of(const std::string& type);
// Which xAI server-side tool produced an item. X search arrives as a
// custom_tool_call whose call_id starts with `xs_`; attachment search with `as_`.
enum class ServerTool { kNone, kWebSearch, kXSearch, kAttachmentSearch, kCodeExecution,
kCollectionsSearch, kMcp, kImageGeneration, kToolSearch };
struct Citation {
std::string url;
std::string title;
std::optional<long> start_index;
std::optional<long> end_index;
};
class OutputItem {
public:
static OutputItem parse(const Json& item);
ItemKind kind() const { return kind_; }
const std::string& type() const { return type_; }
std::string id() const { return jx::str(raw_, "id"); }
std::string status() const { return jx::str(raw_, "status"); }
const Json& raw() const { return raw_; }
// message: concatenated output_text, refusal, citations (url_citation annotations)
std::string text() const;
std::string refusal() const;
std::vector<Citation> citations() const;
std::string role() const { return jx::str(raw_, "role"); }
// function_call / custom_tool_call / mcp_call / shell_call
std::string call_id() const { return jx::str(raw_, "call_id"); }
std::string name() const { return jx::str(raw_, "name"); }
std::string arguments_text() const; // the JSON string as sent
Json arguments() const; // parsed; {} when empty or invalid
bool arguments_valid() const;
// reasoning
std::string reasoning_text() const; // content[].text
std::string summary_text() const; // summary[].text
std::string encrypted_content() const { return jx::str(raw_, "encrypted_content"); }
// server-side tools
ServerTool server_tool() const;
bool is_client_call() const { return kind_ == ItemKind::kFunctionCall; }
std::string web_search_query() const; // web_search_call.action.query
std::vector<std::string> web_search_sources() const;
std::string code() const { return jx::str(raw_, "code"); } // code_interpreter_call
std::string image_base64() const { return jx::str(raw_, "result"); } // image_generation_call
private:
ItemKind kind_ = ItemKind::kUnknown;
std::string type_;
Json raw_;
};
struct ResponseError {
std::string code;
std::string message;
};
class ModelResponse {
public:
static ModelResponse parse(const Json& doc);
const std::string& id() const { return id_; }
const std::string& model() const { return model_; }
const std::string& status() const { return status_; } // completed | in_progress | incomplete | failed
long created_at() const { return created_at_; }
long completed_at() const { return completed_at_; }
const std::string& previous_response_id() const { return previous_; }
const std::string& service_tier() const { return service_tier_; }
bool stored() const { return store_; }
const std::vector<OutputItem>& output() const { return output_; }
const TokenUsage& usage() const { return usage_; }
const std::optional<ResponseError>& error() const { return error_; }
const std::string& incomplete_reason() const { return incomplete_reason_; }
const Json& raw() const { return raw_; }
bool completed() const { return status_ == "completed"; }
bool incomplete() const { return status_ == "incomplete"; }
bool failed() const { return status_ == "failed" || error_.has_value(); }
std::string text() const; // all message text, in order
std::string reasoning_text() const; // raw reasoning content, if the model returned it
std::string summary_text() const;
std::vector<Citation> citations() const;
std::vector<OutputItem> function_calls() const; // client-side, need local execution
std::vector<OutputItem> server_tool_calls() const; // handled by xAI
// Items to pass back as `input` on the next turn when the history is managed
// by the caller (reasoning with encrypted_content, function calls, outputs).
std::vector<Json> replay_items() const;
private:
std::string id_, model_, status_, previous_, service_tier_, incomplete_reason_;
long created_at_ = 0, completed_at_ = 0;
bool store_ = false;
std::vector<OutputItem> output_;
TokenUsage usage_;
std::optional<ResponseError> error_;
Json raw_;
};
struct DeletedResponse {
std::string id;
bool deleted = false;
std::string object;
static DeletedResponse parse(const Json& doc);
};
// One page of GET /v1/responses/{id}/input_items.
struct InputItemList {
std::vector<Json> data;
std::string first_id, last_id, object;
bool has_more = false;
bool truncated = false; // set by the walker when a cap stopped it
static InputItemList parse(const Json& doc);
};
struct CompactResult {
std::string id, model, object;
long created_at = 0;
std::vector<Json> output; // pass verbatim into the next request's input
long input_tokens = 0, output_tokens = 0, total_tokens = 0, dropped_message_count = 0;
Json raw;
static CompactResult parse(const Json& doc);
};
} // namespace molo::xai::core
+79
View File
@@ -0,0 +1,79 @@
#include "xai/core/response_format.h"
namespace molo::xai::core {
namespace {
void walk(const Json& node, int depth) {
if (depth > ResponseFormat::kMaxSchemaDepth) Check::fail("schema is nested deeper than 64 levels");
if (!node.is_object()) return;
for (const auto& [key, value] : node.as_object()) {
if ((key == "enum" || key == "anyOf" || key == "oneOf") && value.is_array() &&
value.as_array().empty())
Check::fail("schema keyword " + key + " has zero variants (xAI answers 400)");
if (key == "minContains" || key == "maxContains")
Check::fail("schema keyword " + key + " is rejected by xAI");
if (key == "items" && value.is_array())
Check::fail("schema `items` must not be an array; use prefixItems (xAI answers 400)");
if (key == "properties" && value.is_object()) {
for (const auto& [pname, pschema] : value.as_object())
if (pschema.is_bool())
Check::fail("property " + pname + " has a boolean schema (xAI answers 400)");
}
if (value.is_object()) walk(value, depth + 1);
if (value.is_array())
for (const Json& e : value.as_array()) walk(e, depth + 1);
}
}
} // namespace
void ResponseFormat::lint(const Json& schema) {
Check::json_schema("response schema", schema);
walk(schema, 0);
}
ResponseFormat ResponseFormat::json_schema(std::string name, Json schema, bool strict) {
Check::token("schema name", name);
lint(schema);
ResponseFormat f(Kind::kJsonSchema);
f.name_ = std::move(name);
f.schema_ = std::move(schema);
f.strict_ = strict;
return f;
}
Json ResponseFormat::to_chat() const {
Json o = Json::object();
switch (kind_) {
case Kind::kText: o.set("type", Json("text")); break;
case Kind::kJsonObject: o.set("type", Json("json_object")); break;
case Kind::kJsonSchema: {
Json js = Json::object();
js.set("name", Json(name_));
js.set("schema", schema_);
js.set("strict", Json(strict_));
o.set("type", Json("json_schema"));
o.set("json_schema", js);
break;
}
}
return o;
}
Json ResponseFormat::to_responses_text() const {
Json f = Json::object();
switch (kind_) {
case Kind::kText: f.set("type", Json("text")); break;
case Kind::kJsonObject: f.set("type", Json("json_object")); break;
case Kind::kJsonSchema:
f.set("type", Json("json_schema"));
f.set("name", Json(name_));
f.set("schema", schema_);
f.set("strict", Json(strict_));
break;
}
Json text = Json::object();
text.set("format", f);
return text;
}
} // namespace molo::xai::core
+45
View File
@@ -0,0 +1,45 @@
#pragma once
// Structured outputs. One value class for the three response formats (text,
// json_object, json_schema) that renders the two wire shapes:
// chat completions response_format: {type:"json_schema", json_schema:{name,schema,strict}}
// Responses API text: {format:{type:"json_schema", name, schema, strict}}
// The schema is linted against the documented rejections (docs: structured
// outputs, "Rejected schemas") so a request xAI would answer with 400 is
// refused locally.
#include <string>
#include "xai/core/params.h"
namespace molo::xai::core {
class ResponseFormat {
public:
enum class Kind { kText, kJsonObject, kJsonSchema };
static ResponseFormat text() { return ResponseFormat(Kind::kText); }
static ResponseFormat json_object() { return ResponseFormat(Kind::kJsonObject); }
// `name` is required by OpenAI-style clients and passed through; xAI marks
// name / description / strict "only included for compatibility".
static ResponseFormat json_schema(std::string name, Json schema, bool strict = true);
Kind kind() const { return kind_; }
Json to_chat() const;
Json to_responses_text() const; // the whole `text` object
// Throws ApiError::invalid for: enum / anyOf / oneOf with zero variants,
// a property schema of true or false, minContains / maxContains, `items`
// as an array, or nesting deeper than kMaxSchemaDepth.
static void lint(const Json& schema);
static constexpr int kMaxSchemaDepth = 64;
private:
explicit ResponseFormat(Kind k) : kind_(k) {}
Kind kind_;
std::string name_;
Json schema_;
bool strict_ = true;
};
} // namespace molo::xai::core
+201
View File
@@ -0,0 +1,201 @@
#include "xai/core/responses.h"
#include "llm/caps.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
ResponseRequest& ResponseRequest::add_tool(const Tool& tool) {
return add_tool_json(tool.to_json());
}
ResponseRequest& ResponseRequest::add_tool_json(Json tool) {
Check::object("tool", tool);
Check::upto("tools", tools_.size(), Caps::kMaxResponsesTools - 1);
tools_.push_back(std::move(tool));
return *this;
}
ResponseRequest& ResponseRequest::include(std::string value) {
Check::token("include", value);
Check::upto("include", include_.size(), Caps::kMaxIncludes - 1);
include_.push_back(std::move(value));
return *this;
}
Json ResponseRequest::to_json(bool stream) const {
Check::model(model_);
Json body = Json::object();
body.set("model", Json(model_));
body.set("input", input_.to_json());
if (instructions_) {
Check::text("instructions", *instructions_, Caps::kMaxTextBytes);
if (previous_) Check::fail("instructions cannot be combined with previous_response_id");
body.set("instructions", Json(*instructions_));
}
if (previous_) {
Check::id("previous_response", *previous_);
body.set("previous_response_id", Json(*previous_));
}
jx::put(body, "store", store_);
if (stream) body.set("stream", Json(true));
if (max_output_tokens_) {
Check::range("max_output_tokens", *max_output_tokens_, 1, 10'000'000);
body.set("max_output_tokens", Json(*max_output_tokens_));
}
if (max_turns_) {
Check::range("max_turns", *max_turns_, 1, Caps::kMaxTurns);
body.set("max_turns", Json(*max_turns_));
}
if (temperature_) {
Check::range("temperature", *temperature_, 0, 2);
body.set("temperature", Json(*temperature_));
}
if (top_p_) {
Check::range("top_p", *top_p_, 0, 1);
body.set("top_p", Json(*top_p_));
}
if (top_k_) {
Check::range("top_k", *top_k_, 1, 1'000'000);
body.set("top_k", Json(*top_k_));
}
if (min_p_) {
Check::range("min_p", *min_p_, 0, 1);
body.set("min_p", Json(*min_p_));
}
jx::put(body, "logprobs", logprobs_);
if (top_logprobs_) {
Check::range("top_logprobs", *top_logprobs_, 0, Caps::kMaxTopLogprobs);
if (!logprobs_ || !*logprobs_) Check::fail("top_logprobs needs logprobs=true");
body.set("top_logprobs", Json(*top_logprobs_));
}
if (!effort_.empty() || !summary_.empty()) {
Json reasoning = Json::object();
if (!effort_.empty()) {
Check::token("reasoning effort", effort_);
reasoning.set("effort", Json(effort_));
}
if (!summary_.empty()) {
Check::one_of("reasoning summary", summary_, {"auto", "concise", "detailed"});
reasoning.set("summary", Json(summary_));
}
body.set("reasoning", reasoning);
}
if (!legacy_effort_.empty()) {
Check::token("reasoning_effort", legacy_effort_);
if (!effort_.empty()) Check::fail("reasoning_effort is only read when reasoning.effort is unset");
body.set("reasoning_effort", Json(legacy_effort_));
}
if (!tools_.empty()) {
Json arr = Json::array();
for (const Json& t : tools_) arr.push(t);
body.set("tools", arr);
}
if (tool_choice_) body.set("tool_choice", tool_choice_->to_responses());
jx::put(body, "parallel_tool_calls", parallel_);
if (format_) body.set("text", format_->to_responses_text());
if (!include_.empty()) body.set("include", jx::string_array(include_));
if (tier_) {
Check::one_of("service_tier", *tier_, {"default", "priority", "fast"});
body.set("service_tier", Json(*tier_));
}
if (user_) { Check::text("user", *user_, 256); body.set("user", Json(*user_)); }
if (safety_) { Check::text("safety_identifier", *safety_, 256); body.set("safety_identifier", Json(*safety_)); }
if (cache_key_) { Check::text("prompt_cache_key", *cache_key_, 256); body.set("prompt_cache_key", Json(*cache_key_)); }
if (search_) body.set("search_parameters", search_->to_json());
if (!guidelines_.empty()) body.set("guidelines", guidelines_json(guidelines_));
return body;
}
ModelResponse ResponsesService::create(const ResponseRequest& request) {
Json body = request.to_json(false);
return ModelResponse::parse(call_json(Call::kResponsesCreate, {}, &body));
}
StreamResult ResponsesService::stream(const ResponseRequest& request, const EventSink& sink) {
Json body = request.to_json(true);
StreamResult result;
call_stream(Call::kResponsesCreate, body, [&](const llm::SseEvent& raw) {
ResponseEvent event = ResponseEvent::parse(raw);
bool keep = result.accumulated.feed(event);
bool user_keep = sink ? sink(event) : true;
return keep && user_keep;
});
const ResponseAccumulator& acc = result.accumulated;
if (acc.failed())
throw llm::ApiError(kVendor, 502, "STREAM_FAILED",
acc.error_message().empty() ? "the response stream failed" : acc.error_message());
if (acc.stopped())
throw llm::ApiError(kVendor, 502, "STREAM_CAP", acc.stop_reason());
result.response = acc.final_response();
result.completed = result.response && result.response->completed();
return result;
}
ModelResponse ResponsesService::retrieve(const std::string& response_id) {
return ModelResponse::parse(call_json(Call::kResponsesGet, {response_id}));
}
DeletedResponse ResponsesService::remove(const std::string& response_id) {
return DeletedResponse::parse(call_json(Call::kResponsesDelete, {response_id}));
}
InputItemList ResponsesService::input_items(const std::string& response_id, std::optional<int> limit,
const std::string& order, const std::string& after) {
Query q;
if (limit) {
Check::range("limit", *limit, 1, Caps::kMaxListLimit);
q.add(Call::kResponsesInputItems, "limit", static_cast<long>(*limit));
}
if (!order.empty()) {
Check::one_of("order", order, {"asc", "desc"});
q.add(Call::kResponsesInputItems, "order", order);
}
if (!after.empty()) {
Check::id("after", after);
q.add(Call::kResponsesInputItems, "after", after);
}
return InputItemList::parse(call_json(Call::kResponsesInputItems, {response_id}, nullptr, q.str()));
}
InputItemList ResponsesService::all_input_items(const std::string& response_id, std::size_t max_items) {
Check::range("max_items", static_cast<double>(max_items), 1, Caps::kMaxAccumulatedItems);
InputItemList all;
std::string after;
for (std::size_t page = 0; page < llm::Caps::kMaxPages; ++page) {
InputItemList p = input_items(response_id, 100, "asc", after);
for (Json& item : p.data) {
if (all.data.size() >= max_items) {
all.truncated = true;
return all;
}
all.data.push_back(std::move(item));
}
if (all.first_id.empty()) all.first_id = p.first_id;
all.last_id = p.last_id;
all.object = p.object;
if (!p.has_more || p.last_id.empty()) {
all.has_more = false;
return all;
}
after = p.last_id;
}
all.has_more = true;
all.truncated = true;
return all;
}
CompactResult ResponsesService::compact(const std::string& model, const Input& input) {
Check::model(model);
Json body = Json::object();
body.set("model", Json(model));
body.set("input", input.to_json());
return CompactResult::parse(call_json(Call::kResponsesCompact, {}, &body));
}
} // namespace molo::xai::core
+128
View File
@@ -0,0 +1,128 @@
#pragma once
// Responses API (POST /v1/responses and the stored-response calls):
// ResponseRequest builds and validates the body, ResponsesService sends it.
#include <functional>
#include <optional>
#include <string>
#include <vector>
#include "xai/core/input.h"
#include "xai/core/response.h"
#include "xai/core/response_format.h"
#include "xai/core/search_parameters.h"
#include "xai/core/service.h"
#include "xai/core/stream_events.h"
#include "xai/core/tools.h"
namespace molo::xai::core {
// Reasoning effort values the guides name. Which values a model accepts is
// model specific: read ModelCapabilities::reasoning_effort from the models API.
struct Effort {
static constexpr const char* kLow = "low";
static constexpr const char* kMedium = "medium";
static constexpr const char* kHigh = "high";
static constexpr const char* kXHigh = "xhigh";
};
// `include` values the guides name for the Responses API.
struct Include {
static constexpr const char* kEncryptedReasoning = "reasoning.encrypted_content";
static constexpr const char* kWebSearchSources = "web_search_call.action.sources";
static constexpr const char* kCodeInterpreterOutputs = "code_interpreter_call.outputs";
static constexpr const char* kFileSearchResults = "file_search_call.results";
};
class ResponseRequest {
public:
explicit ResponseRequest(std::string model) : model_(std::move(model)) {}
ResponseRequest& input(Input in) { input_ = std::move(in); return *this; }
ResponseRequest& input_text(std::string text) { input_ = Input(std::move(text)); return *this; }
ResponseRequest& add_input(InputItem item) { input_.add(std::move(item)); return *this; }
// Cannot be combined with previous_response_id (the stored system prompt is reused).
ResponseRequest& instructions(std::string s) { instructions_ = std::move(s); return *this; }
ResponseRequest& previous_response_id(std::string id) { previous_ = std::move(id); return *this; }
ResponseRequest& store(bool v) { store_ = v; return *this; }
ResponseRequest& max_output_tokens(int n) { max_output_tokens_ = n; return *this; }
ResponseRequest& max_turns(int n) { max_turns_ = n; return *this; }
ResponseRequest& temperature(double v) { temperature_ = v; return *this; }
ResponseRequest& top_p(double v) { top_p_ = v; return *this; }
ResponseRequest& top_k(int v) { top_k_ = v; return *this; }
ResponseRequest& min_p(double v) { min_p_ = v; return *this; }
ResponseRequest& logprobs(bool v) { logprobs_ = v; return *this; }
ResponseRequest& top_logprobs(int v) { top_logprobs_ = v; return *this; }
// Reasoning controls: `reasoning.effort` (preferred) and `reasoning.summary`.
ResponseRequest& reasoning_effort(std::string effort) { effort_ = std::move(effort); return *this; }
ResponseRequest& reasoning_summary(std::string summary) { summary_ = std::move(summary); return *this; }
// The non-standard top-level `reasoning_effort` field instead of reasoning.effort.
ResponseRequest& legacy_reasoning_effort(std::string effort) { legacy_effort_ = std::move(effort); return *this; }
ResponseRequest& add_tool(const Tool& tool);
ResponseRequest& add_tool_json(Json tool);
ResponseRequest& tool_choice(ToolChoice choice) { tool_choice_ = std::move(choice); return *this; }
ResponseRequest& parallel_tool_calls(bool v) { parallel_ = v; return *this; }
ResponseRequest& response_format(ResponseFormat f) { format_ = std::move(f); return *this; }
ResponseRequest& include(std::string value);
ResponseRequest& service_tier(std::string tier) { tier_ = std::move(tier); return *this; }
ResponseRequest& user(std::string u) { user_ = std::move(u); return *this; }
ResponseRequest& safety_identifier(std::string s) { safety_ = std::move(s); return *this; }
ResponseRequest& prompt_cache_key(std::string k) { cache_key_ = std::move(k); return *this; }
ResponseRequest& search_parameters(SearchParameters p) { search_ = std::move(p); return *this; }
ResponseRequest& guideline(Guideline g) { guidelines_.push_back(std::move(g)); return *this; }
const std::string& model() const { return model_; }
// Validates every field and returns the wire body. `stream` adds stream:true.
Json to_json(bool stream = false) const;
private:
std::string model_;
Input input_;
std::optional<std::string> instructions_, previous_, tier_, user_, safety_, cache_key_;
std::optional<bool> store_, logprobs_, parallel_;
std::optional<int> max_output_tokens_, max_turns_, top_k_, top_logprobs_;
std::optional<double> temperature_, top_p_, min_p_;
std::string effort_, summary_, legacy_effort_;
std::vector<Json> tools_;
std::optional<ToolChoice> tool_choice_;
std::optional<ResponseFormat> format_;
std::vector<std::string> include_;
std::optional<SearchParameters> search_;
std::vector<Guideline> guidelines_;
};
// What a streamed call returns once the stream ends.
struct StreamResult {
ResponseAccumulator accumulated;
std::optional<ModelResponse> response; // the final document when the stream delivered one
bool completed = false;
};
class ResponsesService : public XaiService {
public:
using EventSink = std::function<bool(const ResponseEvent&)>; // false stops the stream
explicit ResponsesService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
ModelResponse create(const ResponseRequest& request);
// Streams events to `sink` (may be empty) and folds them. Throws ApiError
// on a stream failure event.
StreamResult stream(const ResponseRequest& request, const EventSink& sink = {});
ModelResponse retrieve(const std::string& response_id);
// Destructive: deletes a stored response. Explicit, no default.
DeletedResponse remove(const std::string& response_id);
// One page of input items. order: asc | desc. limit 1..Caps::kMaxListLimit.
InputItemList input_items(const std::string& response_id, std::optional<int> limit = std::nullopt,
const std::string& order = "", const std::string& after = "");
// Walks pages up to Caps::kMaxPages and max_items; truncated is set when a cap stopped it.
InputItemList all_input_items(const std::string& response_id, std::size_t max_items = 1000);
CompactResult compact(const std::string& model, const Input& input);
};
} // namespace molo::xai::core
+97
View File
@@ -0,0 +1,97 @@
#include "xai/core/search_parameters.h"
#include <set>
#include "xai/core/caps.h"
namespace molo::xai::core {
SearchSource SearchSource::web() { SearchSource s; s.type = "web"; return s; }
SearchSource SearchSource::x() { SearchSource s; s.type = "x"; return s; }
SearchSource SearchSource::news() { SearchSource s; s.type = "news"; return s; }
SearchSource SearchSource::rss(std::vector<std::string> links) {
SearchSource s;
s.type = "rss";
s.links = std::move(links);
return s;
}
Json SearchSource::to_json() const {
Check::one_of("source type", type, {"web", "x", "news", "rss"});
Json o = Json::object();
o.set("type", Json(type));
if (type == "web" || type == "news") {
if (!country.empty()) {
if (country.size() != 2) Check::fail("country must be a 2-letter ISO code");
o.set("country", Json(country));
}
Check::strings("excluded_websites", excluded_websites, Caps::kMaxDomains, 253);
jx::put_strings(o, "excluded_websites", excluded_websites);
jx::put(o, "safe_search", safe_search);
if (type == "web") {
Check::strings("allowed_websites", allowed_websites, Caps::kMaxDomains, 253);
if (!allowed_websites.empty() && !excluded_websites.empty())
Check::fail("allowed_websites and excluded_websites are mutually exclusive");
jx::put_strings(o, "allowed_websites", allowed_websites);
}
} else if (type == "x") {
Check::strings("included_x_handles", included_x_handles, Caps::kMaxXHandles, 64);
Check::strings("excluded_x_handles", excluded_x_handles, Caps::kMaxXHandles, 64);
if (!included_x_handles.empty() && !excluded_x_handles.empty())
Check::fail("included and excluded x handles are mutually exclusive");
jx::put_strings(o, "included_x_handles", included_x_handles);
jx::put_strings(o, "excluded_x_handles", excluded_x_handles);
jx::put(o, "post_favorite_count", post_favorite_count);
jx::put(o, "post_view_count", post_view_count);
} else {
Check::count("rss links", links.size(), 1, 1);
for (const auto& l : links) Check::http_url("rss link", l);
o.set("links", jx::string_array(links));
}
return o;
}
Json SearchParameters::to_json() const {
Json o = Json::object();
if (!mode_.empty()) {
Check::one_of("search mode", mode_, {"off", "on", "auto"});
o.set("mode", Json(mode_));
}
if (!from_.empty()) { Check::date("from_date", from_); o.set("from_date", Json(from_)); }
if (!to_.empty()) { Check::date("to_date", to_); o.set("to_date", Json(to_)); }
if (!from_.empty() && !to_.empty() && from_ > to_) Check::fail("from_date is after to_date");
if (max_results_) {
Check::range("max_search_results", *max_results_, 1, 100);
o.set("max_search_results", Json(*max_results_));
}
jx::put(o, "return_citations", citations_);
if (!sources_.empty()) {
Check::upto("sources", sources_.size(), 8);
Json arr = Json::array();
for (const auto& s : sources_) arr.push(s.to_json());
o.set("sources", arr);
}
return o;
}
Json Guideline::to_json() const {
Check::one_of("guideline category", category, {"nsfw", "violence", "profanity"});
Json o = Json::object();
o.set("type", Json("policy"));
o.set("category", Json(category));
o.set("behavior", Json("decline"));
return o;
}
Json guidelines_json(const std::vector<Guideline>& g) {
Check::upto("guidelines", g.size(), Caps::kMaxGuidelines);
std::set<std::string> seen;
Json arr = Json::array();
for (const auto& e : g) {
if (!seen.insert(e.category).second) Check::fail("each guideline category may appear once");
arr.push(e.to_json());
}
return arr;
}
} // namespace molo::xai::core
+60
View File
@@ -0,0 +1,60 @@
#pragma once
// Live Search parameters (`search_parameters`) shared by chat completions and
// the Responses API, plus the per-category output guidelines. The server-side
// tools (WebSearchTool, XSearchTool) are the current way to search; this is
// the older request-level switch the spec still carries.
#include <optional>
#include <string>
#include <vector>
#include "xai/core/params.h"
namespace molo::xai::core {
struct SearchSource {
// type: "web", "x", "news" or "rss".
std::string type;
std::string country; // web, news: ISO alpha-2
std::vector<std::string> allowed_websites; // web
std::vector<std::string> excluded_websites; // web, news
std::optional<bool> safe_search; // web, news
std::vector<std::string> included_x_handles; // x
std::vector<std::string> excluded_x_handles; // x
std::optional<long> post_favorite_count; // x
std::optional<long> post_view_count; // x
std::vector<std::string> links; // rss (required, 1 link)
static SearchSource web();
static SearchSource x();
static SearchSource news();
static SearchSource rss(std::vector<std::string> links);
Json to_json() const;
};
class SearchParameters {
public:
SearchParameters& mode(std::string m) { mode_ = std::move(m); return *this; } // off | on | auto
SearchParameters& from_date(std::string d) { from_ = std::move(d); return *this; }
SearchParameters& to_date(std::string d) { to_ = std::move(d); return *this; }
SearchParameters& max_results(int n) { max_results_ = n; return *this; }
SearchParameters& citations(bool v) { citations_ = v; return *this; }
SearchParameters& add_source(SearchSource s) { sources_.push_back(std::move(s)); return *this; }
Json to_json() const;
private:
std::string mode_, from_, to_;
std::optional<int> max_results_;
std::optional<bool> citations_;
std::vector<SearchSource> sources_;
};
// {type:"policy", category, behavior:"decline"}; one entry per category.
struct Guideline {
std::string category; // nsfw | violence | profanity
Json to_json() const;
};
Json guidelines_json(const std::vector<Guideline>& g);
} // namespace molo::xai::core
+80
View File
@@ -0,0 +1,80 @@
#include "xai/core/service.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
llm::ApiClientConfig XaiService::default_config() {
llm::ApiClientConfig c;
c.vendor = kVendor;
c.base_url = kBaseUrl;
c.key_env = kKeyEnv;
c.auth = llm::AuthStyle::kBearer;
return c;
}
std::shared_ptr<llm::ApiClient> XaiService::client_from_env() {
return llm::ApiClient::from_env(default_config());
}
Plan XaiService::plan(Call call, const std::vector<std::string>& ids, const Json* body,
const std::string& query) {
Plan p;
p.method = method_of(call);
p.path = render(call, ids) + query;
if (body) {
if (!body->is_object()) Check::fail("request body must be a JSON object");
for (const auto& [key, value] : body->as_object()) {
(void)value;
if (!declares_body_field(call, key))
Check::fail("field " + key + " is not declared for " + spec_of(call).path);
}
p.body = *body;
p.has_body = true;
}
return p;
}
llm::RawResponse XaiService::send_raw(const Plan& p) {
if (p.has_body) {
std::string text = p.body.dump();
return client_->request_raw(p.method, p.path, &text, "application/json");
}
return client_->request_raw(p.method, p.path);
}
Json XaiService::parse_body(const llm::RawResponse& raw) {
if (raw.body.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
try {
return Json::parse(raw.body);
} catch (const std::exception& e) {
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", e.what());
}
}
Json XaiService::send(const Plan& p) {
return p.has_body ? client_->request(p.method, p.path, &p.body)
: client_->request(p.method, p.path);
}
void XaiService::send_stream(const Plan& p, const llm::SseDecoder::Sink& sink) {
client_->stream(p.path, p.body, sink);
}
Json XaiService::call_json(Call call, const std::vector<std::string>& ids, const Json* body,
const std::string& query) {
return send(plan(call, ids, body, query));
}
llm::RawResponse XaiService::call_raw(Call call, const std::vector<std::string>& ids,
const Json* body, const std::string& query) {
return send_raw(plan(call, ids, body, query));
}
void XaiService::call_stream(Call call, const Json& body, const llm::SseDecoder::Sink& sink) {
Plan p = plan(call, {}, &body);
if (p.method != clients::Method::kPost) Check::fail("only POST calls stream");
send_stream(p, sink);
}
} // namespace molo::xai::core
+56
View File
@@ -0,0 +1,56 @@
#pragma once
// XaiService: base of every xAI core service. Owns the call-table dispatch:
// every request goes through Call -> (method, path) from calls.h, with the
// body checked against the row's declared fields before any network traffic.
#include <functional>
#include <memory>
#include <string>
#include <vector>
#include "llm/api_client.h"
#include "xai/core/calls.h"
#include "xai/core/params.h"
namespace molo::xai::core {
// A request ready to send, observable without a network (tests, dry runs).
struct Plan {
clients::Method method = clients::Method::kGet;
std::string path; // includes the query string
Json body; // null when the call has no JSON body
bool has_body = false;
};
class XaiService : public llm::Service {
public:
explicit XaiService(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
// vendor "xai", https://api.x.ai, XAI_API_KEY as a bearer token.
static llm::ApiClientConfig default_config();
static std::shared_ptr<llm::ApiClient> client_from_env();
static constexpr const char* kBaseUrl = "https://api.x.ai";
static constexpr const char* kKeyEnv = "XAI_API_KEY";
// Build the plan for a call: renders the path, checks body fields against
// the call table (an undeclared top-level field is rejected loudly).
static Plan plan(Call call, const std::vector<std::string>& ids = {},
const Json* body = nullptr, const std::string& query = "");
protected:
Json send(const Plan& plan);
llm::RawResponse send_raw(const Plan& plan);
void send_stream(const Plan& plan, const llm::SseDecoder::Sink& sink);
Json call_json(Call call, const std::vector<std::string>& ids = {}, const Json* body = nullptr,
const std::string& query = "");
// Raw reply with the status preserved (202 pending polls, binary content).
llm::RawResponse call_raw(Call call, const std::vector<std::string>& ids = {},
const Json* body = nullptr, const std::string& query = "");
void call_stream(Call call, const Json& body, const llm::SseDecoder::Sink& sink);
static Json parse_body(const llm::RawResponse& raw); // {} for an empty body
};
} // namespace molo::xai::core
+193
View File
@@ -0,0 +1,193 @@
#include "xai/core/stream_events.h"
#include <cstring>
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
struct Row {
const char* type;
EventKind kind;
};
const Row kRows[] = {
{"response.created", EventKind::kCreated},
{"response.in_progress", EventKind::kInProgress},
{"response.completed", EventKind::kCompleted},
{"response.incomplete", EventKind::kIncomplete},
{"response.failed", EventKind::kFailed},
{"error", EventKind::kError},
{"response.output_item.added", EventKind::kOutputItemAdded},
{"response.output_item.done", EventKind::kOutputItemDone},
{"response.content_part.added", EventKind::kContentPartAdded},
{"response.content_part.done", EventKind::kContentPartDone},
{"response.output_text.delta", EventKind::kOutputTextDelta},
{"response.text.delta", EventKind::kOutputTextDelta}, // xAI: identical to output_text.delta
{"response.output_text.done", EventKind::kOutputTextDone},
{"response.refusal.delta", EventKind::kRefusalDelta},
{"response.refusal.done", EventKind::kRefusalDone},
{"response.output_text.annotation.added", EventKind::kAnnotationAdded},
{"response.reasoning_text.delta", EventKind::kReasoningTextDelta},
{"response.reasoning_text.done", EventKind::kReasoningTextDone},
{"response.reasoning_summary_text.delta", EventKind::kReasoningSummaryTextDelta},
{"response.reasoning_summary_text.done", EventKind::kReasoningSummaryTextDone},
{"response.reasoning_summary_part.added", EventKind::kReasoningSummaryPartAdded},
{"response.reasoning_summary_part.done", EventKind::kReasoningSummaryPartDone},
{"response.function_call_arguments.delta", EventKind::kFunctionCallArgumentsDelta},
{"response.function_call_arguments.done", EventKind::kFunctionCallArgumentsDone},
{"response.mcp_call_arguments.delta", EventKind::kMcpCallArgumentsDelta},
{"response.mcp_call_arguments.done", EventKind::kMcpCallArgumentsDone},
{"response.mcp_call.in_progress", EventKind::kMcpCallInProgress},
{"response.mcp_call.completed", EventKind::kMcpCallCompleted},
{"response.mcp_call.failed", EventKind::kMcpCallFailed},
{"response.mcp_list_tools.in_progress", EventKind::kMcpListToolsInProgress},
{"response.mcp_list_tools.completed", EventKind::kMcpListToolsCompleted},
{"response.mcp_list_tools.failed", EventKind::kMcpListToolsFailed},
{"response.web_search_call.in_progress", EventKind::kWebSearchInProgress},
{"response.web_search_call.searching", EventKind::kWebSearchSearching},
{"response.web_search_call.completed", EventKind::kWebSearchCompleted},
{"response.file_search_call.in_progress", EventKind::kFileSearchInProgress},
{"response.file_search_call.searching", EventKind::kFileSearchSearching},
{"response.file_search_call.completed", EventKind::kFileSearchCompleted},
{"response.code_interpreter_call.in_progress", EventKind::kCodeInterpreterInProgress},
{"response.code_interpreter_call.interpreting", EventKind::kCodeInterpreterInterpreting},
{"response.code_interpreter_call.completed", EventKind::kCodeInterpreterCompleted},
{"response.code_interpreter_call_code.delta", EventKind::kCodeInterpreterCodeDelta},
{"response.code_interpreter_call_code.done", EventKind::kCodeInterpreterCodeDone},
{"response.image_generation_call.in_progress", EventKind::kImageGenerationInProgress},
{"response.image_generation_call.generating", EventKind::kImageGenerationGenerating},
{"response.image_generation_call.completed", EventKind::kImageGenerationCompleted},
{"response.custom_tool_call_input.delta", EventKind::kCustomToolInputDelta},
{"response.custom_tool_call_input.done", EventKind::kCustomToolInputDone},
};
bool is_tool_status(EventKind k) {
switch (k) {
case EventKind::kMcpCallInProgress: case EventKind::kMcpCallCompleted:
case EventKind::kMcpCallFailed: case EventKind::kWebSearchInProgress:
case EventKind::kWebSearchSearching: case EventKind::kWebSearchCompleted:
case EventKind::kFileSearchInProgress: case EventKind::kFileSearchSearching:
case EventKind::kFileSearchCompleted: case EventKind::kCodeInterpreterInProgress:
case EventKind::kCodeInterpreterInterpreting: case EventKind::kCodeInterpreterCompleted:
case EventKind::kImageGenerationInProgress: case EventKind::kImageGenerationGenerating:
case EventKind::kImageGenerationCompleted:
return true;
default:
return false;
}
}
} // namespace
EventKind event_kind_of(const std::string& type) {
for (const Row& r : kRows)
if (type == r.type) return r.kind;
return EventKind::kUnknown;
}
const char* event_type_of(EventKind kind) {
for (const Row& r : kRows)
if (r.kind == kind) return r.type;
return "";
}
ResponseEvent ResponseEvent::parse(const llm::SseEvent& event) {
ResponseEvent e;
e.raw = event.data;
e.type = jx::str(event.data, "type");
if (e.type.empty()) e.type = event.event;
e.kind = event_kind_of(e.type);
e.sequence = jx::num(event.data, "sequence_number", -1);
e.output_index = jx::num(event.data, "output_index", -1);
e.content_index = jx::num(event.data, "content_index", -1);
e.item_id = jx::str(event.data, "item_id");
e.delta = jx::str(event.data, "delta");
e.text = jx::str(event.data, "text");
if (e.text.empty()) e.text = jx::str(event.data, "arguments");
if (e.text.empty()) e.text = jx::str(event.data, "code");
if (const Json* item = jx::get(event.data, "item")) e.item = *item;
if (const Json* resp = jx::get(event.data, "response")) e.response = *resp;
if (e.item_id.empty() && e.item.is_object()) e.item_id = jx::str(e.item, "id");
return e;
}
bool ResponseAccumulator::feed(const ResponseEvent& e) {
if (stopped_) return false;
if (++events_ > static_cast<long>(Caps::kMaxStreamEvents)) {
stopped_ = true;
stop_reason_ = "more than the stream event cap";
return false;
}
auto grow = [&](std::string& target, const std::string& add) {
if (target.size() + add.size() > Caps::kMaxStreamTextBytes) {
stopped_ = true;
stop_reason_ = "stream text over the byte cap";
return;
}
target += add;
};
switch (e.kind) {
case EventKind::kOutputTextDelta: grow(text_, e.delta); break;
case EventKind::kReasoningTextDelta: grow(reasoning_, e.delta); break;
case EventKind::kReasoningSummaryTextDelta: grow(summary_, e.delta); break;
case EventKind::kFunctionCallArgumentsDelta:
case EventKind::kMcpCallArgumentsDelta:
case EventKind::kCustomToolInputDelta:
if (!e.item_id.empty()) {
if (arguments_.size() >= Caps::kMaxAccumulatedItems && !arguments_.count(e.item_id)) {
stopped_ = true;
stop_reason_ = "more than the accumulated item cap";
break;
}
grow(arguments_[e.item_id], e.delta);
}
break;
case EventKind::kFunctionCallArgumentsDone:
case EventKind::kMcpCallArgumentsDone:
case EventKind::kCustomToolInputDone:
if (!e.item_id.empty() && !e.text.empty()) arguments_[e.item_id] = e.text;
break;
case EventKind::kOutputItemDone:
if (e.item.is_object()) {
if (items_.size() >= Caps::kMaxAccumulatedItems) {
stopped_ = true;
stop_reason_ = "more than the accumulated item cap";
break;
}
items_.push_back(OutputItem::parse(e.item));
}
break;
case EventKind::kCompleted:
case EventKind::kIncomplete:
if (e.response.is_object()) final_ = ModelResponse::parse(e.response);
break;
case EventKind::kFailed:
failed_ = true;
if (e.response.is_object()) {
final_ = ModelResponse::parse(e.response);
if (final_->error()) error_message_ = final_->error()->message;
}
break;
case EventKind::kError:
failed_ = true;
error_message_ = jx::str(e.raw, "message");
if (error_message_.empty()) error_message_ = jx::str(jx::object(e.raw, "error"), "message");
break;
default:
if (is_tool_status(e.kind) && !e.item_id.empty()) {
const std::string t = e.type;
tool_status_[e.item_id] = t.substr(t.rfind('.') + 1);
}
}
return !stopped_ && !e.terminal();
}
std::vector<std::string> ResponseAccumulator::images() const {
std::vector<std::string> out;
for (const auto& i : items_)
if (i.kind() == ItemKind::kImageGenerationCall && !i.image_base64().empty())
out.push_back(i.image_base64());
return out;
}
} // namespace molo::xai::core
+103
View File
@@ -0,0 +1,103 @@
#pragma once
// Streaming events of the Responses API (`stream: true`). xAI documents the
// stream as OpenAI-Responses compatible and names these events in its guides:
// response.created / completed / incomplete, response.output_text.delta,
// response.reasoning_text.delta, response.reasoning_summary_text.delta,
// response.output_item.added / done, response.function_call_arguments.*,
// response.mcp_call*, response.image_generation_call.*. The remaining kinds
// follow the OpenAI event vocabulary xAI declares compatibility with and are
// listed under Unverified in the SPEC. Anything else decodes as kUnknown and
// stays reachable through raw().
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "llm/sse.h"
#include "xai/core/params.h"
#include "xai/core/response.h"
namespace molo::xai::core {
enum class EventKind {
kCreated, kInProgress, kCompleted, kIncomplete, kFailed, kError,
kOutputItemAdded, kOutputItemDone, kContentPartAdded, kContentPartDone,
kOutputTextDelta, kOutputTextDone, kRefusalDelta, kRefusalDone, kAnnotationAdded,
kReasoningTextDelta, kReasoningTextDone, kReasoningSummaryTextDelta, kReasoningSummaryTextDone,
kReasoningSummaryPartAdded, kReasoningSummaryPartDone,
kFunctionCallArgumentsDelta, kFunctionCallArgumentsDone,
kMcpCallArgumentsDelta, kMcpCallArgumentsDone, kMcpCallInProgress, kMcpCallCompleted,
kMcpCallFailed, kMcpListToolsInProgress, kMcpListToolsCompleted, kMcpListToolsFailed,
kWebSearchInProgress, kWebSearchSearching, kWebSearchCompleted,
kFileSearchInProgress, kFileSearchSearching, kFileSearchCompleted,
kCodeInterpreterInProgress, kCodeInterpreterInterpreting, kCodeInterpreterCompleted,
kCodeInterpreterCodeDelta, kCodeInterpreterCodeDone,
kImageGenerationInProgress, kImageGenerationGenerating, kImageGenerationCompleted,
kCustomToolInputDelta, kCustomToolInputDone,
kUnknown
};
EventKind event_kind_of(const std::string& type);
const char* event_type_of(EventKind kind); // wire name, "" for kUnknown
struct ResponseEvent {
EventKind kind = EventKind::kUnknown;
std::string type; // wire type string
long sequence = -1; // sequence_number
long output_index = -1;
long content_index = -1;
std::string item_id;
std::string delta; // *.delta
std::string text; // *.done (full text / arguments)
Json item; // output_item.* (null otherwise)
Json response; // created / completed / incomplete / failed
Json raw;
static ResponseEvent parse(const llm::SseEvent& event);
bool terminal() const {
return kind == EventKind::kCompleted || kind == EventKind::kIncomplete ||
kind == EventKind::kFailed || kind == EventKind::kError;
}
};
// Folds events into the final picture: assembled text, reasoning, tool call
// arguments, finished items, final response and usage. Bounded: text over
// Caps::kMaxStreamTextBytes, more than Caps::kMaxStreamEvents events or more
// than Caps::kMaxAccumulatedItems items stop the fold (stopped() + reason()).
class ResponseAccumulator {
public:
// Returns false once a cap was hit or a terminal event was folded.
bool feed(const ResponseEvent& event);
const std::string& text() const { return text_; }
const std::string& reasoning() const { return reasoning_; }
const std::string& summary() const { return summary_; }
// Finished output items by arrival; complete function calls are included.
const std::vector<OutputItem>& items() const { return items_; }
// Argument text assembled so far per item id (also for unfinished calls).
const std::map<std::string, std::string>& arguments() const { return arguments_; }
const std::optional<ModelResponse>& final_response() const { return final_; }
long events() const { return events_; }
bool stopped() const { return stopped_; }
const std::string& stop_reason() const { return stop_reason_; }
std::string error_message() const { return error_message_; }
bool failed() const { return failed_; }
// Progress statuses of server-side tools: item id -> last status event.
const std::map<std::string, std::string>& tool_status() const { return tool_status_; }
// Base64 image payloads from finished image_generation_call items.
std::vector<std::string> images() const;
private:
std::string text_, reasoning_, summary_, stop_reason_, error_message_;
std::vector<OutputItem> items_;
std::map<std::string, std::string> arguments_;
std::map<std::string, std::string> tool_status_;
std::optional<ModelResponse> final_;
long events_ = 0;
bool stopped_ = false;
bool failed_ = false;
};
} // namespace molo::xai::core
+231
View File
@@ -0,0 +1,231 @@
#include "xai/core/tools.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
void check_handle(const std::string& h) {
if (h.empty() || h.size() > 64) Check::fail("x handle must be 1..64 characters");
for (char c : h) {
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_';
if (!ok) Check::fail("x handle has an invalid character (give it without @)");
}
}
void check_domain(const std::string& d) {
if (d.empty() || d.size() > 253) Check::fail("domain must be 1..253 characters");
if (d.find("://") != std::string::npos || d.find('/') != std::string::npos)
Check::fail("domain is given without protocol or path: " + d);
}
Json make_function(const std::string& name, const std::string& description, const Json& parameters,
const std::optional<bool>& strict, const std::optional<bool>& defer) {
Check::id("function", name);
if (name.size() > 64) Check::fail("function name over 64 characters");
Check::text("function description", description, 16u * 1024u, true);
Check::json_schema("function parameters", parameters);
Json f = Json::object();
f.set("name", Json(name));
if (!description.empty()) f.set("description", Json(description));
f.set("parameters", parameters);
jx::put(f, "strict", strict);
jx::put(f, "defer_loading", defer);
return f;
}
} // namespace
FunctionTool::FunctionTool(std::string name, std::string description, Json parameters)
: name_(std::move(name)), description_(std::move(description)), parameters_(std::move(parameters)) {}
Json FunctionTool::to_json() const {
Json f = make_function(name_, description_, parameters_, strict_, defer_loading_);
f.set("type", Json("function"));
return f;
}
Json FunctionTool::to_chat_json() const {
Json t = Json::object();
t.set("type", Json("function"));
t.set("function", make_function(name_, description_, parameters_, strict_, defer_loading_));
return t;
}
Json WebSearchTool::to_json() const {
if (!allowed_.empty() && !excluded_.empty())
Check::fail("web search allow_domains and exclude_domains are mutually exclusive");
Check::upto("allowed_domains", allowed_.size(), Caps::kMaxDomains);
Check::upto("excluded_domains", excluded_.size(), Caps::kMaxDomains);
for (const auto& d : allowed_) check_domain(d);
for (const auto& d : excluded_) check_domain(d);
Json t = Json::object();
t.set("type", Json("web_search"));
jx::put_strings(t, "allowed_domains", allowed_);
jx::put_strings(t, "excluded_domains", excluded_);
jx::put(t, "enable_image_search", image_search_);
jx::put(t, "enable_image_understanding", image_understanding_);
if (location_) {
const UserLocation& l = *location_;
if (!l.country.empty() && l.country.size() != 2) Check::fail("country must be a 2-letter ISO code");
Json loc = Json::object();
loc.set("type", Json("approximate"));
if (!l.country.empty()) loc.set("country", Json(l.country));
if (!l.city.empty()) loc.set("city", Json(l.city));
if (!l.region.empty()) loc.set("region", Json(l.region));
if (!l.timezone.empty()) loc.set("timezone", Json(l.timezone));
t.set("user_location", loc);
}
return t;
}
Json XSearchTool::to_json() const {
if (!allowed_.empty() && !excluded_.empty())
Check::fail("x search allow_handles and exclude_handles are mutually exclusive");
Check::upto("allowed_x_handles", allowed_.size(), Caps::kMaxXHandles);
Check::upto("excluded_x_handles", excluded_.size(), Caps::kMaxXHandles);
for (const auto& h : allowed_) check_handle(h);
for (const auto& h : excluded_) check_handle(h);
if (!from_.empty()) Check::date("from_date", from_);
if (!to_.empty()) Check::date("to_date", to_);
if (!from_.empty() && !to_.empty() && from_ > to_) Check::fail("from_date is after to_date");
Json t = Json::object();
t.set("type", Json("x_search"));
jx::put_strings(t, "allowed_x_handles", allowed_);
jx::put_strings(t, "excluded_x_handles", excluded_);
if (!from_.empty()) t.set("from_date", Json(from_));
if (!to_.empty()) t.set("to_date", Json(to_));
jx::put(t, "enable_image_understanding", image_);
jx::put(t, "enable_video_understanding", video_);
return t;
}
Json CodeExecutionTool::to_json() const {
Json t = Json::object();
t.set("type", Json("code_interpreter"));
if (!container_.empty()) {
Check::id("container", container_);
t.set("container", Json(container_));
}
return t;
}
Json CollectionsSearchTool::to_json() const {
Check::count("collection ids", ids_.size(), 1, Caps::kMaxCollections);
for (const auto& id : ids_) Check::id("collection", id);
Json t = Json::object();
t.set("type", Json("file_search"));
t.set("vector_store_ids", jx::string_array(ids_));
if (max_results_) {
Check::range("max_num_results", *max_results_, 1, 100);
t.set("max_num_results", Json(*max_results_));
}
return t;
}
McpTool& McpTool::header(std::string name, std::string value) {
headers_.emplace_back(std::move(name), std::move(value));
return *this;
}
Json McpTool::to_json() const {
Check::token("server_label", label_);
Check::http_url("server_url", url_);
Check::text("server_description", description_, 4096, true);
Check::strings("allowed_tools", allowed_, Caps::kMaxAllowedTools, 128);
Check::upto("headers", headers_.size(), Caps::kMaxMcpHeaders);
Json t = Json::object();
t.set("type", Json("mcp"));
t.set("server_label", Json(label_));
t.set("server_url", Json(url_));
if (!description_.empty()) t.set("server_description", Json(description_));
jx::put_strings(t, "allowed_tools", allowed_);
if (!authorization_.empty()) {
Check::text("authorization", authorization_, 8192);
t.set("authorization", Json(authorization_));
}
if (!headers_.empty()) {
Json h = Json::object();
for (const auto& [k, v] : headers_) {
Check::token("header name", k);
Check::text("header value", v, 8192, true);
h.set(k, Json(v));
}
t.set("headers", h);
}
jx::put(t, "defer_loading", defer_loading_);
return t;
}
Json ImageGenerationTool::to_json() const {
Json t = Json::object();
t.set("type", Json("image_generation"));
if (!action_.empty()) {
Check::one_of("action", action_, {"auto", "generate", "edit"});
t.set("action", Json(action_));
}
return t;
}
Json ShellTool::to_json() const {
Json env = Json::object();
env.set("type", Json("local"));
if (!skills_.empty()) {
Check::upto("skills", skills_.size(), 64);
Json arr = Json::array();
for (const auto& s : skills_) {
Check::token("skill name", s.name);
Check::text("skill description", s.description, 4096);
Check::text("skill path", s.path, 4096);
Json o = Json::object();
o.set("name", Json(s.name));
o.set("description", Json(s.description));
o.set("path", Json(s.path));
arr.push(o);
}
env.set("skills", arr);
}
Json t = Json::object();
t.set("type", Json("shell"));
t.set("environment", env);
return t;
}
Json ToolSearchTool::to_json() const {
Json t = Json::object();
t.set("type", Json("tool_search"));
t.set("execution", Json("server"));
return t;
}
ToolChoice ToolChoice::function(const std::string& name) {
Check::id("function", name);
return ToolChoice("function", name);
}
std::optional<ToolChoice> ToolChoice::from_neutral(const std::string& value) {
if (value.empty()) return std::nullopt;
if (value == "auto") return automatic();
if (value == "none") return none();
if (value == "required") return required();
return function(value);
}
Json ToolChoice::to_responses() const {
if (mode_ != "function") return Json(mode_);
Json o = Json::object();
o.set("type", Json("function"));
o.set("name", Json(name_));
return o;
}
Json ToolChoice::to_chat() const {
if (mode_ != "function") return Json(mode_);
Json fn = Json::object();
fn.set("name", Json(name_));
Json o = Json::object();
o.set("type", Json("function"));
o.set("function", fn);
return o;
}
} // namespace molo::xai::core
+194
View File
@@ -0,0 +1,194 @@
#pragma once
// Tool definitions for the Responses API: client-side function calling and
// the server-side tools xAI runs itself (web search, X search, code
// execution, collections search, remote MCP, image generation, shell, tool
// search). One class per tool; to_json() validates and returns the wire
// object. The spec's `ModelTool` variants and the TOOLS table in calls.cpp
// are checked by tools/check_spec.py.
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "xai/core/params.h"
namespace molo::xai::core {
class Tool {
public:
virtual ~Tool() = default;
virtual const char* type() const = 0;
virtual Json to_json() const = 0;
};
// Client-side function. Responses wire: {type, name, description, parameters};
// chat completions wire: {type, function:{...}} (to_chat_json()).
class FunctionTool : public Tool {
public:
FunctionTool(std::string name, std::string description, Json parameters);
const char* type() const override { return "function"; }
Json to_json() const override;
Json to_chat_json() const;
FunctionTool& strict(bool value) { strict_ = value; return *this; }
// Hide the definition from the prompt; the model loads it through a
// tool_search step (spec: FunctionDefinition.defer_loading).
FunctionTool& defer_loading(bool value) { defer_loading_ = value; return *this; }
const std::string& name() const { return name_; }
private:
std::string name_;
std::string description_;
Json parameters_;
std::optional<bool> strict_;
std::optional<bool> defer_loading_;
};
struct UserLocation {
std::string country; // ISO 3166-1 alpha-2
std::string city;
std::string region;
std::string timezone; // IANA
};
class WebSearchTool : public Tool {
public:
const char* type() const override { return "web_search"; }
Json to_json() const override;
// At most 5 each, and not both (spec).
WebSearchTool& allow_domains(std::vector<std::string> d) { allowed_ = std::move(d); return *this; }
WebSearchTool& exclude_domains(std::vector<std::string> d) { excluded_ = std::move(d); return *this; }
WebSearchTool& image_search(bool v) { image_search_ = v; return *this; }
WebSearchTool& image_understanding(bool v) { image_understanding_ = v; return *this; }
WebSearchTool& location(UserLocation l) { location_ = std::move(l); return *this; }
private:
std::vector<std::string> allowed_, excluded_;
std::optional<bool> image_search_, image_understanding_;
std::optional<UserLocation> location_;
};
class XSearchTool : public Tool {
public:
const char* type() const override { return "x_search"; }
Json to_json() const override;
// Handles without the leading @; at most 10 each (docs), not both (spec).
XSearchTool& allow_handles(std::vector<std::string> h) { allowed_ = std::move(h); return *this; }
XSearchTool& exclude_handles(std::vector<std::string> h) { excluded_ = std::move(h); return *this; }
XSearchTool& from_date(std::string d) { from_ = std::move(d); return *this; } // YYYY-MM-DD
XSearchTool& to_date(std::string d) { to_ = std::move(d); return *this; }
XSearchTool& image_understanding(bool v) { image_ = v; return *this; }
XSearchTool& video_understanding(bool v) { video_ = v; return *this; }
private:
std::vector<std::string> allowed_, excluded_;
std::string from_, to_;
std::optional<bool> image_, video_;
};
// Code execution (wire type `code_interpreter`).
class CodeExecutionTool : public Tool {
public:
const char* type() const override { return "code_interpreter"; }
Json to_json() const override;
// Container id; the spec allows an id or an object, the guides show neither
// (SPEC Unverified), so only the id form is offered.
CodeExecutionTool& container(std::string id) { container_ = std::move(id); return *this; }
private:
std::string container_;
};
// Collections search (wire type `file_search`, vector_store_ids are
// collection ids).
class CollectionsSearchTool : public Tool {
public:
explicit CollectionsSearchTool(std::vector<std::string> collection_ids)
: ids_(std::move(collection_ids)) {}
const char* type() const override { return "file_search"; }
Json to_json() const override;
CollectionsSearchTool& max_results(int n) { max_results_ = n; return *this; }
private:
std::vector<std::string> ids_;
std::optional<int> max_results_;
};
class McpTool : public Tool {
public:
McpTool(std::string server_label, std::string server_url)
: label_(std::move(server_label)), url_(std::move(server_url)) {}
const char* type() const override { return "mcp"; }
Json to_json() const override;
McpTool& description(std::string d) { description_ = std::move(d); return *this; }
McpTool& allow_tools(std::vector<std::string> t) { allowed_ = std::move(t); return *this; }
// Sent as the Authorization header value toward the MCP server. Never logged.
McpTool& authorization(std::string a) { authorization_ = std::move(a); return *this; }
McpTool& header(std::string name, std::string value);
McpTool& defer_loading(bool v) { defer_loading_ = v; return *this; }
private:
std::string label_, url_, description_, authorization_;
std::vector<std::string> allowed_;
std::vector<std::pair<std::string, std::string>> headers_;
std::optional<bool> defer_loading_;
};
class ImageGenerationTool : public Tool {
public:
const char* type() const override { return "image_generation"; }
Json to_json() const override;
// "auto" (default), "generate" or "edit".
ImageGenerationTool& action(std::string a) { action_ = std::move(a); return *this; }
private:
std::string action_;
};
struct ShellSkill {
std::string name, description, path;
};
// Client-executed shell: the model emits shell_call items, the caller runs
// them and answers with shell_call_output.
class ShellTool : public Tool {
public:
const char* type() const override { return "shell"; }
Json to_json() const override;
ShellTool& skill(ShellSkill s) { skills_.push_back(std::move(s)); return *this; }
private:
std::vector<ShellSkill> skills_;
};
class ToolSearchTool : public Tool {
public:
const char* type() const override { return "tool_search"; }
Json to_json() const override; // execution:"server" (the only supported value)
};
// tool_choice: "auto", "none", "required" or one named function.
class ToolChoice {
public:
static ToolChoice automatic() { return ToolChoice("auto", ""); }
static ToolChoice none() { return ToolChoice("none", ""); }
static ToolChoice required() { return ToolChoice("required", ""); }
static ToolChoice function(const std::string& name);
// From the neutral string: "", "auto", "none", "required" or a function name.
static std::optional<ToolChoice> from_neutral(const std::string& value);
Json to_responses() const; // "auto" | {type:"function", name}
Json to_chat() const; // "auto" | {type:"function", function:{name}}
private:
ToolChoice(std::string mode, std::string name) : mode_(std::move(mode)), name_(std::move(name)) {}
std::string mode_;
std::string name_;
};
} // namespace molo::xai::core
+189
View File
@@ -0,0 +1,189 @@
#!/usr/bin/env python3
"""Check src/xai/core/calls.cpp against xAI's OpenAPI document and REST reference.
Fails (exit 1) when a declared call is absent or differs (HTTP method, path,
operationId, query parameter, request body field, request / response schema),
when a schema field or tool field we use is not in the document, when a
batch (docs-only) row or a documented-only field is not on its documentation
page, or when the document has an operation that is neither in the table nor
in the explicit exclusion list (owned by the xAI platform package).
Python 3 stdlib only, no credentials needed.
python3 -I src/xai/core/tools/check_spec.py # fetch live
python3 -I src/xai/core/tools/check_spec.py --file spec.json --no-docs # offline
"""
import hashlib
import json
import os
import re
import sys
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
CALLS = os.path.join(HERE, "..", "calls.cpp")
SPEC_URL = "https://api.x.ai/api-docs/openapi.json"
DOCS = "https://docs.x.ai/developers/%s.md"
BATCH_PAGE = "rest-api-reference/inference/batches"
# Operations of the document that belong to src/xai/platform (files, collections search).
EXCLUDED_OPS = {
"handle_list_files_request", "handle_upload_file_request", "handle_retrieve_file_request",
"handle_delete_file_request", "handle_download_file_content_request",
"handle_create_public_url_request", "handle_revoke_public_url_request",
"handle_document_search_request_v2",
}
ROW = re.compile(r'^\s*\{(.*)\},?\s*$')
STR = re.compile(r'"([^"]*)"')
def block(text, begin, end):
a, b = text.index(begin), text.index(end)
rows = []
for line in text[a:b].splitlines()[1:]:
m = ROW.match(line)
if m and '"' in line:
rows.append(STR.findall(m.group(1)))
return rows
def fetch(url):
with urllib.request.urlopen(url, timeout=60) as r:
return r.read().decode("utf-8")
def resolve(spec, node):
while isinstance(node, dict) and "$ref" in node:
node = spec["components"]["schemas"][node["$ref"].split("/")[-1]]
return node
def props(spec, node, depth=0):
"""All property names of a schema, through $ref / allOf / oneOf / anyOf."""
out = {}
node = resolve(spec, node)
if not isinstance(node, dict) or depth > 8:
return out
out.update(node.get("properties", {}))
for key in ("allOf", "oneOf", "anyOf"):
for sub in node.get(key, []):
out.update(props(spec, sub, depth + 1))
return out
def ref_name(node):
return node["$ref"].split("/")[-1] if isinstance(node, dict) and "$ref" in node else ""
def tool_variants(spec):
"""type const -> property names, for every ModelTool variant."""
out = {}
for variant in spec["components"]["schemas"]["ModelTool"]["oneOf"]:
p = props(spec, variant)
t = p.get("type", {}).get("enum", [None])[0]
if t:
out[t] = p
return out
def main():
argv = sys.argv[1:]
path = argv[argv.index("--file") + 1] if "--file" in argv else None
docs_on = "--no-docs" not in argv
raw = open(path).read() if path else fetch(SPEC_URL)
spec = json.loads(raw)
text = open(CALLS).read()
calls = block(text, "// CALLS-BEGIN", "// CALLS-END")
schemas = block(text, "// SCHEMAS-BEGIN", "// SCHEMAS-END")
tools = block(text, "// TOOLS-BEGIN", "// TOOLS-END")
S = spec["components"]["schemas"]
ops = {}
for p, item in spec["paths"].items():
for m, o in item.items():
if m in ("get", "post", "put", "delete", "patch"):
ops[(m.upper(), p)] = o
bad, covered = [], set()
docs_cache = {}
def page(name):
if name not in docs_cache:
docs_cache[name] = fetch(DOCS % name)
return docs_cache[name]
for method, p, op_id, body, query, doc_fields, req, resp, source in calls:
if source == "openapi":
o = ops.get((method, p))
if not o:
bad.append("missing operation %s %s" % (method, p))
continue
covered.add(o["operationId"])
if o["operationId"] != op_id:
bad.append("%s %s: operationId %s != %s" % (method, p, op_id, o["operationId"]))
have_q = {x["name"] for x in o.get("parameters", []) if x["in"] == "query"}
for q in filter(None, query.split(",")):
if q not in have_q:
bad.append("%s %s: query param %s absent" % (method, p, q))
content = o.get("requestBody", {}).get("content", {})
rschema = next(iter(content.values()), {}).get("schema") if content else None
if rschema is None and (body or req):
bad.append("%s %s: no request body in spec" % (method, p))
if rschema is not None:
if ref_name(rschema) != req:
bad.append("%s %s: request schema %s != %s" % (method, p, req, ref_name(rschema)))
pr = props(spec, rschema)
for f in filter(None, body.split(",")):
if f not in pr:
bad.append("%s %s: body field %s absent in %s" % (method, p, f, req))
ok = o["responses"].get("200", {}).get("content", {})
got = ref_name(next(iter(ok.values()), {}).get("schema", {})) or "string"
if got != resp:
bad.append("%s %s: response schema %s != %s" % (method, p, resp, got))
elif docs_on:
md = page(BATCH_PAGE).replace("\\", "")
head = "## %s %s" % (method, p)
if head not in md:
bad.append("docs: heading %r absent from the batches reference" % head)
continue
section = md.split(head, 1)[1].split("\n## ", 1)[0]
for f in filter(None, (body + "," + query).split(",")):
if "`%s`" % f not in section:
bad.append("docs: %s %s field %s absent" % (method, p, f))
if docs_on:
for entry in filter(None, doc_fields.split(",")):
f, _, pg = entry.partition("@")
if f not in page(pg):
bad.append("docs: field %s absent from %s" % (f, pg))
for o_key, o in ops.items():
if o["operationId"] not in covered and o["operationId"] not in EXCLUDED_OPS:
bad.append("unwrapped operation %s %s (%s)" % (o_key[0], o_key[1], o["operationId"]))
for name, fields in schemas:
if name not in S:
bad.append("missing schema " + name)
continue
pr = props(spec, S[name])
for f in fields.split(","):
if f not in pr:
bad.append("schema %s.%s absent" % (name, f))
variants = tool_variants(spec)
for t, fields in tools:
if t not in variants:
bad.append("tool type %s absent from ModelTool" % t)
continue
for f in fields.split(","):
if f not in variants[t]:
bad.append("tool %s field %s absent" % (t, f))
digest = hashlib.sha256(raw.encode()).hexdigest()[:16]
info = "spec %s v%s sha256:%s, %d operations" % (spec["info"]["title"], spec["info"]["version"], digest, len(ops))
if bad:
print("SPEC CHECK FAILED (%s)" % info)
for b in bad:
print(" " + b)
return 1
print("spec ok: %d calls (%d openapi, %d docs), %d schemas, %d tool types, %d platform operations excluded; %s"
% (len(calls), sum(1 for c in calls if c[8] == "openapi"), sum(1 for c in calls if c[8] == "docs"),
len(schemas), len(tools), len(EXCLUDED_OPS), info))
return 0
if __name__ == "__main__":
sys.exit(main())
+91
View File
@@ -0,0 +1,91 @@
#include "xai/core/usage.h"
namespace molo::xai::core {
ToolUsage ToolUsage::from_json(const Json& doc) {
ToolUsage t;
t.code_interpreter_calls = jx::num(doc, "code_interpreter_calls");
t.document_search_calls = jx::num(doc, "document_search_calls");
t.file_search_calls = jx::num(doc, "file_search_calls");
t.image_generation_calls = jx::num(doc, "image_generation_calls");
t.mcp_calls = jx::num(doc, "mcp_calls");
t.web_search_calls = jx::num(doc, "web_search_calls");
t.x_posts_fetched = jx::num(doc, "x_posts_fetched");
t.x_search_calls = jx::num(doc, "x_search_calls");
t.x_users_fetched = jx::num(doc, "x_users_fetched");
return t;
}
long ToolUsage::total_calls() const {
return code_interpreter_calls + document_search_calls + file_search_calls +
image_generation_calls + mcp_calls + web_search_calls + x_search_calls;
}
TokenUsage TokenUsage::from_chat(const Json& usage) {
TokenUsage u;
if (!usage.is_object()) return u;
u.present = true;
u.input_tokens = jx::num(usage, "prompt_tokens");
u.output_tokens = jx::num(usage, "completion_tokens");
u.total_tokens = jx::num(usage, "total_tokens", u.input_tokens + u.output_tokens);
Json prompt = jx::object(usage, "prompt_tokens_details");
u.cached_tokens = jx::num(prompt, "cached_tokens");
u.text_tokens = jx::num(prompt, "text_tokens");
u.image_tokens = jx::num(prompt, "image_tokens");
u.audio_tokens = jx::num(prompt, "audio_tokens");
u.reasoning_tokens = jx::num(jx::object(usage, "completion_tokens_details"), "reasoning_tokens");
u.num_sources_used = jx::num(usage, "num_sources_used");
u.cost_usd_ticks = jx::num(usage, "cost_in_usd_ticks");
u.tools = ToolUsage::from_json(jx::object(usage, "server_side_tool_usage_details"));
u.num_server_side_tools_used = u.tools.total_calls();
return u;
}
TokenUsage TokenUsage::from_responses(const Json& usage) {
TokenUsage u;
if (!usage.is_object()) return u;
u.present = true;
u.input_tokens = jx::num(usage, "input_tokens");
u.output_tokens = jx::num(usage, "output_tokens");
u.total_tokens = jx::num(usage, "total_tokens", u.input_tokens + u.output_tokens);
u.cached_tokens = jx::num(jx::object(usage, "input_tokens_details"), "cached_tokens");
u.reasoning_tokens = jx::num(jx::object(usage, "output_tokens_details"), "reasoning_tokens");
u.num_sources_used = jx::num(usage, "num_sources_used");
u.num_server_side_tools_used = jx::num(usage, "num_server_side_tools_used");
u.cost_usd_ticks = jx::num(usage, "cost_in_usd_ticks");
if (const Json* nano = jx::get(usage, "cost_in_nano_usd"); nano && nano->is_number())
u.cost_nano_usd = nano->as_int();
Json ctx = jx::object(usage, "context_details");
if (const Json* v = jx::get(ctx, "input_tokens"); v && v->is_number()) u.context_input_tokens = v->as_int();
if (const Json* v = jx::get(ctx, "output_tokens"); v && v->is_number()) u.context_output_tokens = v->as_int();
u.tools = ToolUsage::from_json(jx::object(usage, "server_side_tool_usage_details"));
return u;
}
llm::Usage TokenUsage::to_neutral() const {
llm::Usage n;
n.input_tokens = static_cast<int>(input_tokens);
n.output_tokens = static_cast<int>(output_tokens);
n.total_tokens = static_cast<int>(total_tokens);
n.cached_tokens = static_cast<int>(cached_tokens);
n.reasoning_tokens = static_cast<int>(reasoning_tokens);
return n;
}
MediaUsage MediaUsage::from_json(const Json& usage) {
MediaUsage m;
if (!usage.is_object()) return m;
m.present = true;
m.cost_usd_ticks = jx::num(usage, "cost_in_usd_ticks");
auto opt = [&](const char* k) -> std::optional<long> {
const Json* v = jx::get(usage, k);
if (v && v->is_number()) return v->as_int();
return std::nullopt;
};
m.input_tokens = opt("input_tokens");
m.output_tokens = opt("output_tokens");
m.total_tokens = opt("total_tokens");
return m;
}
} // namespace molo::xai::core
+69
View File
@@ -0,0 +1,69 @@
#pragma once
// Token, tool and cost accounting as xAI reports it. Chat completions use
// `prompt_tokens` / `completion_tokens`, the Responses API uses
// `input_tokens` / `output_tokens`; TokenUsage reads both into one shape.
// Cost is reported in "ticks": 1e10 ticks = 1 USD (spec: one USD cent equals
// 100,000,000 ticks).
#include <optional>
#include "llm/chat_types.h"
#include "xai/core/params.h"
namespace molo::xai::core {
constexpr double kTicksPerUsd = 1e10;
struct ToolUsage {
long code_interpreter_calls = 0;
long document_search_calls = 0;
long file_search_calls = 0;
long image_generation_calls = 0;
long mcp_calls = 0;
long web_search_calls = 0;
long x_posts_fetched = 0;
long x_search_calls = 0;
long x_users_fetched = 0;
static ToolUsage from_json(const Json& doc);
long total_calls() const;
};
struct TokenUsage {
long input_tokens = 0;
long output_tokens = 0;
long total_tokens = 0;
long cached_tokens = 0;
long reasoning_tokens = 0;
long text_tokens = 0;
long image_tokens = 0;
long audio_tokens = 0;
long num_sources_used = 0;
long num_server_side_tools_used = 0;
long cost_usd_ticks = 0;
std::optional<long> cost_nano_usd;
std::optional<long> context_input_tokens; // Responses: latest context window
std::optional<long> context_output_tokens;
ToolUsage tools;
bool present = false;
static TokenUsage from_chat(const Json& usage); // prompt_tokens / completion_tokens
static TokenUsage from_responses(const Json& usage); // input_tokens / output_tokens
double cost_usd() const { return static_cast<double>(cost_usd_ticks) / kTicksPerUsd; }
llm::Usage to_neutral() const;
};
// Image and video usage (cost plus optional token counts).
struct MediaUsage {
long cost_usd_ticks = 0;
std::optional<long> input_tokens;
std::optional<long> output_tokens;
std::optional<long> total_tokens;
bool present = false;
static MediaUsage from_json(const Json& usage);
double cost_usd() const { return static_cast<double>(cost_usd_ticks) / kTicksPerUsd; }
};
} // namespace molo::xai::core
+183
View File
@@ -0,0 +1,183 @@
#include "xai/core/videos.h"
#include "xai/core/caps.h"
namespace molo::xai::core {
namespace {
void write_common(Json& body, const std::string& model, const std::optional<StorageOptions>& storage,
const std::string& upload_url, const std::string& user) {
if (!model.empty()) {
Check::model(model);
body.set("model", Json(model));
}
if (storage) body.set("storage_options", storage->to_json());
if (!upload_url.empty()) {
Check::http_url("upload url", upload_url);
Json out = Json::object();
out.set("upload_url", Json(upload_url));
body.set("output", out);
}
if (!user.empty()) {
Check::text("user", user, 256);
body.set("user", Json(user));
}
}
std::string request_id_of(const Json& reply) {
std::string id = jx::str(reply, "request_id");
if (id.empty()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "reply has no request_id");
return id;
}
} // namespace
Json VideoGenerationRequest::to_json() const {
bool has_refs = !reference_images.empty() || !reference_audios.empty();
bool has_frames = image || last_frame || !keyframes.empty();
if (prompt.empty() && !has_refs && !has_frames) Check::fail("prompt is required without images or references");
Check::text("prompt", prompt, Caps::kMaxPromptBytes, true);
Json body = Json::object();
if (!prompt.empty()) body.set("prompt", Json(prompt));
if (duration) {
Check::range("duration", *duration, Caps::kMinVideoSeconds, Caps::kMaxVideoSeconds);
body.set("duration", Json(*duration));
}
if (!aspect_ratio.empty()) {
Check::one_of("aspect_ratio", aspect_ratio, {"1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3", "21:9", "5:2"});
body.set("aspect_ratio", Json(aspect_ratio));
}
if (!resolution.empty()) {
Check::one_of("resolution", resolution, {"480p", "720p", "1080p"});
body.set("resolution", Json(resolution));
}
if (image) body.set("image", image->to_json());
if (last_frame) body.set("last_frame", last_frame->to_json());
Check::upto("reference_images", reference_images.size(), Caps::kMaxReferenceImages);
if (!reference_images.empty()) {
Json arr = Json::array();
for (const auto& r : reference_images) arr.push(r.to_json());
body.set("reference_images", arr);
}
Check::upto("reference_audios", reference_audios.size(), Caps::kMaxReferenceAudios);
if (!reference_audios.empty()) {
Json arr = Json::array();
for (const auto& r : reference_audios) arr.push(r.to_json());
body.set("reference_audios", arr);
}
Check::upto("keyframes", keyframes.size(), Caps::kMaxKeyframes);
if (!keyframes.empty()) {
Json arr = Json::array();
for (const auto& k : keyframes) {
if (k.image.empty()) Check::fail("keyframe image is required");
double limit = duration ? *duration : Caps::kMaxVideoSeconds;
if (!(k.timestamp_s > 0 && k.timestamp_s < limit)) Check::fail("keyframe timestamp_s must be inside the clip");
Json o = Json::object();
o.set("image", k.image.to_json());
o.set("timestamp_s", Json(k.timestamp_s));
arr.push(o);
}
body.set("keyframes", arr);
}
write_common(body, model, storage, upload_url, user);
return body;
}
Json VideoEditRequest::to_json() const {
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
if (video.empty()) Check::fail("video is required");
Json body = Json::object();
body.set("prompt", Json(prompt));
body.set("video", video.to_json());
write_common(body, model, storage, upload_url, user);
return body;
}
Json VideoExtendRequest::to_json() const {
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
if (video.empty()) Check::fail("video is required");
Json body = Json::object();
body.set("prompt", Json(prompt));
body.set("video", video.to_json());
if (duration) {
Check::range("duration", *duration, Caps::kMinExtensionSeconds, Caps::kMaxExtensionSeconds);
body.set("duration", Json(*duration));
}
write_common(body, model, storage, upload_url, "");
return body;
}
VideoJob VideoJob::parse(const Json& doc, const std::string& request_id) {
VideoJob j;
j.request_id = request_id;
j.raw = doc;
j.status = jx::str(doc, "status");
j.model = jx::str(doc, "model");
if (const Json* p = jx::get(doc, "progress"); p && p->is_number()) j.progress = p->as_int();
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) j.usage = MediaUsage::from_json(*u);
j.error = MediaError::parse(doc, "error");
if (const Json* v = jx::get(doc, "video"); v && v->is_object()) {
GeneratedVideo g;
g.url = jx::str(*v, "url");
g.storage_error = jx::str(*v, "storage_error");
g.duration = jx::num(*v, "duration");
g.respect_moderation = jx::flag(*v, "respect_moderation");
if (const Json* f = jx::get(*v, "file_output"); f && f->is_object()) g.file = StoredFile::parse(*f);
j.video = std::move(g);
}
if (j.status.empty()) j.status = j.error ? "failed" : (j.video ? "done" : "pending");
return j;
}
std::string VideoService::start_generate(const VideoGenerationRequest& request) {
Json body = request.to_json();
return request_id_of(call_json(Call::kVideosGenerate, {}, &body));
}
std::string VideoService::start_edit(const VideoEditRequest& request) {
Json body = request.to_json();
return request_id_of(call_json(Call::kVideosEdit, {}, &body));
}
std::string VideoService::start_extend(const VideoExtendRequest& request) {
Json body = request.to_json();
return request_id_of(call_json(Call::kVideosExtend, {}, &body));
}
VideoJob VideoService::poll(const std::string& request_id) {
llm::RawResponse raw = call_raw(Call::kVideosGet, {request_id});
if (raw.status == 202) {
VideoJob j;
j.request_id = request_id;
j.status = "pending";
return j;
}
return VideoJob::parse(parse_body(raw), request_id);
}
VideoJob VideoService::wait(const std::string& request_id, const WaitPolicy& policy, const Sleeper& sleeper) {
return wait(request_id, policy, sleeper, Progress{});
}
VideoJob VideoService::wait(const std::string& request_id, const WaitPolicy& policy, const Sleeper& sleeper,
const Progress& progress) {
Check::id("request", request_id);
bool cancelled = false;
std::function<std::optional<VideoJob>()> step = [&]() -> std::optional<VideoJob> {
VideoJob j = poll(request_id);
if (progress && !progress(j)) {
cancelled = true;
return j;
}
if (j.pending()) return std::nullopt;
return j;
};
std::optional<VideoJob> j = poll_until<VideoJob>(policy, step, sleeper);
if (!j) throw llm::ApiError(kVendor, 504, "WAIT_TIMEOUT", "video not ready within the wait budget");
if (cancelled && j->pending()) throw llm::ApiError(kVendor, 499, "WAIT_CANCELLED", "wait stopped by the caller");
if (j->failed())
throw llm::ApiError(kVendor, 502, "GENERATION_FAILED",
j->error ? j->error->code + ": " + j->error->message : "video generation failed");
return *j;
}
} // namespace molo::xai::core
+100
View File
@@ -0,0 +1,100 @@
#pragma once
// Video generation (async): text-to-video, image-to-video, reference-to-video
// (/v1/videos/generations), editing (/v1/videos/edits), extension
// (/v1/videos/extensions), and the poll (GET /v1/videos/{request_id}).
// Every start call returns a request id immediately; the video is polled.
// Video costs real money per second of output; callers confirm before use.
#include <optional>
#include <string>
#include <vector>
#include "xai/core/media.h"
#include "xai/core/poll.h"
#include "xai/core/service.h"
namespace molo::xai::core {
struct VideoKeyframe {
MediaSource image;
double timestamp_s = 0; // strictly inside the clip: 0 < t < duration
};
struct VideoGenerationRequest {
std::string model;
std::string prompt; // required for text-to-video and reference-to-video
std::optional<int> duration; // seconds 1..15 (default 8)
std::string aspect_ratio; // 1:1 16:9 9:16 4:3 3:4 3:2 2:3 21:9 5:2
std::string resolution; // 480p | 720p | 1080p
std::optional<MediaSource> image; // first frame (image-to-video)
std::optional<MediaSource> last_frame; // documented in the guides, not in the OpenAPI document
std::vector<MediaSource> reference_images; // up to 14
std::vector<MediaSource> reference_audios; // up to 3 (preset voices)
std::vector<VideoKeyframe> keyframes; // up to 4
std::optional<StorageOptions> storage;
std::string upload_url; // output.upload_url: signed PUT target
std::string user;
Json to_json() const;
};
struct VideoEditRequest {
std::string model;
std::string prompt;
MediaSource video; // .mp4, up to 8.7 s per the guides
std::optional<StorageOptions> storage;
std::string upload_url;
std::string user;
Json to_json() const;
};
struct VideoExtendRequest {
std::string model;
std::string prompt;
MediaSource video;
std::optional<int> duration; // seconds 2..10 (default 6)
std::optional<StorageOptions> storage;
std::string upload_url;
Json to_json() const;
};
struct GeneratedVideo {
std::string url, storage_error;
long duration = 0;
bool respect_moderation = false;
std::optional<StoredFile> file;
};
struct VideoJob {
std::string request_id;
std::string status; // pending | done | failed (expired is possible; see SPEC Unverified)
std::string model;
std::optional<long> progress; // 0..100
std::optional<GeneratedVideo> video;
MediaUsage usage;
std::optional<MediaError> error;
Json raw;
bool done() const { return status == "done"; }
bool failed() const { return status == "failed" || error.has_value(); }
bool pending() const { return !done() && !failed(); }
static VideoJob parse(const Json& doc, const std::string& request_id);
};
class VideoService : public XaiService {
public:
explicit VideoService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
std::string start_generate(const VideoGenerationRequest& request);
std::string start_edit(const VideoEditRequest& request);
std::string start_extend(const VideoExtendRequest& request);
VideoJob poll(const std::string& request_id);
// Polls within the policy; throws 504 WAIT_TIMEOUT or 502 GENERATION_FAILED.
VideoJob wait(const std::string& request_id, const WaitPolicy& policy = {},
const Sleeper& sleeper = real_sleeper());
// Cooperative variant for UIs: reports every poll, return false to stop waiting.
using Progress = std::function<bool(const VideoJob&)>;
VideoJob wait(const std::string& request_id, const WaitPolicy& policy, const Sleeper& sleeper,
const Progress& progress);
};
} // namespace molo::xai::core