Merge xai core package
This commit is contained in:
@@ -0,0 +1,164 @@
|
||||
# xAI core (`molo::xai::core`)
|
||||
|
||||
Status 2026-10-24. Grok API platform inference host `https://api.x.ai`,
|
||||
credential `XAI_API_KEY` (bearer) read through `llm::Config`. Owned by plan
|
||||
17, package "xai core". Voice, collections, documents search, files and the
|
||||
management API belong to `src/xai/platform` and are not implemented here.
|
||||
|
||||
## Scope
|
||||
|
||||
Chat completions, the Responses API (create, stream, retrieve, delete, input
|
||||
items, compact), tools (function calling, web search, X search, code
|
||||
execution, collections search, remote MCP, image generation, shell, tool
|
||||
search) with their stream events, structured outputs, reasoning controls,
|
||||
Live Search parameters and guidelines, image generation and editing, video
|
||||
generation / editing / extension (async), embeddings, models (generic,
|
||||
language, image, video, embedding, with pricing and capability fields),
|
||||
tokenize-text, deferred completions, the Batch API, API key info and `/v1/me`,
|
||||
hosted Skills, the Anthropic-compatible messages endpoint, the legacy text
|
||||
completion endpoints, and `XaiChatModel : llm::ChatModel`.
|
||||
|
||||
## Module map
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | every static ceiling, marked (spec), (docs) or ours |
|
||||
| `params.*` | `Check` validators and `jx` tolerant JSON readers |
|
||||
| `calls.*` | the ONE call table (43 rows), `render`, `Query`, tool and schema parity tables |
|
||||
| `service.*` | `XaiService`: dispatch from the table, `Plan` (testable without network), body field check |
|
||||
| `usage.*` | `TokenUsage` (chat and Responses shapes), `ToolUsage`, `MediaUsage`, cost ticks |
|
||||
| `tools.*` | one class per tool, `ToolChoice` |
|
||||
| `response_format.*` | `ResponseFormat` (text, json_object, json_schema) and the schema lint |
|
||||
| `search_parameters.*` | Live Search `search_parameters`, `Guideline` |
|
||||
| `input.*` | Responses input items and parts |
|
||||
| `response.*` | `ModelResponse`, `OutputItem`, citations, server tool classification, replay items |
|
||||
| `stream_events.*` | `ResponseEvent` decoder, `ResponseAccumulator` |
|
||||
| `responses.*` | `ResponseRequest`, `ResponsesService` |
|
||||
| `chat.*` | chat completions request, response, chunks, accumulator, `ChatService` (deferred) |
|
||||
| `messages.*` | Anthropic-compatible `/v1/messages`, legacy `/v1/complete` and `/v1/completions` |
|
||||
| `media.*`, `images.*`, `videos.*` | media sources, storage options, image and video services |
|
||||
| `embeddings.*` | embeddings, base64 float decoding, tokenize |
|
||||
| `models.*` | `ModelInfo`, `ModelsService` |
|
||||
| `account.*` | `/v1/api-key`, `/v1/me`, Skills |
|
||||
| `batches.*` | Batch API |
|
||||
| `poll.*` | bounded polling (`WaitPolicy`, injectable sleeper) |
|
||||
| `chat_model.*` | `XaiChatModel` (Responses wire by default, chat completions optional) |
|
||||
| `client.*` | `XaiClient`: one `ApiClient`, every service |
|
||||
| `tools/check_spec.py` | spec checker |
|
||||
|
||||
## Contracts
|
||||
|
||||
- All validation (ids, ranges, counts, enums, URLs, dates, schema lint) runs
|
||||
before the credential check, so bad input is 422 in the demo with or
|
||||
without a key and never reaches xAI. Without a key every call throws
|
||||
`NotConfiguredError` before network traffic.
|
||||
- `XaiService::plan` rejects any top-level body field that the call table does
|
||||
not declare. `llm::ChatRequest::extra` is merged and then checked the same
|
||||
way: an unknown field is refused, never silently sent.
|
||||
- Destructive calls are explicit methods: `ResponsesService::remove`,
|
||||
`SkillsService::remove`, `BatchService::cancel`. Image, video and batch
|
||||
submission cost money; the demo makes them dry runs unless `confirm:true`.
|
||||
- Lists report truncation (`truncated`) when a cap stops them; page walks are
|
||||
bounded by `llm::Caps::kMaxPages` or `Caps::kMaxBatchPagesWalked`.
|
||||
- Waits (`wait_deferred`, `ImageService::wait`, `VideoService::wait`) take a
|
||||
`WaitPolicy` (interval 1..60 s, total at most 30 min) and fail with 504
|
||||
`WAIT_TIMEOUT`; a failed job is 502 `GENERATION_FAILED`.
|
||||
- Deferred chat results are fetchable once within 24 h (docs); `poll_deferred`
|
||||
returns `nullopt` on 202.
|
||||
- `XaiChatModel` sends `store=false` on the Responses wire, refuses `stop`
|
||||
there (not in the Responses schema), and maps reasoning, tool calls, images,
|
||||
JSON mode and `response_schema`.
|
||||
- Cost is `usage.cost_in_usd_ticks`; 1e10 ticks = 1 USD.
|
||||
|
||||
## Excluded fields
|
||||
|
||||
Sent by no builder because the spec marks them (Unsupported) or "only
|
||||
maintained for compatibility": Responses `background`, `metadata`,
|
||||
`truncation`, `context_management`; chat `logit_bias`, `web_search_options`;
|
||||
legacy completions `best_of`, `frequency_penalty`, `logit_bias`, `suffix`;
|
||||
messages `top_k`; embeddings `preview`; web search tool `external_web_access`
|
||||
and `search_context_size` (the request is rejected when set); MCP
|
||||
`require_approval` and `connector_id` (docs: not supported).
|
||||
|
||||
## Sources
|
||||
|
||||
1. OpenAPI 3.1 document `https://api.x.ai/api-docs/openapi.json` (identical
|
||||
bytes at `https://docs.x.ai/openapi.json`): 39 paths, 44 operations, 203
|
||||
schemas, sha256 prefix `f34f5c20564fefd1`.
|
||||
2. REST reference markdown pages `docs.x.ai/developers/rest-api-reference/inference/{responses,chat-completions,images,videos,batches,models,other,legacy}.md`.
|
||||
3. Guides: `advanced-api-usage/{batch-api,deferred-chat-completions}`, `model-capabilities/{text/*,images/*,video/*}`, `tools/*`, `models`, `pricing`; `docs.x.ai/llms.txt` index.
|
||||
4. Unauthenticated probes of `api.x.ai` (below).
|
||||
5. Not read: the xai-sdk-python and OpenAI-compatible SDK sources (not fetched in this round; the OpenAPI document plus two doc sources cover every row).
|
||||
|
||||
## Verified against spec
|
||||
|
||||
`python3 -I src/xai/core/tools/check_spec.py` on 2026-10-24:
|
||||
`spec ok: 43 calls (36 openapi, 7 docs), 66 schemas, 9 tool types, 8 platform operations excluded; spec xAI's REST API v1.0.0 sha256:f34f5c20564fefd1, 44 operations`.
|
||||
It checks every OpenAPI row (method, path, operationId, query parameters,
|
||||
request body fields, request and response schema names), the schema fields
|
||||
this module reads, the tool fields the builders write against the `ModelTool`
|
||||
variants, the 7 Batch rows and documented-only fields against the docs pages,
|
||||
and fails on any spec operation that is neither in the table nor in the
|
||||
exclusion list (7 files operations and `documents/search`, owned by
|
||||
`src/xai/platform`).
|
||||
|
||||
## Verified live (unauthenticated, 2026-10-24, no key exists)
|
||||
|
||||
- `api.x.ai` resolves, TLS verifies, HTTP/2, behind Cloudflare. `GET /` answers 421 with a welcome text.
|
||||
- Missing credential: 401 `{"code":"unauthenticated:no-credentials","error":"No credentials presented."}` on `/v1/models`, `/v1/api-key`, `/v1/me`, `/v1/responses/{id}` (GET, DELETE), `/v1/responses/{id}/input_items`, `/v1/images/{id}`, `/v1/videos/{id}`, `/v1/chat/deferred-completion/{id}`, `/v1/language-models[/{id}]`, `/v1/embedding-models`, `/v1/batches`, `/v1/batches/{id}`, `/v1/batches/{id}/requests`, `/v1/batches/{id}/results`, `POST /v1/batches/{id}:cancel`.
|
||||
- `POST` routes without `Content-Type: application/json` answer 415 (`/v1/embeddings`, `/v1/images/generations`, `/v1/videos/generations`, `/v1/videos/extensions`, `/v1/tokenize-text`, `/v1/responses/compact`, `/v1/batches/{id}/requests`): the routes exist.
|
||||
- `POST /v1/responses` with `{}` and no key answers 422 `{"error":"Failed to deserialize the JSON body into the target type: missing field `input`"}` (validation runs before authentication). `POST /v1/chat/completions` with a bogus key and `{}` answers 400 `{"code":"400","error":"Bad data: Messages cannot be empty"}`.
|
||||
- Wrong key: 400 `{"code":"invalid-argument","error":"Incorrect API key provided. ..."}` (400, not 401).
|
||||
- Absent routes answer 404: `POST /v1/batches/{id}/cancel` (the real route is `:cancel`), `POST /v1/messages/count_tokens`, `POST /v1/responses/{id}/cancel`.
|
||||
- `GET /v1/skills` answers 404 although the OpenAPI document lists it (see Unverified).
|
||||
- Error body shape: `{"code": ..., "error": "text"}` (error is a string).
|
||||
|
||||
Opt-in live tests (`tests/src/xai/core/test_xai_core.cpp`, suite `xai_core_live`):
|
||||
`MOLO_LIVE_NET=1` plus `XAI_API_KEY` runs a read-only pass (key info, every
|
||||
model list, batch list) and one smallest completion; `MOLO_LIVE_PAID=1`
|
||||
additionally runs one image generation. They SKIP loudly otherwise. Not run:
|
||||
no key exists.
|
||||
|
||||
## Unverified
|
||||
|
||||
- Every response shape was read from the OpenAPI document and the docs, never
|
||||
seen from a live call with a key.
|
||||
- Responses stream event names beyond those the guides name (see
|
||||
`stream_events.h`): the remaining kinds follow the OpenAI Responses event
|
||||
vocabulary xAI declares compatibility with. The `error` event payload is
|
||||
assumed `{type:"error", message}`.
|
||||
- Anthropic stream events of `/v1/messages` (`message_start`,
|
||||
`content_block_*`, `message_delta`, `message_stop`) are Anthropic's; xAI's
|
||||
reference only says the stream ends with `data: [DONE]`. xAI marks the whole
|
||||
Anthropic compatibility "fully deprecated".
|
||||
- `GET /v1/skills` is listed in the OpenAPI document but answered 404 live;
|
||||
the Skills service follows the document.
|
||||
- Batch routes are not in the OpenAPI document. Request entry keys differ
|
||||
between sources: the reference shows `chat_get_completion`, the guide shows
|
||||
`responses`, `image_generation`, `image_edit`, `video_generation`,
|
||||
`video_extension`. `BatchEntry` offers both; a video edit is sent under
|
||||
`video_generation` as in the guide. `input_file_id` on batch create (JSONL
|
||||
file batches) is guide-only and not exposed.
|
||||
- Documented but absent from the OpenAPI document: image `quality`
|
||||
(`low|medium|auto`), video `last_frame`. They are in the table's doc-only
|
||||
column and checked against the guides.
|
||||
- `ModelToolChoice`: the schema says `{type:"function", name}`, its
|
||||
description says `{type:"function", function:{name}}`. The schema is used on
|
||||
the Responses wire; the chat wire uses the OpenAI nesting.
|
||||
- Code execution `container` accepts "an id or an object" per the spec, shown
|
||||
by no guide; only the id form is offered.
|
||||
- Video job statuses beyond `pending|done|failed` (an expired status) are not
|
||||
documented; unknown statuses read as pending.
|
||||
- Rate limits and per-model batch support are model specific; not modelled.
|
||||
- `reasoning_effort` accepted values are model specific (`ModelInfo::accepts_effort`).
|
||||
|
||||
## Gaps
|
||||
|
||||
- No streaming for the legacy `/v1/completions` and `/v1/complete` calls.
|
||||
- No WebSocket mode for the Responses API (docs: advanced-api-usage/websocket-mode); owned by the platform package if wanted.
|
||||
- File-backed batches (`input_file_id`) need the Files API in `src/xai/platform`.
|
||||
- `llm::error_from_response` drops the top-level `code` when `error` is a
|
||||
string, so xAI errors surface with type `UNKNOWN` and the message intact
|
||||
(foundation gap in `src/llm/error.cpp`, not edited here).
|
||||
- `llm::ApiClient` has no per-request timeout setting; reasoning requests can
|
||||
run long.
|
||||
@@ -0,0 +1,109 @@
|
||||
#include "xai/core/account.h"
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
ApiKeyInfo ApiKeyInfo::parse(const Json& d) {
|
||||
ApiKeyInfo k;
|
||||
k.api_key_id = jx::str(d, "api_key_id");
|
||||
k.name = jx::str(d, "name");
|
||||
k.redacted_api_key = jx::str(d, "redacted_api_key");
|
||||
k.team_id = jx::str(d, "team_id");
|
||||
k.user_id = jx::str(d, "user_id");
|
||||
k.create_time = jx::str(d, "create_time");
|
||||
k.modify_time = jx::str(d, "modify_time");
|
||||
k.modified_by = jx::str(d, "modified_by");
|
||||
k.acls = jx::strings(d, "acls");
|
||||
k.api_key_blocked = jx::flag(d, "api_key_blocked");
|
||||
k.api_key_disabled = jx::flag(d, "api_key_disabled");
|
||||
k.team_blocked = jx::flag(d, "team_blocked");
|
||||
return k;
|
||||
}
|
||||
|
||||
MeInfo MeInfo::parse(const Json& d) {
|
||||
MeInfo m;
|
||||
m.team_id = jx::str(d, "team_id");
|
||||
m.user_id = jx::str(d, "user_id");
|
||||
m.zdr_status = jx::str(d, "zdr_status");
|
||||
m.team_blocked = jx::flag(d, "team_blocked");
|
||||
if (const Json* k = jx::get(d, "api_key"); k && k->is_object()) {
|
||||
m.has_api_key = true;
|
||||
m.api_key_id = jx::str(*k, "api_key_id");
|
||||
m.redacted_api_key = jx::str(*k, "redacted_api_key");
|
||||
m.api_key_blocked = jx::flag(*k, "blocked");
|
||||
m.api_key_disabled = jx::flag(*k, "disabled");
|
||||
}
|
||||
if (const Json* o = jx::get(d, "oauth"); o && o->is_object()) {
|
||||
m.has_oauth = true;
|
||||
m.oauth_client_id = jx::str(*o, "client_id");
|
||||
}
|
||||
return m;
|
||||
}
|
||||
|
||||
SkillInfo SkillInfo::parse(const Json& d) {
|
||||
SkillInfo s;
|
||||
s.id = jx::str(d, "id");
|
||||
s.name = jx::str(d, "name");
|
||||
s.description = jx::str(d, "description");
|
||||
s.default_version = jx::str(d, "default_version");
|
||||
s.latest_version = jx::str(d, "latest_version");
|
||||
s.object = jx::str(d, "object");
|
||||
s.created_at = jx::num(d, "created_at");
|
||||
return s;
|
||||
}
|
||||
|
||||
ApiKeyInfo AccountService::api_key() { return ApiKeyInfo::parse(call_json(Call::kApiKeyInfo)); }
|
||||
MeInfo AccountService::me() { return MeInfo::parse(call_json(Call::kMe)); }
|
||||
|
||||
SkillList SkillsService::list(std::optional<int> limit, const std::string& after, const std::string& order) {
|
||||
Query q;
|
||||
if (limit) {
|
||||
Check::range("limit", *limit, 1, Caps::kMaxListLimit);
|
||||
q.add(Call::kSkillsList, "limit", static_cast<long>(*limit));
|
||||
}
|
||||
if (!after.empty()) {
|
||||
Check::id("after", after);
|
||||
q.add(Call::kSkillsList, "after", after);
|
||||
}
|
||||
if (!order.empty()) {
|
||||
Check::one_of("order", order, {"asc", "desc"});
|
||||
q.add(Call::kSkillsList, "order", order);
|
||||
}
|
||||
Json doc = call_json(Call::kSkillsList, {}, nullptr, q.str());
|
||||
SkillList out;
|
||||
const Json* data = jx::get(doc, "data");
|
||||
if (data && data->is_array())
|
||||
for (const Json& e : data->as_array()) {
|
||||
if (out.data.size() >= Caps::kMaxAccumulatedItems) break;
|
||||
out.data.push_back(SkillInfo::parse(e));
|
||||
}
|
||||
out.first_id = jx::str(doc, "first_id");
|
||||
out.last_id = jx::str(doc, "last_id");
|
||||
out.has_more = jx::flag(doc, "has_more");
|
||||
return out;
|
||||
}
|
||||
|
||||
SkillInfo SkillsService::get(const std::string& skill_id) {
|
||||
return SkillInfo::parse(call_json(Call::kSkillsGet, {skill_id}));
|
||||
}
|
||||
|
||||
SkillInfo SkillsService::upload_zip(const std::string& filename, const std::string& zip_bytes) {
|
||||
Check::text("filename", filename, 255);
|
||||
Check::text("zip", zip_bytes, llm::Caps::kMaxUploadBytes);
|
||||
plan(Call::kSkillsUpload); // table check
|
||||
clients::body::MultipartBody form;
|
||||
form.add_file("files", filename, "application/zip", zip_bytes);
|
||||
return SkillInfo::parse(client_->post_multipart(render(Call::kSkillsUpload), form));
|
||||
}
|
||||
|
||||
bool SkillsService::remove(const std::string& skill_id) {
|
||||
return jx::flag(call_json(Call::kSkillsDelete, {skill_id}), "deleted");
|
||||
}
|
||||
|
||||
std::string SkillsService::download(const std::string& skill_id) {
|
||||
return call_raw(Call::kSkillsContent, {skill_id}).body;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,66 @@
|
||||
#pragma once
|
||||
|
||||
// Account calls on the inference host: GET /v1/api-key (key metadata, never
|
||||
// the key itself: xAI returns a redacted form) and GET /v1/me (caller
|
||||
// identity, works for API keys and OAuth tokens), plus hosted Skills
|
||||
// (/v1/skills). Management, billing, collections and files belong to
|
||||
// src/xai/platform.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/service.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
struct ApiKeyInfo {
|
||||
std::string api_key_id, name, redacted_api_key, team_id, user_id, create_time, modify_time, modified_by;
|
||||
std::vector<std::string> acls;
|
||||
bool api_key_blocked = false, api_key_disabled = false, team_blocked = false;
|
||||
static ApiKeyInfo parse(const Json& doc);
|
||||
bool usable() const { return !api_key_blocked && !api_key_disabled && !team_blocked; }
|
||||
};
|
||||
|
||||
struct MeInfo {
|
||||
std::string team_id, user_id, zdr_status; // no_zdr | zdr | pii_scrubbing
|
||||
bool team_blocked = false;
|
||||
bool has_api_key = false, has_oauth = false;
|
||||
std::string api_key_id, redacted_api_key, oauth_client_id;
|
||||
bool api_key_blocked = false, api_key_disabled = false;
|
||||
static MeInfo parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct SkillInfo {
|
||||
std::string id, name, description, default_version, latest_version, object;
|
||||
long created_at = 0;
|
||||
static SkillInfo parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct SkillList {
|
||||
std::vector<SkillInfo> data;
|
||||
std::string first_id, last_id;
|
||||
bool has_more = false;
|
||||
};
|
||||
|
||||
class AccountService : public XaiService {
|
||||
public:
|
||||
explicit AccountService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
ApiKeyInfo api_key();
|
||||
MeInfo me();
|
||||
};
|
||||
|
||||
class SkillsService : public XaiService {
|
||||
public:
|
||||
explicit SkillsService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
SkillList list(std::optional<int> limit = std::nullopt, const std::string& after = "",
|
||||
const std::string& order = "");
|
||||
SkillInfo get(const std::string& skill_id);
|
||||
// Upload a zip (a skill directory zipped; SKILL.md inside). Bounded by llm::Caps::kMaxUploadBytes.
|
||||
SkillInfo upload_zip(const std::string& filename, const std::string& zip_bytes);
|
||||
// Destructive, explicit.
|
||||
bool remove(const std::string& skill_id);
|
||||
// The skill as a zip.
|
||||
std::string download(const std::string& skill_id);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,183 @@
|
||||
#include "xai/core/batches.h"
|
||||
|
||||
#include <set>
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
void check_batch_id(const std::string& id) {
|
||||
Check::id("batch", id);
|
||||
if (id.find(':') != std::string::npos) Check::fail("bad batch id");
|
||||
}
|
||||
|
||||
Query page_query(Call call, std::optional<int> limit, const std::string& token) {
|
||||
Query q;
|
||||
if (limit) {
|
||||
Check::range("limit", *limit, 1, Caps::kMaxBatchPageLimit);
|
||||
q.add(call, "limit", static_cast<long>(*limit));
|
||||
}
|
||||
if (!token.empty()) {
|
||||
Check::text("pagination_token", token, Caps::kMaxQueryValueLen);
|
||||
q.add(call, "pagination_token", token);
|
||||
}
|
||||
return q;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
BatchEntry BatchEntry::make(const std::string& id, const char* kind, Json body) {
|
||||
Check::id("batch request", id);
|
||||
Json inner = Json::object();
|
||||
inner.set(kind, std::move(body));
|
||||
Json o = Json::object();
|
||||
o.set("batch_request_id", Json(id));
|
||||
o.set("batch_request", inner);
|
||||
BatchEntry e(std::move(o));
|
||||
if (e.bytes() > Caps::kMaxBatchRequestBytes) Check::fail("batch request over 25 MB");
|
||||
return e;
|
||||
}
|
||||
|
||||
BatchEntry BatchEntry::chat(const std::string& id, const ChatRequest& r) { return make(id, "chat_get_completion", r.to_json()); }
|
||||
BatchEntry BatchEntry::responses(const std::string& id, const ResponseRequest& r) { return make(id, "responses", r.to_json()); }
|
||||
BatchEntry BatchEntry::image_generation(const std::string& id, const ImageGenerationRequest& r) { return make(id, "image_generation", r.to_json()); }
|
||||
BatchEntry BatchEntry::image_edit(const std::string& id, const ImageEditRequest& r) { return make(id, "image_edit", r.to_json()); }
|
||||
BatchEntry BatchEntry::video_generation(const std::string& id, const VideoGenerationRequest& r) { return make(id, "video_generation", r.to_json()); }
|
||||
BatchEntry BatchEntry::video_edit(const std::string& id, const VideoEditRequest& r) { return make(id, "video_generation", r.to_json()); }
|
||||
BatchEntry BatchEntry::video_extension(const std::string& id, const VideoExtendRequest& r) { return make(id, "video_extension", r.to_json()); }
|
||||
|
||||
Batch Batch::parse(const Json& d) {
|
||||
Batch b;
|
||||
b.raw = d;
|
||||
b.batch_id = jx::str(d, "batch_id");
|
||||
b.name = jx::str(d, "name");
|
||||
b.create_time = jx::str(d, "create_time");
|
||||
b.expire_time = jx::str(d, "expire_time");
|
||||
b.cancel_time = jx::str(d, "cancel_time");
|
||||
b.cancel_by_xai_message = jx::str(d, "cancel_by_xai_message");
|
||||
b.create_api_key_id = jx::str(d, "create_api_key_id");
|
||||
Json s = jx::object(d, "state");
|
||||
b.state = {jx::num(s, "num_requests"), jx::num(s, "num_pending"), jx::num(s, "num_success"),
|
||||
jx::num(s, "num_error"), jx::num(s, "num_cancelled")};
|
||||
return b;
|
||||
}
|
||||
|
||||
Batch BatchService::create(const std::string& name) {
|
||||
Check::text("name", name, Caps::kMaxNameLen);
|
||||
Json body = Json::object();
|
||||
body.set("name", Json(name));
|
||||
return Batch::parse(call_json(Call::kBatchCreate, {}, &body));
|
||||
}
|
||||
|
||||
Batch BatchService::add_requests(const std::string& batch_id, const std::vector<BatchEntry>& entries) {
|
||||
check_batch_id(batch_id);
|
||||
Check::count("batch requests", entries.size(), 1, Caps::kMaxBatchRequestsPerAdd);
|
||||
std::set<std::string> seen;
|
||||
Json arr = Json::array();
|
||||
for (const auto& e : entries) {
|
||||
if (!seen.insert(jx::str(e.json(), "batch_request_id")).second)
|
||||
Check::fail("batch_request_id must be unique within the call");
|
||||
arr.push(e.json());
|
||||
}
|
||||
Json body = Json::object();
|
||||
body.set("batch_requests", arr);
|
||||
call_json(Call::kBatchRequestsAdd, {batch_id}, &body);
|
||||
return get(batch_id);
|
||||
}
|
||||
|
||||
Batch BatchService::get(const std::string& batch_id) {
|
||||
check_batch_id(batch_id);
|
||||
return Batch::parse(call_json(Call::kBatchGet, {batch_id}));
|
||||
}
|
||||
|
||||
BatchPage BatchService::list(std::optional<int> limit, const std::string& token) {
|
||||
Json doc = call_json(Call::kBatchList, {}, nullptr, page_query(Call::kBatchList, limit, token).str());
|
||||
BatchPage p;
|
||||
const Json* arr = jx::get(doc, "batches");
|
||||
if (arr && arr->is_array())
|
||||
for (const Json& b : arr->as_array()) {
|
||||
if (p.batches.size() >= static_cast<std::size_t>(Caps::kMaxBatchPageLimit)) break;
|
||||
p.batches.push_back(Batch::parse(b));
|
||||
}
|
||||
p.pagination_token = jx::str(doc, "pagination_token");
|
||||
return p;
|
||||
}
|
||||
|
||||
BatchRequestPage BatchService::list_requests(const std::string& batch_id, std::optional<int> limit,
|
||||
const std::string& token) {
|
||||
check_batch_id(batch_id);
|
||||
Json doc = call_json(Call::kBatchRequestsList, {batch_id}, nullptr,
|
||||
page_query(Call::kBatchRequestsList, limit, token).str());
|
||||
BatchRequestPage p;
|
||||
const Json* arr = jx::get(doc, "batch_request_metadata");
|
||||
if (arr && arr->is_array())
|
||||
for (const Json& m : arr->as_array()) {
|
||||
if (p.requests.size() >= static_cast<std::size_t>(Caps::kMaxBatchPageLimit)) break;
|
||||
p.requests.push_back({jx::str(m, "batch_request_id"), jx::str(m, "endpoint"), jx::str(m, "model"),
|
||||
jx::str(m, "state"), jx::str(m, "create_time"), jx::str(m, "finish_time")});
|
||||
}
|
||||
p.pagination_token = jx::str(doc, "pagination_token");
|
||||
return p;
|
||||
}
|
||||
|
||||
BatchResultPage BatchService::results(const std::string& batch_id, std::optional<int> limit,
|
||||
const std::string& token) {
|
||||
check_batch_id(batch_id);
|
||||
Json doc = call_json(Call::kBatchResults, {batch_id}, nullptr, page_query(Call::kBatchResults, limit, token).str());
|
||||
BatchResultPage p;
|
||||
const Json* arr = jx::get(doc, "results");
|
||||
if (arr && arr->is_array())
|
||||
for (const Json& r : arr->as_array()) {
|
||||
if (p.results.size() >= static_cast<std::size_t>(Caps::kMaxBatchPageLimit)) break;
|
||||
BatchResult o;
|
||||
o.batch_request_id = jx::str(r, "batch_request_id");
|
||||
Json result = jx::object(r, "batch_result");
|
||||
Json response = jx::object(result, "response");
|
||||
if (response.is_object() && !response.as_object().empty()) {
|
||||
// The response object carries exactly one typed member.
|
||||
const auto& members = response.as_object();
|
||||
auto it = members.begin();
|
||||
o.kind = it->first;
|
||||
o.response = it->second;
|
||||
o.succeeded = it->second.is_object();
|
||||
if (o.kind == "chat_get_completion")
|
||||
o.cost_usd_ticks = jx::num(jx::object(o.response, "usage"), "cost_in_usd_ticks");
|
||||
}
|
||||
std::string err = jx::str(result, "error");
|
||||
if (err.empty()) err = jx::str(r, "error_message");
|
||||
if (!err.empty()) {
|
||||
o.succeeded = false;
|
||||
o.error = err;
|
||||
}
|
||||
p.results.push_back(std::move(o));
|
||||
}
|
||||
p.pagination_token = jx::str(doc, "pagination_token");
|
||||
return p;
|
||||
}
|
||||
|
||||
BatchResultSet BatchService::all_results(const std::string& batch_id, std::size_t max_items) {
|
||||
Check::range("max_items", static_cast<double>(max_items), 1, 100000);
|
||||
BatchResultSet out;
|
||||
std::string token;
|
||||
for (std::size_t page = 0; page < Caps::kMaxBatchPagesWalked; ++page) {
|
||||
BatchResultPage p = results(batch_id, 1000, token);
|
||||
for (auto& r : p.results) {
|
||||
if (out.results.size() >= max_items) {
|
||||
out.truncated = true;
|
||||
return out;
|
||||
}
|
||||
out.results.push_back(std::move(r));
|
||||
}
|
||||
if (p.pagination_token.empty()) return out;
|
||||
token = p.pagination_token;
|
||||
}
|
||||
out.truncated = true;
|
||||
return out;
|
||||
}
|
||||
|
||||
Batch BatchService::cancel(const std::string& batch_id) {
|
||||
check_batch_id(batch_id);
|
||||
return Batch::parse(call_json(Call::kBatchCancel, {batch_id}));
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,106 @@
|
||||
#pragma once
|
||||
|
||||
// Batch API (/v1/batches...): a batch is a container; requests are added to
|
||||
// it, processed in the background at reduced price, and results are paged.
|
||||
// The OpenAPI document does not list these routes; they come from the REST
|
||||
// reference and the Batch API guide (see SPEC). The reference names the chat
|
||||
// entry `chat_get_completion`, the guide names `responses`, `image_generation`,
|
||||
// `image_edit`, `video_generation`, `video_extension`; both are offered and the
|
||||
// difference is recorded under Unverified.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/chat.h"
|
||||
#include "xai/core/images.h"
|
||||
#include "xai/core/responses.h"
|
||||
#include "xai/core/service.h"
|
||||
#include "xai/core/videos.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class BatchEntry {
|
||||
public:
|
||||
static BatchEntry chat(const std::string& batch_request_id, const ChatRequest& request);
|
||||
static BatchEntry responses(const std::string& batch_request_id, const ResponseRequest& request);
|
||||
static BatchEntry image_generation(const std::string& batch_request_id, const ImageGenerationRequest& request);
|
||||
static BatchEntry image_edit(const std::string& batch_request_id, const ImageEditRequest& request);
|
||||
static BatchEntry video_generation(const std::string& batch_request_id, const VideoGenerationRequest& request);
|
||||
static BatchEntry video_edit(const std::string& batch_request_id, const VideoEditRequest& request);
|
||||
static BatchEntry video_extension(const std::string& batch_request_id, const VideoExtendRequest& request);
|
||||
const Json& json() const { return json_; }
|
||||
std::size_t bytes() const { return json_.dump().size(); }
|
||||
|
||||
private:
|
||||
explicit BatchEntry(Json j) : json_(std::move(j)) {}
|
||||
static BatchEntry make(const std::string& id, const char* kind, Json body);
|
||||
Json json_;
|
||||
};
|
||||
|
||||
struct BatchState {
|
||||
long num_requests = 0, num_pending = 0, num_success = 0, num_error = 0, num_cancelled = 0;
|
||||
};
|
||||
|
||||
struct Batch {
|
||||
std::string batch_id, name, create_time, expire_time, cancel_time, cancel_by_xai_message, create_api_key_id;
|
||||
BatchState state;
|
||||
Json raw;
|
||||
static Batch parse(const Json& doc);
|
||||
// True when nothing is pending and at least one request was added.
|
||||
bool finished() const { return state.num_requests > 0 && state.num_pending == 0; }
|
||||
};
|
||||
|
||||
struct BatchPage {
|
||||
std::vector<Batch> batches;
|
||||
std::string pagination_token; // empty on the last page
|
||||
};
|
||||
|
||||
struct BatchRequestMeta {
|
||||
std::string batch_request_id, endpoint, model, state, create_time, finish_time;
|
||||
};
|
||||
|
||||
struct BatchRequestPage {
|
||||
std::vector<BatchRequestMeta> requests;
|
||||
std::string pagination_token;
|
||||
};
|
||||
|
||||
struct BatchResult {
|
||||
std::string batch_request_id;
|
||||
bool succeeded = false;
|
||||
std::string error; // when failed
|
||||
std::string kind; // chat_get_completion | image_generation | video_generation | responses ...
|
||||
Json response; // the typed response document for `kind`
|
||||
long cost_usd_ticks = 0; // chat results: usage.cost_in_usd_ticks
|
||||
};
|
||||
|
||||
struct BatchResultPage {
|
||||
std::vector<BatchResult> results;
|
||||
std::string pagination_token;
|
||||
};
|
||||
|
||||
struct BatchResultSet {
|
||||
std::vector<BatchResult> results;
|
||||
bool truncated = false; // a page or item cap stopped the walk
|
||||
};
|
||||
|
||||
class BatchService : public XaiService {
|
||||
public:
|
||||
explicit BatchService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
|
||||
Batch create(const std::string& name);
|
||||
// Adds 1..Caps::kMaxBatchRequestsPerAdd entries; each is capped at 25 MB, ids unique.
|
||||
Batch add_requests(const std::string& batch_id, const std::vector<BatchEntry>& entries);
|
||||
Batch get(const std::string& batch_id);
|
||||
BatchPage list(std::optional<int> limit = std::nullopt, const std::string& pagination_token = "");
|
||||
BatchRequestPage list_requests(const std::string& batch_id, std::optional<int> limit = std::nullopt,
|
||||
const std::string& pagination_token = "");
|
||||
BatchResultPage results(const std::string& batch_id, std::optional<int> limit = std::nullopt,
|
||||
const std::string& pagination_token = "");
|
||||
// Walks result pages up to Caps::kMaxBatchPagesWalked and max_items; truncated is loud.
|
||||
BatchResultSet all_results(const std::string& batch_id, std::size_t max_items = 10000);
|
||||
// Destructive: pending requests are not processed; finished results stay.
|
||||
Batch cancel(const std::string& batch_id);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,255 @@
|
||||
#include "xai/core/calls.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
// Row: call, method, path, operationId (or "docs"), body fields, query
|
||||
// params, doc-only body fields ("field@docs page"), request schema, response
|
||||
// schema, source. Fields the spec marks (Unsupported) or "only maintained for
|
||||
// compatibility" are not sent and not listed (see SPEC "Excluded fields").
|
||||
//
|
||||
// CALLS-BEGIN (parsed by tools/check_spec.py; one row per line)
|
||||
static const CallSpec kCalls[] = {
|
||||
{Call::kChatCreate, "POST", "/v1/chat/completions", "handle_generic_completion_request", "deferred,frequency_penalty,guidelines,logprobs,max_completion_tokens,max_tokens,messages,model,n,parallel_tool_calls,presence_penalty,prompt_cache_key,reasoning_effort,response_format,safety_identifier,search_parameters,seed,service_tier,stop,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user", "", "", "ChatRequest", "ChatResponse", "openapi"},
|
||||
{Call::kChatDeferredGet, "GET", "/v1/chat/deferred-completion/{request_id}", "handle_get_deferred_completion_request", "", "", "", "", "ChatResponse", "openapi"},
|
||||
{Call::kCompletionsLegacy, "POST", "/v1/completions", "handle_sample_request", "echo,logprobs,max_tokens,model,n,presence_penalty,prompt,seed,stop,temperature,top_p,user", "", "", "SampleRequest", "SampleResponse", "openapi"},
|
||||
{Call::kCompleteLegacy, "POST", "/v1/complete", "handle_generic_complete_request", "max_tokens_to_sample,metadata,model,prompt,stop_sequences,temperature,top_p", "", "", "CompleteRequest", "CompleteResponse", "openapi"},
|
||||
{Call::kMessagesCreate, "POST", "/v1/messages", "handle_generic_messages_request", "max_tokens,messages,metadata,model,stop_sequences,stream,system,temperature,tool_choice,tools,top_p", "", "", "MessageRequest", "MessageResponse", "openapi"},
|
||||
{Call::kResponsesCreate, "POST", "/v1/responses", "handle_generic_model_request", "guidelines,include,input,instructions,logprobs,max_output_tokens,max_turns,min_p,model,parallel_tool_calls,previous_response_id,prompt_cache_key,reasoning,reasoning_effort,safety_identifier,search_parameters,service_tier,store,stream,temperature,text,tool_choice,tools,top_k,top_logprobs,top_p,user", "", "", "ModelRequest", "ModelResponse", "openapi"},
|
||||
{Call::kResponsesGet, "GET", "/v1/responses/{response_id}", "handle_get_stored_completion_request", "", "", "", "", "ModelResponse", "openapi"},
|
||||
{Call::kResponsesDelete, "DELETE", "/v1/responses/{response_id}", "handle_delete_stored_completion_request", "", "", "", "", "DeleteStoredCompletionResponse", "openapi"},
|
||||
{Call::kResponsesInputItems, "GET", "/v1/responses/{response_id}/input_items", "handle_list_input_items", "", "limit,order,after", "", "", "ListInputItemsResponse", "openapi"},
|
||||
{Call::kResponsesCompact, "POST", "/v1/responses/compact", "handle_compact_request", "input,model", "", "", "CompactRequest", "CompactResponse", "openapi"},
|
||||
{Call::kImagesGenerate, "POST", "/v1/images/generations", "handle_generate_image_request", "aspect_ratio,deferred,model,n,output,prompt,resolution,response_format,storage_options,user", "", "quality@model-capabilities/images/generation", "GenerateImageRequest", "GeneratedImageResponse", "openapi"},
|
||||
{Call::kImagesEdit, "POST", "/v1/images/edits", "handle_edit_image_request", "aspect_ratio,deferred,image,images,model,n,output,prompt,resolution,response_format,storage_options,user", "", "", "EditImageRequest", "GeneratedImageResponse", "openapi"},
|
||||
{Call::kImagesGet, "GET", "/v1/images/{request_id}", "handle_get_deferred_image_request", "", "", "", "", "GetDeferredImageResponse", "openapi"},
|
||||
{Call::kVideosGenerate, "POST", "/v1/videos/generations", "handle_generate_video_request", "aspect_ratio,duration,image,keyframes,model,output,prompt,reference_audios,reference_images,resolution,storage_options,user", "", "last_frame@model-capabilities/video/reference-to-video", "GenerateVideoRequest", "StartDeferredResponse", "openapi"},
|
||||
{Call::kVideosEdit, "POST", "/v1/videos/edits", "handle_edit_video_request", "model,output,prompt,storage_options,user,video", "", "", "EditVideoRequest", "StartDeferredResponse", "openapi"},
|
||||
{Call::kVideosExtend, "POST", "/v1/videos/extensions", "handle_extend_video_request", "duration,model,output,prompt,storage_options,video", "", "", "ExtendVideoRequest", "StartDeferredResponse", "openapi"},
|
||||
{Call::kVideosGet, "GET", "/v1/videos/{request_id}", "handle_get_deferred_video_request", "", "", "", "", "VideoResponse", "openapi"},
|
||||
{Call::kEmbeddingsCreate, "POST", "/v1/embeddings", "handle_embedding_request", "dimensions,encoding_format,input,model,user", "", "", "EmbeddingRequest", "EmbeddingResponse", "openapi"},
|
||||
{Call::kTokenizeText, "POST", "/v1/tokenize-text", "handle_tokenize_text_request", "model,text,user", "", "", "TokenizeRequest", "TokenizeResponse", "openapi"},
|
||||
{Call::kModelsList, "GET", "/v1/models", "handle_models_list_request", "", "", "", "", "ListModelsResponse", "openapi"},
|
||||
{Call::kModelsGet, "GET", "/v1/models/{model_id}", "handle_model_get_request", "", "", "", "", "Model", "openapi"},
|
||||
{Call::kLanguageModelsList, "GET", "/v1/language-models", "handle_language_models_list_request", "", "", "", "", "ListLanguageModelsResponse", "openapi"},
|
||||
{Call::kLanguageModelsGet, "GET", "/v1/language-models/{model_id}", "handle_language_model_get_request", "", "", "", "", "LanguageModel", "openapi"},
|
||||
{Call::kImageModelsList, "GET", "/v1/image-generation-models", "handle_image_generation_models_list_request", "", "", "", "", "ListImageGenerationModelsResponse", "openapi"},
|
||||
{Call::kImageModelsGet, "GET", "/v1/image-generation-models/{model_id}", "handle_image_generation_model_get_request", "", "", "", "", "ImageGenerationModel", "openapi"},
|
||||
{Call::kVideoModelsList, "GET", "/v1/video-generation-models", "handle_video_generation_models_list_request", "", "", "", "", "ListVideoGenerationModelsResponse", "openapi"},
|
||||
{Call::kVideoModelsGet, "GET", "/v1/video-generation-models/{model_id}", "handle_video_generation_model_get_request", "", "", "", "", "VideoGenerationModel", "openapi"},
|
||||
{Call::kEmbeddingModelsList, "GET", "/v1/embedding-models", "handle_embedding_models_list_request", "", "", "", "", "ListEmbeddingModelsResponse", "openapi"},
|
||||
{Call::kEmbeddingModelsGet, "GET", "/v1/embedding-models/{model_id}", "handle_embedding_model_get_request", "", "", "", "", "EmbeddingModel", "openapi"},
|
||||
{Call::kApiKeyInfo, "GET", "/v1/api-key", "handle_get_api_key_info_request", "", "", "", "", "ApiKey", "openapi"},
|
||||
{Call::kMe, "GET", "/v1/me", "handle_get_me_request", "", "", "", "", "GetMeResponse", "openapi"},
|
||||
{Call::kSkillsList, "GET", "/v1/skills", "handle_list_skills_request", "", "limit,after,order", "", "", "SkillList", "openapi"},
|
||||
{Call::kSkillsUpload, "POST", "/v1/skills", "handle_upload_skill_request", "files", "", "", "UploadSkillMultipartRequest", "Skill", "openapi"},
|
||||
{Call::kSkillsGet, "GET", "/v1/skills/{skill_id}", "handle_retrieve_skill_request", "", "", "", "", "Skill", "openapi"},
|
||||
{Call::kSkillsDelete, "DELETE", "/v1/skills/{skill_id}", "handle_delete_skill_request", "", "", "", "", "DeletedSkill", "openapi"},
|
||||
{Call::kSkillsContent, "GET", "/v1/skills/{skill_id}/content", "handle_download_skill_content_request", "", "", "", "", "string", "openapi"},
|
||||
{Call::kBatchCreate, "POST", "/v1/batches", "POST /v1/batches", "name", "", "input_file_id@advanced-api-usage/batch-api", "", "", "docs"},
|
||||
{Call::kBatchList, "GET", "/v1/batches", "GET /v1/batches", "", "limit,pagination_token", "", "", "", "docs"},
|
||||
{Call::kBatchGet, "GET", "/v1/batches/{batch_id}", "GET /v1/batches/{batch_id}", "", "", "", "", "", "docs"},
|
||||
{Call::kBatchRequestsAdd, "POST", "/v1/batches/{batch_id}/requests", "POST /v1/batches/{batch_id}/requests", "batch_requests", "", "", "", "", "docs"},
|
||||
{Call::kBatchRequestsList, "GET", "/v1/batches/{batch_id}/requests", "GET /v1/batches/{batch_id}/requests", "", "limit,pagination_token", "", "", "", "docs"},
|
||||
{Call::kBatchResults, "GET", "/v1/batches/{batch_id}/results", "GET /v1/batches/{batch_id}/results", "", "limit,pagination_token", "", "", "", "docs"},
|
||||
{Call::kBatchCancel, "POST", "/v1/batches/{batch_id}:cancel", "POST /v1/batches/{batch_id}:cancel", "", "", "", "", "", "docs"},
|
||||
};
|
||||
// CALLS-END
|
||||
|
||||
// Response schema fields this module reads (checked against the OpenAPI
|
||||
// component schemas; a renamed or removed field fails the checker).
|
||||
// SCHEMAS-BEGIN
|
||||
static const SchemaFields kSchemas[] = {
|
||||
{"ChatResponse", "choices,citations,created,id,model,object,output_files,service_tier,system_fingerprint,usage"},
|
||||
{"Choice", "finish_reason,index,logprobs,message"},
|
||||
{"ChoiceMessage", "content,reasoning_content,refusal,role,tool_calls"},
|
||||
{"ChatResponseChunk", "choices,citations,created,id,model,object,output_files,service_tier,system_fingerprint,usage"},
|
||||
{"ChoiceChunk", "delta,finish_reason,index,logprobs"},
|
||||
{"Delta", "content,images,reasoning_content,role,tool_calls"},
|
||||
{"ToolCall", "function,id,index,type"},
|
||||
{"Function", "arguments,name"},
|
||||
{"Usage", "completion_tokens,completion_tokens_details,cost_in_usd_ticks,num_sources_used,prompt_tokens,prompt_tokens_details,server_side_tool_usage_details,total_tokens"},
|
||||
{"CompletionUsageDetail", "accepted_prediction_tokens,audio_tokens,reasoning_tokens,rejected_prediction_tokens"},
|
||||
{"PromptUsageDetail", "audio_tokens,cached_tokens,image_tokens,text_tokens"},
|
||||
{"ServerSideToolUsageDetails", "code_interpreter_calls,document_search_calls,file_search_calls,image_generation_calls,mcp_calls,web_search_calls,x_posts_fetched,x_search_calls,x_users_fetched"},
|
||||
{"ModelResponse", "background,completed_at,created_at,error,id,incomplete_details,instructions,max_output_tokens,max_tool_calls,model,object,output,parallel_tool_calls,previous_response_id,reasoning,service_tier,status,store,temperature,text,tool_choice,tools,top_p,usage,user"},
|
||||
{"ModelUsage", "context_details,cost_in_nano_usd,cost_in_usd_ticks,input_tokens,input_tokens_details,num_server_side_tools_used,num_sources_used,output_tokens,output_tokens_details,server_side_tool_usage_details,total_tokens"},
|
||||
{"OutputMessage", "content,id,role,status,type"},
|
||||
{"Annotation", "end_index,start_index,title,type,url"},
|
||||
{"FunctionToolCall", "arguments,call_id,id,name,status,type"},
|
||||
{"Reasoning", "content,encrypted_content,id,status,summary,type"},
|
||||
{"WebSearchCall", "action,id,status,type"},
|
||||
{"FileSearchCall", "id,queries,results,status,type"},
|
||||
{"CodeInterpreterCall", "code,id,outputs,status,type"},
|
||||
{"McpCall", "arguments,error,id,name,output,server_label,status,type"},
|
||||
{"CustomToolCall", "call_id,id,input,name,status,type"},
|
||||
{"ShellCall", "action,call_id,environment,id,status,type"},
|
||||
{"ImageGenerationCall", "id,prompt,result,status,type"},
|
||||
{"ToolSearchCall", "arguments,call_id,execution,id,status,type"},
|
||||
{"ToolSearchOutput", "call_id,execution,id,status,tools,type"},
|
||||
{"DeleteStoredCompletionResponse", "deleted,id,object"},
|
||||
{"ListInputItemsResponse", "data,first_id,has_more,last_id,object"},
|
||||
{"CompactResponse", "created_at,id,model,object,output,usage"},
|
||||
{"CompactUsage", "dropped_message_count,input_tokens,output_tokens,total_tokens"},
|
||||
{"GeneratedImageResponse", "data,usage"},
|
||||
{"GeneratedImage", "b64_json,file_output,mime_type,storage_error,url"},
|
||||
{"FileOutput", "expires_at,file_id,filename,public_url,public_url_error,public_url_expires_at"},
|
||||
{"GetDeferredImageResponse", "data,error,request_id,status,usage"},
|
||||
{"ImageError", "code,message"},
|
||||
{"MediaUsage", "cost_in_usd_ticks,input_tokens,input_tokens_details,output_tokens,output_tokens_details,total_tokens"},
|
||||
{"StartDeferredResponse", "request_id"},
|
||||
{"StartDeferredChatResponse", "request_id"},
|
||||
{"VideoResponse", "error,model,progress,status,usage,video"},
|
||||
{"GeneratedVideo", "duration,file_output,respect_moderation,storage_error,url"},
|
||||
{"VideoError", "code,message"},
|
||||
{"EmbeddingResponse", "data,model,object,usage"},
|
||||
{"Embedding", "embedding,index,object"},
|
||||
{"EmbeddingUsage", "prompt_tokens,total_tokens"},
|
||||
{"TokenizeResponse", "token_ids"},
|
||||
{"TokenizeResponseToken", "string_token,token_bytes,token_id"},
|
||||
{"Model", "aliases,cached_prompt_text_token_price,cached_prompt_text_token_price_long_context,capabilities,completion_text_token_price,completion_text_token_price_long_context,context_length,created,id,image_price,long_context_threshold,object,owned_by,pricing,prompt_image_token_price,prompt_text_token_price,prompt_text_token_price_long_context"},
|
||||
{"ListModelsResponse", "data,object"},
|
||||
{"LanguageModel", "aliases,cached_prompt_text_token_price,cached_prompt_text_token_price_long_context,capabilities,completion_text_token_price,completion_text_token_price_long_context,created,fingerprint,id,input_modalities,long_context_threshold,object,output_modalities,owned_by,prompt_image_token_price,prompt_text_token_price,prompt_text_token_price_long_context,search_price,version"},
|
||||
{"ModelCapabilities", "default_reasoning_effort,reasoning_effort"},
|
||||
{"ImageGenerationModel", "aliases,created,fingerprint,id,image_price,input_modalities,max_prompt_length,object,output_modalities,owned_by,pricing,version"},
|
||||
{"ImagePricingTier", "price_per_image,quality,resolution"},
|
||||
{"VideoGenerationModel", "aliases,created,fingerprint,id,input_modalities,object,output_modalities,owned_by,version"},
|
||||
{"EmbeddingModel", "aliases,created,fingerprint,id,input_modalities,object,output_modalities,owned_by,prompt_image_token_price,prompt_text_token_price,version"},
|
||||
{"ApiKey", "acls,api_key_blocked,api_key_disabled,api_key_id,create_time,modified_by,modify_time,name,redacted_api_key,team_blocked,team_id,user_id"},
|
||||
{"GetMeResponse", "api_key,oauth,team_blocked,team_id,user_id,zdr_status"},
|
||||
{"MeApiKeyInfo", "api_key_id,blocked,disabled,redacted_api_key"},
|
||||
{"Skill", "created_at,default_version,description,id,latest_version,name,object"},
|
||||
{"SkillList", "data,first_id,has_more,last_id,object"},
|
||||
{"DeletedSkill", "deleted,id,object"},
|
||||
{"MessageResponse", "content,id,model,role,stop_reason,stop_sequence,type,usage"},
|
||||
{"MessageUsage", "cache_creation_input_tokens,cache_read_input_tokens,input_tokens,output_tokens"},
|
||||
{"CompleteResponse", "completion,id,model,stop_reason,type"},
|
||||
{"SampleResponse", "choices,created,id,model,object,system_fingerprint,usage"},
|
||||
{"SampleChoice", "finish_reason,index,text"},
|
||||
};
|
||||
// SCHEMAS-END
|
||||
|
||||
// Server-side and client tool types the builders write, with the fields each
|
||||
// builder sets (checked against the ModelTool variants).
|
||||
// TOOLS-BEGIN
|
||||
static const ToolFields kTools[] = {
|
||||
{"function", "name,description,parameters,strict,defer_loading"},
|
||||
{"web_search", "allowed_domains,excluded_domains,enable_image_search,enable_image_understanding,user_location"},
|
||||
{"x_search", "allowed_x_handles,excluded_x_handles,from_date,to_date,enable_image_understanding,enable_video_understanding"},
|
||||
{"image_generation", "action"},
|
||||
{"file_search", "vector_store_ids,max_num_results"},
|
||||
{"code_interpreter", "container"},
|
||||
{"mcp", "server_label,server_url,server_description,allowed_tools,authorization,headers,defer_loading"},
|
||||
{"shell", "environment"},
|
||||
{"tool_search", "execution"},
|
||||
};
|
||||
// TOOLS-END
|
||||
|
||||
const CallSpec& spec_at(std::size_t index) { return kCalls[index]; }
|
||||
std::size_t call_count() { return sizeof(kCalls) / sizeof(kCalls[0]); }
|
||||
std::size_t tool_field_count() { return sizeof(kTools) / sizeof(kTools[0]); }
|
||||
const ToolFields& tool_fields_at(std::size_t index) { return kTools[index]; }
|
||||
|
||||
const CallSpec& spec_of(Call call) {
|
||||
for (const CallSpec& s : kCalls)
|
||||
if (s.call == call) return s;
|
||||
Check::fail("call is not in the table");
|
||||
}
|
||||
|
||||
clients::Method method_of(Call call) {
|
||||
const char* m = spec_of(call).http_method;
|
||||
if (!std::strcmp(m, "GET")) return clients::Method::kGet;
|
||||
if (!std::strcmp(m, "DELETE")) return clients::Method::kDelete;
|
||||
return clients::Method::kPost;
|
||||
}
|
||||
|
||||
namespace {
|
||||
bool in_list(const char* list, const std::string& item) {
|
||||
std::string l(list);
|
||||
std::size_t pos = 0;
|
||||
while (pos <= l.size()) {
|
||||
std::size_t comma = l.find(',', pos);
|
||||
std::string part = l.substr(pos, comma == std::string::npos ? std::string::npos : comma - pos);
|
||||
std::size_t at = part.find('@');
|
||||
if (at != std::string::npos) part = part.substr(0, at);
|
||||
if (part == item) return true;
|
||||
if (comma == std::string::npos) break;
|
||||
pos = comma + 1;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string percent_encode(const std::string& value) {
|
||||
static const char* hex = "0123456789ABCDEF";
|
||||
std::string out;
|
||||
for (unsigned char c : value) {
|
||||
bool keep = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '-' || c == '_' || c == '.' || c == '~';
|
||||
if (keep) {
|
||||
out += static_cast<char>(c);
|
||||
} else {
|
||||
out += '%';
|
||||
out += hex[c >> 4];
|
||||
out += hex[c & 15];
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
bool declares_body_field(Call call, const std::string& field) {
|
||||
const CallSpec& s = spec_of(call);
|
||||
return in_list(s.body_fields, field) || in_list(s.doc_fields, field);
|
||||
}
|
||||
|
||||
bool declares_query_param(Call call, const std::string& name) {
|
||||
return in_list(spec_of(call).query_params, name);
|
||||
}
|
||||
|
||||
std::string render(Call call, const std::vector<std::string>& ids) {
|
||||
std::string tmpl = spec_of(call).path;
|
||||
std::string out;
|
||||
std::size_t used = 0;
|
||||
for (std::size_t i = 0; i < tmpl.size(); ++i) {
|
||||
if (tmpl[i] != '{') {
|
||||
out += tmpl[i];
|
||||
continue;
|
||||
}
|
||||
std::size_t close = tmpl.find('}', i);
|
||||
if (close == std::string::npos) Check::fail("bad path template");
|
||||
std::string name = tmpl.substr(i + 1, close - i - 1);
|
||||
if (used >= ids.size()) Check::fail("missing " + name);
|
||||
std::string label = name.size() > 3 && name.compare(name.size() - 3, 3, "_id") == 0
|
||||
? name.substr(0, name.size() - 3)
|
||||
: name;
|
||||
Check::id(label, ids[used]);
|
||||
out += ids[used++];
|
||||
i = close;
|
||||
}
|
||||
if (used != ids.size()) Check::fail("surplus path ids");
|
||||
return out;
|
||||
}
|
||||
|
||||
Query& Query::add(Call call, const std::string& name, const std::string& value) {
|
||||
if (!declares_query_param(call, name)) Check::fail("query parameter " + name + " is not in the call table");
|
||||
if (pairs_.size() >= Caps::kMaxQueryParams) Check::fail("too many query parameters");
|
||||
if (value.empty() || value.size() > Caps::kMaxQueryValueLen) Check::fail("bad value for " + name);
|
||||
pairs_.emplace_back(name, value);
|
||||
return *this;
|
||||
}
|
||||
|
||||
Query& Query::add(Call call, const std::string& name, long value) {
|
||||
return add(call, name, std::to_string(value));
|
||||
}
|
||||
|
||||
std::string Query::str() const {
|
||||
std::string out;
|
||||
for (const auto& [k, v] : pairs_) out += (out.empty() ? "?" : "&") + k + "=" + percent_encode(v);
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,90 @@
|
||||
#pragma once
|
||||
|
||||
// The ONE table of REST calls this module makes. Every row is checked
|
||||
// against xAI's OpenAPI document (https://api.x.ai/api-docs/openapi.json) or,
|
||||
// for the batch API that the document does not list, against the REST
|
||||
// reference page, by tools/check_spec.py. Code builds every URL from this
|
||||
// table, never from a literal.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/request.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
enum class Call {
|
||||
// Text
|
||||
kChatCreate, kChatDeferredGet, kCompletionsLegacy, kCompleteLegacy, kMessagesCreate,
|
||||
kResponsesCreate, kResponsesGet, kResponsesDelete, kResponsesInputItems, kResponsesCompact,
|
||||
// Media
|
||||
kImagesGenerate, kImagesEdit, kImagesGet,
|
||||
kVideosGenerate, kVideosEdit, kVideosExtend, kVideosGet,
|
||||
// Embeddings, tokenizer
|
||||
kEmbeddingsCreate, kTokenizeText,
|
||||
// Models
|
||||
kModelsList, kModelsGet, kLanguageModelsList, kLanguageModelsGet, kImageModelsList,
|
||||
kImageModelsGet, kVideoModelsList, kVideoModelsGet, kEmbeddingModelsList, kEmbeddingModelsGet,
|
||||
// Account
|
||||
kApiKeyInfo, kMe,
|
||||
// Skills
|
||||
kSkillsList, kSkillsUpload, kSkillsGet, kSkillsDelete, kSkillsContent,
|
||||
// Batch (documented in the REST reference, absent from the OpenAPI document)
|
||||
kBatchCreate, kBatchList, kBatchGet, kBatchRequestsAdd, kBatchRequestsList, kBatchResults,
|
||||
kBatchCancel,
|
||||
};
|
||||
|
||||
struct CallSpec {
|
||||
Call call;
|
||||
const char* http_method; // GET POST DELETE
|
||||
const char* path; // template: /v1/responses/{response_id}
|
||||
const char* op_id; // OpenAPI operationId, or "docs" for documentation-only rows
|
||||
const char* body_fields; // comma list: request body fields we send (in the spec)
|
||||
const char* query_params; // comma list: query parameters we send
|
||||
const char* doc_fields; // comma list: fields only the guides document (see SPEC Unverified)
|
||||
const char* request_schema; // OpenAPI schema name of the request body, "" for none
|
||||
const char* response_schema; // OpenAPI schema name of the response body
|
||||
const char* source; // "openapi" or "docs"
|
||||
};
|
||||
|
||||
struct SchemaFields {
|
||||
const char* schema; // OpenAPI schema name
|
||||
const char* fields; // comma list: fields this module reads
|
||||
};
|
||||
|
||||
struct ToolFields {
|
||||
const char* type; // tool `type` discriminator
|
||||
const char* fields; // comma list: fields the builder writes
|
||||
};
|
||||
|
||||
const CallSpec& spec_of(Call call);
|
||||
clients::Method method_of(Call call);
|
||||
std::size_t call_count();
|
||||
const CallSpec& spec_at(std::size_t index);
|
||||
|
||||
// Path for the call. `ids` fill the {placeholders} in order; each is
|
||||
// validated with Service::valid_id. Throws ApiError::invalid on a missing,
|
||||
// surplus or bad id.
|
||||
std::string render(Call call, const std::vector<std::string>& ids = {});
|
||||
|
||||
// True when `field` is a body field or doc-only field the row declares.
|
||||
bool declares_body_field(Call call, const std::string& field);
|
||||
bool declares_query_param(Call call, const std::string& name);
|
||||
|
||||
// Query string builder: percent-encodes, validates names against the row.
|
||||
class Query {
|
||||
public:
|
||||
Query& add(Call call, const std::string& name, const std::string& value);
|
||||
Query& add(Call call, const std::string& name, long value);
|
||||
bool empty() const { return pairs_.empty(); }
|
||||
std::string str() const; // "" or "?a=b&c=d"
|
||||
|
||||
private:
|
||||
std::vector<std::pair<std::string, std::string>> pairs_;
|
||||
};
|
||||
|
||||
// Parity data for the checker and tests.
|
||||
std::size_t tool_field_count();
|
||||
const ToolFields& tool_fields_at(std::size_t index);
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,81 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for the xAI core module. Never autotuned. Ceilings that
|
||||
// come from the vendor are marked (spec) or (docs); the others are ours and
|
||||
// exist so that no count, size, wait or page walk is unbounded.
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
struct Caps {
|
||||
// Text and structure
|
||||
static constexpr std::size_t kMaxModelLen = 128;
|
||||
static constexpr std::size_t kMaxNameLen = 256;
|
||||
static constexpr std::size_t kMaxTextBytes = 4u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxPromptBytes = 64u * 1024u; // image / video prompts
|
||||
static constexpr std::size_t kMaxUrlLen = 8u * 1024u * 1024u; // data: URLs carry images
|
||||
static constexpr std::size_t kMaxTokenLen = 128; // enum-like strings
|
||||
static constexpr std::size_t kMaxQueryValueLen = 2048;
|
||||
static constexpr std::size_t kMaxQueryParams = 16;
|
||||
|
||||
// Conversation
|
||||
static constexpr std::size_t kMaxInputItems = 4096;
|
||||
static constexpr std::size_t kMaxContentParts = 256;
|
||||
static constexpr std::size_t kMaxChatTools = 128; // (docs) chat completions
|
||||
static constexpr std::size_t kMaxResponsesTools = 1000; // (spec) responses
|
||||
static constexpr std::size_t kMaxStop = 4; // (spec)
|
||||
static constexpr std::size_t kMaxIncludes = 16;
|
||||
static constexpr std::size_t kMaxChoices = 16; // chat `n`, ours
|
||||
static constexpr int kMaxTopLogprobs = 8; // (spec)
|
||||
static constexpr std::size_t kMaxGuidelines = 3; // (spec) one per category
|
||||
static constexpr int kMaxTurns = 1000; // responses max_turns, ours
|
||||
|
||||
// Tools
|
||||
static constexpr std::size_t kMaxDomains = 5; // (spec) web search allow / exclude
|
||||
static constexpr std::size_t kMaxXHandles = 10; // (docs) x search
|
||||
static constexpr std::size_t kMaxCollections = 10; // ours, file_search vector_store_ids
|
||||
static constexpr std::size_t kMaxMcpHeaders = 32;
|
||||
static constexpr std::size_t kMaxAllowedTools = 256;
|
||||
|
||||
// Media
|
||||
static constexpr int kMaxImages = 10; // (docs) n = 1..10
|
||||
static constexpr int kMinVideoSeconds = 1; // (spec)
|
||||
static constexpr int kMaxVideoSeconds = 15; // (spec)
|
||||
static constexpr int kMinExtensionSeconds = 2; // (spec)
|
||||
static constexpr int kMaxExtensionSeconds = 10; // (spec)
|
||||
static constexpr std::size_t kMaxReferenceImages = 14; // (docs) grok-imagine-video-1.5
|
||||
static constexpr std::size_t kMaxReferenceAudios = 3; // (docs)
|
||||
static constexpr std::size_t kMaxKeyframes = 4; // (docs)
|
||||
static constexpr std::size_t kMaxEditImages = 16; // ours, multi-image edits
|
||||
static constexpr long kMaxStorageExpirySeconds = 2592000; // (spec) 30 days
|
||||
static constexpr long kMinPublicUrlSeconds = 3600; // (spec)
|
||||
|
||||
// Embeddings and tokenize
|
||||
static constexpr std::size_t kMaxEmbeddingInputs = 1024;
|
||||
static constexpr int kMaxEmbeddingDimensions = 65536;
|
||||
static constexpr std::size_t kMaxTokenizeBytes = 1u * 1024u * 1024u;
|
||||
|
||||
// Batch
|
||||
static constexpr std::size_t kMaxBatchRequestsPerAdd = 1000; // ours
|
||||
static constexpr int kMaxBatchPageLimit = 1000; // (docs)
|
||||
static constexpr std::size_t kMaxBatchPagesWalked = 100;
|
||||
static constexpr std::size_t kMaxBatchRequestBytes = 25u * 1024u * 1024u; // (docs)
|
||||
|
||||
// Lists
|
||||
static constexpr int kMaxListLimit = 1000;
|
||||
|
||||
// Waits (deferred results, video)
|
||||
static constexpr int kMinPollIntervalMs = 1000;
|
||||
static constexpr int kMaxPollIntervalMs = 60000;
|
||||
static constexpr int kMaxWaitMs = 30 * 60 * 1000;
|
||||
static constexpr int kDefaultWaitMs = 15 * 60 * 1000;
|
||||
static constexpr int kDefaultPollIntervalMs = 5000;
|
||||
|
||||
// Streaming
|
||||
static constexpr std::size_t kMaxStreamEvents = 200000;
|
||||
static constexpr std::size_t kMaxStreamTextBytes = 16u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxAccumulatedItems = 4096;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,428 @@
|
||||
#include "xai/core/chat.h"
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
Json obj(const char* type) {
|
||||
Json o = Json::object();
|
||||
o.set("type", Json(type));
|
||||
return o;
|
||||
}
|
||||
|
||||
ChatToolCall parse_tool_call(const Json& tc) {
|
||||
ChatToolCall c;
|
||||
c.id = jx::str(tc, "id");
|
||||
c.type = jx::str(tc, "type");
|
||||
Json fn = jx::object(tc, "function");
|
||||
c.name = jx::str(fn, "name");
|
||||
const Json* args = jx::get(fn, "arguments");
|
||||
if (args && args->is_string()) c.arguments = args->as_string();
|
||||
else if (args && args->is_object()) c.arguments = args->dump();
|
||||
c.index = jx::num(tc, "index", -1);
|
||||
return c;
|
||||
}
|
||||
|
||||
std::vector<ChatToolCall> parse_tool_calls(const Json& doc, const char* key) {
|
||||
std::vector<ChatToolCall> out;
|
||||
const Json* arr = jx::get(doc, key);
|
||||
if (arr && arr->is_array())
|
||||
for (const Json& tc : arr->as_array()) {
|
||||
if (out.size() >= Caps::kMaxAccumulatedItems) break;
|
||||
out.push_back(parse_tool_call(tc));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<OutputFile> parse_files(const Json& doc) {
|
||||
std::vector<OutputFile> out;
|
||||
const Json* arr = jx::get(doc, "output_files");
|
||||
if (arr && arr->is_array())
|
||||
for (const Json& f : arr->as_array()) {
|
||||
if (out.size() >= Caps::kMaxAccumulatedItems) break;
|
||||
out.push_back({jx::str(f, "file_id"), jx::str(f, "name")});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
Json content_value(const std::vector<ChatPart>& parts) {
|
||||
Check::count("content parts", parts.size(), 1, Caps::kMaxContentParts);
|
||||
Json arr = Json::array();
|
||||
for (const auto& p : parts) arr.push(p.json());
|
||||
return arr;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
ChatPart ChatPart::text(std::string text) {
|
||||
Check::text("text", text, Caps::kMaxTextBytes, true);
|
||||
Json o = obj("text");
|
||||
o.set("text", Json(std::move(text)));
|
||||
return ChatPart(std::move(o));
|
||||
}
|
||||
|
||||
ChatPart ChatPart::image(std::string url) {
|
||||
Check::url("image url", url);
|
||||
Json u = Json::object();
|
||||
u.set("url", Json(std::move(url)));
|
||||
Json o = obj("image_url");
|
||||
o.set("image_url", u);
|
||||
return ChatPart(std::move(o));
|
||||
}
|
||||
|
||||
ChatPart ChatPart::image_file(std::string file_id) {
|
||||
Check::id("file", file_id);
|
||||
Json u = Json::object();
|
||||
u.set("file_id", Json(std::move(file_id)));
|
||||
Json o = obj("image_url");
|
||||
o.set("image_url", u);
|
||||
return ChatPart(std::move(o));
|
||||
}
|
||||
|
||||
ChatPart ChatPart::file(std::string file_id) {
|
||||
Check::id("file", file_id);
|
||||
Json f = Json::object();
|
||||
f.set("file_id", Json(std::move(file_id)));
|
||||
Json o = obj("file");
|
||||
o.set("file", f);
|
||||
return ChatPart(std::move(o));
|
||||
}
|
||||
|
||||
ChatPart ChatPart::file_url(std::string url) {
|
||||
Check::http_url("file url", url);
|
||||
Json f = Json::object();
|
||||
f.set("url", Json(std::move(url)));
|
||||
Json o = obj("file");
|
||||
o.set("file", f);
|
||||
return ChatPart(std::move(o));
|
||||
}
|
||||
|
||||
Json ChatToolCall::arguments_json() const {
|
||||
if (arguments.empty()) return Json::object();
|
||||
try {
|
||||
Json j = Json::parse(arguments);
|
||||
return j.is_object() ? j : Json::object();
|
||||
} catch (const std::exception&) {
|
||||
return Json::object();
|
||||
}
|
||||
}
|
||||
|
||||
namespace {
|
||||
Json text_message(const char* role, std::string text, bool allow_empty) {
|
||||
Check::text("message", text, Caps::kMaxTextBytes, allow_empty);
|
||||
Json o = Json::object();
|
||||
o.set("role", Json(role));
|
||||
o.set("content", Json(std::move(text)));
|
||||
return o;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
ChatMessage ChatMessage::system(std::string text) {
|
||||
return ChatMessage(text_message("system", std::move(text), false));
|
||||
}
|
||||
ChatMessage ChatMessage::developer(std::string text) {
|
||||
return ChatMessage(text_message("developer", std::move(text), false));
|
||||
}
|
||||
ChatMessage ChatMessage::user(std::string text) {
|
||||
return ChatMessage(text_message("user", std::move(text), false));
|
||||
}
|
||||
ChatMessage ChatMessage::user(const std::vector<ChatPart>& parts) {
|
||||
Json o = Json::object();
|
||||
o.set("role", Json("user"));
|
||||
o.set("content", content_value(parts));
|
||||
return ChatMessage(std::move(o));
|
||||
}
|
||||
ChatMessage ChatMessage::assistant(std::string text) {
|
||||
return ChatMessage(text_message("assistant", std::move(text), true));
|
||||
}
|
||||
|
||||
ChatMessage ChatMessage::assistant_calls(std::string text, const std::vector<ChatToolCall>& calls,
|
||||
std::string reasoning_content) {
|
||||
Check::text("message", text, Caps::kMaxTextBytes, true);
|
||||
Check::count("tool calls", calls.size(), 1, Caps::kMaxChatTools);
|
||||
Json o = Json::object();
|
||||
o.set("role", Json("assistant"));
|
||||
if (!text.empty()) o.set("content", Json(std::move(text)));
|
||||
if (!reasoning_content.empty()) o.set("reasoning_content", Json(std::move(reasoning_content)));
|
||||
Json arr = Json::array();
|
||||
for (const auto& c : calls) {
|
||||
Check::id("tool call", c.id);
|
||||
Check::id("function", c.name);
|
||||
Json fn = Json::object();
|
||||
fn.set("name", Json(c.name));
|
||||
fn.set("arguments", Json(c.arguments.empty() ? std::string("{}") : c.arguments));
|
||||
Json tc = Json::object();
|
||||
tc.set("id", Json(c.id));
|
||||
tc.set("type", Json("function"));
|
||||
tc.set("function", fn);
|
||||
arr.push(tc);
|
||||
}
|
||||
o.set("tool_calls", arr);
|
||||
return ChatMessage(std::move(o));
|
||||
}
|
||||
|
||||
ChatMessage ChatMessage::tool_result(const std::string& tool_call_id, std::string content) {
|
||||
Check::id("tool call", tool_call_id);
|
||||
Check::text("tool result", content, Caps::kMaxTextBytes, true);
|
||||
Json o = Json::object();
|
||||
o.set("role", Json("tool"));
|
||||
o.set("tool_call_id", Json(tool_call_id));
|
||||
o.set("content", Json(std::move(content)));
|
||||
return ChatMessage(std::move(o));
|
||||
}
|
||||
|
||||
ChatMessage& ChatMessage::name(std::string n) {
|
||||
if (!n.empty()) {
|
||||
Check::text("name", n, 256);
|
||||
json_.set("name", Json(std::move(n)));
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
|
||||
std::string ChatMessage::role() const { return jx::str(json_, "role"); }
|
||||
|
||||
ChatRequest& ChatRequest::add(ChatMessage m) {
|
||||
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
|
||||
messages_.push_back(std::move(m));
|
||||
return *this;
|
||||
}
|
||||
|
||||
ChatRequest& ChatRequest::add_tool(const FunctionTool& tool) {
|
||||
Check::upto("tools", tools_.size(), Caps::kMaxChatTools - 1);
|
||||
tools_.push_back(tool.to_chat_json());
|
||||
return *this;
|
||||
}
|
||||
|
||||
Json ChatRequest::to_json(bool stream, bool deferred) const {
|
||||
Check::model(model_);
|
||||
Check::count("messages", messages_.size(), 1, llm::Caps::kMaxMessages);
|
||||
if (stream && deferred) Check::fail("a request cannot be both streamed and deferred");
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model_));
|
||||
Json msgs = Json::array();
|
||||
for (const auto& m : messages_) msgs.push(m.json());
|
||||
body.set("messages", msgs);
|
||||
if (stream) {
|
||||
body.set("stream", Json(true));
|
||||
if (include_usage_) {
|
||||
Json so = Json::object();
|
||||
so.set("include_usage", Json(*include_usage_));
|
||||
body.set("stream_options", so);
|
||||
}
|
||||
} else if (include_usage_) {
|
||||
Check::fail("include_usage only applies to streamed requests");
|
||||
}
|
||||
if (deferred) body.set("deferred", Json(true));
|
||||
|
||||
if (max_completion_ && max_tokens_) Check::fail("max_tokens is deprecated; set only max_completion_tokens");
|
||||
if (max_completion_) {
|
||||
Check::range("max_completion_tokens", *max_completion_, 1, 10'000'000);
|
||||
body.set("max_completion_tokens", Json(*max_completion_));
|
||||
}
|
||||
if (max_tokens_) {
|
||||
Check::range("max_tokens", *max_tokens_, 1, 10'000'000);
|
||||
body.set("max_tokens", Json(*max_tokens_));
|
||||
}
|
||||
if (temperature_) { Check::range("temperature", *temperature_, 0, 2); body.set("temperature", Json(*temperature_)); }
|
||||
if (top_p_) { Check::range("top_p", *top_p_, 0, 1); body.set("top_p", Json(*top_p_)); }
|
||||
if (frequency_) { Check::range("frequency_penalty", *frequency_, -2, 2); body.set("frequency_penalty", Json(*frequency_)); }
|
||||
if (presence_) { Check::range("presence_penalty", *presence_, -2, 2); body.set("presence_penalty", Json(*presence_)); }
|
||||
if (n_) { Check::range("n", *n_, 1, static_cast<double>(Caps::kMaxChoices)); body.set("n", Json(*n_)); }
|
||||
jx::put(body, "seed", seed_);
|
||||
Check::strings("stop", stop_, Caps::kMaxStop, 256);
|
||||
jx::put_strings(body, "stop", stop_);
|
||||
jx::put(body, "logprobs", logprobs_);
|
||||
if (top_logprobs_) {
|
||||
Check::range("top_logprobs", *top_logprobs_, 0, Caps::kMaxTopLogprobs);
|
||||
if (!logprobs_ || !*logprobs_) Check::fail("top_logprobs needs logprobs=true");
|
||||
body.set("top_logprobs", Json(*top_logprobs_));
|
||||
}
|
||||
if (!effort_.empty()) { Check::token("reasoning_effort", effort_); body.set("reasoning_effort", Json(effort_)); }
|
||||
if (format_) body.set("response_format", format_->to_chat());
|
||||
if (!tools_.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const Json& t : tools_) arr.push(t);
|
||||
body.set("tools", arr);
|
||||
}
|
||||
if (tool_choice_) body.set("tool_choice", tool_choice_->to_chat());
|
||||
jx::put(body, "parallel_tool_calls", parallel_);
|
||||
if (search_) body.set("search_parameters", search_->to_json());
|
||||
if (!guidelines_.empty()) body.set("guidelines", guidelines_json(guidelines_));
|
||||
if (tier_) { Check::one_of("service_tier", *tier_, {"default", "priority", "fast"}); body.set("service_tier", Json(*tier_)); }
|
||||
if (user_) { Check::text("user", *user_, 256); body.set("user", Json(*user_)); }
|
||||
if (safety_) { Check::text("safety_identifier", *safety_, 256); body.set("safety_identifier", Json(*safety_)); }
|
||||
if (cache_key_) { Check::text("prompt_cache_key", *cache_key_, 256); body.set("prompt_cache_key", Json(*cache_key_)); }
|
||||
return body;
|
||||
}
|
||||
|
||||
ChatCompletion ChatCompletion::parse(const Json& doc) {
|
||||
ChatCompletion c;
|
||||
c.raw = doc;
|
||||
c.id = jx::str(doc, "id");
|
||||
c.model = jx::str(doc, "model");
|
||||
c.object = jx::str(doc, "object");
|
||||
c.created = jx::num(doc, "created");
|
||||
c.system_fingerprint = jx::str(doc, "system_fingerprint");
|
||||
c.service_tier = jx::str(doc, "service_tier");
|
||||
const Json* choices = jx::get(doc, "choices");
|
||||
if (choices && choices->is_array())
|
||||
for (const Json& ch : choices->as_array()) {
|
||||
if (c.choices.size() >= Caps::kMaxChoices) break;
|
||||
ChatChoice o;
|
||||
o.index = jx::num(ch, "index");
|
||||
o.finish_reason = jx::str(ch, "finish_reason");
|
||||
if (const Json* lp = jx::get(ch, "logprobs"); lp && lp->is_object()) o.logprobs = *lp;
|
||||
Json msg = jx::object(ch, "message");
|
||||
o.role = jx::str(msg, "role");
|
||||
o.content = jx::str(msg, "content");
|
||||
o.reasoning_content = jx::str(msg, "reasoning_content");
|
||||
o.refusal = jx::str(msg, "refusal");
|
||||
o.tool_calls = parse_tool_calls(msg, "tool_calls");
|
||||
c.choices.push_back(std::move(o));
|
||||
}
|
||||
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) c.usage = TokenUsage::from_chat(*u);
|
||||
c.citations = jx::strings(doc, "citations");
|
||||
c.output_files = parse_files(doc);
|
||||
return c;
|
||||
}
|
||||
|
||||
std::vector<ChatToolCall> ChatCompletion::function_calls() const {
|
||||
std::vector<ChatToolCall> out;
|
||||
if (!first()) return out;
|
||||
for (const auto& c : first()->tool_calls)
|
||||
if (c.type.empty() || c.type == "function") out.push_back(c);
|
||||
return out;
|
||||
}
|
||||
|
||||
CompletionChunk CompletionChunk::parse(const Json& doc) {
|
||||
CompletionChunk c;
|
||||
c.id = jx::str(doc, "id");
|
||||
c.model = jx::str(doc, "model");
|
||||
c.created = jx::num(doc, "created");
|
||||
const Json* choices = jx::get(doc, "choices");
|
||||
if (choices && choices->is_array())
|
||||
for (const Json& ch : choices->as_array()) {
|
||||
if (c.choices.size() >= Caps::kMaxChoices) break;
|
||||
ChatDelta d;
|
||||
d.index = jx::num(ch, "index");
|
||||
d.finish_reason = jx::str(ch, "finish_reason");
|
||||
Json delta = jx::object(ch, "delta");
|
||||
d.role = jx::str(delta, "role");
|
||||
d.content = jx::str(delta, "content");
|
||||
d.reasoning_content = jx::str(delta, "reasoning_content");
|
||||
d.images = jx::strings(delta, "images");
|
||||
d.tool_calls = parse_tool_calls(delta, "tool_calls");
|
||||
c.choices.push_back(std::move(d));
|
||||
}
|
||||
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) c.usage = TokenUsage::from_chat(*u);
|
||||
c.citations = jx::strings(doc, "citations");
|
||||
c.output_files = parse_files(doc);
|
||||
return c;
|
||||
}
|
||||
|
||||
bool ChatStreamAccumulator::feed(const CompletionChunk& chunk) {
|
||||
if (stopped_) return false;
|
||||
if (++chunks_ > static_cast<long>(Caps::kMaxStreamEvents)) {
|
||||
stopped_ = true;
|
||||
return false;
|
||||
}
|
||||
if (!chunk.id.empty()) id_ = chunk.id;
|
||||
if (!chunk.model.empty()) model_ = chunk.model;
|
||||
if (chunk.created) created_ = chunk.created;
|
||||
if (chunk.usage) usage_ = *chunk.usage;
|
||||
if (!chunk.citations.empty()) citations_ = chunk.citations;
|
||||
if (!chunk.output_files.empty()) files_ = chunk.output_files;
|
||||
for (const ChatDelta& d : chunk.choices) {
|
||||
if (d.index < 0 || d.index >= static_cast<long>(Caps::kMaxChoices)) continue;
|
||||
if (choices_.size() <= static_cast<std::size_t>(d.index)) choices_.resize(d.index + 1);
|
||||
ChatChoice& c = choices_[d.index];
|
||||
c.index = d.index;
|
||||
if (!d.role.empty()) c.role = d.role;
|
||||
if (c.content.size() + d.content.size() + c.reasoning_content.size() + d.reasoning_content.size() >
|
||||
Caps::kMaxStreamTextBytes) {
|
||||
stopped_ = true;
|
||||
return false;
|
||||
}
|
||||
c.content += d.content;
|
||||
c.reasoning_content += d.reasoning_content;
|
||||
if (!d.finish_reason.empty()) c.finish_reason = d.finish_reason;
|
||||
for (const ChatToolCall& frag : d.tool_calls) {
|
||||
// Fragments carry an index; the first one has id and name, later ones only arguments.
|
||||
long slot = frag.index >= 0 ? frag.index : static_cast<long>(c.tool_calls.size()) - 1;
|
||||
if (frag.index < 0 && !frag.id.empty()) slot = static_cast<long>(c.tool_calls.size());
|
||||
if (slot < 0) slot = 0;
|
||||
if (slot >= static_cast<long>(Caps::kMaxChatTools)) {
|
||||
stopped_ = true;
|
||||
return false;
|
||||
}
|
||||
if (c.tool_calls.size() <= static_cast<std::size_t>(slot)) c.tool_calls.resize(slot + 1);
|
||||
ChatToolCall& t = c.tool_calls[slot];
|
||||
t.index = slot;
|
||||
if (!frag.id.empty()) t.id = frag.id;
|
||||
if (!frag.type.empty()) t.type = frag.type;
|
||||
if (!frag.name.empty()) t.name = frag.name;
|
||||
t.arguments += frag.arguments;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
ChatCompletion ChatStreamAccumulator::result() const {
|
||||
ChatCompletion c;
|
||||
c.id = id_;
|
||||
c.model = model_;
|
||||
c.created = created_;
|
||||
c.object = "chat.completion";
|
||||
c.choices = choices_;
|
||||
c.usage = usage_;
|
||||
c.citations = citations_;
|
||||
c.output_files = files_;
|
||||
return c;
|
||||
}
|
||||
|
||||
ChatCompletion ChatService::create(const ChatRequest& request) {
|
||||
Json body = request.to_json(false, false);
|
||||
return ChatCompletion::parse(call_json(Call::kChatCreate, {}, &body));
|
||||
}
|
||||
|
||||
ChatCompletion ChatService::stream(const ChatRequest& request, const ChunkSink& sink) {
|
||||
Json body = request.to_json(true, false);
|
||||
ChatStreamAccumulator acc;
|
||||
call_stream(Call::kChatCreate, body, [&](const llm::SseEvent& ev) {
|
||||
CompletionChunk chunk = CompletionChunk::parse(ev.data);
|
||||
bool keep = acc.feed(chunk);
|
||||
bool user_keep = sink ? sink(chunk) : true;
|
||||
return keep && user_keep;
|
||||
});
|
||||
if (acc.stopped()) throw llm::ApiError(kVendor, 502, "STREAM_CAP", "chat stream hit a cap");
|
||||
return acc.result();
|
||||
}
|
||||
|
||||
std::string ChatService::start_deferred(const ChatRequest& request) {
|
||||
Json body = request.to_json(false, true);
|
||||
Json reply = call_json(Call::kChatCreate, {}, &body);
|
||||
std::string id = jx::str(reply, "request_id");
|
||||
if (id.empty()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "deferred reply has no request_id");
|
||||
return id;
|
||||
}
|
||||
|
||||
std::optional<ChatCompletion> ChatService::poll_deferred(const std::string& request_id) {
|
||||
llm::RawResponse raw = call_raw(Call::kChatDeferredGet, {request_id});
|
||||
if (raw.status == 202) return std::nullopt;
|
||||
ChatCompletion c = ChatCompletion::parse(parse_body(raw));
|
||||
if (c.pending()) return std::nullopt;
|
||||
return c;
|
||||
}
|
||||
|
||||
ChatCompletion ChatService::wait_deferred(const std::string& request_id, const WaitPolicy& policy,
|
||||
const Sleeper& sleeper) {
|
||||
Check::id("request", request_id);
|
||||
std::function<std::optional<ChatCompletion>()> step = [&]() { return poll_deferred(request_id); };
|
||||
std::optional<ChatCompletion> r = poll_until<ChatCompletion>(policy, step, sleeper);
|
||||
if (!r) throw llm::ApiError(kVendor, 504, "WAIT_TIMEOUT", "deferred completion not ready within the wait budget");
|
||||
return *r;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
#pragma once
|
||||
|
||||
// Chat completions (POST /v1/chat/completions), the legacy wire xAI keeps for
|
||||
// OpenAI-SDK compatibility: request builder, typed response and chunk
|
||||
// parsing, stream accumulation, and the deferred-completion calls.
|
||||
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/poll.h"
|
||||
#include "xai/core/response_format.h"
|
||||
#include "xai/core/search_parameters.h"
|
||||
#include "xai/core/service.h"
|
||||
#include "xai/core/tools.h"
|
||||
#include "xai/core/usage.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class ChatPart {
|
||||
public:
|
||||
static ChatPart text(std::string text);
|
||||
static ChatPart image(std::string url); // public or data: URL
|
||||
static ChatPart image_file(std::string file_id);
|
||||
static ChatPart file(std::string file_id);
|
||||
static ChatPart file_url(std::string url);
|
||||
const Json& json() const { return json_; }
|
||||
|
||||
private:
|
||||
explicit ChatPart(Json j) : json_(std::move(j)) {}
|
||||
Json json_;
|
||||
};
|
||||
|
||||
struct ChatToolCall {
|
||||
std::string id;
|
||||
std::string type; // "function" or a server-side tool type
|
||||
std::string name;
|
||||
std::string arguments; // JSON string as sent by the model
|
||||
long index = -1;
|
||||
Json arguments_json() const; // parsed; {} when empty or invalid
|
||||
};
|
||||
|
||||
class ChatMessage {
|
||||
public:
|
||||
static ChatMessage system(std::string text);
|
||||
static ChatMessage developer(std::string text);
|
||||
static ChatMessage user(std::string text);
|
||||
static ChatMessage user(const std::vector<ChatPart>& parts);
|
||||
static ChatMessage assistant(std::string text);
|
||||
// An assistant turn that called tools (content may be empty).
|
||||
static ChatMessage assistant_calls(std::string text, const std::vector<ChatToolCall>& calls,
|
||||
std::string reasoning_content = "");
|
||||
static ChatMessage tool_result(const std::string& tool_call_id, std::string content);
|
||||
|
||||
ChatMessage& name(std::string n);
|
||||
const Json& json() const { return json_; }
|
||||
std::string role() const;
|
||||
|
||||
private:
|
||||
explicit ChatMessage(Json j) : json_(std::move(j)) {}
|
||||
Json json_;
|
||||
};
|
||||
|
||||
class ChatRequest {
|
||||
public:
|
||||
explicit ChatRequest(std::string model) : model_(std::move(model)) {}
|
||||
|
||||
ChatRequest& add(ChatMessage m);
|
||||
ChatRequest& max_completion_tokens(int n) { max_completion_ = n; return *this; }
|
||||
ChatRequest& max_tokens_deprecated(int n) { max_tokens_ = n; return *this; }
|
||||
ChatRequest& temperature(double v) { temperature_ = v; return *this; }
|
||||
ChatRequest& top_p(double v) { top_p_ = v; return *this; }
|
||||
ChatRequest& n(int v) { n_ = v; return *this; }
|
||||
ChatRequest& seed(long v) { seed_ = v; return *this; }
|
||||
ChatRequest& stop(std::vector<std::string> s) { stop_ = std::move(s); return *this; }
|
||||
ChatRequest& frequency_penalty(double v) { frequency_ = v; return *this; }
|
||||
ChatRequest& presence_penalty(double v) { presence_ = v; return *this; }
|
||||
ChatRequest& logprobs(bool v) { logprobs_ = v; return *this; }
|
||||
ChatRequest& top_logprobs(int v) { top_logprobs_ = v; return *this; }
|
||||
ChatRequest& reasoning_effort(std::string e) { effort_ = std::move(e); return *this; }
|
||||
ChatRequest& response_format(ResponseFormat f) { format_ = std::move(f); return *this; }
|
||||
ChatRequest& add_tool(const FunctionTool& tool);
|
||||
ChatRequest& tool_choice(ToolChoice c) { tool_choice_ = std::move(c); return *this; }
|
||||
ChatRequest& parallel_tool_calls(bool v) { parallel_ = v; return *this; }
|
||||
ChatRequest& search_parameters(SearchParameters p) { search_ = std::move(p); return *this; }
|
||||
ChatRequest& guideline(Guideline g) { guidelines_.push_back(std::move(g)); return *this; }
|
||||
ChatRequest& service_tier(std::string t) { tier_ = std::move(t); return *this; }
|
||||
ChatRequest& user(std::string u) { user_ = std::move(u); return *this; }
|
||||
ChatRequest& safety_identifier(std::string s) { safety_ = std::move(s); return *this; }
|
||||
ChatRequest& prompt_cache_key(std::string k) { cache_key_ = std::move(k); return *this; }
|
||||
ChatRequest& include_usage(bool v) { include_usage_ = v; return *this; } // stream_options
|
||||
|
||||
const std::string& model() const { return model_; }
|
||||
// Validates and returns the wire body. `stream` sets stream:true (and
|
||||
// stream_options.include_usage when requested); `deferred` sets deferred:true.
|
||||
Json to_json(bool stream = false, bool deferred = false) const;
|
||||
|
||||
private:
|
||||
std::string model_;
|
||||
std::vector<ChatMessage> messages_;
|
||||
std::optional<int> max_completion_, max_tokens_, n_, top_logprobs_;
|
||||
std::optional<double> temperature_, top_p_, frequency_, presence_;
|
||||
std::optional<long> seed_;
|
||||
std::optional<bool> logprobs_, parallel_, include_usage_;
|
||||
std::vector<std::string> stop_;
|
||||
std::string effort_;
|
||||
std::optional<ResponseFormat> format_;
|
||||
std::vector<Json> tools_;
|
||||
std::optional<ToolChoice> tool_choice_;
|
||||
std::optional<SearchParameters> search_;
|
||||
std::vector<Guideline> guidelines_;
|
||||
std::optional<std::string> tier_, user_, safety_, cache_key_;
|
||||
};
|
||||
|
||||
struct ChatChoice {
|
||||
long index = 0;
|
||||
std::string finish_reason;
|
||||
std::string role;
|
||||
std::string content;
|
||||
std::string reasoning_content;
|
||||
std::string refusal;
|
||||
std::vector<ChatToolCall> tool_calls;
|
||||
Json logprobs;
|
||||
};
|
||||
|
||||
struct OutputFile {
|
||||
std::string file_id;
|
||||
std::string name;
|
||||
};
|
||||
|
||||
struct ChatCompletion {
|
||||
std::string id, model, object, system_fingerprint, service_tier;
|
||||
long created = 0;
|
||||
std::vector<ChatChoice> choices;
|
||||
TokenUsage usage;
|
||||
std::vector<std::string> citations;
|
||||
std::vector<OutputFile> output_files;
|
||||
Json raw;
|
||||
|
||||
static ChatCompletion parse(const Json& doc);
|
||||
const ChatChoice* first() const { return choices.empty() ? nullptr : &choices.front(); }
|
||||
std::string text() const { return first() ? first()->content : ""; }
|
||||
std::string reasoning() const { return first() ? first()->reasoning_content : ""; }
|
||||
std::string finish_reason() const { return first() ? first()->finish_reason : ""; }
|
||||
std::vector<ChatToolCall> function_calls() const; // type function (or empty type)
|
||||
bool pending() const { return id.empty() && choices.empty(); }
|
||||
};
|
||||
|
||||
// One streamed chunk (`chat.completion.chunk`).
|
||||
struct ChatDelta {
|
||||
long index = 0;
|
||||
std::string role, content, reasoning_content, finish_reason;
|
||||
std::vector<std::string> images;
|
||||
std::vector<ChatToolCall> tool_calls; // fragments: id/name once, arguments in pieces
|
||||
};
|
||||
|
||||
struct CompletionChunk {
|
||||
std::string id, model;
|
||||
long created = 0;
|
||||
std::vector<ChatDelta> choices;
|
||||
std::optional<TokenUsage> usage;
|
||||
std::vector<std::string> citations;
|
||||
std::vector<OutputFile> output_files;
|
||||
static CompletionChunk parse(const Json& doc);
|
||||
};
|
||||
|
||||
// Folds chunks into a ChatCompletion (choice 0 and further `n` choices).
|
||||
class ChatStreamAccumulator {
|
||||
public:
|
||||
bool feed(const CompletionChunk& chunk); // false when a cap stopped the fold
|
||||
ChatCompletion result() const;
|
||||
bool stopped() const { return stopped_; }
|
||||
long chunks() const { return chunks_; }
|
||||
|
||||
private:
|
||||
std::string id_, model_;
|
||||
long created_ = 0, chunks_ = 0;
|
||||
std::vector<ChatChoice> choices_;
|
||||
TokenUsage usage_;
|
||||
std::vector<std::string> citations_;
|
||||
std::vector<OutputFile> files_;
|
||||
bool stopped_ = false;
|
||||
};
|
||||
|
||||
class ChatService : public XaiService {
|
||||
public:
|
||||
using ChunkSink = std::function<bool(const CompletionChunk&)>; // false stops
|
||||
|
||||
explicit ChatService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
|
||||
ChatCompletion create(const ChatRequest& request);
|
||||
ChatCompletion stream(const ChatRequest& request, const ChunkSink& sink = {});
|
||||
|
||||
// Deferred completions: the result is fetchable exactly once within 24 hours.
|
||||
std::string start_deferred(const ChatRequest& request);
|
||||
// nullopt while the API answers 202 (not ready).
|
||||
std::optional<ChatCompletion> poll_deferred(const std::string& request_id);
|
||||
// Polls within the policy; throws ApiError 504 WAIT_TIMEOUT when the budget is spent.
|
||||
ChatCompletion wait_deferred(const std::string& request_id, const WaitPolicy& policy = {},
|
||||
const Sleeper& sleeper = real_sleeper());
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,280 @@
|
||||
#include "xai/core/chat_model.h"
|
||||
|
||||
#include <map>
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "security/crypto/Base64.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
std::string image_url_of(const llm::ImageData& img) {
|
||||
if (!img.url.empty()) return img.url;
|
||||
if (img.bytes.empty()) Check::fail("image has neither url nor bytes");
|
||||
std::string mime = img.mime.empty() ? "image/png" : img.mime;
|
||||
return "data:" + mime + ";base64," + security::crypto::Base64::encode(img.bytes);
|
||||
}
|
||||
|
||||
std::string tool_args_text(const llm::ToolCall& c) {
|
||||
return c.arguments.is_object() ? c.arguments.dump() : std::string("{}");
|
||||
}
|
||||
|
||||
std::string finish_of(const std::string& vendor_reason, bool has_calls) {
|
||||
if (has_calls) return "tool_calls";
|
||||
if (vendor_reason == "length" || vendor_reason == "max_output_tokens") return "length";
|
||||
if (vendor_reason == "content_filter") return "content_filter";
|
||||
return "stop";
|
||||
}
|
||||
|
||||
std::optional<ResponseFormat> format_of(const llm::ChatRequest& r) {
|
||||
if (r.response_schema.is_object()) return ResponseFormat::json_schema("response", r.response_schema);
|
||||
if (r.json_mode) return ResponseFormat::json_object();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
llm::ChatResponse from_responses(const ModelResponse& r) {
|
||||
llm::ChatResponse out;
|
||||
out.id = r.id();
|
||||
out.model = r.model();
|
||||
out.message.role = llm::Role::kAssistant;
|
||||
std::string reasoning = r.reasoning_text();
|
||||
if (reasoning.empty()) reasoning = r.summary_text();
|
||||
if (!reasoning.empty()) {
|
||||
llm::ContentPart p;
|
||||
p.reasoning = reasoning;
|
||||
out.message.parts.push_back(std::move(p));
|
||||
}
|
||||
std::string text = r.text();
|
||||
if (!text.empty()) out.message.parts.push_back(llm::ContentPart::of_text(text));
|
||||
bool calls = false;
|
||||
for (const OutputItem& c : r.function_calls()) {
|
||||
llm::ToolCall tc;
|
||||
tc.id = c.call_id();
|
||||
tc.name = c.name();
|
||||
tc.arguments = c.arguments();
|
||||
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
|
||||
calls = true;
|
||||
}
|
||||
out.finish_reason = finish_of(r.incomplete_reason(), calls);
|
||||
out.usage = r.usage().to_neutral();
|
||||
out.raw = r.raw();
|
||||
return out;
|
||||
}
|
||||
|
||||
llm::ChatResponse from_chat(const ChatCompletion& c) {
|
||||
llm::ChatResponse out;
|
||||
out.id = c.id;
|
||||
out.model = c.model;
|
||||
out.message.role = llm::Role::kAssistant;
|
||||
if (const ChatChoice* ch = c.first()) {
|
||||
if (!ch->reasoning_content.empty()) {
|
||||
llm::ContentPart p;
|
||||
p.reasoning = ch->reasoning_content;
|
||||
out.message.parts.push_back(std::move(p));
|
||||
}
|
||||
if (!ch->content.empty()) out.message.parts.push_back(llm::ContentPart::of_text(ch->content));
|
||||
for (const ChatToolCall& t : c.function_calls()) {
|
||||
llm::ToolCall tc;
|
||||
tc.id = t.id;
|
||||
tc.name = t.name;
|
||||
tc.arguments = t.arguments_json();
|
||||
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
|
||||
}
|
||||
out.finish_reason = finish_of(ch->finish_reason, !c.function_calls().empty());
|
||||
}
|
||||
out.usage = c.usage.to_neutral();
|
||||
out.raw = c.raw;
|
||||
return out;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
XaiChatModel::XaiChatModel(std::shared_ptr<llm::ApiClient> client, std::string default_model, Wire wire)
|
||||
: client_(std::move(client)), default_model_(std::move(default_model)), wire_(wire) {}
|
||||
|
||||
std::string XaiChatModel::model_of(const llm::ChatRequest& r) const {
|
||||
return r.model.empty() ? default_model_ : r.model;
|
||||
}
|
||||
|
||||
Json XaiChatModel::responses_body(const llm::ChatRequest& r, bool stream) const {
|
||||
if (!r.stop.empty())
|
||||
Check::fail("stop sequences are not supported by the Responses API; use Wire::kChatCompletions");
|
||||
ResponseRequest req(model_of(r));
|
||||
req.store(false);
|
||||
if (!r.system.empty()) req.instructions(r.system);
|
||||
Check::count("messages", r.messages.size(), 1, llm::Caps::kMaxMessages);
|
||||
for (const llm::Message& m : r.messages) {
|
||||
if (m.role == llm::Role::kTool) {
|
||||
for (const auto& p : m.parts)
|
||||
if (p.tool_result) req.add_input(InputItem::function_output(p.tool_result->call_id, p.tool_result->content));
|
||||
continue;
|
||||
}
|
||||
std::vector<InputPart> parts;
|
||||
std::string role = llm::role_name(m.role);
|
||||
bool assistant = m.role == llm::Role::kAssistant;
|
||||
for (const auto& p : m.parts) {
|
||||
if (!p.text.empty()) parts.push_back(assistant ? InputPart::assistant_text(p.text) : InputPart::text(p.text));
|
||||
if (p.image) parts.push_back(InputPart::image(image_url_of(*p.image)));
|
||||
}
|
||||
if (!parts.empty()) req.add_input(InputItem::message(role, parts));
|
||||
for (const auto& p : m.parts)
|
||||
if (p.tool_call) {
|
||||
Json fc = Json::object();
|
||||
fc.set("type", Json("function_call"));
|
||||
fc.set("call_id", Json(p.tool_call->id));
|
||||
fc.set("name", Json(p.tool_call->name));
|
||||
fc.set("arguments", Json(tool_args_text(*p.tool_call)));
|
||||
req.add_input(InputItem::replay(fc));
|
||||
}
|
||||
}
|
||||
for (const llm::ToolSpec& t : r.tools) req.add_tool(FunctionTool(t.name, t.description, t.parameters));
|
||||
if (auto tc = ToolChoice::from_neutral(r.tool_choice)) req.tool_choice(*tc);
|
||||
if (r.max_tokens) req.max_output_tokens(*r.max_tokens);
|
||||
if (r.temperature) req.temperature(*r.temperature);
|
||||
if (r.top_p) req.top_p(*r.top_p);
|
||||
if (auto f = format_of(r)) req.response_format(*f);
|
||||
if (!r.reasoning_effort.empty()) req.reasoning_effort(r.reasoning_effort);
|
||||
Json body = req.to_json(stream);
|
||||
jx::merge(body, r.extra);
|
||||
return body;
|
||||
}
|
||||
|
||||
Json XaiChatModel::chat_body(const llm::ChatRequest& r, bool stream) const {
|
||||
ChatRequest req(model_of(r));
|
||||
if (!r.system.empty()) req.add(ChatMessage::system(r.system));
|
||||
for (const llm::Message& m : r.messages) {
|
||||
if (m.role == llm::Role::kTool) {
|
||||
for (const auto& p : m.parts)
|
||||
if (p.tool_result) req.add(ChatMessage::tool_result(p.tool_result->call_id, p.tool_result->content));
|
||||
continue;
|
||||
}
|
||||
std::string text;
|
||||
std::vector<ChatPart> parts;
|
||||
std::vector<ChatToolCall> calls;
|
||||
std::string reasoning;
|
||||
bool has_image = false;
|
||||
for (const auto& p : m.parts) {
|
||||
if (!p.text.empty()) { text += p.text; parts.push_back(ChatPart::text(p.text)); }
|
||||
if (p.image) { has_image = true; parts.push_back(ChatPart::image(image_url_of(*p.image))); }
|
||||
if (p.tool_call) calls.push_back({p.tool_call->id, "function", p.tool_call->name, tool_args_text(*p.tool_call), -1});
|
||||
reasoning += p.reasoning;
|
||||
}
|
||||
switch (m.role) {
|
||||
case llm::Role::kSystem: req.add(ChatMessage::system(text)); break;
|
||||
case llm::Role::kUser:
|
||||
req.add(has_image ? ChatMessage::user(parts) : ChatMessage::user(text));
|
||||
break;
|
||||
case llm::Role::kAssistant:
|
||||
req.add(calls.empty() ? ChatMessage::assistant(text)
|
||||
: ChatMessage::assistant_calls(text, calls, reasoning));
|
||||
break;
|
||||
default: break;
|
||||
}
|
||||
}
|
||||
for (const llm::ToolSpec& t : r.tools) req.add_tool(FunctionTool(t.name, t.description, t.parameters));
|
||||
if (auto tc = ToolChoice::from_neutral(r.tool_choice)) req.tool_choice(*tc);
|
||||
if (r.max_tokens) req.max_completion_tokens(*r.max_tokens);
|
||||
if (r.temperature) req.temperature(*r.temperature);
|
||||
if (r.top_p) req.top_p(*r.top_p);
|
||||
if (!r.stop.empty()) req.stop(r.stop);
|
||||
if (auto f = format_of(r)) req.response_format(*f);
|
||||
if (!r.reasoning_effort.empty()) req.reasoning_effort(r.reasoning_effort);
|
||||
if (stream) req.include_usage(true);
|
||||
Json body = req.to_json(stream, false);
|
||||
jx::merge(body, r.extra);
|
||||
return body;
|
||||
}
|
||||
|
||||
llm::ChatResponse XaiChatModel::generate(const llm::ChatRequest& request) {
|
||||
if (wire_ == Wire::kResponses) {
|
||||
Json body = responses_body(request, false);
|
||||
return from_responses(ModelResponse::parse(
|
||||
client_->request(clients::Method::kPost, XaiService::plan(Call::kResponsesCreate, {}, &body).path, &body)));
|
||||
}
|
||||
Json body = chat_body(request, false);
|
||||
return from_chat(ChatCompletion::parse(
|
||||
client_->request(clients::Method::kPost, XaiService::plan(Call::kChatCreate, {}, &body).path, &body)));
|
||||
}
|
||||
|
||||
void XaiChatModel::stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
|
||||
if (wire_ == Wire::kChatCompletions) {
|
||||
Json body = chat_body(request, true);
|
||||
std::string path = XaiService::plan(Call::kChatCreate, {}, &body).path;
|
||||
ChatStreamAccumulator acc;
|
||||
std::map<long, std::string> ids;
|
||||
client_->stream(path, body, [&](const llm::SseEvent& ev) {
|
||||
CompletionChunk chunk = CompletionChunk::parse(ev.data);
|
||||
if (!acc.feed(chunk)) return false;
|
||||
bool cont = true;
|
||||
for (const ChatDelta& d : chunk.choices) {
|
||||
llm::ChatChunk out;
|
||||
out.text_delta = d.content;
|
||||
out.reasoning_delta = d.reasoning_content;
|
||||
out.finish_reason = d.finish_reason.empty() ? "" : finish_of(d.finish_reason, d.finish_reason == "tool_calls");
|
||||
for (const ChatToolCall& t : d.tool_calls) {
|
||||
llm::ChatChunk tc;
|
||||
tc.tool_call_id = t.id;
|
||||
tc.tool_call_name = t.name;
|
||||
tc.tool_args_delta = t.arguments;
|
||||
cont = on_chunk(tc) && cont;
|
||||
}
|
||||
if (!out.text_delta.empty() || !out.reasoning_delta.empty() || !out.finish_reason.empty())
|
||||
cont = on_chunk(out) && cont;
|
||||
}
|
||||
if (chunk.usage) {
|
||||
llm::ChatChunk u;
|
||||
u.usage = chunk.usage->to_neutral();
|
||||
cont = on_chunk(u) && cont;
|
||||
}
|
||||
return cont;
|
||||
});
|
||||
return;
|
||||
}
|
||||
Json body = responses_body(request, true);
|
||||
std::string path = XaiService::plan(Call::kResponsesCreate, {}, &body).path;
|
||||
ResponseAccumulator acc;
|
||||
std::map<std::string, std::string> call_of_item;
|
||||
client_->stream(path, body, [&](const llm::SseEvent& ev) {
|
||||
ResponseEvent e = ResponseEvent::parse(ev);
|
||||
bool keep = acc.feed(e);
|
||||
llm::ChatChunk out;
|
||||
bool emit = false;
|
||||
switch (e.kind) {
|
||||
case EventKind::kOutputTextDelta: out.text_delta = e.delta; emit = true; break;
|
||||
case EventKind::kReasoningTextDelta:
|
||||
case EventKind::kReasoningSummaryTextDelta: out.reasoning_delta = e.delta; emit = true; break;
|
||||
case EventKind::kOutputItemAdded:
|
||||
if (jx::str(e.item, "type") == "function_call") {
|
||||
call_of_item[e.item_id] = jx::str(e.item, "call_id");
|
||||
out.tool_call_id = jx::str(e.item, "call_id");
|
||||
out.tool_call_name = jx::str(e.item, "name");
|
||||
emit = true;
|
||||
}
|
||||
break;
|
||||
case EventKind::kFunctionCallArgumentsDelta:
|
||||
out.tool_call_id = call_of_item.count(e.item_id) ? call_of_item[e.item_id] : e.item_id;
|
||||
out.tool_args_delta = e.delta;
|
||||
emit = true;
|
||||
break;
|
||||
case EventKind::kCompleted:
|
||||
case EventKind::kIncomplete: {
|
||||
ModelResponse r = ModelResponse::parse(e.response);
|
||||
out.finish_reason = finish_of(r.incomplete_reason(), !r.function_calls().empty());
|
||||
out.usage = r.usage().to_neutral();
|
||||
emit = true;
|
||||
break;
|
||||
}
|
||||
case EventKind::kFailed:
|
||||
case EventKind::kError:
|
||||
throw llm::ApiError(kVendor, 502, "STREAM_FAILED",
|
||||
acc.error_message().empty() ? "the response stream failed" : acc.error_message());
|
||||
default: break;
|
||||
}
|
||||
bool cont = emit ? on_chunk(out) : true;
|
||||
return keep && cont;
|
||||
});
|
||||
if (acc.stopped()) throw llm::ApiError(kVendor, 502, "STREAM_CAP", acc.stop_reason());
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,42 @@
|
||||
#pragma once
|
||||
|
||||
// XaiChatModel: llm::ChatModel over xAI. Default wire is the Responses API
|
||||
// (the recommended one, sent with store=false so nothing is retained); the
|
||||
// chat-completions wire is selectable. The vendor-neutral `extra` member of
|
||||
// llm::ChatRequest is merged into the body and checked against the call
|
||||
// table: an undeclared field is rejected, never silently sent.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "llm/chat_types.h"
|
||||
#include "xai/core/chat.h"
|
||||
#include "xai/core/responses.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class XaiChatModel : public llm::ChatModel {
|
||||
public:
|
||||
enum class Wire { kResponses, kChatCompletions };
|
||||
|
||||
XaiChatModel(std::shared_ptr<llm::ApiClient> client, std::string default_model,
|
||||
Wire wire = Wire::kResponses);
|
||||
|
||||
std::string vendor() const override { return "xai"; }
|
||||
llm::ChatResponse generate(const llm::ChatRequest& request) override;
|
||||
void stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
|
||||
|
||||
// The wire bodies, exposed so callers and tests can inspect the mapping.
|
||||
Json responses_body(const llm::ChatRequest& request, bool stream) const;
|
||||
Json chat_body(const llm::ChatRequest& request, bool stream) const;
|
||||
|
||||
Wire wire() const { return wire_; }
|
||||
|
||||
private:
|
||||
std::string model_of(const llm::ChatRequest& request) const;
|
||||
std::shared_ptr<llm::ApiClient> client_;
|
||||
std::string default_model_;
|
||||
Wire wire_;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,19 @@
|
||||
#include "xai/core/client.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
XaiClient::XaiClient(std::shared_ptr<llm::ApiClient> api)
|
||||
: responses(api), chat(api), legacy(api), messages(api), images(api), videos(api),
|
||||
embeddings(api), tokenizer(api), models(api), account(api), skills(api), batches(api),
|
||||
api_(std::move(api)) {}
|
||||
|
||||
std::shared_ptr<XaiClient> XaiClient::from_env() {
|
||||
return std::make_shared<XaiClient>(XaiService::client_from_env());
|
||||
}
|
||||
|
||||
std::shared_ptr<XaiChatModel> XaiClient::chat_model(const std::string& default_model,
|
||||
XaiChatModel::Wire wire) const {
|
||||
return std::make_shared<XaiChatModel>(api_, default_model, wire);
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,51 @@
|
||||
#pragma once
|
||||
|
||||
// XaiClient: one shared ApiClient (XAI_API_KEY via llm::Config) and every core
|
||||
// service built on it.
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include "xai/core/account.h"
|
||||
#include "xai/core/batches.h"
|
||||
#include "xai/core/chat.h"
|
||||
#include "xai/core/chat_model.h"
|
||||
#include "xai/core/embeddings.h"
|
||||
#include "xai/core/images.h"
|
||||
#include "xai/core/messages.h"
|
||||
#include "xai/core/models.h"
|
||||
#include "xai/core/responses.h"
|
||||
#include "xai/core/videos.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class XaiClient {
|
||||
public:
|
||||
explicit XaiClient(std::shared_ptr<llm::ApiClient> api);
|
||||
// Key from XAI_API_KEY (environment, then the env files); a missing key is
|
||||
// fine: every call then throws NotConfiguredError before any traffic.
|
||||
static std::shared_ptr<XaiClient> from_env();
|
||||
|
||||
bool configured() const { return api_->configured(); }
|
||||
std::shared_ptr<llm::ApiClient> api() const { return api_; }
|
||||
|
||||
ResponsesService responses;
|
||||
ChatService chat;
|
||||
LegacyService legacy;
|
||||
MessagesService messages;
|
||||
ImageService images;
|
||||
VideoService videos;
|
||||
EmbeddingService embeddings;
|
||||
TokenizeService tokenizer;
|
||||
ModelsService models;
|
||||
AccountService account;
|
||||
SkillsService skills;
|
||||
BatchService batches;
|
||||
|
||||
std::shared_ptr<XaiChatModel> chat_model(const std::string& default_model,
|
||||
XaiChatModel::Wire wire = XaiChatModel::Wire::kResponses) const;
|
||||
|
||||
private:
|
||||
std::shared_ptr<llm::ApiClient> api_;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,125 @@
|
||||
#include "xai/core/embeddings.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
#include "security/crypto/Base64.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
Json EmbeddingRequest::to_json() const {
|
||||
Check::model(model);
|
||||
if (!texts.empty() && !token_arrays.empty()) Check::fail("give texts or token_arrays, not both");
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model));
|
||||
if (!token_arrays.empty()) {
|
||||
Check::upto("token arrays", token_arrays.size(), Caps::kMaxEmbeddingInputs);
|
||||
Json outer = Json::array();
|
||||
for (const auto& arr : token_arrays) {
|
||||
Check::count("tokens", arr.size(), 1, 1u << 20);
|
||||
Json inner = Json::array();
|
||||
for (int t : arr) {
|
||||
if (t < 0) Check::fail("token ids are non-negative");
|
||||
inner.push(Json(t));
|
||||
}
|
||||
outer.push(inner);
|
||||
}
|
||||
body.set("input", token_arrays.size() == 1 ? outer.as_array().front() : outer);
|
||||
} else {
|
||||
Check::count("texts", texts.size(), 1, Caps::kMaxEmbeddingInputs);
|
||||
for (const auto& t : texts) Check::text("text", t, Caps::kMaxTextBytes);
|
||||
body.set("input", texts.size() == 1 ? Json(texts.front()) : jx::string_array(texts));
|
||||
}
|
||||
if (dimensions) {
|
||||
Check::range("dimensions", *dimensions, 1, Caps::kMaxEmbeddingDimensions);
|
||||
body.set("dimensions", Json(*dimensions));
|
||||
}
|
||||
if (!encoding_format.empty()) {
|
||||
Check::one_of("encoding_format", encoding_format, {"float", "base64"});
|
||||
body.set("encoding_format", Json(encoding_format));
|
||||
}
|
||||
if (!user.empty()) {
|
||||
Check::text("user", user, 256);
|
||||
body.set("user", Json(user));
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
std::vector<float> floats_from_base64(const std::string& text) {
|
||||
std::string raw;
|
||||
std::vector<float> out;
|
||||
if (text.empty() || !security::crypto::Base64::decode(text, raw) || raw.size() % 4 != 0) return out;
|
||||
out.resize(raw.size() / 4);
|
||||
std::memcpy(out.data(), raw.data(), raw.size()); // little-endian hosts
|
||||
return out;
|
||||
}
|
||||
|
||||
EmbeddingResult EmbeddingResult::parse(const Json& doc) {
|
||||
EmbeddingResult r;
|
||||
r.raw = doc;
|
||||
r.model = jx::str(doc, "model");
|
||||
r.object = jx::str(doc, "object");
|
||||
const Json* data = jx::get(doc, "data");
|
||||
if (data && data->is_array())
|
||||
for (const Json& d : data->as_array()) {
|
||||
if (r.data.size() >= Caps::kMaxEmbeddingInputs) break;
|
||||
EmbeddingVector v;
|
||||
v.index = jx::num(d, "index");
|
||||
const Json* e = jx::get(d, "embedding");
|
||||
if (e && e->is_array()) {
|
||||
for (const Json& x : e->as_array())
|
||||
if (x.is_number()) v.values.push_back(static_cast<float>(x.as_number()));
|
||||
} else if (e && e->is_string()) {
|
||||
v.base64 = e->as_string();
|
||||
v.values = floats_from_base64(v.base64);
|
||||
}
|
||||
r.data.push_back(std::move(v));
|
||||
}
|
||||
Json usage = jx::object(doc, "usage");
|
||||
r.prompt_tokens = jx::num(usage, "prompt_tokens");
|
||||
r.total_tokens = jx::num(usage, "total_tokens");
|
||||
return r;
|
||||
}
|
||||
|
||||
EmbeddingResult EmbeddingService::embed(const EmbeddingRequest& request) {
|
||||
Json body = request.to_json();
|
||||
return EmbeddingResult::parse(call_json(Call::kEmbeddingsCreate, {}, &body));
|
||||
}
|
||||
|
||||
EmbeddingResult EmbeddingService::embed(const std::string& model, const std::vector<std::string>& texts) {
|
||||
EmbeddingRequest r;
|
||||
r.model = model;
|
||||
r.texts = texts;
|
||||
return embed(r);
|
||||
}
|
||||
|
||||
std::vector<TokenizeToken> TokenizeService::tokenize(const std::string& model, const std::string& text,
|
||||
const std::string& user) {
|
||||
Check::model(model);
|
||||
Check::text("text", text, Caps::kMaxTokenizeBytes);
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model));
|
||||
body.set("text", Json(text));
|
||||
if (!user.empty()) {
|
||||
Check::text("user", user, 256);
|
||||
body.set("user", Json(user));
|
||||
}
|
||||
Json reply = call_json(Call::kTokenizeText, {}, &body);
|
||||
std::vector<TokenizeToken> out;
|
||||
const Json* ids = jx::get(reply, "token_ids");
|
||||
if (ids && ids->is_array())
|
||||
for (const Json& t : ids->as_array()) {
|
||||
if (out.size() >= Caps::kMaxTokenizeBytes) break;
|
||||
TokenizeToken tok;
|
||||
tok.string_token = jx::str(t, "string_token");
|
||||
tok.token_id = jx::num(t, "token_id");
|
||||
const Json* bytes = jx::get(t, "token_bytes");
|
||||
if (bytes && bytes->is_array())
|
||||
for (const Json& b : bytes->as_array())
|
||||
if (b.is_number()) tok.token_bytes.push_back(static_cast<int>(b.as_int()));
|
||||
out.push_back(std::move(tok));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,64 @@
|
||||
#pragma once
|
||||
|
||||
// Embeddings (POST /v1/embeddings) and tokenize-text (POST /v1/tokenize-text).
|
||||
// The xAI guidance for retrieval: prepend "query: " to query text and
|
||||
// "passage: " to document text (spec: EmbeddingInput.String).
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/service.h"
|
||||
#include "xai/core/usage.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
struct EmbeddingRequest {
|
||||
std::string model;
|
||||
std::vector<std::string> texts; // strings to embed ...
|
||||
std::vector<std::vector<int>> token_arrays; // ... or token id arrays (exclusive)
|
||||
std::optional<int> dimensions;
|
||||
std::string encoding_format; // "float" (default) | "base64"
|
||||
std::string user;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct EmbeddingVector {
|
||||
long index = 0;
|
||||
std::vector<float> values; // filled for float and for decoded base64 output
|
||||
std::string base64; // as received when encoding_format is base64
|
||||
};
|
||||
|
||||
struct EmbeddingResult {
|
||||
std::string model, object;
|
||||
std::vector<EmbeddingVector> data;
|
||||
long prompt_tokens = 0, total_tokens = 0;
|
||||
Json raw;
|
||||
static EmbeddingResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
// Little-endian float32 array in base64 -> floats; empty on invalid input.
|
||||
std::vector<float> floats_from_base64(const std::string& text);
|
||||
|
||||
struct TokenizeToken {
|
||||
std::string string_token;
|
||||
std::vector<int> token_bytes;
|
||||
long token_id = 0;
|
||||
};
|
||||
|
||||
class EmbeddingService : public XaiService {
|
||||
public:
|
||||
explicit EmbeddingService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
EmbeddingResult embed(const EmbeddingRequest& request);
|
||||
// Convenience: one string per input, default encoding.
|
||||
EmbeddingResult embed(const std::string& model, const std::vector<std::string>& texts);
|
||||
};
|
||||
|
||||
class TokenizeService : public XaiService {
|
||||
public:
|
||||
explicit TokenizeService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
std::vector<TokenizeToken> tokenize(const std::string& model, const std::string& text,
|
||||
const std::string& user = "");
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,174 @@
|
||||
#include "xai/core/images.h"
|
||||
|
||||
#include "security/crypto/Base64.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
void check_ratio(const std::string& r) {
|
||||
Check::one_of("aspect_ratio", r,
|
||||
{"1:1", "3:4", "4:3", "9:16", "16:9", "2:3", "3:2", "9:19.5", "19.5:9", "9:20",
|
||||
"20:9", "1:2", "2:1", "21:9", "5:2", "auto"});
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void ImageOptions::write(Json& body, bool deferred) const {
|
||||
if (!model.empty()) {
|
||||
Check::model(model);
|
||||
body.set("model", Json(model));
|
||||
}
|
||||
Check::range("n", n, 1, Caps::kMaxImages);
|
||||
if (n != 1) body.set("n", Json(n));
|
||||
if (!aspect_ratio.empty()) {
|
||||
check_ratio(aspect_ratio);
|
||||
body.set("aspect_ratio", Json(aspect_ratio));
|
||||
}
|
||||
if (!resolution.empty()) {
|
||||
Check::one_of("resolution", resolution, {"1k", "1.5k", "2k"});
|
||||
body.set("resolution", Json(resolution));
|
||||
}
|
||||
if (!response_format.empty()) {
|
||||
Check::one_of("response_format", response_format, {"url", "b64_json"});
|
||||
if (deferred && response_format != "url") Check::fail("deferred image requests support response_format url only");
|
||||
body.set("response_format", Json(response_format));
|
||||
}
|
||||
if (!quality.empty()) {
|
||||
Check::one_of("quality", quality, {"low", "medium", "auto"});
|
||||
body.set("quality", Json(quality));
|
||||
}
|
||||
if (storage) body.set("storage_options", storage->to_json());
|
||||
if (!upload_urls.empty()) {
|
||||
if (upload_urls.size() != static_cast<std::size_t>(n))
|
||||
Check::fail("output.upload_urls must hold exactly n URLs");
|
||||
for (const auto& u : upload_urls) Check::http_url("upload url", u);
|
||||
Json out = Json::object();
|
||||
out.set("upload_urls", jx::string_array(upload_urls));
|
||||
body.set("output", out);
|
||||
}
|
||||
if (!user.empty()) {
|
||||
Check::text("user", user, 256);
|
||||
body.set("user", Json(user));
|
||||
}
|
||||
if (deferred) body.set("deferred", Json(true));
|
||||
}
|
||||
|
||||
Json ImageGenerationRequest::to_json(bool deferred) const {
|
||||
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
|
||||
Json body = Json::object();
|
||||
body.set("prompt", Json(prompt));
|
||||
options.write(body, deferred);
|
||||
return body;
|
||||
}
|
||||
|
||||
Json ImageEditRequest::to_json(bool deferred) const {
|
||||
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
|
||||
if (image && !images.empty()) Check::fail("image and images are mutually exclusive");
|
||||
if (!image && images.empty()) Check::fail("an edit needs image or images");
|
||||
Json body = Json::object();
|
||||
body.set("prompt", Json(prompt));
|
||||
if (image) body.set("image", image->to_json());
|
||||
if (!images.empty()) {
|
||||
Check::upto("images", images.size(), Caps::kMaxEditImages);
|
||||
Json arr = Json::array();
|
||||
for (const auto& i : images) arr.push(i.to_json());
|
||||
body.set("images", arr);
|
||||
}
|
||||
options.write(body, deferred);
|
||||
return body;
|
||||
}
|
||||
|
||||
std::string GeneratedImage::bytes() const {
|
||||
std::string out;
|
||||
if (b64_json.empty()) return out;
|
||||
if (!security::crypto::Base64::decode(b64_json, out)) return "";
|
||||
return out;
|
||||
}
|
||||
|
||||
namespace {
|
||||
GeneratedImage parse_image(const Json& d) {
|
||||
GeneratedImage g;
|
||||
g.url = jx::str(d, "url");
|
||||
g.b64_json = jx::str(d, "b64_json");
|
||||
g.mime_type = jx::str(d, "mime_type");
|
||||
g.storage_error = jx::str(d, "storage_error");
|
||||
if (const Json* f = jx::get(d, "file_output"); f && f->is_object()) g.file = StoredFile::parse(*f);
|
||||
return g;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
ImageResult ImageResult::parse(const Json& doc) {
|
||||
ImageResult r;
|
||||
r.raw = doc;
|
||||
const Json* data = jx::get(doc, "data");
|
||||
if (data && data->is_array())
|
||||
for (const Json& d : data->as_array()) {
|
||||
if (r.data.size() >= static_cast<std::size_t>(Caps::kMaxImages)) break;
|
||||
r.data.push_back(parse_image(d));
|
||||
}
|
||||
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) r.usage = MediaUsage::from_json(*u);
|
||||
return r;
|
||||
}
|
||||
|
||||
DeferredImage DeferredImage::parse(const Json& doc, const std::string& request_id, int http_status) {
|
||||
DeferredImage d;
|
||||
d.request_id = request_id;
|
||||
if (http_status == 202) {
|
||||
d.status = "pending";
|
||||
return d;
|
||||
}
|
||||
d.status = jx::str(doc, "status");
|
||||
if (d.status.empty()) d.status = jx::get(doc, "data") ? "done" : (jx::get(doc, "error") ? "failed" : "pending");
|
||||
d.error = MediaError::parse(doc, "error");
|
||||
if (d.status == "done") d.result = ImageResult::parse(doc);
|
||||
return d;
|
||||
}
|
||||
|
||||
ImageResult ImageService::generate(const ImageGenerationRequest& request) {
|
||||
Json body = request.to_json(false);
|
||||
return ImageResult::parse(call_json(Call::kImagesGenerate, {}, &body));
|
||||
}
|
||||
|
||||
ImageResult ImageService::edit(const ImageEditRequest& request) {
|
||||
Json body = request.to_json(false);
|
||||
return ImageResult::parse(call_json(Call::kImagesEdit, {}, &body));
|
||||
}
|
||||
|
||||
static std::string request_id_of(const Json& reply) {
|
||||
std::string id = jx::str(reply, "request_id");
|
||||
if (id.empty()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "deferred reply has no request_id");
|
||||
return id;
|
||||
}
|
||||
|
||||
std::string ImageService::start_generate(const ImageGenerationRequest& request) {
|
||||
Json body = request.to_json(true);
|
||||
return request_id_of(call_json(Call::kImagesGenerate, {}, &body));
|
||||
}
|
||||
|
||||
std::string ImageService::start_edit(const ImageEditRequest& request) {
|
||||
Json body = request.to_json(true);
|
||||
return request_id_of(call_json(Call::kImagesEdit, {}, &body));
|
||||
}
|
||||
|
||||
DeferredImage ImageService::poll(const std::string& request_id) {
|
||||
llm::RawResponse raw = call_raw(Call::kImagesGet, {request_id});
|
||||
return DeferredImage::parse(parse_body(raw), request_id, raw.status);
|
||||
}
|
||||
|
||||
ImageResult ImageService::wait(const std::string& request_id, const WaitPolicy& policy,
|
||||
const Sleeper& sleeper) {
|
||||
Check::id("request", request_id);
|
||||
std::function<std::optional<DeferredImage>()> step = [&]() -> std::optional<DeferredImage> {
|
||||
DeferredImage d = poll(request_id);
|
||||
if (d.pending()) return std::nullopt;
|
||||
return d;
|
||||
};
|
||||
std::optional<DeferredImage> d = poll_until<DeferredImage>(policy, step, sleeper);
|
||||
if (!d) throw llm::ApiError(kVendor, 504, "WAIT_TIMEOUT", "image not ready within the wait budget");
|
||||
if (d->failed())
|
||||
throw llm::ApiError(kVendor, 502, "GENERATION_FAILED",
|
||||
d->error ? d->error->code + ": " + d->error->message : "image generation failed");
|
||||
return *d->result;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,88 @@
|
||||
#pragma once
|
||||
|
||||
// Image generation and editing (POST /v1/images/generations, /v1/images/edits)
|
||||
// and the deferred poll (GET /v1/images/{request_id}). Generation costs money
|
||||
// (see MediaUsage::cost_usd_ticks); nothing here retries.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/media.h"
|
||||
#include "xai/core/poll.h"
|
||||
#include "xai/core/service.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
// Shared by generation and editing.
|
||||
struct ImageOptions {
|
||||
std::string model; // optional; the server picks a default
|
||||
int n = 1; // 1..10
|
||||
std::string aspect_ratio; // "1:1", "16:9", ..., "auto" (grok-imagine models)
|
||||
std::string resolution; // "1k" | "1.5k" | "2k" (grok-imagine models)
|
||||
std::string response_format; // "url" (default) | "b64_json"
|
||||
std::string quality; // "low" | "medium" | "auto"; documented in the guides, not in the OpenAPI document
|
||||
std::optional<StorageOptions> storage;
|
||||
std::vector<std::string> upload_urls; // signed PUT URLs, exactly n of them (output.upload_urls)
|
||||
std::string user;
|
||||
|
||||
void write(Json& body, bool deferred) const;
|
||||
};
|
||||
|
||||
struct ImageGenerationRequest {
|
||||
std::string prompt;
|
||||
ImageOptions options;
|
||||
Json to_json(bool deferred = false) const;
|
||||
};
|
||||
|
||||
struct ImageEditRequest {
|
||||
std::string prompt;
|
||||
std::optional<MediaSource> image; // single input
|
||||
std::vector<MediaSource> images; // multi-reference input; exclusive with `image`
|
||||
ImageOptions options;
|
||||
Json to_json(bool deferred = false) const;
|
||||
};
|
||||
|
||||
struct GeneratedImage {
|
||||
std::string url, b64_json, mime_type, storage_error;
|
||||
std::optional<StoredFile> file;
|
||||
// Decoded bytes of b64_json; empty when absent or invalid.
|
||||
std::string bytes() const;
|
||||
};
|
||||
|
||||
struct ImageResult {
|
||||
std::vector<GeneratedImage> data;
|
||||
MediaUsage usage;
|
||||
Json raw;
|
||||
static ImageResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct DeferredImage {
|
||||
std::string request_id;
|
||||
std::string status; // pending | done | failed
|
||||
std::optional<ImageResult> result;
|
||||
std::optional<MediaError> error;
|
||||
bool done() const { return status == "done"; }
|
||||
bool failed() const { return status == "failed"; }
|
||||
bool pending() const { return !done() && !failed(); }
|
||||
static DeferredImage parse(const Json& doc, const std::string& request_id, int http_status);
|
||||
};
|
||||
|
||||
class ImageService : public XaiService {
|
||||
public:
|
||||
explicit ImageService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
|
||||
ImageResult generate(const ImageGenerationRequest& request);
|
||||
ImageResult edit(const ImageEditRequest& request);
|
||||
// Deferred: returns the request id; only response_format url is supported.
|
||||
std::string start_generate(const ImageGenerationRequest& request);
|
||||
std::string start_edit(const ImageEditRequest& request);
|
||||
// 202 while pending; 200 carries data (done) or error (failed).
|
||||
DeferredImage poll(const std::string& request_id);
|
||||
// Polls within the policy; throws ApiError 504 WAIT_TIMEOUT when the budget is spent
|
||||
// and 502 GENERATION_FAILED when the job failed.
|
||||
ImageResult wait(const std::string& request_id, const WaitPolicy& policy = {},
|
||||
const Sleeper& sleeper = real_sleeper());
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,176 @@
|
||||
#include "xai/core/input.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
Json part_object(const char* type) {
|
||||
Json o = Json::object();
|
||||
o.set("type", Json(type));
|
||||
return o;
|
||||
}
|
||||
|
||||
Json parts_array(const std::vector<InputPart>& parts) {
|
||||
Check::count("content parts", parts.size(), 1, Caps::kMaxContentParts);
|
||||
Json arr = Json::array();
|
||||
for (const auto& p : parts) arr.push(p.json());
|
||||
return arr;
|
||||
}
|
||||
|
||||
void check_role(const std::string& role) {
|
||||
Check::one_of("role", role, {"user", "assistant", "system", "developer"});
|
||||
}
|
||||
|
||||
Json shell_result(std::string out, std::string err, Json outcome) {
|
||||
Check::text("stdout", out, Caps::kMaxTextBytes, true);
|
||||
Check::text("stderr", err, Caps::kMaxTextBytes, true);
|
||||
Json r = Json::object();
|
||||
r.set("stdout", Json(std::move(out)));
|
||||
r.set("stderr", Json(std::move(err)));
|
||||
r.set("outcome", std::move(outcome));
|
||||
return r;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
InputPart InputPart::text(std::string text) {
|
||||
Check::text("text", text, Caps::kMaxTextBytes, true);
|
||||
Json o = part_object("input_text");
|
||||
o.set("text", Json(std::move(text)));
|
||||
return InputPart(std::move(o));
|
||||
}
|
||||
|
||||
InputPart InputPart::assistant_text(std::string text) {
|
||||
Check::text("text", text, Caps::kMaxTextBytes, true);
|
||||
Json o = part_object("output_text");
|
||||
o.set("text", Json(std::move(text)));
|
||||
return InputPart(std::move(o));
|
||||
}
|
||||
|
||||
InputPart InputPart::image(std::string url) {
|
||||
Check::url("image_url", url);
|
||||
Json o = part_object("input_image");
|
||||
o.set("image_url", Json(std::move(url)));
|
||||
return InputPart(std::move(o));
|
||||
}
|
||||
|
||||
InputPart InputPart::file_id(std::string id) {
|
||||
Check::id("file", id);
|
||||
Json o = part_object("input_file");
|
||||
o.set("file_id", Json(std::move(id)));
|
||||
return InputPart(std::move(o));
|
||||
}
|
||||
|
||||
InputPart InputPart::file_url(std::string url) {
|
||||
Check::http_url("file_url", url);
|
||||
Json o = part_object("input_file");
|
||||
o.set("file_url", Json(std::move(url)));
|
||||
return InputPart(std::move(o));
|
||||
}
|
||||
|
||||
InputPart InputPart::file_data(std::string filename, std::string base64, std::string mime_type) {
|
||||
Check::text("filename", filename, 255);
|
||||
Check::text("file_data", base64, Caps::kMaxUrlLen);
|
||||
Json o = part_object("input_file");
|
||||
o.set("filename", Json(std::move(filename)));
|
||||
o.set("file_data", Json(std::move(base64)));
|
||||
if (!mime_type.empty()) {
|
||||
Check::token("mime_type", mime_type);
|
||||
o.set("mime_type", Json(std::move(mime_type)));
|
||||
}
|
||||
return InputPart(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::message(const std::string& role, std::string text) {
|
||||
check_role(role);
|
||||
Check::text("message", text, Caps::kMaxTextBytes, true);
|
||||
Json o = Json::object();
|
||||
o.set("role", Json(role));
|
||||
o.set("content", Json(std::move(text)));
|
||||
return InputItem(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::message(const std::string& role, const std::vector<InputPart>& parts) {
|
||||
check_role(role);
|
||||
Json o = Json::object();
|
||||
o.set("role", Json(role));
|
||||
o.set("content", parts_array(parts));
|
||||
return InputItem(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::function_output(const std::string& call_id, std::string output) {
|
||||
Check::id("call", call_id);
|
||||
Check::text("output", output, Caps::kMaxTextBytes, true);
|
||||
Json o = part_object("function_call_output");
|
||||
o.set("call_id", Json(call_id));
|
||||
o.set("output", Json(std::move(output)));
|
||||
return InputItem(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::function_output(const std::string& call_id, const std::vector<InputPart>& parts) {
|
||||
Check::id("call", call_id);
|
||||
Json o = part_object("function_call_output");
|
||||
o.set("call_id", Json(call_id));
|
||||
o.set("output", parts_array(parts));
|
||||
return InputItem(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::shell_output(const std::string& call_id, std::string out, std::string err,
|
||||
int exit_code) {
|
||||
Check::id("call", call_id);
|
||||
Json outcome = part_object("exit");
|
||||
outcome.set("exit_code", Json(exit_code));
|
||||
Json results = Json::array();
|
||||
results.push(shell_result(std::move(out), std::move(err), std::move(outcome)));
|
||||
Json o = part_object("shell_call_output");
|
||||
o.set("call_id", Json(call_id));
|
||||
o.set("output", results);
|
||||
return InputItem(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::shell_timeout(const std::string& call_id, std::string out, std::string err) {
|
||||
Check::id("call", call_id);
|
||||
Json results = Json::array();
|
||||
results.push(shell_result(std::move(out), std::move(err), part_object("timeout")));
|
||||
Json o = part_object("shell_call_output");
|
||||
o.set("call_id", Json(call_id));
|
||||
o.set("output", results);
|
||||
return InputItem(std::move(o));
|
||||
}
|
||||
|
||||
InputItem InputItem::replay(const Json& item) {
|
||||
Check::object("replayed item", item);
|
||||
std::string type = jx::str(item, "type");
|
||||
Check::one_of("replayed item type", type,
|
||||
{"message", "reasoning", "function_call", "web_search_call", "file_search_call",
|
||||
"code_interpreter_call", "mcp_call", "custom_tool_call", "shell_call",
|
||||
"image_generation_call", "tool_search_call", "tool_search_output", "compaction",
|
||||
"function_call_output", "shell_call_output"});
|
||||
if (item.dump().size() > Caps::kMaxTextBytes) Check::fail("replayed item is over the size cap");
|
||||
return InputItem(item);
|
||||
}
|
||||
|
||||
std::string InputItem::type() const {
|
||||
std::string t = jx::str(json_, "type");
|
||||
return t.empty() ? "message" : t;
|
||||
}
|
||||
|
||||
Input& Input::add(InputItem item) {
|
||||
if (has_text_) Check::fail("input is a plain string; cannot add items");
|
||||
Check::upto("input items", items_.size(), Caps::kMaxInputItems - 1);
|
||||
items_.push_back(std::move(item));
|
||||
return *this;
|
||||
}
|
||||
|
||||
Json Input::to_json() const {
|
||||
if (has_text_) {
|
||||
Check::text("input", text_, Caps::kMaxTextBytes);
|
||||
return Json(text_);
|
||||
}
|
||||
Check::count("input items", items_.size(), 1, Caps::kMaxInputItems);
|
||||
Json arr = Json::array();
|
||||
for (const auto& i : items_) arr.push(i.json());
|
||||
return arr;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,72 @@
|
||||
#pragma once
|
||||
|
||||
// Input items of the Responses API (`ModelInputPart`): messages with text,
|
||||
// image and file parts, function / shell tool outputs, and verbatim replay of
|
||||
// earlier output items (encrypted reasoning, compaction, tool calls).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
// One piece of message content: input_text, output_text, input_image, input_file.
|
||||
class InputPart {
|
||||
public:
|
||||
static InputPart text(std::string text);
|
||||
static InputPart assistant_text(std::string text); // output_text, for replayed assistant turns
|
||||
// Public https URL or base64 data: URL (JPEG, PNG, WebP; max 20 MiB per the models guide).
|
||||
static InputPart image(std::string url);
|
||||
// Exactly one source: a Files API id, a public URL, or inline bytes.
|
||||
static InputPart file_id(std::string id);
|
||||
static InputPart file_url(std::string url);
|
||||
static InputPart file_data(std::string filename, std::string base64, std::string mime_type = "");
|
||||
|
||||
const Json& json() const { return json_; }
|
||||
|
||||
private:
|
||||
explicit InputPart(Json j) : json_(std::move(j)) {}
|
||||
Json json_;
|
||||
};
|
||||
|
||||
class InputItem {
|
||||
public:
|
||||
// role: user, assistant, system or developer.
|
||||
static InputItem message(const std::string& role, std::string text);
|
||||
static InputItem message(const std::string& role, const std::vector<InputPart>& parts);
|
||||
static InputItem function_output(const std::string& call_id, std::string output);
|
||||
static InputItem function_output(const std::string& call_id, const std::vector<InputPart>& parts);
|
||||
// Result of a shell_call: outcome is an exit code, or timeout.
|
||||
static InputItem shell_output(const std::string& call_id, std::string stdout_text,
|
||||
std::string stderr_text, int exit_code);
|
||||
static InputItem shell_timeout(const std::string& call_id, std::string stdout_text,
|
||||
std::string stderr_text);
|
||||
// Replays an item the API returned (unchanged). Only known item types.
|
||||
static InputItem replay(const Json& item);
|
||||
|
||||
const Json& json() const { return json_; }
|
||||
std::string type() const;
|
||||
|
||||
private:
|
||||
explicit InputItem(Json j) : json_(std::move(j)) {}
|
||||
Json json_;
|
||||
};
|
||||
|
||||
// The `input` field: a bare string or an array of items.
|
||||
class Input {
|
||||
public:
|
||||
Input() = default;
|
||||
explicit Input(std::string text) : text_(std::move(text)), has_text_(true) {}
|
||||
Input& add(InputItem item);
|
||||
bool empty() const { return !has_text_ && items_.empty(); }
|
||||
std::size_t size() const { return has_text_ ? 1 : items_.size(); }
|
||||
Json to_json() const;
|
||||
|
||||
private:
|
||||
std::string text_;
|
||||
bool has_text_ = false;
|
||||
std::vector<InputItem> items_;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,89 @@
|
||||
#include "xai/core/media.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
Json with(const char* key, const std::string& value) {
|
||||
Json o = Json::object();
|
||||
o.set(key, Json(value));
|
||||
return o;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
MediaSource MediaSource::image_url(std::string url) {
|
||||
Check::url("image url", url);
|
||||
return MediaSource(with("url", url));
|
||||
}
|
||||
|
||||
MediaSource MediaSource::image_file(std::string file_id) {
|
||||
Check::id("file", file_id);
|
||||
return MediaSource(with("file_id", file_id));
|
||||
}
|
||||
|
||||
MediaSource MediaSource::video_url(std::string url) {
|
||||
Check::url("video url", url);
|
||||
if (url.rfind("data:", 0) != 0) {
|
||||
std::string path = url.substr(0, url.find_first_of("?#"));
|
||||
if (path.size() < 4 || path.compare(path.size() - 4, 4, ".mp4") != 0)
|
||||
Check::fail("video url must end in .mp4");
|
||||
}
|
||||
return MediaSource(with("url", url));
|
||||
}
|
||||
|
||||
MediaSource MediaSource::video_file(std::string file_id) {
|
||||
Check::id("file", file_id);
|
||||
return MediaSource(with("file_id", file_id));
|
||||
}
|
||||
|
||||
MediaSource MediaSource::audio_url(std::string url) {
|
||||
Check::url("audio url", url);
|
||||
return MediaSource(with("url", url));
|
||||
}
|
||||
|
||||
MediaSource MediaSource::audio_voice(std::string voice_id) {
|
||||
Check::id("voice", voice_id);
|
||||
return MediaSource(with("voice_id", voice_id));
|
||||
}
|
||||
|
||||
Json StorageOptions::to_json() const {
|
||||
Check::text("filename", filename, 255);
|
||||
if (filename.find('/') != std::string::npos || filename.find('\\') != std::string::npos)
|
||||
Check::fail("filename must not contain a path");
|
||||
Json o = Json::object();
|
||||
o.set("filename", Json(filename));
|
||||
if (expires_after) {
|
||||
Check::range("expires_after", static_cast<double>(*expires_after), 1, Caps::kMaxStorageExpirySeconds);
|
||||
o.set("expires_after", Json(*expires_after));
|
||||
}
|
||||
if (public_url_expires_after) {
|
||||
Check::range("public url expires_after", static_cast<double>(*public_url_expires_after),
|
||||
static_cast<double>(Caps::kMinPublicUrlSeconds), Caps::kMaxStorageExpirySeconds);
|
||||
Json p = Json::object();
|
||||
p.set("expires_after", Json(*public_url_expires_after));
|
||||
o.set("public_url", p);
|
||||
} else if (public_url) {
|
||||
o.set("public_url", Json(true));
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
StoredFile StoredFile::parse(const Json& doc) {
|
||||
StoredFile f;
|
||||
f.file_id = jx::str(doc, "file_id");
|
||||
f.filename = jx::str(doc, "filename");
|
||||
f.public_url = jx::str(doc, "public_url");
|
||||
f.public_url_error = jx::str(doc, "public_url_error");
|
||||
if (const Json* v = jx::get(doc, "expires_at"); v && v->is_number()) f.expires_at = v->as_int();
|
||||
if (const Json* v = jx::get(doc, "public_url_expires_at"); v && v->is_number()) f.public_url_expires_at = v->as_int();
|
||||
return f;
|
||||
}
|
||||
|
||||
std::optional<MediaError> MediaError::parse(const Json& doc, const char* key) {
|
||||
const Json* e = jx::get(doc, key);
|
||||
if (!e || !e->is_object()) return std::nullopt;
|
||||
return MediaError{jx::str(*e, "code"), jx::str(*e, "message")};
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,65 @@
|
||||
#pragma once
|
||||
|
||||
// Pieces shared by image and video generation: media sources (URL, data URL
|
||||
// or Files API id), file storage options, uploaded output, stored-file
|
||||
// references and generation errors.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/params.h"
|
||||
#include "xai/core/usage.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
// `ImageUrl` / `VideoUrl`: exactly one of a URL (public or base64 data URL)
|
||||
// or a Files API id.
|
||||
class MediaSource {
|
||||
public:
|
||||
MediaSource() : json_(Json::object()) {} // empty: rejected wherever a source is required
|
||||
bool empty() const { return json_.as_object().empty(); }
|
||||
static MediaSource image_url(std::string url);
|
||||
static MediaSource image_file(std::string file_id);
|
||||
// Video sources must be .mp4 (spec), 2..15 s for extensions (docs).
|
||||
static MediaSource video_url(std::string url);
|
||||
static MediaSource video_file(std::string file_id);
|
||||
static MediaSource audio_url(std::string url); // reference audio: data: URL or http(s)
|
||||
static MediaSource audio_voice(std::string voice_id); // preset voice id (same as TTS voices)
|
||||
|
||||
Json to_json() const { return json_; }
|
||||
|
||||
private:
|
||||
explicit MediaSource(Json j) : json_(std::move(j)) {}
|
||||
Json json_;
|
||||
};
|
||||
|
||||
// `storage_options`: keep the generated media as a Files API file.
|
||||
struct StorageOptions {
|
||||
std::string filename; // required
|
||||
std::optional<long> expires_after; // seconds, <= 2592000
|
||||
bool public_url = false; // create a public URL
|
||||
std::optional<long> public_url_expires_after; // seconds, 3600..2592000
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
// `output`: have the API upload the media to signed URLs instead of
|
||||
// hosting it (one URL per image, a single one for a video).
|
||||
struct UploadOutput {
|
||||
std::vector<std::string> urls;
|
||||
};
|
||||
|
||||
// `FileOutput` in a result: the stored file.
|
||||
struct StoredFile {
|
||||
std::string file_id, filename, public_url, public_url_error;
|
||||
std::optional<long> expires_at, public_url_expires_at;
|
||||
static StoredFile parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct MediaError {
|
||||
std::string code; // invalid_argument | permission_denied | failed_precondition | service_unavailable | internal_error
|
||||
std::string message;
|
||||
static std::optional<MediaError> parse(const Json& doc, const char* key);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,393 @@
|
||||
#include "xai/core/messages.h"
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
Json block(const char* type) {
|
||||
Json o = Json::object();
|
||||
o.set("type", Json(type));
|
||||
return o;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
MessageBlock MessageBlock::text(std::string text) {
|
||||
Check::text("text", text, Caps::kMaxTextBytes, true);
|
||||
Json o = block("text");
|
||||
o.set("text", Json(std::move(text)));
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessageBlock MessageBlock::image_base64(std::string media_type, std::string data) {
|
||||
Check::one_of("media_type", media_type, {"image/jpeg", "image/png", "image/webp"});
|
||||
Check::text("image data", data, Caps::kMaxUrlLen);
|
||||
Json src = block("base64");
|
||||
src.set("media_type", Json(std::move(media_type)));
|
||||
src.set("data", Json(std::move(data)));
|
||||
Json o = block("image");
|
||||
o.set("source", src);
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessageBlock MessageBlock::image_url(std::string url) {
|
||||
Check::http_url("image url", url);
|
||||
Json src = block("url");
|
||||
src.set("url", Json(std::move(url)));
|
||||
Json o = block("image");
|
||||
o.set("source", src);
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessageBlock MessageBlock::tool_use(std::string id, std::string name, Json input) {
|
||||
Check::id("tool use", id);
|
||||
Check::id("tool", name);
|
||||
Check::object("tool input", input);
|
||||
Json o = block("tool_use");
|
||||
o.set("id", Json(std::move(id)));
|
||||
o.set("name", Json(std::move(name)));
|
||||
o.set("input", std::move(input));
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessageBlock MessageBlock::tool_result(std::string tool_use_id, std::string content, bool is_error) {
|
||||
Check::id("tool use", tool_use_id);
|
||||
Check::text("tool result", content, Caps::kMaxTextBytes, true);
|
||||
Json o = block("tool_result");
|
||||
o.set("tool_use_id", Json(std::move(tool_use_id)));
|
||||
o.set("content", Json(std::move(content)));
|
||||
if (is_error) o.set("is_error", Json(true));
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessageBlock MessageBlock::thinking(std::string thinking, std::string signature) {
|
||||
Check::text("thinking", thinking, Caps::kMaxTextBytes, true);
|
||||
Json o = block("thinking");
|
||||
o.set("thinking", Json(std::move(thinking)));
|
||||
if (!signature.empty()) o.set("signature", Json(std::move(signature)));
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessageBlock MessageBlock::redacted_thinking(std::string data) {
|
||||
Check::text("redacted data", data, Caps::kMaxTextBytes);
|
||||
Json o = block("redacted_thinking");
|
||||
o.set("data", Json(std::move(data)));
|
||||
return MessageBlock(std::move(o));
|
||||
}
|
||||
|
||||
MessagesRequest& MessagesRequest::user(std::string text) {
|
||||
Check::text("message", text, Caps::kMaxTextBytes);
|
||||
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("user"));
|
||||
m.set("content", Json(std::move(text)));
|
||||
messages_.push_back(std::move(m));
|
||||
return *this;
|
||||
}
|
||||
|
||||
MessagesRequest& MessagesRequest::assistant(std::string text) {
|
||||
Check::text("message", text, Caps::kMaxTextBytes);
|
||||
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
|
||||
Json m = Json::object();
|
||||
m.set("role", Json("assistant"));
|
||||
m.set("content", Json(std::move(text)));
|
||||
messages_.push_back(std::move(m));
|
||||
return *this;
|
||||
}
|
||||
|
||||
MessagesRequest& MessagesRequest::add(const std::string& role, const std::vector<MessageBlock>& blocks) {
|
||||
Check::one_of("role", role, {"user", "assistant"});
|
||||
Check::count("content blocks", blocks.size(), 1, Caps::kMaxContentParts);
|
||||
Check::upto("messages", messages_.size(), llm::Caps::kMaxMessages - 1);
|
||||
Json arr = Json::array();
|
||||
for (const auto& b : blocks) arr.push(b.json());
|
||||
Json m = Json::object();
|
||||
m.set("role", Json(role));
|
||||
m.set("content", arr);
|
||||
messages_.push_back(std::move(m));
|
||||
return *this;
|
||||
}
|
||||
|
||||
MessagesRequest& MessagesRequest::add_tool(std::string name, std::string description, Json input_schema) {
|
||||
Check::id("tool", name);
|
||||
Check::text("tool description", description, 16u * 1024u);
|
||||
Check::json_schema("input_schema", input_schema);
|
||||
Check::upto("tools", tools_.size(), Caps::kMaxResponsesTools - 1);
|
||||
Json t = Json::object();
|
||||
t.set("name", Json(std::move(name)));
|
||||
t.set("description", Json(std::move(description)));
|
||||
t.set("input_schema", std::move(input_schema));
|
||||
tools_.push_back(std::move(t));
|
||||
return *this;
|
||||
}
|
||||
|
||||
Json MessagesRequest::to_json(bool stream) const {
|
||||
Check::model(model_);
|
||||
Check::range("max_tokens", max_tokens_, 1, 10'000'000);
|
||||
Check::count("messages", messages_.size(), 1, llm::Caps::kMaxMessages);
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model_));
|
||||
body.set("max_tokens", Json(max_tokens_));
|
||||
Json msgs = Json::array();
|
||||
for (const Json& m : messages_) msgs.push(m);
|
||||
body.set("messages", msgs);
|
||||
if (!system_.empty()) {
|
||||
Check::text("system", system_, Caps::kMaxTextBytes);
|
||||
body.set("system", Json(system_));
|
||||
}
|
||||
if (stream) body.set("stream", Json(true));
|
||||
if (temperature_) { Check::range("temperature", *temperature_, 0, 2); body.set("temperature", Json(*temperature_)); }
|
||||
if (top_p_) { Check::range("top_p", *top_p_, 0, 1); body.set("top_p", Json(*top_p_)); }
|
||||
Check::strings("stop_sequences", stop_, Caps::kMaxStop, 256);
|
||||
jx::put_strings(body, "stop_sequences", stop_);
|
||||
if (user_id_) {
|
||||
Check::text("user_id", *user_id_, 256);
|
||||
Json meta = Json::object();
|
||||
meta.set("user_id", Json(*user_id_));
|
||||
body.set("metadata", meta);
|
||||
}
|
||||
if (!tools_.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const Json& t : tools_) arr.push(t);
|
||||
body.set("tools", arr);
|
||||
}
|
||||
if (!choice_.empty()) {
|
||||
Json c = Json::object();
|
||||
c.set("type", Json(choice_));
|
||||
if (choice_ == "tool") {
|
||||
Check::id("tool", choice_name_);
|
||||
c.set("name", Json(choice_name_));
|
||||
}
|
||||
body.set("tool_choice", c);
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
MessagesResponse MessagesResponse::parse(const Json& doc) {
|
||||
MessagesResponse r;
|
||||
r.raw = doc;
|
||||
r.id = jx::str(doc, "id");
|
||||
r.model = jx::str(doc, "model");
|
||||
r.role = jx::str(doc, "role");
|
||||
r.type = jx::str(doc, "type");
|
||||
r.stop_reason = jx::str(doc, "stop_reason");
|
||||
r.stop_sequence = jx::str(doc, "stop_sequence");
|
||||
const Json* content = jx::get(doc, "content");
|
||||
if (content && content->is_array())
|
||||
for (const Json& c : content->as_array()) {
|
||||
if (r.content.size() >= Caps::kMaxAccumulatedItems) break;
|
||||
MessageContent m;
|
||||
m.type = jx::str(c, "type");
|
||||
m.text = m.type == "thinking" ? jx::str(c, "thinking") : jx::str(c, "text");
|
||||
m.signature = jx::str(c, "signature");
|
||||
m.data = jx::str(c, "data");
|
||||
m.id = jx::str(c, "id");
|
||||
m.name = jx::str(c, "name");
|
||||
if (const Json* in = jx::get(c, "input"); in && in->is_object()) m.input = *in;
|
||||
r.content.push_back(std::move(m));
|
||||
}
|
||||
Json u = jx::object(doc, "usage");
|
||||
r.input_tokens = jx::num(u, "input_tokens");
|
||||
r.output_tokens = jx::num(u, "output_tokens");
|
||||
r.cache_read_input_tokens = jx::num(u, "cache_read_input_tokens");
|
||||
r.cache_creation_input_tokens = jx::num(u, "cache_creation_input_tokens");
|
||||
return r;
|
||||
}
|
||||
|
||||
std::string MessagesResponse::text() const {
|
||||
std::string out;
|
||||
for (const auto& c : content)
|
||||
if (c.type == "text") out += c.text;
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string MessagesResponse::thinking() const {
|
||||
std::string out;
|
||||
for (const auto& c : content)
|
||||
if (c.type == "thinking") out += c.text;
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<MessageContent> MessagesResponse::tool_uses() const {
|
||||
std::vector<MessageContent> out;
|
||||
for (const auto& c : content)
|
||||
if (c.type == "tool_use") out.push_back(c);
|
||||
return out;
|
||||
}
|
||||
|
||||
bool MessagesStreamAccumulator::feed(const std::string& event_name, const Json& data) {
|
||||
if (stopped_ || done_) return false;
|
||||
if (++events_ > static_cast<long>(Caps::kMaxStreamEvents)) {
|
||||
stopped_ = true;
|
||||
return false;
|
||||
}
|
||||
std::string type = jx::str(data, "type");
|
||||
if (type.empty()) type = event_name;
|
||||
if (type == "message_start") {
|
||||
Json msg = jx::object(data, "message");
|
||||
MessagesResponse head = MessagesResponse::parse(msg);
|
||||
response_.id = head.id;
|
||||
response_.model = head.model;
|
||||
response_.role = head.role;
|
||||
response_.type = head.type;
|
||||
response_.input_tokens = head.input_tokens;
|
||||
response_.output_tokens = head.output_tokens;
|
||||
response_.cache_read_input_tokens = head.cache_read_input_tokens;
|
||||
} else if (type == "content_block_start") {
|
||||
long index = jx::num(data, "index");
|
||||
if (index < 0 || index >= static_cast<long>(Caps::kMaxAccumulatedItems)) { stopped_ = true; return false; }
|
||||
if (response_.content.size() <= static_cast<std::size_t>(index)) {
|
||||
response_.content.resize(index + 1);
|
||||
input_json_.resize(index + 1);
|
||||
}
|
||||
Json cb = jx::object(data, "content_block");
|
||||
MessageContent& m = response_.content[index];
|
||||
m.type = jx::str(cb, "type");
|
||||
m.text = jx::str(cb, m.type == "thinking" ? "thinking" : "text");
|
||||
m.id = jx::str(cb, "id");
|
||||
m.name = jx::str(cb, "name");
|
||||
m.data = jx::str(cb, "data");
|
||||
} else if (type == "content_block_delta") {
|
||||
long index = jx::num(data, "index");
|
||||
if (index < 0 || index >= static_cast<long>(response_.content.size())) return true;
|
||||
Json d = jx::object(data, "delta");
|
||||
MessageContent& m = response_.content[index];
|
||||
std::string dt = jx::str(d, "type");
|
||||
std::size_t add = 0;
|
||||
if (dt == "text_delta") { m.text += jx::str(d, "text"); add = jx::str(d, "text").size(); }
|
||||
else if (dt == "thinking_delta") { m.text += jx::str(d, "thinking"); add = jx::str(d, "thinking").size(); }
|
||||
else if (dt == "signature_delta") m.signature += jx::str(d, "signature");
|
||||
else if (dt == "input_json_delta") { input_json_[index] += jx::str(d, "partial_json"); add = jx::str(d, "partial_json").size(); }
|
||||
if (add > Caps::kMaxStreamTextBytes || m.text.size() > Caps::kMaxStreamTextBytes) { stopped_ = true; return false; }
|
||||
} else if (type == "content_block_stop") {
|
||||
long index = jx::num(data, "index");
|
||||
if (index >= 0 && index < static_cast<long>(input_json_.size()) && !input_json_[index].empty()) {
|
||||
try {
|
||||
Json parsed = Json::parse(input_json_[index]);
|
||||
if (parsed.is_object()) response_.content[index].input = parsed;
|
||||
} catch (const std::exception&) {
|
||||
// Invalid tool input JSON stays empty; the caller sees an empty input object.
|
||||
}
|
||||
}
|
||||
} else if (type == "message_delta") {
|
||||
Json d = jx::object(data, "delta");
|
||||
if (jx::has_string(d, "stop_reason")) response_.stop_reason = jx::str(d, "stop_reason");
|
||||
if (jx::has_string(d, "stop_sequence")) response_.stop_sequence = jx::str(d, "stop_sequence");
|
||||
Json u = jx::object(data, "usage");
|
||||
if (jx::get(u, "output_tokens")) response_.output_tokens = jx::num(u, "output_tokens");
|
||||
if (jx::get(u, "input_tokens")) response_.input_tokens = jx::num(u, "input_tokens");
|
||||
} else if (type == "message_stop") {
|
||||
done_ = true;
|
||||
} else if (type == "error") {
|
||||
failed_ = true;
|
||||
error_ = jx::str(jx::object(data, "error"), "message");
|
||||
if (error_.empty()) error_ = jx::str(data, "message");
|
||||
done_ = true;
|
||||
}
|
||||
return !done_;
|
||||
}
|
||||
|
||||
MessagesResponse MessagesStreamAccumulator::result() const { return response_; }
|
||||
|
||||
MessagesResponse MessagesService::create(const MessagesRequest& request) {
|
||||
Json body = request.to_json(false);
|
||||
return MessagesResponse::parse(call_json(Call::kMessagesCreate, {}, &body));
|
||||
}
|
||||
|
||||
MessagesResponse MessagesService::stream(const MessagesRequest& request, const EventSink& sink) {
|
||||
Json body = request.to_json(true);
|
||||
MessagesStreamAccumulator acc;
|
||||
call_stream(Call::kMessagesCreate, body, [&](const llm::SseEvent& ev) {
|
||||
bool keep = acc.feed(ev.event, ev.data);
|
||||
bool user_keep = sink ? sink(ev.event, ev.data) : true;
|
||||
return keep && user_keep;
|
||||
});
|
||||
if (acc.failed()) throw llm::ApiError(kVendor, 502, "STREAM_FAILED", acc.error_message());
|
||||
if (acc.stopped()) throw llm::ApiError(kVendor, 502, "STREAM_CAP", "messages stream hit a cap");
|
||||
return acc.result();
|
||||
}
|
||||
|
||||
Json TextCompletionRequest::to_json() const {
|
||||
Check::model(model);
|
||||
Check::count("prompt", prompt.size(), 1, 64);
|
||||
for (const auto& p : prompt) Check::text("prompt", p, Caps::kMaxTextBytes, true);
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model));
|
||||
if (prompt.size() == 1) body.set("prompt", Json(prompt.front()));
|
||||
else body.set("prompt", jx::string_array(prompt));
|
||||
if (max_tokens) { Check::range("max_tokens", *max_tokens, 1, 10'000'000); body.set("max_tokens", Json(*max_tokens)); }
|
||||
if (n) { Check::range("n", *n, 1, static_cast<double>(Caps::kMaxChoices)); body.set("n", Json(*n)); }
|
||||
if (temperature) { Check::range("temperature", *temperature, 0, 2); body.set("temperature", Json(*temperature)); }
|
||||
if (top_p) { Check::range("top_p", *top_p, 0, 1); body.set("top_p", Json(*top_p)); }
|
||||
if (presence_penalty) { Check::range("presence_penalty", *presence_penalty, -2, 2); body.set("presence_penalty", Json(*presence_penalty)); }
|
||||
jx::put(body, "seed", seed);
|
||||
Check::strings("stop", stop, Caps::kMaxStop, 256);
|
||||
jx::put_strings(body, "stop", stop);
|
||||
jx::put(body, "echo", echo);
|
||||
jx::put(body, "logprobs", logprobs);
|
||||
if (!user.empty()) { Check::text("user", user, 256); body.set("user", Json(user)); }
|
||||
return body;
|
||||
}
|
||||
|
||||
TextCompletion TextCompletion::parse(const Json& doc) {
|
||||
TextCompletion t;
|
||||
t.raw = doc;
|
||||
t.id = jx::str(doc, "id");
|
||||
t.model = jx::str(doc, "model");
|
||||
t.object = jx::str(doc, "object");
|
||||
t.system_fingerprint = jx::str(doc, "system_fingerprint");
|
||||
t.created = jx::num(doc, "created");
|
||||
const Json* choices = jx::get(doc, "choices");
|
||||
if (choices && choices->is_array())
|
||||
for (const Json& c : choices->as_array()) {
|
||||
if (t.choices.size() >= Caps::kMaxChoices) break;
|
||||
t.choices.push_back({jx::num(c, "index"), jx::str(c, "text"), jx::str(c, "finish_reason")});
|
||||
}
|
||||
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) t.usage = TokenUsage::from_chat(*u);
|
||||
return t;
|
||||
}
|
||||
|
||||
Json AnthropicCompleteRequest::to_json() const {
|
||||
Check::model(model);
|
||||
Check::text("prompt", prompt, Caps::kMaxTextBytes);
|
||||
Check::range("max_tokens_to_sample", max_tokens_to_sample, 1, 10'000'000);
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model));
|
||||
body.set("prompt", Json(prompt));
|
||||
body.set("max_tokens_to_sample", Json(max_tokens_to_sample));
|
||||
if (temperature) { Check::range("temperature", *temperature, 0, 2); body.set("temperature", Json(*temperature)); }
|
||||
if (top_p) { Check::range("top_p", *top_p, 0, 1); body.set("top_p", Json(*top_p)); }
|
||||
Check::strings("stop_sequences", stop_sequences, Caps::kMaxStop, 256);
|
||||
jx::put_strings(body, "stop_sequences", stop_sequences);
|
||||
if (!user_id.empty()) {
|
||||
Check::text("user_id", user_id, 256);
|
||||
Json meta = Json::object();
|
||||
meta.set("user_id", Json(user_id));
|
||||
body.set("metadata", meta);
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
AnthropicComplete AnthropicComplete::parse(const Json& doc) {
|
||||
AnthropicComplete c;
|
||||
c.raw = doc;
|
||||
c.id = jx::str(doc, "id");
|
||||
c.model = jx::str(doc, "model");
|
||||
c.type = jx::str(doc, "type");
|
||||
c.completion = jx::str(doc, "completion");
|
||||
c.stop_reason = jx::str(doc, "stop_reason");
|
||||
return c;
|
||||
}
|
||||
|
||||
TextCompletion LegacyService::completions(const TextCompletionRequest& request) {
|
||||
Json body = request.to_json();
|
||||
return TextCompletion::parse(call_json(Call::kCompletionsLegacy, {}, &body));
|
||||
}
|
||||
|
||||
AnthropicComplete LegacyService::complete(const AnthropicCompleteRequest& request) {
|
||||
Json body = request.to_json();
|
||||
return AnthropicComplete::parse(call_json(Call::kCompleteLegacy, {}, &body));
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,167 @@
|
||||
#pragma once
|
||||
|
||||
// Anthropic-compatible Messages endpoint (POST /v1/messages) and the legacy
|
||||
// text completion endpoints (/v1/complete Anthropic style, /v1/completions
|
||||
// OpenAI style). xAI marks the Anthropic SDK compatibility "fully deprecated"
|
||||
// in favour of the Responses API; it is still in the OpenAPI document, so it
|
||||
// is covered here. The stream event vocabulary is Anthropic's and is listed
|
||||
// under Unverified in the SPEC.
|
||||
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/service.h"
|
||||
#include "xai/core/usage.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class MessageBlock {
|
||||
public:
|
||||
static MessageBlock text(std::string text);
|
||||
static MessageBlock image_base64(std::string media_type, std::string data); // jpeg, png, webp
|
||||
static MessageBlock image_url(std::string url);
|
||||
static MessageBlock tool_use(std::string id, std::string name, Json input);
|
||||
static MessageBlock tool_result(std::string tool_use_id, std::string content, bool is_error = false);
|
||||
static MessageBlock thinking(std::string thinking, std::string signature = "");
|
||||
static MessageBlock redacted_thinking(std::string data);
|
||||
const Json& json() const { return json_; }
|
||||
|
||||
private:
|
||||
explicit MessageBlock(Json j) : json_(std::move(j)) {}
|
||||
Json json_;
|
||||
};
|
||||
|
||||
class MessagesRequest {
|
||||
public:
|
||||
// max_tokens is required by the endpoint.
|
||||
MessagesRequest(std::string model, int max_tokens) : model_(std::move(model)), max_tokens_(max_tokens) {}
|
||||
|
||||
MessagesRequest& system(std::string text) { system_ = std::move(text); return *this; }
|
||||
MessagesRequest& user(std::string text);
|
||||
MessagesRequest& assistant(std::string text);
|
||||
MessagesRequest& add(const std::string& role, const std::vector<MessageBlock>& blocks);
|
||||
MessagesRequest& temperature(double v) { temperature_ = v; return *this; }
|
||||
MessagesRequest& top_p(double v) { top_p_ = v; return *this; }
|
||||
MessagesRequest& stop_sequences(std::vector<std::string> s) { stop_ = std::move(s); return *this; }
|
||||
MessagesRequest& user_id(std::string id) { user_id_ = std::move(id); return *this; }
|
||||
MessagesRequest& add_tool(std::string name, std::string description, Json input_schema);
|
||||
// auto | any | tool (named).
|
||||
MessagesRequest& tool_choice_auto() { choice_ = "auto"; return *this; }
|
||||
MessagesRequest& tool_choice_any() { choice_ = "any"; return *this; }
|
||||
MessagesRequest& tool_choice_tool(std::string name) { choice_ = "tool"; choice_name_ = std::move(name); return *this; }
|
||||
|
||||
Json to_json(bool stream = false) const;
|
||||
|
||||
private:
|
||||
std::string model_;
|
||||
int max_tokens_;
|
||||
std::string system_, choice_, choice_name_;
|
||||
std::vector<Json> messages_;
|
||||
std::vector<Json> tools_;
|
||||
std::optional<double> temperature_, top_p_;
|
||||
std::vector<std::string> stop_;
|
||||
std::optional<std::string> user_id_;
|
||||
};
|
||||
|
||||
struct MessageContent {
|
||||
std::string type; // text | thinking | redacted_thinking | tool_use
|
||||
std::string text; // text, or thinking text
|
||||
std::string signature; // thinking
|
||||
std::string data; // redacted_thinking
|
||||
std::string id, name; // tool_use
|
||||
Json input; // tool_use
|
||||
};
|
||||
|
||||
struct MessagesResponse {
|
||||
std::string id, model, role, type, stop_reason, stop_sequence;
|
||||
std::vector<MessageContent> content;
|
||||
long input_tokens = 0, output_tokens = 0, cache_read_input_tokens = 0, cache_creation_input_tokens = 0;
|
||||
Json raw;
|
||||
|
||||
static MessagesResponse parse(const Json& doc);
|
||||
std::string text() const;
|
||||
std::string thinking() const;
|
||||
std::vector<MessageContent> tool_uses() const;
|
||||
};
|
||||
|
||||
// Folds Anthropic-style stream events (message_start, content_block_start,
|
||||
// content_block_delta with text_delta / thinking_delta / signature_delta /
|
||||
// input_json_delta, content_block_stop, message_delta, message_stop, ping,
|
||||
// error) into a MessagesResponse.
|
||||
class MessagesStreamAccumulator {
|
||||
public:
|
||||
bool feed(const std::string& event_name, const Json& data); // false when a cap stopped it
|
||||
MessagesResponse result() const;
|
||||
bool stopped() const { return stopped_; }
|
||||
bool failed() const { return failed_; }
|
||||
const std::string& error_message() const { return error_; }
|
||||
bool done() const { return done_; }
|
||||
|
||||
private:
|
||||
MessagesResponse response_;
|
||||
std::vector<std::string> input_json_; // per block index
|
||||
std::string error_;
|
||||
long events_ = 0;
|
||||
bool stopped_ = false, failed_ = false, done_ = false;
|
||||
};
|
||||
|
||||
class MessagesService : public XaiService {
|
||||
public:
|
||||
// false stops; delta text is "" for non-text events.
|
||||
using EventSink = std::function<bool(const std::string& event, const Json& data)>;
|
||||
|
||||
explicit MessagesService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
MessagesResponse create(const MessagesRequest& request);
|
||||
MessagesResponse stream(const MessagesRequest& request, const EventSink& sink = {});
|
||||
};
|
||||
|
||||
// ---- Legacy text completions --------------------------------------------------
|
||||
|
||||
struct TextCompletionRequest { // POST /v1/completions
|
||||
std::string model;
|
||||
std::vector<std::string> prompt; // one string, or several
|
||||
std::optional<int> max_tokens, n;
|
||||
std::optional<double> temperature, top_p, presence_penalty;
|
||||
std::optional<long> seed;
|
||||
std::vector<std::string> stop;
|
||||
std::optional<bool> echo, logprobs;
|
||||
std::string user;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct TextCompletion {
|
||||
std::string id, model, object, system_fingerprint;
|
||||
long created = 0;
|
||||
struct Choice { long index = 0; std::string text, finish_reason; };
|
||||
std::vector<Choice> choices;
|
||||
TokenUsage usage;
|
||||
Json raw;
|
||||
static TextCompletion parse(const Json& doc);
|
||||
std::string text() const { return choices.empty() ? "" : choices.front().text; }
|
||||
};
|
||||
|
||||
struct AnthropicCompleteRequest { // POST /v1/complete
|
||||
std::string model, prompt;
|
||||
int max_tokens_to_sample = 0;
|
||||
std::optional<double> temperature, top_p;
|
||||
std::vector<std::string> stop_sequences;
|
||||
std::string user_id;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct AnthropicComplete {
|
||||
std::string id, model, type, completion, stop_reason;
|
||||
Json raw;
|
||||
static AnthropicComplete parse(const Json& doc);
|
||||
};
|
||||
|
||||
class LegacyService : public XaiService {
|
||||
public:
|
||||
explicit LegacyService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
TextCompletion completions(const TextCompletionRequest& request);
|
||||
AnthropicComplete complete(const AnthropicCompleteRequest& request);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,94 @@
|
||||
#include "xai/core/models.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
const char* const kPriceKeys[] = {
|
||||
"prompt_text_token_price", "cached_prompt_text_token_price", "completion_text_token_price",
|
||||
"prompt_text_token_price_long_context", "cached_prompt_text_token_price_long_context",
|
||||
"completion_text_token_price_long_context", "prompt_image_token_price", "search_price",
|
||||
"image_price"};
|
||||
|
||||
Call list_call(ModelKind k) {
|
||||
switch (k) {
|
||||
case ModelKind::kLanguage: return Call::kLanguageModelsList;
|
||||
case ModelKind::kImage: return Call::kImageModelsList;
|
||||
case ModelKind::kVideo: return Call::kVideoModelsList;
|
||||
case ModelKind::kEmbedding: return Call::kEmbeddingModelsList;
|
||||
default: return Call::kModelsList;
|
||||
}
|
||||
}
|
||||
Call get_call(ModelKind k) {
|
||||
switch (k) {
|
||||
case ModelKind::kLanguage: return Call::kLanguageModelsGet;
|
||||
case ModelKind::kImage: return Call::kImageModelsGet;
|
||||
case ModelKind::kVideo: return Call::kVideoModelsGet;
|
||||
case ModelKind::kEmbedding: return Call::kEmbeddingModelsGet;
|
||||
default: return Call::kModelsGet;
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
|
||||
ModelInfo ModelInfo::parse(ModelKind kind, const Json& d) {
|
||||
ModelInfo m;
|
||||
m.kind = kind;
|
||||
m.raw = d;
|
||||
m.id = jx::str(d, "id");
|
||||
m.object = jx::str(d, "object");
|
||||
m.owned_by = jx::str(d, "owned_by");
|
||||
m.version = jx::str(d, "version");
|
||||
m.fingerprint = jx::str(d, "fingerprint");
|
||||
m.created = jx::num(d, "created");
|
||||
m.aliases = jx::strings(d, "aliases");
|
||||
m.input_modalities = jx::strings(d, "input_modalities");
|
||||
m.output_modalities = jx::strings(d, "output_modalities");
|
||||
auto opt = [&](const char* k) -> std::optional<long> {
|
||||
const Json* v = jx::get(d, k);
|
||||
if (v && v->is_number()) return v->as_int();
|
||||
return std::nullopt;
|
||||
};
|
||||
m.context_length = opt("context_length");
|
||||
m.long_context_threshold = opt("long_context_threshold");
|
||||
m.max_prompt_length = opt("max_prompt_length");
|
||||
for (const char* k : kPriceKeys)
|
||||
if (auto v = opt(k)) m.prices[k] = *v;
|
||||
const Json* pricing = jx::get(d, "pricing");
|
||||
if (pricing && pricing->is_array())
|
||||
for (const Json& t : pricing->as_array()) {
|
||||
if (m.image_pricing.size() >= 64) break;
|
||||
m.image_pricing.push_back({jx::str(t, "quality"), jx::str(t, "resolution"), jx::num(t, "price_per_image")});
|
||||
}
|
||||
Json caps = jx::object(d, "capabilities");
|
||||
m.reasoning_efforts = jx::strings(caps, "reasoning_effort");
|
||||
m.default_reasoning_effort = jx::str(caps, "default_reasoning_effort");
|
||||
return m;
|
||||
}
|
||||
|
||||
bool ModelInfo::accepts_effort(const std::string& effort) const {
|
||||
for (const auto& e : reasoning_efforts)
|
||||
if (e == effort) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
ModelList ModelsService::list(ModelKind kind) {
|
||||
Json doc = call_json(list_call(kind));
|
||||
ModelList out;
|
||||
const Json* arr = jx::get(doc, kind == ModelKind::kGeneric ? "data" : "models");
|
||||
if (arr && arr->is_array())
|
||||
for (const Json& e : arr->as_array()) {
|
||||
if (out.models.size() >= Caps::kMaxAccumulatedItems) {
|
||||
out.truncated = true;
|
||||
break;
|
||||
}
|
||||
out.models.push_back(ModelInfo::parse(kind, e));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
ModelInfo ModelsService::get(ModelKind kind, const std::string& model_id) {
|
||||
return ModelInfo::parse(kind, call_json(get_call(kind), {model_id}));
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,58 @@
|
||||
#pragma once
|
||||
|
||||
// Models API: /v1/models (generic, OpenAI-compatible), /v1/language-models,
|
||||
// /v1/image-generation-models, /v1/video-generation-models,
|
||||
// /v1/embedding-models. One ModelInfo covers all five shapes; fields a shape
|
||||
// does not carry stay at their defaults. Prices are as xAI reports them
|
||||
// (USD cents per 100 million tokens unless stated; embedding prices per
|
||||
// million tokens; image pricing tiers in 1/100,000,000 USD cent).
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/service.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
enum class ModelKind { kGeneric, kLanguage, kImage, kVideo, kEmbedding };
|
||||
|
||||
struct ImagePriceTier {
|
||||
std::string quality, resolution;
|
||||
long price_per_image = 0;
|
||||
};
|
||||
|
||||
struct ModelInfo {
|
||||
ModelKind kind = ModelKind::kGeneric;
|
||||
std::string id, object, owned_by, version, fingerprint;
|
||||
long created = 0;
|
||||
std::vector<std::string> aliases, input_modalities, output_modalities;
|
||||
std::optional<long> context_length; // generic /v1/models only
|
||||
std::optional<long> long_context_threshold;
|
||||
// Named prices: prompt_text_token_price, cached_prompt_text_token_price,
|
||||
// completion_text_token_price, *_long_context, prompt_image_token_price,
|
||||
// search_price, image_price.
|
||||
std::map<std::string, long> prices;
|
||||
std::vector<ImagePriceTier> image_pricing;
|
||||
std::optional<long> max_prompt_length; // image models
|
||||
std::vector<std::string> reasoning_efforts; // capabilities.reasoning_effort
|
||||
std::string default_reasoning_effort;
|
||||
Json raw;
|
||||
|
||||
static ModelInfo parse(ModelKind kind, const Json& doc);
|
||||
bool accepts_effort(const std::string& effort) const;
|
||||
};
|
||||
|
||||
struct ModelList {
|
||||
std::vector<ModelInfo> models;
|
||||
bool truncated = false; // more entries than Caps::kMaxAccumulatedItems
|
||||
};
|
||||
|
||||
class ModelsService : public XaiService {
|
||||
public:
|
||||
explicit ModelsService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
ModelList list(ModelKind kind = ModelKind::kGeneric);
|
||||
ModelInfo get(ModelKind kind, const std::string& model_id);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,197 @@
|
||||
#include "xai/core/params.h"
|
||||
|
||||
#include <cmath>
|
||||
|
||||
#include "llm/api_client.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
void Check::fail(const std::string& message) {
|
||||
throw llm::ApiError::invalid(kVendor, message);
|
||||
}
|
||||
|
||||
void Check::model(const std::string& model) {
|
||||
if (model.empty() || model.size() > Caps::kMaxModelLen) fail("model must be 1.." + std::to_string(Caps::kMaxModelLen) + " characters");
|
||||
for (unsigned char c : model)
|
||||
if (c <= 0x20 || c >= 0x7f) fail("model has an invalid character");
|
||||
}
|
||||
|
||||
void Check::id(const std::string& what, const std::string& id) {
|
||||
if (!llm::Service::valid_id(id)) fail("bad " + what + " id");
|
||||
}
|
||||
|
||||
void Check::non_empty(const char* name, const std::string& value) {
|
||||
if (value.empty()) fail(std::string(name) + " is required");
|
||||
}
|
||||
|
||||
void Check::text(const char* name, const std::string& value, std::size_t max_bytes,
|
||||
bool allow_empty) {
|
||||
if (value.empty() && !allow_empty) fail(std::string(name) + " is required");
|
||||
if (value.size() > max_bytes)
|
||||
fail(std::string(name) + " is over " + std::to_string(max_bytes) + " bytes");
|
||||
}
|
||||
|
||||
void Check::token(const char* name, const std::string& value) {
|
||||
if (value.empty() || value.size() > Caps::kMaxTokenLen)
|
||||
fail(std::string(name) + " must be 1.." + std::to_string(Caps::kMaxTokenLen) + " characters");
|
||||
for (char c : value) {
|
||||
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
|
||||
c == '.' || c == '_' || c == ':' || c == '/' || c == '+' || c == '-';
|
||||
if (!ok) fail(std::string(name) + " has an invalid character");
|
||||
}
|
||||
}
|
||||
|
||||
void Check::one_of(const char* name, const std::string& value,
|
||||
std::initializer_list<const char*> allowed) {
|
||||
for (const char* a : allowed)
|
||||
if (value == a) return;
|
||||
std::string list;
|
||||
for (const char* a : allowed) list += (list.empty() ? "" : ", ") + std::string(a);
|
||||
fail(std::string(name) + " must be one of " + list);
|
||||
}
|
||||
|
||||
void Check::range(const char* name, double value, double lo, double hi) {
|
||||
if (std::isnan(value) || value < lo || value > hi)
|
||||
fail(std::string(name) + " must be between " + std::to_string(lo) + " and " + std::to_string(hi));
|
||||
}
|
||||
|
||||
void Check::count(const char* name, std::size_t n, std::size_t lo, std::size_t hi) {
|
||||
if (n < lo || n > hi)
|
||||
fail(std::string(name) + " must hold " + std::to_string(lo) + ".." + std::to_string(hi) + " items");
|
||||
}
|
||||
|
||||
void Check::upto(const char* name, std::size_t n, std::size_t hi) {
|
||||
if (n > hi) fail(std::string(name) + " holds more than " + std::to_string(hi) + " items");
|
||||
}
|
||||
|
||||
void Check::url(const char* name, const std::string& value) {
|
||||
non_empty(name, value);
|
||||
if (value.size() > Caps::kMaxUrlLen) fail(std::string(name) + " is over the size cap");
|
||||
bool ok = value.rfind("https://", 0) == 0 || value.rfind("http://", 0) == 0 ||
|
||||
value.rfind("data:", 0) == 0;
|
||||
if (!ok) fail(std::string(name) + " must be an http(s) or data: URL");
|
||||
}
|
||||
|
||||
void Check::http_url(const char* name, const std::string& value) {
|
||||
non_empty(name, value);
|
||||
if (value.size() > 4096) fail(std::string(name) + " is over 4096 characters");
|
||||
if (value.rfind("https://", 0) != 0 && value.rfind("http://", 0) != 0)
|
||||
fail(std::string(name) + " must be an http(s) URL");
|
||||
for (unsigned char c : value)
|
||||
if (c <= 0x20 || c >= 0x7f) fail(std::string(name) + " has an invalid character");
|
||||
}
|
||||
|
||||
void Check::date(const char* name, const std::string& value) {
|
||||
bool ok = value.size() == 10 && value[4] == '-' && value[7] == '-';
|
||||
for (std::size_t i = 0; ok && i < value.size(); ++i)
|
||||
if (i != 4 && i != 7 && (value[i] < '0' || value[i] > '9')) ok = false;
|
||||
if (ok) {
|
||||
int month = std::stoi(value.substr(5, 2));
|
||||
int day = std::stoi(value.substr(8, 2));
|
||||
ok = month >= 1 && month <= 12 && day >= 1 && day <= 31;
|
||||
}
|
||||
if (!ok) fail(std::string(name) + " must be an ISO-8601 date YYYY-MM-DD");
|
||||
}
|
||||
|
||||
void Check::strings(const char* name, const std::vector<std::string>& v, std::size_t max,
|
||||
std::size_t each_max) {
|
||||
upto(name, v.size(), max);
|
||||
for (const auto& s : v) {
|
||||
if (s.empty()) fail(std::string(name) + " has an empty entry");
|
||||
if (s.size() > each_max) fail(std::string(name) + " has an entry over " + std::to_string(each_max) + " bytes");
|
||||
}
|
||||
}
|
||||
|
||||
void Check::object(const char* name, const Json& value) {
|
||||
if (!value.is_object()) fail(std::string(name) + " must be a JSON object");
|
||||
}
|
||||
|
||||
void Check::json_schema(const char* name, const Json& value) {
|
||||
object(name, value);
|
||||
if (value.dump().size() > Caps::kMaxTextBytes) fail(std::string(name) + " is over the size cap");
|
||||
}
|
||||
|
||||
namespace jx {
|
||||
|
||||
const Json* get(const Json& doc, const char* key) { return doc.is_object() ? doc.find(key) : nullptr; }
|
||||
|
||||
std::string str(const Json& doc, const char* key, const std::string& fallback) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_string() ? v->as_string() : fallback;
|
||||
}
|
||||
|
||||
long num(const Json& doc, const char* key, long fallback) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_number() ? v->as_int() : fallback;
|
||||
}
|
||||
|
||||
double dbl(const Json& doc, const char* key, double fallback) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_number() ? v->as_number() : fallback;
|
||||
}
|
||||
|
||||
bool flag(const Json& doc, const char* key, bool fallback) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_bool() ? v->as_bool() : fallback;
|
||||
}
|
||||
|
||||
std::vector<std::string> strings(const Json& doc, const char* key) {
|
||||
std::vector<std::string> out;
|
||||
const Json* v = get(doc, key);
|
||||
if (v && v->is_array())
|
||||
for (const Json& e : v->as_array())
|
||||
if (e.is_string()) out.push_back(e.as_string());
|
||||
return out;
|
||||
}
|
||||
|
||||
Json object(const Json& doc, const char* key) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_object() ? *v : Json::object();
|
||||
}
|
||||
|
||||
Json array(const Json& doc, const char* key) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_array() ? *v : Json::array();
|
||||
}
|
||||
|
||||
bool has_string(const Json& doc, const char* key) {
|
||||
const Json* v = get(doc, key);
|
||||
return v && v->is_string();
|
||||
}
|
||||
|
||||
void put(Json& doc, const char* key, const std::optional<std::string>& v) {
|
||||
if (v) doc.set(key, Json(*v));
|
||||
}
|
||||
void put(Json& doc, const char* key, const std::optional<int>& v) {
|
||||
if (v) doc.set(key, Json(*v));
|
||||
}
|
||||
void put(Json& doc, const char* key, const std::optional<long>& v) {
|
||||
if (v) doc.set(key, Json(*v));
|
||||
}
|
||||
void put(Json& doc, const char* key, const std::optional<double>& v) {
|
||||
if (v) doc.set(key, Json(*v));
|
||||
}
|
||||
void put(Json& doc, const char* key, const std::optional<bool>& v) {
|
||||
if (v) doc.set(key, Json(*v));
|
||||
}
|
||||
|
||||
Json string_array(const std::vector<std::string>& v) {
|
||||
Json arr = Json::array();
|
||||
for (const auto& s : v) arr.push(Json(s));
|
||||
return arr;
|
||||
}
|
||||
|
||||
void put_strings(Json& doc, const char* key, const std::vector<std::string>& v) {
|
||||
if (!v.empty()) doc.set(key, string_array(v));
|
||||
}
|
||||
|
||||
void merge(Json& doc, const Json& extra) {
|
||||
if (!extra.is_object()) return;
|
||||
if (extra.as_object().size() > 64) Check::fail("extra holds more than 64 members");
|
||||
for (const auto& [k, v] : extra.as_object()) doc.set(k, v);
|
||||
}
|
||||
|
||||
} // namespace jx
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,75 @@
|
||||
#pragma once
|
||||
|
||||
// Input validation (Check) and tolerant JSON readers (jx) shared by every
|
||||
// xAI core class. Validation throws llm::ApiError::invalid (HTTP 400,
|
||||
// INVALID_ARGUMENT) before any network traffic; readers never throw on a
|
||||
// missing field, they return the stated default.
|
||||
|
||||
#include <initializer_list>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
using Json = clients::body::Json;
|
||||
|
||||
inline constexpr const char* kVendor = "xai";
|
||||
|
||||
class Check {
|
||||
public:
|
||||
[[noreturn]] static void fail(const std::string& message);
|
||||
|
||||
// 1..kMaxModelLen printable ASCII without spaces.
|
||||
static void model(const std::string& model);
|
||||
// llm::Service::valid_id, reported as "bad <what> id".
|
||||
static void id(const std::string& what, const std::string& id);
|
||||
static void non_empty(const char* name, const std::string& value);
|
||||
static void text(const char* name, const std::string& value, std::size_t max_bytes,
|
||||
bool allow_empty = false);
|
||||
// Enum-like token: 1..kMaxTokenLen of [A-Za-z0-9._:/+-].
|
||||
static void token(const char* name, const std::string& value);
|
||||
static void one_of(const char* name, const std::string& value,
|
||||
std::initializer_list<const char*> allowed);
|
||||
static void range(const char* name, double value, double lo, double hi);
|
||||
static void count(const char* name, std::size_t n, std::size_t lo, std::size_t hi);
|
||||
static void upto(const char* name, std::size_t n, std::size_t hi);
|
||||
// http(s):// or data: URL, bounded.
|
||||
static void url(const char* name, const std::string& value);
|
||||
static void http_url(const char* name, const std::string& value);
|
||||
// YYYY-MM-DD with a plausible month and day.
|
||||
static void date(const char* name, const std::string& value);
|
||||
static void strings(const char* name, const std::vector<std::string>& v, std::size_t max,
|
||||
std::size_t each_max);
|
||||
// A JSON object (not null, not array).
|
||||
static void object(const char* name, const Json& value);
|
||||
static void json_schema(const char* name, const Json& value);
|
||||
};
|
||||
|
||||
// Tolerant readers.
|
||||
namespace jx {
|
||||
const Json* get(const Json& doc, const char* key);
|
||||
std::string str(const Json& doc, const char* key, const std::string& fallback = "");
|
||||
long num(const Json& doc, const char* key, long fallback = 0);
|
||||
double dbl(const Json& doc, const char* key, double fallback = 0);
|
||||
bool flag(const Json& doc, const char* key, bool fallback = false);
|
||||
std::vector<std::string> strings(const Json& doc, const char* key);
|
||||
Json object(const Json& doc, const char* key); // {} when absent
|
||||
Json array(const Json& doc, const char* key); // [] when absent
|
||||
bool has_string(const Json& doc, const char* key);
|
||||
// Sets only when the value is present.
|
||||
void put(Json& doc, const char* key, const std::optional<std::string>& v);
|
||||
void put(Json& doc, const char* key, const std::optional<int>& v);
|
||||
void put(Json& doc, const char* key, const std::optional<long>& v);
|
||||
void put(Json& doc, const char* key, const std::optional<double>& v);
|
||||
void put(Json& doc, const char* key, const std::optional<bool>& v);
|
||||
void put_strings(Json& doc, const char* key, const std::vector<std::string>& v);
|
||||
Json string_array(const std::vector<std::string>& v);
|
||||
// Merge `extra` object members into `doc` (extra wins) with a hard cap on members.
|
||||
void merge(Json& doc, const Json& extra);
|
||||
} // namespace jx
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,12 @@
|
||||
#include "xai/core/poll.h"
|
||||
|
||||
#include <chrono>
|
||||
#include <thread>
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
Sleeper real_sleeper() {
|
||||
return [](int ms) { std::this_thread::sleep_for(std::chrono::milliseconds(ms)); };
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
// Bounded polling for deferred results (chat, image, video). Elapsed time is
|
||||
// the sum of the sleeps, so the bound is deterministic and testable with a
|
||||
// fake sleeper. Nothing here waits unbounded.
|
||||
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
struct WaitPolicy {
|
||||
int interval_ms = Caps::kDefaultPollIntervalMs;
|
||||
int timeout_ms = Caps::kDefaultWaitMs;
|
||||
|
||||
void validate() const {
|
||||
Check::range("poll interval ms", interval_ms, Caps::kMinPollIntervalMs, Caps::kMaxPollIntervalMs);
|
||||
Check::range("wait timeout ms", timeout_ms, Caps::kMinPollIntervalMs, Caps::kMaxWaitMs);
|
||||
}
|
||||
// Upper bound of step() calls: one immediately, then one per interval.
|
||||
int max_polls() const { return timeout_ms / interval_ms + 1; }
|
||||
};
|
||||
|
||||
using Sleeper = std::function<void(int ms)>;
|
||||
Sleeper real_sleeper(); // std::this_thread::sleep_for
|
||||
|
||||
// Calls `step` until it yields a value. Returns nullopt when the policy's
|
||||
// budget is spent (the caller reports the timeout loudly).
|
||||
template <typename T>
|
||||
std::optional<T> poll_until(const WaitPolicy& policy, const std::function<std::optional<T>()>& step,
|
||||
const Sleeper& sleeper) {
|
||||
policy.validate();
|
||||
const int polls = policy.max_polls();
|
||||
for (int i = 0; i < polls; ++i) {
|
||||
if (std::optional<T> v = step()) return v;
|
||||
if (i + 1 < polls) sleeper(policy.interval_ms);
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,287 @@
|
||||
#include "xai/core/response.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
ItemKind item_kind_of(const std::string& type) {
|
||||
if (type == "message") return ItemKind::kMessage;
|
||||
if (type == "function_call") return ItemKind::kFunctionCall;
|
||||
if (type == "reasoning") return ItemKind::kReasoning;
|
||||
if (type == "web_search_call") return ItemKind::kWebSearchCall;
|
||||
if (type == "file_search_call") return ItemKind::kFileSearchCall;
|
||||
if (type == "code_interpreter_call") return ItemKind::kCodeInterpreterCall;
|
||||
if (type == "mcp_call") return ItemKind::kMcpCall;
|
||||
if (type == "custom_tool_call") return ItemKind::kCustomToolCall;
|
||||
if (type == "shell_call") return ItemKind::kShellCall;
|
||||
if (type == "image_generation_call") return ItemKind::kImageGenerationCall;
|
||||
if (type == "tool_search_call") return ItemKind::kToolSearchCall;
|
||||
if (type == "tool_search_output") return ItemKind::kToolSearchOutput;
|
||||
return ItemKind::kUnknown;
|
||||
}
|
||||
|
||||
OutputItem OutputItem::parse(const Json& item) {
|
||||
OutputItem o;
|
||||
o.raw_ = item;
|
||||
o.type_ = jx::str(item, "type");
|
||||
// Input-style messages omit `type`; role marks them.
|
||||
if (o.type_.empty() && jx::has_string(item, "role")) o.type_ = "message";
|
||||
o.kind_ = item_kind_of(o.type_);
|
||||
return o;
|
||||
}
|
||||
|
||||
std::string OutputItem::text() const {
|
||||
std::string out;
|
||||
if (kind_ != ItemKind::kMessage) return out;
|
||||
const Json* content = jx::get(raw_, "content");
|
||||
if (content && content->is_string()) return content->as_string();
|
||||
if (content && content->is_array())
|
||||
for (const Json& part : content->as_array())
|
||||
if (jx::str(part, "type") == "output_text" || jx::str(part, "type") == "input_text")
|
||||
out += jx::str(part, "text");
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string OutputItem::refusal() const {
|
||||
std::string out;
|
||||
const Json* content = jx::get(raw_, "content");
|
||||
if (content && content->is_array())
|
||||
for (const Json& part : content->as_array())
|
||||
if (jx::str(part, "type") == "refusal") out += jx::str(part, "refusal");
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<Citation> OutputItem::citations() const {
|
||||
std::vector<Citation> out;
|
||||
const Json* content = jx::get(raw_, "content");
|
||||
if (!content || !content->is_array()) return out;
|
||||
for (const Json& part : content->as_array()) {
|
||||
const Json* ann = jx::get(part, "annotations");
|
||||
if (!ann || !ann->is_array()) continue;
|
||||
for (const Json& a : ann->as_array()) {
|
||||
if (out.size() >= Caps::kMaxAccumulatedItems) return out;
|
||||
Citation c;
|
||||
c.url = jx::str(a, "url");
|
||||
c.title = jx::str(a, "title");
|
||||
if (const Json* s = jx::get(a, "start_index"); s && s->is_number()) c.start_index = s->as_int();
|
||||
if (const Json* e = jx::get(a, "end_index"); e && e->is_number()) c.end_index = e->as_int();
|
||||
out.push_back(std::move(c));
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string OutputItem::arguments_text() const {
|
||||
const Json* a = jx::get(raw_, "arguments");
|
||||
if (!a) a = jx::get(raw_, "input");
|
||||
if (a && a->is_string()) return a->as_string();
|
||||
if (a && a->is_object()) return a->dump();
|
||||
return "";
|
||||
}
|
||||
|
||||
bool OutputItem::arguments_valid() const {
|
||||
std::string t = arguments_text();
|
||||
if (t.empty()) return true;
|
||||
try {
|
||||
return Json::parse(t).is_object();
|
||||
} catch (const std::exception&) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
Json OutputItem::arguments() const {
|
||||
std::string t = arguments_text();
|
||||
if (t.empty()) return Json::object();
|
||||
try {
|
||||
Json j = Json::parse(t);
|
||||
return j.is_object() ? j : Json::object();
|
||||
} catch (const std::exception&) {
|
||||
return Json::object();
|
||||
}
|
||||
}
|
||||
|
||||
std::string OutputItem::reasoning_text() const {
|
||||
std::string out;
|
||||
const Json* c = jx::get(raw_, "content");
|
||||
if (c && c->is_array())
|
||||
for (const Json& e : c->as_array()) out += jx::str(e, "text");
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string OutputItem::summary_text() const {
|
||||
std::string out;
|
||||
const Json* c = jx::get(raw_, "summary");
|
||||
if (c && c->is_array())
|
||||
for (const Json& e : c->as_array()) out += jx::str(e, "text");
|
||||
return out;
|
||||
}
|
||||
|
||||
ServerTool OutputItem::server_tool() const {
|
||||
switch (kind_) {
|
||||
case ItemKind::kWebSearchCall: return ServerTool::kWebSearch;
|
||||
case ItemKind::kFileSearchCall: return ServerTool::kCollectionsSearch;
|
||||
case ItemKind::kCodeInterpreterCall: return ServerTool::kCodeExecution;
|
||||
case ItemKind::kMcpCall: return ServerTool::kMcp;
|
||||
case ItemKind::kImageGenerationCall: return ServerTool::kImageGeneration;
|
||||
case ItemKind::kToolSearchCall:
|
||||
case ItemKind::kToolSearchOutput: return ServerTool::kToolSearch;
|
||||
case ItemKind::kCustomToolCall: {
|
||||
std::string id = call_id();
|
||||
if (id.rfind("xs_", 0) == 0) return ServerTool::kXSearch;
|
||||
if (id.rfind("as_", 0) == 0) return ServerTool::kAttachmentSearch;
|
||||
return ServerTool::kNone;
|
||||
}
|
||||
default: return ServerTool::kNone;
|
||||
}
|
||||
}
|
||||
|
||||
std::string OutputItem::web_search_query() const {
|
||||
return jx::str(jx::object(raw_, "action"), "query");
|
||||
}
|
||||
|
||||
std::vector<std::string> OutputItem::web_search_sources() const {
|
||||
std::vector<std::string> out;
|
||||
Json action = jx::object(raw_, "action");
|
||||
const Json* s = jx::get(action, "sources");
|
||||
if (s && s->is_array())
|
||||
for (const Json& e : s->as_array())
|
||||
if (jx::has_string(e, "url")) out.push_back(jx::str(e, "url"));
|
||||
return out;
|
||||
}
|
||||
|
||||
ModelResponse ModelResponse::parse(const Json& doc) {
|
||||
ModelResponse r;
|
||||
r.raw_ = doc;
|
||||
r.id_ = jx::str(doc, "id");
|
||||
r.model_ = jx::str(doc, "model");
|
||||
r.status_ = jx::str(doc, "status");
|
||||
r.created_at_ = jx::num(doc, "created_at");
|
||||
r.completed_at_ = jx::num(doc, "completed_at");
|
||||
r.previous_ = jx::str(doc, "previous_response_id");
|
||||
r.service_tier_ = jx::str(doc, "service_tier");
|
||||
r.store_ = jx::flag(doc, "store");
|
||||
r.incomplete_reason_ = jx::str(jx::object(doc, "incomplete_details"), "reason");
|
||||
const Json* out = jx::get(doc, "output");
|
||||
if (out && out->is_array())
|
||||
for (const Json& item : out->as_array()) {
|
||||
if (r.output_.size() >= Caps::kMaxAccumulatedItems) break;
|
||||
r.output_.push_back(OutputItem::parse(item));
|
||||
}
|
||||
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) r.usage_ = TokenUsage::from_responses(*u);
|
||||
if (const Json* e = jx::get(doc, "error"); e && e->is_object()) {
|
||||
ResponseError err;
|
||||
err.code = jx::str(*e, "code");
|
||||
err.message = jx::str(*e, "message");
|
||||
r.error_ = err;
|
||||
} else if (e && e->is_string()) {
|
||||
r.error_ = ResponseError{"", e->as_string()};
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
std::string ModelResponse::text() const {
|
||||
std::string out;
|
||||
for (const auto& i : output_)
|
||||
if (i.kind() == ItemKind::kMessage) out += i.text();
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string ModelResponse::reasoning_text() const {
|
||||
std::string out;
|
||||
for (const auto& i : output_)
|
||||
if (i.kind() == ItemKind::kReasoning) out += i.reasoning_text();
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string ModelResponse::summary_text() const {
|
||||
std::string out;
|
||||
for (const auto& i : output_)
|
||||
if (i.kind() == ItemKind::kReasoning) out += i.summary_text();
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<Citation> ModelResponse::citations() const {
|
||||
std::vector<Citation> out;
|
||||
for (const auto& i : output_)
|
||||
if (i.kind() == ItemKind::kMessage)
|
||||
for (auto& c : i.citations()) out.push_back(std::move(c));
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<OutputItem> ModelResponse::function_calls() const {
|
||||
std::vector<OutputItem> out;
|
||||
for (const auto& i : output_)
|
||||
if (i.is_client_call()) out.push_back(i);
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<OutputItem> ModelResponse::server_tool_calls() const {
|
||||
std::vector<OutputItem> out;
|
||||
for (const auto& i : output_)
|
||||
if (i.server_tool() != ServerTool::kNone) out.push_back(i);
|
||||
return out;
|
||||
}
|
||||
|
||||
std::vector<Json> ModelResponse::replay_items() const {
|
||||
std::vector<Json> out;
|
||||
for (const auto& i : output_) {
|
||||
switch (i.kind()) {
|
||||
case ItemKind::kReasoning:
|
||||
if (!i.encrypted_content().empty()) out.push_back(i.raw());
|
||||
break;
|
||||
case ItemKind::kFunctionCall:
|
||||
case ItemKind::kMessage:
|
||||
out.push_back(i.raw());
|
||||
break;
|
||||
default:
|
||||
if (i.server_tool() != ServerTool::kNone) out.push_back(i.raw());
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
DeletedResponse DeletedResponse::parse(const Json& doc) {
|
||||
DeletedResponse d;
|
||||
d.id = jx::str(doc, "id");
|
||||
d.deleted = jx::flag(doc, "deleted");
|
||||
d.object = jx::str(doc, "object");
|
||||
return d;
|
||||
}
|
||||
|
||||
InputItemList InputItemList::parse(const Json& doc) {
|
||||
InputItemList l;
|
||||
const Json* data = jx::get(doc, "data");
|
||||
if (data && data->is_array())
|
||||
for (const Json& e : data->as_array()) {
|
||||
if (l.data.size() >= Caps::kMaxAccumulatedItems) { l.truncated = true; break; }
|
||||
l.data.push_back(e);
|
||||
}
|
||||
l.first_id = jx::str(doc, "first_id");
|
||||
l.last_id = jx::str(doc, "last_id");
|
||||
l.object = jx::str(doc, "object");
|
||||
l.has_more = jx::flag(doc, "has_more");
|
||||
return l;
|
||||
}
|
||||
|
||||
CompactResult CompactResult::parse(const Json& doc) {
|
||||
CompactResult c;
|
||||
c.raw = doc;
|
||||
c.id = jx::str(doc, "id");
|
||||
c.model = jx::str(doc, "model");
|
||||
c.object = jx::str(doc, "object");
|
||||
c.created_at = jx::num(doc, "created_at");
|
||||
const Json* out = jx::get(doc, "output");
|
||||
if (out && out->is_array())
|
||||
for (const Json& e : out->as_array()) {
|
||||
if (c.output.size() >= Caps::kMaxAccumulatedItems) break;
|
||||
c.output.push_back(e);
|
||||
}
|
||||
Json u = jx::object(doc, "usage");
|
||||
c.input_tokens = jx::num(u, "input_tokens");
|
||||
c.output_tokens = jx::num(u, "output_tokens");
|
||||
c.total_tokens = jx::num(u, "total_tokens");
|
||||
c.dropped_message_count = jx::num(u, "dropped_message_count");
|
||||
return c;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,151 @@
|
||||
#pragma once
|
||||
|
||||
// Typed view of a Responses API document (`ModelResponse`): output items
|
||||
// (message, function_call, reasoning, the server-side tool calls), usage,
|
||||
// status and error. Parsing is tolerant: unknown item types stay reachable
|
||||
// through raw().
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/params.h"
|
||||
#include "xai/core/usage.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
enum class ItemKind {
|
||||
kMessage, kFunctionCall, kReasoning, kWebSearchCall, kFileSearchCall, kCodeInterpreterCall,
|
||||
kMcpCall, kCustomToolCall, kShellCall, kImageGenerationCall, kToolSearchCall,
|
||||
kToolSearchOutput, kUnknown
|
||||
};
|
||||
|
||||
ItemKind item_kind_of(const std::string& type);
|
||||
|
||||
// Which xAI server-side tool produced an item. X search arrives as a
|
||||
// custom_tool_call whose call_id starts with `xs_`; attachment search with `as_`.
|
||||
enum class ServerTool { kNone, kWebSearch, kXSearch, kAttachmentSearch, kCodeExecution,
|
||||
kCollectionsSearch, kMcp, kImageGeneration, kToolSearch };
|
||||
|
||||
struct Citation {
|
||||
std::string url;
|
||||
std::string title;
|
||||
std::optional<long> start_index;
|
||||
std::optional<long> end_index;
|
||||
};
|
||||
|
||||
class OutputItem {
|
||||
public:
|
||||
static OutputItem parse(const Json& item);
|
||||
|
||||
ItemKind kind() const { return kind_; }
|
||||
const std::string& type() const { return type_; }
|
||||
std::string id() const { return jx::str(raw_, "id"); }
|
||||
std::string status() const { return jx::str(raw_, "status"); }
|
||||
const Json& raw() const { return raw_; }
|
||||
|
||||
// message: concatenated output_text, refusal, citations (url_citation annotations)
|
||||
std::string text() const;
|
||||
std::string refusal() const;
|
||||
std::vector<Citation> citations() const;
|
||||
std::string role() const { return jx::str(raw_, "role"); }
|
||||
|
||||
// function_call / custom_tool_call / mcp_call / shell_call
|
||||
std::string call_id() const { return jx::str(raw_, "call_id"); }
|
||||
std::string name() const { return jx::str(raw_, "name"); }
|
||||
std::string arguments_text() const; // the JSON string as sent
|
||||
Json arguments() const; // parsed; {} when empty or invalid
|
||||
bool arguments_valid() const;
|
||||
|
||||
// reasoning
|
||||
std::string reasoning_text() const; // content[].text
|
||||
std::string summary_text() const; // summary[].text
|
||||
std::string encrypted_content() const { return jx::str(raw_, "encrypted_content"); }
|
||||
|
||||
// server-side tools
|
||||
ServerTool server_tool() const;
|
||||
bool is_client_call() const { return kind_ == ItemKind::kFunctionCall; }
|
||||
std::string web_search_query() const; // web_search_call.action.query
|
||||
std::vector<std::string> web_search_sources() const;
|
||||
std::string code() const { return jx::str(raw_, "code"); } // code_interpreter_call
|
||||
std::string image_base64() const { return jx::str(raw_, "result"); } // image_generation_call
|
||||
|
||||
private:
|
||||
ItemKind kind_ = ItemKind::kUnknown;
|
||||
std::string type_;
|
||||
Json raw_;
|
||||
};
|
||||
|
||||
struct ResponseError {
|
||||
std::string code;
|
||||
std::string message;
|
||||
};
|
||||
|
||||
class ModelResponse {
|
||||
public:
|
||||
static ModelResponse parse(const Json& doc);
|
||||
|
||||
const std::string& id() const { return id_; }
|
||||
const std::string& model() const { return model_; }
|
||||
const std::string& status() const { return status_; } // completed | in_progress | incomplete | failed
|
||||
long created_at() const { return created_at_; }
|
||||
long completed_at() const { return completed_at_; }
|
||||
const std::string& previous_response_id() const { return previous_; }
|
||||
const std::string& service_tier() const { return service_tier_; }
|
||||
bool stored() const { return store_; }
|
||||
const std::vector<OutputItem>& output() const { return output_; }
|
||||
const TokenUsage& usage() const { return usage_; }
|
||||
const std::optional<ResponseError>& error() const { return error_; }
|
||||
const std::string& incomplete_reason() const { return incomplete_reason_; }
|
||||
const Json& raw() const { return raw_; }
|
||||
|
||||
bool completed() const { return status_ == "completed"; }
|
||||
bool incomplete() const { return status_ == "incomplete"; }
|
||||
bool failed() const { return status_ == "failed" || error_.has_value(); }
|
||||
|
||||
std::string text() const; // all message text, in order
|
||||
std::string reasoning_text() const; // raw reasoning content, if the model returned it
|
||||
std::string summary_text() const;
|
||||
std::vector<Citation> citations() const;
|
||||
std::vector<OutputItem> function_calls() const; // client-side, need local execution
|
||||
std::vector<OutputItem> server_tool_calls() const; // handled by xAI
|
||||
// Items to pass back as `input` on the next turn when the history is managed
|
||||
// by the caller (reasoning with encrypted_content, function calls, outputs).
|
||||
std::vector<Json> replay_items() const;
|
||||
|
||||
private:
|
||||
std::string id_, model_, status_, previous_, service_tier_, incomplete_reason_;
|
||||
long created_at_ = 0, completed_at_ = 0;
|
||||
bool store_ = false;
|
||||
std::vector<OutputItem> output_;
|
||||
TokenUsage usage_;
|
||||
std::optional<ResponseError> error_;
|
||||
Json raw_;
|
||||
};
|
||||
|
||||
struct DeletedResponse {
|
||||
std::string id;
|
||||
bool deleted = false;
|
||||
std::string object;
|
||||
static DeletedResponse parse(const Json& doc);
|
||||
};
|
||||
|
||||
// One page of GET /v1/responses/{id}/input_items.
|
||||
struct InputItemList {
|
||||
std::vector<Json> data;
|
||||
std::string first_id, last_id, object;
|
||||
bool has_more = false;
|
||||
bool truncated = false; // set by the walker when a cap stopped it
|
||||
static InputItemList parse(const Json& doc);
|
||||
};
|
||||
|
||||
struct CompactResult {
|
||||
std::string id, model, object;
|
||||
long created_at = 0;
|
||||
std::vector<Json> output; // pass verbatim into the next request's input
|
||||
long input_tokens = 0, output_tokens = 0, total_tokens = 0, dropped_message_count = 0;
|
||||
Json raw;
|
||||
static CompactResult parse(const Json& doc);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,79 @@
|
||||
#include "xai/core/response_format.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
void walk(const Json& node, int depth) {
|
||||
if (depth > ResponseFormat::kMaxSchemaDepth) Check::fail("schema is nested deeper than 64 levels");
|
||||
if (!node.is_object()) return;
|
||||
for (const auto& [key, value] : node.as_object()) {
|
||||
if ((key == "enum" || key == "anyOf" || key == "oneOf") && value.is_array() &&
|
||||
value.as_array().empty())
|
||||
Check::fail("schema keyword " + key + " has zero variants (xAI answers 400)");
|
||||
if (key == "minContains" || key == "maxContains")
|
||||
Check::fail("schema keyword " + key + " is rejected by xAI");
|
||||
if (key == "items" && value.is_array())
|
||||
Check::fail("schema `items` must not be an array; use prefixItems (xAI answers 400)");
|
||||
if (key == "properties" && value.is_object()) {
|
||||
for (const auto& [pname, pschema] : value.as_object())
|
||||
if (pschema.is_bool())
|
||||
Check::fail("property " + pname + " has a boolean schema (xAI answers 400)");
|
||||
}
|
||||
if (value.is_object()) walk(value, depth + 1);
|
||||
if (value.is_array())
|
||||
for (const Json& e : value.as_array()) walk(e, depth + 1);
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void ResponseFormat::lint(const Json& schema) {
|
||||
Check::json_schema("response schema", schema);
|
||||
walk(schema, 0);
|
||||
}
|
||||
|
||||
ResponseFormat ResponseFormat::json_schema(std::string name, Json schema, bool strict) {
|
||||
Check::token("schema name", name);
|
||||
lint(schema);
|
||||
ResponseFormat f(Kind::kJsonSchema);
|
||||
f.name_ = std::move(name);
|
||||
f.schema_ = std::move(schema);
|
||||
f.strict_ = strict;
|
||||
return f;
|
||||
}
|
||||
|
||||
Json ResponseFormat::to_chat() const {
|
||||
Json o = Json::object();
|
||||
switch (kind_) {
|
||||
case Kind::kText: o.set("type", Json("text")); break;
|
||||
case Kind::kJsonObject: o.set("type", Json("json_object")); break;
|
||||
case Kind::kJsonSchema: {
|
||||
Json js = Json::object();
|
||||
js.set("name", Json(name_));
|
||||
js.set("schema", schema_);
|
||||
js.set("strict", Json(strict_));
|
||||
o.set("type", Json("json_schema"));
|
||||
o.set("json_schema", js);
|
||||
break;
|
||||
}
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
Json ResponseFormat::to_responses_text() const {
|
||||
Json f = Json::object();
|
||||
switch (kind_) {
|
||||
case Kind::kText: f.set("type", Json("text")); break;
|
||||
case Kind::kJsonObject: f.set("type", Json("json_object")); break;
|
||||
case Kind::kJsonSchema:
|
||||
f.set("type", Json("json_schema"));
|
||||
f.set("name", Json(name_));
|
||||
f.set("schema", schema_);
|
||||
f.set("strict", Json(strict_));
|
||||
break;
|
||||
}
|
||||
Json text = Json::object();
|
||||
text.set("format", f);
|
||||
return text;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,45 @@
|
||||
#pragma once
|
||||
|
||||
// Structured outputs. One value class for the three response formats (text,
|
||||
// json_object, json_schema) that renders the two wire shapes:
|
||||
// chat completions response_format: {type:"json_schema", json_schema:{name,schema,strict}}
|
||||
// Responses API text: {format:{type:"json_schema", name, schema, strict}}
|
||||
// The schema is linted against the documented rejections (docs: structured
|
||||
// outputs, "Rejected schemas") so a request xAI would answer with 400 is
|
||||
// refused locally.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class ResponseFormat {
|
||||
public:
|
||||
enum class Kind { kText, kJsonObject, kJsonSchema };
|
||||
|
||||
static ResponseFormat text() { return ResponseFormat(Kind::kText); }
|
||||
static ResponseFormat json_object() { return ResponseFormat(Kind::kJsonObject); }
|
||||
// `name` is required by OpenAI-style clients and passed through; xAI marks
|
||||
// name / description / strict "only included for compatibility".
|
||||
static ResponseFormat json_schema(std::string name, Json schema, bool strict = true);
|
||||
|
||||
Kind kind() const { return kind_; }
|
||||
Json to_chat() const;
|
||||
Json to_responses_text() const; // the whole `text` object
|
||||
|
||||
// Throws ApiError::invalid for: enum / anyOf / oneOf with zero variants,
|
||||
// a property schema of true or false, minContains / maxContains, `items`
|
||||
// as an array, or nesting deeper than kMaxSchemaDepth.
|
||||
static void lint(const Json& schema);
|
||||
static constexpr int kMaxSchemaDepth = 64;
|
||||
|
||||
private:
|
||||
explicit ResponseFormat(Kind k) : kind_(k) {}
|
||||
Kind kind_;
|
||||
std::string name_;
|
||||
Json schema_;
|
||||
bool strict_ = true;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,201 @@
|
||||
#include "xai/core/responses.h"
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
ResponseRequest& ResponseRequest::add_tool(const Tool& tool) {
|
||||
return add_tool_json(tool.to_json());
|
||||
}
|
||||
|
||||
ResponseRequest& ResponseRequest::add_tool_json(Json tool) {
|
||||
Check::object("tool", tool);
|
||||
Check::upto("tools", tools_.size(), Caps::kMaxResponsesTools - 1);
|
||||
tools_.push_back(std::move(tool));
|
||||
return *this;
|
||||
}
|
||||
|
||||
ResponseRequest& ResponseRequest::include(std::string value) {
|
||||
Check::token("include", value);
|
||||
Check::upto("include", include_.size(), Caps::kMaxIncludes - 1);
|
||||
include_.push_back(std::move(value));
|
||||
return *this;
|
||||
}
|
||||
|
||||
Json ResponseRequest::to_json(bool stream) const {
|
||||
Check::model(model_);
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model_));
|
||||
body.set("input", input_.to_json());
|
||||
|
||||
if (instructions_) {
|
||||
Check::text("instructions", *instructions_, Caps::kMaxTextBytes);
|
||||
if (previous_) Check::fail("instructions cannot be combined with previous_response_id");
|
||||
body.set("instructions", Json(*instructions_));
|
||||
}
|
||||
if (previous_) {
|
||||
Check::id("previous_response", *previous_);
|
||||
body.set("previous_response_id", Json(*previous_));
|
||||
}
|
||||
jx::put(body, "store", store_);
|
||||
if (stream) body.set("stream", Json(true));
|
||||
|
||||
if (max_output_tokens_) {
|
||||
Check::range("max_output_tokens", *max_output_tokens_, 1, 10'000'000);
|
||||
body.set("max_output_tokens", Json(*max_output_tokens_));
|
||||
}
|
||||
if (max_turns_) {
|
||||
Check::range("max_turns", *max_turns_, 1, Caps::kMaxTurns);
|
||||
body.set("max_turns", Json(*max_turns_));
|
||||
}
|
||||
if (temperature_) {
|
||||
Check::range("temperature", *temperature_, 0, 2);
|
||||
body.set("temperature", Json(*temperature_));
|
||||
}
|
||||
if (top_p_) {
|
||||
Check::range("top_p", *top_p_, 0, 1);
|
||||
body.set("top_p", Json(*top_p_));
|
||||
}
|
||||
if (top_k_) {
|
||||
Check::range("top_k", *top_k_, 1, 1'000'000);
|
||||
body.set("top_k", Json(*top_k_));
|
||||
}
|
||||
if (min_p_) {
|
||||
Check::range("min_p", *min_p_, 0, 1);
|
||||
body.set("min_p", Json(*min_p_));
|
||||
}
|
||||
jx::put(body, "logprobs", logprobs_);
|
||||
if (top_logprobs_) {
|
||||
Check::range("top_logprobs", *top_logprobs_, 0, Caps::kMaxTopLogprobs);
|
||||
if (!logprobs_ || !*logprobs_) Check::fail("top_logprobs needs logprobs=true");
|
||||
body.set("top_logprobs", Json(*top_logprobs_));
|
||||
}
|
||||
|
||||
if (!effort_.empty() || !summary_.empty()) {
|
||||
Json reasoning = Json::object();
|
||||
if (!effort_.empty()) {
|
||||
Check::token("reasoning effort", effort_);
|
||||
reasoning.set("effort", Json(effort_));
|
||||
}
|
||||
if (!summary_.empty()) {
|
||||
Check::one_of("reasoning summary", summary_, {"auto", "concise", "detailed"});
|
||||
reasoning.set("summary", Json(summary_));
|
||||
}
|
||||
body.set("reasoning", reasoning);
|
||||
}
|
||||
if (!legacy_effort_.empty()) {
|
||||
Check::token("reasoning_effort", legacy_effort_);
|
||||
if (!effort_.empty()) Check::fail("reasoning_effort is only read when reasoning.effort is unset");
|
||||
body.set("reasoning_effort", Json(legacy_effort_));
|
||||
}
|
||||
|
||||
if (!tools_.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const Json& t : tools_) arr.push(t);
|
||||
body.set("tools", arr);
|
||||
}
|
||||
if (tool_choice_) body.set("tool_choice", tool_choice_->to_responses());
|
||||
jx::put(body, "parallel_tool_calls", parallel_);
|
||||
if (format_) body.set("text", format_->to_responses_text());
|
||||
|
||||
if (!include_.empty()) body.set("include", jx::string_array(include_));
|
||||
if (tier_) {
|
||||
Check::one_of("service_tier", *tier_, {"default", "priority", "fast"});
|
||||
body.set("service_tier", Json(*tier_));
|
||||
}
|
||||
if (user_) { Check::text("user", *user_, 256); body.set("user", Json(*user_)); }
|
||||
if (safety_) { Check::text("safety_identifier", *safety_, 256); body.set("safety_identifier", Json(*safety_)); }
|
||||
if (cache_key_) { Check::text("prompt_cache_key", *cache_key_, 256); body.set("prompt_cache_key", Json(*cache_key_)); }
|
||||
if (search_) body.set("search_parameters", search_->to_json());
|
||||
if (!guidelines_.empty()) body.set("guidelines", guidelines_json(guidelines_));
|
||||
return body;
|
||||
}
|
||||
|
||||
ModelResponse ResponsesService::create(const ResponseRequest& request) {
|
||||
Json body = request.to_json(false);
|
||||
return ModelResponse::parse(call_json(Call::kResponsesCreate, {}, &body));
|
||||
}
|
||||
|
||||
StreamResult ResponsesService::stream(const ResponseRequest& request, const EventSink& sink) {
|
||||
Json body = request.to_json(true);
|
||||
StreamResult result;
|
||||
call_stream(Call::kResponsesCreate, body, [&](const llm::SseEvent& raw) {
|
||||
ResponseEvent event = ResponseEvent::parse(raw);
|
||||
bool keep = result.accumulated.feed(event);
|
||||
bool user_keep = sink ? sink(event) : true;
|
||||
return keep && user_keep;
|
||||
});
|
||||
const ResponseAccumulator& acc = result.accumulated;
|
||||
if (acc.failed())
|
||||
throw llm::ApiError(kVendor, 502, "STREAM_FAILED",
|
||||
acc.error_message().empty() ? "the response stream failed" : acc.error_message());
|
||||
if (acc.stopped())
|
||||
throw llm::ApiError(kVendor, 502, "STREAM_CAP", acc.stop_reason());
|
||||
result.response = acc.final_response();
|
||||
result.completed = result.response && result.response->completed();
|
||||
return result;
|
||||
}
|
||||
|
||||
ModelResponse ResponsesService::retrieve(const std::string& response_id) {
|
||||
return ModelResponse::parse(call_json(Call::kResponsesGet, {response_id}));
|
||||
}
|
||||
|
||||
DeletedResponse ResponsesService::remove(const std::string& response_id) {
|
||||
return DeletedResponse::parse(call_json(Call::kResponsesDelete, {response_id}));
|
||||
}
|
||||
|
||||
InputItemList ResponsesService::input_items(const std::string& response_id, std::optional<int> limit,
|
||||
const std::string& order, const std::string& after) {
|
||||
Query q;
|
||||
if (limit) {
|
||||
Check::range("limit", *limit, 1, Caps::kMaxListLimit);
|
||||
q.add(Call::kResponsesInputItems, "limit", static_cast<long>(*limit));
|
||||
}
|
||||
if (!order.empty()) {
|
||||
Check::one_of("order", order, {"asc", "desc"});
|
||||
q.add(Call::kResponsesInputItems, "order", order);
|
||||
}
|
||||
if (!after.empty()) {
|
||||
Check::id("after", after);
|
||||
q.add(Call::kResponsesInputItems, "after", after);
|
||||
}
|
||||
return InputItemList::parse(call_json(Call::kResponsesInputItems, {response_id}, nullptr, q.str()));
|
||||
}
|
||||
|
||||
InputItemList ResponsesService::all_input_items(const std::string& response_id, std::size_t max_items) {
|
||||
Check::range("max_items", static_cast<double>(max_items), 1, Caps::kMaxAccumulatedItems);
|
||||
InputItemList all;
|
||||
std::string after;
|
||||
for (std::size_t page = 0; page < llm::Caps::kMaxPages; ++page) {
|
||||
InputItemList p = input_items(response_id, 100, "asc", after);
|
||||
for (Json& item : p.data) {
|
||||
if (all.data.size() >= max_items) {
|
||||
all.truncated = true;
|
||||
return all;
|
||||
}
|
||||
all.data.push_back(std::move(item));
|
||||
}
|
||||
if (all.first_id.empty()) all.first_id = p.first_id;
|
||||
all.last_id = p.last_id;
|
||||
all.object = p.object;
|
||||
if (!p.has_more || p.last_id.empty()) {
|
||||
all.has_more = false;
|
||||
return all;
|
||||
}
|
||||
after = p.last_id;
|
||||
}
|
||||
all.has_more = true;
|
||||
all.truncated = true;
|
||||
return all;
|
||||
}
|
||||
|
||||
CompactResult ResponsesService::compact(const std::string& model, const Input& input) {
|
||||
Check::model(model);
|
||||
Json body = Json::object();
|
||||
body.set("model", Json(model));
|
||||
body.set("input", input.to_json());
|
||||
return CompactResult::parse(call_json(Call::kResponsesCompact, {}, &body));
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,128 @@
|
||||
#pragma once
|
||||
|
||||
// Responses API (POST /v1/responses and the stored-response calls):
|
||||
// ResponseRequest builds and validates the body, ResponsesService sends it.
|
||||
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/input.h"
|
||||
#include "xai/core/response.h"
|
||||
#include "xai/core/response_format.h"
|
||||
#include "xai/core/search_parameters.h"
|
||||
#include "xai/core/service.h"
|
||||
#include "xai/core/stream_events.h"
|
||||
#include "xai/core/tools.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
// Reasoning effort values the guides name. Which values a model accepts is
|
||||
// model specific: read ModelCapabilities::reasoning_effort from the models API.
|
||||
struct Effort {
|
||||
static constexpr const char* kLow = "low";
|
||||
static constexpr const char* kMedium = "medium";
|
||||
static constexpr const char* kHigh = "high";
|
||||
static constexpr const char* kXHigh = "xhigh";
|
||||
};
|
||||
|
||||
// `include` values the guides name for the Responses API.
|
||||
struct Include {
|
||||
static constexpr const char* kEncryptedReasoning = "reasoning.encrypted_content";
|
||||
static constexpr const char* kWebSearchSources = "web_search_call.action.sources";
|
||||
static constexpr const char* kCodeInterpreterOutputs = "code_interpreter_call.outputs";
|
||||
static constexpr const char* kFileSearchResults = "file_search_call.results";
|
||||
};
|
||||
|
||||
class ResponseRequest {
|
||||
public:
|
||||
explicit ResponseRequest(std::string model) : model_(std::move(model)) {}
|
||||
|
||||
ResponseRequest& input(Input in) { input_ = std::move(in); return *this; }
|
||||
ResponseRequest& input_text(std::string text) { input_ = Input(std::move(text)); return *this; }
|
||||
ResponseRequest& add_input(InputItem item) { input_.add(std::move(item)); return *this; }
|
||||
// Cannot be combined with previous_response_id (the stored system prompt is reused).
|
||||
ResponseRequest& instructions(std::string s) { instructions_ = std::move(s); return *this; }
|
||||
ResponseRequest& previous_response_id(std::string id) { previous_ = std::move(id); return *this; }
|
||||
ResponseRequest& store(bool v) { store_ = v; return *this; }
|
||||
|
||||
ResponseRequest& max_output_tokens(int n) { max_output_tokens_ = n; return *this; }
|
||||
ResponseRequest& max_turns(int n) { max_turns_ = n; return *this; }
|
||||
ResponseRequest& temperature(double v) { temperature_ = v; return *this; }
|
||||
ResponseRequest& top_p(double v) { top_p_ = v; return *this; }
|
||||
ResponseRequest& top_k(int v) { top_k_ = v; return *this; }
|
||||
ResponseRequest& min_p(double v) { min_p_ = v; return *this; }
|
||||
ResponseRequest& logprobs(bool v) { logprobs_ = v; return *this; }
|
||||
ResponseRequest& top_logprobs(int v) { top_logprobs_ = v; return *this; }
|
||||
|
||||
// Reasoning controls: `reasoning.effort` (preferred) and `reasoning.summary`.
|
||||
ResponseRequest& reasoning_effort(std::string effort) { effort_ = std::move(effort); return *this; }
|
||||
ResponseRequest& reasoning_summary(std::string summary) { summary_ = std::move(summary); return *this; }
|
||||
// The non-standard top-level `reasoning_effort` field instead of reasoning.effort.
|
||||
ResponseRequest& legacy_reasoning_effort(std::string effort) { legacy_effort_ = std::move(effort); return *this; }
|
||||
|
||||
ResponseRequest& add_tool(const Tool& tool);
|
||||
ResponseRequest& add_tool_json(Json tool);
|
||||
ResponseRequest& tool_choice(ToolChoice choice) { tool_choice_ = std::move(choice); return *this; }
|
||||
ResponseRequest& parallel_tool_calls(bool v) { parallel_ = v; return *this; }
|
||||
|
||||
ResponseRequest& response_format(ResponseFormat f) { format_ = std::move(f); return *this; }
|
||||
ResponseRequest& include(std::string value);
|
||||
ResponseRequest& service_tier(std::string tier) { tier_ = std::move(tier); return *this; }
|
||||
ResponseRequest& user(std::string u) { user_ = std::move(u); return *this; }
|
||||
ResponseRequest& safety_identifier(std::string s) { safety_ = std::move(s); return *this; }
|
||||
ResponseRequest& prompt_cache_key(std::string k) { cache_key_ = std::move(k); return *this; }
|
||||
ResponseRequest& search_parameters(SearchParameters p) { search_ = std::move(p); return *this; }
|
||||
ResponseRequest& guideline(Guideline g) { guidelines_.push_back(std::move(g)); return *this; }
|
||||
|
||||
const std::string& model() const { return model_; }
|
||||
// Validates every field and returns the wire body. `stream` adds stream:true.
|
||||
Json to_json(bool stream = false) const;
|
||||
|
||||
private:
|
||||
std::string model_;
|
||||
Input input_;
|
||||
std::optional<std::string> instructions_, previous_, tier_, user_, safety_, cache_key_;
|
||||
std::optional<bool> store_, logprobs_, parallel_;
|
||||
std::optional<int> max_output_tokens_, max_turns_, top_k_, top_logprobs_;
|
||||
std::optional<double> temperature_, top_p_, min_p_;
|
||||
std::string effort_, summary_, legacy_effort_;
|
||||
std::vector<Json> tools_;
|
||||
std::optional<ToolChoice> tool_choice_;
|
||||
std::optional<ResponseFormat> format_;
|
||||
std::vector<std::string> include_;
|
||||
std::optional<SearchParameters> search_;
|
||||
std::vector<Guideline> guidelines_;
|
||||
};
|
||||
|
||||
// What a streamed call returns once the stream ends.
|
||||
struct StreamResult {
|
||||
ResponseAccumulator accumulated;
|
||||
std::optional<ModelResponse> response; // the final document when the stream delivered one
|
||||
bool completed = false;
|
||||
};
|
||||
|
||||
class ResponsesService : public XaiService {
|
||||
public:
|
||||
using EventSink = std::function<bool(const ResponseEvent&)>; // false stops the stream
|
||||
|
||||
explicit ResponsesService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
|
||||
ModelResponse create(const ResponseRequest& request);
|
||||
// Streams events to `sink` (may be empty) and folds them. Throws ApiError
|
||||
// on a stream failure event.
|
||||
StreamResult stream(const ResponseRequest& request, const EventSink& sink = {});
|
||||
|
||||
ModelResponse retrieve(const std::string& response_id);
|
||||
// Destructive: deletes a stored response. Explicit, no default.
|
||||
DeletedResponse remove(const std::string& response_id);
|
||||
// One page of input items. order: asc | desc. limit 1..Caps::kMaxListLimit.
|
||||
InputItemList input_items(const std::string& response_id, std::optional<int> limit = std::nullopt,
|
||||
const std::string& order = "", const std::string& after = "");
|
||||
// Walks pages up to Caps::kMaxPages and max_items; truncated is set when a cap stopped it.
|
||||
InputItemList all_input_items(const std::string& response_id, std::size_t max_items = 1000);
|
||||
CompactResult compact(const std::string& model, const Input& input);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,97 @@
|
||||
#include "xai/core/search_parameters.h"
|
||||
|
||||
#include <set>
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
SearchSource SearchSource::web() { SearchSource s; s.type = "web"; return s; }
|
||||
SearchSource SearchSource::x() { SearchSource s; s.type = "x"; return s; }
|
||||
SearchSource SearchSource::news() { SearchSource s; s.type = "news"; return s; }
|
||||
SearchSource SearchSource::rss(std::vector<std::string> links) {
|
||||
SearchSource s;
|
||||
s.type = "rss";
|
||||
s.links = std::move(links);
|
||||
return s;
|
||||
}
|
||||
|
||||
Json SearchSource::to_json() const {
|
||||
Check::one_of("source type", type, {"web", "x", "news", "rss"});
|
||||
Json o = Json::object();
|
||||
o.set("type", Json(type));
|
||||
if (type == "web" || type == "news") {
|
||||
if (!country.empty()) {
|
||||
if (country.size() != 2) Check::fail("country must be a 2-letter ISO code");
|
||||
o.set("country", Json(country));
|
||||
}
|
||||
Check::strings("excluded_websites", excluded_websites, Caps::kMaxDomains, 253);
|
||||
jx::put_strings(o, "excluded_websites", excluded_websites);
|
||||
jx::put(o, "safe_search", safe_search);
|
||||
if (type == "web") {
|
||||
Check::strings("allowed_websites", allowed_websites, Caps::kMaxDomains, 253);
|
||||
if (!allowed_websites.empty() && !excluded_websites.empty())
|
||||
Check::fail("allowed_websites and excluded_websites are mutually exclusive");
|
||||
jx::put_strings(o, "allowed_websites", allowed_websites);
|
||||
}
|
||||
} else if (type == "x") {
|
||||
Check::strings("included_x_handles", included_x_handles, Caps::kMaxXHandles, 64);
|
||||
Check::strings("excluded_x_handles", excluded_x_handles, Caps::kMaxXHandles, 64);
|
||||
if (!included_x_handles.empty() && !excluded_x_handles.empty())
|
||||
Check::fail("included and excluded x handles are mutually exclusive");
|
||||
jx::put_strings(o, "included_x_handles", included_x_handles);
|
||||
jx::put_strings(o, "excluded_x_handles", excluded_x_handles);
|
||||
jx::put(o, "post_favorite_count", post_favorite_count);
|
||||
jx::put(o, "post_view_count", post_view_count);
|
||||
} else {
|
||||
Check::count("rss links", links.size(), 1, 1);
|
||||
for (const auto& l : links) Check::http_url("rss link", l);
|
||||
o.set("links", jx::string_array(links));
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
Json SearchParameters::to_json() const {
|
||||
Json o = Json::object();
|
||||
if (!mode_.empty()) {
|
||||
Check::one_of("search mode", mode_, {"off", "on", "auto"});
|
||||
o.set("mode", Json(mode_));
|
||||
}
|
||||
if (!from_.empty()) { Check::date("from_date", from_); o.set("from_date", Json(from_)); }
|
||||
if (!to_.empty()) { Check::date("to_date", to_); o.set("to_date", Json(to_)); }
|
||||
if (!from_.empty() && !to_.empty() && from_ > to_) Check::fail("from_date is after to_date");
|
||||
if (max_results_) {
|
||||
Check::range("max_search_results", *max_results_, 1, 100);
|
||||
o.set("max_search_results", Json(*max_results_));
|
||||
}
|
||||
jx::put(o, "return_citations", citations_);
|
||||
if (!sources_.empty()) {
|
||||
Check::upto("sources", sources_.size(), 8);
|
||||
Json arr = Json::array();
|
||||
for (const auto& s : sources_) arr.push(s.to_json());
|
||||
o.set("sources", arr);
|
||||
}
|
||||
return o;
|
||||
}
|
||||
|
||||
Json Guideline::to_json() const {
|
||||
Check::one_of("guideline category", category, {"nsfw", "violence", "profanity"});
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("policy"));
|
||||
o.set("category", Json(category));
|
||||
o.set("behavior", Json("decline"));
|
||||
return o;
|
||||
}
|
||||
|
||||
Json guidelines_json(const std::vector<Guideline>& g) {
|
||||
Check::upto("guidelines", g.size(), Caps::kMaxGuidelines);
|
||||
std::set<std::string> seen;
|
||||
Json arr = Json::array();
|
||||
for (const auto& e : g) {
|
||||
if (!seen.insert(e.category).second) Check::fail("each guideline category may appear once");
|
||||
arr.push(e.to_json());
|
||||
}
|
||||
return arr;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,60 @@
|
||||
#pragma once
|
||||
|
||||
// Live Search parameters (`search_parameters`) shared by chat completions and
|
||||
// the Responses API, plus the per-category output guidelines. The server-side
|
||||
// tools (WebSearchTool, XSearchTool) are the current way to search; this is
|
||||
// the older request-level switch the spec still carries.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
struct SearchSource {
|
||||
// type: "web", "x", "news" or "rss".
|
||||
std::string type;
|
||||
std::string country; // web, news: ISO alpha-2
|
||||
std::vector<std::string> allowed_websites; // web
|
||||
std::vector<std::string> excluded_websites; // web, news
|
||||
std::optional<bool> safe_search; // web, news
|
||||
std::vector<std::string> included_x_handles; // x
|
||||
std::vector<std::string> excluded_x_handles; // x
|
||||
std::optional<long> post_favorite_count; // x
|
||||
std::optional<long> post_view_count; // x
|
||||
std::vector<std::string> links; // rss (required, 1 link)
|
||||
|
||||
static SearchSource web();
|
||||
static SearchSource x();
|
||||
static SearchSource news();
|
||||
static SearchSource rss(std::vector<std::string> links);
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
class SearchParameters {
|
||||
public:
|
||||
SearchParameters& mode(std::string m) { mode_ = std::move(m); return *this; } // off | on | auto
|
||||
SearchParameters& from_date(std::string d) { from_ = std::move(d); return *this; }
|
||||
SearchParameters& to_date(std::string d) { to_ = std::move(d); return *this; }
|
||||
SearchParameters& max_results(int n) { max_results_ = n; return *this; }
|
||||
SearchParameters& citations(bool v) { citations_ = v; return *this; }
|
||||
SearchParameters& add_source(SearchSource s) { sources_.push_back(std::move(s)); return *this; }
|
||||
Json to_json() const;
|
||||
|
||||
private:
|
||||
std::string mode_, from_, to_;
|
||||
std::optional<int> max_results_;
|
||||
std::optional<bool> citations_;
|
||||
std::vector<SearchSource> sources_;
|
||||
};
|
||||
|
||||
// {type:"policy", category, behavior:"decline"}; one entry per category.
|
||||
struct Guideline {
|
||||
std::string category; // nsfw | violence | profanity
|
||||
Json to_json() const;
|
||||
};
|
||||
Json guidelines_json(const std::vector<Guideline>& g);
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,80 @@
|
||||
#include "xai/core/service.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
llm::ApiClientConfig XaiService::default_config() {
|
||||
llm::ApiClientConfig c;
|
||||
c.vendor = kVendor;
|
||||
c.base_url = kBaseUrl;
|
||||
c.key_env = kKeyEnv;
|
||||
c.auth = llm::AuthStyle::kBearer;
|
||||
return c;
|
||||
}
|
||||
|
||||
std::shared_ptr<llm::ApiClient> XaiService::client_from_env() {
|
||||
return llm::ApiClient::from_env(default_config());
|
||||
}
|
||||
|
||||
Plan XaiService::plan(Call call, const std::vector<std::string>& ids, const Json* body,
|
||||
const std::string& query) {
|
||||
Plan p;
|
||||
p.method = method_of(call);
|
||||
p.path = render(call, ids) + query;
|
||||
if (body) {
|
||||
if (!body->is_object()) Check::fail("request body must be a JSON object");
|
||||
for (const auto& [key, value] : body->as_object()) {
|
||||
(void)value;
|
||||
if (!declares_body_field(call, key))
|
||||
Check::fail("field " + key + " is not declared for " + spec_of(call).path);
|
||||
}
|
||||
p.body = *body;
|
||||
p.has_body = true;
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
llm::RawResponse XaiService::send_raw(const Plan& p) {
|
||||
if (p.has_body) {
|
||||
std::string text = p.body.dump();
|
||||
return client_->request_raw(p.method, p.path, &text, "application/json");
|
||||
}
|
||||
return client_->request_raw(p.method, p.path);
|
||||
}
|
||||
|
||||
Json XaiService::parse_body(const llm::RawResponse& raw) {
|
||||
if (raw.body.find_first_not_of(" \t\r\n") == std::string::npos) return Json::object();
|
||||
try {
|
||||
return Json::parse(raw.body);
|
||||
} catch (const std::exception& e) {
|
||||
throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", e.what());
|
||||
}
|
||||
}
|
||||
|
||||
Json XaiService::send(const Plan& p) {
|
||||
return p.has_body ? client_->request(p.method, p.path, &p.body)
|
||||
: client_->request(p.method, p.path);
|
||||
}
|
||||
|
||||
void XaiService::send_stream(const Plan& p, const llm::SseDecoder::Sink& sink) {
|
||||
client_->stream(p.path, p.body, sink);
|
||||
}
|
||||
|
||||
Json XaiService::call_json(Call call, const std::vector<std::string>& ids, const Json* body,
|
||||
const std::string& query) {
|
||||
return send(plan(call, ids, body, query));
|
||||
}
|
||||
|
||||
llm::RawResponse XaiService::call_raw(Call call, const std::vector<std::string>& ids,
|
||||
const Json* body, const std::string& query) {
|
||||
return send_raw(plan(call, ids, body, query));
|
||||
}
|
||||
|
||||
void XaiService::call_stream(Call call, const Json& body, const llm::SseDecoder::Sink& sink) {
|
||||
Plan p = plan(call, {}, &body);
|
||||
if (p.method != clients::Method::kPost) Check::fail("only POST calls stream");
|
||||
send_stream(p, sink);
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,56 @@
|
||||
#pragma once
|
||||
|
||||
// XaiService: base of every xAI core service. Owns the call-table dispatch:
|
||||
// every request goes through Call -> (method, path) from calls.h, with the
|
||||
// body checked against the row's declared fields before any network traffic.
|
||||
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "llm/api_client.h"
|
||||
#include "xai/core/calls.h"
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
// A request ready to send, observable without a network (tests, dry runs).
|
||||
struct Plan {
|
||||
clients::Method method = clients::Method::kGet;
|
||||
std::string path; // includes the query string
|
||||
Json body; // null when the call has no JSON body
|
||||
bool has_body = false;
|
||||
};
|
||||
|
||||
class XaiService : public llm::Service {
|
||||
public:
|
||||
explicit XaiService(std::shared_ptr<llm::ApiClient> client) : llm::Service(std::move(client)) {}
|
||||
|
||||
// vendor "xai", https://api.x.ai, XAI_API_KEY as a bearer token.
|
||||
static llm::ApiClientConfig default_config();
|
||||
static std::shared_ptr<llm::ApiClient> client_from_env();
|
||||
static constexpr const char* kBaseUrl = "https://api.x.ai";
|
||||
static constexpr const char* kKeyEnv = "XAI_API_KEY";
|
||||
|
||||
// Build the plan for a call: renders the path, checks body fields against
|
||||
// the call table (an undeclared top-level field is rejected loudly).
|
||||
static Plan plan(Call call, const std::vector<std::string>& ids = {},
|
||||
const Json* body = nullptr, const std::string& query = "");
|
||||
|
||||
protected:
|
||||
Json send(const Plan& plan);
|
||||
llm::RawResponse send_raw(const Plan& plan);
|
||||
void send_stream(const Plan& plan, const llm::SseDecoder::Sink& sink);
|
||||
|
||||
Json call_json(Call call, const std::vector<std::string>& ids = {}, const Json* body = nullptr,
|
||||
const std::string& query = "");
|
||||
// Raw reply with the status preserved (202 pending polls, binary content).
|
||||
llm::RawResponse call_raw(Call call, const std::vector<std::string>& ids = {},
|
||||
const Json* body = nullptr, const std::string& query = "");
|
||||
void call_stream(Call call, const Json& body, const llm::SseDecoder::Sink& sink);
|
||||
|
||||
static Json parse_body(const llm::RawResponse& raw); // {} for an empty body
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,193 @@
|
||||
#include "xai/core/stream_events.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
struct Row {
|
||||
const char* type;
|
||||
EventKind kind;
|
||||
};
|
||||
const Row kRows[] = {
|
||||
{"response.created", EventKind::kCreated},
|
||||
{"response.in_progress", EventKind::kInProgress},
|
||||
{"response.completed", EventKind::kCompleted},
|
||||
{"response.incomplete", EventKind::kIncomplete},
|
||||
{"response.failed", EventKind::kFailed},
|
||||
{"error", EventKind::kError},
|
||||
{"response.output_item.added", EventKind::kOutputItemAdded},
|
||||
{"response.output_item.done", EventKind::kOutputItemDone},
|
||||
{"response.content_part.added", EventKind::kContentPartAdded},
|
||||
{"response.content_part.done", EventKind::kContentPartDone},
|
||||
{"response.output_text.delta", EventKind::kOutputTextDelta},
|
||||
{"response.text.delta", EventKind::kOutputTextDelta}, // xAI: identical to output_text.delta
|
||||
{"response.output_text.done", EventKind::kOutputTextDone},
|
||||
{"response.refusal.delta", EventKind::kRefusalDelta},
|
||||
{"response.refusal.done", EventKind::kRefusalDone},
|
||||
{"response.output_text.annotation.added", EventKind::kAnnotationAdded},
|
||||
{"response.reasoning_text.delta", EventKind::kReasoningTextDelta},
|
||||
{"response.reasoning_text.done", EventKind::kReasoningTextDone},
|
||||
{"response.reasoning_summary_text.delta", EventKind::kReasoningSummaryTextDelta},
|
||||
{"response.reasoning_summary_text.done", EventKind::kReasoningSummaryTextDone},
|
||||
{"response.reasoning_summary_part.added", EventKind::kReasoningSummaryPartAdded},
|
||||
{"response.reasoning_summary_part.done", EventKind::kReasoningSummaryPartDone},
|
||||
{"response.function_call_arguments.delta", EventKind::kFunctionCallArgumentsDelta},
|
||||
{"response.function_call_arguments.done", EventKind::kFunctionCallArgumentsDone},
|
||||
{"response.mcp_call_arguments.delta", EventKind::kMcpCallArgumentsDelta},
|
||||
{"response.mcp_call_arguments.done", EventKind::kMcpCallArgumentsDone},
|
||||
{"response.mcp_call.in_progress", EventKind::kMcpCallInProgress},
|
||||
{"response.mcp_call.completed", EventKind::kMcpCallCompleted},
|
||||
{"response.mcp_call.failed", EventKind::kMcpCallFailed},
|
||||
{"response.mcp_list_tools.in_progress", EventKind::kMcpListToolsInProgress},
|
||||
{"response.mcp_list_tools.completed", EventKind::kMcpListToolsCompleted},
|
||||
{"response.mcp_list_tools.failed", EventKind::kMcpListToolsFailed},
|
||||
{"response.web_search_call.in_progress", EventKind::kWebSearchInProgress},
|
||||
{"response.web_search_call.searching", EventKind::kWebSearchSearching},
|
||||
{"response.web_search_call.completed", EventKind::kWebSearchCompleted},
|
||||
{"response.file_search_call.in_progress", EventKind::kFileSearchInProgress},
|
||||
{"response.file_search_call.searching", EventKind::kFileSearchSearching},
|
||||
{"response.file_search_call.completed", EventKind::kFileSearchCompleted},
|
||||
{"response.code_interpreter_call.in_progress", EventKind::kCodeInterpreterInProgress},
|
||||
{"response.code_interpreter_call.interpreting", EventKind::kCodeInterpreterInterpreting},
|
||||
{"response.code_interpreter_call.completed", EventKind::kCodeInterpreterCompleted},
|
||||
{"response.code_interpreter_call_code.delta", EventKind::kCodeInterpreterCodeDelta},
|
||||
{"response.code_interpreter_call_code.done", EventKind::kCodeInterpreterCodeDone},
|
||||
{"response.image_generation_call.in_progress", EventKind::kImageGenerationInProgress},
|
||||
{"response.image_generation_call.generating", EventKind::kImageGenerationGenerating},
|
||||
{"response.image_generation_call.completed", EventKind::kImageGenerationCompleted},
|
||||
{"response.custom_tool_call_input.delta", EventKind::kCustomToolInputDelta},
|
||||
{"response.custom_tool_call_input.done", EventKind::kCustomToolInputDone},
|
||||
};
|
||||
|
||||
bool is_tool_status(EventKind k) {
|
||||
switch (k) {
|
||||
case EventKind::kMcpCallInProgress: case EventKind::kMcpCallCompleted:
|
||||
case EventKind::kMcpCallFailed: case EventKind::kWebSearchInProgress:
|
||||
case EventKind::kWebSearchSearching: case EventKind::kWebSearchCompleted:
|
||||
case EventKind::kFileSearchInProgress: case EventKind::kFileSearchSearching:
|
||||
case EventKind::kFileSearchCompleted: case EventKind::kCodeInterpreterInProgress:
|
||||
case EventKind::kCodeInterpreterInterpreting: case EventKind::kCodeInterpreterCompleted:
|
||||
case EventKind::kImageGenerationInProgress: case EventKind::kImageGenerationGenerating:
|
||||
case EventKind::kImageGenerationCompleted:
|
||||
return true;
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
|
||||
EventKind event_kind_of(const std::string& type) {
|
||||
for (const Row& r : kRows)
|
||||
if (type == r.type) return r.kind;
|
||||
return EventKind::kUnknown;
|
||||
}
|
||||
|
||||
const char* event_type_of(EventKind kind) {
|
||||
for (const Row& r : kRows)
|
||||
if (r.kind == kind) return r.type;
|
||||
return "";
|
||||
}
|
||||
|
||||
ResponseEvent ResponseEvent::parse(const llm::SseEvent& event) {
|
||||
ResponseEvent e;
|
||||
e.raw = event.data;
|
||||
e.type = jx::str(event.data, "type");
|
||||
if (e.type.empty()) e.type = event.event;
|
||||
e.kind = event_kind_of(e.type);
|
||||
e.sequence = jx::num(event.data, "sequence_number", -1);
|
||||
e.output_index = jx::num(event.data, "output_index", -1);
|
||||
e.content_index = jx::num(event.data, "content_index", -1);
|
||||
e.item_id = jx::str(event.data, "item_id");
|
||||
e.delta = jx::str(event.data, "delta");
|
||||
e.text = jx::str(event.data, "text");
|
||||
if (e.text.empty()) e.text = jx::str(event.data, "arguments");
|
||||
if (e.text.empty()) e.text = jx::str(event.data, "code");
|
||||
if (const Json* item = jx::get(event.data, "item")) e.item = *item;
|
||||
if (const Json* resp = jx::get(event.data, "response")) e.response = *resp;
|
||||
if (e.item_id.empty() && e.item.is_object()) e.item_id = jx::str(e.item, "id");
|
||||
return e;
|
||||
}
|
||||
|
||||
bool ResponseAccumulator::feed(const ResponseEvent& e) {
|
||||
if (stopped_) return false;
|
||||
if (++events_ > static_cast<long>(Caps::kMaxStreamEvents)) {
|
||||
stopped_ = true;
|
||||
stop_reason_ = "more than the stream event cap";
|
||||
return false;
|
||||
}
|
||||
auto grow = [&](std::string& target, const std::string& add) {
|
||||
if (target.size() + add.size() > Caps::kMaxStreamTextBytes) {
|
||||
stopped_ = true;
|
||||
stop_reason_ = "stream text over the byte cap";
|
||||
return;
|
||||
}
|
||||
target += add;
|
||||
};
|
||||
switch (e.kind) {
|
||||
case EventKind::kOutputTextDelta: grow(text_, e.delta); break;
|
||||
case EventKind::kReasoningTextDelta: grow(reasoning_, e.delta); break;
|
||||
case EventKind::kReasoningSummaryTextDelta: grow(summary_, e.delta); break;
|
||||
case EventKind::kFunctionCallArgumentsDelta:
|
||||
case EventKind::kMcpCallArgumentsDelta:
|
||||
case EventKind::kCustomToolInputDelta:
|
||||
if (!e.item_id.empty()) {
|
||||
if (arguments_.size() >= Caps::kMaxAccumulatedItems && !arguments_.count(e.item_id)) {
|
||||
stopped_ = true;
|
||||
stop_reason_ = "more than the accumulated item cap";
|
||||
break;
|
||||
}
|
||||
grow(arguments_[e.item_id], e.delta);
|
||||
}
|
||||
break;
|
||||
case EventKind::kFunctionCallArgumentsDone:
|
||||
case EventKind::kMcpCallArgumentsDone:
|
||||
case EventKind::kCustomToolInputDone:
|
||||
if (!e.item_id.empty() && !e.text.empty()) arguments_[e.item_id] = e.text;
|
||||
break;
|
||||
case EventKind::kOutputItemDone:
|
||||
if (e.item.is_object()) {
|
||||
if (items_.size() >= Caps::kMaxAccumulatedItems) {
|
||||
stopped_ = true;
|
||||
stop_reason_ = "more than the accumulated item cap";
|
||||
break;
|
||||
}
|
||||
items_.push_back(OutputItem::parse(e.item));
|
||||
}
|
||||
break;
|
||||
case EventKind::kCompleted:
|
||||
case EventKind::kIncomplete:
|
||||
if (e.response.is_object()) final_ = ModelResponse::parse(e.response);
|
||||
break;
|
||||
case EventKind::kFailed:
|
||||
failed_ = true;
|
||||
if (e.response.is_object()) {
|
||||
final_ = ModelResponse::parse(e.response);
|
||||
if (final_->error()) error_message_ = final_->error()->message;
|
||||
}
|
||||
break;
|
||||
case EventKind::kError:
|
||||
failed_ = true;
|
||||
error_message_ = jx::str(e.raw, "message");
|
||||
if (error_message_.empty()) error_message_ = jx::str(jx::object(e.raw, "error"), "message");
|
||||
break;
|
||||
default:
|
||||
if (is_tool_status(e.kind) && !e.item_id.empty()) {
|
||||
const std::string t = e.type;
|
||||
tool_status_[e.item_id] = t.substr(t.rfind('.') + 1);
|
||||
}
|
||||
}
|
||||
return !stopped_ && !e.terminal();
|
||||
}
|
||||
|
||||
std::vector<std::string> ResponseAccumulator::images() const {
|
||||
std::vector<std::string> out;
|
||||
for (const auto& i : items_)
|
||||
if (i.kind() == ItemKind::kImageGenerationCall && !i.image_base64().empty())
|
||||
out.push_back(i.image_base64());
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,103 @@
|
||||
#pragma once
|
||||
|
||||
// Streaming events of the Responses API (`stream: true`). xAI documents the
|
||||
// stream as OpenAI-Responses compatible and names these events in its guides:
|
||||
// response.created / completed / incomplete, response.output_text.delta,
|
||||
// response.reasoning_text.delta, response.reasoning_summary_text.delta,
|
||||
// response.output_item.added / done, response.function_call_arguments.*,
|
||||
// response.mcp_call*, response.image_generation_call.*. The remaining kinds
|
||||
// follow the OpenAI event vocabulary xAI declares compatibility with and are
|
||||
// listed under Unverified in the SPEC. Anything else decodes as kUnknown and
|
||||
// stays reachable through raw().
|
||||
|
||||
#include <map>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "llm/sse.h"
|
||||
#include "xai/core/params.h"
|
||||
#include "xai/core/response.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
enum class EventKind {
|
||||
kCreated, kInProgress, kCompleted, kIncomplete, kFailed, kError,
|
||||
kOutputItemAdded, kOutputItemDone, kContentPartAdded, kContentPartDone,
|
||||
kOutputTextDelta, kOutputTextDone, kRefusalDelta, kRefusalDone, kAnnotationAdded,
|
||||
kReasoningTextDelta, kReasoningTextDone, kReasoningSummaryTextDelta, kReasoningSummaryTextDone,
|
||||
kReasoningSummaryPartAdded, kReasoningSummaryPartDone,
|
||||
kFunctionCallArgumentsDelta, kFunctionCallArgumentsDone,
|
||||
kMcpCallArgumentsDelta, kMcpCallArgumentsDone, kMcpCallInProgress, kMcpCallCompleted,
|
||||
kMcpCallFailed, kMcpListToolsInProgress, kMcpListToolsCompleted, kMcpListToolsFailed,
|
||||
kWebSearchInProgress, kWebSearchSearching, kWebSearchCompleted,
|
||||
kFileSearchInProgress, kFileSearchSearching, kFileSearchCompleted,
|
||||
kCodeInterpreterInProgress, kCodeInterpreterInterpreting, kCodeInterpreterCompleted,
|
||||
kCodeInterpreterCodeDelta, kCodeInterpreterCodeDone,
|
||||
kImageGenerationInProgress, kImageGenerationGenerating, kImageGenerationCompleted,
|
||||
kCustomToolInputDelta, kCustomToolInputDone,
|
||||
kUnknown
|
||||
};
|
||||
|
||||
EventKind event_kind_of(const std::string& type);
|
||||
const char* event_type_of(EventKind kind); // wire name, "" for kUnknown
|
||||
|
||||
struct ResponseEvent {
|
||||
EventKind kind = EventKind::kUnknown;
|
||||
std::string type; // wire type string
|
||||
long sequence = -1; // sequence_number
|
||||
long output_index = -1;
|
||||
long content_index = -1;
|
||||
std::string item_id;
|
||||
std::string delta; // *.delta
|
||||
std::string text; // *.done (full text / arguments)
|
||||
Json item; // output_item.* (null otherwise)
|
||||
Json response; // created / completed / incomplete / failed
|
||||
Json raw;
|
||||
|
||||
static ResponseEvent parse(const llm::SseEvent& event);
|
||||
bool terminal() const {
|
||||
return kind == EventKind::kCompleted || kind == EventKind::kIncomplete ||
|
||||
kind == EventKind::kFailed || kind == EventKind::kError;
|
||||
}
|
||||
};
|
||||
|
||||
// Folds events into the final picture: assembled text, reasoning, tool call
|
||||
// arguments, finished items, final response and usage. Bounded: text over
|
||||
// Caps::kMaxStreamTextBytes, more than Caps::kMaxStreamEvents events or more
|
||||
// than Caps::kMaxAccumulatedItems items stop the fold (stopped() + reason()).
|
||||
class ResponseAccumulator {
|
||||
public:
|
||||
// Returns false once a cap was hit or a terminal event was folded.
|
||||
bool feed(const ResponseEvent& event);
|
||||
|
||||
const std::string& text() const { return text_; }
|
||||
const std::string& reasoning() const { return reasoning_; }
|
||||
const std::string& summary() const { return summary_; }
|
||||
// Finished output items by arrival; complete function calls are included.
|
||||
const std::vector<OutputItem>& items() const { return items_; }
|
||||
// Argument text assembled so far per item id (also for unfinished calls).
|
||||
const std::map<std::string, std::string>& arguments() const { return arguments_; }
|
||||
const std::optional<ModelResponse>& final_response() const { return final_; }
|
||||
long events() const { return events_; }
|
||||
bool stopped() const { return stopped_; }
|
||||
const std::string& stop_reason() const { return stop_reason_; }
|
||||
std::string error_message() const { return error_message_; }
|
||||
bool failed() const { return failed_; }
|
||||
// Progress statuses of server-side tools: item id -> last status event.
|
||||
const std::map<std::string, std::string>& tool_status() const { return tool_status_; }
|
||||
// Base64 image payloads from finished image_generation_call items.
|
||||
std::vector<std::string> images() const;
|
||||
|
||||
private:
|
||||
std::string text_, reasoning_, summary_, stop_reason_, error_message_;
|
||||
std::vector<OutputItem> items_;
|
||||
std::map<std::string, std::string> arguments_;
|
||||
std::map<std::string, std::string> tool_status_;
|
||||
std::optional<ModelResponse> final_;
|
||||
long events_ = 0;
|
||||
bool stopped_ = false;
|
||||
bool failed_ = false;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,231 @@
|
||||
#include "xai/core/tools.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
void check_handle(const std::string& h) {
|
||||
if (h.empty() || h.size() > 64) Check::fail("x handle must be 1..64 characters");
|
||||
for (char c : h) {
|
||||
bool ok = (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_';
|
||||
if (!ok) Check::fail("x handle has an invalid character (give it without @)");
|
||||
}
|
||||
}
|
||||
|
||||
void check_domain(const std::string& d) {
|
||||
if (d.empty() || d.size() > 253) Check::fail("domain must be 1..253 characters");
|
||||
if (d.find("://") != std::string::npos || d.find('/') != std::string::npos)
|
||||
Check::fail("domain is given without protocol or path: " + d);
|
||||
}
|
||||
|
||||
Json make_function(const std::string& name, const std::string& description, const Json& parameters,
|
||||
const std::optional<bool>& strict, const std::optional<bool>& defer) {
|
||||
Check::id("function", name);
|
||||
if (name.size() > 64) Check::fail("function name over 64 characters");
|
||||
Check::text("function description", description, 16u * 1024u, true);
|
||||
Check::json_schema("function parameters", parameters);
|
||||
Json f = Json::object();
|
||||
f.set("name", Json(name));
|
||||
if (!description.empty()) f.set("description", Json(description));
|
||||
f.set("parameters", parameters);
|
||||
jx::put(f, "strict", strict);
|
||||
jx::put(f, "defer_loading", defer);
|
||||
return f;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
FunctionTool::FunctionTool(std::string name, std::string description, Json parameters)
|
||||
: name_(std::move(name)), description_(std::move(description)), parameters_(std::move(parameters)) {}
|
||||
|
||||
Json FunctionTool::to_json() const {
|
||||
Json f = make_function(name_, description_, parameters_, strict_, defer_loading_);
|
||||
f.set("type", Json("function"));
|
||||
return f;
|
||||
}
|
||||
|
||||
Json FunctionTool::to_chat_json() const {
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("function"));
|
||||
t.set("function", make_function(name_, description_, parameters_, strict_, defer_loading_));
|
||||
return t;
|
||||
}
|
||||
|
||||
Json WebSearchTool::to_json() const {
|
||||
if (!allowed_.empty() && !excluded_.empty())
|
||||
Check::fail("web search allow_domains and exclude_domains are mutually exclusive");
|
||||
Check::upto("allowed_domains", allowed_.size(), Caps::kMaxDomains);
|
||||
Check::upto("excluded_domains", excluded_.size(), Caps::kMaxDomains);
|
||||
for (const auto& d : allowed_) check_domain(d);
|
||||
for (const auto& d : excluded_) check_domain(d);
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("web_search"));
|
||||
jx::put_strings(t, "allowed_domains", allowed_);
|
||||
jx::put_strings(t, "excluded_domains", excluded_);
|
||||
jx::put(t, "enable_image_search", image_search_);
|
||||
jx::put(t, "enable_image_understanding", image_understanding_);
|
||||
if (location_) {
|
||||
const UserLocation& l = *location_;
|
||||
if (!l.country.empty() && l.country.size() != 2) Check::fail("country must be a 2-letter ISO code");
|
||||
Json loc = Json::object();
|
||||
loc.set("type", Json("approximate"));
|
||||
if (!l.country.empty()) loc.set("country", Json(l.country));
|
||||
if (!l.city.empty()) loc.set("city", Json(l.city));
|
||||
if (!l.region.empty()) loc.set("region", Json(l.region));
|
||||
if (!l.timezone.empty()) loc.set("timezone", Json(l.timezone));
|
||||
t.set("user_location", loc);
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
Json XSearchTool::to_json() const {
|
||||
if (!allowed_.empty() && !excluded_.empty())
|
||||
Check::fail("x search allow_handles and exclude_handles are mutually exclusive");
|
||||
Check::upto("allowed_x_handles", allowed_.size(), Caps::kMaxXHandles);
|
||||
Check::upto("excluded_x_handles", excluded_.size(), Caps::kMaxXHandles);
|
||||
for (const auto& h : allowed_) check_handle(h);
|
||||
for (const auto& h : excluded_) check_handle(h);
|
||||
if (!from_.empty()) Check::date("from_date", from_);
|
||||
if (!to_.empty()) Check::date("to_date", to_);
|
||||
if (!from_.empty() && !to_.empty() && from_ > to_) Check::fail("from_date is after to_date");
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("x_search"));
|
||||
jx::put_strings(t, "allowed_x_handles", allowed_);
|
||||
jx::put_strings(t, "excluded_x_handles", excluded_);
|
||||
if (!from_.empty()) t.set("from_date", Json(from_));
|
||||
if (!to_.empty()) t.set("to_date", Json(to_));
|
||||
jx::put(t, "enable_image_understanding", image_);
|
||||
jx::put(t, "enable_video_understanding", video_);
|
||||
return t;
|
||||
}
|
||||
|
||||
Json CodeExecutionTool::to_json() const {
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("code_interpreter"));
|
||||
if (!container_.empty()) {
|
||||
Check::id("container", container_);
|
||||
t.set("container", Json(container_));
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
Json CollectionsSearchTool::to_json() const {
|
||||
Check::count("collection ids", ids_.size(), 1, Caps::kMaxCollections);
|
||||
for (const auto& id : ids_) Check::id("collection", id);
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("file_search"));
|
||||
t.set("vector_store_ids", jx::string_array(ids_));
|
||||
if (max_results_) {
|
||||
Check::range("max_num_results", *max_results_, 1, 100);
|
||||
t.set("max_num_results", Json(*max_results_));
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
McpTool& McpTool::header(std::string name, std::string value) {
|
||||
headers_.emplace_back(std::move(name), std::move(value));
|
||||
return *this;
|
||||
}
|
||||
|
||||
Json McpTool::to_json() const {
|
||||
Check::token("server_label", label_);
|
||||
Check::http_url("server_url", url_);
|
||||
Check::text("server_description", description_, 4096, true);
|
||||
Check::strings("allowed_tools", allowed_, Caps::kMaxAllowedTools, 128);
|
||||
Check::upto("headers", headers_.size(), Caps::kMaxMcpHeaders);
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("mcp"));
|
||||
t.set("server_label", Json(label_));
|
||||
t.set("server_url", Json(url_));
|
||||
if (!description_.empty()) t.set("server_description", Json(description_));
|
||||
jx::put_strings(t, "allowed_tools", allowed_);
|
||||
if (!authorization_.empty()) {
|
||||
Check::text("authorization", authorization_, 8192);
|
||||
t.set("authorization", Json(authorization_));
|
||||
}
|
||||
if (!headers_.empty()) {
|
||||
Json h = Json::object();
|
||||
for (const auto& [k, v] : headers_) {
|
||||
Check::token("header name", k);
|
||||
Check::text("header value", v, 8192, true);
|
||||
h.set(k, Json(v));
|
||||
}
|
||||
t.set("headers", h);
|
||||
}
|
||||
jx::put(t, "defer_loading", defer_loading_);
|
||||
return t;
|
||||
}
|
||||
|
||||
Json ImageGenerationTool::to_json() const {
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("image_generation"));
|
||||
if (!action_.empty()) {
|
||||
Check::one_of("action", action_, {"auto", "generate", "edit"});
|
||||
t.set("action", Json(action_));
|
||||
}
|
||||
return t;
|
||||
}
|
||||
|
||||
Json ShellTool::to_json() const {
|
||||
Json env = Json::object();
|
||||
env.set("type", Json("local"));
|
||||
if (!skills_.empty()) {
|
||||
Check::upto("skills", skills_.size(), 64);
|
||||
Json arr = Json::array();
|
||||
for (const auto& s : skills_) {
|
||||
Check::token("skill name", s.name);
|
||||
Check::text("skill description", s.description, 4096);
|
||||
Check::text("skill path", s.path, 4096);
|
||||
Json o = Json::object();
|
||||
o.set("name", Json(s.name));
|
||||
o.set("description", Json(s.description));
|
||||
o.set("path", Json(s.path));
|
||||
arr.push(o);
|
||||
}
|
||||
env.set("skills", arr);
|
||||
}
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("shell"));
|
||||
t.set("environment", env);
|
||||
return t;
|
||||
}
|
||||
|
||||
Json ToolSearchTool::to_json() const {
|
||||
Json t = Json::object();
|
||||
t.set("type", Json("tool_search"));
|
||||
t.set("execution", Json("server"));
|
||||
return t;
|
||||
}
|
||||
|
||||
ToolChoice ToolChoice::function(const std::string& name) {
|
||||
Check::id("function", name);
|
||||
return ToolChoice("function", name);
|
||||
}
|
||||
|
||||
std::optional<ToolChoice> ToolChoice::from_neutral(const std::string& value) {
|
||||
if (value.empty()) return std::nullopt;
|
||||
if (value == "auto") return automatic();
|
||||
if (value == "none") return none();
|
||||
if (value == "required") return required();
|
||||
return function(value);
|
||||
}
|
||||
|
||||
Json ToolChoice::to_responses() const {
|
||||
if (mode_ != "function") return Json(mode_);
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("function"));
|
||||
o.set("name", Json(name_));
|
||||
return o;
|
||||
}
|
||||
|
||||
Json ToolChoice::to_chat() const {
|
||||
if (mode_ != "function") return Json(mode_);
|
||||
Json fn = Json::object();
|
||||
fn.set("name", Json(name_));
|
||||
Json o = Json::object();
|
||||
o.set("type", Json("function"));
|
||||
o.set("function", fn);
|
||||
return o;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,194 @@
|
||||
#pragma once
|
||||
|
||||
// Tool definitions for the Responses API: client-side function calling and
|
||||
// the server-side tools xAI runs itself (web search, X search, code
|
||||
// execution, collections search, remote MCP, image generation, shell, tool
|
||||
// search). One class per tool; to_json() validates and returns the wire
|
||||
// object. The spec's `ModelTool` variants and the TOOLS table in calls.cpp
|
||||
// are checked by tools/check_spec.py.
|
||||
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
class Tool {
|
||||
public:
|
||||
virtual ~Tool() = default;
|
||||
virtual const char* type() const = 0;
|
||||
virtual Json to_json() const = 0;
|
||||
};
|
||||
|
||||
// Client-side function. Responses wire: {type, name, description, parameters};
|
||||
// chat completions wire: {type, function:{...}} (to_chat_json()).
|
||||
class FunctionTool : public Tool {
|
||||
public:
|
||||
FunctionTool(std::string name, std::string description, Json parameters);
|
||||
const char* type() const override { return "function"; }
|
||||
Json to_json() const override;
|
||||
Json to_chat_json() const;
|
||||
|
||||
FunctionTool& strict(bool value) { strict_ = value; return *this; }
|
||||
// Hide the definition from the prompt; the model loads it through a
|
||||
// tool_search step (spec: FunctionDefinition.defer_loading).
|
||||
FunctionTool& defer_loading(bool value) { defer_loading_ = value; return *this; }
|
||||
const std::string& name() const { return name_; }
|
||||
|
||||
private:
|
||||
std::string name_;
|
||||
std::string description_;
|
||||
Json parameters_;
|
||||
std::optional<bool> strict_;
|
||||
std::optional<bool> defer_loading_;
|
||||
};
|
||||
|
||||
struct UserLocation {
|
||||
std::string country; // ISO 3166-1 alpha-2
|
||||
std::string city;
|
||||
std::string region;
|
||||
std::string timezone; // IANA
|
||||
};
|
||||
|
||||
class WebSearchTool : public Tool {
|
||||
public:
|
||||
const char* type() const override { return "web_search"; }
|
||||
Json to_json() const override;
|
||||
|
||||
// At most 5 each, and not both (spec).
|
||||
WebSearchTool& allow_domains(std::vector<std::string> d) { allowed_ = std::move(d); return *this; }
|
||||
WebSearchTool& exclude_domains(std::vector<std::string> d) { excluded_ = std::move(d); return *this; }
|
||||
WebSearchTool& image_search(bool v) { image_search_ = v; return *this; }
|
||||
WebSearchTool& image_understanding(bool v) { image_understanding_ = v; return *this; }
|
||||
WebSearchTool& location(UserLocation l) { location_ = std::move(l); return *this; }
|
||||
|
||||
private:
|
||||
std::vector<std::string> allowed_, excluded_;
|
||||
std::optional<bool> image_search_, image_understanding_;
|
||||
std::optional<UserLocation> location_;
|
||||
};
|
||||
|
||||
class XSearchTool : public Tool {
|
||||
public:
|
||||
const char* type() const override { return "x_search"; }
|
||||
Json to_json() const override;
|
||||
|
||||
// Handles without the leading @; at most 10 each (docs), not both (spec).
|
||||
XSearchTool& allow_handles(std::vector<std::string> h) { allowed_ = std::move(h); return *this; }
|
||||
XSearchTool& exclude_handles(std::vector<std::string> h) { excluded_ = std::move(h); return *this; }
|
||||
XSearchTool& from_date(std::string d) { from_ = std::move(d); return *this; } // YYYY-MM-DD
|
||||
XSearchTool& to_date(std::string d) { to_ = std::move(d); return *this; }
|
||||
XSearchTool& image_understanding(bool v) { image_ = v; return *this; }
|
||||
XSearchTool& video_understanding(bool v) { video_ = v; return *this; }
|
||||
|
||||
private:
|
||||
std::vector<std::string> allowed_, excluded_;
|
||||
std::string from_, to_;
|
||||
std::optional<bool> image_, video_;
|
||||
};
|
||||
|
||||
// Code execution (wire type `code_interpreter`).
|
||||
class CodeExecutionTool : public Tool {
|
||||
public:
|
||||
const char* type() const override { return "code_interpreter"; }
|
||||
Json to_json() const override;
|
||||
// Container id; the spec allows an id or an object, the guides show neither
|
||||
// (SPEC Unverified), so only the id form is offered.
|
||||
CodeExecutionTool& container(std::string id) { container_ = std::move(id); return *this; }
|
||||
|
||||
private:
|
||||
std::string container_;
|
||||
};
|
||||
|
||||
// Collections search (wire type `file_search`, vector_store_ids are
|
||||
// collection ids).
|
||||
class CollectionsSearchTool : public Tool {
|
||||
public:
|
||||
explicit CollectionsSearchTool(std::vector<std::string> collection_ids)
|
||||
: ids_(std::move(collection_ids)) {}
|
||||
const char* type() const override { return "file_search"; }
|
||||
Json to_json() const override;
|
||||
CollectionsSearchTool& max_results(int n) { max_results_ = n; return *this; }
|
||||
|
||||
private:
|
||||
std::vector<std::string> ids_;
|
||||
std::optional<int> max_results_;
|
||||
};
|
||||
|
||||
class McpTool : public Tool {
|
||||
public:
|
||||
McpTool(std::string server_label, std::string server_url)
|
||||
: label_(std::move(server_label)), url_(std::move(server_url)) {}
|
||||
const char* type() const override { return "mcp"; }
|
||||
Json to_json() const override;
|
||||
|
||||
McpTool& description(std::string d) { description_ = std::move(d); return *this; }
|
||||
McpTool& allow_tools(std::vector<std::string> t) { allowed_ = std::move(t); return *this; }
|
||||
// Sent as the Authorization header value toward the MCP server. Never logged.
|
||||
McpTool& authorization(std::string a) { authorization_ = std::move(a); return *this; }
|
||||
McpTool& header(std::string name, std::string value);
|
||||
McpTool& defer_loading(bool v) { defer_loading_ = v; return *this; }
|
||||
|
||||
private:
|
||||
std::string label_, url_, description_, authorization_;
|
||||
std::vector<std::string> allowed_;
|
||||
std::vector<std::pair<std::string, std::string>> headers_;
|
||||
std::optional<bool> defer_loading_;
|
||||
};
|
||||
|
||||
class ImageGenerationTool : public Tool {
|
||||
public:
|
||||
const char* type() const override { return "image_generation"; }
|
||||
Json to_json() const override;
|
||||
// "auto" (default), "generate" or "edit".
|
||||
ImageGenerationTool& action(std::string a) { action_ = std::move(a); return *this; }
|
||||
|
||||
private:
|
||||
std::string action_;
|
||||
};
|
||||
|
||||
struct ShellSkill {
|
||||
std::string name, description, path;
|
||||
};
|
||||
|
||||
// Client-executed shell: the model emits shell_call items, the caller runs
|
||||
// them and answers with shell_call_output.
|
||||
class ShellTool : public Tool {
|
||||
public:
|
||||
const char* type() const override { return "shell"; }
|
||||
Json to_json() const override;
|
||||
ShellTool& skill(ShellSkill s) { skills_.push_back(std::move(s)); return *this; }
|
||||
|
||||
private:
|
||||
std::vector<ShellSkill> skills_;
|
||||
};
|
||||
|
||||
class ToolSearchTool : public Tool {
|
||||
public:
|
||||
const char* type() const override { return "tool_search"; }
|
||||
Json to_json() const override; // execution:"server" (the only supported value)
|
||||
};
|
||||
|
||||
// tool_choice: "auto", "none", "required" or one named function.
|
||||
class ToolChoice {
|
||||
public:
|
||||
static ToolChoice automatic() { return ToolChoice("auto", ""); }
|
||||
static ToolChoice none() { return ToolChoice("none", ""); }
|
||||
static ToolChoice required() { return ToolChoice("required", ""); }
|
||||
static ToolChoice function(const std::string& name);
|
||||
// From the neutral string: "", "auto", "none", "required" or a function name.
|
||||
static std::optional<ToolChoice> from_neutral(const std::string& value);
|
||||
|
||||
Json to_responses() const; // "auto" | {type:"function", name}
|
||||
Json to_chat() const; // "auto" | {type:"function", function:{name}}
|
||||
|
||||
private:
|
||||
ToolChoice(std::string mode, std::string name) : mode_(std::move(mode)), name_(std::move(name)) {}
|
||||
std::string mode_;
|
||||
std::string name_;
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check src/xai/core/calls.cpp against xAI's OpenAPI document and REST reference.
|
||||
|
||||
Fails (exit 1) when a declared call is absent or differs (HTTP method, path,
|
||||
operationId, query parameter, request body field, request / response schema),
|
||||
when a schema field or tool field we use is not in the document, when a
|
||||
batch (docs-only) row or a documented-only field is not on its documentation
|
||||
page, or when the document has an operation that is neither in the table nor
|
||||
in the explicit exclusion list (owned by the xAI platform package).
|
||||
Python 3 stdlib only, no credentials needed.
|
||||
|
||||
python3 -I src/xai/core/tools/check_spec.py # fetch live
|
||||
python3 -I src/xai/core/tools/check_spec.py --file spec.json --no-docs # offline
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
CALLS = os.path.join(HERE, "..", "calls.cpp")
|
||||
SPEC_URL = "https://api.x.ai/api-docs/openapi.json"
|
||||
DOCS = "https://docs.x.ai/developers/%s.md"
|
||||
BATCH_PAGE = "rest-api-reference/inference/batches"
|
||||
|
||||
# Operations of the document that belong to src/xai/platform (files, collections search).
|
||||
EXCLUDED_OPS = {
|
||||
"handle_list_files_request", "handle_upload_file_request", "handle_retrieve_file_request",
|
||||
"handle_delete_file_request", "handle_download_file_content_request",
|
||||
"handle_create_public_url_request", "handle_revoke_public_url_request",
|
||||
"handle_document_search_request_v2",
|
||||
}
|
||||
|
||||
ROW = re.compile(r'^\s*\{(.*)\},?\s*$')
|
||||
STR = re.compile(r'"([^"]*)"')
|
||||
|
||||
|
||||
def block(text, begin, end):
|
||||
a, b = text.index(begin), text.index(end)
|
||||
rows = []
|
||||
for line in text[a:b].splitlines()[1:]:
|
||||
m = ROW.match(line)
|
||||
if m and '"' in line:
|
||||
rows.append(STR.findall(m.group(1)))
|
||||
return rows
|
||||
|
||||
|
||||
def fetch(url):
|
||||
with urllib.request.urlopen(url, timeout=60) as r:
|
||||
return r.read().decode("utf-8")
|
||||
|
||||
|
||||
def resolve(spec, node):
|
||||
while isinstance(node, dict) and "$ref" in node:
|
||||
node = spec["components"]["schemas"][node["$ref"].split("/")[-1]]
|
||||
return node
|
||||
|
||||
|
||||
def props(spec, node, depth=0):
|
||||
"""All property names of a schema, through $ref / allOf / oneOf / anyOf."""
|
||||
out = {}
|
||||
node = resolve(spec, node)
|
||||
if not isinstance(node, dict) or depth > 8:
|
||||
return out
|
||||
out.update(node.get("properties", {}))
|
||||
for key in ("allOf", "oneOf", "anyOf"):
|
||||
for sub in node.get(key, []):
|
||||
out.update(props(spec, sub, depth + 1))
|
||||
return out
|
||||
|
||||
|
||||
def ref_name(node):
|
||||
return node["$ref"].split("/")[-1] if isinstance(node, dict) and "$ref" in node else ""
|
||||
|
||||
|
||||
def tool_variants(spec):
|
||||
"""type const -> property names, for every ModelTool variant."""
|
||||
out = {}
|
||||
for variant in spec["components"]["schemas"]["ModelTool"]["oneOf"]:
|
||||
p = props(spec, variant)
|
||||
t = p.get("type", {}).get("enum", [None])[0]
|
||||
if t:
|
||||
out[t] = p
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
argv = sys.argv[1:]
|
||||
path = argv[argv.index("--file") + 1] if "--file" in argv else None
|
||||
docs_on = "--no-docs" not in argv
|
||||
raw = open(path).read() if path else fetch(SPEC_URL)
|
||||
spec = json.loads(raw)
|
||||
text = open(CALLS).read()
|
||||
calls = block(text, "// CALLS-BEGIN", "// CALLS-END")
|
||||
schemas = block(text, "// SCHEMAS-BEGIN", "// SCHEMAS-END")
|
||||
tools = block(text, "// TOOLS-BEGIN", "// TOOLS-END")
|
||||
S = spec["components"]["schemas"]
|
||||
ops = {}
|
||||
for p, item in spec["paths"].items():
|
||||
for m, o in item.items():
|
||||
if m in ("get", "post", "put", "delete", "patch"):
|
||||
ops[(m.upper(), p)] = o
|
||||
bad, covered = [], set()
|
||||
docs_cache = {}
|
||||
|
||||
def page(name):
|
||||
if name not in docs_cache:
|
||||
docs_cache[name] = fetch(DOCS % name)
|
||||
return docs_cache[name]
|
||||
|
||||
for method, p, op_id, body, query, doc_fields, req, resp, source in calls:
|
||||
if source == "openapi":
|
||||
o = ops.get((method, p))
|
||||
if not o:
|
||||
bad.append("missing operation %s %s" % (method, p))
|
||||
continue
|
||||
covered.add(o["operationId"])
|
||||
if o["operationId"] != op_id:
|
||||
bad.append("%s %s: operationId %s != %s" % (method, p, op_id, o["operationId"]))
|
||||
have_q = {x["name"] for x in o.get("parameters", []) if x["in"] == "query"}
|
||||
for q in filter(None, query.split(",")):
|
||||
if q not in have_q:
|
||||
bad.append("%s %s: query param %s absent" % (method, p, q))
|
||||
content = o.get("requestBody", {}).get("content", {})
|
||||
rschema = next(iter(content.values()), {}).get("schema") if content else None
|
||||
if rschema is None and (body or req):
|
||||
bad.append("%s %s: no request body in spec" % (method, p))
|
||||
if rschema is not None:
|
||||
if ref_name(rschema) != req:
|
||||
bad.append("%s %s: request schema %s != %s" % (method, p, req, ref_name(rschema)))
|
||||
pr = props(spec, rschema)
|
||||
for f in filter(None, body.split(",")):
|
||||
if f not in pr:
|
||||
bad.append("%s %s: body field %s absent in %s" % (method, p, f, req))
|
||||
ok = o["responses"].get("200", {}).get("content", {})
|
||||
got = ref_name(next(iter(ok.values()), {}).get("schema", {})) or "string"
|
||||
if got != resp:
|
||||
bad.append("%s %s: response schema %s != %s" % (method, p, resp, got))
|
||||
elif docs_on:
|
||||
md = page(BATCH_PAGE).replace("\\", "")
|
||||
head = "## %s %s" % (method, p)
|
||||
if head not in md:
|
||||
bad.append("docs: heading %r absent from the batches reference" % head)
|
||||
continue
|
||||
section = md.split(head, 1)[1].split("\n## ", 1)[0]
|
||||
for f in filter(None, (body + "," + query).split(",")):
|
||||
if "`%s`" % f not in section:
|
||||
bad.append("docs: %s %s field %s absent" % (method, p, f))
|
||||
if docs_on:
|
||||
for entry in filter(None, doc_fields.split(",")):
|
||||
f, _, pg = entry.partition("@")
|
||||
if f not in page(pg):
|
||||
bad.append("docs: field %s absent from %s" % (f, pg))
|
||||
for o_key, o in ops.items():
|
||||
if o["operationId"] not in covered and o["operationId"] not in EXCLUDED_OPS:
|
||||
bad.append("unwrapped operation %s %s (%s)" % (o_key[0], o_key[1], o["operationId"]))
|
||||
for name, fields in schemas:
|
||||
if name not in S:
|
||||
bad.append("missing schema " + name)
|
||||
continue
|
||||
pr = props(spec, S[name])
|
||||
for f in fields.split(","):
|
||||
if f not in pr:
|
||||
bad.append("schema %s.%s absent" % (name, f))
|
||||
variants = tool_variants(spec)
|
||||
for t, fields in tools:
|
||||
if t not in variants:
|
||||
bad.append("tool type %s absent from ModelTool" % t)
|
||||
continue
|
||||
for f in fields.split(","):
|
||||
if f not in variants[t]:
|
||||
bad.append("tool %s field %s absent" % (t, f))
|
||||
digest = hashlib.sha256(raw.encode()).hexdigest()[:16]
|
||||
info = "spec %s v%s sha256:%s, %d operations" % (spec["info"]["title"], spec["info"]["version"], digest, len(ops))
|
||||
if bad:
|
||||
print("SPEC CHECK FAILED (%s)" % info)
|
||||
for b in bad:
|
||||
print(" " + b)
|
||||
return 1
|
||||
print("spec ok: %d calls (%d openapi, %d docs), %d schemas, %d tool types, %d platform operations excluded; %s"
|
||||
% (len(calls), sum(1 for c in calls if c[8] == "openapi"), sum(1 for c in calls if c[8] == "docs"),
|
||||
len(schemas), len(tools), len(EXCLUDED_OPS), info))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,91 @@
|
||||
#include "xai/core/usage.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
ToolUsage ToolUsage::from_json(const Json& doc) {
|
||||
ToolUsage t;
|
||||
t.code_interpreter_calls = jx::num(doc, "code_interpreter_calls");
|
||||
t.document_search_calls = jx::num(doc, "document_search_calls");
|
||||
t.file_search_calls = jx::num(doc, "file_search_calls");
|
||||
t.image_generation_calls = jx::num(doc, "image_generation_calls");
|
||||
t.mcp_calls = jx::num(doc, "mcp_calls");
|
||||
t.web_search_calls = jx::num(doc, "web_search_calls");
|
||||
t.x_posts_fetched = jx::num(doc, "x_posts_fetched");
|
||||
t.x_search_calls = jx::num(doc, "x_search_calls");
|
||||
t.x_users_fetched = jx::num(doc, "x_users_fetched");
|
||||
return t;
|
||||
}
|
||||
|
||||
long ToolUsage::total_calls() const {
|
||||
return code_interpreter_calls + document_search_calls + file_search_calls +
|
||||
image_generation_calls + mcp_calls + web_search_calls + x_search_calls;
|
||||
}
|
||||
|
||||
TokenUsage TokenUsage::from_chat(const Json& usage) {
|
||||
TokenUsage u;
|
||||
if (!usage.is_object()) return u;
|
||||
u.present = true;
|
||||
u.input_tokens = jx::num(usage, "prompt_tokens");
|
||||
u.output_tokens = jx::num(usage, "completion_tokens");
|
||||
u.total_tokens = jx::num(usage, "total_tokens", u.input_tokens + u.output_tokens);
|
||||
Json prompt = jx::object(usage, "prompt_tokens_details");
|
||||
u.cached_tokens = jx::num(prompt, "cached_tokens");
|
||||
u.text_tokens = jx::num(prompt, "text_tokens");
|
||||
u.image_tokens = jx::num(prompt, "image_tokens");
|
||||
u.audio_tokens = jx::num(prompt, "audio_tokens");
|
||||
u.reasoning_tokens = jx::num(jx::object(usage, "completion_tokens_details"), "reasoning_tokens");
|
||||
u.num_sources_used = jx::num(usage, "num_sources_used");
|
||||
u.cost_usd_ticks = jx::num(usage, "cost_in_usd_ticks");
|
||||
u.tools = ToolUsage::from_json(jx::object(usage, "server_side_tool_usage_details"));
|
||||
u.num_server_side_tools_used = u.tools.total_calls();
|
||||
return u;
|
||||
}
|
||||
|
||||
TokenUsage TokenUsage::from_responses(const Json& usage) {
|
||||
TokenUsage u;
|
||||
if (!usage.is_object()) return u;
|
||||
u.present = true;
|
||||
u.input_tokens = jx::num(usage, "input_tokens");
|
||||
u.output_tokens = jx::num(usage, "output_tokens");
|
||||
u.total_tokens = jx::num(usage, "total_tokens", u.input_tokens + u.output_tokens);
|
||||
u.cached_tokens = jx::num(jx::object(usage, "input_tokens_details"), "cached_tokens");
|
||||
u.reasoning_tokens = jx::num(jx::object(usage, "output_tokens_details"), "reasoning_tokens");
|
||||
u.num_sources_used = jx::num(usage, "num_sources_used");
|
||||
u.num_server_side_tools_used = jx::num(usage, "num_server_side_tools_used");
|
||||
u.cost_usd_ticks = jx::num(usage, "cost_in_usd_ticks");
|
||||
if (const Json* nano = jx::get(usage, "cost_in_nano_usd"); nano && nano->is_number())
|
||||
u.cost_nano_usd = nano->as_int();
|
||||
Json ctx = jx::object(usage, "context_details");
|
||||
if (const Json* v = jx::get(ctx, "input_tokens"); v && v->is_number()) u.context_input_tokens = v->as_int();
|
||||
if (const Json* v = jx::get(ctx, "output_tokens"); v && v->is_number()) u.context_output_tokens = v->as_int();
|
||||
u.tools = ToolUsage::from_json(jx::object(usage, "server_side_tool_usage_details"));
|
||||
return u;
|
||||
}
|
||||
|
||||
llm::Usage TokenUsage::to_neutral() const {
|
||||
llm::Usage n;
|
||||
n.input_tokens = static_cast<int>(input_tokens);
|
||||
n.output_tokens = static_cast<int>(output_tokens);
|
||||
n.total_tokens = static_cast<int>(total_tokens);
|
||||
n.cached_tokens = static_cast<int>(cached_tokens);
|
||||
n.reasoning_tokens = static_cast<int>(reasoning_tokens);
|
||||
return n;
|
||||
}
|
||||
|
||||
MediaUsage MediaUsage::from_json(const Json& usage) {
|
||||
MediaUsage m;
|
||||
if (!usage.is_object()) return m;
|
||||
m.present = true;
|
||||
m.cost_usd_ticks = jx::num(usage, "cost_in_usd_ticks");
|
||||
auto opt = [&](const char* k) -> std::optional<long> {
|
||||
const Json* v = jx::get(usage, k);
|
||||
if (v && v->is_number()) return v->as_int();
|
||||
return std::nullopt;
|
||||
};
|
||||
m.input_tokens = opt("input_tokens");
|
||||
m.output_tokens = opt("output_tokens");
|
||||
m.total_tokens = opt("total_tokens");
|
||||
return m;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,69 @@
|
||||
#pragma once
|
||||
|
||||
// Token, tool and cost accounting as xAI reports it. Chat completions use
|
||||
// `prompt_tokens` / `completion_tokens`, the Responses API uses
|
||||
// `input_tokens` / `output_tokens`; TokenUsage reads both into one shape.
|
||||
// Cost is reported in "ticks": 1e10 ticks = 1 USD (spec: one USD cent equals
|
||||
// 100,000,000 ticks).
|
||||
|
||||
#include <optional>
|
||||
|
||||
#include "llm/chat_types.h"
|
||||
#include "xai/core/params.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
constexpr double kTicksPerUsd = 1e10;
|
||||
|
||||
struct ToolUsage {
|
||||
long code_interpreter_calls = 0;
|
||||
long document_search_calls = 0;
|
||||
long file_search_calls = 0;
|
||||
long image_generation_calls = 0;
|
||||
long mcp_calls = 0;
|
||||
long web_search_calls = 0;
|
||||
long x_posts_fetched = 0;
|
||||
long x_search_calls = 0;
|
||||
long x_users_fetched = 0;
|
||||
|
||||
static ToolUsage from_json(const Json& doc);
|
||||
long total_calls() const;
|
||||
};
|
||||
|
||||
struct TokenUsage {
|
||||
long input_tokens = 0;
|
||||
long output_tokens = 0;
|
||||
long total_tokens = 0;
|
||||
long cached_tokens = 0;
|
||||
long reasoning_tokens = 0;
|
||||
long text_tokens = 0;
|
||||
long image_tokens = 0;
|
||||
long audio_tokens = 0;
|
||||
long num_sources_used = 0;
|
||||
long num_server_side_tools_used = 0;
|
||||
long cost_usd_ticks = 0;
|
||||
std::optional<long> cost_nano_usd;
|
||||
std::optional<long> context_input_tokens; // Responses: latest context window
|
||||
std::optional<long> context_output_tokens;
|
||||
ToolUsage tools;
|
||||
bool present = false;
|
||||
|
||||
static TokenUsage from_chat(const Json& usage); // prompt_tokens / completion_tokens
|
||||
static TokenUsage from_responses(const Json& usage); // input_tokens / output_tokens
|
||||
double cost_usd() const { return static_cast<double>(cost_usd_ticks) / kTicksPerUsd; }
|
||||
llm::Usage to_neutral() const;
|
||||
};
|
||||
|
||||
// Image and video usage (cost plus optional token counts).
|
||||
struct MediaUsage {
|
||||
long cost_usd_ticks = 0;
|
||||
std::optional<long> input_tokens;
|
||||
std::optional<long> output_tokens;
|
||||
std::optional<long> total_tokens;
|
||||
bool present = false;
|
||||
|
||||
static MediaUsage from_json(const Json& usage);
|
||||
double cost_usd() const { return static_cast<double>(cost_usd_ticks) / kTicksPerUsd; }
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,183 @@
|
||||
#include "xai/core/videos.h"
|
||||
|
||||
#include "xai/core/caps.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
namespace {
|
||||
void write_common(Json& body, const std::string& model, const std::optional<StorageOptions>& storage,
|
||||
const std::string& upload_url, const std::string& user) {
|
||||
if (!model.empty()) {
|
||||
Check::model(model);
|
||||
body.set("model", Json(model));
|
||||
}
|
||||
if (storage) body.set("storage_options", storage->to_json());
|
||||
if (!upload_url.empty()) {
|
||||
Check::http_url("upload url", upload_url);
|
||||
Json out = Json::object();
|
||||
out.set("upload_url", Json(upload_url));
|
||||
body.set("output", out);
|
||||
}
|
||||
if (!user.empty()) {
|
||||
Check::text("user", user, 256);
|
||||
body.set("user", Json(user));
|
||||
}
|
||||
}
|
||||
|
||||
std::string request_id_of(const Json& reply) {
|
||||
std::string id = jx::str(reply, "request_id");
|
||||
if (id.empty()) throw llm::ApiError(kVendor, 502, "BAD_RESPONSE", "reply has no request_id");
|
||||
return id;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json VideoGenerationRequest::to_json() const {
|
||||
bool has_refs = !reference_images.empty() || !reference_audios.empty();
|
||||
bool has_frames = image || last_frame || !keyframes.empty();
|
||||
if (prompt.empty() && !has_refs && !has_frames) Check::fail("prompt is required without images or references");
|
||||
Check::text("prompt", prompt, Caps::kMaxPromptBytes, true);
|
||||
Json body = Json::object();
|
||||
if (!prompt.empty()) body.set("prompt", Json(prompt));
|
||||
if (duration) {
|
||||
Check::range("duration", *duration, Caps::kMinVideoSeconds, Caps::kMaxVideoSeconds);
|
||||
body.set("duration", Json(*duration));
|
||||
}
|
||||
if (!aspect_ratio.empty()) {
|
||||
Check::one_of("aspect_ratio", aspect_ratio, {"1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3", "21:9", "5:2"});
|
||||
body.set("aspect_ratio", Json(aspect_ratio));
|
||||
}
|
||||
if (!resolution.empty()) {
|
||||
Check::one_of("resolution", resolution, {"480p", "720p", "1080p"});
|
||||
body.set("resolution", Json(resolution));
|
||||
}
|
||||
if (image) body.set("image", image->to_json());
|
||||
if (last_frame) body.set("last_frame", last_frame->to_json());
|
||||
Check::upto("reference_images", reference_images.size(), Caps::kMaxReferenceImages);
|
||||
if (!reference_images.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const auto& r : reference_images) arr.push(r.to_json());
|
||||
body.set("reference_images", arr);
|
||||
}
|
||||
Check::upto("reference_audios", reference_audios.size(), Caps::kMaxReferenceAudios);
|
||||
if (!reference_audios.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const auto& r : reference_audios) arr.push(r.to_json());
|
||||
body.set("reference_audios", arr);
|
||||
}
|
||||
Check::upto("keyframes", keyframes.size(), Caps::kMaxKeyframes);
|
||||
if (!keyframes.empty()) {
|
||||
Json arr = Json::array();
|
||||
for (const auto& k : keyframes) {
|
||||
if (k.image.empty()) Check::fail("keyframe image is required");
|
||||
double limit = duration ? *duration : Caps::kMaxVideoSeconds;
|
||||
if (!(k.timestamp_s > 0 && k.timestamp_s < limit)) Check::fail("keyframe timestamp_s must be inside the clip");
|
||||
Json o = Json::object();
|
||||
o.set("image", k.image.to_json());
|
||||
o.set("timestamp_s", Json(k.timestamp_s));
|
||||
arr.push(o);
|
||||
}
|
||||
body.set("keyframes", arr);
|
||||
}
|
||||
write_common(body, model, storage, upload_url, user);
|
||||
return body;
|
||||
}
|
||||
|
||||
Json VideoEditRequest::to_json() const {
|
||||
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
|
||||
if (video.empty()) Check::fail("video is required");
|
||||
Json body = Json::object();
|
||||
body.set("prompt", Json(prompt));
|
||||
body.set("video", video.to_json());
|
||||
write_common(body, model, storage, upload_url, user);
|
||||
return body;
|
||||
}
|
||||
|
||||
Json VideoExtendRequest::to_json() const {
|
||||
Check::text("prompt", prompt, Caps::kMaxPromptBytes);
|
||||
if (video.empty()) Check::fail("video is required");
|
||||
Json body = Json::object();
|
||||
body.set("prompt", Json(prompt));
|
||||
body.set("video", video.to_json());
|
||||
if (duration) {
|
||||
Check::range("duration", *duration, Caps::kMinExtensionSeconds, Caps::kMaxExtensionSeconds);
|
||||
body.set("duration", Json(*duration));
|
||||
}
|
||||
write_common(body, model, storage, upload_url, "");
|
||||
return body;
|
||||
}
|
||||
|
||||
VideoJob VideoJob::parse(const Json& doc, const std::string& request_id) {
|
||||
VideoJob j;
|
||||
j.request_id = request_id;
|
||||
j.raw = doc;
|
||||
j.status = jx::str(doc, "status");
|
||||
j.model = jx::str(doc, "model");
|
||||
if (const Json* p = jx::get(doc, "progress"); p && p->is_number()) j.progress = p->as_int();
|
||||
if (const Json* u = jx::get(doc, "usage"); u && u->is_object()) j.usage = MediaUsage::from_json(*u);
|
||||
j.error = MediaError::parse(doc, "error");
|
||||
if (const Json* v = jx::get(doc, "video"); v && v->is_object()) {
|
||||
GeneratedVideo g;
|
||||
g.url = jx::str(*v, "url");
|
||||
g.storage_error = jx::str(*v, "storage_error");
|
||||
g.duration = jx::num(*v, "duration");
|
||||
g.respect_moderation = jx::flag(*v, "respect_moderation");
|
||||
if (const Json* f = jx::get(*v, "file_output"); f && f->is_object()) g.file = StoredFile::parse(*f);
|
||||
j.video = std::move(g);
|
||||
}
|
||||
if (j.status.empty()) j.status = j.error ? "failed" : (j.video ? "done" : "pending");
|
||||
return j;
|
||||
}
|
||||
|
||||
std::string VideoService::start_generate(const VideoGenerationRequest& request) {
|
||||
Json body = request.to_json();
|
||||
return request_id_of(call_json(Call::kVideosGenerate, {}, &body));
|
||||
}
|
||||
|
||||
std::string VideoService::start_edit(const VideoEditRequest& request) {
|
||||
Json body = request.to_json();
|
||||
return request_id_of(call_json(Call::kVideosEdit, {}, &body));
|
||||
}
|
||||
|
||||
std::string VideoService::start_extend(const VideoExtendRequest& request) {
|
||||
Json body = request.to_json();
|
||||
return request_id_of(call_json(Call::kVideosExtend, {}, &body));
|
||||
}
|
||||
|
||||
VideoJob VideoService::poll(const std::string& request_id) {
|
||||
llm::RawResponse raw = call_raw(Call::kVideosGet, {request_id});
|
||||
if (raw.status == 202) {
|
||||
VideoJob j;
|
||||
j.request_id = request_id;
|
||||
j.status = "pending";
|
||||
return j;
|
||||
}
|
||||
return VideoJob::parse(parse_body(raw), request_id);
|
||||
}
|
||||
|
||||
VideoJob VideoService::wait(const std::string& request_id, const WaitPolicy& policy, const Sleeper& sleeper) {
|
||||
return wait(request_id, policy, sleeper, Progress{});
|
||||
}
|
||||
|
||||
VideoJob VideoService::wait(const std::string& request_id, const WaitPolicy& policy, const Sleeper& sleeper,
|
||||
const Progress& progress) {
|
||||
Check::id("request", request_id);
|
||||
bool cancelled = false;
|
||||
std::function<std::optional<VideoJob>()> step = [&]() -> std::optional<VideoJob> {
|
||||
VideoJob j = poll(request_id);
|
||||
if (progress && !progress(j)) {
|
||||
cancelled = true;
|
||||
return j;
|
||||
}
|
||||
if (j.pending()) return std::nullopt;
|
||||
return j;
|
||||
};
|
||||
std::optional<VideoJob> j = poll_until<VideoJob>(policy, step, sleeper);
|
||||
if (!j) throw llm::ApiError(kVendor, 504, "WAIT_TIMEOUT", "video not ready within the wait budget");
|
||||
if (cancelled && j->pending()) throw llm::ApiError(kVendor, 499, "WAIT_CANCELLED", "wait stopped by the caller");
|
||||
if (j->failed())
|
||||
throw llm::ApiError(kVendor, 502, "GENERATION_FAILED",
|
||||
j->error ? j->error->code + ": " + j->error->message : "video generation failed");
|
||||
return *j;
|
||||
}
|
||||
|
||||
} // namespace molo::xai::core
|
||||
@@ -0,0 +1,100 @@
|
||||
#pragma once
|
||||
|
||||
// Video generation (async): text-to-video, image-to-video, reference-to-video
|
||||
// (/v1/videos/generations), editing (/v1/videos/edits), extension
|
||||
// (/v1/videos/extensions), and the poll (GET /v1/videos/{request_id}).
|
||||
// Every start call returns a request id immediately; the video is polled.
|
||||
// Video costs real money per second of output; callers confirm before use.
|
||||
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "xai/core/media.h"
|
||||
#include "xai/core/poll.h"
|
||||
#include "xai/core/service.h"
|
||||
|
||||
namespace molo::xai::core {
|
||||
|
||||
struct VideoKeyframe {
|
||||
MediaSource image;
|
||||
double timestamp_s = 0; // strictly inside the clip: 0 < t < duration
|
||||
};
|
||||
|
||||
struct VideoGenerationRequest {
|
||||
std::string model;
|
||||
std::string prompt; // required for text-to-video and reference-to-video
|
||||
std::optional<int> duration; // seconds 1..15 (default 8)
|
||||
std::string aspect_ratio; // 1:1 16:9 9:16 4:3 3:4 3:2 2:3 21:9 5:2
|
||||
std::string resolution; // 480p | 720p | 1080p
|
||||
std::optional<MediaSource> image; // first frame (image-to-video)
|
||||
std::optional<MediaSource> last_frame; // documented in the guides, not in the OpenAPI document
|
||||
std::vector<MediaSource> reference_images; // up to 14
|
||||
std::vector<MediaSource> reference_audios; // up to 3 (preset voices)
|
||||
std::vector<VideoKeyframe> keyframes; // up to 4
|
||||
std::optional<StorageOptions> storage;
|
||||
std::string upload_url; // output.upload_url: signed PUT target
|
||||
std::string user;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct VideoEditRequest {
|
||||
std::string model;
|
||||
std::string prompt;
|
||||
MediaSource video; // .mp4, up to 8.7 s per the guides
|
||||
std::optional<StorageOptions> storage;
|
||||
std::string upload_url;
|
||||
std::string user;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct VideoExtendRequest {
|
||||
std::string model;
|
||||
std::string prompt;
|
||||
MediaSource video;
|
||||
std::optional<int> duration; // seconds 2..10 (default 6)
|
||||
std::optional<StorageOptions> storage;
|
||||
std::string upload_url;
|
||||
Json to_json() const;
|
||||
};
|
||||
|
||||
struct GeneratedVideo {
|
||||
std::string url, storage_error;
|
||||
long duration = 0;
|
||||
bool respect_moderation = false;
|
||||
std::optional<StoredFile> file;
|
||||
};
|
||||
|
||||
struct VideoJob {
|
||||
std::string request_id;
|
||||
std::string status; // pending | done | failed (expired is possible; see SPEC Unverified)
|
||||
std::string model;
|
||||
std::optional<long> progress; // 0..100
|
||||
std::optional<GeneratedVideo> video;
|
||||
MediaUsage usage;
|
||||
std::optional<MediaError> error;
|
||||
Json raw;
|
||||
bool done() const { return status == "done"; }
|
||||
bool failed() const { return status == "failed" || error.has_value(); }
|
||||
bool pending() const { return !done() && !failed(); }
|
||||
static VideoJob parse(const Json& doc, const std::string& request_id);
|
||||
};
|
||||
|
||||
class VideoService : public XaiService {
|
||||
public:
|
||||
explicit VideoService(std::shared_ptr<llm::ApiClient> client) : XaiService(std::move(client)) {}
|
||||
|
||||
std::string start_generate(const VideoGenerationRequest& request);
|
||||
std::string start_edit(const VideoEditRequest& request);
|
||||
std::string start_extend(const VideoExtendRequest& request);
|
||||
VideoJob poll(const std::string& request_id);
|
||||
// Polls within the policy; throws 504 WAIT_TIMEOUT or 502 GENERATION_FAILED.
|
||||
VideoJob wait(const std::string& request_id, const WaitPolicy& policy = {},
|
||||
const Sleeper& sleeper = real_sleeper());
|
||||
// Cooperative variant for UIs: reports every poll, return false to stop waiting.
|
||||
using Progress = std::function<bool(const VideoJob&)>;
|
||||
VideoJob wait(const std::string& request_id, const WaitPolicy& policy, const Sleeper& sleeper,
|
||||
const Progress& progress);
|
||||
};
|
||||
|
||||
} // namespace molo::xai::core
|
||||
Reference in New Issue
Block a user