Merge openai core package

This commit is contained in:
2026-10-24 07:45:38 +02:00
68 changed files with 8399 additions and 3 deletions
+190
View File
@@ -0,0 +1,190 @@
# OpenAI core - Specification
Owned directories: `src/openai/core/`, `tests/src/openai/core/`,
`example/app/ai/OpenaiDemo.cpp`, `example/assets/ai_openai_core.js`,
`tests/example/ai/openai/core/`. Namespace `molo::openai::core`. Plan:
`plans/17-llm-vendor-platforms.md` (package "openai core"). Foundation:
`src/llm/` (`ApiClient`, `Service`, `SseDecoder`, `ChatModel`), not edited.
## 1. Scope
The core of the OpenAI API platform at `https://api.openai.com/v1`:
- Responses API: create, streaming (all 59 event types), background mode
with polling, retrieve, delete, cancel, input items (single page and capped
walk), input token preview, compaction, `previous_response_id` and
`conversation`, reasoning controls, structured outputs, stored prompts.
- Chat Completions: create, streaming chunks, stored completions (get, list,
update metadata, delete, messages).
- Legacy Completions (plain and streaming).
- Tools: function, custom (text and grammar), web search, file search, code
interpreter, computer use, image generation, remote MCP and connectors,
local shell, shell, apply patch, tool search (builders plus validation of
the 18 tool types of the spec's Tool union).
- Images: generations, edits (multipart upload or JSON references),
variations, streaming of generations and JSON edits.
- Audio: speech (blob and SSE), transcriptions, translations, custom voices
and voice consents.
- Videos (Sora): create, edit, extend, remix, characters, list, retrieve,
delete, content download, polling.
- Embeddings, Moderations, Models (list, retrieve, delete a fine-tune).
- `/decisions` and `/content_provenance_checks` (single-call core endpoints
nobody else owns).
- `OpenaiChatModel : llm::ChatModel` over Responses (default) or Chat
Completions.
Not here (other packages): Realtime and Live, Files, Uploads, Batch,
Fine-tuning, Vector stores, Evals, Conversations, Containers, Assistants and
Threads, ChatKit, Skills (`openai/platform`); Organization admin, projects,
webhooks, audit logs (`openai/admin`); Agents and Vaults surfaces of the spec
(no package owns them; listed in `tools/check_spec.py` EXCLUDED).
## 2. Module map
| File | Concern |
|---|---|
| `caps.h` | Static ceilings (`CoreCaps`) |
| `calls.h/.cpp` | The ONE table of 44 REST calls, `render`, `accepts_field` |
| `check.h/.cpp` | `Check`: validation helpers (all failures are `ApiError::invalid`) |
| `body.h/.cpp` | `Body` (request builder, `extra` merge guard), `Doc` (read helpers) |
| `form_body.h/.cpp`, `upload_file.h` | `FormBody` multipart builder, `UploadFile` |
| `client.h/.cpp` | `OpenaiClient`: `ApiClient` config, org and project headers |
| `service.h/.cpp` | `OpenaiService` base, `Query`, `ListParams`, `Listing` |
| `responses_request.*` | `ResponseRequest`, `TokenCountRequest`, `CompactRequest` |
| `responses.*` | `ResponsesService` |
| `chat_request.*` | `ChatMessages`, `ChatCompletionRequest`, `CompletionRequest` |
| `chat_completions.*`, `completions.*` | `ChatCompletionsService`, `CompletionsService` |
| `embeddings.*`, `moderations.*`, `models.*` | the like-named services |
| `images.*`, `speech.*`, `transcriptions.*`, `voices.*`, `videos.*` | media services and their requests |
| `decisions.*` | `DecisionsService` (decisions, provenance checks) |
| `chat_model.*` | `OpenaiChatModel`, `Dialect` |
| `wire/tool_schemas.*` | `ToolSchemas`: tool and tool_choice JSON, reusable |
| `wire/input_items.*` | `InputItems`: Responses input items, neutral message mapping |
| `wire/response_config.*` | `TextFormat`, `ReasoningConfig`, `StrictSchema` |
| `wire/chat_wire.*` | `ChatWire`, `ResponsesWire`, `ChatChunkDecoder`, `WireOptions` |
| `wire/response_events.*` | event table, `ResponseEvent`, `ResponseAccumulator`, `ResponsesChunkDecoder` |
| `wire/neutral_map.*` | usage and finish reason mapping |
| `tools/check_spec.py` | spec conformance checker |
The `wire/` classes depend only on `llm` and `clients::body::Json`, so other
packages that speak OpenAI's wire format can reuse them.
## 3. Contracts
- Credential: `OPENAI_API_KEY` via `llm::Config`, bearer. Optional
`OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` become `OpenAI-Organization` and
`OpenAI-Project` headers. Without a key every call throws
`NotConfiguredError` (503 `NOT_CONFIGURED`) before any traffic.
- Validation happens before the network: `ApiError::invalid` (400
`INVALID_ARGUMENT`). Ids go through `Service::valid_id` before a path is
built. `extra` on a request merges only properties the call lists in the
table and never overrides a typed member (loud error otherwise).
- Destructive calls are explicit methods with no default: `remove` (responses,
stored chat completions, models, videos, voices, consents) and `cancel`.
`ModelsService::remove` refuses ids that do not start with `ft:`.
- List walks (`*_all`) are capped at 20 pages (default 5) and return
`Listing::truncated` plus `next_after`; a missing cursor with `has_more`
also reports truncation.
- Streams are capped at 200000 events and by the foundation byte caps.
- Waiting (`ResponsesService::wait`, `VideosService::wait`) has a 1 s floor
between polls, a 60 s ceiling per poll interval, 600 s total, 600 polls.
- Background responses require `store=true`. `previous_response_id` and
`conversation` are mutually exclusive. Strict structured outputs are
checked locally (additionalProperties false, all properties required,
depth 10, 5000 properties).
- `OpenaiChatModel`: system text becomes `instructions` (Responses) or a
system/developer message (Chat). `max_tokens` becomes `max_output_tokens`
(min 16) or `max_completion_tokens`. Chat streaming sets
`stream_options.include_usage`. Refusals surface as text. A failed response
or `error` event throws `ApiError`. The Responses API has no stop
sequences: a request with `stop` is refused in that dialect.
## 4. Static caps
See `caps.h`. Notable: input 10485760 chars (spec), 128 tools, 16 metadata
pairs (64/512 chars), 4 stop sequences, n 1..128, 2048 embedding inputs,
10 images per call, 16 edit images (50 MiB each, 4 MiB for variations),
4096 speech chars, 25 MiB audio uploads, 10 MiB voice samples, 32000 video
prompt chars, 100 per page.
## Sources
- OpenAPI document: github.com/openai/openai-openapi `openapi.json`, pinned
at commit 8243cc8fa6f04430fb1f1a6a2cb926fbeb4f7f26 (2026-10-24), OpenAPI
3.0, `info.version` 2.3.0. The branch `manual_spec` is stale (April 2025)
and was not used. The official SDKs' `.stats.yml` (openai-python
`.castiron.stats.yml`) names the same Stainless generation; the
Stainless-hosted copy returned 404 and was not used.
- Cross-check of paths, headers and error shapes: platform.openai.com
reference as mirrored by the spec's `x-oaiMeta` examples (used as test
fixtures), and the unauthenticated probes below.
- The spec was read for: paths and operation ids, request properties,
query parameters, response media types, the Tool, ResponseStreamEvent,
InputItem, OutputItem, IncludeEnum and ReasoningEffort schemas, enums for
voices, video sizes and seconds, image options.
## Verified against spec
`python3 -I src/openai/core/tools/check_spec.py` (2026-10-24): `spec ok: 44
calls, 371 spec operations seen, 18 tool types, 59 stream events, 8 include
values, openapi 2.3.0 (8243cc8fa6)`. It fails on a wrong method, path,
operation id, request schema, content type, response kind, request property
list or query list, on any spec operation that is neither in the table nor in
`EXCLUDED`, and when the tool types, stream events or include values in the
C++ sources differ from the spec.
The `?beta=true` variants (`beta_createResponse` etc., header
`OpenAI-Beta: responses_multi_agent=v1`) use the same paths; the table holds
the plain rows, `ResponseRequest::betas` sends the header. Their `Beta*`
schemas are not modeled.
## Verified live
No OpenAI key exists here. Unauthenticated probes, 2026-10-24:
- `api.openai.com` resolves, TLS works (HTTP/2, Cloudflare).
- `GET /v1/models`, `/v1/responses`, `/v1/chat/completions`, `/v1/completions`,
`/v1/embeddings`, `/v1/audio/speech` answer 401 with
`{"error":{"message":"Missing bearer authentication in header","type":"invalid_request_error","param":null,"code":null}}`
and `www-authenticate: Bearer realm="OpenAI API"`.
- `GET /v1/images/generations` answers 405 (POST-only route exists).
- `GET /v1/videos` and `GET /v1/moderations` answered 404 without a key
(a documented route; not understood, probably unauthenticated routing).
Unexplained, not a conformance claim.
- The opt-in test `openai_live.unauthenticated_probe_is_401_with_error_body`
(MOLO_LIVE_NET=1) sent a bogus key to `GET /models` and got 401, error
type `invalid_request_error`; `ApiError` mapped it correctly.
- Key-gated live tests (`MOLO_LIVE_NET=1` plus `OPENAI_API_KEY`; the one
completion also `MOLO_LIVE_PAID=1`) are written and SKIP loudly here.
## Unverified
Everything that needs a key: success response shapes beyond the documented
examples used as fixtures, real streaming transcripts (only the documented
chat and Responses streams were decoded), rate-limit and 429 behavior,
background polling, multipart field spelling for edits (`image` for one file,
`image[]` for several), `type=audio_sample` sent with voice-from-sample
multipart, the `include[]` and `metadata[key]` query spelling (taken from the
official SDKs; the client percent-encodes the brackets), the 25 MiB audio and
50 MiB image caps (platform guides, not the spec), the `gpt-4.1-nano` model
named in the paid live test, `tts-1` SSE exclusion, `dall-e-3` n=1 and
prompt-length rules (guides). Usage of base64 embeddings assumes
little-endian float32.
## Gaps
- `GET /responses/{id}?stream=true&starting_after=N` (stream replay) cannot go
through `llm::ApiClient`, which streams POST only. `retrieve` never streams.
- Streaming transcription (`stream=true`, multipart SSE) and streaming
multipart image edits are not offered for the same reason; JSON image edits
stream.
- Raw multipart (`transcribe`, `translate`) is bound by the 32 MiB raw body
cap of the foundation; JSON-answer multipart calls use the 512 MiB path.
- Response bodies over 64 MiB (long videos) exceed the foundation response
cap; `VideosService::content` fails loudly there.
- `Beta*` schema variants are not modeled (see above).
- Query arrays could not be verified against a live server.
- Responses output items are exposed as raw JSON (`ChatResponse::raw`) except
messages, function and custom calls and reasoning summaries; hosted tool
call items (web search, file search, code interpreter, image generation,
MCP, computer, shell) are not typed.
+91
View File
@@ -0,0 +1,91 @@
#include "openai/core/body.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
void Body::guard_key(const std::string& key) const {
if (!accepts_field(call_, key))
Check::fail("field '" + key + "' is not a request property of " + spec_of(call_).operation_id);
}
Body& Body::put(const std::string& key, Json value) {
guard_key(key);
obj_.set(key, std::move(value));
return *this;
}
Body& Body::put_text(const std::string& key, const std::string& value) {
if (!value.empty()) put(key, Json(value));
return *this;
}
Body& Body::put_int(const std::string& key, const std::optional<int>& value) {
if (value) put(key, Json(*value));
return *this;
}
Body& Body::put_num(const std::string& key, const std::optional<double>& value) {
if (value) put(key, Json(*value));
return *this;
}
Body& Body::put_bool(const std::string& key, const std::optional<bool>& value) {
if (value) put(key, Json(*value));
return *this;
}
Body& Body::put_strings(const std::string& key, const std::vector<std::string>& values) {
if (values.empty()) return *this;
Json a = Json::array();
for (const std::string& v : values) a.push(Json(v));
return put(key, std::move(a));
}
Body& Body::put_map(const std::string& key, const std::map<std::string, std::string>& values) {
if (values.empty()) return *this;
Check::metadata(values);
Json o = Json::object();
for (const auto& [k, v] : values) o.set(k, Json(v));
return put(key, std::move(o));
}
Body& Body::merge_extra(const Json& extra) {
if (extra.is_null()) return *this;
if (!extra.is_object()) Check::fail("extra must be a JSON object");
for (const auto& [k, v] : extra.as_object()) {
guard_key(k);
if (obj_.has(k)) Check::fail("extra field '" + k + "' is already set by a typed member");
obj_.set(k, v);
}
return *this;
}
std::string Doc::str(const Json& j, const std::string& key, const std::string& fallback) {
const Json* v = j.is_object() ? j.find(key) : nullptr;
return v && v->is_string() ? v->as_string() : fallback;
}
long Doc::num(const Json& j, const std::string& key, long fallback) {
const Json* v = j.is_object() ? j.find(key) : nullptr;
return v && v->is_number() ? v->as_int() : fallback;
}
bool Doc::flag(const Json& j, const std::string& key, bool fallback) {
const Json* v = j.is_object() ? j.find(key) : nullptr;
return v && v->is_bool() ? v->as_bool() : fallback;
}
const Json* Doc::obj(const Json& j, const std::string& key) {
const Json* v = j.is_object() ? j.find(key) : nullptr;
return v && v->is_object() ? v : nullptr;
}
const Json* Doc::arr(const Json& j, const std::string& key) {
const Json* v = j.is_object() ? j.find(key) : nullptr;
return v && v->is_array() ? v : nullptr;
}
} // namespace molo::openai::core
+53
View File
@@ -0,0 +1,53 @@
#pragma once
// Request body builder over a JSON object. Typed members are written with
// put*; anything else the spec lists for the call goes in through
// merge_extra, which rejects a key the call does not accept and a key a
// typed member already set (loud, never a silent override).
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "openai/core/calls.h"
namespace molo::openai::core {
class Body {
public:
explicit Body(Call call) : call_(call), obj_(clients::body::Json::object()) {}
Body& put(const std::string& key, clients::body::Json value);
Body& put_text(const std::string& key, const std::string& value); // skipped when empty
Body& put_int(const std::string& key, const std::optional<int>& value);
Body& put_num(const std::string& key, const std::optional<double>& value);
Body& put_bool(const std::string& key, const std::optional<bool>& value);
Body& put_strings(const std::string& key, const std::vector<std::string>& values);
Body& put_map(const std::string& key, const std::map<std::string, std::string>& values);
Body& merge_extra(const clients::body::Json& extra);
bool has(const std::string& key) const { return obj_.has(key); }
const clients::body::Json& json() const { return obj_; }
clients::body::Json build() const { return obj_; }
private:
void guard_key(const std::string& key) const;
Call call_;
clients::body::Json obj_;
};
// Read helpers shared by the response parsers.
class Doc {
public:
static std::string str(const clients::body::Json& j, const std::string& key,
const std::string& fallback = "");
static long num(const clients::body::Json& j, const std::string& key, long fallback = 0);
static bool flag(const clients::body::Json& j, const std::string& key, bool fallback = false);
static const clients::body::Json* obj(const clients::body::Json& j, const std::string& key);
static const clients::body::Json* arr(const clients::body::Json& j, const std::string& key);
};
} // namespace molo::openai::core
+115
View File
@@ -0,0 +1,115 @@
#include "openai/core/calls.h"
#include <sstream>
#include "llm/api_client.h"
#include "llm/error.h"
namespace molo::openai::core {
// CALLS-BEGIN (parsed by tools/check_spec.py; one row per line)
static const CallSpec kCalls[] = {
{Call::kSpeech, "POST", "/audio/speech", "createSpeech", "CreateSpeechRequest", "json", "binary+sse", "input,instructions,model,response_format,speed,stream_format,voice", ""},
{Call::kTranscription, "POST", "/audio/transcriptions", "createTranscription", "CreateTranscriptionRequest", "multipart", "json+sse+text", "chunking_strategy,file,include,keywords,known_speaker_names,known_speaker_references,language,languages,model,prompt,response_format,stream,temperature,timestamp_granularities", ""},
{Call::kTranslation, "POST", "/audio/translations", "createTranslation", "CreateTranslationRequest", "multipart", "json+text", "file,model,prompt,response_format,temperature", ""},
{Call::kVoiceConsentCreate, "POST", "/audio/voice_consents", "createVoiceConsent", "CreateVoiceConsentRequest", "multipart", "json", "language,name,recording", ""},
{Call::kVoiceConsentList, "GET", "/audio/voice_consents", "listVoiceConsents", "", "none", "json", "", "after,limit"},
{Call::kVoiceConsentGet, "GET", "/audio/voice_consents/{consent_id}", "getVoiceConsent", "", "none", "json", "", ""},
{Call::kVoiceConsentUpdate, "POST", "/audio/voice_consents/{consent_id}", "updateVoiceConsent", "UpdateVoiceConsentRequest", "json", "json", "name", ""},
{Call::kVoiceConsentDelete, "DELETE", "/audio/voice_consents/{consent_id}", "deleteVoiceConsent", "", "none", "json", "", ""},
{Call::kVoiceCreate, "POST", "/audio/voices", "createVoice", "CreateVoiceRequest", "json+multipart", "json", "audio_sample,consent,model,name,prompt,script_hint,type", ""},
{Call::kVoiceDelete, "DELETE", "/audio/voices/{voice_id}", "deleteVoice", "", "none", "json", "", ""},
{Call::kChatList, "GET", "/chat/completions", "listChatCompletions", "", "none", "json", "", "model,metadata,after,limit,order"},
{Call::kChatCreate, "POST", "/chat/completions", "createChatCompletion", "CreateChatCompletionRequest", "json", "json+sse", "audio,frequency_penalty,function_call,functions,logit_bias,logprobs,max_completion_tokens,max_tokens,messages,metadata,modalities,model,moderation,n,parallel_tool_calls,prediction,presence_penalty,prompt_cache_key,prompt_cache_options,prompt_cache_retention,reasoning_effort,response_format,safety_identifier,seed,service_tier,stop,store,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user,verbosity,web_search_options", ""},
{Call::kChatGet, "GET", "/chat/completions/{completion_id}", "getChatCompletion", "", "none", "json", "", ""},
{Call::kChatUpdate, "POST", "/chat/completions/{completion_id}", "updateChatCompletion", "inline", "json", "json", "metadata", ""},
{Call::kChatDelete, "DELETE", "/chat/completions/{completion_id}", "deleteChatCompletion", "", "none", "json", "", ""},
{Call::kChatMessages, "GET", "/chat/completions/{completion_id}/messages", "getChatCompletionMessages", "", "none", "json", "", "after,limit,order"},
{Call::kCompletionCreate, "POST", "/completions", "createCompletion", "CreateCompletionRequest", "json", "json+sse", "best_of,echo,frequency_penalty,logit_bias,logprobs,max_tokens,model,n,presence_penalty,prompt,seed,stop,stream,stream_options,suffix,temperature,top_p,user", ""},
{Call::kEmbeddingCreate, "POST", "/embeddings", "createEmbedding", "CreateEmbeddingRequest", "json", "json", "dimensions,encoding_format,input,model,user", ""},
{Call::kImageEdit, "POST", "/images/edits", "createImageEdit", "EditImageBodyJsonParam", "json+multipart", "json+sse", "background,image,images,input_fidelity,mask,model,moderation,n,output_compression,output_format,partial_images,prompt,quality,response_format,safety_identifier,size,stream,user", ""},
{Call::kImageGenerate, "POST", "/images/generations", "createImage", "CreateImageRequest", "json", "json+sse", "background,model,moderation,n,output_compression,output_format,partial_images,prompt,quality,response_format,safety_identifier,size,stream,style,user", ""},
{Call::kImageVariation, "POST", "/images/variations", "createImageVariation", "CreateImageVariationRequest", "multipart", "json", "image,model,n,response_format,size,user", ""},
{Call::kModelList, "GET", "/models", "listModels", "", "none", "json", "", ""},
{Call::kModelGet, "GET", "/models/{model}", "retrieveModel", "", "none", "json", "", ""},
{Call::kModelDelete, "DELETE", "/models/{model}", "deleteModel", "", "none", "json", "", ""},
{Call::kModerationCreate, "POST", "/moderations", "createModeration", "CreateModerationRequest", "json", "json", "input,model", ""},
{Call::kResponseCreate, "POST", "/responses", "createResponse", "CreateResponse", "json", "json+sse", "access_programs,background,context_management,conversation,include,input,instructions,max_output_tokens,max_tool_calls,metadata,model,moderation,parallel_tool_calls,previous_response_id,prompt,prompt_cache_key,prompt_cache_options,prompt_cache_retention,reasoning,safety_identifier,service_tier,store,stream,stream_options,temperature,text,tool_choice,tools,top_logprobs,top_p,truncation,user", ""},
{Call::kResponseGet, "GET", "/responses/{response_id}", "getResponse", "", "none", "json+sse", "", "include,stream,starting_after,include_obfuscation"},
{Call::kResponseDelete, "DELETE", "/responses/{response_id}", "deleteResponse", "", "none", "json", "", ""},
{Call::kResponseCancel, "POST", "/responses/{response_id}/cancel", "cancelResponse", "", "none", "json", "", ""},
{Call::kResponseInputItems, "GET", "/responses/{response_id}/input_items", "listInputItems", "", "none", "json", "", "limit,order,after,include"},
{Call::kProvenanceCheck, "POST", "/content_provenance_checks", "Createcontentprovenancecheck", "CreateContentProvenanceBody", "multipart", "json", "file", ""},
{Call::kVideoCreate, "POST", "/videos", "createVideo", "CreateVideoJsonBody", "json+multipart", "json", "input_reference,model,prompt,seconds,size", ""},
{Call::kVideoList, "GET", "/videos", "ListVideos", "", "none", "json", "", "limit,order,after"},
{Call::kVideoCharacterCreate, "POST", "/videos/characters", "CreateVideoCharacter", "CreateVideoCharacterBody", "multipart", "json", "name,video", ""},
{Call::kVideoCharacterGet, "GET", "/videos/characters/{character_id}", "GetVideoCharacter", "", "none", "json", "", ""},
{Call::kVideoEdit, "POST", "/videos/edits", "CreateVideoEdit", "CreateVideoEditJsonBody", "json+multipart", "json", "prompt,video", ""},
{Call::kVideoExtend, "POST", "/videos/extensions", "CreateVideoExtend", "CreateVideoExtendJsonBody", "json+multipart", "json", "prompt,seconds,video", ""},
{Call::kVideoGet, "GET", "/videos/{video_id}", "GetVideo", "", "none", "json", "", ""},
{Call::kVideoDelete, "DELETE", "/videos/{video_id}", "DeleteVideo", "", "none", "json", "", ""},
{Call::kVideoContent, "GET", "/videos/{video_id}/content", "RetrieveVideoContent", "", "none", "binary+json", "", "variant"},
{Call::kVideoRemix, "POST", "/videos/{video_id}/remix", "CreateVideoRemix", "CreateVideoRemixBody", "json+multipart", "json", "prompt", ""},
{Call::kResponseInputTokens, "POST", "/responses/input_tokens", "Getinputtokencounts", "TokenCountsBody", "json", "json", "conversation,input,instructions,model,parallel_tool_calls,personality,previous_response_id,reasoning,text,tool_choice,tools,truncation", ""},
{Call::kResponseCompact, "POST", "/responses/compact", "Compactconversation", "CompactResponseMethodPublicBody", "json", "json", "input,instructions,model,previous_response_id,prompt_cache_key,prompt_cache_options,prompt_cache_retention,service_tier", ""},
{Call::kDecisionCreate, "POST", "/decisions", "createDecision", "DecisionRequest", "json", "json", "input,model,prompt_cache_key,prompt_cache_options,questions,safety_identifier", ""},
};
// CALLS-END
namespace {
bool listed(const char* csv, const std::string& item) {
std::string s = std::string(",") + csv + ",";
return s.find("," + item + ",") != std::string::npos;
}
} // namespace
std::size_t call_count() { return sizeof(kCalls) / sizeof(kCalls[0]); }
const CallSpec& spec_at(std::size_t index) { return kCalls[index]; }
const CallSpec& spec_of(Call call) {
for (const CallSpec& c : kCalls)
if (c.call == call) return c;
throw llm::ApiError::invalid("openai", "call is not in the table");
}
clients::Method method_of(Call call) {
std::string m = spec_of(call).http_method;
if (m == "GET") return clients::Method::kGet;
if (m == "POST") return clients::Method::kPost;
if (m == "DELETE") return clients::Method::kDelete;
throw llm::ApiError::invalid("openai", "unsupported HTTP method in the table");
}
bool accepts_field(Call call, const std::string& field) {
return listed(spec_of(call).body_fields, field);
}
bool accepts_query(Call call, const std::string& name) {
return listed(spec_of(call).query_params, name);
}
bool answers(Call call, const std::string& kind) {
std::string s = std::string("+") + spec_of(call).response + "+";
return s.find("+" + kind + "+") != std::string::npos;
}
std::string render(Call call, const std::vector<std::string>& ids) {
const std::string path = spec_of(call).path;
std::string out;
std::size_t next = 0;
for (std::size_t i = 0; i < path.size(); ++i) {
if (path[i] != '{') {
out += path[i];
continue;
}
std::size_t close = path.find('}', i);
if (close == std::string::npos || next >= ids.size())
throw llm::ApiError::invalid("openai", "path needs more ids than given");
if (!llm::Service::valid_id(ids[next]))
throw llm::ApiError::invalid("openai", "bad " + path.substr(i + 1, close - i - 1));
out += ids[next++];
i = close;
}
if (next != ids.size()) throw llm::ApiError::invalid("openai", "more ids than the path takes");
return out;
}
} // namespace molo::openai::core
+54
View File
@@ -0,0 +1,54 @@
#pragma once
// The ONE table of REST calls this module makes. Every row is checked
// against the OpenAI OpenAPI document by tools/check_spec.py. Code builds
// every path from this table, never from a literal.
#include <cstddef>
#include <string>
#include <vector>
#include "clients/request.h"
namespace molo::openai::core {
enum class Call {
kSpeech, kTranscription, kTranslation, kVoiceConsentCreate, kVoiceConsentList,
kVoiceConsentGet, kVoiceConsentUpdate, kVoiceConsentDelete, kVoiceCreate, kVoiceDelete,
kChatList, kChatCreate, kChatGet, kChatUpdate, kChatDelete, kChatMessages, kCompletionCreate,
kEmbeddingCreate, kImageEdit, kImageGenerate, kImageVariation, kModelList, kModelGet,
kModelDelete, kModerationCreate, kResponseCreate, kResponseGet, kResponseDelete,
kResponseCancel, kResponseInputItems, kProvenanceCheck, kVideoCreate, kVideoList,
kVideoCharacterCreate, kVideoCharacterGet, kVideoEdit, kVideoExtend, kVideoGet, kVideoDelete,
kVideoContent, kVideoRemix, kResponseInputTokens, kResponseCompact, kDecisionCreate,
};
struct CallSpec {
Call call;
const char* http_method; // GET POST DELETE
const char* path; // spec path under /v1, e.g. "/responses/{response_id}"
const char* operation_id; // spec operationId
const char* request_schema; // spec request body schema, "" when none
const char* content; // "json", "multipart", "json+multipart" or "none"
const char* response; // kinds the spec lists: json, sse, text, binary
const char* body_fields; // every top-level request property of the spec
const char* query_params; // query parameters of the spec
};
const CallSpec& spec_of(Call call);
const CallSpec& spec_at(std::size_t index);
std::size_t call_count();
clients::Method method_of(Call call);
// True when the spec lists `field` as a request property / `name` as a query
// parameter of the call.
bool accepts_field(Call call, const std::string& field);
bool accepts_query(Call call, const std::string& name);
// True when the spec answers this call with the given response kind.
bool answers(Call call, const std::string& kind);
// Fills the {placeholders} of the path in order. Every id is validated as a
// single URL path segment first (llm::Service::valid_id), else ApiError.
std::string render(Call call, const std::vector<std::string>& ids = {});
} // namespace molo::openai::core
+65
View File
@@ -0,0 +1,65 @@
#pragma once
// Static ceilings for the OpenAI core module. Never autotuned. Numbers come
// from the OpenAPI document (maxLength / maximum fields) or the platform
// guides; each is named for the knob it limits.
#include <cstddef>
namespace molo::openai::core {
struct CoreCaps {
// Text. Responses input strings are capped at 10485760 chars by the spec.
static constexpr std::size_t kMaxInputChars = 10485760;
static constexpr std::size_t kMaxInstructionChars = 1048576;
static constexpr std::size_t kMaxInputItems = 4096;
static constexpr std::size_t kMaxTools = 128;
static constexpr std::size_t kMaxIncludes = 16;
static constexpr std::size_t kMaxMetadataPairs = 16;
static constexpr std::size_t kMaxMetadataKeyChars = 64;
static constexpr std::size_t kMaxMetadataValueChars = 512;
static constexpr std::size_t kMaxStopSequences = 4;
static constexpr std::size_t kMaxLogitBias = 300;
static constexpr std::size_t kMaxSafetyIdChars = 128;
static constexpr std::size_t kMaxPromptCacheKeyChars = 64;
static constexpr int kMinOutputTokens = 16; // Responses max_output_tokens floor
static constexpr int kMaxTopLogprobs = 20;
static constexpr int kMaxChoices = 128; // chat / completions n
static constexpr int kMaxLegacyLogprobs = 5;
// Lists.
static constexpr int kMaxPageLimit = 100;
static constexpr std::size_t kMaxListPages = 20;
// Streams.
static constexpr std::size_t kMaxStreamEvents = 200000;
// Embeddings.
static constexpr std::size_t kMaxEmbeddingInputs = 2048;
static constexpr std::size_t kMaxEmbeddingDimensions = 16384;
// Moderation.
static constexpr std::size_t kMaxModerationInputs = 128;
// Images.
static constexpr int kMaxImagesN = 10;
static constexpr int kMaxPartialImages = 3;
static constexpr std::size_t kMaxImagePromptChars = 32000;
static constexpr std::size_t kMaxEditImages = 16;
static constexpr std::size_t kMaxImageBytes = 50u * 1024u * 1024u;
static constexpr std::size_t kMaxVariationBytes = 4u * 1024u * 1024u;
// Audio.
static constexpr std::size_t kMaxSpeechChars = 4096;
static constexpr std::size_t kMaxAudioFileBytes = 25u * 1024u * 1024u;
static constexpr std::size_t kMaxSpeechBytes = 64u * 1024u * 1024u;
static constexpr std::size_t kMaxTranscriptionLists = 64;
static constexpr std::size_t kMaxVoiceNameChars = 256;
static constexpr std::size_t kMaxVoicePromptChars = 2000;
// Video.
static constexpr std::size_t kMaxVideoPromptChars = 32000;
static constexpr std::size_t kMaxVideoReferenceBytes = 50u * 1024u * 1024u;
static constexpr std::size_t kMaxCharacterNameChars = 80;
// Waiting on background work (responses, videos). The floor is the
// shortest sleep between polls, the ceiling bounds one wait call.
static constexpr int kMinPollSeconds = 1;
static constexpr int kMaxPollSeconds = 60;
static constexpr int kMaxWaitSeconds = 600;
static constexpr int kMaxPolls = 600;
};
} // namespace molo::openai::core
+84
View File
@@ -0,0 +1,84 @@
#include "openai/core/chat_completions.h"
#include "openai/core/body.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
Json ChatCompletionsService::create(const ChatCompletionRequest& request) {
Json body = request.to_json(false);
return json_call(Call::kChatCreate, {}, &body);
}
void ChatCompletionsService::stream(const ChatCompletionRequest& request,
const ChunkHandler& handler) {
Json body = request.to_json(true);
stream_call(Call::kChatCreate, {}, body,
[&](const llm::SseEvent& ev) { return handler(ev.data); });
}
Json ChatCompletionsService::retrieve(const std::string& id) {
Check::id("completion", id);
return json_call(Call::kChatGet, {id});
}
static Query list_query(const StoredListParams& p) {
p.page.validate();
Check::max_chars("model", p.model, 256);
Query q(Call::kChatList);
q.add("model", p.model);
q.add_map("metadata", p.metadata);
q.add_int("limit", p.page.limit);
q.add("order", p.page.order);
q.add("after", p.page.after);
return q;
}
Json ChatCompletionsService::list(const StoredListParams& params) {
Query q = list_query(params);
return json_call(Call::kChatList, {}, nullptr, &q);
}
Listing ChatCompletionsService::list_all(const StoredListParams& params, std::size_t max_pages) {
Query base(Call::kChatList);
Check::max_chars("model", params.model, 256);
base.add("model", params.model);
base.add_map("metadata", params.metadata);
return walk(Call::kChatList, {}, params.page, base, max_pages);
}
Json ChatCompletionsService::update(const std::string& id,
const std::map<std::string, std::string>& metadata) {
Check::id("completion", id);
Check::metadata(metadata);
Json m = Json::object();
for (const auto& [k, v] : metadata) m.set(k, Json(v));
Json body = Json::object();
body.set("metadata", m);
return json_call(Call::kChatUpdate, {id}, &body);
}
Json ChatCompletionsService::remove(const std::string& id) {
Check::id("completion", id);
return json_call(Call::kChatDelete, {id});
}
Json ChatCompletionsService::messages(const std::string& id, const ListParams& params) {
Check::id("completion", id);
params.validate();
Query q(Call::kChatMessages);
q.add_int("limit", params.limit);
q.add("order", params.order);
q.add("after", params.after);
return json_call(Call::kChatMessages, {id}, nullptr, &q);
}
Listing ChatCompletionsService::messages_all(const std::string& id, const ListParams& params,
std::size_t max_pages) {
Check::id("completion", id);
return walk(Call::kChatMessages, {id}, params, Query(Call::kChatMessages), max_pages);
}
} // namespace molo::openai::core
+45
View File
@@ -0,0 +1,45 @@
#pragma once
// Chat Completions: create (plain and streaming), and the stored completion
// calls (retrieve, list, update metadata, delete, messages). Stored calls
// only see completions created with store=true.
#include <functional>
#include <map>
#include <string>
#include "openai/core/chat_request.h"
#include "openai/core/service.h"
namespace molo::openai::core {
struct StoredListParams {
ListParams page;
std::string model; // filter
std::map<std::string, std::string> metadata; // filter, name[key]=value
};
class ChatCompletionsService : public OpenaiService {
public:
using ChunkHandler = std::function<bool(const clients::body::Json& chunk)>; // false: stop
using OpenaiService::OpenaiService;
clients::body::Json create(const ChatCompletionRequest& request);
// Each `chat.completion.chunk` document reaches the handler; [DONE] ends.
void stream(const ChatCompletionRequest& request, const ChunkHandler& handler);
clients::body::Json retrieve(const std::string& completion_id);
clients::body::Json list(const StoredListParams& params = {});
Listing list_all(const StoredListParams& params = {}, std::size_t max_pages = 5);
// Replaces the metadata of a stored completion (an empty map clears it).
clients::body::Json update(const std::string& completion_id,
const std::map<std::string, std::string>& metadata);
// DELETE: only completions created with store=true can be deleted.
clients::body::Json remove(const std::string& completion_id);
clients::body::Json messages(const std::string& completion_id, const ListParams& params = {});
Listing messages_all(const std::string& completion_id, const ListParams& params = {},
std::size_t max_pages = 5);
};
} // namespace molo::openai::core
+55
View File
@@ -0,0 +1,55 @@
#include "openai/core/chat_model.h"
#include "openai/core/check.h"
#include "openai/core/client.h"
namespace molo::openai::core {
OpenaiChatModel::OpenaiChatModel(std::shared_ptr<llm::ApiClient> client, ChatModelConfig config)
: client_(client), config_(std::move(config)), responses_(client), chat_(client) {}
std::shared_ptr<OpenaiChatModel> OpenaiChatModel::from_env(ChatModelConfig config) {
return std::make_shared<OpenaiChatModel>(OpenaiClient::from_env(), std::move(config));
}
void OpenaiChatModel::check(const llm::ChatRequest& request) const {
Check::non_empty("model", request.model);
Check::require(!request.messages.empty(), "messages is required");
}
llm::ChatResponse OpenaiChatModel::generate(const llm::ChatRequest& request) {
check(request);
if (config_.dialect == Dialect::kResponses) {
ResponseRequest r = ResponsesWire::request(request, config_.wire);
return ResponsesWire::response(responses_.create(r));
}
ChatCompletionRequest r = ChatWire::request(request, config_.wire);
return ChatWire::response(chat_.create(r));
}
void OpenaiChatModel::stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
check(request);
if (config_.dialect == Dialect::kResponses) {
ResponseRequest r = ResponsesWire::request(request, config_.wire);
ResponsesChunkDecoder decoder;
responses_.stream(r, [&](const ResponseEvent& e) {
llm::ChatChunk chunk;
if (!decoder.decode(e, chunk)) return true;
return on_chunk(chunk);
});
return;
}
ChatCompletionRequest r = ChatWire::request(request, config_.wire);
r.include_usage = true;
ChatChunkDecoder decoder;
chat_.stream(r, [&](const clients::body::Json& doc) {
std::vector<llm::ChatChunk> chunks;
decoder.decode(doc, chunks);
for (const llm::ChatChunk& c : chunks)
if (!on_chunk(c)) return false;
return true;
});
}
} // namespace molo::openai::core
+47
View File
@@ -0,0 +1,47 @@
#pragma once
// OpenaiChatModel: llm::ChatModel over the OpenAI API in either dialect,
// the Responses API (default) or Chat Completions.
#include <functional>
#include <memory>
#include <string>
#include "llm/chat_types.h"
#include "openai/core/chat_completions.h"
#include "openai/core/responses.h"
#include "openai/core/wire/chat_wire.h"
namespace molo::openai::core {
enum class Dialect { kResponses, kChatCompletions };
struct ChatModelConfig {
Dialect dialect = Dialect::kResponses;
WireOptions wire;
};
class OpenaiChatModel : public llm::ChatModel {
public:
explicit OpenaiChatModel(std::shared_ptr<llm::ApiClient> client, ChatModelConfig config = {});
// OPENAI_API_KEY from llm::Config; not configured without it.
static std::shared_ptr<OpenaiChatModel> from_env(ChatModelConfig config = {});
std::string vendor() const override { return "openai"; }
llm::ChatResponse generate(const llm::ChatRequest& request) override;
void stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
Dialect dialect() const { return config_.dialect; }
bool configured() const { return client_->configured(); }
private:
void check(const llm::ChatRequest& request) const;
std::shared_ptr<llm::ApiClient> client_;
ChatModelConfig config_;
ResponsesService responses_;
ChatCompletionsService chat_;
};
} // namespace molo::openai::core
+337
View File
@@ -0,0 +1,337 @@
#include "openai/core/chat_request.h"
#include "llm/caps.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/wire/input_items.h"
#include "openai/core/wire/neutral_map.h"
#include "openai/core/wire/tool_schemas.h"
#include "security/crypto/Base64.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
Json obj(const char* k, Json v) {
Json o = Json::object();
o.set(k, std::move(v));
return o;
}
Json role_msg(const char* role, const std::string& text) {
Check::max_chars("message", text, CoreCaps::kMaxInputChars);
Json o = Json::object();
o.set("role", Json(role));
o.set("content", Json(text));
return o;
}
Json array_of(const std::vector<Json>& v) {
Json a = Json::array();
for (const Json& j : v) a.push(j);
return a;
}
Json stop_value(const std::vector<std::string>& stop) {
Json a = Json::array();
for (const std::string& s : stop) {
Check::non_empty("stop sequence", s);
a.push(Json(s));
}
return a;
}
Json bias_value(const std::map<std::string, int>& bias) {
Check::max_count("logit_bias", bias.size(), CoreCaps::kMaxLogitBias);
Json o = Json::object();
for (const auto& [k, v] : bias) {
Check::non_empty("logit_bias token", k);
Check::range("logit_bias value", v, -100, 100);
o.set(k, Json(v));
}
return o;
}
} // namespace
Json ChatMessages::system(const std::string& text) { return role_msg("system", text); }
Json ChatMessages::developer(const std::string& text) { return role_msg("developer", text); }
Json ChatMessages::user(const std::string& text) { return role_msg("user", text); }
Json ChatMessages::assistant(const std::string& text) { return role_msg("assistant", text); }
Json ChatMessages::user(const std::vector<Json>& parts) {
Check::require(!parts.empty(), "user message needs content");
Json o = Json::object();
o.set("role", Json("user"));
o.set("content", array_of(parts));
return o;
}
Json ChatMessages::assistant_with_calls(const std::string& text, const std::vector<Json>& calls) {
Check::require(!calls.empty() || !text.empty(), "assistant message needs text or tool calls");
Json o = Json::object();
o.set("role", Json("assistant"));
o.set("content", text.empty() ? Json() : Json(text));
if (!calls.empty()) o.set("tool_calls", array_of(calls));
return o;
}
Json ChatMessages::tool(const std::string& call_id, const std::string& content) {
Check::non_empty("tool_call_id", call_id);
Check::max_chars("tool content", content, CoreCaps::kMaxInputChars);
Json o = Json::object();
o.set("role", Json("tool"));
o.set("tool_call_id", Json(call_id));
o.set("content", Json(content));
return o;
}
Json ChatMessages::tool_call(const std::string& id, const std::string& name,
const std::string& arguments_json) {
Check::non_empty("tool call id", id);
ToolSchemas::check_function_name(name);
Json f = Json::object();
f.set("name", Json(name));
f.set("arguments", Json(arguments_json));
Json o = Json::object();
o.set("id", Json(id));
o.set("type", Json("function"));
o.set("function", f);
return o;
}
Json ChatMessages::part_text(const std::string& text) {
Json o = Json::object();
o.set("type", Json("text"));
o.set("text", Json(text));
return o;
}
Json ChatMessages::part_image_url(const std::string& url, const std::string& detail) {
Check::non_empty("image url", url);
Check::in_set("image detail", detail, {"auto", "low", "high"});
Json img = Json::object();
img.set("url", Json(url));
img.set("detail", Json(detail));
Json o = Json::object();
o.set("type", Json("image_url"));
o.set("image_url", img);
return o;
}
Json ChatMessages::part_image(const llm::ImageData& image, const std::string& detail) {
return part_image_url(InputItems::data_uri(image), detail);
}
Json ChatMessages::part_audio(const std::string& base64_data, const std::string& format) {
Check::non_empty("audio data", base64_data);
Check::in_set("audio format", format, {"wav", "mp3"});
Json a = Json::object();
a.set("data", Json(base64_data));
a.set("format", Json(format));
Json o = Json::object();
o.set("type", Json("input_audio"));
o.set("input_audio", a);
return o;
}
Json ChatMessages::part_file_id(const std::string& file_id) {
Check::id("file", file_id);
Json o = Json::object();
o.set("type", Json("file"));
o.set("file", obj("file_id", Json(file_id)));
return o;
}
Json ChatMessages::part_file_data(const std::string& filename, const std::string& bytes,
const std::string& mime) {
Check::non_empty("filename", filename);
Check::require(!bytes.empty() && bytes.size() <= security::crypto::Base64::kMaxInputBytes,
"inline file must be 1 byte to 16 MiB");
Json f = Json::object();
f.set("filename", Json(filename));
f.set("file_data", Json("data:" + mime + ";base64," + security::crypto::Base64::encode(bytes)));
Json o = Json::object();
o.set("type", Json("file"));
o.set("file", f);
return o;
}
std::vector<Json> ChatMessages::from_neutral(const llm::Message& m, bool developer_role) {
std::vector<Json> out;
std::vector<Json> parts;
std::vector<Json> calls;
std::string text;
bool has_image = false;
auto flush = [&]() {
if (m.role == llm::Role::kAssistant) {
if (!text.empty() || !calls.empty()) out.push_back(assistant_with_calls(text, calls));
} else if (!parts.empty()) {
Json o = Json::object();
const char* role = m.role == llm::Role::kSystem ? (developer_role ? "developer" : "system") : "user";
o.set("role", Json(role));
if (!has_image) {
std::string joined;
for (const Json& p : parts) joined += p.find("text")->as_string();
o.set("content", Json(joined));
} else {
o.set("content", array_of(parts));
}
if (!m.name.empty()) o.set("name", Json(m.name));
out.push_back(o);
}
parts.clear();
calls.clear();
text.clear();
has_image = false;
};
for (const llm::ContentPart& p : m.parts) {
if (!p.text.empty()) {
if (m.role == llm::Role::kAssistant) text += p.text;
else parts.push_back(part_text(p.text));
}
if (p.image) {
Check::require(m.role == llm::Role::kUser, "only user messages can carry images");
parts.push_back(part_image(*p.image));
has_image = true;
}
if (p.tool_call) {
Check::require(m.role == llm::Role::kAssistant, "only assistant messages carry tool calls");
calls.push_back(tool_call(p.tool_call->id, p.tool_call->name, p.tool_call->arguments.dump()));
}
if (p.tool_result) {
flush();
out.push_back(tool(p.tool_result->call_id, p.tool_result->content));
}
}
flush();
return out;
}
void ChatCompletionRequest::validate(bool stream) const {
Check::non_empty("model", model);
Check::max_chars("model", model, 256);
Check::require(!messages.empty(), "messages is required");
Check::max_count("messages", messages.size(), llm::Caps::kMaxMessages);
for (const Json& m : messages) Check::require(m.is_object(), "every message must be a JSON object");
if (!tools.empty()) ToolSchemas::validate_chat_tools(tools);
if (!tool_choice.is_null()) {
Check::require(!tools.empty(), "tool_choice needs tools");
Check::require(tool_choice.is_string() || tool_choice.is_object(), "tool_choice must be a string or object");
if (tool_choice.is_string()) Check::in_set("tool_choice", tool_choice.as_string(), {"none", "auto", "required"});
}
Check::require(!parallel_tool_calls || !tools.empty(), "parallel_tool_calls needs tools");
if (temperature) Check::range("temperature", *temperature, 0, 2);
if (top_p) Check::range("top_p", *top_p, 0, 1);
if (n) Check::range("n", *n, 1, CoreCaps::kMaxChoices);
Check::max_count("stop", stop.size(), CoreCaps::kMaxStopSequences);
if (max_completion_tokens) Check::require(*max_completion_tokens >= 1, "max_completion_tokens must be positive");
if (frequency_penalty) Check::range("frequency_penalty", *frequency_penalty, -2, 2);
if (presence_penalty) Check::range("presence_penalty", *presence_penalty, -2, 2);
if (top_logprobs) {
Check::range("top_logprobs", *top_logprobs, 0, CoreCaps::kMaxTopLogprobs);
Check::require(logprobs && *logprobs, "top_logprobs needs logprobs=true");
}
if (response_format) response_format->validate();
Check::in_set_or_empty("reasoning_effort", reasoning_effort, ReasoningConfig::efforts());
Check::in_set_or_empty("verbosity", verbosity, {"low", "medium", "high"});
Check::in_set_or_empty("prompt_cache_retention", prompt_cache_retention, {"in_memory", "24h"});
Check::in_set_or_empty("service_tier", service_tier, {"auto", "default", "flex", "scale", "priority", "fast"});
for (const std::string& m : modalities) Check::in_set("modality", m, {"text", "audio"});
Check::require(audio.is_null() || Check::contains(modalities, "audio"), "audio options need modalities to include audio");
Check::metadata(metadata);
Check::max_chars("safety_identifier", safety_identifier, CoreCaps::kMaxSafetyIdChars);
Check::require(!include_usage || stream, "include_usage applies to streaming only");
}
Json ChatCompletionRequest::to_json(bool stream) const {
validate(stream);
Body b(Call::kChatCreate);
b.put_text("model", model);
b.put("messages", array_of(messages));
if (!tools.empty()) b.put("tools", array_of(tools));
if (!tool_choice.is_null()) b.put("tool_choice", tool_choice);
b.put_bool("parallel_tool_calls", parallel_tool_calls);
b.put_num("temperature", temperature);
b.put_num("top_p", top_p);
b.put_int("n", n);
if (stop.size() == 1) b.put("stop", Json(stop[0]));
else if (!stop.empty()) b.put("stop", stop_value(stop));
b.put_int("max_completion_tokens", max_completion_tokens);
b.put_num("frequency_penalty", frequency_penalty);
b.put_num("presence_penalty", presence_penalty);
if (!logit_bias.empty()) b.put("logit_bias", bias_value(logit_bias));
b.put_bool("logprobs", logprobs);
b.put_int("top_logprobs", top_logprobs);
b.put_int("seed", seed);
if (response_format) b.put("response_format", response_format->to_chat());
b.put_text("reasoning_effort", reasoning_effort);
b.put_text("verbosity", verbosity);
b.put_bool("store", store);
b.put_map("metadata", metadata);
b.put_text("user", user);
b.put_text("safety_identifier", safety_identifier);
b.put_text("prompt_cache_key", prompt_cache_key);
b.put_text("prompt_cache_retention", prompt_cache_retention);
b.put_text("service_tier", service_tier);
b.put_strings("modalities", modalities);
if (!audio.is_null()) b.put("audio", audio);
if (!prediction.is_null()) b.put("prediction", prediction);
if (!web_search_options.is_null()) b.put("web_search_options", web_search_options);
if (stream) {
b.put("stream", Json(true));
if (include_usage) b.put("stream_options", obj("include_usage", Json(*include_usage)));
}
b.merge_extra(extra);
return b.build();
}
void CompletionRequest::validate(bool stream) const {
Check::non_empty("model", model);
Check::require(prompt.empty() != prompts.empty(), "give prompt or prompts (exactly one)");
Check::max_chars("prompt", prompt, CoreCaps::kMaxInputChars);
Check::max_count("prompts", prompts.size(), CoreCaps::kMaxEmbeddingInputs);
if (best_of) Check::range("best_of", *best_of, 0, 20);
if (frequency_penalty) Check::range("frequency_penalty", *frequency_penalty, -2, 2);
if (presence_penalty) Check::range("presence_penalty", *presence_penalty, -2, 2);
if (logprobs) Check::range("logprobs", *logprobs, 0, CoreCaps::kMaxLegacyLogprobs);
if (max_tokens) Check::require(*max_tokens >= 0, "max_tokens must not be negative");
if (n) Check::range("n", *n, 1, CoreCaps::kMaxChoices);
if (temperature) Check::range("temperature", *temperature, 0, 2);
if (top_p) Check::range("top_p", *top_p, 0, 1);
Check::max_count("stop", stop.size(), CoreCaps::kMaxStopSequences);
Check::require(!include_usage || stream, "include_usage applies to streaming only");
Check::require(!best_of || !stream, "best_of cannot be combined with streaming");
}
Json CompletionRequest::to_json(bool stream) const {
validate(stream);
Body b(Call::kCompletionCreate);
b.put_text("model", model);
if (!prompt.empty()) b.put("prompt", Json(prompt));
else b.put_strings("prompt", prompts);
b.put_int("best_of", best_of);
b.put_bool("echo", echo);
b.put_num("frequency_penalty", frequency_penalty);
b.put_num("presence_penalty", presence_penalty);
if (!logit_bias.empty()) b.put("logit_bias", bias_value(logit_bias));
b.put_int("logprobs", logprobs);
b.put_int("max_tokens", max_tokens);
b.put_int("n", n);
b.put_int("seed", seed);
if (stop.size() == 1) b.put("stop", Json(stop[0]));
else if (!stop.empty()) b.put("stop", stop_value(stop));
b.put_text("suffix", suffix);
b.put_num("temperature", temperature);
b.put_num("top_p", top_p);
b.put_text("user", user);
if (stream) {
b.put("stream", Json(true));
if (include_usage) b.put("stream_options", obj("include_usage", Json(*include_usage)));
}
b.merge_extra(extra);
return b.build();
}
} // namespace molo::openai::core
+111
View File
@@ -0,0 +1,111 @@
#pragma once
// POST /chat/completions request, wire message builders, and legacy
// POST /completions request. Typed members for common fields, `extra` for
// any other spec property.
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
#include "openai/core/wire/response_config.h"
namespace molo::openai::core {
// Chat Completions wire messages and content parts.
class ChatMessages {
public:
static clients::body::Json system(const std::string& text);
static clients::body::Json developer(const std::string& text);
static clients::body::Json user(const std::string& text);
static clients::body::Json user(const std::vector<clients::body::Json>& parts);
static clients::body::Json assistant(const std::string& text);
static clients::body::Json assistant_with_calls(const std::string& text,
const std::vector<clients::body::Json>& calls);
static clients::body::Json tool(const std::string& call_id, const std::string& content);
static clients::body::Json tool_call(const std::string& id, const std::string& name,
const std::string& arguments_json);
// Content parts.
static clients::body::Json part_text(const std::string& text);
static clients::body::Json part_image_url(const std::string& url,
const std::string& detail = "auto");
static clients::body::Json part_image(const llm::ImageData& image,
const std::string& detail = "auto");
static clients::body::Json part_audio(const std::string& base64_data, const std::string& format);
static clients::body::Json part_file_id(const std::string& file_id);
static clients::body::Json part_file_data(const std::string& filename,
const std::string& bytes,
const std::string& mime = "application/pdf");
// Neutral message to wire messages (one tool message per tool result).
static std::vector<clients::body::Json> from_neutral(const llm::Message& message,
bool developer_role = false);
};
struct ChatCompletionRequest {
std::string model;
std::vector<clients::body::Json> messages; // wire messages (ChatMessages)
std::vector<clients::body::Json> tools; // function / custom (ToolSchemas::chat_*)
clients::body::Json tool_choice; // "auto" / "none" / "required" or object
std::optional<bool> parallel_tool_calls;
std::optional<double> temperature; // 0..2
std::optional<double> top_p; // 0..1
std::optional<int> n; // 1..128
std::vector<std::string> stop; // up to 4
std::optional<int> max_completion_tokens;
std::optional<double> frequency_penalty; // -2..2
std::optional<double> presence_penalty; // -2..2
std::map<std::string, int> logit_bias; // token id -> -100..100
std::optional<bool> logprobs;
std::optional<int> top_logprobs; // 0..20, needs logprobs
std::optional<int> seed;
std::optional<TextFormat> response_format;
std::string reasoning_effort; // ReasoningConfig::efforts()
std::string verbosity; // low, medium, high
std::optional<bool> store; // true: kept for the stored-completions calls
std::map<std::string, std::string> metadata;
std::string user;
std::string safety_identifier;
std::string prompt_cache_key;
std::string prompt_cache_retention;
std::string service_tier;
std::vector<std::string> modalities; // text, audio
clients::body::Json audio; // {voice, format} when modalities has audio
clients::body::Json prediction;
clients::body::Json web_search_options;
std::optional<bool> include_usage; // stream_options.include_usage (streaming)
clients::body::Json extra;
void validate(bool stream) const;
clients::body::Json to_json(bool stream) const;
};
// Legacy POST /completions.
struct CompletionRequest {
std::string model; // gpt-3.5-turbo-instruct, davinci-002, babbage-002, ...
std::string prompt; // one prompt, or
std::vector<std::string> prompts; // several
std::optional<int> best_of; // 0..20
std::optional<bool> echo;
std::optional<double> frequency_penalty;
std::optional<double> presence_penalty;
std::map<std::string, int> logit_bias;
std::optional<int> logprobs; // 0..5
std::optional<int> max_tokens;
std::optional<int> n;
std::optional<int> seed;
std::vector<std::string> stop;
std::string suffix;
std::optional<double> temperature;
std::optional<double> top_p;
std::string user;
std::optional<bool> include_usage;
clients::body::Json extra;
void validate(bool stream) const;
clients::body::Json to_json(bool stream) const;
};
} // namespace molo::openai::core
+70
View File
@@ -0,0 +1,70 @@
#include "openai/core/check.h"
#include <algorithm>
#include "llm/api_client.h"
#include "llm/error.h"
#include "openai/core/caps.h"
namespace molo::openai::core {
void Check::fail(const std::string& why) { throw llm::ApiError::invalid("openai", why); }
void Check::require(bool ok, const std::string& why) {
if (!ok) fail(why);
}
void Check::non_empty(const std::string& what, const std::string& value) {
if (value.empty()) fail(what + " is required");
}
void Check::max_chars(const std::string& what, const std::string& value, std::size_t cap) {
if (value.size() > cap) fail(what + " is over " + std::to_string(cap) + " characters");
}
void Check::max_count(const std::string& what, std::size_t count, std::size_t cap) {
if (count > cap) fail(what + " has more than " + std::to_string(cap) + " entries");
}
void Check::range(const std::string& what, double value, double lo, double hi) {
if (!(value >= lo && value <= hi))
fail(what + " must be between " + std::to_string(lo) + " and " + std::to_string(hi));
}
bool Check::contains(const std::vector<std::string>& set, const std::string& value) {
return std::find(set.begin(), set.end(), value) != set.end();
}
void Check::in_set(const std::string& what, const std::string& value,
const std::vector<std::string>& allowed) {
if (!contains(allowed, value)) {
std::string list;
for (const std::string& a : allowed) list += (list.empty() ? "" : ", ") + a;
fail(what + " must be one of: " + list);
}
}
void Check::in_set_or_empty(const std::string& what, const std::string& value,
const std::vector<std::string>& allowed) {
if (!value.empty()) in_set(what, value, allowed);
}
void Check::id(const std::string& what, const std::string& value) {
if (!llm::Service::valid_id(value)) fail("bad " + what + " id");
}
void Check::no_control(const std::string& what, const std::string& value) {
for (char c : value)
if (static_cast<unsigned char>(c) < 0x20 && c != '\n' && c != '\r' && c != '\t')
fail(what + " has control characters");
}
void Check::metadata(const std::map<std::string, std::string>& pairs) {
max_count("metadata", pairs.size(), CoreCaps::kMaxMetadataPairs);
for (const auto& [k, v] : pairs) {
if (k.empty() || k.size() > CoreCaps::kMaxMetadataKeyChars) fail("metadata key length");
if (v.size() > CoreCaps::kMaxMetadataValueChars) fail("metadata value length");
}
}
} // namespace molo::openai::core
+32
View File
@@ -0,0 +1,32 @@
#pragma once
// Request validation helpers. Every failure is a llm::ApiError::invalid
// ("openai", 400 INVALID_ARGUMENT) thrown before any network traffic.
#include <map>
#include <optional>
#include <string>
#include <vector>
namespace molo::openai::core {
class Check {
public:
[[noreturn]] static void fail(const std::string& why);
static void require(bool ok, const std::string& why);
static void non_empty(const std::string& what, const std::string& value);
static void max_chars(const std::string& what, const std::string& value, std::size_t cap);
static void max_count(const std::string& what, std::size_t count, std::size_t cap);
static void range(const std::string& what, double value, double lo, double hi);
static void in_set(const std::string& what, const std::string& value,
const std::vector<std::string>& allowed);
// `value` may be empty (not set); a set value must be listed.
static void in_set_or_empty(const std::string& what, const std::string& value,
const std::vector<std::string>& allowed);
static void id(const std::string& what, const std::string& value);
static void metadata(const std::map<std::string, std::string>& pairs);
static void no_control(const std::string& what, const std::string& value);
static bool contains(const std::vector<std::string>& set, const std::string& value);
};
} // namespace molo::openai::core
+41
View File
@@ -0,0 +1,41 @@
#include "openai/core/client.h"
#include "llm/config.h"
namespace molo::openai::core {
namespace {
void apply_scope(llm::ApiClientConfig& c, const std::string& org, const std::string& project) {
if (!org.empty()) c.default_headers["OpenAI-Organization"] = org;
if (!project.empty()) c.default_headers["OpenAI-Project"] = project;
}
llm::ApiClientConfig base() {
llm::ApiClientConfig c;
c.vendor = OpenaiClient::kVendor;
c.base_url = OpenaiClient::kBaseUrl;
c.key_env = OpenaiClient::kKeyEnv;
c.auth = llm::AuthStyle::kBearer;
return c;
}
} // namespace
llm::ApiClientConfig OpenaiClient::config() {
llm::ApiClientConfig c = base();
apply_scope(c, llm::Config::get(kOrgEnv), llm::Config::get(kProjectEnv));
return c;
}
std::shared_ptr<llm::ApiClient> OpenaiClient::from_env() {
return llm::ApiClient::from_env(config());
}
std::shared_ptr<llm::ApiClient> OpenaiClient::with_transport(
std::unique_ptr<clients::Transport> transport, const std::string& key,
const std::string& organization, const std::string& project) {
llm::ApiClientConfig c = base();
apply_scope(c, organization, project);
return std::make_shared<llm::ApiClient>(std::move(transport), std::move(c), key);
}
} // namespace molo::openai::core
+33
View File
@@ -0,0 +1,33 @@
#pragma once
// OpenaiClient: builds the llm::ApiClient for api.openai.com/v1. Credential
// OPENAI_API_KEY (bearer). Optional OPENAI_ORG_ID and OPENAI_PROJECT_ID turn
// into the OpenAI-Organization / OpenAI-Project headers.
#include <memory>
#include <string>
#include "llm/api_client.h"
namespace molo::openai::core {
class OpenaiClient {
public:
static constexpr const char* kVendor = "openai";
static constexpr const char* kBaseUrl = "https://api.openai.com/v1";
static constexpr const char* kKeyEnv = "OPENAI_API_KEY";
static constexpr const char* kOrgEnv = "OPENAI_ORG_ID";
static constexpr const char* kProjectEnv = "OPENAI_PROJECT_ID";
// The ApiClientConfig with optional organization / project headers filled
// from llm::Config (environment, then the .env.json files).
static llm::ApiClientConfig config();
// Real HTTPS transport, key from OPENAI_API_KEY (empty: not configured).
static std::shared_ptr<llm::ApiClient> from_env();
// Same config over any transport and key (tests, other seams).
static std::shared_ptr<llm::ApiClient> with_transport(
std::unique_ptr<clients::Transport> transport, const std::string& key,
const std::string& organization = "", const std::string& project = "");
};
} // namespace molo::openai::core
+18
View File
@@ -0,0 +1,18 @@
#include "openai/core/completions.h"
namespace molo::openai::core {
using clients::body::Json;
Json CompletionsService::create(const CompletionRequest& request) {
Json body = request.to_json(false);
return json_call(Call::kCompletionCreate, {}, &body);
}
void CompletionsService::stream(const CompletionRequest& request, const ChunkHandler& handler) {
Json body = request.to_json(true);
stream_call(Call::kCompletionCreate, {}, body,
[&](const llm::SseEvent& ev) { return handler(ev.data); });
}
} // namespace molo::openai::core
+23
View File
@@ -0,0 +1,23 @@
#pragma once
// Legacy POST /completions (gpt-3.5-turbo-instruct, davinci-002, babbage-002).
#include <functional>
#include "openai/core/chat_request.h"
#include "openai/core/service.h"
namespace molo::openai::core {
class CompletionsService : public OpenaiService {
public:
using ChunkHandler = std::function<bool(const clients::body::Json& chunk)>; // false: stop
using OpenaiService::OpenaiService;
clients::body::Json create(const CompletionRequest& request);
// Each streamed text_completion chunk reaches the handler; [DONE] ends.
void stream(const CompletionRequest& request, const ChunkHandler& handler);
};
} // namespace molo::openai::core
+117
View File
@@ -0,0 +1,117 @@
#include "openai/core/decisions.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/form_body.h"
namespace molo::openai::core {
using clients::body::Json;
DecisionQuestion DecisionQuestion::predicate(std::string name, std::string instructions) {
DecisionQuestion q;
q.name = std::move(name);
q.instructions = std::move(instructions);
return q;
}
DecisionQuestion DecisionQuestion::choice(std::string name, std::string instructions,
std::vector<std::string> choices) {
DecisionQuestion q = predicate(std::move(name), std::move(instructions));
q.kind = Kind::kChoice;
q.choices = std::move(choices);
return q;
}
DecisionQuestion DecisionQuestion::score(std::string name, std::string instructions,
std::vector<std::string> levels) {
DecisionQuestion q = predicate(std::move(name), std::move(instructions));
q.kind = Kind::kScore;
q.levels = std::move(levels);
return q;
}
Json DecisionQuestion::to_json() const {
Check::non_empty("question instructions", instructions);
Check::max_chars("question instructions", instructions, CoreCaps::kMaxInstructionChars);
Check::max_chars("question name", name, 1024);
Json o = Json::object();
o.set("instructions", Json(instructions));
if (!name.empty()) o.set("name", Json(name));
switch (kind) {
case Kind::kPredicate: o.set("type", Json("predicate")); break;
case Kind::kChoice: {
Check::require(choices.size() >= 2, "a choice question needs at least two choices");
o.set("type", Json("choice"));
Json a = Json::array();
for (const std::string& c : choices) {
Check::non_empty("choice", c);
Json item = Json::object();
item.set("value", Json(c));
a.push(item);
}
o.set("choices", a);
break;
}
case Kind::kScore: {
Check::require(levels.size() >= 2, "a score question needs at least two levels");
o.set("type", Json("score"));
Json a = Json::array();
for (const std::string& l : levels) {
Check::non_empty("level", l);
Json item = Json::object();
item.set("label", Json(l));
a.push(item);
}
o.set("levels", a);
break;
}
}
return o;
}
void DecisionRequest::validate() const {
Check::non_empty("model", model);
Check::non_empty("input", input_text);
Check::max_chars("input", input_text, CoreCaps::kMaxInputChars);
Check::require(!questions.empty(), "at least one question is required");
Check::max_count("questions", questions.size(), 64);
Check::max_chars("safety_identifier", safety_identifier, CoreCaps::kMaxSafetyIdChars);
Check::max_chars("prompt_cache_key", prompt_cache_key, CoreCaps::kMaxPromptCacheKeyChars);
}
Json DecisionRequest::to_json() const {
validate();
Body b(Call::kDecisionCreate);
b.put_text("model", model).put_text("input", input_text);
Json qs = Json::array();
for (const DecisionQuestion& q : questions) qs.push(q.to_json());
b.put("questions", qs);
b.put_text("safety_identifier", safety_identifier).put_text("prompt_cache_key", prompt_cache_key);
b.merge_extra(extra);
return b.build();
}
void ProvenanceRequest::validate() const {
Check::require(file.content_type.rfind("image/", 0) == 0 || file.content_type.rfind("audio/", 0) == 0,
"provenance checks take an image or audio file");
}
clients::body::MultipartBody ProvenanceRequest::to_multipart() const {
validate();
FormBody f(Call::kProvenanceCheck);
f.file("file", file, "", CoreCaps::kMaxImageBytes);
return f.form();
}
Json DecisionsService::decide(const DecisionRequest& request) {
Json body = request.to_json();
return json_call(Call::kDecisionCreate, {}, &body);
}
Json DecisionsService::check_provenance(const ProvenanceRequest& request) {
return multipart_call(Call::kProvenanceCheck, {}, request.to_multipart());
}
} // namespace molo::openai::core
+56
View File
@@ -0,0 +1,56 @@
#pragma once
// POST /decisions (classification and scoring questions about an input) and
// POST /content_provenance_checks (does a file carry OpenAI provenance
// signals). Both are small single-call surfaces of the core API.
#include <string>
#include <vector>
#include "openai/core/service.h"
#include "openai/core/upload_file.h"
namespace molo::openai::core {
struct DecisionQuestion {
enum class Kind { kPredicate, kChoice, kScore };
Kind kind = Kind::kPredicate;
std::string name;
std::string instructions;
std::vector<std::string> choices; // kChoice: values
std::vector<std::string> levels; // kScore: labels, lowest first
static DecisionQuestion predicate(std::string name, std::string instructions);
static DecisionQuestion choice(std::string name, std::string instructions,
std::vector<std::string> choices);
static DecisionQuestion score(std::string name, std::string instructions,
std::vector<std::string> levels);
clients::body::Json to_json() const;
};
struct DecisionRequest {
std::string model;
std::string input_text;
std::vector<DecisionQuestion> questions; // at least one
std::string safety_identifier;
std::string prompt_cache_key;
clients::body::Json extra;
void validate() const;
clients::body::Json to_json() const;
};
struct ProvenanceRequest {
UploadFile file; // image or audio
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
class DecisionsService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
clients::body::Json decide(const DecisionRequest& request);
clients::body::Json check_provenance(const ProvenanceRequest& request);
};
} // namespace molo::openai::core
+88
View File
@@ -0,0 +1,88 @@
#include "openai/core/embeddings.h"
#include <cstring>
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "security/crypto/Base64.h"
namespace molo::openai::core {
using clients::body::Json;
void EmbeddingRequest::validate() const {
Check::non_empty("model", model);
Check::require(inputs.empty() != token_inputs.empty(), "give inputs or token_inputs (exactly one)");
Check::max_count("inputs", inputs.size() + token_inputs.size(), CoreCaps::kMaxEmbeddingInputs);
for (const std::string& s : inputs) {
Check::non_empty("input", s);
Check::max_chars("input", s, CoreCaps::kMaxInputChars);
}
for (const auto& t : token_inputs) Check::require(!t.empty(), "token input is empty");
Check::in_set_or_empty("encoding_format", encoding_format, {"float", "base64"});
if (dimensions) Check::range("dimensions", *dimensions, 1, static_cast<double>(CoreCaps::kMaxEmbeddingDimensions));
}
Json EmbeddingRequest::to_json() const {
validate();
Body b(Call::kEmbeddingCreate);
b.put_text("model", model);
if (!inputs.empty()) {
if (inputs.size() == 1) b.put("input", Json(inputs[0]));
else b.put_strings("input", inputs);
} else {
Json outer = Json::array();
for (const auto& t : token_inputs) {
Json inner = Json::array();
for (int v : t) inner.push(Json(v));
outer.push(inner);
}
b.put("input", token_inputs.size() == 1 ? outer.as_array()[0] : outer);
}
b.put_text("encoding_format", encoding_format);
b.put_int("dimensions", dimensions);
b.put_text("user", user);
b.merge_extra(extra);
return b.build();
}
EmbeddingResult EmbeddingsService::parse(const Json& doc) {
EmbeddingResult r;
r.raw = doc;
r.model = Doc::str(doc, "model");
if (const Json* u = Doc::obj(doc, "usage")) {
r.prompt_tokens = static_cast<int>(Doc::num(*u, "prompt_tokens"));
r.total_tokens = static_cast<int>(Doc::num(*u, "total_tokens"));
}
const Json* data = Doc::arr(doc, "data");
if (!data) throw llm::ApiError("openai", 502, "BAD_RESPONSE", "embedding list has no data");
r.vectors.resize(data->as_array().size());
for (const Json& e : data->as_array()) {
long index = Doc::num(e, "index", -1);
if (index < 0 || static_cast<std::size_t>(index) >= r.vectors.size())
throw llm::ApiError("openai", 502, "BAD_RESPONSE", "embedding index out of range");
const Json* emb = e.find("embedding");
std::vector<float>& out = r.vectors[static_cast<std::size_t>(index)];
if (emb && emb->is_array()) {
for (const Json& v : emb->as_array()) out.push_back(static_cast<float>(v.as_number()));
} else if (emb && emb->is_string()) {
std::string raw;
if (!security::crypto::Base64::decode(emb->as_string(), raw) || raw.size() % 4 != 0)
throw llm::ApiError("openai", 502, "BAD_RESPONSE", "bad base64 embedding");
out.resize(raw.size() / 4);
std::memcpy(out.data(), raw.data(), raw.size()); // little-endian float32 on the wire
} else {
throw llm::ApiError("openai", 502, "BAD_RESPONSE", "embedding has no vector");
}
}
return r;
}
EmbeddingResult EmbeddingsService::create(const EmbeddingRequest& request) {
Json body = request.to_json();
return parse(json_call(Call::kEmbeddingCreate, {}, &body));
}
} // namespace molo::openai::core
+42
View File
@@ -0,0 +1,42 @@
#pragma once
// POST /embeddings. float vectors, or base64 (little-endian float32) which
// is decoded for you.
#include <string>
#include <vector>
#include "openai/core/service.h"
namespace molo::openai::core {
struct EmbeddingRequest {
std::string model; // text-embedding-3-small, ...-large, ada-002
std::vector<std::string> inputs; // texts, or
std::vector<std::vector<int>> token_inputs; // pre-tokenized inputs
std::string encoding_format; // "", float, base64
std::optional<int> dimensions; // text-embedding-3 and later
std::string user;
clients::body::Json extra;
void validate() const;
clients::body::Json to_json() const;
};
struct EmbeddingResult {
std::string model;
std::vector<std::vector<float>> vectors; // in input order
int prompt_tokens = 0;
int total_tokens = 0;
clients::body::Json raw;
};
class EmbeddingsService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
EmbeddingResult create(const EmbeddingRequest& request);
// Parses an Embedding list document (float arrays or base64 strings).
static EmbeddingResult parse(const clients::body::Json& doc);
};
} // namespace molo::openai::core
+63
View File
@@ -0,0 +1,63 @@
#include "openai/core/form_body.h"
#include "openai/core/check.h"
namespace molo::openai::core {
void FormBody::guard(const std::string& name) const {
std::string base = name;
if (base.size() > 2 && base.compare(base.size() - 2, 2, "[]") == 0) base.resize(base.size() - 2);
if (!accepts_field(call_, base))
Check::fail("field '" + base + "' is not a request property of " + spec_of(call_).operation_id);
}
FormBody& FormBody::text(const std::string& name, const std::string& value) {
if (value.empty()) return *this;
guard(name);
Check::no_control(name, value);
form_.add_text(name, value);
return *this;
}
FormBody& FormBody::integer(const std::string& name, const std::optional<int>& value) {
if (value) text(name, std::to_string(*value));
return *this;
}
FormBody& FormBody::number(const std::string& name, const std::optional<double>& value) {
if (value) {
std::string s = std::to_string(*value);
while (s.size() > 1 && s.back() == '0' && s.find('.') != std::string::npos) s.pop_back();
if (!s.empty() && s.back() == '.') s.pop_back();
text(name, s);
}
return *this;
}
FormBody& FormBody::flag(const std::string& name, const std::optional<bool>& value) {
if (value) text(name, *value ? "true" : "false");
return *this;
}
FormBody& FormBody::list(const std::string& name, const std::vector<std::string>& values) {
for (const std::string& v : values) text(name + "[]", v);
return *this;
}
FormBody& FormBody::file(const std::string& name, const UploadFile& f, const std::string& mime_prefix,
std::size_t cap) {
guard(name);
Check::non_empty(name + " filename", f.filename);
Check::require(f.filename.find_first_of("\"\r\n\\/") == std::string::npos,
name + " filename has forbidden characters");
Check::require(!f.bytes.empty(), name + " file is empty");
Check::require(f.bytes.size() <= cap, name + " file is over " + std::to_string(cap) + " bytes");
Check::require(f.content_type.find('/') != std::string::npos, name + " needs a content type");
if (!mime_prefix.empty())
Check::require(f.content_type.rfind(mime_prefix, 0) == 0,
name + " content type must start with " + mime_prefix);
form_.add_file(name, f.filename, f.content_type, f.bytes);
return *this;
}
} // namespace molo::openai::core
+37
View File
@@ -0,0 +1,37 @@
#pragma once
// Multipart request builder: field names are checked against the call row
// (a trailing "[]" is ignored for the check), file parts against a byte cap.
#include <optional>
#include <string>
#include <vector>
#include "clients/body/form.h"
#include "openai/core/calls.h"
#include "openai/core/upload_file.h"
namespace molo::openai::core {
class FormBody {
public:
explicit FormBody(Call call) : call_(call) {}
FormBody& text(const std::string& name, const std::string& value); // skipped when empty
FormBody& integer(const std::string& name, const std::optional<int>& value);
FormBody& number(const std::string& name, const std::optional<double>& value);
FormBody& flag(const std::string& name, const std::optional<bool>& value);
FormBody& list(const std::string& name, const std::vector<std::string>& values); // name[]
// A file part. `mime_prefix` ("image/", "audio/", "video/", "") restricts the
// content type; `cap` bounds the bytes.
FormBody& file(const std::string& name, const UploadFile& file, const std::string& mime_prefix,
std::size_t cap);
const clients::body::MultipartBody& form() const { return form_; }
private:
void guard(const std::string& name) const;
Call call_;
clients::body::MultipartBody form_;
};
} // namespace molo::openai::core
+233
View File
@@ -0,0 +1,233 @@
#include "openai/core/images.h"
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/form_body.h"
#include "security/crypto/Base64.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
const std::vector<std::string> kQualities = {"low", "medium", "high", "xhigh", "max",
"auto", "standard", "hd"};
bool starts(const std::string& s, const char* p) { return s.rfind(p, 0) == 0; }
void check_size(const std::string& size) {
if (size.empty() || size == "auto") return;
std::size_t x = size.find('x');
Check::require(x != std::string::npos && x > 0 && x + 1 < size.size() &&
size.find_first_not_of("0123456789x") == std::string::npos &&
size.find('x', x + 1) == std::string::npos,
"size must be auto or WIDTHxHEIGHT");
}
void check_common(const std::string& prompt, const std::string& model, const std::optional<int>& n,
const std::string& quality, const std::string& output_format,
const std::optional<int>& compression, const std::optional<int>& partial,
const std::string& moderation, const std::string& background,
const std::string& size, const std::string& response_format, bool stream) {
Check::non_empty("prompt", prompt);
std::size_t cap = CoreCaps::kMaxImagePromptChars;
if (starts(model, "dall-e-2")) cap = 1000;
else if (starts(model, "dall-e-3")) cap = 4000;
Check::max_chars("prompt", prompt, cap);
if (n) Check::range("n", *n, 1, CoreCaps::kMaxImagesN);
if (starts(model, "dall-e-3") && n) Check::require(*n == 1, "dall-e-3 generates one image per request");
Check::in_set_or_empty("quality", quality, kQualities);
Check::in_set_or_empty("output_format", output_format, {"png", "jpeg", "webp"});
if (compression) Check::range("output_compression", *compression, 0, 100);
if (partial) {
Check::range("partial_images", *partial, 0, CoreCaps::kMaxPartialImages);
Check::require(stream, "partial_images applies to streaming only");
}
Check::in_set_or_empty("moderation", moderation, {"low", "auto"});
Check::in_set_or_empty("background", background, {"transparent", "opaque", "auto"});
Check::require(background != "transparent" || output_format != "jpeg",
"transparent background needs png or webp output");
Check::in_set_or_empty("response_format", response_format, {"url", "b64_json"});
check_size(size);
}
} // namespace
void ImageGenerateRequest::validate(bool stream) const {
Check::non_empty("model", model);
check_common(prompt, model, n, quality, output_format, output_compression, partial_images,
moderation, background, size, response_format, stream);
Check::in_set_or_empty("style", style, {"vivid", "natural"});
Check::max_chars("safety_identifier", safety_identifier, CoreCaps::kMaxSafetyIdChars);
}
Json ImageGenerateRequest::to_json(bool stream) const {
validate(stream);
Body b(Call::kImageGenerate);
b.put_text("prompt", prompt);
b.put_text("model", model);
b.put_int("n", n);
b.put_text("quality", quality);
b.put_text("response_format", response_format);
b.put_text("output_format", output_format);
b.put_int("output_compression", output_compression);
b.put_int("partial_images", partial_images);
b.put_text("size", size);
b.put_text("moderation", moderation);
b.put_text("background", background);
b.put_text("style", style);
b.put_text("user", user);
b.put_text("safety_identifier", safety_identifier);
if (stream) b.put("stream", Json(true));
b.merge_extra(extra);
return b.build();
}
void ImageEditRequest::validate(bool stream) const {
check_common(prompt, model, n, quality, output_format, output_compression, partial_images,
moderation, background, size, response_format, stream);
Check::in_set_or_empty("input_fidelity", input_fidelity, {"high", "low"});
Check::max_chars("safety_identifier", safety_identifier, CoreCaps::kMaxSafetyIdChars);
Check::require(images.empty() != image_refs.empty(), "give image files (multipart) or image_refs (JSON), exactly one");
Check::max_count("images", images.size() + image_refs.size(), CoreCaps::kMaxEditImages);
if (json_mode()) {
Check::require(!mask, "JSON mode takes mask_ref, not a mask file");
for (const Json& r : image_refs)
Check::require(r.is_object() && (r.has("image_url") != r.has("file_id")),
"every image ref needs exactly one of image_url and file_id");
Check::require(response_format.empty(), "response_format is multipart only");
Check::require(mask_ref.is_null() || mask_ref.is_object(), "mask_ref must be an object");
} else {
Check::require(mask_ref.is_null(), "multipart mode takes a mask file, not mask_ref");
Check::require(extra.is_null() || (extra.is_object() && extra.as_object().empty()),
"extra fields are JSON mode only");
Check::require(!stream, "streaming edits need JSON mode (image_refs)");
}
}
Json ImageEditRequest::to_json(bool stream) const {
Check::require(json_mode(), "to_json needs JSON mode (image_refs)");
validate(stream);
Body b(Call::kImageEdit);
Json imgs = Json::array();
for (const Json& r : image_refs) imgs.push(r);
b.put("images", imgs);
if (mask_ref.is_object()) b.put("mask", mask_ref);
b.put_text("prompt", prompt);
b.put_text("model", model);
b.put_int("n", n);
b.put_text("quality", quality);
b.put_text("input_fidelity", input_fidelity);
b.put_text("size", size);
b.put_text("output_format", output_format);
b.put_int("output_compression", output_compression);
b.put_text("moderation", moderation);
b.put_text("background", background);
b.put_int("partial_images", partial_images);
b.put_text("user", user);
b.put_text("safety_identifier", safety_identifier);
if (stream) b.put("stream", Json(true));
b.merge_extra(extra);
return b.build();
}
clients::body::MultipartBody ImageEditRequest::to_multipart() const {
Check::require(!json_mode(), "to_multipart needs image files, not image_refs");
validate(false);
FormBody f(Call::kImageEdit);
const std::string field = images.size() == 1 ? "image" : "image[]";
// The guard in FormBody checks the base name "image".
for (const UploadFile& i : images) f.file(field, i, "image/", CoreCaps::kMaxImageBytes);
if (mask) f.file("mask", *mask, "image/", CoreCaps::kMaxImageBytes);
f.text("prompt", prompt).text("model", model).integer("n", n).text("quality", quality);
f.text("input_fidelity", input_fidelity).text("size", size).text("output_format", output_format);
f.integer("output_compression", output_compression).text("moderation", moderation);
f.text("background", background).text("response_format", response_format);
f.text("user", user).text("safety_identifier", safety_identifier);
return f.form();
}
void ImageVariationRequest::validate() const {
Check::in_set_or_empty("model", model, {"dall-e-2"});
if (n) Check::range("n", *n, 1, CoreCaps::kMaxImagesN);
Check::in_set_or_empty("response_format", response_format, {"url", "b64_json"});
Check::in_set_or_empty("size", size, {"256x256", "512x512", "1024x1024"});
Check::require(image.content_type == "image/png", "variation image must be image/png");
}
clients::body::MultipartBody ImageVariationRequest::to_multipart() const {
validate();
FormBody f(Call::kImageVariation);
f.file("image", image, "image/", CoreCaps::kMaxVariationBytes);
f.text("model", model).integer("n", n).text("response_format", response_format);
f.text("size", size).text("user", user);
return f.form();
}
std::string ImageItem::bytes() const {
std::string out;
if (b64_json.empty() || !security::crypto::Base64::decode(b64_json, out)) return "";
return out;
}
ImagesResult ImagesService::parse(const Json& doc) {
ImagesResult r;
r.raw = doc;
r.created = Doc::num(doc, "created");
if (const Json* u = Doc::obj(doc, "usage")) r.usage = *u;
if (const Json* data = Doc::arr(doc, "data"))
for (const Json& d : data->as_array()) {
ImageItem item;
item.url = Doc::str(d, "url");
item.b64_json = Doc::str(d, "b64_json");
item.revised_prompt = Doc::str(d, "revised_prompt");
r.data.push_back(std::move(item));
}
return r;
}
ImageStreamEvent ImageStreamEvent::from_sse(const llm::SseEvent& event) {
ImageStreamEvent e;
e.raw = event.data;
e.type = Doc::str(event.data, "type", event.event);
e.b64_json = Doc::str(event.data, "b64_json");
e.partial_index = Doc::num(event.data, "partial_image_index", -1);
e.completed = e.type.size() >= 10 && e.type.compare(e.type.size() - 10, 10, ".completed") == 0;
return e;
}
ImagesResult ImagesService::generate(const ImageGenerateRequest& request) {
Json body = request.to_json(false);
return parse(json_call(Call::kImageGenerate, {}, &body));
}
void ImagesService::stream_generate(const ImageGenerateRequest& request, const EventHandler& handler) {
Json body = request.to_json(true);
stream_call(Call::kImageGenerate, {}, body, [&](const llm::SseEvent& ev) {
ImageStreamEvent e = ImageStreamEvent::from_sse(ev);
return handler(e);
});
}
ImagesResult ImagesService::edit(const ImageEditRequest& request) {
if (request.json_mode()) {
Json body = request.to_json(false);
return parse(json_call(Call::kImageEdit, {}, &body));
}
return parse(multipart_call(Call::kImageEdit, {}, request.to_multipart()));
}
void ImagesService::stream_edit(const ImageEditRequest& request, const EventHandler& handler) {
Json body = request.to_json(true);
stream_call(Call::kImageEdit, {}, body, [&](const llm::SseEvent& ev) {
ImageStreamEvent e = ImageStreamEvent::from_sse(ev);
return handler(e);
});
}
ImagesResult ImagesService::variation(const ImageVariationRequest& request) {
return parse(multipart_call(Call::kImageVariation, {}, request.to_multipart()));
}
} // namespace molo::openai::core
+119
View File
@@ -0,0 +1,119 @@
#pragma once
// Images: generations, edits (multipart upload or JSON references),
// variations, and streaming of generations / JSON edits. Image generation
// costs money: callers decide when to spend.
#include <functional>
#include <optional>
#include <string>
#include <vector>
#include "openai/core/service.h"
#include "openai/core/upload_file.h"
namespace molo::openai::core {
struct ImageGenerateRequest {
std::string prompt;
std::string model; // gpt-image-*, dall-e-3, dall-e-2
std::optional<int> n; // 1..10 (dall-e-3: 1)
std::string quality;
std::string response_format; // url | b64_json (dall-e models only)
std::string output_format; // png | jpeg | webp
std::optional<int> output_compression; // 0..100 (jpeg, webp)
std::optional<int> partial_images; // 0..3, streaming only
std::string size;
std::string moderation; // low | auto
std::string background; // transparent | opaque | auto
std::string style; // vivid | natural (dall-e-3)
std::string user;
std::string safety_identifier;
clients::body::Json extra;
void validate(bool stream) const;
clients::body::Json to_json(bool stream) const;
};
struct ImageEditRequest {
std::string prompt;
std::string model;
// Multipart mode: files. JSON mode: references (image_url or file_id).
std::vector<UploadFile> images;
std::optional<UploadFile> mask;
std::vector<clients::body::Json> image_refs; // {"image_url":...} or {"file_id":...}
clients::body::Json mask_ref; // same shape, JSON mode
std::optional<int> n;
std::string quality;
std::string input_fidelity; // high | low
std::string size;
std::string output_format;
std::optional<int> output_compression;
std::string moderation;
std::string background;
std::optional<int> partial_images;
std::string response_format; // multipart only (dall-e-2)
std::string user;
std::string safety_identifier;
clients::body::Json extra; // JSON mode only; multipart ignores it loudly
bool json_mode() const { return !image_refs.empty(); }
void validate(bool stream) const;
clients::body::Json to_json(bool stream) const; // JSON mode body
clients::body::MultipartBody to_multipart() const; // multipart mode body
};
struct ImageVariationRequest {
UploadFile image; // square PNG under 4 MB
std::string model; // dall-e-2
std::optional<int> n; // 1..10
std::string response_format; // url | b64_json
std::string size; // 256x256 | 512x512 | 1024x1024
std::string user;
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
struct ImageItem {
std::string url;
std::string b64_json;
std::string revised_prompt;
// Decoded bytes of b64_json; empty when the item is a URL.
std::string bytes() const;
};
struct ImagesResult {
long created = 0;
std::vector<ImageItem> data;
clients::body::Json usage;
clients::body::Json raw;
};
// One event of a streamed generation or edit.
struct ImageStreamEvent {
std::string type; // image_generation.partial_image / .completed, image_edit.* equivalents
std::string b64_json;
long partial_index = -1;
bool completed = false;
clients::body::Json raw;
static ImageStreamEvent from_sse(const llm::SseEvent& event);
};
class ImagesService : public OpenaiService {
public:
using EventHandler = std::function<bool(const ImageStreamEvent&)>; // false: stop
using OpenaiService::OpenaiService;
ImagesResult generate(const ImageGenerateRequest& request);
void stream_generate(const ImageGenerateRequest& request, const EventHandler& handler);
// Multipart upload (files) or JSON references, chosen by the request.
ImagesResult edit(const ImageEditRequest& request);
// Streaming edits need JSON mode (llm::ApiClient streams JSON bodies only).
void stream_edit(const ImageEditRequest& request, const EventHandler& handler);
ImagesResult variation(const ImageVariationRequest& request);
static ImagesResult parse(const clients::body::Json& doc);
};
} // namespace molo::openai::core
+31
View File
@@ -0,0 +1,31 @@
#include "openai/core/models.h"
#include "openai/core/body.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
Json ModelsService::list() { return json_call(Call::kModelList, {}); }
std::vector<std::string> ModelsService::ids() {
Json doc = list();
std::vector<std::string> out;
if (const Json* data = Doc::arr(doc, "data"))
for (const Json& m : data->as_array()) out.push_back(Doc::str(m, "id"));
return out;
}
Json ModelsService::retrieve(const std::string& model) {
Check::id("model", model);
return json_call(Call::kModelGet, {model});
}
Json ModelsService::remove(const std::string& model) {
Check::id("model", model);
Check::require(model.rfind("ft:", 0) == 0, "only fine-tuned models (ft:...) can be deleted");
return json_call(Call::kModelDelete, {model});
}
} // namespace molo::openai::core
+25
View File
@@ -0,0 +1,25 @@
#pragma once
// Models: list, retrieve, delete a fine-tuned model.
#include <string>
#include <vector>
#include "openai/core/service.h"
namespace molo::openai::core {
class ModelsService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
// GET /models (the whole list, not paginated).
clients::body::Json list();
std::vector<std::string> ids();
// GET /models/{model}. Fine-tuned ids contain ':' (ft:gpt-4o:org::id).
clients::body::Json retrieve(const std::string& model);
// DELETE /models/{model}: only fine-tuned models owned by your organization.
clients::body::Json remove(const std::string& fine_tuned_model);
};
} // namespace molo::openai::core
+82
View File
@@ -0,0 +1,82 @@
#include "openai/core/moderations.h"
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
Json ModerationRequest::text_part(const std::string& text) {
Check::non_empty("text", text);
Json o = Json::object();
o.set("type", Json("text"));
o.set("text", Json(text));
return o;
}
Json ModerationRequest::image_part(const std::string& url) {
Check::non_empty("image url", url);
Json u = Json::object();
u.set("url", Json(url));
Json o = Json::object();
o.set("type", Json("image_url"));
o.set("image_url", u);
return o;
}
void ModerationRequest::validate() const {
Check::require(texts.empty() != parts.empty(), "give texts or parts (exactly one)");
Check::max_count("inputs", texts.size() + parts.size(), CoreCaps::kMaxModerationInputs);
for (const std::string& t : texts) {
Check::non_empty("input", t);
Check::max_chars("input", t, CoreCaps::kMaxInputChars);
}
for (const Json& p : parts) Check::require(p.is_object(), "every part must be a JSON object");
}
Json ModerationRequest::to_json() const {
validate();
Body b(Call::kModerationCreate);
b.put_text("model", model);
if (!texts.empty()) {
if (texts.size() == 1) b.put("input", Json(texts[0]));
else b.put_strings("input", texts);
} else {
Json a = Json::array();
for (const Json& p : parts) a.push(p);
b.put("input", a);
}
b.merge_extra(extra);
return b.build();
}
ModerationResult ModerationsService::parse(const Json& doc) {
ModerationResult r;
r.raw = doc;
r.id = Doc::str(doc, "id");
r.model = Doc::str(doc, "model");
const Json* results = Doc::arr(doc, "results");
if (!results) throw llm::ApiError("openai", 502, "BAD_RESPONSE", "moderation has no results");
for (const Json& item : results->as_array()) {
ModerationVerdict v;
v.flagged = Doc::flag(item, "flagged");
if (const Json* c = Doc::obj(item, "categories"))
for (const auto& [k, val] : c->as_object())
if (val.is_bool()) v.categories[k] = val.as_bool();
if (const Json* s = Doc::obj(item, "category_scores"))
for (const auto& [k, val] : s->as_object())
if (val.is_number()) v.scores[k] = val.as_number();
r.results.push_back(std::move(v));
}
return r;
}
ModerationResult ModerationsService::create(const ModerationRequest& request) {
Json body = request.to_json();
return parse(json_call(Call::kModerationCreate, {}, &body));
}
} // namespace molo::openai::core
+45
View File
@@ -0,0 +1,45 @@
#pragma once
// POST /moderations: text and image inputs.
#include <map>
#include <string>
#include <vector>
#include "openai/core/service.h"
namespace molo::openai::core {
struct ModerationRequest {
std::string model; // omni-moderation-latest by default on the server
std::vector<std::string> texts; // plain text inputs, or
std::vector<clients::body::Json> parts; // multimodal parts (ModerationRequest::text_part / image_part)
clients::body::Json extra;
static clients::body::Json text_part(const std::string& text);
static clients::body::Json image_part(const std::string& url); // https URL or data URI
void validate() const;
clients::body::Json to_json() const;
};
struct ModerationVerdict {
bool flagged = false;
std::map<std::string, bool> categories;
std::map<std::string, double> scores;
};
struct ModerationResult {
std::string id;
std::string model;
std::vector<ModerationVerdict> results;
clients::body::Json raw;
};
class ModerationsService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
ModerationResult create(const ModerationRequest& request);
static ModerationResult parse(const clients::body::Json& doc);
};
} // namespace molo::openai::core
+121
View File
@@ -0,0 +1,121 @@
#include "openai/core/responses.h"
#include <chrono>
#include <thread>
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
void WaitPolicy::validate() const {
Check::range("interval_seconds", interval_seconds, CoreCaps::kMinPollSeconds, CoreCaps::kMaxPollSeconds);
Check::range("max_wait_seconds", max_wait_seconds, 1, CoreCaps::kMaxWaitSeconds);
}
Json ResponsesService::create(const ResponseRequest& request) {
Json body = request.to_json(false);
return json_call(Call::kResponseCreate, {}, &body, nullptr, beta_headers(request.betas));
}
void ResponsesService::stream(const ResponseRequest& request, const EventHandler& handler) {
Json body = request.to_json(true);
stream_call(Call::kResponseCreate, {}, body,
[&](const llm::SseEvent& ev) {
ResponseEvent e = ResponseEvent::from_sse(ev);
bool keep = handler(e);
return keep && !e.terminal;
},
beta_headers(request.betas));
}
ResponseAccumulator ResponsesService::stream_collect(const ResponseRequest& request,
const EventHandler& observer) {
ResponseAccumulator acc;
stream(request, [&](const ResponseEvent& e) {
acc.add(e);
return observer ? observer(e) : true;
});
if (acc.errored())
throw llm::ApiError("openai", 502, "response_failed",
acc.error_message().empty() ? "stream ended with an error" : acc.error_message());
return acc;
}
Json ResponsesService::retrieve(const std::string& id, const std::vector<std::string>& include) {
Check::id("response", id);
Query q(Call::kResponseGet);
for (const std::string& i : include) Check::in_set("include", i, ResponseRequest::include_values());
q.add_list("include", include);
return json_call(Call::kResponseGet, {id}, nullptr, &q);
}
Json ResponsesService::remove(const std::string& id) {
Check::id("response", id);
return json_call(Call::kResponseDelete, {id});
}
Json ResponsesService::cancel(const std::string& id) {
Check::id("response", id);
return json_call(Call::kResponseCancel, {id});
}
Json ResponsesService::input_items(const std::string& id, const ListParams& params,
const std::vector<std::string>& include) {
Check::id("response", id);
params.validate();
Query q(Call::kResponseInputItems);
for (const std::string& i : include) Check::in_set("include", i, ResponseRequest::include_values());
q.add_list("include", include);
q.add_int("limit", params.limit);
q.add("order", params.order);
q.add("after", params.after);
return json_call(Call::kResponseInputItems, {id}, nullptr, &q);
}
Listing ResponsesService::input_items_all(const std::string& id, const ListParams& params,
const std::vector<std::string>& include,
std::size_t max_pages) {
Check::id("response", id);
Query base(Call::kResponseInputItems);
for (const std::string& i : include) Check::in_set("include", i, ResponseRequest::include_values());
base.add_list("include", include);
return walk(Call::kResponseInputItems, {id}, params, base, max_pages);
}
Json ResponsesService::input_tokens(const TokenCountRequest& request) {
Json body = request.to_json();
return json_call(Call::kResponseInputTokens, {}, &body);
}
Json ResponsesService::compact(const CompactRequest& request) {
Json body = request.to_json();
return json_call(Call::kResponseCompact, {}, &body);
}
WaitResult ResponsesService::wait(const std::string& id, const WaitPolicy& policy) {
Check::id("response", id);
policy.validate();
WaitResult out;
int waited = 0;
while (true) {
out.response = retrieve(id);
++out.polls;
out.status = Doc::str(out.response, "status");
if (out.status != "queued" && out.status != "in_progress") return out;
if (waited + policy.interval_seconds > policy.max_wait_seconds ||
out.polls >= CoreCaps::kMaxPolls) {
out.timed_out = true;
return out;
}
if (policy.sleeper) policy.sleeper(policy.interval_seconds);
else std::this_thread::sleep_for(std::chrono::seconds(policy.interval_seconds));
waited += policy.interval_seconds;
}
}
} // namespace molo::openai::core
+67
View File
@@ -0,0 +1,67 @@
#pragma once
// Responses API: create (plain, streaming, background), retrieve, delete,
// cancel, input items, input token preview, compaction, polling.
#include <functional>
#include <string>
#include <vector>
#include "openai/core/responses_request.h"
#include "openai/core/service.h"
#include "openai/core/wire/response_events.h"
namespace molo::openai::core {
struct WaitPolicy {
int interval_seconds = 2; // floor 1 (CoreCaps::kMinPollSeconds), max 60
int max_wait_seconds = 120; // max 600
// Sleeps between polls; replaceable so callers (and tests) control time.
std::function<void(int seconds)> sleeper;
void validate() const;
};
struct WaitResult {
clients::body::Json response;
std::string status; // completed, failed, incomplete, cancelled, queued, in_progress
bool timed_out = false; // still queued / in_progress when the wait ended
int polls = 0;
};
class ResponsesService : public OpenaiService {
public:
using EventHandler = std::function<bool(const ResponseEvent&)>; // false: stop
using OpenaiService::OpenaiService;
// POST /responses (stream=false). Returns the Response document.
clients::body::Json create(const ResponseRequest& request);
// POST /responses with stream=true; every event reaches the handler.
void stream(const ResponseRequest& request, const EventHandler& handler);
// stream() into a ResponseAccumulator; throws ApiError when the stream
// ends with response.failed or error. `observer` may be empty.
ResponseAccumulator stream_collect(const ResponseRequest& request,
const EventHandler& observer = {});
// GET /responses/{id}. Replaying a stream (stream=true, starting_after) is
// not available through llm::ApiClient (POST-only streaming); see SPEC gaps.
clients::body::Json retrieve(const std::string& id,
const std::vector<std::string>& include = {});
// DELETE /responses/{id}: removes the stored response. No default.
clients::body::Json remove(const std::string& id);
// POST /responses/{id}/cancel: only responses created with background=true.
clients::body::Json cancel(const std::string& id);
// GET /responses/{id}/input_items: one page.
clients::body::Json input_items(const std::string& id, const ListParams& params = {},
const std::vector<std::string>& include = {});
// Walks the pages (max_pages <= 20); truncation is reported in the result.
Listing input_items_all(const std::string& id, const ListParams& params = {},
const std::vector<std::string>& include = {}, std::size_t max_pages = 5);
clients::body::Json input_tokens(const TokenCountRequest& request);
clients::body::Json compact(const CompactRequest& request);
// Polls retrieve() until the response leaves queued / in_progress or the
// policy runs out. Never waits longer than WaitPolicy::max_wait_seconds.
WaitResult wait(const std::string& id, const WaitPolicy& policy = {});
};
} // namespace molo::openai::core
+202
View File
@@ -0,0 +1,202 @@
#include "openai/core/responses_request.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/wire/tool_schemas.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
const std::vector<std::string> kServiceTiers = {"auto", "default", "flex", "scale", "priority",
"fast", "ultrafast"};
Json items_array(const std::vector<Json>& items) {
Json a = Json::array();
for (const Json& i : items) a.push(i);
return a;
}
void check_input(const std::string& text, const std::vector<Json>& items) {
Check::require(text.empty() || items.empty(), "give input_text or input_items, not both");
Check::max_chars("input", text, CoreCaps::kMaxInputChars);
Check::max_count("input items", items.size(), CoreCaps::kMaxInputItems);
for (const Json& i : items) Check::require(i.is_object(), "every input item must be a JSON object");
}
void put_input(Body& b, const std::string& text, const std::vector<Json>& items) {
if (!text.empty()) b.put("input", Json(text));
else if (!items.empty()) b.put("input", items_array(items));
}
void put_text_block(Body& b, const std::optional<TextFormat>& format, const std::string& verbosity) {
if (!format && verbosity.empty()) return;
Json t = Json::object();
if (format) t.set("format", format->to_responses());
if (!verbosity.empty()) t.set("verbosity", Json(verbosity));
b.put("text", t);
}
void put_tools(Body& b, const std::vector<Json>& tools, const Json& choice) {
if (!tools.empty()) {
ToolSchemas::validate_response_tools(tools);
b.put("tools", items_array(tools));
}
if (!choice.is_null()) {
Check::require(choice.is_string() || choice.is_object(), "tool_choice must be a string or object");
if (choice.is_string())
Check::in_set("tool_choice", choice.as_string(), {"none", "auto", "required"});
b.put("tool_choice", choice);
}
}
} // namespace
const std::vector<std::string>& ResponseRequest::include_values() {
// INCLUDE-BEGIN (checked against the spec's IncludeEnum by tools/check_spec.py)
static const std::vector<std::string> kValues = {
"file_search_call.results", "web_search_call.results", "web_search_call.action.sources",
"message.input_image.image_url", "computer_call_output.output.image_url",
"code_interpreter_call.outputs", "reasoning.encrypted_content",
"message.output_text.logprobs"};
// INCLUDE-END
return kValues;
}
void ResponseRequest::validate(bool stream) const {
Check::require(!model.empty() || !prompt_id.empty(), "model is required (or a stored prompt id)");
if (!model.empty()) Check::max_chars("model", model, 256);
check_input(input_text, input_items);
Check::max_chars("instructions", instructions, CoreCaps::kMaxInstructionChars);
Check::require(previous_response_id.empty() || conversation_id.empty(),
"previous_response_id and conversation are mutually exclusive");
if (!previous_response_id.empty()) Check::id("previous_response", previous_response_id);
if (!conversation_id.empty()) Check::id("conversation", conversation_id);
if (max_output_tokens) Check::require(*max_output_tokens >= CoreCaps::kMinOutputTokens,
"max_output_tokens must be at least 16");
if (max_tool_calls) Check::require(*max_tool_calls >= 1, "max_tool_calls must be at least 1");
if (top_logprobs) Check::range("top_logprobs", *top_logprobs, 0, CoreCaps::kMaxTopLogprobs);
if (temperature) Check::range("temperature", *temperature, 0, 2);
if (top_p) Check::range("top_p", *top_p, 0, 1);
reasoning.validate();
Check::in_set_or_empty("verbosity", verbosity, {"low", "medium", "high"});
Check::in_set_or_empty("truncation", truncation, {"auto", "disabled"});
Check::in_set_or_empty("prompt_cache_retention", prompt_cache_retention, {"in_memory", "24h"});
Check::in_set_or_empty("service_tier", service_tier, kServiceTiers);
Check::max_count("include", include.size(), CoreCaps::kMaxIncludes);
for (const std::string& i : include) Check::in_set("include", i, include_values());
Check::metadata(metadata);
Check::max_chars("safety_identifier", safety_identifier, CoreCaps::kMaxSafetyIdChars);
Check::max_chars("prompt_cache_key", prompt_cache_key, 1024);
Check::require(prompt_version.empty() || !prompt_id.empty(), "prompt_version needs prompt_id");
Check::require(prompt_variables.is_null() || (!prompt_id.empty() && prompt_variables.is_object()),
"prompt_variables needs prompt_id and must be an object");
Check::require(context_management.is_null() || context_management.is_array(),
"context_management must be an array");
if (background && *background) {
Check::require(!store || *store, "background responses need store=true");
}
if (include_obfuscation) Check::require(stream, "include_obfuscation applies to streaming only");
}
Json ResponseRequest::to_json(bool stream) const {
validate(stream);
Body b(Call::kResponseCreate);
b.put_text("model", model);
put_input(b, input_text, input_items);
b.put_text("instructions", instructions);
b.put_text("previous_response_id", previous_response_id);
if (!conversation_id.empty()) b.put("conversation", Json(conversation_id));
put_tools(b, tools, tool_choice);
b.put_bool("parallel_tool_calls", parallel_tool_calls);
b.put_int("max_output_tokens", max_output_tokens);
b.put_int("max_tool_calls", max_tool_calls);
b.put_int("top_logprobs", top_logprobs);
b.put_num("temperature", temperature);
b.put_num("top_p", top_p);
if (!reasoning.empty()) b.put("reasoning", reasoning.to_json());
put_text_block(b, text_format, verbosity);
b.put_text("truncation", truncation);
b.put_bool("store", store);
b.put_bool("background", background);
b.put_strings("include", include);
b.put_map("metadata", metadata);
b.put_text("user", user);
b.put_text("safety_identifier", safety_identifier);
b.put_text("prompt_cache_key", prompt_cache_key);
b.put_text("prompt_cache_retention", prompt_cache_retention);
b.put_text("service_tier", service_tier);
if (!prompt_id.empty()) {
Json p = Json::object();
p.set("id", Json(prompt_id));
if (!prompt_version.empty()) p.set("version", Json(prompt_version));
if (prompt_variables.is_object()) p.set("variables", prompt_variables);
b.put("prompt", p);
}
if (stream) {
b.put("stream", Json(true));
if (include_obfuscation) {
Json so = Json::object();
so.set("include_obfuscation", Json(*include_obfuscation));
b.put("stream_options", so);
}
}
if (context_management.is_array()) b.put("context_management", context_management);
b.merge_extra(extra);
return b.build();
}
void TokenCountRequest::validate() const {
check_input(input_text, input_items);
Check::max_chars("instructions", instructions, CoreCaps::kMaxInstructionChars);
Check::require(previous_response_id.empty() || conversation_id.empty(),
"previous_response_id and conversation are mutually exclusive");
if (!previous_response_id.empty()) Check::id("previous_response", previous_response_id);
if (!conversation_id.empty()) Check::id("conversation", conversation_id);
reasoning.validate();
Check::in_set_or_empty("truncation", truncation, {"auto", "disabled"});
}
Json TokenCountRequest::to_json() const {
validate();
Body b(Call::kResponseInputTokens);
b.put_text("model", model);
put_input(b, input_text, input_items);
b.put_text("previous_response_id", previous_response_id);
b.put_text("instructions", instructions);
put_tools(b, tools, tool_choice);
if (!reasoning.empty()) b.put("reasoning", reasoning.to_json());
put_text_block(b, text_format, "");
b.put_text("truncation", truncation);
if (!conversation_id.empty()) b.put("conversation", Json(conversation_id));
b.put_bool("parallel_tool_calls", parallel_tool_calls);
b.merge_extra(extra);
return b.build();
}
void CompactRequest::validate() const {
Check::non_empty("model", model);
check_input(input_text, input_items);
Check::max_chars("instructions", instructions, CoreCaps::kMaxInstructionChars);
if (!previous_response_id.empty()) Check::id("previous_response", previous_response_id);
Check::max_chars("prompt_cache_key", prompt_cache_key, CoreCaps::kMaxPromptCacheKeyChars);
Check::in_set_or_empty("prompt_cache_retention", prompt_cache_retention, {"in_memory", "24h"});
Check::in_set_or_empty("service_tier", service_tier, kServiceTiers);
}
Json CompactRequest::to_json() const {
validate();
Body b(Call::kResponseCompact);
b.put_text("model", model);
put_input(b, input_text, input_items);
b.put_text("previous_response_id", previous_response_id);
b.put_text("instructions", instructions);
b.put_text("prompt_cache_key", prompt_cache_key);
b.put_text("prompt_cache_retention", prompt_cache_retention);
b.put_text("service_tier", service_tier);
b.merge_extra(extra);
return b.build();
}
} // namespace molo::openai::core
+95
View File
@@ -0,0 +1,95 @@
#pragma once
// POST /responses request: typed members for the common fields, `extra` for
// any other property the spec lists (merged without overriding typed ones).
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "openai/core/wire/response_config.h"
namespace molo::openai::core {
struct ResponseRequest {
std::string model;
// Input: text or items (never both). Either may be empty when a stored
// prompt template or previous_response_id supplies the context.
std::string input_text;
std::vector<clients::body::Json> input_items;
std::string instructions;
std::string previous_response_id;
std::string conversation_id; // conv_...; exclusive with previous_response_id
std::vector<clients::body::Json> tools;
clients::body::Json tool_choice; // null, "auto" / "none" / "required", or an object
std::optional<bool> parallel_tool_calls;
std::optional<int> max_output_tokens; // >= 16
std::optional<int> max_tool_calls;
std::optional<int> top_logprobs; // 0..20
std::optional<double> temperature; // 0..2
std::optional<double> top_p; // 0..1
ReasoningConfig reasoning;
std::optional<TextFormat> text_format;
std::string verbosity; // low, medium, high
std::string truncation; // auto, disabled
std::optional<bool> store;
std::optional<bool> background; // needs store, poll with ResponsesService::wait
std::vector<std::string> include; // IncludeEnum values
std::map<std::string, std::string> metadata;
std::string user;
std::string safety_identifier;
std::string prompt_cache_key;
std::string prompt_cache_retention; // in_memory, 24h
std::string service_tier;
std::string prompt_id; // stored prompt template
std::string prompt_version;
clients::body::Json prompt_variables; // object or null
std::optional<bool> include_obfuscation; // stream_options.include_obfuscation
clients::body::Json context_management; // array or null
std::vector<std::string> betas; // OpenAI-Beta header values
clients::body::Json extra; // other spec properties
static const std::vector<std::string>& include_values();
void validate(bool stream) const;
clients::body::Json to_json(bool stream) const;
};
// POST /responses/input_tokens (preview the input token count).
struct TokenCountRequest {
std::string model;
std::string input_text;
std::vector<clients::body::Json> input_items;
std::string previous_response_id;
std::string instructions;
std::vector<clients::body::Json> tools;
clients::body::Json tool_choice;
ReasoningConfig reasoning;
std::optional<TextFormat> text_format;
std::string truncation;
std::string conversation_id;
std::optional<bool> parallel_tool_calls;
clients::body::Json extra;
void validate() const;
clients::body::Json to_json() const;
};
// POST /responses/compact (compact a conversation).
struct CompactRequest {
std::string model; // required
std::string input_text;
std::vector<clients::body::Json> input_items;
std::string previous_response_id;
std::string instructions;
std::string prompt_cache_key;
std::string prompt_cache_retention;
std::string service_tier;
clients::body::Json extra;
void validate() const;
clients::body::Json to_json() const;
};
} // namespace molo::openai::core
+155
View File
@@ -0,0 +1,155 @@
#include "openai/core/service.h"
#include "clients/url.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
void Query::guard(const std::string& name) const {
if (!accepts_query(call_, name))
Check::fail("query '" + name + "' is not a parameter of " + spec_of(call_).operation_id);
}
Query& Query::add(const std::string& name, const std::string& value) {
if (value.empty()) return *this;
guard(name);
pairs_.emplace_back(name, value);
return *this;
}
Query& Query::add_int(const std::string& name, const std::optional<int>& value) {
if (value) add(name, std::to_string(*value));
return *this;
}
Query& Query::add_bool(const std::string& name, const std::optional<bool>& value) {
if (value) add(name, *value ? "true" : "false");
return *this;
}
Query& Query::add_list(const std::string& name, const std::vector<std::string>& values) {
Check::max_count(name, values.size(), CoreCaps::kMaxIncludes);
if (!values.empty()) guard(name);
for (const std::string& v : values) {
Check::non_empty(name, v);
pairs_.emplace_back(name + "[]", v);
}
return *this;
}
Query& Query::add_map(const std::string& name, const std::map<std::string, std::string>& values) {
if (!values.empty()) guard(name);
Check::metadata(values);
for (const auto& [k, v] : values) pairs_.emplace_back(name + "[" + k + "]", v);
return *this;
}
std::string Query::str() const {
std::string out;
for (const auto& [k, v] : pairs_) {
out += out.empty() ? "?" : "&";
// name[] / name[key] go out as the SDKs send them; the URL layer percent-encodes the brackets.
std::string name;
for (char c : k) name += (c == '[' || c == ']') ? std::string(1, c) : clients::Url::encode_component(std::string(1, c));
out += name + "=" + clients::Url::encode_component(v);
}
return out;
}
void ListParams::validate() const {
if (limit) Check::range("limit", *limit, 1, CoreCaps::kMaxPageLimit);
Check::in_set_or_empty("order", order, {"asc", "desc"});
if (!after.empty()) Check::id("after", after);
}
llm::Headers OpenaiService::beta_headers(const std::vector<std::string>& betas) {
llm::Headers h;
if (betas.empty()) return h;
std::string v;
for (const std::string& b : betas) {
Check::no_control("beta", b);
Check::non_empty("beta", b);
v += (v.empty() ? "" : ",") + b;
}
h["OpenAI-Beta"] = v;
return h;
}
Json OpenaiService::json_call(Call call, const std::vector<std::string>& ids, const Json* body,
const Query* query, const llm::Headers& headers) {
std::string path = render(call, ids) + (query ? query->str() : "");
return client_->request(method_of(call), path, body, headers);
}
llm::RawResponse OpenaiService::raw_call(Call call, const std::vector<std::string>& ids,
const std::string* body, const std::string& content_type,
const Query* query, const llm::Headers& headers) {
std::string path = render(call, ids) + (query ? query->str() : "");
return client_->request_raw(method_of(call), path, body, content_type, headers);
}
Json OpenaiService::multipart_call(Call call, const std::vector<std::string>& ids,
const clients::body::MultipartBody& form,
const llm::Headers& headers) {
return client_->post_multipart(render(call, ids), form, headers);
}
llm::RawResponse OpenaiService::multipart_raw(Call call, const std::vector<std::string>& ids,
const clients::body::MultipartBody& form,
const llm::Headers& headers) {
std::string encoded = form.encode();
return client_->request_raw(method_of(call), render(call, ids), &encoded, form.content_type(),
headers);
}
void OpenaiService::stream_call(Call call, const std::vector<std::string>& ids, const Json& body,
const EventSink& sink, const llm::Headers& headers) {
std::size_t seen = 0;
client_->stream(render(call, ids), body,
[&](const llm::SseEvent& ev) {
if (++seen > CoreCaps::kMaxStreamEvents) {
throw llm::ApiError("openai", 502, "BAD_RESPONSE",
"stream over the event cap");
}
return sink(ev);
},
headers);
}
Listing OpenaiService::walk(Call call, const std::vector<std::string>& ids,
const ListParams& params, const Query& base, std::size_t max_pages) {
params.validate();
Check::range("max_pages", static_cast<double>(max_pages), 1, CoreCaps::kMaxListPages);
Listing out;
std::string after = params.after;
for (std::size_t page = 0; page < max_pages; ++page) {
Query q = base;
q.add_int("limit", params.limit);
q.add("order", params.order);
q.add("after", after);
Json doc = json_call(call, ids, nullptr, &q);
++out.pages;
if (const Json* data = Doc::arr(doc, "data"))
for (const Json& item : data->as_array()) out.items.push_back(item);
bool more = Doc::flag(doc, "has_more");
std::string last = Doc::str(doc, "last_id");
if (!more) return out;
if (last.empty() || last == after) {
// The server says there is more but gave no usable cursor: stop loudly.
out.truncated = true;
return out;
}
after = last;
if (page + 1 == max_pages) {
out.truncated = true;
out.next_after = last;
}
}
return out;
}
} // namespace molo::openai::core
+92
View File
@@ -0,0 +1,92 @@
#pragma once
// OpenaiService: base of every core service. Sends calls described by the
// table (calls.h): method and path come from the row, ids are validated
// before a path is built, query names must be listed by the spec, streams
// are capped, list walks are capped and report truncation.
#include <functional>
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/form.h"
#include "clients/body/json.h"
#include "llm/api_client.h"
#include "llm/sse.h"
#include "openai/core/calls.h"
namespace molo::openai::core {
// Query string builder: names are checked against the call row, values are
// percent-encoded. Arrays use name[]=v (what the official SDKs send), maps
// use name[key]=v.
class Query {
public:
explicit Query(Call call) : call_(call) {}
Query& add(const std::string& name, const std::string& value); // skipped when empty
Query& add_int(const std::string& name, const std::optional<int>& value);
Query& add_bool(const std::string& name, const std::optional<bool>& value);
Query& add_list(const std::string& name, const std::vector<std::string>& values);
Query& add_map(const std::string& name, const std::map<std::string, std::string>& values);
std::string str() const; // "" or "?a=b&c=d"
bool empty() const { return pairs_.empty(); }
private:
void guard(const std::string& name) const;
Call call_;
std::vector<std::pair<std::string, std::string>> pairs_;
};
// Cursor pagination parameters shared by the list calls.
struct ListParams {
std::optional<int> limit; // 1..100
std::string after; // cursor (an object id)
std::string order; // "", "asc", "desc"
void validate() const;
};
// Result of walking a cursor list: loud about truncation.
struct Listing {
std::vector<clients::body::Json> items;
bool truncated = false; // the page cap was hit while the server had more
std::string next_after; // cursor to continue from when truncated
std::size_t pages = 0;
};
using EventSink = std::function<bool(const llm::SseEvent&)>;
class OpenaiService : public llm::Service {
public:
using llm::Service::Service;
protected:
clients::body::Json json_call(Call call, const std::vector<std::string>& ids,
const clients::body::Json* body = nullptr,
const Query* query = nullptr,
const llm::Headers& headers = {});
llm::RawResponse raw_call(Call call, const std::vector<std::string>& ids,
const std::string* body = nullptr,
const std::string& content_type = "", const Query* query = nullptr,
const llm::Headers& headers = {});
clients::body::Json multipart_call(Call call, const std::vector<std::string>& ids,
const clients::body::MultipartBody& form,
const llm::Headers& headers = {});
// Multipart POST whose answer may not be JSON (text, srt, vtt). The encoded
// form is bound by llm::Caps::kMaxRequestBytes (32 MiB), as for any raw body.
llm::RawResponse multipart_raw(Call call, const std::vector<std::string>& ids,
const clients::body::MultipartBody& form,
const llm::Headers& headers = {});
// POST with a JSON body decoded as SSE. At most CoreCaps::kMaxStreamEvents
// events reach the sink; the sink returns false to stop.
void stream_call(Call call, const std::vector<std::string>& ids,
const clients::body::Json& body, const EventSink& sink,
const llm::Headers& headers = {});
Listing walk(Call call, const std::vector<std::string>& ids, const ListParams& params,
const Query& base, std::size_t max_pages);
static llm::Headers beta_headers(const std::vector<std::string>& betas);
};
} // namespace molo::openai::core
+94
View File
@@ -0,0 +1,94 @@
#include "openai/core/speech.h"
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "security/crypto/Base64.h"
namespace molo::openai::core {
using clients::body::Json;
const std::vector<std::string>& SpeechRequest::builtin_voices() {
static const std::vector<std::string> kVoices = {
"alloy", "ash", "ballad", "coral", "echo", "fable", "onyx", "nova",
"sage", "shimmer", "verse", "marin", "cedar"};
return kVoices;
}
void SpeechRequest::validate(bool stream) const {
Check::non_empty("model", model);
Check::non_empty("input", input);
Check::max_chars("input", input, CoreCaps::kMaxSpeechChars);
Check::require(voice.empty() != custom_voice_id.empty(), "give voice or custom_voice_id (exactly one)");
if (!voice.empty()) Check::in_set("voice", voice, builtin_voices());
if (!custom_voice_id.empty()) Check::id("voice", custom_voice_id);
Check::max_chars("instructions", instructions, CoreCaps::kMaxSpeechChars);
const bool legacy = model.rfind("tts-1", 0) == 0;
Check::require(instructions.empty() || !legacy, "instructions are not supported by tts-1 and tts-1-hd");
Check::in_set_or_empty("response_format", response_format, {"mp3", "opus", "aac", "flac", "wav", "pcm"});
if (speed) Check::range("speed", *speed, 0.25, 4);
Check::in_set_or_empty("stream_format", stream_format, {"sse", "audio"});
if (stream) Check::require(!legacy, "SSE streaming is not supported by tts-1 and tts-1-hd");
if (!stream) Check::require(stream_format != "sse", "use stream() for stream_format=sse");
}
Json SpeechRequest::to_json(bool stream) const {
validate(stream);
Body b(Call::kSpeech);
b.put_text("model", model);
b.put_text("input", input);
if (!custom_voice_id.empty()) {
Json v = Json::object();
v.set("id", Json(custom_voice_id));
b.put("voice", v);
} else {
b.put_text("voice", voice);
}
b.put_text("instructions", instructions);
b.put_text("response_format", response_format);
b.put_num("speed", speed);
if (stream) b.put("stream_format", Json("sse"));
else b.put_text("stream_format", stream_format);
b.merge_extra(extra);
return b.build();
}
std::string SpeechStreamEvent::audio_bytes() const {
std::string out;
if (audio_b64.empty() || !security::crypto::Base64::decode(audio_b64, out)) return "";
return out;
}
SpeechStreamEvent SpeechStreamEvent::from_sse(const llm::SseEvent& event) {
SpeechStreamEvent e;
e.raw = event.data;
e.type = Doc::str(event.data, "type", event.event);
e.audio_b64 = Doc::str(event.data, "audio");
e.done = e.type == "speech.audio.done";
if (const Json* u = Doc::obj(event.data, "usage")) e.usage = *u;
return e;
}
SpeechAudio SpeechService::speak(const SpeechRequest& request) {
Json body = request.to_json(false);
std::string text = body.dump();
llm::RawResponse raw = raw_call(Call::kSpeech, {}, &text, "application/json");
if (raw.body.size() > CoreCaps::kMaxSpeechBytes)
throw llm::ApiError("openai", 502, "BAD_RESPONSE", "audio over the byte cap");
SpeechAudio out;
out.bytes = std::move(raw.body);
out.content_type = raw.content_type;
return out;
}
void SpeechService::stream(const SpeechRequest& request, const EventHandler& handler) {
Json body = request.to_json(true);
stream_call(Call::kSpeech, {}, body, [&](const llm::SseEvent& ev) {
SpeechStreamEvent e = SpeechStreamEvent::from_sse(ev);
return handler(e) && !e.done;
});
}
} // namespace molo::openai::core
+57
View File
@@ -0,0 +1,57 @@
#pragma once
// POST /audio/speech: text to speech, as one audio blob or an SSE stream of
// audio chunks. Speech costs money per character.
#include <functional>
#include <optional>
#include <string>
#include "openai/core/service.h"
namespace molo::openai::core {
struct SpeechRequest {
std::string model; // tts-1, tts-1-hd, gpt-4o-mini-tts, ...
std::string input; // 1..4096 characters
std::string voice; // built-in voice name, or
std::string custom_voice_id; // a custom voice id (voice_...), exactly one of the two
std::string instructions; // style hints, not for tts-1 / tts-1-hd
std::string response_format; // mp3, opus, aac, flac, wav, pcm
std::optional<double> speed; // 0.25..4
std::string stream_format; // "" / audio, or sse
clients::body::Json extra;
static const std::vector<std::string>& builtin_voices();
void validate(bool stream) const;
clients::body::Json to_json(bool stream) const;
};
struct SpeechAudio {
std::string bytes;
std::string content_type; // audio/mpeg, audio/wav, ...
};
struct SpeechStreamEvent {
std::string type; // speech.audio.delta / speech.audio.done
std::string audio_b64; // delta
bool done = false;
clients::body::Json usage;
clients::body::Json raw;
std::string audio_bytes() const; // decoded delta
static SpeechStreamEvent from_sse(const llm::SseEvent& event);
};
class SpeechService : public OpenaiService {
public:
using EventHandler = std::function<bool(const SpeechStreamEvent&)>; // false: stop
using OpenaiService::OpenaiService;
// Whole audio file (capped at CoreCaps::kMaxSpeechBytes).
SpeechAudio speak(const SpeechRequest& request);
// stream_format=sse is forced; not available on tts-1 / tts-1-hd.
void stream(const SpeechRequest& request, const EventHandler& handler);
};
} // namespace molo::openai::core
+175
View File
@@ -0,0 +1,175 @@
#!/usr/bin/env python3
"""Check src/openai/core against the OpenAI OpenAPI document.
python3 -I src/openai/core/tools/check_spec.py [path/to/openapi.json]
Fails (exit 1) when a row of calls.cpp has a wrong method, path, operation id,
request schema, request property list or query list; when an operation of the
spec is neither in the table nor in EXCLUDED; or when the tool type list, the
stream event list or the include list in the C++ sources differ from the
spec's Tool, ResponseStreamEvent and IncludeEnum. Python 3 stdlib only.
"""
import json
import os
import re
import sys
import urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
CORE = os.path.join(HERE, "..")
SPEC_SHA = "8243cc8fa6f04430fb1f1a6a2cb926fbeb4f7f26"
URL = "https://raw.githubusercontent.com/openai/openai-openapi/%s/openapi.json" % SPEC_SHA
# Path prefixes owned by other packages or products outside this module.
EXCLUDED = (
"/organization", "/projects", "/realtime", "/live", "/files", "/uploads", "/batches",
"/fine_tuning", "/vector_stores", "/evals", "/conversations", "/containers", "/assistants",
"/threads", "/chatkit", "/skills", "/webhook", "/agents", "/vaults", "/safety",
)
# Same operations with the OpenAI-Beta header (responses_multi_agent=v1): the
# calls are in the table under their plain path; callers add the header.
BETA_SUFFIX = "?beta=true"
VERBS = ("get", "post", "delete", "put", "patch")
STR = re.compile(r'"([^"]*)"')
def rows(text, begin, end):
a, b = text.index(begin), text.index(end)
out = []
for line in text[a:b].splitlines()[1:]:
s = line.strip()
if s.startswith("{") and '"' in s:
out.append(STR.findall(s))
return out
def props(spec, node):
schemas = spec["components"]["schemas"]
if "$ref" in node:
node = schemas[node["$ref"].split("/")[-1]]
out = set(node.get("properties", {}))
for key in ("allOf", "oneOf", "anyOf"):
for x in node.get(key, []):
out |= props(spec, x)
return out
def enum_values(spec, node):
schemas = spec["components"]["schemas"]
if "$ref" in node:
node = schemas[node["$ref"].split("/")[-1]]
out = list(node.get("enum", []))
for key in ("anyOf", "oneOf", "allOf"):
for x in node.get(key, []):
out += enum_values(spec, x)
return out
def union_types(spec, name):
schemas = spec["components"]["schemas"]
out = []
for x in schemas[name].get("oneOf") or schemas[name]["anyOf"]:
t = props_type(spec, x)
out += t
return out
def props_type(spec, node):
schemas = spec["components"]["schemas"]
if "$ref" in node:
node = schemas[node["$ref"].split("/")[-1]]
t = {}
for key in ("allOf",):
for x in node.get(key, []):
t.update(node_props(spec, x))
t.update(node.get("properties", {}))
return enum_values(spec, t["type"]) if "type" in t else []
def node_props(spec, node):
schemas = spec["components"]["schemas"]
if "$ref" in node:
node = schemas[node["$ref"].split("/")[-1]]
return node.get("properties", {})
def main():
if len(sys.argv) > 1:
spec = json.load(open(sys.argv[1]))
else:
with urllib.request.urlopen(URL, timeout=120) as r:
spec = json.load(r)
calls_text = open(os.path.join(CORE, "calls.cpp")).read()
table = rows(calls_text, "// CALLS-BEGIN", "// CALLS-END")
bad = []
seen = set()
for r in table:
http, path, op, schema, content, resp, body, query = r
seen.add((http.lower(), path))
node = spec["paths"].get(path, {}).get(http.lower())
if not node:
bad.append("missing operation %s %s" % (http, path))
continue
if node.get("operationId") != op:
bad.append("%s %s: operationId %s != %s" % (http, path, node.get("operationId"), op))
rb = node.get("requestBody", {}).get("content", {})
want_content = "+".join(sorted({"json" if c == "application/json" else "multipart" for c in rb})) or "none"
if want_content != content:
bad.append("%s: content %s != %s" % (op, content, want_content))
fields = set()
refs = []
for c, x in rb.items():
refs.append(x["schema"].get("$ref", "").split("/")[-1] or "inline")
fields |= props(spec, x["schema"])
if (refs or [""])[0] != schema and schema not in refs:
bad.append("%s: request schema %s not in %s" % (op, schema, refs))
if set(filter(None, body.split(","))) != fields:
bad.append("%s: body fields differ: only-in-table %s only-in-spec %s" % (
op, sorted(set(filter(None, body.split(","))) - fields),
sorted(fields - set(filter(None, body.split(","))))))
q = {p["name"] for p in node.get("parameters", []) if p["in"] == "query"}
if set(filter(None, query.split(","))) != q:
bad.append("%s: query params differ: table %s spec %s" % (op, query, sorted(q)))
kinds = set()
for c in node["responses"].get("200", {}).get("content", {}):
kinds.add("sse" if c == "text/event-stream" else "json" if c == "application/json"
else "text" if c == "text/plain" else "binary")
if "+".join(sorted(kinds)) != resp:
bad.append("%s: response kinds %s != %s" % (op, resp, "+".join(sorted(kinds))))
ops = 0
for path, item in spec["paths"].items():
for m in VERBS:
if m not in item:
continue
ops += 1
if (m, path) in seen or path.endswith(BETA_SUFFIX) or path.startswith(EXCLUDED):
continue
bad.append("unwrapped spec operation %s %s (%s)" % (m.upper(), path, item[m].get("operationId")))
# lists in C++ sources
tools = open(os.path.join(CORE, "wire", "tool_schemas.cpp")).read()
cpp_tools = set(STR.findall(tools[tools.index("TOOL-TYPES-BEGIN"):tools.index("TOOL-TYPES-END")]))
spec_tools = set(union_types(spec, "Tool"))
if cpp_tools != spec_tools:
bad.append("tool types differ: %s / %s" % (sorted(cpp_tools - spec_tools), sorted(spec_tools - cpp_tools)))
ev = open(os.path.join(CORE, "wire", "response_events.cpp")).read()
cpp_ev = {r[0] for r in rows(ev, "// EVENTS-BEGIN", "// EVENTS-END")}
spec_ev = set(union_types(spec, "ResponseStreamEvent"))
if cpp_ev != spec_ev:
bad.append("stream events differ: %s / %s" % (sorted(cpp_ev - spec_ev), sorted(spec_ev - cpp_ev)))
req = open(os.path.join(CORE, "responses_request.cpp")).read()
cpp_inc = set(STR.findall(req[req.index("INCLUDE-BEGIN"):req.index("INCLUDE-END")]))
spec_inc = set(enum_values(spec, spec["components"]["schemas"]["IncludeEnum"]))
if cpp_inc != spec_inc:
bad.append("include values differ: %s / %s" % (sorted(cpp_inc - spec_inc), sorted(spec_inc - cpp_inc)))
if bad:
print("SPEC CHECK FAILED")
for b in bad:
print(" " + b)
return 1
print("spec ok: %d calls, %d spec operations seen, %d tool types, %d stream events, %d include values, openapi %s (%s)"
% (len(table), ops, len(spec_tools), len(spec_ev), len(spec_inc), spec["info"]["version"], SPEC_SHA[:10]))
return 0
if __name__ == "__main__":
sys.exit(main())
+103
View File
@@ -0,0 +1,103 @@
#include "openai/core/transcriptions.h"
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/form_body.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
bool json_format(const std::string& f) {
return f.empty() || f == "json" || f == "verbose_json" || f == "diarized_json";
}
TextResult finish(const llm::RawResponse& raw, const std::string& format) {
TextResult r;
r.format = format.empty() ? "json" : format;
if (json_format(format)) {
try {
r.raw = Json::parse(raw.body);
} catch (const std::exception& e) {
throw llm::ApiError("openai", 502, "BAD_RESPONSE", e.what());
}
r.text = Doc::str(r.raw, "text");
} else {
r.text = raw.body;
}
return r;
}
void check_file(const UploadFile& f) {
Check::non_empty("file", f.bytes);
Check::require(f.bytes.size() <= CoreCaps::kMaxAudioFileBytes, "audio file is over 25 MiB");
}
} // namespace
void TranscriptionRequest::validate() const {
check_file(file);
Check::non_empty("model", model);
Check::in_set_or_empty("response_format", response_format,
{"json", "text", "srt", "verbose_json", "vtt", "diarized_json"});
if (temperature) Check::range("temperature", *temperature, 0, 1);
for (const std::string& i : include) Check::in_set("include", i, {"logprobs"});
Check::max_count("timestamp_granularities", timestamp_granularities.size(), 2);
for (const std::string& g : timestamp_granularities) Check::in_set("timestamp granularity", g, {"word", "segment"});
Check::require(timestamp_granularities.empty() || response_format == "verbose_json",
"timestamp_granularities needs response_format=verbose_json");
Check::in_set_or_empty("chunking_strategy", chunking_strategy, {"auto"});
Check::max_count("languages", languages.size(), CoreCaps::kMaxTranscriptionLists);
Check::max_count("keywords", keywords.size(), CoreCaps::kMaxTranscriptionLists);
Check::max_count("known_speaker_names", known_speaker_names.size(), 4);
Check::require(known_speaker_names.size() == known_speaker_references.size(),
"known_speaker_names and known_speaker_references must pair up");
for (const std::string& r : known_speaker_references)
Check::require(r.rfind("data:", 0) == 0, "speaker references must be data URLs");
Check::max_chars("prompt", prompt, 4096);
}
clients::body::MultipartBody TranscriptionRequest::to_multipart() const {
validate();
FormBody f(Call::kTranscription);
f.file("file", file, "", CoreCaps::kMaxAudioFileBytes);
f.text("model", model).text("language", language).text("prompt", prompt);
f.list("languages", languages).list("keywords", keywords);
f.text("response_format", response_format).number("temperature", temperature);
f.list("include", include).list("timestamp_granularities", timestamp_granularities);
f.text("chunking_strategy", chunking_strategy);
f.list("known_speaker_names", known_speaker_names);
f.list("known_speaker_references", known_speaker_references);
return f.form();
}
void TranslationRequest::validate() const {
check_file(file);
Check::in_set("model", model, {"whisper-1"});
Check::in_set_or_empty("response_format", response_format, {"json", "text", "srt", "verbose_json", "vtt"});
if (temperature) Check::range("temperature", *temperature, 0, 1);
Check::max_chars("prompt", prompt, 4096);
}
clients::body::MultipartBody TranslationRequest::to_multipart() const {
validate();
FormBody f(Call::kTranslation);
f.file("file", file, "", CoreCaps::kMaxAudioFileBytes);
f.text("model", model).text("prompt", prompt).text("response_format", response_format);
f.number("temperature", temperature);
return f.form();
}
TextResult TranscriptionsService::transcribe(const TranscriptionRequest& request) {
clients::body::MultipartBody form = request.to_multipart();
return finish(multipart_raw(Call::kTranscription, {}, form), request.response_format);
}
TextResult TranscriptionsService::translate(const TranslationRequest& request) {
clients::body::MultipartBody form = request.to_multipart();
return finish(multipart_raw(Call::kTranslation, {}, form), request.response_format);
}
} // namespace molo::openai::core
+59
View File
@@ -0,0 +1,59 @@
#pragma once
// POST /audio/transcriptions and /audio/translations (multipart upload).
// Streaming transcription (stream=true) needs multipart SSE, which
// llm::ApiClient does not offer, so there is no stream member (see SPEC gaps).
#include <optional>
#include <string>
#include <vector>
#include "openai/core/service.h"
#include "openai/core/upload_file.h"
namespace molo::openai::core {
struct TranscriptionRequest {
UploadFile file; // flac, mp3, mp4, mpeg, mpga, m4a, ogg, wav, webm; <= 25 MiB
std::string model; // whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe, ...
std::string language; // ISO-639-1
std::vector<std::string> languages; // gpt-transcribe: possible languages
std::vector<std::string> keywords; // gpt-transcribe: guide words
std::string prompt;
std::string response_format; // json, text, srt, verbose_json, vtt, diarized_json
std::optional<double> temperature; // 0..1
std::vector<std::string> include; // logprobs
std::vector<std::string> timestamp_granularities; // word, segment (verbose_json)
std::string chunking_strategy; // "" or "auto"
std::vector<std::string> known_speaker_names; // diarize model, up to 4
std::vector<std::string> known_speaker_references; // data URLs, matching names
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
struct TranslationRequest {
UploadFile file;
std::string model; // whisper-1
std::string prompt;
std::string response_format; // json, text, srt, verbose_json, vtt
std::optional<double> temperature;
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
struct TextResult {
std::string text; // the transcript (also for text / srt / vtt bodies)
std::string format; // the response_format that was asked
clients::body::Json raw; // the JSON document for json formats, else null
};
class TranscriptionsService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
TextResult transcribe(const TranscriptionRequest& request);
TextResult translate(const TranslationRequest& request);
};
} // namespace molo::openai::core
+14
View File
@@ -0,0 +1,14 @@
#pragma once
#include <string>
namespace molo::openai::core {
// A file part of a multipart request (image, audio, video, document).
struct UploadFile {
std::string filename; // e.g. "photo.png"; no path separators or quotes
std::string content_type; // e.g. "image/png"
std::string bytes;
};
} // namespace molo::openai::core
+221
View File
@@ -0,0 +1,221 @@
#include "openai/core/videos.h"
#include <chrono>
#include <thread>
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/form_body.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
const std::vector<std::string> kSeconds = {"4", "8", "12"};
const std::vector<std::string> kSizes = {"720x1280", "1280x720", "1024x1792", "1792x1024"};
void check_prompt(const std::string& p) {
Check::non_empty("prompt", p);
Check::max_chars("prompt", p, CoreCaps::kMaxVideoPromptChars);
}
Json ref(const char* key, const std::string& value) {
Json o = Json::object();
o.set(key, Json(value));
return o;
}
} // namespace
void VideoCreateRequest::validate() const {
check_prompt(prompt);
Check::in_set_or_empty("seconds", seconds, kSeconds);
Check::in_set_or_empty("size", size, kSizes);
int refs = (!reference_image_url.empty() ? 1 : 0) + (!reference_file_id.empty() ? 1 : 0) +
(reference_upload ? 1 : 0);
Check::require(refs <= 1, "give at most one reference: image url, file id or upload");
if (!reference_file_id.empty()) Check::id("file", reference_file_id);
Check::require(!reference_upload || extra.is_null() || (extra.is_object() && extra.as_object().empty()),
"extra fields are JSON mode only");
}
Json VideoCreateRequest::to_json() const {
Check::require(!multipart(), "to_json needs JSON mode (no upload)");
validate();
Body b(Call::kVideoCreate);
b.put_text("prompt", prompt).put_text("model", model).put_text("seconds", seconds).put_text("size", size);
if (!reference_image_url.empty()) b.put("input_reference", ref("image_url", reference_image_url));
if (!reference_file_id.empty()) b.put("input_reference", ref("file_id", reference_file_id));
b.merge_extra(extra);
return b.build();
}
clients::body::MultipartBody VideoCreateRequest::to_multipart() const {
Check::require(multipart(), "to_multipart needs a reference upload");
validate();
FormBody f(Call::kVideoCreate);
f.text("prompt", prompt).text("model", model).text("seconds", seconds).text("size", size);
f.file("input_reference", *reference_upload, "image/", CoreCaps::kMaxVideoReferenceBytes);
return f.form();
}
void VideoEditRequest::validate() const {
check_prompt(prompt);
Check::require(video_id.empty() != !video.has_value(), "give video_id or a video upload (exactly one)");
if (!video_id.empty()) Check::id("video", video_id);
}
Json VideoEditRequest::to_json() const {
Check::require(!multipart(), "to_json needs video_id (no upload)");
validate();
Body b(Call::kVideoEdit);
b.put("video", ref("id", video_id)).put_text("prompt", prompt);
return b.build();
}
clients::body::MultipartBody VideoEditRequest::to_multipart() const {
Check::require(multipart(), "to_multipart needs a video upload");
validate();
FormBody f(Call::kVideoEdit);
f.file("video", *video, "video/", CoreCaps::kMaxVideoReferenceBytes);
f.text("prompt", prompt);
return f.form();
}
void VideoExtendRequest::validate() const {
check_prompt(prompt);
Check::non_empty("seconds", seconds);
Check::in_set("seconds", seconds, kSeconds);
Check::require(video_id.empty() != !video.has_value(), "give video_id or a video upload (exactly one)");
if (!video_id.empty()) Check::id("video", video_id);
}
Json VideoExtendRequest::to_json() const {
Check::require(!multipart(), "to_json needs video_id (no upload)");
validate();
Body b(Call::kVideoExtend);
b.put("video", ref("id", video_id)).put_text("prompt", prompt).put_text("seconds", seconds);
return b.build();
}
clients::body::MultipartBody VideoExtendRequest::to_multipart() const {
Check::require(multipart(), "to_multipart needs a video upload");
validate();
FormBody f(Call::kVideoExtend);
f.file("video", *video, "video/", CoreCaps::kMaxVideoReferenceBytes);
f.text("prompt", prompt).text("seconds", seconds);
return f.form();
}
void VideoCharacterRequest::validate() const {
Check::non_empty("name", name);
Check::max_chars("name", name, CoreCaps::kMaxCharacterNameChars);
Check::no_control("name", name);
}
clients::body::MultipartBody VideoCharacterRequest::to_multipart() const {
validate();
FormBody f(Call::kVideoCharacterCreate);
f.file("video", video, "video/", CoreCaps::kMaxVideoReferenceBytes);
f.text("name", name);
return f.form();
}
void VideoWaitPolicy::validate() const {
Check::range("interval_seconds", interval_seconds, CoreCaps::kMinPollSeconds, CoreCaps::kMaxPollSeconds);
Check::range("max_wait_seconds", max_wait_seconds, 1, CoreCaps::kMaxWaitSeconds);
}
Json VideosService::create(const VideoCreateRequest& request) {
if (request.multipart()) return multipart_call(Call::kVideoCreate, {}, request.to_multipart());
Json body = request.to_json();
return json_call(Call::kVideoCreate, {}, &body);
}
Json VideosService::edit(const VideoEditRequest& request) {
if (request.multipart()) return multipart_call(Call::kVideoEdit, {}, request.to_multipart());
Json body = request.to_json();
return json_call(Call::kVideoEdit, {}, &body);
}
Json VideosService::extend(const VideoExtendRequest& request) {
if (request.multipart()) return multipart_call(Call::kVideoExtend, {}, request.to_multipart());
Json body = request.to_json();
return json_call(Call::kVideoExtend, {}, &body);
}
Json VideosService::remix(const std::string& id, const std::string& prompt) {
Check::id("video", id);
check_prompt(prompt);
Json body = Json::object();
body.set("prompt", Json(prompt));
return json_call(Call::kVideoRemix, {id}, &body);
}
Json VideosService::create_character(const VideoCharacterRequest& request) {
return multipart_call(Call::kVideoCharacterCreate, {}, request.to_multipart());
}
Json VideosService::get_character(const std::string& id) {
Check::id("character", id);
return json_call(Call::kVideoCharacterGet, {id});
}
Json VideosService::retrieve(const std::string& id) {
Check::id("video", id);
return json_call(Call::kVideoGet, {id});
}
Json VideosService::list(const ListParams& params) {
params.validate();
Query q(Call::kVideoList);
q.add_int("limit", params.limit);
q.add("order", params.order);
q.add("after", params.after);
return json_call(Call::kVideoList, {}, nullptr, &q);
}
Listing VideosService::list_all(const ListParams& params, std::size_t max_pages) {
return walk(Call::kVideoList, {}, params, Query(Call::kVideoList), max_pages);
}
Json VideosService::remove(const std::string& id) {
Check::id("video", id);
return json_call(Call::kVideoDelete, {id});
}
VideoContent VideosService::content(const std::string& id, const std::string& variant) {
Check::id("video", id);
Check::in_set("variant", variant, {"video", "thumbnail", "spritesheet"});
Query q(Call::kVideoContent);
q.add("variant", variant == "video" ? "" : variant);
llm::RawResponse raw = raw_call(Call::kVideoContent, {id}, nullptr, "", &q);
VideoContent out;
out.bytes = std::move(raw.body);
out.content_type = raw.content_type;
out.variant = variant;
return out;
}
VideoWaitResult VideosService::wait(const std::string& id, const VideoWaitPolicy& policy) {
Check::id("video", id);
policy.validate();
VideoWaitResult out;
int waited = 0;
while (true) {
out.video = retrieve(id);
++out.polls;
out.status = Doc::str(out.video, "status");
if (out.status != "queued" && out.status != "in_progress") return out;
if (waited + policy.interval_seconds > policy.max_wait_seconds || out.polls >= CoreCaps::kMaxPolls) {
out.timed_out = true;
return out;
}
if (policy.sleeper) policy.sleeper(policy.interval_seconds);
else std::this_thread::sleep_for(std::chrono::seconds(policy.interval_seconds));
waited += policy.interval_seconds;
}
}
} // namespace molo::openai::core
+100
View File
@@ -0,0 +1,100 @@
#pragma once
// Videos (Sora): create, edit, extend, remix, characters, list, retrieve,
// delete, content download, polling. Video generation costs real money.
#include <functional>
#include <optional>
#include <string>
#include "openai/core/service.h"
#include "openai/core/upload_file.h"
namespace molo::openai::core {
struct VideoCreateRequest {
std::string prompt; // <= 32000
std::string model; // sora-2, sora-2-pro, dated snapshots
std::string seconds; // "4", "8", "12"
std::string size; // 720x1280, 1280x720, 1024x1792, 1792x1024
// Optional reference image: a URL or file id (JSON), or an upload (multipart).
std::string reference_image_url;
std::string reference_file_id;
std::optional<UploadFile> reference_upload;
clients::body::Json extra; // JSON mode only
bool multipart() const { return reference_upload.has_value(); }
void validate() const;
clients::body::Json to_json() const;
clients::body::MultipartBody to_multipart() const;
};
struct VideoEditRequest {
std::string prompt;
std::string video_id; // an existing video, or
std::optional<UploadFile> video; // an uploaded video (multipart)
bool multipart() const { return video.has_value(); }
void validate() const;
clients::body::Json to_json() const;
clients::body::MultipartBody to_multipart() const;
};
struct VideoExtendRequest {
std::string prompt;
std::string seconds;
std::string video_id;
std::optional<UploadFile> video;
bool multipart() const { return video.has_value(); }
void validate() const;
clients::body::Json to_json() const;
clients::body::MultipartBody to_multipart() const;
};
struct VideoCharacterRequest {
std::string name; // <= 80
UploadFile video;
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
struct VideoContent {
std::string bytes;
std::string content_type; // video/mp4, image/webp
std::string variant; // video, thumbnail, spritesheet
};
struct VideoWaitPolicy {
int interval_seconds = 5; // floor 1, max 60
int max_wait_seconds = 600; // max 600
std::function<void(int seconds)> sleeper;
void validate() const;
};
struct VideoWaitResult {
clients::body::Json video;
std::string status; // queued, in_progress, completed, failed
bool timed_out = false;
int polls = 0;
};
class VideosService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
clients::body::Json create(const VideoCreateRequest& request);
clients::body::Json edit(const VideoEditRequest& request);
clients::body::Json extend(const VideoExtendRequest& request);
clients::body::Json remix(const std::string& video_id, const std::string& prompt);
clients::body::Json create_character(const VideoCharacterRequest& request);
clients::body::Json get_character(const std::string& character_id);
clients::body::Json retrieve(const std::string& video_id);
clients::body::Json list(const ListParams& params = {});
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
// DELETE: removes the video and its assets. No default.
clients::body::Json remove(const std::string& video_id);
// GET /videos/{id}/content: the rendered video, or a thumbnail / spritesheet.
VideoContent content(const std::string& video_id, const std::string& variant = "video");
VideoWaitResult wait(const std::string& video_id, const VideoWaitPolicy& policy = {});
};
} // namespace molo::openai::core
+110
View File
@@ -0,0 +1,110 @@
#include "openai/core/voices.h"
#include "openai/core/body.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "openai/core/form_body.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
constexpr std::size_t kMaxSampleBytes = 10u * 1024u * 1024u;
void check_name(const std::string& name) {
Check::non_empty("name", name);
Check::max_chars("name", name, CoreCaps::kMaxVoiceNameChars);
Check::no_control("name", name);
}
} // namespace
void VoiceConsentRequest::validate() const {
check_name(name);
Check::non_empty("language", language);
Check::max_chars("language", language, 35);
}
clients::body::MultipartBody VoiceConsentRequest::to_multipart() const {
validate();
FormBody f(Call::kVoiceConsentCreate);
f.text("name", name).file("recording", recording, "audio/", kMaxSampleBytes).text("language", language);
return f.form();
}
void VoiceFromSampleRequest::validate() const {
check_name(name);
Check::id("consent", consent_id);
}
clients::body::MultipartBody VoiceFromSampleRequest::to_multipart() const {
validate();
FormBody f(Call::kVoiceCreate);
f.text("type", "audio_sample").text("name", name);
f.file("audio_sample", audio_sample, "audio/", kMaxSampleBytes).text("consent", consent_id);
return f.form();
}
void VoiceFromPromptRequest::validate() const {
check_name(name);
Check::non_empty("prompt", prompt);
Check::max_chars("prompt", prompt, CoreCaps::kMaxVoicePromptChars);
Check::max_chars("script_hint", script_hint, CoreCaps::kMaxVoicePromptChars);
Check::in_set_or_empty("model", model, {"auto", "2026-10-08"});
}
Json VoiceFromPromptRequest::to_json() const {
validate();
Body b(Call::kVoiceCreate);
b.put("type", Json("prompt"));
b.put_text("name", name).put_text("prompt", prompt).put_text("script_hint", script_hint);
b.put_text("model", model);
return b.build();
}
Json VoicesService::create_consent(const VoiceConsentRequest& request) {
return multipart_call(Call::kVoiceConsentCreate, {}, request.to_multipart());
}
Json VoicesService::list_consents(const ListParams& params) {
params.validate();
Query q(Call::kVoiceConsentList);
q.add_int("limit", params.limit);
q.add("after", params.after);
Check::require(params.order.empty(), "voice consents have no order parameter");
return json_call(Call::kVoiceConsentList, {}, nullptr, &q);
}
Json VoicesService::get_consent(const std::string& id) {
Check::id("consent", id);
return json_call(Call::kVoiceConsentGet, {id});
}
Json VoicesService::update_consent(const std::string& id, const std::string& name) {
Check::id("consent", id);
check_name(name);
Json body = Json::object();
body.set("name", Json(name));
return json_call(Call::kVoiceConsentUpdate, {id}, &body);
}
Json VoicesService::remove_consent(const std::string& id) {
Check::id("consent", id);
return json_call(Call::kVoiceConsentDelete, {id});
}
Json VoicesService::create_voice(const VoiceFromSampleRequest& request) {
return multipart_call(Call::kVoiceCreate, {}, request.to_multipart());
}
Json VoicesService::create_voice(const VoiceFromPromptRequest& request) {
Json body = request.to_json();
return json_call(Call::kVoiceCreate, {}, &body);
}
Json VoicesService::remove_voice(const std::string& id) {
Check::id("voice", id);
return json_call(Call::kVoiceDelete, {id});
}
} // namespace molo::openai::core
+55
View File
@@ -0,0 +1,55 @@
#pragma once
// Custom voices: consent recordings (create, list, get, update, delete) and
// voices (from a consented audio sample, or from a text prompt).
#include <string>
#include "openai/core/service.h"
#include "openai/core/upload_file.h"
namespace molo::openai::core {
struct VoiceConsentRequest {
std::string name; // 1..256
UploadFile recording; // audio, <= 10 MiB
std::string language; // BCP 47, e.g. en-US
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
struct VoiceFromSampleRequest {
std::string name;
UploadFile audio_sample; // audio, <= 10 MiB
std::string consent_id; // cons_...
void validate() const;
clients::body::MultipartBody to_multipart() const;
};
struct VoiceFromPromptRequest {
std::string name;
std::string prompt; // <= 2000
std::string script_hint; // <= 2000
std::string model; // auto | 2026-10-08
void validate() const;
clients::body::Json to_json() const;
};
class VoicesService : public OpenaiService {
public:
using OpenaiService::OpenaiService;
clients::body::Json create_consent(const VoiceConsentRequest& request);
clients::body::Json list_consents(const ListParams& params = {});
clients::body::Json get_consent(const std::string& consent_id);
clients::body::Json update_consent(const std::string& consent_id, const std::string& name);
// DELETE: removes the consent recording. No default.
clients::body::Json remove_consent(const std::string& consent_id);
clients::body::Json create_voice(const VoiceFromSampleRequest& request);
clients::body::Json create_voice(const VoiceFromPromptRequest& request);
// DELETE: removes the custom voice. No default.
clients::body::Json remove_voice(const std::string& voice_id);
};
} // namespace molo::openai::core
+218
View File
@@ -0,0 +1,218 @@
#include "openai/core/wire/chat_wire.h"
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/check.h"
#include "openai/core/wire/input_items.h"
#include "openai/core/wire/neutral_map.h"
#include "openai/core/wire/tool_schemas.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
void apply_response_format(const llm::ChatRequest& in, const WireOptions& opt,
std::optional<TextFormat>& out) {
if (!in.response_schema.is_null()) {
out = TextFormat::json_schema("response", in.response_schema, opt.strict_schema);
} else if (in.json_mode) {
out = TextFormat::json_object();
}
}
} // namespace
ChatCompletionRequest ChatWire::request(const llm::ChatRequest& in, const WireOptions& opt) {
ChatCompletionRequest r;
r.model = in.model;
if (!in.system.empty())
r.messages.push_back(opt.developer_role ? ChatMessages::developer(in.system)
: ChatMessages::system(in.system));
for (const llm::Message& m : in.messages)
for (Json& j : ChatMessages::from_neutral(m, opt.developer_role)) r.messages.push_back(std::move(j));
for (const llm::ToolSpec& t : in.tools) r.tools.push_back(ToolSchemas::chat_from_neutral(t, opt.strict_tools));
r.tool_choice = ToolSchemas::choice_from_neutral(in.tool_choice, true);
r.max_completion_tokens = in.max_tokens;
r.temperature = in.temperature;
r.top_p = in.top_p;
r.stop = in.stop;
apply_response_format(in, opt, r.response_format);
r.reasoning_effort = in.reasoning_effort;
r.store = opt.store;
r.extra = in.extra;
return r;
}
llm::ChatResponse ChatWire::response(const Json& doc) {
llm::ChatResponse out;
out.raw = doc;
out.id = Doc::str(doc, "id");
out.model = Doc::str(doc, "model");
out.message.role = llm::Role::kAssistant;
if (const Json* u = Doc::obj(doc, "usage")) out.usage = NeutralMap::usage_chat(*u);
const Json* choices = Doc::arr(doc, "choices");
if (!choices || choices->as_array().empty())
throw llm::ApiError("openai", 502, "BAD_RESPONSE", "completion has no choices");
const Json& choice = choices->as_array()[0];
out.finish_reason = NeutralMap::finish_chat(Doc::str(choice, "finish_reason"));
const Json* msg = Doc::obj(choice, "message");
if (!msg) return out;
std::string text = Doc::str(*msg, "content");
if (text.empty()) text = Doc::str(*msg, "refusal");
if (!text.empty()) out.message.parts.push_back(llm::ContentPart::of_text(text));
if (const Json* calls = Doc::arr(*msg, "tool_calls")) {
for (const Json& c : calls->as_array()) {
llm::ToolCall tc;
tc.id = Doc::str(c, "id");
const std::string type = Doc::str(c, "type", "function");
if (type == "custom") {
if (const Json* cu = Doc::obj(c, "custom")) {
tc.name = Doc::str(*cu, "name");
tc.arguments = Json::object();
tc.arguments.set("input", Json(Doc::str(*cu, "input")));
}
} else if (const Json* f = Doc::obj(c, "function")) {
tc.name = Doc::str(*f, "name");
tc.arguments = NeutralMap::parse_arguments(Doc::str(*f, "arguments"));
}
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
}
} else if (const Json* fc = Doc::obj(*msg, "function_call")) {
llm::ToolCall tc;
tc.id = "function_call";
tc.name = Doc::str(*fc, "name");
tc.arguments = NeutralMap::parse_arguments(Doc::str(*fc, "arguments"));
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
}
return out;
}
void ChatChunkDecoder::decode(const Json& chunk, std::vector<llm::ChatChunk>& out) {
if (const Json* err = Doc::obj(chunk, "error"))
throw llm::ApiError("openai", 502, Doc::str(*err, "type", "stream_error"),
Doc::str(*err, "message", "stream error"));
const Json* choices = Doc::arr(chunk, "choices");
if (choices) {
for (const Json& choice : choices->as_array()) {
// Only the first choice is surfaced; callers of the neutral seam use n=1.
if (Doc::num(choice, "index") != 0) continue;
if (const Json* delta = Doc::obj(choice, "delta")) {
std::string text = Doc::str(*delta, "content");
if (text.empty()) text = Doc::str(*delta, "refusal");
if (!text.empty()) {
llm::ChatChunk c;
c.text_delta = text;
out.push_back(std::move(c));
}
std::string reasoning = Doc::str(*delta, "reasoning_content");
if (!reasoning.empty()) {
llm::ChatChunk c;
c.reasoning_delta = reasoning;
out.push_back(std::move(c));
}
if (const Json* calls = Doc::arr(*delta, "tool_calls")) {
for (const Json& tc : calls->as_array()) {
llm::ChatChunk c;
const long index = Doc::num(tc, "index");
std::string id = Doc::str(tc, "id");
if (!id.empty()) ids_[index] = id;
c.tool_call_id = ids_.count(index) ? ids_[index] : std::to_string(index);
if (const Json* f = Doc::obj(tc, "function")) {
c.tool_call_name = Doc::str(*f, "name");
c.tool_args_delta = Doc::str(*f, "arguments");
}
out.push_back(std::move(c));
}
}
}
std::string finish = Doc::str(choice, "finish_reason");
if (!finish.empty()) {
llm::ChatChunk c;
c.finish_reason = NeutralMap::finish_chat(finish);
out.push_back(std::move(c));
}
}
}
if (const Json* u = Doc::obj(chunk, "usage")) {
llm::ChatChunk c;
c.usage = NeutralMap::usage_chat(*u);
out.push_back(std::move(c));
}
}
ResponseRequest ResponsesWire::request(const llm::ChatRequest& in, const WireOptions& opt) {
ResponseRequest r;
r.model = in.model;
r.instructions = in.system;
r.input_items = InputItems::from_neutral(in.messages);
for (const llm::ToolSpec& t : in.tools) r.tools.push_back(ToolSchemas::from_neutral(t, opt.strict_tools));
r.tool_choice = ToolSchemas::choice_from_neutral(in.tool_choice, false);
r.max_output_tokens = in.max_tokens;
r.temperature = in.temperature;
r.top_p = in.top_p;
if (!in.stop.empty())
Check::fail("the Responses API has no stop sequences; use Chat Completions mode");
apply_response_format(in, opt, r.text_format);
r.reasoning.effort = in.reasoning_effort;
r.store = opt.store;
r.extra = in.extra;
return r;
}
llm::ChatResponse ResponsesWire::response(const Json& doc) {
llm::ChatResponse out;
out.raw = doc;
out.id = Doc::str(doc, "id");
out.model = Doc::str(doc, "model");
out.message.role = llm::Role::kAssistant;
const std::string status = Doc::str(doc, "status");
if (status == "failed") {
std::string msg = "response failed";
std::string code = "response_failed";
if (const Json* e = Doc::obj(doc, "error")) {
msg = Doc::str(*e, "message", msg);
code = Doc::str(*e, "code", code);
}
throw llm::ApiError("openai", 502, code, msg);
}
out.finish_reason = (status == "queued" || status == "in_progress") ? status
: NeutralMap::finish_response(doc);
if (const Json* u = Doc::obj(doc, "usage")) out.usage = NeutralMap::usage_responses(*u);
const Json* items = Doc::arr(doc, "output");
if (!items) return out;
for (const Json& item : items->as_array()) {
const std::string type = Doc::str(item, "type");
if (type == "message") {
if (const Json* content = Doc::arr(item, "content"))
for (const Json& part : content->as_array()) {
const std::string pt = Doc::str(part, "type");
if (pt == "output_text") out.message.parts.push_back(llm::ContentPart::of_text(Doc::str(part, "text")));
else if (pt == "refusal") out.message.parts.push_back(llm::ContentPart::of_text(Doc::str(part, "refusal")));
}
} else if (type == "function_call") {
llm::ToolCall tc;
tc.id = Doc::str(item, "call_id", Doc::str(item, "id"));
tc.name = Doc::str(item, "name");
tc.arguments = NeutralMap::parse_arguments(Doc::str(item, "arguments"));
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
} else if (type == "custom_tool_call") {
llm::ToolCall tc;
tc.id = Doc::str(item, "call_id", Doc::str(item, "id"));
tc.name = Doc::str(item, "name");
tc.arguments = Json::object();
tc.arguments.set("input", Json(Doc::str(item, "input")));
out.message.parts.push_back(llm::ContentPart::of_call(std::move(tc)));
} else if (type == "reasoning") {
llm::ContentPart p;
if (const Json* summary = Doc::arr(item, "summary"))
for (const Json& s : summary->as_array()) {
if (!p.reasoning.empty()) p.reasoning += "\n";
p.reasoning += Doc::str(s, "text");
}
if (!p.reasoning.empty()) out.message.parts.push_back(std::move(p));
}
}
return out;
}
} // namespace molo::openai::core
+51
View File
@@ -0,0 +1,51 @@
#pragma once
// Neutral llm::ChatRequest <-> OpenAI wire formats. Two dialects, one
// shared options struct: Chat Completions (ChatWire) and Responses
// (ResponsesWire). Both are reusable by any package whose API speaks the
// same shapes.
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
#include "openai/core/chat_request.h"
#include "openai/core/responses_request.h"
namespace molo::openai::core {
struct WireOptions {
bool developer_role = false; // send system text as role "developer"
bool strict_tools = false; // tools[].strict (Responses default is strict)
bool strict_schema = true; // response_schema as strict structured output
std::optional<bool> store; // unset: the server default (Responses stores, Chat does not)
};
class ChatWire {
public:
static ChatCompletionRequest request(const llm::ChatRequest& request,
const WireOptions& options = {});
static llm::ChatResponse response(const clients::body::Json& completion);
};
// Chat Completions stream chunks to neutral chunks. Stateful: tool call
// fragments carry only an index after the first one.
class ChatChunkDecoder {
public:
// Appends zero or more neutral chunks for one `chat.completion.chunk`.
void decode(const clients::body::Json& chunk, std::vector<llm::ChatChunk>& out);
private:
std::map<long, std::string> ids_; // tool call index -> call id
};
class ResponsesWire {
public:
static ResponseRequest request(const llm::ChatRequest& request, const WireOptions& options = {});
static llm::ChatResponse response(const clients::body::Json& response);
};
} // namespace molo::openai::core
+214
View File
@@ -0,0 +1,214 @@
#include "openai/core/wire/input_items.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
#include "security/crypto/Base64.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
Json typed(const char* type) {
Json o = Json::object();
o.set("type", Json(type));
return o;
}
} // namespace
Json InputItems::text(const std::string& text) {
Check::max_chars("text", text, CoreCaps::kMaxInputChars);
Json o = typed("input_text");
o.set("text", Json(text));
return o;
}
Json InputItems::output_text(const std::string& text) {
Check::max_chars("text", text, CoreCaps::kMaxInputChars);
Json o = typed("output_text");
o.set("text", Json(text));
o.set("annotations", Json::array());
return o;
}
Json InputItems::image_url(const std::string& url, const std::string& detail) {
Check::non_empty("image url", url);
Check::max_chars("image url", url, 20971520);
Check::in_set("image detail", detail, {"low", "high", "auto", "original"});
Json o = typed("input_image");
o.set("image_url", Json(url));
o.set("detail", Json(detail));
return o;
}
Json InputItems::image_file(const std::string& file_id, const std::string& detail) {
Check::id("file", file_id);
Check::in_set("image detail", detail, {"low", "high", "auto", "original"});
Json o = typed("input_image");
o.set("file_id", Json(file_id));
o.set("detail", Json(detail));
return o;
}
std::string InputItems::data_uri(const llm::ImageData& image) {
if (!image.url.empty()) return image.url;
Check::non_empty("image mime", image.mime);
Check::require(image.mime.rfind("image/", 0) == 0, "image mime must start with image/");
Check::require(!image.bytes.empty(), "image has neither url nor bytes");
Check::require(image.bytes.size() <= security::crypto::Base64::kMaxInputBytes,
"inline image over 16 MiB");
return "data:" + image.mime + ";base64," + security::crypto::Base64::encode(image.bytes);
}
Json InputItems::image_data(const llm::ImageData& image, const std::string& detail) {
return image_url(data_uri(image), detail);
}
Json InputItems::file_id(const std::string& id) {
Check::id("file", id);
Json o = typed("input_file");
o.set("file_id", Json(id));
return o;
}
Json InputItems::file_url(const std::string& url, const std::string& detail) {
Check::non_empty("file url", url);
Check::in_set_or_empty("file detail", detail, {"auto", "low", "high"});
Json o = typed("input_file");
o.set("file_url", Json(url));
if (!detail.empty()) o.set("detail", Json(detail));
return o;
}
Json InputItems::file_data(const std::string& filename, const std::string& bytes,
const std::string& mime) {
Check::non_empty("filename", filename);
Check::require(mime.find('/') != std::string::npos && mime.find(';') == std::string::npos,
"file mime must look like type/subtype");
Check::require(!bytes.empty() && bytes.size() <= security::crypto::Base64::kMaxInputBytes,
"inline file must be 1 byte to 16 MiB");
Json o = typed("input_file");
o.set("filename", Json(filename));
o.set("file_data", Json("data:" + mime + ";base64," +
security::crypto::Base64::encode(bytes)));
return o;
}
Json InputItems::message(const std::string& role, const std::string& text) {
Check::in_set("role", role, {"user", "assistant", "system", "developer"});
Check::max_chars("text", text, CoreCaps::kMaxInputChars);
Json o = typed("message");
o.set("role", Json(role));
o.set("content", Json(text));
return o;
}
Json InputItems::message(const std::string& role, const std::vector<Json>& parts) {
Check::in_set("role", role, {"user", "assistant", "system", "developer"});
Check::require(!parts.empty(), "message needs content");
Json o = typed("message");
o.set("role", Json(role));
Json a = Json::array();
for (const Json& p : parts) a.push(p);
o.set("content", a);
return o;
}
Json InputItems::function_call(const std::string& call_id, const std::string& name,
const std::string& arguments_json) {
Check::non_empty("call_id", call_id);
Check::max_chars("call_id", call_id, 64);
Check::non_empty("function name", name);
Json o = typed("function_call");
o.set("call_id", Json(call_id));
o.set("name", Json(name));
o.set("arguments", Json(arguments_json));
return o;
}
Json InputItems::function_call_output(const std::string& call_id, const std::string& output) {
Check::non_empty("call_id", call_id);
Check::max_chars("call_id", call_id, 64);
Check::max_chars("output", output, CoreCaps::kMaxInputChars);
Json o = typed("function_call_output");
o.set("call_id", Json(call_id));
o.set("output", Json(output));
return o;
}
Json InputItems::custom_tool_call_output(const std::string& call_id, const std::string& output) {
Check::non_empty("call_id", call_id);
Check::max_chars("output", output, CoreCaps::kMaxInputChars);
Json o = typed("custom_tool_call_output");
o.set("call_id", Json(call_id));
o.set("output", Json(output));
return o;
}
Json InputItems::computer_call_output(const std::string& call_id, const std::string& screenshot_url,
const std::vector<Json>& acknowledged_checks) {
Check::non_empty("call_id", call_id);
Check::non_empty("screenshot url", screenshot_url);
Json shot = typed("computer_screenshot");
shot.set("image_url", Json(screenshot_url));
Json o = typed("computer_call_output");
o.set("call_id", Json(call_id));
o.set("output", shot);
if (!acknowledged_checks.empty()) {
Json a = Json::array();
for (const Json& c : acknowledged_checks) a.push(c);
o.set("acknowledged_safety_checks", a);
}
return o;
}
Json InputItems::mcp_approval_response(const std::string& approval_request_id, bool approve,
const std::string& reason) {
Check::non_empty("approval_request_id", approval_request_id);
Json o = typed("mcp_approval_response");
o.set("approval_request_id", Json(approval_request_id));
o.set("approve", Json(approve));
if (!reason.empty()) o.set("reason", Json(reason));
return o;
}
Json InputItems::item_reference(const std::string& id) {
Check::id("item", id);
Json o = typed("item_reference");
o.set("id", Json(id));
return o;
}
std::vector<Json> InputItems::from_neutral(const std::vector<llm::Message>& messages) {
Check::max_count("messages", messages.size(), CoreCaps::kMaxInputItems);
std::vector<Json> out;
for (const llm::Message& m : messages) {
const bool assistant = m.role == llm::Role::kAssistant;
std::vector<Json> parts;
auto flush = [&]() {
if (parts.empty()) return;
const char* role = m.role == llm::Role::kSystem ? "system" : assistant ? "assistant" : "user";
out.push_back(message(role, parts));
parts.clear();
};
for (const llm::ContentPart& p : m.parts) {
if (!p.text.empty()) parts.push_back(assistant ? output_text(p.text) : text(p.text));
if (p.image) {
Check::require(!assistant, "assistant messages cannot carry images");
parts.push_back(image_data(*p.image));
}
if (p.tool_call) {
flush();
out.push_back(function_call(p.tool_call->id, p.tool_call->name, p.tool_call->arguments.dump()));
}
if (p.tool_result) {
flush();
out.push_back(function_call_output(p.tool_result->call_id, p.tool_result->content));
}
}
flush();
}
return out;
}
} // namespace molo::openai::core
+53
View File
@@ -0,0 +1,53 @@
#pragma once
// Builders for Responses API input items and content parts, plus the
// neutral llm::Message to input item mapping. Reusable wire pieces.
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
namespace molo::openai::core {
class InputItems {
public:
// Content parts.
static clients::body::Json text(const std::string& text);
static clients::body::Json image_url(const std::string& url, const std::string& detail = "auto");
static clients::body::Json image_file(const std::string& file_id,
const std::string& detail = "auto");
static clients::body::Json image_data(const llm::ImageData& image,
const std::string& detail = "auto");
static clients::body::Json file_id(const std::string& id);
static clients::body::Json file_url(const std::string& url, const std::string& detail = "");
static clients::body::Json file_data(const std::string& filename, const std::string& bytes,
const std::string& mime = "application/pdf");
static clients::body::Json output_text(const std::string& text);
// Items. role is user, system, developer or assistant.
static clients::body::Json message(const std::string& role, const std::string& text);
static clients::body::Json message(const std::string& role,
const std::vector<clients::body::Json>& parts);
static clients::body::Json function_call(const std::string& call_id, const std::string& name,
const std::string& arguments_json);
static clients::body::Json function_call_output(const std::string& call_id,
const std::string& output);
static clients::body::Json custom_tool_call_output(const std::string& call_id,
const std::string& output);
static clients::body::Json computer_call_output(const std::string& call_id,
const std::string& screenshot_url,
const std::vector<clients::body::Json>& acknowledged_checks = {});
static clients::body::Json mcp_approval_response(const std::string& approval_request_id,
bool approve, const std::string& reason = "");
static clients::body::Json item_reference(const std::string& id);
// llm::Message list to input items (system messages become developer-less
// plain "system" messages; ChatRequest::system is handled by the caller as
// `instructions`).
static std::vector<clients::body::Json> from_neutral(const std::vector<llm::Message>& messages);
static std::string data_uri(const llm::ImageData& image);
};
} // namespace molo::openai::core
+70
View File
@@ -0,0 +1,70 @@
#include "openai/core/wire/neutral_map.h"
#include "openai/core/body.h"
namespace molo::openai::core {
using clients::body::Json;
llm::Usage NeutralMap::usage_chat(const Json& usage) {
llm::Usage u;
if (!usage.is_object()) return u;
u.input_tokens = static_cast<int>(Doc::num(usage, "prompt_tokens"));
u.output_tokens = static_cast<int>(Doc::num(usage, "completion_tokens"));
u.total_tokens = static_cast<int>(Doc::num(usage, "total_tokens"));
if (const Json* d = Doc::obj(usage, "prompt_tokens_details"))
u.cached_tokens = static_cast<int>(Doc::num(*d, "cached_tokens"));
if (const Json* d = Doc::obj(usage, "completion_tokens_details"))
u.reasoning_tokens = static_cast<int>(Doc::num(*d, "reasoning_tokens"));
return u;
}
llm::Usage NeutralMap::usage_responses(const Json& usage) {
llm::Usage u;
if (!usage.is_object()) return u;
u.input_tokens = static_cast<int>(Doc::num(usage, "input_tokens"));
u.output_tokens = static_cast<int>(Doc::num(usage, "output_tokens"));
u.total_tokens = static_cast<int>(Doc::num(usage, "total_tokens"));
if (const Json* d = Doc::obj(usage, "input_tokens_details"))
u.cached_tokens = static_cast<int>(Doc::num(*d, "cached_tokens"));
if (const Json* d = Doc::obj(usage, "output_tokens_details"))
u.reasoning_tokens = static_cast<int>(Doc::num(*d, "reasoning_tokens"));
return u;
}
std::string NeutralMap::finish_chat(const std::string& reason) {
if (reason == "function_call") return "tool_calls";
return reason; // stop, length, tool_calls, content_filter pass through
}
std::string NeutralMap::finish_response(const Json& response) {
const std::string status = Doc::str(response, "status");
if (status == "incomplete") {
std::string why;
if (const Json* d = Doc::obj(response, "incomplete_details")) why = Doc::str(*d, "reason");
if (why == "max_output_tokens") return "length";
if (why == "content_filter") return "content_filter";
return why.empty() ? "length" : why;
}
if (status == "failed") return "error";
if (status == "cancelled") return "cancelled";
if (const Json* out = Doc::arr(response, "output"))
for (const Json& item : out->as_array()) {
const std::string t = Doc::str(item, "type");
if (t == "function_call" || t == "custom_tool_call") return "tool_calls";
}
return "stop";
}
Json NeutralMap::parse_arguments(const std::string& text) {
if (text.empty()) return Json::object();
try {
return Json::parse(text);
} catch (const std::exception&) {
Json o = Json::object();
o.set("_raw", Json(text));
return o;
}
}
} // namespace molo::openai::core
+27
View File
@@ -0,0 +1,27 @@
#pragma once
// Small shared mappings from OpenAI documents to the neutral llm types:
// usage objects and finish reasons.
#include <string>
#include "clients/body/json.h"
#include "llm/chat_types.h"
namespace molo::openai::core {
class NeutralMap {
public:
// Chat Completions / legacy Completions usage (prompt_tokens, ...).
static llm::Usage usage_chat(const clients::body::Json& usage);
// Responses usage (input_tokens, ...).
static llm::Usage usage_responses(const clients::body::Json& usage);
// Chat finish_reason: stop, length, tool_calls, content_filter, function_call.
static std::string finish_chat(const std::string& reason);
// Responses: from status, incomplete_details.reason and the output items.
static std::string finish_response(const clients::body::Json& response);
// Parses a tool-call arguments string; invalid JSON is kept as {"_raw": text}.
static clients::body::Json parse_arguments(const std::string& text);
};
} // namespace molo::openai::core
+145
View File
@@ -0,0 +1,145 @@
#include "openai/core/wire/response_config.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
void walk(const Json& node, int depth, std::size_t& props) {
Check::require(depth <= StrictSchema::kMaxDepth, "schema nests deeper than 10 levels");
if (!node.is_object()) return;
const Json* type = node.find("type");
const Json* properties = node.find("properties");
const bool is_object = (type && type->is_string() && type->as_string() == "object") ||
(properties && properties->is_object());
if (is_object) {
const Json* ap = node.find("additionalProperties");
Check::require(ap && ap->is_bool() && !ap->as_bool(),
"strict schema: every object needs additionalProperties:false");
std::vector<std::string> required;
if (const Json* r = node.find("required"); r && r->is_array())
for (const Json& v : r->as_array())
if (v.is_string()) required.push_back(v.as_string());
if (properties && properties->is_object()) {
for (const auto& [k, v] : properties->as_object()) {
Check::require(Check::contains(required, k),
"strict schema: property '" + k + "' must be listed in required");
++props;
Check::require(props <= StrictSchema::kMaxProperties, "schema has over 5000 properties");
walk(v, depth + 1, props);
}
}
}
if (const Json* items = node.find("items")) walk(*items, depth + 1, props);
for (const char* key : {"anyOf", "oneOf", "allOf"})
if (const Json* alts = node.find(key); alts && alts->is_array())
for (const Json& a : alts->as_array()) walk(a, depth + 1, props);
for (const char* key : {"$defs", "definitions"})
if (const Json* defs = node.find(key); defs && defs->is_object())
for (const auto& [k, v] : defs->as_object()) walk(v, depth + 1, props);
}
} // namespace
void StrictSchema::check(const Json& schema) {
Check::require(schema.is_object(), "schema must be a JSON object");
std::size_t props = 0;
walk(schema, 1, props);
}
void StrictSchema::check_name(const std::string& name) {
Check::require(!name.empty() && name.size() <= 64, "schema name must be 1..64 characters");
for (char c : name)
Check::require((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '_' || c == '-',
"schema name may only use letters, digits, underscore and hyphen");
}
TextFormat TextFormat::text() { return TextFormat(); }
TextFormat TextFormat::json_object() {
TextFormat f;
f.kind = Kind::kJsonObject;
return f;
}
TextFormat TextFormat::json_schema(std::string name, Json schema, bool strict,
std::string description) {
TextFormat f;
f.kind = Kind::kJsonSchema;
f.name = std::move(name);
f.schema = std::move(schema);
f.strict = strict;
f.description = std::move(description);
return f;
}
void TextFormat::validate() const {
if (kind != Kind::kJsonSchema) return;
StrictSchema::check_name(name);
Check::require(schema.is_object(), "json_schema needs a schema object");
if (strict) StrictSchema::check(schema);
}
Json TextFormat::to_responses() const {
validate();
Json o = Json::object();
switch (kind) {
case Kind::kText: o.set("type", Json("text")); break;
case Kind::kJsonObject: o.set("type", Json("json_object")); break;
case Kind::kJsonSchema:
o.set("type", Json("json_schema"));
o.set("name", Json(name));
o.set("schema", schema);
o.set("strict", Json(strict));
if (!description.empty()) o.set("description", Json(description));
break;
}
return o;
}
Json TextFormat::to_chat() const {
validate();
Json o = Json::object();
switch (kind) {
case Kind::kText: o.set("type", Json("text")); break;
case Kind::kJsonObject: o.set("type", Json("json_object")); break;
case Kind::kJsonSchema: {
o.set("type", Json("json_schema"));
Json s = Json::object();
s.set("name", Json(name));
s.set("schema", schema);
s.set("strict", Json(strict));
if (!description.empty()) s.set("description", Json(description));
o.set("json_schema", s);
break;
}
}
return o;
}
const std::vector<std::string>& ReasoningConfig::efforts() {
static const std::vector<std::string> kEfforts = {"none", "minimal", "low", "medium",
"high", "xhigh", "max"};
return kEfforts;
}
void ReasoningConfig::validate() const {
Check::in_set_or_empty("reasoning effort", effort, efforts());
Check::in_set_or_empty("reasoning summary", summary, {"auto", "concise", "detailed"});
Check::in_set_or_empty("reasoning context", context, {"auto", "current_turn", "all_turns"});
Check::in_set_or_empty("reasoning mode", mode, {"standard", "pro"});
}
Json ReasoningConfig::to_json() const {
validate();
Json o = Json::object();
if (!effort.empty()) o.set("effort", Json(effort));
if (!summary.empty()) o.set("summary", Json(summary));
if (!context.empty()) o.set("context", Json(context));
if (!mode.empty()) o.set("mode", Json(mode));
return o;
}
} // namespace molo::openai::core
+56
View File
@@ -0,0 +1,56 @@
#pragma once
// Output shaping pieces shared by Responses and Chat Completions: structured
// output formats (JSON mode, JSON Schema) and reasoning controls.
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::openai::core {
// Structured Outputs strict-mode rules, checked locally so a bad schema is a
// clear error instead of a 400 round trip: every object sets
// additionalProperties:false and lists all its properties as required;
// nesting depth and total property count are capped.
class StrictSchema {
public:
static constexpr int kMaxDepth = 10;
static constexpr std::size_t kMaxProperties = 5000;
static void check(const clients::body::Json& schema);
static void check_name(const std::string& name);
};
struct TextFormat {
enum class Kind { kText, kJsonObject, kJsonSchema };
Kind kind = Kind::kText;
std::string name; // json_schema: 1..64 of [A-Za-z0-9_-]
std::string description;
clients::body::Json schema;
bool strict = true;
static TextFormat text();
static TextFormat json_object();
static TextFormat json_schema(std::string name, clients::body::Json schema, bool strict = true,
std::string description = "");
void validate() const;
// Responses API: the value of text.format.
clients::body::Json to_responses() const;
// Chat Completions: the value of response_format.
clients::body::Json to_chat() const;
};
struct ReasoningConfig {
std::string effort; // none, minimal, low, medium, high, xhigh, max
std::string summary; // auto, concise, detailed
std::string context; // auto, current_turn, all_turns
std::string mode; // standard, pro
bool empty() const { return effort.empty() && summary.empty() && context.empty() && mode.empty(); }
void validate() const;
clients::body::Json to_json() const;
static const std::vector<std::string>& efforts();
};
} // namespace molo::openai::core
+215
View File
@@ -0,0 +1,215 @@
#include "openai/core/wire/response_events.h"
#include "llm/error.h"
#include "openai/core/body.h"
#include "openai/core/wire/neutral_map.h"
namespace molo::openai::core {
using clients::body::Json;
// EVENTS-BEGIN (checked against the spec's ResponseStreamEvent union by tools/check_spec.py)
static const EventInfo kEvents[] = {
{"response.audio.delta", EventKind::kAudio, false},
{"response.audio.done", EventKind::kAudio, false},
{"response.audio.transcript.delta", EventKind::kAudio, false},
{"response.audio.transcript.done", EventKind::kAudio, false},
{"response.code_interpreter_call_code.delta", EventKind::kCodeInterpreter, false},
{"response.code_interpreter_call_code.done", EventKind::kCodeInterpreter, false},
{"response.code_interpreter_call.completed", EventKind::kCodeInterpreter, false},
{"response.code_interpreter_call.in_progress", EventKind::kCodeInterpreter, false},
{"response.code_interpreter_call.interpreting", EventKind::kCodeInterpreter, false},
{"response.compaction.compacting", EventKind::kCompaction, false},
{"response.completed", EventKind::kLifecycle, true},
{"response.content_part.added", EventKind::kContentPart, false},
{"response.content_part.done", EventKind::kContentPart, false},
{"response.created", EventKind::kLifecycle, false},
{"error", EventKind::kError, true},
{"response.file_search_call.completed", EventKind::kToolStatus, false},
{"response.file_search_call.in_progress", EventKind::kToolStatus, false},
{"response.file_search_call.searching", EventKind::kToolStatus, false},
{"response.function_call_arguments.delta", EventKind::kToolArguments, false},
{"response.function_call_arguments.done", EventKind::kToolArguments, false},
{"response.shell_call_command.added", EventKind::kShell, false},
{"response.shell_call_command.delta", EventKind::kShell, false},
{"response.shell_call_command.done", EventKind::kShell, false},
{"response.shell_call_output_content.delta", EventKind::kShell, false},
{"response.shell_call_output_content.done", EventKind::kShell, false},
{"response.in_progress", EventKind::kLifecycle, false},
{"response.failed", EventKind::kLifecycle, true},
{"response.incomplete", EventKind::kLifecycle, true},
{"response.output_item.added", EventKind::kOutputItem, false},
{"response.output_item.done", EventKind::kOutputItem, false},
{"response.reasoning_summary_part.added", EventKind::kReasoning, false},
{"response.reasoning_summary_part.done", EventKind::kReasoning, false},
{"response.reasoning_summary_text.delta", EventKind::kReasoning, false},
{"response.reasoning_summary_text.done", EventKind::kReasoning, false},
{"response.reasoning_text.delta", EventKind::kReasoning, false},
{"response.reasoning_text.done", EventKind::kReasoning, false},
{"response.refusal.delta", EventKind::kRefusal, false},
{"response.refusal.done", EventKind::kRefusal, false},
{"response.output_text.delta", EventKind::kText, false},
{"response.output_text.done", EventKind::kText, false},
{"response.web_search_call.completed", EventKind::kToolStatus, false},
{"response.web_search_call.in_progress", EventKind::kToolStatus, false},
{"response.web_search_call.searching", EventKind::kToolStatus, false},
{"response.image_generation_call.completed", EventKind::kImage, false},
{"response.image_generation_call.generating", EventKind::kImage, false},
{"response.image_generation_call.in_progress", EventKind::kImage, false},
{"response.image_generation_call.partial_image", EventKind::kImage, false},
{"response.mcp_call_arguments.delta", EventKind::kToolArguments, false},
{"response.mcp_call_arguments.done", EventKind::kToolArguments, false},
{"response.mcp_call.completed", EventKind::kMcp, false},
{"response.mcp_call.failed", EventKind::kMcp, false},
{"response.mcp_call.in_progress", EventKind::kMcp, false},
{"response.mcp_list_tools.completed", EventKind::kMcp, false},
{"response.mcp_list_tools.failed", EventKind::kMcp, false},
{"response.mcp_list_tools.in_progress", EventKind::kMcp, false},
{"response.output_text.annotation.added", EventKind::kAnnotation, false},
{"response.queued", EventKind::kLifecycle, false},
{"response.custom_tool_call_input.delta", EventKind::kToolArguments, false},
{"response.custom_tool_call_input.done", EventKind::kToolArguments, false},
};
// EVENTS-END
std::size_t ResponseEvents::count() { return sizeof(kEvents) / sizeof(kEvents[0]); }
const EventInfo& ResponseEvents::at(std::size_t index) { return kEvents[index]; }
const EventInfo* ResponseEvents::find(const std::string& type) {
for (const EventInfo& e : kEvents)
if (type == e.type) return &e;
return nullptr;
}
ResponseEvent ResponseEvent::from_sse(const llm::SseEvent& sse) {
ResponseEvent e;
e.data = sse.data;
e.type = Doc::str(sse.data, "type", sse.event);
e.sequence = Doc::num(sse.data, "sequence_number", -1);
if (const EventInfo* info = ResponseEvents::find(e.type)) {
e.known = true;
e.kind = info->kind;
e.terminal = info->terminal;
}
return e;
}
std::string ResponseEvent::text_delta() const {
if (type == "response.output_text.delta" || type == "response.refusal.delta")
return Doc::str(data, "delta");
return "";
}
const Json* ResponseEvent::response() const { return Doc::obj(data, "response"); }
void ResponseAccumulator::add(const ResponseEvent& e) {
++events_;
++counts_[e.type];
if (!e.known) {
++unknown_;
return;
}
if (e.type == "response.output_text.delta") {
text_ += Doc::str(e.data, "delta");
} else if (e.type == "response.refusal.delta") {
refusal_ += Doc::str(e.data, "delta");
} else if (e.type == "response.reasoning_summary_text.delta") {
reasoning_summary_ += Doc::str(e.data, "delta");
} else if (e.type == "response.reasoning_summary_part.done" && !reasoning_summary_.empty() &&
reasoning_summary_.back() != '\n') {
reasoning_summary_ += "\n";
} else if (e.type == "response.output_item.done") {
if (const Json* item = Doc::obj(e.data, "item")) {
const std::string t = Doc::str(*item, "type");
if (t == "function_call") {
llm::ToolCall c;
c.id = Doc::str(*item, "call_id", Doc::str(*item, "id"));
c.name = Doc::str(*item, "name");
c.arguments = NeutralMap::parse_arguments(Doc::str(*item, "arguments"));
calls_.push_back(std::move(c));
} else if (t == "custom_tool_call") {
llm::ToolCall c;
c.id = Doc::str(*item, "call_id", Doc::str(*item, "id"));
c.name = Doc::str(*item, "name");
c.arguments = Json::object();
c.arguments.set("input", Json(Doc::str(*item, "input")));
calls_.push_back(std::move(c));
}
}
} else if (e.type == "response.created" || e.type == "response.in_progress" ||
e.type == "response.queued") {
if (const Json* r = e.response()) status_ = Doc::str(*r, "status", status_);
} else if (e.type == "response.completed" || e.type == "response.incomplete") {
if (const Json* r = e.response()) {
final_ = *r;
status_ = Doc::str(*r, "status", e.type == "response.completed" ? "completed" : "incomplete");
}
done_ = true;
} else if (e.type == "response.failed") {
errored_ = true;
done_ = true;
status_ = "failed";
if (const Json* r = e.response()) {
final_ = *r;
if (const Json* err = Doc::obj(*r, "error")) error_message_ = Doc::str(*err, "message");
}
} else if (e.type == "error") {
errored_ = true;
done_ = true;
error_message_ = Doc::str(e.data, "message");
}
}
bool ResponsesChunkDecoder::decode(const ResponseEvent& e, llm::ChatChunk& chunk) {
chunk = llm::ChatChunk();
if (!e.known) return false;
const std::string& t = e.type;
if (t == "response.output_text.delta") {
chunk.text_delta = Doc::str(e.data, "delta");
return true;
}
if (t == "response.reasoning_summary_text.delta" || t == "response.reasoning_text.delta") {
chunk.reasoning_delta = Doc::str(e.data, "delta");
return true;
}
if (t == "response.output_item.added") {
if (const Json* item = Doc::obj(e.data, "item")) {
const std::string it = Doc::str(*item, "type");
if (it == "function_call" || it == "custom_tool_call") {
std::string call_id = Doc::str(*item, "call_id", Doc::str(*item, "id"));
std::string name = Doc::str(*item, "name");
items_[Doc::str(*item, "id")] = {call_id, name};
chunk.tool_call_id = call_id;
chunk.tool_call_name = name;
return true;
}
}
return false;
}
if (t == "response.function_call_arguments.delta" || t == "response.custom_tool_call_input.delta") {
auto it = items_.find(Doc::str(e.data, "item_id"));
chunk.tool_call_id = it == items_.end() ? Doc::str(e.data, "item_id") : it->second.first;
chunk.tool_args_delta = Doc::str(e.data, "delta");
return true;
}
if (t == "response.completed" || t == "response.incomplete") {
if (const Json* r = e.response()) {
chunk.finish_reason = NeutralMap::finish_response(*r);
if (const Json* u = Doc::obj(*r, "usage")) chunk.usage = NeutralMap::usage_responses(*u);
}
return true;
}
if (t == "response.failed") {
std::string msg = "response failed";
if (const Json* r = e.response())
if (const Json* err = Doc::obj(*r, "error")) msg = Doc::str(*err, "message", msg);
throw llm::ApiError("openai", 502, "response_failed", msg);
}
if (t == "error") {
throw llm::ApiError("openai", 502, Doc::str(e.data, "code", "stream_error"),
Doc::str(e.data, "message", "stream error"));
}
return false;
}
} // namespace molo::openai::core
+97
View File
@@ -0,0 +1,97 @@
#pragma once
// Responses API streaming: the table of every event type the spec lists, a
// typed event wrapper, an accumulator that rebuilds text / reasoning / tool
// calls from the stream, and a decoder to neutral llm::ChatChunk.
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
#include "llm/sse.h"
namespace molo::openai::core {
enum class EventKind {
kLifecycle, kOutputItem, kContentPart, kText, kRefusal, kReasoning, kToolArguments,
kToolStatus, kAudio, kImage, kMcp, kCodeInterpreter, kShell, kAnnotation, kCompaction, kError,
kOther
};
struct EventInfo {
const char* type;
EventKind kind;
bool terminal; // the stream ends with this event
};
class ResponseEvents {
public:
static std::size_t count();
static const EventInfo& at(std::size_t index);
static const EventInfo* find(const std::string& type);
static bool known(const std::string& type) { return find(type) != nullptr; }
};
struct ResponseEvent {
std::string type;
long sequence = -1;
EventKind kind = EventKind::kOther;
bool known = false;
bool terminal = false;
clients::body::Json data;
static ResponseEvent from_sse(const llm::SseEvent& event);
// Convenience reads on the payload.
std::string text_delta() const; // output_text.delta / refusal.delta payload
const clients::body::Json* response() const; // the "response" object of lifecycle events
};
// Rebuilds the response pieces from events. Unknown event types are counted,
// never fatal (the platform adds events over time).
class ResponseAccumulator {
public:
void add(const ResponseEvent& event);
bool done() const { return done_; }
const std::string& status() const { return status_; } // completed, failed, incomplete, ...
const std::string& text() const { return text_; }
const std::string& refusal() const { return refusal_; }
const std::string& reasoning_summary() const { return reasoning_summary_; }
const std::vector<llm::ToolCall>& tool_calls() const { return calls_; }
const clients::body::Json& final_response() const { return final_; }
bool errored() const { return errored_; }
std::string error_message() const { return error_message_; }
std::size_t events() const { return events_; }
std::size_t unknown_events() const { return unknown_; }
std::map<std::string, std::size_t> counts() const { return counts_; }
private:
bool done_ = false;
bool errored_ = false;
std::string status_;
std::string text_;
std::string refusal_;
std::string reasoning_summary_;
std::string error_message_;
std::vector<llm::ToolCall> calls_;
clients::body::Json final_;
std::size_t events_ = 0;
std::size_t unknown_ = 0;
std::map<std::string, std::size_t> counts_;
};
// Responses events to neutral chunks. Stateful: remembers which output item
// is which function call so argument deltas carry the call id.
class ResponsesChunkDecoder {
public:
// Returns true and fills `chunk` when the event produces neutral output.
bool decode(const ResponseEvent& event, llm::ChatChunk& chunk);
private:
std::map<std::string, std::pair<std::string, std::string>> items_; // item id -> (call id, name)
};
} // namespace molo::openai::core
+367
View File
@@ -0,0 +1,367 @@
#include "openai/core/wire/tool_schemas.h"
#include "openai/core/caps.h"
#include "openai/core/check.h"
namespace molo::openai::core {
using clients::body::Json;
namespace {
Json typed(const std::string& type) {
Json o = Json::object();
o.set("type", Json(type));
return o;
}
void set_text(Json& o, const std::string& key, const std::string& v) {
if (!v.empty()) o.set(key, Json(v));
}
Json string_array(const std::vector<std::string>& v) {
Json a = Json::array();
for (const std::string& s : v) a.push(Json(s));
return a;
}
} // namespace
const std::vector<std::string>& ToolSchemas::response_tool_types() {
// TOOL-TYPES-BEGIN (checked against the spec's Tool union by tools/check_spec.py)
static const std::vector<std::string> kTypes = {
"function", "file_search", "computer", "computer_use_preview", "web_search",
"web_search_2025_08_26", "mcp", "code_interpreter", "programmatic_tool_calling",
"image_generation", "local_shell", "shell", "custom", "namespace", "tool_search",
"web_search_preview", "web_search_preview_2025_03_11", "apply_patch"};
// TOOL-TYPES-END
return kTypes;
}
const std::vector<std::string>& ToolSchemas::connector_ids() {
static const std::vector<std::string> kIds = {
"connector_dropbox", "connector_gmail", "connector_googlecalendar", "connector_googledrive",
"connector_microsoftteams", "connector_outlookcalendar", "connector_outlookemail",
"connector_sharepoint"};
return kIds;
}
bool ToolSchemas::is_response_tool_type(const std::string& type) {
return Check::contains(response_tool_types(), type);
}
void ToolSchemas::check_function_name(const std::string& name) {
Check::require(!name.empty() && name.size() <= 64, "tool name must be 1..64 characters");
for (char c : name)
Check::require((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') ||
c == '_' || c == '-',
"tool name may only use letters, digits, underscore and hyphen");
}
Json ToolSchemas::function(const std::string& name, const std::string& description,
const Json& parameters, bool strict) {
check_function_name(name);
Check::require(parameters.is_object() || parameters.is_null(), "parameters must be a JSON Schema object");
Json o = typed("function");
o.set("name", Json(name));
set_text(o, "description", description);
o.set("parameters", parameters.is_null() ? Json::object() : parameters);
o.set("strict", Json(strict));
return o;
}
Json ToolSchemas::text_format() { return typed("text"); }
Json ToolSchemas::grammar_format(const std::string& syntax, const std::string& definition) {
Check::in_set("grammar syntax", syntax, {"lark", "regex"});
Check::non_empty("grammar definition", definition);
Json o = typed("grammar");
o.set("syntax", Json(syntax));
o.set("definition", Json(definition));
return o;
}
Json ToolSchemas::custom(const std::string& name, const std::string& description,
const Json& format) {
check_function_name(name);
Json o = typed("custom");
o.set("name", Json(name));
set_text(o, "description", description);
if (!format.is_null()) {
Check::require(format.is_object(), "custom tool format must be an object");
o.set("format", format);
}
return o;
}
Json ToolSchemas::web_search(const WebSearchOptions& opt) {
Check::require(opt.type.rfind("web_search", 0) == 0, "web search tool type must start with web_search");
Check::require(is_response_tool_type(opt.type), "unknown web search tool type " + opt.type);
Check::in_set_or_empty("search_context_size", opt.search_context_size, {"low", "medium", "high"});
Json o = typed(opt.type);
set_text(o, "search_context_size", opt.search_context_size);
if (opt.external_web_access) o.set("external_web_access", Json(*opt.external_web_access));
if (opt.user_location) {
Json loc = typed("approximate");
set_text(loc, "country", opt.user_location->country);
set_text(loc, "region", opt.user_location->region);
set_text(loc, "city", opt.user_location->city);
set_text(loc, "timezone", opt.user_location->timezone);
o.set("user_location", loc);
}
if (!opt.allowed_domains.empty()) {
Check::require(opt.type.rfind("web_search_preview", 0) != 0,
"allowed_domains is not available on web_search_preview");
Json f = Json::object();
f.set("allowed_domains", string_array(opt.allowed_domains));
o.set("filters", f);
}
return o;
}
Json ToolSchemas::file_search(const std::vector<std::string>& vector_store_ids,
std::optional<int> max_num_results, const Json& filters,
const Json& ranking_options) {
Check::require(!vector_store_ids.empty(), "file_search needs at least one vector store id");
for (const std::string& id : vector_store_ids) Check::id("vector store", id);
Json o = typed("file_search");
o.set("vector_store_ids", string_array(vector_store_ids));
if (max_num_results) {
Check::range("max_num_results", *max_num_results, 1, 50);
o.set("max_num_results", Json(*max_num_results));
}
if (!filters.is_null()) o.set("filters", filters);
if (!ranking_options.is_null()) o.set("ranking_options", ranking_options);
return o;
}
Json ToolSchemas::code_interpreter_auto(const std::vector<std::string>& file_ids,
const std::string& memory_limit) {
Json c = typed("auto");
if (!file_ids.empty()) {
for (const std::string& id : file_ids) Check::id("file", id);
c.set("file_ids", string_array(file_ids));
}
set_text(c, "memory_limit", memory_limit);
Json o = typed("code_interpreter");
o.set("container", c);
return o;
}
Json ToolSchemas::code_interpreter_container(const std::string& container_id) {
Check::id("container", container_id);
Json o = typed("code_interpreter");
o.set("container", Json(container_id));
return o;
}
Json ToolSchemas::computer_use_preview(const std::string& environment, int display_width,
int display_height) {
Check::in_set("environment", environment, {"windows", "mac", "linux", "ubuntu", "browser"});
Check::range("display_width", display_width, 1, 16384);
Check::range("display_height", display_height, 1, 16384);
Json o = typed("computer_use_preview");
o.set("environment", Json(environment));
o.set("display_width", Json(display_width));
o.set("display_height", Json(display_height));
return o;
}
Json ToolSchemas::computer() { return typed("computer"); }
Json ToolSchemas::image_generation(const ImageGenOptions& opt) {
Check::in_set_or_empty("quality", opt.quality, {"low", "medium", "high", "xhigh", "max", "auto"});
Check::in_set_or_empty("output_format", opt.output_format, {"png", "webp", "jpeg"});
Check::in_set_or_empty("moderation", opt.moderation, {"auto", "low"});
Check::in_set_or_empty("background", opt.background, {"transparent", "opaque", "auto"});
Check::in_set_or_empty("input_fidelity", opt.input_fidelity, {"high", "low"});
Json o = typed("image_generation");
set_text(o, "model", opt.model);
set_text(o, "quality", opt.quality);
set_text(o, "size", opt.size);
set_text(o, "output_format", opt.output_format);
if (opt.output_compression) {
Check::range("output_compression", *opt.output_compression, 0, 100);
o.set("output_compression", Json(*opt.output_compression));
}
set_text(o, "moderation", opt.moderation);
set_text(o, "background", opt.background);
set_text(o, "input_fidelity", opt.input_fidelity);
if (opt.partial_images) {
Check::range("partial_images", *opt.partial_images, 0, CoreCaps::kMaxPartialImages);
o.set("partial_images", Json(*opt.partial_images));
}
set_text(o, "action", opt.action);
return o;
}
Json ToolSchemas::mcp(const McpOptions& opt) {
Check::non_empty("server_label", opt.server_label);
Check::require(opt.server_url.empty() != opt.connector_id.empty(),
"mcp needs exactly one of server_url and connector_id");
Check::in_set_or_empty("connector_id", opt.connector_id, connector_ids());
if (!opt.server_url.empty())
Check::require(opt.server_url.rfind("https://", 0) == 0 || opt.server_url.rfind("http://", 0) == 0,
"server_url must be an http(s) URL");
Check::in_set_or_empty("require_approval", opt.require_approval, {"always", "never"});
Json o = typed("mcp");
o.set("server_label", Json(opt.server_label));
set_text(o, "server_url", opt.server_url);
set_text(o, "connector_id", opt.connector_id);
set_text(o, "authorization", opt.authorization);
set_text(o, "server_description", opt.server_description);
if (!opt.headers.empty()) {
Json h = Json::object();
for (const auto& [k, v] : opt.headers) {
Check::no_control("header", k + v);
h.set(k, Json(v));
}
o.set("headers", h);
}
if (!opt.allowed_tools.empty()) o.set("allowed_tools", string_array(opt.allowed_tools));
set_text(o, "require_approval", opt.require_approval);
if (opt.defer_loading) o.set("defer_loading", Json(*opt.defer_loading));
return o;
}
Json ToolSchemas::local_shell() { return typed("local_shell"); }
Json ToolSchemas::shell(const Json& environment) {
Json o = typed("shell");
if (!environment.is_null()) o.set("environment", environment);
return o;
}
Json ToolSchemas::apply_patch() { return typed("apply_patch"); }
Json ToolSchemas::tool_search() { return typed("tool_search"); }
Json ToolSchemas::from_neutral(const llm::ToolSpec& spec, bool strict) {
return function(spec.name, spec.description, spec.parameters, strict);
}
Json ToolSchemas::chat_function(const std::string& name, const std::string& description,
const Json& parameters, bool strict) {
check_function_name(name);
Check::require(parameters.is_object() || parameters.is_null(), "parameters must be a JSON Schema object");
Json f = Json::object();
f.set("name", Json(name));
set_text(f, "description", description);
if (!parameters.is_null()) f.set("parameters", parameters);
if (strict) f.set("strict", Json(true));
Json o = typed("function");
o.set("function", f);
return o;
}
Json ToolSchemas::chat_custom(const std::string& name, const std::string& description,
const Json& format) {
check_function_name(name);
Json c = Json::object();
c.set("name", Json(name));
set_text(c, "description", description);
if (!format.is_null()) c.set("format", format);
Json o = typed("custom");
o.set("custom", c);
return o;
}
Json ToolSchemas::chat_from_neutral(const llm::ToolSpec& spec, bool strict) {
return chat_function(spec.name, spec.description, spec.parameters, strict);
}
Json ToolSchemas::choice_function(const std::string& name) {
check_function_name(name);
Json o = typed("function");
o.set("name", Json(name));
return o;
}
Json ToolSchemas::choice_custom(const std::string& name) {
check_function_name(name);
Json o = typed("custom");
o.set("name", Json(name));
return o;
}
Json ToolSchemas::choice_hosted(const std::string& type) {
Check::in_set("hosted tool choice", type,
{"file_search", "web_search_preview", "computer", "computer_use_preview",
"computer_use", "web_search_preview_2025_03_11", "image_generation",
"code_interpreter", "apply_patch", "shell", "programmatic_tool_calling"});
return typed(type);
}
Json ToolSchemas::choice_mcp(const std::string& server_label, const std::string& name) {
Check::non_empty("server_label", server_label);
Json o = typed("mcp");
o.set("server_label", Json(server_label));
set_text(o, "name", name);
return o;
}
Json ToolSchemas::choice_allowed(const std::string& mode, const std::vector<Json>& tools) {
Check::in_set("allowed_tools mode", mode, {"auto", "required"});
Check::require(!tools.empty(), "allowed_tools needs at least one tool");
Json o = typed("allowed_tools");
o.set("mode", Json(mode));
Json a = Json::array();
for (const Json& t : tools) a.push(t);
o.set("tools", a);
return o;
}
Json ToolSchemas::chat_choice_function(const std::string& name) {
check_function_name(name);
Json f = Json::object();
f.set("name", Json(name));
Json o = typed("function");
o.set("function", f);
return o;
}
Json ToolSchemas::chat_choice_custom(const std::string& name) {
check_function_name(name);
Json f = Json::object();
f.set("name", Json(name));
Json o = typed("custom");
o.set("custom", f);
return o;
}
Json ToolSchemas::choice_from_neutral(const std::string& choice, bool chat) {
if (choice.empty()) return Json();
if (choice == "auto" || choice == "none" || choice == "required") return Json(choice);
return chat ? chat_choice_function(choice) : choice_function(choice);
}
void ToolSchemas::validate_response_tools(const std::vector<Json>& tools) {
Check::max_count("tools", tools.size(), CoreCaps::kMaxTools);
for (const Json& t : tools) {
Check::require(t.is_object(), "every tool must be a JSON object");
const Json* type = t.find("type");
Check::require(type && type->is_string(), "every tool needs a type");
Check::require(is_response_tool_type(type->as_string()),
"unknown tool type " + type->as_string());
if (type->as_string() == "function" || type->as_string() == "custom") {
const Json* name = t.find("name");
Check::require(name && name->is_string(), "function and custom tools need a name");
check_function_name(name->as_string());
}
}
}
void ToolSchemas::validate_chat_tools(const std::vector<Json>& tools) {
Check::max_count("tools", tools.size(), CoreCaps::kMaxTools);
for (const Json& t : tools) {
Check::require(t.is_object(), "every tool must be a JSON object");
const Json* type = t.find("type");
Check::require(type && type->is_string() &&
(type->as_string() == "function" || type->as_string() == "custom"),
"Chat Completions tools are function or custom");
const Json* body = t.find(type->as_string());
Check::require(body && body->is_object(), "tool is missing its " + type->as_string() + " object");
const Json* name = body->find("name");
Check::require(name && name->is_string(), "tool needs a name");
check_function_name(name->as_string());
}
}
} // namespace molo::openai::core
+122
View File
@@ -0,0 +1,122 @@
#pragma once
// OpenAI tool schemas as JSON, for the Responses API (flat tools) and for
// Chat Completions (function / custom only). Reusable by any package that
// speaks OpenAI's wire format. Builders validate what the spec bounds and
// keep everything else raw.
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/json.h"
#include "llm/chat_types.h"
namespace molo::openai::core {
struct UserLocation {
std::string country; // ISO 3166-1 alpha-2
std::string region;
std::string city;
std::string timezone; // IANA
};
struct WebSearchOptions {
std::string type = "web_search"; // web_search, web_search_2025_08_26, web_search_preview, ...
std::string search_context_size; // "", low, medium, high
std::optional<UserLocation> user_location;
std::vector<std::string> allowed_domains; // filters.allowed_domains
std::optional<bool> external_web_access;
};
struct ImageGenOptions {
std::string model;
std::string quality; // low, medium, high, xhigh, max, auto
std::string size;
std::string output_format; // png, webp, jpeg
std::optional<int> output_compression; // 0..100
std::string moderation; // auto, low
std::string background; // transparent, opaque, auto
std::string input_fidelity;
std::optional<int> partial_images; // 0..3
std::string action;
};
struct McpOptions {
std::string server_label; // required
std::string server_url; // remote server, or
std::string connector_id; // a built-in connector (connector_dropbox, ...)
std::string authorization; // OAuth access token
std::string server_description;
std::map<std::string, std::string> headers;
std::vector<std::string> allowed_tools; // empty: all
std::string require_approval; // "", always, never
std::optional<bool> defer_loading;
};
class ToolSchemas {
public:
// Every tool type of the spec's Tool union (Responses API).
static const std::vector<std::string>& response_tool_types();
static const std::vector<std::string>& connector_ids();
static bool is_response_tool_type(const std::string& type);
// Responses API tools.
static clients::body::Json function(const std::string& name, const std::string& description,
const clients::body::Json& parameters, bool strict = true);
static clients::body::Json custom(const std::string& name, const std::string& description,
const clients::body::Json& format = clients::body::Json());
static clients::body::Json text_format();
static clients::body::Json grammar_format(const std::string& syntax,
const std::string& definition); // lark | regex
static clients::body::Json web_search(const WebSearchOptions& options = {});
static clients::body::Json file_search(const std::vector<std::string>& vector_store_ids,
std::optional<int> max_num_results = std::nullopt,
const clients::body::Json& filters = clients::body::Json(),
const clients::body::Json& ranking_options =
clients::body::Json());
static clients::body::Json code_interpreter_auto(const std::vector<std::string>& file_ids = {},
const std::string& memory_limit = "");
static clients::body::Json code_interpreter_container(const std::string& container_id);
static clients::body::Json computer_use_preview(const std::string& environment,
int display_width, int display_height);
static clients::body::Json computer();
static clients::body::Json image_generation(const ImageGenOptions& options = {});
static clients::body::Json mcp(const McpOptions& options);
static clients::body::Json local_shell();
static clients::body::Json shell(const clients::body::Json& environment = clients::body::Json());
static clients::body::Json apply_patch();
static clients::body::Json tool_search();
static clients::body::Json from_neutral(const llm::ToolSpec& spec, bool strict = true);
// Chat Completions tools.
static clients::body::Json chat_function(const std::string& name,
const std::string& description,
const clients::body::Json& parameters,
bool strict = false);
static clients::body::Json chat_custom(const std::string& name, const std::string& description,
const clients::body::Json& format = clients::body::Json());
static clients::body::Json chat_from_neutral(const llm::ToolSpec& spec, bool strict = false);
// tool_choice values. Responses: "auto" / "none" / "required" strings or
// these objects. Chat Completions nests the name under "function".
static clients::body::Json choice_function(const std::string& name);
static clients::body::Json choice_custom(const std::string& name);
static clients::body::Json choice_hosted(const std::string& type);
static clients::body::Json choice_mcp(const std::string& server_label,
const std::string& name = "");
static clients::body::Json choice_allowed(const std::string& mode,
const std::vector<clients::body::Json>& tools);
static clients::body::Json chat_choice_function(const std::string& name);
static clients::body::Json chat_choice_custom(const std::string& name);
// Neutral ChatRequest::tool_choice ("", auto, none, required, or a name).
static clients::body::Json choice_from_neutral(const std::string& choice, bool chat);
// Validation of a finished tool array.
static void validate_response_tools(const std::vector<clients::body::Json>& tools);
static void validate_chat_tools(const std::vector<clients::body::Json>& tools);
static void check_function_name(const std::string& name);
};
} // namespace molo::openai::core