Merge branch 'worktree-agent-a404596e1b4db6b5d'

This commit is contained in:
2026-10-25 15:24:28 +01:00
67 changed files with 6607 additions and 3 deletions
+274
View File
@@ -0,0 +1,274 @@
# Azure OpenAI and AI Foundry - Specification
Owned directory: `src/vendors/azure/`. Namespace `molo::azure`. Depends on
`molo::llm` (`ApiClient`, `Service`, `SseDecoder`, `ChatModel`, `Config`),
`molo::clients`, the WebSocket client of `src/google/live` (realtime) and the
OpenAI neutral wire mapping `openai::core::ChatWire` / `ResponsesWire` (the
chat model only: Azure OpenAI speaks the same shapes). Plan:
`plans/18-breadth.md` section 2, method of `plans/17-llm-vendor-platforms.md`.
Demo: `example/app/ai/AzureDemo.cpp`, routes `/ai/azure`.
## 1. Scope
One resource, three surfaces, one credential:
| Surface | Host (from the specs) | Prefix | Covers |
|---|---|---|---|
| OpenAI | `https://<resource>.openai.azure.com` | `/openai/v1` (v1 API) or `/openai` (dated API) | chat, completions, responses, embeddings, images (generate, edit, variation), audio (speech, transcription, translation), realtime, assistants / threads / messages / runs / steps, batch, fine-tuning (jobs, events, checkpoints, permissions, copy), files, uploads, vector stores, models, containers, conversations, evals, graders, videos |
| Foundry | `https://<resource>.services.ai.azure.com` | `/models` | Azure AI model inference for catalog models: chat completions, embeddings, image embeddings, model info |
| Safety | `https://<resource>.cognitiveservices.azure.com` | `/contentsafety` | Content Safety: analyze text / image, Prompt Shields, protected material, text blocklists |
Not covered, and said so: the Azure management plane (ARM: creating
resources, deployments, keys, quotas), Azure AI Agents / Projects
(`ai/data-plane/AIAgents`, `AIProjects`), Voice Live, Speech / Translator /
Language services. Deployments are therefore not listable here: `GET /models`
lists models, a deployment name must be known to the caller.
## 2. Addressing and api-version (the core rule)
`AZURE_OPENAI_API_VERSION` selects one of three modes (`ApiVersion`):
| Value | Mode | OpenAI-surface URL | Deployment goes |
|---|---|---|---|
| unset / `v1` | v1 | `<endpoint>/openai/v1/<path>` (no api-version query) | body `model` |
| `preview` | v1 preview | `<endpoint>/openai/v1/<path>?api-version=preview` | body `model` |
| `YYYY-MM-DD[-preview]` | dated | `<endpoint>/openai/<path>?api-version=<v>` | URL `/deployments/{d}/...` (8 operations), body `model` for the rest |
The 8 deployment-scoped operations are exactly the SDK list
(openai-python `lib/azure.py` `_deployments_endpoints`): completions, chat
completions, embeddings, audio transcriptions / translations / speech, image
generations / edits. The caller always passes the deployment once; the gateway
(`AzureService`) places it. A body `model` that differs from the deployment is
rejected; on the dated API it is removed.
An operation that exists on one API only is sent to that API with its
fallback version: dated-only operations (assistants, uploads, files import,
realtime sessions under `/realtimeapi`, fine-tuning job delete) under a v1
config use `2025-04-01-preview`; v1-only operations (containers,
conversations, evals, graders, videos, realtime client secrets and calls,
checkpoint permissions, image variations) under a dated config use
`/openai/v1` with `api-version=preview`. Foundry uses `2024-05-01-preview`,
Safety `2024-09-01` (both overridable in `Settings`, independent of the OpenAI
version).
## 3. Authentication
| Mode | Chosen when | Header |
|---|---|---|
| API key | `AZURE_OPENAI_API_KEY` set (wins) | `api-key: <key>` (OpenAI, Foundry); `Ocp-Apim-Subscription-Key: <key>` (Safety, key `AZURE_CONTENT_SAFETY_KEY` or the same key) |
| Microsoft Entra | tenant, client id and secret set, no key | `Authorization: Bearer <token>` on every surface |
Entra is its own class pair (`entra.h`): `EntraTokenSource` posts
`client_id`, `client_secret`, `scope`, `grant_type=client_credentials` (form
encoded) to `<authority>/<tenant>/oauth2/v2.0/token` (authority default
`https://login.microsoftonline.com`, `AZURE_AUTHORITY_HOST` for sovereign
clouds), scope `https://cognitiveservices.azure.com/.default` (the scope the
specs' OAuth2 schemes name), caches the token, refreshes it 300 s before
expiry, and after a failed exchange refuses (without network) for 5 s
(`TOKEN_BACKOFF`, a floor). `EntraTransport` decorates the transport so the
bearer reaches every request, including streams and the realtime handshake.
Secrets never appear in URLs, errors or the demo output (the demo's `entra`
route reports expiry only).
## 4. Module map (one concern per file)
| File | Concern |
|---|---|
| `caps.h` | Static ceilings |
| `api_version.*` | `ApiVersion`: v1 / preview / dated parsing |
| `settings.*` | `Settings`, `EntraConfig`: env, endpoint rules, auth kind |
| `calls.*`, `calls_table.inc` | The ONE call table (generated), `Calls::render/accepts_*` |
| `entra.*` | `EntraTokenSource`, `EntraTransport` |
| `client.*` | `AzureClient`: one `llm::ApiClient` per surface, shared credential |
| `service.*` | `AzureService` gateway: side choice, paths, field/query checks, JSON / multipart / raw / SSE, list walk |
| `check.*` | Validation helpers |
| `chat.*` | `ChatService`, `CompletionsService` |
| `responses.*` | `ResponsesService` |
| `embeddings.*` | `EmbeddingsService` |
| `images.*` | `ImagesService` |
| `audio.*` | `AudioService` |
| `realtime.*` | `RealtimeService` (REST), `RealtimeUrl`, `RealtimeSocket` (WebSocket) |
| `assistants.*` | `AssistantsService` |
| `batches.*`, `fine_tuning.*`, `files.*`, `vector_stores.*`, `models.*` | one service each |
| `foundry.*` | `FoundryService` (model inference) |
| `safety.*` | `SafetyService` (Content Safety) |
| `operations.*` | `OperationsService`: every table operation by name |
| `chat_model.*` | `AzureChatModel : llm::ChatModel` (dialects chat completions, responses, Foundry) |
| `upload_file.h` | `UploadFile` |
| `tools/specdoc.py`, `gen_calls.py`, `check_spec.py` | spec reader, table generator, checker |
## 5. Static caps (see caps.h)
| Knob | Value |
|---|---|
| Deployment name | 1..64 of `[A-Za-z0-9._-]` |
| Endpoint | 256 chars, scheme and host only |
| Page limit / page walk | 100 / 20 pages |
| Stream events | 200000 |
| Embedding inputs | 2048 |
| Audio file / image file / speech text / image prompt | 25 MiB / 50 MiB / 4096 / 32000 chars |
| Single file upload | 512 MiB (`llm::Caps`), upload part 64 MiB |
| Content Safety text / image / blocklist items / Shield documents | 10000 chars / 7.5 MiB / 100 / 5 |
| Entra token | refresh skew 300 s, failure floor 5 s, lifetime <= 24 h, answer <= 16 KiB |
| Realtime event / events per session / bytes / seconds | 1 MiB / 100000 / 64 MiB / 3600 |
| Client secret lifetime | 10..7200 s |
## 6. Contracts
- No endpoint, or neither key nor complete Entra credentials: every call
throws `NotConfiguredError` (503 `NOT_CONFIGURED`) before any network
traffic. A malformed `AZURE_*` value throws `ApiError::invalid` from
`Settings::from_env`.
- Every request field and query name must be listed by the spec of the API
side in use (`Calls::accepts_field/accepts_query`), else 400 before traffic.
Ids are single URL path segments (`Service::valid_id`).
- Destructive calls (delete, cancel, remove, hangup, reject, pause) are
explicit methods with no defaults; the demo asks for `confirm:true` for them
and for everything that costs money or changes a resource.
- Lists report truncation (`Listing::truncated`, `next_after`); a server that
says `has_more` without a usable cursor is reported truncated, not looped.
- Error bodies of the form `{"error":{"code","message"}}` keep the code as
`ApiError::type()`.
## 7. Sources
All machine readable, pinned to azure-rest-api-specs commit
`ba3eef39e5be55cd5ed1a2e5c6c9bb1a17c36b9a` (2026-10-23):
| Short name | Document |
|---|---|
| `v1`, `v1p` | `specification/ai/data-plane/OpenAI.v1/azure-v1-v1-generated.json`, `azure-v1-preview-generated.json` (OpenAPI 3, server `{endpoint}/openai/v1`) |
| `inf25` | `cognitiveservices/data-plane/OpenAIInference/preview/2025-04-01-preview/inference.json` |
| `auth24` | `cognitiveservices/data-plane/OpenAIAuthoring/stable/2024-10-21/azureopenai.json` (files, batches, fine-tuning, models, uploads) |
| `cs24` | `cognitiveservices/data-plane/ContentSafety/stable/2024-09-01/contentsafety.json` |
| `mi` | `ai/data-plane/ModelInference/preview/2024-05-01-preview/openapi.json` |
Cross-checks: openai-python `src/openai/lib/azure.py` (deployment endpoint
list, `api-key` / `Authorization: Bearer` headers, `/openai` and
`/openai/deployments/...` base URLs, `AZURE_OPENAI_ENDPOINT` /
`AZURE_OPENAI_API_KEY`; the SDK reads the version from `OPENAI_API_VERSION`,
this package from `AZURE_OPENAI_API_VERSION` as the plan names it); the specs'
own security schemes (header names, OAuth scope) and `info.version`; the
Microsoft identity platform client-credentials documentation (token request
and response bodies).
The older dated revisions (2022-12-01 ... 2025-03-01-preview, 2024-02-15
etc.) are not read: the table covers the newest stable authoring revision and
the newest preview inference revision; a field only older versions accept is
rejected by the gateway.
## 8. Verified against spec
`python3 -I src/vendors/azure/tools/check_spec.py` (2026-10-25):
`spec ok: 147 rows, 209 table sides verified, 315 spec operations seen in 6 documents`.
Per side it checks method, path, operationId, request content kind, the full
top-level request property list and the query list; it fails on any spec
operation of a covered document missing from the table, duplicate names, the
deployment-scoped set against the SDK list, the api-version constants against
each document's `info.version`, the v1 `api-version` enum (`v1`, `preview`),
the key header names and the Entra scope. The table is generated by
`tools/gen_calls.py` from the same documents (`calls_table.inc` is never
hand edited).
## 9. Verified live
Unauthenticated probes, 2026-10-25 (`MOLO_LIVE_NET=1 make ai-test`, suite
`azure_live`, passing):
- `POST https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token` with the
documented form and a client id that does not exist: HTTP 400,
`{"error":"unauthorized_client","error_description":"AADSTS700038: ...","error_codes":[700038],"trace_id":...}`.
`EntraTokenSource` maps it to `ApiError(400, "unauthorized_client")` with the
trace id as request id. The same body is the parse fixture in `test_entra.cpp`.
- `GET https://eastus.api.cognitive.microsoft.com/openai/v1/models` with a bad
`api-key`: HTTP 401 `{"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint..."}}`.
`POST .../contentsafety/text:analyze?api-version=2024-09-01` with a bad
`Ocp-Apim-Subscription-Key`: the same 401 body. So the v1 path, the
Content Safety path and both key headers are the right shape on the real host.
- `<name>.openai.azure.com`, `.services.ai.azure.com`, `.cognitiveservices.azure.com`
do not resolve for an unknown resource name (NXDOMAIN); a resource host
cannot be probed without a resource.
Opt-in authenticated live tests (`azure_live.models_list_and_one_tiny_completion`):
need `MOLO_LIVE_NET=1`, `AZURE_OPENAI_ENDPOINT` and a key or Entra triple;
models list is read-only; one smallest completion additionally needs
`MOLO_LIVE_PAID=1` and `AZURE_OPENAI_DEPLOYMENT`. Loud SKIP otherwise.
Nothing is created, updated or deleted.
## 10. Unverified
No Azure resource or key exists in this environment: no success response body
from any Azure host was seen. Everything below is read from specs, the SDK or
documentation but never confirmed against a live resource.
- Success shapes: chat completion, embeddings, models, responses and stream
parsing use the documented OpenAI-shaped examples (labelled in the tests).
- v1 `api-version`: omitted for `v1` per the Microsoft Learn v1 guide (the spec
says optional, default `v1`); whether a real resource accepts both is unseen.
- Multipart vs JSON: the v1 spec lists `application/json` for audio
transcriptions / translations and image edits / variations (generator
artefact: a `file` property typed `object`), and the dated 2025-04-01-preview
spec lists JSON for image edits. This package sends multipart (the dated
spec says multipart for transcription; every official SDK sends multipart
for these). Field names are checked against the spec lists.
- Assistants: the v1 documents contain threads, messages and runs but no
`/assistants` path; assistants are sent to the dated API (fallback version),
threads to the configured API. Microsoft has announced the retirement of the
Assistants API; whether a given resource still serves it is unseen.
- Realtime WebSocket URLs (`/openai/v1/realtime?model=`, dated
`/openai/realtime?api-version=&deployment=`) and the `api-key` /
Bearer handshake headers come from the SDK shape and Microsoft Learn, not
from the OpenAPI documents (WebSocket is not in them); the session over a
loopback server proves the client, not the service. The realtime event
vocabulary is not catalogued (events are JSON objects with a `type`).
- Foundry: `https://<resource>.services.ai.azure.com/models` host and the
bearer scope `https://cognitiveservices.azure.com/.default` for it (the spec
names this scope; the Learn page for Foundry models also mentions
`https://ai.azure.com/.default` for some flows). `AZURE_AI_ENDPOINT` and
`AZURE_CONTENT_SAFETY_ENDPOINT` default to the OpenAI endpoint, which only
works on a multi-service Azure AI Services resource that serves all three
hosts under one name. How `model` picks a deployment on a multi-model
endpoint is not described by the spec.
- Content Safety: Entra scope (spec), `application/merge-patch+json` for the
blocklist PATCH (spec `consumes`), response shapes of analyze / Shield.
- Entra: the `expires_in` number form (documented); sovereign authority hosts
(`login.microsoftonline.us`, `.cn`) via `AZURE_AUTHORITY_HOST` (documented
hosts, never called). Managed identity, certificates and workload identity
federation are not implemented.
- Rate limits, `Retry-After` and quota errors (429) are passed through as
`ApiError` without any retry (the `llm` foundation drops `Retry-After`).
## 11. Gaps
- No ARM (management plane): cannot list or create deployments, keys, quotas
(needs `management.azure.com` and a different token scope).
- No AAD token for Foundry-only resources via `https://ai.azure.com/.default`.
- Realtime has no event catalogue and no audio helpers (owner: this package,
`realtime.h`); the `openai/platform` catalogue could be reused once Azure
event differences are known.
- `llm::ApiClient` has no per-call timeout, no GET stream (response replay),
and no unauthenticated request (`src/llm/SPEC.md` section 4); the Entra
exchange therefore uses its own `clients::Client`.
- `clients::Url` percent-encodes `[` and `]` in query names (`include[]`
goes out as `include%5B%5D`); servers decode it, not proven on Azure.
- Batch input blobs (`input_blob`, `output_folder` of the v1 batch API) are
passed through as JSON fields only.
- `ListParams::before` exists, but the spec lists `before` for some lists only;
the gateway rejects it where the spec does not.
- Newer dated revisions than 2025-04-01-preview (if Microsoft ships them) need
`gen_calls.py` re-run; `check_spec.py` fails on drift of the pinned documents.
## 12. Credentials
| Variable | Meaning |
|---|---|
| `AZURE_OPENAI_ENDPOINT` | `https://<resource>.openai.azure.com` (scheme and host only) |
| `AZURE_OPENAI_API_KEY` | resource key (Azure portal, Keys and Endpoint) |
| `AZURE_OPENAI_API_VERSION` | `v1` (default), `preview`, or a dated version |
| `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET` | Entra app registration with the "Cognitive Services OpenAI User" (or Contributor) role on the resource |
| `AZURE_AI_ENDPOINT` | optional, Foundry model inference host |
| `AZURE_CONTENT_SAFETY_ENDPOINT`, `AZURE_CONTENT_SAFETY_KEY` | optional, Content Safety host and key |
| `AZURE_AUTHORITY_HOST` | optional, default `https://login.microsoftonline.com` |
| `AZURE_OPENAI_DEPLOYMENT` | live tests only: the deployment for the one paid completion |
Console: https://portal.azure.com (resource, Keys and Endpoint; Entra: App
registrations). Not read: the portal needs a sign-in.
+32
View File
@@ -0,0 +1,32 @@
#include "vendors/azure/api_version.h"
#include "llm/error.h"
namespace molo::azure {
bool ApiVersion::is_date(const std::string& t) {
auto digit = [&](std::size_t i) { return i < t.size() && t[i] >= '0' && t[i] <= '9'; };
if (t.size() < 10) return false;
for (std::size_t i : {0u, 1u, 2u, 3u, 5u, 6u, 8u, 9u})
if (!digit(i)) return false;
if (t[4] != '-' || t[7] != '-') return false;
if (t.size() == 10) return true;
return t.substr(10) == "-preview";
}
ApiVersion ApiVersion::parse(const std::string& text) {
ApiVersion v;
if (text.empty() || text == "v1") return v;
if (text == "preview") {
v.mode_ = Mode::kPreview;
v.value_ = "preview";
return v;
}
if (!is_date(text))
throw llm::ApiError::invalid("azure", "api version must be v1, preview or YYYY-MM-DD[-preview]");
v.mode_ = Mode::kDated;
v.value_ = text;
return v;
}
} // namespace molo::azure
+39
View File
@@ -0,0 +1,39 @@
#pragma once
// ApiVersion: the three ways an Azure OpenAI resource is addressed.
// v1 /openai/v1/... no api-version query (the GA v1 API)
// preview /openai/v1/... ?api-version=preview (v1 preview features)
// dated /openai/... ?api-version=YYYY-MM-DD[-preview], deployment URLs
// Env AZURE_OPENAI_API_VERSION selects one; unset means v1.
#include <string>
namespace molo::azure {
class ApiVersion {
public:
enum class Mode { kV1, kPreview, kDated };
static constexpr const char* kDatedFallback = "2025-04-01-preview"; // inference doc revision
static constexpr const char* kFoundryDefault = "2024-05-01-preview";
static constexpr const char* kSafetyDefault = "2024-09-01";
ApiVersion() = default;
// "", "v1", "preview", or a date; anything else throws ApiError::invalid.
static ApiVersion parse(const std::string& text);
// True for YYYY-MM-DD and YYYY-MM-DD-preview.
static bool is_date(const std::string& text);
Mode mode() const { return mode_; }
const std::string& value() const { return value_; }
bool is_dated() const { return mode_ == Mode::kDated; }
bool is_v1_family() const { return mode_ != Mode::kDated; }
// Query value to send on the v1 path: "" for v1 (omitted), "preview".
std::string v1_query() const { return mode_ == Mode::kPreview ? "preview" : ""; }
private:
Mode mode_ = Mode::kV1;
std::string value_ = "v1";
};
} // namespace molo::azure
+146
View File
@@ -0,0 +1,146 @@
#include "vendors/azure/assistants.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
namespace {
Query page(const ListParams& p) {
p.validate();
Query q;
p.apply(q);
return q;
}
} // namespace
Json AssistantsService::create_assistant(const Json& body) {
Check::has_string(body, "model");
Check::deployment(body.find("model")->as_string());
return call(Op::kCreateAssistant, {}, &body);
}
Json AssistantsService::get_assistant(const std::string& id) {
Check::id("assistant", id);
return call(Op::kGetAssistant, {"", {id}});
}
Json AssistantsService::modify_assistant(const std::string& id, const Json& body) {
Check::id("assistant", id);
return call(Op::kModifyAssistant, {"", {id}}, &body);
}
Json AssistantsService::list_assistants(const ListParams& params) {
Query q = page(params);
return call(Op::kListAssistants, {}, nullptr, &q);
}
Json AssistantsService::remove_assistant(const std::string& id) {
Check::id("assistant", id);
return call(Op::kDeleteAssistant, {"", {id}});
}
Json AssistantsService::create_thread(const Json& body) { return call(Op::kCreateThread, {}, &body); }
Json AssistantsService::get_thread(const std::string& id) {
Check::id("thread", id);
return call(Op::kRetrieveThread, {"", {id}});
}
Json AssistantsService::modify_thread(const std::string& id, const Json& body) {
Check::id("thread", id);
return call(Op::kModifyThread, {"", {id}}, &body);
}
Json AssistantsService::remove_thread(const std::string& id) {
Check::id("thread", id);
return call(Op::kDeleteThread, {"", {id}});
}
Json AssistantsService::create_thread_and_run(const Json& body) {
Check::has_string(body, "assistant_id");
return call(Op::kCreateThreadAndRun, {}, &body);
}
Json AssistantsService::create_message(const std::string& thread, const Json& body) {
Check::id("thread", thread);
Check::has_string(body, "role");
Check::has_field(body, "content");
return call(Op::kCreateMessage, {"", {thread}}, &body);
}
Json AssistantsService::get_message(const std::string& thread, const std::string& message) {
Check::id("thread", thread);
Check::id("message", message);
return call(Op::kRetrieveMessage, {"", {thread, message}});
}
Json AssistantsService::modify_message(const std::string& thread, const std::string& message,
const Json& body) {
Check::id("thread", thread);
Check::id("message", message);
return call(Op::kModifyMessage, {"", {thread, message}}, &body);
}
Json AssistantsService::list_messages(const std::string& thread, const ListParams& params) {
Check::id("thread", thread);
Query q = page(params);
return call(Op::kListMessages, {"", {thread}}, nullptr, &q);
}
Listing AssistantsService::list_messages_all(const std::string& thread, const ListParams& params,
std::size_t max_pages) {
Check::id("thread", thread);
return walk(Op::kListMessages, {"", {thread}}, params, Query(), max_pages);
}
Json AssistantsService::remove_message(const std::string& thread, const std::string& message) {
Check::id("thread", thread);
Check::id("message", message);
return call(Op::kDeleteMessage, {"", {thread, message}});
}
Json AssistantsService::create_run(const std::string& thread, const Json& body) {
Check::id("thread", thread);
Check::has_string(body, "assistant_id");
return call(Op::kCreateRun, {"", {thread}}, &body);
}
void AssistantsService::stream_run(const std::string& thread, const Json& body, const EventHandler& handler) {
Check::id("thread", thread);
Check::has_string(body, "assistant_id");
call_stream(Op::kCreateRun, {"", {thread}}, body, handler);
}
Json AssistantsService::get_run(const std::string& thread, const std::string& run) {
Check::id("thread", thread);
Check::id("run", run);
return call(Op::kRetrieveRun, {"", {thread, run}});
}
Json AssistantsService::modify_run(const std::string& thread, const std::string& run, const Json& body) {
Check::id("thread", thread);
Check::id("run", run);
return call(Op::kModifyRun, {"", {thread, run}}, &body);
}
Json AssistantsService::list_runs(const std::string& thread, const ListParams& params) {
Check::id("thread", thread);
Query q = page(params);
return call(Op::kListRuns, {"", {thread}}, nullptr, &q);
}
Json AssistantsService::cancel_run(const std::string& thread, const std::string& run) {
Check::id("thread", thread);
Check::id("run", run);
return call(Op::kCancelRun, {"", {thread, run}});
}
Json AssistantsService::submit_tool_outputs(const std::string& thread, const std::string& run,
const Json& tool_outputs) {
Check::id("thread", thread);
Check::id("run", run);
Check::require(tool_outputs.is_array() && !tool_outputs.as_array().empty(),
"tool_outputs must be a non-empty array");
Json body = Json::object();
body.set("tool_outputs", tool_outputs);
return call(Op::kSubmitToolOutputsToRun, {"", {thread, run}}, &body);
}
Json AssistantsService::list_run_steps(const std::string& thread, const std::string& run,
const ListParams& params) {
Check::id("thread", thread);
Check::id("run", run);
Query q = page(params);
return call(Op::kListRunSteps, {"", {thread, run}}, nullptr, &q);
}
Json AssistantsService::get_run_step(const std::string& thread, const std::string& run,
const std::string& step) {
Check::id("thread", thread);
Check::id("run", run);
Check::id("step", step);
return call(Op::kGetRunStep, {"", {thread, run, step}});
}
} // namespace molo::azure
+63
View File
@@ -0,0 +1,63 @@
#pragma once
// Assistants, threads, messages, runs, run steps (OpenAI Assistants API on
// Azure). The v1 documents have threads / runs but no /assistants path; when
// an operation exists on one API only, the gateway uses that API with its
// fallback api-version (dated: 2025-04-01-preview, v1: preview). Microsoft
// announced the retirement of this API in favour of Responses / Agents.
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class AssistantsService : public AzureService {
public:
using EventHandler = std::function<bool(const llm::SseEvent&)>; // false: stop
explicit AssistantsService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// Assistants. create needs `model` (the deployment name).
clients::body::Json create_assistant(const clients::body::Json& body);
clients::body::Json get_assistant(const std::string& id);
clients::body::Json modify_assistant(const std::string& id, const clients::body::Json& body);
clients::body::Json list_assistants(const ListParams& params = {});
clients::body::Json remove_assistant(const std::string& id); // destructive
// Threads.
clients::body::Json create_thread(const clients::body::Json& body = clients::body::Json::object());
clients::body::Json get_thread(const std::string& id);
clients::body::Json modify_thread(const std::string& id, const clients::body::Json& body);
clients::body::Json remove_thread(const std::string& id); // destructive
// Thread and run in one call; body needs assistant_id.
clients::body::Json create_thread_and_run(const clients::body::Json& body);
// Messages.
clients::body::Json create_message(const std::string& thread, const clients::body::Json& body);
clients::body::Json get_message(const std::string& thread, const std::string& message);
clients::body::Json modify_message(const std::string& thread, const std::string& message,
const clients::body::Json& body);
clients::body::Json list_messages(const std::string& thread, const ListParams& params = {});
Listing list_messages_all(const std::string& thread, const ListParams& params = {},
std::size_t max_pages = 5);
clients::body::Json remove_message(const std::string& thread, const std::string& message); // destructive
// Runs. body needs assistant_id.
clients::body::Json create_run(const std::string& thread, const clients::body::Json& body);
void stream_run(const std::string& thread, const clients::body::Json& body, const EventHandler& handler);
clients::body::Json get_run(const std::string& thread, const std::string& run);
clients::body::Json modify_run(const std::string& thread, const std::string& run,
const clients::body::Json& body);
clients::body::Json list_runs(const std::string& thread, const ListParams& params = {});
clients::body::Json cancel_run(const std::string& thread, const std::string& run);
clients::body::Json submit_tool_outputs(const std::string& thread, const std::string& run,
const clients::body::Json& tool_outputs);
clients::body::Json list_run_steps(const std::string& thread, const std::string& run,
const ListParams& params = {});
clients::body::Json get_run_step(const std::string& thread, const std::string& run,
const std::string& step);
};
} // namespace molo::azure
+49
View File
@@ -0,0 +1,49 @@
#include "vendors/azure/audio.h"
#include "vendors/azure/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
llm::RawResponse AudioService::speech(const std::string& deployment, const Json& body) {
Check::deployment(deployment);
Check::object("body", body);
Check::has_string(body, "input");
Check::has_string(body, "voice");
Check::max_chars("input", body.find("input")->as_string(), AzureCaps::kMaxSpeechChars);
return call_json_raw(Op::kCreateSpeech, {deployment, {}}, body);
}
llm::RawResponse AudioService::upload(Op op, const std::string& deployment, const UploadFile& file,
const std::map<std::string, std::string>& fields) {
Check::deployment(deployment);
Check::non_empty("audio filename", file.filename);
Check::no_control("filename", file.filename);
Check::require(!file.data.empty(), "audio file is empty");
Check::require(file.data.size() <= AzureCaps::kMaxAudioFileBytes, "audio file over 25 MiB");
clients::body::MultipartBody form;
form.add_file("file", file.filename, file.content_type, file.data);
for (const auto& [k, v] : fields) form.add_text(k, v);
return call_multipart_raw(op, {deployment, {}}, form);
}
llm::RawResponse AudioService::transcribe(const std::string& deployment, const UploadFile& file,
const std::map<std::string, std::string>& fields) {
return upload(Op::kCreateTranscription, deployment, file, fields);
}
llm::RawResponse AudioService::translate(const std::string& deployment, const UploadFile& file,
const std::map<std::string, std::string>& fields) {
return upload(Op::kCreateTranslation, deployment, file, fields);
}
Json AudioService::as_json(const llm::RawResponse& answer) {
if (answer.content_type.find("json") != std::string::npos) return Json::parse(answer.body);
Json o = Json::object();
o.set("text", Json(answer.body));
return o;
}
} // namespace molo::azure
+38
View File
@@ -0,0 +1,38 @@
#pragma once
// Audio on a deployment: speech (tts, tts-hd, gpt-4o-mini-tts), transcription
// (whisper, gpt-4o-transcribe) and translation. Transcription and translation
// are multipart on both APIs (the v1 document lists JSON for them, see SPEC
// Unverified). Answers may be text (srt, vtt, text), so they come back raw.
#include <map>
#include <string>
#include "vendors/azure/service.h"
#include "vendors/azure/upload_file.h"
namespace molo::azure {
class AudioService : public AzureService {
public:
explicit AudioService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: input (<= 4096 chars) and voice (required), plus response_format,
// speed, instructions. The answer is audio bytes (content_type tells which).
llm::RawResponse speech(const std::string& deployment, const clients::body::Json& body);
// fields: language, prompt, response_format, temperature, ... (spec names).
// The file is capped at 25 MiB.
llm::RawResponse transcribe(const std::string& deployment, const UploadFile& file,
const std::map<std::string, std::string>& fields = {});
llm::RawResponse translate(const std::string& deployment, const UploadFile& file,
const std::map<std::string, std::string>& fields = {});
// The answer parsed as JSON when its content type says so, else {"text": body}.
static clients::body::Json as_json(const llm::RawResponse& answer);
private:
llm::RawResponse upload(Op op, const std::string& deployment, const UploadFile& file,
const std::map<std::string, std::string>& fields);
};
} // namespace molo::azure
+38
View File
@@ -0,0 +1,38 @@
#include "vendors/azure/batches.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
Json BatchesService::create(const Json& body) {
Check::has_string(body, "endpoint");
Check::has_string(body, "completion_window");
Check::require(body.find("input_file_id") || body.find("input_blob"),
"input_file_id (or input_blob) is required");
return call(Op::kCreateBatch, {}, &body);
}
Json BatchesService::get(const std::string& id) {
Check::id("batch", id);
return call(Op::kRetrieveBatch, {"", {id}});
}
Json BatchesService::list(const ListParams& params) {
params.validate();
Query q;
params.apply(q);
return call(Op::kListBatches, {}, nullptr, &q);
}
Listing BatchesService::list_all(const ListParams& params, std::size_t max_pages) {
return walk(Op::kListBatches, {}, params, Query(), max_pages);
}
Json BatchesService::cancel(const std::string& id) {
Check::id("batch", id);
return call(Op::kCancelBatch, {"", {id}});
}
} // namespace molo::azure
+27
View File
@@ -0,0 +1,27 @@
#pragma once
// Batch (Global Batch): jobs over an uploaded JSONL file of requests.
// Creating a batch is paid work: the demo runs it as a dry run unless
// confirmed.
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class BatchesService : public AzureService {
public:
explicit BatchesService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: endpoint (/chat/completions, /embeddings, /v1/chat/completions ...),
// completion_window ("24h"), and input_file_id (or input_blob on the v1 API).
clients::body::Json create(const clients::body::Json& body);
clients::body::Json get(const std::string& id);
clients::body::Json list(const ListParams& params = {});
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
clients::body::Json cancel(const std::string& id); // destructive
};
} // namespace molo::azure
+114
View File
@@ -0,0 +1,114 @@
#include "vendors/azure/calls.h"
#include "llm/api_client.h"
#include "llm/error.h"
namespace molo::azure {
namespace {
const CallSpec kCalls[] = {
#define AZ_OP(name, surface, method, dep, d1, p1, o1, c1, b1, q1, d2, p2, o2, c2, b2, q2) \
{Op::k##name, #name, surface, method, dep != 0, {d1, p1, o1, c1, b1, q1}, {d2, p2, o2, c2, b2, q2}},
#include "vendors/azure/calls_table.inc"
#undef AZ_OP
};
constexpr std::size_t kCallCount = sizeof(kCalls) / sizeof(kCalls[0]);
bool listed(const std::string& csv, const std::string& item) {
std::size_t pos = 0;
while (pos <= csv.size()) {
std::size_t end = csv.find(',', pos);
if (end == std::string::npos) end = csv.size();
if (csv.compare(pos, end - pos, item) == 0 && end - pos == item.size()) return true;
pos = end + 1;
}
return false;
}
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
} // namespace
const CallSpec& Calls::spec(Op op) { return kCalls[static_cast<std::size_t>(op)]; }
const CallSpec& Calls::at(std::size_t index) { return kCalls[index]; }
std::size_t Calls::count() { return kCallCount; }
const CallSpec* Calls::find(const std::string& name) {
for (const CallSpec& c : kCalls)
if (name == c.name) return &c;
return nullptr;
}
clients::Method Calls::method(Op op) {
std::string m = spec(op).method;
if (m == "GET") return clients::Method::kGet;
if (m == "POST") return clients::Method::kPost;
if (m == "DELETE") return clients::Method::kDelete;
if (m == "PATCH") return clients::Method::kPatch;
if (m == "PUT") return clients::Method::kPut;
bad("unknown method in table: " + m);
}
bool Calls::accepts_field(Op op, Side side, const std::string& field) {
return listed(spec(op).side(side).body, field);
}
bool Calls::accepts_query(Op op, Side side, const std::string& name) {
std::string n = name;
if (n.size() > 2 && n.compare(n.size() - 2, 2, "[]") == 0) {
// The v1 spec names array parameters include[]; the dated one include.
if (listed(spec(op).side(side).query, n)) return true;
n.resize(n.size() - 2);
}
return listed(spec(op).side(side).query, n) || listed(spec(op).side(side).query, n + "[]");
}
std::size_t Calls::id_count(Op op, Side side) {
std::string path = spec(op).side(side).path;
std::size_t n = 0;
for (std::size_t i = 0; i < path.size(); ++i)
if (path[i] == '{') ++n;
if (side == Side::kDated && spec(op).deployment && n > 0) --n;
return n;
}
std::string Calls::render(Op op, Side side, const std::vector<std::string>& ids,
const std::string& deployment) {
const CallSpec& c = spec(op);
const SideSpec& s = c.side(side);
if (!s.present()) bad(std::string(c.name) + " is not available on this API side");
std::string path = s.path;
std::string out;
std::size_t used = 0;
bool first_placeholder = true;
for (std::size_t i = 0; i < path.size();) {
if (path[i] != '{') {
out += path[i++];
continue;
}
std::size_t close = path.find('}', i);
if (close == std::string::npos) bad("malformed table path");
if (side == Side::kDated && c.deployment && first_placeholder) {
if (!llm::Service::valid_id(deployment)) bad("bad deployment name");
out += deployment;
} else {
if (used >= ids.size()) bad(std::string(c.name) + " needs more ids");
if (!llm::Service::valid_id(ids[used])) bad("bad id");
out += ids[used++];
}
first_placeholder = false;
i = close + 1;
}
if (used != ids.size()) bad(std::string(c.name) + " got too many ids");
return out;
}
std::string Calls::prefix(Op op, Side side) {
switch (spec(op).surface) {
case Surface::kFoundry: return "/models";
case Surface::kSafety: return "/contentsafety";
case Surface::kOpenAi: break;
}
return side == Side::kV1 ? "/openai/v1" : "/openai";
}
} // namespace molo::azure
+76
View File
@@ -0,0 +1,76 @@
#pragma once
// The ONE table of REST operations this package makes (calls_table.inc,
// generated by tools/gen_calls.py from the pinned azure-rest-api-specs and
// verified by tools/check_spec.py). Code builds every path from this table.
//
// An OpenAI-surface operation has up to two sides: the v1 API (/openai/v1,
// document "v1" / "v1p") and the dated API (/openai, documents "inf25" /
// "auth24", deployment URLs). Foundry model inference and Content Safety are
// single-sided (the dated slot).
#include <cstddef>
#include <string>
#include <vector>
#include "clients/request.h"
namespace molo::azure {
enum class Surface { kOpenAi, kFoundry, kSafety };
enum class Side { kV1, kDated };
enum class Op {
#define AZ_OP(name, ...) k##name,
#include "vendors/azure/calls_table.inc"
#undef AZ_OP
};
struct SideSpec {
const char* doc; // spec document short name, "" when the side is absent
const char* path; // spec path under the server base
const char* op_id; // spec operationId ("" when the spec has none)
const char* content; // json, multipart, json+multipart, none (as the spec lists it)
const char* body; // every top-level request property, comma separated
const char* query; // query parameters except api-version, comma separated
bool present() const { return path[0] != '\0'; }
};
struct CallSpec {
Op op;
const char* name;
Surface surface;
const char* method;
bool deployment; // dated path is /deployments/{deployment-id}/...
SideSpec v1;
SideSpec dated;
const SideSpec& side(Side s) const { return s == Side::kV1 ? v1 : dated; }
};
class Calls {
public:
static const CallSpec& spec(Op op);
static const CallSpec& at(std::size_t index);
static std::size_t count();
// By table name ("CreateChatCompletion"); nullptr when unknown.
static const CallSpec* find(const std::string& name);
static clients::Method method(Op op);
static bool has(Op op, Side side) { return spec(op).side(side).present(); }
// True when the spec of that side lists `field` as a request property.
static bool accepts_field(Op op, Side side, const std::string& field);
// True when the spec lists `name` as a query parameter ("include[]" matches "include").
static bool accepts_query(Op op, Side side, const std::string& name);
// Number of {placeholders} the caller supplies as ids (the deployment of a
// dated deployment-scoped path is not one of them).
static std::size_t id_count(Op op, Side side);
// Spec path with ids substituted (each validated as one URL path segment),
// without the surface prefix and without the deployment segment.
// `deployment` is inserted for a dated deployment-scoped side.
static std::string render(Op op, Side side, const std::vector<std::string>& ids,
const std::string& deployment = "");
// Server base path of a side: "/openai/v1", "/openai", "/models", "/contentsafety".
static std::string prefix(Op op, Side side);
};
} // namespace molo::azure
+152
View File
@@ -0,0 +1,152 @@
// GENERATED by tools/gen_calls.py from azure-rest-api-specs ba3eef39e5be55cd5ed1a2e5c6c9bb1a17c36b9a. Do not edit.
// AZ_OP(name, surface, method, deployment_scoped, then twice (v1 side, dated side):
// doc, path, operation_id, content, body_fields, query_params)
// AZ-ROWS-BEGIN
AZ_OP(DeleteAssistant, Surface::kOpenAi, "DELETE", 0, "", "", "", "", "", "", "inf25", "/assistants/{assistant_id}", "Delete_Assistant", "none", "", "")
AZ_OP(DeleteContainer, Surface::kOpenAi, "DELETE", 0, "v1p", "/containers/{container_id}", "deleteContainer", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteContainerFile, Surface::kOpenAi, "DELETE", 0, "v1p", "/containers/{container_id}/files/{file_id}", "deleteContainerFile", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteConversation, Surface::kOpenAi, "DELETE", 0, "v1p", "/conversations/{conversation_id}", "deleteConversation", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteConversationItem, Surface::kOpenAi, "DELETE", 0, "v1p", "/conversations/{conversation_id}/items/{item_id}", "deleteConversationItem", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteEval, Surface::kOpenAi, "DELETE", 0, "v1p", "/evals/{eval_id}", "deleteEval", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteEvalRun, Surface::kOpenAi, "DELETE", 0, "v1p", "/evals/{eval_id}/runs/{run_id}", "deleteEvalRun", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteFile, Surface::kOpenAi, "DELETE", 0, "v1p", "/files/{file_id}", "deleteFile", "none", "", "", "auth24", "/files/{file-id}", "Files_Delete", "none", "", "")
AZ_OP(DeleteFineTuningCheckpointPermission, Surface::kOpenAi, "DELETE", 0, "v1p", "/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}", "deleteFineTuningCheckpointPermission", "none", "", "", "", "", "", "", "", "")
AZ_OP(FineTuningDelete, Surface::kOpenAi, "DELETE", 0, "", "", "", "", "", "", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}", "FineTuning_Delete", "none", "", "")
AZ_OP(DeleteModel, Surface::kOpenAi, "DELETE", 0, "v1p", "/models/{model}", "deleteModel", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteResponse, Surface::kOpenAi, "DELETE", 0, "v1p", "/responses/{response_id}", "deleteResponse", "none", "", "", "inf25", "/responses/{response_id}", "deleteResponse", "none", "", "")
AZ_OP(DeleteThread, Surface::kOpenAi, "DELETE", 0, "v1p", "/threads/{thread_id}", "deleteThread", "none", "", "", "inf25", "/threads/{thread_id}", "Delete_Thread", "none", "", "")
AZ_OP(DeleteMessage, Surface::kOpenAi, "DELETE", 0, "v1p", "/threads/{thread_id}/messages/{message_id}", "deleteMessage", "none", "", "", "", "", "", "", "", "")
AZ_OP(DeleteVectorStore, Surface::kOpenAi, "DELETE", 0, "v1p", "/vector_stores/{vector_store_id}", "deleteVectorStore", "none", "", "", "inf25", "/vector_stores/{vector_store_id}", "Delete_Vector_Store", "none", "", "")
AZ_OP(DeleteVectorStoreFile, Surface::kOpenAi, "DELETE", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}", "deleteVectorStoreFile", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}", "Delete_Vector_Store_File", "none", "", "")
AZ_OP(VideosDelete, Surface::kOpenAi, "DELETE", 0, "v1p", "/videos/{video-id}", "Videos_Delete", "none", "", "", "", "", "", "", "", "")
AZ_OP(ListAssistants, Surface::kOpenAi, "GET", 0, "", "", "", "", "", "", "inf25", "/assistants", "List_Assistants", "none", "", "after,before,limit,order")
AZ_OP(GetAssistant, Surface::kOpenAi, "GET", 0, "", "", "", "", "", "", "inf25", "/assistants/{assistant_id}", "Get_Assistant", "none", "", "")
AZ_OP(ListBatches, Surface::kOpenAi, "GET", 0, "v1p", "/batches", "listBatches", "none", "", "$filter,$orderby,after,limit", "auth24", "/batches", "Batch_List", "none", "", "after,limit")
AZ_OP(RetrieveBatch, Surface::kOpenAi, "GET", 0, "v1p", "/batches/{batch_id}", "retrieveBatch", "none", "", "", "auth24", "/batches/{batch-id}", "Batch_Get", "none", "", "")
AZ_OP(ListContainers, Surface::kOpenAi, "GET", 0, "v1p", "/containers", "listContainers", "none", "", "after,before,limit,order", "", "", "", "", "", "")
AZ_OP(RetrieveContainer, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}", "retrieveContainer", "none", "", "", "", "", "", "", "", "")
AZ_OP(ListContainerFiles, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}/files", "listContainerFiles", "none", "", "after,before,limit,order", "", "", "", "", "", "")
AZ_OP(RetrieveContainerFile, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}/files/{file_id}", "retrieveContainerFile", "none", "", "", "", "", "", "", "", "")
AZ_OP(RetrieveContainerFileContent, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}/files/{file_id}/content", "retrieveContainerFileContent", "none", "", "", "", "", "", "", "", "")
AZ_OP(RetrieveConversation, Surface::kOpenAi, "GET", 0, "v1p", "/conversations/{conversation_id}", "retrieveConversation", "none", "", "", "", "", "", "", "", "")
AZ_OP(ListConversationItems, Surface::kOpenAi, "GET", 0, "v1p", "/conversations/{conversation_id}/items", "listConversationItems", "none", "", "after,include,limit,order", "", "", "", "", "", "")
AZ_OP(RetrieveConversationItem, Surface::kOpenAi, "GET", 0, "v1p", "/conversations/{conversation_id}/items/{item_id}", "retrieveConversationItem", "none", "", "include", "", "", "", "", "", "")
AZ_OP(ListEvals, Surface::kOpenAi, "GET", 0, "v1p", "/evals", "listEvals", "none", "", "after,limit,order,order_by", "", "", "", "", "", "")
AZ_OP(GetEval, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}", "getEval", "none", "", "", "", "", "", "", "", "")
AZ_OP(GetEvalRuns, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs", "getEvalRuns", "none", "", "after,limit,order,status", "", "", "", "", "", "")
AZ_OP(GetEvalRun, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs/{run_id}", "getEvalRun", "none", "", "", "", "", "", "", "", "")
AZ_OP(GetEvalRunOutputItems, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs/{run_id}/output_items", "getEvalRunOutputItems", "none", "", "after,limit,order,status", "", "", "", "", "", "")
AZ_OP(GetEvalRunOutputItem, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}", "getEvalRunOutputItem", "none", "", "", "", "", "", "", "", "")
AZ_OP(ListFiles, Surface::kOpenAi, "GET", 0, "v1p", "/files", "listFiles", "none", "", "after,limit,order,purpose", "auth24", "/files", "Files_List", "none", "", "purpose")
AZ_OP(RetrieveFile, Surface::kOpenAi, "GET", 0, "v1p", "/files/{file_id}", "retrieveFile", "none", "", "", "auth24", "/files/{file-id}", "Files_Get", "none", "", "")
AZ_OP(DownloadFile, Surface::kOpenAi, "GET", 0, "v1p", "/files/{file_id}/content", "downloadFile", "none", "", "", "auth24", "/files/{file-id}/content", "Files_GetContent", "none", "", "")
AZ_OP(ListFineTuningCheckpointPermissions, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions", "listFineTuningCheckpointPermissions", "none", "", "after,limit,order,project_id", "", "", "", "", "", "")
AZ_OP(ListPaginatedFineTuningJobs, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs", "listPaginatedFineTuningJobs", "none", "", "after,limit", "auth24", "/fine_tuning/jobs", "FineTuning_List", "none", "", "after,limit")
AZ_OP(RetrieveFineTuningJob, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}", "retrieveFineTuningJob", "none", "", "", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}", "FineTuning_Get", "none", "", "")
AZ_OP(ListFineTuningJobCheckpoints, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints", "listFineTuningJobCheckpoints", "none", "", "after,limit", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}/checkpoints", "FineTuning_GetCheckpoints", "none", "", "after,limit")
AZ_OP(FineTuningGetCheckpoint, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints/{fine_tuning_checkpoint_id}/copy", "FineTuning_GetCheckpoint", "none", "", "", "", "", "", "", "", "")
AZ_OP(ListFineTuningEvents, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/events", "listFineTuningEvents", "none", "", "after,limit", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}/events", "FineTuning_GetEvents", "none", "", "after,limit")
AZ_OP(ListModels, Surface::kOpenAi, "GET", 0, "v1p", "/models", "listModels", "none", "", "", "auth24", "/models", "Models_List", "none", "", "")
AZ_OP(RetrieveModel, Surface::kOpenAi, "GET", 0, "v1p", "/models/{model}", "retrieveModel", "none", "", "", "auth24", "/models/{model-id}", "Models_Get", "none", "", "")
AZ_OP(GetResponse, Surface::kOpenAi, "GET", 0, "v1p", "/responses/{response_id}", "getResponse", "none", "", "include[],include_obfuscation,starting_after,stream", "inf25", "/responses/{response_id}", "", "none", "", "include")
AZ_OP(ListInputItems, Surface::kOpenAi, "GET", 0, "v1p", "/responses/{response_id}/input_items", "listInputItems", "none", "", "after,before,limit,order", "inf25", "/responses/{response_id}/input_items", "", "none", "", "after,before,limit,order")
AZ_OP(RetrieveThread, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}", "retrieveThread", "none", "", "", "inf25", "/threads/{thread_id}", "Get_Thread", "none", "", "")
AZ_OP(ListMessages, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/messages", "listMessages", "none", "", "after,before,limit,order,run_id", "inf25", "/threads/{thread_id}/messages", "List_Messages", "none", "", "after,before,limit,order,run_id")
AZ_OP(RetrieveMessage, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/messages/{message_id}", "retrieveMessage", "none", "", "", "inf25", "/threads/{thread_id}/messages/{message_id}", "Get_Message", "none", "", "")
AZ_OP(ListRuns, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs", "listRuns", "none", "", "after,before,limit,order", "inf25", "/threads/{thread_id}/runs", "List_Runs", "none", "", "after,before,limit,order")
AZ_OP(RetrieveRun, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs/{run_id}", "retrieveRun", "none", "", "", "inf25", "/threads/{thread_id}/runs/{run_id}", "Get_Run", "none", "", "")
AZ_OP(ListRunSteps, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/steps", "listRunSteps", "none", "", "after,before,limit,order", "inf25", "/threads/{thread_id}/runs/{run_id}/steps", "List_Run_Steps", "none", "", "after,before,include[],limit,order")
AZ_OP(GetRunStep, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/steps/{step_id}", "getRunStep", "none", "", "", "inf25", "/threads/{thread_id}/runs/{run_id}/steps/{step_id}", "Get_Run_Step", "none", "", "include[]")
AZ_OP(ListVectorStores, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores", "listVectorStores", "none", "", "after,before,limit,order", "inf25", "/vector_stores", "List_Vector_Stores", "none", "", "after,before,limit,order")
AZ_OP(GetVectorStore, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}", "getVectorStore", "none", "", "", "inf25", "/vector_stores/{vector_store_id}", "Get_Vector_Store", "none", "", "")
AZ_OP(GetVectorStoreFileBatch, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches/{batch_id}", "getVectorStoreFileBatch", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/file_batches/{batch_id}", "Get_Vector_Store_File_Batch", "none", "", "")
AZ_OP(ListFilesInVectorStoreBatch, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/files", "listFilesInVectorStoreBatch", "none", "", "after,before,filter,limit,order", "inf25", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/files", "List_Vector_Store_File_Batch_Files", "none", "", "after,before,filter,limit,order")
AZ_OP(ListVectorStoreFiles, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/files", "listVectorStoreFiles", "none", "", "after,before,filter,limit,order", "inf25", "/vector_stores/{vector_store_id}/files", "List_Vector_Store_Files", "none", "", "after,before,filter,limit,order")
AZ_OP(GetVectorStoreFile, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}", "getVectorStoreFile", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}", "Get_Vector_Store_File", "none", "", "")
AZ_OP(RetrieveVectorStoreFileContent, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}/content", "retrieveVectorStoreFileContent", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}/content", "retrieveVectorStoreFileContent", "none", "", "")
AZ_OP(VideosList, Surface::kOpenAi, "GET", 0, "v1p", "/videos", "Videos_List", "none", "", "after,limit,order", "", "", "", "", "", "")
AZ_OP(VideosGet, Surface::kOpenAi, "GET", 0, "v1p", "/videos/{video-id}", "Videos_Get", "none", "", "", "", "", "", "", "", "")
AZ_OP(VideosRetrieveContent, Surface::kOpenAi, "GET", 0, "v1p", "/videos/{video-id}/content", "Videos_RetrieveContent", "none", "", "variant", "", "", "", "", "", "")
AZ_OP(CreateAssistant, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/assistants", "Create_Assistant", "json", "description,instructions,metadata,model,name,response_format,temperature,tool_resources,tools,top_p", "")
AZ_OP(ModifyAssistant, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/assistants/{assistant_id}", "Modify_Assistant", "json", "description,instructions,metadata,model,name,response_format,temperature,tool_resources,tools,top_p", "")
AZ_OP(CreateSpeech, Surface::kOpenAi, "POST", 1, "v1p", "/audio/speech", "createSpeech", "json", "input,instructions,model,response_format,speed,stream_format,voice", "", "inf25", "/deployments/{deployment-id}/audio/speech", "Speech_Create", "multipart", "input,response_format,speed,voice", "")
AZ_OP(CreateTranscription, Surface::kOpenAi, "POST", 1, "v1p", "/audio/transcriptions", "createTranscription", "json", "chunking_strategy,file,filename,include,known_speaker_names,known_speaker_references,language,model,prompt,response_format,stream,temperature,timestamp_granularities", "", "inf25", "/deployments/{deployment-id}/audio/transcriptions", "Transcriptions_Create", "multipart", "file,language,prompt,response_format,temperature,timestamp_granularities[]", "")
AZ_OP(CreateTranslation, Surface::kOpenAi, "POST", 1, "v1p", "/audio/translations", "createTranslation", "json", "file,filename,model,prompt,response_format,temperature", "", "inf25", "/deployments/{deployment-id}/audio/translations", "Translations_Create", "multipart", "file,prompt,response_format,temperature", "")
AZ_OP(CreateBatch, Surface::kOpenAi, "POST", 0, "v1p", "/batches", "createBatch", "json", "completion_window,endpoint,input_blob,input_file_id,output_expires_after,output_folder", "", "auth24", "/batches", "Batch_Create", "json", "completion_window,endpoint,input_file_id,metadata", "")
AZ_OP(CancelBatch, Surface::kOpenAi, "POST", 0, "v1p", "/batches/{batch_id}/cancel", "cancelBatch", "none", "", "", "auth24", "/batches/{batch-id}/cancel", "Batch_Cancel", "none", "", "")
AZ_OP(CreateChatCompletion, Surface::kOpenAi, "POST", 1, "v1p", "/chat/completions", "createChatCompletion", "json", "audio,frequency_penalty,function_call,functions,logit_bias,logprobs,max_completion_tokens,max_tokens,messages,metadata,modalities,model,n,parallel_tool_calls,prediction,presence_penalty,prompt_cache_key,prompt_cache_retention,reasoning_effort,response_format,safety_identifier,seed,stop,store,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user,user_security_context,verbosity", "", "inf25", "/deployments/{deployment-id}/chat/completions", "ChatCompletions_Create", "json", "audio,data_sources,frequency_penalty,function_call,functions,logit_bias,logprobs,max_completion_tokens,max_tokens,messages,metadata,modalities,n,parallel_tool_calls,prediction,presence_penalty,reasoning_effort,response_format,seed,stop,store,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user,user_security_context", "")
AZ_OP(CreateCompletion, Surface::kOpenAi, "POST", 1, "v1p", "/completions", "createCompletion", "json", "best_of,echo,frequency_penalty,logit_bias,logprobs,max_tokens,model,n,presence_penalty,prompt,seed,stop,stream,stream_options,suffix,temperature,top_p,user", "", "inf25", "/deployments/{deployment-id}/completions", "Completions_Create", "json", "best_of,echo,frequency_penalty,logit_bias,logprobs,max_tokens,n,presence_penalty,prompt,seed,stop,stream,suffix,temperature,top_p,user", "")
AZ_OP(CreateContainer, Surface::kOpenAi, "POST", 0, "v1p", "/containers", "createContainer", "json", "expires_after,file_ids,memory_limit,name,network_policy,skills", "", "", "", "", "", "", "")
AZ_OP(CreateContainerFile, Surface::kOpenAi, "POST", 0, "v1p", "/containers/{container_id}/files", "createContainerFile", "multipart", "file,file_id", "", "", "", "", "", "", "")
AZ_OP(CreateConversation, Surface::kOpenAi, "POST", 0, "v1p", "/conversations", "createConversation", "json", "items,metadata", "", "", "", "", "", "", "")
AZ_OP(UpdateConversation, Surface::kOpenAi, "POST", 0, "v1p", "/conversations/{conversation_id}", "updateConversation", "json", "metadata", "", "", "", "", "", "", "")
AZ_OP(CreateConversationItems, Surface::kOpenAi, "POST", 0, "v1p", "/conversations/{conversation_id}/items", "createConversationItems", "json", "items", "include", "", "", "", "", "", "")
AZ_OP(CreateEmbedding, Surface::kOpenAi, "POST", 1, "v1p", "/embeddings", "createEmbedding", "json", "dimensions,encoding_format,input,model,user", "", "inf25", "/deployments/{deployment-id}/embeddings", "embeddings_create", "json", "dimensions,encoding_format,input,input_type,user", "")
AZ_OP(CreateEval, Surface::kOpenAi, "POST", 0, "v1p", "/evals", "createEval", "json", "data_source_config,metadata,name,statusCode,testing_criteria", "", "", "", "", "", "", "")
AZ_OP(UpdateEval, Surface::kOpenAi, "POST", 0, "v1p", "/evals/{eval_id}", "updateEval", "json", "metadata,name", "", "", "", "", "", "", "")
AZ_OP(CreateEvalRun, Surface::kOpenAi, "POST", 0, "v1p", "/evals/{eval_id}/runs", "createEvalRun", "json", "data_source,metadata,name", "", "", "", "", "", "", "")
AZ_OP(CancelEvalRun, Surface::kOpenAi, "POST", 0, "v1p", "/evals/{eval_id}/runs/{run_id}", "cancelEvalRun", "none", "", "", "", "", "", "", "", "")
AZ_OP(CreateFile, Surface::kOpenAi, "POST", 0, "v1p", "/files", "createFile", "multipart", "expires_after,file,purpose", "", "auth24", "/files", "Files_Upload", "multipart", "file,purpose", "")
AZ_OP(FilesImport, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/files/import", "Files_Import", "json", "content_url,filename,purpose", "")
AZ_OP(RunGrader, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/alpha/graders/run", "runGrader", "json", "grader,item,model_sample", "", "", "", "", "", "", "")
AZ_OP(ValidateGrader, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/alpha/graders/validate", "validateGrader", "json", "grader", "", "", "", "", "", "", "")
AZ_OP(CreateFineTuningCheckpointPermission, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions", "createFineTuningCheckpointPermission", "json", "project_ids", "", "", "", "", "", "", "")
AZ_OP(CreateFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs", "createFineTuningJob", "json", "hyperparameters,integrations,metadata,method,model,seed,suffix,training_file,validation_file", "", "auth24", "/fine_tuning/jobs", "FineTuning_Create", "json", "hyperparameters,model,seed,suffix,training_file,validation_file", "")
AZ_OP(CancelFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/cancel", "cancelFineTuningJob", "none", "", "", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}/cancel", "FineTuning_Cancel", "none", "", "")
AZ_OP(FineTuningCopyCheckpoint, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints/{fine_tuning_checkpoint_id}/copy", "FineTuning_CopyCheckpoint", "json", "destinationResourceId,region", "", "", "", "", "", "", "")
AZ_OP(PauseFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/pause", "pauseFineTuningJob", "none", "", "", "", "", "", "", "", "")
AZ_OP(ResumeFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/resume", "resumeFineTuningJob", "none", "", "", "", "", "", "", "", "")
AZ_OP(CreateImageEdit, Surface::kOpenAi, "POST", 1, "v1p", "/images/edits", "createImageEdit", "json", "background,image,input_fidelity,mask,model,n,output_compression,output_format,partial_images,prompt,quality,response_format,size,stream,user", "", "inf25", "/deployments/{deployment-id}/images/edits", "ImageGenerations_Edit", "json", "image,mask,n,prompt,quality,response_format,size,user", "")
AZ_OP(CreateImage, Surface::kOpenAi, "POST", 1, "v1p", "/images/generations", "createImage", "json", "background,model,moderation,n,output_compression,output_format,partial_images,prompt,quality,response_format,size,stream,style,user", "", "inf25", "/deployments/{deployment-id}/images/generations", "ImageGenerations_Create", "json", "background,n,output_compression,output_format,prompt,quality,response_format,size,style,user", "")
AZ_OP(CreateImageVariation, Surface::kOpenAi, "POST", 0, "v1p", "/images/variations", "createImageVariation", "json", "image,model,n,response_format,size,user", "", "", "", "", "", "", "")
AZ_OP(CreateRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls", "createRealtimeCall", "multipart", "sdp,session", "", "", "", "", "", "", "")
AZ_OP(AcceptRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/accept", "acceptRealtimeCall", "json", "audio,include,instructions,max_output_tokens,model,output_modalities,parallel_tool_calls,prompt,reasoning,tool_choice,tools,tracing,truncation,type", "", "", "", "", "", "", "")
AZ_OP(HangupRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/hangup", "hangupRealtimeCall", "none", "", "", "", "", "", "", "", "")
AZ_OP(ReferRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/refer", "referRealtimeCall", "json", "target_uri", "", "", "", "", "", "", "")
AZ_OP(RejectRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/reject", "rejectRealtimeCall", "json", "status_code", "", "", "", "", "", "", "")
AZ_OP(CreateRealtimeClientSecret, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/client_secrets", "createRealtimeClientSecret", "json", "expires_after,session", "", "", "", "", "", "", "")
AZ_OP(CreateRealtimeSession, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/sessions", "createRealtimeSession", "json", "client_secret,input_audio_format,input_audio_transcription,instructions,max_response_output_tokens,modalities,output_audio_format,prompt,speed,temperature,tool_choice,tools,tracing,truncation,turn_detection,voice", "", "", "", "", "", "", "")
AZ_OP(CreateRealtimeTranscriptionSession, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/transcription_sessions", "createRealtimeTranscriptionSession", "json", "include,input_audio_format,input_audio_noise_reduction,input_audio_transcription,turn_detection", "", "", "", "", "", "", "")
AZ_OP(CreateRealtimeSessionDated, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/realtimeapi/sessions", "create_realtimeSession", "json", "input_audio_format,input_audio_noise_reduction,input_audio_transcription,instructions,max_response_output_tokens,modalities,model,output_audio_format,temperature,tool_choice,tools,turn_detection,voice", "")
AZ_OP(CreateTranscriptionRealtimeSession, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/realtimeapi/transcription_sessions", "create_transcriptionRealtimeSession", "json", "include,input_audio_format,input_audio_noise_reduction,input_audio_transcription,modalities,turn_detection", "")
AZ_OP(CreateResponse, Surface::kOpenAi, "POST", 0, "v1p", "/responses", "createResponse", "json", "background,context_management,conversation,include,input,instructions,max_output_tokens,max_tool_calls,metadata,model,parallel_tool_calls,previous_response_id,prompt,prompt_cache_key,prompt_cache_retention,reasoning,safety_identifier,store,stream,stream_options,temperature,text,tool_choice,tools,top_logprobs,top_p,truncation,user", "", "inf25", "/responses", "", "json", "include,input,instructions,max_output_tokens,metadata,model,parallel_tool_calls,previous_response_id,reasoning,store,stream,temperature,text,tool_choice,tools,top_p,truncation,user", "")
AZ_OP(CancelResponse, Surface::kOpenAi, "POST", 0, "v1p", "/responses/{response_id}/cancel", "cancelResponse", "none", "", "", "", "", "", "", "", "")
AZ_OP(CreateThread, Surface::kOpenAi, "POST", 0, "v1p", "/threads", "createThread", "json", "messages,metadata,tool_resources", "", "inf25", "/threads", "Create_Thread", "json", "messages,metadata,tool_resources", "")
AZ_OP(CreateThreadAndRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/runs", "createThreadAndRun", "json", "assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,response_format,stream,temperature,thread,tool_choice,tool_resources,tools,top_p,truncation_strategy", "", "inf25", "/threads/runs", "Create_Thread_And_Run", "json", "assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,response_format,stream,stream_options,temperature,thread,tool_choice,tool_resources,tools,top_p,truncation_strategy", "")
AZ_OP(ModifyThread, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}", "modifyThread", "json", "metadata,tool_resources", "", "inf25", "/threads/{thread_id}", "Modify_Thread", "json", "metadata,tool_resources", "")
AZ_OP(CreateMessage, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/messages", "createMessage", "json", "attachments,content,metadata,role", "", "inf25", "/threads/{thread_id}/messages", "Create_Message", "json", "attachments,content,metadata,role", "")
AZ_OP(ModifyMessage, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/messages/{message_id}", "modifyMessage", "json", "metadata", "", "inf25", "/threads/{thread_id}/messages/{message_id}", "Modify_Message", "json", "metadata", "")
AZ_OP(CreateRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs", "createRun", "json", "additional_instructions,additional_messages,assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,reasoning_effort,response_format,stream,temperature,tool_choice,tools,top_p,truncation_strategy", "", "inf25", "/threads/{thread_id}/runs", "Create_Run", "json", "additional_instructions,additional_messages,assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,response_format,stream,temperature,tool_choice,tools,top_p,truncation_strategy", "include[]")
AZ_OP(ModifyRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs/{run_id}", "modifyRun", "json", "metadata", "", "inf25", "/threads/{thread_id}/runs/{run_id}", "Modify_Run", "json", "metadata", "")
AZ_OP(CancelRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/cancel", "cancelRun", "none", "", "", "inf25", "/threads/{thread_id}/runs/{run_id}/cancel", "Cancel_Run", "none", "", "")
AZ_OP(SubmitToolOutputsToRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/submit_tool_outputs", "submitToolOutputsToRun", "json", "stream,tool_outputs", "", "inf25", "/threads/{thread_id}/runs/{run_id}/submit_tool_outputs", "Submit_Tool_Outputs_To_Run", "json", "stream,tool_outputs", "")
AZ_OP(UploadFileStart, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads", "UploadFile_Start", "json", "bytes,filename,mime_type,purpose", "")
AZ_OP(UploadFileCancel, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads/{upload-id}/cancel", "UploadFile_Cancel", "none", "", "")
AZ_OP(UploadFileComplete, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads/{upload-id}/complete", "UploadFile_Complete", "json", "md5,part_ids", "")
AZ_OP(UploadFilePart, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads/{upload-id}/parts", "UploadFile_Part", "multipart", "data", "")
AZ_OP(CreateVectorStore, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores", "createVectorStore", "json", "chunking_strategy,description,expires_after,file_ids,metadata,name", "", "inf25", "/vector_stores", "Create_Vector_Store", "json", "chunking_strategy,expires_after,file_ids,metadata,name", "")
AZ_OP(ModifyVectorStore, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}", "modifyVectorStore", "json", "expires_after,metadata,name", "", "inf25", "/vector_stores/{vector_store_id}", "Modify_Vector_Store", "json", "expires_after,metadata,name", "")
AZ_OP(CreateVectorStoreFileBatch, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches", "createVectorStoreFileBatch", "json", "attributes,chunking_strategy,file_ids,files", "", "inf25", "/vector_stores/{vector_store_id}/file_batches", "Create_Vector_Store_File_Batch", "json", "chunking_strategy,file_ids", "")
AZ_OP(CancelVectorStoreFileBatch, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/cancel", "cancelVectorStoreFileBatch", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/cancel", "Cancel_Vector_Store_File_Batch", "none", "", "")
AZ_OP(CreateVectorStoreFile, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/files", "createVectorStoreFile", "json", "attributes,chunking_strategy,file_id", "", "inf25", "/vector_stores/{vector_store_id}/files", "Create_Vector_Store_File", "json", "chunking_strategy,file_id", "")
AZ_OP(UpdateVectorStoreFileAttributes, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}", "updateVectorStoreFileAttributes", "json", "attributes", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}", "updateVectorStoreFileAttributes", "json", "attributes", "")
AZ_OP(SearchVectorStore, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/search", "searchVectorStore", "json", "filters,max_num_results,query,ranking_options,rewrite_query", "", "inf25", "/vector_stores/{vector_store_id}/search", "searchVectorStore", "json", "filters,max_num_results,query,ranking_options,rewrite_query", "")
AZ_OP(VideosCreate, Surface::kOpenAi, "POST", 0, "v1p", "/videos", "Videos_Create", "json+multipart", "input_reference,model,prompt,seconds,size", "", "", "", "", "", "", "")
AZ_OP(VideosRemix, Surface::kOpenAi, "POST", 0, "v1p", "/videos/{video-id}/remix", "Videos_Remix", "json", "prompt", "", "", "", "", "", "", "")
AZ_OP(FoundryGetModelInfo, Surface::kFoundry, "GET", 0, "", "", "", "", "", "", "mi", "/info", "GetModelInfo", "none", "", "")
AZ_OP(FoundryGetChatCompletions, Surface::kFoundry, "POST", 0, "", "", "", "", "", "", "mi", "/chat/completions", "GetChatCompletions", "json", "frequency_penalty,max_tokens,messages,modalities,model,presence_penalty,response_format,seed,stop,stream,temperature,tool_choice,tools,top_p", "")
AZ_OP(FoundryGetEmbeddings, Surface::kFoundry, "POST", 0, "", "", "", "", "", "", "mi", "/embeddings", "GetEmbeddings", "json", "dimensions,encoding_format,input,input_type,model", "")
AZ_OP(FoundryGetImageEmbeddings, Surface::kFoundry, "POST", 0, "", "", "", "", "", "", "mi", "/images/embeddings", "GetImageEmbeddings", "json", "dimensions,encoding_format,input,input_type,model", "")
AZ_OP(SafetyTextBlocklistsDeleteTextBlocklist, Surface::kSafety, "DELETE", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}", "TextBlocklists_DeleteTextBlocklist", "none", "", "")
AZ_OP(SafetyTextBlocklistsListTextBlocklists, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists", "TextBlocklists_ListTextBlocklists", "none", "", "")
AZ_OP(SafetyTextBlocklistsGetTextBlocklist, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}", "TextBlocklists_GetTextBlocklist", "none", "", "")
AZ_OP(SafetyTextBlocklistsListTextBlocklistItems, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}/blocklistItems", "TextBlocklists_ListTextBlocklistItems", "none", "", "maxpagesize,skip,top")
AZ_OP(SafetyTextBlocklistsGetTextBlocklistItem, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}/blocklistItems/{blocklistItemId}", "TextBlocklists_GetTextBlocklistItem", "none", "", "")
AZ_OP(SafetyTextBlocklistsCreateOrUpdateTextBlocklist, Surface::kSafety, "PATCH", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}", "TextBlocklists_CreateOrUpdateTextBlocklist", "json", "blocklistName,description", "")
AZ_OP(SafetyImageOperationsAnalyzeImage, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/image:analyze", "ImageOperations_AnalyzeImage", "json", "categories,image,outputType", "")
AZ_OP(SafetyTextBlocklistsAddOrUpdateBlocklistItems, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}:addOrUpdateBlocklistItems", "TextBlocklists_AddOrUpdateBlocklistItems", "json", "blocklistItems", "")
AZ_OP(SafetyTextBlocklistsRemoveBlocklistItems, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}:removeBlocklistItems", "TextBlocklists_RemoveBlocklistItems", "json", "blocklistItemIds", "")
AZ_OP(SafetyTextOperationsAnalyzeText, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text:analyze", "TextOperations_AnalyzeText", "json", "blocklistNames,categories,haltOnBlocklistHit,outputType,text", "")
AZ_OP(SafetyTextOperationsDetectTextProtectedMaterial, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text:detectProtectedMaterial", "TextOperations_DetectTextProtectedMaterial", "json", "text", "")
AZ_OP(SafetyTextOperationsShieldPrompt, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text:shieldPrompt", "TextOperations_ShieldPrompt", "json", "documents,userPrompt", "")
// AZ-ROWS-END
+48
View File
@@ -0,0 +1,48 @@
#pragma once
// Static ceilings for the Azure package. Never autotuned. Numbers come from
// Microsoft Learn limits pages and the OpenAPI documents (see SPEC.md).
#include <cstddef>
namespace molo::azure {
struct AzureCaps {
// Names and endpoints.
static constexpr std::size_t kMaxEndpointChars = 256;
static constexpr std::size_t kMaxDeploymentChars = 64; // Azure deployment name limit
static constexpr std::size_t kMaxIdChars = 256;
static constexpr std::size_t kMaxSecretChars = 1024;
// Lists.
static constexpr int kMaxPageLimit = 100;
static constexpr std::size_t kMaxListPages = 20;
// Streams.
static constexpr std::size_t kMaxStreamEvents = 200000;
// Request content.
static constexpr std::size_t kMaxEmbeddingInputs = 2048;
static constexpr std::size_t kMaxAudioFileBytes = 25u * 1024u * 1024u;
static constexpr std::size_t kMaxImageFileBytes = 50u * 1024u * 1024u;
static constexpr std::size_t kMaxSpeechChars = 4096;
static constexpr std::size_t kMaxImagePromptChars = 32000;
// Content Safety (documented limits: 10k chars per text, 7.5 MiB image).
static constexpr std::size_t kMaxSafetyTextChars = 10000;
static constexpr std::size_t kMaxSafetyImageBytes = 7864320;
static constexpr std::size_t kMaxBlocklistItems = 100;
static constexpr std::size_t kMaxBlocklistItemChars = 128;
static constexpr std::size_t kMaxShieldDocuments = 5;
// Entra token exchange. The skew refreshes before expiry; the gap is the
// floor between two exchanges after a failure (the source throws without
// network traffic until it passes). Responses over the byte cap are refused.
static constexpr long kTokenSkewSeconds = 300;
static constexpr long kMinExchangeGapSeconds = 5;
static constexpr long kMaxTokenLifetimeSeconds = 24 * 3600;
static constexpr std::size_t kMaxTokenBytes = 16 * 1024;
// Realtime client secrets.
static constexpr int kMaxClientSecretSeconds = 7200;
// Waiting on background work.
static constexpr int kMinPollSeconds = 1;
static constexpr int kMaxPollSeconds = 60;
static constexpr int kMaxWaitSeconds = 600;
};
} // namespace molo::azure
+38
View File
@@ -0,0 +1,38 @@
#include "vendors/azure/chat.h"
#include "llm/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
void ChatService::validate(const std::string& deployment, const Json& body) const {
Check::deployment(deployment);
Check::object("body", body);
const Json* m = body.find("messages");
Check::require(m && m->is_array() && !m->as_array().empty(), "messages is required");
Check::max_count("messages", m->as_array().size(), llm::Caps::kMaxMessages);
}
Json ChatService::create(const std::string& deployment, const Json& body) {
validate(deployment, body);
const Json* s = body.find("stream");
Check::require(!s || (s->is_bool() && !s->as_bool()), "use stream() for stream=true");
return call(Op::kCreateChatCompletion, {deployment, {}}, &body);
}
void ChatService::stream(const std::string& deployment, const Json& body, const ChunkHandler& handler) {
validate(deployment, body);
call_stream(Op::kCreateChatCompletion, {deployment, {}}, body,
[&](const llm::SseEvent& ev) { return handler(ev.data); });
}
Json CompletionsService::create(const std::string& deployment, const Json& body) {
Check::deployment(deployment);
Check::object("body", body);
Check::has_field(body, "prompt");
return call(Op::kCreateCompletion, {deployment, {}}, &body);
}
} // namespace molo::azure
+42
View File
@@ -0,0 +1,42 @@
#pragma once
// Chat Completions and legacy Completions on a deployment.
// v1 API POST /openai/v1/chat/completions (deployment = body model)
// dated API POST /openai/deployments/{d}/chat/completions?api-version=...
// The caller passes the deployment name once; the gateway puts it where the
// configured API needs it.
#include <functional>
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class ChatService : public AzureService {
public:
using ChunkHandler = std::function<bool(const clients::body::Json& chunk)>; // false: stop
explicit ChatService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: messages (required, 1..4096) plus any field of the spec. `stream`
// must be absent or false here; use stream().
clients::body::Json create(const std::string& deployment, const clients::body::Json& body);
// Every `chat.completion.chunk` document reaches the handler; [DONE] ends.
void stream(const std::string& deployment, const clients::body::Json& body,
const ChunkHandler& handler);
private:
void validate(const std::string& deployment, const clients::body::Json& body) const;
};
class CompletionsService : public AzureService {
public:
explicit CompletionsService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: prompt (required) plus any field of the spec.
clients::body::Json create(const std::string& deployment, const clients::body::Json& body);
};
} // namespace molo::azure
+83
View File
@@ -0,0 +1,83 @@
#include "vendors/azure/chat_model.h"
#include "openai/core/wire/response_events.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
namespace oc = openai::core;
AzureChatModel::AzureChatModel(std::shared_ptr<AzureClient> azure, ChatModelConfig config)
: azure_(azure), config_(std::move(config)), chat_(azure), responses_(azure), foundry_(azure) {}
std::shared_ptr<AzureChatModel> AzureChatModel::from_env(ChatModelConfig config) {
return std::make_shared<AzureChatModel>(AzureClient::from_env(), std::move(config));
}
void AzureChatModel::check(const llm::ChatRequest& request) const {
Check::deployment(request.model);
Check::require(!request.messages.empty(), "messages is required");
}
Json AzureChatModel::wire_body(const llm::ChatRequest& request, bool stream) const {
check(request);
if (config_.dialect == Dialect::kResponses)
return oc::ResponsesWire::request(request, config_.wire).to_json(stream);
oc::ChatCompletionRequest r = oc::ChatWire::request(request, config_.wire);
if (stream) r.include_usage = true;
Json body = r.to_json(stream);
if (config_.dialect == Dialect::kFoundry) {
// The model inference API names the cap max_tokens.
Json::Object& o = body.as_object();
if (auto it = o.find("max_completion_tokens"); it != o.end()) {
o["max_tokens"] = it->second;
o.erase("max_completion_tokens");
}
if (!stream) o.erase("stream");
}
return body;
}
llm::ChatResponse AzureChatModel::generate(const llm::ChatRequest& request) {
Json body = wire_body(request, false);
switch (config_.dialect) {
case Dialect::kResponses:
return oc::ResponsesWire::response(responses_.create(body));
case Dialect::kFoundry:
return oc::ChatWire::response(foundry_.chat(body, config_.extra_parameters));
case Dialect::kChatCompletions:
break;
}
return oc::ChatWire::response(chat_.create(request.model, body));
}
void AzureChatModel::stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
Json body = wire_body(request, true);
if (config_.dialect == Dialect::kResponses) {
oc::ResponsesChunkDecoder decoder;
responses_.stream(body, [&](const llm::SseEvent& ev) {
oc::ResponseEvent e = oc::ResponseEvent::from_sse(ev);
llm::ChatChunk chunk;
if (!decoder.decode(e, chunk)) return true;
return on_chunk(chunk);
});
return;
}
oc::ChatChunkDecoder decoder;
auto handle = [&](const Json& doc) {
std::vector<llm::ChatChunk> chunks;
decoder.decode(doc, chunks);
for (const llm::ChatChunk& c : chunks)
if (!on_chunk(c)) return false;
return true;
};
if (config_.dialect == Dialect::kFoundry) {
foundry_.chat_stream(body, handle, config_.extra_parameters);
return;
}
chat_.stream(request.model, body, handle);
}
} // namespace molo::azure
+57
View File
@@ -0,0 +1,57 @@
#pragma once
// AzureChatModel: llm::ChatModel over a deployment of an Azure resource.
// The request's `model` is the deployment name. Three dialects:
// kChatCompletions /chat/completions (default; the widest model support)
// kResponses /responses
// kFoundry Azure AI model inference /models/chat/completions
// The neutral <-> wire mapping is the OpenAI one (openai::core::ChatWire /
// ResponsesWire): Azure OpenAI speaks the same shapes.
#include <functional>
#include <memory>
#include <string>
#include "llm/chat_types.h"
#include "openai/core/wire/chat_wire.h"
#include "vendors/azure/chat.h"
#include "vendors/azure/foundry.h"
#include "vendors/azure/responses.h"
namespace molo::azure {
enum class Dialect { kChatCompletions, kResponses, kFoundry };
struct ChatModelConfig {
Dialect dialect = Dialect::kChatCompletions;
openai::core::WireOptions wire;
std::string extra_parameters; // Foundry dialect: error, drop, pass-through
};
class AzureChatModel : public llm::ChatModel {
public:
explicit AzureChatModel(std::shared_ptr<AzureClient> azure, ChatModelConfig config = {});
// Settings from the environment; not configured without endpoint and credential.
static std::shared_ptr<AzureChatModel> from_env(ChatModelConfig config = {});
std::string vendor() const override { return "azure"; }
llm::ChatResponse generate(const llm::ChatRequest& request) override;
void stream(const llm::ChatRequest& request,
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
Dialect dialect() const { return config_.dialect; }
bool configured() const { return azure_->configured(); }
// The wire body generate() would send for a request (no traffic).
clients::body::Json wire_body(const llm::ChatRequest& request, bool stream) const;
private:
void check(const llm::ChatRequest& request) const;
std::shared_ptr<AzureClient> azure_;
ChatModelConfig config_;
ChatService chat_;
ResponsesService responses_;
FoundryService foundry_;
};
} // namespace molo::azure
+63
View File
@@ -0,0 +1,63 @@
#include "vendors/azure/check.h"
#include "llm/api_client.h"
#include "llm/error.h"
#include "vendors/azure/caps.h"
namespace molo::azure {
using clients::body::Json;
void Check::fail(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
void Check::require(bool ok, const std::string& why) {
if (!ok) fail(why);
}
void Check::non_empty(const std::string& what, const std::string& value) {
if (value.empty()) fail(what + " is required");
}
void Check::max_chars(const std::string& what, const std::string& value, std::size_t cap) {
if (value.size() > cap) fail(what + " is over " + std::to_string(cap) + " characters");
}
void Check::max_count(const std::string& what, std::size_t count, std::size_t cap) {
if (count > cap) fail(what + " has more than " + std::to_string(cap) + " entries");
}
void Check::range(const std::string& what, double value, double lo, double hi) {
if (value < lo || value > hi)
fail(what + " must be " + std::to_string(static_cast<long>(lo)) + ".." + std::to_string(static_cast<long>(hi)));
}
void Check::in_set(const std::string& what, const std::string& value,
const std::vector<std::string>& allowed) {
for (const std::string& a : allowed)
if (a == value) return;
std::string list;
for (const std::string& a : allowed) list += (list.empty() ? "" : ", ") + a;
fail(what + " must be one of: " + list);
}
void Check::deployment(const std::string& value) {
if (value.empty() || value.size() > AzureCaps::kMaxDeploymentChars || value == "." || value == "..")
fail("deployment must be 1.." + std::to_string(AzureCaps::kMaxDeploymentChars) + " characters");
for (char c : value)
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
c == '_' || c == '.'))
fail("deployment has invalid characters");
}
void Check::id(const std::string& what, const std::string& value) {
if (!llm::Service::valid_id(value)) fail("bad " + what + " id");
}
void Check::object(const std::string& what, const Json& value) {
if (!value.is_object()) fail(what + " must be a JSON object");
}
void Check::has_string(const Json& body, const std::string& key) {
const Json* v = body.is_object() ? body.find(key) : nullptr;
if (!v || !v->is_string() || v->as_string().empty()) fail(key + " is required");
}
void Check::has_field(const Json& body, const std::string& key) {
const Json* v = body.is_object() ? body.find(key) : nullptr;
if (!v || v->is_null()) fail(key + " is required");
}
void Check::no_control(const std::string& what, const std::string& value) {
for (char c : value)
if (static_cast<unsigned char>(c) < 0x20) fail(what + " has control characters");
}
} // namespace molo::azure
+34
View File
@@ -0,0 +1,34 @@
#pragma once
// Request validation helpers. Every failure is llm::ApiError::invalid
// ("azure", 400 INVALID_ARGUMENT), thrown before any network traffic.
#include <string>
#include <vector>
#include "clients/body/json.h"
namespace molo::azure {
class Check {
public:
[[noreturn]] static void fail(const std::string& why);
static void require(bool ok, const std::string& why);
static void non_empty(const std::string& what, const std::string& value);
static void max_chars(const std::string& what, const std::string& value, std::size_t cap);
static void max_count(const std::string& what, std::size_t count, std::size_t cap);
static void range(const std::string& what, double value, double lo, double hi);
static void in_set(const std::string& what, const std::string& value,
const std::vector<std::string>& allowed);
// A deployment name: 1..64 of [A-Za-z0-9._-], not "." or "..".
static void deployment(const std::string& value);
// One URL path segment id (llm::Service::valid_id).
static void id(const std::string& what, const std::string& value);
static void object(const std::string& what, const clients::body::Json& value);
// The body object has the string field `key` (non-empty).
static void has_string(const clients::body::Json& body, const std::string& key);
static void has_field(const clients::body::Json& body, const std::string& key);
static void no_control(const std::string& what, const std::string& value);
};
} // namespace molo::azure
+67
View File
@@ -0,0 +1,67 @@
#include "vendors/azure/client.h"
#include "clients/transports/http_transport.h"
namespace molo::azure {
namespace {
constexpr const char* kPlaceholderKey = "entra-managed"; // never sent: EntraTransport overwrites it
llm::ApiClientConfig base(const std::string& endpoint, const char* key_env, bool entra,
const char* key_header) {
llm::ApiClientConfig c;
c.vendor = Settings::kVendor;
// An unset endpoint cannot be parsed as a URL; the empty key stops every call first.
c.base_url = endpoint.empty() ? "https://unset.invalid" : endpoint;
c.key_env = endpoint.empty() ? Settings::kEndpointEnv : key_env;
if (entra) {
c.auth = llm::AuthStyle::kBearer;
} else {
c.auth = llm::AuthStyle::kHeader;
c.auth_header = key_header;
}
return c;
}
} // namespace
std::shared_ptr<AzureClient> AzureClient::from_env() {
return create(Settings::from_env(), [](Channel) -> std::unique_ptr<clients::Transport> {
return std::make_unique<clients::transports::HttpTransport>();
});
}
std::shared_ptr<AzureClient> AzureClient::create(Settings settings, const TransportFactory& factory,
EntraTokenSource::Clock clock) {
settings.validate();
std::shared_ptr<AzureClient> self(new AzureClient());
self->settings_ = std::move(settings);
const Settings& s = self->settings_;
bool entra = s.auth() == AuthKind::kEntra;
if (entra) self->tokens_ = std::make_shared<EntraTokenSource>(s.entra, factory(Channel::kToken), clock);
auto wire = [&](Channel ch, const std::string& endpoint, const char* key_env, const char* header,
const std::string& key) {
std::unique_ptr<clients::Transport> t = factory(ch);
if (entra) t = std::make_unique<EntraTransport>(std::move(t), self->tokens_);
std::string k = entra ? (endpoint.empty() ? "" : kPlaceholderKey) : (endpoint.empty() ? "" : key);
return std::make_shared<llm::ApiClient>(std::move(t), base(endpoint, key_env, entra, header), k);
};
self->openai_ = wire(Channel::kOpenAi, s.endpoint, Settings::kKeyEnv, "api-key", s.api_key);
self->foundry_ = wire(Channel::kFoundry, s.foundry_endpoint, Settings::kKeyEnv, "api-key", s.api_key);
self->safety_ = wire(Channel::kSafety, s.safety_endpoint, Settings::kSafetyKeyEnv,
"Ocp-Apim-Subscription-Key", s.safety_key);
return self;
}
std::shared_ptr<llm::ApiClient> AzureClient::api(Surface surface) const {
switch (surface) {
case Surface::kFoundry: return foundry_;
case Surface::kSafety: return safety_;
case Surface::kOpenAi: break;
}
return openai_;
}
bool AzureClient::configured(Surface surface) const { return api(surface)->configured(); }
} // namespace molo::azure
+51
View File
@@ -0,0 +1,51 @@
#pragma once
// AzureClient: one llm::ApiClient per surface (OpenAI resource, Foundry model
// inference, Content Safety) plus the shared credential. Key mode sends the
// api-key header (Ocp-Apim-Subscription-Key for Content Safety); Entra mode
// puts a managed bearer token on every request through EntraTransport.
// Not configured (no endpoint, or neither key nor complete Entra client
// credentials): every call throws NotConfiguredError before any traffic.
#include <functional>
#include <memory>
#include <string>
#include "llm/api_client.h"
#include "vendors/azure/calls.h"
#include "vendors/azure/entra.h"
#include "vendors/azure/settings.h"
namespace molo::azure {
// Which wire a transport serves. kToken is the Entra token exchange.
enum class Channel { kOpenAi, kFoundry, kSafety, kToken };
class AzureClient {
public:
using TransportFactory = std::function<std::unique_ptr<clients::Transport>(Channel)>;
// Real HTTPS transports, settings from the environment.
static std::shared_ptr<AzureClient> from_env();
// Any transports (tests).
static std::shared_ptr<AzureClient> create(Settings settings, const TransportFactory& factory,
EntraTokenSource::Clock clock = nullptr);
const Settings& settings() const { return settings_; }
bool configured() const { return settings_.configured(); }
AuthKind auth() const { return settings_.auth(); }
std::shared_ptr<llm::ApiClient> api(Surface surface) const;
std::shared_ptr<EntraTokenSource> tokens() const { return tokens_; }
// The configured credential for a surface is present (Safety may have its own key).
bool configured(Surface surface) const;
private:
AzureClient() = default;
Settings settings_;
std::shared_ptr<EntraTokenSource> tokens_;
std::shared_ptr<llm::ApiClient> openai_;
std::shared_ptr<llm::ApiClient> foundry_;
std::shared_ptr<llm::ApiClient> safety_;
};
} // namespace molo::azure
+53
View File
@@ -0,0 +1,53 @@
#include "vendors/azure/embeddings.h"
#include "vendors/azure/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
Json EmbeddingsService::create(const std::string& deployment, const Json& body) {
Check::deployment(deployment);
Check::object("body", body);
const Json* in = body.find("input");
Check::require(in && !in->is_null(), "input is required");
if (in->is_array()) {
Check::require(!in->as_array().empty(), "input must not be empty");
Check::max_count("input", in->as_array().size(), AzureCaps::kMaxEmbeddingInputs);
} else {
Check::require(in->is_string() && !in->as_string().empty(), "input must be a non-empty string or an array");
}
return call(Op::kCreateEmbedding, {deployment, {}}, &body);
}
std::vector<std::vector<double>> EmbeddingsService::embed(const std::string& deployment,
const std::vector<std::string>& texts,
int dimensions) {
Check::require(!texts.empty(), "texts must not be empty");
Json body = Json::object();
Json arr = Json::array();
for (const std::string& t : texts) {
Check::non_empty("text", t);
arr.push(Json(t));
}
body.set("input", arr);
body.set("encoding_format", Json("float"));
if (dimensions > 0) body.set("dimensions", Json(dimensions));
Json doc = create(deployment, body);
std::vector<std::vector<double>> out;
const Json* data = doc.find("data");
if (!data || !data->is_array())
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "embeddings answer has no data array");
for (const Json& item : data->as_array()) {
std::vector<double> v;
const Json* e = item.find("embedding");
if (!e || !e->is_array())
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "embedding item has no vector");
for (const Json& x : e->as_array()) v.push_back(x.as_number());
out.push_back(std::move(v));
}
return out;
}
} // namespace molo::azure
+25
View File
@@ -0,0 +1,25 @@
#pragma once
// Embeddings on a deployment (text-embedding-3-*, ada-002).
#include <string>
#include <vector>
#include "vendors/azure/service.h"
namespace molo::azure {
class EmbeddingsService : public AzureService {
public:
explicit EmbeddingsService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body.input: a string or an array of 1..2048 strings / token arrays.
clients::body::Json create(const std::string& deployment, const clients::body::Json& body);
// Convenience: texts in, body built, vectors out (encoding_format float).
std::vector<std::vector<double>> embed(const std::string& deployment,
const std::vector<std::string>& texts,
int dimensions = 0);
};
} // namespace molo::azure
+146
View File
@@ -0,0 +1,146 @@
#include "vendors/azure/entra.h"
#include <chrono>
#include "clients/body/form.h"
#include "clients/body/json.h"
#include "llm/error.h"
#include "vendors/azure/caps.h"
namespace molo::azure {
using clients::body::Json;
namespace {
clients::ClientConfig make_config() {
clients::ClientConfig c;
c.persist_cookies = false;
return c;
}
std::int64_t system_now() {
return std::chrono::duration_cast<std::chrono::seconds>(
std::chrono::system_clock::now().time_since_epoch())
.count();
}
std::string first_line(const std::string& s, std::size_t cap) {
std::string out = s.substr(0, s.find_first_of("\r\n"));
if (out.size() > cap) out.resize(cap);
return out;
}
} // namespace
EntraTokenSource::EntraTokenSource(EntraConfig config, std::unique_ptr<clients::Transport> transport,
Clock clock)
: config_(std::move(config)),
client_(std::move(transport), make_config()),
clock_(clock ? std::move(clock) : Clock(system_now)) {
config_.validate();
}
std::string EntraTokenSource::request_body() const {
clients::body::UrlEncodedForm f;
f.add("client_id", config_.client_id);
f.add("client_secret", config_.client_secret);
f.add("scope", config_.scope);
f.add("grant_type", "client_credentials");
return f.encode();
}
EntraTokenSource::Parsed EntraTokenSource::parse(int status, const std::string& body) {
if (body.size() > AzureCaps::kMaxTokenBytes)
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "token response over the byte cap");
Json doc;
try {
doc = Json::parse(body);
} catch (const std::exception&) {
throw llm::ApiError("azure", status >= 400 ? status : 502, "BAD_RESPONSE",
"token endpoint answered with a non-JSON body");
}
if (status < 200 || status >= 300 || !doc.is_object() || doc.has("error")) {
std::string code = "token_error";
std::string desc = "HTTP " + std::to_string(status);
std::string trace;
if (doc.is_object()) {
if (const Json* e = doc.find("error"); e && e->is_string()) code = e->as_string();
if (const Json* d = doc.find("error_description"); d && d->is_string())
desc = first_line(d->as_string(), 300);
if (const Json* t = doc.find("trace_id"); t && t->is_string()) trace = t->as_string();
}
throw llm::ApiError("azure", status >= 400 ? status : 502, code, desc, trace);
}
Parsed out;
const Json* t = doc.find("access_token");
if (!t || !t->is_string() || t->as_string().empty())
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "token response has no access_token");
out.access_token = t->as_string();
const Json* ex = doc.find("expires_in");
// The v2.0 endpoint documents expires_in as a number; some stacks send a string.
if (ex && ex->is_number()) out.expires_in = ex->as_int();
else if (ex && ex->is_string()) out.expires_in = std::atol(ex->as_string().c_str());
if (out.expires_in <= 0 || out.expires_in > AzureCaps::kMaxTokenLifetimeSeconds)
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "token response has an unusable expires_in");
return out;
}
std::string EntraTokenSource::token() {
std::lock_guard<std::mutex> lock(mutex_);
std::int64_t now = clock_();
if (!token_.empty() && now < expires_at_ - AzureCaps::kTokenSkewSeconds) return token_;
if (last_failure_at_ != 0 && now - last_failure_at_ < AzureCaps::kMinExchangeGapSeconds)
throw llm::ApiError("azure", 503, "TOKEN_BACKOFF",
"token exchange failed " + std::to_string(now - last_failure_at_) +
"s ago (" + last_failure_ + "), not retrying yet");
clients::ClientRequest req(clients::Method::kPost, clients::Url::parse(config_.token_url()));
req.form(clients::body::UrlEncodedForm::decode(request_body()));
++exchanges_;
try {
clients::ClientResponse resp = client_.send_blocking(std::move(req));
Parsed p = parse(resp.status(), resp.body());
token_ = p.access_token;
expires_at_ = now + p.expires_in;
last_failure_at_ = 0;
return token_;
} catch (const llm::ApiError& e) {
last_failure_at_ = now;
last_failure_ = e.type();
token_.clear();
throw;
} catch (const std::exception& e) {
last_failure_at_ = now;
last_failure_ = "UNAVAILABLE";
token_.clear();
throw llm::ApiError("azure", 502, "UNAVAILABLE", e.what());
}
}
std::size_t EntraTokenSource::exchanges() const {
std::lock_guard<std::mutex> lock(mutex_);
return exchanges_;
}
std::int64_t EntraTokenSource::expires_at() const {
std::lock_guard<std::mutex> lock(mutex_);
return expires_at_;
}
EntraTransport::EntraTransport(std::unique_ptr<clients::Transport> inner,
std::shared_ptr<EntraTokenSource> source)
: inner_(std::move(inner)), source_(std::move(source)) {}
clients::Task<clients::ClientResponse> EntraTransport::send(const clients::ClientRequest& request) {
clients::ClientRequest copy = request;
copy.bearer(source_->token());
co_return co_await inner_->send(copy);
}
clients::Task<void> EntraTransport::send_stream(const clients::ClientRequest& request,
clients::body::ChunkSink sink) {
clients::ClientRequest copy = request;
copy.bearer(source_->token());
co_await inner_->send_stream(copy, std::move(sink));
co_return;
}
} // namespace molo::azure
+76
View File
@@ -0,0 +1,76 @@
#pragma once
// Microsoft Entra ID client-credentials flow (OAuth2, RFC 6749 section 4.4):
// POST <authority>/<tenant>/oauth2/v2.0/token with client_id, client_secret,
// scope and grant_type=client_credentials. EntraTokenSource caches the token
// and refreshes it AzureCaps::kTokenSkewSeconds before expiry; after a failed
// exchange it refuses to call again for AzureCaps::kMinExchangeGapSeconds
// (a floor: a broken secret must not hammer the tenant). EntraTransport puts
// the current token on every request that passes through a transport.
#include <cstdint>
#include <functional>
#include <memory>
#include <mutex>
#include <string>
#include "clients/client.h"
#include "clients/transport.h"
#include "vendors/azure/settings.h"
namespace molo::azure {
class EntraTokenSource {
public:
using Clock = std::function<std::int64_t()>; // unix seconds
EntraTokenSource(EntraConfig config, std::unique_ptr<clients::Transport> transport,
Clock clock = nullptr);
// A valid access token, exchanging when none is cached or it is about to
// expire. Throws llm::ApiError (type = the AAD error code, e.g.
// "invalid_client") on a failed exchange. Thread safe.
std::string token();
// The form body of the exchange request (contains the secret: for tests of
// the wire shape only, never log it).
std::string request_body() const;
const EntraConfig& config() const { return config_; }
std::size_t exchanges() const;
std::int64_t expires_at() const;
// Parses a token endpoint answer (documented success and error bodies).
struct Parsed {
std::string access_token;
long expires_in = 0;
};
static Parsed parse(int status, const std::string& body);
private:
EntraConfig config_;
clients::Client client_;
Clock clock_;
mutable std::mutex mutex_;
std::string token_;
std::int64_t expires_at_ = 0;
std::int64_t last_failure_at_ = 0;
std::string last_failure_;
std::size_t exchanges_ = 0;
};
// Transport decorator: sets `Authorization: Bearer <token>` from the source on
// every request, whatever the caller put there.
class EntraTransport : public clients::Transport {
public:
EntraTransport(std::unique_ptr<clients::Transport> inner, std::shared_ptr<EntraTokenSource> source);
const char* name() const override { return "entra"; }
clients::Task<clients::ClientResponse> send(const clients::ClientRequest& request) override;
clients::Task<void> send_stream(const clients::ClientRequest& request,
clients::body::ChunkSink sink) override;
void close() override { inner_->close(); }
private:
std::unique_ptr<clients::Transport> inner_;
std::shared_ptr<EntraTokenSource> source_;
};
} // namespace molo::azure
+87
View File
@@ -0,0 +1,87 @@
#include "vendors/azure/files.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
Json FilesService::upload(const UploadFile& file, const std::string& purpose) {
Check::non_empty("filename", file.filename);
Check::no_control("filename", file.filename);
Check::non_empty("purpose", purpose);
Check::require(!file.data.empty(), "file is empty");
Check::require(file.data.size() <= kMaxFileBytes, "file over 512 MiB");
clients::body::MultipartBody form;
form.add_text("purpose", purpose);
form.add_file("file", file.filename, file.content_type, file.data);
return call_multipart(Op::kCreateFile, {}, form);
}
Json FilesService::get(const std::string& id) {
Check::id("file", id);
return call(Op::kRetrieveFile, {"", {id}});
}
Json FilesService::list(const std::string& purpose, const ListParams& params) {
params.validate();
Query q;
q.add("purpose", purpose);
params.apply(q);
return call(Op::kListFiles, {}, nullptr, &q);
}
llm::RawResponse FilesService::content(const std::string& id) {
Check::id("file", id);
return call_raw(Op::kDownloadFile, {"", {id}});
}
Json FilesService::remove(const std::string& id) {
Check::id("file", id);
return call(Op::kDeleteFile, {"", {id}});
}
Json FilesService::import_from_url(const Json& body) {
Check::has_string(body, "purpose");
Check::has_string(body, "filename");
Check::has_string(body, "content_url");
return call(Op::kFilesImport, {}, &body);
}
Json FilesService::start_upload(const Json& body) {
Check::has_string(body, "purpose");
Check::has_string(body, "filename");
Check::has_field(body, "bytes");
return call(Op::kUploadFileStart, {}, &body);
}
Json FilesService::add_part(const std::string& upload_id, const UploadFile& part) {
Check::id("upload", upload_id);
Check::require(!part.data.empty(), "part is empty");
Check::require(part.data.size() <= 64u * 1024u * 1024u, "part over 64 MiB");
clients::body::MultipartBody form;
form.add_file("data", part.filename.empty() ? "part" : part.filename, part.content_type, part.data);
return call_multipart(Op::kUploadFilePart, {"", {upload_id}}, form);
}
Json FilesService::complete_upload(const std::string& upload_id, const std::vector<std::string>& part_ids,
const std::string& md5) {
Check::id("upload", upload_id);
Check::require(!part_ids.empty() && part_ids.size() <= 10000, "part_ids must hold 1..10000 ids");
Json body = Json::object();
Json ids = Json::array();
for (const std::string& p : part_ids) {
Check::id("part", p);
ids.push(Json(p));
}
body.set("part_ids", ids);
if (!md5.empty()) body.set("md5", Json(md5));
return call(Op::kUploadFileComplete, {"", {upload_id}}, &body);
}
Json FilesService::cancel_upload(const std::string& upload_id) {
Check::id("upload", upload_id);
return call(Op::kUploadFileCancel, {"", {upload_id}});
}
} // namespace molo::azure
+39
View File
@@ -0,0 +1,39 @@
#pragma once
// Files and multipart Uploads. Upload purposes: fine-tune, assistants, batch,
// user_data, vision, evals (the spec lists the accepted ones per API).
#include <string>
#include <vector>
#include "vendors/azure/service.h"
#include "vendors/azure/upload_file.h"
namespace molo::azure {
class FilesService : public AzureService {
public:
static constexpr std::size_t kMaxFileBytes = 512u * 1024u * 1024u; // llm::Caps::kMaxUploadBytes
explicit FilesService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// Uploads one file (single request, capped at 512 MiB). purpose is required.
clients::body::Json upload(const UploadFile& file, const std::string& purpose);
clients::body::Json get(const std::string& id);
// GET /files; the dated API takes `purpose`, the v1 API also after/limit/order.
clients::body::Json list(const std::string& purpose = "", const ListParams& params = {});
llm::RawResponse content(const std::string& id);
clients::body::Json remove(const std::string& id); // destructive
// Dated API only: import from a URL (body: purpose, filename, content_url).
clients::body::Json import_from_url(const clients::body::Json& body);
// Multipart uploads of large files (dated API).
clients::body::Json start_upload(const clients::body::Json& body);
clients::body::Json add_part(const std::string& upload_id, const UploadFile& part);
clients::body::Json complete_upload(const std::string& upload_id, const std::vector<std::string>& part_ids,
const std::string& md5 = "");
clients::body::Json cancel_upload(const std::string& upload_id); // destructive
};
} // namespace molo::azure
+89
View File
@@ -0,0 +1,89 @@
#include "vendors/azure/fine_tuning.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
namespace {
Query page(const ListParams& p) {
p.validate();
Query q;
p.apply(q);
return q;
}
} // namespace
Json FineTuningService::create(const Json& body) {
Check::has_string(body, "model");
Check::has_string(body, "training_file");
return call(Op::kCreateFineTuningJob, {}, &body);
}
Json FineTuningService::get(const std::string& job) {
Check::id("job", job);
return call(Op::kRetrieveFineTuningJob, {"", {job}});
}
Json FineTuningService::list(const ListParams& params) {
Query q = page(params);
return call(Op::kListPaginatedFineTuningJobs, {}, nullptr, &q);
}
Listing FineTuningService::list_all(const ListParams& params, std::size_t max_pages) {
return walk(Op::kListPaginatedFineTuningJobs, {}, params, Query(), max_pages);
}
Json FineTuningService::events(const std::string& job, const ListParams& params) {
Check::id("job", job);
Query q = page(params);
return call(Op::kListFineTuningEvents, {"", {job}}, nullptr, &q);
}
Json FineTuningService::checkpoints(const std::string& job, const ListParams& params) {
Check::id("job", job);
Query q = page(params);
return call(Op::kListFineTuningJobCheckpoints, {"", {job}}, nullptr, &q);
}
Json FineTuningService::cancel(const std::string& job) {
Check::id("job", job);
return call(Op::kCancelFineTuningJob, {"", {job}});
}
Json FineTuningService::pause(const std::string& job) {
Check::id("job", job);
return call(Op::kPauseFineTuningJob, {"", {job}});
}
Json FineTuningService::resume(const std::string& job) {
Check::id("job", job);
return call(Op::kResumeFineTuningJob, {"", {job}});
}
Json FineTuningService::remove(const std::string& job) {
Check::id("job", job);
return call(Op::kFineTuningDelete, {"", {job}});
}
Json FineTuningService::copy_checkpoint(const std::string& job, const std::string& checkpoint,
const Json& body) {
Check::id("job", job);
Check::id("checkpoint", checkpoint);
Check::has_string(body, "destinationResourceId");
Check::has_string(body, "region");
return call(Op::kFineTuningCopyCheckpoint, {"", {job, checkpoint}}, &body);
}
Json FineTuningService::get_checkpoint_copy(const std::string& job, const std::string& checkpoint) {
Check::id("job", job);
Check::id("checkpoint", checkpoint);
return call(Op::kFineTuningGetCheckpoint, {"", {job, checkpoint}});
}
Json FineTuningService::list_permissions(const std::string& model, const ListParams& params) {
Check::id("checkpoint model", model);
Query q = page(params);
return call(Op::kListFineTuningCheckpointPermissions, {"", {model}}, nullptr, &q);
}
Json FineTuningService::create_permission(const std::string& model, const Json& body) {
Check::id("checkpoint model", model);
Check::has_field(body, "project_ids");
return call(Op::kCreateFineTuningCheckpointPermission, {"", {model}}, &body);
}
Json FineTuningService::remove_permission(const std::string& model, const std::string& permission) {
Check::id("checkpoint model", model);
Check::id("permission", permission);
return call(Op::kDeleteFineTuningCheckpointPermission, {"", {model, permission}});
}
} // namespace molo::azure
+40
View File
@@ -0,0 +1,40 @@
#pragma once
// Fine-tuning jobs, events, checkpoints, checkpoint permissions and the
// cross-resource checkpoint copy. Creating a job costs money.
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class FineTuningService : public AzureService {
public:
explicit FineTuningService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: model and training_file (required) plus any spec field.
clients::body::Json create(const clients::body::Json& body);
clients::body::Json get(const std::string& job);
clients::body::Json list(const ListParams& params = {});
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
clients::body::Json events(const std::string& job, const ListParams& params = {});
clients::body::Json checkpoints(const std::string& job, const ListParams& params = {});
clients::body::Json cancel(const std::string& job); // destructive
clients::body::Json pause(const std::string& job);
clients::body::Json resume(const std::string& job);
// Dated API only: DELETE a job record.
clients::body::Json remove(const std::string& job); // destructive
// v1 API: copy a checkpoint to another resource (body: destinationResourceId, region).
clients::body::Json copy_checkpoint(const std::string& job, const std::string& checkpoint,
const clients::body::Json& body);
clients::body::Json get_checkpoint_copy(const std::string& job, const std::string& checkpoint);
// Checkpoint permissions (v1 API).
clients::body::Json list_permissions(const std::string& checkpoint_model, const ListParams& params = {});
clients::body::Json create_permission(const std::string& checkpoint_model, const clients::body::Json& body);
clients::body::Json remove_permission(const std::string& checkpoint_model,
const std::string& permission); // destructive
};
} // namespace molo::azure
+55
View File
@@ -0,0 +1,55 @@
#include "vendors/azure/foundry.h"
#include "llm/caps.h"
#include "vendors/azure/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
llm::Headers FoundryService::headers(const std::string& extra) {
llm::Headers h;
if (extra.empty()) return h;
Check::in_set("extra_parameters", extra, {"error", "drop", "pass-through"});
h["extra-parameters"] = extra;
return h;
}
Json FoundryService::info() { return call(Op::kFoundryGetModelInfo); }
Json FoundryService::chat(const Json& body, const std::string& extra) {
Check::object("body", body);
const Json* m = body.find("messages");
Check::require(m && m->is_array() && !m->as_array().empty(), "messages is required");
Check::max_count("messages", m->as_array().size(), llm::Caps::kMaxMessages);
const Json* s = body.find("stream");
Check::require(!s || (s->is_bool() && !s->as_bool()), "use chat_stream() for stream=true");
return call(Op::kFoundryGetChatCompletions, {}, &body, nullptr, headers(extra));
}
void FoundryService::chat_stream(const Json& body, const ChunkHandler& handler, const std::string& extra) {
Check::object("body", body);
const Json* m = body.find("messages");
Check::require(m && m->is_array() && !m->as_array().empty(), "messages is required");
call_stream(Op::kFoundryGetChatCompletions, {}, body,
[&](const llm::SseEvent& ev) { return handler(ev.data); }, headers(extra));
}
Json FoundryService::embeddings(const Json& body, const std::string& extra) {
Check::object("body", body);
const Json* in = body.find("input");
Check::require(in && in->is_array() && !in->as_array().empty(), "input must be a non-empty array of strings");
Check::max_count("input", in->as_array().size(), AzureCaps::kMaxEmbeddingInputs);
return call(Op::kFoundryGetEmbeddings, {}, &body, nullptr, headers(extra));
}
Json FoundryService::image_embeddings(const Json& body, const std::string& extra) {
Check::object("body", body);
const Json* in = body.find("input");
Check::require(in && in->is_array() && !in->as_array().empty(), "input must be a non-empty array of images");
Check::max_count("input", in->as_array().size(), AzureCaps::kMaxEmbeddingInputs);
return call(Op::kFoundryGetImageEmbeddings, {}, &body, nullptr, headers(extra));
}
} // namespace molo::azure
+39
View File
@@ -0,0 +1,39 @@
#pragma once
// Azure AI Foundry model inference (the Azure AI Model Inference API,
// https://<resource>.services.ai.azure.com/models, api-version
// 2024-05-01-preview): chat completions, text embeddings, image embeddings
// and model info for the model catalog models (Llama, Mistral, Phi, Cohere,
// DeepSeek, ...). Same credential as the OpenAI resource (api-key header or
// Entra bearer). The `model` body field picks the deployment on a multi-model
// endpoint.
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class FoundryService : public AzureService {
public:
using ChunkHandler = std::function<bool(const clients::body::Json& chunk)>; // false: stop
explicit FoundryService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kFoundry) {}
// GET /info: model_name, model_type, model_provider_name of the endpoint's model.
clients::body::Json info();
// extra_parameters: "" (service default: error), "error", "drop" or "pass-through"
// (the extra-parameters header of the spec).
clients::body::Json chat(const clients::body::Json& body, const std::string& extra_parameters = "");
void chat_stream(const clients::body::Json& body, const ChunkHandler& handler,
const std::string& extra_parameters = "");
clients::body::Json embeddings(const clients::body::Json& body, const std::string& extra_parameters = "");
clients::body::Json image_embeddings(const clients::body::Json& body,
const std::string& extra_parameters = "");
private:
static llm::Headers headers(const std::string& extra_parameters);
};
} // namespace molo::azure
+54
View File
@@ -0,0 +1,54 @@
#include "vendors/azure/images.h"
#include "vendors/azure/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
using clients::body::MultipartBody;
namespace {
void add_file(MultipartBody& form, const std::string& field, const UploadFile& f) {
Check::non_empty("image filename", f.filename);
Check::no_control("filename", f.filename);
Check::require(!f.data.empty(), "image file is empty");
Check::require(f.data.size() <= AzureCaps::kMaxImageFileBytes, "image file over 50 MiB");
form.add_file(field, f.filename, f.content_type, f.data);
}
} // namespace
Json ImagesService::generate(const std::string& deployment, const Json& body) {
Check::deployment(deployment);
Check::object("body", body);
Check::has_string(body, "prompt");
Check::max_chars("prompt", body.find("prompt")->as_string(), AzureCaps::kMaxImagePromptChars);
return call(Op::kCreateImage, {deployment, {}}, &body);
}
Json ImagesService::edit(const std::string& deployment, const std::string& prompt,
const std::vector<UploadFile>& images, const std::optional<UploadFile>& mask,
const std::map<std::string, std::string>& fields) {
Check::deployment(deployment);
Check::non_empty("prompt", prompt);
Check::max_chars("prompt", prompt, AzureCaps::kMaxImagePromptChars);
Check::require(!images.empty() && images.size() <= 16, "images must hold 1..16 files");
MultipartBody form;
form.add_text("prompt", prompt);
for (const UploadFile& f : images) add_file(form, images.size() > 1 ? "image[]" : "image", f);
if (mask) add_file(form, "mask", *mask);
for (const auto& [k, v] : fields) form.add_text(k, v);
return call_multipart(Op::kCreateImageEdit, {deployment, {}}, form);
}
Json ImagesService::variation(const std::string& deployment, const UploadFile& image,
const std::map<std::string, std::string>& fields) {
Check::deployment(deployment);
MultipartBody form;
add_file(form, "image", image);
for (const auto& [k, v] : fields) form.add_text(k, v);
form.add_text("model", deployment);
return call_multipart(Op::kCreateImageVariation, {}, form);
}
} // namespace molo::azure
+35
View File
@@ -0,0 +1,35 @@
#pragma once
// Images on a deployment (gpt-image-1, dall-e-3): generations (JSON), edits
// and variations (multipart; the spec of the dated API lists JSON for edits,
// the service and every official SDK send multipart, see SPEC Unverified).
#include <map>
#include <optional>
#include <string>
#include <vector>
#include "vendors/azure/service.h"
#include "vendors/azure/upload_file.h"
namespace molo::azure {
class ImagesService : public AzureService {
public:
explicit ImagesService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: prompt (required, <= 32000) plus any field of the spec (n, size, quality...).
clients::body::Json generate(const std::string& deployment, const clients::body::Json& body);
// fields: text form fields of the spec (n, size, quality, response_format...).
// images: 1..16 files (sent as image[] when more than one); mask optional.
clients::body::Json edit(const std::string& deployment, const std::string& prompt,
const std::vector<UploadFile>& images,
const std::optional<UploadFile>& mask = std::nullopt,
const std::map<std::string, std::string>& fields = {});
// v1 API only (the dated documents have no variations operation).
clients::body::Json variation(const std::string& deployment, const UploadFile& image,
const std::map<std::string, std::string>& fields = {});
};
} // namespace molo::azure
+30
View File
@@ -0,0 +1,30 @@
#include "vendors/azure/models.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
Json ModelsService::list() { return call(Op::kListModels); }
std::vector<std::string> ModelsService::ids() {
std::vector<std::string> out;
Json doc = list();
if (const Json* data = doc.find("data"); data && data->is_array())
for (const Json& m : data->as_array())
if (const Json* id = m.find("id"); id && id->is_string()) out.push_back(id->as_string());
return out;
}
Json ModelsService::get(const std::string& id) {
Check::id("model", id);
return call(Op::kRetrieveModel, {"", {id}});
}
Json ModelsService::remove(const std::string& id) {
Check::id("model", id);
return call(Op::kDeleteModel, {"", {id}});
}
} // namespace molo::azure
+26
View File
@@ -0,0 +1,26 @@
#pragma once
// Models available to the resource (base models and fine-tuned ones). On
// Azure this lists models, not deployments: deployments are managed through
// the Azure management plane (ARM), which this package does not call.
#include <string>
#include <vector>
#include "vendors/azure/service.h"
namespace molo::azure {
class ModelsService : public AzureService {
public:
explicit ModelsService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
clients::body::Json list();
std::vector<std::string> ids();
clients::body::Json get(const std::string& id);
// v1 API only: deletes a fine-tuned model. Destructive, no default.
clients::body::Json remove(const std::string& id);
};
} // namespace molo::azure
+90
View File
@@ -0,0 +1,90 @@
#include "vendors/azure/operations.h"
#include <cstring>
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
OperationsService::OperationsService(std::shared_ptr<AzureClient> azure)
: azure_(azure),
openai_(azure, Surface::kOpenAi),
foundry_(azure, Surface::kFoundry),
safety_(azure, Surface::kSafety) {}
AzureService& OperationsService::gate(Surface s) const {
switch (s) {
case Surface::kFoundry: return foundry_;
case Surface::kSafety: return safety_;
case Surface::kOpenAi: break;
}
return openai_;
}
const CallSpec& OperationsService::lookup(const std::string& name) const {
const CallSpec* c = Calls::find(name);
if (!c) Check::fail("unknown operation '" + name + "'");
return *c;
}
bool OperationsService::destructive(const CallSpec& c) {
if (std::strcmp(c.method, "DELETE") == 0) return true;
std::string n = c.name;
for (const char* p : {"Cancel", "Delete", "Reject", "Hangup", "Pause", "Refer", "Remove"})
if (n.find(p) != std::string::npos) return true;
return false;
}
bool OperationsService::generic(const CallSpec& c) {
for (const SideSpec* s : {&c.v1, &c.dated}) {
if (!s->present()) continue;
std::string content = s->content;
if (content == "multipart") return false;
}
std::string n = c.name;
// Binary or streaming answers belong to the typed services.
return n != "CreateSpeech" && n != "DownloadFile" && n != "VideosRetrieveContent" &&
n != "RetrieveContainerFileContent" && n != "CreateRealtimeCall";
}
Query OperationsService::to_query(const std::map<std::string, std::string>& query) const {
Query q;
for (const auto& [k, v] : query) q.add(k, v);
return q;
}
OperationsService::Plan OperationsService::plan(const std::string& name, const Target& target,
const Json* body,
const std::map<std::string, std::string>& query) const {
const CallSpec& c = lookup(name);
if (!generic(c)) Check::fail(name + " needs its typed service (multipart or binary)");
AzureService& g = gate(c.surface);
Plan p;
p.method = c.method;
p.destructive = destructive(c);
Route r = g.route(c.op, target);
p.side = r.side == Side::kV1 ? "v1" : "dated";
Query q = to_query(query);
p.path = g.request_path(c.op, target, &q);
if (body) {
if (std::strcmp(c.side(r.side).content, "none") == 0) Check::fail(name + " takes no request body");
p.body = g.prepare_body(c.op, target, *body);
p.has_body = true;
} else if (c.deployment && std::strstr(c.side(r.side).content, "json") && std::strcmp(c.method, "POST") == 0) {
p.body = g.prepare_body(c.op, target, Json::object());
p.has_body = true;
}
return p;
}
Json OperationsService::invoke(const std::string& name, const Target& target, const Json* body,
const std::map<std::string, std::string>& query) {
const CallSpec& c = lookup(name);
if (!generic(c)) Check::fail(name + " needs its typed service (multipart or binary)");
Query q = to_query(query);
return gate(c.surface).call(c.op, target, body, &q);
}
} // namespace molo::azure
+53
View File
@@ -0,0 +1,53 @@
#pragma once
// OperationsService: every operation of the call table by name, for the ones
// that have no typed class (containers, conversations, evals, graders,
// videos, checkpoint tools) and for the demo's generic route. Same gateway as
// the typed services: ids validated, fields and query names checked against
// the spec, api side and api-version chosen from the settings.
#include <map>
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class OperationsService {
public:
explicit OperationsService(std::shared_ptr<AzureClient> azure);
// JSON in, JSON out. `body` may be null. `query` values are strings.
// Throws ApiError::invalid for an unknown name, wrong ids, unknown fields or
// query names, or an operation that needs multipart / streams / answers bytes
// (use the typed service for those).
clients::body::Json invoke(const std::string& name, const Target& target,
const clients::body::Json* body = nullptr,
const std::map<std::string, std::string>& query = {});
// Route and body exactly as invoke() would send them; no traffic.
struct Plan {
std::string method;
std::string path; // with query and api-version
clients::body::Json body;
bool has_body = false;
bool destructive = false;
std::string side; // "v1" or "dated"
};
Plan plan(const std::string& name, const Target& target, const clients::body::Json* body = nullptr,
const std::map<std::string, std::string>& query = {}) const;
// DELETE, cancel, reject, hangup, pause, remove: the demo asks for confirm.
static bool destructive(const CallSpec& spec);
// Calls the generic JSON path can make (not multipart-only, not binary).
static bool generic(const CallSpec& spec);
private:
AzureService& gate(Surface s) const;
const CallSpec& lookup(const std::string& name) const;
Query to_query(const std::map<std::string, std::string>& query) const;
std::shared_ptr<AzureClient> azure_;
mutable AzureService openai_, foundry_, safety_;
};
} // namespace molo::azure
+209
View File
@@ -0,0 +1,209 @@
#include "vendors/azure/realtime.h"
#include <chrono>
#include "clients/url.h"
#include "google/live/ws_error.h"
#include "google/live/ws_handshake.h"
#include "google/live/ws_url.h"
#include "llm/error.h"
#include "vendors/azure/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
namespace live = google::live;
namespace {
long long now_ms() {
return std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now().time_since_epoch())
.count();
}
[[noreturn]] void rethrow(const live::WsError& e) {
int status = 502;
std::string type = "UNAVAILABLE";
switch (e.kind()) {
case live::WsError::Kind::Invalid: status = 400; type = "INVALID_ARGUMENT"; break;
case live::WsError::Kind::Timeout: status = 504; type = "DEADLINE_EXCEEDED"; break;
case live::WsError::Kind::Handshake:
if (e.http_status() >= 400) {
status = e.http_status();
type = status == 401 ? "authentication_error" : "HANDSHAKE_REFUSED";
} else {
type = "HANDSHAKE_FAILED";
}
break;
case live::WsError::Kind::Closed: type = "CLOSED"; break;
case live::WsError::Kind::TooBig: type = "EVENT_TOO_BIG"; break;
default: break;
}
throw llm::ApiError("azure", status, type, e.what());
}
} // namespace
Json RealtimeService::create_client_secret(const Json& body) {
if (body.is_object() && body.find("expires_after")) {
const Json* ea = body.find("expires_after");
const Json* secs = ea->is_object() ? ea->find("seconds") : nullptr;
Check::require(secs && secs->is_number(), "expires_after.seconds is required");
Check::range("expires_after.seconds", secs->as_number(), 10, AzureCaps::kMaxClientSecretSeconds);
}
return call(Op::kCreateRealtimeClientSecret, {}, &body);
}
Json RealtimeService::create_session(const Json& body) {
Op op = azure_->settings().api_version.is_dated() ? Op::kCreateRealtimeSessionDated : Op::kCreateRealtimeSession;
return call(op, {}, &body);
}
Json RealtimeService::create_transcription_session(const Json& body) {
Op op = azure_->settings().api_version.is_dated() ? Op::kCreateTranscriptionRealtimeSession
: Op::kCreateRealtimeTranscriptionSession;
return call(op, {}, &body);
}
Json RealtimeService::accept_call(const std::string& call_id, const Json& body) {
Check::id("call", call_id);
Check::has_string(body, "type");
return call(Op::kAcceptRealtimeCall, {"", {call_id}}, &body);
}
Json RealtimeService::refer_call(const std::string& call_id, const std::string& target_uri) {
Check::id("call", call_id);
Check::non_empty("target_uri", target_uri);
Check::no_control("target_uri", target_uri);
Json body = Json::object();
body.set("target_uri", Json(target_uri));
return call(Op::kReferRealtimeCall, {"", {call_id}}, &body);
}
Json RealtimeService::reject_call(const std::string& call_id, int status_code) {
Check::id("call", call_id);
Check::range("status_code", status_code, 400, 699);
Json body = Json::object();
body.set("status_code", Json(status_code));
return call(Op::kRejectRealtimeCall, {"", {call_id}}, &body);
}
Json RealtimeService::hangup_call(const std::string& call_id) {
Check::id("call", call_id);
return call(Op::kHangupRealtimeCall, {"", {call_id}});
}
std::string RealtimeUrl::build(const Settings& s, const std::string& deployment) {
Check::deployment(deployment);
Check::require(!s.endpoint.empty(), "endpoint is not set");
std::string host = s.endpoint;
if (host.rfind("https://", 0) == 0) host = "wss://" + host.substr(8);
else if (host.rfind("http://", 0) == 0) host = "ws://" + host.substr(7);
std::string dep = clients::Url::encode_component(deployment);
if (s.api_version.is_dated())
return host + "/openai/realtime?api-version=" + clients::Url::encode_component(s.api_version.value()) +
"&deployment=" + dep;
std::string url = host + "/openai/v1/realtime?model=" + dep;
if (!s.api_version.v1_query().empty()) url += "&api-version=" + s.api_version.v1_query();
return url;
}
RealtimeSocket::RealtimeSocket(std::shared_ptr<AzureClient> azure, live::TlsFactory tls)
: azure_(std::move(azure)), tls_(std::move(tls)) {}
RealtimeSocket::~RealtimeSocket() {
try {
close();
} catch (...) {
}
}
void RealtimeSocket::connect(const std::string& deployment) {
Check::require(!ws_, "socket already connected");
const Settings& s = azure_->settings();
if (s.endpoint.empty()) throw llm::NotConfiguredError("azure", Settings::kEndpointEnv);
if (s.auth() == AuthKind::kNone) throw llm::NotConfiguredError("azure", Settings::kKeyEnv);
std::string url_text = RealtimeUrl::build(s, deployment);
live::WsHeaders headers;
if (s.auth() == AuthKind::kApiKey) headers.emplace_back("api-key", s.api_key);
else headers.emplace_back("Authorization", "Bearer " + azure_->tokens()->token());
try {
live::WsUrl url = live::WsUrl::parse(url_text);
auto client = std::make_unique<live::WsClient>(live::open_stream(url, 10000, 10000, tls_));
client->handshake(url, headers, 10000);
client->set_read_timeout_ms(250);
ws_ = std::move(client);
} catch (const live::WsError& e) {
rethrow(e);
}
}
void RealtimeSocket::send(const Json& event) {
Check::require(ws_ != nullptr, "socket is not connected");
const Json* t = event.is_object() ? event.find("type") : nullptr;
Check::require(t && t->is_string() && !t->as_string().empty(), "event needs a string type");
std::string text = event.dump();
Check::require(text.size() <= kMaxEventBytes, "event over the byte cap");
try {
ws_->send_text(text);
} catch (const live::WsError& e) {
rethrow(e);
}
}
RealtimeStats RealtimeSocket::run(const Sink& sink, int seconds) {
Check::require(ws_ != nullptr, "socket is not connected");
Check::range("seconds", seconds, 1, kMaxSessionSeconds);
RealtimeStats stats;
long long deadline = now_ms() + static_cast<long long>(seconds) * 1000;
try {
for (;;) {
if (now_ms() >= deadline) {
stats.stop_reason = "timeout";
return stats;
}
auto m = ws_->receive();
if (!m) continue;
if (m->kind == live::WsMessage::Kind::Close) {
stats.stop_reason = "closed";
stats.close_code = m->code;
return stats;
}
if (m->kind == live::WsMessage::Kind::Binary) continue;
++stats.events;
stats.bytes += m->data.size();
Json ev;
try {
ev = Json::parse(m->data);
} catch (const std::exception&) {
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "server event is not JSON");
}
if (!sink(ev)) {
stats.stop_reason = "sink_stopped";
return stats;
}
if (stats.events >= kMaxSessionEvents) {
stats.stop_reason = "event_cap";
return stats;
}
if (stats.bytes >= kMaxSessionBytes) {
stats.stop_reason = "byte_cap";
return stats;
}
}
} catch (const live::WsError& e) {
rethrow(e);
}
}
void RealtimeSocket::close() {
if (!ws_) return;
try {
ws_->close(1000, "");
} catch (...) {
}
ws_->shutdown();
ws_.reset();
}
} // namespace molo::azure
+85
View File
@@ -0,0 +1,85 @@
#pragma once
// Realtime (gpt-realtime, gpt-4o-realtime-preview): the REST creators of
// sessions and ephemeral client secrets, the WebSocket URL builder, and a
// small WebSocket session over the RFC 6455 client of src/google/live
// (WsClient, reused, not copied).
//
// v1 wss://<host>/openai/v1/realtime?model=<deployment>
// dated wss://<host>/openai/realtime?api-version=<v>&deployment=<deployment>
// The credential travels in a header (api-key, or Authorization: Bearer for
// Entra), never in the URL.
#include <functional>
#include <memory>
#include <string>
#include "google/live/ws_client.h"
#include "google/live/ws_stream.h"
#include "vendors/azure/service.h"
namespace molo::azure {
class RealtimeService : public AzureService {
public:
explicit RealtimeService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// Ephemeral client secret for browsers and devices (v1 API). The answer
// holds the secret in `value`: hand it to the client, never log it.
// body: session, expires_after ({"anchor":"created_at","seconds":10..7200}).
clients::body::Json create_client_secret(const clients::body::Json& body = clients::body::Json::object());
// Session creator: /realtime/sessions (v1) or /realtimeapi/sessions (dated).
clients::body::Json create_session(const clients::body::Json& body);
clients::body::Json create_transcription_session(
const clients::body::Json& body = clients::body::Json::object());
// SIP / WebRTC call control (v1 API). Destructive ones have no defaults.
clients::body::Json accept_call(const std::string& call_id, const clients::body::Json& body);
clients::body::Json refer_call(const std::string& call_id, const std::string& target_uri);
clients::body::Json reject_call(const std::string& call_id, int status_code);
clients::body::Json hangup_call(const std::string& call_id);
};
class RealtimeUrl {
public:
// The WebSocket URL without credentials. Throws on a bad deployment.
static std::string build(const Settings& settings, const std::string& deployment);
};
struct RealtimeStats {
std::size_t events = 0;
std::size_t bytes = 0;
std::string stop_reason; // sink_stopped, closed, timeout, event_cap, byte_cap
int close_code = 0;
};
class RealtimeSocket {
public:
static constexpr std::size_t kMaxEventBytes = 1024 * 1024;
static constexpr std::size_t kMaxSessionEvents = 100000;
static constexpr std::size_t kMaxSessionBytes = 64u * 1024u * 1024u;
static constexpr int kMaxSessionSeconds = 3600;
using Sink = std::function<bool(const clients::body::Json& event)>; // false: stop
explicit RealtimeSocket(std::shared_ptr<AzureClient> azure, google::live::TlsFactory tls = nullptr);
~RealtimeSocket();
bool connected() const { return ws_ != nullptr; }
// Connects and upgrades. Throws NotConfiguredError (503) before any traffic
// when there is no credential or endpoint.
void connect(const std::string& deployment);
// Sends one client event (a JSON object with a string `type`).
void send(const clients::body::Json& event);
// Receives until the sink returns false, the server closes, `seconds` pass
// (1..3600) or a cap is hit.
RealtimeStats run(const Sink& sink, int seconds);
void close();
private:
std::shared_ptr<AzureClient> azure_;
google::live::TlsFactory tls_;
std::unique_ptr<google::live::WsClient> ws_;
};
} // namespace molo::azure
+58
View File
@@ -0,0 +1,58 @@
#include "vendors/azure/responses.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
void ResponsesService::validate(const Json& body) {
Check::object("body", body);
Check::has_string(body, "model");
Check::deployment(body.find("model")->as_string());
}
Json ResponsesService::create(const Json& body) {
validate(body);
const Json* s = body.find("stream");
Check::require(!s || (s->is_bool() && !s->as_bool()), "use stream() for stream=true");
return call(Op::kCreateResponse, {}, &body);
}
void ResponsesService::stream(const Json& body, const EventHandler& handler) {
validate(body);
call_stream(Op::kCreateResponse, {}, body, handler);
}
Json ResponsesService::retrieve(const std::string& id, const std::vector<std::string>& include) {
Check::id("response", id);
Query q;
q.add_list("include", include);
return call(Op::kGetResponse, {"", {id}}, nullptr, &q);
}
Json ResponsesService::remove(const std::string& id) {
Check::id("response", id);
return call(Op::kDeleteResponse, {"", {id}});
}
Json ResponsesService::cancel(const std::string& id) {
Check::id("response", id);
return call(Op::kCancelResponse, {"", {id}});
}
Json ResponsesService::input_items(const std::string& id, const ListParams& params) {
Check::id("response", id);
params.validate();
Query q;
params.apply(q);
return call(Op::kListInputItems, {"", {id}}, nullptr, &q);
}
Listing ResponsesService::input_items_all(const std::string& id, const ListParams& params,
std::size_t max_pages) {
Check::id("response", id);
return walk(Op::kListInputItems, {"", {id}}, params, Query(), max_pages);
}
} // namespace molo::azure
+37
View File
@@ -0,0 +1,37 @@
#pragma once
// Responses API. On Azure the `model` field of the body is the deployment
// name on both APIs (v1: /openai/v1/responses, dated: /openai/responses).
#include <functional>
#include <string>
#include <vector>
#include "vendors/azure/service.h"
namespace molo::azure {
class ResponsesService : public AzureService {
public:
using EventHandler = std::function<bool(const llm::SseEvent&)>; // false: stop
explicit ResponsesService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
// body: model (the deployment, required), input, plus any spec field.
clients::body::Json create(const clients::body::Json& body);
void stream(const clients::body::Json& body, const EventHandler& handler);
clients::body::Json retrieve(const std::string& id, const std::vector<std::string>& include = {});
// DELETE: removes the stored response. No default.
clients::body::Json remove(const std::string& id);
// Only responses created with background=true can be cancelled.
clients::body::Json cancel(const std::string& id);
clients::body::Json input_items(const std::string& id, const ListParams& params = {});
Listing input_items_all(const std::string& id, const ListParams& params = {},
std::size_t max_pages = 5);
private:
static void validate(const clients::body::Json& body);
};
} // namespace molo::azure
+121
View File
@@ -0,0 +1,121 @@
#include "vendors/azure/safety.h"
#include "vendors/azure/caps.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
void SafetyService::check_blocklist_name(const std::string& name) {
Check::require(!name.empty() && name.size() <= 64, "blocklist name must be 1..64 characters");
for (char c : name)
if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '.' ||
c == '_' || c == '~' || c == '-'))
Check::fail("blocklist name allows only 0-9 A-Z a-z . _ ~ -");
}
Json SafetyService::analyze_text(const Json& body) {
Check::has_string(body, "text");
Check::max_chars("text", body.find("text")->as_string(), AzureCaps::kMaxSafetyTextChars);
return call(Op::kSafetyTextOperationsAnalyzeText, {}, &body);
}
Json SafetyService::analyze_image(const Json& body) {
Check::object("body", body);
const Json* img = body.find("image");
Check::require(img && img->is_object(), "image is required");
const Json* content = img->find("content");
Check::require(content || img->find("blobUrl"), "image needs content (base64) or blobUrl");
if (content && content->is_string())
Check::require(content->as_string().size() <= AzureCaps::kMaxSafetyImageBytes * 4 / 3 + 8,
"image over 7.5 MiB");
return call(Op::kSafetyImageOperationsAnalyzeImage, {}, &body);
}
Json SafetyService::shield_prompt(const Json& body) {
Check::object("body", body);
Check::require(body.find("userPrompt") || body.find("documents"), "userPrompt or documents is required");
if (const Json* d = body.find("documents"); d && d->is_array())
Check::max_count("documents", d->as_array().size(), AzureCaps::kMaxShieldDocuments);
return call(Op::kSafetyTextOperationsShieldPrompt, {}, &body);
}
Json SafetyService::detect_protected_material(const std::string& text) {
Check::non_empty("text", text);
Check::max_chars("text", text, AzureCaps::kMaxSafetyTextChars);
Json body = Json::object();
body.set("text", Json(text));
return call(Op::kSafetyTextOperationsDetectTextProtectedMaterial, {}, &body);
}
Json SafetyService::list_blocklists() { return call(Op::kSafetyTextBlocklistsListTextBlocklists); }
Json SafetyService::get_blocklist(const std::string& name) {
check_blocklist_name(name);
return call(Op::kSafetyTextBlocklistsGetTextBlocklist, {"", {name}});
}
Json SafetyService::create_or_update_blocklist(const std::string& name, const std::string& description) {
check_blocklist_name(name);
Check::max_chars("description", description, 1024);
Json body = Json::object();
if (!description.empty()) body.set("description", Json(description));
Op op = Op::kSafetyTextBlocklistsCreateOrUpdateTextBlocklist;
Json prepared = prepare_body(op, {"", {name}}, body);
std::string text = prepared.dump();
llm::RawResponse r = client_->request_raw(Calls::method(op), request_path(op, {"", {name}}), &text,
"application/merge-patch+json");
return r.body.find_first_not_of(" \t\r\n") == std::string::npos ? Json::object() : Json::parse(r.body);
}
Json SafetyService::remove_blocklist(const std::string& name) {
check_blocklist_name(name);
return call(Op::kSafetyTextBlocklistsDeleteTextBlocklist, {"", {name}});
}
Json SafetyService::add_blocklist_items(const std::string& name, const Json& items) {
check_blocklist_name(name);
Check::require(items.is_array() && !items.as_array().empty(), "items must be a non-empty array");
Check::max_count("items", items.as_array().size(), AzureCaps::kMaxBlocklistItems);
for (const Json& it : items.as_array()) {
Check::has_string(it, "text");
Check::max_chars("item text", it.find("text")->as_string(), AzureCaps::kMaxBlocklistItemChars);
}
Json body = Json::object();
body.set("blocklistItems", items);
return call(Op::kSafetyTextBlocklistsAddOrUpdateBlocklistItems, {"", {name}}, &body);
}
Json SafetyService::remove_blocklist_items(const std::string& name, const std::vector<std::string>& item_ids) {
check_blocklist_name(name);
Check::require(!item_ids.empty(), "item_ids must not be empty");
Check::max_count("item_ids", item_ids.size(), AzureCaps::kMaxBlocklistItems);
Json ids = Json::array();
for (const std::string& i : item_ids) {
Check::id("item", i);
ids.push(Json(i));
}
Json body = Json::object();
body.set("blocklistItemIds", ids);
return call(Op::kSafetyTextBlocklistsRemoveBlocklistItems, {"", {name}}, &body);
}
Json SafetyService::list_blocklist_items(const std::string& name, std::optional<int> maxpagesize,
std::optional<int> skip) {
check_blocklist_name(name);
if (maxpagesize) Check::range("maxpagesize", *maxpagesize, 1, 1000);
if (skip) Check::range("skip", *skip, 0, 1000000);
Query q;
q.add_int("maxpagesize", maxpagesize);
q.add_int("skip", skip);
return call(Op::kSafetyTextBlocklistsListTextBlocklistItems, {"", {name}}, nullptr, &q);
}
Json SafetyService::get_blocklist_item(const std::string& name, const std::string& item_id) {
check_blocklist_name(name);
Check::id("item", item_id);
return call(Op::kSafetyTextBlocklistsGetTextBlocklistItem, {"", {name, item_id}});
}
} // namespace molo::azure
+47
View File
@@ -0,0 +1,47 @@
#pragma once
// Azure AI Content Safety (api-version 2024-09-01): analyze text and images
// for harm categories (Hate, SelfHarm, Sexual, Violence), Prompt Shields,
// protected material detection, and custom text blocklists. Own endpoint
// (https://<resource>.cognitiveservices.azure.com/contentsafety) and key
// header (Ocp-Apim-Subscription-Key), or an Entra bearer token.
#include <string>
#include <vector>
#include "vendors/azure/service.h"
namespace molo::azure {
class SafetyService : public AzureService {
public:
explicit SafetyService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kSafety) {}
// body: text (required, <= 10000 chars), categories, blocklistNames,
// haltOnBlocklistHit, outputType.
clients::body::Json analyze_text(const clients::body::Json& body);
// body: image ({content: base64} or {blobUrl}), categories, outputType.
clients::body::Json analyze_image(const clients::body::Json& body);
// Prompt Shields: userPrompt and/or documents (<= 5).
clients::body::Json shield_prompt(const clients::body::Json& body);
clients::body::Json detect_protected_material(const std::string& text);
// Blocklists. Names: 1..64 of [0-9A-Za-z._~-].
clients::body::Json list_blocklists();
clients::body::Json get_blocklist(const std::string& name);
// PATCH (merge-patch) creates or updates; description <= 1024 chars.
clients::body::Json create_or_update_blocklist(const std::string& name, const std::string& description);
clients::body::Json remove_blocklist(const std::string& name); // destructive
// items: [{description, text}] (1..100, text <= 128 chars).
clients::body::Json add_blocklist_items(const std::string& name, const clients::body::Json& items);
clients::body::Json remove_blocklist_items(const std::string& name, const std::vector<std::string>& item_ids);
// One page; `truncated` stays visible in the answer as nextLink.
clients::body::Json list_blocklist_items(const std::string& name, std::optional<int> maxpagesize = std::nullopt,
std::optional<int> skip = std::nullopt);
clients::body::Json get_blocklist_item(const std::string& name, const std::string& item_id);
static void check_blocklist_name(const std::string& name);
};
} // namespace molo::azure
+291
View File
@@ -0,0 +1,291 @@
#include "vendors/azure/service.h"
#include <cstring>
#include "clients/url.h"
#include "llm/error.h"
#include "vendors/azure/caps.h"
namespace molo::azure {
using clients::body::Json;
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
std::string strip_brackets(const std::string& name) {
if (name.size() > 2 && name.compare(name.size() - 2, 2, "[]") == 0) return name.substr(0, name.size() - 2);
return name;
}
std::string encode_name(const std::string& k) {
std::string name;
for (char c : k)
name += (c == '[' || c == ']') ? std::string(1, c) : clients::Url::encode_component(std::string(1, c));
return name;
}
bool has_json(const char* content) { return std::strstr(content, "json") != nullptr; }
} // namespace
Query& Query::add(const std::string& name, const std::string& value) {
if (value.empty()) return *this;
for (char c : value)
if (static_cast<unsigned char>(c) < 0x20) bad("query value has control characters");
if (value.size() > 1024) bad("query value is too long");
pairs_.emplace_back(name, value);
return *this;
}
Query& Query::add_int(const std::string& name, const std::optional<int>& value) {
if (value) add(name, std::to_string(*value));
return *this;
}
Query& Query::add_bool(const std::string& name, const std::optional<bool>& value) {
if (value) add(name, *value ? "true" : "false");
return *this;
}
Query& Query::add_list(const std::string& name, const std::vector<std::string>& values) {
if (values.size() > 16) bad(name + ": too many values");
for (const std::string& v : values) {
if (v.empty()) bad(name + ": empty value");
pairs_.emplace_back(name + "[]", v);
}
return *this;
}
void ListParams::validate() const {
if (limit && (*limit < 1 || *limit > AzureCaps::kMaxPageLimit))
bad("limit must be 1.." + std::to_string(AzureCaps::kMaxPageLimit));
if (!order.empty() && order != "asc" && order != "desc") bad("order must be asc or desc");
if (!after.empty() && !llm::Service::valid_id(after)) bad("bad after cursor");
if (!before.empty() && !llm::Service::valid_id(before)) bad("bad before cursor");
}
void ListParams::apply(Query& q) const {
q.add_int("limit", limit);
q.add("order", order);
q.add("after", after);
q.add("before", before);
}
AzureService::AzureService(std::shared_ptr<AzureClient> azure, Surface surface)
: llm::Service(azure->api(surface)), azure_(azure), surface_(surface) {}
void AzureService::guard(Op op) const {
if (Calls::spec(op).surface != surface_)
bad(std::string(Calls::spec(op).name) + " does not belong to this service");
}
Route AzureService::route(Op op, const Target& target) const {
const CallSpec& c = Calls::spec(op);
const Settings& s = azure_->settings();
Route r;
switch (c.surface) {
case Surface::kFoundry:
r.side = Side::kDated;
r.version = s.foundry_version;
break;
case Surface::kSafety:
r.side = Side::kDated;
r.version = s.safety_version;
break;
case Surface::kOpenAi: {
const ApiVersion& v = s.api_version;
Side pref = v.is_dated() ? Side::kDated : Side::kV1;
Side other = pref == Side::kDated ? Side::kV1 : Side::kDated;
r.side = Calls::has(op, pref) ? pref : other;
if (r.side == Side::kV1) r.version = v.is_v1_family() ? v.v1_query() : "preview";
else r.version = v.is_dated() ? v.value() : ApiVersion::kDatedFallback;
break;
}
}
if (c.deployment && r.side == Side::kDated) {
if (target.deployment.empty() || target.deployment.size() > AzureCaps::kMaxDeploymentChars)
bad("a deployment name of 1.." + std::to_string(AzureCaps::kMaxDeploymentChars) + " characters is required");
}
r.path = Calls::prefix(op, r.side) + Calls::render(op, r.side, target.ids, target.deployment);
return r;
}
std::string AzureService::request_path(Op op, const Target& target, const Query* query) const {
Route r = route(op, target);
std::string out = r.path;
char sep = '?';
if (query) {
for (const auto& [k, v] : query->pairs()) {
if (!Calls::accepts_query(op, r.side, k))
bad("query '" + k + "' is not a parameter of " + Calls::spec(op).side(r.side).op_id +
" on the " + (r.side == Side::kV1 ? "v1" : "dated") + " API");
out += sep;
out += encode_name(k) + "=" + clients::Url::encode_component(v);
sep = '&';
}
}
if (!r.version.empty()) {
out += sep;
out += "api-version=" + clients::Url::encode_component(r.version);
}
return out;
}
Json AzureService::prepare_body(Op op, const Target& target, const Json& body) const {
if (!body.is_object()) bad("body must be a JSON object");
Route r = route(op, target);
const CallSpec& c = Calls::spec(op);
Json out = body;
Json::Object& obj = out.as_object();
if (c.deployment) {
auto it = obj.find("model");
if (it != obj.end()) {
if (!it->second.is_string() || it->second.as_string() != target.deployment)
bad("model must equal the deployment name (or be left out)");
if (r.side == Side::kDated) obj.erase(it);
} else if (r.side == Side::kV1) {
if (!llm::Service::valid_id(target.deployment) ||
target.deployment.size() > AzureCaps::kMaxDeploymentChars)
bad("a deployment name is required");
obj["model"] = Json(target.deployment);
}
}
for (const auto& [k, v] : obj) {
(void)v;
if (!Calls::accepts_field(op, r.side, k))
bad("field '" + k + "' is not accepted by " + c.side(r.side).op_id + " on the " +
(r.side == Side::kV1 ? "v1" : "dated") + " API");
}
return out;
}
clients::body::MultipartBody AzureService::prepare_form(
Op op, const Target& target, const clients::body::MultipartBody& form) const {
Route r = route(op, target);
const CallSpec& c = Calls::spec(op);
clients::body::MultipartBody out = form;
bool has_model = false;
for (const auto& part : form.parts()) {
std::string name = strip_brackets(part.name);
if (name == "model" && c.deployment) {
has_model = true;
if (part.data != target.deployment) bad("model must equal the deployment name (or be left out)");
if (r.side == Side::kDated) bad("model is not a form field of the dated API (the deployment is in the URL)");
continue;
}
if (!Calls::accepts_field(op, r.side, name))
bad("form field '" + name + "' is not accepted by " + c.side(r.side).op_id);
}
if (c.deployment && r.side == Side::kV1 && !has_model) out.add_text("model", target.deployment);
return out;
}
Json AzureService::call(Op op, const Target& target, const Json* body, const Query* query,
const llm::Headers& headers) {
guard(op);
const CallSpec& c = Calls::spec(op);
Route r = route(op, target);
std::string path = request_path(op, target, query);
Json prepared;
const Json* send = nullptr;
if (body) {
if (std::strcmp(c.side(r.side).content, "none") == 0)
bad(std::string(c.name) + " takes no request body");
prepared = prepare_body(op, target, *body);
send = &prepared;
} else if (has_json(c.side(r.side).content) && std::strcmp(c.method, "POST") == 0 &&
c.deployment) {
prepared = prepare_body(op, target, Json::object());
send = &prepared;
}
return client_->request(Calls::method(op), path, send, headers);
}
llm::RawResponse AzureService::call_raw(Op op, const Target& target, const Query* query,
const llm::Headers& headers) {
guard(op);
return client_->request_raw(Calls::method(op), request_path(op, target, query), nullptr, "",
headers);
}
llm::RawResponse AzureService::call_json_raw(Op op, const Target& target, const Json& body,
const llm::Headers& headers) {
guard(op);
Json prepared = prepare_body(op, target, body);
std::string text = prepared.dump();
return client_->request_raw(Calls::method(op), request_path(op, target), &text,
"application/json", headers);
}
Json AzureService::call_multipart(Op op, const Target& target,
const clients::body::MultipartBody& form,
const llm::Headers& headers) {
guard(op);
clients::body::MultipartBody prepared = prepare_form(op, target, form);
return client_->post_multipart(request_path(op, target), prepared, headers);
}
llm::RawResponse AzureService::call_multipart_raw(Op op, const Target& target,
const clients::body::MultipartBody& form,
const llm::Headers& headers) {
guard(op);
clients::body::MultipartBody prepared = prepare_form(op, target, form);
std::string encoded = prepared.encode();
return client_->request_raw(clients::Method::kPost, request_path(op, target), &encoded,
prepared.content_type(), headers);
}
void AzureService::call_stream(Op op, const Target& target, const Json& body, const EventSink& sink,
const llm::Headers& headers) {
guard(op);
Route r = route(op, target);
Json prepared = prepare_body(op, target, body);
if (!Calls::accepts_field(op, r.side, "stream"))
bad(std::string(Calls::spec(op).name) + " does not stream");
prepared.set("stream", Json(true));
std::size_t seen = 0;
client_->stream(request_path(op, target), prepared,
[&](const llm::SseEvent& ev) {
if (++seen > AzureCaps::kMaxStreamEvents)
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "stream over the event cap");
return sink(ev);
},
headers);
}
Listing AzureService::walk(Op op, const Target& target, const ListParams& params, const Query& base,
std::size_t max_pages) {
guard(op);
params.validate();
if (max_pages < 1 || max_pages > AzureCaps::kMaxListPages)
bad("max_pages must be 1.." + std::to_string(AzureCaps::kMaxListPages));
Listing out;
std::string after = params.after;
for (std::size_t page = 0; page < max_pages; ++page) {
Query q = base;
q.add_int("limit", params.limit);
q.add("order", params.order);
q.add("before", params.before);
q.add("after", after);
Json doc = call(op, target, nullptr, &q);
++out.pages;
if (const Json* data = doc.find("data"); data && data->is_array())
for (const Json& item : data->as_array()) out.items.push_back(item);
const Json* more = doc.find("has_more");
bool has_more = more && more->is_bool() && more->as_bool();
std::string last = string_at(doc, "last_id");
if (!has_more) return out;
if (last.empty() || last == after) {
out.truncated = true; // more pages, no usable cursor: stop loudly
return out;
}
after = last;
if (page + 1 == max_pages) {
out.truncated = true;
out.next_after = last;
}
}
return out;
}
} // namespace molo::azure
+117
View File
@@ -0,0 +1,117 @@
#pragma once
// AzureService: the gateway every service class goes through. It
// - picks the API side for an operation (v1 or dated) from the configured
// api version, and the api-version query value that side needs;
// - builds the path from the call table (ids validated as single path
// segments, the deployment placed in the URL on the dated API and in the
// body `model` on the v1 API);
// - rejects request fields and query names the spec of that side does not
// list (loud, before any traffic);
// - sends JSON / multipart / raw / SSE through llm::ApiClient;
// - walks cursor lists with a page cap and reports truncation.
#include <functional>
#include <optional>
#include <string>
#include <vector>
#include "clients/body/form.h"
#include "clients/body/json.h"
#include "llm/api_client.h"
#include "llm/sse.h"
#include "vendors/azure/calls.h"
#include "vendors/azure/client.h"
namespace molo::azure {
// Where an operation goes: the deployment (or empty) and the path ids in order.
struct Target {
std::string deployment;
std::vector<std::string> ids;
};
// Query builder; names are checked against the call row when the path is built.
class Query {
public:
Query& add(const std::string& name, const std::string& value); // skipped when empty
Query& add_int(const std::string& name, const std::optional<int>& value);
Query& add_bool(const std::string& name, const std::optional<bool>& value);
Query& add_list(const std::string& name, const std::vector<std::string>& values);
const std::vector<std::pair<std::string, std::string>>& pairs() const { return pairs_; }
bool empty() const { return pairs_.empty(); }
private:
std::vector<std::pair<std::string, std::string>> pairs_;
};
struct ListParams {
std::optional<int> limit; // 1..100
std::string after; // cursor (an object id)
std::string before;
std::string order; // "", "asc", "desc"
void validate() const;
void apply(Query& q) const;
};
struct Listing {
std::vector<clients::body::Json> items;
bool truncated = false; // the page cap was hit while the server had more
std::string next_after; // cursor to continue from when truncated
std::size_t pages = 0;
};
struct Route {
Side side = Side::kV1;
std::string path; // prefix + rendered spec path
std::string version; // api-version query value, empty = omitted
};
using EventSink = std::function<bool(const llm::SseEvent&)>; // false: stop
class AzureService : public llm::Service {
public:
AzureService(std::shared_ptr<AzureClient> azure, Surface surface);
const AzureClient& azure() const { return *azure_; }
Surface surface() const { return surface_; }
// Side, path and api-version of an operation for a target (no traffic).
Route route(Op op, const Target& target = {}) const;
// Route path plus ?query&api-version=... exactly as sent.
std::string request_path(Op op, const Target& target = {}, const Query* query = nullptr) const;
// The JSON body as sent (deployment moved, fields checked). Throws on a bad body.
clients::body::Json prepare_body(Op op, const Target& target, const clients::body::Json& body) const;
// Any operation of the surface by table entry. Used by the typed services
// and by the generic route of the demo.
clients::body::Json call(Op op, const Target& target = {},
const clients::body::Json* body = nullptr, const Query* query = nullptr,
const llm::Headers& headers = {});
llm::RawResponse call_raw(Op op, const Target& target = {}, const Query* query = nullptr,
const llm::Headers& headers = {});
clients::body::Json call_multipart(Op op, const Target& target,
const clients::body::MultipartBody& form,
const llm::Headers& headers = {});
// Multipart POST whose answer may be text (srt, vtt) or binary.
llm::RawResponse call_multipart_raw(Op op, const Target& target,
const clients::body::MultipartBody& form,
const llm::Headers& headers = {});
// POST with `stream: true` decoded as SSE; at most AzureCaps::kMaxStreamEvents events.
void call_stream(Op op, const Target& target, const clients::body::Json& body,
const EventSink& sink, const llm::Headers& headers = {});
// Binary-answer POST (speech): the body as JSON, the answer as bytes.
llm::RawResponse call_json_raw(Op op, const Target& target, const clients::body::Json& body,
const llm::Headers& headers = {});
Listing walk(Op op, const Target& target, const ListParams& params, const Query& base,
std::size_t max_pages);
protected:
void guard(Op op) const;
clients::body::MultipartBody prepare_form(Op op, const Target& target,
const clients::body::MultipartBody& form) const;
std::shared_ptr<AzureClient> azure_;
Surface surface_;
};
} // namespace molo::azure
+112
View File
@@ -0,0 +1,112 @@
#include "vendors/azure/settings.h"
#include "llm/config.h"
#include "llm/error.h"
#include "vendors/azure/caps.h"
namespace molo::azure {
namespace {
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
bool name_char(char c) {
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
c == '.' || c == '_';
}
void check_name(const std::string& what, const std::string& value) {
if (value.empty() || value.size() > 128) bad(what + " must be 1..128 characters");
for (char c : value)
if (!name_char(c)) bad(what + " has invalid characters");
}
std::string trimmed(std::string s) {
while (!s.empty() && (s.back() == ' ' || s.back() == '\t' || s.back() == '\n' || s.back() == '\r'))
s.pop_back();
std::size_t i = 0;
while (i < s.size() && (s[i] == ' ' || s[i] == '\t')) ++i;
return s.substr(i);
}
} // namespace
void EntraConfig::validate() const {
check_name("tenant id", tenant_id);
check_name("client id", client_id);
if (client_secret.empty() || client_secret.size() > AzureCaps::kMaxSecretChars)
bad("client secret must be 1.." + std::to_string(AzureCaps::kMaxSecretChars) + " characters");
for (char c : client_secret)
if (static_cast<unsigned char>(c) < 0x20) bad("client secret has control characters");
Settings::normalize_endpoint(authority);
if (scope.empty() || scope.size() > 256) bad("scope must be 1..256 characters");
for (char c : scope)
if (static_cast<unsigned char>(c) <= 0x20) bad("scope has invalid characters");
}
std::string EntraConfig::token_url() const {
return Settings::normalize_endpoint(authority) + "/" + tenant_id + "/oauth2/v2.0/token";
}
std::string Settings::normalize_endpoint(const std::string& raw) {
std::string e = trimmed(raw);
if (e.empty()) return e;
if (e.size() > AzureCaps::kMaxEndpointChars) bad("endpoint is too long");
std::string scheme, rest;
if (e.rfind("https://", 0) == 0) {
scheme = "https://";
} else if (e.rfind("http://127.0.0.1", 0) == 0 || e.rfind("http://localhost", 0) == 0) {
scheme = "http://";
} else {
bad("endpoint must start with https://");
}
rest = e.substr(scheme.size());
while (!rest.empty() && rest.back() == '/') rest.pop_back();
if (rest.empty()) bad("endpoint has no host");
e = scheme + rest;
for (char c : rest) {
if (static_cast<unsigned char>(c) <= 0x20) bad("endpoint has invalid characters");
if (c == '/' || c == '?' || c == '#' || c == '@' || c == '\\')
bad("endpoint must be scheme and host only (no path, query or credentials)");
}
return e;
}
AuthKind Settings::auth() const {
if (!api_key.empty()) return AuthKind::kApiKey;
if (entra.complete()) return AuthKind::kEntra;
return AuthKind::kNone;
}
void Settings::validate() const {
normalize_endpoint(endpoint);
normalize_endpoint(foundry_endpoint);
normalize_endpoint(safety_endpoint);
for (const std::string* k : {&api_key, &safety_key}) {
if (k->size() > AzureCaps::kMaxSecretChars) bad("api key is too long");
for (char c : *k)
if (static_cast<unsigned char>(c) <= 0x20) bad("api key has invalid characters");
}
if (entra.complete()) entra.validate();
}
Settings Settings::from_env() {
using llm::Config;
Settings s;
s.endpoint = normalize_endpoint(Config::get(kEndpointEnv));
s.foundry_endpoint = normalize_endpoint(Config::get(kFoundryEndpointEnv));
if (s.foundry_endpoint.empty()) s.foundry_endpoint = s.endpoint;
s.safety_endpoint = normalize_endpoint(Config::get(kSafetyEndpointEnv));
if (s.safety_endpoint.empty()) s.safety_endpoint = s.endpoint;
s.api_key = trimmed(Config::get(kKeyEnv));
s.safety_key = trimmed(Config::get(kSafetyKeyEnv));
if (s.safety_key.empty()) s.safety_key = s.api_key;
s.api_version = ApiVersion::parse(trimmed(Config::get(kVersionEnv)));
s.entra.tenant_id = trimmed(Config::get(kTenantEnv));
s.entra.client_id = trimmed(Config::get(kClientIdEnv));
s.entra.client_secret = Config::get(kClientSecretEnv);
std::string authority = trimmed(Config::get(kAuthorityEnv));
if (!authority.empty()) s.entra.authority = authority;
s.validate();
return s;
}
} // namespace molo::azure
+82
View File
@@ -0,0 +1,82 @@
#pragma once
// Settings: everything the package needs to address and authenticate against
// an Azure resource, read from llm::Config (environment, then the .env.json
// files). Validation is loud and happens before any network traffic.
//
// AZURE_OPENAI_ENDPOINT https://<resource>.openai.azure.com (no path)
// AZURE_OPENAI_API_KEY resource key, sent as the api-key header
// AZURE_OPENAI_API_VERSION v1 (default), preview, or YYYY-MM-DD[-preview]
// AZURE_TENANT_ID / AZURE_CLIENT_ID / AZURE_CLIENT_SECRET
// Microsoft Entra client credentials (used when
// no api key is set)
// Optional extras (same style):
// AZURE_AI_ENDPOINT Foundry model inference host, default = endpoint
// AZURE_CONTENT_SAFETY_ENDPOINT / AZURE_CONTENT_SAFETY_KEY
// default = endpoint / api key
// AZURE_AUTHORITY_HOST default https://login.microsoftonline.com
#include <string>
#include "vendors/azure/api_version.h"
namespace molo::azure {
enum class AuthKind { kNone, kApiKey, kEntra };
struct EntraConfig {
static constexpr const char* kDefaultAuthority = "https://login.microsoftonline.com";
static constexpr const char* kDefaultScope = "https://cognitiveservices.azure.com/.default";
std::string tenant_id;
std::string client_id;
std::string client_secret;
std::string authority = kDefaultAuthority;
std::string scope = kDefaultScope;
bool complete() const {
return !tenant_id.empty() && !client_id.empty() && !client_secret.empty();
}
// Throws ApiError::invalid on a malformed tenant / client id / authority.
void validate() const;
// POST target: <authority>/<tenant>/oauth2/v2.0/token
std::string token_url() const;
};
struct Settings {
static constexpr const char* kVendor = "azure";
static constexpr const char* kEndpointEnv = "AZURE_OPENAI_ENDPOINT";
static constexpr const char* kKeyEnv = "AZURE_OPENAI_API_KEY";
static constexpr const char* kVersionEnv = "AZURE_OPENAI_API_VERSION";
static constexpr const char* kTenantEnv = "AZURE_TENANT_ID";
static constexpr const char* kClientIdEnv = "AZURE_CLIENT_ID";
static constexpr const char* kClientSecretEnv = "AZURE_CLIENT_SECRET";
static constexpr const char* kFoundryEndpointEnv = "AZURE_AI_ENDPOINT";
static constexpr const char* kSafetyEndpointEnv = "AZURE_CONTENT_SAFETY_ENDPOINT";
static constexpr const char* kSafetyKeyEnv = "AZURE_CONTENT_SAFETY_KEY";
static constexpr const char* kAuthorityEnv = "AZURE_AUTHORITY_HOST";
std::string endpoint; // OpenAI resource
std::string foundry_endpoint; // model inference (services.ai.azure.com)
std::string safety_endpoint; // content safety (cognitiveservices.azure.com)
std::string api_key;
std::string safety_key;
ApiVersion api_version;
std::string foundry_version = ApiVersion::kFoundryDefault;
std::string safety_version = ApiVersion::kSafetyDefault;
EntraConfig entra;
// Reads the environment. Malformed values throw ApiError::invalid; missing
// ones leave the field empty (the clients then report not configured).
static Settings from_env();
// Validates and normalizes: https only (http for loopback), host only, no
// path, query, fragment or userinfo, trailing slash removed. Empty stays empty.
static std::string normalize_endpoint(const std::string& endpoint);
AuthKind auth() const;
bool configured() const { return !endpoint.empty() && auth() != AuthKind::kNone; }
// Throws ApiError::invalid when a field is malformed (used by from_env and tests).
void validate() const;
};
} // namespace molo::azure
Binary file not shown.
+151
View File
@@ -0,0 +1,151 @@
#!/usr/bin/env python3
"""Check src/vendors/azure against the pinned azure-rest-api-specs documents.
python3 -I src/vendors/azure/tools/check_spec.py [local_spec_dir]
Fails (exit 1) when a row of calls_table.inc has a wrong method, path,
operationId, request content kind, request property list or query list on
either side; when a spec operation of a covered document is not in the table;
when the table lists a name twice; or when the constants in the C++ sources
(api versions, header names, Entra scope) disagree with the documents.
Python 3 stdlib only. local_spec_dir may hold <name>.json copies of the
documents (v1.json, v1p.json, inf25.json, auth24.json, cs24.json, mi.json);
without it the pinned revision is fetched from raw.githubusercontent.com.
"""
import os
import re
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import specdoc # noqa: E402
PKG = os.path.join(HERE, "..")
STR = re.compile(r'"([^"]*)"')
DEP = re.compile(r"^/deployments/\{[^}]+\}")
# Spec operations deliberately not in the table: none. (Older dated revisions
# than the ones in specdoc.DOCS are not read at all; see SPEC.md.)
EXCLUDED = set()
def norm(method, path):
return method.upper() + " " + re.sub(r"\{[^}]*\}", "{}", DEP.sub("", path))
def read_rows():
text = open(os.path.join(PKG, "calls_table.inc")).read()
body = text[text.index("// AZ-ROWS-BEGIN"):text.index("// AZ-ROWS-END")]
rows = []
for line in body.splitlines()[1:]:
if not line.startswith("AZ_OP("):
continue
name = line[len("AZ_OP("):].split(",")[0]
surface = re.search(r"Surface::(\w+)", line).group(1)
strs = STR.findall(line)
dep = int(re.search(r'"[A-Z]+", (\d),', line).group(1))
method = strs[0]
sides = [strs[1:7], strs[7:13]]
rows.append({"name": name, "surface": surface, "method": method, "dep": dep, "sides": sides})
return rows
def main():
local = sys.argv[1] if len(sys.argv) > 1 else None
docs = {n: specdoc.load(n, local) for n in specdoc.DOCS}
rows = read_rows()
bad = []
names = set()
covered = set() # (doc, method, path)
sides_checked = 0
for r in rows:
if r["name"] in names:
bad.append("duplicate row name " + r["name"])
names.add(r["name"])
for idx, s in enumerate(r["sides"]):
doc_name, path, opid, content, body, query = s
if not path:
continue
sides_checked += 1
doc = docs.get(doc_name)
if doc is None:
bad.append("%s: unknown document %s" % (r["name"], doc_name))
continue
item = doc["paths"].get(path)
op = item.get(r["method"].lower()) if item else None
if op is None:
bad.append("%s: %s %s missing in %s" % (r["name"], r["method"], path, doc_name))
continue
covered.add((doc_name, r["method"].lower(), path))
f = specdoc.op_facts(doc, item, op)
if f["opid"] != opid:
bad.append("%s: operationId %r != %r" % (r["name"], f["opid"], opid))
if f["content"] != content:
bad.append("%s: content %s != %s" % (r["name"], f["content"], content))
if sorted(filter(None, body.split(","))) != f["body"]:
bad.append("%s: body fields differ: table-only %s spec-only %s" % (
r["name"], sorted(set(filter(None, body.split(","))) - set(f["body"])),
sorted(set(f["body"]) - set(filter(None, body.split(","))))))
if sorted(filter(None, query.split(","))) != f["query"]:
bad.append("%s: query differs: table %s spec %s" % (r["name"], query, f["query"]))
if r["surface"] != "kOpenAi" and idx == 0:
bad.append("%s: single-sided surface uses the v1 slot" % r["name"])
if r["dep"] and idx == 1 and not DEP.match(path):
bad.append("%s: deployment flag but no deployment path" % r["name"])
# Every spec operation must be reachable through the table.
keys = set()
for r in rows:
for s in r["sides"]:
if s[1]:
keys.add((r["surface"], norm(r["method"], s[1])))
total_ops = 0
for surface, names_ in specdoc.SURFACE_DOCS.items():
sk = {"openai": "kOpenAi", "foundry": "kFoundry", "safety": "kSafety"}[surface]
for dn in names_:
for m, path, item, op in specdoc.operations(docs[dn]):
total_ops += 1
if (sk, norm(m, path)) not in keys and (dn, m, path) not in EXCLUDED:
bad.append("unwrapped spec operation %s %s %s (%s)" % (dn, m.upper(), path, op.get("operationId")))
# Second source: openai-python src/openai/lib/azure.py `_deployments_endpoints`
# (read 2026-10-25): the operations the official SDK moves under /deployments/{model}.
sdk_deployment_paths = {"/completions", "/chat/completions", "/embeddings", "/audio/transcriptions",
"/audio/translations", "/audio/speech", "/images/generations", "/images/edits"}
table_deployment_paths = {DEP.sub("", s[1]) for r in rows if r["dep"] for s in r["sides"][1:] if s[1]}
if table_deployment_paths != sdk_deployment_paths:
bad.append("deployment-scoped operations differ from the SDK: %s / %s" % (
sorted(table_deployment_paths - sdk_deployment_paths), sorted(sdk_deployment_paths - table_deployment_paths)))
if len([r for r in rows if r["dep"]]) != len(sdk_deployment_paths):
bad.append("deployment flag count differs from the SDK list")
# Constants in C++ sources.
read = lambda p: open(os.path.join(PKG, p)).read()
ver = read("api_version.h")
for const, doc_name in (("kDatedFallback", "inf25"), ("kFoundryDefault", "mi"), ("kSafetyDefault", "cs24")):
v = re.search(const + r' = "([^"]+)"', ver).group(1)
spec_v = docs[doc_name]["info"]["version"]
if v != spec_v:
bad.append("%s %s != info.version %s of %s" % (const, v, spec_v, doc_name))
enum = docs["v1p"]["components"]["schemas"]["AzureAIFoundryModelsApiVersion"]["enum"]
if sorted(enum) != ["preview", "v1"]:
bad.append("v1 api-version enum is now %s (ApiVersion knows v1, preview)" % enum)
sec = docs["v1p"]["components"]["securitySchemes"]
if sec["ApiKeyAuth"]["name"] != "api-key":
bad.append("OpenAI api key header is no longer api-key")
if docs["cs24"]["securityDefinitions"]["ApiKeyAuth"]["name"] != "Ocp-Apim-Subscription-Key":
bad.append("Content Safety key header changed")
scope = list(docs["cs24"]["securityDefinitions"]["OAuth2Auth"]["scopes"])[0]
if scope not in read("settings.h"):
bad.append("Entra scope %s not in settings.h" % scope)
if "Ocp-Apim-Subscription-Key" not in read("client.cpp"):
bad.append("client.cpp does not use Ocp-Apim-Subscription-Key")
if bad:
print("SPEC CHECK FAILED")
for b in bad:
print(" " + b)
return 1
print("spec ok: %d rows, %d table sides verified, %d spec operations seen in %d documents (rev %s)"
% (len(rows), sides_checked, total_ops, len(docs), specdoc.REV[:10]))
return 0
if __name__ == "__main__":
sys.exit(main())
+109
View File
@@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""Generates src/vendors/azure/calls_table.inc from the pinned Azure specs.
python3 -I src/vendors/azure/tools/gen_calls.py [local_spec_dir]
One row per logical operation. The OpenAI surface has two sides, "v1" (the
/openai/v1 API) and "dated" (the api-version=YYYY-MM-DD[-preview] API);
operations present in both are one row. Foundry model inference and Content
Safety are single-sided (dated slot). check_spec.py re-verifies every row.
"""
import os
import re
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import specdoc # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
OUT = os.path.join(HERE, "..", "calls_table.inc")
DEP = re.compile(r"^/deployments/\{[^}]+\}")
def norm_key(method, path):
p = DEP.sub("", path)
return method + " " + re.sub(r"\{[^}]*\}", "{}", p)
def collect(names, local):
"""(method, path) -> (doc name, facts), first document wins."""
out = {}
for n in names:
doc = specdoc.load(n, local)
for m, path, item, op in specdoc.operations(doc):
if (m, path) in out:
continue
f = specdoc.op_facts(doc, item, op)
f["doc"] = n
f["path"] = path
out[(m, path)] = f
return out
def side_args(f):
if f is None:
return ['""'] * 6
return ['"%s"' % f["doc"], '"%s"' % f["path"], '"%s"' % f["opid"], '"%s"' % f["content"],
'"%s"' % ",".join(f["body"]), '"%s"' % ",".join(f["query"])]
def main():
local = sys.argv[1] if len(sys.argv) > 1 else None
rows = [] # (surface, method, v1, dated)
v1 = collect(("v1p", "v1"), local)
dated = collect(("inf25", "auth24"), local)
groups = {}
for (m, path), f in v1.items():
groups.setdefault(norm_key(m, path), [None, None, m])[0] = f
for (m, path), f in dated.items():
k = norm_key(m, path)
g = groups.setdefault(k, [None, None, m])
if g[1] is not None:
sys.exit("two dated operations map to %s" % k)
g[1] = f
for k in sorted(groups):
a, b, m = groups[k]
rows.append(("kOpenAi", m, a, b))
for surface, doc in (("kFoundry", "mi"), ("kSafety", "cs24")):
for (m, path), f in sorted(collect((doc,), local).items()):
rows.append((surface, m, None, f))
names = {}
lines = []
for surface, m, a, b in rows:
pick = a if a is not None and a["opid"] else (b if b is not None and b["opid"] else None)
if pick is not None:
name = specdoc.camel(pick["opid"])
else:
name = specdoc.camel(m + " " + (a or b)["path"])
if surface == "kFoundry":
name = "Foundry" + name
if surface == "kSafety":
name = "Safety" + name
if name in names:
alt = specdoc.camel(b["opid"]) if b is not None and b["opid"] else None
name = alt if alt and alt not in names else name + "Dated"
names[name] = 1
if a is not None and b is not None:
na = len(re.findall(r"\{", DEP.sub("", a["path"])))
nb = len(re.findall(r"\{", DEP.sub("", b["path"])))
if na != nb:
sys.exit("placeholder count differs for %s" % name)
dep = 1 if (b is not None and DEP.match(b["path"])) else 0
if a is not None and DEP.match(a["path"]):
sys.exit("v1 path with deployment: " + name)
args = [name, "Surface::" + surface, '"%s"' % m.upper(), str(dep)] + side_args(a) + side_args(b)
lines.append("AZ_OP(" + ", ".join(args) + ")")
with open(OUT, "w") as fh:
fh.write("// GENERATED by tools/gen_calls.py from azure-rest-api-specs %s. Do not edit.\n"
% specdoc.REV)
fh.write("// AZ_OP(name, surface, method, deployment_scoped, then twice (v1 side, dated side):\n")
fh.write("// doc, path, operation_id, content, body_fields, query_params)\n")
fh.write("// AZ-ROWS-BEGIN\n")
for ln in lines:
fh.write(ln + "\n")
fh.write("// AZ-ROWS-END\n")
print("wrote %d rows" % len(lines))
if __name__ == "__main__":
main()
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env python3
"""Shared reader for the Azure OpenAPI / Swagger documents (stdlib only).
Used by gen_calls.py (writes calls_table.inc) and check_spec.py (verifies it).
Every document is pinned to a commit of Azure/azure-rest-api-specs.
"""
import json
import os
import re
import urllib.request
# Pinned revision of Azure/azure-rest-api-specs (main at the time of reading).
REV = "ba3eef39e5be55cd5ed1a2e5c6c9bb1a17c36b9a" # committed 2026-10-23T22:56:13Z
BASE = "https://raw.githubusercontent.com/Azure/azure-rest-api-specs/%s/specification/" % REV
# Short name -> path under specification/
DOCS = {
"v1": "ai/data-plane/OpenAI.v1/azure-v1-v1-generated.json",
"v1p": "ai/data-plane/OpenAI.v1/azure-v1-preview-generated.json",
"inf25": "cognitiveservices/data-plane/OpenAIInference/preview/2025-04-01-preview/inference.json",
"auth24": "cognitiveservices/data-plane/OpenAIAuthoring/stable/2024-10-21/azureopenai.json",
"cs24": "cognitiveservices/data-plane/ContentSafety/stable/2024-09-01/contentsafety.json",
"mi": "ai/data-plane/ModelInference/preview/2024-05-01-preview/openapi.json",
}
# Documents per surface: a spec operation in one of these must be in the table.
SURFACE_DOCS = {"openai": ("v1p", "v1", "inf25", "auth24"), "foundry": ("mi",), "safety": ("cs24",)}
VERBS = ("get", "post", "put", "patch", "delete")
def load(name, local_dir=None):
"""Loads a document by short name, from local_dir/<name>.json when given."""
if local_dir:
p = os.path.join(local_dir, name + ".json")
if os.path.exists(p):
return json.load(open(p))
with urllib.request.urlopen(BASE + DOCS[name], timeout=180) as r:
return json.load(r)
def deref(doc, node):
seen = 0
while isinstance(node, dict) and "$ref" in node and seen < 20:
cur = doc
for part in node["$ref"].lstrip("#/").split("/"):
cur = cur[part]
node = cur
seen += 1
return node
def schema_props(doc, node, depth=0):
if depth > 12 or not isinstance(node, dict):
return set()
node = deref(doc, node)
out = set(node.get("properties", {}))
for key in ("allOf", "oneOf", "anyOf"):
for x in node.get(key, []):
out |= schema_props(doc, x, depth + 1)
return out
def params_of(doc, path_item, op):
out = []
for p in list(path_item.get("parameters", [])) + list(op.get("parameters", [])):
out.append(deref(doc, p))
return out
def norm_content(c):
if c.startswith("application/json") or c.endswith("+json"):
return "json"
if c.startswith("multipart/"):
return "multipart"
if c.startswith("application/x-www-form-urlencoded"):
return "form"
return c
def op_facts(doc, path_item, op):
"""Returns dict(opid, content, body, query) for one operation."""
params = params_of(doc, path_item, op)
query = sorted({p["name"] for p in params if p.get("in") == "query" and p["name"] != "api-version"})
body = set()
kinds = set()
rb = op.get("requestBody")
if rb:
rb = deref(doc, rb)
for c, x in rb.get("content", {}).items():
kinds.add(norm_content(c))
body |= schema_props(doc, x.get("schema", {}))
for p in params:
if p.get("in") == "body":
kinds.add("json")
body |= schema_props(doc, p.get("schema", {}))
if p.get("in") == "formData":
kinds.add("multipart")
body.add(p["name"])
return {
"opid": op.get("operationId") or "",
"content": "+".join(sorted(kinds)) or "none",
"body": sorted(body),
"query": query,
}
def operations(doc):
for path, item in doc["paths"].items():
for m in VERBS:
if m in item:
yield m, path, item, item[m]
def camel(text):
parts = re.split(r"[^A-Za-z0-9]+", text)
out = ""
for p in parts:
if p:
out += p[0].upper() + p[1:]
return out
+15
View File
@@ -0,0 +1,15 @@
#pragma once
// A file for a multipart form: name, media type, bytes.
#include <string>
namespace molo::azure {
struct UploadFile {
std::string filename;
std::string content_type = "application/octet-stream";
std::string data;
};
} // namespace molo::azure
+107
View File
@@ -0,0 +1,107 @@
#include "vendors/azure/vector_stores.h"
#include "vendors/azure/check.h"
namespace molo::azure {
using clients::body::Json;
namespace {
Query page(const ListParams& p, const std::string& filter = "") {
p.validate();
Query q;
p.apply(q);
if (!filter.empty()) {
Check::in_set("filter", filter, {"in_progress", "completed", "failed", "cancelled"});
q.add("filter", filter);
}
return q;
}
} // namespace
Json VectorStoresService::create(const Json& body) { return call(Op::kCreateVectorStore, {}, &body); }
Json VectorStoresService::get(const std::string& s) {
Check::id("vector store", s);
return call(Op::kGetVectorStore, {"", {s}});
}
Json VectorStoresService::modify(const std::string& s, const Json& body) {
Check::id("vector store", s);
return call(Op::kModifyVectorStore, {"", {s}}, &body);
}
Json VectorStoresService::list(const ListParams& params) {
Query q = page(params);
return call(Op::kListVectorStores, {}, nullptr, &q);
}
Listing VectorStoresService::list_all(const ListParams& params, std::size_t max_pages) {
return walk(Op::kListVectorStores, {}, params, Query(), max_pages);
}
Json VectorStoresService::remove(const std::string& s) {
Check::id("vector store", s);
return call(Op::kDeleteVectorStore, {"", {s}});
}
Json VectorStoresService::search(const std::string& s, const Json& body) {
Check::id("vector store", s);
Check::has_field(body, "query");
return call(Op::kSearchVectorStore, {"", {s}}, &body);
}
Json VectorStoresService::add_file(const std::string& s, const Json& body) {
Check::id("vector store", s);
Check::has_string(body, "file_id");
return call(Op::kCreateVectorStoreFile, {"", {s}}, &body);
}
Json VectorStoresService::get_file(const std::string& s, const std::string& f) {
Check::id("vector store", s);
Check::id("file", f);
return call(Op::kGetVectorStoreFile, {"", {s, f}});
}
Json VectorStoresService::list_files(const std::string& s, const ListParams& params,
const std::string& filter) {
Check::id("vector store", s);
Query q = page(params, filter);
return call(Op::kListVectorStoreFiles, {"", {s}}, nullptr, &q);
}
Json VectorStoresService::update_file_attributes(const std::string& s, const std::string& f,
const Json& attributes) {
Check::id("vector store", s);
Check::id("file", f);
Check::object("attributes", attributes);
Json body = Json::object();
body.set("attributes", attributes);
return call(Op::kUpdateVectorStoreFileAttributes, {"", {s, f}}, &body);
}
Json VectorStoresService::file_content(const std::string& s, const std::string& f) {
Check::id("vector store", s);
Check::id("file", f);
return call(Op::kRetrieveVectorStoreFileContent, {"", {s, f}});
}
Json VectorStoresService::remove_file(const std::string& s, const std::string& f) {
Check::id("vector store", s);
Check::id("file", f);
return call(Op::kDeleteVectorStoreFile, {"", {s, f}});
}
Json VectorStoresService::create_file_batch(const std::string& s, const Json& body) {
Check::id("vector store", s);
Check::require(body.find("file_ids") || body.find("files"), "file_ids (or files) is required");
return call(Op::kCreateVectorStoreFileBatch, {"", {s}}, &body);
}
Json VectorStoresService::get_file_batch(const std::string& s, const std::string& b) {
Check::id("vector store", s);
Check::id("batch", b);
return call(Op::kGetVectorStoreFileBatch, {"", {s, b}});
}
Json VectorStoresService::cancel_file_batch(const std::string& s, const std::string& b) {
Check::id("vector store", s);
Check::id("batch", b);
return call(Op::kCancelVectorStoreFileBatch, {"", {s, b}});
}
Json VectorStoresService::list_batch_files(const std::string& s, const std::string& b,
const ListParams& params, const std::string& filter) {
Check::id("vector store", s);
Check::id("batch", b);
Query q = page(params, filter);
return call(Op::kListFilesInVectorStoreBatch, {"", {s, b}}, nullptr, &q);
}
} // namespace molo::azure
+41
View File
@@ -0,0 +1,41 @@
#pragma once
// Vector stores: stores, files, file batches, search, file content.
#include <string>
#include "vendors/azure/service.h"
namespace molo::azure {
class VectorStoresService : public AzureService {
public:
explicit VectorStoresService(std::shared_ptr<AzureClient> azure)
: AzureService(std::move(azure), Surface::kOpenAi) {}
clients::body::Json create(const clients::body::Json& body = clients::body::Json::object());
clients::body::Json get(const std::string& store);
clients::body::Json modify(const std::string& store, const clients::body::Json& body);
clients::body::Json list(const ListParams& params = {});
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
clients::body::Json remove(const std::string& store); // destructive
// body: query (required) plus filters, max_num_results, ranking_options, rewrite_query.
clients::body::Json search(const std::string& store, const clients::body::Json& body);
clients::body::Json add_file(const std::string& store, const clients::body::Json& body); // file_id required
clients::body::Json get_file(const std::string& store, const std::string& file);
clients::body::Json list_files(const std::string& store, const ListParams& params = {},
const std::string& filter = "");
clients::body::Json update_file_attributes(const std::string& store, const std::string& file,
const clients::body::Json& attributes);
clients::body::Json file_content(const std::string& store, const std::string& file);
clients::body::Json remove_file(const std::string& store, const std::string& file); // destructive
clients::body::Json create_file_batch(const std::string& store, const clients::body::Json& body);
clients::body::Json get_file_batch(const std::string& store, const std::string& batch);
clients::body::Json cancel_file_batch(const std::string& store, const std::string& batch); // destructive
clients::body::Json list_batch_files(const std::string& store, const std::string& batch,
const ListParams& params = {}, const std::string& filter = "");
};
} // namespace molo::azure