feat(clients): azure: Azure OpenAI and AI Foundry package, 147-row spec table, Entra auth, demo
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Vendored
+274
@@ -0,0 +1,274 @@
|
||||
# Azure OpenAI and AI Foundry - Specification
|
||||
|
||||
Owned directory: `src/vendors/azure/`. Namespace `molo::azure`. Depends on
|
||||
`molo::llm` (`ApiClient`, `Service`, `SseDecoder`, `ChatModel`, `Config`),
|
||||
`molo::clients`, the WebSocket client of `src/google/live` (realtime) and the
|
||||
OpenAI neutral wire mapping `openai::core::ChatWire` / `ResponsesWire` (the
|
||||
chat model only: Azure OpenAI speaks the same shapes). Plan:
|
||||
`plans/18-breadth.md` section 2, method of `plans/17-llm-vendor-platforms.md`.
|
||||
Demo: `example/app/ai/AzureDemo.cpp`, routes `/ai/azure`.
|
||||
|
||||
## 1. Scope
|
||||
|
||||
One resource, three surfaces, one credential:
|
||||
|
||||
| Surface | Host (from the specs) | Prefix | Covers |
|
||||
|---|---|---|---|
|
||||
| OpenAI | `https://<resource>.openai.azure.com` | `/openai/v1` (v1 API) or `/openai` (dated API) | chat, completions, responses, embeddings, images (generate, edit, variation), audio (speech, transcription, translation), realtime, assistants / threads / messages / runs / steps, batch, fine-tuning (jobs, events, checkpoints, permissions, copy), files, uploads, vector stores, models, containers, conversations, evals, graders, videos |
|
||||
| Foundry | `https://<resource>.services.ai.azure.com` | `/models` | Azure AI model inference for catalog models: chat completions, embeddings, image embeddings, model info |
|
||||
| Safety | `https://<resource>.cognitiveservices.azure.com` | `/contentsafety` | Content Safety: analyze text / image, Prompt Shields, protected material, text blocklists |
|
||||
|
||||
Not covered, and said so: the Azure management plane (ARM: creating
|
||||
resources, deployments, keys, quotas), Azure AI Agents / Projects
|
||||
(`ai/data-plane/AIAgents`, `AIProjects`), Voice Live, Speech / Translator /
|
||||
Language services. Deployments are therefore not listable here: `GET /models`
|
||||
lists models, a deployment name must be known to the caller.
|
||||
|
||||
## 2. Addressing and api-version (the core rule)
|
||||
|
||||
`AZURE_OPENAI_API_VERSION` selects one of three modes (`ApiVersion`):
|
||||
|
||||
| Value | Mode | OpenAI-surface URL | Deployment goes |
|
||||
|---|---|---|---|
|
||||
| unset / `v1` | v1 | `<endpoint>/openai/v1/<path>` (no api-version query) | body `model` |
|
||||
| `preview` | v1 preview | `<endpoint>/openai/v1/<path>?api-version=preview` | body `model` |
|
||||
| `YYYY-MM-DD[-preview]` | dated | `<endpoint>/openai/<path>?api-version=<v>` | URL `/deployments/{d}/...` (8 operations), body `model` for the rest |
|
||||
|
||||
The 8 deployment-scoped operations are exactly the SDK list
|
||||
(openai-python `lib/azure.py` `_deployments_endpoints`): completions, chat
|
||||
completions, embeddings, audio transcriptions / translations / speech, image
|
||||
generations / edits. The caller always passes the deployment once; the gateway
|
||||
(`AzureService`) places it. A body `model` that differs from the deployment is
|
||||
rejected; on the dated API it is removed.
|
||||
|
||||
An operation that exists on one API only is sent to that API with its
|
||||
fallback version: dated-only operations (assistants, uploads, files import,
|
||||
realtime sessions under `/realtimeapi`, fine-tuning job delete) under a v1
|
||||
config use `2025-04-01-preview`; v1-only operations (containers,
|
||||
conversations, evals, graders, videos, realtime client secrets and calls,
|
||||
checkpoint permissions, image variations) under a dated config use
|
||||
`/openai/v1` with `api-version=preview`. Foundry uses `2024-05-01-preview`,
|
||||
Safety `2024-09-01` (both overridable in `Settings`, independent of the OpenAI
|
||||
version).
|
||||
|
||||
## 3. Authentication
|
||||
|
||||
| Mode | Chosen when | Header |
|
||||
|---|---|---|
|
||||
| API key | `AZURE_OPENAI_API_KEY` set (wins) | `api-key: <key>` (OpenAI, Foundry); `Ocp-Apim-Subscription-Key: <key>` (Safety, key `AZURE_CONTENT_SAFETY_KEY` or the same key) |
|
||||
| Microsoft Entra | tenant, client id and secret set, no key | `Authorization: Bearer <token>` on every surface |
|
||||
|
||||
Entra is its own class pair (`entra.h`): `EntraTokenSource` posts
|
||||
`client_id`, `client_secret`, `scope`, `grant_type=client_credentials` (form
|
||||
encoded) to `<authority>/<tenant>/oauth2/v2.0/token` (authority default
|
||||
`https://login.microsoftonline.com`, `AZURE_AUTHORITY_HOST` for sovereign
|
||||
clouds), scope `https://cognitiveservices.azure.com/.default` (the scope the
|
||||
specs' OAuth2 schemes name), caches the token, refreshes it 300 s before
|
||||
expiry, and after a failed exchange refuses (without network) for 5 s
|
||||
(`TOKEN_BACKOFF`, a floor). `EntraTransport` decorates the transport so the
|
||||
bearer reaches every request, including streams and the realtime handshake.
|
||||
Secrets never appear in URLs, errors or the demo output (the demo's `entra`
|
||||
route reports expiry only).
|
||||
|
||||
## 4. Module map (one concern per file)
|
||||
|
||||
| File | Concern |
|
||||
|---|---|
|
||||
| `caps.h` | Static ceilings |
|
||||
| `api_version.*` | `ApiVersion`: v1 / preview / dated parsing |
|
||||
| `settings.*` | `Settings`, `EntraConfig`: env, endpoint rules, auth kind |
|
||||
| `calls.*`, `calls_table.inc` | The ONE call table (generated), `Calls::render/accepts_*` |
|
||||
| `entra.*` | `EntraTokenSource`, `EntraTransport` |
|
||||
| `client.*` | `AzureClient`: one `llm::ApiClient` per surface, shared credential |
|
||||
| `service.*` | `AzureService` gateway: side choice, paths, field/query checks, JSON / multipart / raw / SSE, list walk |
|
||||
| `check.*` | Validation helpers |
|
||||
| `chat.*` | `ChatService`, `CompletionsService` |
|
||||
| `responses.*` | `ResponsesService` |
|
||||
| `embeddings.*` | `EmbeddingsService` |
|
||||
| `images.*` | `ImagesService` |
|
||||
| `audio.*` | `AudioService` |
|
||||
| `realtime.*` | `RealtimeService` (REST), `RealtimeUrl`, `RealtimeSocket` (WebSocket) |
|
||||
| `assistants.*` | `AssistantsService` |
|
||||
| `batches.*`, `fine_tuning.*`, `files.*`, `vector_stores.*`, `models.*` | one service each |
|
||||
| `foundry.*` | `FoundryService` (model inference) |
|
||||
| `safety.*` | `SafetyService` (Content Safety) |
|
||||
| `operations.*` | `OperationsService`: every table operation by name |
|
||||
| `chat_model.*` | `AzureChatModel : llm::ChatModel` (dialects chat completions, responses, Foundry) |
|
||||
| `upload_file.h` | `UploadFile` |
|
||||
| `tools/specdoc.py`, `gen_calls.py`, `check_spec.py` | spec reader, table generator, checker |
|
||||
|
||||
## 5. Static caps (see caps.h)
|
||||
|
||||
| Knob | Value |
|
||||
|---|---|
|
||||
| Deployment name | 1..64 of `[A-Za-z0-9._-]` |
|
||||
| Endpoint | 256 chars, scheme and host only |
|
||||
| Page limit / page walk | 100 / 20 pages |
|
||||
| Stream events | 200000 |
|
||||
| Embedding inputs | 2048 |
|
||||
| Audio file / image file / speech text / image prompt | 25 MiB / 50 MiB / 4096 / 32000 chars |
|
||||
| Single file upload | 512 MiB (`llm::Caps`), upload part 64 MiB |
|
||||
| Content Safety text / image / blocklist items / Shield documents | 10000 chars / 7.5 MiB / 100 / 5 |
|
||||
| Entra token | refresh skew 300 s, failure floor 5 s, lifetime <= 24 h, answer <= 16 KiB |
|
||||
| Realtime event / events per session / bytes / seconds | 1 MiB / 100000 / 64 MiB / 3600 |
|
||||
| Client secret lifetime | 10..7200 s |
|
||||
|
||||
## 6. Contracts
|
||||
|
||||
- No endpoint, or neither key nor complete Entra credentials: every call
|
||||
throws `NotConfiguredError` (503 `NOT_CONFIGURED`) before any network
|
||||
traffic. A malformed `AZURE_*` value throws `ApiError::invalid` from
|
||||
`Settings::from_env`.
|
||||
- Every request field and query name must be listed by the spec of the API
|
||||
side in use (`Calls::accepts_field/accepts_query`), else 400 before traffic.
|
||||
Ids are single URL path segments (`Service::valid_id`).
|
||||
- Destructive calls (delete, cancel, remove, hangup, reject, pause) are
|
||||
explicit methods with no defaults; the demo asks for `confirm:true` for them
|
||||
and for everything that costs money or changes a resource.
|
||||
- Lists report truncation (`Listing::truncated`, `next_after`); a server that
|
||||
says `has_more` without a usable cursor is reported truncated, not looped.
|
||||
- Error bodies of the form `{"error":{"code","message"}}` keep the code as
|
||||
`ApiError::type()`.
|
||||
|
||||
## 7. Sources
|
||||
|
||||
All machine readable, pinned to azure-rest-api-specs commit
|
||||
`ba3eef39e5be55cd5ed1a2e5c6c9bb1a17c36b9a` (2026-10-23):
|
||||
|
||||
| Short name | Document |
|
||||
|---|---|
|
||||
| `v1`, `v1p` | `specification/ai/data-plane/OpenAI.v1/azure-v1-v1-generated.json`, `azure-v1-preview-generated.json` (OpenAPI 3, server `{endpoint}/openai/v1`) |
|
||||
| `inf25` | `cognitiveservices/data-plane/OpenAIInference/preview/2025-04-01-preview/inference.json` |
|
||||
| `auth24` | `cognitiveservices/data-plane/OpenAIAuthoring/stable/2024-10-21/azureopenai.json` (files, batches, fine-tuning, models, uploads) |
|
||||
| `cs24` | `cognitiveservices/data-plane/ContentSafety/stable/2024-09-01/contentsafety.json` |
|
||||
| `mi` | `ai/data-plane/ModelInference/preview/2024-05-01-preview/openapi.json` |
|
||||
|
||||
Cross-checks: openai-python `src/openai/lib/azure.py` (deployment endpoint
|
||||
list, `api-key` / `Authorization: Bearer` headers, `/openai` and
|
||||
`/openai/deployments/...` base URLs, `AZURE_OPENAI_ENDPOINT` /
|
||||
`AZURE_OPENAI_API_KEY`; the SDK reads the version from `OPENAI_API_VERSION`,
|
||||
this package from `AZURE_OPENAI_API_VERSION` as the plan names it); the specs'
|
||||
own security schemes (header names, OAuth scope) and `info.version`; the
|
||||
Microsoft identity platform client-credentials documentation (token request
|
||||
and response bodies).
|
||||
The older dated revisions (2022-12-01 ... 2025-03-01-preview, 2024-02-15
|
||||
etc.) are not read: the table covers the newest stable authoring revision and
|
||||
the newest preview inference revision; a field only older versions accept is
|
||||
rejected by the gateway.
|
||||
|
||||
## 8. Verified against spec
|
||||
|
||||
`python3 -I src/vendors/azure/tools/check_spec.py` (2026-10-25):
|
||||
`spec ok: 147 rows, 209 table sides verified, 315 spec operations seen in 6 documents`.
|
||||
Per side it checks method, path, operationId, request content kind, the full
|
||||
top-level request property list and the query list; it fails on any spec
|
||||
operation of a covered document missing from the table, duplicate names, the
|
||||
deployment-scoped set against the SDK list, the api-version constants against
|
||||
each document's `info.version`, the v1 `api-version` enum (`v1`, `preview`),
|
||||
the key header names and the Entra scope. The table is generated by
|
||||
`tools/gen_calls.py` from the same documents (`calls_table.inc` is never
|
||||
hand edited).
|
||||
|
||||
## 9. Verified live
|
||||
|
||||
Unauthenticated probes, 2026-10-25 (`MOLO_LIVE_NET=1 make ai-test`, suite
|
||||
`azure_live`, passing):
|
||||
|
||||
- `POST https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token` with the
|
||||
documented form and a client id that does not exist: HTTP 400,
|
||||
`{"error":"unauthorized_client","error_description":"AADSTS700038: ...","error_codes":[700038],"trace_id":...}`.
|
||||
`EntraTokenSource` maps it to `ApiError(400, "unauthorized_client")` with the
|
||||
trace id as request id. The same body is the parse fixture in `test_entra.cpp`.
|
||||
- `GET https://eastus.api.cognitive.microsoft.com/openai/v1/models` with a bad
|
||||
`api-key`: HTTP 401 `{"error":{"code":"401","message":"Access denied due to invalid subscription key or wrong API endpoint..."}}`.
|
||||
`POST .../contentsafety/text:analyze?api-version=2024-09-01` with a bad
|
||||
`Ocp-Apim-Subscription-Key`: the same 401 body. So the v1 path, the
|
||||
Content Safety path and both key headers are the right shape on the real host.
|
||||
- `<name>.openai.azure.com`, `.services.ai.azure.com`, `.cognitiveservices.azure.com`
|
||||
do not resolve for an unknown resource name (NXDOMAIN); a resource host
|
||||
cannot be probed without a resource.
|
||||
|
||||
Opt-in authenticated live tests (`azure_live.models_list_and_one_tiny_completion`):
|
||||
need `MOLO_LIVE_NET=1`, `AZURE_OPENAI_ENDPOINT` and a key or Entra triple;
|
||||
models list is read-only; one smallest completion additionally needs
|
||||
`MOLO_LIVE_PAID=1` and `AZURE_OPENAI_DEPLOYMENT`. Loud SKIP otherwise.
|
||||
Nothing is created, updated or deleted.
|
||||
|
||||
## 10. Unverified
|
||||
|
||||
No Azure resource or key exists in this environment: no success response body
|
||||
from any Azure host was seen. Everything below is read from specs, the SDK or
|
||||
documentation but never confirmed against a live resource.
|
||||
|
||||
- Success shapes: chat completion, embeddings, models, responses and stream
|
||||
parsing use the documented OpenAI-shaped examples (labelled in the tests).
|
||||
- v1 `api-version`: omitted for `v1` per the Microsoft Learn v1 guide (the spec
|
||||
says optional, default `v1`); whether a real resource accepts both is unseen.
|
||||
- Multipart vs JSON: the v1 spec lists `application/json` for audio
|
||||
transcriptions / translations and image edits / variations (generator
|
||||
artefact: a `file` property typed `object`), and the dated 2025-04-01-preview
|
||||
spec lists JSON for image edits. This package sends multipart (the dated
|
||||
spec says multipart for transcription; every official SDK sends multipart
|
||||
for these). Field names are checked against the spec lists.
|
||||
- Assistants: the v1 documents contain threads, messages and runs but no
|
||||
`/assistants` path; assistants are sent to the dated API (fallback version),
|
||||
threads to the configured API. Microsoft has announced the retirement of the
|
||||
Assistants API; whether a given resource still serves it is unseen.
|
||||
- Realtime WebSocket URLs (`/openai/v1/realtime?model=`, dated
|
||||
`/openai/realtime?api-version=&deployment=`) and the `api-key` /
|
||||
Bearer handshake headers come from the SDK shape and Microsoft Learn, not
|
||||
from the OpenAPI documents (WebSocket is not in them); the session over a
|
||||
loopback server proves the client, not the service. The realtime event
|
||||
vocabulary is not catalogued (events are JSON objects with a `type`).
|
||||
- Foundry: `https://<resource>.services.ai.azure.com/models` host and the
|
||||
bearer scope `https://cognitiveservices.azure.com/.default` for it (the spec
|
||||
names this scope; the Learn page for Foundry models also mentions
|
||||
`https://ai.azure.com/.default` for some flows). `AZURE_AI_ENDPOINT` and
|
||||
`AZURE_CONTENT_SAFETY_ENDPOINT` default to the OpenAI endpoint, which only
|
||||
works on a multi-service Azure AI Services resource that serves all three
|
||||
hosts under one name. How `model` picks a deployment on a multi-model
|
||||
endpoint is not described by the spec.
|
||||
- Content Safety: Entra scope (spec), `application/merge-patch+json` for the
|
||||
blocklist PATCH (spec `consumes`), response shapes of analyze / Shield.
|
||||
- Entra: the `expires_in` number form (documented); sovereign authority hosts
|
||||
(`login.microsoftonline.us`, `.cn`) via `AZURE_AUTHORITY_HOST` (documented
|
||||
hosts, never called). Managed identity, certificates and workload identity
|
||||
federation are not implemented.
|
||||
- Rate limits, `Retry-After` and quota errors (429) are passed through as
|
||||
`ApiError` without any retry (the `llm` foundation drops `Retry-After`).
|
||||
|
||||
## 11. Gaps
|
||||
|
||||
- No ARM (management plane): cannot list or create deployments, keys, quotas
|
||||
(needs `management.azure.com` and a different token scope).
|
||||
- No AAD token for Foundry-only resources via `https://ai.azure.com/.default`.
|
||||
- Realtime has no event catalogue and no audio helpers (owner: this package,
|
||||
`realtime.h`); the `openai/platform` catalogue could be reused once Azure
|
||||
event differences are known.
|
||||
- `llm::ApiClient` has no per-call timeout, no GET stream (response replay),
|
||||
and no unauthenticated request (`src/llm/SPEC.md` section 4); the Entra
|
||||
exchange therefore uses its own `clients::Client`.
|
||||
- `clients::Url` percent-encodes `[` and `]` in query names (`include[]`
|
||||
goes out as `include%5B%5D`); servers decode it, not proven on Azure.
|
||||
- Batch input blobs (`input_blob`, `output_folder` of the v1 batch API) are
|
||||
passed through as JSON fields only.
|
||||
- `ListParams::before` exists, but the spec lists `before` for some lists only;
|
||||
the gateway rejects it where the spec does not.
|
||||
- Newer dated revisions than 2025-04-01-preview (if Microsoft ships them) need
|
||||
`gen_calls.py` re-run; `check_spec.py` fails on drift of the pinned documents.
|
||||
|
||||
## 12. Credentials
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `AZURE_OPENAI_ENDPOINT` | `https://<resource>.openai.azure.com` (scheme and host only) |
|
||||
| `AZURE_OPENAI_API_KEY` | resource key (Azure portal, Keys and Endpoint) |
|
||||
| `AZURE_OPENAI_API_VERSION` | `v1` (default), `preview`, or a dated version |
|
||||
| `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET` | Entra app registration with the "Cognitive Services OpenAI User" (or Contributor) role on the resource |
|
||||
| `AZURE_AI_ENDPOINT` | optional, Foundry model inference host |
|
||||
| `AZURE_CONTENT_SAFETY_ENDPOINT`, `AZURE_CONTENT_SAFETY_KEY` | optional, Content Safety host and key |
|
||||
| `AZURE_AUTHORITY_HOST` | optional, default `https://login.microsoftonline.com` |
|
||||
| `AZURE_OPENAI_DEPLOYMENT` | live tests only: the deployment for the one paid completion |
|
||||
|
||||
Console: https://portal.azure.com (resource, Keys and Endpoint; Entra: App
|
||||
registrations). Not read: the portal needs a sign-in.
|
||||
Vendored
+32
@@ -0,0 +1,32 @@
|
||||
#include "vendors/azure/api_version.h"
|
||||
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
bool ApiVersion::is_date(const std::string& t) {
|
||||
auto digit = [&](std::size_t i) { return i < t.size() && t[i] >= '0' && t[i] <= '9'; };
|
||||
if (t.size() < 10) return false;
|
||||
for (std::size_t i : {0u, 1u, 2u, 3u, 5u, 6u, 8u, 9u})
|
||||
if (!digit(i)) return false;
|
||||
if (t[4] != '-' || t[7] != '-') return false;
|
||||
if (t.size() == 10) return true;
|
||||
return t.substr(10) == "-preview";
|
||||
}
|
||||
|
||||
ApiVersion ApiVersion::parse(const std::string& text) {
|
||||
ApiVersion v;
|
||||
if (text.empty() || text == "v1") return v;
|
||||
if (text == "preview") {
|
||||
v.mode_ = Mode::kPreview;
|
||||
v.value_ = "preview";
|
||||
return v;
|
||||
}
|
||||
if (!is_date(text))
|
||||
throw llm::ApiError::invalid("azure", "api version must be v1, preview or YYYY-MM-DD[-preview]");
|
||||
v.mode_ = Mode::kDated;
|
||||
v.value_ = text;
|
||||
return v;
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+39
@@ -0,0 +1,39 @@
|
||||
#pragma once
|
||||
|
||||
// ApiVersion: the three ways an Azure OpenAI resource is addressed.
|
||||
// v1 /openai/v1/... no api-version query (the GA v1 API)
|
||||
// preview /openai/v1/... ?api-version=preview (v1 preview features)
|
||||
// dated /openai/... ?api-version=YYYY-MM-DD[-preview], deployment URLs
|
||||
// Env AZURE_OPENAI_API_VERSION selects one; unset means v1.
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class ApiVersion {
|
||||
public:
|
||||
enum class Mode { kV1, kPreview, kDated };
|
||||
|
||||
static constexpr const char* kDatedFallback = "2025-04-01-preview"; // inference doc revision
|
||||
static constexpr const char* kFoundryDefault = "2024-05-01-preview";
|
||||
static constexpr const char* kSafetyDefault = "2024-09-01";
|
||||
|
||||
ApiVersion() = default;
|
||||
// "", "v1", "preview", or a date; anything else throws ApiError::invalid.
|
||||
static ApiVersion parse(const std::string& text);
|
||||
// True for YYYY-MM-DD and YYYY-MM-DD-preview.
|
||||
static bool is_date(const std::string& text);
|
||||
|
||||
Mode mode() const { return mode_; }
|
||||
const std::string& value() const { return value_; }
|
||||
bool is_dated() const { return mode_ == Mode::kDated; }
|
||||
bool is_v1_family() const { return mode_ != Mode::kDated; }
|
||||
// Query value to send on the v1 path: "" for v1 (omitted), "preview".
|
||||
std::string v1_query() const { return mode_ == Mode::kPreview ? "preview" : ""; }
|
||||
|
||||
private:
|
||||
Mode mode_ = Mode::kV1;
|
||||
std::string value_ = "v1";
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+146
@@ -0,0 +1,146 @@
|
||||
#include "vendors/azure/assistants.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
Query page(const ListParams& p) {
|
||||
p.validate();
|
||||
Query q;
|
||||
p.apply(q);
|
||||
return q;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json AssistantsService::create_assistant(const Json& body) {
|
||||
Check::has_string(body, "model");
|
||||
Check::deployment(body.find("model")->as_string());
|
||||
return call(Op::kCreateAssistant, {}, &body);
|
||||
}
|
||||
Json AssistantsService::get_assistant(const std::string& id) {
|
||||
Check::id("assistant", id);
|
||||
return call(Op::kGetAssistant, {"", {id}});
|
||||
}
|
||||
Json AssistantsService::modify_assistant(const std::string& id, const Json& body) {
|
||||
Check::id("assistant", id);
|
||||
return call(Op::kModifyAssistant, {"", {id}}, &body);
|
||||
}
|
||||
Json AssistantsService::list_assistants(const ListParams& params) {
|
||||
Query q = page(params);
|
||||
return call(Op::kListAssistants, {}, nullptr, &q);
|
||||
}
|
||||
Json AssistantsService::remove_assistant(const std::string& id) {
|
||||
Check::id("assistant", id);
|
||||
return call(Op::kDeleteAssistant, {"", {id}});
|
||||
}
|
||||
|
||||
Json AssistantsService::create_thread(const Json& body) { return call(Op::kCreateThread, {}, &body); }
|
||||
Json AssistantsService::get_thread(const std::string& id) {
|
||||
Check::id("thread", id);
|
||||
return call(Op::kRetrieveThread, {"", {id}});
|
||||
}
|
||||
Json AssistantsService::modify_thread(const std::string& id, const Json& body) {
|
||||
Check::id("thread", id);
|
||||
return call(Op::kModifyThread, {"", {id}}, &body);
|
||||
}
|
||||
Json AssistantsService::remove_thread(const std::string& id) {
|
||||
Check::id("thread", id);
|
||||
return call(Op::kDeleteThread, {"", {id}});
|
||||
}
|
||||
Json AssistantsService::create_thread_and_run(const Json& body) {
|
||||
Check::has_string(body, "assistant_id");
|
||||
return call(Op::kCreateThreadAndRun, {}, &body);
|
||||
}
|
||||
|
||||
Json AssistantsService::create_message(const std::string& thread, const Json& body) {
|
||||
Check::id("thread", thread);
|
||||
Check::has_string(body, "role");
|
||||
Check::has_field(body, "content");
|
||||
return call(Op::kCreateMessage, {"", {thread}}, &body);
|
||||
}
|
||||
Json AssistantsService::get_message(const std::string& thread, const std::string& message) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("message", message);
|
||||
return call(Op::kRetrieveMessage, {"", {thread, message}});
|
||||
}
|
||||
Json AssistantsService::modify_message(const std::string& thread, const std::string& message,
|
||||
const Json& body) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("message", message);
|
||||
return call(Op::kModifyMessage, {"", {thread, message}}, &body);
|
||||
}
|
||||
Json AssistantsService::list_messages(const std::string& thread, const ListParams& params) {
|
||||
Check::id("thread", thread);
|
||||
Query q = page(params);
|
||||
return call(Op::kListMessages, {"", {thread}}, nullptr, &q);
|
||||
}
|
||||
Listing AssistantsService::list_messages_all(const std::string& thread, const ListParams& params,
|
||||
std::size_t max_pages) {
|
||||
Check::id("thread", thread);
|
||||
return walk(Op::kListMessages, {"", {thread}}, params, Query(), max_pages);
|
||||
}
|
||||
Json AssistantsService::remove_message(const std::string& thread, const std::string& message) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("message", message);
|
||||
return call(Op::kDeleteMessage, {"", {thread, message}});
|
||||
}
|
||||
|
||||
Json AssistantsService::create_run(const std::string& thread, const Json& body) {
|
||||
Check::id("thread", thread);
|
||||
Check::has_string(body, "assistant_id");
|
||||
return call(Op::kCreateRun, {"", {thread}}, &body);
|
||||
}
|
||||
void AssistantsService::stream_run(const std::string& thread, const Json& body, const EventHandler& handler) {
|
||||
Check::id("thread", thread);
|
||||
Check::has_string(body, "assistant_id");
|
||||
call_stream(Op::kCreateRun, {"", {thread}}, body, handler);
|
||||
}
|
||||
Json AssistantsService::get_run(const std::string& thread, const std::string& run) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("run", run);
|
||||
return call(Op::kRetrieveRun, {"", {thread, run}});
|
||||
}
|
||||
Json AssistantsService::modify_run(const std::string& thread, const std::string& run, const Json& body) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("run", run);
|
||||
return call(Op::kModifyRun, {"", {thread, run}}, &body);
|
||||
}
|
||||
Json AssistantsService::list_runs(const std::string& thread, const ListParams& params) {
|
||||
Check::id("thread", thread);
|
||||
Query q = page(params);
|
||||
return call(Op::kListRuns, {"", {thread}}, nullptr, &q);
|
||||
}
|
||||
Json AssistantsService::cancel_run(const std::string& thread, const std::string& run) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("run", run);
|
||||
return call(Op::kCancelRun, {"", {thread, run}});
|
||||
}
|
||||
Json AssistantsService::submit_tool_outputs(const std::string& thread, const std::string& run,
|
||||
const Json& tool_outputs) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("run", run);
|
||||
Check::require(tool_outputs.is_array() && !tool_outputs.as_array().empty(),
|
||||
"tool_outputs must be a non-empty array");
|
||||
Json body = Json::object();
|
||||
body.set("tool_outputs", tool_outputs);
|
||||
return call(Op::kSubmitToolOutputsToRun, {"", {thread, run}}, &body);
|
||||
}
|
||||
Json AssistantsService::list_run_steps(const std::string& thread, const std::string& run,
|
||||
const ListParams& params) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("run", run);
|
||||
Query q = page(params);
|
||||
return call(Op::kListRunSteps, {"", {thread, run}}, nullptr, &q);
|
||||
}
|
||||
Json AssistantsService::get_run_step(const std::string& thread, const std::string& run,
|
||||
const std::string& step) {
|
||||
Check::id("thread", thread);
|
||||
Check::id("run", run);
|
||||
Check::id("step", step);
|
||||
return call(Op::kGetRunStep, {"", {thread, run, step}});
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+63
@@ -0,0 +1,63 @@
|
||||
#pragma once
|
||||
|
||||
// Assistants, threads, messages, runs, run steps (OpenAI Assistants API on
|
||||
// Azure). The v1 documents have threads / runs but no /assistants path; when
|
||||
// an operation exists on one API only, the gateway uses that API with its
|
||||
// fallback api-version (dated: 2025-04-01-preview, v1: preview). Microsoft
|
||||
// announced the retirement of this API in favour of Responses / Agents.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class AssistantsService : public AzureService {
|
||||
public:
|
||||
using EventHandler = std::function<bool(const llm::SseEvent&)>; // false: stop
|
||||
|
||||
explicit AssistantsService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// Assistants. create needs `model` (the deployment name).
|
||||
clients::body::Json create_assistant(const clients::body::Json& body);
|
||||
clients::body::Json get_assistant(const std::string& id);
|
||||
clients::body::Json modify_assistant(const std::string& id, const clients::body::Json& body);
|
||||
clients::body::Json list_assistants(const ListParams& params = {});
|
||||
clients::body::Json remove_assistant(const std::string& id); // destructive
|
||||
|
||||
// Threads.
|
||||
clients::body::Json create_thread(const clients::body::Json& body = clients::body::Json::object());
|
||||
clients::body::Json get_thread(const std::string& id);
|
||||
clients::body::Json modify_thread(const std::string& id, const clients::body::Json& body);
|
||||
clients::body::Json remove_thread(const std::string& id); // destructive
|
||||
// Thread and run in one call; body needs assistant_id.
|
||||
clients::body::Json create_thread_and_run(const clients::body::Json& body);
|
||||
|
||||
// Messages.
|
||||
clients::body::Json create_message(const std::string& thread, const clients::body::Json& body);
|
||||
clients::body::Json get_message(const std::string& thread, const std::string& message);
|
||||
clients::body::Json modify_message(const std::string& thread, const std::string& message,
|
||||
const clients::body::Json& body);
|
||||
clients::body::Json list_messages(const std::string& thread, const ListParams& params = {});
|
||||
Listing list_messages_all(const std::string& thread, const ListParams& params = {},
|
||||
std::size_t max_pages = 5);
|
||||
clients::body::Json remove_message(const std::string& thread, const std::string& message); // destructive
|
||||
|
||||
// Runs. body needs assistant_id.
|
||||
clients::body::Json create_run(const std::string& thread, const clients::body::Json& body);
|
||||
void stream_run(const std::string& thread, const clients::body::Json& body, const EventHandler& handler);
|
||||
clients::body::Json get_run(const std::string& thread, const std::string& run);
|
||||
clients::body::Json modify_run(const std::string& thread, const std::string& run,
|
||||
const clients::body::Json& body);
|
||||
clients::body::Json list_runs(const std::string& thread, const ListParams& params = {});
|
||||
clients::body::Json cancel_run(const std::string& thread, const std::string& run);
|
||||
clients::body::Json submit_tool_outputs(const std::string& thread, const std::string& run,
|
||||
const clients::body::Json& tool_outputs);
|
||||
clients::body::Json list_run_steps(const std::string& thread, const std::string& run,
|
||||
const ListParams& params = {});
|
||||
clients::body::Json get_run_step(const std::string& thread, const std::string& run,
|
||||
const std::string& step);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+49
@@ -0,0 +1,49 @@
|
||||
#include "vendors/azure/audio.h"
|
||||
|
||||
#include "vendors/azure/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
llm::RawResponse AudioService::speech(const std::string& deployment, const Json& body) {
|
||||
Check::deployment(deployment);
|
||||
Check::object("body", body);
|
||||
Check::has_string(body, "input");
|
||||
Check::has_string(body, "voice");
|
||||
Check::max_chars("input", body.find("input")->as_string(), AzureCaps::kMaxSpeechChars);
|
||||
return call_json_raw(Op::kCreateSpeech, {deployment, {}}, body);
|
||||
}
|
||||
|
||||
llm::RawResponse AudioService::upload(Op op, const std::string& deployment, const UploadFile& file,
|
||||
const std::map<std::string, std::string>& fields) {
|
||||
Check::deployment(deployment);
|
||||
Check::non_empty("audio filename", file.filename);
|
||||
Check::no_control("filename", file.filename);
|
||||
Check::require(!file.data.empty(), "audio file is empty");
|
||||
Check::require(file.data.size() <= AzureCaps::kMaxAudioFileBytes, "audio file over 25 MiB");
|
||||
clients::body::MultipartBody form;
|
||||
form.add_file("file", file.filename, file.content_type, file.data);
|
||||
for (const auto& [k, v] : fields) form.add_text(k, v);
|
||||
return call_multipart_raw(op, {deployment, {}}, form);
|
||||
}
|
||||
|
||||
llm::RawResponse AudioService::transcribe(const std::string& deployment, const UploadFile& file,
|
||||
const std::map<std::string, std::string>& fields) {
|
||||
return upload(Op::kCreateTranscription, deployment, file, fields);
|
||||
}
|
||||
|
||||
llm::RawResponse AudioService::translate(const std::string& deployment, const UploadFile& file,
|
||||
const std::map<std::string, std::string>& fields) {
|
||||
return upload(Op::kCreateTranslation, deployment, file, fields);
|
||||
}
|
||||
|
||||
Json AudioService::as_json(const llm::RawResponse& answer) {
|
||||
if (answer.content_type.find("json") != std::string::npos) return Json::parse(answer.body);
|
||||
Json o = Json::object();
|
||||
o.set("text", Json(answer.body));
|
||||
return o;
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+38
@@ -0,0 +1,38 @@
|
||||
#pragma once
|
||||
|
||||
// Audio on a deployment: speech (tts, tts-hd, gpt-4o-mini-tts), transcription
|
||||
// (whisper, gpt-4o-transcribe) and translation. Transcription and translation
|
||||
// are multipart on both APIs (the v1 document lists JSON for them, see SPEC
|
||||
// Unverified). Answers may be text (srt, vtt, text), so they come back raw.
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
#include "vendors/azure/upload_file.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class AudioService : public AzureService {
|
||||
public:
|
||||
explicit AudioService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body: input (<= 4096 chars) and voice (required), plus response_format,
|
||||
// speed, instructions. The answer is audio bytes (content_type tells which).
|
||||
llm::RawResponse speech(const std::string& deployment, const clients::body::Json& body);
|
||||
// fields: language, prompt, response_format, temperature, ... (spec names).
|
||||
// The file is capped at 25 MiB.
|
||||
llm::RawResponse transcribe(const std::string& deployment, const UploadFile& file,
|
||||
const std::map<std::string, std::string>& fields = {});
|
||||
llm::RawResponse translate(const std::string& deployment, const UploadFile& file,
|
||||
const std::map<std::string, std::string>& fields = {});
|
||||
// The answer parsed as JSON when its content type says so, else {"text": body}.
|
||||
static clients::body::Json as_json(const llm::RawResponse& answer);
|
||||
|
||||
private:
|
||||
llm::RawResponse upload(Op op, const std::string& deployment, const UploadFile& file,
|
||||
const std::map<std::string, std::string>& fields);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+38
@@ -0,0 +1,38 @@
|
||||
#include "vendors/azure/batches.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
Json BatchesService::create(const Json& body) {
|
||||
Check::has_string(body, "endpoint");
|
||||
Check::has_string(body, "completion_window");
|
||||
Check::require(body.find("input_file_id") || body.find("input_blob"),
|
||||
"input_file_id (or input_blob) is required");
|
||||
return call(Op::kCreateBatch, {}, &body);
|
||||
}
|
||||
|
||||
Json BatchesService::get(const std::string& id) {
|
||||
Check::id("batch", id);
|
||||
return call(Op::kRetrieveBatch, {"", {id}});
|
||||
}
|
||||
|
||||
Json BatchesService::list(const ListParams& params) {
|
||||
params.validate();
|
||||
Query q;
|
||||
params.apply(q);
|
||||
return call(Op::kListBatches, {}, nullptr, &q);
|
||||
}
|
||||
|
||||
Listing BatchesService::list_all(const ListParams& params, std::size_t max_pages) {
|
||||
return walk(Op::kListBatches, {}, params, Query(), max_pages);
|
||||
}
|
||||
|
||||
Json BatchesService::cancel(const std::string& id) {
|
||||
Check::id("batch", id);
|
||||
return call(Op::kCancelBatch, {"", {id}});
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+27
@@ -0,0 +1,27 @@
|
||||
#pragma once
|
||||
|
||||
// Batch (Global Batch): jobs over an uploaded JSONL file of requests.
|
||||
// Creating a batch is paid work: the demo runs it as a dry run unless
|
||||
// confirmed.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class BatchesService : public AzureService {
|
||||
public:
|
||||
explicit BatchesService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body: endpoint (/chat/completions, /embeddings, /v1/chat/completions ...),
|
||||
// completion_window ("24h"), and input_file_id (or input_blob on the v1 API).
|
||||
clients::body::Json create(const clients::body::Json& body);
|
||||
clients::body::Json get(const std::string& id);
|
||||
clients::body::Json list(const ListParams& params = {});
|
||||
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
|
||||
clients::body::Json cancel(const std::string& id); // destructive
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+114
@@ -0,0 +1,114 @@
|
||||
#include "vendors/azure/calls.h"
|
||||
|
||||
#include "llm/api_client.h"
|
||||
#include "llm/error.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
namespace {
|
||||
const CallSpec kCalls[] = {
|
||||
#define AZ_OP(name, surface, method, dep, d1, p1, o1, c1, b1, q1, d2, p2, o2, c2, b2, q2) \
|
||||
{Op::k##name, #name, surface, method, dep != 0, {d1, p1, o1, c1, b1, q1}, {d2, p2, o2, c2, b2, q2}},
|
||||
#include "vendors/azure/calls_table.inc"
|
||||
#undef AZ_OP
|
||||
};
|
||||
constexpr std::size_t kCallCount = sizeof(kCalls) / sizeof(kCalls[0]);
|
||||
|
||||
bool listed(const std::string& csv, const std::string& item) {
|
||||
std::size_t pos = 0;
|
||||
while (pos <= csv.size()) {
|
||||
std::size_t end = csv.find(',', pos);
|
||||
if (end == std::string::npos) end = csv.size();
|
||||
if (csv.compare(pos, end - pos, item) == 0 && end - pos == item.size()) return true;
|
||||
pos = end + 1;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
|
||||
} // namespace
|
||||
|
||||
const CallSpec& Calls::spec(Op op) { return kCalls[static_cast<std::size_t>(op)]; }
|
||||
const CallSpec& Calls::at(std::size_t index) { return kCalls[index]; }
|
||||
std::size_t Calls::count() { return kCallCount; }
|
||||
|
||||
const CallSpec* Calls::find(const std::string& name) {
|
||||
for (const CallSpec& c : kCalls)
|
||||
if (name == c.name) return &c;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
clients::Method Calls::method(Op op) {
|
||||
std::string m = spec(op).method;
|
||||
if (m == "GET") return clients::Method::kGet;
|
||||
if (m == "POST") return clients::Method::kPost;
|
||||
if (m == "DELETE") return clients::Method::kDelete;
|
||||
if (m == "PATCH") return clients::Method::kPatch;
|
||||
if (m == "PUT") return clients::Method::kPut;
|
||||
bad("unknown method in table: " + m);
|
||||
}
|
||||
|
||||
bool Calls::accepts_field(Op op, Side side, const std::string& field) {
|
||||
return listed(spec(op).side(side).body, field);
|
||||
}
|
||||
|
||||
bool Calls::accepts_query(Op op, Side side, const std::string& name) {
|
||||
std::string n = name;
|
||||
if (n.size() > 2 && n.compare(n.size() - 2, 2, "[]") == 0) {
|
||||
// The v1 spec names array parameters include[]; the dated one include.
|
||||
if (listed(spec(op).side(side).query, n)) return true;
|
||||
n.resize(n.size() - 2);
|
||||
}
|
||||
return listed(spec(op).side(side).query, n) || listed(spec(op).side(side).query, n + "[]");
|
||||
}
|
||||
|
||||
std::size_t Calls::id_count(Op op, Side side) {
|
||||
std::string path = spec(op).side(side).path;
|
||||
std::size_t n = 0;
|
||||
for (std::size_t i = 0; i < path.size(); ++i)
|
||||
if (path[i] == '{') ++n;
|
||||
if (side == Side::kDated && spec(op).deployment && n > 0) --n;
|
||||
return n;
|
||||
}
|
||||
|
||||
std::string Calls::render(Op op, Side side, const std::vector<std::string>& ids,
|
||||
const std::string& deployment) {
|
||||
const CallSpec& c = spec(op);
|
||||
const SideSpec& s = c.side(side);
|
||||
if (!s.present()) bad(std::string(c.name) + " is not available on this API side");
|
||||
std::string path = s.path;
|
||||
std::string out;
|
||||
std::size_t used = 0;
|
||||
bool first_placeholder = true;
|
||||
for (std::size_t i = 0; i < path.size();) {
|
||||
if (path[i] != '{') {
|
||||
out += path[i++];
|
||||
continue;
|
||||
}
|
||||
std::size_t close = path.find('}', i);
|
||||
if (close == std::string::npos) bad("malformed table path");
|
||||
if (side == Side::kDated && c.deployment && first_placeholder) {
|
||||
if (!llm::Service::valid_id(deployment)) bad("bad deployment name");
|
||||
out += deployment;
|
||||
} else {
|
||||
if (used >= ids.size()) bad(std::string(c.name) + " needs more ids");
|
||||
if (!llm::Service::valid_id(ids[used])) bad("bad id");
|
||||
out += ids[used++];
|
||||
}
|
||||
first_placeholder = false;
|
||||
i = close + 1;
|
||||
}
|
||||
if (used != ids.size()) bad(std::string(c.name) + " got too many ids");
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string Calls::prefix(Op op, Side side) {
|
||||
switch (spec(op).surface) {
|
||||
case Surface::kFoundry: return "/models";
|
||||
case Surface::kSafety: return "/contentsafety";
|
||||
case Surface::kOpenAi: break;
|
||||
}
|
||||
return side == Side::kV1 ? "/openai/v1" : "/openai";
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+76
@@ -0,0 +1,76 @@
|
||||
#pragma once
|
||||
|
||||
// The ONE table of REST operations this package makes (calls_table.inc,
|
||||
// generated by tools/gen_calls.py from the pinned azure-rest-api-specs and
|
||||
// verified by tools/check_spec.py). Code builds every path from this table.
|
||||
//
|
||||
// An OpenAI-surface operation has up to two sides: the v1 API (/openai/v1,
|
||||
// document "v1" / "v1p") and the dated API (/openai, documents "inf25" /
|
||||
// "auth24", deployment URLs). Foundry model inference and Content Safety are
|
||||
// single-sided (the dated slot).
|
||||
|
||||
#include <cstddef>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/request.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
enum class Surface { kOpenAi, kFoundry, kSafety };
|
||||
enum class Side { kV1, kDated };
|
||||
|
||||
enum class Op {
|
||||
#define AZ_OP(name, ...) k##name,
|
||||
#include "vendors/azure/calls_table.inc"
|
||||
#undef AZ_OP
|
||||
};
|
||||
|
||||
struct SideSpec {
|
||||
const char* doc; // spec document short name, "" when the side is absent
|
||||
const char* path; // spec path under the server base
|
||||
const char* op_id; // spec operationId ("" when the spec has none)
|
||||
const char* content; // json, multipart, json+multipart, none (as the spec lists it)
|
||||
const char* body; // every top-level request property, comma separated
|
||||
const char* query; // query parameters except api-version, comma separated
|
||||
bool present() const { return path[0] != '\0'; }
|
||||
};
|
||||
|
||||
struct CallSpec {
|
||||
Op op;
|
||||
const char* name;
|
||||
Surface surface;
|
||||
const char* method;
|
||||
bool deployment; // dated path is /deployments/{deployment-id}/...
|
||||
SideSpec v1;
|
||||
SideSpec dated;
|
||||
const SideSpec& side(Side s) const { return s == Side::kV1 ? v1 : dated; }
|
||||
};
|
||||
|
||||
class Calls {
|
||||
public:
|
||||
static const CallSpec& spec(Op op);
|
||||
static const CallSpec& at(std::size_t index);
|
||||
static std::size_t count();
|
||||
// By table name ("CreateChatCompletion"); nullptr when unknown.
|
||||
static const CallSpec* find(const std::string& name);
|
||||
static clients::Method method(Op op);
|
||||
|
||||
static bool has(Op op, Side side) { return spec(op).side(side).present(); }
|
||||
// True when the spec of that side lists `field` as a request property.
|
||||
static bool accepts_field(Op op, Side side, const std::string& field);
|
||||
// True when the spec lists `name` as a query parameter ("include[]" matches "include").
|
||||
static bool accepts_query(Op op, Side side, const std::string& name);
|
||||
// Number of {placeholders} the caller supplies as ids (the deployment of a
|
||||
// dated deployment-scoped path is not one of them).
|
||||
static std::size_t id_count(Op op, Side side);
|
||||
// Spec path with ids substituted (each validated as one URL path segment),
|
||||
// without the surface prefix and without the deployment segment.
|
||||
// `deployment` is inserted for a dated deployment-scoped side.
|
||||
static std::string render(Op op, Side side, const std::vector<std::string>& ids,
|
||||
const std::string& deployment = "");
|
||||
// Server base path of a side: "/openai/v1", "/openai", "/models", "/contentsafety".
|
||||
static std::string prefix(Op op, Side side);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+152
@@ -0,0 +1,152 @@
|
||||
// GENERATED by tools/gen_calls.py from azure-rest-api-specs ba3eef39e5be55cd5ed1a2e5c6c9bb1a17c36b9a. Do not edit.
|
||||
// AZ_OP(name, surface, method, deployment_scoped, then twice (v1 side, dated side):
|
||||
// doc, path, operation_id, content, body_fields, query_params)
|
||||
// AZ-ROWS-BEGIN
|
||||
AZ_OP(DeleteAssistant, Surface::kOpenAi, "DELETE", 0, "", "", "", "", "", "", "inf25", "/assistants/{assistant_id}", "Delete_Assistant", "none", "", "")
|
||||
AZ_OP(DeleteContainer, Surface::kOpenAi, "DELETE", 0, "v1p", "/containers/{container_id}", "deleteContainer", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteContainerFile, Surface::kOpenAi, "DELETE", 0, "v1p", "/containers/{container_id}/files/{file_id}", "deleteContainerFile", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteConversation, Surface::kOpenAi, "DELETE", 0, "v1p", "/conversations/{conversation_id}", "deleteConversation", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteConversationItem, Surface::kOpenAi, "DELETE", 0, "v1p", "/conversations/{conversation_id}/items/{item_id}", "deleteConversationItem", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteEval, Surface::kOpenAi, "DELETE", 0, "v1p", "/evals/{eval_id}", "deleteEval", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteEvalRun, Surface::kOpenAi, "DELETE", 0, "v1p", "/evals/{eval_id}/runs/{run_id}", "deleteEvalRun", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteFile, Surface::kOpenAi, "DELETE", 0, "v1p", "/files/{file_id}", "deleteFile", "none", "", "", "auth24", "/files/{file-id}", "Files_Delete", "none", "", "")
|
||||
AZ_OP(DeleteFineTuningCheckpointPermission, Surface::kOpenAi, "DELETE", 0, "v1p", "/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions/{permission_id}", "deleteFineTuningCheckpointPermission", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(FineTuningDelete, Surface::kOpenAi, "DELETE", 0, "", "", "", "", "", "", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}", "FineTuning_Delete", "none", "", "")
|
||||
AZ_OP(DeleteModel, Surface::kOpenAi, "DELETE", 0, "v1p", "/models/{model}", "deleteModel", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteResponse, Surface::kOpenAi, "DELETE", 0, "v1p", "/responses/{response_id}", "deleteResponse", "none", "", "", "inf25", "/responses/{response_id}", "deleteResponse", "none", "", "")
|
||||
AZ_OP(DeleteThread, Surface::kOpenAi, "DELETE", 0, "v1p", "/threads/{thread_id}", "deleteThread", "none", "", "", "inf25", "/threads/{thread_id}", "Delete_Thread", "none", "", "")
|
||||
AZ_OP(DeleteMessage, Surface::kOpenAi, "DELETE", 0, "v1p", "/threads/{thread_id}/messages/{message_id}", "deleteMessage", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(DeleteVectorStore, Surface::kOpenAi, "DELETE", 0, "v1p", "/vector_stores/{vector_store_id}", "deleteVectorStore", "none", "", "", "inf25", "/vector_stores/{vector_store_id}", "Delete_Vector_Store", "none", "", "")
|
||||
AZ_OP(DeleteVectorStoreFile, Surface::kOpenAi, "DELETE", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}", "deleteVectorStoreFile", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}", "Delete_Vector_Store_File", "none", "", "")
|
||||
AZ_OP(VideosDelete, Surface::kOpenAi, "DELETE", 0, "v1p", "/videos/{video-id}", "Videos_Delete", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ListAssistants, Surface::kOpenAi, "GET", 0, "", "", "", "", "", "", "inf25", "/assistants", "List_Assistants", "none", "", "after,before,limit,order")
|
||||
AZ_OP(GetAssistant, Surface::kOpenAi, "GET", 0, "", "", "", "", "", "", "inf25", "/assistants/{assistant_id}", "Get_Assistant", "none", "", "")
|
||||
AZ_OP(ListBatches, Surface::kOpenAi, "GET", 0, "v1p", "/batches", "listBatches", "none", "", "$filter,$orderby,after,limit", "auth24", "/batches", "Batch_List", "none", "", "after,limit")
|
||||
AZ_OP(RetrieveBatch, Surface::kOpenAi, "GET", 0, "v1p", "/batches/{batch_id}", "retrieveBatch", "none", "", "", "auth24", "/batches/{batch-id}", "Batch_Get", "none", "", "")
|
||||
AZ_OP(ListContainers, Surface::kOpenAi, "GET", 0, "v1p", "/containers", "listContainers", "none", "", "after,before,limit,order", "", "", "", "", "", "")
|
||||
AZ_OP(RetrieveContainer, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}", "retrieveContainer", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ListContainerFiles, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}/files", "listContainerFiles", "none", "", "after,before,limit,order", "", "", "", "", "", "")
|
||||
AZ_OP(RetrieveContainerFile, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}/files/{file_id}", "retrieveContainerFile", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(RetrieveContainerFileContent, Surface::kOpenAi, "GET", 0, "v1p", "/containers/{container_id}/files/{file_id}/content", "retrieveContainerFileContent", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(RetrieveConversation, Surface::kOpenAi, "GET", 0, "v1p", "/conversations/{conversation_id}", "retrieveConversation", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ListConversationItems, Surface::kOpenAi, "GET", 0, "v1p", "/conversations/{conversation_id}/items", "listConversationItems", "none", "", "after,include,limit,order", "", "", "", "", "", "")
|
||||
AZ_OP(RetrieveConversationItem, Surface::kOpenAi, "GET", 0, "v1p", "/conversations/{conversation_id}/items/{item_id}", "retrieveConversationItem", "none", "", "include", "", "", "", "", "", "")
|
||||
AZ_OP(ListEvals, Surface::kOpenAi, "GET", 0, "v1p", "/evals", "listEvals", "none", "", "after,limit,order,order_by", "", "", "", "", "", "")
|
||||
AZ_OP(GetEval, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}", "getEval", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(GetEvalRuns, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs", "getEvalRuns", "none", "", "after,limit,order,status", "", "", "", "", "", "")
|
||||
AZ_OP(GetEvalRun, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs/{run_id}", "getEvalRun", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(GetEvalRunOutputItems, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs/{run_id}/output_items", "getEvalRunOutputItems", "none", "", "after,limit,order,status", "", "", "", "", "", "")
|
||||
AZ_OP(GetEvalRunOutputItem, Surface::kOpenAi, "GET", 0, "v1p", "/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}", "getEvalRunOutputItem", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ListFiles, Surface::kOpenAi, "GET", 0, "v1p", "/files", "listFiles", "none", "", "after,limit,order,purpose", "auth24", "/files", "Files_List", "none", "", "purpose")
|
||||
AZ_OP(RetrieveFile, Surface::kOpenAi, "GET", 0, "v1p", "/files/{file_id}", "retrieveFile", "none", "", "", "auth24", "/files/{file-id}", "Files_Get", "none", "", "")
|
||||
AZ_OP(DownloadFile, Surface::kOpenAi, "GET", 0, "v1p", "/files/{file_id}/content", "downloadFile", "none", "", "", "auth24", "/files/{file-id}/content", "Files_GetContent", "none", "", "")
|
||||
AZ_OP(ListFineTuningCheckpointPermissions, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions", "listFineTuningCheckpointPermissions", "none", "", "after,limit,order,project_id", "", "", "", "", "", "")
|
||||
AZ_OP(ListPaginatedFineTuningJobs, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs", "listPaginatedFineTuningJobs", "none", "", "after,limit", "auth24", "/fine_tuning/jobs", "FineTuning_List", "none", "", "after,limit")
|
||||
AZ_OP(RetrieveFineTuningJob, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}", "retrieveFineTuningJob", "none", "", "", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}", "FineTuning_Get", "none", "", "")
|
||||
AZ_OP(ListFineTuningJobCheckpoints, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints", "listFineTuningJobCheckpoints", "none", "", "after,limit", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}/checkpoints", "FineTuning_GetCheckpoints", "none", "", "after,limit")
|
||||
AZ_OP(FineTuningGetCheckpoint, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints/{fine_tuning_checkpoint_id}/copy", "FineTuning_GetCheckpoint", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ListFineTuningEvents, Surface::kOpenAi, "GET", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/events", "listFineTuningEvents", "none", "", "after,limit", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}/events", "FineTuning_GetEvents", "none", "", "after,limit")
|
||||
AZ_OP(ListModels, Surface::kOpenAi, "GET", 0, "v1p", "/models", "listModels", "none", "", "", "auth24", "/models", "Models_List", "none", "", "")
|
||||
AZ_OP(RetrieveModel, Surface::kOpenAi, "GET", 0, "v1p", "/models/{model}", "retrieveModel", "none", "", "", "auth24", "/models/{model-id}", "Models_Get", "none", "", "")
|
||||
AZ_OP(GetResponse, Surface::kOpenAi, "GET", 0, "v1p", "/responses/{response_id}", "getResponse", "none", "", "include[],include_obfuscation,starting_after,stream", "inf25", "/responses/{response_id}", "", "none", "", "include")
|
||||
AZ_OP(ListInputItems, Surface::kOpenAi, "GET", 0, "v1p", "/responses/{response_id}/input_items", "listInputItems", "none", "", "after,before,limit,order", "inf25", "/responses/{response_id}/input_items", "", "none", "", "after,before,limit,order")
|
||||
AZ_OP(RetrieveThread, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}", "retrieveThread", "none", "", "", "inf25", "/threads/{thread_id}", "Get_Thread", "none", "", "")
|
||||
AZ_OP(ListMessages, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/messages", "listMessages", "none", "", "after,before,limit,order,run_id", "inf25", "/threads/{thread_id}/messages", "List_Messages", "none", "", "after,before,limit,order,run_id")
|
||||
AZ_OP(RetrieveMessage, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/messages/{message_id}", "retrieveMessage", "none", "", "", "inf25", "/threads/{thread_id}/messages/{message_id}", "Get_Message", "none", "", "")
|
||||
AZ_OP(ListRuns, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs", "listRuns", "none", "", "after,before,limit,order", "inf25", "/threads/{thread_id}/runs", "List_Runs", "none", "", "after,before,limit,order")
|
||||
AZ_OP(RetrieveRun, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs/{run_id}", "retrieveRun", "none", "", "", "inf25", "/threads/{thread_id}/runs/{run_id}", "Get_Run", "none", "", "")
|
||||
AZ_OP(ListRunSteps, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/steps", "listRunSteps", "none", "", "after,before,limit,order", "inf25", "/threads/{thread_id}/runs/{run_id}/steps", "List_Run_Steps", "none", "", "after,before,include[],limit,order")
|
||||
AZ_OP(GetRunStep, Surface::kOpenAi, "GET", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/steps/{step_id}", "getRunStep", "none", "", "", "inf25", "/threads/{thread_id}/runs/{run_id}/steps/{step_id}", "Get_Run_Step", "none", "", "include[]")
|
||||
AZ_OP(ListVectorStores, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores", "listVectorStores", "none", "", "after,before,limit,order", "inf25", "/vector_stores", "List_Vector_Stores", "none", "", "after,before,limit,order")
|
||||
AZ_OP(GetVectorStore, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}", "getVectorStore", "none", "", "", "inf25", "/vector_stores/{vector_store_id}", "Get_Vector_Store", "none", "", "")
|
||||
AZ_OP(GetVectorStoreFileBatch, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches/{batch_id}", "getVectorStoreFileBatch", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/file_batches/{batch_id}", "Get_Vector_Store_File_Batch", "none", "", "")
|
||||
AZ_OP(ListFilesInVectorStoreBatch, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/files", "listFilesInVectorStoreBatch", "none", "", "after,before,filter,limit,order", "inf25", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/files", "List_Vector_Store_File_Batch_Files", "none", "", "after,before,filter,limit,order")
|
||||
AZ_OP(ListVectorStoreFiles, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/files", "listVectorStoreFiles", "none", "", "after,before,filter,limit,order", "inf25", "/vector_stores/{vector_store_id}/files", "List_Vector_Store_Files", "none", "", "after,before,filter,limit,order")
|
||||
AZ_OP(GetVectorStoreFile, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}", "getVectorStoreFile", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}", "Get_Vector_Store_File", "none", "", "")
|
||||
AZ_OP(RetrieveVectorStoreFileContent, Surface::kOpenAi, "GET", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}/content", "retrieveVectorStoreFileContent", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}/content", "retrieveVectorStoreFileContent", "none", "", "")
|
||||
AZ_OP(VideosList, Surface::kOpenAi, "GET", 0, "v1p", "/videos", "Videos_List", "none", "", "after,limit,order", "", "", "", "", "", "")
|
||||
AZ_OP(VideosGet, Surface::kOpenAi, "GET", 0, "v1p", "/videos/{video-id}", "Videos_Get", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(VideosRetrieveContent, Surface::kOpenAi, "GET", 0, "v1p", "/videos/{video-id}/content", "Videos_RetrieveContent", "none", "", "variant", "", "", "", "", "", "")
|
||||
AZ_OP(CreateAssistant, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/assistants", "Create_Assistant", "json", "description,instructions,metadata,model,name,response_format,temperature,tool_resources,tools,top_p", "")
|
||||
AZ_OP(ModifyAssistant, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/assistants/{assistant_id}", "Modify_Assistant", "json", "description,instructions,metadata,model,name,response_format,temperature,tool_resources,tools,top_p", "")
|
||||
AZ_OP(CreateSpeech, Surface::kOpenAi, "POST", 1, "v1p", "/audio/speech", "createSpeech", "json", "input,instructions,model,response_format,speed,stream_format,voice", "", "inf25", "/deployments/{deployment-id}/audio/speech", "Speech_Create", "multipart", "input,response_format,speed,voice", "")
|
||||
AZ_OP(CreateTranscription, Surface::kOpenAi, "POST", 1, "v1p", "/audio/transcriptions", "createTranscription", "json", "chunking_strategy,file,filename,include,known_speaker_names,known_speaker_references,language,model,prompt,response_format,stream,temperature,timestamp_granularities", "", "inf25", "/deployments/{deployment-id}/audio/transcriptions", "Transcriptions_Create", "multipart", "file,language,prompt,response_format,temperature,timestamp_granularities[]", "")
|
||||
AZ_OP(CreateTranslation, Surface::kOpenAi, "POST", 1, "v1p", "/audio/translations", "createTranslation", "json", "file,filename,model,prompt,response_format,temperature", "", "inf25", "/deployments/{deployment-id}/audio/translations", "Translations_Create", "multipart", "file,prompt,response_format,temperature", "")
|
||||
AZ_OP(CreateBatch, Surface::kOpenAi, "POST", 0, "v1p", "/batches", "createBatch", "json", "completion_window,endpoint,input_blob,input_file_id,output_expires_after,output_folder", "", "auth24", "/batches", "Batch_Create", "json", "completion_window,endpoint,input_file_id,metadata", "")
|
||||
AZ_OP(CancelBatch, Surface::kOpenAi, "POST", 0, "v1p", "/batches/{batch_id}/cancel", "cancelBatch", "none", "", "", "auth24", "/batches/{batch-id}/cancel", "Batch_Cancel", "none", "", "")
|
||||
AZ_OP(CreateChatCompletion, Surface::kOpenAi, "POST", 1, "v1p", "/chat/completions", "createChatCompletion", "json", "audio,frequency_penalty,function_call,functions,logit_bias,logprobs,max_completion_tokens,max_tokens,messages,metadata,modalities,model,n,parallel_tool_calls,prediction,presence_penalty,prompt_cache_key,prompt_cache_retention,reasoning_effort,response_format,safety_identifier,seed,stop,store,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user,user_security_context,verbosity", "", "inf25", "/deployments/{deployment-id}/chat/completions", "ChatCompletions_Create", "json", "audio,data_sources,frequency_penalty,function_call,functions,logit_bias,logprobs,max_completion_tokens,max_tokens,messages,metadata,modalities,n,parallel_tool_calls,prediction,presence_penalty,reasoning_effort,response_format,seed,stop,store,stream,stream_options,temperature,tool_choice,tools,top_logprobs,top_p,user,user_security_context", "")
|
||||
AZ_OP(CreateCompletion, Surface::kOpenAi, "POST", 1, "v1p", "/completions", "createCompletion", "json", "best_of,echo,frequency_penalty,logit_bias,logprobs,max_tokens,model,n,presence_penalty,prompt,seed,stop,stream,stream_options,suffix,temperature,top_p,user", "", "inf25", "/deployments/{deployment-id}/completions", "Completions_Create", "json", "best_of,echo,frequency_penalty,logit_bias,logprobs,max_tokens,n,presence_penalty,prompt,seed,stop,stream,suffix,temperature,top_p,user", "")
|
||||
AZ_OP(CreateContainer, Surface::kOpenAi, "POST", 0, "v1p", "/containers", "createContainer", "json", "expires_after,file_ids,memory_limit,name,network_policy,skills", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateContainerFile, Surface::kOpenAi, "POST", 0, "v1p", "/containers/{container_id}/files", "createContainerFile", "multipart", "file,file_id", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateConversation, Surface::kOpenAi, "POST", 0, "v1p", "/conversations", "createConversation", "json", "items,metadata", "", "", "", "", "", "", "")
|
||||
AZ_OP(UpdateConversation, Surface::kOpenAi, "POST", 0, "v1p", "/conversations/{conversation_id}", "updateConversation", "json", "metadata", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateConversationItems, Surface::kOpenAi, "POST", 0, "v1p", "/conversations/{conversation_id}/items", "createConversationItems", "json", "items", "include", "", "", "", "", "", "")
|
||||
AZ_OP(CreateEmbedding, Surface::kOpenAi, "POST", 1, "v1p", "/embeddings", "createEmbedding", "json", "dimensions,encoding_format,input,model,user", "", "inf25", "/deployments/{deployment-id}/embeddings", "embeddings_create", "json", "dimensions,encoding_format,input,input_type,user", "")
|
||||
AZ_OP(CreateEval, Surface::kOpenAi, "POST", 0, "v1p", "/evals", "createEval", "json", "data_source_config,metadata,name,statusCode,testing_criteria", "", "", "", "", "", "", "")
|
||||
AZ_OP(UpdateEval, Surface::kOpenAi, "POST", 0, "v1p", "/evals/{eval_id}", "updateEval", "json", "metadata,name", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateEvalRun, Surface::kOpenAi, "POST", 0, "v1p", "/evals/{eval_id}/runs", "createEvalRun", "json", "data_source,metadata,name", "", "", "", "", "", "", "")
|
||||
AZ_OP(CancelEvalRun, Surface::kOpenAi, "POST", 0, "v1p", "/evals/{eval_id}/runs/{run_id}", "cancelEvalRun", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateFile, Surface::kOpenAi, "POST", 0, "v1p", "/files", "createFile", "multipart", "expires_after,file,purpose", "", "auth24", "/files", "Files_Upload", "multipart", "file,purpose", "")
|
||||
AZ_OP(FilesImport, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/files/import", "Files_Import", "json", "content_url,filename,purpose", "")
|
||||
AZ_OP(RunGrader, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/alpha/graders/run", "runGrader", "json", "grader,item,model_sample", "", "", "", "", "", "", "")
|
||||
AZ_OP(ValidateGrader, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/alpha/graders/validate", "validateGrader", "json", "grader", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateFineTuningCheckpointPermission, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/checkpoints/{fine_tuned_model_checkpoint}/permissions", "createFineTuningCheckpointPermission", "json", "project_ids", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs", "createFineTuningJob", "json", "hyperparameters,integrations,metadata,method,model,seed,suffix,training_file,validation_file", "", "auth24", "/fine_tuning/jobs", "FineTuning_Create", "json", "hyperparameters,model,seed,suffix,training_file,validation_file", "")
|
||||
AZ_OP(CancelFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/cancel", "cancelFineTuningJob", "none", "", "", "auth24", "/fine_tuning/jobs/{fine-tuning-job-id}/cancel", "FineTuning_Cancel", "none", "", "")
|
||||
AZ_OP(FineTuningCopyCheckpoint, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/checkpoints/{fine_tuning_checkpoint_id}/copy", "FineTuning_CopyCheckpoint", "json", "destinationResourceId,region", "", "", "", "", "", "", "")
|
||||
AZ_OP(PauseFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/pause", "pauseFineTuningJob", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ResumeFineTuningJob, Surface::kOpenAi, "POST", 0, "v1p", "/fine_tuning/jobs/{fine_tuning_job_id}/resume", "resumeFineTuningJob", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateImageEdit, Surface::kOpenAi, "POST", 1, "v1p", "/images/edits", "createImageEdit", "json", "background,image,input_fidelity,mask,model,n,output_compression,output_format,partial_images,prompt,quality,response_format,size,stream,user", "", "inf25", "/deployments/{deployment-id}/images/edits", "ImageGenerations_Edit", "json", "image,mask,n,prompt,quality,response_format,size,user", "")
|
||||
AZ_OP(CreateImage, Surface::kOpenAi, "POST", 1, "v1p", "/images/generations", "createImage", "json", "background,model,moderation,n,output_compression,output_format,partial_images,prompt,quality,response_format,size,stream,style,user", "", "inf25", "/deployments/{deployment-id}/images/generations", "ImageGenerations_Create", "json", "background,n,output_compression,output_format,prompt,quality,response_format,size,style,user", "")
|
||||
AZ_OP(CreateImageVariation, Surface::kOpenAi, "POST", 0, "v1p", "/images/variations", "createImageVariation", "json", "image,model,n,response_format,size,user", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls", "createRealtimeCall", "multipart", "sdp,session", "", "", "", "", "", "", "")
|
||||
AZ_OP(AcceptRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/accept", "acceptRealtimeCall", "json", "audio,include,instructions,max_output_tokens,model,output_modalities,parallel_tool_calls,prompt,reasoning,tool_choice,tools,tracing,truncation,type", "", "", "", "", "", "", "")
|
||||
AZ_OP(HangupRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/hangup", "hangupRealtimeCall", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(ReferRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/refer", "referRealtimeCall", "json", "target_uri", "", "", "", "", "", "", "")
|
||||
AZ_OP(RejectRealtimeCall, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/calls/{call_id}/reject", "rejectRealtimeCall", "json", "status_code", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateRealtimeClientSecret, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/client_secrets", "createRealtimeClientSecret", "json", "expires_after,session", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateRealtimeSession, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/sessions", "createRealtimeSession", "json", "client_secret,input_audio_format,input_audio_transcription,instructions,max_response_output_tokens,modalities,output_audio_format,prompt,speed,temperature,tool_choice,tools,tracing,truncation,turn_detection,voice", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateRealtimeTranscriptionSession, Surface::kOpenAi, "POST", 0, "v1p", "/realtime/transcription_sessions", "createRealtimeTranscriptionSession", "json", "include,input_audio_format,input_audio_noise_reduction,input_audio_transcription,turn_detection", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateRealtimeSessionDated, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/realtimeapi/sessions", "create_realtimeSession", "json", "input_audio_format,input_audio_noise_reduction,input_audio_transcription,instructions,max_response_output_tokens,modalities,model,output_audio_format,temperature,tool_choice,tools,turn_detection,voice", "")
|
||||
AZ_OP(CreateTranscriptionRealtimeSession, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "inf25", "/realtimeapi/transcription_sessions", "create_transcriptionRealtimeSession", "json", "include,input_audio_format,input_audio_noise_reduction,input_audio_transcription,modalities,turn_detection", "")
|
||||
AZ_OP(CreateResponse, Surface::kOpenAi, "POST", 0, "v1p", "/responses", "createResponse", "json", "background,context_management,conversation,include,input,instructions,max_output_tokens,max_tool_calls,metadata,model,parallel_tool_calls,previous_response_id,prompt,prompt_cache_key,prompt_cache_retention,reasoning,safety_identifier,store,stream,stream_options,temperature,text,tool_choice,tools,top_logprobs,top_p,truncation,user", "", "inf25", "/responses", "", "json", "include,input,instructions,max_output_tokens,metadata,model,parallel_tool_calls,previous_response_id,reasoning,store,stream,temperature,text,tool_choice,tools,top_p,truncation,user", "")
|
||||
AZ_OP(CancelResponse, Surface::kOpenAi, "POST", 0, "v1p", "/responses/{response_id}/cancel", "cancelResponse", "none", "", "", "", "", "", "", "", "")
|
||||
AZ_OP(CreateThread, Surface::kOpenAi, "POST", 0, "v1p", "/threads", "createThread", "json", "messages,metadata,tool_resources", "", "inf25", "/threads", "Create_Thread", "json", "messages,metadata,tool_resources", "")
|
||||
AZ_OP(CreateThreadAndRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/runs", "createThreadAndRun", "json", "assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,response_format,stream,temperature,thread,tool_choice,tool_resources,tools,top_p,truncation_strategy", "", "inf25", "/threads/runs", "Create_Thread_And_Run", "json", "assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,response_format,stream,stream_options,temperature,thread,tool_choice,tool_resources,tools,top_p,truncation_strategy", "")
|
||||
AZ_OP(ModifyThread, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}", "modifyThread", "json", "metadata,tool_resources", "", "inf25", "/threads/{thread_id}", "Modify_Thread", "json", "metadata,tool_resources", "")
|
||||
AZ_OP(CreateMessage, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/messages", "createMessage", "json", "attachments,content,metadata,role", "", "inf25", "/threads/{thread_id}/messages", "Create_Message", "json", "attachments,content,metadata,role", "")
|
||||
AZ_OP(ModifyMessage, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/messages/{message_id}", "modifyMessage", "json", "metadata", "", "inf25", "/threads/{thread_id}/messages/{message_id}", "Modify_Message", "json", "metadata", "")
|
||||
AZ_OP(CreateRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs", "createRun", "json", "additional_instructions,additional_messages,assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,reasoning_effort,response_format,stream,temperature,tool_choice,tools,top_p,truncation_strategy", "", "inf25", "/threads/{thread_id}/runs", "Create_Run", "json", "additional_instructions,additional_messages,assistant_id,instructions,max_completion_tokens,max_prompt_tokens,metadata,model,parallel_tool_calls,response_format,stream,temperature,tool_choice,tools,top_p,truncation_strategy", "include[]")
|
||||
AZ_OP(ModifyRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs/{run_id}", "modifyRun", "json", "metadata", "", "inf25", "/threads/{thread_id}/runs/{run_id}", "Modify_Run", "json", "metadata", "")
|
||||
AZ_OP(CancelRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/cancel", "cancelRun", "none", "", "", "inf25", "/threads/{thread_id}/runs/{run_id}/cancel", "Cancel_Run", "none", "", "")
|
||||
AZ_OP(SubmitToolOutputsToRun, Surface::kOpenAi, "POST", 0, "v1p", "/threads/{thread_id}/runs/{run_id}/submit_tool_outputs", "submitToolOutputsToRun", "json", "stream,tool_outputs", "", "inf25", "/threads/{thread_id}/runs/{run_id}/submit_tool_outputs", "Submit_Tool_Outputs_To_Run", "json", "stream,tool_outputs", "")
|
||||
AZ_OP(UploadFileStart, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads", "UploadFile_Start", "json", "bytes,filename,mime_type,purpose", "")
|
||||
AZ_OP(UploadFileCancel, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads/{upload-id}/cancel", "UploadFile_Cancel", "none", "", "")
|
||||
AZ_OP(UploadFileComplete, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads/{upload-id}/complete", "UploadFile_Complete", "json", "md5,part_ids", "")
|
||||
AZ_OP(UploadFilePart, Surface::kOpenAi, "POST", 0, "", "", "", "", "", "", "auth24", "/uploads/{upload-id}/parts", "UploadFile_Part", "multipart", "data", "")
|
||||
AZ_OP(CreateVectorStore, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores", "createVectorStore", "json", "chunking_strategy,description,expires_after,file_ids,metadata,name", "", "inf25", "/vector_stores", "Create_Vector_Store", "json", "chunking_strategy,expires_after,file_ids,metadata,name", "")
|
||||
AZ_OP(ModifyVectorStore, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}", "modifyVectorStore", "json", "expires_after,metadata,name", "", "inf25", "/vector_stores/{vector_store_id}", "Modify_Vector_Store", "json", "expires_after,metadata,name", "")
|
||||
AZ_OP(CreateVectorStoreFileBatch, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches", "createVectorStoreFileBatch", "json", "attributes,chunking_strategy,file_ids,files", "", "inf25", "/vector_stores/{vector_store_id}/file_batches", "Create_Vector_Store_File_Batch", "json", "chunking_strategy,file_ids", "")
|
||||
AZ_OP(CancelVectorStoreFileBatch, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/cancel", "cancelVectorStoreFileBatch", "none", "", "", "inf25", "/vector_stores/{vector_store_id}/file_batches/{batch_id}/cancel", "Cancel_Vector_Store_File_Batch", "none", "", "")
|
||||
AZ_OP(CreateVectorStoreFile, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/files", "createVectorStoreFile", "json", "attributes,chunking_strategy,file_id", "", "inf25", "/vector_stores/{vector_store_id}/files", "Create_Vector_Store_File", "json", "chunking_strategy,file_id", "")
|
||||
AZ_OP(UpdateVectorStoreFileAttributes, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/files/{file_id}", "updateVectorStoreFileAttributes", "json", "attributes", "", "inf25", "/vector_stores/{vector_store_id}/files/{file_id}", "updateVectorStoreFileAttributes", "json", "attributes", "")
|
||||
AZ_OP(SearchVectorStore, Surface::kOpenAi, "POST", 0, "v1p", "/vector_stores/{vector_store_id}/search", "searchVectorStore", "json", "filters,max_num_results,query,ranking_options,rewrite_query", "", "inf25", "/vector_stores/{vector_store_id}/search", "searchVectorStore", "json", "filters,max_num_results,query,ranking_options,rewrite_query", "")
|
||||
AZ_OP(VideosCreate, Surface::kOpenAi, "POST", 0, "v1p", "/videos", "Videos_Create", "json+multipart", "input_reference,model,prompt,seconds,size", "", "", "", "", "", "", "")
|
||||
AZ_OP(VideosRemix, Surface::kOpenAi, "POST", 0, "v1p", "/videos/{video-id}/remix", "Videos_Remix", "json", "prompt", "", "", "", "", "", "", "")
|
||||
AZ_OP(FoundryGetModelInfo, Surface::kFoundry, "GET", 0, "", "", "", "", "", "", "mi", "/info", "GetModelInfo", "none", "", "")
|
||||
AZ_OP(FoundryGetChatCompletions, Surface::kFoundry, "POST", 0, "", "", "", "", "", "", "mi", "/chat/completions", "GetChatCompletions", "json", "frequency_penalty,max_tokens,messages,modalities,model,presence_penalty,response_format,seed,stop,stream,temperature,tool_choice,tools,top_p", "")
|
||||
AZ_OP(FoundryGetEmbeddings, Surface::kFoundry, "POST", 0, "", "", "", "", "", "", "mi", "/embeddings", "GetEmbeddings", "json", "dimensions,encoding_format,input,input_type,model", "")
|
||||
AZ_OP(FoundryGetImageEmbeddings, Surface::kFoundry, "POST", 0, "", "", "", "", "", "", "mi", "/images/embeddings", "GetImageEmbeddings", "json", "dimensions,encoding_format,input,input_type,model", "")
|
||||
AZ_OP(SafetyTextBlocklistsDeleteTextBlocklist, Surface::kSafety, "DELETE", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}", "TextBlocklists_DeleteTextBlocklist", "none", "", "")
|
||||
AZ_OP(SafetyTextBlocklistsListTextBlocklists, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists", "TextBlocklists_ListTextBlocklists", "none", "", "")
|
||||
AZ_OP(SafetyTextBlocklistsGetTextBlocklist, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}", "TextBlocklists_GetTextBlocklist", "none", "", "")
|
||||
AZ_OP(SafetyTextBlocklistsListTextBlocklistItems, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}/blocklistItems", "TextBlocklists_ListTextBlocklistItems", "none", "", "maxpagesize,skip,top")
|
||||
AZ_OP(SafetyTextBlocklistsGetTextBlocklistItem, Surface::kSafety, "GET", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}/blocklistItems/{blocklistItemId}", "TextBlocklists_GetTextBlocklistItem", "none", "", "")
|
||||
AZ_OP(SafetyTextBlocklistsCreateOrUpdateTextBlocklist, Surface::kSafety, "PATCH", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}", "TextBlocklists_CreateOrUpdateTextBlocklist", "json", "blocklistName,description", "")
|
||||
AZ_OP(SafetyImageOperationsAnalyzeImage, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/image:analyze", "ImageOperations_AnalyzeImage", "json", "categories,image,outputType", "")
|
||||
AZ_OP(SafetyTextBlocklistsAddOrUpdateBlocklistItems, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}:addOrUpdateBlocklistItems", "TextBlocklists_AddOrUpdateBlocklistItems", "json", "blocklistItems", "")
|
||||
AZ_OP(SafetyTextBlocklistsRemoveBlocklistItems, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text/blocklists/{blocklistName}:removeBlocklistItems", "TextBlocklists_RemoveBlocklistItems", "json", "blocklistItemIds", "")
|
||||
AZ_OP(SafetyTextOperationsAnalyzeText, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text:analyze", "TextOperations_AnalyzeText", "json", "blocklistNames,categories,haltOnBlocklistHit,outputType,text", "")
|
||||
AZ_OP(SafetyTextOperationsDetectTextProtectedMaterial, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text:detectProtectedMaterial", "TextOperations_DetectTextProtectedMaterial", "json", "text", "")
|
||||
AZ_OP(SafetyTextOperationsShieldPrompt, Surface::kSafety, "POST", 0, "", "", "", "", "", "", "cs24", "/text:shieldPrompt", "TextOperations_ShieldPrompt", "json", "documents,userPrompt", "")
|
||||
// AZ-ROWS-END
|
||||
Vendored
+48
@@ -0,0 +1,48 @@
|
||||
#pragma once
|
||||
|
||||
// Static ceilings for the Azure package. Never autotuned. Numbers come from
|
||||
// Microsoft Learn limits pages and the OpenAPI documents (see SPEC.md).
|
||||
|
||||
#include <cstddef>
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
struct AzureCaps {
|
||||
// Names and endpoints.
|
||||
static constexpr std::size_t kMaxEndpointChars = 256;
|
||||
static constexpr std::size_t kMaxDeploymentChars = 64; // Azure deployment name limit
|
||||
static constexpr std::size_t kMaxIdChars = 256;
|
||||
static constexpr std::size_t kMaxSecretChars = 1024;
|
||||
// Lists.
|
||||
static constexpr int kMaxPageLimit = 100;
|
||||
static constexpr std::size_t kMaxListPages = 20;
|
||||
// Streams.
|
||||
static constexpr std::size_t kMaxStreamEvents = 200000;
|
||||
// Request content.
|
||||
static constexpr std::size_t kMaxEmbeddingInputs = 2048;
|
||||
static constexpr std::size_t kMaxAudioFileBytes = 25u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxImageFileBytes = 50u * 1024u * 1024u;
|
||||
static constexpr std::size_t kMaxSpeechChars = 4096;
|
||||
static constexpr std::size_t kMaxImagePromptChars = 32000;
|
||||
// Content Safety (documented limits: 10k chars per text, 7.5 MiB image).
|
||||
static constexpr std::size_t kMaxSafetyTextChars = 10000;
|
||||
static constexpr std::size_t kMaxSafetyImageBytes = 7864320;
|
||||
static constexpr std::size_t kMaxBlocklistItems = 100;
|
||||
static constexpr std::size_t kMaxBlocklistItemChars = 128;
|
||||
static constexpr std::size_t kMaxShieldDocuments = 5;
|
||||
// Entra token exchange. The skew refreshes before expiry; the gap is the
|
||||
// floor between two exchanges after a failure (the source throws without
|
||||
// network traffic until it passes). Responses over the byte cap are refused.
|
||||
static constexpr long kTokenSkewSeconds = 300;
|
||||
static constexpr long kMinExchangeGapSeconds = 5;
|
||||
static constexpr long kMaxTokenLifetimeSeconds = 24 * 3600;
|
||||
static constexpr std::size_t kMaxTokenBytes = 16 * 1024;
|
||||
// Realtime client secrets.
|
||||
static constexpr int kMaxClientSecretSeconds = 7200;
|
||||
// Waiting on background work.
|
||||
static constexpr int kMinPollSeconds = 1;
|
||||
static constexpr int kMaxPollSeconds = 60;
|
||||
static constexpr int kMaxWaitSeconds = 600;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+38
@@ -0,0 +1,38 @@
|
||||
#include "vendors/azure/chat.h"
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
void ChatService::validate(const std::string& deployment, const Json& body) const {
|
||||
Check::deployment(deployment);
|
||||
Check::object("body", body);
|
||||
const Json* m = body.find("messages");
|
||||
Check::require(m && m->is_array() && !m->as_array().empty(), "messages is required");
|
||||
Check::max_count("messages", m->as_array().size(), llm::Caps::kMaxMessages);
|
||||
}
|
||||
|
||||
Json ChatService::create(const std::string& deployment, const Json& body) {
|
||||
validate(deployment, body);
|
||||
const Json* s = body.find("stream");
|
||||
Check::require(!s || (s->is_bool() && !s->as_bool()), "use stream() for stream=true");
|
||||
return call(Op::kCreateChatCompletion, {deployment, {}}, &body);
|
||||
}
|
||||
|
||||
void ChatService::stream(const std::string& deployment, const Json& body, const ChunkHandler& handler) {
|
||||
validate(deployment, body);
|
||||
call_stream(Op::kCreateChatCompletion, {deployment, {}}, body,
|
||||
[&](const llm::SseEvent& ev) { return handler(ev.data); });
|
||||
}
|
||||
|
||||
Json CompletionsService::create(const std::string& deployment, const Json& body) {
|
||||
Check::deployment(deployment);
|
||||
Check::object("body", body);
|
||||
Check::has_field(body, "prompt");
|
||||
return call(Op::kCreateCompletion, {deployment, {}}, &body);
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+42
@@ -0,0 +1,42 @@
|
||||
#pragma once
|
||||
|
||||
// Chat Completions and legacy Completions on a deployment.
|
||||
// v1 API POST /openai/v1/chat/completions (deployment = body model)
|
||||
// dated API POST /openai/deployments/{d}/chat/completions?api-version=...
|
||||
// The caller passes the deployment name once; the gateway puts it where the
|
||||
// configured API needs it.
|
||||
|
||||
#include <functional>
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class ChatService : public AzureService {
|
||||
public:
|
||||
using ChunkHandler = std::function<bool(const clients::body::Json& chunk)>; // false: stop
|
||||
|
||||
explicit ChatService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body: messages (required, 1..4096) plus any field of the spec. `stream`
|
||||
// must be absent or false here; use stream().
|
||||
clients::body::Json create(const std::string& deployment, const clients::body::Json& body);
|
||||
// Every `chat.completion.chunk` document reaches the handler; [DONE] ends.
|
||||
void stream(const std::string& deployment, const clients::body::Json& body,
|
||||
const ChunkHandler& handler);
|
||||
|
||||
private:
|
||||
void validate(const std::string& deployment, const clients::body::Json& body) const;
|
||||
};
|
||||
|
||||
class CompletionsService : public AzureService {
|
||||
public:
|
||||
explicit CompletionsService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
// body: prompt (required) plus any field of the spec.
|
||||
clients::body::Json create(const std::string& deployment, const clients::body::Json& body);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+83
@@ -0,0 +1,83 @@
|
||||
#include "vendors/azure/chat_model.h"
|
||||
|
||||
#include "openai/core/wire/response_events.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
namespace oc = openai::core;
|
||||
|
||||
AzureChatModel::AzureChatModel(std::shared_ptr<AzureClient> azure, ChatModelConfig config)
|
||||
: azure_(azure), config_(std::move(config)), chat_(azure), responses_(azure), foundry_(azure) {}
|
||||
|
||||
std::shared_ptr<AzureChatModel> AzureChatModel::from_env(ChatModelConfig config) {
|
||||
return std::make_shared<AzureChatModel>(AzureClient::from_env(), std::move(config));
|
||||
}
|
||||
|
||||
void AzureChatModel::check(const llm::ChatRequest& request) const {
|
||||
Check::deployment(request.model);
|
||||
Check::require(!request.messages.empty(), "messages is required");
|
||||
}
|
||||
|
||||
Json AzureChatModel::wire_body(const llm::ChatRequest& request, bool stream) const {
|
||||
check(request);
|
||||
if (config_.dialect == Dialect::kResponses)
|
||||
return oc::ResponsesWire::request(request, config_.wire).to_json(stream);
|
||||
oc::ChatCompletionRequest r = oc::ChatWire::request(request, config_.wire);
|
||||
if (stream) r.include_usage = true;
|
||||
Json body = r.to_json(stream);
|
||||
if (config_.dialect == Dialect::kFoundry) {
|
||||
// The model inference API names the cap max_tokens.
|
||||
Json::Object& o = body.as_object();
|
||||
if (auto it = o.find("max_completion_tokens"); it != o.end()) {
|
||||
o["max_tokens"] = it->second;
|
||||
o.erase("max_completion_tokens");
|
||||
}
|
||||
if (!stream) o.erase("stream");
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
llm::ChatResponse AzureChatModel::generate(const llm::ChatRequest& request) {
|
||||
Json body = wire_body(request, false);
|
||||
switch (config_.dialect) {
|
||||
case Dialect::kResponses:
|
||||
return oc::ResponsesWire::response(responses_.create(body));
|
||||
case Dialect::kFoundry:
|
||||
return oc::ChatWire::response(foundry_.chat(body, config_.extra_parameters));
|
||||
case Dialect::kChatCompletions:
|
||||
break;
|
||||
}
|
||||
return oc::ChatWire::response(chat_.create(request.model, body));
|
||||
}
|
||||
|
||||
void AzureChatModel::stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) {
|
||||
Json body = wire_body(request, true);
|
||||
if (config_.dialect == Dialect::kResponses) {
|
||||
oc::ResponsesChunkDecoder decoder;
|
||||
responses_.stream(body, [&](const llm::SseEvent& ev) {
|
||||
oc::ResponseEvent e = oc::ResponseEvent::from_sse(ev);
|
||||
llm::ChatChunk chunk;
|
||||
if (!decoder.decode(e, chunk)) return true;
|
||||
return on_chunk(chunk);
|
||||
});
|
||||
return;
|
||||
}
|
||||
oc::ChatChunkDecoder decoder;
|
||||
auto handle = [&](const Json& doc) {
|
||||
std::vector<llm::ChatChunk> chunks;
|
||||
decoder.decode(doc, chunks);
|
||||
for (const llm::ChatChunk& c : chunks)
|
||||
if (!on_chunk(c)) return false;
|
||||
return true;
|
||||
};
|
||||
if (config_.dialect == Dialect::kFoundry) {
|
||||
foundry_.chat_stream(body, handle, config_.extra_parameters);
|
||||
return;
|
||||
}
|
||||
chat_.stream(request.model, body, handle);
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+57
@@ -0,0 +1,57 @@
|
||||
#pragma once
|
||||
|
||||
// AzureChatModel: llm::ChatModel over a deployment of an Azure resource.
|
||||
// The request's `model` is the deployment name. Three dialects:
|
||||
// kChatCompletions /chat/completions (default; the widest model support)
|
||||
// kResponses /responses
|
||||
// kFoundry Azure AI model inference /models/chat/completions
|
||||
// The neutral <-> wire mapping is the OpenAI one (openai::core::ChatWire /
|
||||
// ResponsesWire): Azure OpenAI speaks the same shapes.
|
||||
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "llm/chat_types.h"
|
||||
#include "openai/core/wire/chat_wire.h"
|
||||
#include "vendors/azure/chat.h"
|
||||
#include "vendors/azure/foundry.h"
|
||||
#include "vendors/azure/responses.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
enum class Dialect { kChatCompletions, kResponses, kFoundry };
|
||||
|
||||
struct ChatModelConfig {
|
||||
Dialect dialect = Dialect::kChatCompletions;
|
||||
openai::core::WireOptions wire;
|
||||
std::string extra_parameters; // Foundry dialect: error, drop, pass-through
|
||||
};
|
||||
|
||||
class AzureChatModel : public llm::ChatModel {
|
||||
public:
|
||||
explicit AzureChatModel(std::shared_ptr<AzureClient> azure, ChatModelConfig config = {});
|
||||
// Settings from the environment; not configured without endpoint and credential.
|
||||
static std::shared_ptr<AzureChatModel> from_env(ChatModelConfig config = {});
|
||||
|
||||
std::string vendor() const override { return "azure"; }
|
||||
llm::ChatResponse generate(const llm::ChatRequest& request) override;
|
||||
void stream(const llm::ChatRequest& request,
|
||||
const std::function<bool(const llm::ChatChunk&)>& on_chunk) override;
|
||||
|
||||
Dialect dialect() const { return config_.dialect; }
|
||||
bool configured() const { return azure_->configured(); }
|
||||
// The wire body generate() would send for a request (no traffic).
|
||||
clients::body::Json wire_body(const llm::ChatRequest& request, bool stream) const;
|
||||
|
||||
private:
|
||||
void check(const llm::ChatRequest& request) const;
|
||||
|
||||
std::shared_ptr<AzureClient> azure_;
|
||||
ChatModelConfig config_;
|
||||
ChatService chat_;
|
||||
ResponsesService responses_;
|
||||
FoundryService foundry_;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+63
@@ -0,0 +1,63 @@
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
#include "llm/api_client.h"
|
||||
#include "llm/error.h"
|
||||
#include "vendors/azure/caps.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
void Check::fail(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
|
||||
void Check::require(bool ok, const std::string& why) {
|
||||
if (!ok) fail(why);
|
||||
}
|
||||
void Check::non_empty(const std::string& what, const std::string& value) {
|
||||
if (value.empty()) fail(what + " is required");
|
||||
}
|
||||
void Check::max_chars(const std::string& what, const std::string& value, std::size_t cap) {
|
||||
if (value.size() > cap) fail(what + " is over " + std::to_string(cap) + " characters");
|
||||
}
|
||||
void Check::max_count(const std::string& what, std::size_t count, std::size_t cap) {
|
||||
if (count > cap) fail(what + " has more than " + std::to_string(cap) + " entries");
|
||||
}
|
||||
void Check::range(const std::string& what, double value, double lo, double hi) {
|
||||
if (value < lo || value > hi)
|
||||
fail(what + " must be " + std::to_string(static_cast<long>(lo)) + ".." + std::to_string(static_cast<long>(hi)));
|
||||
}
|
||||
void Check::in_set(const std::string& what, const std::string& value,
|
||||
const std::vector<std::string>& allowed) {
|
||||
for (const std::string& a : allowed)
|
||||
if (a == value) return;
|
||||
std::string list;
|
||||
for (const std::string& a : allowed) list += (list.empty() ? "" : ", ") + a;
|
||||
fail(what + " must be one of: " + list);
|
||||
}
|
||||
void Check::deployment(const std::string& value) {
|
||||
if (value.empty() || value.size() > AzureCaps::kMaxDeploymentChars || value == "." || value == "..")
|
||||
fail("deployment must be 1.." + std::to_string(AzureCaps::kMaxDeploymentChars) + " characters");
|
||||
for (char c : value)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
|
||||
c == '_' || c == '.'))
|
||||
fail("deployment has invalid characters");
|
||||
}
|
||||
void Check::id(const std::string& what, const std::string& value) {
|
||||
if (!llm::Service::valid_id(value)) fail("bad " + what + " id");
|
||||
}
|
||||
void Check::object(const std::string& what, const Json& value) {
|
||||
if (!value.is_object()) fail(what + " must be a JSON object");
|
||||
}
|
||||
void Check::has_string(const Json& body, const std::string& key) {
|
||||
const Json* v = body.is_object() ? body.find(key) : nullptr;
|
||||
if (!v || !v->is_string() || v->as_string().empty()) fail(key + " is required");
|
||||
}
|
||||
void Check::has_field(const Json& body, const std::string& key) {
|
||||
const Json* v = body.is_object() ? body.find(key) : nullptr;
|
||||
if (!v || v->is_null()) fail(key + " is required");
|
||||
}
|
||||
void Check::no_control(const std::string& what, const std::string& value) {
|
||||
for (char c : value)
|
||||
if (static_cast<unsigned char>(c) < 0x20) fail(what + " has control characters");
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+34
@@ -0,0 +1,34 @@
|
||||
#pragma once
|
||||
|
||||
// Request validation helpers. Every failure is llm::ApiError::invalid
|
||||
// ("azure", 400 INVALID_ARGUMENT), thrown before any network traffic.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/json.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class Check {
|
||||
public:
|
||||
[[noreturn]] static void fail(const std::string& why);
|
||||
static void require(bool ok, const std::string& why);
|
||||
static void non_empty(const std::string& what, const std::string& value);
|
||||
static void max_chars(const std::string& what, const std::string& value, std::size_t cap);
|
||||
static void max_count(const std::string& what, std::size_t count, std::size_t cap);
|
||||
static void range(const std::string& what, double value, double lo, double hi);
|
||||
static void in_set(const std::string& what, const std::string& value,
|
||||
const std::vector<std::string>& allowed);
|
||||
// A deployment name: 1..64 of [A-Za-z0-9._-], not "." or "..".
|
||||
static void deployment(const std::string& value);
|
||||
// One URL path segment id (llm::Service::valid_id).
|
||||
static void id(const std::string& what, const std::string& value);
|
||||
static void object(const std::string& what, const clients::body::Json& value);
|
||||
// The body object has the string field `key` (non-empty).
|
||||
static void has_string(const clients::body::Json& body, const std::string& key);
|
||||
static void has_field(const clients::body::Json& body, const std::string& key);
|
||||
static void no_control(const std::string& what, const std::string& value);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+67
@@ -0,0 +1,67 @@
|
||||
#include "vendors/azure/client.h"
|
||||
|
||||
#include "clients/transports/http_transport.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
namespace {
|
||||
constexpr const char* kPlaceholderKey = "entra-managed"; // never sent: EntraTransport overwrites it
|
||||
|
||||
llm::ApiClientConfig base(const std::string& endpoint, const char* key_env, bool entra,
|
||||
const char* key_header) {
|
||||
llm::ApiClientConfig c;
|
||||
c.vendor = Settings::kVendor;
|
||||
// An unset endpoint cannot be parsed as a URL; the empty key stops every call first.
|
||||
c.base_url = endpoint.empty() ? "https://unset.invalid" : endpoint;
|
||||
c.key_env = endpoint.empty() ? Settings::kEndpointEnv : key_env;
|
||||
if (entra) {
|
||||
c.auth = llm::AuthStyle::kBearer;
|
||||
} else {
|
||||
c.auth = llm::AuthStyle::kHeader;
|
||||
c.auth_header = key_header;
|
||||
}
|
||||
return c;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::shared_ptr<AzureClient> AzureClient::from_env() {
|
||||
return create(Settings::from_env(), [](Channel) -> std::unique_ptr<clients::Transport> {
|
||||
return std::make_unique<clients::transports::HttpTransport>();
|
||||
});
|
||||
}
|
||||
|
||||
std::shared_ptr<AzureClient> AzureClient::create(Settings settings, const TransportFactory& factory,
|
||||
EntraTokenSource::Clock clock) {
|
||||
settings.validate();
|
||||
std::shared_ptr<AzureClient> self(new AzureClient());
|
||||
self->settings_ = std::move(settings);
|
||||
const Settings& s = self->settings_;
|
||||
bool entra = s.auth() == AuthKind::kEntra;
|
||||
if (entra) self->tokens_ = std::make_shared<EntraTokenSource>(s.entra, factory(Channel::kToken), clock);
|
||||
|
||||
auto wire = [&](Channel ch, const std::string& endpoint, const char* key_env, const char* header,
|
||||
const std::string& key) {
|
||||
std::unique_ptr<clients::Transport> t = factory(ch);
|
||||
if (entra) t = std::make_unique<EntraTransport>(std::move(t), self->tokens_);
|
||||
std::string k = entra ? (endpoint.empty() ? "" : kPlaceholderKey) : (endpoint.empty() ? "" : key);
|
||||
return std::make_shared<llm::ApiClient>(std::move(t), base(endpoint, key_env, entra, header), k);
|
||||
};
|
||||
self->openai_ = wire(Channel::kOpenAi, s.endpoint, Settings::kKeyEnv, "api-key", s.api_key);
|
||||
self->foundry_ = wire(Channel::kFoundry, s.foundry_endpoint, Settings::kKeyEnv, "api-key", s.api_key);
|
||||
self->safety_ = wire(Channel::kSafety, s.safety_endpoint, Settings::kSafetyKeyEnv,
|
||||
"Ocp-Apim-Subscription-Key", s.safety_key);
|
||||
return self;
|
||||
}
|
||||
|
||||
std::shared_ptr<llm::ApiClient> AzureClient::api(Surface surface) const {
|
||||
switch (surface) {
|
||||
case Surface::kFoundry: return foundry_;
|
||||
case Surface::kSafety: return safety_;
|
||||
case Surface::kOpenAi: break;
|
||||
}
|
||||
return openai_;
|
||||
}
|
||||
|
||||
bool AzureClient::configured(Surface surface) const { return api(surface)->configured(); }
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+51
@@ -0,0 +1,51 @@
|
||||
#pragma once
|
||||
|
||||
// AzureClient: one llm::ApiClient per surface (OpenAI resource, Foundry model
|
||||
// inference, Content Safety) plus the shared credential. Key mode sends the
|
||||
// api-key header (Ocp-Apim-Subscription-Key for Content Safety); Entra mode
|
||||
// puts a managed bearer token on every request through EntraTransport.
|
||||
// Not configured (no endpoint, or neither key nor complete Entra client
|
||||
// credentials): every call throws NotConfiguredError before any traffic.
|
||||
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "llm/api_client.h"
|
||||
#include "vendors/azure/calls.h"
|
||||
#include "vendors/azure/entra.h"
|
||||
#include "vendors/azure/settings.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
// Which wire a transport serves. kToken is the Entra token exchange.
|
||||
enum class Channel { kOpenAi, kFoundry, kSafety, kToken };
|
||||
|
||||
class AzureClient {
|
||||
public:
|
||||
using TransportFactory = std::function<std::unique_ptr<clients::Transport>(Channel)>;
|
||||
|
||||
// Real HTTPS transports, settings from the environment.
|
||||
static std::shared_ptr<AzureClient> from_env();
|
||||
// Any transports (tests).
|
||||
static std::shared_ptr<AzureClient> create(Settings settings, const TransportFactory& factory,
|
||||
EntraTokenSource::Clock clock = nullptr);
|
||||
|
||||
const Settings& settings() const { return settings_; }
|
||||
bool configured() const { return settings_.configured(); }
|
||||
AuthKind auth() const { return settings_.auth(); }
|
||||
std::shared_ptr<llm::ApiClient> api(Surface surface) const;
|
||||
std::shared_ptr<EntraTokenSource> tokens() const { return tokens_; }
|
||||
// The configured credential for a surface is present (Safety may have its own key).
|
||||
bool configured(Surface surface) const;
|
||||
|
||||
private:
|
||||
AzureClient() = default;
|
||||
Settings settings_;
|
||||
std::shared_ptr<EntraTokenSource> tokens_;
|
||||
std::shared_ptr<llm::ApiClient> openai_;
|
||||
std::shared_ptr<llm::ApiClient> foundry_;
|
||||
std::shared_ptr<llm::ApiClient> safety_;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+53
@@ -0,0 +1,53 @@
|
||||
#include "vendors/azure/embeddings.h"
|
||||
|
||||
#include "vendors/azure/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
Json EmbeddingsService::create(const std::string& deployment, const Json& body) {
|
||||
Check::deployment(deployment);
|
||||
Check::object("body", body);
|
||||
const Json* in = body.find("input");
|
||||
Check::require(in && !in->is_null(), "input is required");
|
||||
if (in->is_array()) {
|
||||
Check::require(!in->as_array().empty(), "input must not be empty");
|
||||
Check::max_count("input", in->as_array().size(), AzureCaps::kMaxEmbeddingInputs);
|
||||
} else {
|
||||
Check::require(in->is_string() && !in->as_string().empty(), "input must be a non-empty string or an array");
|
||||
}
|
||||
return call(Op::kCreateEmbedding, {deployment, {}}, &body);
|
||||
}
|
||||
|
||||
std::vector<std::vector<double>> EmbeddingsService::embed(const std::string& deployment,
|
||||
const std::vector<std::string>& texts,
|
||||
int dimensions) {
|
||||
Check::require(!texts.empty(), "texts must not be empty");
|
||||
Json body = Json::object();
|
||||
Json arr = Json::array();
|
||||
for (const std::string& t : texts) {
|
||||
Check::non_empty("text", t);
|
||||
arr.push(Json(t));
|
||||
}
|
||||
body.set("input", arr);
|
||||
body.set("encoding_format", Json("float"));
|
||||
if (dimensions > 0) body.set("dimensions", Json(dimensions));
|
||||
Json doc = create(deployment, body);
|
||||
std::vector<std::vector<double>> out;
|
||||
const Json* data = doc.find("data");
|
||||
if (!data || !data->is_array())
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "embeddings answer has no data array");
|
||||
for (const Json& item : data->as_array()) {
|
||||
std::vector<double> v;
|
||||
const Json* e = item.find("embedding");
|
||||
if (!e || !e->is_array())
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "embedding item has no vector");
|
||||
for (const Json& x : e->as_array()) v.push_back(x.as_number());
|
||||
out.push_back(std::move(v));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+25
@@ -0,0 +1,25 @@
|
||||
#pragma once
|
||||
|
||||
// Embeddings on a deployment (text-embedding-3-*, ada-002).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class EmbeddingsService : public AzureService {
|
||||
public:
|
||||
explicit EmbeddingsService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body.input: a string or an array of 1..2048 strings / token arrays.
|
||||
clients::body::Json create(const std::string& deployment, const clients::body::Json& body);
|
||||
// Convenience: texts in, body built, vectors out (encoding_format float).
|
||||
std::vector<std::vector<double>> embed(const std::string& deployment,
|
||||
const std::vector<std::string>& texts,
|
||||
int dimensions = 0);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+146
@@ -0,0 +1,146 @@
|
||||
#include "vendors/azure/entra.h"
|
||||
|
||||
#include <chrono>
|
||||
|
||||
#include "clients/body/form.h"
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/error.h"
|
||||
#include "vendors/azure/caps.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
clients::ClientConfig make_config() {
|
||||
clients::ClientConfig c;
|
||||
c.persist_cookies = false;
|
||||
return c;
|
||||
}
|
||||
|
||||
std::int64_t system_now() {
|
||||
return std::chrono::duration_cast<std::chrono::seconds>(
|
||||
std::chrono::system_clock::now().time_since_epoch())
|
||||
.count();
|
||||
}
|
||||
|
||||
std::string first_line(const std::string& s, std::size_t cap) {
|
||||
std::string out = s.substr(0, s.find_first_of("\r\n"));
|
||||
if (out.size() > cap) out.resize(cap);
|
||||
return out;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
EntraTokenSource::EntraTokenSource(EntraConfig config, std::unique_ptr<clients::Transport> transport,
|
||||
Clock clock)
|
||||
: config_(std::move(config)),
|
||||
client_(std::move(transport), make_config()),
|
||||
clock_(clock ? std::move(clock) : Clock(system_now)) {
|
||||
config_.validate();
|
||||
}
|
||||
|
||||
std::string EntraTokenSource::request_body() const {
|
||||
clients::body::UrlEncodedForm f;
|
||||
f.add("client_id", config_.client_id);
|
||||
f.add("client_secret", config_.client_secret);
|
||||
f.add("scope", config_.scope);
|
||||
f.add("grant_type", "client_credentials");
|
||||
return f.encode();
|
||||
}
|
||||
|
||||
EntraTokenSource::Parsed EntraTokenSource::parse(int status, const std::string& body) {
|
||||
if (body.size() > AzureCaps::kMaxTokenBytes)
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "token response over the byte cap");
|
||||
Json doc;
|
||||
try {
|
||||
doc = Json::parse(body);
|
||||
} catch (const std::exception&) {
|
||||
throw llm::ApiError("azure", status >= 400 ? status : 502, "BAD_RESPONSE",
|
||||
"token endpoint answered with a non-JSON body");
|
||||
}
|
||||
if (status < 200 || status >= 300 || !doc.is_object() || doc.has("error")) {
|
||||
std::string code = "token_error";
|
||||
std::string desc = "HTTP " + std::to_string(status);
|
||||
std::string trace;
|
||||
if (doc.is_object()) {
|
||||
if (const Json* e = doc.find("error"); e && e->is_string()) code = e->as_string();
|
||||
if (const Json* d = doc.find("error_description"); d && d->is_string())
|
||||
desc = first_line(d->as_string(), 300);
|
||||
if (const Json* t = doc.find("trace_id"); t && t->is_string()) trace = t->as_string();
|
||||
}
|
||||
throw llm::ApiError("azure", status >= 400 ? status : 502, code, desc, trace);
|
||||
}
|
||||
Parsed out;
|
||||
const Json* t = doc.find("access_token");
|
||||
if (!t || !t->is_string() || t->as_string().empty())
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "token response has no access_token");
|
||||
out.access_token = t->as_string();
|
||||
const Json* ex = doc.find("expires_in");
|
||||
// The v2.0 endpoint documents expires_in as a number; some stacks send a string.
|
||||
if (ex && ex->is_number()) out.expires_in = ex->as_int();
|
||||
else if (ex && ex->is_string()) out.expires_in = std::atol(ex->as_string().c_str());
|
||||
if (out.expires_in <= 0 || out.expires_in > AzureCaps::kMaxTokenLifetimeSeconds)
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "token response has an unusable expires_in");
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string EntraTokenSource::token() {
|
||||
std::lock_guard<std::mutex> lock(mutex_);
|
||||
std::int64_t now = clock_();
|
||||
if (!token_.empty() && now < expires_at_ - AzureCaps::kTokenSkewSeconds) return token_;
|
||||
if (last_failure_at_ != 0 && now - last_failure_at_ < AzureCaps::kMinExchangeGapSeconds)
|
||||
throw llm::ApiError("azure", 503, "TOKEN_BACKOFF",
|
||||
"token exchange failed " + std::to_string(now - last_failure_at_) +
|
||||
"s ago (" + last_failure_ + "), not retrying yet");
|
||||
clients::ClientRequest req(clients::Method::kPost, clients::Url::parse(config_.token_url()));
|
||||
req.form(clients::body::UrlEncodedForm::decode(request_body()));
|
||||
++exchanges_;
|
||||
try {
|
||||
clients::ClientResponse resp = client_.send_blocking(std::move(req));
|
||||
Parsed p = parse(resp.status(), resp.body());
|
||||
token_ = p.access_token;
|
||||
expires_at_ = now + p.expires_in;
|
||||
last_failure_at_ = 0;
|
||||
return token_;
|
||||
} catch (const llm::ApiError& e) {
|
||||
last_failure_at_ = now;
|
||||
last_failure_ = e.type();
|
||||
token_.clear();
|
||||
throw;
|
||||
} catch (const std::exception& e) {
|
||||
last_failure_at_ = now;
|
||||
last_failure_ = "UNAVAILABLE";
|
||||
token_.clear();
|
||||
throw llm::ApiError("azure", 502, "UNAVAILABLE", e.what());
|
||||
}
|
||||
}
|
||||
|
||||
std::size_t EntraTokenSource::exchanges() const {
|
||||
std::lock_guard<std::mutex> lock(mutex_);
|
||||
return exchanges_;
|
||||
}
|
||||
|
||||
std::int64_t EntraTokenSource::expires_at() const {
|
||||
std::lock_guard<std::mutex> lock(mutex_);
|
||||
return expires_at_;
|
||||
}
|
||||
|
||||
EntraTransport::EntraTransport(std::unique_ptr<clients::Transport> inner,
|
||||
std::shared_ptr<EntraTokenSource> source)
|
||||
: inner_(std::move(inner)), source_(std::move(source)) {}
|
||||
|
||||
clients::Task<clients::ClientResponse> EntraTransport::send(const clients::ClientRequest& request) {
|
||||
clients::ClientRequest copy = request;
|
||||
copy.bearer(source_->token());
|
||||
co_return co_await inner_->send(copy);
|
||||
}
|
||||
|
||||
clients::Task<void> EntraTransport::send_stream(const clients::ClientRequest& request,
|
||||
clients::body::ChunkSink sink) {
|
||||
clients::ClientRequest copy = request;
|
||||
copy.bearer(source_->token());
|
||||
co_await inner_->send_stream(copy, std::move(sink));
|
||||
co_return;
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+76
@@ -0,0 +1,76 @@
|
||||
#pragma once
|
||||
|
||||
// Microsoft Entra ID client-credentials flow (OAuth2, RFC 6749 section 4.4):
|
||||
// POST <authority>/<tenant>/oauth2/v2.0/token with client_id, client_secret,
|
||||
// scope and grant_type=client_credentials. EntraTokenSource caches the token
|
||||
// and refreshes it AzureCaps::kTokenSkewSeconds before expiry; after a failed
|
||||
// exchange it refuses to call again for AzureCaps::kMinExchangeGapSeconds
|
||||
// (a floor: a broken secret must not hammer the tenant). EntraTransport puts
|
||||
// the current token on every request that passes through a transport.
|
||||
|
||||
#include <cstdint>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <string>
|
||||
|
||||
#include "clients/client.h"
|
||||
#include "clients/transport.h"
|
||||
#include "vendors/azure/settings.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class EntraTokenSource {
|
||||
public:
|
||||
using Clock = std::function<std::int64_t()>; // unix seconds
|
||||
|
||||
EntraTokenSource(EntraConfig config, std::unique_ptr<clients::Transport> transport,
|
||||
Clock clock = nullptr);
|
||||
|
||||
// A valid access token, exchanging when none is cached or it is about to
|
||||
// expire. Throws llm::ApiError (type = the AAD error code, e.g.
|
||||
// "invalid_client") on a failed exchange. Thread safe.
|
||||
std::string token();
|
||||
// The form body of the exchange request (contains the secret: for tests of
|
||||
// the wire shape only, never log it).
|
||||
std::string request_body() const;
|
||||
const EntraConfig& config() const { return config_; }
|
||||
std::size_t exchanges() const;
|
||||
std::int64_t expires_at() const;
|
||||
|
||||
// Parses a token endpoint answer (documented success and error bodies).
|
||||
struct Parsed {
|
||||
std::string access_token;
|
||||
long expires_in = 0;
|
||||
};
|
||||
static Parsed parse(int status, const std::string& body);
|
||||
|
||||
private:
|
||||
EntraConfig config_;
|
||||
clients::Client client_;
|
||||
Clock clock_;
|
||||
mutable std::mutex mutex_;
|
||||
std::string token_;
|
||||
std::int64_t expires_at_ = 0;
|
||||
std::int64_t last_failure_at_ = 0;
|
||||
std::string last_failure_;
|
||||
std::size_t exchanges_ = 0;
|
||||
};
|
||||
|
||||
// Transport decorator: sets `Authorization: Bearer <token>` from the source on
|
||||
// every request, whatever the caller put there.
|
||||
class EntraTransport : public clients::Transport {
|
||||
public:
|
||||
EntraTransport(std::unique_ptr<clients::Transport> inner, std::shared_ptr<EntraTokenSource> source);
|
||||
const char* name() const override { return "entra"; }
|
||||
clients::Task<clients::ClientResponse> send(const clients::ClientRequest& request) override;
|
||||
clients::Task<void> send_stream(const clients::ClientRequest& request,
|
||||
clients::body::ChunkSink sink) override;
|
||||
void close() override { inner_->close(); }
|
||||
|
||||
private:
|
||||
std::unique_ptr<clients::Transport> inner_;
|
||||
std::shared_ptr<EntraTokenSource> source_;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+87
@@ -0,0 +1,87 @@
|
||||
#include "vendors/azure/files.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
Json FilesService::upload(const UploadFile& file, const std::string& purpose) {
|
||||
Check::non_empty("filename", file.filename);
|
||||
Check::no_control("filename", file.filename);
|
||||
Check::non_empty("purpose", purpose);
|
||||
Check::require(!file.data.empty(), "file is empty");
|
||||
Check::require(file.data.size() <= kMaxFileBytes, "file over 512 MiB");
|
||||
clients::body::MultipartBody form;
|
||||
form.add_text("purpose", purpose);
|
||||
form.add_file("file", file.filename, file.content_type, file.data);
|
||||
return call_multipart(Op::kCreateFile, {}, form);
|
||||
}
|
||||
|
||||
Json FilesService::get(const std::string& id) {
|
||||
Check::id("file", id);
|
||||
return call(Op::kRetrieveFile, {"", {id}});
|
||||
}
|
||||
|
||||
Json FilesService::list(const std::string& purpose, const ListParams& params) {
|
||||
params.validate();
|
||||
Query q;
|
||||
q.add("purpose", purpose);
|
||||
params.apply(q);
|
||||
return call(Op::kListFiles, {}, nullptr, &q);
|
||||
}
|
||||
|
||||
llm::RawResponse FilesService::content(const std::string& id) {
|
||||
Check::id("file", id);
|
||||
return call_raw(Op::kDownloadFile, {"", {id}});
|
||||
}
|
||||
|
||||
Json FilesService::remove(const std::string& id) {
|
||||
Check::id("file", id);
|
||||
return call(Op::kDeleteFile, {"", {id}});
|
||||
}
|
||||
|
||||
Json FilesService::import_from_url(const Json& body) {
|
||||
Check::has_string(body, "purpose");
|
||||
Check::has_string(body, "filename");
|
||||
Check::has_string(body, "content_url");
|
||||
return call(Op::kFilesImport, {}, &body);
|
||||
}
|
||||
|
||||
Json FilesService::start_upload(const Json& body) {
|
||||
Check::has_string(body, "purpose");
|
||||
Check::has_string(body, "filename");
|
||||
Check::has_field(body, "bytes");
|
||||
return call(Op::kUploadFileStart, {}, &body);
|
||||
}
|
||||
|
||||
Json FilesService::add_part(const std::string& upload_id, const UploadFile& part) {
|
||||
Check::id("upload", upload_id);
|
||||
Check::require(!part.data.empty(), "part is empty");
|
||||
Check::require(part.data.size() <= 64u * 1024u * 1024u, "part over 64 MiB");
|
||||
clients::body::MultipartBody form;
|
||||
form.add_file("data", part.filename.empty() ? "part" : part.filename, part.content_type, part.data);
|
||||
return call_multipart(Op::kUploadFilePart, {"", {upload_id}}, form);
|
||||
}
|
||||
|
||||
Json FilesService::complete_upload(const std::string& upload_id, const std::vector<std::string>& part_ids,
|
||||
const std::string& md5) {
|
||||
Check::id("upload", upload_id);
|
||||
Check::require(!part_ids.empty() && part_ids.size() <= 10000, "part_ids must hold 1..10000 ids");
|
||||
Json body = Json::object();
|
||||
Json ids = Json::array();
|
||||
for (const std::string& p : part_ids) {
|
||||
Check::id("part", p);
|
||||
ids.push(Json(p));
|
||||
}
|
||||
body.set("part_ids", ids);
|
||||
if (!md5.empty()) body.set("md5", Json(md5));
|
||||
return call(Op::kUploadFileComplete, {"", {upload_id}}, &body);
|
||||
}
|
||||
|
||||
Json FilesService::cancel_upload(const std::string& upload_id) {
|
||||
Check::id("upload", upload_id);
|
||||
return call(Op::kUploadFileCancel, {"", {upload_id}});
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+39
@@ -0,0 +1,39 @@
|
||||
#pragma once
|
||||
|
||||
// Files and multipart Uploads. Upload purposes: fine-tune, assistants, batch,
|
||||
// user_data, vision, evals (the spec lists the accepted ones per API).
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
#include "vendors/azure/upload_file.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class FilesService : public AzureService {
|
||||
public:
|
||||
static constexpr std::size_t kMaxFileBytes = 512u * 1024u * 1024u; // llm::Caps::kMaxUploadBytes
|
||||
|
||||
explicit FilesService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// Uploads one file (single request, capped at 512 MiB). purpose is required.
|
||||
clients::body::Json upload(const UploadFile& file, const std::string& purpose);
|
||||
clients::body::Json get(const std::string& id);
|
||||
// GET /files; the dated API takes `purpose`, the v1 API also after/limit/order.
|
||||
clients::body::Json list(const std::string& purpose = "", const ListParams& params = {});
|
||||
llm::RawResponse content(const std::string& id);
|
||||
clients::body::Json remove(const std::string& id); // destructive
|
||||
// Dated API only: import from a URL (body: purpose, filename, content_url).
|
||||
clients::body::Json import_from_url(const clients::body::Json& body);
|
||||
|
||||
// Multipart uploads of large files (dated API).
|
||||
clients::body::Json start_upload(const clients::body::Json& body);
|
||||
clients::body::Json add_part(const std::string& upload_id, const UploadFile& part);
|
||||
clients::body::Json complete_upload(const std::string& upload_id, const std::vector<std::string>& part_ids,
|
||||
const std::string& md5 = "");
|
||||
clients::body::Json cancel_upload(const std::string& upload_id); // destructive
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+89
@@ -0,0 +1,89 @@
|
||||
#include "vendors/azure/fine_tuning.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
Query page(const ListParams& p) {
|
||||
p.validate();
|
||||
Query q;
|
||||
p.apply(q);
|
||||
return q;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json FineTuningService::create(const Json& body) {
|
||||
Check::has_string(body, "model");
|
||||
Check::has_string(body, "training_file");
|
||||
return call(Op::kCreateFineTuningJob, {}, &body);
|
||||
}
|
||||
Json FineTuningService::get(const std::string& job) {
|
||||
Check::id("job", job);
|
||||
return call(Op::kRetrieveFineTuningJob, {"", {job}});
|
||||
}
|
||||
Json FineTuningService::list(const ListParams& params) {
|
||||
Query q = page(params);
|
||||
return call(Op::kListPaginatedFineTuningJobs, {}, nullptr, &q);
|
||||
}
|
||||
Listing FineTuningService::list_all(const ListParams& params, std::size_t max_pages) {
|
||||
return walk(Op::kListPaginatedFineTuningJobs, {}, params, Query(), max_pages);
|
||||
}
|
||||
Json FineTuningService::events(const std::string& job, const ListParams& params) {
|
||||
Check::id("job", job);
|
||||
Query q = page(params);
|
||||
return call(Op::kListFineTuningEvents, {"", {job}}, nullptr, &q);
|
||||
}
|
||||
Json FineTuningService::checkpoints(const std::string& job, const ListParams& params) {
|
||||
Check::id("job", job);
|
||||
Query q = page(params);
|
||||
return call(Op::kListFineTuningJobCheckpoints, {"", {job}}, nullptr, &q);
|
||||
}
|
||||
Json FineTuningService::cancel(const std::string& job) {
|
||||
Check::id("job", job);
|
||||
return call(Op::kCancelFineTuningJob, {"", {job}});
|
||||
}
|
||||
Json FineTuningService::pause(const std::string& job) {
|
||||
Check::id("job", job);
|
||||
return call(Op::kPauseFineTuningJob, {"", {job}});
|
||||
}
|
||||
Json FineTuningService::resume(const std::string& job) {
|
||||
Check::id("job", job);
|
||||
return call(Op::kResumeFineTuningJob, {"", {job}});
|
||||
}
|
||||
Json FineTuningService::remove(const std::string& job) {
|
||||
Check::id("job", job);
|
||||
return call(Op::kFineTuningDelete, {"", {job}});
|
||||
}
|
||||
Json FineTuningService::copy_checkpoint(const std::string& job, const std::string& checkpoint,
|
||||
const Json& body) {
|
||||
Check::id("job", job);
|
||||
Check::id("checkpoint", checkpoint);
|
||||
Check::has_string(body, "destinationResourceId");
|
||||
Check::has_string(body, "region");
|
||||
return call(Op::kFineTuningCopyCheckpoint, {"", {job, checkpoint}}, &body);
|
||||
}
|
||||
Json FineTuningService::get_checkpoint_copy(const std::string& job, const std::string& checkpoint) {
|
||||
Check::id("job", job);
|
||||
Check::id("checkpoint", checkpoint);
|
||||
return call(Op::kFineTuningGetCheckpoint, {"", {job, checkpoint}});
|
||||
}
|
||||
Json FineTuningService::list_permissions(const std::string& model, const ListParams& params) {
|
||||
Check::id("checkpoint model", model);
|
||||
Query q = page(params);
|
||||
return call(Op::kListFineTuningCheckpointPermissions, {"", {model}}, nullptr, &q);
|
||||
}
|
||||
Json FineTuningService::create_permission(const std::string& model, const Json& body) {
|
||||
Check::id("checkpoint model", model);
|
||||
Check::has_field(body, "project_ids");
|
||||
return call(Op::kCreateFineTuningCheckpointPermission, {"", {model}}, &body);
|
||||
}
|
||||
Json FineTuningService::remove_permission(const std::string& model, const std::string& permission) {
|
||||
Check::id("checkpoint model", model);
|
||||
Check::id("permission", permission);
|
||||
return call(Op::kDeleteFineTuningCheckpointPermission, {"", {model, permission}});
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+40
@@ -0,0 +1,40 @@
|
||||
#pragma once
|
||||
|
||||
// Fine-tuning jobs, events, checkpoints, checkpoint permissions and the
|
||||
// cross-resource checkpoint copy. Creating a job costs money.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class FineTuningService : public AzureService {
|
||||
public:
|
||||
explicit FineTuningService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body: model and training_file (required) plus any spec field.
|
||||
clients::body::Json create(const clients::body::Json& body);
|
||||
clients::body::Json get(const std::string& job);
|
||||
clients::body::Json list(const ListParams& params = {});
|
||||
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
|
||||
clients::body::Json events(const std::string& job, const ListParams& params = {});
|
||||
clients::body::Json checkpoints(const std::string& job, const ListParams& params = {});
|
||||
clients::body::Json cancel(const std::string& job); // destructive
|
||||
clients::body::Json pause(const std::string& job);
|
||||
clients::body::Json resume(const std::string& job);
|
||||
// Dated API only: DELETE a job record.
|
||||
clients::body::Json remove(const std::string& job); // destructive
|
||||
// v1 API: copy a checkpoint to another resource (body: destinationResourceId, region).
|
||||
clients::body::Json copy_checkpoint(const std::string& job, const std::string& checkpoint,
|
||||
const clients::body::Json& body);
|
||||
clients::body::Json get_checkpoint_copy(const std::string& job, const std::string& checkpoint);
|
||||
// Checkpoint permissions (v1 API).
|
||||
clients::body::Json list_permissions(const std::string& checkpoint_model, const ListParams& params = {});
|
||||
clients::body::Json create_permission(const std::string& checkpoint_model, const clients::body::Json& body);
|
||||
clients::body::Json remove_permission(const std::string& checkpoint_model,
|
||||
const std::string& permission); // destructive
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+55
@@ -0,0 +1,55 @@
|
||||
#include "vendors/azure/foundry.h"
|
||||
|
||||
#include "llm/caps.h"
|
||||
#include "vendors/azure/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
llm::Headers FoundryService::headers(const std::string& extra) {
|
||||
llm::Headers h;
|
||||
if (extra.empty()) return h;
|
||||
Check::in_set("extra_parameters", extra, {"error", "drop", "pass-through"});
|
||||
h["extra-parameters"] = extra;
|
||||
return h;
|
||||
}
|
||||
|
||||
Json FoundryService::info() { return call(Op::kFoundryGetModelInfo); }
|
||||
|
||||
Json FoundryService::chat(const Json& body, const std::string& extra) {
|
||||
Check::object("body", body);
|
||||
const Json* m = body.find("messages");
|
||||
Check::require(m && m->is_array() && !m->as_array().empty(), "messages is required");
|
||||
Check::max_count("messages", m->as_array().size(), llm::Caps::kMaxMessages);
|
||||
const Json* s = body.find("stream");
|
||||
Check::require(!s || (s->is_bool() && !s->as_bool()), "use chat_stream() for stream=true");
|
||||
return call(Op::kFoundryGetChatCompletions, {}, &body, nullptr, headers(extra));
|
||||
}
|
||||
|
||||
void FoundryService::chat_stream(const Json& body, const ChunkHandler& handler, const std::string& extra) {
|
||||
Check::object("body", body);
|
||||
const Json* m = body.find("messages");
|
||||
Check::require(m && m->is_array() && !m->as_array().empty(), "messages is required");
|
||||
call_stream(Op::kFoundryGetChatCompletions, {}, body,
|
||||
[&](const llm::SseEvent& ev) { return handler(ev.data); }, headers(extra));
|
||||
}
|
||||
|
||||
Json FoundryService::embeddings(const Json& body, const std::string& extra) {
|
||||
Check::object("body", body);
|
||||
const Json* in = body.find("input");
|
||||
Check::require(in && in->is_array() && !in->as_array().empty(), "input must be a non-empty array of strings");
|
||||
Check::max_count("input", in->as_array().size(), AzureCaps::kMaxEmbeddingInputs);
|
||||
return call(Op::kFoundryGetEmbeddings, {}, &body, nullptr, headers(extra));
|
||||
}
|
||||
|
||||
Json FoundryService::image_embeddings(const Json& body, const std::string& extra) {
|
||||
Check::object("body", body);
|
||||
const Json* in = body.find("input");
|
||||
Check::require(in && in->is_array() && !in->as_array().empty(), "input must be a non-empty array of images");
|
||||
Check::max_count("input", in->as_array().size(), AzureCaps::kMaxEmbeddingInputs);
|
||||
return call(Op::kFoundryGetImageEmbeddings, {}, &body, nullptr, headers(extra));
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+39
@@ -0,0 +1,39 @@
|
||||
#pragma once
|
||||
|
||||
// Azure AI Foundry model inference (the Azure AI Model Inference API,
|
||||
// https://<resource>.services.ai.azure.com/models, api-version
|
||||
// 2024-05-01-preview): chat completions, text embeddings, image embeddings
|
||||
// and model info for the model catalog models (Llama, Mistral, Phi, Cohere,
|
||||
// DeepSeek, ...). Same credential as the OpenAI resource (api-key header or
|
||||
// Entra bearer). The `model` body field picks the deployment on a multi-model
|
||||
// endpoint.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class FoundryService : public AzureService {
|
||||
public:
|
||||
using ChunkHandler = std::function<bool(const clients::body::Json& chunk)>; // false: stop
|
||||
|
||||
explicit FoundryService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kFoundry) {}
|
||||
|
||||
// GET /info: model_name, model_type, model_provider_name of the endpoint's model.
|
||||
clients::body::Json info();
|
||||
// extra_parameters: "" (service default: error), "error", "drop" or "pass-through"
|
||||
// (the extra-parameters header of the spec).
|
||||
clients::body::Json chat(const clients::body::Json& body, const std::string& extra_parameters = "");
|
||||
void chat_stream(const clients::body::Json& body, const ChunkHandler& handler,
|
||||
const std::string& extra_parameters = "");
|
||||
clients::body::Json embeddings(const clients::body::Json& body, const std::string& extra_parameters = "");
|
||||
clients::body::Json image_embeddings(const clients::body::Json& body,
|
||||
const std::string& extra_parameters = "");
|
||||
|
||||
private:
|
||||
static llm::Headers headers(const std::string& extra_parameters);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+54
@@ -0,0 +1,54 @@
|
||||
#include "vendors/azure/images.h"
|
||||
|
||||
#include "vendors/azure/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
using clients::body::MultipartBody;
|
||||
|
||||
namespace {
|
||||
void add_file(MultipartBody& form, const std::string& field, const UploadFile& f) {
|
||||
Check::non_empty("image filename", f.filename);
|
||||
Check::no_control("filename", f.filename);
|
||||
Check::require(!f.data.empty(), "image file is empty");
|
||||
Check::require(f.data.size() <= AzureCaps::kMaxImageFileBytes, "image file over 50 MiB");
|
||||
form.add_file(field, f.filename, f.content_type, f.data);
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json ImagesService::generate(const std::string& deployment, const Json& body) {
|
||||
Check::deployment(deployment);
|
||||
Check::object("body", body);
|
||||
Check::has_string(body, "prompt");
|
||||
Check::max_chars("prompt", body.find("prompt")->as_string(), AzureCaps::kMaxImagePromptChars);
|
||||
return call(Op::kCreateImage, {deployment, {}}, &body);
|
||||
}
|
||||
|
||||
Json ImagesService::edit(const std::string& deployment, const std::string& prompt,
|
||||
const std::vector<UploadFile>& images, const std::optional<UploadFile>& mask,
|
||||
const std::map<std::string, std::string>& fields) {
|
||||
Check::deployment(deployment);
|
||||
Check::non_empty("prompt", prompt);
|
||||
Check::max_chars("prompt", prompt, AzureCaps::kMaxImagePromptChars);
|
||||
Check::require(!images.empty() && images.size() <= 16, "images must hold 1..16 files");
|
||||
MultipartBody form;
|
||||
form.add_text("prompt", prompt);
|
||||
for (const UploadFile& f : images) add_file(form, images.size() > 1 ? "image[]" : "image", f);
|
||||
if (mask) add_file(form, "mask", *mask);
|
||||
for (const auto& [k, v] : fields) form.add_text(k, v);
|
||||
return call_multipart(Op::kCreateImageEdit, {deployment, {}}, form);
|
||||
}
|
||||
|
||||
Json ImagesService::variation(const std::string& deployment, const UploadFile& image,
|
||||
const std::map<std::string, std::string>& fields) {
|
||||
Check::deployment(deployment);
|
||||
MultipartBody form;
|
||||
add_file(form, "image", image);
|
||||
for (const auto& [k, v] : fields) form.add_text(k, v);
|
||||
form.add_text("model", deployment);
|
||||
return call_multipart(Op::kCreateImageVariation, {}, form);
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+35
@@ -0,0 +1,35 @@
|
||||
#pragma once
|
||||
|
||||
// Images on a deployment (gpt-image-1, dall-e-3): generations (JSON), edits
|
||||
// and variations (multipart; the spec of the dated API lists JSON for edits,
|
||||
// the service and every official SDK send multipart, see SPEC Unverified).
|
||||
|
||||
#include <map>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
#include "vendors/azure/upload_file.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class ImagesService : public AzureService {
|
||||
public:
|
||||
explicit ImagesService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body: prompt (required, <= 32000) plus any field of the spec (n, size, quality...).
|
||||
clients::body::Json generate(const std::string& deployment, const clients::body::Json& body);
|
||||
// fields: text form fields of the spec (n, size, quality, response_format...).
|
||||
// images: 1..16 files (sent as image[] when more than one); mask optional.
|
||||
clients::body::Json edit(const std::string& deployment, const std::string& prompt,
|
||||
const std::vector<UploadFile>& images,
|
||||
const std::optional<UploadFile>& mask = std::nullopt,
|
||||
const std::map<std::string, std::string>& fields = {});
|
||||
// v1 API only (the dated documents have no variations operation).
|
||||
clients::body::Json variation(const std::string& deployment, const UploadFile& image,
|
||||
const std::map<std::string, std::string>& fields = {});
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+30
@@ -0,0 +1,30 @@
|
||||
#include "vendors/azure/models.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
Json ModelsService::list() { return call(Op::kListModels); }
|
||||
|
||||
std::vector<std::string> ModelsService::ids() {
|
||||
std::vector<std::string> out;
|
||||
Json doc = list();
|
||||
if (const Json* data = doc.find("data"); data && data->is_array())
|
||||
for (const Json& m : data->as_array())
|
||||
if (const Json* id = m.find("id"); id && id->is_string()) out.push_back(id->as_string());
|
||||
return out;
|
||||
}
|
||||
|
||||
Json ModelsService::get(const std::string& id) {
|
||||
Check::id("model", id);
|
||||
return call(Op::kRetrieveModel, {"", {id}});
|
||||
}
|
||||
|
||||
Json ModelsService::remove(const std::string& id) {
|
||||
Check::id("model", id);
|
||||
return call(Op::kDeleteModel, {"", {id}});
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+26
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
// Models available to the resource (base models and fine-tuned ones). On
|
||||
// Azure this lists models, not deployments: deployments are managed through
|
||||
// the Azure management plane (ARM), which this package does not call.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class ModelsService : public AzureService {
|
||||
public:
|
||||
explicit ModelsService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
clients::body::Json list();
|
||||
std::vector<std::string> ids();
|
||||
clients::body::Json get(const std::string& id);
|
||||
// v1 API only: deletes a fine-tuned model. Destructive, no default.
|
||||
clients::body::Json remove(const std::string& id);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+90
@@ -0,0 +1,90 @@
|
||||
#include "vendors/azure/operations.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
OperationsService::OperationsService(std::shared_ptr<AzureClient> azure)
|
||||
: azure_(azure),
|
||||
openai_(azure, Surface::kOpenAi),
|
||||
foundry_(azure, Surface::kFoundry),
|
||||
safety_(azure, Surface::kSafety) {}
|
||||
|
||||
AzureService& OperationsService::gate(Surface s) const {
|
||||
switch (s) {
|
||||
case Surface::kFoundry: return foundry_;
|
||||
case Surface::kSafety: return safety_;
|
||||
case Surface::kOpenAi: break;
|
||||
}
|
||||
return openai_;
|
||||
}
|
||||
|
||||
const CallSpec& OperationsService::lookup(const std::string& name) const {
|
||||
const CallSpec* c = Calls::find(name);
|
||||
if (!c) Check::fail("unknown operation '" + name + "'");
|
||||
return *c;
|
||||
}
|
||||
|
||||
bool OperationsService::destructive(const CallSpec& c) {
|
||||
if (std::strcmp(c.method, "DELETE") == 0) return true;
|
||||
std::string n = c.name;
|
||||
for (const char* p : {"Cancel", "Delete", "Reject", "Hangup", "Pause", "Refer", "Remove"})
|
||||
if (n.find(p) != std::string::npos) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
bool OperationsService::generic(const CallSpec& c) {
|
||||
for (const SideSpec* s : {&c.v1, &c.dated}) {
|
||||
if (!s->present()) continue;
|
||||
std::string content = s->content;
|
||||
if (content == "multipart") return false;
|
||||
}
|
||||
std::string n = c.name;
|
||||
// Binary or streaming answers belong to the typed services.
|
||||
return n != "CreateSpeech" && n != "DownloadFile" && n != "VideosRetrieveContent" &&
|
||||
n != "RetrieveContainerFileContent" && n != "CreateRealtimeCall";
|
||||
}
|
||||
|
||||
Query OperationsService::to_query(const std::map<std::string, std::string>& query) const {
|
||||
Query q;
|
||||
for (const auto& [k, v] : query) q.add(k, v);
|
||||
return q;
|
||||
}
|
||||
|
||||
OperationsService::Plan OperationsService::plan(const std::string& name, const Target& target,
|
||||
const Json* body,
|
||||
const std::map<std::string, std::string>& query) const {
|
||||
const CallSpec& c = lookup(name);
|
||||
if (!generic(c)) Check::fail(name + " needs its typed service (multipart or binary)");
|
||||
AzureService& g = gate(c.surface);
|
||||
Plan p;
|
||||
p.method = c.method;
|
||||
p.destructive = destructive(c);
|
||||
Route r = g.route(c.op, target);
|
||||
p.side = r.side == Side::kV1 ? "v1" : "dated";
|
||||
Query q = to_query(query);
|
||||
p.path = g.request_path(c.op, target, &q);
|
||||
if (body) {
|
||||
if (std::strcmp(c.side(r.side).content, "none") == 0) Check::fail(name + " takes no request body");
|
||||
p.body = g.prepare_body(c.op, target, *body);
|
||||
p.has_body = true;
|
||||
} else if (c.deployment && std::strstr(c.side(r.side).content, "json") && std::strcmp(c.method, "POST") == 0) {
|
||||
p.body = g.prepare_body(c.op, target, Json::object());
|
||||
p.has_body = true;
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
Json OperationsService::invoke(const std::string& name, const Target& target, const Json* body,
|
||||
const std::map<std::string, std::string>& query) {
|
||||
const CallSpec& c = lookup(name);
|
||||
if (!generic(c)) Check::fail(name + " needs its typed service (multipart or binary)");
|
||||
Query q = to_query(query);
|
||||
return gate(c.surface).call(c.op, target, body, &q);
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+53
@@ -0,0 +1,53 @@
|
||||
#pragma once
|
||||
|
||||
// OperationsService: every operation of the call table by name, for the ones
|
||||
// that have no typed class (containers, conversations, evals, graders,
|
||||
// videos, checkpoint tools) and for the demo's generic route. Same gateway as
|
||||
// the typed services: ids validated, fields and query names checked against
|
||||
// the spec, api side and api-version chosen from the settings.
|
||||
|
||||
#include <map>
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class OperationsService {
|
||||
public:
|
||||
explicit OperationsService(std::shared_ptr<AzureClient> azure);
|
||||
|
||||
// JSON in, JSON out. `body` may be null. `query` values are strings.
|
||||
// Throws ApiError::invalid for an unknown name, wrong ids, unknown fields or
|
||||
// query names, or an operation that needs multipart / streams / answers bytes
|
||||
// (use the typed service for those).
|
||||
clients::body::Json invoke(const std::string& name, const Target& target,
|
||||
const clients::body::Json* body = nullptr,
|
||||
const std::map<std::string, std::string>& query = {});
|
||||
// Route and body exactly as invoke() would send them; no traffic.
|
||||
struct Plan {
|
||||
std::string method;
|
||||
std::string path; // with query and api-version
|
||||
clients::body::Json body;
|
||||
bool has_body = false;
|
||||
bool destructive = false;
|
||||
std::string side; // "v1" or "dated"
|
||||
};
|
||||
Plan plan(const std::string& name, const Target& target, const clients::body::Json* body = nullptr,
|
||||
const std::map<std::string, std::string>& query = {}) const;
|
||||
|
||||
// DELETE, cancel, reject, hangup, pause, remove: the demo asks for confirm.
|
||||
static bool destructive(const CallSpec& spec);
|
||||
// Calls the generic JSON path can make (not multipart-only, not binary).
|
||||
static bool generic(const CallSpec& spec);
|
||||
|
||||
private:
|
||||
AzureService& gate(Surface s) const;
|
||||
const CallSpec& lookup(const std::string& name) const;
|
||||
Query to_query(const std::map<std::string, std::string>& query) const;
|
||||
|
||||
std::shared_ptr<AzureClient> azure_;
|
||||
mutable AzureService openai_, foundry_, safety_;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+209
@@ -0,0 +1,209 @@
|
||||
#include "vendors/azure/realtime.h"
|
||||
|
||||
#include <chrono>
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "google/live/ws_error.h"
|
||||
#include "google/live/ws_handshake.h"
|
||||
#include "google/live/ws_url.h"
|
||||
#include "llm/error.h"
|
||||
#include "vendors/azure/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
namespace live = google::live;
|
||||
|
||||
namespace {
|
||||
long long now_ms() {
|
||||
return std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now().time_since_epoch())
|
||||
.count();
|
||||
}
|
||||
|
||||
[[noreturn]] void rethrow(const live::WsError& e) {
|
||||
int status = 502;
|
||||
std::string type = "UNAVAILABLE";
|
||||
switch (e.kind()) {
|
||||
case live::WsError::Kind::Invalid: status = 400; type = "INVALID_ARGUMENT"; break;
|
||||
case live::WsError::Kind::Timeout: status = 504; type = "DEADLINE_EXCEEDED"; break;
|
||||
case live::WsError::Kind::Handshake:
|
||||
if (e.http_status() >= 400) {
|
||||
status = e.http_status();
|
||||
type = status == 401 ? "authentication_error" : "HANDSHAKE_REFUSED";
|
||||
} else {
|
||||
type = "HANDSHAKE_FAILED";
|
||||
}
|
||||
break;
|
||||
case live::WsError::Kind::Closed: type = "CLOSED"; break;
|
||||
case live::WsError::Kind::TooBig: type = "EVENT_TOO_BIG"; break;
|
||||
default: break;
|
||||
}
|
||||
throw llm::ApiError("azure", status, type, e.what());
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json RealtimeService::create_client_secret(const Json& body) {
|
||||
if (body.is_object() && body.find("expires_after")) {
|
||||
const Json* ea = body.find("expires_after");
|
||||
const Json* secs = ea->is_object() ? ea->find("seconds") : nullptr;
|
||||
Check::require(secs && secs->is_number(), "expires_after.seconds is required");
|
||||
Check::range("expires_after.seconds", secs->as_number(), 10, AzureCaps::kMaxClientSecretSeconds);
|
||||
}
|
||||
return call(Op::kCreateRealtimeClientSecret, {}, &body);
|
||||
}
|
||||
|
||||
Json RealtimeService::create_session(const Json& body) {
|
||||
Op op = azure_->settings().api_version.is_dated() ? Op::kCreateRealtimeSessionDated : Op::kCreateRealtimeSession;
|
||||
return call(op, {}, &body);
|
||||
}
|
||||
|
||||
Json RealtimeService::create_transcription_session(const Json& body) {
|
||||
Op op = azure_->settings().api_version.is_dated() ? Op::kCreateTranscriptionRealtimeSession
|
||||
: Op::kCreateRealtimeTranscriptionSession;
|
||||
return call(op, {}, &body);
|
||||
}
|
||||
|
||||
Json RealtimeService::accept_call(const std::string& call_id, const Json& body) {
|
||||
Check::id("call", call_id);
|
||||
Check::has_string(body, "type");
|
||||
return call(Op::kAcceptRealtimeCall, {"", {call_id}}, &body);
|
||||
}
|
||||
|
||||
Json RealtimeService::refer_call(const std::string& call_id, const std::string& target_uri) {
|
||||
Check::id("call", call_id);
|
||||
Check::non_empty("target_uri", target_uri);
|
||||
Check::no_control("target_uri", target_uri);
|
||||
Json body = Json::object();
|
||||
body.set("target_uri", Json(target_uri));
|
||||
return call(Op::kReferRealtimeCall, {"", {call_id}}, &body);
|
||||
}
|
||||
|
||||
Json RealtimeService::reject_call(const std::string& call_id, int status_code) {
|
||||
Check::id("call", call_id);
|
||||
Check::range("status_code", status_code, 400, 699);
|
||||
Json body = Json::object();
|
||||
body.set("status_code", Json(status_code));
|
||||
return call(Op::kRejectRealtimeCall, {"", {call_id}}, &body);
|
||||
}
|
||||
|
||||
Json RealtimeService::hangup_call(const std::string& call_id) {
|
||||
Check::id("call", call_id);
|
||||
return call(Op::kHangupRealtimeCall, {"", {call_id}});
|
||||
}
|
||||
|
||||
std::string RealtimeUrl::build(const Settings& s, const std::string& deployment) {
|
||||
Check::deployment(deployment);
|
||||
Check::require(!s.endpoint.empty(), "endpoint is not set");
|
||||
std::string host = s.endpoint;
|
||||
if (host.rfind("https://", 0) == 0) host = "wss://" + host.substr(8);
|
||||
else if (host.rfind("http://", 0) == 0) host = "ws://" + host.substr(7);
|
||||
std::string dep = clients::Url::encode_component(deployment);
|
||||
if (s.api_version.is_dated())
|
||||
return host + "/openai/realtime?api-version=" + clients::Url::encode_component(s.api_version.value()) +
|
||||
"&deployment=" + dep;
|
||||
std::string url = host + "/openai/v1/realtime?model=" + dep;
|
||||
if (!s.api_version.v1_query().empty()) url += "&api-version=" + s.api_version.v1_query();
|
||||
return url;
|
||||
}
|
||||
|
||||
RealtimeSocket::RealtimeSocket(std::shared_ptr<AzureClient> azure, live::TlsFactory tls)
|
||||
: azure_(std::move(azure)), tls_(std::move(tls)) {}
|
||||
|
||||
RealtimeSocket::~RealtimeSocket() {
|
||||
try {
|
||||
close();
|
||||
} catch (...) {
|
||||
}
|
||||
}
|
||||
|
||||
void RealtimeSocket::connect(const std::string& deployment) {
|
||||
Check::require(!ws_, "socket already connected");
|
||||
const Settings& s = azure_->settings();
|
||||
if (s.endpoint.empty()) throw llm::NotConfiguredError("azure", Settings::kEndpointEnv);
|
||||
if (s.auth() == AuthKind::kNone) throw llm::NotConfiguredError("azure", Settings::kKeyEnv);
|
||||
std::string url_text = RealtimeUrl::build(s, deployment);
|
||||
live::WsHeaders headers;
|
||||
if (s.auth() == AuthKind::kApiKey) headers.emplace_back("api-key", s.api_key);
|
||||
else headers.emplace_back("Authorization", "Bearer " + azure_->tokens()->token());
|
||||
try {
|
||||
live::WsUrl url = live::WsUrl::parse(url_text);
|
||||
auto client = std::make_unique<live::WsClient>(live::open_stream(url, 10000, 10000, tls_));
|
||||
client->handshake(url, headers, 10000);
|
||||
client->set_read_timeout_ms(250);
|
||||
ws_ = std::move(client);
|
||||
} catch (const live::WsError& e) {
|
||||
rethrow(e);
|
||||
}
|
||||
}
|
||||
|
||||
void RealtimeSocket::send(const Json& event) {
|
||||
Check::require(ws_ != nullptr, "socket is not connected");
|
||||
const Json* t = event.is_object() ? event.find("type") : nullptr;
|
||||
Check::require(t && t->is_string() && !t->as_string().empty(), "event needs a string type");
|
||||
std::string text = event.dump();
|
||||
Check::require(text.size() <= kMaxEventBytes, "event over the byte cap");
|
||||
try {
|
||||
ws_->send_text(text);
|
||||
} catch (const live::WsError& e) {
|
||||
rethrow(e);
|
||||
}
|
||||
}
|
||||
|
||||
RealtimeStats RealtimeSocket::run(const Sink& sink, int seconds) {
|
||||
Check::require(ws_ != nullptr, "socket is not connected");
|
||||
Check::range("seconds", seconds, 1, kMaxSessionSeconds);
|
||||
RealtimeStats stats;
|
||||
long long deadline = now_ms() + static_cast<long long>(seconds) * 1000;
|
||||
try {
|
||||
for (;;) {
|
||||
if (now_ms() >= deadline) {
|
||||
stats.stop_reason = "timeout";
|
||||
return stats;
|
||||
}
|
||||
auto m = ws_->receive();
|
||||
if (!m) continue;
|
||||
if (m->kind == live::WsMessage::Kind::Close) {
|
||||
stats.stop_reason = "closed";
|
||||
stats.close_code = m->code;
|
||||
return stats;
|
||||
}
|
||||
if (m->kind == live::WsMessage::Kind::Binary) continue;
|
||||
++stats.events;
|
||||
stats.bytes += m->data.size();
|
||||
Json ev;
|
||||
try {
|
||||
ev = Json::parse(m->data);
|
||||
} catch (const std::exception&) {
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "server event is not JSON");
|
||||
}
|
||||
if (!sink(ev)) {
|
||||
stats.stop_reason = "sink_stopped";
|
||||
return stats;
|
||||
}
|
||||
if (stats.events >= kMaxSessionEvents) {
|
||||
stats.stop_reason = "event_cap";
|
||||
return stats;
|
||||
}
|
||||
if (stats.bytes >= kMaxSessionBytes) {
|
||||
stats.stop_reason = "byte_cap";
|
||||
return stats;
|
||||
}
|
||||
}
|
||||
} catch (const live::WsError& e) {
|
||||
rethrow(e);
|
||||
}
|
||||
}
|
||||
|
||||
void RealtimeSocket::close() {
|
||||
if (!ws_) return;
|
||||
try {
|
||||
ws_->close(1000, "");
|
||||
} catch (...) {
|
||||
}
|
||||
ws_->shutdown();
|
||||
ws_.reset();
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+85
@@ -0,0 +1,85 @@
|
||||
#pragma once
|
||||
|
||||
// Realtime (gpt-realtime, gpt-4o-realtime-preview): the REST creators of
|
||||
// sessions and ephemeral client secrets, the WebSocket URL builder, and a
|
||||
// small WebSocket session over the RFC 6455 client of src/google/live
|
||||
// (WsClient, reused, not copied).
|
||||
//
|
||||
// v1 wss://<host>/openai/v1/realtime?model=<deployment>
|
||||
// dated wss://<host>/openai/realtime?api-version=<v>&deployment=<deployment>
|
||||
// The credential travels in a header (api-key, or Authorization: Bearer for
|
||||
// Entra), never in the URL.
|
||||
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
|
||||
#include "google/live/ws_client.h"
|
||||
#include "google/live/ws_stream.h"
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class RealtimeService : public AzureService {
|
||||
public:
|
||||
explicit RealtimeService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// Ephemeral client secret for browsers and devices (v1 API). The answer
|
||||
// holds the secret in `value`: hand it to the client, never log it.
|
||||
// body: session, expires_after ({"anchor":"created_at","seconds":10..7200}).
|
||||
clients::body::Json create_client_secret(const clients::body::Json& body = clients::body::Json::object());
|
||||
// Session creator: /realtime/sessions (v1) or /realtimeapi/sessions (dated).
|
||||
clients::body::Json create_session(const clients::body::Json& body);
|
||||
clients::body::Json create_transcription_session(
|
||||
const clients::body::Json& body = clients::body::Json::object());
|
||||
// SIP / WebRTC call control (v1 API). Destructive ones have no defaults.
|
||||
clients::body::Json accept_call(const std::string& call_id, const clients::body::Json& body);
|
||||
clients::body::Json refer_call(const std::string& call_id, const std::string& target_uri);
|
||||
clients::body::Json reject_call(const std::string& call_id, int status_code);
|
||||
clients::body::Json hangup_call(const std::string& call_id);
|
||||
};
|
||||
|
||||
class RealtimeUrl {
|
||||
public:
|
||||
// The WebSocket URL without credentials. Throws on a bad deployment.
|
||||
static std::string build(const Settings& settings, const std::string& deployment);
|
||||
};
|
||||
|
||||
struct RealtimeStats {
|
||||
std::size_t events = 0;
|
||||
std::size_t bytes = 0;
|
||||
std::string stop_reason; // sink_stopped, closed, timeout, event_cap, byte_cap
|
||||
int close_code = 0;
|
||||
};
|
||||
|
||||
class RealtimeSocket {
|
||||
public:
|
||||
static constexpr std::size_t kMaxEventBytes = 1024 * 1024;
|
||||
static constexpr std::size_t kMaxSessionEvents = 100000;
|
||||
static constexpr std::size_t kMaxSessionBytes = 64u * 1024u * 1024u;
|
||||
static constexpr int kMaxSessionSeconds = 3600;
|
||||
|
||||
using Sink = std::function<bool(const clients::body::Json& event)>; // false: stop
|
||||
|
||||
explicit RealtimeSocket(std::shared_ptr<AzureClient> azure, google::live::TlsFactory tls = nullptr);
|
||||
~RealtimeSocket();
|
||||
|
||||
bool connected() const { return ws_ != nullptr; }
|
||||
// Connects and upgrades. Throws NotConfiguredError (503) before any traffic
|
||||
// when there is no credential or endpoint.
|
||||
void connect(const std::string& deployment);
|
||||
// Sends one client event (a JSON object with a string `type`).
|
||||
void send(const clients::body::Json& event);
|
||||
// Receives until the sink returns false, the server closes, `seconds` pass
|
||||
// (1..3600) or a cap is hit.
|
||||
RealtimeStats run(const Sink& sink, int seconds);
|
||||
void close();
|
||||
|
||||
private:
|
||||
std::shared_ptr<AzureClient> azure_;
|
||||
google::live::TlsFactory tls_;
|
||||
std::unique_ptr<google::live::WsClient> ws_;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+58
@@ -0,0 +1,58 @@
|
||||
#include "vendors/azure/responses.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
void ResponsesService::validate(const Json& body) {
|
||||
Check::object("body", body);
|
||||
Check::has_string(body, "model");
|
||||
Check::deployment(body.find("model")->as_string());
|
||||
}
|
||||
|
||||
Json ResponsesService::create(const Json& body) {
|
||||
validate(body);
|
||||
const Json* s = body.find("stream");
|
||||
Check::require(!s || (s->is_bool() && !s->as_bool()), "use stream() for stream=true");
|
||||
return call(Op::kCreateResponse, {}, &body);
|
||||
}
|
||||
|
||||
void ResponsesService::stream(const Json& body, const EventHandler& handler) {
|
||||
validate(body);
|
||||
call_stream(Op::kCreateResponse, {}, body, handler);
|
||||
}
|
||||
|
||||
Json ResponsesService::retrieve(const std::string& id, const std::vector<std::string>& include) {
|
||||
Check::id("response", id);
|
||||
Query q;
|
||||
q.add_list("include", include);
|
||||
return call(Op::kGetResponse, {"", {id}}, nullptr, &q);
|
||||
}
|
||||
|
||||
Json ResponsesService::remove(const std::string& id) {
|
||||
Check::id("response", id);
|
||||
return call(Op::kDeleteResponse, {"", {id}});
|
||||
}
|
||||
|
||||
Json ResponsesService::cancel(const std::string& id) {
|
||||
Check::id("response", id);
|
||||
return call(Op::kCancelResponse, {"", {id}});
|
||||
}
|
||||
|
||||
Json ResponsesService::input_items(const std::string& id, const ListParams& params) {
|
||||
Check::id("response", id);
|
||||
params.validate();
|
||||
Query q;
|
||||
params.apply(q);
|
||||
return call(Op::kListInputItems, {"", {id}}, nullptr, &q);
|
||||
}
|
||||
|
||||
Listing ResponsesService::input_items_all(const std::string& id, const ListParams& params,
|
||||
std::size_t max_pages) {
|
||||
Check::id("response", id);
|
||||
return walk(Op::kListInputItems, {"", {id}}, params, Query(), max_pages);
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+37
@@ -0,0 +1,37 @@
|
||||
#pragma once
|
||||
|
||||
// Responses API. On Azure the `model` field of the body is the deployment
|
||||
// name on both APIs (v1: /openai/v1/responses, dated: /openai/responses).
|
||||
|
||||
#include <functional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class ResponsesService : public AzureService {
|
||||
public:
|
||||
using EventHandler = std::function<bool(const llm::SseEvent&)>; // false: stop
|
||||
|
||||
explicit ResponsesService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
// body: model (the deployment, required), input, plus any spec field.
|
||||
clients::body::Json create(const clients::body::Json& body);
|
||||
void stream(const clients::body::Json& body, const EventHandler& handler);
|
||||
clients::body::Json retrieve(const std::string& id, const std::vector<std::string>& include = {});
|
||||
// DELETE: removes the stored response. No default.
|
||||
clients::body::Json remove(const std::string& id);
|
||||
// Only responses created with background=true can be cancelled.
|
||||
clients::body::Json cancel(const std::string& id);
|
||||
clients::body::Json input_items(const std::string& id, const ListParams& params = {});
|
||||
Listing input_items_all(const std::string& id, const ListParams& params = {},
|
||||
std::size_t max_pages = 5);
|
||||
|
||||
private:
|
||||
static void validate(const clients::body::Json& body);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+121
@@ -0,0 +1,121 @@
|
||||
#include "vendors/azure/safety.h"
|
||||
|
||||
#include "vendors/azure/caps.h"
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
void SafetyService::check_blocklist_name(const std::string& name) {
|
||||
Check::require(!name.empty() && name.size() <= 64, "blocklist name must be 1..64 characters");
|
||||
for (char c : name)
|
||||
if (!((c >= '0' && c <= '9') || (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c == '.' ||
|
||||
c == '_' || c == '~' || c == '-'))
|
||||
Check::fail("blocklist name allows only 0-9 A-Z a-z . _ ~ -");
|
||||
}
|
||||
|
||||
Json SafetyService::analyze_text(const Json& body) {
|
||||
Check::has_string(body, "text");
|
||||
Check::max_chars("text", body.find("text")->as_string(), AzureCaps::kMaxSafetyTextChars);
|
||||
return call(Op::kSafetyTextOperationsAnalyzeText, {}, &body);
|
||||
}
|
||||
|
||||
Json SafetyService::analyze_image(const Json& body) {
|
||||
Check::object("body", body);
|
||||
const Json* img = body.find("image");
|
||||
Check::require(img && img->is_object(), "image is required");
|
||||
const Json* content = img->find("content");
|
||||
Check::require(content || img->find("blobUrl"), "image needs content (base64) or blobUrl");
|
||||
if (content && content->is_string())
|
||||
Check::require(content->as_string().size() <= AzureCaps::kMaxSafetyImageBytes * 4 / 3 + 8,
|
||||
"image over 7.5 MiB");
|
||||
return call(Op::kSafetyImageOperationsAnalyzeImage, {}, &body);
|
||||
}
|
||||
|
||||
Json SafetyService::shield_prompt(const Json& body) {
|
||||
Check::object("body", body);
|
||||
Check::require(body.find("userPrompt") || body.find("documents"), "userPrompt or documents is required");
|
||||
if (const Json* d = body.find("documents"); d && d->is_array())
|
||||
Check::max_count("documents", d->as_array().size(), AzureCaps::kMaxShieldDocuments);
|
||||
return call(Op::kSafetyTextOperationsShieldPrompt, {}, &body);
|
||||
}
|
||||
|
||||
Json SafetyService::detect_protected_material(const std::string& text) {
|
||||
Check::non_empty("text", text);
|
||||
Check::max_chars("text", text, AzureCaps::kMaxSafetyTextChars);
|
||||
Json body = Json::object();
|
||||
body.set("text", Json(text));
|
||||
return call(Op::kSafetyTextOperationsDetectTextProtectedMaterial, {}, &body);
|
||||
}
|
||||
|
||||
Json SafetyService::list_blocklists() { return call(Op::kSafetyTextBlocklistsListTextBlocklists); }
|
||||
|
||||
Json SafetyService::get_blocklist(const std::string& name) {
|
||||
check_blocklist_name(name);
|
||||
return call(Op::kSafetyTextBlocklistsGetTextBlocklist, {"", {name}});
|
||||
}
|
||||
|
||||
Json SafetyService::create_or_update_blocklist(const std::string& name, const std::string& description) {
|
||||
check_blocklist_name(name);
|
||||
Check::max_chars("description", description, 1024);
|
||||
Json body = Json::object();
|
||||
if (!description.empty()) body.set("description", Json(description));
|
||||
Op op = Op::kSafetyTextBlocklistsCreateOrUpdateTextBlocklist;
|
||||
Json prepared = prepare_body(op, {"", {name}}, body);
|
||||
std::string text = prepared.dump();
|
||||
llm::RawResponse r = client_->request_raw(Calls::method(op), request_path(op, {"", {name}}), &text,
|
||||
"application/merge-patch+json");
|
||||
return r.body.find_first_not_of(" \t\r\n") == std::string::npos ? Json::object() : Json::parse(r.body);
|
||||
}
|
||||
|
||||
Json SafetyService::remove_blocklist(const std::string& name) {
|
||||
check_blocklist_name(name);
|
||||
return call(Op::kSafetyTextBlocklistsDeleteTextBlocklist, {"", {name}});
|
||||
}
|
||||
|
||||
Json SafetyService::add_blocklist_items(const std::string& name, const Json& items) {
|
||||
check_blocklist_name(name);
|
||||
Check::require(items.is_array() && !items.as_array().empty(), "items must be a non-empty array");
|
||||
Check::max_count("items", items.as_array().size(), AzureCaps::kMaxBlocklistItems);
|
||||
for (const Json& it : items.as_array()) {
|
||||
Check::has_string(it, "text");
|
||||
Check::max_chars("item text", it.find("text")->as_string(), AzureCaps::kMaxBlocklistItemChars);
|
||||
}
|
||||
Json body = Json::object();
|
||||
body.set("blocklistItems", items);
|
||||
return call(Op::kSafetyTextBlocklistsAddOrUpdateBlocklistItems, {"", {name}}, &body);
|
||||
}
|
||||
|
||||
Json SafetyService::remove_blocklist_items(const std::string& name, const std::vector<std::string>& item_ids) {
|
||||
check_blocklist_name(name);
|
||||
Check::require(!item_ids.empty(), "item_ids must not be empty");
|
||||
Check::max_count("item_ids", item_ids.size(), AzureCaps::kMaxBlocklistItems);
|
||||
Json ids = Json::array();
|
||||
for (const std::string& i : item_ids) {
|
||||
Check::id("item", i);
|
||||
ids.push(Json(i));
|
||||
}
|
||||
Json body = Json::object();
|
||||
body.set("blocklistItemIds", ids);
|
||||
return call(Op::kSafetyTextBlocklistsRemoveBlocklistItems, {"", {name}}, &body);
|
||||
}
|
||||
|
||||
Json SafetyService::list_blocklist_items(const std::string& name, std::optional<int> maxpagesize,
|
||||
std::optional<int> skip) {
|
||||
check_blocklist_name(name);
|
||||
if (maxpagesize) Check::range("maxpagesize", *maxpagesize, 1, 1000);
|
||||
if (skip) Check::range("skip", *skip, 0, 1000000);
|
||||
Query q;
|
||||
q.add_int("maxpagesize", maxpagesize);
|
||||
q.add_int("skip", skip);
|
||||
return call(Op::kSafetyTextBlocklistsListTextBlocklistItems, {"", {name}}, nullptr, &q);
|
||||
}
|
||||
|
||||
Json SafetyService::get_blocklist_item(const std::string& name, const std::string& item_id) {
|
||||
check_blocklist_name(name);
|
||||
Check::id("item", item_id);
|
||||
return call(Op::kSafetyTextBlocklistsGetTextBlocklistItem, {"", {name, item_id}});
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+47
@@ -0,0 +1,47 @@
|
||||
#pragma once
|
||||
|
||||
// Azure AI Content Safety (api-version 2024-09-01): analyze text and images
|
||||
// for harm categories (Hate, SelfHarm, Sexual, Violence), Prompt Shields,
|
||||
// protected material detection, and custom text blocklists. Own endpoint
|
||||
// (https://<resource>.cognitiveservices.azure.com/contentsafety) and key
|
||||
// header (Ocp-Apim-Subscription-Key), or an Entra bearer token.
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class SafetyService : public AzureService {
|
||||
public:
|
||||
explicit SafetyService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kSafety) {}
|
||||
|
||||
// body: text (required, <= 10000 chars), categories, blocklistNames,
|
||||
// haltOnBlocklistHit, outputType.
|
||||
clients::body::Json analyze_text(const clients::body::Json& body);
|
||||
// body: image ({content: base64} or {blobUrl}), categories, outputType.
|
||||
clients::body::Json analyze_image(const clients::body::Json& body);
|
||||
// Prompt Shields: userPrompt and/or documents (<= 5).
|
||||
clients::body::Json shield_prompt(const clients::body::Json& body);
|
||||
clients::body::Json detect_protected_material(const std::string& text);
|
||||
|
||||
// Blocklists. Names: 1..64 of [0-9A-Za-z._~-].
|
||||
clients::body::Json list_blocklists();
|
||||
clients::body::Json get_blocklist(const std::string& name);
|
||||
// PATCH (merge-patch) creates or updates; description <= 1024 chars.
|
||||
clients::body::Json create_or_update_blocklist(const std::string& name, const std::string& description);
|
||||
clients::body::Json remove_blocklist(const std::string& name); // destructive
|
||||
// items: [{description, text}] (1..100, text <= 128 chars).
|
||||
clients::body::Json add_blocklist_items(const std::string& name, const clients::body::Json& items);
|
||||
clients::body::Json remove_blocklist_items(const std::string& name, const std::vector<std::string>& item_ids);
|
||||
// One page; `truncated` stays visible in the answer as nextLink.
|
||||
clients::body::Json list_blocklist_items(const std::string& name, std::optional<int> maxpagesize = std::nullopt,
|
||||
std::optional<int> skip = std::nullopt);
|
||||
clients::body::Json get_blocklist_item(const std::string& name, const std::string& item_id);
|
||||
|
||||
static void check_blocklist_name(const std::string& name);
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+291
@@ -0,0 +1,291 @@
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
#include "clients/url.h"
|
||||
#include "llm/error.h"
|
||||
#include "vendors/azure/caps.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
|
||||
|
||||
std::string strip_brackets(const std::string& name) {
|
||||
if (name.size() > 2 && name.compare(name.size() - 2, 2, "[]") == 0) return name.substr(0, name.size() - 2);
|
||||
return name;
|
||||
}
|
||||
|
||||
std::string encode_name(const std::string& k) {
|
||||
std::string name;
|
||||
for (char c : k)
|
||||
name += (c == '[' || c == ']') ? std::string(1, c) : clients::Url::encode_component(std::string(1, c));
|
||||
return name;
|
||||
}
|
||||
|
||||
bool has_json(const char* content) { return std::strstr(content, "json") != nullptr; }
|
||||
} // namespace
|
||||
|
||||
Query& Query::add(const std::string& name, const std::string& value) {
|
||||
if (value.empty()) return *this;
|
||||
for (char c : value)
|
||||
if (static_cast<unsigned char>(c) < 0x20) bad("query value has control characters");
|
||||
if (value.size() > 1024) bad("query value is too long");
|
||||
pairs_.emplace_back(name, value);
|
||||
return *this;
|
||||
}
|
||||
|
||||
Query& Query::add_int(const std::string& name, const std::optional<int>& value) {
|
||||
if (value) add(name, std::to_string(*value));
|
||||
return *this;
|
||||
}
|
||||
|
||||
Query& Query::add_bool(const std::string& name, const std::optional<bool>& value) {
|
||||
if (value) add(name, *value ? "true" : "false");
|
||||
return *this;
|
||||
}
|
||||
|
||||
Query& Query::add_list(const std::string& name, const std::vector<std::string>& values) {
|
||||
if (values.size() > 16) bad(name + ": too many values");
|
||||
for (const std::string& v : values) {
|
||||
if (v.empty()) bad(name + ": empty value");
|
||||
pairs_.emplace_back(name + "[]", v);
|
||||
}
|
||||
return *this;
|
||||
}
|
||||
|
||||
void ListParams::validate() const {
|
||||
if (limit && (*limit < 1 || *limit > AzureCaps::kMaxPageLimit))
|
||||
bad("limit must be 1.." + std::to_string(AzureCaps::kMaxPageLimit));
|
||||
if (!order.empty() && order != "asc" && order != "desc") bad("order must be asc or desc");
|
||||
if (!after.empty() && !llm::Service::valid_id(after)) bad("bad after cursor");
|
||||
if (!before.empty() && !llm::Service::valid_id(before)) bad("bad before cursor");
|
||||
}
|
||||
|
||||
void ListParams::apply(Query& q) const {
|
||||
q.add_int("limit", limit);
|
||||
q.add("order", order);
|
||||
q.add("after", after);
|
||||
q.add("before", before);
|
||||
}
|
||||
|
||||
AzureService::AzureService(std::shared_ptr<AzureClient> azure, Surface surface)
|
||||
: llm::Service(azure->api(surface)), azure_(azure), surface_(surface) {}
|
||||
|
||||
void AzureService::guard(Op op) const {
|
||||
if (Calls::spec(op).surface != surface_)
|
||||
bad(std::string(Calls::spec(op).name) + " does not belong to this service");
|
||||
}
|
||||
|
||||
Route AzureService::route(Op op, const Target& target) const {
|
||||
const CallSpec& c = Calls::spec(op);
|
||||
const Settings& s = azure_->settings();
|
||||
Route r;
|
||||
switch (c.surface) {
|
||||
case Surface::kFoundry:
|
||||
r.side = Side::kDated;
|
||||
r.version = s.foundry_version;
|
||||
break;
|
||||
case Surface::kSafety:
|
||||
r.side = Side::kDated;
|
||||
r.version = s.safety_version;
|
||||
break;
|
||||
case Surface::kOpenAi: {
|
||||
const ApiVersion& v = s.api_version;
|
||||
Side pref = v.is_dated() ? Side::kDated : Side::kV1;
|
||||
Side other = pref == Side::kDated ? Side::kV1 : Side::kDated;
|
||||
r.side = Calls::has(op, pref) ? pref : other;
|
||||
if (r.side == Side::kV1) r.version = v.is_v1_family() ? v.v1_query() : "preview";
|
||||
else r.version = v.is_dated() ? v.value() : ApiVersion::kDatedFallback;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (c.deployment && r.side == Side::kDated) {
|
||||
if (target.deployment.empty() || target.deployment.size() > AzureCaps::kMaxDeploymentChars)
|
||||
bad("a deployment name of 1.." + std::to_string(AzureCaps::kMaxDeploymentChars) + " characters is required");
|
||||
}
|
||||
r.path = Calls::prefix(op, r.side) + Calls::render(op, r.side, target.ids, target.deployment);
|
||||
return r;
|
||||
}
|
||||
|
||||
std::string AzureService::request_path(Op op, const Target& target, const Query* query) const {
|
||||
Route r = route(op, target);
|
||||
std::string out = r.path;
|
||||
char sep = '?';
|
||||
if (query) {
|
||||
for (const auto& [k, v] : query->pairs()) {
|
||||
if (!Calls::accepts_query(op, r.side, k))
|
||||
bad("query '" + k + "' is not a parameter of " + Calls::spec(op).side(r.side).op_id +
|
||||
" on the " + (r.side == Side::kV1 ? "v1" : "dated") + " API");
|
||||
out += sep;
|
||||
out += encode_name(k) + "=" + clients::Url::encode_component(v);
|
||||
sep = '&';
|
||||
}
|
||||
}
|
||||
if (!r.version.empty()) {
|
||||
out += sep;
|
||||
out += "api-version=" + clients::Url::encode_component(r.version);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
Json AzureService::prepare_body(Op op, const Target& target, const Json& body) const {
|
||||
if (!body.is_object()) bad("body must be a JSON object");
|
||||
Route r = route(op, target);
|
||||
const CallSpec& c = Calls::spec(op);
|
||||
Json out = body;
|
||||
Json::Object& obj = out.as_object();
|
||||
if (c.deployment) {
|
||||
auto it = obj.find("model");
|
||||
if (it != obj.end()) {
|
||||
if (!it->second.is_string() || it->second.as_string() != target.deployment)
|
||||
bad("model must equal the deployment name (or be left out)");
|
||||
if (r.side == Side::kDated) obj.erase(it);
|
||||
} else if (r.side == Side::kV1) {
|
||||
if (!llm::Service::valid_id(target.deployment) ||
|
||||
target.deployment.size() > AzureCaps::kMaxDeploymentChars)
|
||||
bad("a deployment name is required");
|
||||
obj["model"] = Json(target.deployment);
|
||||
}
|
||||
}
|
||||
for (const auto& [k, v] : obj) {
|
||||
(void)v;
|
||||
if (!Calls::accepts_field(op, r.side, k))
|
||||
bad("field '" + k + "' is not accepted by " + c.side(r.side).op_id + " on the " +
|
||||
(r.side == Side::kV1 ? "v1" : "dated") + " API");
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
clients::body::MultipartBody AzureService::prepare_form(
|
||||
Op op, const Target& target, const clients::body::MultipartBody& form) const {
|
||||
Route r = route(op, target);
|
||||
const CallSpec& c = Calls::spec(op);
|
||||
clients::body::MultipartBody out = form;
|
||||
bool has_model = false;
|
||||
for (const auto& part : form.parts()) {
|
||||
std::string name = strip_brackets(part.name);
|
||||
if (name == "model" && c.deployment) {
|
||||
has_model = true;
|
||||
if (part.data != target.deployment) bad("model must equal the deployment name (or be left out)");
|
||||
if (r.side == Side::kDated) bad("model is not a form field of the dated API (the deployment is in the URL)");
|
||||
continue;
|
||||
}
|
||||
if (!Calls::accepts_field(op, r.side, name))
|
||||
bad("form field '" + name + "' is not accepted by " + c.side(r.side).op_id);
|
||||
}
|
||||
if (c.deployment && r.side == Side::kV1 && !has_model) out.add_text("model", target.deployment);
|
||||
return out;
|
||||
}
|
||||
|
||||
Json AzureService::call(Op op, const Target& target, const Json* body, const Query* query,
|
||||
const llm::Headers& headers) {
|
||||
guard(op);
|
||||
const CallSpec& c = Calls::spec(op);
|
||||
Route r = route(op, target);
|
||||
std::string path = request_path(op, target, query);
|
||||
Json prepared;
|
||||
const Json* send = nullptr;
|
||||
if (body) {
|
||||
if (std::strcmp(c.side(r.side).content, "none") == 0)
|
||||
bad(std::string(c.name) + " takes no request body");
|
||||
prepared = prepare_body(op, target, *body);
|
||||
send = &prepared;
|
||||
} else if (has_json(c.side(r.side).content) && std::strcmp(c.method, "POST") == 0 &&
|
||||
c.deployment) {
|
||||
prepared = prepare_body(op, target, Json::object());
|
||||
send = &prepared;
|
||||
}
|
||||
return client_->request(Calls::method(op), path, send, headers);
|
||||
}
|
||||
|
||||
llm::RawResponse AzureService::call_raw(Op op, const Target& target, const Query* query,
|
||||
const llm::Headers& headers) {
|
||||
guard(op);
|
||||
return client_->request_raw(Calls::method(op), request_path(op, target, query), nullptr, "",
|
||||
headers);
|
||||
}
|
||||
|
||||
llm::RawResponse AzureService::call_json_raw(Op op, const Target& target, const Json& body,
|
||||
const llm::Headers& headers) {
|
||||
guard(op);
|
||||
Json prepared = prepare_body(op, target, body);
|
||||
std::string text = prepared.dump();
|
||||
return client_->request_raw(Calls::method(op), request_path(op, target), &text,
|
||||
"application/json", headers);
|
||||
}
|
||||
|
||||
Json AzureService::call_multipart(Op op, const Target& target,
|
||||
const clients::body::MultipartBody& form,
|
||||
const llm::Headers& headers) {
|
||||
guard(op);
|
||||
clients::body::MultipartBody prepared = prepare_form(op, target, form);
|
||||
return client_->post_multipart(request_path(op, target), prepared, headers);
|
||||
}
|
||||
|
||||
llm::RawResponse AzureService::call_multipart_raw(Op op, const Target& target,
|
||||
const clients::body::MultipartBody& form,
|
||||
const llm::Headers& headers) {
|
||||
guard(op);
|
||||
clients::body::MultipartBody prepared = prepare_form(op, target, form);
|
||||
std::string encoded = prepared.encode();
|
||||
return client_->request_raw(clients::Method::kPost, request_path(op, target), &encoded,
|
||||
prepared.content_type(), headers);
|
||||
}
|
||||
|
||||
void AzureService::call_stream(Op op, const Target& target, const Json& body, const EventSink& sink,
|
||||
const llm::Headers& headers) {
|
||||
guard(op);
|
||||
Route r = route(op, target);
|
||||
Json prepared = prepare_body(op, target, body);
|
||||
if (!Calls::accepts_field(op, r.side, "stream"))
|
||||
bad(std::string(Calls::spec(op).name) + " does not stream");
|
||||
prepared.set("stream", Json(true));
|
||||
std::size_t seen = 0;
|
||||
client_->stream(request_path(op, target), prepared,
|
||||
[&](const llm::SseEvent& ev) {
|
||||
if (++seen > AzureCaps::kMaxStreamEvents)
|
||||
throw llm::ApiError("azure", 502, "BAD_RESPONSE", "stream over the event cap");
|
||||
return sink(ev);
|
||||
},
|
||||
headers);
|
||||
}
|
||||
|
||||
Listing AzureService::walk(Op op, const Target& target, const ListParams& params, const Query& base,
|
||||
std::size_t max_pages) {
|
||||
guard(op);
|
||||
params.validate();
|
||||
if (max_pages < 1 || max_pages > AzureCaps::kMaxListPages)
|
||||
bad("max_pages must be 1.." + std::to_string(AzureCaps::kMaxListPages));
|
||||
Listing out;
|
||||
std::string after = params.after;
|
||||
for (std::size_t page = 0; page < max_pages; ++page) {
|
||||
Query q = base;
|
||||
q.add_int("limit", params.limit);
|
||||
q.add("order", params.order);
|
||||
q.add("before", params.before);
|
||||
q.add("after", after);
|
||||
Json doc = call(op, target, nullptr, &q);
|
||||
++out.pages;
|
||||
if (const Json* data = doc.find("data"); data && data->is_array())
|
||||
for (const Json& item : data->as_array()) out.items.push_back(item);
|
||||
const Json* more = doc.find("has_more");
|
||||
bool has_more = more && more->is_bool() && more->as_bool();
|
||||
std::string last = string_at(doc, "last_id");
|
||||
if (!has_more) return out;
|
||||
if (last.empty() || last == after) {
|
||||
out.truncated = true; // more pages, no usable cursor: stop loudly
|
||||
return out;
|
||||
}
|
||||
after = last;
|
||||
if (page + 1 == max_pages) {
|
||||
out.truncated = true;
|
||||
out.next_after = last;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+117
@@ -0,0 +1,117 @@
|
||||
#pragma once
|
||||
|
||||
// AzureService: the gateway every service class goes through. It
|
||||
// - picks the API side for an operation (v1 or dated) from the configured
|
||||
// api version, and the api-version query value that side needs;
|
||||
// - builds the path from the call table (ids validated as single path
|
||||
// segments, the deployment placed in the URL on the dated API and in the
|
||||
// body `model` on the v1 API);
|
||||
// - rejects request fields and query names the spec of that side does not
|
||||
// list (loud, before any traffic);
|
||||
// - sends JSON / multipart / raw / SSE through llm::ApiClient;
|
||||
// - walks cursor lists with a page cap and reports truncation.
|
||||
|
||||
#include <functional>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "clients/body/form.h"
|
||||
#include "clients/body/json.h"
|
||||
#include "llm/api_client.h"
|
||||
#include "llm/sse.h"
|
||||
#include "vendors/azure/calls.h"
|
||||
#include "vendors/azure/client.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
// Where an operation goes: the deployment (or empty) and the path ids in order.
|
||||
struct Target {
|
||||
std::string deployment;
|
||||
std::vector<std::string> ids;
|
||||
};
|
||||
|
||||
// Query builder; names are checked against the call row when the path is built.
|
||||
class Query {
|
||||
public:
|
||||
Query& add(const std::string& name, const std::string& value); // skipped when empty
|
||||
Query& add_int(const std::string& name, const std::optional<int>& value);
|
||||
Query& add_bool(const std::string& name, const std::optional<bool>& value);
|
||||
Query& add_list(const std::string& name, const std::vector<std::string>& values);
|
||||
const std::vector<std::pair<std::string, std::string>>& pairs() const { return pairs_; }
|
||||
bool empty() const { return pairs_.empty(); }
|
||||
|
||||
private:
|
||||
std::vector<std::pair<std::string, std::string>> pairs_;
|
||||
};
|
||||
|
||||
struct ListParams {
|
||||
std::optional<int> limit; // 1..100
|
||||
std::string after; // cursor (an object id)
|
||||
std::string before;
|
||||
std::string order; // "", "asc", "desc"
|
||||
void validate() const;
|
||||
void apply(Query& q) const;
|
||||
};
|
||||
|
||||
struct Listing {
|
||||
std::vector<clients::body::Json> items;
|
||||
bool truncated = false; // the page cap was hit while the server had more
|
||||
std::string next_after; // cursor to continue from when truncated
|
||||
std::size_t pages = 0;
|
||||
};
|
||||
|
||||
struct Route {
|
||||
Side side = Side::kV1;
|
||||
std::string path; // prefix + rendered spec path
|
||||
std::string version; // api-version query value, empty = omitted
|
||||
};
|
||||
|
||||
using EventSink = std::function<bool(const llm::SseEvent&)>; // false: stop
|
||||
|
||||
class AzureService : public llm::Service {
|
||||
public:
|
||||
AzureService(std::shared_ptr<AzureClient> azure, Surface surface);
|
||||
|
||||
const AzureClient& azure() const { return *azure_; }
|
||||
Surface surface() const { return surface_; }
|
||||
|
||||
// Side, path and api-version of an operation for a target (no traffic).
|
||||
Route route(Op op, const Target& target = {}) const;
|
||||
// Route path plus ?query&api-version=... exactly as sent.
|
||||
std::string request_path(Op op, const Target& target = {}, const Query* query = nullptr) const;
|
||||
// The JSON body as sent (deployment moved, fields checked). Throws on a bad body.
|
||||
clients::body::Json prepare_body(Op op, const Target& target, const clients::body::Json& body) const;
|
||||
|
||||
// Any operation of the surface by table entry. Used by the typed services
|
||||
// and by the generic route of the demo.
|
||||
clients::body::Json call(Op op, const Target& target = {},
|
||||
const clients::body::Json* body = nullptr, const Query* query = nullptr,
|
||||
const llm::Headers& headers = {});
|
||||
llm::RawResponse call_raw(Op op, const Target& target = {}, const Query* query = nullptr,
|
||||
const llm::Headers& headers = {});
|
||||
clients::body::Json call_multipart(Op op, const Target& target,
|
||||
const clients::body::MultipartBody& form,
|
||||
const llm::Headers& headers = {});
|
||||
// Multipart POST whose answer may be text (srt, vtt) or binary.
|
||||
llm::RawResponse call_multipart_raw(Op op, const Target& target,
|
||||
const clients::body::MultipartBody& form,
|
||||
const llm::Headers& headers = {});
|
||||
// POST with `stream: true` decoded as SSE; at most AzureCaps::kMaxStreamEvents events.
|
||||
void call_stream(Op op, const Target& target, const clients::body::Json& body,
|
||||
const EventSink& sink, const llm::Headers& headers = {});
|
||||
// Binary-answer POST (speech): the body as JSON, the answer as bytes.
|
||||
llm::RawResponse call_json_raw(Op op, const Target& target, const clients::body::Json& body,
|
||||
const llm::Headers& headers = {});
|
||||
Listing walk(Op op, const Target& target, const ListParams& params, const Query& base,
|
||||
std::size_t max_pages);
|
||||
|
||||
protected:
|
||||
void guard(Op op) const;
|
||||
clients::body::MultipartBody prepare_form(Op op, const Target& target,
|
||||
const clients::body::MultipartBody& form) const;
|
||||
std::shared_ptr<AzureClient> azure_;
|
||||
Surface surface_;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+112
@@ -0,0 +1,112 @@
|
||||
#include "vendors/azure/settings.h"
|
||||
|
||||
#include "llm/config.h"
|
||||
#include "llm/error.h"
|
||||
#include "vendors/azure/caps.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
namespace {
|
||||
[[noreturn]] void bad(const std::string& why) { throw llm::ApiError::invalid("azure", why); }
|
||||
|
||||
bool name_char(char c) {
|
||||
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '-' ||
|
||||
c == '.' || c == '_';
|
||||
}
|
||||
|
||||
void check_name(const std::string& what, const std::string& value) {
|
||||
if (value.empty() || value.size() > 128) bad(what + " must be 1..128 characters");
|
||||
for (char c : value)
|
||||
if (!name_char(c)) bad(what + " has invalid characters");
|
||||
}
|
||||
|
||||
std::string trimmed(std::string s) {
|
||||
while (!s.empty() && (s.back() == ' ' || s.back() == '\t' || s.back() == '\n' || s.back() == '\r'))
|
||||
s.pop_back();
|
||||
std::size_t i = 0;
|
||||
while (i < s.size() && (s[i] == ' ' || s[i] == '\t')) ++i;
|
||||
return s.substr(i);
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void EntraConfig::validate() const {
|
||||
check_name("tenant id", tenant_id);
|
||||
check_name("client id", client_id);
|
||||
if (client_secret.empty() || client_secret.size() > AzureCaps::kMaxSecretChars)
|
||||
bad("client secret must be 1.." + std::to_string(AzureCaps::kMaxSecretChars) + " characters");
|
||||
for (char c : client_secret)
|
||||
if (static_cast<unsigned char>(c) < 0x20) bad("client secret has control characters");
|
||||
Settings::normalize_endpoint(authority);
|
||||
if (scope.empty() || scope.size() > 256) bad("scope must be 1..256 characters");
|
||||
for (char c : scope)
|
||||
if (static_cast<unsigned char>(c) <= 0x20) bad("scope has invalid characters");
|
||||
}
|
||||
|
||||
std::string EntraConfig::token_url() const {
|
||||
return Settings::normalize_endpoint(authority) + "/" + tenant_id + "/oauth2/v2.0/token";
|
||||
}
|
||||
|
||||
std::string Settings::normalize_endpoint(const std::string& raw) {
|
||||
std::string e = trimmed(raw);
|
||||
if (e.empty()) return e;
|
||||
if (e.size() > AzureCaps::kMaxEndpointChars) bad("endpoint is too long");
|
||||
std::string scheme, rest;
|
||||
if (e.rfind("https://", 0) == 0) {
|
||||
scheme = "https://";
|
||||
} else if (e.rfind("http://127.0.0.1", 0) == 0 || e.rfind("http://localhost", 0) == 0) {
|
||||
scheme = "http://";
|
||||
} else {
|
||||
bad("endpoint must start with https://");
|
||||
}
|
||||
rest = e.substr(scheme.size());
|
||||
while (!rest.empty() && rest.back() == '/') rest.pop_back();
|
||||
if (rest.empty()) bad("endpoint has no host");
|
||||
e = scheme + rest;
|
||||
for (char c : rest) {
|
||||
if (static_cast<unsigned char>(c) <= 0x20) bad("endpoint has invalid characters");
|
||||
if (c == '/' || c == '?' || c == '#' || c == '@' || c == '\\')
|
||||
bad("endpoint must be scheme and host only (no path, query or credentials)");
|
||||
}
|
||||
return e;
|
||||
}
|
||||
|
||||
AuthKind Settings::auth() const {
|
||||
if (!api_key.empty()) return AuthKind::kApiKey;
|
||||
if (entra.complete()) return AuthKind::kEntra;
|
||||
return AuthKind::kNone;
|
||||
}
|
||||
|
||||
void Settings::validate() const {
|
||||
normalize_endpoint(endpoint);
|
||||
normalize_endpoint(foundry_endpoint);
|
||||
normalize_endpoint(safety_endpoint);
|
||||
for (const std::string* k : {&api_key, &safety_key}) {
|
||||
if (k->size() > AzureCaps::kMaxSecretChars) bad("api key is too long");
|
||||
for (char c : *k)
|
||||
if (static_cast<unsigned char>(c) <= 0x20) bad("api key has invalid characters");
|
||||
}
|
||||
if (entra.complete()) entra.validate();
|
||||
}
|
||||
|
||||
Settings Settings::from_env() {
|
||||
using llm::Config;
|
||||
Settings s;
|
||||
s.endpoint = normalize_endpoint(Config::get(kEndpointEnv));
|
||||
s.foundry_endpoint = normalize_endpoint(Config::get(kFoundryEndpointEnv));
|
||||
if (s.foundry_endpoint.empty()) s.foundry_endpoint = s.endpoint;
|
||||
s.safety_endpoint = normalize_endpoint(Config::get(kSafetyEndpointEnv));
|
||||
if (s.safety_endpoint.empty()) s.safety_endpoint = s.endpoint;
|
||||
s.api_key = trimmed(Config::get(kKeyEnv));
|
||||
s.safety_key = trimmed(Config::get(kSafetyKeyEnv));
|
||||
if (s.safety_key.empty()) s.safety_key = s.api_key;
|
||||
s.api_version = ApiVersion::parse(trimmed(Config::get(kVersionEnv)));
|
||||
s.entra.tenant_id = trimmed(Config::get(kTenantEnv));
|
||||
s.entra.client_id = trimmed(Config::get(kClientIdEnv));
|
||||
s.entra.client_secret = Config::get(kClientSecretEnv);
|
||||
std::string authority = trimmed(Config::get(kAuthorityEnv));
|
||||
if (!authority.empty()) s.entra.authority = authority;
|
||||
s.validate();
|
||||
return s;
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+82
@@ -0,0 +1,82 @@
|
||||
#pragma once
|
||||
|
||||
// Settings: everything the package needs to address and authenticate against
|
||||
// an Azure resource, read from llm::Config (environment, then the .env.json
|
||||
// files). Validation is loud and happens before any network traffic.
|
||||
//
|
||||
// AZURE_OPENAI_ENDPOINT https://<resource>.openai.azure.com (no path)
|
||||
// AZURE_OPENAI_API_KEY resource key, sent as the api-key header
|
||||
// AZURE_OPENAI_API_VERSION v1 (default), preview, or YYYY-MM-DD[-preview]
|
||||
// AZURE_TENANT_ID / AZURE_CLIENT_ID / AZURE_CLIENT_SECRET
|
||||
// Microsoft Entra client credentials (used when
|
||||
// no api key is set)
|
||||
// Optional extras (same style):
|
||||
// AZURE_AI_ENDPOINT Foundry model inference host, default = endpoint
|
||||
// AZURE_CONTENT_SAFETY_ENDPOINT / AZURE_CONTENT_SAFETY_KEY
|
||||
// default = endpoint / api key
|
||||
// AZURE_AUTHORITY_HOST default https://login.microsoftonline.com
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/api_version.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
enum class AuthKind { kNone, kApiKey, kEntra };
|
||||
|
||||
struct EntraConfig {
|
||||
static constexpr const char* kDefaultAuthority = "https://login.microsoftonline.com";
|
||||
static constexpr const char* kDefaultScope = "https://cognitiveservices.azure.com/.default";
|
||||
|
||||
std::string tenant_id;
|
||||
std::string client_id;
|
||||
std::string client_secret;
|
||||
std::string authority = kDefaultAuthority;
|
||||
std::string scope = kDefaultScope;
|
||||
|
||||
bool complete() const {
|
||||
return !tenant_id.empty() && !client_id.empty() && !client_secret.empty();
|
||||
}
|
||||
// Throws ApiError::invalid on a malformed tenant / client id / authority.
|
||||
void validate() const;
|
||||
// POST target: <authority>/<tenant>/oauth2/v2.0/token
|
||||
std::string token_url() const;
|
||||
};
|
||||
|
||||
struct Settings {
|
||||
static constexpr const char* kVendor = "azure";
|
||||
static constexpr const char* kEndpointEnv = "AZURE_OPENAI_ENDPOINT";
|
||||
static constexpr const char* kKeyEnv = "AZURE_OPENAI_API_KEY";
|
||||
static constexpr const char* kVersionEnv = "AZURE_OPENAI_API_VERSION";
|
||||
static constexpr const char* kTenantEnv = "AZURE_TENANT_ID";
|
||||
static constexpr const char* kClientIdEnv = "AZURE_CLIENT_ID";
|
||||
static constexpr const char* kClientSecretEnv = "AZURE_CLIENT_SECRET";
|
||||
static constexpr const char* kFoundryEndpointEnv = "AZURE_AI_ENDPOINT";
|
||||
static constexpr const char* kSafetyEndpointEnv = "AZURE_CONTENT_SAFETY_ENDPOINT";
|
||||
static constexpr const char* kSafetyKeyEnv = "AZURE_CONTENT_SAFETY_KEY";
|
||||
static constexpr const char* kAuthorityEnv = "AZURE_AUTHORITY_HOST";
|
||||
|
||||
std::string endpoint; // OpenAI resource
|
||||
std::string foundry_endpoint; // model inference (services.ai.azure.com)
|
||||
std::string safety_endpoint; // content safety (cognitiveservices.azure.com)
|
||||
std::string api_key;
|
||||
std::string safety_key;
|
||||
ApiVersion api_version;
|
||||
std::string foundry_version = ApiVersion::kFoundryDefault;
|
||||
std::string safety_version = ApiVersion::kSafetyDefault;
|
||||
EntraConfig entra;
|
||||
|
||||
// Reads the environment. Malformed values throw ApiError::invalid; missing
|
||||
// ones leave the field empty (the clients then report not configured).
|
||||
static Settings from_env();
|
||||
// Validates and normalizes: https only (http for loopback), host only, no
|
||||
// path, query, fragment or userinfo, trailing slash removed. Empty stays empty.
|
||||
static std::string normalize_endpoint(const std::string& endpoint);
|
||||
|
||||
AuthKind auth() const;
|
||||
bool configured() const { return !endpoint.empty() && auth() != AuthKind::kNone; }
|
||||
// Throws ApiError::invalid when a field is malformed (used by from_env and tests).
|
||||
void validate() const;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Binary file not shown.
+151
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check src/vendors/azure against the pinned azure-rest-api-specs documents.
|
||||
|
||||
python3 -I src/vendors/azure/tools/check_spec.py [local_spec_dir]
|
||||
|
||||
Fails (exit 1) when a row of calls_table.inc has a wrong method, path,
|
||||
operationId, request content kind, request property list or query list on
|
||||
either side; when a spec operation of a covered document is not in the table;
|
||||
when the table lists a name twice; or when the constants in the C++ sources
|
||||
(api versions, header names, Entra scope) disagree with the documents.
|
||||
Python 3 stdlib only. local_spec_dir may hold <name>.json copies of the
|
||||
documents (v1.json, v1p.json, inf25.json, auth24.json, cs24.json, mi.json);
|
||||
without it the pinned revision is fetched from raw.githubusercontent.com.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
import specdoc # noqa: E402
|
||||
|
||||
PKG = os.path.join(HERE, "..")
|
||||
STR = re.compile(r'"([^"]*)"')
|
||||
DEP = re.compile(r"^/deployments/\{[^}]+\}")
|
||||
|
||||
# Spec operations deliberately not in the table: none. (Older dated revisions
|
||||
# than the ones in specdoc.DOCS are not read at all; see SPEC.md.)
|
||||
EXCLUDED = set()
|
||||
|
||||
|
||||
def norm(method, path):
|
||||
return method.upper() + " " + re.sub(r"\{[^}]*\}", "{}", DEP.sub("", path))
|
||||
|
||||
|
||||
def read_rows():
|
||||
text = open(os.path.join(PKG, "calls_table.inc")).read()
|
||||
body = text[text.index("// AZ-ROWS-BEGIN"):text.index("// AZ-ROWS-END")]
|
||||
rows = []
|
||||
for line in body.splitlines()[1:]:
|
||||
if not line.startswith("AZ_OP("):
|
||||
continue
|
||||
name = line[len("AZ_OP("):].split(",")[0]
|
||||
surface = re.search(r"Surface::(\w+)", line).group(1)
|
||||
strs = STR.findall(line)
|
||||
dep = int(re.search(r'"[A-Z]+", (\d),', line).group(1))
|
||||
method = strs[0]
|
||||
sides = [strs[1:7], strs[7:13]]
|
||||
rows.append({"name": name, "surface": surface, "method": method, "dep": dep, "sides": sides})
|
||||
return rows
|
||||
|
||||
|
||||
def main():
|
||||
local = sys.argv[1] if len(sys.argv) > 1 else None
|
||||
docs = {n: specdoc.load(n, local) for n in specdoc.DOCS}
|
||||
rows = read_rows()
|
||||
bad = []
|
||||
names = set()
|
||||
covered = set() # (doc, method, path)
|
||||
sides_checked = 0
|
||||
for r in rows:
|
||||
if r["name"] in names:
|
||||
bad.append("duplicate row name " + r["name"])
|
||||
names.add(r["name"])
|
||||
for idx, s in enumerate(r["sides"]):
|
||||
doc_name, path, opid, content, body, query = s
|
||||
if not path:
|
||||
continue
|
||||
sides_checked += 1
|
||||
doc = docs.get(doc_name)
|
||||
if doc is None:
|
||||
bad.append("%s: unknown document %s" % (r["name"], doc_name))
|
||||
continue
|
||||
item = doc["paths"].get(path)
|
||||
op = item.get(r["method"].lower()) if item else None
|
||||
if op is None:
|
||||
bad.append("%s: %s %s missing in %s" % (r["name"], r["method"], path, doc_name))
|
||||
continue
|
||||
covered.add((doc_name, r["method"].lower(), path))
|
||||
f = specdoc.op_facts(doc, item, op)
|
||||
if f["opid"] != opid:
|
||||
bad.append("%s: operationId %r != %r" % (r["name"], f["opid"], opid))
|
||||
if f["content"] != content:
|
||||
bad.append("%s: content %s != %s" % (r["name"], f["content"], content))
|
||||
if sorted(filter(None, body.split(","))) != f["body"]:
|
||||
bad.append("%s: body fields differ: table-only %s spec-only %s" % (
|
||||
r["name"], sorted(set(filter(None, body.split(","))) - set(f["body"])),
|
||||
sorted(set(f["body"]) - set(filter(None, body.split(","))))))
|
||||
if sorted(filter(None, query.split(","))) != f["query"]:
|
||||
bad.append("%s: query differs: table %s spec %s" % (r["name"], query, f["query"]))
|
||||
if r["surface"] != "kOpenAi" and idx == 0:
|
||||
bad.append("%s: single-sided surface uses the v1 slot" % r["name"])
|
||||
if r["dep"] and idx == 1 and not DEP.match(path):
|
||||
bad.append("%s: deployment flag but no deployment path" % r["name"])
|
||||
# Every spec operation must be reachable through the table.
|
||||
keys = set()
|
||||
for r in rows:
|
||||
for s in r["sides"]:
|
||||
if s[1]:
|
||||
keys.add((r["surface"], norm(r["method"], s[1])))
|
||||
total_ops = 0
|
||||
for surface, names_ in specdoc.SURFACE_DOCS.items():
|
||||
sk = {"openai": "kOpenAi", "foundry": "kFoundry", "safety": "kSafety"}[surface]
|
||||
for dn in names_:
|
||||
for m, path, item, op in specdoc.operations(docs[dn]):
|
||||
total_ops += 1
|
||||
if (sk, norm(m, path)) not in keys and (dn, m, path) not in EXCLUDED:
|
||||
bad.append("unwrapped spec operation %s %s %s (%s)" % (dn, m.upper(), path, op.get("operationId")))
|
||||
# Second source: openai-python src/openai/lib/azure.py `_deployments_endpoints`
|
||||
# (read 2026-10-25): the operations the official SDK moves under /deployments/{model}.
|
||||
sdk_deployment_paths = {"/completions", "/chat/completions", "/embeddings", "/audio/transcriptions",
|
||||
"/audio/translations", "/audio/speech", "/images/generations", "/images/edits"}
|
||||
table_deployment_paths = {DEP.sub("", s[1]) for r in rows if r["dep"] for s in r["sides"][1:] if s[1]}
|
||||
if table_deployment_paths != sdk_deployment_paths:
|
||||
bad.append("deployment-scoped operations differ from the SDK: %s / %s" % (
|
||||
sorted(table_deployment_paths - sdk_deployment_paths), sorted(sdk_deployment_paths - table_deployment_paths)))
|
||||
if len([r for r in rows if r["dep"]]) != len(sdk_deployment_paths):
|
||||
bad.append("deployment flag count differs from the SDK list")
|
||||
# Constants in C++ sources.
|
||||
read = lambda p: open(os.path.join(PKG, p)).read()
|
||||
ver = read("api_version.h")
|
||||
for const, doc_name in (("kDatedFallback", "inf25"), ("kFoundryDefault", "mi"), ("kSafetyDefault", "cs24")):
|
||||
v = re.search(const + r' = "([^"]+)"', ver).group(1)
|
||||
spec_v = docs[doc_name]["info"]["version"]
|
||||
if v != spec_v:
|
||||
bad.append("%s %s != info.version %s of %s" % (const, v, spec_v, doc_name))
|
||||
enum = docs["v1p"]["components"]["schemas"]["AzureAIFoundryModelsApiVersion"]["enum"]
|
||||
if sorted(enum) != ["preview", "v1"]:
|
||||
bad.append("v1 api-version enum is now %s (ApiVersion knows v1, preview)" % enum)
|
||||
sec = docs["v1p"]["components"]["securitySchemes"]
|
||||
if sec["ApiKeyAuth"]["name"] != "api-key":
|
||||
bad.append("OpenAI api key header is no longer api-key")
|
||||
if docs["cs24"]["securityDefinitions"]["ApiKeyAuth"]["name"] != "Ocp-Apim-Subscription-Key":
|
||||
bad.append("Content Safety key header changed")
|
||||
scope = list(docs["cs24"]["securityDefinitions"]["OAuth2Auth"]["scopes"])[0]
|
||||
if scope not in read("settings.h"):
|
||||
bad.append("Entra scope %s not in settings.h" % scope)
|
||||
if "Ocp-Apim-Subscription-Key" not in read("client.cpp"):
|
||||
bad.append("client.cpp does not use Ocp-Apim-Subscription-Key")
|
||||
if bad:
|
||||
print("SPEC CHECK FAILED")
|
||||
for b in bad:
|
||||
print(" " + b)
|
||||
return 1
|
||||
print("spec ok: %d rows, %d table sides verified, %d spec operations seen in %d documents (rev %s)"
|
||||
% (len(rows), sides_checked, total_ops, len(docs), specdoc.REV[:10]))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Vendored
+109
@@ -0,0 +1,109 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generates src/vendors/azure/calls_table.inc from the pinned Azure specs.
|
||||
|
||||
python3 -I src/vendors/azure/tools/gen_calls.py [local_spec_dir]
|
||||
|
||||
One row per logical operation. The OpenAI surface has two sides, "v1" (the
|
||||
/openai/v1 API) and "dated" (the api-version=YYYY-MM-DD[-preview] API);
|
||||
operations present in both are one row. Foundry model inference and Content
|
||||
Safety are single-sided (dated slot). check_spec.py re-verifies every row.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import specdoc # noqa: E402
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
OUT = os.path.join(HERE, "..", "calls_table.inc")
|
||||
DEP = re.compile(r"^/deployments/\{[^}]+\}")
|
||||
|
||||
|
||||
def norm_key(method, path):
|
||||
p = DEP.sub("", path)
|
||||
return method + " " + re.sub(r"\{[^}]*\}", "{}", p)
|
||||
|
||||
|
||||
def collect(names, local):
|
||||
"""(method, path) -> (doc name, facts), first document wins."""
|
||||
out = {}
|
||||
for n in names:
|
||||
doc = specdoc.load(n, local)
|
||||
for m, path, item, op in specdoc.operations(doc):
|
||||
if (m, path) in out:
|
||||
continue
|
||||
f = specdoc.op_facts(doc, item, op)
|
||||
f["doc"] = n
|
||||
f["path"] = path
|
||||
out[(m, path)] = f
|
||||
return out
|
||||
|
||||
|
||||
def side_args(f):
|
||||
if f is None:
|
||||
return ['""'] * 6
|
||||
return ['"%s"' % f["doc"], '"%s"' % f["path"], '"%s"' % f["opid"], '"%s"' % f["content"],
|
||||
'"%s"' % ",".join(f["body"]), '"%s"' % ",".join(f["query"])]
|
||||
|
||||
|
||||
def main():
|
||||
local = sys.argv[1] if len(sys.argv) > 1 else None
|
||||
rows = [] # (surface, method, v1, dated)
|
||||
v1 = collect(("v1p", "v1"), local)
|
||||
dated = collect(("inf25", "auth24"), local)
|
||||
groups = {}
|
||||
for (m, path), f in v1.items():
|
||||
groups.setdefault(norm_key(m, path), [None, None, m])[0] = f
|
||||
for (m, path), f in dated.items():
|
||||
k = norm_key(m, path)
|
||||
g = groups.setdefault(k, [None, None, m])
|
||||
if g[1] is not None:
|
||||
sys.exit("two dated operations map to %s" % k)
|
||||
g[1] = f
|
||||
for k in sorted(groups):
|
||||
a, b, m = groups[k]
|
||||
rows.append(("kOpenAi", m, a, b))
|
||||
for surface, doc in (("kFoundry", "mi"), ("kSafety", "cs24")):
|
||||
for (m, path), f in sorted(collect((doc,), local).items()):
|
||||
rows.append((surface, m, None, f))
|
||||
names = {}
|
||||
lines = []
|
||||
for surface, m, a, b in rows:
|
||||
pick = a if a is not None and a["opid"] else (b if b is not None and b["opid"] else None)
|
||||
if pick is not None:
|
||||
name = specdoc.camel(pick["opid"])
|
||||
else:
|
||||
name = specdoc.camel(m + " " + (a or b)["path"])
|
||||
if surface == "kFoundry":
|
||||
name = "Foundry" + name
|
||||
if surface == "kSafety":
|
||||
name = "Safety" + name
|
||||
if name in names:
|
||||
alt = specdoc.camel(b["opid"]) if b is not None and b["opid"] else None
|
||||
name = alt if alt and alt not in names else name + "Dated"
|
||||
names[name] = 1
|
||||
if a is not None and b is not None:
|
||||
na = len(re.findall(r"\{", DEP.sub("", a["path"])))
|
||||
nb = len(re.findall(r"\{", DEP.sub("", b["path"])))
|
||||
if na != nb:
|
||||
sys.exit("placeholder count differs for %s" % name)
|
||||
dep = 1 if (b is not None and DEP.match(b["path"])) else 0
|
||||
if a is not None and DEP.match(a["path"]):
|
||||
sys.exit("v1 path with deployment: " + name)
|
||||
args = [name, "Surface::" + surface, '"%s"' % m.upper(), str(dep)] + side_args(a) + side_args(b)
|
||||
lines.append("AZ_OP(" + ", ".join(args) + ")")
|
||||
with open(OUT, "w") as fh:
|
||||
fh.write("// GENERATED by tools/gen_calls.py from azure-rest-api-specs %s. Do not edit.\n"
|
||||
% specdoc.REV)
|
||||
fh.write("// AZ_OP(name, surface, method, deployment_scoped, then twice (v1 side, dated side):\n")
|
||||
fh.write("// doc, path, operation_id, content, body_fields, query_params)\n")
|
||||
fh.write("// AZ-ROWS-BEGIN\n")
|
||||
for ln in lines:
|
||||
fh.write(ln + "\n")
|
||||
fh.write("// AZ-ROWS-END\n")
|
||||
print("wrote %d rows" % len(lines))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Vendored
+119
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Shared reader for the Azure OpenAPI / Swagger documents (stdlib only).
|
||||
|
||||
Used by gen_calls.py (writes calls_table.inc) and check_spec.py (verifies it).
|
||||
Every document is pinned to a commit of Azure/azure-rest-api-specs.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import urllib.request
|
||||
|
||||
# Pinned revision of Azure/azure-rest-api-specs (main at the time of reading).
|
||||
REV = "ba3eef39e5be55cd5ed1a2e5c6c9bb1a17c36b9a" # committed 2026-10-23T22:56:13Z
|
||||
BASE = "https://raw.githubusercontent.com/Azure/azure-rest-api-specs/%s/specification/" % REV
|
||||
|
||||
# Short name -> path under specification/
|
||||
DOCS = {
|
||||
"v1": "ai/data-plane/OpenAI.v1/azure-v1-v1-generated.json",
|
||||
"v1p": "ai/data-plane/OpenAI.v1/azure-v1-preview-generated.json",
|
||||
"inf25": "cognitiveservices/data-plane/OpenAIInference/preview/2025-04-01-preview/inference.json",
|
||||
"auth24": "cognitiveservices/data-plane/OpenAIAuthoring/stable/2024-10-21/azureopenai.json",
|
||||
"cs24": "cognitiveservices/data-plane/ContentSafety/stable/2024-09-01/contentsafety.json",
|
||||
"mi": "ai/data-plane/ModelInference/preview/2024-05-01-preview/openapi.json",
|
||||
}
|
||||
# Documents per surface: a spec operation in one of these must be in the table.
|
||||
SURFACE_DOCS = {"openai": ("v1p", "v1", "inf25", "auth24"), "foundry": ("mi",), "safety": ("cs24",)}
|
||||
VERBS = ("get", "post", "put", "patch", "delete")
|
||||
|
||||
|
||||
def load(name, local_dir=None):
|
||||
"""Loads a document by short name, from local_dir/<name>.json when given."""
|
||||
if local_dir:
|
||||
p = os.path.join(local_dir, name + ".json")
|
||||
if os.path.exists(p):
|
||||
return json.load(open(p))
|
||||
with urllib.request.urlopen(BASE + DOCS[name], timeout=180) as r:
|
||||
return json.load(r)
|
||||
|
||||
|
||||
def deref(doc, node):
|
||||
seen = 0
|
||||
while isinstance(node, dict) and "$ref" in node and seen < 20:
|
||||
cur = doc
|
||||
for part in node["$ref"].lstrip("#/").split("/"):
|
||||
cur = cur[part]
|
||||
node = cur
|
||||
seen += 1
|
||||
return node
|
||||
|
||||
|
||||
def schema_props(doc, node, depth=0):
|
||||
if depth > 12 or not isinstance(node, dict):
|
||||
return set()
|
||||
node = deref(doc, node)
|
||||
out = set(node.get("properties", {}))
|
||||
for key in ("allOf", "oneOf", "anyOf"):
|
||||
for x in node.get(key, []):
|
||||
out |= schema_props(doc, x, depth + 1)
|
||||
return out
|
||||
|
||||
|
||||
def params_of(doc, path_item, op):
|
||||
out = []
|
||||
for p in list(path_item.get("parameters", [])) + list(op.get("parameters", [])):
|
||||
out.append(deref(doc, p))
|
||||
return out
|
||||
|
||||
|
||||
def norm_content(c):
|
||||
if c.startswith("application/json") or c.endswith("+json"):
|
||||
return "json"
|
||||
if c.startswith("multipart/"):
|
||||
return "multipart"
|
||||
if c.startswith("application/x-www-form-urlencoded"):
|
||||
return "form"
|
||||
return c
|
||||
|
||||
|
||||
def op_facts(doc, path_item, op):
|
||||
"""Returns dict(opid, content, body, query) for one operation."""
|
||||
params = params_of(doc, path_item, op)
|
||||
query = sorted({p["name"] for p in params if p.get("in") == "query" and p["name"] != "api-version"})
|
||||
body = set()
|
||||
kinds = set()
|
||||
rb = op.get("requestBody")
|
||||
if rb:
|
||||
rb = deref(doc, rb)
|
||||
for c, x in rb.get("content", {}).items():
|
||||
kinds.add(norm_content(c))
|
||||
body |= schema_props(doc, x.get("schema", {}))
|
||||
for p in params:
|
||||
if p.get("in") == "body":
|
||||
kinds.add("json")
|
||||
body |= schema_props(doc, p.get("schema", {}))
|
||||
if p.get("in") == "formData":
|
||||
kinds.add("multipart")
|
||||
body.add(p["name"])
|
||||
return {
|
||||
"opid": op.get("operationId") or "",
|
||||
"content": "+".join(sorted(kinds)) or "none",
|
||||
"body": sorted(body),
|
||||
"query": query,
|
||||
}
|
||||
|
||||
|
||||
def operations(doc):
|
||||
for path, item in doc["paths"].items():
|
||||
for m in VERBS:
|
||||
if m in item:
|
||||
yield m, path, item, item[m]
|
||||
|
||||
|
||||
def camel(text):
|
||||
parts = re.split(r"[^A-Za-z0-9]+", text)
|
||||
out = ""
|
||||
for p in parts:
|
||||
if p:
|
||||
out += p[0].upper() + p[1:]
|
||||
return out
|
||||
Vendored
+15
@@ -0,0 +1,15 @@
|
||||
#pragma once
|
||||
|
||||
// A file for a multipart form: name, media type, bytes.
|
||||
|
||||
#include <string>
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
struct UploadFile {
|
||||
std::string filename;
|
||||
std::string content_type = "application/octet-stream";
|
||||
std::string data;
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+107
@@ -0,0 +1,107 @@
|
||||
#include "vendors/azure/vector_stores.h"
|
||||
|
||||
#include "vendors/azure/check.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
using clients::body::Json;
|
||||
|
||||
namespace {
|
||||
Query page(const ListParams& p, const std::string& filter = "") {
|
||||
p.validate();
|
||||
Query q;
|
||||
p.apply(q);
|
||||
if (!filter.empty()) {
|
||||
Check::in_set("filter", filter, {"in_progress", "completed", "failed", "cancelled"});
|
||||
q.add("filter", filter);
|
||||
}
|
||||
return q;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Json VectorStoresService::create(const Json& body) { return call(Op::kCreateVectorStore, {}, &body); }
|
||||
Json VectorStoresService::get(const std::string& s) {
|
||||
Check::id("vector store", s);
|
||||
return call(Op::kGetVectorStore, {"", {s}});
|
||||
}
|
||||
Json VectorStoresService::modify(const std::string& s, const Json& body) {
|
||||
Check::id("vector store", s);
|
||||
return call(Op::kModifyVectorStore, {"", {s}}, &body);
|
||||
}
|
||||
Json VectorStoresService::list(const ListParams& params) {
|
||||
Query q = page(params);
|
||||
return call(Op::kListVectorStores, {}, nullptr, &q);
|
||||
}
|
||||
Listing VectorStoresService::list_all(const ListParams& params, std::size_t max_pages) {
|
||||
return walk(Op::kListVectorStores, {}, params, Query(), max_pages);
|
||||
}
|
||||
Json VectorStoresService::remove(const std::string& s) {
|
||||
Check::id("vector store", s);
|
||||
return call(Op::kDeleteVectorStore, {"", {s}});
|
||||
}
|
||||
Json VectorStoresService::search(const std::string& s, const Json& body) {
|
||||
Check::id("vector store", s);
|
||||
Check::has_field(body, "query");
|
||||
return call(Op::kSearchVectorStore, {"", {s}}, &body);
|
||||
}
|
||||
|
||||
Json VectorStoresService::add_file(const std::string& s, const Json& body) {
|
||||
Check::id("vector store", s);
|
||||
Check::has_string(body, "file_id");
|
||||
return call(Op::kCreateVectorStoreFile, {"", {s}}, &body);
|
||||
}
|
||||
Json VectorStoresService::get_file(const std::string& s, const std::string& f) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("file", f);
|
||||
return call(Op::kGetVectorStoreFile, {"", {s, f}});
|
||||
}
|
||||
Json VectorStoresService::list_files(const std::string& s, const ListParams& params,
|
||||
const std::string& filter) {
|
||||
Check::id("vector store", s);
|
||||
Query q = page(params, filter);
|
||||
return call(Op::kListVectorStoreFiles, {"", {s}}, nullptr, &q);
|
||||
}
|
||||
Json VectorStoresService::update_file_attributes(const std::string& s, const std::string& f,
|
||||
const Json& attributes) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("file", f);
|
||||
Check::object("attributes", attributes);
|
||||
Json body = Json::object();
|
||||
body.set("attributes", attributes);
|
||||
return call(Op::kUpdateVectorStoreFileAttributes, {"", {s, f}}, &body);
|
||||
}
|
||||
Json VectorStoresService::file_content(const std::string& s, const std::string& f) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("file", f);
|
||||
return call(Op::kRetrieveVectorStoreFileContent, {"", {s, f}});
|
||||
}
|
||||
Json VectorStoresService::remove_file(const std::string& s, const std::string& f) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("file", f);
|
||||
return call(Op::kDeleteVectorStoreFile, {"", {s, f}});
|
||||
}
|
||||
|
||||
Json VectorStoresService::create_file_batch(const std::string& s, const Json& body) {
|
||||
Check::id("vector store", s);
|
||||
Check::require(body.find("file_ids") || body.find("files"), "file_ids (or files) is required");
|
||||
return call(Op::kCreateVectorStoreFileBatch, {"", {s}}, &body);
|
||||
}
|
||||
Json VectorStoresService::get_file_batch(const std::string& s, const std::string& b) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("batch", b);
|
||||
return call(Op::kGetVectorStoreFileBatch, {"", {s, b}});
|
||||
}
|
||||
Json VectorStoresService::cancel_file_batch(const std::string& s, const std::string& b) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("batch", b);
|
||||
return call(Op::kCancelVectorStoreFileBatch, {"", {s, b}});
|
||||
}
|
||||
Json VectorStoresService::list_batch_files(const std::string& s, const std::string& b,
|
||||
const ListParams& params, const std::string& filter) {
|
||||
Check::id("vector store", s);
|
||||
Check::id("batch", b);
|
||||
Query q = page(params, filter);
|
||||
return call(Op::kListFilesInVectorStoreBatch, {"", {s, b}}, nullptr, &q);
|
||||
}
|
||||
|
||||
} // namespace molo::azure
|
||||
Vendored
+41
@@ -0,0 +1,41 @@
|
||||
#pragma once
|
||||
|
||||
// Vector stores: stores, files, file batches, search, file content.
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vendors/azure/service.h"
|
||||
|
||||
namespace molo::azure {
|
||||
|
||||
class VectorStoresService : public AzureService {
|
||||
public:
|
||||
explicit VectorStoresService(std::shared_ptr<AzureClient> azure)
|
||||
: AzureService(std::move(azure), Surface::kOpenAi) {}
|
||||
|
||||
clients::body::Json create(const clients::body::Json& body = clients::body::Json::object());
|
||||
clients::body::Json get(const std::string& store);
|
||||
clients::body::Json modify(const std::string& store, const clients::body::Json& body);
|
||||
clients::body::Json list(const ListParams& params = {});
|
||||
Listing list_all(const ListParams& params = {}, std::size_t max_pages = 5);
|
||||
clients::body::Json remove(const std::string& store); // destructive
|
||||
// body: query (required) plus filters, max_num_results, ranking_options, rewrite_query.
|
||||
clients::body::Json search(const std::string& store, const clients::body::Json& body);
|
||||
|
||||
clients::body::Json add_file(const std::string& store, const clients::body::Json& body); // file_id required
|
||||
clients::body::Json get_file(const std::string& store, const std::string& file);
|
||||
clients::body::Json list_files(const std::string& store, const ListParams& params = {},
|
||||
const std::string& filter = "");
|
||||
clients::body::Json update_file_attributes(const std::string& store, const std::string& file,
|
||||
const clients::body::Json& attributes);
|
||||
clients::body::Json file_content(const std::string& store, const std::string& file);
|
||||
clients::body::Json remove_file(const std::string& store, const std::string& file); // destructive
|
||||
|
||||
clients::body::Json create_file_batch(const std::string& store, const clients::body::Json& body);
|
||||
clients::body::Json get_file_batch(const std::string& store, const std::string& batch);
|
||||
clients::body::Json cancel_file_batch(const std::string& store, const std::string& batch); // destructive
|
||||
clients::body::Json list_batch_files(const std::string& store, const std::string& batch,
|
||||
const ListParams& params = {}, const std::string& filter = "");
|
||||
};
|
||||
|
||||
} // namespace molo::azure
|
||||
Reference in New Issue
Block a user