# retoor <retoor@molodetz.nl>
from .._shared import endpoint, field
GROUP = {
"slug": "gateway",
"title": "OpenAI Gateway",
"intro": """
# OpenAI Gateway
An OpenAI-compatible proxy mounted at `/openai/v1`. It forwards requests to a configured
upstream using the gateway's own credentials, so no DevPlace key is required - but an
administrator must enable the `openai` service first. Point any OpenAI-compatible client at
`{{ base }}/openai/v1`.
This gateway is the **single point of truth for AI** on the platform. Every other DevPlace
service (news, bots, Devii) calls it by default instead of an external provider, sends the
generic model name `molodetz`, and authenticates with an internal key that is auto-generated
on first boot. The real provider URLs, models, and keys (DeepSeek, OpenRouter) live only here,
so an operator switches providers or backends in one place. `DEEPSEEK_API_KEY` and
`OPENROUTER_API_KEY` are migrated into the editable settings on boot and the value in use is
shown. Because `Force model` is on by default, the upstream always receives the configured
model regardless of what a client (or `molodetz`) requests.
The gateway also performs **vision** augmentation: when a request includes an image, the gateway
describes the image with a configured vision model and rewrites it to text, so a vision-less
upstream still works. The vision model, URL, and key are configured alongside the other gateway
settings.
The gateway additionally serves **text embeddings** at `/openai/v1/embeddings`. Clients request the
generic model `molodetz~embed`, which the gateway maps to the configured embedding model (OpenRouter's
Qwen3 8B embedding model by default). Usage and cost are tracked per call exactly like chat and vision.
The gateway also serves **image generation** at `/openai/v1/images/generations`. Clients request the
generic model `molodetz-img-small`, which the gateway maps to the configured image model (OpenRouter's
Flux 1.1 Pro by default). Cost is tracked per call with a flat per-image price when the upstream
returns no native cost.
## Quick start
Copy the command below and paste it into a terminal. If you are signed in the `{{ api_key }}`
and `{{ app_reference }}` placeholders are already filled in with your own values; otherwise
replace them with the API key from your [profile](/profile) page and any application identifier.
```bash
curl -X POST "{{ base }}/openai/v1/chat/completions" \
-H "Authorization: Bearer {{ api_key }}" \
-H "X-App-Reference: {{ app_reference }}" \
-H "Content-Type: application/json" \
-d '{
"model": "molodetz",
"messages": [{"role": "user", "content": "Hello, how are you?"}]
}'
```
The response carries `X-Gateway-*` headers with token counts and dollar cost for the call.
For streaming, add `"stream": true` to the JSON body.
## Model routing and providers
On top of the single default upstream above, an administrator can register additional named
**providers** and map any number of requested **model names** onto them, so one gateway can front
many models across many backends. A model route binds a source model name (what a client sends) to a
target provider and upstream model, and carries:
- its **own pricing economy** (input, output, and cache-hit / cache-miss prices per million tokens),
used to compute that call's cost when the upstream returns no native cost;
- an optional **vision model**, which turns on the image-to-text merge for that route (so a text-only
model can answer about images);
- an optional **context window** used for the context-utilization header.
Resolution is transparent to clients: when the requested `model` matches an active route, the gateway
forwards to that route's provider and target model and meters the call against the route's economy.
When it matches no route, the request falls through to the default upstream unchanged (so `molodetz`,
`molodetz~embed`, and any existing client keep working exactly as before). Providers and routes are
managed by administrators on the **Gateway** page (`/admin/gateway`).
## Per-call cost and usage headers
Every gateway response - chat, embeddings, and passthrough, on both success and error - carries
`X-Gateway-*` response headers describing that single call, so a client can read its own token usage
and dollar cost directly from the response with no extra request:
| Header | Meaning |
|--------|---------|
| `X-Gateway-Model` | Upstream model actually used for the call |
| `X-Gateway-Backend` | Backend that served it: `chat`, `embed`, `image`, or passthrough |
| `X-Gateway-Prompt-Tokens` | Input (prompt) tokens |
| `X-Gateway-Completion-Tokens` | Output (completion) tokens |
| `X-Gateway-Total-Tokens` | Total tokens (prompt + completion) |
| `X-Gateway-Cache-Hit-Tokens` | Prompt tokens served from the upstream prompt cache |
| `X-Gateway-Cache-Miss-Tokens` | Prompt tokens not served from cache |
| `X-Gateway-Reasoning-Tokens` | Reasoning tokens, when the model reports them |
| `X-Gateway-Cost-USD` | Total cost of the call in US dollars |
| `X-Gateway-Input-Cost-USD` | Input portion of the cost in US dollars |
| `X-Gateway-Output-Cost-USD` | Output portion of the cost in US dollars |
| `X-Gateway-Cost-Native` | `1` if the dollar cost is the upstream's own reported cost, `0` if computed from the configured per-million pricing |
| `X-Gateway-Tokens-Per-Second` | Output tokens per second for the call |
| `X-Gateway-Upstream-Latency-Ms` | Upstream round-trip latency in milliseconds |
| `X-Gateway-Total-Latency-Ms` | Full end-to-end gateway time for the call in milliseconds |
| `X-Gateway-Gateway-Overhead-Ms` | Gateway processing time minus the upstream and queue wait, in milliseconds |
| `X-Gateway-Queue-Wait-Ms` | Time spent waiting on the concurrency semaphore before dispatch, in milliseconds |
| `X-Gateway-Connect-Ms` | Upstream connection establishment time in milliseconds |
| `X-Gateway-Context-Window` | The model's context window in tokens, when known |
| `X-Gateway-Context-Utilization` | Total tokens as a fraction of the context window, when known |
Dollar costs use the upstream's native `cost` field when it returns one
(`X-Gateway-Cost-Native: 1`); otherwise they are computed from the per-million prices of the matched
model route, falling back to the prices configured on the `openai` service when no route matches. The
denied paths that make no upstream call (embeddings or image generation disabled) return no usage headers.
## Request header `X-App-Reference`
Clients **SHOULD** send an `X-App-Reference` header to identify themselves for cost attribution.
The value is a free-form slug (max 30 characters, letters, digits, `_`, `.`, `-`). When missing or
invalid, the gateway defaults to `default`. The value is recorded in every usage ledger row and can
be queried alongside owner-kind and owner-id to attribute spending per application.
```
X-App-Reference: devplace-devii-v-1-0-0
```
Administrators enable and configure this gateway under [Background Services](/docs/services.html)
(the `openai` service).
The gateway is exempt from rate limiting, but every other endpoint follows the shared
[Conventions & Errors](/docs/conventions.html); see [Authentication](/docs/authentication.html)
for signing DevPlace's own requests.
""",
"endpoints": [
endpoint(
id="gateway-chat",
method="POST",
path="/openai/v1/chat/completions",
title="Chat completions",
summary="OpenAI-compatible chat completion. Supports streaming.",
auth="public",
encoding="json",
params=[
field(
"model",
"json",
"string",
False,
"gpt-4o-mini",
"Model id. When it matches a configured model route the gateway forwards to that route's provider and upstream model; otherwise it falls back to the configured default upstream model.",
),
field(
"messages",
"json",
"string",
True,
'[{"role":"user","content":"Hello"}]',
"Chat messages array.",
),
field(
"stream",
"json",
"string",
False,
"false",
"Set true for a streamed SSE response.",
),
],
notes=[
"Returns `503` when the gateway service is not running.",
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above), including the streamed SSE response.",
"If `model` matches a configured model route it is forwarded to that route's provider, upstream model, and per-model pricing (with an optional vision model); otherwise it falls through to the default upstream (see Model routing and providers above).",
],
),
endpoint(
id="gateway-embeddings",
method="POST",
path="/openai/v1/embeddings",
title="Embeddings",
summary="OpenAI-compatible text embeddings. Request model molodetz~embed.",
auth="public",
encoding="json",
params=[
field(
"model",
"json",
"string",
False,
"molodetz~embed",
"Embedding model id; the gateway maps molodetz~embed to the configured model, or to a matching embed model route's provider and target model.",
),
field(
"input",
"json",
"string",
True,
'"some text to embed"',
"String or array of strings to embed.",
),
field(
"dimensions",
"json",
"string",
False,
"4096",
"Optional output vector size (Matryoshka, 32-4096).",
),
],
notes=[
"Returns `503` when the gateway service is not running or embeddings are disabled.",
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
"If `model` matches a configured embed model route it is forwarded to that route's provider and upstream model; otherwise it falls back to the configured default embedding model.",
],
),
endpoint(
id="gateway-images",
method="POST",
path="/openai/v1/images/generations",
title="Image generation",
summary="OpenAI-compatible image generation. Request model molodetz-img-small.",
auth="public",
encoding="json",
params=[
field(
"model",
"json",
"string",
False,
"molodetz-img-small",
"Image model id; the gateway maps molodetz-img-small to the configured model, or to a matching image model route's provider and target model.",
),
field(
"prompt",
"json",
"string",
True,
'"a decorative developer award emblem"',
"Text prompt describing the image to generate.",
),
field(
"size",
"json",
"string",
False,
"512x512",
"Output dimensions (provider-dependent).",
),
field(
"response_format",
"json",
"string",
False,
"b64_json",
"Return format: url or b64_json.",
),
],
notes=[
"Returns `503` when the gateway service is not running or image generation is disabled.",
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
"If `model` matches a configured image model route it is forwarded to that route's provider and upstream model; otherwise it falls back to the configured default image model.",
],
),
endpoint(
id="gateway-passthrough",
method="POST",
path="/openai/v1/{path}",
title="Passthrough",
summary="Any other /v1 path is forwarded to the upstream as-is.",
auth="public",
interactive=False,
params=[
field(
"path",
"path",
"string",
True,
"models",
"Upstream API path after /v1/.",
)
],
),
],
}
Reference in New Issue View Git Blame Copy Permalink