# retoor <retoor@molodetz.nl>
from .._shared import endpoint, field
GROUP = {
"slug": "gateway",
"title": "OpenAI Gateway",
"admin": True,
"intro": """
# OpenAI Gateway
An OpenAI-compatible proxy mounted at `/openai/v1`. It forwards requests to a configured
upstream using the gateway's own credentials, so no DevPlace key is required - but an
administrator must enable the `openai` service first. Point any OpenAI-compatible client at
`{{ base }}/openai/v1`.
This gateway is the **single point of truth for AI** on the platform. Every other DevPlace
service (news, bots, Devii) calls it by default instead of an external provider, sends the
generic model name `molodetz`, and authenticates with an internal key that is auto-generated
on first boot. The real provider URLs, models, and keys (DeepSeek, OpenRouter) live only here,
so an operator switches providers or backends in one place. `DEEPSEEK_API_KEY` and
`OPENROUTER_API_KEY` are migrated into the editable settings on boot and the value in use is
shown. Because `Force model` is on by default, the upstream always receives the configured
model regardless of what a client (or `molodetz`) requests.
The gateway also performs **vision** augmentation: when a request includes an image, the gateway
describes the image with a configured vision model and rewrites it to text, so a vision-less
upstream still works. The vision model, URL, and key are configured alongside the other gateway
settings.
The gateway additionally serves **text embeddings** at `/openai/v1/embeddings`. Clients request the
generic model `molodetz~embed`, which the gateway maps to the configured embedding model (OpenRouter's
Qwen3 8B embedding model by default). Usage and cost are tracked per call exactly like chat and vision.
## Model routing and providers
On top of the single default upstream above, an administrator can register additional named
**providers** and map any number of requested **model names** onto them, so one gateway can front
many models across many backends. A model route binds a source model name (what a client sends) to a
target provider and upstream model, and carries:
- its **own pricing economy** (input, output, and cache-hit / cache-miss prices per million tokens),
used to compute that call's cost when the upstream returns no native cost;
- an optional **vision model**, which turns on the image-to-text merge for that route (so a text-only
model can answer about images);
- an optional **context window** used for the context-utilization header.
Resolution is transparent to clients: when the requested `model` matches an active route, the gateway
forwards to that route's provider and target model and meters the call against the route's economy.
When it matches no route, the request falls through to the default upstream unchanged (so `molodetz`,
`molodetz~embed`, and any existing client keep working exactly as before). Providers and routes are
managed by administrators on the **Gateway** page (`/admin/gateway`).
## Per-call cost and usage headers
Every gateway response - chat, embeddings, and passthrough, on both success and error - carries
`X-Gateway-*` response headers describing that single call, so a client can read its own token usage
and dollar cost directly from the response with no extra request:
| Header | Meaning |
|--------|---------|
| `X-Gateway-Model` | Upstream model actually used for the call |
| `X-Gateway-Backend` | Backend that served it: `chat`, `embed`, or passthrough |
| `X-Gateway-Prompt-Tokens` | Input (prompt) tokens |
| `X-Gateway-Completion-Tokens` | Output (completion) tokens |
| `X-Gateway-Total-Tokens` | Total tokens (prompt + completion) |
| `X-Gateway-Cache-Hit-Tokens` | Prompt tokens served from the upstream prompt cache |
| `X-Gateway-Cache-Miss-Tokens` | Prompt tokens not served from cache |
| `X-Gateway-Reasoning-Tokens` | Reasoning tokens, when the model reports them |
| `X-Gateway-Cost-USD` | Total cost of the call in US dollars |
| `X-Gateway-Input-Cost-USD` | Input portion of the cost in US dollars |
| `X-Gateway-Output-Cost-USD` | Output portion of the cost in US dollars |
| `X-Gateway-Cost-Native` | `1` if the dollar cost is the upstream's own reported cost, `0` if computed from the configured per-million pricing |
| `X-Gateway-Tokens-Per-Second` | Output tokens per second for the call |
| `X-Gateway-Upstream-Latency-Ms` | Upstream round-trip latency in milliseconds |
| `X-Gateway-Total-Latency-Ms` | Full end-to-end gateway time for the call in milliseconds |
| `X-Gateway-Gateway-Overhead-Ms` | Gateway processing time minus the upstream and queue wait, in milliseconds |
| `X-Gateway-Queue-Wait-Ms` | Time spent waiting on the concurrency semaphore before dispatch, in milliseconds |
| `X-Gateway-Connect-Ms` | Upstream connection establishment time in milliseconds |
| `X-Gateway-Context-Window` | The model's context window in tokens, when known |
| `X-Gateway-Context-Utilization` | Total tokens as a fraction of the context window, when known |
Dollar costs use the upstream's native `cost` field when it returns one
(`X-Gateway-Cost-Native: 1`); otherwise they are computed from the per-million prices of the matched
model route, falling back to the prices configured on the `openai` service when no route matches. The
one denied path that makes no upstream call (embeddings disabled) returns no usage headers.
Administrators enable and configure this gateway under [Background Services](/docs/services.html)
(the `openai` service).
The gateway is exempt from rate limiting, but every other endpoint follows the shared
[Conventions & Errors](/docs/conventions.html); see [Authentication](/docs/authentication.html)
for signing DevPlace's own requests.
""",
"endpoints": [
endpoint(
id="gateway-chat",
method="POST",
path="/openai/v1/chat/completions",
title="Chat completions",
summary="OpenAI-compatible chat completion. Supports streaming.",
auth="user",
encoding="json",
params=[
field(
"model",
"json",
"string",
False,
"gpt-4o-mini",
"Model id. When it matches a configured model route the gateway forwards to that route's provider and upstream model; otherwise it uses the default upstream model.",
),
field(
"messages",
"json",
"string",
True,
'[{"role":"user","content":"Hello"}]',
"Chat messages array.",
),
field(
"stream",
"json",
"string",
False,
"false",
"Set true for a streamed SSE response.",
),
],
notes=[
"Returns `503` when the gateway service is not running.",
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above), including the streamed SSE response.",
"If `model` matches a configured model route it is forwarded to that route's provider, upstream model, and per-model pricing (with an optional vision model); otherwise it falls through to the default upstream (see Model routing and providers above).",
],
),
endpoint(
id="gateway-embeddings",
method="POST",
path="/openai/v1/embeddings",
title="Embeddings",
summary="OpenAI-compatible text embeddings. Request model molodetz~embed.",
auth="user",
encoding="json",
params=[
field(
"model",
"json",
"string",
False,
"molodetz~embed",
"Embedding model id; the gateway maps molodetz~embed to the configured model, or to a matching embed model route's provider and target model.",
),
field(
"input",
"json",
"string",
True,
'"some text to embed"',
"String or array of strings to embed.",
),
field(
"dimensions",
"json",
"string",
False,
"4096",
"Optional output vector size (Matryoshka, 32-4096).",
),
],
notes=[
"Returns `503` when the gateway service is not running or embeddings are disabled.",
"Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
],
),
endpoint(
id="gateway-passthrough",
method="POST",
path="/openai/v1/{path}",
title="Passthrough",
summary="Any other /v1 path is forwarded to the upstream as-is.",
auth="user",
interactive=False,
params=[
field(
"path",
"path",
"string",
True,
"models",
"Upstream API path after /v1/.",
)
],
),
],
}
Reference in New Issue View Git Blame Copy Permalink