# GatewayService (AI) `name` `openai` - enabled by default - 30s interval (minimum 5s). The gateway is an OpenAI-compatible endpoint at `/openai/v1/*` and the **single source of truth for all AI** on the platform. Every other AI consumer (news grading, the bots, Devii) routes through it, so all spend is attributed, priced, and limitable in one place. Source: `services/openai_gateway/`. See [Reading service data](/docs/services-data.html). > Audience: administrators and maintainers. This page describes credentials, pricing, and cost. ## What it does It forwards chat-completions and other `/v1/*` calls to a configured upstream (DeepSeek by default), optionally describing image content with a vision model first so vision-less upstreams still work, records every call to a usage ledger, computes cost, retries transient failures, and trips a circuit breaker on sustained upstream errors. The 30s service loop samples concurrency and prunes old usage rows; the real work happens per request. ## Endpoints ``` POST /openai/v1/chat/completions POST /openai/v1/embeddings GET|POST|PUT|DELETE|PATCH /openai/v1/{path} passthrough ``` Chat requests are augmented (vision), forwarded, retried, priced, and ledgered. Embeddings forward to the configured embeddings upstream. Other paths pass through to the upstream. ## Authentication and attribution `authorize()` accepts a request when any of these hold (and `Require authentication` is on): - the presented key (sent as `X-API-KEY` or `Authorization: Bearer`) equals the **static access key** (`gateway_access_key`), or - the presented key equals the auto-generated **internal key** (`gateway_internal_key`), which is how DevPlace's own services authenticate, or - the request carries a session for an admin (and `Allow admins` is on) or any user (and `Allow users` is on). `resolve_owner()` then records who made the call: `("internal", "devii")` for the internal key, `("key", "access")` for the access key, `("admin", uid)` or `("user", uid)` for a session user, else `("anonymous", "anonymous")`. This owner tuple is written on every ledger row so usage rolls up per user. Devii operating a signed-in user authenticates with that user's own API key, so their full gateway spend rolls up under their uid. ## Configuration fields **Upstream:** `gateway_upstream_url` (default DeepSeek chat-completions), `gateway_model` (`deepseek-v4-flash`), `gateway_force_model` (override the client-requested model, default on), `gateway_api_key` (auto-migrated from `DEEPSEEK_API_KEY`/`OPENROUTER_API_KEY`), `gateway_timeout` (default and minimum 300s), `gateway_instances` (max concurrent forwards per worker, default 4, 1 to 64; this is the connection pool plus semaphore). **Prompt:** `gateway_system_preamble` (multiline text, default empty) - operator text prepended ahead of every chat request's system message (see "System-message composition" below). **Vision:** `gateway_vision_enabled` (default on), `gateway_vision_url` (OpenRouter by default), `gateway_vision_model` (`google/gemma-3-12b-it`), `gateway_vision_key` (auto-migrated from `OPENROUTER_API_KEY`), `gateway_vision_cache_size` (LRU entries, default 256, 0 disables). **Access:** `gateway_require_auth` (default on), `gateway_allow_admins`, `gateway_allow_users` (both default on), `gateway_access_key` (secret, write-only), `gateway_internal_key` (auto-generated on boot; clear and restart to rotate). **Pricing (USD per 1M tokens):** `gateway_price_cache_hit_per_m` (0.0028), `gateway_price_cache_miss_per_m` (0.14), `gateway_price_output_per_m` (0.28), `gateway_vision_price_input_per_m` (0.0), `gateway_vision_price_output_per_m` (0.0). Used only when the upstream returns no native cost. **Reliability:** `gateway_max_retries` (default 2, 0 to 10), `gateway_retry_backoff_ms` (250, linear multiplied by attempt number), `gateway_circuit_threshold` (consecutive failures before opening, default 5, 0 disables), `gateway_circuit_cooldown_seconds` (default 30). **Tracking:** `gateway_usage_retention_hours` (default 720), `gateway_model_context_map` (JSON mapping model name to context window for utilization tracking). ## Model routing and providers The single upstream above is the implicit `default` provider. On top of it, an administrator can register additional named **providers** and map any number of requested model names onto them, so one gateway can front many models across many backends. Routing is managed on the **Gateway** page (`/admin/gateway`) and stored in two tables (`gateway_providers`, `gateway_models`); it is admin configuration, not user content. - A **provider** is a named upstream: `name`, a chat-completions `base_url`, and an `api_key`. The embeddings URL is derived from the base URL by swapping `/chat/completions` for `/embeddings`. - A **model route** maps a `source_model` (what a client requests) to a `provider` (blank uses the default upstream) and a `target_model` (sent upstream), with a `kind` of `chat` or `embed`, an optional `context_window`, an optional vision provider and model, and its own per-model **economy** (USD per 1M tokens): `price_cache_hit_per_m`, `price_cache_miss_per_m`, `price_output_per_m`, and `price_input_per_m` (the input price covers embeddings and the vision description). When a request's model matches an active route of the right kind, the gateway forwards to that route's provider and target model and meters the call against that route's economy. A route with a vision model turns on the **text-and-vision merge** for that route, so a text-only model can answer about images. When no route matches, the request falls through to the default upstream unchanged, so `molodetz`, `molodetz~embed`, and every existing client keep behaving exactly as before. Resolution is a per-request overlay onto the service config, so all of the behavior below (authentication, system-message composition, vision augmentation, retries, the circuit breaker, the ledger, and the `X-Gateway-*` headers) applies identically to routed requests. Routes and providers are also managed conversationally by an administrator through Devii (`gateway_providers`, `gateway_models` to read; `gateway_provider_set`, `gateway_provider_delete`, `gateway_model_set`, `gateway_model_delete` to manage, with deletes confirmation-gated). ## Cost computation If the upstream response carries a numeric `cost` field, that value is used verbatim and split proportionally into input and output (the row is flagged `native_cost`). Otherwise cost is computed from the per-1M pricing: for chat, `cache_hit_tokens` and `cache_miss_tokens` are priced separately on input plus `completion_tokens` on output; for vision, input and output use the vision rates. When the requested model matches a model route, that route's per-model prices are used instead of the service-level pricing fields, so each model carries its own economy. Prompt-cache hit rate directly drives cost: at the default rates cache hits are roughly fifty times cheaper than misses. ## Response headers Every gateway response (chat, embeddings, and passthrough; success and error) carries `X-Gateway-*` headers built from the same computed ledger row, so any caller can read the token usage and dollar cost of its own call straight from the response without querying the admin analytics. `GatewayUsageLedger.record()` returns the computed row and each handler maps it through `usage.usage_response_headers()` onto the response: `X-Gateway-Model`, `X-Gateway-Backend`, `X-Gateway-Prompt-Tokens`, `X-Gateway-Completion-Tokens`, `X-Gateway-Total-Tokens`, `X-Gateway-Cache-Hit-Tokens`, `X-Gateway-Cache-Miss-Tokens`, `X-Gateway-Reasoning-Tokens`, `X-Gateway-Cost-USD`, `X-Gateway-Input-Cost-USD`, `X-Gateway-Output-Cost-USD`, `X-Gateway-Cost-Native` (`1` when the dollar cost is the upstream's own value, `0` when computed from the per-1M pricing), `X-Gateway-Tokens-Per-Second`, `X-Gateway-Upstream-Latency-Ms`, `X-Gateway-Total-Latency-Ms` (full end-to-end gateway time), `X-Gateway-Gateway-Overhead-Ms` (gateway processing minus upstream and queue wait), `X-Gateway-Queue-Wait-Ms` (time waiting on the concurrency semaphore), `X-Gateway-Connect-Ms` (upstream connection establishment), and, when the model's window is known, `X-Gateway-Context-Window` and `X-Gateway-Context-Utilization`. The denied embeddings-disabled path makes no upstream call and returns no usage headers. The AI content correction worker reads `X-Gateway-Cost-USD` and the token headers off its own correction call to accumulate per-user totals (see [AI content correction](/docs/ai-correction.html)). ## Reliability layer `retry_send()` retries on connection errors, timeouts, and upstream 5xx, up to `max_retries`, with a linear backoff of `backoff_ms * attempt`. The `CircuitBreaker` counts consecutive failures: at `threshold` it opens and new requests are rejected immediately with `503` and error type `circuit_open`; after `cooldown_seconds` it half-opens to test recovery and closes on the next success. Setting the threshold to 0 disables the breaker. ## System-message composition Before a chat request is forwarded (and only for `chat/completions`, never embeddings or passthrough), the gateway composes a single system message in the fixed order: operator preamble, then a date line, then the client's original system content. The operator preamble (`gateway_system_preamble`) is prepended on every chat call so all calls carry your preferences. If the client sent no system message at all, the composed content becomes a new leading system message; a blank or whitespace-only preamble is a no-op. The current date is injected as `Current date: DD/MM/YYYY` (EU format) only when the composed text contains no date in any common format already (ISO `YYYY-MM-DD`, `D/M/YYYY`, `DD/MM/YYYY`, `MM/DD/YYYY`, `DD-MM-YYYY`, `DD.MM.YYYY`, and textual months such as `14 June 2026`, `June 14, 2026`, `14 Jun 2026`). The date is intentionally date-only with no time: keeping it stable for the whole day preserves upstream prompt and token caching. ## Vision augmentation Before forwarding a chat request, image blocks are described by the vision model and replaced with a text description, so a text-only upstream still answers questions about images. Descriptions are cached in an LRU keyed by image hash; vision calls are themselves ledgered with `backend="vision"`. If vision is disabled or the key is missing, images become a placeholder note rather than failing the request. ## Data it offers **Live metrics** (in `describe()`, at `GET /admin/services/data`): runtime counters (`requests`, `errors`, `in_flight`, `peak_in_flight`, `vision_calls`, `last_status`, `last_latency_ms`, `pool`, `circuit_open`) and a 24h summary (`requests`, `success_pct`, `error_pct`, `cost_hour`, `cost_24h`, `tokens_24h`, `avg_latency_ms`, `avg_tps`, `peak_concurrency`, `top_model`, `top_caller`). **Ledger** `gateway_usage_ledger`: one row per upstream call with owner, backend, requested and actual model, status, success, error category, the full latency breakdown (`upstream_latency_ms`, `gateway_overhead_ms`, `queue_wait_ms`, `connect_ms`, `total_latency_ms`), token counts (prompt, completion, cache hit and miss, reasoning, total), `tokens_per_second`, context window and utilization, the cost fields (`cost_usd`, `input_cost_usd`, `output_cost_usd`, `native_cost`), request shape (`stream_requested`, `temperature`, `top_p`, `max_tokens`, `has_tools`), and reliability flags (`retries_attempted`, `retry_succeeded`, `circuit_open`). Pruned to `gateway_usage_retention_hours`. **Concurrency samples** `gateway_concurrency_samples`: `in_flight` sampled once per service tick. **Deep analytics** at `GET /admin/ai-usage/data` (`GatewayUsageOut`): per-hour buckets, token and latency percentiles, error breakdowns, cost (this hour, 24h, projected monthly, caching savings, by model, by caller), and behavior. TTFT and inter-token latency are null because the gateway calls the upstream non-streaming. Per-user usage is at `GET /admin/users/{uid}/ai-usage`. It is all rendered on `/admin/ai-usage` by `AiUsageMonitor.js`.