# GatewayService (AI) `name` `openai` - enabled by default - 30s interval (minimum 5s). The gateway is an OpenAI-compatible endpoint at `/openai/v1/*` and the **single source of truth for all AI** on the platform. Every other AI consumer (news grading, the bots, Devii) routes through it, so all spend is attributed, priced, and limitable in one place. Source: `services/openai_gateway/`. See [Reading service data](/docs/services-data.html). > Audience: administrators and maintainers. This page describes credentials, pricing, and cost. ## What it does It forwards chat-completions and other `/v1/*` calls to a configured upstream (DeepSeek by default), optionally describing image content with a vision model first so vision-less upstreams still work, records every call to a usage ledger, computes cost, retries transient failures, and trips a circuit breaker on sustained upstream errors. The 30s service loop samples concurrency and prunes old usage rows; the real work happens per request. ## Endpoints ``` POST /openai/v1/chat/completions GET|POST|PUT|DELETE|PATCH /openai/v1/{path} passthrough ``` Chat requests are augmented (vision), forwarded, retried, priced, and ledgered. Other paths pass through to the upstream. ## Authentication and attribution `authorize()` accepts a request when any of these hold (and `Require authentication` is on): - the presented key (sent as `X-API-KEY` or `Authorization: Bearer`) equals the **static access key** (`gateway_access_key`), or - the presented key equals the auto-generated **internal key** (`gateway_internal_key`), which is how DevPlace's own services authenticate, or - the request carries a session for an admin (and `Allow admins` is on) or any user (and `Allow users` is on). `resolve_owner()` then records who made the call: `("internal", "devii")` for the internal key, `("key", "access")` for the access key, `("admin", uid)` or `("user", uid)` for a session user, else `("anonymous", "anonymous")`. This owner tuple is written on every ledger row so usage rolls up per user. Devii operating a signed-in user authenticates with that user's own API key, so their full gateway spend rolls up under their uid. ## Configuration fields **Upstream:** `gateway_upstream_url` (default DeepSeek chat-completions), `gateway_model` (`deepseek-v4-flash`), `gateway_force_model` (override the client-requested model, default on), `gateway_api_key` (auto-migrated from `DEEPSEEK_API_KEY`/`OPENROUTER_API_KEY`), `gateway_timeout` (default and minimum 300s), `gateway_instances` (max concurrent forwards per worker, default 4, 1 to 64; this is the connection pool plus semaphore). **Prompt:** `gateway_system_preamble` (multiline text, default empty) - operator text prepended ahead of every chat request's system message (see "System-message composition" below). **Vision:** `gateway_vision_enabled` (default on), `gateway_vision_url` (OpenRouter by default), `gateway_vision_model` (`google/gemma-3-12b-it`), `gateway_vision_key` (auto-migrated from `OPENROUTER_API_KEY`), `gateway_vision_cache_size` (LRU entries, default 256, 0 disables). **Access:** `gateway_require_auth` (default on), `gateway_allow_admins`, `gateway_allow_users` (both default on), `gateway_access_key` (secret, write-only), `gateway_internal_key` (auto-generated on boot; clear and restart to rotate). **Pricing (USD per 1M tokens):** `gateway_price_cache_hit_per_m` (0.0028), `gateway_price_cache_miss_per_m` (0.14), `gateway_price_output_per_m` (0.28), `gateway_vision_price_input_per_m` (0.0), `gateway_vision_price_output_per_m` (0.0). Used only when the upstream returns no native cost. **Reliability:** `gateway_max_retries` (default 2, 0 to 10), `gateway_retry_backoff_ms` (250, linear multiplied by attempt number), `gateway_circuit_threshold` (consecutive failures before opening, default 5, 0 disables), `gateway_circuit_cooldown_seconds` (default 30). **Tracking:** `gateway_usage_retention_hours` (default 720), `gateway_model_context_map` (JSON mapping model name to context window for utilization tracking). ## Cost computation If the upstream response carries a numeric `cost` field, that value is used verbatim and split proportionally into input and output (the row is flagged `native_cost`). Otherwise cost is computed from the per-1M pricing: for chat, `cache_hit_tokens` and `cache_miss_tokens` are priced separately on input plus `completion_tokens` on output; for vision, input and output use the vision rates. Prompt-cache hit rate directly drives cost: at the default rates cache hits are roughly fifty times cheaper than misses. ## Response headers Every gateway response (chat, embeddings, and passthrough; success and error) carries `X-Gateway-*` headers built from the same computed ledger row, so any caller can read the token usage and dollar cost of its own call straight from the response without querying the admin analytics. `GatewayUsageLedger.record()` returns the computed row and each handler maps it through `usage.usage_response_headers()` onto the response: `X-Gateway-Model`, `X-Gateway-Backend`, `X-Gateway-Prompt-Tokens`, `X-Gateway-Completion-Tokens`, `X-Gateway-Total-Tokens`, `X-Gateway-Cache-Hit-Tokens`, `X-Gateway-Cache-Miss-Tokens`, `X-Gateway-Reasoning-Tokens`, `X-Gateway-Cost-USD`, `X-Gateway-Input-Cost-USD`, `X-Gateway-Output-Cost-USD`, `X-Gateway-Cost-Native` (`1` when the dollar cost is the upstream's own value, `0` when computed from the per-1M pricing), `X-Gateway-Tokens-Per-Second`, `X-Gateway-Upstream-Latency-Ms`, `X-Gateway-Total-Latency-Ms` (full end-to-end gateway time), `X-Gateway-Gateway-Overhead-Ms` (gateway processing minus upstream and queue wait), `X-Gateway-Queue-Wait-Ms` (time waiting on the concurrency semaphore), `X-Gateway-Connect-Ms` (upstream connection establishment), and, when the model's window is known, `X-Gateway-Context-Window` and `X-Gateway-Context-Utilization`. The denied embeddings-disabled path makes no upstream call and returns no usage headers. The AI content correction worker reads `X-Gateway-Cost-USD` and the token headers off its own correction call to accumulate per-user totals (see [AI content correction](/docs/ai-correction.html)). ## Reliability layer `retry_send()` retries on connection errors, timeouts, and upstream 5xx, up to `max_retries`, with a linear backoff of `backoff_ms * attempt`. The `CircuitBreaker` counts consecutive failures: at `threshold` it opens and new requests are rejected immediately with `503` and error type `circuit_open`; after `cooldown_seconds` it half-opens to test recovery and closes on the next success. Setting the threshold to 0 disables the breaker. ## System-message composition Before a chat request is forwarded (and only for `chat/completions`, never embeddings or passthrough), the gateway composes a single system message in the fixed order: operator preamble, then a date line, then the client's original system content. The operator preamble (`gateway_system_preamble`) is prepended on every chat call so all calls carry your preferences. If the client sent no system message at all, the composed content becomes a new leading system message; a blank or whitespace-only preamble is a no-op. The current date is injected as `Current date: DD/MM/YYYY` (EU format) only when the composed text contains no date in any common format already (ISO `YYYY-MM-DD`, `D/M/YYYY`, `DD/MM/YYYY`, `MM/DD/YYYY`, `DD-MM-YYYY`, `DD.MM.YYYY`, and textual months such as `14 June 2026`, `June 14, 2026`, `14 Jun 2026`). The date is intentionally date-only with no time: keeping it stable for the whole day preserves upstream prompt and token caching. ## Vision augmentation Before forwarding a chat request, image blocks are described by the vision model and replaced with a text description, so a text-only upstream still answers questions about images. Descriptions are cached in an LRU keyed by image hash; vision calls are themselves ledgered with `backend="vision"`. If vision is disabled or the key is missing, images become a placeholder note rather than failing the request. ## Data it offers **Live metrics** (in `describe()`, at `GET /admin/services/data`): runtime counters (`requests`, `errors`, `in_flight`, `peak_in_flight`, `vision_calls`, `last_status`, `last_latency_ms`, `pool`, `circuit_open`) and a 24h summary (`requests`, `success_pct`, `error_pct`, `cost_hour`, `cost_24h`, `tokens_24h`, `avg_latency_ms`, `avg_tps`, `peak_concurrency`, `top_model`, `top_caller`). **Ledger** `gateway_usage_ledger`: one row per upstream call with owner, backend, requested and actual model, status, success, error category, the full latency breakdown (`upstream_latency_ms`, `gateway_overhead_ms`, `queue_wait_ms`, `connect_ms`, `total_latency_ms`), token counts (prompt, completion, cache hit and miss, reasoning, total), `tokens_per_second`, context window and utilization, the cost fields (`cost_usd`, `input_cost_usd`, `output_cost_usd`, `native_cost`), request shape (`stream_requested`, `temperature`, `top_p`, `max_tokens`, `has_tools`), and reliability flags (`retries_attempted`, `retry_succeeded`, `circuit_open`). Pruned to `gateway_usage_retention_hours`. **Concurrency samples** `gateway_concurrency_samples`: `in_flight` sampled once per service tick. **Deep analytics** at `GET /admin/ai-usage/data` (`GatewayUsageOut`): per-hour buckets, token and latency percentiles, error breakdowns, cost (this hour, 24h, projected monthly, caching savings, by model, by caller), and behavior. TTFT and inter-token latency are null because the gateway calls the upstream non-streaming. Per-user usage is at `GET /admin/users/{uid}/ai-usage`. It is all rendered on `/admin/ai-usage` by `AiUsageMonitor.js`.