Trust only the upstream X-Gateway-Model header in the AI gateway

An upstream the gateway forwards to may itself emit X-Gateway-* headers
(e.g. another DevPlace-style gateway), which can collide with the ones
about to be built for the response. Only X-Gateway-Model is ever trusted
from upstream and relayed as-is - it is the one field an upstream can
legitimately know better than we do (it may have resolved an alias or
served a different pinned version). Every other header (cost, tokens,
latency, context, app-reference) is always our own measurement and is
never overwritten, since blending in an upstream's own accounting would
corrupt the usage ledger's per-model rollups and the quota math built on
top of it.

usage.upstream_reported_model() extracts that one header defensively
(case-insensitive lookup, rejects anything oversized or containing a
control character) and gateway._apply_served_model() applies it, display-
only, at the tail of every response-header build across chat, streaming,
embeddings, images, and passthrough.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BWJy6PrMMt5hwWxQwia2rd
This commit is contained in:
2026-09-08 03:44:06 +02:00
co-authored by Claude Sonnet 5
parent 7880bf4b31
commit c0e6abb923
6 changed files with 420 additions and 12 deletions
@@ -14,6 +14,15 @@ Every NON-STREAMING gateway response (chat, embeddings, images, passthrough; suc
**A STREAMING chat response (`stream: true`) is the one exception and carries only `X-Gateway-Model`/`X-Gateway-Backend`/`X-App-Reference`** - HTTP headers are sent before the body, and cost/tokens for a streamed call are only known once the stream ends, so they cannot be headers on that same response. The call is still fully metered: the ledger row (including `ttft_ms`/`inter_token_ms`) is written server-side once the stream completes (see "Real upstream streaming" below), and a client that requests `stream_options.include_usage` still gets the upstream's real `usage` object on the final SSE chunk, exactly as the OpenAI streaming API itself works - it is simply not summarized into response headers.
### Upstream header collision policy (`usage.upstream_reported_model`, `gateway._apply_served_model`)
An upstream this gateway forwards to may itself be a DevPlace-style gateway (or any provider that happens to emit the same `X-Gateway-*` header names on its own response), so its raw response headers can collide with the ones we are about to build. **Exactly one upstream header is ever trusted and relayed: `X-Gateway-Model`.** Every other `X-Gateway-*`/`X-App-Reference` header is always our own measurement, computed as if the upstream had sent nothing at all - an upstream cannot be an authority on our own cost table, our own tokenizer counts, our own socket timings, or our own caller attribution.
- **Why only the model name.** It is the one field an upstream can legitimately know better than we do: it may have resolved an alias, redirected to a fallback, or served a different pinned version than what we requested. Cost/tokens/latency/context/app-reference are, by construction, things WE measured about OUR call (our pricing table, our tokenizer expectations, our socket, our caller) - an upstream's own figures for those describe its own accounting, not ours, and blending them in would corrupt `gateway_usage_ledger`'s per-model cost/token rollups and the quota math that reads these exact header names back out (`usage.parse_usage_headers`, consumed by `services/correction.py`/`services/ai_modifier.py`).
- **`usage.upstream_reported_model(headers)`** extracts that one value defensively: it treats the upstream response as untrusted input and never raises. It looks up `X-Gateway-Model` case-insensitively (a direct `.get`, then a manual case-insensitive scan for a plain non-case-insensitive mapping), and rejects anything that is not a non-empty string, exceeds 200 characters, or contains a control character (`\r`/`\n`/`\x00`, a header-injection guard) - any of those makes it return `None`, i.e. "upstream said nothing usable."
- **`gateway._apply_served_model(headers, resp)`** is the one place this is applied: called at the tail of every response-header build (chat non-streaming and streaming, embeddings, images, passthrough), it overwrites `headers["X-Gateway-Model"]` with the upstream's claim ONLY when one was found, and leaves every other header untouched. It never touches the ledger row itself (`GatewayUsageLedger.record`'s `model` field, and therefore `context_utilization`/pricing lookups, stay keyed on the model WE resolved and billed against) - the override is display-only, informing the client which model actually answered without altering what we measured or charged for.
- **Tests:** `tests/unit/services/openai_gateway/usage.py` (`upstream_reported_model` - case-insensitivity, missing/oversized/control-character rejection, non-dict/`None` inputs) and `tests/unit/services/openai_gateway/gateway.py` (`test_upstream_x_gateway_model_header_is_forwarded_*` - chat/streaming/embeddings/images/passthrough all forward an upstream-reported model while every cost/token header stays ours; `test_upstream_x_gateway_model_header_absent_keeps_our_own_model` - no upstream header is a no-op).
## App-reference header (`X-App-Reference`)
Callers may send an optional `X-App-Reference` header to tag gateway calls by application. The value is validated and stored in the `app_reference` column of `gateway_usage_ledger`, surfaced in analytics and admin reporting.
+26 -9
View File
@@ -31,6 +31,7 @@ from devplacepy.services.openai_gateway.usage import (
extract_params,
parse_context_map,
pricing_from_cfg,
upstream_reported_model,
usage_response_headers,
)
from devplacepy.services.openai_gateway.vision import VisionAugmenter, VisionCache
@@ -44,6 +45,19 @@ def _call_failed(resp, exc, timing: dict) -> bool:
return resp is not None and resp.status_code >= 400
def _apply_served_model(headers: dict, resp) -> dict:
# Everything else in `headers` (cost, tokens, latency, context) is our
# own measurement and must never be overwritten by an upstream that
# happens to echo the same X-Gateway-* header names - only the served
# model name is worth trusting from upstream. See usage.upstream_reported_model.
if not headers or resp is None:
return headers
served = upstream_reported_model(getattr(resp, "headers", None))
if served:
headers["X-Gateway-Model"] = served
return headers
def _fallback_headers(cfg: dict, key_field: str) -> dict:
headers = {"Content-Type": "application/json"}
if cfg.get(key_field):
@@ -407,7 +421,7 @@ class GatewayRuntime:
chat_cost = float((row or {}).get("cost_usd") or 0.0)
headers["X-Gateway-Cost-USD"] = f"{chat_cost + vision_cost:.8f}"
headers["X-Gateway-Vision-Cost-USD"] = f"{vision_cost:.8f}"
return headers
return _apply_served_model(headers, resp)
if timing["circuit_open"]:
resp_headers = finalize(503, False, "circuit_open")
@@ -465,11 +479,14 @@ class GatewayRuntime:
log,
),
media_type="text/event-stream",
headers={
"X-Gateway-Model": model,
"X-Gateway-Backend": "chat",
"X-App-Reference": app_reference or "default",
},
headers=_apply_served_model(
{
"X-Gateway-Model": model,
"X-Gateway-Backend": "chat",
"X-App-Reference": app_reference or "default",
},
resp,
),
)
try:
data = resp.json()
@@ -734,7 +751,7 @@ class GatewayRuntime:
chat_cost = float((row or {}).get("cost_usd") or 0.0)
headers["X-Gateway-Cost-USD"] = f"{chat_cost + vision_cost:.8f}"
headers["X-Gateway-Vision-Cost-USD"] = f"{vision_cost:.8f}"
return headers
return _apply_served_model(headers, resp)
if timing["circuit_open"]:
resp_headers = finalize(503, False, "circuit_open")
@@ -939,7 +956,7 @@ class GatewayRuntime:
base["error_category"] = category
base["usage"] = usage
row = self._ledger.record(base, pricing, context_map)
return usage_response_headers(row)
return _apply_served_model(usage_response_headers(row), resp)
if timing["circuit_open"]:
resp_headers = finalize(503, False, "circuit_open")
@@ -1064,7 +1081,7 @@ class GatewayRuntime:
chat_cost = float((row or {}).get("cost_usd") or 0.0)
headers["X-Gateway-Cost-USD"] = f"{chat_cost + vision_cost:.8f}"
headers["X-Gateway-Vision-Cost-USD"] = f"{vision_cost:.8f}"
return headers
return _apply_served_model(headers, resp)
if timing["circuit_open"]:
resp_headers = finalize(503, False, "circuit_open")
@@ -408,6 +408,49 @@ def usage_response_headers(row: Optional[dict]) -> dict:
return headers
UPSTREAM_MODEL_HEADER_MAX_LEN = 200
def upstream_reported_model(headers: Any) -> Optional[str]:
"""Extract a served-model name an upstream (which may itself be another
DevPlace-style gateway) reported on its own X-Gateway-Model response
header, if present and well-formed.
This is the ONE upstream response header this gateway ever trusts and
relays: model name is the single field an upstream can know better than
we do (it may have resolved an alias/fallback on its own side). Every
other X-Gateway-* figure (cost, tokens, latency, context) stays our own
measurement regardless of what an upstream claims - see usage_response_headers.
An upstream response is untrusted input, so this never raises: a missing,
malformed, oversized, or control-character-laced value is treated as
absent rather than propagated or allowed to break our own response.
"""
if not headers:
return None
value = None
try:
value = headers.get("X-Gateway-Model")
except Exception:
value = None
if value is None:
try:
for key, candidate in headers.items():
if isinstance(key, str) and key.lower() == "x-gateway-model":
value = candidate
break
except Exception:
value = None
if not isinstance(value, str):
return None
value = value.strip()
if not value or len(value) > UPSTREAM_MODEL_HEADER_MAX_LEN:
return None
if any(ch in value for ch in ("\r", "\n", "\x00")):
return None
return value
def parse_usage_headers(headers) -> Optional[dict]:
if not headers or "X-Gateway-Cost-USD" not in headers:
return None