This commit is contained in:
2026-07-22 23:55:46 +02:00
parent 34f76aad65
commit 77f043640e
23 changed files with 993 additions and 11 deletions
@@ -61,6 +61,20 @@ The gateway records one row per upstream call (chat, vision, passthrough) and su
**Financial data is admin-only everywhere.** Any monetary figure (USD cost, pricing, spend, limit) is restricted to administrators; members and guests see only the percentage of quota used - this rule is enforced consistently across the profile card, `ai_correction`/`ai_modifier` usage displays, and the Devii cost tools.
## Quota rules (`quota.py`, admin `/admin/gateway` "Quota rules" section)
**This caps `/openai/v1/*` itself, independent of Devii's own daily cap.** Devii's `devii_user_daily_usd`/`devii_guest_daily_usd`/`devii_admin_daily_usd` (documented in `devplacepy/services/devii/CLAUDE.md`) only gate turns that go *through* Devii. A caller hitting the gateway directly with their own `api_key` bypasses that entirely - `quota.py` is the root-level enforcement that closes this, checked in `GatewayService.handle()` right after owner/`app_reference` resolution and before every billed dispatch (chat, embeddings, images, passthrough; `GET /v1/models` is exempt, it makes no upstream call).
**Two layers, same shape as provider/model routing above.** Layer A is five flat `config_fields` on `GatewayService` (`gateway_default_user_daily_usd` $1.00, `gateway_default_admin_daily_usd` $0/unlimited, `gateway_default_guest_daily_usd` $0.05, `gateway_default_internal_daily_usd` $0/unlimited, `gateway_default_key_daily_usd` $0/unlimited - group **Quota**) applied per specific caller (`owner_id`) when no rule matches; internal/key default unlimited so shipping this never starts blocking DevPlace's own news/bots/Devii-guest/correction traffic on the internal key. Layer B is the `gateway_quota_rules` table (`ensure_tables()`, called from `init_db()` alongside `routing.ensure_tables()`; hard CRUD, not in `SOFT_DELETE_TABLES`, cross-worker cache-invalidated under the `"gateway_quota"` name): each row scopes by **any combination** of `owner_kind` (internal/key/user/admin/anonymous - DevPlace's only "roles" here), a specific `owner_id`, and `app_reference` (the `X-App-Reference` label), each nullable = wildcard; a `QuotaRuleIn` Pydantic validator rejects a rule with all three blank (that belongs in Layer A). `quota.resolve(owner_kind, owner_id, app_reference, cfg)` gathers every active rule whose non-null dimensions all equal the request, picks the one with the most non-null dimensions (ties broken toward the smaller limit, unlimited `0` never wins a tie against a finite cap), and returns `(limit_usd, scope, rule)` where `scope` is the exact `(owner_kind, owner_id, app_reference)` triple - each possibly `None` - that spend must be summed over. Layer A is internally just the maximally-specific implicit scope `(owner_kind, owner_id, None)`, so one code path (`quota.spent_24h(*scope)`, a plain `SUM(cost_usd)` over `gateway_usage_ledger` filtered by whichever scope dimensions are non-null) serves both layers.
**A wildcard dimension means a shared pool, by design.** A rule scoped only by `app_reference` caps that app's combined spend across every caller using it; a rule scoped only by `owner_kind` caps that whole role's combined spend. Pin `owner_id` to get a true per-caller cap (the Layer A default's own behavior). `anonymous`/`internal`/`key` owner_ids are already fixed constants (`"anonymous"`/`"devii"`/`"access"`, from `resolve_owner()`), not per-caller identities, so any cap on those kinds is inherently pooled - there is no per-guest identity at this layer (unlike Devii's own guest-cookie-scoped ledger).
**No lock, no hold, bounded overshoot by design - this is deliberate, not an oversight.** Cost is only known after the upstream call returns, so a true atomic pre-authorization would need a reserve-then-reconcile ("hold") mechanism, and a bug in releasing a hold is exactly the kind of thing that gets a caller stuck forever. Instead this mirrors Devii's own already-shipped mechanism exactly: read the 24h sum, compare, `raise HTTPException(429, ...)` if already at/over - a single `SELECT` and a conditional raise, nothing held, nothing to leak, structurally impossible to deadlock. The tradeoff is a small, bounded overshoot (at most a few concurrent in-flight calls' worth of cost past the cap before the next request sees the updated sum and blocks) - acceptable and industry-standard for a cost whose exact size isn't known until the call finishes, and it is the property actually being enforced: once tripped, every subsequent separate request stays blocked until the 24h window rolls off or an admin adjusts the rule.
**429 body never carries a dollar figure**, admin or not (`{"detail": "AI gateway daily quota exceeded"}`) - mirrors Devii's own over-limit WS message, which likewise never states a number. The admin-only services log line and the `ai.quota.exceeded` audit row (`GatewayService._audit_quota_exceeded`, reusing `usage.audit_actor_for`) do carry the spend/limit/matched-rule-uid, since those are admin-only surfaces.
**CRUD.** Admin JSON at `/admin/gateway/quota-rules` (`routers/admin/gateway_configs.py`, list returns each rule's live `spent_24h_usd` plus the Layer A defaults for context), audited `gateway.quota_rule.update`/`gateway.quota_rule.delete` (category `ai`, both already in `events.md`), rendered in the **Quota rules** section of `/admin/gateway` (`GatewayAdmin.js`, mirrors the providers/models CRUD tables). Devii tools `gateway_quota_rules`/`gateway_quota_rule_set`/`gateway_quota_rule_delete` (`requires_admin=True`, delete is `CONFIRM_REQUIRED`) proxy the same endpoints via `handler="http"`, same as the provider/model tools. CLI: `devplace gateway quota list|set|delete`.
## Image generation
`POST /openai/v1/images/generations` exposes an OpenAI-compatible image-generation endpoint. Clients send the generic model `molodetz-img-small` (`config.INTERNAL_IMAGE_MODEL`), which `handle_images` remaps to `gateway_image_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty or `molodetz-img`). It defaults to OpenRouter's `black-forest-labs/flux-1.1-pro` at `https://openrouter.ai/api/v1/images/generations` (`config.IMAGE_*_DEFAULT`, $0.04 per image fallback). `handle_images` mirrors `handle_embeddings`: build the payload, forward via `_send`, and record one ledger row. The config fields are the **Images** group (`gateway_image_enabled` default on, `gateway_image_url`, `gateway_image_model`, `gateway_image_key`) plus the Pricing-group `gateway_image_price_per_call`. `effective_config()` falls the image key back to `gateway_api_key` then `OPENROUTER_API_KEY`. Usage is recorded with **`backend="image"`**; `usage.compute_cost` adds an `image` branch (flat per-call, native OpenRouter `cost` still preferred via `extract_image_usage`). `routing.image_overlay` resolves per-route provider/url/key and uses `price_input_per_m` as the per-image price. `routing.seed_default_image_routes()` (from `migrate_ai_gateway_settings`) idempotently seeds `molodetz-img-small` -> Flux on the `openrouter` provider when `OPENROUTER_API_KEY` is set. When `gateway_image_enabled` is off the endpoint returns 503 with no ledger row.
@@ -11,6 +11,7 @@ import httpx
from fastapi.responses import JSONResponse, Response, StreamingResponse
from devplacepy import stealth
from devplacepy.services.background import background
from devplacepy.services.openai_gateway import config
from devplacepy.services.openai_gateway.reliability import CircuitBreaker, retry_send
from devplacepy.services.openai_gateway.routing import (
@@ -33,6 +34,19 @@ from devplacepy.services.openai_gateway.vision import VisionAugmenter, VisionCac
logger = logging.getLogger(__name__)
def _notify_gateway_status(is_open: bool) -> None:
from devplacepy.database import get_table
from devplacepy.utils import create_notification
message = (
"AI gateway circuit breaker opened - upstream calls are being rejected"
if is_open
else "AI gateway circuit breaker closed - upstream calls have resumed"
)
for admin in get_table("users").find(role="Admin"):
create_notification(admin["uid"], "system", message, "gateway", "/admin/services/openai")
def _fake_stream(data: dict, model: str, include_usage: bool = False):
chunk_id = data.get("id", f"chatcmpl-{uuid.uuid4().hex[:12]}")
created = data.get("created", int(time.time()))
@@ -219,15 +233,24 @@ class GatewayRuntime:
self.last_latency_ms = int(timing["upstream_latency_ms"])
if exc is not None:
self.errors += 1
was_open = self._breaker.is_open
self._breaker.record_failure()
if not was_open and self._breaker.is_open:
background.submit(_notify_gateway_status, True)
log(f"{method} {url} connection failed after {attempts} attempt(s): {exc}")
return None, exc, timing
self.last_status = resp.status_code
if resp.status_code >= 500:
self.errors += 1
was_open = self._breaker.is_open
self._breaker.record_failure()
if not was_open and self._breaker.is_open:
background.submit(_notify_gateway_status, True)
else:
was_open = self._breaker.is_open
self._breaker.record_success()
if was_open and not self._breaker.is_open:
background.submit(_notify_gateway_status, False)
timing["retry_succeeded"] = attempts > 1
return resp, None, timing
+317
View File
@@ -0,0 +1,317 @@
# retoor <retoor@molodetz.nl>
from __future__ import annotations
import logging
import re
import uuid
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Optional
from pydantic import BaseModel, Field, field_validator, model_validator
from devplacepy.database import bump_cache_version, db, get_table, sync_local_cache
logger = logging.getLogger(__name__)
RULES_TABLE = "gateway_quota_rules"
CACHE_NAME = "gateway_quota"
APP_REFERENCE_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]{1,30}$")
OWNER_KINDS = ("internal", "key", "user", "admin", "anonymous")
FIELD_DEFAULT_USER = "gateway_default_user_daily_usd"
FIELD_DEFAULT_ADMIN = "gateway_default_admin_daily_usd"
FIELD_DEFAULT_GUEST = "gateway_default_guest_daily_usd"
FIELD_DEFAULT_INTERNAL = "gateway_default_internal_daily_usd"
FIELD_DEFAULT_KEY = "gateway_default_key_daily_usd"
_DEFAULT_FIELD_BY_KIND = {
"user": FIELD_DEFAULT_USER,
"admin": FIELD_DEFAULT_ADMIN,
"anonymous": FIELD_DEFAULT_GUEST,
"internal": FIELD_DEFAULT_INTERNAL,
"key": FIELD_DEFAULT_KEY,
}
_QUOTA_CACHE: dict = {}
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def ensure_tables() -> None:
db.query(
"CREATE TABLE IF NOT EXISTS "
+ RULES_TABLE
+ " (id INTEGER PRIMARY KEY, uid TEXT, owner_kind TEXT, owner_id TEXT, "
"app_reference TEXT, limit_usd REAL DEFAULT 0, is_active INTEGER DEFAULT 1, "
"label TEXT, created_by TEXT, created_at TEXT, updated_at TEXT)"
)
try:
db.query(
"CREATE UNIQUE INDEX IF NOT EXISTS idx_gateway_quota_rules_uid ON "
+ RULES_TABLE
+ " (uid)"
)
db.query(
"CREATE INDEX IF NOT EXISTS idx_gateway_quota_rules_lookup ON "
+ RULES_TABLE
+ " (owner_kind, owner_id, app_reference)"
)
except Exception as exc:
logger.warning("gateway quota rule index creation failed: %s", exc)
class QuotaRuleIn(BaseModel):
owner_kind: Optional[str] = None
owner_id: Optional[str] = Field(default=None, max_length=64)
app_reference: Optional[str] = Field(default=None, max_length=30)
limit_usd: float = Field(default=0.0, ge=0)
is_active: bool = True
label: str = Field(default="", max_length=200)
@field_validator("owner_kind")
@classmethod
def _clean_owner_kind(cls, value: Optional[str]) -> Optional[str]:
if not value:
return None
value = value.strip().lower()
if value not in OWNER_KINDS:
raise ValueError(f"owner_kind must be one of {', '.join(OWNER_KINDS)}")
return value
@field_validator("owner_id")
@classmethod
def _clean_owner_id(cls, value: Optional[str]) -> Optional[str]:
if not value:
return None
return value.strip()
@field_validator("app_reference")
@classmethod
def _clean_app_reference(cls, value: Optional[str]) -> Optional[str]:
if not value:
return None
value = value.strip()
if not APP_REFERENCE_PATTERN.match(value):
raise ValueError("app_reference must match ^[a-zA-Z0-9_.-]{1,30}$")
return value
@field_validator("label")
@classmethod
def _clean_label(cls, value: str) -> str:
return (value or "").strip()
@model_validator(mode="after")
def _require_a_dimension(self) -> "QuotaRuleIn":
if self.owner_kind is None and self.owner_id is None and self.app_reference is None:
raise ValueError(
"At least one of owner_kind, owner_id, or app_reference is required - "
"an unscoped cap belongs in the global default fields, not a rule"
)
return self
@dataclass(frozen=True)
class QuotaRule:
uid: str
owner_kind: Optional[str]
owner_id: Optional[str]
app_reference: Optional[str]
limit_usd: float
is_active: bool
label: str
created_by: str
created_at: str
updated_at: str
@property
def specificity(self) -> int:
return sum(
1 for v in (self.owner_kind, self.owner_id, self.app_reference) if v is not None
)
def matches(self, owner_kind: str, owner_id: str, app_reference: str) -> bool:
if not self.is_active:
return False
if self.owner_kind is not None and self.owner_kind != owner_kind:
return False
if self.owner_id is not None and self.owner_id != owner_id:
return False
if self.app_reference is not None and self.app_reference != app_reference:
return False
return True
def as_dict(self) -> dict:
return {
"uid": self.uid,
"owner_kind": self.owner_kind,
"owner_id": self.owner_id,
"app_reference": self.app_reference,
"limit_usd": self.limit_usd,
"is_active": self.is_active,
"label": self.label,
"created_by": self.created_by,
"created_at": self.created_at,
"updated_at": self.updated_at,
"specificity": self.specificity,
}
def _row_to_rule(row: dict) -> QuotaRule:
return QuotaRule(
uid=str(row.get("uid") or ""),
owner_kind=row.get("owner_kind") or None,
owner_id=row.get("owner_id") or None,
app_reference=row.get("app_reference") or None,
limit_usd=float(row.get("limit_usd") or 0.0),
is_active=bool(row.get("is_active", 1)),
label=str(row.get("label") or ""),
created_by=str(row.get("created_by") or ""),
created_at=str(row.get("created_at") or ""),
updated_at=str(row.get("updated_at") or ""),
)
def _load() -> list[QuotaRule]:
sync_local_cache(CACHE_NAME, _QUOTA_CACHE)
if "rules" not in _QUOTA_CACHE:
rules: list[QuotaRule] = []
try:
if RULES_TABLE in db.tables:
for row in get_table(RULES_TABLE).all():
if row.get("uid"):
rules.append(_row_to_rule(row))
except Exception as exc:
logger.warning("gateway quota rule load failed: %s", exc)
_QUOTA_CACHE["rules"] = rules
return _QUOTA_CACHE["rules"]
class QuotaRuleStore:
def list(self) -> list[dict]:
rules = sorted(_load(), key=lambda r: (-r.specificity, r.created_at))
return [r.as_dict() for r in rules]
def get(self, uid: str) -> Optional[QuotaRule]:
if not uid:
return None
for rule in _load():
if rule.uid == uid:
return rule
return None
def count(self) -> int:
return len(_load())
def set(
self, payload: QuotaRuleIn, *, uid: Optional[str] = None, created_by: str = ""
) -> dict:
ensure_tables()
table = get_table(RULES_TABLE)
existing = table.find_one(uid=uid) if uid else None
record: dict = {
"owner_kind": payload.owner_kind,
"owner_id": payload.owner_id,
"app_reference": payload.app_reference,
"limit_usd": payload.limit_usd,
"is_active": 1 if payload.is_active else 0,
"label": payload.label,
"updated_at": _now(),
}
if existing:
record["uid"] = existing["uid"]
record["created_by"] = existing.get("created_by") or created_by
record["created_at"] = existing.get("created_at") or _now()
table.update({**record, "id": existing["id"]}, ["id"])
else:
record["uid"] = uid or uuid.uuid4().hex
record["created_by"] = created_by
record["created_at"] = _now()
table.insert(record)
bump_cache_version(CACHE_NAME)
_QUOTA_CACHE.clear()
saved = self.get(record["uid"])
return saved.as_dict() if saved else record
def remove(self, uid: str) -> bool:
uid = (uid or "").strip()
if not uid or RULES_TABLE not in db.tables:
return False
removed = int(get_table(RULES_TABLE).delete(uid=uid))
if removed:
bump_cache_version(CACHE_NAME)
_QUOTA_CACHE.clear()
return bool(removed)
quota_rule_store = QuotaRuleStore()
def default_limit(owner_kind: str, cfg: dict) -> float:
field = _DEFAULT_FIELD_BY_KIND.get(owner_kind, FIELD_DEFAULT_USER)
return float(cfg.get(field, 0.0) or 0.0)
def resolve(
owner_kind: str, owner_id: str, app_reference: str, cfg: dict
) -> tuple[float, tuple[Optional[str], Optional[str], Optional[str]], Optional[QuotaRule]]:
matches = [r for r in _load() if r.matches(owner_kind, owner_id, app_reference)]
if matches:
def _sort_key(rule: QuotaRule):
tie = float("inf") if rule.limit_usd == 0 else rule.limit_usd
return (-rule.specificity, tie)
best = sorted(matches, key=_sort_key)[0]
return best.limit_usd, (best.owner_kind, best.owner_id, best.app_reference), best
return default_limit(owner_kind, cfg), (owner_kind, owner_id, None), None
def resolve_for_owner(
owner_kind: str, owner_id: str, cfg: dict
) -> tuple[float, tuple[Optional[str], Optional[str], None], Optional[QuotaRule]]:
matches = [
r
for r in _load()
if r.app_reference is None and r.matches(owner_kind, owner_id, "")
]
if matches:
def _sort_key(rule: QuotaRule):
tie = float("inf") if rule.limit_usd == 0 else rule.limit_usd
return (-rule.specificity, tie)
best = sorted(matches, key=_sort_key)[0]
return best.limit_usd, (best.owner_kind, best.owner_id, None), best
return default_limit(owner_kind, cfg), (owner_kind, owner_id, None), None
def spent_24h(
owner_kind: Optional[str], owner_id: Optional[str], app_reference: Optional[str]
) -> float:
from devplacepy.services.openai_gateway.usage import GATEWAY_LEDGER
if GATEWAY_LEDGER not in db.tables:
return 0.0
clauses = ["created_at >= :cutoff"]
params: dict = {
"cutoff": (datetime.now(timezone.utc) - timedelta(hours=24)).isoformat()
}
if owner_kind is not None:
clauses.append("owner_kind = :owner_kind")
params["owner_kind"] = owner_kind
if owner_id is not None:
clauses.append("owner_id = :owner_id")
params["owner_id"] = owner_id
if app_reference is not None:
clauses.append("app_reference = :app_reference")
params["app_reference"] = app_reference
where = " AND ".join(clauses)
rows = list(
db.query(
f"SELECT COALESCE(SUM(cost_usd), 0) AS spent FROM {GATEWAY_LEDGER} WHERE {where}",
**params,
)
)
return float(rows[0].get("spent") or 0.0) if rows else 0.0
+95 -1
View File
@@ -11,7 +11,7 @@ from fastapi.responses import JSONResponse
from devplacepy.database import get_int_setting
from devplacepy.services.base import BaseService, ConfigField
from devplacepy.services.openai_gateway import config
from devplacepy.services.openai_gateway import config, quota
from devplacepy.services.openai_gateway.analytics import summary_metrics
from devplacepy.services.openai_gateway.gateway import GatewayRuntime
from devplacepy.services.openai_gateway.routing import model_store
@@ -262,6 +262,58 @@ class GatewayService(BaseService):
"with this key. Clear it and restart to rotate.",
group="Access",
),
ConfigField(
quota.FIELD_DEFAULT_USER,
"Default per-user daily cap ($)",
type="float",
default=1.0,
minimum=0,
help="Rolling 24h cap applied to a signed-in member with no matching quota rule. "
"0 = unlimited. Overridable per user/app-reference on /admin/gateway.",
group="Quota",
),
ConfigField(
quota.FIELD_DEFAULT_ADMIN,
"Default per-admin daily cap ($)",
type="float",
default=0.0,
minimum=0,
help="Rolling 24h cap applied to an administrator with no matching quota rule. "
"0 = unlimited (the default - admins are exempt unless a rule says otherwise).",
group="Quota",
),
ConfigField(
quota.FIELD_DEFAULT_GUEST,
"Default per-guest daily cap ($)",
type="float",
default=0.05,
minimum=0,
help="Rolling 24h cap applied to an unauthenticated caller with no matching quota "
"rule (only reachable when authentication is not required). 0 = unlimited.",
group="Quota",
),
ConfigField(
quota.FIELD_DEFAULT_INTERNAL,
"Default per-internal-key daily cap ($)",
type="float",
default=0.0,
minimum=0,
help="Rolling 24h cap applied to calls authenticated with the auto-generated internal "
"key (DevPlace's own services: news, bots, Devii guests, correction/modifier). "
"0 = unlimited (the default - do not cap this without a matching quota rule, or "
"internal platform traffic will start failing).",
group="Quota",
),
ConfigField(
quota.FIELD_DEFAULT_KEY,
"Default per-access-key daily cap ($)",
type="float",
default=0.0,
minimum=0,
help="Rolling 24h cap applied to calls authenticated with the static access key. "
"0 = unlimited by default (it is an admin-provisioned trusted secret).",
group="Quota",
),
ConfigField(
"gateway_price_cache_hit_per_m",
"Chat price cache-hit / 1M ($)",
@@ -463,6 +515,37 @@ class GatewayService(BaseService):
return (kind, user.get("uid") or "unknown")
return ("anonymous", "anonymous")
def _audit_quota_exceeded(
self,
owner_kind: str,
owner_id: str,
app_reference: str,
spent: float,
limit: float,
rule,
) -> None:
from devplacepy.services.audit import record as audit
from devplacepy.services.openai_gateway.usage import audit_actor_for
actor_kind, actor_uid, actor_role = audit_actor_for(owner_kind, owner_id)
audit.record_system(
"ai.quota.exceeded",
actor_kind=actor_kind,
actor_uid=actor_uid,
actor_role=actor_role,
origin="api",
result="denied",
summary=f"AI gateway call by {owner_kind}/{owner_id} blocked - 24h quota reached",
metadata={
"owner_kind": owner_kind,
"owner_id": owner_id,
"app_reference": app_reference,
"spent_usd": round(spent, 6),
"limit_usd": limit,
"rule_uid": rule.uid if rule else None,
},
)
def _models_response(self) -> JSONResponse:
created = int(time.time())
seen: set = set()
@@ -501,6 +584,17 @@ class GatewayService(BaseService):
)
if subpath == "models" and request.method == "GET":
return self._models_response()
limit, scope, rule = quota.resolve(owner[0], owner[1], app_reference, cfg)
if limit > 0:
spent = quota.spent_24h(*scope)
if spent >= limit:
self.log(
f"Rejected {owner[0]}:{owner[1]} app={app_reference}: "
f"quota exceeded (${spent:.4f}/${limit:.4f}"
f"{' rule ' + rule.uid if rule else ' default'})"
)
self._audit_quota_exceeded(owner[0], owner[1], app_reference, spent, limit, rule)
raise HTTPException(status_code=429, detail="AI gateway daily quota exceeded")
if subpath == "chat/completions" and request.method == "POST":
try:
body = await request.json()