forked from retoor/devplacepy
yex
This commit is contained in:
@@ -61,6 +61,20 @@ The gateway records one row per upstream call (chat, vision, passthrough) and su
|
||||
|
||||
**Financial data is admin-only everywhere.** Any monetary figure (USD cost, pricing, spend, limit) is restricted to administrators; members and guests see only the percentage of quota used - this rule is enforced consistently across the profile card, `ai_correction`/`ai_modifier` usage displays, and the Devii cost tools.
|
||||
|
||||
## Quota rules (`quota.py`, admin `/admin/gateway` "Quota rules" section)
|
||||
|
||||
**This caps `/openai/v1/*` itself, independent of Devii's own daily cap.** Devii's `devii_user_daily_usd`/`devii_guest_daily_usd`/`devii_admin_daily_usd` (documented in `devplacepy/services/devii/CLAUDE.md`) only gate turns that go *through* Devii. A caller hitting the gateway directly with their own `api_key` bypasses that entirely - `quota.py` is the root-level enforcement that closes this, checked in `GatewayService.handle()` right after owner/`app_reference` resolution and before every billed dispatch (chat, embeddings, images, passthrough; `GET /v1/models` is exempt, it makes no upstream call).
|
||||
|
||||
**Two layers, same shape as provider/model routing above.** Layer A is five flat `config_fields` on `GatewayService` (`gateway_default_user_daily_usd` $1.00, `gateway_default_admin_daily_usd` $0/unlimited, `gateway_default_guest_daily_usd` $0.05, `gateway_default_internal_daily_usd` $0/unlimited, `gateway_default_key_daily_usd` $0/unlimited - group **Quota**) applied per specific caller (`owner_id`) when no rule matches; internal/key default unlimited so shipping this never starts blocking DevPlace's own news/bots/Devii-guest/correction traffic on the internal key. Layer B is the `gateway_quota_rules` table (`ensure_tables()`, called from `init_db()` alongside `routing.ensure_tables()`; hard CRUD, not in `SOFT_DELETE_TABLES`, cross-worker cache-invalidated under the `"gateway_quota"` name): each row scopes by **any combination** of `owner_kind` (internal/key/user/admin/anonymous - DevPlace's only "roles" here), a specific `owner_id`, and `app_reference` (the `X-App-Reference` label), each nullable = wildcard; a `QuotaRuleIn` Pydantic validator rejects a rule with all three blank (that belongs in Layer A). `quota.resolve(owner_kind, owner_id, app_reference, cfg)` gathers every active rule whose non-null dimensions all equal the request, picks the one with the most non-null dimensions (ties broken toward the smaller limit, unlimited `0` never wins a tie against a finite cap), and returns `(limit_usd, scope, rule)` where `scope` is the exact `(owner_kind, owner_id, app_reference)` triple - each possibly `None` - that spend must be summed over. Layer A is internally just the maximally-specific implicit scope `(owner_kind, owner_id, None)`, so one code path (`quota.spent_24h(*scope)`, a plain `SUM(cost_usd)` over `gateway_usage_ledger` filtered by whichever scope dimensions are non-null) serves both layers.
|
||||
|
||||
**A wildcard dimension means a shared pool, by design.** A rule scoped only by `app_reference` caps that app's combined spend across every caller using it; a rule scoped only by `owner_kind` caps that whole role's combined spend. Pin `owner_id` to get a true per-caller cap (the Layer A default's own behavior). `anonymous`/`internal`/`key` owner_ids are already fixed constants (`"anonymous"`/`"devii"`/`"access"`, from `resolve_owner()`), not per-caller identities, so any cap on those kinds is inherently pooled - there is no per-guest identity at this layer (unlike Devii's own guest-cookie-scoped ledger).
|
||||
|
||||
**No lock, no hold, bounded overshoot by design - this is deliberate, not an oversight.** Cost is only known after the upstream call returns, so a true atomic pre-authorization would need a reserve-then-reconcile ("hold") mechanism, and a bug in releasing a hold is exactly the kind of thing that gets a caller stuck forever. Instead this mirrors Devii's own already-shipped mechanism exactly: read the 24h sum, compare, `raise HTTPException(429, ...)` if already at/over - a single `SELECT` and a conditional raise, nothing held, nothing to leak, structurally impossible to deadlock. The tradeoff is a small, bounded overshoot (at most a few concurrent in-flight calls' worth of cost past the cap before the next request sees the updated sum and blocks) - acceptable and industry-standard for a cost whose exact size isn't known until the call finishes, and it is the property actually being enforced: once tripped, every subsequent separate request stays blocked until the 24h window rolls off or an admin adjusts the rule.
|
||||
|
||||
**429 body never carries a dollar figure**, admin or not (`{"detail": "AI gateway daily quota exceeded"}`) - mirrors Devii's own over-limit WS message, which likewise never states a number. The admin-only services log line and the `ai.quota.exceeded` audit row (`GatewayService._audit_quota_exceeded`, reusing `usage.audit_actor_for`) do carry the spend/limit/matched-rule-uid, since those are admin-only surfaces.
|
||||
|
||||
**CRUD.** Admin JSON at `/admin/gateway/quota-rules` (`routers/admin/gateway_configs.py`, list returns each rule's live `spent_24h_usd` plus the Layer A defaults for context), audited `gateway.quota_rule.update`/`gateway.quota_rule.delete` (category `ai`, both already in `events.md`), rendered in the **Quota rules** section of `/admin/gateway` (`GatewayAdmin.js`, mirrors the providers/models CRUD tables). Devii tools `gateway_quota_rules`/`gateway_quota_rule_set`/`gateway_quota_rule_delete` (`requires_admin=True`, delete is `CONFIRM_REQUIRED`) proxy the same endpoints via `handler="http"`, same as the provider/model tools. CLI: `devplace gateway quota list|set|delete`.
|
||||
|
||||
## Image generation
|
||||
|
||||
`POST /openai/v1/images/generations` exposes an OpenAI-compatible image-generation endpoint. Clients send the generic model `molodetz-img-small` (`config.INTERNAL_IMAGE_MODEL`), which `handle_images` remaps to `gateway_image_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty or `molodetz-img`). It defaults to OpenRouter's `black-forest-labs/flux-1.1-pro` at `https://openrouter.ai/api/v1/images/generations` (`config.IMAGE_*_DEFAULT`, $0.04 per image fallback). `handle_images` mirrors `handle_embeddings`: build the payload, forward via `_send`, and record one ledger row. The config fields are the **Images** group (`gateway_image_enabled` default on, `gateway_image_url`, `gateway_image_model`, `gateway_image_key`) plus the Pricing-group `gateway_image_price_per_call`. `effective_config()` falls the image key back to `gateway_api_key` then `OPENROUTER_API_KEY`. Usage is recorded with **`backend="image"`**; `usage.compute_cost` adds an `image` branch (flat per-call, native OpenRouter `cost` still preferred via `extract_image_usage`). `routing.image_overlay` resolves per-route provider/url/key and uses `price_input_per_m` as the per-image price. `routing.seed_default_image_routes()` (from `migrate_ai_gateway_settings`) idempotently seeds `molodetz-img-small` -> Flux on the `openrouter` provider when `OPENROUTER_API_KEY` is set. When `gateway_image_enabled` is off the endpoint returns 503 with no ledger row.
|
||||
|
||||
@@ -11,6 +11,7 @@ import httpx
|
||||
from fastapi.responses import JSONResponse, Response, StreamingResponse
|
||||
|
||||
from devplacepy import stealth
|
||||
from devplacepy.services.background import background
|
||||
from devplacepy.services.openai_gateway import config
|
||||
from devplacepy.services.openai_gateway.reliability import CircuitBreaker, retry_send
|
||||
from devplacepy.services.openai_gateway.routing import (
|
||||
@@ -33,6 +34,19 @@ from devplacepy.services.openai_gateway.vision import VisionAugmenter, VisionCac
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _notify_gateway_status(is_open: bool) -> None:
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.utils import create_notification
|
||||
|
||||
message = (
|
||||
"AI gateway circuit breaker opened - upstream calls are being rejected"
|
||||
if is_open
|
||||
else "AI gateway circuit breaker closed - upstream calls have resumed"
|
||||
)
|
||||
for admin in get_table("users").find(role="Admin"):
|
||||
create_notification(admin["uid"], "system", message, "gateway", "/admin/services/openai")
|
||||
|
||||
|
||||
def _fake_stream(data: dict, model: str, include_usage: bool = False):
|
||||
chunk_id = data.get("id", f"chatcmpl-{uuid.uuid4().hex[:12]}")
|
||||
created = data.get("created", int(time.time()))
|
||||
@@ -219,15 +233,24 @@ class GatewayRuntime:
|
||||
self.last_latency_ms = int(timing["upstream_latency_ms"])
|
||||
if exc is not None:
|
||||
self.errors += 1
|
||||
was_open = self._breaker.is_open
|
||||
self._breaker.record_failure()
|
||||
if not was_open and self._breaker.is_open:
|
||||
background.submit(_notify_gateway_status, True)
|
||||
log(f"{method} {url} connection failed after {attempts} attempt(s): {exc}")
|
||||
return None, exc, timing
|
||||
self.last_status = resp.status_code
|
||||
if resp.status_code >= 500:
|
||||
self.errors += 1
|
||||
was_open = self._breaker.is_open
|
||||
self._breaker.record_failure()
|
||||
if not was_open and self._breaker.is_open:
|
||||
background.submit(_notify_gateway_status, True)
|
||||
else:
|
||||
was_open = self._breaker.is_open
|
||||
self._breaker.record_success()
|
||||
if was_open and not self._breaker.is_open:
|
||||
background.submit(_notify_gateway_status, False)
|
||||
timing["retry_succeeded"] = attempts > 1
|
||||
return resp, None, timing
|
||||
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
|
||||
from devplacepy.database import bump_cache_version, db, get_table, sync_local_cache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
RULES_TABLE = "gateway_quota_rules"
|
||||
CACHE_NAME = "gateway_quota"
|
||||
APP_REFERENCE_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]{1,30}$")
|
||||
OWNER_KINDS = ("internal", "key", "user", "admin", "anonymous")
|
||||
|
||||
FIELD_DEFAULT_USER = "gateway_default_user_daily_usd"
|
||||
FIELD_DEFAULT_ADMIN = "gateway_default_admin_daily_usd"
|
||||
FIELD_DEFAULT_GUEST = "gateway_default_guest_daily_usd"
|
||||
FIELD_DEFAULT_INTERNAL = "gateway_default_internal_daily_usd"
|
||||
FIELD_DEFAULT_KEY = "gateway_default_key_daily_usd"
|
||||
|
||||
_DEFAULT_FIELD_BY_KIND = {
|
||||
"user": FIELD_DEFAULT_USER,
|
||||
"admin": FIELD_DEFAULT_ADMIN,
|
||||
"anonymous": FIELD_DEFAULT_GUEST,
|
||||
"internal": FIELD_DEFAULT_INTERNAL,
|
||||
"key": FIELD_DEFAULT_KEY,
|
||||
}
|
||||
|
||||
_QUOTA_CACHE: dict = {}
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def ensure_tables() -> None:
|
||||
db.query(
|
||||
"CREATE TABLE IF NOT EXISTS "
|
||||
+ RULES_TABLE
|
||||
+ " (id INTEGER PRIMARY KEY, uid TEXT, owner_kind TEXT, owner_id TEXT, "
|
||||
"app_reference TEXT, limit_usd REAL DEFAULT 0, is_active INTEGER DEFAULT 1, "
|
||||
"label TEXT, created_by TEXT, created_at TEXT, updated_at TEXT)"
|
||||
)
|
||||
try:
|
||||
db.query(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS idx_gateway_quota_rules_uid ON "
|
||||
+ RULES_TABLE
|
||||
+ " (uid)"
|
||||
)
|
||||
db.query(
|
||||
"CREATE INDEX IF NOT EXISTS idx_gateway_quota_rules_lookup ON "
|
||||
+ RULES_TABLE
|
||||
+ " (owner_kind, owner_id, app_reference)"
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("gateway quota rule index creation failed: %s", exc)
|
||||
|
||||
|
||||
class QuotaRuleIn(BaseModel):
|
||||
owner_kind: Optional[str] = None
|
||||
owner_id: Optional[str] = Field(default=None, max_length=64)
|
||||
app_reference: Optional[str] = Field(default=None, max_length=30)
|
||||
limit_usd: float = Field(default=0.0, ge=0)
|
||||
is_active: bool = True
|
||||
label: str = Field(default="", max_length=200)
|
||||
|
||||
@field_validator("owner_kind")
|
||||
@classmethod
|
||||
def _clean_owner_kind(cls, value: Optional[str]) -> Optional[str]:
|
||||
if not value:
|
||||
return None
|
||||
value = value.strip().lower()
|
||||
if value not in OWNER_KINDS:
|
||||
raise ValueError(f"owner_kind must be one of {', '.join(OWNER_KINDS)}")
|
||||
return value
|
||||
|
||||
@field_validator("owner_id")
|
||||
@classmethod
|
||||
def _clean_owner_id(cls, value: Optional[str]) -> Optional[str]:
|
||||
if not value:
|
||||
return None
|
||||
return value.strip()
|
||||
|
||||
@field_validator("app_reference")
|
||||
@classmethod
|
||||
def _clean_app_reference(cls, value: Optional[str]) -> Optional[str]:
|
||||
if not value:
|
||||
return None
|
||||
value = value.strip()
|
||||
if not APP_REFERENCE_PATTERN.match(value):
|
||||
raise ValueError("app_reference must match ^[a-zA-Z0-9_.-]{1,30}$")
|
||||
return value
|
||||
|
||||
@field_validator("label")
|
||||
@classmethod
|
||||
def _clean_label(cls, value: str) -> str:
|
||||
return (value or "").strip()
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _require_a_dimension(self) -> "QuotaRuleIn":
|
||||
if self.owner_kind is None and self.owner_id is None and self.app_reference is None:
|
||||
raise ValueError(
|
||||
"At least one of owner_kind, owner_id, or app_reference is required - "
|
||||
"an unscoped cap belongs in the global default fields, not a rule"
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QuotaRule:
|
||||
uid: str
|
||||
owner_kind: Optional[str]
|
||||
owner_id: Optional[str]
|
||||
app_reference: Optional[str]
|
||||
limit_usd: float
|
||||
is_active: bool
|
||||
label: str
|
||||
created_by: str
|
||||
created_at: str
|
||||
updated_at: str
|
||||
|
||||
@property
|
||||
def specificity(self) -> int:
|
||||
return sum(
|
||||
1 for v in (self.owner_kind, self.owner_id, self.app_reference) if v is not None
|
||||
)
|
||||
|
||||
def matches(self, owner_kind: str, owner_id: str, app_reference: str) -> bool:
|
||||
if not self.is_active:
|
||||
return False
|
||||
if self.owner_kind is not None and self.owner_kind != owner_kind:
|
||||
return False
|
||||
if self.owner_id is not None and self.owner_id != owner_id:
|
||||
return False
|
||||
if self.app_reference is not None and self.app_reference != app_reference:
|
||||
return False
|
||||
return True
|
||||
|
||||
def as_dict(self) -> dict:
|
||||
return {
|
||||
"uid": self.uid,
|
||||
"owner_kind": self.owner_kind,
|
||||
"owner_id": self.owner_id,
|
||||
"app_reference": self.app_reference,
|
||||
"limit_usd": self.limit_usd,
|
||||
"is_active": self.is_active,
|
||||
"label": self.label,
|
||||
"created_by": self.created_by,
|
||||
"created_at": self.created_at,
|
||||
"updated_at": self.updated_at,
|
||||
"specificity": self.specificity,
|
||||
}
|
||||
|
||||
|
||||
def _row_to_rule(row: dict) -> QuotaRule:
|
||||
return QuotaRule(
|
||||
uid=str(row.get("uid") or ""),
|
||||
owner_kind=row.get("owner_kind") or None,
|
||||
owner_id=row.get("owner_id") or None,
|
||||
app_reference=row.get("app_reference") or None,
|
||||
limit_usd=float(row.get("limit_usd") or 0.0),
|
||||
is_active=bool(row.get("is_active", 1)),
|
||||
label=str(row.get("label") or ""),
|
||||
created_by=str(row.get("created_by") or ""),
|
||||
created_at=str(row.get("created_at") or ""),
|
||||
updated_at=str(row.get("updated_at") or ""),
|
||||
)
|
||||
|
||||
|
||||
def _load() -> list[QuotaRule]:
|
||||
sync_local_cache(CACHE_NAME, _QUOTA_CACHE)
|
||||
if "rules" not in _QUOTA_CACHE:
|
||||
rules: list[QuotaRule] = []
|
||||
try:
|
||||
if RULES_TABLE in db.tables:
|
||||
for row in get_table(RULES_TABLE).all():
|
||||
if row.get("uid"):
|
||||
rules.append(_row_to_rule(row))
|
||||
except Exception as exc:
|
||||
logger.warning("gateway quota rule load failed: %s", exc)
|
||||
_QUOTA_CACHE["rules"] = rules
|
||||
return _QUOTA_CACHE["rules"]
|
||||
|
||||
|
||||
class QuotaRuleStore:
|
||||
def list(self) -> list[dict]:
|
||||
rules = sorted(_load(), key=lambda r: (-r.specificity, r.created_at))
|
||||
return [r.as_dict() for r in rules]
|
||||
|
||||
def get(self, uid: str) -> Optional[QuotaRule]:
|
||||
if not uid:
|
||||
return None
|
||||
for rule in _load():
|
||||
if rule.uid == uid:
|
||||
return rule
|
||||
return None
|
||||
|
||||
def count(self) -> int:
|
||||
return len(_load())
|
||||
|
||||
def set(
|
||||
self, payload: QuotaRuleIn, *, uid: Optional[str] = None, created_by: str = ""
|
||||
) -> dict:
|
||||
ensure_tables()
|
||||
table = get_table(RULES_TABLE)
|
||||
existing = table.find_one(uid=uid) if uid else None
|
||||
record: dict = {
|
||||
"owner_kind": payload.owner_kind,
|
||||
"owner_id": payload.owner_id,
|
||||
"app_reference": payload.app_reference,
|
||||
"limit_usd": payload.limit_usd,
|
||||
"is_active": 1 if payload.is_active else 0,
|
||||
"label": payload.label,
|
||||
"updated_at": _now(),
|
||||
}
|
||||
if existing:
|
||||
record["uid"] = existing["uid"]
|
||||
record["created_by"] = existing.get("created_by") or created_by
|
||||
record["created_at"] = existing.get("created_at") or _now()
|
||||
table.update({**record, "id": existing["id"]}, ["id"])
|
||||
else:
|
||||
record["uid"] = uid or uuid.uuid4().hex
|
||||
record["created_by"] = created_by
|
||||
record["created_at"] = _now()
|
||||
table.insert(record)
|
||||
bump_cache_version(CACHE_NAME)
|
||||
_QUOTA_CACHE.clear()
|
||||
saved = self.get(record["uid"])
|
||||
return saved.as_dict() if saved else record
|
||||
|
||||
def remove(self, uid: str) -> bool:
|
||||
uid = (uid or "").strip()
|
||||
if not uid or RULES_TABLE not in db.tables:
|
||||
return False
|
||||
removed = int(get_table(RULES_TABLE).delete(uid=uid))
|
||||
if removed:
|
||||
bump_cache_version(CACHE_NAME)
|
||||
_QUOTA_CACHE.clear()
|
||||
return bool(removed)
|
||||
|
||||
|
||||
quota_rule_store = QuotaRuleStore()
|
||||
|
||||
|
||||
def default_limit(owner_kind: str, cfg: dict) -> float:
|
||||
field = _DEFAULT_FIELD_BY_KIND.get(owner_kind, FIELD_DEFAULT_USER)
|
||||
return float(cfg.get(field, 0.0) or 0.0)
|
||||
|
||||
|
||||
def resolve(
|
||||
owner_kind: str, owner_id: str, app_reference: str, cfg: dict
|
||||
) -> tuple[float, tuple[Optional[str], Optional[str], Optional[str]], Optional[QuotaRule]]:
|
||||
matches = [r for r in _load() if r.matches(owner_kind, owner_id, app_reference)]
|
||||
if matches:
|
||||
def _sort_key(rule: QuotaRule):
|
||||
tie = float("inf") if rule.limit_usd == 0 else rule.limit_usd
|
||||
return (-rule.specificity, tie)
|
||||
|
||||
best = sorted(matches, key=_sort_key)[0]
|
||||
return best.limit_usd, (best.owner_kind, best.owner_id, best.app_reference), best
|
||||
return default_limit(owner_kind, cfg), (owner_kind, owner_id, None), None
|
||||
|
||||
|
||||
def resolve_for_owner(
|
||||
owner_kind: str, owner_id: str, cfg: dict
|
||||
) -> tuple[float, tuple[Optional[str], Optional[str], None], Optional[QuotaRule]]:
|
||||
matches = [
|
||||
r
|
||||
for r in _load()
|
||||
if r.app_reference is None and r.matches(owner_kind, owner_id, "")
|
||||
]
|
||||
if matches:
|
||||
def _sort_key(rule: QuotaRule):
|
||||
tie = float("inf") if rule.limit_usd == 0 else rule.limit_usd
|
||||
return (-rule.specificity, tie)
|
||||
|
||||
best = sorted(matches, key=_sort_key)[0]
|
||||
return best.limit_usd, (best.owner_kind, best.owner_id, None), best
|
||||
return default_limit(owner_kind, cfg), (owner_kind, owner_id, None), None
|
||||
|
||||
|
||||
def spent_24h(
|
||||
owner_kind: Optional[str], owner_id: Optional[str], app_reference: Optional[str]
|
||||
) -> float:
|
||||
from devplacepy.services.openai_gateway.usage import GATEWAY_LEDGER
|
||||
|
||||
if GATEWAY_LEDGER not in db.tables:
|
||||
return 0.0
|
||||
clauses = ["created_at >= :cutoff"]
|
||||
params: dict = {
|
||||
"cutoff": (datetime.now(timezone.utc) - timedelta(hours=24)).isoformat()
|
||||
}
|
||||
if owner_kind is not None:
|
||||
clauses.append("owner_kind = :owner_kind")
|
||||
params["owner_kind"] = owner_kind
|
||||
if owner_id is not None:
|
||||
clauses.append("owner_id = :owner_id")
|
||||
params["owner_id"] = owner_id
|
||||
if app_reference is not None:
|
||||
clauses.append("app_reference = :app_reference")
|
||||
params["app_reference"] = app_reference
|
||||
where = " AND ".join(clauses)
|
||||
rows = list(
|
||||
db.query(
|
||||
f"SELECT COALESCE(SUM(cost_usd), 0) AS spent FROM {GATEWAY_LEDGER} WHERE {where}",
|
||||
**params,
|
||||
)
|
||||
)
|
||||
return float(rows[0].get("spent") or 0.0) if rows else 0.0
|
||||
@@ -11,7 +11,7 @@ from fastapi.responses import JSONResponse
|
||||
|
||||
from devplacepy.database import get_int_setting
|
||||
from devplacepy.services.base import BaseService, ConfigField
|
||||
from devplacepy.services.openai_gateway import config
|
||||
from devplacepy.services.openai_gateway import config, quota
|
||||
from devplacepy.services.openai_gateway.analytics import summary_metrics
|
||||
from devplacepy.services.openai_gateway.gateway import GatewayRuntime
|
||||
from devplacepy.services.openai_gateway.routing import model_store
|
||||
@@ -262,6 +262,58 @@ class GatewayService(BaseService):
|
||||
"with this key. Clear it and restart to rotate.",
|
||||
group="Access",
|
||||
),
|
||||
ConfigField(
|
||||
quota.FIELD_DEFAULT_USER,
|
||||
"Default per-user daily cap ($)",
|
||||
type="float",
|
||||
default=1.0,
|
||||
minimum=0,
|
||||
help="Rolling 24h cap applied to a signed-in member with no matching quota rule. "
|
||||
"0 = unlimited. Overridable per user/app-reference on /admin/gateway.",
|
||||
group="Quota",
|
||||
),
|
||||
ConfigField(
|
||||
quota.FIELD_DEFAULT_ADMIN,
|
||||
"Default per-admin daily cap ($)",
|
||||
type="float",
|
||||
default=0.0,
|
||||
minimum=0,
|
||||
help="Rolling 24h cap applied to an administrator with no matching quota rule. "
|
||||
"0 = unlimited (the default - admins are exempt unless a rule says otherwise).",
|
||||
group="Quota",
|
||||
),
|
||||
ConfigField(
|
||||
quota.FIELD_DEFAULT_GUEST,
|
||||
"Default per-guest daily cap ($)",
|
||||
type="float",
|
||||
default=0.05,
|
||||
minimum=0,
|
||||
help="Rolling 24h cap applied to an unauthenticated caller with no matching quota "
|
||||
"rule (only reachable when authentication is not required). 0 = unlimited.",
|
||||
group="Quota",
|
||||
),
|
||||
ConfigField(
|
||||
quota.FIELD_DEFAULT_INTERNAL,
|
||||
"Default per-internal-key daily cap ($)",
|
||||
type="float",
|
||||
default=0.0,
|
||||
minimum=0,
|
||||
help="Rolling 24h cap applied to calls authenticated with the auto-generated internal "
|
||||
"key (DevPlace's own services: news, bots, Devii guests, correction/modifier). "
|
||||
"0 = unlimited (the default - do not cap this without a matching quota rule, or "
|
||||
"internal platform traffic will start failing).",
|
||||
group="Quota",
|
||||
),
|
||||
ConfigField(
|
||||
quota.FIELD_DEFAULT_KEY,
|
||||
"Default per-access-key daily cap ($)",
|
||||
type="float",
|
||||
default=0.0,
|
||||
minimum=0,
|
||||
help="Rolling 24h cap applied to calls authenticated with the static access key. "
|
||||
"0 = unlimited by default (it is an admin-provisioned trusted secret).",
|
||||
group="Quota",
|
||||
),
|
||||
ConfigField(
|
||||
"gateway_price_cache_hit_per_m",
|
||||
"Chat price cache-hit / 1M ($)",
|
||||
@@ -463,6 +515,37 @@ class GatewayService(BaseService):
|
||||
return (kind, user.get("uid") or "unknown")
|
||||
return ("anonymous", "anonymous")
|
||||
|
||||
def _audit_quota_exceeded(
|
||||
self,
|
||||
owner_kind: str,
|
||||
owner_id: str,
|
||||
app_reference: str,
|
||||
spent: float,
|
||||
limit: float,
|
||||
rule,
|
||||
) -> None:
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.openai_gateway.usage import audit_actor_for
|
||||
|
||||
actor_kind, actor_uid, actor_role = audit_actor_for(owner_kind, owner_id)
|
||||
audit.record_system(
|
||||
"ai.quota.exceeded",
|
||||
actor_kind=actor_kind,
|
||||
actor_uid=actor_uid,
|
||||
actor_role=actor_role,
|
||||
origin="api",
|
||||
result="denied",
|
||||
summary=f"AI gateway call by {owner_kind}/{owner_id} blocked - 24h quota reached",
|
||||
metadata={
|
||||
"owner_kind": owner_kind,
|
||||
"owner_id": owner_id,
|
||||
"app_reference": app_reference,
|
||||
"spent_usd": round(spent, 6),
|
||||
"limit_usd": limit,
|
||||
"rule_uid": rule.uid if rule else None,
|
||||
},
|
||||
)
|
||||
|
||||
def _models_response(self) -> JSONResponse:
|
||||
created = int(time.time())
|
||||
seen: set = set()
|
||||
@@ -501,6 +584,17 @@ class GatewayService(BaseService):
|
||||
)
|
||||
if subpath == "models" and request.method == "GET":
|
||||
return self._models_response()
|
||||
limit, scope, rule = quota.resolve(owner[0], owner[1], app_reference, cfg)
|
||||
if limit > 0:
|
||||
spent = quota.spent_24h(*scope)
|
||||
if spent >= limit:
|
||||
self.log(
|
||||
f"Rejected {owner[0]}:{owner[1]} app={app_reference}: "
|
||||
f"quota exceeded (${spent:.4f}/${limit:.4f}"
|
||||
f"{' rule ' + rule.uid if rule else ' default'})"
|
||||
)
|
||||
self._audit_quota_exceeded(owner[0], owner[1], app_reference, spent, limit, rule)
|
||||
raise HTTPException(status_code=429, detail="AI gateway daily quota exceeded")
|
||||
if subpath == "chat/completions" and request.method == "POST":
|
||||
try:
|
||||
body = await request.json()
|
||||
|
||||
Reference in New Issue
Block a user