yex

devplacepy/services/openai_gateway/CLAUDE.md

@@ -61,6 +61,20 @@ The gateway records one row per upstream call (chat, vision, passthrough) and su
 
 **Financial data is admin-only everywhere.** Any monetary figure (USD cost, pricing, spend, limit) is restricted to administrators; members and guests see only the percentage of quota used - this rule is enforced consistently across the profile card, `ai_correction`/`ai_modifier` usage displays, and the Devii cost tools.
 
+## Quota rules (`quota.py`, admin `/admin/gateway` "Quota rules" section)
+
+**This caps `/openai/v1/*` itself, independent of Devii's own daily cap.** Devii's `devii_user_daily_usd`/`devii_guest_daily_usd`/`devii_admin_daily_usd` (documented in `devplacepy/services/devii/CLAUDE.md`) only gate turns that go *through* Devii. A caller hitting the gateway directly with their own `api_key` bypasses that entirely - `quota.py` is the root-level enforcement that closes this, checked in `GatewayService.handle()` right after owner/`app_reference` resolution and before every billed dispatch (chat, embeddings, images, passthrough; `GET /v1/models` is exempt, it makes no upstream call).
+
+**Two layers, same shape as provider/model routing above.** Layer A is five flat `config_fields` on `GatewayService` (`gateway_default_user_daily_usd` $1.00, `gateway_default_admin_daily_usd` $0/unlimited, `gateway_default_guest_daily_usd` $0.05, `gateway_default_internal_daily_usd` $0/unlimited, `gateway_default_key_daily_usd` $0/unlimited - group **Quota**) applied per specific caller (`owner_id`) when no rule matches; internal/key default unlimited so shipping this never starts blocking DevPlace's own news/bots/Devii-guest/correction traffic on the internal key. Layer B is the `gateway_quota_rules` table (`ensure_tables()`, called from `init_db()` alongside `routing.ensure_tables()`; hard CRUD, not in `SOFT_DELETE_TABLES`, cross-worker cache-invalidated under the `"gateway_quota"` name): each row scopes by **any combination** of `owner_kind` (internal/key/user/admin/anonymous - DevPlace's only "roles" here), a specific `owner_id`, and `app_reference` (the `X-App-Reference` label), each nullable = wildcard; a `QuotaRuleIn` Pydantic validator rejects a rule with all three blank (that belongs in Layer A). `quota.resolve(owner_kind, owner_id, app_reference, cfg)` gathers every active rule whose non-null dimensions all equal the request, picks the one with the most non-null dimensions (ties broken toward the smaller limit, unlimited `0` never wins a tie against a finite cap), and returns `(limit_usd, scope, rule)` where `scope` is the exact `(owner_kind, owner_id, app_reference)` triple - each possibly `None` - that spend must be summed over. Layer A is internally just the maximally-specific implicit scope `(owner_kind, owner_id, None)`, so one code path (`quota.spent_24h(*scope)`, a plain `SUM(cost_usd)` over `gateway_usage_ledger` filtered by whichever scope dimensions are non-null) serves both layers.
+
+**A wildcard dimension means a shared pool, by design.** A rule scoped only by `app_reference` caps that app's combined spend across every caller using it; a rule scoped only by `owner_kind` caps that whole role's combined spend. Pin `owner_id` to get a true per-caller cap (the Layer A default's own behavior). `anonymous`/`internal`/`key` owner_ids are already fixed constants (`"anonymous"`/`"devii"`/`"access"`, from `resolve_owner()`), not per-caller identities, so any cap on those kinds is inherently pooled - there is no per-guest identity at this layer (unlike Devii's own guest-cookie-scoped ledger).
+
+**No lock, no hold, bounded overshoot by design - this is deliberate, not an oversight.** Cost is only known after the upstream call returns, so a true atomic pre-authorization would need a reserve-then-reconcile ("hold") mechanism, and a bug in releasing a hold is exactly the kind of thing that gets a caller stuck forever. Instead this mirrors Devii's own already-shipped mechanism exactly: read the 24h sum, compare, `raise HTTPException(429, ...)` if already at/over - a single `SELECT` and a conditional raise, nothing held, nothing to leak, structurally impossible to deadlock. The tradeoff is a small, bounded overshoot (at most a few concurrent in-flight calls' worth of cost past the cap before the next request sees the updated sum and blocks) - acceptable and industry-standard for a cost whose exact size isn't known until the call finishes, and it is the property actually being enforced: once tripped, every subsequent separate request stays blocked until the 24h window rolls off or an admin adjusts the rule.
+
+**429 body never carries a dollar figure**, admin or not (`{"detail": "AI gateway daily quota exceeded"}`) - mirrors Devii's own over-limit WS message, which likewise never states a number. The admin-only services log line and the `ai.quota.exceeded` audit row (`GatewayService._audit_quota_exceeded`, reusing `usage.audit_actor_for`) do carry the spend/limit/matched-rule-uid, since those are admin-only surfaces.
+
+**CRUD.** Admin JSON at `/admin/gateway/quota-rules` (`routers/admin/gateway_configs.py`, list returns each rule's live `spent_24h_usd` plus the Layer A defaults for context), audited `gateway.quota_rule.update`/`gateway.quota_rule.delete` (category `ai`, both already in `events.md`), rendered in the **Quota rules** section of `/admin/gateway` (`GatewayAdmin.js`, mirrors the providers/models CRUD tables). Devii tools `gateway_quota_rules`/`gateway_quota_rule_set`/`gateway_quota_rule_delete` (`requires_admin=True`, delete is `CONFIRM_REQUIRED`) proxy the same endpoints via `handler="http"`, same as the provider/model tools. CLI: `devplace gateway quota list|set|delete`.
+
 ## Image generation
 
 `POST /openai/v1/images/generations` exposes an OpenAI-compatible image-generation endpoint. Clients send the generic model `molodetz-img-small` (`config.INTERNAL_IMAGE_MODEL`), which `handle_images` remaps to `gateway_image_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty or `molodetz-img`). It defaults to OpenRouter's `black-forest-labs/flux-1.1-pro` at `https://openrouter.ai/api/v1/images/generations` (`config.IMAGE_*_DEFAULT`, $0.04 per image fallback). `handle_images` mirrors `handle_embeddings`: build the payload, forward via `_send`, and record one ledger row. The config fields are the **Images** group (`gateway_image_enabled` default on, `gateway_image_url`, `gateway_image_model`, `gateway_image_key`) plus the Pricing-group `gateway_image_price_per_call`. `effective_config()` falls the image key back to `gateway_api_key` then `OPENROUTER_API_KEY`. Usage is recorded with **`backend="image"`**; `usage.compute_cost` adds an `image` branch (flat per-call, native OpenRouter `cost` still preferred via `extract_image_usage`). `routing.image_overlay` resolves per-route provider/url/key and uses `price_input_per_m` as the per-image price. `routing.seed_default_image_routes()` (from `migrate_ai_gateway_settings`) idempotently seeds `molodetz-img-small` -> Flux on the `openrouter` provider when `OPENROUTER_API_KEY` is set. When `gateway_image_enabled` is off the endpoint returns 503 with no ledger row.