Add personal notes, DeepSearch history, backup offload, and gateway auth throttling
DevPlace CI / test (push) Failing after 28m20s

Also streamline the top navigation: drop the Tools dropdown, make Quizzes
and Battles icon-only entries, and remove the Workspace, Containers and
Editor entry points from the project detail page.
This commit is contained in:
2026-09-12 20:32:31 +02:00
parent c4f7d01b2d
commit cae139ead9
158 changed files with 6156 additions and 377 deletions
+20 -1
View File
@@ -42,6 +42,16 @@ Callers may send an optional `X-App-Reference` header to tag gateway calls by ap
**Per-user attribution:** the gateway also accepts a real user's `api_key` (Bearer / X-API-KEY / session), gated by `gateway_allow_users` (default **on**); `resolve_owner()` records `(owner_kind, owner_id)` per call, so usage is attributed and limitable per user. With `gateway_allow_users` off, only internal/access/admin keys work and per-user attribution never happens. Devii operating a signed-in user authenticates its LLM calls with that user's own `api_key` (set as the session's `ai_key` in `build_settings(..., owner_kind="user")`), so a user's full gateway spend (Devii and direct API calls) rolls up under their uid - surfaced admin-only on the profile page via `build_user_usage()` / `GET /admin/users/{uid}/ai-usage`. Guests keep the internal key.
### Failed-auth throttle (`services/openai_gateway/auth_throttle.py`)
A dedicated per-IP throttle protects `/openai/v1/*` against unauthenticated probing WITHOUT IP-allowlisting the endpoint (which would break legitimate external API consumers using `gateway_access_key`/`gateway_allow_users`). It is scoped ONLY to failed authentication attempts, never to general traffic - the existing global rate limiter already exempts `/openai` entirely (`main.py`), and this throttle does not change that exemption.
- **Config (`gateway_auth_throttle_enabled` default on, `gateway_auth_throttle_max_failures` default 10, `gateway_auth_throttle_window_seconds` default 60, all group "Access").** An admin can retune or disable the throttle with no code change, same as every other gateway setting.
- **Counter.** `auth_throttle.py` is a small in-process sliding-window store (`dict[ip] -> list[float]` timestamps), the same shape as the global rate limiter's `_rate_limit_store` in `main.py`, but a separate, dedicated store local to the gateway rather than reusing that one (the two throttles key on different events - all mutating traffic vs. failed gateway auth - and `/openai` is deliberately exempt from the general limiter). `record_failure(ip, window)` appends a timestamp and returns the running in-window count; `is_throttled(ip, threshold, window)` checks without recording. Per-worker, like every other in-process cache in this codebase - not divided by worker count, so the effective aggregate threshold across `N` workers is up to `N x max_failures`; this is an accepted tradeoff (it still closes the probe, just at a slightly higher aggregate bound) rather than importing `main.WEB_WORKERS` into a leaf service module.
- **`GatewayService.authorize()` records a failure on every deny path** (a cheap static-key/internal-key match still short-circuits first and is never counted), so a wrong or garbage key counts exactly like presenting nothing. **The fast 429 short-circuit fires only when the request carries no credential at all** (no `X-API-KEY`, no `Authorization` header of any scheme, no `session` cookie) AND the IP is already over threshold - this is the "skip the rest of `authorize()`" defense against wasted CPU on a blind flood. A request presenting ANY credential - even one that turns out invalid - always runs the full check, so it can never be blocked by another caller's failures sharing its IP (a NAT/proxy caller with its own valid key is never punished for a neighbor's failed probes); only a request presenting nothing is fast-denied once its IP has tripped.
- **Audit.** Tripping the throttle (the exact request whose failure brings the window count to the configured threshold) records `ai.gateway_auth_throttle.tripped` (category `ai`, `result="denied"`, metadata `ip`/`failed_attempts`/`threshold`/`window_seconds`) via `audit.record(request, ...)`. Every subsequent fast-denied request in the same trip is silent (no audit spam under an actual flood) - the trip itself is the visible signal for admins.
- **Internal traffic cannot trip this by construction.** Every internal consumer (Devii guests, AI correction/modifier, quiz grading, SEO metadata generation, news, bots) authenticates with `database.internal_gateway_key()`, which is the exact same value `authorize()` compares against as `gateway_internal_key` - that comparison is the FIRST check in `authorize()`, before the throttle is even consulted, so internal self-dial calls always succeed there and never reach the failure-recording branch.
## Config fields
All config is `config_fields` (upstream url/model/key, force-model, the Prompt group's `gateway_system_preamble` and `gateway_thinking` (default off), timeout, instances, vision url/model/key/cache/toggle, the Embeddings group (`gateway_embed_enabled`/`_url`/`_model`/`_key`), the auth toggles + static key + internal key, plus the Pricing/Reliability/Tracking groups); live metrics (requests/errors/in-flight/vision-calls/embed-calls/latency plus 24h cost/tokens/success rollups) via `collect_metrics`.
@@ -57,6 +67,15 @@ Two behaviors:
Embeddings/passthrough are untouched by this composition step.
### Per-call operator preamble bypass (`bypass_preamble`, internal-only)
A structured/deterministic internal caller (one that expects a strict, machine-parsed output shape - grading JSON, a corrected text field, generated SEO metadata) can ask the gateway to skip `gateway_system_preamble` for that one call, since an operator preamble written for conversational tone can conflict with a task that demands exact output discipline. The date line and the client's own system content are unaffected either way.
- **Signal.** The chat request body carries `"bypass_preamble": true`. `GatewayRuntime.handle_chat` pops it off `body` before it is ever copied into the upstream `payload` (so it never leaks to the upstream provider) and only honors it when the caller also proved internal credentials for that same request (see below); otherwise it is silently discarded, exactly like an ordinary, unsupported field.
- **Internal-only gate.** `GatewayService.internal_bypass_allowed(request, cfg)` requires the request to carry the `X-Gateway-Internal-Key` header equal to the configured `gateway_internal_key` setting. This is a SEPARATE header from `Authorization`/`X-API-KEY` on purpose: a caller that authenticates with a real user's own `api_key` (correction, the `@ai` modifier, quiz free-text grading - all per-user-attributed on purpose, for correct spend attribution) still proves it is DevPlace's own server code, not the user's browser or a malicious client holding that same api_key, by additionally presenting the internal key on this header. A caller that authenticates AS the internal key directly (SEO metadata generation, via `database.internal_gateway_key()`) can present the same header too - one mechanism covers both shapes of internal caller. `GatewayService.handle()` computes this boolean once per request and passes it into `handle_chat` as `bypass_allowed`; a request with no valid internal credentials gets `bypass_allowed=False` and the body field is ignored even if present - it can never come from an external API consumer or a public-facing client call.
- **Callers that set it.** The four structured/deterministic internal callers all route through the single shared `services/correction.py::gateway_complete(..., bypass_preamble=True)` helper, which sets both the body field and the `X-Gateway-Internal-Key` header (via `database.internal_gateway_key()`) in one place: `services/correction.py::correct_text` (AI content correction), `services/ai_modifier.py::modify_text` (the `@ai` inline modifier), `services/quiz/grading.py::grade_free_text` (quiz free-text grading), and `services/jobs/seo_meta_service.py`'s generation call (SEO metadata). **Genuinely conversational internal callers are deliberately left unchanged** and keep receiving the operator preamble: Devii chat sessions and the docs chat do not call `gateway_complete` at all, so they are unaffected by construction.
- **Other internal self-dial callers found but NOT updated** (each builds its own request instead of using `gateway_complete`, and each is a candidate for the same treatment in a follow-up): `services/gitea/enhance.py::enhance_ticket` and `services/gitea/planning.py::generate_plan` (both structured, single-shot ticket text generation), `services/dbapi/nl2sql.py::design_query` (NL-to-SQL, strictly structured), `services/deepsearch/llm.py`/`services/jobs/deepsearch/enhance.py` (planner/synthesis calls), and `services/jobs/isslop/agent/llm.py` (per-file classification). None of these were in scope for this change.
## Thinking default (fast path)
`thinking.py` (`apply_thinking`) runs in `handle_chat` after `stream_options` is stripped, and in the vision describe call. DeepSeek V4 **enables thinking by default**; leaving the payload alone is the slow path. The gateway therefore always writes an explicit thinking field in the upstream dialect:
@@ -127,7 +146,7 @@ The human-facing **Quota rules** tab of `/admin/gateway` is a third, independent
## Embeddings
`POST /openai/v1/embeddings` exposes an OpenAI-compatible text-embeddings model. Clients send the generic model `molodetz~embed` (`config.INTERNAL_EMBED_MODEL`), which `handle_embeddings` remaps to `gateway_embed_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty). It defaults to OpenRouter's `qwen/qwen3-embedding-8b` at `https://openrouter.ai/api/v1/embeddings` (`config.EMBED_*_DEFAULT`, $0.01 per 1M input tokens). `handle_embeddings` mirrors `handle_chat` but is simpler: no vision augmentation and no streaming - build the payload, forward via `_send`, and record one ledger row through the same `finalize(...)` closure. The config fields are the **Embeddings** group (`gateway_embed_enabled` default on, `gateway_embed_url`, `gateway_embed_model`, `gateway_embed_key`) plus the Pricing-group `gateway_embed_price_input_per_m`. `effective_config()` falls the embed key back to `gateway_vision_key` then `OPENROUTER_API_KEY` (NOT `gateway_api_key`: that is the DeepSeek chat upstream key, whereas embeddings target OpenRouter like vision does). Usage is recorded with **`backend="embed"`**; `usage.compute_cost` adds an `embed` branch (input-only, completion always 0, native OpenRouter `cost` still preferred) and `Pricing` gained `embed_input_per_m`. `analytics.py` groups by `backend` generically, so embed rows roll up automatically; `caching_savings` counts only **non-native** chat rows (native-priced rows did not use the configured cache-hit/miss rates, so folding them in would report a fictional saving). When `gateway_embed_enabled` is off the endpoint returns 503 with no ledger row.
`POST /openai/v1/embeddings` exposes an OpenAI-compatible text-embeddings model. Clients send the generic model `molodetz~embed` (`config.INTERNAL_EMBED_MODEL`), which `handle_embeddings` remaps to `gateway_embed_model` exactly like chat remaps `molodetz` -> `gateway_model` (also remapped when `gateway_force_model` is on or the model is empty). It defaults to OpenRouter's `qwen/qwen3-embedding-8b` at `https://openrouter.ai/api/v1/embeddings` (`config.EMBED_*_DEFAULT`, $0.01 per 1M input tokens). `handle_embeddings` mirrors `handle_chat` but is simpler: no vision augmentation and no streaming - build the payload, forward via `_send`, and record one ledger row through the same `finalize(...)` closure. The config fields are the **Embeddings** group (`gateway_embed_enabled` default on, `gateway_embed_url`, `gateway_embed_model`, `gateway_embed_key`) plus the Pricing-group `gateway_embed_price_input_per_m`. `effective_config()` falls the embed key back to `gateway_vision_key` then `OPENROUTER_API_KEY` (NOT `gateway_api_key`: that is the DeepSeek chat upstream key, whereas embeddings target OpenRouter like vision does). Usage is recorded with **`backend="embed"`**; `usage.compute_cost` adds an `embed` branch (input-only, completion always 0, native OpenRouter `cost` still preferred) and `Pricing` gained `embed_input_per_m`. `analytics.py` groups by `backend` generically, so embed rows roll up automatically; `caching_savings` counts only **non-native** chat rows (native-priced rows did not use the configured cache-hit/miss rates, so folding them in would report a fictional saving). When `gateway_embed_enabled` is off the endpoint returns 503 with no ledger row. **When the resolved `gateway_embed_key` is still blank after the `effective_config()`/route-overlay merge** (all of `gateway_embed_key`, `gateway_vision_key`, and `OPENROUTER_API_KEY` are unset, or a per-route provider was configured with no `api_key` of its own), `handle_embeddings` returns the same `503` shape (`{"error": {"message": "Embeddings are not configured", "type": "embeddings_not_configured"}}`) with an `ai.gateway.call` audit row (`result="denied"`, `summary="no embeddings key configured"`) instead of forwarding an unauthenticated request upstream - mirrors the `gateway_embed_enabled` disabled-check immediately above it, no ledger row either. `routing.embed_overlay` only overlays `gateway_embed_key` when the matched route's provider actually has a non-blank `api_key`, so a misconfigured route can never downgrade an already-good fallback key to empty.
## Single point of truth for AI
@@ -0,0 +1,56 @@
# retoor <retoor@molodetz.nl>
import time
from collections import defaultdict
_failures: dict[str, list[float]] = defaultdict(list)
_last_sweep = 0.0
SWEEP_INTERVAL_SECONDS = 60.0
def _window_slice(ip: str, window_seconds: int, now: float) -> list[float]:
window_start = now - window_seconds
timestamps = [t for t in _failures.get(ip, ()) if t > window_start]
if timestamps:
_failures[ip] = timestamps
elif ip in _failures:
del _failures[ip]
return timestamps
def _sweep(window_start: float) -> None:
stale = [
ip
for ip, timestamps in _failures.items()
if not timestamps or timestamps[-1] <= window_start
]
for ip in stale:
del _failures[ip]
def is_throttled(ip: str, threshold: int, window_seconds: int) -> bool:
now = time.time()
return len(_window_slice(ip, window_seconds, now)) >= threshold
def record_failure(ip: str, window_seconds: int) -> int:
global _last_sweep
now = time.time()
window_start = now - window_seconds
if now - _last_sweep >= SWEEP_INTERVAL_SECONDS:
_sweep(window_start)
_last_sweep = now
timestamps = _window_slice(ip, window_seconds, now)
timestamps.append(now)
_failures[ip] = timestamps
return len(timestamps)
def reset(ip: str) -> None:
_failures.pop(ip, None)
def clear() -> None:
_failures.clear()
global _last_sweep
_last_sweep = 0.0
@@ -6,6 +6,9 @@ TIMEOUT_DEFAULT = 300
TIMEOUT_MIN = 300
INSTANCES_DEFAULT = 4
AUTH_THROTTLE_MAX_FAILURES_DEFAULT = 10
AUTH_THROTTLE_WINDOW_SECONDS_DEFAULT = 60
SYSTEM_PREAMBLE_DEFAULT = ""
THINKING_DEFAULT = False
+41 -8
View File
@@ -350,9 +350,17 @@ class GatewayRuntime:
return resp, None, timing
async def handle_chat(
self, body: dict, cfg: dict, owner: tuple, user_agent: str, app_reference: str, log=None
self,
body: dict,
cfg: dict,
owner: tuple,
user_agent: str,
app_reference: str,
log=None,
bypass_allowed: bool = False,
):
log = log or (lambda message: None)
bypass_preamble = bool(body.pop("bypass_preamble", False)) and bypass_allowed
overlay = chat_overlay(body.get("model"), cfg)
base_cfg = cfg
if overlay:
@@ -391,7 +399,8 @@ class GatewayRuntime:
self.vision_calls += augmenter.calls
vision_cost = augmenter.cost_usd
messages = apply_system_directives(messages, cfg.get("gateway_system_preamble", ""))
preamble = "" if bypass_preamble else cfg.get("gateway_system_preamble", "")
messages = apply_system_directives(messages, preamble)
requested = body.get("model")
allow_client_model = bool(cfg.get("gateway_allow_client_model"))
@@ -842,6 +851,35 @@ class GatewayRuntime:
}
},
)
if not cfg["gateway_embed_key"]:
from devplacepy.services.audit import record as audit
from devplacepy.services.openai_gateway.usage import audit_actor_for
actor_kind, actor_uid, actor_role = audit_actor_for(owner[0], owner[1])
audit.record_system(
"ai.gateway.call",
actor_kind=actor_kind,
actor_uid=actor_uid,
actor_role=actor_role,
origin="api",
result="denied",
summary="no embeddings key configured",
metadata={
"backend": "embed",
"endpoint": "embeddings",
"owner_kind": owner[0],
"owner_id": owner[1],
},
)
return JSONResponse(
status_code=503,
content={
"error": {
"message": "Embeddings are not configured",
"type": "embeddings_not_configured",
}
},
)
client, sem = self._ensure(cfg)
params = extract_params(body)
handle_start = time.monotonic()
@@ -866,13 +904,8 @@ class GatewayRuntime:
"Content-Type": "application/json",
**_attribution_headers(),
**_extra_provider_headers(cfg),
"Authorization": f"Bearer {cfg['gateway_embed_key']}",
}
if cfg["gateway_embed_key"]:
headers["Authorization"] = f"Bearer {cfg['gateway_embed_key']}"
else:
log(
"No upstream embeddings API key configured (gateway_embed_key / gateway_vision_key / OPENROUTER_API_KEY); upstream will likely reject the request"
)
resp, exc, timing = await self._send(
client,
+86 -2
View File
@@ -11,16 +11,19 @@ from fastapi.responses import JSONResponse
from devplacepy.database import get_int_setting
from devplacepy.services.base import BaseService, ConfigField
from devplacepy.services.openai_gateway import config, quota
from devplacepy.services.openai_gateway import auth_throttle, config, quota
from devplacepy.services.openai_gateway.analytics import summary_metrics
from devplacepy.services.openai_gateway.gateway import GatewayRuntime
from devplacepy.services.openai_gateway.routing import model_store
from devplacepy.utils import get_current_user
from devplacepy.utils.auth import _request_has_auth
from devplacepy.utils.guards import client_ip
logger = logging.getLogger(__name__)
APP_REFERENCE_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]{1,30}$")
DEFAULT_APP_REFERENCE = "default"
INTERNAL_KEY_HEADER = "X-Gateway-Internal-Key"
USER_CONTENT_OWNER_KINDS = ("user", "admin")
@@ -324,6 +327,36 @@ class GatewayService(BaseService):
"with this key. Clear it and restart to rotate.",
group="Access",
),
ConfigField(
"gateway_auth_throttle_enabled",
"Failed-auth throttle",
type="bool",
default=True,
help="Track failed authentication attempts per IP and block further "
"unauthenticated attempts from an IP once it crosses the failure "
"threshold within the window. Never blocks a request that presents "
"valid credentials, regardless of what its IP has done.",
group="Access",
),
ConfigField(
"gateway_auth_throttle_max_failures",
"Failed-auth threshold",
type="int",
default=config.AUTH_THROTTLE_MAX_FAILURES_DEFAULT,
minimum=1,
help="Failed authentication attempts allowed from one IP within the "
"window before further unauthenticated attempts are blocked with 429.",
group="Access",
),
ConfigField(
"gateway_auth_throttle_window_seconds",
"Failed-auth window (seconds)",
type="int",
default=config.AUTH_THROTTLE_WINDOW_SECONDS_DEFAULT,
minimum=1,
help="Sliding window the failed-auth threshold is counted over.",
group="Access",
),
ConfigField(
quota.FIELD_DEFAULT_USER,
"Default per-user daily cap ($)",
@@ -537,6 +570,24 @@ class GatewayService(BaseService):
)
return cfg
def _audit_throttle_tripped(
self, request: Request, ip: str, count: int, threshold: int, window: int
) -> None:
from devplacepy.services.audit import record as audit
audit.record(
request,
"ai.gateway_auth_throttle.tripped",
result="denied",
summary=f"AI gateway auth throttle tripped for {ip} after {count} failed attempts",
metadata={
"ip": ip,
"failed_attempts": count,
"threshold": threshold,
"window_seconds": window,
},
)
def authorize(self, request: Request) -> bool:
cfg = self.get_config()
if not cfg["gateway_require_auth"]:
@@ -548,12 +599,35 @@ class GatewayService(BaseService):
return True
if presented and internal_key and presented == internal_key:
return True
throttle_enabled = cfg.get("gateway_auth_throttle_enabled", True)
threshold = max(1, int(cfg.get("gateway_auth_throttle_max_failures", 10)))
window = max(1, int(cfg.get("gateway_auth_throttle_window_seconds", 60)))
has_credential = (
bool(presented)
or bool(request.cookies.get("session"))
or _request_has_auth(request)
)
ip = client_ip(request, default="unknown")
if (
throttle_enabled
and not has_credential
and auth_throttle.is_throttled(ip, threshold, window)
):
raise HTTPException(
status_code=429,
detail="Too many failed authentication attempts. Try again later.",
headers={"Retry-After": str(window)},
)
user = get_current_user(request)
if user:
if user.get("role") == "Admin" and cfg["gateway_allow_admins"]:
return True
if cfg["gateway_allow_users"]:
return True
if throttle_enabled:
count = auth_throttle.record_failure(ip, window)
if count == threshold:
self._audit_throttle_tripped(request, ip, count, threshold, window)
return False
def resolve_owner(self, request: Request) -> tuple:
@@ -577,6 +651,13 @@ class GatewayService(BaseService):
return (kind, user.get("uid") or "unknown")
return ("anonymous", "anonymous")
def internal_bypass_allowed(self, request: Request, cfg: dict) -> bool:
internal_key = cfg.get("gateway_internal_key")
if not internal_key:
return False
presented = request.headers.get(INTERNAL_KEY_HEADER, "").strip()
return bool(presented) and presented == internal_key
def user_content_owner(self, owner: tuple) -> str:
if owner[0] in USER_CONTENT_OWNER_KINDS and owner[1]:
return owner[1]
@@ -722,7 +803,10 @@ class GatewayService(BaseService):
if not isinstance(body, dict):
self.log("Rejected chat request: JSON body was not an object")
raise HTTPException(status_code=400, detail="Invalid JSON body")
return await runtime.handle_chat(body, cfg, owner, user_agent, app_reference, self.log)
bypass_allowed = self.internal_bypass_allowed(request, cfg)
return await runtime.handle_chat(
body, cfg, owner, user_agent, app_reference, self.log, bypass_allowed
)
if subpath == "embeddings" and request.method == "POST":
try:
body = await request.json()