Party Hardy

CLAUDE.md

@@ -183,9 +183,9 @@ The AI usage analyzer (`devplace isslop analyze`, the `/tools/isslop` job servic
 
 ### Request pipeline
 
-`main.py` mounts `/static`, registers every router with its prefix, installs middlewares (security headers, a per-IP rate limit in an in-process `defaultdict`, a maintenance gate), and registers global 404/500 handlers. Rate limiting reads `rate_limit_per_minute`/`rate_limit_window_seconds` from `site_settings` (default 60/60s) and **applies only to mutating methods** (`POST`/`PUT`/`DELETE`/`PATCH`); reads and the whole `/openai` gateway are exempt. Bucket key is `X-Real-IP` falling back to `request.client.host`; an over-limit request gets `429` with `Retry-After`. The maintenance middleware short-circuits non-admin requests with a 503 when `maintenance_mode="1"`, but always allows `/static`, `/avatar`, `/auth`, `/admin`, and admin users. The outermost middleware is `response_timing`: stamps `request.state.request_start` and sets `X-Response-Time` on every response; the `response_time_ms(request)` Jinja global renders it as a fixed bottom-left badge in `base.html`. This is the single generic timing mechanism - never re-time per route.
+`main.py` mounts `/static`, registers every router with its prefix, installs middlewares from `middleware/` (`register()` in addition order: snapshot, corrections, security headers, rate limit, maintenance, terms, presence, visits, timing, tunnel dispatch, gzip), and registers the global handlers from `errors.py` (404/500/`ContentRefused`/`RequestValidationError`). The per-IP rate limit state lives in `middleware/state.py`; app lifespan (locks, `SERVICE_CLASSES`, startup/shutdown) lives in `lifespan.py`; the home route lives in `routers/landing.py`; the static-file classes live in `static_files.py`. Rate limiting reads `rate_limit_per_minute`/`rate_limit_window_seconds` from `site_settings` (default 60/60s) and **applies only to mutating methods** (`POST`/`PUT`/`DELETE`/`PATCH`); reads and the whole `/openai` gateway are exempt. Bucket key is `X-Real-IP` falling back to `request.client.host`; an over-limit request gets `429` with `Retry-After`. The maintenance middleware short-circuits non-admin requests with a 503 when `maintenance_mode="1"`, but always allows `/static`, `/avatar`, `/auth`, `/admin`, and admin users. The outermost middleware is `response_timing`: stamps `request.state.request_start` and sets `X-Response-Time` on every response; the `response_time_ms(request)` Jinja global renders it as a fixed bottom-left badge in `base.html`. This is the single generic timing mechanism - never re-time per route.
 
-`@app.on_event("startup")` calls `init_db()` and, unless `DEVPLACE_DISABLE_SERVICES` is set, registers `NewsService` and kicks off `start_all()`. `GET /` never redirects: guests get the marketing splash, authenticated users get a personalized home, both sharing Latest Posts + Developer News. The feed and Latest Posts enforce **author diversity** by interleaving authors (never dropping posts) via `interleave_by_author`/`paginate_diverse` in `database/`.
+`lifespan.py` `lifespan()` calls `init_db()` and, unless `DEVPLACE_DISABLE_SERVICES` is set, registers every service in `SERVICE_CLASSES` and supervises them in the lock owner. `GET /` never redirects: guests get the marketing splash, authenticated users get a personalized home, both sharing Latest Posts + Developer News. The feed and Latest Posts enforce **author diversity** by interleaving authors (never dropping posts) via `interleave_by_author`/`paginate_diverse` in `database/`.
 
 ### Routing layout
 
@@ -201,6 +201,7 @@ Routers in `devplacepy/routers/` are organised as a **directory tree that mirror
 | `/messages` | messages.py - see `services/messaging/CLAUDE.md` |
 | `/notifications`, `/votes`, `/reactions`, `/bookmarks`, `/polls`, `/avatar`, `/follow`, `/leaderboard` | flat files |
 | (none) | relations.py - block/mute |
+| `/` | landing.py - home route (guests get the splash, users the dashboard) |
 | `/admin`, `/admin/services`, `/admin/containers` | admin/ package |
 | `/issues` | issues/ package - see `services/gitea/CLAUDE.md` |
 | `/gists`, `/news`, `/uploads`, `/media` | flat files |
@@ -297,7 +298,7 @@ The escape hatch is deliberately two-factor and must never be self-served: after
 - **No comments, no docstrings in source.** Code is self-documenting.
 - **Forbidden variable name patterns:** `_new`, `_old`, `_temp`, `_v2`, `better_`, `my_`, `the_`.
 - **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, web push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Two exceptions keep a plain `httpx.AsyncClient`: (1) the reverse-proxy forwarding core `services/containers/forward.py` (and the three routers that call it - `routers/proxy.py`, `routers/tunnel.py`, the workspace editor route), which relays the user's own headers verbatim - bolting the Chrome identity onto it would overwrite the very headers it exists to forward; (2) the APNs provider (`push/providers/apns.py` `gateway_client` / `delivery_client`) which talks to Apple's HTTP/2 provider API with `httpx.AsyncClient(http2=True, trust_env=False)` - Chrome impersonation, HTTP/2 PRIORITY frames, HPACK indexing of `:path`/`authorization`, extra browser headers, and the outbound proxy all break or starve that API. Web Push stays on stealth. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
-- **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
+- **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models/`, one module per domain, re-exported from `models/__init__.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
 - **`RedirectResponse(url=..., status_code=302)`** - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
 - **Ownership checks:** compare `resource["user_uid"] == user["uid"]` before edit/delete (use `content.is_owner`). **Deletes additionally allow any admin:** `is_owner(...) or is_admin(user)` on every content delete endpoint, so an administrator may delete any member's content while members are limited to their own. Edits stay owner-only. Deletes are **soft** and cascade under one shared stamp - see `devplacepy/database/CLAUDE.md`.
 - **All dates shown to users are DD/MM/YYYY** (European), rendered in the viewer's own timezone client-side. Timestamps are stored/emitted as UTC ISO. Use the `local_dt(iso, mode)`/`dt_ago(iso)` Jinja globals for any user-facing instant - they emit `<time data-dt>` and `static/js/LocalTime.js` reformats to local timezone with a `MutationObserver` for dynamic content. `format_date()`/`time_ago()` stay as plain-text helpers for JSON responses, no-JS fallbacks, and non-timestamp date fields (e.g. project `release_date`) - do NOT wrap those in `local_dt`.
@@ -368,7 +369,7 @@ Every feature in DevPlace is **one data source fanning out into several consumer
 A new public read almost always needs all four. The cardinal failure mode is changing one face and forgetting a connected one - a real worked example (followers/following listing) touched nine files across data helpers, output schema, route, view, agent tool, API docs, and the docs trio below for one conceptual feature; that count is the norm, not the exception. If a change touches only one file, confirm none of the four faces were missed. Checklist, ordered by data flow:
 
 1. **Understand.** Read the router, template, matching tests (per the tier naming rule above), and the matching nested `CLAUDE.md`. Trace input model -> router -> data helper -> response (HTML and JSON).
-2. **Data layer.** `database/` for query/batch helpers (never inline N+1 loops - use `get_users_by_uids`, `build_pagination`, `_in_clause`; guard raw SQL with `if "table" in db.tables`). `models.py` for the Pydantic `Form` input model. `schemas/` for the `*Out` JSON response model - every context key a JSON route exposes via `respond(..., model=XOut)` MUST exist on `XOut` or it is silently dropped. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
+2. **Data layer.** `database/` for query/batch helpers (never inline N+1 loops - use `get_users_by_uids`, `build_pagination`, `_in_clause`; guard raw SQL with `if "table" in db.tables`). `models/` for the Pydantic `Form` input model (add the class to the matching domain module and to the re-export list plus `__all__` in `models/__init__.py`). `schemas/` for the `*Out` JSON response model - every context key a JSON route exposes via `respond(..., model=XOut)` MUST exist on `XOut` or it is silently dropped. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
 3. **Server layer.** Handler with the right auth guard (`get_current_user` public read, `require_user` member POST, `require_admin` admin). Specific paths (`/{username}/followers`) declared before catch-alls (`/{username}`). Return HTML+JSON via `respond(request, template, ctx, model=XOut)` or pure JSON via `JSONResponse`. Register any NEW router in `main.py`.
 4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
 5. **Safety layer (skip only when the feature stores nothing and shows nothing).** If the feature adds a surface that carries user-generated content, register it in `database/moderation.py` `REPORTABLE_TARGETS` (or in `UNREPORTABLE_TABLES` with a reason), resolve it in `resolve_object_url`, and include `_report_button.html` in its action bar - the registry test and the e2e coverage test both fail otherwise. If it collects or transmits a new *category* of personal data, or sends anything new to a third party, update `/docs/privacy.html` and the app-store privacy declarations in the same change; that disclosure is part of the feature, not a follow-up.

README.md

@@ -42,12 +42,15 @@ sudo apt-get install -y libglib2.0-0 libpango-1.0-0 libpangoft2-1.0-0 libharfbuz
 ```
 devplacepy/
   main.py             # FastAPI app, router registration
+  middleware/         # HTTP middlewares + shared request state (package)
+  errors.py           # Global exception handlers
+  lifespan.py         # Startup/shutdown, service registry
   config.py           # Settings from env vars + .env
   database/           # dataset connection, index creation (package)
   templating.py       # Shared Jinja2 environment + globals
   avatar.py           # Multiavatar generation, URL builder
   utils/              # Password hashing, session mgmt, time_ago, notification hook (package)
-  models.py           # Pydantic schemas
+  models/             # Pydantic Form models, one module per domain (package)
   push/               # Push delivery: provider protocol, Web Push, APNs, registrations
   routers/            # One file per domain (auth, feed, posts, push, ...)
   templates/          # Jinja2 HTML templates

devplacepy/errors.py

@@ -0,0 +1,154 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+
+from fastapi import Request
+from fastapi.exceptions import RequestValidationError
+from fastapi.responses import JSONResponse, RedirectResponse
+
+from devplacepy import happy404
+from devplacepy.responses import json_error, wants_json
+from devplacepy.schemas import ValidationErrorOut
+from devplacepy.seo import base_seo_context
+from devplacepy.services.moderation.screening import ContentRefused
+from devplacepy.templating import templates
+from devplacepy.utils import safe_next
+
+logger = logging.getLogger(__name__)
+
+
+async def not_found(request: Request, exc):
+    if wants_json(request):
+        return json_error(404, "Not found")
+    happy_response = happy404.render(request)
+    if happy_response is not None:
+        return happy_response
+    seo_ctx = base_seo_context(
+        request,
+        title="Not Found - DevPlace",
+        description="The page you requested does not exist.",
+        robots="noindex",
+    )
+    return templates.TemplateResponse(
+        request,
+        "error.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "error_code": 404,
+            "error_message": "Page not found",
+        },
+        status_code=404,
+    )
+
+
+async def server_error(request: Request, exc):
+    logger.exception("500 error on %s %s", request.method, request.url.path)
+    if wants_json(request):
+        return json_error(500, "Internal server error")
+    seo_ctx = base_seo_context(
+        request,
+        title="Server Error - DevPlace",
+        description="Something went wrong.",
+        robots="noindex",
+    )
+    return templates.TemplateResponse(
+        request,
+        "error.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "error_code": 500,
+            "error_message": "Internal server error",
+        },
+        status_code=500,
+    )
+
+
+async def content_refused(request: Request, exc: ContentRefused):
+    logger.info("content refused on %s %s: %s", request.method, request.url.path, exc.message)
+    if wants_json(request):
+        return json_error(400, exc.message, categories=list(exc.categories))
+    seo_ctx = base_seo_context(
+        request,
+        title="Content not published - DevPlace",
+        description=exc.message,
+        robots="noindex",
+    )
+    return templates.TemplateResponse(
+        request,
+        "error.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "error_code": 400,
+            "error_message": exc.message,
+        },
+        status_code=400,
+    )
+
+
+_AUTH_FORM_PAGES = {
+    "/auth/signup": ("signup.html", "Join DevPlace"),
+    "/auth/login": ("login.html", "Sign In"),
+    "/auth/forgot-password": ("forgot_password.html", "Reset Password"),
+}
+
+_FRIENDLY_ERRORS = {
+    ("username", "too_short"): "Username must be between 3 and 32 characters",
+    ("username", "too_long"): "Username must be between 3 and 32 characters",
+    ("password", "too_short"): "Password must be at least 6 characters",
+}
+
+
+def _friendly_error(err):
+    field = err["loc"][-1] if err.get("loc") else ""
+    key = (field, err.get("type", "").replace("string_", ""))
+    if key in _FRIENDLY_ERRORS:
+        return _FRIENDLY_ERRORS[key]
+    msg = err.get("msg", "Invalid input")
+    prefix = "Value error, "
+    return msg[len(prefix) :] if msg.startswith(prefix) else msg
+
+
+async def on_validation_error(request: Request, exc: RequestValidationError):
+    errors = [_friendly_error(e) for e in exc.errors()]
+    if wants_json(request):
+        fields: dict = {}
+        for raw, friendly in zip(exc.errors(), errors):
+            name = raw["loc"][-1] if raw.get("loc") else "_"
+            fields.setdefault(str(name), []).append(friendly)
+        return JSONResponse(
+            ValidationErrorOut(fields=fields, messages=errors).model_dump(mode="json"),
+            status_code=422,
+        )
+    path = request.url.path
+    page = _AUTH_FORM_PAGES.get(path)
+    if page is None and path.startswith("/auth/reset-password/"):
+        page = ("reset_password.html", "Set New Password")
+    if page:
+        template_name, title = page
+        context = {
+            **base_seo_context(request, title=title, robots="noindex,nofollow"),
+            "request": request,
+            "errors": errors,
+        }
+        try:
+            form = await request.form()
+            context.update({k: v for k, v in form.items() if isinstance(v, str)})
+        except Exception:
+            pass
+        if "token" in request.path_params:
+            context["token"] = request.path_params["token"]
+        return templates.TemplateResponse(
+            request, template_name, context, status_code=400
+        )
+    referer = safe_next(request.headers.get("referer"), "/feed")
+    return RedirectResponse(url=referer, status_code=303)
+
+
+def register(app):
+    app.exception_handler(404)(not_found)
+    app.exception_handler(500)(server_error)
+    app.exception_handler(ContentRefused)(content_refused)
+    app.exception_handler(RequestValidationError)(on_validation_error)

devplacepy/lifespan.py

@@ -0,0 +1,154 @@
+# retoor <retoor@molodetz.nl>
+
+import fcntl
+import logging
+import os
+from contextlib import asynccontextmanager, contextmanager
+
+from fastapi import FastAPI
+
+from devplacepy.config import (
+    INIT_LOCK_FILE,
+    PORT,
+    SERVICE_LOCK_FILE,
+    ensure_data_dirs,
+)
+from devplacepy.database import init_db
+from devplacepy.services.acceptance.service import AcceptanceService
+from devplacepy.services.audit import AuditService
+from devplacepy.services.backup import BackupService
+from devplacepy.services.background import background
+from devplacepy.services.bot import BotsService
+from devplacepy.services.community.service import CommunityService
+from devplacepy.services.containers.service import ContainerService
+from devplacepy.services.containers.workspace_service import WorkspaceService
+from devplacepy.services.dbapi.service import DbApiJobService
+from devplacepy.services.devii import DeviiService
+from devplacepy.services.gitea.service import IssueTrackerService
+from devplacepy.services.jobs.award_service import AwardService
+from devplacepy.services.jobs.deepsearch.service import DeepsearchService
+from devplacepy.services.jobs.fork_service import ForkService
+from devplacepy.services.jobs.issue_create_service import IssueCreateService
+from devplacepy.services.jobs.isslop.service import IsslopService
+from devplacepy.services.jobs.planning_service import PlanningReportService
+from devplacepy.services.jobs.seo.service import SeoService
+from devplacepy.services.jobs.seo_meta_service import SeoMetaService
+from devplacepy.services.jobs.zip_service import ZipService
+from devplacepy.services.live_view_relay import LiveViewRelayService
+from devplacepy.services.manager import service_manager
+from devplacepy.services.moderation.service import ModerationService
+from devplacepy.services.news import NewsService
+from devplacepy.services.nodes.service import NodesService
+from devplacepy.services.notification_relay import NotificationRelayService
+from devplacepy.services.openai_gateway import GatewayService
+from devplacepy.services.opinionwar.service import OpinionWarService
+from devplacepy.services.presence_relay import PresenceRelayService
+from devplacepy.services.pubsub import PubSubService
+from devplacepy.services.push import PushService
+from devplacepy.services.telegram import TelegramService
+from devplacepy.services.telegram.outbox_service import TelegramOutboxService
+from devplacepy.services.xmlrpc import XmlrpcService
+
+logger = logging.getLogger(__name__)
+
+SERVICE_CLASSES = [
+    NewsService,
+    BotsService,
+    GatewayService,
+    DeviiService,
+    ZipService,
+    ForkService,
+    SeoService,
+    SeoMetaService,
+    AwardService,
+    BackupService,
+    DbApiJobService,
+    PubSubService,
+    NotificationRelayService,
+    OpinionWarService,
+    LiveViewRelayService,
+    PresenceRelayService,
+    DeepsearchService,
+    IsslopService,
+    IssueCreateService,
+    PlanningReportService,
+    IssueTrackerService,
+    ContainerService,
+    WorkspaceService,
+    XmlrpcService,
+    AuditService,
+    ModerationService,
+    AcceptanceService,
+    CommunityService,
+    NodesService,
+    PushService,
+    TelegramService,
+    TelegramOutboxService,
+]
+
+_service_lock_handle = None
+
+
+@contextmanager
+def init_lock():
+    handle = open(INIT_LOCK_FILE, "w")
+    try:
+        fcntl.flock(handle, fcntl.LOCK_EX)
+        yield
+    finally:
+        fcntl.flock(handle, fcntl.LOCK_UN)
+        handle.close()
+
+
+def acquire_service_lock() -> bool:
+    global _service_lock_handle
+    handle = open(SERVICE_LOCK_FILE, "w")
+    try:
+        fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
+    except OSError:
+        handle.close()
+        return False
+    _service_lock_handle = handle
+    return True
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    ensure_data_dirs()
+    with init_lock():
+        init_db()
+        from devplacepy.push import ensure_certificates
+
+        ensure_certificates()
+        from devplacepy.services.openai_gateway import model_health
+
+        model_health.seed_from_ledger()
+    for service_cls in SERVICE_CLASSES:
+        service_manager.register(service_cls())
+    if not os.environ.get("DEVPLACE_DISABLE_SERVICES"):
+        await background.start()
+        if acquire_service_lock():
+            service_manager.set_lock_owner(True)
+            service_manager.supervise()
+            logger.info(f"Background services started in worker pid {os.getpid()}")
+        else:
+            logger.info(
+                f"Worker pid {os.getpid()} declined service lock; another worker owns background services"
+            )
+    from devplacepy.services.statistics.tracking import start_visit_flusher
+
+    start_visit_flusher()
+    logger.info(f"DevPlace started on port {PORT}")
+    yield
+    logger.info("Shutting down services...")
+    from devplacepy.services.statistics.tracking import flush_visits
+
+    flush_visits()
+    await service_manager.shutdown_all()
+    await background.stop()
+    from devplacepy.services.containers import forward
+
+    await forward.close_client()
+    from devplacepy import push
+
+    await push.shutdown_providers()

devplacepy/main.py

@@ -1,54 +1,18 @@
 # retoor <retoor@molodetz.nl>
 
-import asyncio
-import fcntl
 import logging
-import os
-import time
-from collections import defaultdict
-from contextlib import asynccontextmanager, contextmanager
-from pathlib import Path
-from fastapi import FastAPI, Request, WebSocket
-from fastapi.responses import HTMLResponse, RedirectResponse
-from fastapi.staticfiles import StaticFiles
-from fastapi.exceptions import RequestValidationError
-from starlette.middleware.gzip import GZipMiddleware
+
+from fastapi import FastAPI
+
+from devplacepy import errors as error_handlers
 from devplacepy.config import (
+    LOG_LEVEL,
     STATIC_DIR,
     STATIC_VERSION,
     UPLOADS_DIR,
-    PORT,
-    SERVICE_LOCK_FILE,
-    INIT_LOCK_FILE,
-    LOG_LEVEL,
-    ensure_data_dirs,
 )
-from devplacepy.database import (
-    init_db,
-    get_table,
-    db,
-    refresh_snapshot,
-    get_users_by_uids,
-    get_comment_counts_by_post_uids,
-    get_vote_counts,
-    get_news_images_by_uids,
-    get_setting,
-    get_int_setting,
-    paginate_diverse,
-    get_user_post_count,
-    get_user_stars,
-    get_blocked_uids,
-    get_top_authors,
-    get_trending_topics,
-)
-from devplacepy.templating import templates, jinja_unread_count
-from devplacepy.cache import TTLCache
-from devplacepy.responses import respond, wants_json, json_error
-from devplacepy import happy404
-from devplacepy.schemas import LandingOut, ValidationErrorOut
-from fastapi.responses import JSONResponse
-from devplacepy.utils import get_current_user, time_ago, safe_next, client_ip
-from devplacepy.seo import base_seo_context, site_url, website_schema
+from devplacepy.lifespan import lifespan
+from devplacepy.middleware import register as register_middleware
 from devplacepy.routers import (
     auth,
     battles,
@@ -96,243 +60,18 @@ from devplacepy.routers import (
     pubsub,
     game,
     workspaces,
+    landing,
+)
+from devplacepy.static_files import (
+    CachedStaticFiles,
+    FallbackStaticFiles,
+    UploadStaticFiles,
 )
-from devplacepy.services.manager import service_manager
-from devplacepy.services.background import background
-from devplacepy.services.news import NewsService
-from devplacepy.services.bot import BotsService
-from devplacepy.services.openai_gateway import GatewayService
-from devplacepy.services.devii import DeviiService
-from devplacepy.services.jobs.zip_service import ZipService
-from devplacepy.services.jobs.fork_service import ForkService
-from devplacepy.services.jobs.issue_create_service import IssueCreateService
-from devplacepy.services.jobs.planning_service import PlanningReportService
-from devplacepy.services.jobs.seo.service import SeoService
-from devplacepy.services.jobs.seo_meta_service import SeoMetaService
-from devplacepy.services.jobs.award_service import AwardService
-from devplacepy.services.backup import BackupService
-from devplacepy.services.dbapi.service import DbApiJobService
-from devplacepy.services.pubsub import PubSubService
-from devplacepy.services.notification_relay import NotificationRelayService
-from devplacepy.services.opinionwar.service import OpinionWarService
-from devplacepy.services.live_view_relay import LiveViewRelayService
-from devplacepy.services.presence_relay import PresenceRelayService
-from devplacepy.services import presence
-from devplacepy.services.correction import PENDING_SCOPE_KEY
-from devplacepy.services.jobs.deepsearch.service import DeepsearchService
-from devplacepy.services.jobs.isslop.service import IsslopService
-from devplacepy.services.gitea.service import IssueTrackerService
-from devplacepy.services.containers.service import ContainerService
-from devplacepy.services.containers.workspace_service import WorkspaceService
-from devplacepy.services.xmlrpc import XmlrpcService
-from devplacepy.services.audit import AuditService
-from devplacepy.services.moderation.service import ModerationService
-from devplacepy.services.moderation.screening import ContentRefused
-from devplacepy.services.acceptance.service import AcceptanceService
-from devplacepy.services.community.service import CommunityService
-from devplacepy.services.nodes.service import NodesService
-from devplacepy.services.audit import record as audit
-from devplacepy.services.push import PushService
-from devplacepy.services.telegram import TelegramService
-from devplacepy.services.telegram.outbox_service import TelegramOutboxService
 
 logging.basicConfig(
     level=getattr(logging, LOG_LEVEL, logging.INFO),
     format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
 )
-logger = logging.getLogger(__name__)
-
-_rate_limit_store = defaultdict(list)
-RATE_LIMIT = int(os.environ.get("DEVPLACE_RATE_LIMIT", "60"))
-RATE_WINDOW = 60
-WEB_WORKERS = max(1, int(os.environ.get("DEVPLACE_WEB_WORKERS", "1")))
-RATE_LIMIT_DISABLED = os.environ.get("DEVPLACE_DISABLE_RATE_LIMIT") == "1"
-
-HOT_SETTINGS_TTL = 2.0
-_hot_settings_value: dict = {}
-_hot_settings_at = 0.0
-_last_rate_sweep = 0.0
-RATE_SWEEP_INTERVAL = 60.0
-
-
-def _hot_settings() -> dict:
-    global _hot_settings_value, _hot_settings_at
-    now = time.monotonic()
-    if not _hot_settings_value or now - _hot_settings_at >= HOT_SETTINGS_TTL:
-        _hot_settings_value = {
-            "maintenance_mode": get_setting("maintenance_mode", "0"),
-            "rate_limit_per_minute": max(
-                1, get_int_setting("rate_limit_per_minute", RATE_LIMIT)
-            ),
-            "rate_limit_window_seconds": max(
-                1, get_int_setting("rate_limit_window_seconds", RATE_WINDOW)
-            ),
-        }
-        _hot_settings_at = now
-    return _hot_settings_value
-
-
-def _sweep_rate_limit_store(window_start: float) -> None:
-    stale = [
-        ip
-        for ip, timestamps in _rate_limit_store.items()
-        if not timestamps or timestamps[-1] <= window_start
-    ]
-    for ip in stale:
-        del _rate_limit_store[ip]
-
-
-def _worker_rate_limit(limit: int) -> int:
-    return max(1, -(-limit // WEB_WORKERS))
-
-
-_service_lock_handle = None
-
-
-@contextmanager
-def init_lock():
-    handle = open(INIT_LOCK_FILE, "w")
-    try:
-        fcntl.flock(handle, fcntl.LOCK_EX)
-        yield
-    finally:
-        fcntl.flock(handle, fcntl.LOCK_UN)
-        handle.close()
-
-
-def acquire_service_lock() -> bool:
-    global _service_lock_handle
-    handle = open(SERVICE_LOCK_FILE, "w")
-    try:
-        fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
-    except OSError:
-        handle.close()
-        return False
-    _service_lock_handle = handle
-    return True
-
-
-INLINE_MEDIA_EXTENSIONS = {
-    ".jpg",
-    ".jpeg",
-    ".png",
-    ".gif",
-    ".webp",
-    ".bmp",
-    ".tiff",
-    ".mp4",
-    ".webm",
-    ".ogv",
-    ".mov",
-    ".m4v",
-    ".mp3",
-}
-
-
-class UploadStaticFiles(StaticFiles):
-    async def get_response(self, path, scope):
-        response = await super().get_response(path, scope)
-        disposition = (
-            "inline"
-            if Path(path).suffix.lower() in INLINE_MEDIA_EXTENSIONS
-            else "attachment"
-        )
-        response.headers["Content-Disposition"] = disposition
-        response.headers["Cache-Control"] = "public, max-age=604800"
-        return response
-
-
-class CachedStaticFiles(StaticFiles):
-    async def get_response(self, path, scope):
-        response = await super().get_response(path, scope)
-        if Path(path).name == "service-worker.js":
-            response.headers["Cache-Control"] = "no-cache"
-        else:
-            response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
-        return response
-
-
-class FallbackStaticFiles(StaticFiles):
-    async def get_response(self, path, scope):
-        response = await super().get_response(path, scope)
-        if Path(path).name == "service-worker.js":
-            response.headers["Cache-Control"] = "no-cache"
-        else:
-            response.headers["Cache-Control"] = "public, max-age=3600"
-        return response
-
-
-@asynccontextmanager
-async def lifespan(app: FastAPI):
-    ensure_data_dirs()
-    with init_lock():
-        init_db()
-        from devplacepy.push import ensure_certificates
-
-        ensure_certificates()
-        from devplacepy.services.openai_gateway import model_health
-
-        model_health.seed_from_ledger()
-    service_manager.register(NewsService())
-    service_manager.register(BotsService())
-    service_manager.register(GatewayService())
-    service_manager.register(DeviiService())
-    service_manager.register(ZipService())
-    service_manager.register(ForkService())
-    service_manager.register(SeoService())
-    service_manager.register(SeoMetaService())
-    service_manager.register(AwardService())
-    service_manager.register(BackupService())
-    service_manager.register(DbApiJobService())
-    service_manager.register(PubSubService())
-    service_manager.register(NotificationRelayService())
-    service_manager.register(OpinionWarService())
-    service_manager.register(LiveViewRelayService())
-    service_manager.register(PresenceRelayService())
-    service_manager.register(DeepsearchService())
-    service_manager.register(IsslopService())
-    service_manager.register(IssueCreateService())
-    service_manager.register(PlanningReportService())
-    service_manager.register(IssueTrackerService())
-    service_manager.register(ContainerService())
-    service_manager.register(WorkspaceService())
-    service_manager.register(XmlrpcService())
-    service_manager.register(AuditService())
-    service_manager.register(ModerationService())
-    service_manager.register(AcceptanceService())
-    service_manager.register(CommunityService())
-    service_manager.register(NodesService())
-    service_manager.register(PushService())
-    service_manager.register(TelegramService())
-    service_manager.register(TelegramOutboxService())
-    if not os.environ.get("DEVPLACE_DISABLE_SERVICES"):
-        await background.start()
-        if acquire_service_lock():
-            service_manager.set_lock_owner(True)
-            service_manager.supervise()
-            logger.info(f"Background services started in worker pid {os.getpid()}")
-        else:
-            logger.info(
-                f"Worker pid {os.getpid()} declined service lock; another worker owns background services"
-            )
-    from devplacepy.services.statistics.tracking import start_visit_flusher
-
-    start_visit_flusher()
-    logger.info(f"DevPlace started on port {PORT}")
-    yield
-    logger.info("Shutting down services...")
-    from devplacepy.services.statistics.tracking import flush_visits
-
-    flush_visits()
-    await service_manager.shutdown_all()
-    await background.stop()
-    from devplacepy.services.containers import forward
-
-    await forward.close_client()
-    from devplacepy import push
-
-    await push.shutdown_providers()
-
 
 app = FastAPI(
     title="DevPlace",
@@ -353,140 +92,7 @@ app.mount(
 )
 app.mount("/static", FallbackStaticFiles(directory=str(STATIC_DIR)), name="static")
 
-
-@app.exception_handler(404)
-async def not_found(request: Request, exc):
-    if wants_json(request):
-        return json_error(404, "Not found")
-    happy_response = happy404.render(request)
-    if happy_response is not None:
-        return happy_response
-    seo_ctx = base_seo_context(
-        request,
-        title="Not Found - DevPlace",
-        description="The page you requested does not exist.",
-        robots="noindex",
-    )
-    return templates.TemplateResponse(
-        request,
-        "error.html",
-        {
-            **seo_ctx,
-            "request": request,
-            "error_code": 404,
-            "error_message": "Page not found",
-        },
-        status_code=404,
-    )
-
-
-@app.exception_handler(500)
-async def server_error(request: Request, exc):
-    logger.exception("500 error on %s %s", request.method, request.url.path)
-    if wants_json(request):
-        return json_error(500, "Internal server error")
-    seo_ctx = base_seo_context(
-        request,
-        title="Server Error - DevPlace",
-        description="Something went wrong.",
-        robots="noindex",
-    )
-    return templates.TemplateResponse(
-        request,
-        "error.html",
-        {
-            **seo_ctx,
-            "request": request,
-            "error_code": 500,
-            "error_message": "Internal server error",
-        },
-        status_code=500,
-    )
-
-
-@app.exception_handler(ContentRefused)
-async def content_refused(request: Request, exc: ContentRefused):
-    logger.info("content refused on %s %s: %s", request.method, request.url.path, exc.message)
-    if wants_json(request):
-        return json_error(400, exc.message, categories=list(exc.categories))
-    seo_ctx = base_seo_context(
-        request,
-        title="Content not published - DevPlace",
-        description=exc.message,
-        robots="noindex",
-    )
-    return templates.TemplateResponse(
-        request,
-        "error.html",
-        {
-            **seo_ctx,
-            "request": request,
-            "error_code": 400,
-            "error_message": exc.message,
-        },
-        status_code=400,
-    )
-
-
-_AUTH_FORM_PAGES = {
-    "/auth/signup": ("signup.html", "Join DevPlace"),
-    "/auth/login": ("login.html", "Sign In"),
-    "/auth/forgot-password": ("forgot_password.html", "Reset Password"),
-}
-
-_FRIENDLY_ERRORS = {
-    ("username", "too_short"): "Username must be between 3 and 32 characters",
-    ("username", "too_long"): "Username must be between 3 and 32 characters",
-    ("password", "too_short"): "Password must be at least 6 characters",
-}
-
-
-def _friendly_error(err):
-    field = err["loc"][-1] if err.get("loc") else ""
-    key = (field, err.get("type", "").replace("string_", ""))
-    if key in _FRIENDLY_ERRORS:
-        return _FRIENDLY_ERRORS[key]
-    msg = err.get("msg", "Invalid input")
-    prefix = "Value error, "
-    return msg[len(prefix) :] if msg.startswith(prefix) else msg
-
-
-@app.exception_handler(RequestValidationError)
-async def on_validation_error(request: Request, exc: RequestValidationError):
-    errors = [_friendly_error(e) for e in exc.errors()]
-    if wants_json(request):
-        fields: dict = {}
-        for raw, friendly in zip(exc.errors(), errors):
-            name = raw["loc"][-1] if raw.get("loc") else "_"
-            fields.setdefault(str(name), []).append(friendly)
-        return JSONResponse(
-            ValidationErrorOut(fields=fields, messages=errors).model_dump(mode="json"),
-            status_code=422,
-        )
-    path = request.url.path
-    page = _AUTH_FORM_PAGES.get(path)
-    if page is None and path.startswith("/auth/reset-password/"):
-        page = ("reset_password.html", "Set New Password")
-    if page:
-        template_name, title = page
-        context = {
-            **base_seo_context(request, title=title, robots="noindex,nofollow"),
-            "request": request,
-            "errors": errors,
-        }
-        try:
-            form = await request.form()
-            context.update({k: v for k, v in form.items() if isinstance(v, str)})
-        except Exception:
-            pass
-        if "token" in request.path_params:
-            context["token"] = request.path_params["token"]
-        return templates.TemplateResponse(
-            request, template_name, context, status_code=400
-        )
-    referer = safe_next(request.headers.get("referer"), "/feed")
-    return RedirectResponse(url=referer, status_code=303)
-
+error_handlers.register(app)
 
 app.include_router(auth.router, prefix="/auth")
 app.include_router(feed.router, prefix="/feed")
@@ -534,359 +140,6 @@ app.include_router(battles.router, prefix="/battles")
 app.include_router(nodes.router, prefix="/nodes")
 app.include_router(sync.router, prefix="/sync")
 app.include_router(workspaces.router, prefix="/workspaces")
+app.include_router(landing.router)
 
-
-@app.middleware("http")
-async def refresh_db_snapshot(request: Request, call_next):
-    if not request.url.path.startswith(("/static", "/avatar")):
-        refresh_snapshot()
-    return await call_next(request)
-
-
-@app.middleware("http")
-async def await_pending_corrections(request: Request, call_next):
-    response = await call_next(request)
-    pending = request.scope.get(PENDING_SCOPE_KEY)
-    if pending:
-        await asyncio.gather(*pending, return_exceptions=True)
-    return response
-
-
-def _frame_ancestors() -> str:
-    from devplacepy.services.containers.workspace import naming
-
-    tunnel_domain = naming.domain()
-    if not tunnel_domain:
-        return "'self'"
-    return f"'self' https://*.{tunnel_domain}"
-
-
-@app.middleware("http")
-async def add_security_headers(request: Request, call_next):
-    response = await call_next(request)
-    if not response.headers.get("X-Robots-Tag"):
-        response.headers["X-Robots-Tag"] = "index, follow"
-    response.headers["X-Content-Type-Options"] = "nosniff"
-    response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains"
-    response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
-    if not request.url.path.startswith("/p/"):
-        response.headers["Content-Security-Policy"] = (
-            "object-src 'none'; base-uri 'self'; "
-            f"frame-ancestors {_frame_ancestors()}; form-action 'self'"
-        )
-    if request.url.path.startswith("/admin"):
-        response.headers["Cache-Control"] = "no-store, no-cache, must-revalidate, max-age=0"
-        response.headers["Pragma"] = "no-cache"
-        response.headers["Expires"] = "0"
-    return response
-
-
-@app.middleware("http")
-async def rate_limit_middleware(request: Request, call_next):
-    if RATE_LIMIT_DISABLED:
-        return await call_next(request)
-    if request.method in (
-        "POST",
-        "PUT",
-        "DELETE",
-        "PATCH",
-    ) and not request.url.path.startswith(("/openai", "/xmlrpc")):
-        settings = _hot_settings()
-        limit = _worker_rate_limit(settings["rate_limit_per_minute"])
-        window = settings["rate_limit_window_seconds"]
-        ip = client_ip(request, default="unknown")
-        now = time.time()
-        window_start = now - window
-        global _last_rate_sweep
-        if now - _last_rate_sweep >= RATE_SWEEP_INTERVAL:
-            _sweep_rate_limit_store(window_start)
-            _last_rate_sweep = now
-        timestamps = [t for t in _rate_limit_store.get(ip, ()) if t > window_start]
-        if len(timestamps) >= limit:
-            _rate_limit_store[ip] = timestamps
-            audit.record(
-                request,
-                "security.rate_limit.block",
-                result="denied",
-                summary=f"request from {ip} blocked by rate limit",
-                metadata={"ip": ip, "limit": limit, "window_seconds": window},
-            )
-            retry_after = {"Retry-After": str(window)}
-            if wants_json(request):
-                response = json_error(429, "Rate limit exceeded. Try again later.")
-                response.headers["Retry-After"] = str(window)
-                return response
-            return HTMLResponse(
-                "Rate limit exceeded. Try again later.",
-                status_code=429,
-                headers=retry_after,
-            )
-        timestamps.append(now)
-        _rate_limit_store[ip] = timestamps
-    return await call_next(request)
-
-
-_MAINTENANCE_ALLOWED_PREFIXES = ("/static", "/avatar", "/auth", "/admin", "/openai")
-
-
-@app.middleware("http")
-async def maintenance_middleware(request: Request, call_next):
-    if _hot_settings()["maintenance_mode"] != "1":
-        return await call_next(request)
-    if request.url.path.startswith(_MAINTENANCE_ALLOWED_PREFIXES):
-        return await call_next(request)
-    user = get_current_user(request)
-    if user and user.get("role") == "Admin":
-        return await call_next(request)
-    message = get_setting(
-        "maintenance_message",
-        "DevPlace is undergoing scheduled maintenance. Please check back shortly.",
-    )
-    audit.record(
-        request,
-        "security.maintenance.block",
-        user=user,
-        result="denied",
-        summary="non-admin request blocked by maintenance mode",
-    )
-    if wants_json(request):
-        return json_error(503, message)
-    seo_ctx = base_seo_context(
-        request, title="Maintenance - DevPlace", description=message, robots="noindex"
-    )
-    return templates.TemplateResponse(
-        request,
-        "error.html",
-        {**seo_ctx, "request": request, "error_code": 503, "error_message": message},
-        status_code=503,
-    )
-
-
-_TERMS_ALLOWED_PREFIXES = (
-    "/static",
-    "/avatar",
-    "/auth",
-    "/docs",
-    "/reports",
-    "/block",
-    "/mute",
-    "/openai",
-)
-
-_TERMS_GATED_METHODS = ("POST", "PUT", "DELETE", "PATCH")
-
-
-def _terms_exempt(path: str) -> bool:
-    if path.startswith(_TERMS_ALLOWED_PREFIXES):
-        return True
-    return path.startswith("/profile/") and (
-        path.endswith("/delete") or path.endswith("/consent")
-    )
-
-
-@app.middleware("http")
-async def terms_acceptance_gate(request: Request, call_next):
-    if request.method not in _TERMS_GATED_METHODS or _terms_exempt(request.url.path):
-        return await call_next(request)
-    from devplacepy.routers.auth.terms import (
-        TERMS_ACCEPTANCE_CODE,
-        current_terms_version,
-        needs_acceptance,
-    )
-
-    user = get_current_user(request)
-    if not needs_acceptance(user):
-        return await call_next(request)
-    message = "Accept the updated Terms of Service to continue."
-    if wants_json(request):
-        return json_error(
-            403,
-            message,
-            code=TERMS_ACCEPTANCE_CODE,
-            redirect="/auth/accept-terms",
-            terms_version=current_terms_version(),
-        )
-    return RedirectResponse(url="/auth/accept-terms", status_code=303)
-
-
-@app.middleware("http")
-async def track_presence(request: Request, call_next):
-    path = request.url.path
-    if not path.startswith(("/static", "/avatar")):
-        user = get_current_user(request)
-        if user:
-            presence.touch(user["uid"])
-    return await call_next(request)
-
-
-@app.middleware("http")
-async def visit_statistics(request: Request, call_next):
-    from devplacepy.services.statistics.tracking import track_visit
-
-    response = await call_next(request)
-    track_visit(request, response.status_code)
-    return response
-
-
-@app.middleware("http")
-async def response_timing(request: Request, call_next):
-    start = time.perf_counter()
-    request.state.request_start = start
-    response = await call_next(request)
-    response.headers["X-Response-Time"] = f"{(time.perf_counter() - start) * 1000:.1f}ms"
-    return response
-
-
-class TunnelDispatchMiddleware:
-    def __init__(self, app):
-        self.app = app
-
-    @staticmethod
-    def _host(scope) -> str:
-        for key, value in scope.get("headers") or []:
-            if key == b"host":
-                return value.decode("latin-1")
-        return ""
-
-    async def __call__(self, scope, receive, send):
-        if scope["type"] not in ("http", "websocket"):
-            await self.app(scope, receive, send)
-            return
-        from devplacepy.routers import tunnel as tunnel_router
-        from devplacepy.services.containers.workspace import naming
-
-        try:
-            if not naming.is_tunnel_host(self._host(scope)):
-                await self.app(scope, receive, send)
-                return
-        except Exception:
-            await self.app(scope, receive, send)
-            return
-        path = (scope.get("path") or "/").lstrip("/")
-        if scope["type"] == "websocket":
-            websocket = WebSocket(scope, receive, send)
-            await tunnel_router.handle_ws(websocket, path)
-            return
-        request = Request(scope, receive)
-        response = await tunnel_router.handle_http(request, path)
-        await response(scope, receive, send)
-
-
-app.add_middleware(TunnelDispatchMiddleware)
-app.add_middleware(GZipMiddleware, minimum_size=512, compresslevel=5)
-
-
-_home_cache = TTLCache(ttl=int(os.environ.get("DEVPLACE_HOME_CACHE_TTL", "60")), max_size=4)
-
-
-def _landing_news():
-    cached = _home_cache.get("news")
-    if cached is not None:
-        return cached
-    articles = []
-    if "news" in db.tables:
-        news_table = get_table("news")
-        raw = list(
-            news_table.find(show_on_landing=1, order_by=["-synced_at"], _limit=6)
-        )
-        images_by_news = get_news_images_by_uids([a["uid"] for a in raw])
-        for a in raw:
-            articles.append(
-                {
-                    "uid": a["uid"],
-                    "slug": a.get("slug", ""),
-                    "title": a.get("title", ""),
-                    "description": (a.get("description", "") or "")[:250],
-                    "url": a.get("url", ""),
-                    "source_name": a.get("source_name", ""),
-                    "grade": a.get("grade", 0),
-                    "featured": a.get("featured", 0),
-                    "synced_at": a.get("synced_at", "") or "",
-                    "time_ago": time_ago(a["synced_at"]),
-                    "image_url": a.get("image_url", "") or images_by_news.get(a["uid"], ""),
-                }
-            )
-    _home_cache.set("news", articles)
-    return articles
-
-
-def _landing_recent_posts(viewer_uid):
-    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
-    if not blocked:
-        cached = _home_cache.get("posts")
-        if cached is not None:
-            return cached
-    posts = []
-    if "posts" in db.tables:
-        posts_table = get_table("posts")
-        raw_posts, _ = paginate_diverse(
-            posts_table, order=["-created_at"], viewer_uid=viewer_uid, limit=6
-        )
-        if raw_posts:
-            post_uids = [p["uid"] for p in raw_posts]
-            author_uids = [p["user_uid"] for p in raw_posts]
-            authors = get_users_by_uids(author_uids)
-            comment_counts = get_comment_counts_by_post_uids(post_uids)
-            upvotes, downvotes = get_vote_counts(post_uids)
-            for p in raw_posts:
-                posts.append(
-                    {
-                        "post": p,
-                        "author": authors.get(p["user_uid"]),
-                        "time_ago": time_ago(p["created_at"]),
-                        "comment_count": comment_counts.get(p["uid"], 0),
-                        "stars": upvotes.get(p["uid"], 0) - downvotes.get(p["uid"], 0),
-                        "slug": p.get("slug", "") or p["uid"],
-                    }
-                )
-    if not blocked:
-        _home_cache.set("posts", posts)
-    return posts
-
-
-@app.get("/")
-async def landing(request: Request):
-    user = get_current_user(request)
-
-    landing_articles = _landing_news()
-    viewer_uid = user["uid"] if user else None
-    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
-    landing_posts = _landing_recent_posts(viewer_uid)
-
-    base = site_url(request)
-    seo_ctx = base_seo_context(
-        request,
-        title="DevPlace - The Developer Social Network",
-        description="Track industry shifts. Discover bold releases. Share what you're building in an open environment built by developers, for developers.",
-        breadcrumbs=[],
-        schemas=[website_schema(base)],
-    )
-    user_xp = user.get("xp", 0) or 0 if user else 0
-    user_level = user.get("level", 1) or 1 if user else 1
-    xp_progress_pct = (user_xp % 100) if user_xp else 0
-    unread_count = jinja_unread_count(user["uid"]) if user else 0
-
-    top_contributors = get_top_authors(5) if not blocked else []
-    trending_topics = get_trending_topics(6) if not blocked else []
-
-    return respond(
-        request,
-        "landing.html",
-        {
-            **seo_ctx,
-            "request": request,
-            "user": user,
-            "is_authenticated": bool(user),
-            "user_post_count": get_user_post_count(user["uid"]) if user else 0,
-            "user_stars": get_user_stars(user["uid"]) if user else 0,
-            "user_xp": user_xp,
-            "user_level": user_level,
-            "xp_progress_pct": xp_progress_pct,
-            "unread_count": unread_count,
-            "landing_articles": landing_articles,
-            "landing_posts": landing_posts,
-            "top_contributors": top_contributors,
-            "trending_topics": trending_topics,
-        },
-        model=LandingOut,
-    )
+register_middleware(app)

devplacepy/middleware/__init__.py

@@ -0,0 +1,34 @@
+# retoor <retoor@molodetz.nl>
+
+from starlette.middleware.gzip import GZipMiddleware
+
+from devplacepy.middleware.guards import (
+    maintenance_middleware,
+    rate_limit_middleware,
+    terms_acceptance_gate,
+)
+from devplacepy.middleware.observe import (
+    response_timing,
+    track_presence,
+    visit_statistics,
+)
+from devplacepy.middleware.pipeline import (
+    add_security_headers,
+    await_pending_corrections,
+    refresh_db_snapshot,
+)
+from devplacepy.middleware.tunnel import TunnelDispatchMiddleware
+
+
+def register(app):
+    app.middleware("http")(refresh_db_snapshot)
+    app.middleware("http")(await_pending_corrections)
+    app.middleware("http")(add_security_headers)
+    app.middleware("http")(rate_limit_middleware)
+    app.middleware("http")(maintenance_middleware)
+    app.middleware("http")(terms_acceptance_gate)
+    app.middleware("http")(track_presence)
+    app.middleware("http")(visit_statistics)
+    app.middleware("http")(response_timing)
+    app.add_middleware(TunnelDispatchMiddleware)
+    app.add_middleware(GZipMiddleware, minimum_size=512, compresslevel=5)

devplacepy/middleware/guards.py

@@ -0,0 +1,138 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+
+from fastapi import Request
+from fastapi.responses import HTMLResponse, RedirectResponse
+
+from devplacepy.database import get_setting
+from devplacepy.middleware import state
+from devplacepy.responses import json_error, wants_json
+from devplacepy.seo import base_seo_context
+from devplacepy.services.audit import record as audit
+from devplacepy.templating import templates
+from devplacepy.utils import client_ip, get_current_user
+
+
+async def rate_limit_middleware(request: Request, call_next):
+    if state.RATE_LIMIT_DISABLED:
+        return await call_next(request)
+    if request.method in (
+        "POST",
+        "PUT",
+        "DELETE",
+        "PATCH",
+    ) and not request.url.path.startswith(("/openai", "/xmlrpc")):
+        settings = state._hot_settings()
+        limit = state._worker_rate_limit(settings["rate_limit_per_minute"])
+        window = settings["rate_limit_window_seconds"]
+        ip = client_ip(request, default="unknown")
+        now = time.time()
+        window_start = now - window
+        if now - state._last_rate_sweep >= state.RATE_SWEEP_INTERVAL:
+            state._sweep_rate_limit_store(window_start)
+            state._last_rate_sweep = now
+        timestamps = [t for t in state._rate_limit_store.get(ip, ()) if t > window_start]
+        if len(timestamps) >= limit:
+            state._rate_limit_store[ip] = timestamps
+            audit.record(
+                request,
+                "security.rate_limit.block",
+                result="denied",
+                summary=f"request from {ip} blocked by rate limit",
+                metadata={"ip": ip, "limit": limit, "window_seconds": window},
+            )
+            retry_after = {"Retry-After": str(window)}
+            if wants_json(request):
+                response = json_error(429, "Rate limit exceeded. Try again later.")
+                response.headers["Retry-After"] = str(window)
+                return response
+            return HTMLResponse(
+                "Rate limit exceeded. Try again later.",
+                status_code=429,
+                headers=retry_after,
+            )
+        timestamps.append(now)
+        state._rate_limit_store[ip] = timestamps
+    return await call_next(request)
+
+
+_MAINTENANCE_ALLOWED_PREFIXES = ("/static", "/avatar", "/auth", "/admin", "/openai")
+
+
+async def maintenance_middleware(request: Request, call_next):
+    if state._hot_settings()["maintenance_mode"] != "1":
+        return await call_next(request)
+    if request.url.path.startswith(_MAINTENANCE_ALLOWED_PREFIXES):
+        return await call_next(request)
+    user = get_current_user(request)
+    if user and user.get("role") == "Admin":
+        return await call_next(request)
+    message = get_setting(
+        "maintenance_message",
+        "DevPlace is undergoing scheduled maintenance. Please check back shortly.",
+    )
+    audit.record(
+        request,
+        "security.maintenance.block",
+        user=user,
+        result="denied",
+        summary="non-admin request blocked by maintenance mode",
+    )
+    if wants_json(request):
+        return json_error(503, message)
+    seo_ctx = base_seo_context(
+        request, title="Maintenance - DevPlace", description=message, robots="noindex"
+    )
+    return templates.TemplateResponse(
+        request,
+        "error.html",
+        {**seo_ctx, "request": request, "error_code": 503, "error_message": message},
+        status_code=503,
+    )
+
+
+_TERMS_ALLOWED_PREFIXES = (
+    "/static",
+    "/avatar",
+    "/auth",
+    "/docs",
+    "/reports",
+    "/block",
+    "/mute",
+    "/openai",
+)
+
+_TERMS_GATED_METHODS = ("POST", "PUT", "DELETE", "PATCH")
+
+
+def _terms_exempt(path: str) -> bool:
+    if path.startswith(_TERMS_ALLOWED_PREFIXES):
+        return True
+    return path.startswith("/profile/") and (
+        path.endswith("/delete") or path.endswith("/consent")
+    )
+
+
+async def terms_acceptance_gate(request: Request, call_next):
+    if request.method not in _TERMS_GATED_METHODS or _terms_exempt(request.url.path):
+        return await call_next(request)
+    from devplacepy.routers.auth.terms import (
+        TERMS_ACCEPTANCE_CODE,
+        current_terms_version,
+        needs_acceptance,
+    )
+
+    user = get_current_user(request)
+    if not needs_acceptance(user):
+        return await call_next(request)
+    message = "Accept the updated Terms of Service to continue."
+    if wants_json(request):
+        return json_error(
+            403,
+            message,
+            code=TERMS_ACCEPTANCE_CODE,
+            redirect="/auth/accept-terms",
+            terms_version=current_terms_version(),
+        )
+    return RedirectResponse(url="/auth/accept-terms", status_code=303)

devplacepy/middleware/observe.py

@@ -0,0 +1,33 @@
+# retoor <retoor@molodetz.nl>
+
+import time
+
+from fastapi import Request
+
+from devplacepy.services import presence
+from devplacepy.utils import get_current_user
+
+
+async def track_presence(request: Request, call_next):
+    path = request.url.path
+    if not path.startswith(("/static", "/avatar")):
+        user = get_current_user(request)
+        if user:
+            presence.touch(user["uid"])
+    return await call_next(request)
+
+
+async def visit_statistics(request: Request, call_next):
+    from devplacepy.services.statistics.tracking import track_visit
+
+    response = await call_next(request)
+    track_visit(request, response.status_code)
+    return response
+
+
+async def response_timing(request: Request, call_next):
+    start = time.perf_counter()
+    request.state.request_start = start
+    response = await call_next(request)
+    response.headers["X-Response-Time"] = f"{(time.perf_counter() - start) * 1000:.1f}ms"
+    return response

devplacepy/middleware/pipeline.py

@@ -0,0 +1,50 @@
+# retoor <retoor@molodetz.nl>
+
+import asyncio
+
+from fastapi import Request
+
+from devplacepy.database import refresh_snapshot
+from devplacepy.services.correction import PENDING_SCOPE_KEY
+
+
+async def refresh_db_snapshot(request: Request, call_next):
+    if not request.url.path.startswith(("/static", "/avatar")):
+        refresh_snapshot()
+    return await call_next(request)
+
+
+async def await_pending_corrections(request: Request, call_next):
+    response = await call_next(request)
+    pending = request.scope.get(PENDING_SCOPE_KEY)
+    if pending:
+        await asyncio.gather(*pending, return_exceptions=True)
+    return response
+
+
+def _frame_ancestors() -> str:
+    from devplacepy.services.containers.workspace import naming
+
+    tunnel_domain = naming.domain()
+    if not tunnel_domain:
+        return "'self'"
+    return f"'self' https://*.{tunnel_domain}"
+
+
+async def add_security_headers(request: Request, call_next):
+    response = await call_next(request)
+    if not response.headers.get("X-Robots-Tag"):
+        response.headers["X-Robots-Tag"] = "index, follow"
+    response.headers["X-Content-Type-Options"] = "nosniff"
+    response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains"
+    response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
+    if not request.url.path.startswith("/p/"):
+        response.headers["Content-Security-Policy"] = (
+            "object-src 'none'; base-uri 'self'; "
+            f"frame-ancestors {_frame_ancestors()}; form-action 'self'"
+        )
+    if request.url.path.startswith("/admin"):
+        response.headers["Cache-Control"] = "no-store, no-cache, must-revalidate, max-age=0"
+        response.headers["Pragma"] = "no-cache"
+        response.headers["Expires"] = "0"
+    return response

devplacepy/middleware/state.py

@@ -0,0 +1,50 @@
+# retoor <retoor@molodetz.nl>
+
+import os
+import time
+from collections import defaultdict
+
+from devplacepy.database import get_int_setting, get_setting
+
+_rate_limit_store = defaultdict(list)
+RATE_LIMIT = int(os.environ.get("DEVPLACE_RATE_LIMIT", "60"))
+RATE_WINDOW = 60
+WEB_WORKERS = max(1, int(os.environ.get("DEVPLACE_WEB_WORKERS", "1")))
+RATE_LIMIT_DISABLED = os.environ.get("DEVPLACE_DISABLE_RATE_LIMIT") == "1"
+
+HOT_SETTINGS_TTL = 2.0
+_hot_settings_value: dict = {}
+_hot_settings_at = 0.0
+_last_rate_sweep = 0.0
+RATE_SWEEP_INTERVAL = 60.0
+
+
+def _hot_settings() -> dict:
+    global _hot_settings_value, _hot_settings_at
+    now = time.monotonic()
+    if not _hot_settings_value or now - _hot_settings_at >= HOT_SETTINGS_TTL:
+        _hot_settings_value = {
+            "maintenance_mode": get_setting("maintenance_mode", "0"),
+            "rate_limit_per_minute": max(
+                1, get_int_setting("rate_limit_per_minute", RATE_LIMIT)
+            ),
+            "rate_limit_window_seconds": max(
+                1, get_int_setting("rate_limit_window_seconds", RATE_WINDOW)
+            ),
+        }
+        _hot_settings_at = now
+    return _hot_settings_value
+
+
+def _sweep_rate_limit_store(window_start: float) -> None:
+    stale = [
+        ip
+        for ip, timestamps in _rate_limit_store.items()
+        if not timestamps or timestamps[-1] <= window_start
+    ]
+    for ip in stale:
+        del _rate_limit_store[ip]
+
+
+def _worker_rate_limit(limit: int) -> int:
+    return max(1, -(-limit // WEB_WORKERS))

devplacepy/middleware/tunnel.py

@@ -0,0 +1,38 @@
+# retoor <retoor@molodetz.nl>
+
+from fastapi import Request, WebSocket
+
+
+class TunnelDispatchMiddleware:
+    def __init__(self, app):
+        self.app = app
+
+    @staticmethod
+    def _host(scope) -> str:
+        for key, value in scope.get("headers") or []:
+            if key == b"host":
+                return value.decode("latin-1")
+        return ""
+
+    async def __call__(self, scope, receive, send):
+        if scope["type"] not in ("http", "websocket"):
+            await self.app(scope, receive, send)
+            return
+        from devplacepy.routers import tunnel as tunnel_router
+        from devplacepy.services.containers.workspace import naming
+
+        try:
+            if not naming.is_tunnel_host(self._host(scope)):
+                await self.app(scope, receive, send)
+                return
+        except Exception:
+            await self.app(scope, receive, send)
+            return
+        path = (scope.get("path") or "/").lstrip("/")
+        if scope["type"] == "websocket":
+            websocket = WebSocket(scope, receive, send)
+            await tunnel_router.handle_ws(websocket, path)
+            return
+        request = Request(scope, receive)
+        response = await tunnel_router.handle_http(request, path)
+        await response(scope, receive, send)

devplacepy/models/__init__.py

@@ -0,0 +1,289 @@
+# retoor <retoor@molodetz.nl>
+
+from devplacepy.models.shared import (
+    normalize_european_date,
+    normalize_website_url,
+    normalize_poll_options,
+    _declared_age,
+    normalize_index_list,
+)
+from devplacepy.models.auth import (
+    SignupForm,
+    LoginForm,
+    ForgotPasswordForm,
+    ResetPasswordForm,
+)
+from devplacepy.models.content import (
+    PostForm,
+    WarJoinForm,
+    PostEditForm,
+    CommentForm,
+    CommentEditForm,
+    NoteForm,
+    VoteForm,
+    ReactionForm,
+    PollVoteForm,
+)
+from devplacepy.models.projects import (
+    ProjectForm,
+    ProjectEditForm,
+    ProjectFlagForm,
+    ProjectScreenshotsForm,
+    ForkForm,
+    ProjectFileWriteForm,
+    ProjectFileMkdirForm,
+    ProjectFileMoveForm,
+    ProjectFileDeleteForm,
+    ProjectFileReplaceLinesForm,
+    ProjectFileInsertLinesForm,
+    ProjectFileDeleteLinesForm,
+    ProjectFileAppendForm,
+)
+from devplacepy.models.profile import (
+    CustomizationToggleForm,
+    AwardGiveForm,
+    NotificationPrefForm,
+    AiCorrectionForm,
+    AiModifierForm,
+    InteractionsForm,
+    TelegramPairForm,
+    ProfileForm,
+    ConsentForm,
+    MaturityForm,
+    MaturePreferenceForm,
+    AccountDeleteForm,
+)
+from devplacepy.models.containers import (
+    ContainerInstanceForm,
+    ContainerAdminCreateForm,
+    ContainerEditForm,
+    ContainerExecForm,
+    ContainerScheduleForm,
+    TunnelForm,
+    EditorPrefsForm,
+)
+from devplacepy.models.community import (
+    SkillInstallForm,
+    SkillEnabledForm,
+    SkillDeleteForm,
+    MessageForm,
+    GistForm,
+    GistEditForm,
+    IssueForm,
+    IssueCommentForm,
+    IssueAttachmentForm,
+    IssueStatusForm,
+)
+from devplacepy.models.uploads import (
+    UploadUrlForm,
+    GenerateImageForm,
+    AttachmentRenameForm,
+)
+from devplacepy.models.tools import (
+    PlanningForm,
+    SeoRunForm,
+    SEO_URL_PATTERN,
+    ISSLOP_URL_PATTERN,
+    ISSLOP_SINGLE_SLASH_PATTERN,
+    ISSLOP_SCHEME_PATTERN,
+    IsslopRunForm,
+    DEEPSEARCH_MIN_DEPTH,
+    DEEPSEARCH_MAX_DEPTH,
+    DEEPSEARCH_DEFAULT_DEPTH,
+    DEEPSEARCH_MIN_PAGES,
+    DEEPSEARCH_MAX_PAGES,
+    DEEPSEARCH_DEFAULT_PAGES,
+    DEEPSEARCH_MIN_QUERY,
+    DEEPSEARCH_MAX_QUERY,
+    DEEPSEARCH_MAX_MESSAGE,
+    DeepsearchRunForm,
+    DeepsearchChatForm,
+)
+from devplacepy.models.quizzes import (
+    QUIZ_KINDS,
+    QUIZ_OPTION_KINDS,
+    QuizForm,
+    QuizQuestionForm,
+    QuizReorderForm,
+    QuizAnswerForm,
+    QuizDocumentOption,
+    QuizDocumentQuestion,
+    QuizDocumentSettings,
+    QuizDocument,
+    QuizImportForm,
+)
+from devplacepy.models.game import (
+    GamePlantForm,
+    GameSlotForm,
+    GamePlotStopForm,
+    GamePerkForm,
+    GameQuestForm,
+    GameLegacyForm,
+    GameInfraForm,
+    GameCosmeticForm,
+    GameMasteryForm,
+    GameEraStartForm,
+)
+from devplacepy.models.admin import (
+    CommunityAddForm,
+    CommunitySchemaForm,
+    CommunityUpdateForm,
+    CommunityRunForm,
+    BackupRunForm,
+    BackupScheduleForm,
+    NotificationDefaultForm,
+    AdminRoleForm,
+    AdminPasswordForm,
+    AdminSettingsForm,
+    WorkspaceQuotaForm,
+    WorkspaceFlagForm,
+    WorkspaceSuspendForm,
+    ReportForm,
+    ReportStatusForm,
+    ModerationDecisionForm,
+    SuspensionForm,
+    BanForm,
+    NodeSettingsForm,
+    NodeFetchForm,
+    NODE_SUSPEND_MIN_MINUTES,
+    NODE_SUSPEND_MAX_MINUTES,
+    NODE_SETTING_FORM_MAX,
+    AdminNodeKillswitchForm,
+    AdminNodeSettingsForm,
+    AdminNodeClassForm,
+    AdminNodeSuspendForm,
+    NodeDeviceForm,
+)
+
+__all__ = [
+    "normalize_european_date",
+    "normalize_website_url",
+    "normalize_poll_options",
+    "_declared_age",
+    "normalize_index_list",
+    "SignupForm",
+    "LoginForm",
+    "ForgotPasswordForm",
+    "ResetPasswordForm",
+    "PostForm",
+    "WarJoinForm",
+    "PostEditForm",
+    "CommentForm",
+    "CommentEditForm",
+    "NoteForm",
+    "VoteForm",
+    "ReactionForm",
+    "PollVoteForm",
+    "ProjectForm",
+    "ProjectEditForm",
+    "ProjectFlagForm",
+    "ProjectScreenshotsForm",
+    "ForkForm",
+    "ProjectFileWriteForm",
+    "ProjectFileMkdirForm",
+    "ProjectFileMoveForm",
+    "ProjectFileDeleteForm",
+    "ProjectFileReplaceLinesForm",
+    "ProjectFileInsertLinesForm",
+    "ProjectFileDeleteLinesForm",
+    "ProjectFileAppendForm",
+    "CustomizationToggleForm",
+    "AwardGiveForm",
+    "NotificationPrefForm",
+    "AiCorrectionForm",
+    "AiModifierForm",
+    "InteractionsForm",
+    "TelegramPairForm",
+    "ProfileForm",
+    "ConsentForm",
+    "MaturityForm",
+    "MaturePreferenceForm",
+    "AccountDeleteForm",
+    "ContainerInstanceForm",
+    "ContainerAdminCreateForm",
+    "ContainerEditForm",
+    "ContainerExecForm",
+    "ContainerScheduleForm",
+    "TunnelForm",
+    "EditorPrefsForm",
+    "SkillInstallForm",
+    "SkillEnabledForm",
+    "SkillDeleteForm",
+    "MessageForm",
+    "GistForm",
+    "GistEditForm",
+    "IssueForm",
+    "IssueCommentForm",
+    "IssueAttachmentForm",
+    "IssueStatusForm",
+    "UploadUrlForm",
+    "GenerateImageForm",
+    "AttachmentRenameForm",
+    "PlanningForm",
+    "SeoRunForm",
+    "SEO_URL_PATTERN",
+    "ISSLOP_URL_PATTERN",
+    "ISSLOP_SINGLE_SLASH_PATTERN",
+    "ISSLOP_SCHEME_PATTERN",
+    "IsslopRunForm",
+    "DEEPSEARCH_MIN_DEPTH",
+    "DEEPSEARCH_MAX_DEPTH",
+    "DEEPSEARCH_DEFAULT_DEPTH",
+    "DEEPSEARCH_MIN_PAGES",
+    "DEEPSEARCH_MAX_PAGES",
+    "DEEPSEARCH_DEFAULT_PAGES",
+    "DEEPSEARCH_MIN_QUERY",
+    "DEEPSEARCH_MAX_QUERY",
+    "DEEPSEARCH_MAX_MESSAGE",
+    "DeepsearchRunForm",
+    "DeepsearchChatForm",
+    "QUIZ_KINDS",
+    "QUIZ_OPTION_KINDS",
+    "QuizForm",
+    "QuizQuestionForm",
+    "QuizReorderForm",
+    "QuizAnswerForm",
+    "QuizDocumentOption",
+    "QuizDocumentQuestion",
+    "QuizDocumentSettings",
+    "QuizDocument",
+    "QuizImportForm",
+    "GamePlantForm",
+    "GameSlotForm",
+    "GamePlotStopForm",
+    "GamePerkForm",
+    "GameQuestForm",
+    "GameLegacyForm",
+    "GameInfraForm",
+    "GameCosmeticForm",
+    "GameMasteryForm",
+    "GameEraStartForm",
+    "CommunityAddForm",
+    "CommunitySchemaForm",
+    "CommunityUpdateForm",
+    "CommunityRunForm",
+    "BackupRunForm",
+    "BackupScheduleForm",
+    "NotificationDefaultForm",
+    "AdminRoleForm",
+    "AdminPasswordForm",
+    "AdminSettingsForm",
+    "WorkspaceQuotaForm",
+    "WorkspaceFlagForm",
+    "WorkspaceSuspendForm",
+    "ReportForm",
+    "ReportStatusForm",
+    "ModerationDecisionForm",
+    "SuspensionForm",
+    "BanForm",
+    "NodeSettingsForm",
+    "NodeFetchForm",
+    "NODE_SUSPEND_MIN_MINUTES",
+    "NODE_SUSPEND_MAX_MINUTES",
+    "NODE_SETTING_FORM_MAX",
+    "AdminNodeKillswitchForm",
+    "AdminNodeSettingsForm",
+    "AdminNodeClassForm",
+    "AdminNodeSuspendForm",
+    "NodeDeviceForm",
+]

devplacepy/models/admin.py

@@ -0,0 +1,299 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Literal, Optional
+from urllib.parse import urlsplit
+from pydantic import BaseModel, Field, field_validator, model_validator
+
+
+class CommunityAddForm(BaseModel):
+    query: str = Field(min_length=2, max_length=200)
+    login_username: str = Field(default="", max_length=200)
+    login_email: str = Field(default="", max_length=254)
+    login_password: str = Field(default="", max_length=500)
+    login_totp_secret: str = Field(default="", max_length=200)
+
+    @field_validator("query")
+    @classmethod
+    def trimmed(cls, value):
+        return value.strip()
+
+
+class CommunitySchemaForm(BaseModel):
+    schema_text: str = Field(min_length=2, max_length=300_000, alias="schema")
+
+    model_config = {"populate_by_name": True}
+
+
+class CommunityUpdateForm(BaseModel):
+    interval_seconds: int = Field(ge=900, le=2592000)
+    enabled: bool = False
+    login_username: str = Field(default="", max_length=200)
+    login_email: str = Field(default="", max_length=254)
+    login_password: str = Field(default="", max_length=500)
+    login_totp_secret: str = Field(default="", max_length=200)
+    clear_login: bool = False
+    alias: Optional[bool] = None
+    translate_to: Optional[str] = Field(default=None, max_length=12)
+    sample_percent: Optional[int] = Field(default=None, ge=1, le=100)
+    min_comments: Optional[int] = Field(default=None, ge=0, le=1000)
+    topics: Optional[str] = Field(default=None, max_length=500)
+    jitter_percent: Optional[int] = Field(default=None, ge=0, le=50)
+    collect_attachments: Optional[bool] = None
+
+
+class CommunityRunForm(BaseModel):
+    dry_run: bool = False
+
+
+class BackupRunForm(BaseModel):
+    target: Literal["database", "uploads", "keys", "full"] = "full"
+
+
+class BackupScheduleForm(BaseModel):
+    name: str = Field(min_length=1, max_length=120)
+    target: Literal["database", "uploads", "keys", "full"] = "full"
+    kind: Literal["interval", "cron"] = "interval"
+    every_seconds: int = Field(default=86400, ge=60)
+    cron: str = Field(default="", max_length=120)
+    keep_last: int = Field(default=7, ge=0, le=1000)
+
+    @model_validator(mode="after")
+    def _validate_schedule(self) -> "BackupScheduleForm":
+        if self.kind == "cron" and not self.cron.strip():
+            raise ValueError("A cron schedule requires a cron expression")
+        return self
+
+
+class NotificationDefaultForm(BaseModel):
+    notification_type: str = Field(min_length=1, max_length=40)
+    channel: Literal["in_app", "push", "telegram"]
+    value: bool = False
+
+
+class AdminRoleForm(BaseModel):
+    role: Literal["member", "admin"]
+
+
+class AdminPasswordForm(BaseModel):
+    password: str = Field(min_length=6, max_length=128)
+
+
+class AdminSettingsForm(BaseModel):
+    site_name: str = Field(default="", max_length=200)
+    site_description: str = Field(default="", max_length=500)
+    site_tagline: str = Field(default="", max_length=500)
+    site_url: str = Field(default="", max_length=300)
+    ios_app_url: str = Field(default="", max_length=300)
+    max_upload_size_mb: str = Field(default="", max_length=10)
+    allowed_file_types: str = Field(default="", max_length=1000)
+    max_attachments_per_resource: str = Field(default="", max_length=10)
+    rate_limit_per_minute: str = Field(default="", max_length=10)
+    rate_limit_window_seconds: str = Field(default="", max_length=10)
+    session_max_age_days: str = Field(default="", max_length=10)
+    session_remember_days: str = Field(default="", max_length=10)
+    registration_open: str = Field(default="", max_length=1)
+    maintenance_mode: str = Field(default="", max_length=1)
+    maintenance_message: str = Field(default="", max_length=300)
+    happy_404_enabled: str = Field(default="", max_length=1)
+    docs_search_mode: str = Field(default="", max_length=20)
+    outbound_proxy_url: str = Field(default="", max_length=500)
+    moderation_sla_hours: str = Field(default="", max_length=10)
+    moderation_filter_mode: str = Field(default="", max_length=10)
+    moderation_minimum_age: str = Field(default="", max_length=3)
+    moderation_mature_default_hidden: str = Field(default="", max_length=1)
+    account_deletion_grace_hours: str = Field(default="", max_length=10)
+    contact_email: str = Field(default="", max_length=200)
+    contact_phone: str = Field(default="", max_length=60)
+    contact_address: str = Field(default="", max_length=500)
+    terms_version: str = Field(default="", max_length=20)
+    privacy_version: str = Field(default="", max_length=20)
+    guidelines_version: str = Field(default="", max_length=20)
+    ai_third_party_provider: str = Field(default="", max_length=120)
+    correction_model: str = Field(default="", max_length=120)
+    modifier_model: str = Field(default="", max_length=120)
+    quiz_grading_model: str = Field(default="", max_length=120)
+    extra_head: str = Field(default="", max_length=50000)
+    sync_enabled: str = Field(default="", max_length=1)
+    sync_origins: str = Field(default="", max_length=1000)
+
+    @field_validator("sync_origins")
+    @classmethod
+    def validate_sync_origins(cls, value):
+        from devplacepy.services.sync.normalize import normalize_origin
+
+        items = [item for item in value.replace("\n", ",").split(",") if item.strip()]
+        return ",".join(dict.fromkeys(normalize_origin(item) for item in items))
+
+    @field_validator("moderation_filter_mode")
+    @classmethod
+    def validate_filter_mode(cls, value):
+        from devplacepy.services.moderation.rules import FILTER_MODES
+
+        if value and value not in FILTER_MODES:
+            raise ValueError(f"Filter mode must be one of {', '.join(FILTER_MODES)}")
+        return value
+
+    @field_validator("outbound_proxy_url")
+    @classmethod
+    def validate_outbound_proxy_url(cls, value):
+        text = value.strip()
+        if not text:
+            return text
+        parsed = urlsplit(text)
+        if parsed.scheme not in ("http", "https", "socks5", "socks5h") or not parsed.hostname:
+            raise ValueError("Proxy URL must be http(s):// or socks5(h):// with a host, e.g. http://user:pass@host:port")
+        return text
+
+
+class WorkspaceQuotaForm(BaseModel):
+    owner_id: str = Field(default="", max_length=36)
+    label: str = Field(default="", max_length=64)
+    max_workspaces: int = Field(default=0, ge=0, le=100)
+    max_tunnels: int = Field(default=0, ge=0, le=100)
+    disk_quota_mb: int = Field(default=0, ge=0)
+    egress_quota_mb: int = Field(default=0, ge=0)
+    idle_stop_minutes: int = Field(default=0, ge=0)
+    retention_days: int = Field(default=0, ge=0)
+    cpu_millicores: int = Field(default=0, ge=0, le=64000)
+    memory_mb: int = Field(default=0, ge=0, le=1048576)
+
+
+class WorkspaceFlagForm(BaseModel):
+    kind: str = Field(default="manual", max_length=40)
+    severity: str = Field(default="warn", max_length=16)
+    detail: str = Field(default="", max_length=500)
+
+
+class WorkspaceSuspendForm(BaseModel):
+    reason: str = Field(default="", max_length=500)
+
+
+class ReportForm(BaseModel):
+    reason: str = Field(min_length=1, max_length=40)
+    detail: str = Field(default="", max_length=2000)
+
+    @field_validator("reason")
+    @classmethod
+    def known_reason(cls, value):
+        from devplacepy.database.moderation import REPORT_REASONS
+
+        if value not in REPORT_REASONS:
+            raise ValueError("Unknown report reason")
+        return value
+
+
+class ReportStatusForm(BaseModel):
+    status: str = Field(default="acknowledged", max_length=20)
+
+    @field_validator("status")
+    @classmethod
+    def known_status(cls, value):
+        from devplacepy.database.moderation import REPORT_STATUSES
+
+        if value not in REPORT_STATUSES:
+            raise ValueError("Unknown report status")
+        return value
+
+
+class ModerationDecisionForm(BaseModel):
+    action: str = Field(min_length=1, max_length=30)
+    reason: str = Field(default="", max_length=200)
+    notes: str = Field(default="", max_length=2000)
+    duration_hours: int = Field(default=24, ge=1, le=8760)
+
+    @field_validator("action")
+    @classmethod
+    def known_action(cls, value):
+        from devplacepy.database.moderation import MODERATION_ACTIONS
+
+        if value not in MODERATION_ACTIONS:
+            raise ValueError("Unknown moderation action")
+        return value
+
+
+class SuspensionForm(BaseModel):
+    reason: str = Field(default="", max_length=200)
+    duration_hours: int = Field(default=24, ge=1, le=8760)
+
+
+class BanForm(BaseModel):
+    reason: str = Field(default="", max_length=200)
+
+
+class NodeSettingsForm(BaseModel):
+    enabled: Optional[bool] = None
+    bandwidth_mb_day: Optional[int] = Field(default=None, ge=1, le=20 * 1024)
+    max_concurrent: Optional[int] = Field(default=None, ge=1, le=16)
+    active_from: Optional[str] = Field(default=None, max_length=5)
+    active_to: Optional[str] = Field(default=None, max_length=5)
+    timezone: Optional[str] = Field(default=None, max_length=64)
+    allowed_platforms: Optional[list[str]] = Field(default=None, max_length=16)
+    pause_on_battery: Optional[bool] = None
+    public_display: Optional[bool] = None
+
+    @field_validator("allowed_platforms", mode="before")
+    @classmethod
+    def split_platforms(cls, value):
+        if isinstance(value, str):
+            return [item.strip() for item in value.split(",") if item.strip()]
+        return value
+
+
+class NodeFetchForm(BaseModel):
+    url: str = Field(max_length=2048)
+    cache: bool = True
+    timeout: int = Field(default=15, ge=1, le=60)
+
+
+NODE_SUSPEND_MIN_MINUTES = 1
+
+
+NODE_SUSPEND_MAX_MINUTES = 10080
+
+
+NODE_SETTING_FORM_MAX = 1_000_000
+
+
+class AdminNodeKillswitchForm(BaseModel):
+    enabled: Optional[bool] = None
+
+
+class AdminNodeSettingsForm(BaseModel):
+    nodes_max_total: Optional[int] = Field(default=None, ge=0, le=NODE_SETTING_FORM_MAX)
+    nodes_daily_quota_user: Optional[int] = Field(default=None, ge=0, le=NODE_SETTING_FORM_MAX)
+    nodes_daily_quota_guest: Optional[int] = Field(default=None, ge=0, le=NODE_SETTING_FORM_MAX)
+    nodes_cache_ttl_seconds: Optional[int] = Field(default=None, ge=0, le=NODE_SETTING_FORM_MAX)
+    nodes_social_via_nodes: Optional[Literal["0", "1"]] = None
+    nodes_daily_social_user: Optional[int] = Field(default=None, ge=0, le=NODE_SETTING_FORM_MAX)
+    nodes_allow_unknown_class: Optional[Literal["0", "1"]] = None
+    nodes_redundancy_percent: Optional[int] = Field(default=None, ge=0, le=NODE_SETTING_FORM_MAX)
+
+
+class AdminNodeClassForm(BaseModel):
+    value: Literal["allow", "clear"]
+
+
+class AdminNodeSuspendForm(BaseModel):
+    minutes: int = Field(ge=NODE_SUSPEND_MIN_MINUTES, le=NODE_SUSPEND_MAX_MINUTES)
+    reason: str = Field(default="", max_length=500)
+
+    @field_validator("reason")
+    @classmethod
+    def trimmed(cls, value):
+        return value.strip()
+
+
+class NodeDeviceForm(BaseModel):
+    label: str = Field(default="", max_length=60)
+    kind: str = Field(default="native", max_length=20)
+    platform: str = Field(default="", max_length=40)
+    client_version: str = Field(default="", max_length=40)
+
+    @field_validator("kind")
+    @classmethod
+    def known_kind(cls, value):
+        from devplacepy.services.nodes.spec import DEVICE_KINDS
+
+        if value not in DEVICE_KINDS:
+            raise ValueError("Unknown device kind")
+        return value

devplacepy/models/auth.py

@@ -0,0 +1,95 @@
+# retoor <retoor@molodetz.nl>
+
+from pydantic import BaseModel, Field, field_validator, model_validator
+from devplacepy.models.shared import _declared_age
+
+
+class SignupForm(BaseModel):
+    username: str = Field(min_length=3, max_length=32)
+    email: str = Field(min_length=1, max_length=255)
+    password: str = Field(min_length=6, max_length=128)
+    confirm_password: str = Field(min_length=1, max_length=128)
+    birth_date: str = Field(min_length=1, max_length=20)
+    accept_terms: str = Field(default="")
+
+    @property
+    def age_band(self) -> str:
+        from devplacepy.database.moderation import age_band_for
+
+        return age_band_for(_declared_age(self.birth_date))
+
+    @field_validator("birth_date")
+    @classmethod
+    def old_enough(cls, value):
+        from devplacepy.database import minimum_age
+
+        minimum = minimum_age()
+        if _declared_age(value) < minimum:
+            raise ValueError(f"You must be at least {minimum} years old to join")
+        return value
+
+    @field_validator("accept_terms")
+    @classmethod
+    def terms_accepted(cls, value):
+        if value.strip().lower() not in ("1", "on", "true", "yes"):
+            raise ValueError(
+                "You must accept the Terms of Service and Community Guidelines"
+            )
+        return value
+
+    @field_validator("username")
+    @classmethod
+    def username_chars(cls, value):
+        if not value.isascii() or not all(
+            c.isalnum() or c in ("-", "_") for c in value
+        ):
+            raise ValueError(
+                "Username can only contain letters, numbers, hyphens, and underscores"
+            )
+        from devplacepy.services.moderation.filter import classify
+
+        if classify(value).verdict == "block":
+            raise ValueError("That username breaks the community guidelines")
+        return value
+
+    @field_validator("email")
+    @classmethod
+    def email_has_at(cls, value):
+        if "@" not in value:
+            raise ValueError("Valid email is required")
+        return value
+
+    @model_validator(mode="after")
+    def passwords_match(self):
+        if self.password != self.confirm_password:
+            raise ValueError("Passwords do not match")
+        return self
+
+
+class LoginForm(BaseModel):
+    email: str = Field(min_length=1, max_length=255)
+    password: str = Field(min_length=1, max_length=128)
+    remember_me: str = ""
+    next: str = ""
+
+
+class ForgotPasswordForm(BaseModel):
+    email: str = Field(min_length=1, max_length=255)
+
+    @field_validator("email")
+    @classmethod
+    def email_has_at(cls, value):
+        if "@" not in value:
+            raise ValueError("Valid email is required")
+        return value
+
+
+class ResetPasswordForm(BaseModel):
+    password: str = Field(min_length=6, max_length=128)
+    confirm_password: str = Field(min_length=1, max_length=128)
+
+    @model_validator(mode="after")
+    def passwords_match(self):
+        if self.password != self.confirm_password:
+            raise ValueError("Passwords do not match")
+        return self

devplacepy/models/community.py

@@ -0,0 +1,57 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Literal, Optional
+from pydantic import BaseModel, Field
+
+
+class SkillInstallForm(BaseModel):
+    url: str = Field(min_length=1, max_length=2000)
+
+
+class SkillEnabledForm(BaseModel):
+    enabled: Optional[bool] = None
+
+
+class SkillDeleteForm(BaseModel):
+    owner: str = Field(default="", max_length=32)
+
+
+class MessageForm(BaseModel):
+    content: str = Field(min_length=0, max_length=2000)
+    receiver_uid: str = Field(min_length=1, max_length=36)
+    attachment_uids: list[str] = []
+    client_id: Optional[str] = Field(default=None, max_length=64)
+
+
+class GistForm(BaseModel):
+    title: str = Field(min_length=1, max_length=200)
+    description: str = Field(default="", max_length=5000)
+    source_code: str = Field(min_length=1, max_length=400000)
+    language: str = Field(default="plaintext", max_length=50)
+    attachment_uids: list[str] = []
+
+
+class GistEditForm(BaseModel):
+    title: str = Field(min_length=1, max_length=200)
+    description: str = Field(default="", max_length=5000)
+    source_code: str = Field(min_length=1, max_length=400000)
+    language: str = Field(default="plaintext", max_length=50)
+
+
+class IssueForm(BaseModel):
+    title: str = Field(min_length=1, max_length=200)
+    description: str = Field(min_length=1, max_length=5000)
+    attachment_uids: list[str] = []
+
+
+class IssueCommentForm(BaseModel):
+    body: str = Field(min_length=1, max_length=5000)
+    attachment_uids: list[str] = []
+
+
+class IssueAttachmentForm(BaseModel):
+    attachment_uids: list[str] = []
+
+
+class IssueStatusForm(BaseModel):
+    status: Literal["open", "closed"]

devplacepy/models/containers.py

@@ -0,0 +1,78 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Optional
+from pydantic import BaseModel, Field, field_validator
+
+
+class ContainerInstanceForm(BaseModel):
+    name: str = Field(min_length=1, max_length=64)
+    boot_command: str = Field(default="", max_length=500)
+    boot_language: str = Field(default="none", max_length=10)
+    boot_script: str = Field(default="", max_length=100000)
+    run_as_uid: str = Field(default="", max_length=36)
+    start_on_boot: bool = False
+    env: str = Field(default="", max_length=10000)
+    cpu_limit: str = Field(default="", max_length=16)
+    mem_limit: str = Field(default="", max_length=16)
+    ports: str = Field(default="", max_length=500)
+    volumes: str = Field(default="", max_length=2000)
+    restart_policy: str = Field(default="never", max_length=20)
+    autostart: bool = True
+    ingress_slug: str = Field(default="", max_length=64)
+    ingress_port: Optional[int] = Field(default=None, ge=1, le=65535)
+
+    @field_validator("ingress_port", mode="before")
+    @classmethod
+    def _blank_ingress_port(cls, value):
+        if value is None or (isinstance(value, str) and not value.strip()):
+            return None
+        return value
+
+
+class ContainerAdminCreateForm(ContainerInstanceForm):
+    project_slug: str = Field(min_length=1, max_length=128)
+
+
+class ContainerEditForm(BaseModel):
+    run_as_uid: str = Field(default="", max_length=36)
+    boot_language: str = Field(default="none", max_length=10)
+    boot_script: str = Field(default="", max_length=100000)
+    boot_command: str = Field(default="", max_length=500)
+    restart_policy: str = Field(default="never", max_length=20)
+    start_on_boot: bool = False
+    cpu_limit: str = Field(default="", max_length=16)
+    mem_limit: str = Field(default="", max_length=16)
+
+
+class ContainerExecForm(BaseModel):
+    command: str = Field(min_length=1, max_length=2000)
+
+
+class ContainerScheduleForm(BaseModel):
+    action: str = Field(max_length=10)
+    kind: str = Field(max_length=10)
+    cron: str = Field(default="", max_length=120)
+    run_at: str = Field(default="", max_length=40)
+    delay_seconds: Optional[int] = Field(default=None, ge=1)
+    every_seconds: Optional[int] = Field(default=None, ge=1)
+    max_runs: Optional[int] = Field(default=None, ge=1)
+
+
+class TunnelForm(BaseModel):
+    label: str = Field(default="", max_length=64)
+    container_port: int = Field(default=0, ge=0, le=65535)
+
+
+class EditorPrefsForm(BaseModel):
+    font_size: int = Field(default=0, ge=0, le=48)
+    terminal_font_size: int = Field(default=0, ge=0, le=48)
+    zoom_level: int = Field(default=-99, ge=-99, le=5)
+    theme: str = Field(default="", max_length=32)
+    layout: str = Field(default="", max_length=32)
+    panel_preset: str = Field(default="", max_length=32)
+    window_mode: str = Field(default="", max_length=32)
+    window_width: int = Field(default=0, ge=0, le=7680)
+    window_height: int = Field(default=0, ge=0, le=4320)
+    boot_agent: str = Field(default="", max_length=32)
+    boot_shell: int = Field(default=-1, ge=-1, le=1)
+    reset: bool = False

devplacepy/models/content.py

@@ -0,0 +1,128 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Literal
+from pydantic import BaseModel, Field, field_validator, model_validator
+from devplacepy.constants import TOPICS
+from devplacepy.rendering import is_single_emoji
+from devplacepy.models.shared import normalize_poll_options
+
+
+class PostForm(BaseModel):
+    content: str = Field(min_length=10, max_length=125000)
+    title: str = Field(default="", max_length=500)
+    topic: str = "random"
+    project_uid: str = Field(default="", max_length=36)
+    attachment_uids: list[str] = []
+    poll_question: str = Field(default="", max_length=200)
+    poll_options: list[str] = []
+    war_faction_a: str = Field(default="", max_length=30)
+    war_faction_b: str = Field(default="", max_length=30)
+
+    @field_validator("poll_options")
+    @classmethod
+    def poll_options_max_length(cls, value):
+        if value is None:
+            return value
+        for opt in value:
+            if isinstance(opt, str) and len(opt) > 200:
+                raise ValueError("Each poll option must be 200 characters or fewer")
+        return value
+
+    @field_validator("topic")
+    @classmethod
+    def valid_topic(cls, value):
+        return value if value in TOPICS else "random"
+
+    @field_validator("poll_options", mode="before")
+    @classmethod
+    def split_poll_options(cls, value):
+        return normalize_poll_options(value)
+
+
+class WarJoinForm(BaseModel):
+    faction: str
+
+    @field_validator("faction")
+    @classmethod
+    def valid_faction(cls, value):
+        if value not in ("a", "b"):
+            raise ValueError("Faction must be a or b")
+        return value
+
+
+class PostEditForm(BaseModel):
+    content: str = Field(min_length=10, max_length=125000)
+    title: str = Field(default="", max_length=500)
+    topic: str = "random"
+    poll_question: str = Field(default="", max_length=200)
+    poll_options: list[str] = []
+
+    @field_validator("poll_options")
+    @classmethod
+    def poll_options_max_length(cls, value):
+        if value is None:
+            return value
+        for opt in value:
+            if isinstance(opt, str) and len(opt) > 200:
+                raise ValueError("Each poll option must be 200 characters or fewer")
+        return value
+
+    @field_validator("topic")
+    @classmethod
+    def valid_topic(cls, value):
+        return value if value in TOPICS else "random"
+
+    @field_validator("poll_options", mode="before")
+    @classmethod
+    def split_poll_options(cls, value):
+        return normalize_poll_options(value)
+
+
+class CommentForm(BaseModel):
+    content: str = Field(min_length=3, max_length=125000)
+    target_uid: str = Field(default="", max_length=36)
+    post_uid: str = Field(default="", max_length=36)
+    target_type: Literal["post", "project", "news", "issue", "gist", "quiz"] = "post"
+    parent_uid: str = Field(default="", max_length=36)
+    attachment_uids: list[str] = []
+
+    @model_validator(mode="after")
+    def require_target(self):
+        if not (self.target_uid or self.post_uid):
+            raise ValueError("A target is required")
+        return self
+
+
+class CommentEditForm(BaseModel):
+    content: str = Field(min_length=3, max_length=125000)
+
+
+class NoteForm(BaseModel):
+    content: str = Field(min_length=1, max_length=4000)
+
+
+class VoteForm(BaseModel):
+    value: int
+
+    @field_validator("value")
+    @classmethod
+    def valid_value(cls, value):
+        if value not in (1, -1, 0):
+            raise ValueError("value must be 1, -1 or 0")
+        return value
+
+
+class ReactionForm(BaseModel):
+    emoji: str = Field(min_length=1, max_length=16)
+
+    @field_validator("emoji")
+    @classmethod
+    def valid_emoji(cls, value):
+        reaction = (value or "").strip()
+        if not is_single_emoji(reaction):
+            raise ValueError("Reaction must be a single emoji")
+        return reaction
+
+
+class PollVoteForm(BaseModel):
+    option_uid: str = Field(min_length=1, max_length=36)

devplacepy/models/game.py

@@ -0,0 +1,47 @@
+# retoor <retoor@molodetz.nl>
+
+from pydantic import BaseModel, Field
+
+
+class GamePlantForm(BaseModel):
+    slot: int = Field(ge=0, le=64)
+    crop: str = Field(min_length=1, max_length=40)
+
+
+class GameSlotForm(BaseModel):
+    slot: int = Field(ge=0, le=64)
+
+
+class GamePlotStopForm(BaseModel):
+    slot: int = Field(ge=0, le=64)
+    stop: bool
+
+
+class GamePerkForm(BaseModel):
+    perk: str = Field(min_length=1, max_length=40)
+
+
+class GameQuestForm(BaseModel):
+    quest: str = Field(min_length=1, max_length=40)
+    scope: str = Field(default="daily", min_length=1, max_length=10)
+
+
+class GameLegacyForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameInfraForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameCosmeticForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameMasteryForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameEraStartForm(BaseModel):
+    name: str = Field(min_length=1, max_length=60)
+    duration_days: int = Field(default=28, ge=1, le=180)

devplacepy/models/profile.py

@@ -0,0 +1,86 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Literal
+from pydantic import BaseModel, Field, field_validator
+from devplacepy.config import (
+    DEFAULT_CORRECTION_PROMPT,
+    DEFAULT_MODIFIER_PROMPT,
+)
+
+
+class CustomizationToggleForm(BaseModel):
+    value: bool = False
+
+
+class AwardGiveForm(BaseModel):
+    description: str = Field(min_length=1, max_length=125)
+
+
+class NotificationPrefForm(BaseModel):
+    notification_type: str = Field(min_length=1, max_length=40)
+    channel: Literal["in_app", "push", "telegram"]
+    value: bool = False
+
+
+class AiCorrectionForm(BaseModel):
+    enabled: bool = False
+    sync: bool = False
+    prompt: str = Field(default=DEFAULT_CORRECTION_PROMPT, max_length=20000)
+
+
+class AiModifierForm(BaseModel):
+    enabled: bool = False
+    sync: bool = False
+    prompt: str = Field(default=DEFAULT_MODIFIER_PROMPT, max_length=20000)
+
+
+class InteractionsForm(BaseModel):
+    enabled: bool = True
+    reset: bool = False
+
+
+class TelegramPairForm(BaseModel):
+    action: str = Field(default="request", max_length=16)
+
+
+class ProfileForm(BaseModel):
+    bio: str = Field(default="", max_length=500)
+    location: str = Field(default="", max_length=200)
+    git_link: str = Field(default="", max_length=500)
+    website: str = Field(default="", max_length=500)
+
+
+class ConsentForm(BaseModel):
+    kind: str = Field(min_length=1, max_length=40)
+    granted: str = Field(default="0", max_length=5)
+
+    @field_validator("kind")
+    @classmethod
+    def known_kind(cls, value):
+        from devplacepy.database.moderation import CONSENT_KINDS
+
+        if value not in CONSENT_KINDS:
+            raise ValueError("Unknown consent kind")
+        return value
+
+
+class MaturityForm(BaseModel):
+    level: str = Field(default="general", max_length=20)
+
+    @field_validator("level")
+    @classmethod
+    def known_level(cls, value):
+        from devplacepy.database.moderation import MATURITY_LEVELS
+
+        if value not in MATURITY_LEVELS:
+            raise ValueError("Unknown maturity level")
+        return value
+
+
+class MaturePreferenceForm(BaseModel):
+    mature_opt_in: str = Field(default="0", max_length=5)
+
+
+class AccountDeleteForm(BaseModel):
+    password: str = Field(min_length=1, max_length=128)
+    confirm_text: str = Field(default="", max_length=40)

devplacepy/models/projects.py

@@ -0,0 +1,113 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Literal
+from pydantic import BaseModel, Field, field_validator
+from devplacepy.models.shared import normalize_european_date, normalize_website_url
+
+
+class ProjectForm(BaseModel):
+    title: str = Field(min_length=1, max_length=200)
+    description: str = Field(min_length=1, max_length=5000)
+    release_date: str = ""
+    demo_date: str = ""
+    project_type: Literal["game", "game_asset", "software", "mobile_app", "website"] = (
+        "software"
+    )
+    platforms: str = Field(default="", max_length=500)
+    status: str = Field(default="In Development", max_length=100)
+    website_url: str = Field(default="", max_length=500)
+    repo_url: str = Field(default="", max_length=500)
+    cover_attachment_uid: str = Field(default="", max_length=64)
+    logo_attachment_uid: str = Field(default="", max_length=64)
+    is_private: bool = False
+    attachment_uids: list[str] = []
+
+    @field_validator("release_date", "demo_date", mode="before")
+    @classmethod
+    def normalize_dates(cls, value):
+        return normalize_european_date(value)
+
+    @field_validator("website_url", "repo_url")
+    @classmethod
+    def valid_link_url(cls, value):
+        return normalize_website_url(value)
+
+
+class ProjectEditForm(BaseModel):
+    title: str = Field(min_length=1, max_length=200)
+    description: str = Field(min_length=1, max_length=5000)
+    release_date: str = ""
+    demo_date: str = ""
+    project_type: Literal["game", "game_asset", "software", "mobile_app", "website"] = (
+        "software"
+    )
+    platforms: str = Field(default="", max_length=500)
+    status: str = Field(default="In Development", max_length=100)
+    website_url: str = Field(default="", max_length=500)
+    repo_url: str = Field(default="", max_length=500)
+    cover_attachment_uid: str = Field(default="", max_length=64)
+    logo_attachment_uid: str = Field(default="", max_length=64)
+
+    @field_validator("release_date", "demo_date", mode="before")
+    @classmethod
+    def normalize_dates(cls, value):
+        return normalize_european_date(value)
+
+    @field_validator("website_url", "repo_url")
+    @classmethod
+    def valid_link_url(cls, value):
+        return normalize_website_url(value)
+
+
+class ProjectFlagForm(BaseModel):
+    value: bool = False
+
+
+class ProjectScreenshotsForm(BaseModel):
+    attachment_uids: list[str] = []
+
+
+class ForkForm(BaseModel):
+    title: str = Field(min_length=1, max_length=200)
+
+
+class ProjectFileWriteForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+    content: str = Field(default="", max_length=400000)
+
+
+class ProjectFileMkdirForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+
+
+class ProjectFileMoveForm(BaseModel):
+    from_path: str = Field(min_length=1, max_length=1024)
+    to_path: str = Field(min_length=1, max_length=1024)
+
+
+class ProjectFileDeleteForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+
+
+class ProjectFileReplaceLinesForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+    start: int = Field(ge=1)
+    end: int = Field(ge=0)
+    content: str = Field(default="", max_length=400000)
+
+
+class ProjectFileInsertLinesForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+    at: int = Field(ge=1)
+    content: str = Field(default="", max_length=400000)
+
+
+class ProjectFileDeleteLinesForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+    start: int = Field(ge=1)
+    end: int = Field(ge=1)
+
+
+class ProjectFileAppendForm(BaseModel):
+    path: str = Field(min_length=1, max_length=1024)
+    content: str = Field(default="", max_length=400000)

devplacepy/models/quizzes.py

@@ -0,0 +1,247 @@
+# retoor <retoor@molodetz.nl>
+
+import json
+from typing import Literal
+from pydantic import BaseModel, Field, field_validator, model_validator
+from devplacepy.config import (
+    QUIZ_ANSWER_MAX_CHARS,
+    QUIZ_MAX_OPTIONS,
+    QUIZ_MAX_QUESTIONS,
+    QUIZ_MAX_TIME_LIMIT_SECONDS,
+)
+from devplacepy.models.shared import normalize_poll_options, normalize_index_list
+
+
+QUIZ_KINDS = (
+    "single_choice",
+    "multiple_choice",
+    "true_false",
+    "free_text",
+    "fill_blank",
+    "numeric",
+    "ordering",
+    "matching",
+)
+
+
+QUIZ_OPTION_KINDS = frozenset(
+    {"single_choice", "multiple_choice", "fill_blank", "ordering", "matching"}
+)
+
+
+class QuizForm(BaseModel):
+    title: str = Field(min_length=3, max_length=200)
+    description: str = Field(default="", max_length=5000)
+    shuffle_questions: bool = False
+    shuffle_options: bool = False
+    reveal_answers: bool = False
+    allow_review: bool = True
+    time_limit_seconds: int = Field(default=0, ge=0, le=QUIZ_MAX_TIME_LIMIT_SECONDS)
+    pass_percent: int = Field(default=0, ge=0, le=100)
+
+
+class QuizQuestionForm(BaseModel):
+    kind: Literal[QUIZ_KINDS]
+    prompt: str = Field(min_length=1, max_length=2000)
+    explanation: str = Field(default="", max_length=2000)
+    points: int = Field(default=1, ge=1, le=100)
+    media_attachment_uid: str = Field(default="", max_length=36)
+    correct_boolean: bool = False
+    expected_answer: str = Field(default="", max_length=2000)
+    grading_criteria: str = Field(default="", max_length=2000)
+    numeric_value: float = 0.0
+    numeric_tolerance: float = Field(default=0.0, ge=0.0)
+    case_sensitive: bool = False
+    options: list[str] = []
+    match_values: list[str] = []
+    correct_indexes: list[int] = []
+
+    @field_validator("options", "match_values", mode="before")
+    @classmethod
+    def split_lists(cls, value):
+        return normalize_poll_options(value)
+
+    @field_validator("correct_indexes", mode="before")
+    @classmethod
+    def split_indexes(cls, value):
+        return normalize_index_list(value)
+
+    @field_validator("options", "match_values")
+    @classmethod
+    def bounded_options(cls, value):
+        if len(value) > QUIZ_MAX_OPTIONS:
+            raise ValueError(f"A question takes at most {QUIZ_MAX_OPTIONS} options")
+        for entry in value:
+            if len(entry) > 500:
+                raise ValueError("Each option must be 500 characters or fewer")
+        return value
+
+    @model_validator(mode="after")
+    def kind_requirements(self):
+        options = [option for option in self.options if option.strip()]
+        if self.kind in QUIZ_OPTION_KINDS and not options:
+            raise ValueError("This question type needs at least one option")
+        if self.kind in ("single_choice", "multiple_choice") and len(options) < 2:
+            raise ValueError("Choice questions need at least two options")
+        if self.kind == "single_choice" and len(self.correct_indexes) != 1:
+            raise ValueError("A single choice question needs exactly one correct option")
+        if self.kind == "multiple_choice" and not self.correct_indexes:
+            raise ValueError("A multiple choice question needs at least one correct option")
+        if self.kind in ("fill_blank", "matching") and len(self.match_values) < len(options):
+            raise ValueError("Every option needs an accepted answer")
+        if self.kind == "matching" and len(options) < 2:
+            raise ValueError("A matching question needs at least two pairs")
+        if self.kind == "ordering" and len(options) < 2:
+            raise ValueError("An ordering question needs at least two items")
+        if self.kind == "free_text" and not (
+            self.expected_answer.strip() or self.grading_criteria.strip()
+        ):
+            raise ValueError("A free text question needs a reference answer or grading criteria")
+        return self
+
+    def option_rows(self) -> list[dict]:
+        rows = []
+        for index, label in enumerate(self.options):
+            if not label.strip():
+                continue
+            match_value = (
+                self.match_values[index] if index < len(self.match_values) else ""
+            )
+            rows.append(
+                {
+                    "label": label.strip(),
+                    "match_value": match_value.strip(),
+                    "is_correct": index in set(self.correct_indexes),
+                }
+            )
+        return rows
+
+
+class QuizReorderForm(BaseModel):
+    order: list[str] = []
+
+    @field_validator("order", mode="before")
+    @classmethod
+    def split_order(cls, value):
+        return normalize_poll_options(value)
+
+    @model_validator(mode="after")
+    def require_order(self):
+        if not self.order:
+            raise ValueError("The new question order is required")
+        return self
+
+
+class QuizAnswerForm(BaseModel):
+    question_uid: str = Field(min_length=1, max_length=36)
+    answer_text: str = Field(default="", max_length=QUIZ_ANSWER_MAX_CHARS)
+    option_uids: list[str] = []
+    blanks: list[str] = []
+    matches: list[str] = []
+
+    @field_validator("option_uids", "blanks", "matches", mode="before")
+    @classmethod
+    def split_lists(cls, value):
+        return normalize_poll_options(value)
+
+    def submission(self) -> dict:
+        if self.blanks:
+            return {
+                "answer_text": json.dumps(self.blanks, ensure_ascii=False),
+                "option_uids": self.option_uids,
+            }
+        if self.matches:
+            pairs = dict(zip(self.option_uids, self.matches))
+            return {
+                "answer_text": json.dumps(pairs, ensure_ascii=False),
+                "option_uids": self.option_uids,
+            }
+        return {"answer_text": self.answer_text, "option_uids": self.option_uids}
+
+
+class QuizDocumentOption(BaseModel):
+    label: str = Field(default="", max_length=500)
+    match_value: str = Field(default="", max_length=500)
+    is_correct: bool = False
+
+
+class QuizDocumentQuestion(BaseModel):
+    kind: Literal[QUIZ_KINDS]
+    prompt: str = Field(min_length=1, max_length=2000)
+    explanation: str = Field(default="", max_length=2000)
+    points: int = Field(default=1, ge=1, le=100)
+    media_attachment_uid: str = Field(default="", max_length=36)
+    correct_boolean: bool = False
+    expected_answer: str = Field(default="", max_length=2000)
+    grading_criteria: str = Field(default="", max_length=2000)
+    numeric_value: float = 0.0
+    numeric_tolerance: float = Field(default=0.0, ge=0.0)
+    case_sensitive: bool = False
+    options: list[QuizDocumentOption] = Field(default_factory=list, max_length=QUIZ_MAX_OPTIONS)
+
+    @model_validator(mode="after")
+    def kind_requirements(self):
+        labelled = [option for option in self.options if option.label.strip()]
+        if self.kind in ("single_choice", "multiple_choice") and len(labelled) < 2:
+            raise ValueError("Choice questions need at least two options")
+        if self.kind == "single_choice" and sum(
+            1 for option in labelled if option.is_correct
+        ) != 1:
+            raise ValueError("A single choice question needs exactly one correct option")
+        if self.kind == "multiple_choice" and not any(
+            option.is_correct for option in labelled
+        ):
+            raise ValueError("A multiple choice question needs at least one correct option")
+        if self.kind in ("ordering", "matching") and len(labelled) < 2:
+            raise ValueError("This question type needs at least two entries")
+        if self.kind == "matching" and any(
+            not option.match_value.strip() for option in labelled
+        ):
+            raise ValueError("Every matching pair needs a right-hand value")
+        if self.kind == "fill_blank" and any(
+            not option.match_value.strip() for option in labelled
+        ):
+            raise ValueError("Every blank needs an accepted answer")
+        if self.kind == "free_text" and not (
+            self.expected_answer.strip() or self.grading_criteria.strip()
+        ):
+            raise ValueError("A free text question needs a reference answer or grading criteria")
+        return self
+
+
+class QuizDocumentSettings(BaseModel):
+    shuffle_questions: bool = False
+    shuffle_options: bool = False
+    reveal_answers: bool = False
+    allow_review: bool = True
+    time_limit_seconds: int = Field(default=0, ge=0, le=QUIZ_MAX_TIME_LIMIT_SECONDS)
+    pass_percent: int = Field(default=0, ge=0, le=100)
+
+
+class QuizDocument(BaseModel):
+    title: str = Field(min_length=3, max_length=200)
+    description: str = Field(default="", max_length=5000)
+    settings: QuizDocumentSettings = Field(default_factory=QuizDocumentSettings)
+    questions: list[QuizDocumentQuestion] = Field(
+        default_factory=list, max_length=QUIZ_MAX_QUESTIONS
+    )
+
+    @model_validator(mode="after")
+    def require_questions(self):
+        if not self.questions:
+            raise ValueError("A quiz document needs at least one question")
+        return self
+
+
+class QuizImportForm(BaseModel):
+    document: QuizDocument
+
+    @field_validator("document", mode="before")
+    @classmethod
+    def parse_document(cls, value):
+        if isinstance(value, str):
+            try:
+                return json.loads(value)
+            except ValueError as exc:
+                raise ValueError("document must be valid JSON") from exc
+        return value

devplacepy/models/shared.py

@@ -0,0 +1,75 @@
+# retoor <retoor@molodetz.nl>
+
+from datetime import datetime
+from urllib.parse import urlparse
+
+
+def normalize_european_date(value):
+    if not value:
+        return ""
+    text = str(value).strip()
+    if not text:
+        return ""
+    for fmt in ("%d/%m/%Y", "%Y-%m-%d"):
+        try:
+            return datetime.strptime(text, fmt).strftime("%Y-%m-%d")
+        except ValueError:
+            continue
+    raise ValueError("Date must be in DD/MM/YYYY format")
+
+
+def normalize_website_url(value):
+    if not value:
+        return ""
+    text = str(value).strip()
+    if not text:
+        return ""
+    if not text.lower().startswith(("http://", "https://")):
+        text = f"https://{text}"
+    parsed = urlparse(text)
+    if parsed.scheme not in ("http", "https") or not parsed.hostname or "." not in parsed.hostname:
+        raise ValueError("Website must be a valid http(s) URL")
+    return text
+
+
+def normalize_poll_options(value):
+    if value is None:
+        return []
+    if isinstance(value, str):
+        value = [value]
+    if not isinstance(value, list):
+        return value
+    if len(value) == 1 and isinstance(value[0], str):
+        single = value[0]
+        separator = "\n" if "\n" in single else ("," if "," in single else "")
+        if separator:
+            return [part.strip() for part in single.split(separator) if part.strip()]
+    return value
+
+
+def _declared_age(birth_date: str) -> int:
+    from datetime import date
+
+    from devplacepy.database.moderation import years_between
+
+    normalized = normalize_european_date(birth_date)
+    if not normalized:
+        raise ValueError("Date of birth is required")
+    born = date.fromisoformat(normalized)
+    today = date.today()
+    if born > today:
+        raise ValueError("Date of birth cannot be in the future")
+    return years_between(born, today)
+
+
+def normalize_index_list(value):
+    parts = normalize_poll_options(value)
+    if not isinstance(parts, list):
+        return []
+    indexes = []
+    for part in parts:
+        try:
+            indexes.append(int(str(part).strip()))
+        except (TypeError, ValueError):
+            continue
+    return indexes

devplacepy/models/tools.py

@@ -0,0 +1,119 @@
+# retoor <retoor@molodetz.nl>
+
+import re
+from typing import Literal
+from pydantic import BaseModel, Field, field_validator
+
+
+class PlanningForm(BaseModel):
+    numbers: str = Field(default="", max_length=4096)
+
+
+class SeoRunForm(BaseModel):
+    url: str = Field(min_length=3, max_length=2000)
+    mode: Literal["url", "sitemap"] = "url"
+    max_pages: int = Field(default=10, ge=1, le=50)
+
+    @field_validator("url")
+    @classmethod
+    def url_scheme(cls, value):
+        text = value.strip()
+        if not text:
+            raise ValueError("A URL is required")
+        if "://" in text:
+            scheme = text.split("://", 1)[0]
+            if scheme not in ("http", "https"):
+                raise ValueError(f"Only http and https URLs are allowed; got '{scheme}://'")
+        else:
+            text = f"https://{text}"
+        if not SEO_URL_PATTERN.match(text):
+            raise ValueError("URL must be a valid http or https source location")
+        return text
+
+
+SEO_URL_PATTERN = re.compile(r"^https?://[a-zA-Z0-9][\w./:@~^?&#%=;-]*$")
+
+
+ISSLOP_URL_PATTERN = re.compile(r"^(https?://|git://|ssh://|git@)[\w./:@~^-]+$", re.IGNORECASE)
+
+
+ISSLOP_SINGLE_SLASH_PATTERN = re.compile(r"^(https?|git|ssh):/(?!/)", re.IGNORECASE)
+
+
+ISSLOP_SCHEME_PATTERN = re.compile(r"^[a-z][a-z0-9+.-]*://", re.IGNORECASE)
+
+
+class IsslopRunForm(BaseModel):
+    url: str = Field(min_length=4, max_length=2048)
+
+    @field_validator("url")
+    @classmethod
+    def url_scheme(cls, value):
+        text = value.strip()
+        text = ISSLOP_SINGLE_SLASH_PATTERN.sub(lambda match: f"{match.group(1)}://", text)
+        if not ISSLOP_SCHEME_PATTERN.match(text) and not text.startswith("git@"):
+            text = f"https://{text}"
+        if not ISSLOP_URL_PATTERN.match(text):
+            raise ValueError("URL must be an http(s), git or ssh source location")
+        return text
+
+
+DEEPSEARCH_MIN_DEPTH = 1
+
+
+DEEPSEARCH_MAX_DEPTH = 4
+
+
+DEEPSEARCH_DEFAULT_DEPTH = 2
+
+
+DEEPSEARCH_MIN_PAGES = 1
+
+
+DEEPSEARCH_MAX_PAGES = 30
+
+
+DEEPSEARCH_DEFAULT_PAGES = 12
+
+
+DEEPSEARCH_MIN_QUERY = 3
+
+
+DEEPSEARCH_MAX_QUERY = 500
+
+
+DEEPSEARCH_MAX_MESSAGE = 2000
+
+
+class DeepsearchRunForm(BaseModel):
+    query: str = Field(min_length=DEEPSEARCH_MIN_QUERY, max_length=DEEPSEARCH_MAX_QUERY)
+    depth: int = Field(
+        default=DEEPSEARCH_DEFAULT_DEPTH,
+        ge=DEEPSEARCH_MIN_DEPTH,
+        le=DEEPSEARCH_MAX_DEPTH,
+    )
+    max_pages: int = Field(
+        default=DEEPSEARCH_DEFAULT_PAGES,
+        ge=DEEPSEARCH_MIN_PAGES,
+        le=DEEPSEARCH_MAX_PAGES,
+    )
+
+    @field_validator("query")
+    @classmethod
+    def query_present(cls, value):
+        text = value.strip()
+        if not text:
+            raise ValueError("A research question is required")
+        return text
+
+
+class DeepsearchChatForm(BaseModel):
+    message: str = Field(min_length=1, max_length=DEEPSEARCH_MAX_MESSAGE)
+
+    @field_validator("message")
+    @classmethod
+    def message_present(cls, value):
+        text = value.strip()
+        if not text:
+            raise ValueError("A message is required")
+        return text

devplacepy/models/uploads.py

@@ -0,0 +1,18 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Optional
+from pydantic import BaseModel, Field
+
+
+class UploadUrlForm(BaseModel):
+    url: str = Field(min_length=1, max_length=2048)
+    filename: Optional[str] = Field(default=None, max_length=255)
+
+
+class GenerateImageForm(BaseModel):
+    prompt: str = Field(min_length=1, max_length=2000)
+    size: Optional[str] = Field(default=None, max_length=16)
+
+
+class AttachmentRenameForm(BaseModel):
+    filename: str = Field(min_length=1, max_length=255)

devplacepy/routers/CLAUDE.md

@@ -74,7 +74,7 @@ Response models live in `devplacepy/schemas.py` (Pydantic v2, `extra="ignore"`,
 
 ## FastAPI patterns
 
-- **All routes are async.** Form data is validated via a typed Pydantic body param: `data: Annotated[SomeForm, Form()]` (models in `models.py`). Read raw `await request.form()` only when also handling an uploaded file (a separate `File()` param would embed the model under its parameter name).
+- **All routes are async.** Form data is validated via a typed Pydantic body param: `data: Annotated[SomeForm, Form()]` (models in `models/`, one module per domain). Read raw `await request.form()` only when also handling an uploaded file (a separate `File()` param would embed the model under its parameter name).
 - **Return `RedirectResponse(url=..., status_code=302)`** for redirects - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
 - **Return `templates.TemplateResponse("name.html", {...})`** from `devplacepy.templating` to render.
 - **Never create your own `Jinja2Templates` instance.** Import the shared one: `from devplacepy.templating import templates`.
@@ -167,7 +167,7 @@ A dedicated `/gists` page for sharing code snippets. Uses `gists` table (auto-cr
 
 ### Politics category
 
-The `politics` topic is available as a feed filter sidebar item (icon `🏛`) and post topic. It defines the CSS variable `--topic-politics: #00bcd4` in `variables.css` and the `.badge-politics` class in `base.css`. It is validated like every topic via the canonical `TOPICS` list in `constants.py` (consumed by `models.py` `valid_topic`, `templating.py` `TOPICS` global, `routers/posts.py`, `docs_api`). Unlike the former `signals` topic, `politics` is also part of the bot fleet's rotation - it is in `services/bot/config.py` `CATEGORIES`/`FEED_TOPICS`, carries a modest per-persona weight in `PERSONA_CATEGORY_WEIGHTS`, and has a writing instruction in `services/bot/llm.py` `category_extras`.
+The `politics` topic is available as a feed filter sidebar item (icon `🏛`) and post topic. It defines the CSS variable `--topic-politics: #00bcd4` in `variables.css` and the `.badge-politics` class in `base.css`. It is validated like every topic via the canonical `TOPICS` list in `constants.py` (consumed by `models/` `valid_topic`, `templating.py` `TOPICS` global, `routers/posts.py`, `docs_api`). Unlike the former `signals` topic, `politics` is also part of the bot fleet's rotation - it is in `services/bot/config.py` `CATEGORIES`/`FEED_TOPICS`, carries a modest per-persona weight in `PERSONA_CATEGORY_WEIGHTS`, and has a writing instruction in `services/bot/llm.py` `category_extras`.
 
 ### Date format
 
@@ -200,7 +200,7 @@ News articles have an internal detail page at `/news/{slug}` with full comment s
 
 ### Home page (`GET /`)
 
-The home route (`main.py` `landing()`) never redirects - it renders `templates/landing.html` for everyone, branching on `user`:
+The home route (`routers/landing.py` `landing()`) never redirects - it renders `templates/landing.html` for everyone, branching on `user`:
 
 - **Guests** get the marketing hero (`Join DevPlace Free` CTA + features grid).
 - **Signed-in users** get a personalized dashboard hero (`.dashboard-welcome`): avatar, "Welcome back, {username}", quicklink buttons (`.dashboard-btn`, with `New Post` -> `/feed` as `.dashboard-btn-primary`, plus Code Farm/Projects/Gists), and a Posts/Stars/Level stat strip (`.dashboard-stats`, `user_post_count` + the user dict's `stars`/`level`). Styles live in the `.dashboard-*` classes in `static/css/landing.css`.
@@ -239,7 +239,7 @@ Every public listing card that shows the last few comments must render them with
 Articles can be toggled to appear on the landing page via `/admin/news/{uid}/landing`:
 
 - **`show_on_landing`** field on `news` table
-- Landing route (`main.py` `GET /`) fetches up to 6 articles with `show_on_landing=1`
+- Landing route (`routers/landing.py` `GET /`) fetches up to 6 articles with `show_on_landing=1`
 - Rendered as a 3-column card grid with image, source, title, date (responsive -> 1 column on mobile)
 - Toggleable individually from the admin news table
 
@@ -333,7 +333,7 @@ All three columns are populated by `routers/posts.py` `post_page_context()`, the
 
 ### Polls
 - A poll rides on a post (one `polls` row keyed by `post_uid`, options in `poll_options`, one-per-user votes in `poll_votes`). Created in `posts.py:create_poll` when `poll_question` plus >= 2 non-empty `poll_options` are submitted (capped at 6). Both `create_post` and `edit_post` accept the poll fields; `edit_post` only attaches a poll when the post has **none** yet (it never replaces an existing poll). The builders live in the create-post modal (`feed.html`) and the edit-post modal (`post.html`, rendered only when the post has no poll) using `data-poll-toggle` / `data-poll-add-option`.
-- `poll_options` accepts either repeated form fields (the web builders, which preserve commas inside an option label) **or** a single newline- or comma-separated string (the API/Devii path). `models.py:normalize_poll_options` splits a lone delimited element - applied as a `mode="before"` validator on `PostForm`/`PostEditForm` - so the documented "one per line or comma separated" agent format actually produces a multi-option poll instead of a single dropped option.
+- `poll_options` accepts either repeated form fields (the web builders, which preserve commas inside an option label) **or** a single newline- or comma-separated string (the API/Devii path). `models/shared.py:normalize_poll_options` splits a lone delimited element - applied as a `mode="before"` validator on `PostForm`/`PostEditForm` - so the documented "one per line or comma separated" agent format actually produces a multi-option poll instead of a single dropped option.
 - `POST /polls/{poll_uid}/vote` toggles the voter's choice and returns the full poll dict: voting on a new option switches, voting the **same** option again **retracts** (one vote per user, like reactions). Results (bars + `%`) are **always shown** to everyone; the chosen option gets `.chosen` + a checkmark. Guests see read-only bars with a "Log in to vote" hint (options `disabled`). Batch reads via `get_polls_by_post_uids(post_uids, user)` / `get_poll_for_post(post_uid, user)`.
 - Composer poll builder (`feed.html`, `PollManager.js`): poll inputs are `disabled` until "Add poll" is opened (so a never-opened or "Remove poll"-collapsed builder submits **nothing**); a capture-phase `submit` listener blocks an incomplete poll (question + >= 2 options) with an inline error instead of silently dropping it - capture phase + `stopImmediatePropagation()` is required so it pre-empts `FormManager`'s bubble-phase submit handlers.
 

devplacepy/routers/landing.py

@@ -0,0 +1,143 @@
+# retoor <retoor@molodetz.nl>
+
+import os
+
+from fastapi import APIRouter, Request
+
+from devplacepy.cache import TTLCache
+from devplacepy.database import (
+    db,
+    get_blocked_uids,
+    get_comment_counts_by_post_uids,
+    get_news_images_by_uids,
+    get_table,
+    get_top_authors,
+    get_trending_topics,
+    get_user_post_count,
+    get_user_stars,
+    get_users_by_uids,
+    get_vote_counts,
+    paginate_diverse,
+)
+from devplacepy.responses import respond
+from devplacepy.schemas import LandingOut
+from devplacepy.seo import base_seo_context, site_url, website_schema
+from devplacepy.templating import jinja_unread_count
+from devplacepy.utils import get_current_user, time_ago
+
+router = APIRouter()
+
+_home_cache = TTLCache(ttl=int(os.environ.get("DEVPLACE_HOME_CACHE_TTL", "60")), max_size=4)
+
+
+def _landing_news():
+    cached = _home_cache.get("news")
+    if cached is not None:
+        return cached
+    articles = []
+    if "news" in db.tables:
+        news_table = get_table("news")
+        raw = list(
+            news_table.find(show_on_landing=1, order_by=["-synced_at"], _limit=6)
+        )
+        images_by_news = get_news_images_by_uids([a["uid"] for a in raw])
+        for a in raw:
+            articles.append(
+                {
+                    "uid": a["uid"],
+                    "slug": a.get("slug", ""),
+                    "title": a.get("title", ""),
+                    "description": (a.get("description", "") or "")[:250],
+                    "url": a.get("url", ""),
+                    "source_name": a.get("source_name", ""),
+                    "grade": a.get("grade", 0),
+                    "featured": a.get("featured", 0),
+                    "synced_at": a.get("synced_at", "") or "",
+                    "time_ago": time_ago(a["synced_at"]),
+                    "image_url": a.get("image_url", "") or images_by_news.get(a["uid"], ""),
+                }
+            )
+    _home_cache.set("news", articles)
+    return articles
+
+
+def _landing_recent_posts(viewer_uid):
+    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
+    if not blocked:
+        cached = _home_cache.get("posts")
+        if cached is not None:
+            return cached
+    posts = []
+    if "posts" in db.tables:
+        posts_table = get_table("posts")
+        raw_posts, _ = paginate_diverse(
+            posts_table, order=["-created_at"], viewer_uid=viewer_uid, limit=6
+        )
+        if raw_posts:
+            post_uids = [p["uid"] for p in raw_posts]
+            author_uids = [p["user_uid"] for p in raw_posts]
+            authors = get_users_by_uids(author_uids)
+            comment_counts = get_comment_counts_by_post_uids(post_uids)
+            upvotes, downvotes = get_vote_counts(post_uids)
+            for p in raw_posts:
+                posts.append(
+                    {
+                        "post": p,
+                        "author": authors.get(p["user_uid"]),
+                        "time_ago": time_ago(p["created_at"]),
+                        "comment_count": comment_counts.get(p["uid"], 0),
+                        "stars": upvotes.get(p["uid"], 0) - downvotes.get(p["uid"], 0),
+                        "slug": p.get("slug", "") or p["uid"],
+                    }
+                )
+    if not blocked:
+        _home_cache.set("posts", posts)
+    return posts
+
+
+@router.get("/")
+async def landing(request: Request):
+    user = get_current_user(request)
+
+    landing_articles = _landing_news()
+    viewer_uid = user["uid"] if user else None
+    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
+    landing_posts = _landing_recent_posts(viewer_uid)
+
+    base = site_url(request)
+    seo_ctx = base_seo_context(
+        request,
+        title="DevPlace - The Developer Social Network",
+        description="Track industry shifts. Discover bold releases. Share what you're building in an open environment built by developers, for developers.",
+        breadcrumbs=[],
+        schemas=[website_schema(base)],
+    )
+    user_xp = user.get("xp", 0) or 0 if user else 0
+    user_level = user.get("level", 1) or 1 if user else 1
+    xp_progress_pct = (user_xp % 100) if user_xp else 0
+    unread_count = jinja_unread_count(user["uid"]) if user else 0
+
+    top_contributors = get_top_authors(5) if not blocked else []
+    trending_topics = get_trending_topics(6) if not blocked else []
+
+    return respond(
+        request,
+        "landing.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "user": user,
+            "is_authenticated": bool(user),
+            "user_post_count": get_user_post_count(user["uid"]) if user else 0,
+            "user_stars": get_user_stars(user["uid"]) if user else 0,
+            "user_xp": user_xp,
+            "user_level": user_level,
+            "xp_progress_pct": xp_progress_pct,
+            "unread_count": unread_count,
+            "landing_articles": landing_articles,
+            "landing_posts": landing_posts,
+            "top_contributors": top_contributors,
+            "trending_topics": trending_topics,
+        },
+        model=LandingOut,
+    )

devplacepy/services/CLAUDE.md

@@ -4,9 +4,9 @@ This file documents the service files that live directly in devplacepy/services/
 
 Generic, lightweight, **fire-and-forget** offload for non-critical side-effects so they leave the request path. The singleton `background` (`from devplacepy.services.background import background`) wraps one in-process `asyncio.Queue` drained by a single consumer task. The whole API is one call: `background.submit(fn, *args, **kwargs)` enqueues a **synchronous** callable and returns immediately (`put_nowait`). The consumer runs each callable inside its own `try/except`, so a failing task is logged and never kills the loop; FIFO order is preserved.
 
-- **Per-worker, not lock-gated (the reason it is NOT a `JobService`/`BaseService`).** Producers run in every uvicorn worker, and process memory is not shared, so the drain must run wherever requests are handled. `main.py` `startup()` calls `await background.start()` for every worker (inside the `if not DEVPLACE_DISABLE_SERVICES` guard, **outside** the `acquire_service_lock()` branch); `shutdown()` calls `await background.stop()`. A `BaseService` supervisor only runs in the lock owner, which would strand records produced by the other worker. The `JobService` queue is also wrong here: it is DB-backed, so deferring a tiny audit insert would *add* writes instead of removing them.
+- **Per-worker, not lock-gated (the reason it is NOT a `JobService`/`BaseService`).** Producers run in every uvicorn worker, and process memory is not shared, so the drain must run wherever requests are handled. `lifespan.py` `lifespan()` calls `await background.start()` for every worker (inside the `if not DEVPLACE_DISABLE_SERVICES` guard, **outside** the `acquire_service_lock()` branch); shutdown calls `await background.stop()`. A `BaseService` supervisor only runs in the lock owner, which would strand records produced by the other worker. The `JobService` queue is also wrong here: it is DB-backed, so deferring a tiny audit insert would *add* writes instead of removing them.
 - **Inline fallback (load-bearing).** When the consumer is not running - tests with `DEVPLACE_DISABLE_SERVICES=1`, unit tests, request-less bootstrap, or a full queue - `submit` runs `fn` **inline and synchronously**. This keeps audit/notification/XP writes deterministic and immediately visible to the test suite (which asserts audit rows right after an action) while production defers them. No test changes are needed.
-- **Best-effort durability (intentional).** In-memory only; `stop()` drains whatever remains synchronously so a **graceful** shutdown loses nothing, but a hard crash drops unflushed items. This matches audit/notifications already being best-effort (the recorder never raises). Do not put response-critical or money-touching work on it.
+- **Best-effort durability (intentional).** In-memory only; `stop()` posts the stop marker with `put_nowait` (a full queue logs and lets the worker keep draining instead of hanging shutdown) and then drains at most `SHUTDOWN_DRAIN_MAX`=1000 leftovers inline - anything beyond that is dropped with a loud warning and counted in `stats()["dropped"]`, so a **graceful** shutdown still loses nothing in the normal case but can never stall on a stuck worker. A hard crash drops unflushed items. This matches audit/notifications already being best-effort (the recorder never raises). Do not put response-critical or money-touching work on it.
 - **Capture plain data, never the `Request`.** Its lifecycle ends with the response - build the row/payload synchronously on the request thread and submit only the resulting dict/scalars.
 - **Consumers today** (deferred at their canonical choke points, so callers need no change):
   - **Audit** - `services/audit/record.py` `_write` builds the row + links synchronously, generates the `uid`/`created_at` eagerly so `record()` still returns the real uid, then `background.submit(_persist, row, links)` does the two `store` inserts off-thread.
@@ -129,7 +129,7 @@ devplace devii reset-quota --all        # Reset every quota (users and guests)
 
 - **In-process caches are never authoritative across workers.** Coordinate invalidation through the `cache_state` version table in `database.py`: call `bump_cache_version(name)` at every write path and `sync_local_cache(name, cache)` before every read path. Existing names: `auth` (guards `_user_cache`; bumped by `clear_user_cache`/`clear_session_cache` on logout, ban, role/password change - a version bump makes EVERY worker clear its WHOLE `_user_cache`, so display-only refreshes (XP/level in `award_xp`, AI-corrected bio) call `clear_user_cache(uid, propagate=False)` for a local pop without the global bump; identity/authz changes (logout, ban, role, password, api-key/token revoke) MUST keep the default propagate=True), `settings` (guards `_settings_cache`; bumped by `set_setting`/`clear_settings_cache`), `relations` (`_relations_cache`), `customizations` (`_customizations_cache`), `notif_prefs` (`_notification_prefs_cache`), `gateway_routing` (`_ROUTING_CACHE`), and **`admins`** (guards `_admins_cache`, which memoizes `get_admin_uids()` / `get_primary_admin_uid()`). The `_cache_version_cache` (`ttl=1`) caches the version reads themselves, so the bump is seen by other workers within ~1s on their next read (the originating worker is immediate); the version read/bump fail open (logged, never raise). When you add a per-process cache whose staleness matters, give it a name and wire both calls - do not invent a second invalidation mechanism.
 - **Admin-set cache invariant (load-bearing).** `_admins_cache` makes `get_admin_uids()` / `get_primary_admin_uid()` (hit on the projects-listing visibility filter, `_owner_is_admin`, and every primary-admin gate: `/dbapi`, backup-archive download) a dict lookup instead of a per-call `SELECT`. It is NOT the authorization gate - `is_admin(user)`/`require_admin` read the role off the user object (independently invalidated via `clear_user_cache`), so a stale admin set cannot grant access. But **any code that writes `users.role` MUST call `database.invalidate_admins_cache()`** (clears local + bumps the `admins` version), exactly like the soft-delete and `with db:` rules. Current role-write sites all do: `routers/admin/users.py` (role change), `cli.py` (`role set`), and `utils._create_account` (first user -> Admin). Omitting the call leaves the admin set stale for up to the 300s TTL.
-- **Deterministic / eventual caches skip version-sync on purpose.** Two caches need no `cache_state` name because correctness does not depend on cross-worker freshness: (1) `docs_prose._render_markdown` is an `@lru_cache` on the **static** prose source string (pure function of template content, so identical on every worker; changes only on deploy), and (2) `main._home_cache` (`TTLCache ttl=60`) memoizes the guest `/` blocks - the featured-news block (identical for everyone) and the latest-posts block **only for the no-block case**; a viewer with a non-empty block set bypasses the cache and is computed fresh, so the original block-filter semantics are preserved byte-for-byte and there is zero cross-user leakage. Worst case is a soft-deleted/edited public post lingering on `/` for <=60s. Use a plain `TTLCache`/`lru_cache` (no version name) ONLY when the value is deterministic or its staleness is purely cosmetic; anything whose staleness affects correctness or permissions MUST use the version-sync pattern above.
+- **Deterministic / eventual caches skip version-sync on purpose.** Two caches need no `cache_state` name because correctness does not depend on cross-worker freshness: (1) `docs_prose._render_markdown` is an `@lru_cache` on the **static** prose source string (pure function of template content, so identical on every worker; changes only on deploy), and (2) `routers/landing.py` `_home_cache` (`TTLCache ttl=60`) memoizes the guest `/` blocks - the featured-news block (identical for everyone) and the latest-posts block **only for the no-block case**; a viewer with a non-empty block set bypasses the cache and is computed fresh, so the original block-filter semantics are preserved byte-for-byte and there is zero cross-user leakage. Worst case is a soft-deleted/edited public post lingering on `/` for <=60s. Use a plain `TTLCache`/`lru_cache` (no version name) ONLY when the value is deterministic or its staleness is purely cosmetic; anything whose staleness affects correctness or permissions MUST use the version-sync pattern above.
 - **Authoritative state lives in the DB**, memory is only a short-TTL accelerator (sessions, the Devii 24h cap via `devii_usage_ledger`, cost counters).
 - **Global rates/quotas are worker-count-aware or DB-backed.** The rate limiter enforces `ceil(limit / DEVPLACE_WEB_WORKERS)` per process so the aggregate matches the configured value with no per-request write. `DEVPLACE_WEB_WORKERS` MUST equal the real `--workers` (set in `make prod`=2 and the Dockerfile=2; default 1 for dev). Update it whenever you change worker count.
 - **First-boot side effects are flock-serialized and idempotent.** `init_db()` runs under a blocking `flock` on `devplace-init.lock` (the seed-if-missing inserts would otherwise race to duplicate rows); `ensure_certificates()` early-returns when keys exist and otherwise generates under `.vapid.lock`. Any new run-once-at-startup work follows the same pattern (flock + exists-check / `INSERT OR IGNORE`).

devplacepy/services/acceptance/CLAUDE.md

@@ -10,7 +10,7 @@ It is off by default and it is never appropriate on a real production host.
 
 ## The two load-bearing ideas
 
-**The application must not know.** There is no request-path branch, no schema, no route, no template, no Jinja global, no Devii tool and no environment flag. The only import of this package anywhere is the one registration line in `main.py`, and `tests/unit/services/acceptance/isolation.py` fails the suite if a second one appears. An environment flag would be exactly the knowledge the application is not allowed to have, which is why there is none.
+**The application must not know.** There is no request-path branch, no schema, no route, no template, no Jinja global, no Devii tool and no environment flag. The only import of this package anywhere is the one entry in `lifespan.py` `SERVICE_CLASSES`, and `tests/unit/services/acceptance/isolation.py` fails the suite if a second one appears. An environment flag would be exactly the knowledge the application is not allowed to have, which is why there is none.
 
 **The decline register needs no storage.** `user_consents` is append-only in effect, so the latest live row for a `(user, kind)` pair already is the register. The service only ever writes `granted`; it follows that any `withdrawn` row in the ledger was written by a human, and the service never touches that pair again. No provenance column, no marker, no flag, and nothing for the application to observe.
 

devplacepy/services/background.py

@@ -11,6 +11,7 @@ logger = logging.getLogger(__name__)
 QUEUE_MAXSIZE = 10000
 JOIN_TIMEOUT_SECONDS = 10
 GET_TIMEOUT_SECONDS = 1
+SHUTDOWN_DRAIN_MAX = 1000
 
 
 class _Stop:
@@ -30,6 +31,7 @@ class BackgroundQueue:
         self._processed = 0
         self._failed = 0
         self._inline = 0
+        self._dropped = 0
 
     @property
     def running(self) -> bool:
@@ -90,17 +92,23 @@ class BackgroundQueue:
         self._thread = None
         self._loop = None
         if thread is not None:
-            self._queue.put(_STOP)
+            try:
+                self._queue.put_nowait(_STOP)
+            except queue.Full:
+                logger.warning("background queue full at shutdown; worker keeps draining")
             await asyncio.to_thread(thread.join, JOIN_TIMEOUT_SECONDS)
         self._flush_remaining()
         logger.info(
-            "background task queue stopped processed=%s failed=%s inline=%s",
+            "background task queue stopped processed=%s failed=%s inline=%s dropped=%s",
             self._processed,
             self._failed,
             self._inline,
+            self._dropped,
         )
 
     def _flush_remaining(self) -> None:
+        drained = 0
+        dropped = 0
         while True:
             try:
                 item = self._queue.get_nowait()
@@ -108,8 +116,20 @@ class BackgroundQueue:
                 break
             if item is _STOP:
                 continue
+            if drained >= SHUTDOWN_DRAIN_MAX:
+                dropped += 1
+                continue
             fn, args, kwargs = item
             self._execute(fn, args, kwargs)
+            drained += 1
+        if dropped:
+            with self._counters_lock:
+                self._dropped += dropped
+            logger.warning(
+                "background queue dropped %s tasks at shutdown after draining %s",
+                dropped,
+                drained,
+            )
 
     def _worker(self) -> None:
         while True:
@@ -130,6 +150,7 @@ class BackgroundQueue:
             "processed": self._processed,
             "failed": self._failed,
             "inline": self._inline,
+            "dropped": self._dropped,
         }
 
 

devplacepy/services/containers/CLAUDE.md

@@ -94,7 +94,7 @@ Devii: `container_create_instance` takes the boot/run-as/start_on_boot args; `co
 
 ## Host port allocation
 
-Host ports are auto-allocated and globally unique. `parse_ports` accepts a bare container port (`8899`, host side `0` = auto) or a pinned `host:container`. `api.assign_host_ports` (called in `create_instance`) resolves every `host=0` to the lowest free port in `[HOST_PORT_MIN=20001, 65535]` that is neither in `used_host_ports()` (the union of every instance's published host ports from `ports_json`) nor currently bound on the host (`_host_port_free` test-binds `0.0.0.0:port`). A pinned host port already published by another instance is rejected up front. This guarantees no two instances ever collide on a host port - previously a silent `docker run` "port is already allocated" failure. The resolved host port is what gets stored in `ports_json` and what the ingress proxy reads.
+Host ports are auto-allocated and globally unique. `parse_ports` accepts a bare container port (`8899`, host side `0` = auto) or a pinned `host:container`, and rejects more than `MAX_PORTS_PER_INSTANCE=32` mappings per instance. `api.assign_host_ports` (called in `create_instance`) resolves every `host=0` to the lowest free port in `[HOST_PORT_MIN=20001, 65535]` that is neither in `used_host_ports()` (the union of every instance's published host ports from `ports_json`) nor currently bound on the host (`_host_port_free` test-binds `0.0.0.0:port`). The ascending scan keeps lowest-free semantics but stops after `MAX_PORT_PROBES=4096` failed bind probes instead of sweeping all 45k ports. A pinned host port already published by another instance is rejected up front. This guarantees no two instances ever collide on a host port - previously a silent `docker run` "port is already allocated" failure. The resolved host port is what gets stored in `ports_json` and what the ingress proxy reads.
 
 ## Floating terminals (interactive shell)
 

devplacepy/services/containers/api.py

@@ -33,6 +33,8 @@ INSTANCE_LABEL = "devplace.instance"
 PROJECT_LABEL = "devplace.project"
 HOST_PORT_MIN = 20001
 HOST_PORT_MAX = 65535
+MAX_PORT_PROBES = 4096
+MAX_PORTS_PER_INSTANCE = 32
 BOOT_LANGUAGES = ("none", "python", "bash")
 BOOT_SCRIPT_FILES = {"python": ".devplace_boot.py", "bash": ".devplace_boot.sh"}
 BOOT_SCRIPT_RUNNERS = {"python": "python", "bash": "bash"}
@@ -114,6 +116,10 @@ def parse_ports(value) -> list:
                 f"port '{item}' must be numeric host:container or a bare container port"
             )
         ports.append(PortMapping(int(host), int(container), proto.strip() or "tcp"))
+    if len(ports) > MAX_PORTS_PER_INSTANCE:
+        raise ContainerError(
+            f"at most {MAX_PORTS_PER_INSTANCE} port mappings per instance"
+        )
     return ports
 
 
@@ -138,11 +144,17 @@ def _host_port_free(port: int) -> bool:
 
 
 def allocate_host_port(reserved: set) -> int:
+    probes = 0
     for port in range(HOST_PORT_MIN, HOST_PORT_MAX + 1):
         if port in reserved:
             continue
         if _host_port_free(port):
             return port
+        probes += 1
+        if probes >= MAX_PORT_PROBES:
+            raise ContainerError(
+                f"no free host port found after probing {MAX_PORT_PROBES} ports"
+            )
     raise ContainerError(
         f"no free host port available in range {HOST_PORT_MIN}-{HOST_PORT_MAX}"
     )

devplacepy/services/devii/CLAUDE.md

@@ -142,7 +142,7 @@ Devii's project filesystem tools include surgical **line-range editing** (`proje
 
 ## Web fetch and arbitrary HTTP (`handler="fetch"`)
 
-`services/devii/fetch/FetchController` (a standalone `httpx.AsyncClient`, not `PlatformClient`) backs two tools: `fetch_url` (read a page as readable text, `requires_auth=False`) and `http_request` (`requires_auth=True`) - the general external HTTP client. `http_request` takes `method` (GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS), `headers` (object), and one body of `json` (object/array, Content-Type set automatically), `form` (url-encoded), or `body` (raw string), and returns `{http_status, headers, content_type, content}`; it does **not** raise on non-2xx so the model can read API errors. Both run through one `_stream()` (`fetch_url` via `_download` with `raise_5xx=True`; `http_request` with `raise_5xx=False`) sharing the SSRF `_guard` (refuses private/loopback unless `DEVII_FETCH_ALLOW_PRIVATE`), `fetch_max_bytes` cap, timeout, and decoding. The `object`-typed params required adding an `object` branch to `Action.tool_schema()` in `spec.py` (emits a valid open-object schema). The `SYSTEM_PROMPT` "WEB REQUESTS AND EXTERNAL APIS" rule tells the agent to call `http_request` DIRECTLY for an API and never wrap it in a self-invoking user-defined tool (a virtual tool only re-runs the agent and cannot do I/O, so wrapping the call recurses into the `MAX_EVAL_DEPTH` guard - the exact failure this fixed). To attach a remote file see `attach_url` / `store_attachment_from_url` under Attachments and media.
+`services/devii/fetch/FetchController` (a standalone `httpx.AsyncClient`, not `PlatformClient`) backs two tools: `fetch_url` (read a page as readable text, `requires_auth=False`) and `http_request` (`requires_auth=True`) - the general external HTTP client. `http_request` takes `method` (GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS), `headers` (object), and one body of `json` (object/array, Content-Type set automatically), `form` (url-encoded), or `body` (raw string), and returns `{http_status, headers, content_type, content}`; it does **not** raise on non-2xx so the model can read API errors. Both run through one `_stream()` (`fetch_url` via `_download` with `raise_5xx=True`; `http_request` with `raise_5xx=False`) sharing the SSRF `_guard` (refuses private/loopback unless `DEVII_FETCH_ALLOW_PRIVATE`), `fetch_max_bytes` cap, timeout, and decoding. Returned content is clamped by `_optional_cap` to `settings.fetch_max_chars` (default 200000, `DEVII_FETCH_MAX_CHARS`) - the model's `max_chars` argument can only narrow below that ceiling, never exceed it, and a missing or invalid value falls back to the ceiling. The `object`-typed params required adding an `object` branch to `Action.tool_schema()` in `spec.py` (emits a valid open-object schema). The `SYSTEM_PROMPT` "WEB REQUESTS AND EXTERNAL APIS" rule tells the agent to call `http_request` DIRECTLY for an API and never wrap it in a self-invoking user-defined tool (a virtual tool only re-runs the agent and cannot do I/O, so wrapping the call recurses into the `MAX_EVAL_DEPTH` guard - the exact failure this fixed). To attach a remote file see `attach_url` / `store_attachment_from_url` under Attachments and media.
 
 ## Remote web tools (rsearch, non-platform)
 

devplacepy/services/devii/fetch/controller.py

@@ -43,12 +43,13 @@ class FetchController:
         raise ToolInputError(f"Unknown fetch tool: {name}")
 
     def _optional_cap(self, requested: Any, content: str) -> str:
+        ceiling = max(MIN_FETCH_CHARS, self._settings.fetch_max_chars)
         if requested is None:
-            return content
+            return content[:ceiling]
         try:
-            cap = max(MIN_FETCH_CHARS, int(requested))
+            cap = min(max(MIN_FETCH_CHARS, int(requested)), ceiling)
         except (TypeError, ValueError):
-            return content
+            return content[:ceiling]
         return content[:cap]
 
     async def _fetch_url(self, arguments: dict[str, Any]) -> str:

devplacepy/services/gitea/CLAUDE.md

@@ -45,4 +45,4 @@ Note: for Gitea to accept every file type the instance `app.ini` `[attachment] A
 
 ### Schemas, forms, and tools
 
-Schemas: `IssueItemOut`/`IssueCommentOut`/`IssueDetailOut`/`IssueAttachmentsOut`/`IssuesOut`/`IssueJobOut` (`schemas.py`). Forms: `IssueForm`/`IssueCommentForm`/`IssueAttachmentForm`/`IssueStatusForm` (`models.py`). Devii tools: `list_issues`, `create_issue`, `view_issue`, `comment_issue`, `set_issue_status` (admin), plus the attachment tools above. Docs: the `issues` API group in `docs_api.py`. Footer link in `base.html` under `.site-footer`.
+Schemas: `IssueItemOut`/`IssueCommentOut`/`IssueDetailOut`/`IssueAttachmentsOut`/`IssuesOut`/`IssueJobOut` (`schemas.py`). Forms: `IssueForm`/`IssueCommentForm`/`IssueAttachmentForm`/`IssueStatusForm` (`models/community.py`). Devii tools: `list_issues`, `create_issue`, `view_issue`, `comment_issue`, `set_issue_status` (admin), plus the attachment tools above. Docs: the `issues` API group in `docs_api.py`. Footer link in `base.html` under `.site-footer`.

devplacepy/services/openai_gateway/CLAUDE.md

@@ -47,7 +47,7 @@ Callers may send an optional `X-App-Reference` header to tag gateway calls by ap
 A dedicated per-IP throttle protects `/openai/v1/*` against unauthenticated probing WITHOUT IP-allowlisting the endpoint (which would break legitimate external API consumers using `gateway_access_key`/`gateway_allow_users`). It is scoped ONLY to failed authentication attempts, never to general traffic - the existing global rate limiter already exempts `/openai` entirely (`main.py`), and this throttle does not change that exemption.
 
 - **Config (`gateway_auth_throttle_enabled` default on, `gateway_auth_throttle_max_failures` default 10, `gateway_auth_throttle_window_seconds` default 60, all group "Access").** An admin can retune or disable the throttle with no code change, same as every other gateway setting.
-- **Counter.** `auth_throttle.py` is a small in-process sliding-window store (`dict[ip] -> list[float]` timestamps), the same shape as the global rate limiter's `_rate_limit_store` in `main.py`, but a separate, dedicated store local to the gateway rather than reusing that one (the two throttles key on different events - all mutating traffic vs. failed gateway auth - and `/openai` is deliberately exempt from the general limiter). `record_failure(ip, window)` appends a timestamp and returns the running in-window count; `is_throttled(ip, threshold, window)` checks without recording. Per-worker, like every other in-process cache in this codebase - not divided by worker count, so the effective aggregate threshold across `N` workers is up to `N x max_failures`; this is an accepted tradeoff (it still closes the probe, just at a slightly higher aggregate bound) rather than importing `main.WEB_WORKERS` into a leaf service module.
+- **Counter.** `auth_throttle.py` is a small in-process sliding-window store (`dict[ip] -> list[float]` timestamps), the same shape as the global rate limiter's `_rate_limit_store` in `middleware/state.py`, but a separate, dedicated store local to the gateway rather than reusing that one (the two throttles key on different events - all mutating traffic vs. failed gateway auth - and `/openai` is deliberately exempt from the general limiter). `record_failure(ip, window)` appends a timestamp and returns the running in-window count; `is_throttled(ip, threshold, window)` checks without recording. Per-worker, like every other in-process cache in this codebase - not divided by worker count, so the effective aggregate threshold across `N` workers is up to `N x max_failures`; this is an accepted tradeoff (it still closes the probe, just at a slightly higher aggregate bound) rather than importing `state.WEB_WORKERS` into a leaf service module.
 - **`GatewayService.authorize()` records a failure on every deny path** (a cheap static-key/internal-key match still short-circuits first and is never counted), so a wrong or garbage key counts exactly like presenting nothing. **The fast 429 short-circuit fires only when the request carries no credential at all** (no `X-API-KEY`, no `Authorization` header of any scheme, no `session` cookie) AND the IP is already over threshold - this is the "skip the rest of `authorize()`" defense against wasted CPU on a blind flood. A request presenting ANY credential - even one that turns out invalid - always runs the full check, so it can never be blocked by another caller's failures sharing its IP (a NAT/proxy caller with its own valid key is never punished for a neighbor's failed probes); only a request presenting nothing is fast-denied once its IP has tripped.
 - **Audit.** Tripping the throttle (the exact request whose failure brings the window count to the configured threshold) records `ai.gateway_auth_throttle.tripped` (category `ai`, `result="denied"`, metadata `ip`/`failed_attempts`/`threshold`/`window_seconds`) via `audit.record(request, ...)`. Every subsequent fast-denied request in the same trip is silent (no audit spam under an actual flood) - the trip itself is the visible signal for admins.
 - **Internal traffic cannot trip this by construction.** Every internal consumer (Devii guests, AI correction/modifier, quiz grading, SEO metadata generation, news, bots) authenticates with `database.internal_gateway_key()`, which is the exact same value `authorize()` compares against as `gateway_internal_key` - that comparison is the FIRST check in `authorize()`, before the throttle is even consulted, so internal self-dial calls always succeed there and never reach the failure-recording branch.

devplacepy/static_files.py

@@ -0,0 +1,54 @@
+# retoor <retoor@molodetz.nl>
+
+from pathlib import Path
+
+from fastapi.staticfiles import StaticFiles
+
+INLINE_MEDIA_EXTENSIONS = {
+    ".jpg",
+    ".jpeg",
+    ".png",
+    ".gif",
+    ".webp",
+    ".bmp",
+    ".tiff",
+    ".mp4",
+    ".webm",
+    ".ogv",
+    ".mov",
+    ".m4v",
+    ".mp3",
+}
+
+
+class UploadStaticFiles(StaticFiles):
+    async def get_response(self, path, scope):
+        response = await super().get_response(path, scope)
+        disposition = (
+            "inline"
+            if Path(path).suffix.lower() in INLINE_MEDIA_EXTENSIONS
+            else "attachment"
+        )
+        response.headers["Content-Disposition"] = disposition
+        response.headers["Cache-Control"] = "public, max-age=604800"
+        return response
+
+
+class CachedStaticFiles(StaticFiles):
+    async def get_response(self, path, scope):
+        response = await super().get_response(path, scope)
+        if Path(path).name == "service-worker.js":
+            response.headers["Cache-Control"] = "no-cache"
+        else:
+            response.headers["Cache-Control"] = "public, max-age=31536000, immutable"
+        return response
+
+
+class FallbackStaticFiles(StaticFiles):
+    async def get_response(self, path, scope):
+        response = await super().get_response(path, scope)
+        if Path(path).name == "service-worker.js":
+            response.headers["Cache-Control"] = "no-cache"
+        else:
+            response.headers["Cache-Control"] = "public, max-age=3600"
+        return response

devplacepy/templates/docs/architecture-backend.html

@@ -7,7 +7,7 @@ How a request flows through middleware to a router, how the database layer is bu
 
 ## Application setup
 
-`main.py` builds the FastAPI app, mounts `/static`, registers every router with its prefix, installs the middlewares, and registers global 404, 500, and `RequestValidationError` handlers that render `error.html` through the shared `templates` instance.
+`main.py` builds the FastAPI app, mounts `/static`, registers every router with its prefix, installs the middlewares from `middleware/` in addition order, and registers the global handlers from `errors.py` (404, 500, `ContentRefused`, `RequestValidationError`) that render `error.html` through the shared `templates` instance. Startup and shutdown live in `lifespan.py`.
 
 ### Happy 404
 
@@ -78,7 +78,7 @@ In-process TTL caches accelerate hot reads but are **never authoritative**: `_se
 
 ## Models, schemas, and responses
 
-- **`models.py`** holds the Pydantic `Form` input models. Routers declare `data: Annotated[SomeForm, Form()]`; FastAPI validates, and the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes.
+- **`models/`** holds the Pydantic `Form` input models (one module per domain, re-exported from `models/__init__.py`). Routers declare `data: Annotated[SomeForm, Form()]`; FastAPI validates, and the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes.
 - **`schemas/`** (a package) holds the `*Out` response projections. They ignore extra keys, so the full context dict can be passed through, but only declared fields reach the JSON. `*Out` models never expose `password_hash`, `email`, or `api_key`.
 - **`responses.py`** provides content negotiation. `wants_json(request)` is true for `Accept: application/json` or `Content-Type: application/json`. Page GETs return `respond(request, "x.html", ctx, model=XOut)` (HTML or validated JSON); action POSTs return `action_result(request, url, data=...)` (a 302 redirect or `{ok, redirect, data}` JSON).
 
@@ -88,5 +88,5 @@ Session cookies named `session` hold a 64-char hex token from `secrets.token_hex
 
 ## Services framework
 
-`BaseService` provides the async run loop, typed `ConfigField` definitions, state persisted to `service_state` (status, last/next run, log deque, metrics), and admin commands (run, clear). `ServiceManager` is a singleton that registers, supervises, and stops services and tracks lock ownership. New services extend `BaseService`, override `async def run_once(self)`, and register in `main.py`; they then appear on `/admin/services`. `GatewayService` is the single source of truth for AI: every other AI consumer routes through it, so usage is attributed and limitable per owner.
+`BaseService` provides the async run loop, typed `ConfigField` definitions, state persisted to `service_state` (status, last/next run, log deque, metrics), and admin commands (run, clear). `ServiceManager` is a singleton that registers, supervises, and stops services and tracks lock ownership. New services extend `BaseService`, override `async def run_once(self)`, and register in `lifespan.py` `SERVICE_CLASSES`; they then appear on `/admin/services`. `GatewayService` is the single source of truth for AI: every other AI consumer routes through it, so usage is attributed and limitable per owner.
 </div>

devplacepy/templates/docs/architecture-conventions.html

@@ -46,7 +46,7 @@ The `comments` table keys on `(target_type, target_uid)`, so the same comment se
 
 ## Validation and redirects
 
-- **Form validation is Pydantic-native.** Declare a typed body param `data: Annotated[SomeForm, Form()]` (models live in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages or redirects other routes.
+- **Form validation is Pydantic-native.** Declare a typed body param `data: Annotated[SomeForm, Form()]` (models live in `models/`, one module per domain); the global `RequestValidationError` handler re-renders auth pages with messages or redirects other routes.
 - **`RedirectResponse(url=..., status_code=302)`** always passes `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
 
 ## SEO

devplacepy/templates/docs/architecture-workflow.html

@@ -9,7 +9,7 @@ How a change moves from understanding to shipped. The workflow is ordered by dat
 ## The feature workflow
 
 1. **Understand.** Read the router for the area (a flat `routers/{area}.py`, or the leaf inside the `routers/{area}/` package that owns the endpoint), the template, the test file, and the matching nested `CLAUDE.md` before writing. Trace the existing input model, router, data helper, and response.
-2. **Data layer.** Add query and batch helpers in the `database/` package (never inline N+1). Add the Pydantic `Form` model to `models.py` and the `*Out` response model to the `schemas/` package. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
+2. **Data layer.** Add query and batch helpers in the `database/` package (never inline N+1). Add the Pydantic `Form` model to the matching `models/` domain module (and its re-export in `models/__init__.py`) and the `*Out` response model to the `schemas/` package. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
 3. **Server layer.** Write the handler with the right guard (`get_current_user` for public reads, `require_user` for member POSTs, `require_admin` for admin). Declare specific routes before catch-alls. Return HTML and JSON from one handler via `respond(..., model=XOut)`. Register any new router in `main.py`.
 4. **View layer.** Import the shared `templates`, extend `base.html`, put page CSS in `{% block extra_head %}` and page JS in `{% block extra_js %}`, and reuse partials and globals.
 5. **Agent and docs layer (the most-forgotten step).** If the route is something a user could ask the assistant to do, add an `Action` to the Devii catalog. Add an `endpoint()` entry to the `docs_api/` package, and a prose page to `DOCS_PAGES` where relevant. Set SEO context for any new public page.

devplacepy/templates/docs/architecture.html

@@ -28,9 +28,11 @@ The recurring theme is deliberate minimalism: no JavaScript framework, no NPM, n
 ```
 devplacepy/
   main.py            app, middleware, router registration, lifecycle
+  middleware/      HTTP middlewares + shared request state (package)
+  errors.py        global exception handlers   lifespan.py   startup/shutdown
   config.py          paths, env vars, constants
   database/          SQLite/dataset, init_db, batch helpers, caching (package)
-  models.py          Pydantic Form input models
+  models/            Pydantic Form input models (package)
   schemas/           Pydantic *Out response projections (package)
   responses.py       respond() / action_result() / wants_json()
   utils/             auth guards, slugs, dates, gamification (package)

devplacepy/templates/docs/getting-started.html

@@ -22,11 +22,13 @@ Open `http://localhost:10500`. The first account you register becomes the admini
 
 ```
 devplacepy/        the application package
-  main.py          mounts routers, middleware, startup/shutdown
+  main.py          mounts routers, middleware, error handlers, lifespan
+  middleware/      HTTP middlewares + shared request state
+  errors.py        global exception handlers   lifespan.py   startup/shutdown
   routers/         one directory per URL domain (mirrors the endpoint tree)
   templates/       Jinja2 pages; static/ holds CSS and ES6 modules
   database/        SQLite via dataset; query and batch helpers
-  models.py        Pydantic Form models   schemas/    *Out JSON models
+  models/          Pydantic Form models   schemas/    *Out JSON models
   services/        background + async-job services, Devii, containers
 tests/             unit / api / e2e, mirroring the route or source path
 ```
@@ -38,7 +40,7 @@ as HTML, served as JSON, called by the Devii assistant as a tool, and described
 docs. The usual failure is changing one face and forgetting a connected one. Work in this
 order so nothing is dropped:
 
-1. **Data** - query helpers in `database.py`, a `Form` model in `models.py`, an `*Out`
+1. **Data** - query helpers in `database.py`, a `Form` model in `models/`, an `*Out`
    model in `schemas.py`.
 2. **Server** - the handler in `routers/`, with the right guard (`get_current_user` for
    public reads, `require_user`, `require_admin`); return both faces via

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "devplacepy"
-version = "1.1.7"
+version = "1.1.8"
 description = "DevPlace - The Developer Social Network"
 requires-python = ">=3.12"
 dependencies = [

tests/api/auth/login.py

@@ -156,18 +156,19 @@ def test_rate_limit_block_recorded(monkeypatch):
     # limiter re-enabled for itself - the same approach as test_ratelimit.py - and
     # asserts the block emits the security.rate_limit.block audit row.
     import devplacepy.main as m
+    import devplacepy.middleware.state as st
     from starlette.testclient import TestClient
 
-    monkeypatch.setattr(m, "RATE_LIMIT_DISABLED", False)
+    monkeypatch.setattr(st, "RATE_LIMIT_DISABLED", False)
     monkeypatch.setattr(
-        m,
+        st,
         "get_int_setting",
         lambda key, default: {
             "rate_limit_per_minute": 1,
             "rate_limit_window_seconds": 60,
         }.get(key, default),
     )
-    m._rate_limit_store.clear()
+    st._rate_limit_store.clear()
     ip = "203.0.113.77"
     client = TestClient(m.app)
     codes = [

tests/api/robotstxt.py

@@ -1,6 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
 import devplacepy.main as m
+import devplacepy.middleware.state as st
 from starlette.testclient import TestClient
 def _patch_limits(monkeypatch, limit, window=60):
     def fake_get_int_setting(key, default):
@@ -9,10 +10,10 @@ def _patch_limits(monkeypatch, limit, window=60):
             "rate_limit_window_seconds": window,
         }.get(key, default)
 
-    monkeypatch.setattr(m, "get_int_setting", fake_get_int_setting)
+    monkeypatch.setattr(st, "get_int_setting", fake_get_int_setting)
     # conftest disables the limiter suite-wide; re-enable it for these tests only.
-    monkeypatch.setattr(m, "RATE_LIMIT_DISABLED", False)
-    m._rate_limit_store.clear()
+    monkeypatch.setattr(st, "RATE_LIMIT_DISABLED", False)
+    st._rate_limit_store.clear()
 import requests
 from tests.conftest import BASE_URL
 def _seed_news_seo():

tests/api/root.py

@@ -3,6 +3,7 @@
 import re
 
 import devplacepy.main as m
+import devplacepy.middleware.state as st
 from starlette.testclient import TestClient
 def _patch_limits(monkeypatch, limit, window=60):
     def fake_get_int_setting(key, default):
@@ -11,14 +12,14 @@ def _patch_limits(monkeypatch, limit, window=60):
             "rate_limit_window_seconds": window,
         }.get(key, default)
 
-    monkeypatch.setattr(m, "get_int_setting", fake_get_int_setting)
+    monkeypatch.setattr(st, "get_int_setting", fake_get_int_setting)
     # conftest disables the limiter suite-wide; re-enable it for these tests only.
-    monkeypatch.setattr(m, "RATE_LIMIT_DISABLED", False)
-    m._rate_limit_store.clear()
+    monkeypatch.setattr(st, "RATE_LIMIT_DISABLED", False)
+    st._rate_limit_store.clear()
     # _hot_settings caches the limit for HOT_SETTINGS_TTL; force a re-read of the
     # patched get_int_setting so a stale value from a prior test cannot leak in.
-    m._hot_settings_value = None
-    m._hot_settings_at = 0.0
+    st._hot_settings_value = None
+    st._hot_settings_at = 0.0
 
 
 def test_rate_limit_blocks_excess(monkeypatch):

tests/unit/services/acceptance/isolation.py

@@ -27,13 +27,13 @@ def references(path):
     return any(needle in body for needle in NEEDLES)
 
 
-def test_only_main_imports_the_acceptance_package():
+def test_only_lifespan_imports_the_acceptance_package():
     importers = [
         path.relative_to(PACKAGE_ROOT).as_posix()
         for path in source_files({".py"})
         if references(path)
     ]
-    assert importers == ["main.py"]
+    assert importers == ["lifespan.py"]
 
 
 def test_no_template_or_static_asset_mentions_the_package():

tests/unit/services/background.py

@@ -0,0 +1,54 @@
+# retoor <retoor@molodetz.nl>
+
+import queue
+import threading
+
+from tests.conftest import run_async
+from devplacepy.services.background import BackgroundQueue
+
+
+def _fill(direct_queue, count, executed):
+    for _ in range(count):
+        direct_queue.put_nowait((executed.append, ("x",), {}))
+
+
+def test_stop_without_start_returns_cleanly():
+    direct = BackgroundQueue()
+    run_async(direct.stop())
+    assert direct.stats()["running"] is False
+
+
+def test_stop_full_queue_does_not_block():
+    direct = BackgroundQueue(maxsize=4)
+    dead = threading.Thread(target=lambda: None)
+    dead.start()
+    dead.join()
+    direct._thread = dead
+    executed = []
+    _fill(direct._queue, 4, executed)
+    run_async(direct.stop())
+    assert executed == ["x"] * 4
+
+
+def test_stop_bounds_shutdown_drain_and_counts_dropped():
+    from devplacepy.services.background import SHUTDOWN_DRAIN_MAX
+
+    direct = BackgroundQueue(maxsize=5000)
+    executed = []
+    _fill(direct._queue, SHUTDOWN_DRAIN_MAX + 200, executed)
+    run_async(direct.stop())
+    stats = direct.stats()
+    assert len(executed) == SHUTDOWN_DRAIN_MAX
+    assert stats["dropped"] == 200
+
+
+def test_start_stop_roundtrip_executes_submitted_task():
+    direct = BackgroundQueue()
+    run_async(direct.start())
+    try:
+        executed = []
+        direct.submit(executed.append, "x")
+    finally:
+        run_async(direct.stop())
+    assert executed == ["x"]
+    assert direct.stats()["running"] is False

tests/unit/services/containers/api.py

@@ -793,3 +793,39 @@ def test_tunnel_target_is_empty_without_a_route_to_the_port():
     instance = {"ports_json": "[]", "container_gateway": "172.17.0.1"}
     assert api.tunnel_target(instance, 3000) == (None, None)
     assert api.tunnel_target({"container_ip": "172.17.0.9"}, 0) == (None, None)
+
+
+def test_allocate_host_port_returns_lowest_free(monkeypatch):
+    monkeypatch.setattr(api, "_host_port_free", lambda port: True)
+    assert api.allocate_host_port(set()) == api.HOST_PORT_MIN
+    assert api.allocate_host_port({20001, 20002}) == 20003
+
+
+def test_allocate_host_port_skips_bound_ports(monkeypatch):
+    monkeypatch.setattr(api, "_host_port_free", lambda port: port != 20001)
+    assert api.allocate_host_port(set()) == 20002
+
+
+def test_allocate_host_port_probe_budget_exhausted(monkeypatch):
+    from devplacepy.services.containers.api import ContainerError
+
+    calls = []
+    monkeypatch.setattr(
+        api, "_host_port_free", lambda port: calls.append(port) or False
+    )
+    with pytest.raises(ContainerError, match="probing"):
+        api.allocate_host_port(set())
+    assert len(calls) == api.MAX_PORT_PROBES
+
+
+def test_parse_ports_accepts_mapping_cap():
+    value = "\n".join(str(8000 + i) for i in range(api.MAX_PORTS_PER_INSTANCE))
+    assert len(api.parse_ports(value)) == api.MAX_PORTS_PER_INSTANCE
+
+
+def test_parse_ports_rejects_too_many_mappings():
+    from devplacepy.services.containers.api import ContainerError
+
+    value = "\n".join(str(8000 + i) for i in range(api.MAX_PORTS_PER_INSTANCE + 1))
+    with pytest.raises(ContainerError, match="at most"):
+        api.parse_ports(value)

tests/unit/services/devii/fetch/__init__.py

@@ -0,0 +1 @@
+# retoor <retoor@molodetz.nl>

tests/unit/services/devii/fetch/controller.py

@@ -0,0 +1,41 @@
+# retoor <retoor@molodetz.nl>
+
+from dataclasses import replace
+
+from devplacepy.services.devii.config import load_settings
+from devplacepy.services.devii.fetch.controller import MIN_FETCH_CHARS, FetchController
+
+
+def _controller(ceiling):
+    return FetchController(replace(load_settings(), fetch_max_chars=ceiling))
+
+
+def test_optional_cap_defaults_to_ceiling():
+    controller = _controller(2000)
+    assert controller._optional_cap(None, "a" * 5000) == "a" * 2000
+
+
+def test_optional_cap_clamps_large_request_to_ceiling():
+    controller = _controller(2000)
+    assert controller._optional_cap(10**9, "b" * 5000) == "b" * 2000
+
+
+def test_optional_cap_honors_smaller_request():
+    controller = _controller(2000)
+    assert controller._optional_cap(1500, "c" * 5000) == "c" * 1500
+
+
+def test_optional_cap_floors_tiny_request():
+    controller = _controller(2000)
+    assert controller._optional_cap(10, "d" * 5000) == "d" * MIN_FETCH_CHARS
+
+
+def test_optional_cap_invalid_request_falls_back_to_ceiling():
+    controller = _controller(2000)
+    assert controller._optional_cap("many", "e" * 5000) == "e" * 2000
+
+
+def test_optional_cap_leaves_short_content_alone():
+    controller = _controller(2000)
+    assert controller._optional_cap(None, "short") == "short"
+    assert controller._optional_cap(1500, "short") == "short"