iUpdate

Restructures the disk-cleanup command to show location, current on-disk
size, and an estimated reclaim per area (attachments, project files,
container workspaces) before touching anything - --dry-run stops there.
A real run re-executes each check, reports actual items/bytes freed,
an "After" size per area, and a final total with elapsed time, flagging
any drift between the estimate and execution passes (e.g. the live
server wrote something in between).

store.gc_workspaces() now reports bytes freed alongside the removed
count (previously count-only), so the containers gc-workspaces CLI
action picks up the same reporting for free.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W5tFWkm3UbcstcbPKFE5gP

.coveragerc

@@ -1,7 +1,7 @@
 # retoor <retoor@molodetz.nl>
 [run]
 source = devplacepy
-parallel = true
+parallel = false
 sigterm = true
 omit =
     tests/*

.env.example

@@ -29,9 +29,16 @@ SECRET_KEY=change-me
 # from the request.
 DEVPLACE_SITE_URL=
 
-# Host port the nginx front door binds.
+# Host port the nginx front door binds (Docker only - the app container's own
+# internal port stays fixed).
 PORT=10500
 
+# Port the uvicorn process itself binds to for `make dev`/`make prod` (bare
+# metal, no Docker/nginx in front). Also what the app calls itself on
+# internally (DEVII_BASE_URL default, INTERNAL_GATEWAY_URL). Unrelated to
+# PORT above - leave unset unless running bare metal on a non-default port.
+# DEVPLACE_PORT=10500
+
 # nginx upload ceiling. Must be >= the admin-configurable max_upload_size_mb.
 NGINX_MAX_BODY_SIZE=50m
 

.githooks/pre-commit

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+# retoor <retoor@molodetz.nl>
+
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+REPO_ROOT = Path(
+    subprocess.run(
+        ["git", "rev-parse", "--show-toplevel"],
+        capture_output=True,
+        text=True,
+        check=True,
+    ).stdout.strip()
+)
+PYPROJECT = REPO_ROOT / "pyproject.toml"
+VERSION_LINE = re.compile(r'^version = "(\d+)\.(\d+)\.(\d+)"$', re.MULTILINE)
+STAGED_VERSION_CHANGE = re.compile(r'^[+-]version = "\d+\.\d+\.\d+"$', re.MULTILINE)
+
+
+def staged_diff(path: Path) -> str:
+    result = subprocess.run(
+        ["git", "diff", "--cached", "--unified=0", "--", str(path)],
+        capture_output=True,
+        text=True,
+        cwd=REPO_ROOT,
+    )
+    return result.stdout
+
+
+def main() -> int:
+    if (REPO_ROOT / ".git" / "MERGE_HEAD").exists():
+        return 0
+    if not PYPROJECT.is_file():
+        return 0
+    if STAGED_VERSION_CHANGE.search(staged_diff(PYPROJECT)):
+        return 0
+    text = PYPROJECT.read_text()
+    match = VERSION_LINE.search(text)
+    if not match:
+        return 0
+    major, minor, patch = (int(part) for part in match.groups())
+    bumped = f'version = "{major}.{minor}.{patch + 1}"'
+    PYPROJECT.write_text(VERSION_LINE.sub(bumped, text, count=1))
+    subprocess.run(["git", "add", str(PYPROJECT)], cwd=REPO_ROOT, check=True)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

.gitignore

@@ -1,3 +1,6 @@
+.dpc
+.devplace
+dpc.log
 .cache
 .local
 .devplace_bots/
@@ -13,6 +16,10 @@ devplace-init.lock
 notification-private.pem
 notification-private.pkcs8.pem
 notification-public.pem
+# HOME=/app in the Docker app container, so an rclone.conf created via
+# `rclone config` or DEVPLACE_RCLONE_CONFIG's default lands here on the
+# bind-mounted host tree - it holds live remote-storage credentials.
+/.config/
 .pytest_cache/
 .ruff_cache/
 .opencode

CLAUDE.md

@@ -19,7 +19,7 @@ DevPlace is a server-rendered social network for developers. FastAPI backend ser
 ## Commands
 
 ```bash
-make install        # pip install -e . + playwright install chromium
+make install        # create .venv if needed, pip install -e ".[dev]", playwright chromium
 make ppy            # build the single shared container image (ppy:latest); run once before launching instances
 make dev            # uvicorn --reload on port 10500, backlog 4096
 make prod           # uvicorn --workers $(WEB_WORKERS) (defaults to nproc), port 10500 (backlog 8192)
@@ -33,6 +33,8 @@ make locust         # Locust load test, interactive web UI
 make locust-headless  # Locust CLI mode for CI
 ```
 
+Every Python make target (`install`, `dev`, `prod`, `test*`, `coverage*`, `locust*`) uses `.venv/bin/python`. If `.venv` is missing, make creates it from `python3`, installs `-e ".[dev]"`, and installs Playwright Chromium before running the target. `make install` refreshes that environment. `make clean` removes `.venv` as well as stray bytecode.
+
 The Makefile exports `PYTHONDONTWRITEBYTECODE=1` for every recipe, so no `.pyc` files or `__pycache__` directories are written by any `make` target. Keep it that way - do not add a target that re-enables bytecode writing. `make clean` removes any stray bytecode left from running Python outside make.
 
 Preliminary validation: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance). These checks are gates on the way to the real validation, never a substitute for it: **every change ends with the full test suite (`make test` - all three tiers, every test) and it must pass.**
@@ -56,6 +58,7 @@ devplace token prune                # soft-delete all expired access tokens
 devplace news clear         # delete all news rows
 devplace news sanitize      # strip HTML from news descriptions/content
 devplace attachments prune  # remove orphan attachment records/files
+devplace system prune [--dry-run]  # safe but aggressive: purges soft-deleted attachment/project-file blobs, sweeps blob files with zero DB reference at all (e.g. left by an interrupted/racing sync), and GCs orphaned container workspace dirs
 devplace devii reset-quota <username>   # reset one user's rolling 24h AI quota
 devplace devii reset-quota --guests     # reset every guest quota
 devplace devii reset-quota --all        # reset every quota (users and guests)
@@ -114,6 +117,7 @@ Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA
 | Var | Default | Purpose |
 |-----|---------|---------|
 | `DEVPLACE_DATABASE_URL` | `sqlite:///<repo>/data/devplace.db` | Override DB path (tests use this) |
+| `DEVPLACE_PORT` | `10500` | Port the app itself binds to bare-metal (`make dev`/`make prod`, wired to uvicorn's `--port`; tests use their own `DEVPLACE_TEST_PORT`, default `10501`). Also `config.PORT`'s source, so `DEVPLACE_INTERNAL_BASE_URL`'s default and Devii's self-dial URL (`DEFAULT_BASE_URL`/`INSTANCE_ORIGIN_DEFAULT`) follow it automatically. The Docker app container pins it to `10500` in `docker-compose.yml` regardless of `.env` - Docker's externally reachable port is the unrelated `PORT` var (nginx's host mapping), never this one. |
 | `SECRET_KEY` | hardcoded fallback | Session signing |
 | `DEVPLACE_DISABLE_SERVICES` | unset | When `1`, NewsService and other background services skip start (set by test conftest) |
 | `PLAYWRIGHT_HEADLESS` | `1` in tests | Toggle headed mode |
@@ -143,7 +147,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
 | `devplacepy/services/moderation/CLAUDE.md` | Trust and safety: the reportable-target registry, the content filter and its five choke points, the report queue and its atomic resolution, enforcement, consent, maturity, account deletion |
 | `devplacepy/services/acceptance/CLAUDE.md` | Acceptance convergence: the opt-in service that grants every policy agreement to every account that has not declined it, its invisibility contract and the ledger-as-decline-register rule |
 | `devplacepy/services/audit/CLAUDE.md` | Audit log: recorders, categories, retention |
-| `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download |
+| `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download, remote offload to Hetzner Storage Box |
 | `devplacepy/services/telegram/CLAUDE.md` | Telegram bot bridge |
 | `devplacepy/services/email/CLAUDE.md` | Devii IMAP/SMTP email tools |
 | `devplacepy/services/gitea/CLAUDE.md` | Issue tracker (Gitea-backed, no local issue store) |
@@ -155,6 +159,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
 | `devplacepy/services/pubsub/CLAUDE.md` | Database-free pub/sub bus |
 | `devplacepy/push/CLAUDE.md` | Push notification providers: the `PushProvider` protocol, the registry, Web Push and APNs, registration storage |
 | `devplacepy/services/game/CLAUDE.md` | Code Farm idle game (economy invariants, raids, the one-pure-function rule) |
+| `devplacepy/services/opinionwar/CLAUDE.md` | Opinion Wars (week-long faction battles on posts: atomic fight/resolve transitions, cooldown-before-coins compensation, event seq allocation, relay-on-lock-owner) |
 | `devplacepy/services/quiz/CLAUDE.md` | Quizzes (the terminal publish lock, attempt atomicity, answer-key withholding, AI free-text grading, the best-attempt scoreboard) |
 | `devplacepy/services/CLAUDE.md` | Background task queue, AI correction/modifier, presence, live view relay, `BaseService`/`ServiceManager` |
 | `devplacepy/database/CLAUDE.md` | Dataset rules, indexing conventions, soft delete, tables, site settings |
@@ -182,6 +187,7 @@ Routers in `devplacepy/routers/` are organised as a **directory tree that mirror
 |--------|--------|
 | `/auth` | auth/ package |
 | `/feed`, `/posts`, `/comments` | flat files |
+| `/topics` | topics.py - crawlable per-topic category index pages (`/topics` hub, `/topics/{topic}` listing) |
 | `/projects`, `/projects/{slug}/files`, `/projects/{slug}/containers` | projects/ package - see `routers/projects/CLAUDE.md` |
 | `/profile` | profile/ package (customization, notifications, ai-correction, ai-modifier, interactions, telegram, usage) |
 | `/messages` | messages.py - see `services/messaging/CLAUDE.md` |
@@ -201,6 +207,7 @@ Routers in `devplacepy/routers/` are organised as a **directory tree that mirror
 | `/game` | game/ package - see `services/game/CLAUDE.md` |
 | `/reports`, `/admin/moderation`, `/workspaces` | reports.py, admin/moderation.py, workspaces.py - see `services/moderation/CLAUDE.md` |
 | `/quizzes` | quizzes/ package - see `services/quiz/CLAUDE.md` |
+| `/battles` | battles.py - see `services/opinionwar/CLAUDE.md` |
 | (none) | push.py (web push/PWA), docs.py (see `routers/docs/CLAUDE.md`) |
 | `/pubsub` | pubsub.py - see `services/pubsub/CLAUDE.md` |
 | (none) | seo.py - `/robots.txt`, `/sitemap.xml` |
@@ -256,7 +263,7 @@ Users and guests inject their own CSS and JS, scoped to a page type or globally,
 
 ### Container manager, Devii assistant, AI gateway, async jobs, audit log
 
-DevPlace ships a Docker-backed **Container manager** (admin-only, one shared `ppy:latest` image, security-hardened rootless workflow via a `sudo` superclone and `aptroot`), the **Devii** in-platform agentic assistant (WebSocket terminal, per-owner-channel sessions, scheduler/reminders, self-learning lessons, user-defined virtual tools, self-configured behavior, browser-automation client tools), the **AI gateway** (`/openai/v1/*`, single point of truth for every AI call, per-user cost attribution, provider/model routing overlay), **async job services** (`JobService` base pattern: zip, fork, SEO diagnostics, SEO metadata generation, DeepSearch, AI Usage Analyzer), and an admin-only append-only **audit log** (`record`/`record_system` entrypoints, `events.md` catalogue of 288 keys, never raises into the caller). These are among the largest subsystems in the codebase - read their dedicated nested `CLAUDE.md` files (see Subsystem map) before working in any of them; do not assume prior knowledge from this summary.
+DevPlace ships a Docker-backed **Container manager** (admin-only, one shared `ppy:latest` image, security-hardened rootless workflow via a `sudo` superclone and `aptroot`), the **Devii** in-platform agentic assistant (WebSocket terminal, per-owner-channel sessions, scheduler/reminders, self-learning lessons, user-defined virtual tools, self-configured behavior, browser-automation client tools), the **AI gateway** (`/openai/v1/*`, single point of truth for every AI call, per-user cost attribution, provider/model routing overlay), **async job services** (`JobService` base pattern: zip, fork, SEO diagnostics, SEO metadata generation, DeepSearch, AI Usage Analyzer), and an admin-only append-only **audit log** (`record`/`record_system` entrypoints, `events.md` catalogue of every recorded event key, never raises into the caller). These are among the largest subsystems in the codebase - read their dedicated nested `CLAUDE.md` files (see Subsystem map) before working in any of them; do not assume prior knowledge from this summary.
 
 ### Telegram bot, email, devRant compatibility API, issue tracker
 
@@ -280,7 +287,7 @@ The escape hatch is deliberately two-factor and must never be self-served: after
 
 - **No comments, no docstrings in source.** Code is self-documenting.
 - **Forbidden variable name patterns:** `_new`, `_old`, `_temp`, `_v2`, `better_`, `my_`, `the_`.
-- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: the reverse-proxy forwarding core `services/containers/forward.py` (and the three routers that call it - `routers/proxy.py`, `routers/tunnel.py`, the workspace editor route), which relays the user's own headers verbatim and therefore keeps a plain `httpx.AsyncClient`; bolting the Chrome identity onto it would overwrite the very headers it exists to forward. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
+- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, web push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Two exceptions keep a plain `httpx.AsyncClient`: (1) the reverse-proxy forwarding core `services/containers/forward.py` (and the three routers that call it - `routers/proxy.py`, `routers/tunnel.py`, the workspace editor route), which relays the user's own headers verbatim - bolting the Chrome identity onto it would overwrite the very headers it exists to forward; (2) the APNs provider (`push/providers/apns.py` `gateway_client` / `delivery_client`) which talks to Apple's HTTP/2 provider API with `httpx.AsyncClient(http2=True, trust_env=False)` - Chrome impersonation, HTTP/2 PRIORITY frames, HPACK indexing of `:path`/`authorization`, extra browser headers, and the outbound proxy all break or starve that API. Web Push stays on stealth. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
 - **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
 - **`RedirectResponse(url=..., status_code=302)`** - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
 - **Ownership checks:** compare `resource["user_uid"] == user["uid"]` before edit/delete (use `content.is_owner`). **Deletes additionally allow any admin:** `is_owner(...) or is_admin(user)` on every content delete endpoint, so an administrator may delete any member's content while members are limited to their own. Edits stay owner-only. Deletes are **soft** and cascade under one shared stamp - see `devplacepy/database/CLAUDE.md`.
@@ -346,7 +353,7 @@ Every feature in DevPlace is **one data source fanning out into several consumer
 
 1. **HTML** - `respond()` returns a rendered template for browsers.
 2. **JSON** - the SAME `respond(..., model=XOut)` returns JSON when `Accept: application/json`. The `*Out` schema is the gate: a context key not declared on `*Out` is silently dropped from JSON even though the template still sees it.
-3. **Agent tool** - `services/devii/actions/catalog.py` exposes the route as an `Action` so Devii can call it on the user's behalf. `requires_auth` mirrors the route guard.
+3. **Agent tool** - `services/devii/actions/catalog/` (the relevant module inside the package, e.g. `posts.py`, `admin.py`) exposes the route as an `Action` so Devii can call it on the user's behalf. `requires_auth` mirrors the route guard.
 4. **Documented endpoint** - `docs_api.py` `endpoint()` describes it (params, auth, `sample_response`) in the right group, rendered at `/docs/{group}.html`.
 
 A new public read almost always needs all four. The cardinal failure mode is changing one face and forgetting a connected one - a real worked example (followers/following listing) touched nine files across data helpers, output schema, route, view, agent tool, API docs, and the docs trio below for one conceptual feature; that count is the norm, not the exception. If a change touches only one file, confirm none of the four faces were missed. Checklist, ordered by data flow:
@@ -356,7 +363,7 @@ A new public read almost always needs all four. The cardinal failure mode is cha
 3. **Server layer.** Handler with the right auth guard (`get_current_user` public read, `require_user` member POST, `require_admin` admin). Specific paths (`/{username}/followers`) declared before catch-alls (`/{username}`). Return HTML+JSON via `respond(request, template, ctx, model=XOut)` or pure JSON via `JSONResponse`. Register any NEW router in `main.py`.
 4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
 5. **Safety layer (skip only when the feature stores nothing and shows nothing).** If the feature adds a surface that carries user-generated content, register it in `database/moderation.py` `REPORTABLE_TARGETS` (or in `UNREPORTABLE_TABLES` with a reason), resolve it in `resolve_object_url`, and include `_report_button.html` in its action bar - the registry test and the e2e coverage test both fail otherwise. If it collects or transmits a new *category* of personal data, or sends anything new to a third party, update `/docs/privacy.html` and the app-store privacy declarations in the same change; that disclosure is part of the feature, not a follow-up.
-6. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog.py` - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
+6. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog/` (the relevant module inside the package) - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
 7. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
 8. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.
 
@@ -366,6 +373,12 @@ Failures at any implementation step block the workflow - never skip a failed ste
 
 Gitea Actions workflow at `.gitea/workflows/test.yaml` runs on every push/PR to `master`: installs dependencies + Playwright Chromium, runs the full suite serially under coverage, publishes coverage HTML as an artifact, uploads failure screenshots. CI must be green before merging. Changes move through DTAP: Development (`make dev`) -> Test (CI suite + coverage on `master`) -> Acceptance (`master` to `production` promotion via `make deploy`) -> Production (Docker Compose stack). Only CI-green `master` commits are promoted to `production`.
 
+## Version bumping
+
+`pyproject.toml` `version` is bumped automatically, by a real git `pre-commit` hook, not by an agent remembering to edit it. The hook lives at `.githooks/pre-commit` (tracked in the repo, plain stdlib Python) and `make install` points git at it with `git config core.hooksPath .githooks` - run `make install` once per clone (or that one `git config` line by hand) to activate it; a clone that has never run `make install` simply gets no auto-bump, which is a safe, backwards-compatible no-op, never a broken commit.
+
+On every commit the hook increments the patch component (`1.0.0` -> `1.0.1`) and stages the change, so the bump rides in the same commit with no extra step. It defers to a deliberate version edit already staged in that same commit (a hand-set major/minor bump in `pyproject.toml` is left exactly as written, never incremented further) and does nothing during a merge (`.git/MERGE_HEAD` present) or when `pyproject.toml` does not exist. It never blocks a commit - a missing or unparsable version line is a silent no-op, not a failure.
+
 ## Diagnosing a production failure (the order that finds it fastest)
 
 This procedure exists because a single "the editor is down" report turned out to be **three unrelated faults stacked on each other** (a stale URL, a firewalled network leg, and a corrupt database), and the investigation wasted hours by guessing before measuring. Work the layers outward from the browser; each step is cheap and each one eliminates a whole class of cause. **Never skip to a hypothesis, and never repair anything before the layer above it is proven healthy.**

Dockerfile

@@ -3,7 +3,7 @@ FROM python:3.13-slim
 WORKDIR /app
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
-    curl ca-certificates \
+    curl ca-certificates rclone \
     libglib2.0-0 libpango-1.0-0 libpangoft2-1.0-0 libharfbuzz0b libfontconfig1 \
     fonts-dejavu-core \
     && rm -rf /var/lib/apt/lists/*

Makefile

@@ -8,21 +8,42 @@ LOCUST_RUN_TIME ?= 120s
 LOCUST_WEB_WORKERS ?= 4
 WEB_WORKERS ?= $(shell nproc 2>/dev/null || echo 2)
 DEVPLACE_RATE_LIMIT ?= 1000000
+DEVPLACE_PORT ?= 10500
+
+VENV ?= $(CURDIR)/.venv
+PYTHON := $(VENV)/bin/python
+VENV_STAMP := $(VENV)/.installed
+BOOTSTRAP_PYTHON := $(shell command -v python3 2>/dev/null || command -v python 2>/dev/null)
 
 PYTHONDONTWRITEBYTECODE := 1
 export PYTHONDONTWRITEBYTECODE
 
-.PHONY: install dev clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless
+.PHONY: venv install dev prod clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless prune prune-dry-run
 
-install:
-	pip install -e .
-	python -m playwright install chromium
+$(PYTHON):
+	@test -n "$(BOOTSTRAP_PYTHON)" || { echo "python3 is required to create $(VENV)"; exit 1; }
+	$(BOOTSTRAP_PYTHON) -m venv $(VENV)
 
-dev:
-	uvicorn devplacepy.main:app --reload --reload-dir devplacepy --host 0.0.0.0 --port 10500 --backlog 4096
+$(VENV_STAMP): $(PYTHON) pyproject.toml
+	$(PYTHON) -m pip install -U pip
+	$(PYTHON) -m pip install -e ".[dev]"
+	$(PYTHON) -m playwright install chromium
+	touch $(VENV_STAMP)
 
-prod:
-	DEVPLACE_STATIC_VERSION=$$(date +%s) DEVPLACE_TEMPLATE_AUTO_RELOAD=0 DEVPLACE_WEB_WORKERS=$(WEB_WORKERS) uvicorn devplacepy.main:app --host 0.0.0.0 --port 10500 --workers $(WEB_WORKERS) --backlog 8192 --proxy-headers --forwarded-allow-ips '*'
+venv: $(VENV_STAMP)
+
+install: $(PYTHON)
+	$(PYTHON) -m pip install -U pip
+	$(PYTHON) -m pip install -e ".[dev]"
+	$(PYTHON) -m playwright install chromium
+	git config core.hooksPath .githooks
+	touch $(VENV_STAMP)
+
+dev: $(VENV_STAMP)
+	DEVPLACE_PORT=$(DEVPLACE_PORT) $(PYTHON) -m uvicorn devplacepy.main:app --reload --reload-dir devplacepy --host 0.0.0.0 --port $(DEVPLACE_PORT) --backlog 4096
+
+prod: $(VENV_STAMP)
+	DEVPLACE_STATIC_VERSION=$$(date +%s) DEVPLACE_TEMPLATE_AUTO_RELOAD=0 DEVPLACE_WEB_WORKERS=$(WEB_WORKERS) DEVPLACE_PORT=$(DEVPLACE_PORT) $(PYTHON) -m uvicorn devplacepy.main:app --host 0.0.0.0 --port $(DEVPLACE_PORT) --workers $(WEB_WORKERS) --backlog 8192 --proxy-headers --forwarded-allow-ips '*'
 
 delete-pyc:
 	find . -name "__pycache__" -type d -prune -exec rm -rf {} + 2>/dev/null || true
@@ -42,78 +63,78 @@ zip:
 	@git ls-files -z | xargs -0 zip -q $(notdir $(CURDIR)).zip
 	@printf 'Wrote %s (%s files)\n' "$(notdir $(CURDIR)).zip" "$$(git ls-files | wc -l)"
 
-test:
-	PLAYWRIGHT_HEADLESS=1 python -m pytest tests/
+test: $(VENV_STAMP)
+	PLAYWRIGHT_HEADLESS=1 $(PYTHON) -m pytest tests/
 
-test-headed:
-	PLAYWRIGHT_HEADLESS=0 python -m pytest tests/
+test-headed: $(VENV_STAMP)
+	PLAYWRIGHT_HEADLESS=0 $(PYTHON) -m pytest tests/
 
-test-unit:
-	python -m pytest tests/unit
+test-unit: $(VENV_STAMP)
+	$(PYTHON) -m pytest tests/unit
 
-test-api:
-	python -m pytest tests/api
+test-api: $(VENV_STAMP)
+	$(PYTHON) -m pytest tests/api
 
-test-e2e:
-	PLAYWRIGHT_HEADLESS=1 python -m pytest tests/e2e
+test-e2e: $(VENV_STAMP)
+	PLAYWRIGHT_HEADLESS=1 $(PYTHON) -m pytest tests/e2e
 
-test-fast:
-	python -m pytest tests/unit tests/api
+test-fast: $(VENV_STAMP)
+	$(PYTHON) -m pytest tests/unit tests/api
 
-test-failed:
-	PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ --last-failed --last-failed-no-failures none
+test-failed: $(VENV_STAMP)
+	PLAYWRIGHT_HEADLESS=1 $(PYTHON) -m pytest tests/ --last-failed --last-failed-no-failures none
 
-test-first-failure:
-	PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ -x
+test-first-failure: $(VENV_STAMP)
+	PLAYWRIGHT_HEADLESS=1 $(PYTHON) -m pytest tests/ -x
 
-test-slowest:
-	PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ --durations=40
+test-slowest: $(VENV_STAMP)
+	PLAYWRIGHT_HEADLESS=1 $(PYTHON) -m pytest tests/ --durations=40
 
-coverage:
+coverage: $(VENV_STAMP)
 	rm -f .coverage .coverage.*
 	COVERAGE_PROCESS_START=$(CURDIR)/.coveragerc PLAYWRIGHT_HEADLESS=1 \
-		python -m coverage run -m pytest tests/
-	python -m coverage combine
-	python -m coverage report
+		$(PYTHON) -m coverage run -m pytest tests/
+	$(PYTHON) -m coverage combine
+	$(PYTHON) -m coverage report
 
-coverage-headed:
+coverage-headed: $(VENV_STAMP)
 	rm -f .coverage .coverage.*
 	COVERAGE_PROCESS_START=$(CURDIR)/.coveragerc PLAYWRIGHT_HEADLESS=0 \
-		python -m coverage run -m pytest tests/
-	python -m coverage combine
-	python -m coverage report
+		$(PYTHON) -m coverage run -m pytest tests/
+	$(PYTHON) -m coverage combine
+	$(PYTHON) -m coverage report
 
 coverage-html: coverage
-	python -m coverage html
+	$(PYTHON) -m coverage html
 	@echo "Report written to htmlcov/index.html"
 
-locust:
+locust: $(VENV_STAMP)
 	export DEVPLACE_DATABASE_URL="sqlite:///$(LOCUST_DB)"; \
 	export DEVPLACE_RATE_LIMIT=$(DEVPLACE_RATE_LIMIT); \
 	fuser -k $(LOCUST_PORT)/tcp 2>/dev/null || true; \
 	sleep 1; \
 	mkdir -p $(LOCUST_DB_DIR); \
 	rm -f $(LOCUST_DB); \
-	DEVPLACE_TEMPLATE_AUTO_RELOAD=0 DEVPLACE_WEB_WORKERS=$(LOCUST_WEB_WORKERS) uvicorn devplacepy.main:app --host 127.0.0.1 --port $(LOCUST_PORT) --workers $(LOCUST_WEB_WORKERS) --backlog 8192 > /tmp/devplace_locust_server.log 2>&1 & \
+	DEVPLACE_TEMPLATE_AUTO_RELOAD=0 DEVPLACE_WEB_WORKERS=$(LOCUST_WEB_WORKERS) $(PYTHON) -m uvicorn devplacepy.main:app --host 127.0.0.1 --port $(LOCUST_PORT) --workers $(LOCUST_WEB_WORKERS) --backlog 8192 > /tmp/devplace_locust_server.log 2>&1 & \
 	PID=$$!; \
 	while kill -0 $$PID 2>/dev/null && ! curl -s http://127.0.0.1:$(LOCUST_PORT)/ > /dev/null 2>&1; do sleep 0.5; done; \
 	if ! kill -0 $$PID 2>/dev/null; then echo "Server failed to start (port $(LOCUST_PORT) busy?). See /tmp/devplace_locust_server.log"; exit 1; fi; \
-	locust --host http://127.0.0.1:$(LOCUST_PORT) --web-port $(LOCUST_WEB_PORT); \
+	$(PYTHON) -m locust --host http://127.0.0.1:$(LOCUST_PORT) --web-port $(LOCUST_WEB_PORT); \
 	kill $$PID 2>/dev/null || true; \
 	rm -rf $(LOCUST_DB_DIR)
 
-locust-headless:
+locust-headless: $(VENV_STAMP)
 	export DEVPLACE_DATABASE_URL="sqlite:///$(LOCUST_DB)"; \
 	export DEVPLACE_RATE_LIMIT=$(DEVPLACE_RATE_LIMIT); \
 	fuser -k $(LOCUST_PORT)/tcp 2>/dev/null || true; \
 	sleep 1; \
 	mkdir -p $(LOCUST_DB_DIR); \
 	rm -f $(LOCUST_DB); \
-	DEVPLACE_TEMPLATE_AUTO_RELOAD=0 DEVPLACE_WEB_WORKERS=$(LOCUST_WEB_WORKERS) uvicorn devplacepy.main:app --host 127.0.0.1 --port $(LOCUST_PORT) --workers $(LOCUST_WEB_WORKERS) --backlog 8192 > /tmp/devplace_locust_server.log 2>&1 & \
+	DEVPLACE_TEMPLATE_AUTO_RELOAD=0 DEVPLACE_WEB_WORKERS=$(LOCUST_WEB_WORKERS) $(PYTHON) -m uvicorn devplacepy.main:app --host 127.0.0.1 --port $(LOCUST_PORT) --workers $(LOCUST_WEB_WORKERS) --backlog 8192 > /tmp/devplace_locust_server.log 2>&1 & \
 	PID=$$!; \
 	while kill -0 $$PID 2>/dev/null && ! curl -s http://127.0.0.1:$(LOCUST_PORT)/ > /dev/null 2>&1; do sleep 0.5; done; \
 	if ! kill -0 $$PID 2>/dev/null; then echo "Server failed to start (port $(LOCUST_PORT) busy?). See /tmp/devplace_locust_server.log"; exit 1; fi; \
-	locust --host http://127.0.0.1:$(LOCUST_PORT) --headless -u $(LOCUST_USERS) -r $(LOCUST_SPAWN_RATE) --run-time $(LOCUST_RUN_TIME) --html=$(LOCUST_DB_DIR)/report.html; \
+	$(PYTHON) -m locust --host http://127.0.0.1:$(LOCUST_PORT) --headless -u $(LOCUST_USERS) -r $(LOCUST_SPAWN_RATE) --run-time $(LOCUST_RUN_TIME) --html=$(LOCUST_DB_DIR)/report.html; \
 	kill $$PID 2>/dev/null || true; \
 	rm -rf $(LOCUST_DB_DIR)
 
@@ -127,6 +148,15 @@ clean:
 test-cache-clean:
 	rm -rf .pytest_cache
 
+# Safe but aggressive disk-space cleanup, acting on the REAL data/devplace.db
+# and data/ tree (never the test database) - see CLAUDE.md "devplace system
+# prune". prune-dry-run reports what would be removed without deleting.
+prune: $(VENV_STAMP)
+	$(PYTHON) -m devplacepy.cli system prune
+
+prune-dry-run: $(VENV_STAMP)
+	$(PYTHON) -m devplacepy.cli system prune --dry-run
+
 # Container Manager works out of the box: the overlay installs the docker CLI in
 # the image and mounts the host socket. DOCKER_GID is read straight from the
 # socket so the UID-1000 app can use it; the data dir is the project's own data/

README.md

@@ -7,7 +7,7 @@ Server-rendered social network for developers. FastAPI backend serving Jinja2 te
 ## Quick start
 
 ```bash
-make install        # pip install -e .
+make install        # create .venv if needed, pip install -e ".[dev]", playwright chromium
 make dev            # uvicorn --reload on port 10500
 make test           # Playwright integration + unit tests, headless, fail-fast
 make test-headed    # same tests in visible browser
@@ -63,6 +63,7 @@ devplacepy/
 | `/` | Home page: marketing splash for guests, personalized home (welcome, feed shortcut, latest posts, news) for signed-in users. Does not redirect. Latest-posts section interleaves authors so no two consecutive posts share an author. |
 | `/auth` | Signup, login, logout, forgot/reset password |
 | `/feed` | Post feed with topic/tab filtering and free-text `search` (title, content, and author username) in the left panel (public). Each page interleaves authors so no two consecutive posts share an author. |
+| `/topics` | Crawlable per-topic category pages (public): `/topics` hub links every topic with a live post count, `/topics/{topic}` lists that topic's posts with its own canonical URL, title, and breadcrumbs |
 | `/news` | Developer news listing, detail page with comments |
 | `/posts` | Post detail, creation |
 | `/gists` | Code gist listing, detail, creation, and editing; left panel offers language filtering and free-text `search` (title, description, and author username), public read |
@@ -82,7 +83,7 @@ devplacepy/
 | `/admin/trash` | Admin **Trash**: review, restore, and permanently purge soft-deleted content (posts, comments, gists, projects, news, project files, attachments) across the platform |
 | `/admin/devii-tasks` | Admin **Devii tasks**: every scheduled task across all owners with its schedule, run count, expiry and failure streak, plus per-task disable and delete |
 | `/notifications` | Notification list, mark read, live unread counts (`/notifications/counts`) |
-| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages, so typing an inline `@ai <instruction>` in a message executes it and the resolved result appears live in the chat for both participants (the `message` WS frame carries an additive `ai_processed` flag when a pending correction/modification was applied). An opened conversation loads its 500 most recent messages; older history is retained in the database. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets |
+| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, reconnect catch-up, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages: the original is delivered immediately, then a second `message` frame with `ai_processed` replaces the bubble for both participants when the rewrite lands (including across workers). An opened conversation loads its 500 most recent messages and can page older rows with `?before=`; switching threads does not drop the socket. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets |
 | `/votes` | Upvote/downvote on posts, comments, projects |
 | `/reactions` | Emoji reactions on posts, comments, gists, projects |
 | `/bookmarks` | Save/unsave content; `/bookmarks/saved` personal list |
@@ -97,6 +98,7 @@ devplacepy/
 | `/leaderboard` | Contributor ranking by total stars earned |
 | `/game` | **Code Farm** cooperative idle game (member-only): plant projects that build over real time, harvest coins and XP, upgrade CI, buy plots, and water friends' builds at `/game/farm/{username}`. Live over pub/sub; every endpoint negotiates JSON |
 | `/quizzes` | **Quizzes**: author quizzes, play them, and climb the cross-quiz scoreboard. Three-column hub with filters (`all`/`todo`/`done`/`mine`/`drafts`), search, per-viewer state badges, and the scoreboard rail; `/quizzes/{slug}` detail, `/quizzes/{slug}/edit` builder, `/quizzes/{slug}/attempts/{uid}` player, `/quizzes/scoreboard` JSON. Publishing is permanent. Every endpoint negotiates JSON |
+| `/battles` | **Opinion Wars**: week-long two-faction battles attached to posts, started from the composer's *Start Opinion War* builder. Members join a side and fight once a day (25 Code Farm coins, level-weighted damage); the pixel-art battle card shows live HP bars, a countdown, top contributors and an event ticker. `/battles` lists battles (`active`/`ended`/`mine` + search); `/battles/{uid}` state, `/battles/{uid}/events` replay, `/battles/{uid}/join` and `/battles/{uid}/fight` actions |
 | `/avatar` | Multiavatar proxy with in-memory cache |
 | `/issues` | Issue tracker backed by Gitea: list/detail/comment/status, AI-enhanced filing, an admin planning report over a selectable set of open tickets (each ticket's full text reproduced verbatim so the document hands straight to a coding agent), and file attachments on open issues and comments (mirrored to the Gitea tracker) |
 | `/admin/services` | Background service management (start/stop, config, status, logs) |
@@ -169,6 +171,25 @@ scoreboard on the right. Guests read published quizzes and see the board; they c
 Retention: completed attempts are permanent; abandoned and expired ones are garbage-collected by
 `devplace quiz prune`.
 
+## Opinion Wars
+
+**Opinion Wars** (`/battles`) are week-long two-faction battles attached to posts, in the spirit of
+old eRepublik battles: settle tabs-versus-spaces by showing up daily and fighting for your side.
+
+- **Start one from the composer.** The *Start Opinion War* button next to *Add poll* names the two
+  factions; the battle runs for exactly 7 days from the moment the post is published.
+- **Join and fight.** Any signed-in member picks a side and may fight once every 24 hours per
+  battle. A fight costs 25 Code Farm coins and deals deterministic, level-weighted damage
+  (100 HP + 10 per site level, capped at level 20) - no randomness, dedication wins wars.
+- **Defection is allowed.** Switch factions any time; damage already dealt stays where it landed.
+- **Live pixel-art card.** The battle renders on the post as a CSS pixel-art battlefield with HP
+  bars, a countdown, your rank, top contributors and a live event ticker (joins, defections,
+  fights, lead changes) over pub/sub with an incremental replay fallback.
+- **Rewards.** When the week ends the bigger total wins: every fighter earns XP, the winning side
+  and the top damage dealer earn bonuses, and battle badges (*Instigator*, *First Blood*,
+  *War Veteran*, *Champion*) mark the milestones. Notifications cover lead changes, the result and
+  your next fight being ready.
+
 ## Code Farm
 
 The **Code Farm** (`/game`) is a cooperative idle game in the spirit of Farmville, themed for developers. Each member owns a farm of plots and plays asynchronously - nothing has to happen in real time.
@@ -202,6 +223,7 @@ The farm refreshes live over the pub/sub bus (a watered build appears on the own
 - **Emoji reactions** - react with **any** emoji on posts, comments, gists, and projects, separate from voting and carrying no ranking weight. A short quick-pick palette covers the common reactions, and a `+` button next to it opens the full searchable emoji picker (every standard emoji, including skin tones), so a reaction is never limited to a preset list. Emoji already used on an item are shown as counted chips beside the palette.
 - **Emoji shortcodes** - typing a `:name:` shortcode in any content (posts, comments, titles, project and gist descriptions, news, and direct messages) renders the matching emoji, using the full GitHub/Discord standard set (for example `:rocket:` becomes a rocket). Server-rendered and live content share one shortcode list; unknown names and shortcodes inside code are left untouched. Documented at `/docs/emoji-shortcodes`. This is distinct from the visual emoji-picker button in the composer, which inserts the literal emoji character.
 - **Polls** - a post can carry a poll (question plus up to six options); results appear as live bars once the viewer votes, one vote per member. A poll can be attached when the post is created or added later by editing a post that has none.
+- **Paste an image to attach it** - pressing Ctrl+V (Cmd+V) with a screenshot or copied image on the clipboard while writing a post, a comment, a direct message, an issue, a gist, or a project attaches it immediately, with no trip through the file picker. The upload, its limits, and the resulting attachment are identical to picking the file by hand.
 - **Bookmarks** - save posts, gists, projects, and news to a personal list at `/bookmarks/saved`.
 - **Private projects** - an owner can mark a project private so it is visible only to them (and administrators) and excluded from listings, profiles, search, the sitemap, and zip access. Set at creation or toggled later from the project page.
 - **Read-only projects** - an owner can mark a project read-only, making its entire virtual filesystem immutable: every write, edit, line-edit, move, delete, and upload is refused from all paths (the web UI, the HTTP API, the Devii agent, and container workspace sync) until read-only is turned off. Devii may toggle read-only only after the user explicitly confirms.
@@ -238,12 +260,14 @@ The log is **administrator-only**. `/admin/audit-log` is a paginated, filterable
 |---------|---------|---------|
 | `DEVPLACE_DATABASE_URL` | `sqlite:///<repo>/data/devplace.db` | Database connection string |
 | `DEVPLACE_DATA_DIR` | `<repo>/data` | Single root for every runtime/user-generated artifact (DB, uploads, VAPID keys, locks, bot state, job staging, container workspaces), outside the package and not served via `/static`. Point at a volume in production. Defined once in `config.py` (`DATA_PATHS` registry, created by `ensure_data_dirs()`) |
+| `DEVPLACE_PORT` | `10500` | Port the app itself binds to (`make dev`/`make prod`, and what `Makefile`'s `dev`/`prod` targets pass to uvicorn's `--port`). Also feeds `config.PORT`, so `DEVPLACE_INTERNAL_BASE_URL`'s default and Devii's own self-dial URL follow it automatically. Bare metal only - the Docker app container always binds its fixed internal port regardless of this var; for Docker, use `PORT` (below) to change the externally reachable port |
+| `PORT` | `10500` | Docker only: the host port `docker-compose.yml` publishes nginx on (`127.0.0.1:${PORT}:80`). Unrelated to `DEVPLACE_PORT` above - it never reaches the app container |
 | `SECRET_KEY` | hardcoded fallback | Session signing key |
 | `DEVPLACE_VAPID_SUB` | `mailto:retoor@molodetz.nl` | Contact address in the VAPID JWT `sub` claim |
 | `DEVPLACE_INTERNAL_BASE_URL` | `http://localhost:10500` | Base URL the platform's own services dial for the AI gateway |
 | `DEVPLACE_XMLRPC_PORT` | `10550` | Loopback port the forking XML-RPC bridge binds; the app and nginx reverse-proxy `/xmlrpc` to it |
 | `DEVPLACE_XMLRPC_BIND` | `127.0.0.1` | Bind address for the XML-RPC bridge (loopback; the app and nginx are the intended front doors) |
-| `DEVPLACE_STATIC_VERSION` | server boot unix timestamp | Cache-busting version stamped into every static asset URL (`/static/v<version>/...`). Set it at launch so multiple workers agree (the `prod` target and Docker image do this); leave unset in dev to refresh on each reload. See [Static asset caching](#static-asset-caching) |
+| `DEVPLACE_STATIC_VERSION` | server boot unix timestamp | The boot-id half of the cache-busting version stamped into every static asset URL (`/static/v<app-version>-<boot-id>/...`, e.g. `/static/v1.0.1-1718040000/...`). Set it at launch so multiple workers agree (the `prod` target and Docker image do this); leave unset in dev to refresh on each reload. See [Static asset caching](#static-asset-caching) |
 | `DEEPSEEK_API_KEY` / `OPENROUTER_API_KEY` | unset | Upstream provider keys; migrated into the gateway settings on first boot |
 | `DEVPLACE_PRESENCE_TIMEOUT_SECONDS` | `60` | Online-presence window: a user counts as online for this many seconds after their last activity. `last_seen` is refreshed by a throttled in-place update at most once per half this interval per worker (no per-load inserts, no data growth) |
 | `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Maximum avatars shown in the feed's live "Online now" panel (ordered alphabetically by username) |
@@ -264,6 +288,7 @@ Operational behavior is tunable live from `/admin/settings` (stored in `site_set
 | `registration_open` | `1` | When `0`, new sign-ups are rejected |
 | `maintenance_mode` | `0` | When `1`, non-admins see the maintenance page; admins retain access |
 | `maintenance_message` | scheduled-maintenance text | Message shown during maintenance |
+| `happy_404_enabled` | `1` | When `1`, an HTML page that would otherwise 404 renders a random existing post instead ("Happy 404"); JSON/API requests always get a real 404 regardless. See [Happy 404](#happy-404) |
 | `customization_enabled` | `1` | When `0`, no user CSS/JS customization is injected on any page |
 | `customization_js_enabled` | `1` | When `0`, user custom CSS is still served but custom JavaScript is suppressed |
 | `audit_log_retention_days` | `90` | Audit rows older than this are pruned daily by the Audit retention service; `0` disables pruning |
@@ -318,6 +343,27 @@ curl -H "Accept: application/json" https://your-host/feed
 curl -H "Accept: application/json" -X POST -d "content=hi&title=T&topic=devlog" https://your-host/posts/create
 ```
 
+## Happy 404
+
+Instead of a bare error page, an HTML request that would 404 (an unmatched route, or an app route
+raising `not_found()` for a missing resource) instead renders a random existing post at that URL,
+using the exact same template and context as the real `/posts/{slug}` page. JSON/API requests are
+unaffected and still get a normal `404` - the substitution only ever applies to a browser HTML
+navigation.
+
+- **Toggle:** `happy_404_enabled` site setting (`/admin/settings`, on by default).
+- **Scope:** any HTML `GET` 404, app-wide - not just under `/posts`.
+- **Performance:** a small pool of random post slugs is cached in-process for a few minutes and
+  refreshed with a fresh random sample on expiry, so every request only does an in-memory pick plus
+  one indexed lookup - no per-request full-table scan - while the pool composition still cycles
+  through the whole `posts` table over time.
+- **SEO safety:** the substituted page is always marked `noindex,nofollow` so the decoy URL is never
+  indexed under the wrong address.
+- **Fail-closed:** any error while building the substitute page falls straight through to the normal
+  404 page - this feature can never turn a real error into a worse one.
+
+Implementation: `devplacepy/happy404.py`, wired into the `404` exception handler in `main.py`.
+
 ## XML-RPC bridge
 
 The full REST API is also reachable over XML-RPC at `/xmlrpc`. A standalone forking XML-RPC
@@ -431,16 +477,16 @@ and its full configuration are documented automatically - including future servi
 - **`ConfigField`** - declarative parameter spec (type, default, validation, secret) a service uses to declare its editable settings
 - **`BaseService`** - abstract class with a reconciling run loop that honors the persisted `enabled`/command/interval state, plus `config_fields`, `get_config()`, `describe()`, and a log buffer
 - **`ServiceManager`** - singleton: `register`, `describe_all`, `set_enabled`, `send_command`, `save_config`, `supervise`, `shutdown_all`
-- **`NewsService`** - a fully automatic, zero-maintenance news pipeline: fetches news from `news.app.molodetz.nl/api`, cleans each article, fetches and perceptually compares the images to reject placeholders and detect uniqueness, grades each one deterministically, reformats every valid article into clean Markdown (paragraphs, headings, lists) with the AI so the source wall of text reads as a proper article, and auto-rotates the best articles to Featured and the landing page. Its AI spend is metered from the gateway response headers and reported on the admin Services page (calls, tokens, total cost, and per-call averages)
+- **`NewsService`** - a fully automatic, zero-maintenance news pipeline: fetches news from `news.app.molodetz.nl/api`, cleans each article, fetches and perceptually compares the images to reject placeholders and detect uniqueness, grades each one against a free, local, deterministic quality model (aquality - no billed LLM call, no API key), optionally reformats articles that clear the publish grade threshold into clean Markdown with a generative AI model (off by default, since the default grading model cannot generate text), and auto-rotates the best articles to Featured and the landing page. Any AI spend it does incur is metered from the gateway response headers and reported on the admin Services page (calls, tokens, total cost, and per-call averages)
 - **`BotsService`** - Playwright fleet of AI personas that browse and interact with a DevPlace instance, with live cost/usage metrics and a live screenshot monitor at `/admin/bots` (opt-in; install the `bots` extra)
-- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected
+- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Thinking is disabled by default** on every chat and vision call (`thinking.type=disabled` on DeepSeek, `reasoning.effort=none` on OpenRouter, `think=false` on Ollama) so the fast path is the default; a client may re-enable it per request, and an administrator may flip `gateway_thinking` on. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected. Each route may also name a **fallback model** - another already-configured model of the same kind, picked from a select box of the public model names (never a free-text provider model id) - that the gateway retries once, automatically, whenever the primary model fails after its own retries are exhausted, so a struggling model degrades to a working one instead of failing the caller
 - **`JobService` / `ZipService` / `ForkService`** - generic async job framework (`services/jobs/`) for heavy, blocking work run off the request path; `ZipService` builds project zip archives in a subprocess, `ForkService` copies a project into a new project owned by the forking user
 - **`ContainerService`** - the admin container manager (`services/containers/`): a reconciling supervisor for container instances, all running one shared prebuilt image
 - **`AcceptanceService`** - grants every policy agreement (Terms of Service, Privacy Policy, third-party AI processing, activity recording, container credentials) to every account that has not declined it, so an instance kept production-identical for extended manual testing never interrupts with an acceptance dialog. Administrator-only, **off by default**, with a separate switch per agreement type and a dry run that reports what it would do without writing. An account that withdrew a consent is never granted it again, with no further action: the consent ledger itself is the decline register. It is not appropriate on a real production host
 
 ### Container manager (admin only)
 
-`services/containers/` runs supervised container instances from the web UI, the HTTP API, and Devii. It drives the `docker` CLI via async subprocesses behind a pluggable `Backend` interface (a `DockerCliBackend` plus a `FakeBackend` for tests). **There is no in-app image building.** Every instance runs ONE shared prebuilt image, `ppy:latest` (override `DEVPLACE_CONTAINER_IMAGE`), built once with **`make ppy`** from `ppy.Dockerfile`: a Python + Playwright base with a broad set of common libraries preinstalled, the `pravda` (uid 1000) user, the sudo superclone, `pagent` at `/usr/bin/pagent.py`, and the `code-server` browser IDE that backs dev workspaces all baked in. Creating an instance is then an instant `docker run` (it fails fast with a clear error if the `ppy` image has not been built yet). `ContainerService` reconciles desired instance state against `docker ps` each tick (containers are labeled `devplace.instance=<uid>`, so orphans are reaped and no state is lost), applies restart policies, fires cron/interval/one-time schedules, and samples metrics. The container's `/app` is bind-mounted to a persistent project workspace (materialized from the project files) and **stays in sync automatically**: the manager runs a bidirectional, newer-wins sync between the project files and the workspace on every start and roughly once a minute while running, so edits made inside the container and edits made in the project file editor converge without manual intervention (the sync only ever creates or overwrites the older copy of a file, never deletes one; a read-only project is export-only). Projects that need extra packages use runtime `pip install` (pravda owns the site-packages, no sudo needed) or `apt install` directly (the `pravda` user runs `apt`/`dpkg` through a fakeroot wrapper, so system packages install without root) or add the library to `ppy.Dockerfile` and rerun `make ppy`. Each instance can run a **boot script** in Python or Bash (written into the workspace and run on launch) or a plain boot command, can be set to **start automatically** whenever the container service starts, and can be configured to **run as** a chosen DevPlace user - which only selects whose identity and API key are injected into the container (the container always runs as the unprivileged `pravda` user). Every status change is recorded and shown as a status history on the instance page. A running instance can be **published** with an `ingress_slug`, making its service reachable (HTTP and WebSocket) at `/p/<slug>` through DevPlace. The manager is reached two ways: the admin **Containers** sidebar entry (`/admin/containers`) lists, creates, edits, and controls every instance across all projects and opens a dedicated detail page per instance, and each project page carries an admin-only **Containers** button to its own instance manager.
+`services/containers/` runs supervised container instances from the web UI, the HTTP API, and Devii. It drives the `docker` CLI via async subprocesses behind a pluggable `Backend` interface (a `DockerCliBackend` plus a `FakeBackend` for tests). **There is no in-app image building.** Every instance runs ONE shared prebuilt image, `ppy:latest` (override `DEVPLACE_CONTAINER_IMAGE`), built once with **`make ppy`** from `ppy.Dockerfile`: a Python + Playwright base with a broad set of common libraries preinstalled, the `pravda` (uid 1000) user, the sudo superclone, `pagent` at `/usr/bin/pagent.py`, and the `code-server` browser IDE that backs dev workspaces all baked in. Creating an instance is then an instant `docker run` (it fails fast with a clear error if the `ppy` image has not been built yet). `ContainerService` reconciles desired instance state against `docker ps` each tick (containers are labeled `devplace.instance=<uid>`, so orphans are reaped and no state is lost), applies restart policies, fires cron/interval/one-time schedules, and samples metrics. The container's `/app` is bind-mounted to a persistent project workspace (materialized from the project files) and **stays in sync automatically**: the manager runs a bidirectional sync between the project files and the workspace on every start and roughly once a minute while running, so edits made inside the container and edits made in the project file editor converge without manual intervention. Deletions propagate too, in both directions: deleting a file inside the container removes it from the project, and deleting it in the project's file editor removes it from the container, tracked against a per-file sync baseline so a genuine deletion is never confused with a file that simply has not been materialized to that workspace yet; an edit made after a conflicting deletion always wins and restores the file (a read-only project is export-only and always mirrors the project verbatim, including removing files the project no longer has). Projects that need extra packages use runtime `pip install` (pravda owns the site-packages, no sudo needed) or `apt install` directly (the `pravda` user runs `apt`/`dpkg` through a fakeroot wrapper, so system packages install without root) or add the library to `ppy.Dockerfile` and rerun `make ppy`. Each instance can run a **boot script** in Python or Bash (written into the workspace and run on launch) or a plain boot command, can be set to **start automatically** whenever the container service starts, and can be configured to **run as** a chosen DevPlace user - which only selects whose identity and API key are injected into the container (the container always runs as the unprivileged `pravda` user). Every status change is recorded and shown as a status history on the instance page. A running instance can be **published** with an `ingress_slug`, making its service reachable (HTTP and WebSocket) at `/p/<slug>` through DevPlace. The manager is reached two ways: the admin **Containers** sidebar entry (`/admin/containers`) lists, creates, edits, and controls every instance across all projects and opens a dedicated detail page per instance, and each project page carries an admin-only **Containers** button to its own instance manager.
 
 **Security:** this requires mounting the Docker socket, which grants host root. Every run, exec, lifecycle, and schedule operation is administrator-only; `--privileged` is never used and all docker calls are argument-list subprocesses. Containers are additionally isolated per user: an instance is managed only by its owner (the administrator who created it, or the owner of its project) and by the primary administrator, who alone sees and manages every instance including those on private projects; other administrators get a read-only view of instances on public projects and none of another user's private-project instances (exec, terminals, schedules, edits, and lifecycle actions are all refused and audited as denied). The service is disabled by default; an admin enables **Containers** on `/admin/services`. CLI: `devplace containers list | reconcile | prune | prune-builds | gc-workspaces` (`prune-builds` is a one-time cleanup that removes legacy per-project images and the old dockerfiles/builds tables).
 
@@ -452,7 +498,17 @@ A **workspace** is a member-facing container running the DevPlace browser editor
 container runtime above. It is opened from a project's **Workspace** page and reached at
 `/projects/{slug}/workspace`; the editor itself is proxied at
 `/projects/{slug}/containers/instances/{uid}/code/`, and an **Editor** button appears on the project
-page whenever the workspace is running.
+page whenever the editor is actually reachable.
+
+**The workspace page reports the editor's real state, live.** A workspace has a *phase* derived on
+the server from its desired state, its container status and a TCP probe of the editor port:
+`stopped`, `starting`, `ready`, `stopping`, `crashed` or `suspended`. The page renders the control
+that matches the phase, so pressing **Start** turns the button into a spinner reading *Starting the
+editor* and the **Open editor** link appears only once code-server answers, never while the
+container is still booting. While a workspace is in transition the page polls every two seconds
+(twenty seconds otherwise) and also receives pushed updates on the owner's private pub/sub topic, so
+the label changes on its own without a reload; the project page's **Editor** button follows the same
+readiness rule. The phase, its label and `editor_ready` are part of the workspace JSON.
 
 The editor is `code-server`, rebranded as DevPlace end to end: the application name, the browser tab
 icon and PWA icons, the login page styling, and `product.json` all carry DevPlace, and a bundled
@@ -465,10 +521,21 @@ coding agent baked into the image, and a plain login shell beside it with the Py
 Swift toolchains on `PATH`. Both are configurable, and `bash` stays the default profile for
 terminals the member opens later.
 
-The workspace opens straight onto the member's files rather than a welcome page, and the editor's
-own built-in chat assistant is suppressed so `dpc` is the only agent on offer and every token it
-spends is ledgered against the member's DevPlace account. `dpc`'s own working files (`.dpc/`,
-`dpc.log`) are in `SYNC_SKIP_NAMES`, so running an agent on every boot never pollutes the project.
+**The first boot of a workspace opens the DevPlace welcome page** beside the terminals: a webview
+introducing the workspace and DevPlace Code (its 900k token context, vision, parallel sub-agents,
+deep research, safety gates and the daily credits it runs on), with example prompts and buttons that
+focus the agent terminal, start the walkthrough, show the public tunnels and open the workspace
+settings. It is shown once per workspace and can be reopened with **DevPlace: Show the welcome
+page**. The editor's own built-in chat assistant and VS Code's own welcome page stay suppressed so
+`dpc` is the only agent on offer and every token it spends is ledgered against the member's DevPlace
+account. `dpc`'s own working files (`.dpc/`, `dpc.log`) are in `SYNC_SKIP_NAMES`, so running an
+agent on every boot never pollutes the project.
+
+**The terminal panel gets about a third of the window by default.** Every preset is a fixed number
+of steps up from the panel's minimum height (`normal`, the default, lands at roughly a third of a
+typical window; `short` at a fifth, `tall` at about half) and `maximized` fills the editor area.
+A preset is applied on the first boot of a workspace and again whenever it changes; a height the
+member drags themselves is kept across restarts.
 
 **Every workspace is trusted.** VS Code Restricted Mode is disabled at the command line and in the
 seeded settings, so nothing prompts and automatic tasks run. This is a deliberate default with a
@@ -503,7 +570,7 @@ restart.
 
 `SeoMetaService` is a separate AI subservice that generates a clean, search-optimised title, description and short keyword list for every published post, project, gist, news article and issue, entirely off the request path so it never slows the web server. The work is queued whenever content is created, edited or published; until the AI value is ready a plain-content default (built from the markdown-stripped text and clamped to safe lengths) fills the fields, so a page's metadata is **always populated, never empty**. The service uses the built-in internal AI gateway and meters its own AI cost and statistics in a dedicated usage table, surfaced together with its live task pipeline on the **Admin -> Services** page. This release also fixes the on-page metadata: the `<meta name="description">` is now stripped of markdown markup (it previously leaked `#`, `**` and `[](...)` from the raw body), a `<meta name="keywords">` tag is emitted (a short honest list, not stuffed), and social-card image dimensions and alt text are added. CLI: `devplace seo-meta prune` / `devplace seo-meta clear` (job rows only; the generated metadata persists).
 
-`DeepsearchService` powers the public **Tools -> DeepSearch** researcher. Given a single research question it plans a set of diverse web search queries, interleaves their results so every angle contributes sources, and crawls the most relevant pages in a subprocess (concurrent fetches, plain HTTP first with a headless-browser fallback for JavaScript-heavy pages, PDF documents streamed and text-extracted, every URL SSRF-guarded). Social sites that block bots (X, YouTube, Reddit and similar) contribute the readable text supplied by the search engine, so their content is not lost to a login wall. A readability-grade extractor isolates the main article content of each page (navigation, cookie banners and footers are discarded), and the configured depth follows the most relevant in-page links for deeper coverage. Content is de-duplicated and indexed into a per-session ChromaDB vector collection (embeddings via the AI gateway with a local fallback). A chain of agents (report writer, findings extractor, linker) then synthesises a thorough cited markdown report grounded on the passages retrieved from that index, with key findings, a confidence score and source diversity; if synthesis fails the report is clearly marked as degraded instead of silently shipping raw source material. Progress streams live over `WS /tools/deepsearch/{uid}/ws`; the report is at `/tools/deepsearch/{uid}/session` (HTML or JSON) and can be exported as Markdown, JSON or PDF. A grounded chat over the session's evidence runs at `WS /tools/deepsearch/{uid}/chat` using hybrid retrieval (vector + keyword/BM25). Runs can be paused, resumed or cancelled. A cross-session URL cache avoids re-fetching pages seen by earlier runs. CLI: `devplace deepsearch prune` / `devplace deepsearch clear`. ChromaDB, weasyprint, pypdf and Playwright are core dependencies.
+`DeepsearchService` powers the public **Tools -> DeepSearch** researcher, an iterative agent modelled on the way Perplexity and OpenAI's own deep-research agents work: search, read, decide what is still missing, search again. Given a single research question it plans a set of diverse web search queries, interleaves their results so every angle contributes sources, and crawls the most relevant pages in a subprocess (concurrent fetches, plain HTTP first with a headless-browser fallback for JavaScript-heavy pages, PDF documents streamed and text-extracted, every URL SSRF-guarded). After the first pass it runs up to two automatic gap-filling refinement rounds - a lightweight planner looks at what has been gathered so far and proposes a few more targeted queries only if real coverage gaps remain, bounded by the page budget, and stops the moment nothing more is needed. Social sites that block bots (X, YouTube, Reddit and similar) contribute the readable text supplied by the search engine, so their content is not lost to a login wall. A readability-grade extractor isolates the main article content of each page (navigation, cookie banners and footers are discarded), and the configured depth follows the most relevant in-page links for deeper coverage. Content is de-duplicated and indexed into a per-session ChromaDB vector collection (embeddings via the AI gateway with a local fallback); retrieval reranks candidate passages with Maximal Marginal Relevance so the report is grounded on genuinely diverse evidence rather than several near-duplicate hits on the same page. A chain of agents (report writer, findings extractor, linker) then synthesises a thorough cited markdown report grounded on the passages retrieved from that index, with key findings, a confidence score, source diversity and a short list of suggested follow-up questions you can ask straight into the chat; if synthesis fails the report is clearly marked as degraded instead of silently shipping raw source material. Progress streams live over `WS /tools/deepsearch/{uid}/ws`; the report is at `/tools/deepsearch/{uid}/session` (HTML or JSON) and can be exported as Markdown, JSON or PDF. A grounded chat over the session's evidence runs at `WS /tools/deepsearch/{uid}/chat` using hybrid retrieval (vector + keyword/BM25). Runs can be paused, resumed or cancelled. A cross-session URL cache avoids re-fetching pages seen by earlier runs. CLI: `devplace deepsearch prune` / `devplace deepsearch clear`. ChromaDB, weasyprint, pypdf and Playwright are core dependencies.
 
 `IsslopService` powers the public **Tools -> AI Usage Analyzer**, which classifies a git repository or website as AI slop, sophisticated AI-assisted work or genuine human work. Sources are acquired in an isolated subprocess (git URLs are probed with `git ls-remote` and shallow-cloned with a 3 GB guard; websites render in a stealth headless browser with an HTTP fallback, bounded by depth, file and byte caps), inventoried with exclusion rules, and scored by a multi-signal static engine (twenty-one detector families, 126 signal types across an origin axis and a quality-deficit axis). For a live website, the analyzer also opens the home page in a headless browser and inspects what actually renders: it fingerprints AI website-builder platforms directly (Lovable, Bolt.new, Framer and others) and checks the page's real computed styles, layout and build artifacts, not just its file contents. Representative files receive an AI review pass and images an AI-generation review through the internal gateway (model `molodetz`, internal key); the static engine stays authoritative when the gateway is unreachable. Every pipeline step is persisted as an ordered event trail and streamed live over the pub/sub topic `public.isslop.{uid}`. The verdict is an A-F authenticity grade with a human/AI split and one of five categories (`ai-slop`, `sophisticated-ai`, `human-clean`, `human-messy`, `uncertain`), published as a persistent report with an embeddable SVG badge. Members keep their analysis history on their account; guest history is session-bound and claimed by the account on the first signed-in visit. Admin settings (private-host allowance, AI/image review toggles, image cap, retention, concurrency) live on `/admin/services`. CLI: `devplace isslop analyze <url>` / `devplace isslop prune` / `devplace isslop clear`. Playwright plus playwright-stealth back the website crawler.
 
@@ -536,9 +603,10 @@ Configuration on the Services tab (`/admin/services`):
 |-----------|---------|---------|
 | `news_grade_threshold` | `7` | Minimum AI grade for auto-publish |
 | `news_api_url` | `https://news.app.molodetz.nl/api` | News source |
-| `news_ai_url` | `http://localhost:10500/openai/v1/chat/completions` | AI grading endpoint (the internal gateway) |
-| `news_ai_model` | `molodetz` | Generic model name; the gateway maps it to the real model |
-| `news_ai_key` | internal key | AI API key (`NEWS_AI_KEY` env, then the auto-generated gateway internal key) |
+| `news_ai_url` | `https://aquality.cloud.pravda.education/v1/chat/completions` | AI grading endpoint - a free, local, deterministic quality model by default |
+| `news_ai_model` | `aquality` | Generic model name; echoed back, not used for routing |
+| `news_ai_key` | internal key | AI API key (`NEWS_AI_KEY` env, then the auto-generated gateway internal key); unused by the default aquality endpoint |
+| `news_format_enabled` | `false` | Reformat article bodies into Markdown with AI after grading; off by default since aquality only scores and cannot generate text - enable when `news_ai_url` points at a generative chat model |
 | `news_service_interval` | `3600` | Seconds between fetch cycles (min 60) |
 
 News articles have detail pages at `/news/{slug}` with full comment support (same component as posts/projects). The landing page can display curated articles toggled from admin.
@@ -635,6 +703,7 @@ Configuration on the Services tab:
 | `gateway_upstream_url` | `https://api.deepseek.com/chat/completions` | Where requests are forwarded |
 | `gateway_model` | `deepseek-v4-flash` | Real model sent upstream (what `molodetz` maps to); 1M-token context, 384K max output |
 | `gateway_force_model` | on | Override the client-requested model (and the `molodetz` alias) |
+| `gateway_thinking` | off | When off, chat completions disable model thinking unless the client explicitly enables it (`think` / `thinking` / `reasoning`). Fastest default. |
 | `gateway_api_key` | migrated from env | Upstream key, shown and editable (auto-migrated from `DEEPSEEK_API_KEY`/`OPENROUTER_API_KEY`) |
 | `gateway_instances` | `4` | Max concurrent upstream forwards per worker (pool + semaphore) |
 | `gateway_timeout` | `300` | Upstream timeout (seconds); minimum five minutes; also bounds the vision describe-image call |
@@ -898,13 +967,17 @@ receives a notification through every provider they hold a live subscription for
 | Provider | Registration | Transport |
 |----------|--------------|-----------|
 | `webpush` | `PushSubscription` from the browser `PushManager` (endpoint + `p256dh`/`auth` keys) | Web Push Protocol, VAPID signed, `aesgcm` encrypted payload |
-| `apns` | Hexadecimal device token | `POST https://api.push.apple.com/3/device/{token}` over HTTP/2, ES256 provider token |
+| `apns` | Hexadecimal device token, optional stable `client_id` | Dedicated HTTP/2 client to `api.push.apple.com` or `api.sandbox.push.apple.com` (`POST /3/device/{token}`), ES256 provider token. Not the stealth/Chrome client. |
 
 `POST /push.json` accepts a registration for any active provider; a body without a
-`provider` field is a `webpush` body, so browsers need no change. `GET /push.json` returns
-the VAPID public key plus the providers currently accepting registrations. A provider that
-is disabled or not fully configured accepts no registrations and is skipped during
-delivery, so an unconfigured provider is inert rather than an error.
+`provider` field is a `webpush` body, so browsers need no change. An APNs body is
+`{"provider": "apns", "token": "...", "client_id": "..."}` - `client_id` is optional and
+identifies the device across Apple token rotations, so a new token updates that row
+instead of inserting another. A token-only body still works and revives a previously
+dead token. `GET /push.json` returns the VAPID public key plus the providers currently
+accepting registrations; when `apns` is active it includes `environment` (`production` or
+`sandbox`). A provider that is disabled or not fully configured accepts no registrations
+and is skipped during delivery, so an unconfigured provider is inert rather than an error.
 
 Every provider setting is edited at **`/admin/services/push`**: per provider an `Enabled`
 toggle, the VAPID subject for `webpush`, and team id, key id, `.p8` auth key (stored as a
@@ -930,6 +1003,7 @@ in real time, bridged onto the in-process pub/sub bus by a lock-owner relay:
 | Direct message received | receiver |
 | Comment on your post | post author |
 | Reply to your comment | comment author |
+| Any comment on a post you've also commented on | every other commenter on that post |
 | `@mention` in any content | mentioned user |
 | Upvote on your content | content owner |
 | New follower | followed user |
@@ -939,13 +1013,15 @@ in real time, bridged onto the in-process pub/sub bus by a lock-owner relay:
 `create_notification` schedules delivery as a fire-and-forget async task, so a dead
 subscription or push-service error never blocks the triggering request. Delivery
 (`push.notify_user`) reads a user's subscriptions once, groups them by provider, builds
-each provider's payload once, and sends over a single shared HTTP client. A subscription
-the push service reports as gone (`404`/`410` for Web Push, `410` or an `Unregistered`
-class reason for APNs) is soft-deleted; any other failure is logged and the subscription is
-kept.
+each provider's payload once, and sends over that provider's own HTTP client (stealth for
+Web Push, a dedicated HTTP/2 client for APNs). A subscription the push service reports as
+gone (`404`/`410` for Web Push, `410` or an `Unregistered` class reason for APNs) is
+soft-deleted and the provider reason is logged; any other failure is logged and the
+subscription is kept. APNs environment is stored per registration so a sandbox debug
+token and a production token can coexist.
 
 A notification is also **marked read automatically when you open the page that shows its
-content** - viewing a post clears its comment, reply, upvote and mention notifications;
+content** - viewing a post clears its comment, reply, thread, upvote and mention notifications;
 opening a conversation clears its direct-message notifications; visiting a profile clears
 the matching follow, badge and level notifications; and the issue, reminder and farm-raid
 notifications clear on their respective pages. You no longer have to dismiss each one by
@@ -1001,7 +1077,7 @@ offline. Installation requires a secure origin (HTTPS, or `localhost` for develo
 |------|------|
 | `devplacepy/push/providers/` | Provider protocol, Web Push (VAPID keys, payload encryption), APNs |
 | `devplacepy/push/store.py` | `push_registration` reads and writes |
-| `devplacepy/push/delivery.py` | `notify_user` - group by provider, deliver, reap dead subscriptions |
+| `devplacepy/push/delivery.py` | `notify_user` / `notify_registration` - group by provider, per-provider client, reap dead subscriptions |
 | `devplacepy/services/push/service.py` | Provider configuration at `/admin/services/push`, retention sweep, metrics |
 | `devplacepy/routers/push.py` | `/push.json`, `/service-worker.js`, `/manifest.json` |
 | `static/js/PushManager.js` | Service-worker registration + subscribe + opt-in UI |
@@ -1148,13 +1224,13 @@ reverse_proxy localhost:10500 {
 
 ### Static asset caching
 
-Static assets (CSS, JS, vendored libraries) are served with a **one-year immutable cache** for the best Lighthouse "efficient cache policy" score, while deploys still take effect immediately. Every app-owned static URL carries a boot-time version path segment, `/static/v<timestamp>/...`, where `<timestamp>` is the unix time the server process started (`config.STATIC_VERSION`). A restart changes the segment, so every asset URL changes and returning browsers refetch on their next page load - no cache purge, no hashing build step.
+Static assets (CSS, JS, vendored libraries) are served with a **one-year immutable cache** for the best Lighthouse "efficient cache policy" score, while deploys still take effect immediately. Every app-owned static URL carries a version path segment, `/static/v<app-version>-<boot-id>/...` (`config.STATIC_VERSION`) - `<app-version>` is `pyproject.toml`'s `version` (auto-bumped on every commit, see "Version bumping" in `CLAUDE.md`) and `<boot-id>` is the unix time the server process started. A restart changes the segment, so every asset URL changes and returning browsers refetch on their next page load - no cache purge, no hashing build step.
 
 The version sits in the **path**, not a query string, because the frontend is unbundled ES6 modules wired with relative imports: a path segment is inherited automatically by every transitively imported module and relative CSS `url()`, so the whole graph busts on deploy. Templates emit URLs through the `static_url` Jinja global and runtime JavaScript through the `assetUrl` helper (`static/js/assetVersion.js`, reading `<meta name="asset-version">`). User uploads under `/static/uploads/` and the `service-worker.js` route are excluded. Set `DEVPLACE_STATIC_VERSION` at launch so multiple workers share one value (the `prod` target and Docker image do this). Full detail: `/docs/static-caching.html`.
 
 ### Bare-metal alternative
 
-`make prod` runs the same app without containers (`uvicorn ... --workers $(WEB_WORKERS) --proxy-headers`, where `WEB_WORKERS` defaults to `nproc`) from the project root, sharing the identical database and files. Note it binds port 10500, so it conflicts with the Docker front door on the same port - run one, or set a different `PORT`.
+`make prod` runs the same app without containers (`uvicorn ... --workers $(WEB_WORKERS) --proxy-headers`, where `WEB_WORKERS` defaults to `nproc`) from the project root, sharing the identical database and files. It binds port 10500 by default, so it conflicts with the Docker front door on the same port - run one, or set `DEVPLACE_PORT=<other-port> make prod` (also honoured by `make dev`).
 
 ### Multi-worker safety
 

devplacepy/attachments.py

@@ -3,6 +3,7 @@
 import asyncio
 import ipaddress
 import logging
+import os
 import socket
 from datetime import datetime, timezone
 from pathlib import Path
@@ -10,7 +11,7 @@ from urllib.parse import urlparse
 from PIL import Image
 from io import BytesIO
 import httpx
-from devplacepy import stealth
+from devplacepy.net_guard import BlockedAddressError, guarded_async_client
 from devplacepy.database import get_table, db, get_setting
 from devplacepy.config import UPLOADS_DIR, ATTACHMENTS_DIR
 from devplacepy.utils import generate_uid
@@ -389,7 +390,7 @@ async def fetch_remote_file(url, filename=None):
     await _guard_public_url(url)
     max_bytes = _get_max_upload_bytes()
     try:
-        async with stealth.stealth_async_client(
+        async with guarded_async_client(
             follow_redirects=True,
             timeout=REMOTE_FETCH_TIMEOUT,
             headers={"User-Agent": REMOTE_FETCH_USER_AGENT},
@@ -412,6 +413,8 @@ async def fetch_remote_file(url, filename=None):
                             413,
                         )
                 data = b"".join(chunks)
+    except BlockedAddressError as exc:
+        raise RemoteFetchError(str(exc), 400) from exc
     except httpx.HTTPError as exc:
         raise RemoteFetchError(f"Could not fetch {url}: {exc}", 400) from exc
 
@@ -453,6 +456,33 @@ def link_attachments(uids, target_type, target_uid):
         )
 
 
+def split_attachment_uids(raw):
+    return [
+        uid.strip() for item in raw or [] for uid in str(item).split(",") if uid.strip()
+    ]
+
+
+def get_orphan_attachments_batch(uids, user, admin=False):
+    if not uids:
+        return []
+    placeholders = ",".join(f":p{i}" for i in range(len(uids)))
+    params = {f"p{i}": uid for i, uid in enumerate(uids)}
+    rows = db.query(
+        f"SELECT * FROM attachments WHERE uid IN ({placeholders}) AND deleted_at IS NULL",
+        **params,
+    )
+    by_uid = {row["uid"]: row for row in rows}
+    owned = []
+    for uid in uids:
+        row = by_uid.get(uid)
+        if not row or row.get("target_uid"):
+            continue
+        if row.get("user_uid") and row["user_uid"] != user["uid"] and not admin:
+            continue
+        owned.append(uid)
+    return owned
+
+
 def set_gitea_asset_id(uid, asset_id):
     get_table("attachments").update(
         {"uid": uid, "gitea_asset_id": int(asset_id)}, ["uid"]
@@ -472,7 +502,7 @@ async def mirror_attachment_to_gitea(uid):
         return None
     path = ATTACHMENTS_DIR / row.get("directory", "") / row.get("stored_name", "")
     try:
-        data = path.read_bytes()
+        data = await asyncio.to_thread(path.read_bytes)
     except OSError as exc:
         logger.warning("Cannot read attachment %s for Gitea mirror: %s", uid, exc)
         return None
@@ -516,6 +546,28 @@ async def remove_gitea_asset(row):
         logger.warning("Gitea asset delete failed for %s: %s", row.get("uid"), exc)
 
 
+_pending_gitea_tasks: set[asyncio.Task] = set()
+
+
+def _fire_and_forget(coro) -> None:
+    try:
+        loop = asyncio.get_running_loop()
+    except RuntimeError:
+        coro.close()
+        return
+    task = loop.create_task(coro)
+    _pending_gitea_tasks.add(task)
+    task.add_done_callback(_pending_gitea_tasks.discard)
+
+
+def schedule_gitea_mirror(uid: str) -> None:
+    _fire_and_forget(mirror_attachment_to_gitea(uid))
+
+
+def schedule_gitea_removal(row: dict) -> None:
+    _fire_and_forget(remove_gitea_asset(row))
+
+
 def _unlink_attachment_files(row):
     stored_name = row.get("stored_name", "")
     directory = row.get("directory", "")
@@ -586,6 +638,73 @@ def restore_attachment(uid):
     return True
 
 
+def purge_soft_deleted_attachments(*, dry_run: bool = False) -> tuple[int, int]:
+    if "attachments" not in db.tables:
+        return 0, 0
+    table = get_table("attachments")
+    rows = list(table.find(table.table.columns.deleted_at.isnot(None)))
+    removed = 0
+    freed = 0
+    for row in rows:
+        directory = row.get("directory")
+        stored_name = row.get("stored_name")
+        if directory and stored_name:
+            path = ATTACHMENTS_DIR / directory / stored_name
+            try:
+                freed += path.stat().st_size
+            except OSError:
+                pass
+            for thumb in (ATTACHMENTS_DIR / directory).glob(
+                f"{Path(stored_name).stem}_thumb.*"
+            ):
+                try:
+                    freed += thumb.stat().st_size
+                except OSError:
+                    pass
+            if not dry_run:
+                _unlink_attachment_files(row)
+        if not dry_run:
+            table.delete(id=row["id"])
+        removed += 1
+    return removed, freed
+
+
+def sweep_orphan_attachment_blobs(*, dry_run: bool = False) -> tuple[int, int]:
+    if not ATTACHMENTS_DIR.exists():
+        return 0, 0
+    referenced_stems = set()
+    if "attachments" in db.tables:
+        for row in get_table("attachments").find():
+            directory = row.get("directory")
+            stored_name = row.get("stored_name")
+            if directory and stored_name:
+                referenced_stems.add((directory, Path(stored_name).stem))
+    removed = 0
+    freed = 0
+    base = str(ATTACHMENTS_DIR)
+    for root, _dirs, files in os.walk(base):
+        directory = os.path.relpath(root, base)
+        for name in files:
+            stem = Path(name).stem
+            if stem.endswith("_thumb"):
+                stem = stem[: -len("_thumb")]
+            if (directory, stem) in referenced_stems:
+                continue
+            file_path = os.path.join(root, name)
+            try:
+                size = os.path.getsize(file_path)
+            except OSError:
+                continue
+            if not dry_run:
+                try:
+                    os.unlink(file_path)
+                except OSError:
+                    continue
+            removed += 1
+            freed += size
+    return removed, freed
+
+
 def soft_delete_target_attachments(target_type, target_uid, deleted_by):
     stamp = datetime.now(timezone.utc).isoformat()
     for row in get_table("attachments").find(

devplacepy/cli/__init__.py

@@ -45,6 +45,7 @@ from devplacepy.cli.containers import (
 )
 from devplacepy.cli.quiz import cmd_quiz_prune
 from devplacepy.cli.migrate import cmd_emoji_sync, cmd_migrate_data
+from devplacepy.cli.system import cmd_system_prune
 
 __all__ = [
     "main",
@@ -91,4 +92,5 @@ __all__ = [
     "cmd_quiz_prune",
     "cmd_emoji_sync",
     "cmd_migrate_data",
+    "cmd_system_prune",
 ]

devplacepy/cli/containers.py

@@ -66,23 +66,27 @@ def cmd_containers_prune_builds(args):
     )
 
 
+def _human_bytes(n):
+    n = float(n)
+    for unit in ("B", "KB", "MB", "GB", "TB"):
+        if n < 1024:
+            return f"{n:.1f}{unit}"
+        n /= 1024
+    return f"{n:.1f}PB"
+
+
 def cmd_containers_gc_workspaces(args):
-    import shutil
-    from pathlib import Path
-    from devplacepy import config
     from devplacepy.services.containers import store
 
-    active = {inst["project_uid"] for inst in store.all_instances()}
-    base = Path(config.CONTAINER_WORKSPACES_DIR)
-    removed = 0
-    if base.is_dir():
-        for child in base.iterdir():
-            if child.is_dir() and child.name not in active:
-                shutil.rmtree(child, ignore_errors=True)
-                removed += 1
-    _audit_cli("cli.containers.gc_workspaces", f"CLI removed {removed} unused workspace dirs", metadata={"count": removed})
+    removed, freed = store.gc_workspaces()
+    _audit_cli(
+        "cli.containers.gc_workspaces",
+        f"CLI removed {removed} unused workspace dirs, freed {_human_bytes(freed)}",
+        metadata={"count": removed, "bytes_freed": freed},
+    )
     print(
         f"Removed {removed} unused workspace director{'y' if removed == 1 else 'ies'}"
+        f" ({_human_bytes(freed)})"
     )
 
 

devplacepy/cli/main.py

@@ -17,6 +17,7 @@ from devplacepy.cli.game import register_game
 from devplacepy.cli.quiz import register_quiz
 from devplacepy.cli.gateway import register_gateway
 from devplacepy.cli.messaging import register_messaging
+from devplacepy.cli.system import register_system
 
 
 def build_parser():
@@ -38,6 +39,7 @@ def build_parser():
     register_gateway(sub)
     register_messaging(sub)
     register_accounts(sub)
+    register_system(sub)
 
     return parser
 

devplacepy/cli/system.py

@@ -0,0 +1,202 @@
+# retoor <retoor@molodetz.nl>
+
+from devplacepy.cli._shared import _audit_cli
+
+RULE_WIDTH = 72
+
+
+def _rule(char: str = "=") -> str:
+    return char * RULE_WIDTH
+
+
+def _human(n) -> str:
+    n = float(n)
+    for unit in ("B", "KB", "MB", "GB", "TB"):
+        if n < 1024:
+            return f"{n:.1f}{unit}"
+        n /= 1024
+    return f"{n:.1f}PB"
+
+
+def _dir_stats(path) -> tuple[int, int]:
+    import os
+
+    count = 0
+    total = 0
+    for root, _dirs, files in os.walk(str(path)):
+        for name in files:
+            count += 1
+            try:
+                total += os.path.getsize(os.path.join(root, name))
+            except OSError:
+                continue
+    return count, total
+
+
+def _section(title: str) -> None:
+    print()
+    print(_rule())
+    print(title)
+    print(_rule())
+
+
+def _build_areas():
+    from pathlib import Path
+
+    from devplacepy import config
+    from devplacepy.attachments import (
+        ATTACHMENTS_DIR,
+        purge_soft_deleted_attachments,
+        sweep_orphan_attachment_blobs,
+    )
+    from devplacepy.project_files import (
+        PROJECT_FILES_DIR,
+        purge_soft_deleted_project_files,
+        sweep_orphan_project_file_blobs,
+    )
+    from devplacepy.services.containers import store as container_store
+
+    return [
+        {
+            "label": "Attachments",
+            "path": ATTACHMENTS_DIR,
+            "checks": [
+                ("soft-deleted attachments", purge_soft_deleted_attachments),
+                (
+                    "orphan attachment blobs (zero DB reference)",
+                    sweep_orphan_attachment_blobs,
+                ),
+            ],
+        },
+        {
+            "label": "Project files",
+            "path": PROJECT_FILES_DIR,
+            "checks": [
+                ("soft-deleted project files", purge_soft_deleted_project_files),
+                (
+                    "orphan project-file blobs (zero DB reference)",
+                    sweep_orphan_project_file_blobs,
+                ),
+            ],
+        },
+        {
+            "label": "Container workspaces",
+            "path": Path(config.CONTAINER_WORKSPACES_DIR),
+            "checks": [
+                (
+                    "orphaned workspace directories (no live instance)",
+                    container_store.gc_workspaces,
+                ),
+            ],
+        },
+    ]
+
+
+def cmd_system_prune(args):
+    import time
+
+    dry_run = bool(args.dry_run)
+    areas = _build_areas()
+
+    print("System prune - safe but aggressive disk-space cleanup")
+    print(
+        "Removes only content with zero live reference: soft-deleted attachment/"
+        "\nproject-file blobs, blob files with no matching database row at all"
+        "\n(orphans, e.g. left by an interrupted or racing sync), and orphaned"
+        "\ncontainer workspace directories. Never touches live content."
+    )
+    if dry_run:
+        print("DRY RUN: this pass only reports; nothing will be deleted.")
+
+    started = time.monotonic()
+
+    _section("Where the data is, and what can be reclaimed (estimate)")
+    estimate_items = 0
+    estimate_bytes = 0
+    for area in areas:
+        count, size = _dir_stats(area["path"])
+        print(f"\n{area['label']}")
+        print(f"  location: {area['path']}")
+        print(f"  currently on disk: {count} file(s), {_human(size)}")
+        for check_label, fn in area["checks"]:
+            c, f = fn(dry_run=True)
+            estimate_items += c
+            estimate_bytes += f
+            print(f"  expected to reclaim - {check_label}: {c} item(s), {_human(f)}")
+
+    print()
+    print(_rule("-"))
+    print(
+        f"Estimated total: {estimate_items} item(s), {_human(estimate_bytes)} reclaimable"
+    )
+    print(_rule("-"))
+
+    if dry_run:
+        print()
+        print("DRY RUN complete: nothing was deleted. Re-run without --dry-run to apply.")
+        return
+
+    _section("Executing")
+    actual_items = 0
+    actual_bytes = 0
+    for area in areas:
+        print(f"\n{area['label']} - {area['path']}")
+        for check_label, fn in area["checks"]:
+            c, f = fn(dry_run=False)
+            actual_items += c
+            actual_bytes += f
+            print(f"  removed - {check_label}: {c} item(s), {_human(f)}")
+
+    _section("After")
+    for area in areas:
+        count, size = _dir_stats(area["path"])
+        print(f"  {area['label']} ({area['path']}): {count} file(s), {_human(size)}")
+
+    elapsed = time.monotonic() - started
+
+    print()
+    print(_rule())
+    print(
+        f"Removed {actual_items} item(s) total, freed {_human(actual_bytes)} "
+        f"in {elapsed:.1f}s"
+    )
+    if estimate_bytes != actual_bytes or estimate_items != actual_items:
+        print(
+            f"(Estimate was {estimate_items} item(s), {_human(estimate_bytes)} - "
+            "state changed between the estimate and execution passes, e.g. the "
+            "live server wrote or soft-deleted something in between.)"
+        )
+    print(_rule())
+
+    _audit_cli(
+        "cli.system.prune",
+        f"CLI system prune removed {actual_items} item(s), freed {_human(actual_bytes)}",
+        metadata={
+            "items": actual_items,
+            "bytes_freed": actual_bytes,
+            "estimated_items": estimate_items,
+            "estimated_bytes": estimate_bytes,
+            "elapsed_seconds": round(elapsed, 3),
+        },
+    )
+
+
+def register_system(subparsers):
+    system = subparsers.add_parser("system", help="Cross-cutting system maintenance")
+    system_sub = system.add_subparsers(title="action", dest="action")
+    prune = system_sub.add_parser(
+        "prune",
+        help=(
+            "Safe but aggressive disk-space cleanup: purges soft-deleted "
+            "attachment/project-file blobs and any blob with zero database "
+            "reference at all, plus orphaned container workspace directories. "
+            "Reports location, current size, and estimated reclaim per area "
+            "before acting, then confirms what was actually freed."
+        ),
+    )
+    prune.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="Report location, size, and estimated reclaim without deleting anything",
+    )
+    prune.set_defaults(func=cmd_system_prune)

devplacepy/config.py

@@ -1,6 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
 import time
+import tomllib
 from pathlib import Path
 from dotenv import load_dotenv
 from os import environ
@@ -22,6 +23,15 @@ ZIP_STAGING_DIR = DATA_DIR / "zip_staging"
 FORK_STAGING_DIR = DATA_DIR / "fork_staging"
 BACKUPS_DIR = DATA_DIR / "backups"
 BACKUP_STAGING_DIR = DATA_DIR / "backup_staging"
+
+RCLONE_BIN = environ.get("DEVPLACE_RCLONE_BIN", "rclone")
+RCLONE_CONFIG_FILE = environ.get(
+    "DEVPLACE_RCLONE_CONFIG",
+    str(Path(environ.get("HOME", "/root")) / ".config" / "rclone" / "rclone.conf"),
+)
+BACKUP_OFFLOAD_REMOTE = environ.get(
+    "DEVPLACE_BACKUP_OFFLOAD_REMOTE", "storagebox:devplacepy-backups"
+)
 SEO_REPORTS_DIR = DATA_DIR / "seo_reports"
 PLANNING_REPORTS_DIR = DATA_DIR / "planning_reports"
 DBAPI_DIR = DATA_DIR / "dbapi"
@@ -45,7 +55,7 @@ SECRET_KEY = environ.get("SECRET_KEY", "devplace-secret-key-change-in-production
 SECONDS_PER_DAY = 86400
 SESSION_MAX_AGE = SECONDS_PER_DAY * 7
 SESSION_MAX_AGE_REMEMBER = SECONDS_PER_DAY * 30
-PORT = 10500
+PORT = int(environ.get("DEVPLACE_PORT", "10500"))
 SITE_URL = environ.get("DEVPLACE_SITE_URL", "").rstrip("/")
 
 PRESENCE_TIMEOUT_SECONDS = int(environ.get("DEVPLACE_PRESENCE_TIMEOUT_SECONDS", "60"))
@@ -59,7 +69,9 @@ PRESENCE_ONLINE_MARGIN_SECONDS = int(
 XMLRPC_BIND = environ.get("DEVPLACE_XMLRPC_BIND", "127.0.0.1")
 XMLRPC_PORT = int(environ.get("DEVPLACE_XMLRPC_PORT", "10550"))
 
-STATIC_VERSION = environ.get("DEVPLACE_STATIC_VERSION") or str(int(time.time()))
+APP_VERSION = tomllib.loads((BASE_DIR / "pyproject.toml").read_text())["project"]["version"]
+BOOT_ID = environ.get("DEVPLACE_STATIC_VERSION") or str(int(time.time()))
+STATIC_VERSION = f"{APP_VERSION}-{BOOT_ID}"
 
 TEMPLATE_AUTO_RELOAD = environ.get("DEVPLACE_TEMPLATE_AUTO_RELOAD", "1") != "0"
 
@@ -72,6 +84,9 @@ INTERNAL_MODEL = "molodetz"
 INTERNAL_EMBED_MODEL = "molodetz~embed"
 INTERNAL_IMAGE_MODEL = "molodetz-img-small"
 
+AQUALITY_NEWS_GRADING_URL = "https://aquality.cloud.pravda.education/v1/chat/completions"
+AQUALITY_NEWS_GRADING_MODEL = "aquality"
+
 AWARD_GIVE_COOLDOWN_HOURS_DEFAULT = 24
 AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT = 24
 AWARD_DISPLAY_HOURS_DEFAULT = 24
@@ -97,6 +112,8 @@ QUIZ_SCOREBOARD_LIMIT = 20
 QUIZ_SCOREBOARD_CACHE_SECONDS = 15
 QUIZ_LIST_PER_PAGE = 20
 
+BATTLES_LIST_PER_PAGE = 10
+
 DEFAULT_CORRECTION_PROMPT = "Leave literary as is, only do punctuation and casing"
 DEFAULT_MODIFIER_PROMPT = (
     "Execute what is behind `@ai` (the prompt) and replace that part including `@ai`"

devplacepy/constants.py

@@ -2,6 +2,16 @@
 
 TOPICS = ["devlog", "showcase", "question", "rant", "fun", "random", "politics"]
 
+TOPIC_LABELS = {
+    "devlog": "Devlog",
+    "showcase": "Showcase",
+    "question": "Question",
+    "rant": "Rant",
+    "fun": "Fun",
+    "random": "Random",
+    "politics": "Politics",
+}
+
 REACTION_EMOJI = [
     "\U0001f44d",
     "❤️",

devplacepy/content.py

@@ -38,6 +38,8 @@ from devplacepy.database import (
     get_int_setting,
     _now_iso,
     db,
+    get_user_recent_items,
+    invalidate_user_recent_cache,
 )
 from devplacepy.utils import (
     time_ago,
@@ -46,6 +48,7 @@ from devplacepy.utils import (
     award_rewards,
     track_action,
     create_notification,
+    create_thread_notifications,
     create_mention_notifications,
     is_admin,
     is_primary_admin,
@@ -53,6 +56,7 @@ from devplacepy.utils import (
     XP_UPVOTE,
 )
 from devplacepy.services.audit import record as audit
+from devplacepy.services.opinionwar import store as war_store
 from devplacepy.services.correction import schedule_correction
 from devplacepy.services.ai_modifier import schedule_modification
 from devplacepy.services.seo_meta import schedule_seo_meta_for_table
@@ -131,6 +135,18 @@ def can_view_project(project: dict | None, user: dict | None) -> bool:
     return not _owner_is_admin(project)
 
 
+def get_user_sidebar_gists(user_uid: str, limit: int = 5) -> list[dict]:
+    return get_user_recent_items("gists", user_uid)[:limit]
+
+
+def get_user_sidebar_projects(
+    user_uid: str, viewer: dict | None, limit: int = 5
+) -> list[dict]:
+    rows = get_user_recent_items("projects", user_uid)
+    visible = [p for p in rows if can_view_project(p, viewer)]
+    return visible[:limit]
+
+
 def owns_instance(
     instance: dict | None, project: dict | None, user: dict | None
 ) -> bool:
@@ -262,6 +278,7 @@ def create_content_item(
         from devplacepy.templating import clear_user_projects_cache
 
         clear_user_projects_cache(user["uid"])
+    invalidate_user_recent_cache(table_name, user["uid"])
     award_rewards(user["uid"], xp, badge)
     if attachment_uids:
         link_attachments(attachment_uids, target_type, uid)
@@ -437,6 +454,7 @@ def create_comment_record(
     comment_url = f"{redirect_url}#comment-{comment_uid}"
 
     if target_type == "post":
+        already_notified = {user["uid"]}
         if parent_uid:
             parent = get_table("comments").find_one(uid=parent_uid, deleted_at=None)
             if parent and parent["user_uid"] != user["uid"]:
@@ -447,6 +465,7 @@ def create_comment_record(
                     user["uid"],
                     comment_url,
                 )
+                already_notified.add(parent["user_uid"])
         else:
             posts = get_table("posts")
             post = posts.find_one(uid=target_uid)
@@ -460,6 +479,9 @@ def create_comment_record(
                     user["uid"],
                     comment_url,
                 )
+                already_notified.add(post["user_uid"])
+
+        create_thread_notifications(target_uid, user["uid"], comment_url, already_notified)
 
     create_mention_notifications(content, user["uid"], comment_url)
     record_screening(
@@ -627,6 +649,7 @@ def detail_context(
         "reactions": detail.get("reactions", {"counts": {}, "mine": []}),
         "bookmarked": detail.get("bookmarked", False),
         "poll": detail.get("poll"),
+        "war": detail.get("war"),
         "project_link": detail.get("project_link"),
         "maturity": detail.get("maturity", "general"),
     }
@@ -670,6 +693,7 @@ def edit_content_item(
         "updated_at": datetime.now(timezone.utc).isoformat(),
     }
     table.update({"uid": item["uid"], **update_fields}, ["uid"])
+    invalidate_user_recent_cache(table_name, item["user_uid"])
     record_screening(
         screening,
         target_type=kind,
@@ -761,6 +785,7 @@ def delete_content_item(
         soft_delete_all_project_files(item["uid"], actor)
         soft_delete_fork_relations(item["uid"], actor)
         clear_user_projects_cache(item["user_uid"])
+    invalidate_user_recent_cache(table_name, item["user_uid"])
     soft_delete(table_name, actor, stamp=stamp, uid=item["uid"])
     logger.info(f"{table_name} {item['uid']} soft-deleted by {user['username']}")
     audit.record(
@@ -817,6 +842,9 @@ def load_detail(
         "reactions": reactions,
         "bookmarked": bookmarked,
         "poll": get_poll_for_post(item["uid"], user) if target_type == "post" else None,
+        "war": war_store.get_war_serialized_for_post(item["uid"], user)
+        if target_type == "post"
+        else None,
         "project_link": get_project_by_uid(item.get("project_uid")) if target_type == "post" else None,
         "maturity": get_maturity(target_type, item["uid"])["level"],
     }

devplacepy/database/CLAUDE.md

@@ -198,8 +198,8 @@ Site settings are seeded on startup (`site_settings` table):
 | `site_name` / `site_description` / `site_tagline` | DevPlace branding | General site metadata |
 | `news_grade_threshold` | `"7"` | Minimum AI grade for auto-publish |
 | `news_api_url` | `"https://news.app.molodetz.nl/api"` | News source API |
-| `news_ai_url` | `"https://openai.app.molodetz.nl/v1/chat/completions"` | AI grading endpoint |
-| `news_ai_model` | `"molodetz"` | AI model identifier |
+| `news_ai_url` | `"https://aquality.cloud.pravda.education/v1/chat/completions"` | AI grading endpoint - the free, local aquality quality model by default (see `devplacepy/services/news/CLAUDE.md`) |
+| `news_ai_model` | `"aquality"` | AI model identifier |
 | `max_upload_size_mb` / `allowed_file_types` / `max_attachments_per_resource` | `"10"` / `""` / `"10"` | Upload limits |
 | `rate_limit_per_minute` | `"60"` | Mutating requests per IP per window (`main.py` middleware); a `429` carries a `Retry-After: <window>` header |
 | `rate_limit_window_seconds` | `"60"` | Rolling window for the rate limit (also the `Retry-After` value) |
@@ -211,6 +211,7 @@ Site settings are seeded on startup (`site_settings` table):
 | `maintenance_message` | scheduled-maintenance text | Body shown on the maintenance 503 page |
 | `customization_enabled` | `"1"` | When `"0"`, `custom_css_tag`/`custom_js_tag` inject nothing (feature off) |
 | `customization_js_enabled` | `"1"` | When `"0"`, custom CSS still serves but custom JS is suppressed |
+| `happy_404_enabled` | `"1"` | When `"0"`, `happy404.render()` is a no-op and a real 404 page is shown; when on, an HTML 404 renders a random existing post instead (see below) |
 | `moderation_sla_hours` | `"24"` | The published moderation response window; the admin queue badge turns red past it |
 | `moderation_filter_mode` | `"review"` | `off`/`label`/`review`/`block` - how the content filter acts on a match |
 | `moderation_filter_review_score` | `"2"` | Rule score at which a match becomes a report rather than a label |
@@ -239,6 +240,7 @@ Operational settings - read sites and rules:
 | `service_<name>_enabled` / `service_<name>_command` / `service_<name>_log_size` | `BaseService` reconciling loop | Generic per-service controls written by the Services tab; the loop reconciles within ~1s |
 | `session_max_age_days` / `session_remember_days` | `auth.py` signup + login | Multiplied by `SECONDS_PER_DAY`; passed to `create_session(uid, max_age)` so the cookie and the DB session row expire together |
 | `registration_open` | `auth.py` `signup_page` (GET) and `signup` (POST) | POST returns before any DB write when closed |
+| `happy_404_enabled` | `happy404.render()`, called from the `404` handler in `main.py` | Gates the whole feature; checked per-request via the normal 60s `get_setting` TTL cache, so a toggle takes effect within a minute across all workers with no restart |
 
 **Booleans are `<select>`, never checkboxes.** The settings save handler (`admin.py`) skips empty form values so empty fields don't clobber existing rows. An unchecked checkbox submits nothing, so it could never be turned off - `registration_open` and `maintenance_mode` use `<option value="1">`/`<option value="0">` so a value is always submitted.
 

devplacepy/database/__init__.py

@@ -74,7 +74,7 @@ from .moderation import (
     years_between,
 )
 from .comments import _drop_blocked, _build_comment_items, load_comments, get_recent_comments_by_target_uids, get_recent_comments_by_post_uids, load_comments_by_target_uids
-from .content import resolve_by_slug, resolve_object_url, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news, get_trending_topics
+from .content import resolve_by_slug, resolve_object_url, get_projects_by_uids, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news, get_featured_topics, get_trending_topics, get_user_recent_items, invalidate_user_recent_cache
 from .attachments_data import get_attachments, get_attachments_by_type, get_news_images_by_uids, delete_attachment_record, delete_attachments, _delete_attachment_file, get_user_media, get_user_attachments, get_user_attachment, get_deleted_media
 from .stats import _stats_cache, get_site_stats, _analytics_cache, get_platform_analytics, _gist_languages_cache, get_gist_languages
 from .schema import BUG_TABLE_RENAMES, migrate_bug_tables_to_issue_tables, init_db, _refresh_query_planner_stats, OLD_GATEWAY_URL, migrate_ai_gateway_settings, backfill_api_keys, _backfill_gamification
@@ -296,10 +296,14 @@ __all__ = [
     "load_comments_by_target_uids",
     "resolve_by_slug",
     "resolve_object_url",
+    "get_projects_by_uids",
     "get_uids_by_username_match",
     "text_search_clause",
     "get_daily_topic",
     "get_featured_news",
+    "get_featured_topics",
+    "get_user_recent_items",
+    "invalidate_user_recent_cache",
     "get_trending_topics",
     "get_attachments",
     "get_attachments_by_type",

devplacepy/database/content.py

@@ -1,5 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
+import os
+import random
 from collections import Counter
 
 from devplacepy.cache import TTLCache
@@ -8,6 +10,34 @@ from .core import db, get_table, or_
 
 _daily_topic_cache = TTLCache(ttl=60, max_size=1)
 _trending_cache = TTLCache(ttl=15, max_size=1)
+FEATURED_TOPICS_POOL_TTL = int(os.environ.get("DEVPLACE_FEATURED_TOPICS_POOL_TTL", "300"))
+FEATURED_TOPICS_POOL_SIZE = 20
+_featured_topics_cache = TTLCache(ttl=FEATURED_TOPICS_POOL_TTL, max_size=1)
+
+USER_RECENT_ITEMS_TTL = int(os.environ.get("DEVPLACE_USER_RECENT_ITEMS_TTL", "15"))
+_user_recent_cache = TTLCache(ttl=USER_RECENT_ITEMS_TTL, max_size=2000)
+
+
+def _user_recent_key(table_name: str, user_uid: str) -> str:
+    return f"{table_name}:{user_uid}"
+
+
+def invalidate_user_recent_cache(table_name: str, user_uid: str) -> None:
+    _user_recent_cache.pop(_user_recent_key(table_name, user_uid))
+
+
+def get_user_recent_items(table_name: str, user_uid: str) -> list[dict]:
+    key = _user_recent_key(table_name, user_uid)
+    cached = _user_recent_cache.get(key)
+    if cached is not None:
+        return cached
+    items = []
+    if table_name in db.tables:
+        rows = list(get_table(table_name).find(user_uid=user_uid, deleted_at=None))
+        rows.sort(key=lambda row: row.get("updated_at") or row.get("created_at") or "", reverse=True)
+        items = rows
+    _user_recent_cache.set(key, items)
+    return items
 
 
 def resolve_by_slug(table, slug, include_deleted=False):
@@ -89,6 +119,11 @@ def resolve_object_url(target_type: str, target_uid: str) -> str:
         if not poll:
             return "/feed"
         return resolve_object_url("post", poll.get("post_uid", ""))
+    if target_type == "battle":
+        war = get_table("opinion_wars").find_one(uid=target_uid)
+        if not war:
+            return "/battles"
+        return resolve_object_url("post", war.get("post_uid", ""))
     if target_type == "workspace":
         instance = get_table("instances").find_one(uid=target_uid)
         return f"/admin/containers/{instance['uid']}" if instance else "/admin/containers"
@@ -97,6 +132,17 @@ def resolve_object_url(target_type: str, target_uid: str) -> str:
     return "/feed"
 
 
+def get_projects_by_uids(uids):
+    if not uids or "projects" not in db.tables:
+        return {}
+    projects = get_table("projects")
+    if "uid" not in projects.columns:
+        return {}
+    seen = set()
+    unique = [u for u in uids if u not in seen and not seen.add(u)]
+    return {p["uid"]: p for p in projects.find(projects.table.columns.uid.in_(unique))}
+
+
 def get_uids_by_username_match(search, limit=200):
     term = (search or "").strip()
     if not term or "users" not in db.tables:
@@ -155,6 +201,49 @@ def _load_daily_topic():
     }
 
 
+def get_featured_topics(count: int = 3) -> list[dict]:
+    pool = _featured_topics_pool()
+    if not pool:
+        return []
+    return random.sample(pool, min(count, len(pool)))
+
+
+def _featured_topics_pool() -> list[dict]:
+    cached = _featured_topics_cache.get("pool")
+    if cached is not None:
+        return cached
+    pool = _load_featured_topics_pool()
+    _featured_topics_cache.set("pool", pool)
+    return pool
+
+
+def _load_featured_topics_pool(limit: int = FEATURED_TOPICS_POOL_SIZE) -> list[dict]:
+    if "news" not in db.tables:
+        return []
+    rows = db["news"].find(
+        featured=1,
+        status="published",
+        deleted_at=None,
+        order_by=["-synced_at"],
+        _limit=limit,
+    )
+    topics = []
+    for article in rows:
+        desc = (article.get("description") or "")[:160] or (
+            article.get("content") or ""
+        )[:160]
+        topics.append(
+            {
+                "title": article.get("title", ""),
+                "summary": desc,
+                "slug": article.get("slug", ""),
+                "url": article.get("url", ""),
+                "image_url": article.get("image_url", "") or "",
+            }
+        )
+    return topics
+
+
 def get_featured_news(limit=5):
     if "news" not in db.tables:
         return []

devplacepy/database/core.py

@@ -9,6 +9,8 @@ from collections import defaultdict
 from datetime import datetime, timedelta, timezone
 from devplacepy.cache import TTLCache
 from devplacepy.config import (
+    AQUALITY_NEWS_GRADING_MODEL,
+    AQUALITY_NEWS_GRADING_URL,
     DATABASE_URL,
     DEFAULT_CORRECTION_PROMPT,
     DEFAULT_MODIFIER_PROMPT,

devplacepy/database/moderation.py

@@ -17,6 +17,7 @@ REPORTABLE_TARGETS: dict[str, str] = {
     "message": "messages",
     "quiz": "quizzes",
     "poll": "polls",
+    "battle": "opinion_wars",
     "award": "awards",
     "user": "users",
     "issue": "issue_tickets",
@@ -48,6 +49,8 @@ UNREPORTABLE_TABLES: dict[str, str] = {
     "bookmarks": "private to the owner",
     "follows": "relationship rows, carry no authored content",
     "poll_votes": "private ballots",
+    "opinion_war_fighters": "membership and damage counters, carry no authored content",
+    "opinion_war_events": "server-composed battle log rows, not authored content",
     "quiz_attempts": "private to the participant",
     "quiz_answers": "private to the participant",
     "sessions": "authentication state",

devplacepy/database/notifications.py

@@ -8,6 +8,7 @@ from .soft_delete import soft_delete
 NOTIFICATION_TYPES = [
     {"key": "comment", "label": "Comments", "description": "Someone comments on your post"},
     {"key": "reply", "label": "Replies", "description": "Someone replies to your comment"},
+    {"key": "thread", "label": "Thread activity", "description": "Someone else comments on a post you've commented on"},
     {"key": "mention", "label": "Mentions", "description": "Someone mentions you with @username"},
     {"key": "vote", "label": "Upvotes", "description": "Someone ++'d your content"},
     {"key": "follow", "label": "Followers", "description": "Someone starts following you"},
@@ -19,6 +20,7 @@ NOTIFICATION_TYPES = [
     {"key": "harvest_stolen", "label": "Farm raids", "description": "Someone steals a ready build from your Code Farm"},
     {"key": "award", "label": "Awards", "description": "Someone gives you an award on your profile"},
     {"key": "quiz_attempt", "label": "Quiz attempts", "description": "Someone completes one of your quizzes"},
+    {"key": "battle", "label": "Opinion Wars", "description": "Lead changes, results and fight-ready alerts for battles you joined"},
     {"key": "workspace", "label": "Dev workspaces", "description": "Idle, quota, retention and moderation notices for your dev workspaces"},
     {"key": "moderation", "label": "Moderation", "description": "Acknowledgements for reports you file and decisions taken on your content"},
     {"key": "system", "label": "System alerts", "description": "Platform infrastructure alerts (e.g. the AI gateway going down)"},

devplacepy/database/pagination.py

@@ -18,6 +18,7 @@ def paginate(
     order=None,
     cursor_field="created_at",
     viewer_uid=None,
+    limit=PAGE_SIZE,
     **filters,
 ):
     order = order or ["-" + cursor_field]
@@ -30,9 +31,9 @@ def paginate(
             clauses.append(table.table.columns.user_uid.notin_(blocked))
     if before:
         clauses.append(table.table.columns[cursor_field] < before)
-    rows = list(table.find(*clauses, **filters, order_by=order, _limit=PAGE_SIZE + 1))
-    has_more = len(rows) > PAGE_SIZE
-    rows = rows[:PAGE_SIZE]
+    rows = list(table.find(*clauses, **filters, order_by=order, _limit=limit + 1))
+    has_more = len(rows) > limit
+    rows = rows[:limit]
     next_cursor = rows[-1][cursor_field] if has_more and rows else None
     return rows, next_cursor
 
@@ -64,6 +65,7 @@ def paginate_diverse(
     cursor_field="created_at",
     uid_key="user_uid",
     viewer_uid=None,
+    limit=PAGE_SIZE,
     **filters,
 ):
     rows, next_cursor = paginate(
@@ -73,6 +75,7 @@ def paginate_diverse(
         order=order,
         cursor_field=cursor_field,
         viewer_uid=viewer_uid,
+        limit=limit,
         **filters,
     )
     return interleave_by_author(rows, uid_key=uid_key), next_cursor

devplacepy/database/ranking.py

@@ -4,7 +4,7 @@ import os
 
 from .core import TTLCache, _in_clause, _now_iso, db, get_table
 from .users import get_users_by_uids
-from .soft_delete import soft_delete, soft_delete_in
+from .soft_delete import soft_delete_in
 
 
 VOTABLE_TARGETS: dict[str, str] = {
@@ -133,6 +133,26 @@ def update_target_stars(target_type: str, target_uid: str, net_stars: int) -> No
         get_table(table_name).update({"uid": target_uid, "stars": net_stars}, ["uid"])
 
 
+def _child_uids(table_name, parent_column, parent_uids, live_only=False):
+    if table_name not in db.tables:
+        return []
+    table = db[table_name]
+    if parent_column not in table.columns:
+        return []
+    clause = table.table.columns[parent_column].in_(parent_uids)
+    if live_only:
+        return [row["uid"] for row in table.find(clause, deleted_at=None)]
+    return [row["uid"] for row in table.find(clause)]
+
+
+def _delete_in(table_name, column, uids):
+    placeholders, params = _in_clause(uids)
+    with db:
+        db.query(
+            f"DELETE FROM {table_name} WHERE {column} IN ({placeholders})", **params
+        )
+
+
 def soft_delete_engagement(target_type: str, target_uids: list, deleted_by: str) -> None:
     uids = [uid for uid in (target_uids or []) if uid]
     if not uids:
@@ -145,11 +165,15 @@ def soft_delete_engagement(target_type: str, target_uids: list, deleted_by: str)
         "bookmarks", "target_uid", uids, deleted_by, stamp=stamp, target_type=target_type
     )
     if target_type == "post" and "polls" in db.tables:
-        for uid in uids:
-            for poll in db["polls"].find(post_uid=uid, deleted_at=None):
-                soft_delete("poll_votes", deleted_by, stamp=stamp, poll_uid=poll["uid"])
-                soft_delete("poll_options", deleted_by, stamp=stamp, poll_uid=poll["uid"])
-            soft_delete("polls", deleted_by, stamp=stamp, post_uid=uid)
+        poll_uids = _child_uids("polls", "post_uid", uids, live_only=True)
+        soft_delete_in("poll_votes", "poll_uid", poll_uids, deleted_by, stamp=stamp)
+        soft_delete_in("poll_options", "poll_uid", poll_uids, deleted_by, stamp=stamp)
+        soft_delete_in("polls", "post_uid", uids, deleted_by, stamp=stamp)
+    if target_type == "post" and "opinion_wars" in db.tables:
+        war_uids = _child_uids("opinion_wars", "post_uid", uids, live_only=True)
+        soft_delete_in("opinion_war_fighters", "war_uid", war_uids, deleted_by, stamp=stamp)
+        soft_delete_in("opinion_war_events", "war_uid", war_uids, deleted_by, stamp=stamp)
+        soft_delete_in("opinion_wars", "post_uid", uids, deleted_by, stamp=stamp)
 
 
 def delete_engagement(target_type: str, target_uids: list) -> None:
@@ -174,13 +198,21 @@ def delete_engagement(target_type: str, target_uids: list) -> None:
                 **params,
             )
     if target_type == "post" and "polls" in tables:
-        for uid in uids:
-            for poll in db["polls"].find(post_uid=uid):
-                if "poll_votes" in tables:
-                    db["poll_votes"].delete(poll_uid=poll["uid"])
-                if "poll_options" in tables:
-                    db["poll_options"].delete(poll_uid=poll["uid"])
-            db["polls"].delete(post_uid=uid)
+        poll_uids = _child_uids("polls", "post_uid", uids)
+        if poll_uids:
+            if "poll_votes" in tables:
+                _delete_in("poll_votes", "poll_uid", poll_uids)
+            if "poll_options" in tables:
+                _delete_in("poll_options", "poll_uid", poll_uids)
+        _delete_in("polls", "post_uid", uids)
+    if target_type == "post" and "opinion_wars" in tables:
+        war_uids = _child_uids("opinion_wars", "post_uid", uids)
+        if war_uids:
+            if "opinion_war_fighters" in tables:
+                _delete_in("opinion_war_fighters", "war_uid", war_uids)
+            if "opinion_war_events" in tables:
+                _delete_in("opinion_war_events", "war_uid", war_uids)
+        _delete_in("opinion_wars", "post_uid", uids)
 
 
 def get_target_owner_uid(target_type: str, target_uid: str) -> str | None:

devplacepy/database/schema.py

@@ -1,6 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
-from .core import DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, _drop_index, _index, _uid_index, db, defaultdict, get_table, logger
+from .core import AQUALITY_NEWS_GRADING_MODEL, AQUALITY_NEWS_GRADING_URL, DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, _drop_index, _index, _uid_index, db, defaultdict, get_table, logger
 from .settings import get_setting, set_setting
 from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns
 from .ranking import _authors_cache
@@ -30,6 +30,54 @@ def migrate_bug_tables_to_issue_tables() -> None:
         logger.info("Dropped table %s after migration", source_name)
 
 
+_TUNNEL_STATUS_RANK = {
+    "active": 4,
+    "provisioning": 3,
+    "pending": 2,
+    "failed": 1,
+    "suspended": 0,
+}
+
+
+def _dedupe_tunnel_hostnames() -> None:
+    if "tunnels" not in db.tables:
+        return
+    table = get_table("tunnels")
+    if not table.has_column("hostname") or not table.has_column("uid"):
+        return
+    groups = list(
+        db.query(
+            "SELECT hostname FROM tunnels "
+            "WHERE hostname IS NOT NULL AND hostname != '' "
+            "GROUP BY hostname HAVING COUNT(*) > 1"
+        )
+    )
+    for group in groups:
+        hostname = group["hostname"]
+        dupes = list(table.find(hostname=hostname))
+        dupes.sort(
+            key=lambda r: (
+                _TUNNEL_STATUS_RANK.get(r.get("status") or "", -1),
+                r.get("deleted_at") is None,
+                r.get("created_at") or "",
+                r.get("id") or 0,
+            ),
+            reverse=True,
+        )
+        losers = [r["uid"] for r in dupes[1:]]
+        if not losers:
+            continue
+        with db:
+            for uid in losers:
+                db.query("DELETE FROM tunnels WHERE uid = :uid", uid=uid)
+        logger.warning(
+            "Removed %d duplicate tunnel row(s) for hostname %s, kept %s",
+            len(losers),
+            hostname,
+            dupes[0]["uid"],
+        )
+
+
 def init_db():
     tables = db.tables
     _index(db, "users", "idx_users_username", ["username"])
@@ -38,6 +86,7 @@ def init_db():
     _index(db, "users", "idx_users_role", ["role", "created_at"])
     _index(db, "users", "idx_users_last_seen", ["last_seen"])
     _index(db, "users", "idx_users_created_at", ["created_at"])
+    _ensure_users_uid()
     _index(db, "posts", "idx_posts_user_uid", ["user_uid"])
     _index(db, "posts", "idx_posts_created_at", ["created_at"])
     _index(db, "posts", "idx_posts_topic", ["topic"])
@@ -129,6 +178,7 @@ def init_db():
         ("content", ""),
         ("read", False),
         ("created_at", ""),
+        ("updated_at", ""),
     ):
         if not messages.has_column(column):
             messages.create_column_by_example(column, example)
@@ -143,6 +193,7 @@ def init_db():
         "idx_messages_conversation_rev",
         ["receiver_uid", "sender_uid"],
     )
+    _index(db, "messages", "idx_messages_updated_at", ["updated_at"])
     _index(db, "notifications", "idx_notifications_user", ["user_uid"])
     _index(db, "notifications", "idx_notifications_user_read", ["user_uid", "read"])
     push_registration = get_table("push_registration")
@@ -154,19 +205,38 @@ def init_db():
         ("key_auth", ""),
         ("key_p256dh", ""),
         ("token", ""),
+        ("client_id", ""),
+        ("environment", ""),
         ("created_at", ""),
+        ("registered_at", ""),
         ("deleted_at", ""),
     ):
         if not push_registration.has_column(column):
             push_registration.create_column_by_example(column, example)
     _index(db, "push_registration", "idx_push_registration_user", ["user_uid"])
     _index(db, "push_registration", "idx_push_registration_provider", ["provider"])
+    _index(
+        db,
+        "push_registration",
+        "idx_push_registration_client",
+        ["user_uid", "provider", "client_id"],
+    )
+    _index(
+        db,
+        "push_registration",
+        "idx_push_registration_token",
+        ["user_uid", "provider", "token"],
+    )
     if "push_registration" in db.tables:
         with db:
             db.query(
                 "UPDATE push_registration SET provider = 'webpush' "
                 "WHERE provider IS NULL OR provider = ''"
             )
+            db.query(
+                "UPDATE push_registration SET registered_at = created_at "
+                "WHERE registered_at IS NULL OR registered_at = ''"
+            )
     _index(db, "sessions", "idx_sessions_token", ["session_token"])
     projects = get_table("projects")
     for column, example in (
@@ -224,6 +294,22 @@ def init_db():
     _index(
         db, "project_files", "idx_project_files_parent", ["project_uid", "parent_path"]
     )
+    project_file_sync_state = get_table("project_file_sync_state")
+    for column, example in (
+        ("project_uid", ""),
+        ("path", ""),
+        ("db_epoch", 0.0),
+        ("fs_epoch", 0.0),
+    ):
+        if not project_file_sync_state.has_column(column):
+            project_file_sync_state.create_column_by_example(column, example)
+    _index(
+        db,
+        "project_file_sync_state",
+        "idx_project_file_sync_state_path",
+        ["project_uid", "path"],
+        unique=True,
+    )
     _index(db, "badges", "idx_badges_user", ["user_uid"])
     _index(db, "badges", "idx_badges_user_name", ["user_uid", "badge_name"])
     _drop_index(db, "idx_follows_follower")
@@ -510,6 +596,51 @@ def init_db():
         "idx_user_cust_scope",
         ["owner_kind", "owner_id", "scope", "lang"],
     )
+    gateway_usage_ledger = get_table("gateway_usage_ledger")
+    for column, example in (
+        ("created_at", ""),
+        ("owner_kind", ""),
+        ("owner_id", ""),
+        ("backend", ""),
+        ("endpoint", ""),
+        ("requested_model", ""),
+        ("model", ""),
+        ("status_code", 0),
+        ("success", 0),
+        ("error_category", ""),
+        ("upstream_latency_ms", 0.0),
+        ("gateway_overhead_ms", 0.0),
+        ("queue_wait_ms", 0.0),
+        ("connect_ms", 0.0),
+        ("total_latency_ms", 0.0),
+        ("prompt_tokens", 0),
+        ("completion_tokens", 0),
+        ("cache_hit_tokens", 0),
+        ("cache_miss_tokens", 0),
+        ("reasoning_tokens", 0),
+        ("total_tokens", 0),
+        ("tokens_per_second", 0.0),
+        ("context_window", 0),
+        ("context_utilization", 0.0),
+        ("cost_usd", 0.0),
+        ("input_cost_usd", 0.0),
+        ("output_cost_usd", 0.0),
+        ("native_cost", 0),
+        ("stream_requested", 0),
+        ("temperature", 0.0),
+        ("top_p", 0.0),
+        ("max_tokens", 0),
+        ("has_tools", 0),
+        ("retries_attempted", 0),
+        ("retry_succeeded", 0),
+        ("circuit_open", 0),
+        ("user_agent", ""),
+        ("app_reference", ""),
+        ("ttft_ms", 0.0),
+        ("inter_token_ms", 0.0),
+    ):
+        if not gateway_usage_ledger.has_column(column):
+            gateway_usage_ledger.create_column_by_example(column, example)
     _index(db, "gateway_usage_ledger", "idx_gw_usage_time", ["created_at"])
     _index(
         db,
@@ -702,7 +833,9 @@ def init_db():
     _index(db, "instances", "idx_instances_workspace", ["is_workspace", "status"])
     _index(db, "instances", "idx_instances_workspace_owner", ["workspace_owner_uid"])
     _index(db, "instances", "idx_instances_tunnel_name", ["tunnel_name"])
-    _index(db, "tunnels", "idx_tunnels_hostname", ["hostname"])
+    _dedupe_tunnel_hostnames()
+    _drop_index(db, "idx_tunnels_hostname")
+    _index(db, "tunnels", "idx_tunnels_hostname", ["hostname"], unique=True)
     _index(db, "tunnels", "idx_tunnels_instance", ["instance_uid"])
     _index(db, "tunnels", "idx_tunnels_user", ["user_uid"])
     _index(db, "tunnels", "idx_tunnels_state", ["desired_state", "status"])
@@ -1580,6 +1713,98 @@ def init_db():
     )
     _index(db, "quiz_answers", "idx_quiz_answers_quiz", ["quiz_uid"])
 
+    opinion_wars = get_table("opinion_wars")
+    for column, example in (
+        ("uid", ""),
+        ("post_uid", ""),
+        ("user_uid", ""),
+        ("faction_a", ""),
+        ("faction_b", ""),
+        ("hp_a", 0),
+        ("hp_b", 0),
+        ("leader", ""),
+        ("status", "active"),
+        ("winner", ""),
+        ("created_at", ""),
+        ("ends_at", ""),
+        ("resolved_at", ""),
+        ("updated_at", ""),
+        ("deleted_at", ""),
+        ("deleted_by", ""),
+    ):
+        if not opinion_wars.has_column(column):
+            opinion_wars.create_column_by_example(column, example)
+    _index(db, "opinion_wars", "idx_opinion_wars_post", ["post_uid"], unique=True)
+    _index(db, "opinion_wars", "idx_opinion_wars_status_ends", ["status", "ends_at"])
+    _index(
+        db,
+        "opinion_wars",
+        "idx_opinion_wars_live_created",
+        ["created_at"],
+        where="deleted_at IS NULL",
+    )
+
+    opinion_war_fighters = get_table("opinion_war_fighters")
+    for column, example in (
+        ("uid", ""),
+        ("war_uid", ""),
+        ("user_uid", ""),
+        ("faction", ""),
+        ("hp_a", 0),
+        ("hp_b", 0),
+        ("fight_count", 0),
+        ("last_fight_at", ""),
+        ("cooldown_notified_at", ""),
+        ("created_at", ""),
+        ("updated_at", ""),
+        ("deleted_at", ""),
+        ("deleted_by", ""),
+    ):
+        if not opinion_war_fighters.has_column(column):
+            opinion_war_fighters.create_column_by_example(column, example)
+    _index(
+        db,
+        "opinion_war_fighters",
+        "idx_opinion_war_fighters_war_user",
+        ["war_uid", "user_uid"],
+        unique=True,
+    )
+    _index(
+        db,
+        "opinion_war_fighters",
+        "idx_opinion_war_fighters_war_faction",
+        ["war_uid", "faction"],
+    )
+    _index(
+        db,
+        "opinion_war_fighters",
+        "idx_opinion_war_fighters_last_fight",
+        ["last_fight_at"],
+    )
+
+    opinion_war_events = get_table("opinion_war_events")
+    for column, example in (
+        ("uid", ""),
+        ("war_uid", ""),
+        ("seq", 0),
+        ("kind", ""),
+        ("message", ""),
+        ("payload", ""),
+        ("actor_uid", ""),
+        ("created_at", ""),
+        ("deleted_at", ""),
+        ("deleted_by", ""),
+    ):
+        if not opinion_war_events.has_column(column):
+            opinion_war_events.create_column_by_example(column, example)
+    _index(
+        db,
+        "opinion_war_events",
+        "idx_opinion_war_events_war_seq",
+        ["war_uid", "seq"],
+        unique=True,
+    )
+
     _index(db, "posts", "idx_posts_user_created", ["user_uid", "created_at"])
     _index(
         db,
@@ -1706,8 +1931,8 @@ def init_db():
         news_defaults = {
             "news_grade_threshold": "7",
             "news_api_url": "https://news.app.molodetz.nl/api",
-            "news_ai_url": INTERNAL_GATEWAY_URL,
-            "news_ai_model": "molodetz",
+            "news_ai_url": AQUALITY_NEWS_GRADING_URL,
+            "news_ai_model": AQUALITY_NEWS_GRADING_MODEL,
         }
         for key, value in news_defaults.items():
             existing = db["site_settings"].find_one(key=key)
@@ -1739,6 +1964,7 @@ def init_db():
             "maintenance_message": "DevPlace is undergoing scheduled maintenance. Please check back shortly.",
             "customization_enabled": "1",
             "customization_js_enabled": "1",
+            "happy_404_enabled": "1",
             "audit_log_retention_days": "90",
             "statistics_tracking_enabled": "1",
             "docs_search_mode": "agent",
@@ -1953,6 +2179,12 @@ def migrate_ai_gateway_settings() -> None:
         if get_setting(key, "") == OLD_GATEWAY_URL:
             set_setting(key, INTERNAL_GATEWAY_URL)
             logger.info(f"Migrated {key} to the internal gateway")
+    if get_setting("news_ai_url", "") == INTERNAL_GATEWAY_URL:
+        set_setting("news_ai_url", AQUALITY_NEWS_GRADING_URL)
+        logger.info("Migrated news_ai_url to the free aquality grading model")
+    if get_setting("news_ai_model", "") in ("molodetz", ""):
+        set_setting("news_ai_model", AQUALITY_NEWS_GRADING_MODEL)
+        logger.info("Migrated news_ai_model to aquality")
     if get_setting("bot_model", "") == "deepseek-chat":
         set_setting("bot_model", "molodetz")
         logger.info("Migrated bot_model to molodetz")
@@ -1967,6 +2199,26 @@ def migrate_ai_gateway_settings() -> None:
     migrate_retired_image_gateway()
 
 
+def _ensure_users_uid() -> int:
+    if "users" not in db.tables:
+        return 0
+    users = get_table("users")
+    if not users.has_column("uid"):
+        users.create_column_by_example("uid", "")
+    import uuid_utils
+
+    updated = 0
+    for user in users.find():
+        if not user.get("uid"):
+            users.update(
+                {"id": user["id"], "uid": str(uuid_utils.uuid7())}, ["id"]
+            )
+            updated += 1
+    if updated:
+        logger.info("Backfilled uid for %s user(s)", updated)
+    return updated
+
+
 def backfill_api_keys() -> int:
     users = get_table("users")
     if not users.has_column("api_key"):

devplacepy/database/soft_delete.py

@@ -51,6 +51,9 @@ SOFT_DELETE_TABLES = [
     "quiz_options",
     "quiz_attempts",
     "quiz_answers",
+    "opinion_wars",
+    "opinion_war_fighters",
+    "opinion_war_events",
     "content_reports",
     "moderation_actions",
     "content_maturity",

devplacepy/database/users.py

@@ -15,8 +15,6 @@ def get_users_by_uids(uids):
 
 
 _admins_cache = TTLCache(ttl=300, max_size=4)
-# The primary administrator must be an account that can actually authenticate, so scan a
-# few of the earliest admins and skip any that are soft-deleted or deactivated.
 PRIMARY_ADMIN_CANDIDATES = 50
 
 
@@ -32,6 +30,9 @@ def get_admin_uids():
         return list(cached)
     if "users" not in db.tables:
         return []
+    users = db["users"]
+    if "uid" not in users.columns or "role" not in users.columns:
+        return []
     rows = db.query("SELECT uid FROM users WHERE role = 'Admin'")
     uids = [row["uid"] for row in rows]
     _admins_cache.set("uids", uids)
@@ -92,6 +93,9 @@ def get_primary_admin_uid():
         return cached or None
     if "users" not in db.tables:
         return None
+    users = db["users"]
+    if "uid" not in users.columns or "role" not in users.columns:
+        return None
     rows = list(
         db.query(
             "SELECT * FROM users WHERE role = 'Admin' "

devplacepy/dependencies.py

@@ -1,9 +1,4 @@
 # retoor <retoor@molodetz.nl>
-"""
-Generic FastAPI dependency that accepts JSON or form-encoded data,
-validated against a Pydantic model.
-"""
-
 import json
 import logging
 from typing import Any, TypeVar, get_origin
@@ -17,21 +12,10 @@ logger = logging.getLogger(__name__)
 
 _TModel = TypeVar("_TModel", bound=BaseModel)
 
-# Container origins recognised as sequence fields that may receive
-# multiple values from form data.
 _SEQUENCE_ORIGINS = frozenset({list, set, tuple, frozenset})
 
 
 def _formdata_to_dict(form: FormData, model: type[BaseModel]) -> dict[str, Any]:
-    """Convert FormData to a dict suitable for Pydantic validation.
-
-    * Sequence-typed model fields collect every submitted value via
-      ``getlist()``; a lone empty string is dropped (browsers emit empty
-      hidden inputs by default).
-    * Scalar fields use ``get()`` (the last value).
-    * Fields absent from the form are omitted so that Pydantic applies
-      the model default.
-    """
     body: dict[str, Any] = {}
     for field_name, field_info in model.model_fields.items():
         origin = get_origin(field_info.annotation)
@@ -50,8 +34,6 @@ def _formdata_to_dict(form: FormData, model: type[BaseModel]) -> dict[str, Any]:
 
 
 class _JsonOrForm:
-    """Internal callable that parses JSON or form data and validates."""
-
     def __init__(self, model: type[BaseModel]):
         self.model = model
 
@@ -70,7 +52,6 @@ class _JsonOrForm:
                         status_code=400, detail="JSON body must be an object"
                     )
                 return self.model.model_validate(body)
-            # Default: form-encoded (multipart or url-encoded)
             try:
                 form = await request.form()
             except Exception as exc:
@@ -85,11 +66,4 @@ class _JsonOrForm:
 
 
 def json_or_form(model: type[_TModel]) -> _JsonOrForm:
-    """Dependency factory: accept JSON or form-encoded data for a Pydantic model.
-
-    Usage:
-        @router.post("/create")
-        async def create(data: Annotated[PostForm, Depends(json_or_form(PostForm))]):
-            ...
-    """
     return _JsonOrForm(model)

devplacepy/docs_api/groups/__init__.py

@@ -22,6 +22,7 @@ from . import (
     admin,
     game,
     quizzes,
+    battles,
 )
 
 ORDERED_GROUPS = [
@@ -46,4 +47,5 @@ ORDERED_GROUPS = [
     admin.GROUP,
     game.GROUP,
     quizzes.GROUP,
+    battles.GROUP,
 ]

devplacepy/docs_api/groups/admin.py

@@ -265,6 +265,29 @@ four ways to sign requests.
                     ),
                 ],
             ),
+            endpoint(
+                id="admin-issues-planning",
+                method="GET",
+                path="/admin/issues/planning",
+                title="Ticket planning report",
+                summary=(
+                    "Admin page listing every open Gitea ticket so an admin can pick a subset "
+                    "and generate a grouped, ordered planning document for a coding agent."
+                ),
+                auth="admin",
+                interactive=False,
+                sample_response={
+                    "configured": True,
+                    "tickets": [
+                        {"number": 42, "title": "Fix login redirect", "labels": ["bug"]},
+                    ],
+                    "tickets_error": False,
+                },
+                notes=[
+                    "`configured` is false when the issue tracker (Gitea) has not been set up in Services yet - `tickets` is then empty.",
+                    "`tickets_error` is true when the tracker is configured but the live fetch failed - retry shortly.",
+                ],
+            ),
             endpoint(
                 id="admin-ai-usage",
                 method="GET",
@@ -300,7 +323,7 @@ four ways to sign requests.
                     ),
                 ],
                 notes=[
-                    "TTFT and inter-token latency are not reported: the gateway forwards non-streaming to the upstream."
+                    "TTFT and inter-token latency (latency.ttft_ms/latency.inter_token_ms) are measured only for calls that requested streaming (stream: true); a zero count means no streaming calls occurred in the window, not that the metric is unavailable."
                 ],
             ),
             endpoint(
@@ -613,6 +636,85 @@ four ways to sign requests.
                 auth="admin",
                 destructive=True,
             ),
+            endpoint(
+                id="admin-gateway-page",
+                method="GET",
+                path="/admin/gateway",
+                title="Gateway routing dashboard",
+                summary="Admin HTML page for managing AI gateway providers and per-model routing.",
+                auth="admin",
+                interactive=False,
+            ),
+            endpoint(
+                id="admin-gateway-providers",
+                method="GET",
+                path="/admin/gateway/providers",
+                title="List AI gateway providers",
+                summary="List every configured upstream provider plus the default provider summary.",
+                auth="admin",
+                interactive=True,
+            ),
+            endpoint(
+                id="admin-gateway-provider-set",
+                method="POST",
+                path="/admin/gateway/providers",
+                title="Create or update an AI gateway provider",
+                summary="Save an upstream provider (base URL, model, and API key) by name. Pass name to update an existing provider.",
+                auth="admin",
+                params=[
+                    field("name", "json", "string", True, "openrouter", "Provider name; existing name updates in place."),
+                    field("base_url", "json", "string", True, "https://openrouter.ai/api/v1", "Upstream chat completions base URL."),
+                    field("model", "json", "string", True, "x-ai/grok-4.3", "Default model for this provider."),
+                    field("api_key", "json", "string", False, "", "Upstream API key. Blank keeps the current key."),
+                ],
+            ),
+            endpoint(
+                id="admin-gateway-provider-delete",
+                method="DELETE",
+                path="/admin/gateway/providers/{name}",
+                title="Delete an AI gateway provider",
+                summary="Delete a configured upstream provider by name.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("name", "path", "string", True, "openrouter", "Provider name."),
+                ],
+            ),
+            endpoint(
+                id="admin-gateway-models",
+                method="GET",
+                path="/admin/gateway/models",
+                title="List AI gateway model routes",
+                summary="List every source-to-target model route plus the configured provider names.",
+                auth="admin",
+                interactive=True,
+            ),
+            endpoint(
+                id="admin-gateway-model-set",
+                method="POST",
+                path="/admin/gateway/models",
+                title="Create or update an AI gateway model route",
+                summary="Route a source model name to a target model, optionally on a specific provider. Pass source_model to update an existing route.",
+                auth="admin",
+                params=[
+                    field("source_model", "json", "string", True, "gpt-4", "Model name callers request."),
+                    field("target_model", "json", "string", True, "x-ai/grok-4.3", "Model actually sent upstream."),
+                    field("provider", "json", "string", False, "openrouter", "Provider name to route through. Blank = the default provider."),
+                    field("fallback_model", "json", "string", False, "molodetz-pro", "Another already-configured source_model of the same kind, tried once automatically when this route fails after its own retries are exhausted. Blank = no fallback."),
+                ],
+            ),
+            endpoint(
+                id="admin-gateway-model-delete",
+                method="DELETE",
+                path="/admin/gateway/models/{source_model}",
+                title="Delete an AI gateway model route",
+                summary="Delete a source-to-target model route by source model name.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("source_model", "path", "string", True, "gpt-4", "Source model name."),
+                ],
+            ),
             endpoint(
                 id="admin-gateway-quota-rules",
                 method="GET",
@@ -975,5 +1077,190 @@ four ways to sign requests.
                 destructive=True,
                 sample_response={"ok": True, "redirect": "/admin/game"},
             ),
+            endpoint(
+                id="admin-workspaces-page",
+                method="GET",
+                path="/admin/workspaces",
+                title="Workspaces dashboard",
+                summary="Admin HTML page listing every dev workspace across all projects, with owner, project, and moderation flags.",
+                auth="admin",
+                interactive=False,
+            ),
+            endpoint(
+                id="admin-workspaces-data",
+                method="GET",
+                path="/admin/workspaces/data",
+                title="List workspaces",
+                summary="Every dev workspace across all projects plus open moderation flags, as JSON.",
+                auth="admin",
+                interactive=True,
+                sample_response={"workspaces": [], "flags": []},
+            ),
+            endpoint(
+                id="admin-workspaces-suspend",
+                method="POST",
+                path="/admin/workspaces/{uid}/suspend",
+                title="Suspend a workspace",
+                summary="Suspend a workspace with a reason shown to its owner. The owner is notified.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                    field("reason", "form", "string", True, "Excessive resource usage", "Shown to the workspace owner."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-unsuspend",
+                method="POST",
+                path="/admin/workspaces/{uid}/unsuspend",
+                title="Unsuspend a workspace",
+                summary="Lift a suspension. The owner is notified the workspace is available again.",
+                auth="admin",
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-stop",
+                method="POST",
+                path="/admin/workspaces/{uid}/stop",
+                title="Stop a workspace",
+                summary="Stop the workspace container. Files and tunnels are kept.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-start",
+                method="POST",
+                path="/admin/workspaces/{uid}/start",
+                title="Start a workspace",
+                summary="Resume a stopped workspace container.",
+                auth="admin",
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-delete",
+                method="POST",
+                path="/admin/workspaces/{uid}/delete",
+                title="Delete a workspace",
+                summary="Remove a workspace and its tunnels.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-flag",
+                method="POST",
+                path="/admin/workspaces/{uid}/flag",
+                title="Flag a workspace",
+                summary="Raise a moderation flag on a workspace. The owner is notified.",
+                auth="admin",
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                    field("kind", "form", "string", False, "manual", "Flag kind."),
+                    field("severity", "form", "string", False, "warn", "Flag severity."),
+                    field("detail", "form", "string", False, "", "Detail shown to the owner."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-flag-resolve",
+                method="POST",
+                path="/admin/workspaces/flags/{flag_uid}/resolve",
+                title="Resolve or dismiss a workspace flag",
+                summary="Set a moderation flag's status.",
+                auth="admin",
+                params=[
+                    field("flag_uid", "path", "string", True, "FLAG_UID", "Flag uid."),
+                    field("status", "query", "string", False, "resolved", "resolved or dismissed."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-editor",
+                method="POST",
+                path="/admin/workspaces/{uid}/editor",
+                title="Set a workspace owner's editor preferences",
+                summary="Change the workspace owner's editor preferences on their behalf, or reset them. Subject to admin seniority.",
+                auth="admin",
+                params=[
+                    field("uid", "path", "string", True, "INSTANCE_UID", "Workspace instance uid."),
+                    field("theme", "form", "string", False, "devplace-dark", "devplace-dark, devplace-light or system."),
+                    field("layout", "form", "string", False, "standard", "standard, terminal-focus or zen."),
+                    field("panel_preset", "form", "string", False, "tall", "short, normal, tall or maximized."),
+                    field("font_size", "form", "integer", False, "14", "Editor font size in pixels."),
+                    field("terminal_font_size", "form", "integer", False, "13", "Terminal font size in pixels."),
+                    field("zoom_level", "form", "integer", False, "0", "Window zoom, -5 to 5."),
+                    field("boot_agent", "form", "string", False, "dpc", "dpc or none."),
+                    field("boot_shell", "form", "integer", False, "1", "1 opens a shell on boot, 0 skips it."),
+                    field("window_mode", "form", "string", False, "tab", "tab, window or fullscreen."),
+                    field("window_width", "form", "integer", False, "1600", "Editor window width in pixels."),
+                    field("window_height", "form", "integer", False, "1000", "Editor window height in pixels."),
+                    field("reset", "form", "boolean", False, "false", "Drop every preference for this owner."),
+                ],
+            ),
+            endpoint(
+                id="admin-workspaces-quota",
+                method="POST",
+                path="/admin/workspaces/quota",
+                title="Set a user's workspace quota override",
+                summary="Set or update a per-user override of the workspace count/disk/egress/idle/retention/CPU/memory limits.",
+                auth="admin",
+                params=[
+                    field("owner_id", "form", "string", True, "USER_UID", "Target user uid."),
+                    field("label", "form", "string", False, "", "Optional admin-facing note."),
+                    field("max_workspaces", "form", "int", False, "0", "0 = use the site default."),
+                    field("max_tunnels", "form", "int", False, "0", "0 = use the site default."),
+                    field("disk_quota_mb", "form", "int", False, "0", "0 = use the site default."),
+                    field("egress_quota_mb", "form", "int", False, "0", "0 = use the site default."),
+                    field("idle_stop_minutes", "form", "int", False, "0", "0 = use the site default."),
+                    field("retention_days", "form", "int", False, "0", "0 = use the site default."),
+                    field("cpu_millicores", "form", "int", False, "0", "0 = use the site default."),
+                    field("memory_mb", "form", "int", False, "0", "0 = use the site default."),
+                ],
+            ),
+            endpoint(
+                id="admin-trash-list",
+                method="GET",
+                path="/admin/trash",
+                title="Trash",
+                summary="Admin HTML page listing soft-deleted rows for one table, with restore/purge controls per row.",
+                auth="admin",
+                interactive=True,
+                params=[
+                    field("table", "query", "string", False, "posts", "Trash table key (posts, comments, gists, projects, news, awards, quizzes, project_files, attachments)."),
+                    field("page", "query", "int", False, "1", "Page number."),
+                ],
+            ),
+            endpoint(
+                id="admin-trash-restore",
+                method="POST",
+                path="/admin/trash/{table}/{uid}/restore",
+                title="Restore a soft-deleted row",
+                summary="Restore every row soft-deleted under the same event timestamp as the given row (a whole delete cascade at once).",
+                auth="admin",
+                params=[
+                    field("table", "path", "string", True, "posts", "Trash table key."),
+                    field("uid", "path", "string", True, "ROW_UID", "Row uid."),
+                ],
+            ),
+            endpoint(
+                id="admin-trash-purge",
+                method="POST",
+                path="/admin/trash/{table}/{uid}/purge",
+                title="Purge a soft-deleted row",
+                summary="Permanently delete every row soft-deleted under the same event timestamp as the given row, unlinking any attachment/project-file blobs.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("table", "path", "string", True, "posts", "Trash table key."),
+                    field("uid", "path", "string", True, "ROW_UID", "Row uid."),
+                ],
+            ),
         ],
 }

devplacepy/docs_api/groups/battles.py

@@ -0,0 +1,149 @@
+# retoor <retoor@molodetz.nl>
+
+from devplacepy.services.opinionwar import rules
+
+from .._shared import endpoint, field
+
+FILTER_KEYS = ["active", "ended", "mine"]
+
+SAMPLE_WAR = {
+    "uid": "0198f2c0-2222-7aaa-8bbb-000000000002",
+    "post_uid": "0198f2c0-3333-7aaa-8bbb-000000000003",
+    "post_url": "/posts/8bbb000000000003-tabs-or-spaces",
+    "post_title": "Tabs or spaces?",
+    "faction_a": "Tabs",
+    "faction_b": "Spaces",
+    "hp_a": 12548,
+    "hp_b": 7362,
+    "pct_a": 63,
+    "pct_b": 37,
+    "leader": "a",
+    "fighter_count": 42,
+    "status": "active",
+    "winner": "",
+    "ends_at": "2026-08-27T12:00:00+00:00",
+    "ends_in": "2d 14h 32m",
+    "last_seq": 87,
+    "fight_cost": rules.FIGHT_COST_COINS,
+    "top_contributors": [
+        {"username": "code_warrior", "faction": "a", "hp": 982},
+    ],
+    "recent_events": [
+        {"seq": 87, "kind": "fight", "message": "code_warrior dealt 300 HP for Tabs", "faction": "a"},
+    ],
+    "viewer": {
+        "faction": "a",
+        "hp": 256,
+        "rank": 7,
+        "can_fight": True,
+        "next_fight_at": "",
+    },
+}
+
+GROUP = {
+    "slug": "battles",
+    "title": "Opinion Wars",
+    "intro": f"""
+# Opinion Wars
+
+An Opinion War is a week-long two-faction battle attached to a post. The creator names
+exactly two factions when creating the post (the `war_faction_a` / `war_faction_b` fields
+on `POST /posts/create`); from that moment the battle runs for exactly
+{rules.WAR_DURATION_DAYS} days.
+
+Any signed-in member joins one of the two factions and may **fight** once every
+{rules.FIGHT_COOLDOWN_HOURS} hours per battle. A fight costs {rules.FIGHT_COST_COINS}
+Code Farm coins and deals deterministic, level-weighted damage for the fighter's faction:
+`{rules.BASE_DAMAGE} + {rules.LEVEL_DAMAGE_STEP} * min(level, {rules.LEVEL_DAMAGE_CAP})`
+HP, so a level 1 member deals {rules.damage_for(1)} HP and the bonus caps at
+{rules.damage_for(rules.LEVEL_DAMAGE_CAP)} HP. There is no randomness. Switching factions
+is allowed at any time; damage already dealt stays with the faction it was dealt to.
+
+When the week is over the faction with more HP wins. Resolution is evaluated lazily on
+read (no background clock): the first read after the deadline freezes the totals, awards
+XP (participation for every fighter with at least one fight, a bonus for the winning
+side, a bonus for the single top damage dealer) and notifies every fighter. Equal totals
+are a draw with participation XP only.
+
+Every battle keeps an ordered event log (kinds `join`, `switch`, `fight`, `lead_change`,
+`result`) replayable with the `after` cursor; live frames are also published on the
+pub/sub topic `public.battle.{{uid}}`.
+
+All endpoints negotiate HTML or JSON. POST bodies are form encoded. Action POSTs answer
+`{{"ok": true, "redirect": "...", "data": {{...}}}}`; a refused action (cooldown, missing
+coins, ended battle) answers `400` as `{{"error": {{"status": 400, "message": "..."}}}}`.
+""",
+    "endpoints": [
+        endpoint(
+            id="battles-list",
+            method="GET",
+            path="/battles",
+            title="Battle listing",
+            summary="Opinion Wars with HP totals, filter counts and the viewer's faction state.",
+            auth="public",
+            negotiation=True,
+            params=[
+                field("search", "query", "string", False, "tabs", "Match a faction name or the creator's username."),
+                field("filter", "query", "enum", False, "active", "Which battles to list.", options=FILTER_KEYS),
+                field("page", "query", "integer", False, "1", "1-based page number."),
+            ],
+            sample_response={"battles": [SAMPLE_WAR], "counts": {"active": 3, "ended": 12, "mine": 1}},
+        ),
+        endpoint(
+            id="battles-get",
+            method="GET",
+            path="/battles/{uid}",
+            title="Battle state",
+            summary="One battle's full serialized state, resolving it first when its week is over.",
+            auth="public",
+            params=[
+                field("uid", "path", "string", True, SAMPLE_WAR["uid"], "Battle uid."),
+            ],
+            sample_response=SAMPLE_WAR,
+        ),
+        endpoint(
+            id="battles-events",
+            method="GET",
+            path="/battles/{uid}/events",
+            title="Battle events",
+            summary="The ordered battle event log, replayable incrementally with the after cursor.",
+            auth="public",
+            params=[
+                field("uid", "path", "string", True, SAMPLE_WAR["uid"], "Battle uid."),
+                field("after", "query", "integer", False, "0", "Return only events with a seq greater than this."),
+                field("limit", "query", "integer", False, "500", "Maximum events to return."),
+            ],
+            sample_response={"events": SAMPLE_WAR["recent_events"], "status": "active"},
+        ),
+        endpoint(
+            id="battles-join",
+            method="POST",
+            path="/battles/{uid}/join",
+            title="Join or switch faction",
+            summary="Join faction a or b, or switch an existing fighter to the other side.",
+            auth="user",
+            encoding="form",
+            params=[
+                field("uid", "path", "string", True, SAMPLE_WAR["uid"], "Battle uid."),
+                field("faction", "form", "enum", True, "a", "Which side to join or switch to.", options=["a", "b"]),
+            ],
+            sample_response={"ok": True, "redirect": SAMPLE_WAR["post_url"], "data": {"war": SAMPLE_WAR}},
+        ),
+        endpoint(
+            id="battles-fight",
+            method="POST",
+            path="/battles/{uid}/fight",
+            title="Fight",
+            summary=(
+                f"Spend {rules.FIGHT_COST_COINS} Code Farm coins and deal level-weighted HP damage "
+                f"for your faction. Once per {rules.FIGHT_COOLDOWN_HOURS} hours per battle."
+            ),
+            auth="user",
+            encoding="form",
+            params=[
+                field("uid", "path", "string", True, SAMPLE_WAR["uid"], "Battle uid."),
+            ],
+            sample_response={"ok": True, "redirect": SAMPLE_WAR["post_url"], "data": {"war": SAMPLE_WAR, "damage": 300}},
+        ),
+    ],
+}

devplacepy/docs_api/groups/content.py

@@ -64,6 +64,33 @@ four ways to sign requests.
                     field("before", "query", "string", False, "", "Pagination cursor."),
                 ],
             ),
+            endpoint(
+                id="topics-hub",
+                method="GET",
+                path="/topics",
+                title="Browse topics",
+                summary="The topics hub - every post topic with its live post count, linking to its own crawlable listing page.",
+                auth="public",
+                interactive=True,
+            ),
+            endpoint(
+                id="topics-list",
+                method="GET",
+                path="/topics/{topic}",
+                title="Browse one topic",
+                summary=(
+                    "A single topic's post listing, on its own permanent, crawlable URL (unlike /feed?topic=, "
+                    "whose canonical collapses back to /feed). Same author-interleaved pagination as the feed."
+                ),
+                auth="public",
+                interactive=True,
+                params=[
+                    field(
+                        "topic", "path", "enum", True, "devlog", "Topic key.", TOPICS
+                    ),
+                    field("before", "query", "string", False, "", "Pagination cursor."),
+                ],
+            ),
             endpoint(
                 id="posts-create",
                 method="POST",
@@ -117,6 +144,22 @@ four ways to sign requests.
                         "",
                         "Repeat the field for each poll option, or send a single newline- or comma-separated string (2-6 options).",
                     ),
+                    field(
+                        "war_faction_a",
+                        "form",
+                        "string",
+                        False,
+                        "",
+                        "Optional Opinion War faction A name (max 30 chars). Both faction names start the week-long battle.",
+                    ),
+                    field(
+                        "war_faction_b",
+                        "form",
+                        "string",
+                        False,
+                        "",
+                        "Optional Opinion War faction B name (max 30 chars). Must differ from faction A.",
+                    ),
                 ],
                 notes=["Returns a `302` redirect to `/posts/{slug}` on success."],
             ),

devplacepy/docs_api/groups/gateway.py

@@ -77,9 +77,16 @@ managed by administrators on the **Gateway** page (`/admin/gateway`).
 
 ## Per-call cost and usage headers
 
-Every gateway response - chat, embeddings, and passthrough, on both success and error - carries
-`X-Gateway-*` response headers describing that single call, so a client can read its own token usage
-and dollar cost directly from the response with no extra request:
+Every NON-STREAMING gateway response - chat, embeddings, images, and passthrough, on both success
+and error - carries `X-Gateway-*` response headers describing that single call, so a client can read
+its own token usage and dollar cost directly from the response with no extra request. A streaming
+chat response (`"stream": true`) is the one exception: it carries only `X-Gateway-Model`,
+`X-Gateway-Backend`, and `X-App-Reference` - HTTP headers must be sent before the body, and cost/token
+counts for a streamed call are only known once the stream ends, so they cannot be response headers on
+that same response. The call is still fully metered server-side, and a client that sends
+`"stream_options": {"include_usage": true}` still receives the upstream's real `usage` object on the
+final SSE chunk, exactly as the underlying provider's own streaming API works - it is just not
+summarized into headers.
 
 | Header | Meaning |
 |--------|---------|
@@ -161,11 +168,20 @@ for signing DevPlace's own requests.
                         "false",
                         "Set true for a streamed SSE response.",
                     ),
+                    field(
+                        "think",
+                        "json",
+                        "string",
+                        False,
+                        "false",
+                        "Optional thinking override. Omitted: the gateway disables thinking (fast path). true / high / medium / low enables it; false disables it. DeepSeek-native `thinking.type` and OpenRouter `reasoning.effort` are also accepted.",
+                    ),
                 ],
                 notes=[
                     "Returns `503` when the gateway service is not running.",
-                    "Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above), including the streamed SSE response.",
+                    "A non-streaming response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above). A streaming response (`stream: true`) is forwarded from the upstream in real time and carries only `X-Gateway-Model`/`X-Gateway-Backend`/`X-App-Reference` - request `stream_options: {\"include_usage\": true}` to receive the real token usage on the final SSE chunk instead.",
                     "If `model` matches a configured model route it is forwarded to that route's provider, upstream model, and per-model pricing (with an optional vision model); otherwise it falls through to the default upstream (see Model routing and providers above).",
+                    "Thinking is disabled by default. Pass `think: true` (or `thinking: {\"type\": \"enabled\"}`) to turn it on for that call.",
                 ],
             ),
             endpoint(

devplacepy/docs_api/groups/messaging.py

@@ -41,6 +41,14 @@ four ways to sign requests.
                         "",
                         "Jump to a conversation by username.",
                     ),
+                    field(
+                        "before",
+                        "query",
+                        "string",
+                        False,
+                        "",
+                        "ISO timestamp. When set, return the page of messages strictly older than this instant (for loading earlier history).",
+                    ),
                 ],
             ),
             endpoint(

devplacepy/docs_api/groups/push.py

@@ -14,9 +14,11 @@ implements the Web Push protocol: fetch the public VAPID key, then register a
 Push Notification service device token and is only offered when an administrator has
 configured it.
 
-`GET /push.json` lists the providers that currently accept registrations. A registration
-body without a `provider` field is a `webpush` registration, so existing clients need no
-change.
+`GET /push.json` lists the providers that currently accept registrations. When `apns` is
+active it includes `environment` (`production` or `sandbox`) so a native client can match
+its build. A registration body without a `provider` field is a `webpush` registration, so
+existing clients need no change. An APNs body may include a stable `client_id` so a later
+token rotation updates the same device instead of inserting another row.
 
 There is no server-side unsubscribe endpoint: unsubscription is handled entirely in the
 browser by calling `PushManager.unsubscribe()` on the subscription. The server stops delivering
@@ -37,7 +39,10 @@ four ways to sign requests.
                 auth="public",
                 sample_response={
                     "publicKey": "BASE64_VAPID_KEY",
-                    "providers": {"webpush": {"publicKey": "BASE64_VAPID_KEY"}},
+                    "providers": {
+                        "webpush": {"publicKey": "BASE64_VAPID_KEY"},
+                        "apns": {"environment": "production"},
+                    },
                 },
             ),
             endpoint(
@@ -80,15 +85,24 @@ four ways to sign requests.
                         "string",
                         False,
                         "a1b2c3...",
-                        "Hexadecimal device token. Required for apns.",
+                        "Hexadecimal device token. Required for apns. Spaces and angle brackets are stripped.",
+                    ),
+                    field(
+                        "client_id",
+                        "json",
+                        "string",
+                        False,
+                        "vendor-uuid",
+                        "Stable per-device id for apns. When present, a new token updates this device instead of inserting a row.",
                     ),
                 ],
                 notes=[
                     'A webpush body is JSON: `{"endpoint": "...", "keys": {"p256dh": "...", "auth": "..."}}`.',
-                    'An APNs body is JSON: `{"provider": "apns", "token": "..."}`.',
+                    'An APNs body is JSON: `{"provider": "apns", "token": "...", "client_id": "..."}`. `client_id` is optional; token-only bodies keep working and revive a previously dead token.',
                     "A provider that is unknown, disabled or unconfigured returns 400.",
+                    "A newly created or revived registration is probed immediately. The response then includes `delivered` and, on failure, `error` with the provider reason. `registered` stays true so existing clients keep working.",
                 ],
-                sample_response={"registered": True},
+                sample_response={"registered": True, "delivered": True},
             ),
         ],
 }

devplacepy/docs_api/groups/tools.py

@@ -226,6 +226,7 @@ status and report.
                         {"title": "Invention", "detail": "...", "confidence": 0.8, "citations": [1]}
                     ],
                     "sources": [{"url": "https://example.com", "title": "Example", "source": "httpx"}],
+                    "follow_up_questions": ["Who else worked on the transistor?", "How did it replace vacuum tubes?"],
                     "chat_ws_url": "/tools/deepsearch/DEEPSEARCH_JOB_UID/chat",
                     "export_md_url": "/tools/deepsearch/DEEPSEARCH_JOB_UID/export.md",
                     "export_json_url": "/tools/deepsearch/DEEPSEARCH_JOB_UID/export.json",

devplacepy/docs_api/groups/workspaces.py

@@ -37,7 +37,10 @@ arrives as a `workspace` notification and states exactly what happens next and w
             title="Read workspace",
             summary=(
                 "State, quota usage, idle countdown, tunnels and open moderation flags "
-                "for your workspace on this project."
+                "for your workspace on this project. The phase (stopped, starting, ready, "
+                "stopping, crashed, suspended) is derived from the desired state, the "
+                "container status and a live probe of the editor port, so editor_ready "
+                "is true only when the editor will actually open."
             ),
             auth="user",
             params=[
@@ -51,7 +54,12 @@ arrives as a `workspace` notification and states exactly what happens next and w
                 "editor_url": "/projects/my-project/containers/instances/INSTANCE_UID/code/",
                 "workspace": {
                     "uid": "INSTANCE_UID",
+                    "owner_uid": "USER_UID",
                     "status": "running",
+                    "desired_state": "running",
+                    "phase": "ready",
+                    "phase_label": "Ready",
+                    "editor_ready": True,
                     "suspended": False,
                     "tunnel_name": "brave-otter",
                     "primary_url": "https://brave-otter.tunnel.pravda.education",
@@ -131,7 +139,7 @@ arrives as a `workspace` notification and states exactly what happens next and w
                     "terminal_font_size": 13,
                     "zoom_level": 0,
                     "layout": "standard",
-                    "panel_preset": "tall",
+                    "panel_preset": "normal",
                     "boot_agent": "dpc",
                     "boot_shell": True,
                     "window_mode": "tab",

devplacepy/docs_api/services_group.py

@@ -117,6 +117,25 @@ def build_services_group(services, base):
             )
         )
     control_endpoints = [
+        endpoint(
+            id="services-page",
+            method="GET",
+            path="/admin/services",
+            title="Services dashboard",
+            summary="Admin HTML index of every registered background service, its status, and controls.",
+            auth="admin",
+            interactive=False,
+        ),
+        endpoint(
+            id="services-detail-page",
+            method="GET",
+            path="/admin/services/{name}",
+            title="Service detail page",
+            summary="Admin HTML detail page for a single service: overview, configuration form, and logs.",
+            auth="admin",
+            interactive=False,
+            params=[field("name", "path", "string", True, "news", "Service name.")],
+        ),
         endpoint(
             id="services-data",
             method="GET",

devplacepy/happy404.py

@@ -0,0 +1,70 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+import logging
+import os
+import random
+
+from fastapi import Request
+from fastapi.responses import HTMLResponse
+
+from devplacepy.cache import TTLCache
+from devplacepy.content import load_detail
+from devplacepy.database import db, get_setting
+from devplacepy.routers.posts import post_page_context
+from devplacepy.templating import templates
+from devplacepy.utils import get_current_user
+
+logger = logging.getLogger("happy404")
+
+POOL_TTL_SECONDS = int(os.environ.get("DEVPLACE_HAPPY_404_POOL_TTL", "300"))
+POOL_SIZE = 100
+
+API_PATH_PREFIXES = ("/api", "/dbapi", "/openai", "/xmlrpc", "/swagger", "/openapi.json")
+
+_pool_cache = TTLCache(ttl=POOL_TTL_SECONDS, max_size=1)
+
+
+def _post_pool() -> list[str]:
+    cached = _pool_cache.get("slugs")
+    if cached is not None:
+        return cached
+    slugs: list[str] = []
+    if "posts" in db.tables:
+        rows = db.query(
+            "SELECT slug, uid FROM posts WHERE deleted_at IS NULL ORDER BY RANDOM() LIMIT :limit",
+            limit=POOL_SIZE,
+        )
+        slugs = [row["slug"] or row["uid"] for row in rows]
+    _pool_cache.set("slugs", slugs)
+    return slugs
+
+
+def _eligible(request: Request) -> bool:
+    if request.method != "GET":
+        return False
+    if request.url.path.startswith(API_PATH_PREFIXES):
+        return False
+    return get_setting("happy_404_enabled", "1") == "1"
+
+
+def render(request: Request) -> HTMLResponse | None:
+    try:
+        if not _eligible(request):
+            return None
+        pool = _post_pool()
+        if not pool:
+            return None
+        slug = random.choice(pool)
+        user = get_current_user(request)
+        detail = load_detail("posts", "post", slug, user)
+        if not detail:
+            return None
+        context = post_page_context(
+            request, user, detail, robots="noindex,nofollow"
+        )
+        return templates.TemplateResponse(request, "post.html", context)
+    except Exception as exc:  # noqa: BLE001 - a happy-404 bug must never break the 404 page
+        logger.warning("happy_404 render failed: %s", exc)
+        return None

devplacepy/main.py

@@ -33,7 +33,7 @@ from devplacepy.database import (
     get_news_images_by_uids,
     get_setting,
     get_int_setting,
-    interleave_by_author,
+    paginate_diverse,
     get_user_post_count,
     get_user_stars,
     get_blocked_uids,
@@ -43,14 +43,17 @@ from devplacepy.database import (
 from devplacepy.templating import templates, jinja_unread_count
 from devplacepy.cache import TTLCache
 from devplacepy.responses import respond, wants_json, json_error
+from devplacepy import happy404
 from devplacepy.schemas import LandingOut, ValidationErrorOut
 from fastapi.responses import JSONResponse
 from devplacepy.utils import get_current_user, time_ago, safe_next, client_ip
 from devplacepy.seo import base_seo_context, site_url, website_schema
 from devplacepy.routers import (
     auth,
+    battles,
     feed,
     posts,
+    topics,
     comments,
     projects,
     profile,
@@ -106,6 +109,7 @@ from devplacepy.services.backup import BackupService
 from devplacepy.services.dbapi.service import DbApiJobService
 from devplacepy.services.pubsub import PubSubService
 from devplacepy.services.notification_relay import NotificationRelayService
+from devplacepy.services.opinionwar.service import OpinionWarService
 from devplacepy.services.live_view_relay import LiveViewRelayService
 from devplacepy.services.presence_relay import PresenceRelayService
 from devplacepy.services import presence
@@ -272,6 +276,7 @@ async def lifespan(app: FastAPI):
     service_manager.register(DbApiJobService())
     service_manager.register(PubSubService())
     service_manager.register(NotificationRelayService())
+    service_manager.register(OpinionWarService())
     service_manager.register(LiveViewRelayService())
     service_manager.register(PresenceRelayService())
     service_manager.register(DeepsearchService())
@@ -312,6 +317,9 @@ async def lifespan(app: FastAPI):
     from devplacepy.services.containers import forward
 
     await forward.close_client()
+    from devplacepy import push
+
+    await push.shutdown_providers()
 
 
 app = FastAPI(
@@ -338,6 +346,9 @@ app.mount("/static", FallbackStaticFiles(directory=str(STATIC_DIR)), name="stati
 async def not_found(request: Request, exc):
     if wants_json(request):
         return json_error(404, "Not found")
+    happy_response = happy404.render(request)
+    if happy_response is not None:
+        return happy_response
     seo_ctx = base_seo_context(
         request,
         title="Not Found - DevPlace",
@@ -468,6 +479,7 @@ async def on_validation_error(request: Request, exc: RequestValidationError):
 app.include_router(auth.router, prefix="/auth")
 app.include_router(feed.router, prefix="/feed")
 app.include_router(posts.router, prefix="/posts")
+app.include_router(topics.router, prefix="/topics")
 app.include_router(comments.router, prefix="/comments")
 app.include_router(projects.router, prefix="/projects")
 app.include_router(profile.router, prefix="/profile")
@@ -504,6 +516,7 @@ app.include_router(dbapi.router, prefix="/dbapi")
 app.include_router(pubsub.router, prefix="/pubsub")
 app.include_router(game.router, prefix="/game")
 app.include_router(quizzes.router, prefix="/quizzes")
+app.include_router(battles.router, prefix="/battles")
 app.include_router(workspaces.router, prefix="/workspaces")
 
 
@@ -781,7 +794,8 @@ def _landing_news():
     return articles
 
 
-def _landing_recent_posts(blocked):
+def _landing_recent_posts(viewer_uid):
+    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
     if not blocked:
         cached = _home_cache.get("posts")
         if cached is not None:
@@ -789,15 +803,9 @@ def _landing_recent_posts(blocked):
     posts = []
     if "posts" in db.tables:
         posts_table = get_table("posts")
-        clauses = []
-        if blocked:
-            clauses.append(posts_table.table.columns.user_uid.notin_(blocked))
-        raw_posts = list(
-            posts_table.find(
-                *clauses, deleted_at=None, order_by=["-created_at"], _limit=6
-            )
+        raw_posts, _ = paginate_diverse(
+            posts_table, order=["-created_at"], viewer_uid=viewer_uid, limit=6
         )
-        raw_posts = interleave_by_author(raw_posts)
         if raw_posts:
             post_uids = [p["uid"] for p in raw_posts]
             author_uids = [p["user_uid"] for p in raw_posts]
@@ -825,8 +833,9 @@ async def landing(request: Request):
     user = get_current_user(request)
 
     landing_articles = _landing_news()
-    blocked = get_blocked_uids(user["uid"]) if user else frozenset()
-    landing_posts = _landing_recent_posts(blocked)
+    viewer_uid = user["uid"] if user else None
+    blocked = get_blocked_uids(viewer_uid) if viewer_uid else frozenset()
+    landing_posts = _landing_recent_posts(viewer_uid)
 
     base = site_url(request)
     seo_ctx = base_seo_context(

devplacepy/models.py

@@ -176,6 +176,8 @@ class PostForm(BaseModel):
     attachment_uids: list[str] = []
     poll_question: str = Field(default="", max_length=200)
     poll_options: list[str] = []
+    war_faction_a: str = Field(default="", max_length=30)
+    war_faction_b: str = Field(default="", max_length=30)
 
     @field_validator("poll_options")
     @classmethod
@@ -198,6 +200,17 @@ class PostForm(BaseModel):
         return normalize_poll_options(value)
 
 
+class WarJoinForm(BaseModel):
+    faction: str
+
+    @field_validator("faction")
+    @classmethod
+    def valid_faction(cls, value):
+        if value not in ("a", "b"):
+            raise ValueError("Faction must be a or b")
+        return value
+
+
 class PostEditForm(BaseModel):
     content: str = Field(min_length=10, max_length=125000)
     title: str = Field(default="", max_length=500)
@@ -668,6 +681,7 @@ class AdminSettingsForm(BaseModel):
     registration_open: str = Field(default="", max_length=1)
     maintenance_mode: str = Field(default="", max_length=1)
     maintenance_message: str = Field(default="", max_length=300)
+    happy_404_enabled: str = Field(default="", max_length=1)
     docs_search_mode: str = Field(default="", max_length=20)
     outbound_proxy_url: str = Field(default="", max_length=500)
     moderation_sla_hours: str = Field(default="", max_length=10)
@@ -682,6 +696,9 @@ class AdminSettingsForm(BaseModel):
     privacy_version: str = Field(default="", max_length=20)
     guidelines_version: str = Field(default="", max_length=20)
     ai_third_party_provider: str = Field(default="", max_length=120)
+    correction_model: str = Field(default="", max_length=120)
+    modifier_model: str = Field(default="", max_length=120)
+    quiz_grading_model: str = Field(default="", max_length=120)
     extra_head: str = Field(default="", max_length=50000)
 
     @field_validator("moderation_filter_mode")

devplacepy/project_files.py

@@ -1,6 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
 import logging
+import os
 import shutil
 from datetime import datetime, timezone
 from pathlib import Path
@@ -477,6 +478,7 @@ def delete_node(project_uid: str, raw_path: str, deleted_by: str = "system") ->
 def soft_delete_all_project_files(project_uid: str, deleted_by: str) -> None:
     if "project_files" not in db.tables:
         return
+    clear_sync_state(project_uid)
     stamp = _now()
     for row in _table().find(project_uid=project_uid, deleted_at=None):
         _table().update(
@@ -486,6 +488,7 @@ def soft_delete_all_project_files(project_uid: str, deleted_by: str) -> None:
 
 
 _SYNC_CLOCK_SKEW_SECONDS = 1.0
+SYNC_STATE_TABLE = "project_file_sync_state"
 
 
 def _epoch_of(iso_timestamp) -> float:
@@ -497,6 +500,48 @@ def _epoch_of(iso_timestamp) -> float:
         return 0.0
 
 
+def _sync_state_table():
+    return get_table(SYNC_STATE_TABLE)
+
+
+def _load_sync_manifest(project_uid: str) -> dict:
+    if SYNC_STATE_TABLE not in db.tables:
+        return {}
+    return {
+        row["path"]: {"db_epoch": row["db_epoch"], "fs_epoch": row["fs_epoch"]}
+        for row in _sync_state_table().find(project_uid=project_uid)
+    }
+
+
+def _save_sync_manifest(project_uid: str, old: dict, new: dict) -> None:
+    table = _sync_state_table()
+    for path in old:
+        if path not in new:
+            table.delete(project_uid=project_uid, path=path)
+    for path, entry in new.items():
+        if old.get(path) == entry:
+            continue
+        table.upsert(
+            {"project_uid": project_uid, "path": path, **entry},
+            ["project_uid", "path"],
+        )
+
+
+def clear_sync_state(project_uid: str) -> None:
+    if SYNC_STATE_TABLE not in db.tables:
+        return
+    _sync_state_table().delete(project_uid=project_uid)
+
+
+def _delete_db_row_for_sync(row: dict, deleted_by: str) -> None:
+    _table().update(
+        {"uid": row["uid"], "deleted_at": _now(), "deleted_by": deleted_by},
+        ["uid"],
+    )
+    if row.get("is_binary"):
+        _unlink_blob(row)
+
+
 def _file_records(project_uid: str) -> dict:
     records: dict = {}
     for row in _table().find(project_uid=project_uid, deleted_at=None):
@@ -510,7 +555,7 @@ def _walk_workspace_files(src, skip):
     for root, dirs, files in src.walk():
         dirs[:] = [d for d in sorted(dirs) if d not in skip]
         for name in sorted(files):
-            if name in skip:
+            if name in skip or Path(name).suffix in IMPORT_SKIP_EXTENSIONS:
                 continue
             full = Path(root) / name
             if full.is_symlink() or not full.is_file():
@@ -536,48 +581,121 @@ def _workspace_records(workspace) -> dict:
 
 
 def sync_dir_bidirectional(project_uid: str, workspace, user: dict) -> dict:
+    empty = {
+        "exported": 0,
+        "imported": 0,
+        "deleted_in_project": 0,
+        "deleted_in_workspace": 0,
+    }
     if "project_files" not in db.tables:
-        return {"exported": 0, "imported": 0}
+        return empty
     readonly = is_readonly(project_uid)
     dest = Path(workspace).resolve()
     dest.mkdir(parents=True, exist_ok=True)
     db_files = _file_records(project_uid)
     fs_files = _workspace_records(dest)
-    exported = 0
-    imported = 0
+    manifest = _load_sync_manifest(project_uid)
+    new_manifest: dict = {}
+    counts = dict(empty)
 
-    for path, row in db_files.items():
+    for path in set(manifest) | set(db_files) | set(fs_files):
+        row = db_files.get(path)
         fs_full = fs_files.get(path)
-        if fs_full is None:
-            _export_node(row, dest)
-            exported += 1
-            continue
-        try:
-            fs_mtime = fs_full.stat().st_mtime
-        except OSError:
-            continue
-        db_mtime = _epoch_of(row.get("updated_at"))
-        if db_mtime >= fs_mtime - _SYNC_CLOCK_SKEW_SECONDS:
-            _export_node(row, dest)
-            exported += 1
-        elif not readonly:
-            if _import_file(project_uid, user, path, fs_full):
-                imported += 1
+        entry = manifest.get(path)
 
-    if not readonly:
-        for path, fs_full in fs_files.items():
-            if path in db_files:
+        if row is not None and fs_full is not None:
+            try:
+                fs_epoch = fs_full.stat().st_mtime
+            except OSError:
                 continue
-            if _import_file(project_uid, user, path, fs_full):
-                imported += 1
+            db_epoch = _epoch_of(row.get("updated_at"))
+            if (
+                entry is not None
+                and abs(db_epoch - entry["db_epoch"]) <= _SYNC_CLOCK_SKEW_SECONDS
+                and abs(fs_epoch - entry["fs_epoch"]) <= _SYNC_CLOCK_SKEW_SECONDS
+            ):
+                new_manifest[path] = entry
+                continue
+            if db_epoch >= fs_epoch - _SYNC_CLOCK_SKEW_SECONDS or readonly:
+                recorded = _record_export(row, dest)
+                if recorded:
+                    counts["exported"] += 1
+                    new_manifest[path] = recorded
+            else:
+                recorded = _record_import(project_uid, user, path, fs_full, fs_epoch)
+                if recorded:
+                    counts["imported"] += 1
+                    new_manifest[path] = recorded
+            continue
 
-    return {"exported": exported, "imported": imported}
+        if row is not None and fs_full is None:
+            db_epoch = _epoch_of(row.get("updated_at"))
+            if (
+                entry is None
+                or readonly
+                or db_epoch > entry["db_epoch"] + _SYNC_CLOCK_SKEW_SECONDS
+            ):
+                recorded = _record_export(row, dest)
+                if recorded:
+                    counts["exported"] += 1
+                    new_manifest[path] = recorded
+            else:
+                _delete_db_row_for_sync(row, user["uid"])
+                counts["deleted_in_project"] += 1
+            continue
+
+        if row is None and fs_full is not None:
+            try:
+                fs_epoch = fs_full.stat().st_mtime
+            except OSError:
+                continue
+            if entry is None:
+                if not readonly:
+                    recorded = _record_import(project_uid, user, path, fs_full, fs_epoch)
+                    if recorded:
+                        counts["imported"] += 1
+                        new_manifest[path] = recorded
+                continue
+            if not readonly and fs_epoch > entry["fs_epoch"] + _SYNC_CLOCK_SKEW_SECONDS:
+                recorded = _record_import(project_uid, user, path, fs_full, fs_epoch)
+                if recorded:
+                    counts["imported"] += 1
+                    new_manifest[path] = recorded
+                continue
+            try:
+                fs_full.unlink()
+                counts["deleted_in_workspace"] += 1
+            except OSError:
+                pass
+            continue
+
+    _save_sync_manifest(project_uid, manifest, new_manifest)
+    return counts
 
 
-def _export_node(row: dict, dest: Path) -> None:
+def _record_export(row: dict, dest: Path):
+    target = _export_node(row, dest)
+    if target is None:
+        return None
+    db_epoch = _epoch_of(row.get("updated_at"))
+    try:
+        fs_epoch = target.stat().st_mtime
+    except OSError:
+        fs_epoch = db_epoch
+    return {"db_epoch": db_epoch, "fs_epoch": fs_epoch}
+
+
+def _record_import(project_uid: str, user: dict, path: str, fs_full: Path, fs_epoch: float):
+    imported = _import_file(project_uid, user, path, fs_full)
+    if imported is None:
+        return None
+    return {"db_epoch": _epoch_of(imported.get("updated_at")), "fs_epoch": fs_epoch}
+
+
+def _export_node(row: dict, dest: Path):
     target = (dest / row["path"]).resolve()
     if target != dest and not target.is_relative_to(dest):
-        return
+        return None
     target.parent.mkdir(parents=True, exist_ok=True)
     if target.is_symlink():
         target.unlink()
@@ -587,20 +705,21 @@ def _export_node(row: dict, dest: Path) -> None:
             shutil.copyfile(src, target)
         except (FileNotFoundError, OSError):
             logger.warning("Blob file missing during export: %s", src)
+            return None
     else:
         target.write_text(row.get("content") or "", encoding="utf-8")
+    return target
 
 
-def _import_file(project_uid: str, user: dict, path: str, fs_full: Path) -> bool:
+def _import_file(project_uid: str, user: dict, path: str, fs_full: Path):
     try:
         data = fs_full.read_bytes()
     except OSError:
-        return False
+        return None
     try:
-        store_upload(project_uid, user, _parent_of(path), _name_of(path), data)
-        return True
+        return store_upload(project_uid, user, _parent_of(path), _name_of(path), data)
     except ProjectFileError:
-        return False
+        return None
 
 
 def node_to_dict(row: dict) -> dict:
@@ -662,6 +781,36 @@ IMPORT_SKIP_NAMES = {
     ".DS_Store",
     ".idea",
     ".cache",
+    "build",
+    "dist",
+    "target",
+    "out",
+    "bin",
+    "obj",
+    ".next",
+    ".nuxt",
+    ".gradle",
+    ".tox",
+    "cmake-build-debug",
+    "cmake-build-release",
+}
+# Compiled/build-artifact extensions, regenerated wholesale on every build -
+# never worth importing/syncing regardless of which directory they land in
+# (unlike IMPORT_SKIP_NAMES, matched by suffix rather than exact name; see
+# _walk_workspace_files). A missing exclusion here is what let an unlocked
+# concurrent sync (see api._sync_dir_bidirectional_locked) leak millions of
+# orphaned blobs from an actively-compiling container workspace.
+IMPORT_SKIP_EXTENSIONS = {
+    ".o",
+    ".obj",
+    ".pyc",
+    ".pyo",
+    ".class",
+    ".so",
+    ".dylib",
+    ".dll",
+    ".a",
+    ".exe",
 }
 SYNC_SKIP_NAMES = IMPORT_SKIP_NAMES | {
     ".devplace_boot.py",
@@ -875,7 +1024,66 @@ def append_lines(project_uid: str, raw_path: str, content: str) -> dict:
 def delete_all_project_files(project_uid: str) -> None:
     if "project_files" not in db.tables:
         return
+    clear_sync_state(project_uid)
     for row in _table().find(project_uid=project_uid):
         if row.get("is_binary"):
             _unlink_blob(row)
     _table().delete(project_uid=project_uid)
+
+
+def purge_soft_deleted_project_files(*, dry_run: bool = False) -> tuple[int, int]:
+    if "project_files" not in db.tables:
+        return 0, 0
+    table = _table()
+    rows = list(table.find(table.table.columns.deleted_at.isnot(None), is_binary=1))
+    removed = 0
+    freed = 0
+    for row in rows:
+        directory = row.get("directory")
+        stored_name = row.get("stored_name")
+        if directory and stored_name:
+            path = PROJECT_FILES_DIR / directory / stored_name
+            try:
+                freed += path.stat().st_size
+            except OSError:
+                pass
+            else:
+                if not dry_run:
+                    _unlink_blob(row)
+        if not dry_run:
+            table.delete(id=row["id"])
+        removed += 1
+    return removed, freed
+
+
+def sweep_orphan_project_file_blobs(*, dry_run: bool = False) -> tuple[int, int]:
+    if not PROJECT_FILES_DIR.exists():
+        return 0, 0
+    referenced = set()
+    if "project_files" in db.tables:
+        for row in _table().find(is_binary=1):
+            directory = row.get("directory")
+            stored_name = row.get("stored_name")
+            if directory and stored_name:
+                referenced.add((directory, stored_name))
+    removed = 0
+    freed = 0
+    base = str(PROJECT_FILES_DIR)
+    for root, _dirs, files in os.walk(base):
+        directory = os.path.relpath(root, base)
+        for name in files:
+            if (directory, name) in referenced:
+                continue
+            file_path = os.path.join(root, name)
+            try:
+                size = os.path.getsize(file_path)
+            except OSError:
+                continue
+            if not dry_run:
+                try:
+                    os.unlink(file_path)
+                except OSError:
+                    continue
+            removed += 1
+            freed += size
+    return removed, freed

devplacepy/push/CLAUDE.md

@@ -2,33 +2,35 @@ This file documents `devplacepy/push/` - push notification delivery and its prov
 
 ## What this package is
 
-One delivery library behind one public surface. `devplacepy.push` exports `notify_user`, `register`, `ensure_certificates`, `public_key_standard_b64` and the Web Push crypto helpers; every caller in the codebase (`main.py`, `utils/notifications.py`, `routers/push.py`) imports only those names. Everything else is internal to the package.
+One delivery library behind one public surface. `devplacepy.push` exports `notify_user`, `notify_registration`, `register`, `ensure_certificates`, `public_key_standard_b64` and the Web Push crypto helpers; every caller in the codebase (`main.py`, `utils/notifications.py`, `routers/push.py`) imports only those names. Everything else is internal to the package.
 
 | Module | Role |
 |---|---|
 | `providers/base.py` | `PushProvider` protocol, the `Delivery` outcome and the three outcome constants |
 | `providers/webpush.py` | VAPID key material, `aesgcm` payload encryption, the Web Push provider |
-| `providers/apns.py` | Apple Push Notification service provider (token based, HTTP/2) |
+| `providers/apns.py` | Apple Push Notification service provider (token based, dedicated HTTP/2 client) |
 | `providers/__init__.py` | `PROVIDERS` registry, `get`, `active`, `is_active`, `admin_fields`, `client_config` |
-| `store.py` | Every `push_registration` read and write |
-| `delivery.py` | `notify_user`: group by provider, one shared client, one prepared body per provider |
+| `store.py` | Every `push_registration` read and write, including identity upsert and revive |
+| `delivery.py` | `notify_user` / `notify_registration`: group by provider, one client per provider, one prepared body per provider |
 
 The admin configuration surface lives in `devplacepy/services/push/service.py` (`PushService`), not here.
 
 ## Adding a provider
 
-1. Write `providers/<name>.py` with a `PushProvider` subclass: `name`, `label`, `config_fields`, `is_configured`, `parse_registration`, `prepare`, `deliver`, optionally `client_config`.
+1. Write `providers/<name>.py` with a `PushProvider` subclass: `name`, `label`, `config_fields`, `is_configured`, `parse_registration`, `prepare`, `deliver`, optionally `client_config`, `stamp_registration`, `delivery_client`.
 2. Add one entry to `PROVIDERS` in `providers/__init__.py`.
 
 That is the whole change. The registration route, the delivery loop, the admin page, the audit record, the metrics and the docs are written against the protocol and need no edit. The `Enabled` toggle (`push_<name>_enabled`) comes from the base class, so a provider never declares its own.
 
+The default `delivery_client` is `stealth_async_client`. Override it only when the destination is a first-party API that must not see Chrome impersonation, PRIORITY frames, extra browser headers, or the outbound proxy. APNs is that case.
+
 ## Invariants
 
-- **Zero cost for the request.** Delivery is reached only through `utils/notifications.py` `_schedule_push`, a fire-and-forget task. Never make a route await `notify_user`, and never add a queue or a table to this path.
+- **Zero cost for the request, except the welcome probe.** Delivery of real notifications is reached only through `utils/notifications.py` `_schedule_push`, a fire-and-forget task. `POST /push.json` awaits `notify_registration` only for a newly created or revived row, so the client can see `delivered` / `error`. Never add a queue or a table to this path.
 - **`deliver` never raises.** Return `Delivery(REJECTED, detail)` instead. `delivery.py` guards anyway, but a raising provider costs a log line per subscription.
 - **A provider that is not configured is inert, never an error.** `is_configured()` is false, `is_active()` is false, the delivery loop skips it, and `POST /push.json` refuses a registration for it with 400. Nothing else in the platform notices.
-- **`DEAD` is the only outcome that touches the database.** It soft-deletes the registration (`deleted_at`), exactly like a `404`/`410` Web Push endpoint always did. `REJECTED` keeps the row.
-- **Every insert writes `deleted_at: None`,** and every read filters `deleted_at IS NULL`. `push_registration` deliberately stays out of `SOFT_DELETE_TABLES` (no `deleted_by`, not restorable from Trash) - a dead device token has no owner action to undo.
+- **`DEAD` is the only outcome that touches the database.** It soft-deletes the registration (`deleted_at`), exactly like a `404`/`410` Web Push endpoint always did - unless `Delivery.dead_before` names an instant the row was proven live again after (APNs 410 `timestamp` vs. `registered_at`, see "APNs specifics"), in which case the delete is skipped. `REJECTED` keeps the row. The Apple/Web Push reason is logged at WARNING (`Push dead via ...: <detail>`); do not drop `outcome.detail`.
+- **Every insert writes `deleted_at: None`,** and every read of the live set filters `deleted_at IS NULL`. Identity lookups for upsert/revive intentionally ignore `deleted_at` so a client that re-registers a previously dead token comes back live. `push_registration` deliberately stays out of `SOFT_DELETE_TABLES` (no `deleted_by`, not restorable from Trash) - a dead device token has no owner action to undo.
 - **A row without a provider is a Web Push row.** `store.provider_of` resolves `None`/`""` to `DEFAULT_PROVIDER`, so a row written by an old worker during a deploy still delivers. `init_db` backfills the column once with a single converging `UPDATE`.
 
 ## Storage
@@ -40,13 +42,31 @@ That is the whole change. The registration route, the delivery loop, the admin p
 | `provider` | `webpush` | `apns` |
 | `endpoint`, `key_auth`, `key_p256dh` | set | `NULL` |
 | `token` | `NULL` | device token |
+| `client_id` | unused | optional stable per-device id |
+| `environment` | unused | `production` or `sandbox`, stamped server-side at register |
+| `registered_at` | stamped on insert/merge | stamped on insert/merge |
 
-Deduplication is generic: `store.register` looks up `user_uid` + `provider` + exactly the fields the provider's `parse_registration` returned, so a provider never writes its own identity rule.
+`registered_at` (ISO, both providers) is stamped on insert and on every `_merge` that actually changes a row (including revival). It exists solely to arbitrate the dead-token race described below - it is not a general "last seen" field.
+
+`store.register` returns `RegistrationWrite(record, created, revived)`. Identity, in order:
+
+1. `user_uid` + `provider` + `client_id` (live or dead), if `client_id` is present.
+2. `user_uid` + `provider` + `token` (or `endpoint` for Web Push), live or dead.
+3. Exact live match on the remaining fields.
+4. Insert.
+
+A match **updates** the row (token, client_id, environment) and clears `deleted_at` if it was dead. Token rotation with the same `client_id` therefore replaces the token on one row. Sibling live rows that share the new token, the previous token, or the same `client_id` are marked dead so one device cannot accumulate duplicates. A body without `client_id` still works: the same token revives, a new token inserts. `push.update` is a real update, not a no-op of an identical POST.
 
 ## APNs specifics
 
-- `POST https://{host}/3/device/{token}` over HTTP/2, host from `push_apns_environment` (`api.push.apple.com` or `api.sandbox.push.apple.com`; an unrecognised value falls back to production). HTTP/2 comes from `stealth_async_client` because the origin is `https` - the cleartext downgrade in `curl_transport` does not apply.
-- Provider token: `ES256`, header `kid` = key id, claims `iss` = team id and `iat`. Cached per credential fingerprint for 45 minutes, so a worker signs at most one token per 45 minutes; Apple refuses tokens regenerated faster than every 20 minutes. Editing any credential changes the fingerprint and takes effect on the next delivery, with no restart.
+- **Persistent HTTP/2 connection, not one per notification.** `apns.cached_client(timeout)` lazily creates ONE module-level `httpx.AsyncClient(http2=True, ...)` and reuses it across every `notify_user`/`notify_registration` call for the lifetime of the process, following Apple's explicit guidance to keep the connection open rather than repeatedly opening/closing (`sending-notification-requests-to-apns`). `ApnsProvider.closes_delivery_client()` returns `False` so `delivery.py` never closes it after a batch (Web Push still opens/closes per call via `stealth_async_client`, `closes_delivery_client()` defaulting `True` on the base class). Closed once, gracefully, in `main.py`'s shutdown via `push.shutdown_providers()` -> `ApnsProvider.aclose()` -> `apns.close_client()`, mirroring the identical `services/containers/forward.py` `client()`/`close_client()` pattern. Not stealth, not curl_cffi Chrome impersonation, not the outbound proxy - Apple's provider API is a first-party HTTP/2 service; browser PRIORITY frames, HPACK indexing of `:path`/`authorization`, extra `sec-ch-ua` headers, and a scraping proxy all violate that contract. This is the second documented exception to the stealth-only outbound rule (the other is container reverse-proxy forwarding). If the admin-configured delivery timeout changes, the cached client is rebuilt with the new timeout on next use and the old one is left for GC (not explicitly closed) - a deliberate, rare-path simplification.
+- `POST https://{host}/3/device/{token}` over HTTP/2. Host comes from the **row's** `environment` if set, else `push_apns_environment` (`api.push.apple.com` or `api.sandbox.push.apple.com`; an unrecognised value falls back to production). Stamping environment per row lets a sandbox debug token and a production TestFlight token coexist.
+- `GET /push.json` advertises `providers.apns.environment` when APNs is active so a native client can refuse to register a sandbox token against production.
+- `parse_registration` requires a hexadecimal `token` (64-200 digits after stripping spaces and `<>`) and optionally `client_id` (string, max 128). A `client_id` that is not a string is a 400, not a silent drop. Server stamps `environment`; the client cannot pick the host.
+- **Provider token (JWT):** `ES256`, header `kid` = key id, claims `iss` = team id and `iat`. Cached per credential fingerprint for 45 minutes (Apple refuses tokens regenerated faster than every 20 minutes, and requires a fresh one at least once an hour). **Shared cross-worker via `site_settings` key `push_apns_shared_token`** (JSON `{fingerprint, token, issued_at}`, read/written through the existing `get_setting`/`set_setting` cache-version machinery, propagating to sibling workers within ~1s like every other settings key) - a worker that finds no valid in-process cache checks the shared copy before signing a new one, so `make prod`'s multiple uvicorn workers converge on presenting Apple the SAME token instead of each independently re-signing on its own clock (which could otherwise interleave closer together than Apple's per-credential minimum spacing, worst case when workers boot near-simultaneously). Editing any credential changes the fingerprint and takes effect on the next delivery, with no restart.
+- **A rejected provider token is evicted immediately, not left to expire from cache.** A 403 status, or any reason in `AUTH_REASONS` (`InvalidProviderToken`, `ExpiredProviderToken`, `BadCertificate`, `BadCertificateEnvironment`, `Forbidden`, `MissingProviderToken` - all provider-credential-shaped per Apple's reason table, never about a specific device), calls `invalidate_provider_token()`: clears the in-process cache AND the shared DB copy, so the very next delivery attempt (any worker) re-signs instead of retrying the same rejected token for up to 45 minutes.
 - A `.p8` that does not parse is cached as a failure for the same window, so a misconfiguration costs one error log per window rather than one parse per notification.
-- `410`, or any status carrying reason `BadDeviceToken`, `Unregistered`, `ExpiredToken`, `DeviceTokenNotForTopic` or `TopicDisallowed`, is `DEAD`. Everything else is `REJECTED`.
+- **`DEAD_REASONS` is deliberately narrow: only `BadDeviceToken` (400), `Unregistered` (410) and `ExpiredToken` (410).** These are the only reasons in Apple's documented table that describe the TOKEN itself as permanently invalid. `DeviceTokenNotForTopic` and `TopicDisallowed` are 400 errors about the **topic/provisioning matching the connection**, not the token - they fire identically for every token when `push_apns_topic` is misconfigured or the certificate/entitlements don't match, so treating them as dead would soft-delete the entire APNs subscriber base (a table deliberately kept OUT of `SOFT_DELETE_TABLES`, hence unrestorable) on the first delivery after a one-field admin typo. Never add a topic/provisioning-shaped reason to `DEAD_REASONS`.
+- **A 410's `timestamp` is honored before deleting.** Apple's 410 body carries `{"reason": "Unregistered", "timestamp": <ms epoch>}` - the instant Apple last confirmed the token dead, which can trail real device state by "several days" per Apple engineering guidance (410 delivery is intentionally non-deterministic; do not use it to infer app-uninstall timing). `apns._dead_before` converts it to ISO and it rides on `Delivery.dead_before`; `store.mark_dead(id, dead_before)` compares it against the row's `registered_at` and **skips the delete** (logs at INFO instead) if the registration was re-registered/revived after that instant - closing the race where an in-flight delivery against a stale row state would otherwise undo a concurrent revival. `dead_before` is only ever set for a 410; the `BadDeviceToken`/`ExpiredToken` paths pass `None` (unconditional delete, as before), since those reasons carry no `timestamp`.
 - The shared payload dict (`title`, `message`, `icon`, `url`) is translated once per batch into `aps.alert` plus the custom `url`/`icon` keys, mirroring what `service-worker.js` does for Web Push. `thread-id` mirrors the service worker's notification `tag`.
+- `POST /push.json` probes a created or revived row immediately via `notify_registration`. The JSON is `{registered: true, delivered: bool, error?: string}`. `registered` stays true so existing clients keep working; `delivered`/`error` surface Apple's reason instead of killing the row silently from the client's point of view. The row is still marked dead on a `DEAD` outcome (subject to the `dead_before` guard above) so later `notify_user` calls do not keep hitting a known-bad token.

devplacepy/push/__init__.py

@@ -1,6 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
-from devplacepy.push.delivery import notify_user
+from devplacepy.push.delivery import notify_registration, notify_user
+from devplacepy.push.providers import shutdown as shutdown_providers
 from devplacepy.push.providers.webpush import (
     browser_base64,
     create_notification_authorization,
@@ -23,7 +24,9 @@ __all__ = [
     "generate_private_key",
     "generate_public_key",
     "hkdf",
+    "notify_registration",
     "notify_user",
     "public_key_standard_b64",
     "register",
+    "shutdown_providers",
 ]

devplacepy/push/delivery.py

@@ -3,9 +3,9 @@
 import logging
 from typing import Any
 
-from devplacepy import stealth
 from devplacepy.database import get_int_setting
 from devplacepy.push import providers, store
+from devplacepy.push.providers.base import Delivery
 
 logger = logging.getLogger(__name__)
 
@@ -29,6 +29,10 @@ def group_by_provider(
     return grouped
 
 
+def _open_client(provider, timeout: float):
+    return provider.delivery_client(timeout)
+
+
 async def notify_user(user_uid: str, payload: dict[str, Any]) -> None:
     registrations = store.active_for_user(user_uid)
     if not registrations:
@@ -36,48 +40,86 @@ async def notify_user(user_uid: str, payload: dict[str, Any]) -> None:
         return
 
     grouped = group_by_provider(registrations)
-    async with stealth.stealth_async_client(timeout=timeout_seconds()) as client:
-        for name, rows in grouped.items():
-            provider = providers.PROVIDERS.get(name)
-            if provider is None:
-                logger.warning(
-                    "Unknown push provider %s on %s subscriptions of user %s",
-                    name,
-                    len(rows),
-                    user_uid,
-                )
-                continue
-            if not providers.is_active(provider):
-                logger.debug(
-                    "Push provider %s is not active; skipping %s subscriptions",
-                    name,
-                    len(rows),
-                )
-                continue
-            try:
-                prepared = provider.prepare(payload)
-            except Exception as exc:
-                logger.error("Push provider %s could not build a payload: %s", name, exc)
-                continue
+    timeout = timeout_seconds()
+    for name, rows in grouped.items():
+        provider = providers.PROVIDERS.get(name)
+        if provider is None:
+            logger.warning(
+                "Unknown push provider %s on %s subscriptions of user %s",
+                name,
+                len(rows),
+                user_uid,
+            )
+            continue
+        if not providers.is_active(provider):
+            logger.debug(
+                "Push provider %s is not active; skipping %s subscriptions",
+                name,
+                len(rows),
+            )
+            continue
+        try:
+            prepared = provider.prepare(payload)
+        except Exception as exc:
+            logger.error("Push provider %s could not build a payload: %s", name, exc)
+            continue
+        try:
+            client = _open_client(provider, timeout)
+        except Exception as exc:
+            logger.error("Push provider %s could not open a client: %s", name, exc)
+            continue
+        try:
             for registration in rows:
                 await _deliver_one(provider, client, registration, prepared, user_uid)
+        finally:
+            if provider.closes_delivery_client():
+                await client.aclose()
 
 
-async def _deliver_one(provider, client, registration, prepared, user_uid) -> None:
+async def notify_registration(
+    registration: dict[str, Any], payload: dict[str, Any]
+) -> Delivery:
+    name = store.provider_of(registration)
+    provider = providers.PROVIDERS.get(name)
+    if provider is None:
+        return Delivery(providers.REJECTED, f"unknown provider {name}")
+    if not providers.is_active(provider):
+        return Delivery(providers.REJECTED, f"provider {name} is not active")
+    try:
+        prepared = provider.prepare(payload)
+    except Exception as exc:
+        return Delivery(providers.REJECTED, str(exc))
+    user_uid = registration.get("user_uid") or ""
+    try:
+        client = _open_client(provider, timeout_seconds())
+    except Exception as exc:
+        return Delivery(providers.REJECTED, str(exc))
+    try:
+        return await _deliver_one(provider, client, registration, prepared, user_uid)
+    finally:
+        if provider.closes_delivery_client():
+            await client.aclose()
+
+
+async def _deliver_one(provider, client, registration, prepared, user_uid) -> Delivery:
     try:
         outcome = await provider.deliver(client, registration, prepared)
     except Exception as exc:
         logger.error("Push provider %s raised for %s: %s", provider.name, user_uid, exc)
-        return
+        return Delivery(providers.REJECTED, str(exc))
     if outcome.status == providers.ACCEPTED:
         logger.debug("Push delivered to %s via %s", user_uid, provider.name)
-        return
+        return outcome
     if outcome.status == providers.DEAD:
+        logger.warning(
+            "Push dead via %s for %s: %s", provider.name, user_uid, outcome.detail
+        )
         try:
-            store.mark_dead(registration["id"])
+            store.mark_dead(registration["id"], outcome.dead_before)
         except Exception as exc:
             logger.error("Could not soft-delete push subscription: %s", exc)
-        return
+        return outcome
     logger.warning(
         "Push rejected by %s for %s: %s", provider.name, user_uid, outcome.detail
     )
+    return outcome

devplacepy/push/providers/__init__.py

@@ -35,6 +35,7 @@ __all__ = [
     "get",
     "is_active",
     "names",
+    "shutdown",
 ]
 
 
@@ -74,3 +75,11 @@ def _client_config(provider: PushProvider) -> dict[str, Any]:
     except Exception as exc:
         logger.error("Push provider %s failed to describe itself: %s", provider.name, exc)
         return {}
+
+
+async def shutdown() -> None:
+    for provider in PROVIDERS.values():
+        try:
+            await provider.aclose()
+        except Exception as exc:
+            logger.error("Push provider %s failed to close: %s", provider.name, exc)

devplacepy/push/providers/apns.py

@@ -5,13 +5,14 @@ import json
 import logging
 import string
 import time
+from datetime import datetime, timezone
 from typing import Any
 
 import httpx
 import jwt
 
 from devplacepy.config import SECONDS_PER_DAY
-from devplacepy.database import get_setting
+from devplacepy.database import get_setting, set_setting
 from devplacepy.push.providers.base import (
     ACCEPTED,
     DEAD,
@@ -44,20 +45,27 @@ ENVIRONMENT_OPTIONS = [
 TOKEN_REFRESH_SECONDS = 45 * 60
 TOKEN_MIN_LENGTH = 64
 TOKEN_MAX_LENGTH = 200
+CLIENT_ID_MAX_LENGTH = 128
 THREAD_ID = "devplace-notification"
 PUSH_TYPE = "alert"
 PRIORITY = "10"
-DEAD_REASONS = frozenset(
+SHARED_TOKEN_KEY = "push_apns_shared_token"
+
+DEAD_REASONS = frozenset({"BadDeviceToken", "ExpiredToken", "Unregistered"})
+
+AUTH_REASONS = frozenset(
     {
-        "BadDeviceToken",
-        "DeviceTokenNotForTopic",
-        "ExpiredToken",
-        "Unregistered",
-        "TopicDisallowed",
+        "BadCertificate",
+        "BadCertificateEnvironment",
+        "ExpiredProviderToken",
+        "Forbidden",
+        "InvalidProviderToken",
+        "MissingProviderToken",
     }
 )
 
 _token_state: dict[str, Any] = {}
+_client: httpx.AsyncClient | None = None
 
 
 def _setting(key: str) -> str:
@@ -70,13 +78,78 @@ def _environment() -> str:
 
 
 def host() -> str:
-    return HOSTS[_environment()]
+    return host_for(_environment())
+
+
+def host_for(environment: str | None) -> str:
+    value = (environment or "").strip() or _environment()
+    return HOSTS[value] if value in HOSTS else HOSTS[DEFAULT_ENVIRONMENT]
+
+
+def gateway_client(timeout: float) -> httpx.AsyncClient:
+    return httpx.AsyncClient(
+        http2=True,
+        timeout=timeout,
+        trust_env=False,
+        verify=True,
+        follow_redirects=False,
+    )
+
+
+def cached_client(timeout: float) -> httpx.AsyncClient:
+    global _client
+    if _client is None or _client.is_closed:
+        _client = gateway_client(timeout)
+    return _client
+
+
+async def close_client() -> None:
+    global _client
+    if _client is not None and not _client.is_closed:
+        await _client.aclose()
+    _client = None
+
+
+def _normalize_token(token: str) -> str:
+    return "".join(
+        character.lower() for character in token if character in string.hexdigits
+    )
 
 
 def _fingerprint(team_id: str, key_id: str, auth_key: str) -> str:
     return hashlib.sha256(f"{team_id}:{key_id}:{auth_key}".encode("utf-8")).hexdigest()
 
 
+def _read_shared_token() -> dict[str, Any] | None:
+    raw = get_setting(SHARED_TOKEN_KEY, "")
+    if not raw:
+        return None
+    try:
+        data = json.loads(raw)
+    except ValueError:
+        return None
+    if (
+        not isinstance(data, dict)
+        or not isinstance(data.get("fingerprint"), str)
+        or not isinstance(data.get("token"), str)
+        or not isinstance(data.get("issued_at"), int)
+    ):
+        return None
+    return data
+
+
+def _write_shared_token(fingerprint: str, token: str, issued_at: int) -> None:
+    set_setting(
+        SHARED_TOKEN_KEY,
+        json.dumps({"fingerprint": fingerprint, "token": token, "issued_at": issued_at}),
+    )
+
+
+def invalidate_provider_token() -> None:
+    _token_state.pop("current", None)
+    set_setting(SHARED_TOKEN_KEY, "")
+
+
 def provider_token(team_id: str, key_id: str, auth_key: str) -> str:
     fingerprint = _fingerprint(team_id, key_id, auth_key)
     issued_at = int(time.time())
@@ -89,6 +162,21 @@ def provider_token(team_id: str, key_id: str, auth_key: str) -> str:
         if state["token"] is None:
             raise ValueError(state["error"])
         return state["token"]
+
+    shared = _read_shared_token()
+    if (
+        shared
+        and shared["fingerprint"] == fingerprint
+        and issued_at - shared["issued_at"] < TOKEN_REFRESH_SECONDS
+    ):
+        _token_state["current"] = {
+            "token": shared["token"],
+            "error": "",
+            "issued_at": shared["issued_at"],
+            "fingerprint": fingerprint,
+        }
+        return shared["token"]
+
     try:
         token = jwt.encode(
             {"iss": team_id, "iat": issued_at},
@@ -112,6 +200,7 @@ def provider_token(team_id: str, key_id: str, auth_key: str) -> str:
         "issued_at": issued_at,
         "fingerprint": fingerprint,
     }
+    _write_shared_token(fingerprint, token, issued_at)
     return token
 
 
@@ -125,6 +214,22 @@ def _reason(response: httpx.Response) -> str:
     return ""
 
 
+def _dead_before(response: httpx.Response) -> str | None:
+    try:
+        body = response.json()
+    except ValueError:
+        return None
+    if not isinstance(body, dict):
+        return None
+    raw = body.get("timestamp")
+    if not isinstance(raw, (int, float)) or isinstance(raw, bool):
+        return None
+    try:
+        return datetime.fromtimestamp(raw / 1000, tz=timezone.utc).isoformat()
+    except (OverflowError, OSError, ValueError):
+        return None
+
+
 class ApnsProvider(PushProvider):
     name = "apns"
     label = PROVIDER_LABEL
@@ -181,16 +286,41 @@ class ApnsProvider(PushProvider):
             and _setting(TOPIC_KEY)
         )
 
+    def client_config(self) -> dict[str, Any]:
+        return {"environment": _environment()}
+
+    def stamp_registration(self, fields: dict[str, Any]) -> dict[str, Any]:
+        return {**fields, "environment": _environment()}
+
+    def delivery_client(self, timeout: float) -> httpx.AsyncClient:
+        return cached_client(timeout)
+
+    def closes_delivery_client(self) -> bool:
+        return False
+
+    async def aclose(self) -> None:
+        await close_client()
+
     def parse_registration(self, body: dict[str, Any]) -> dict[str, Any] | None:
         token = body.get("token")
         if not isinstance(token, str):
             return None
-        token = token.strip()
+        token = _normalize_token(token)
         if not TOKEN_MIN_LENGTH <= len(token) <= TOKEN_MAX_LENGTH:
             return None
-        if any(character not in string.hexdigits for character in token):
+        fields: dict[str, Any] = {"token": token}
+        client_id = body.get("client_id")
+        if client_id is None:
+            return fields
+        if not isinstance(client_id, str):
             return None
-        return {"token": token}
+        client_id = client_id.strip()
+        if not client_id:
+            return fields
+        if len(client_id) > CLIENT_ID_MAX_LENGTH:
+            return None
+        fields["client_id"] = client_id
+        return fields
 
     def prepare(self, payload: dict[str, Any]) -> str:
         return json.dumps(
@@ -228,7 +358,7 @@ class ApnsProvider(PushProvider):
         try:
             headers = self.headers()
             response = await client.post(
-                f"https://{host()}/3/device/{token}",
+                f"https://{host_for(registration.get('environment'))}/3/device/{token}",
                 headers=headers,
                 content=prepared.encode("utf-8"),
             )
@@ -238,6 +368,9 @@ class ApnsProvider(PushProvider):
             return Delivery(ACCEPTED)
         reason = _reason(response)
         detail = f"{response.status_code} {reason}".strip()
+        if response.status_code == 403 or reason in AUTH_REASONS:
+            invalidate_provider_token()
         if response.status_code == 410 or reason in DEAD_REASONS:
-            return Delivery(DEAD, detail)
+            dead_before = _dead_before(response) if response.status_code == 410 else None
+            return Delivery(DEAD, detail, dead_before)
         return Delivery(REJECTED, detail)

devplacepy/push/providers/base.py

@@ -18,6 +18,7 @@ REJECTED = "rejected"
 class Delivery:
     status: str
     detail: str = ""
+    dead_before: str | None = None
 
 
 class PushProvider(ABC):
@@ -51,6 +52,20 @@ class PushProvider(ABC):
     def client_config(self) -> dict[str, Any]:
         return {}
 
+    def stamp_registration(self, fields: dict[str, Any]) -> dict[str, Any]:
+        return fields
+
+    def delivery_client(self, timeout: float) -> httpx.AsyncClient:
+        from devplacepy import stealth
+
+        return stealth.stealth_async_client(timeout=timeout)
+
+    def closes_delivery_client(self) -> bool:
+        return True
+
+    async def aclose(self) -> None:
+        return None
+
     @abstractmethod
     def is_configured(self) -> bool: ...
 

devplacepy/push/store.py

@@ -1,6 +1,7 @@
 # retoor <retoor@molodetz.nl>
 
 import logging
+from dataclasses import dataclass
 from datetime import datetime, timezone
 from typing import Any
 
@@ -13,6 +14,17 @@ logger = logging.getLogger(__name__)
 TABLE = "push_registration"
 
 
+@dataclass(frozen=True)
+class RegistrationWrite:
+    record: dict[str, Any]
+    created: bool
+    revived: bool
+
+    @property
+    def probe(self) -> bool:
+        return self.created or self.revived
+
+
 def table():
     return get_table(TABLE)
 
@@ -25,31 +37,161 @@ def active_for_user(user_uid: str) -> list[dict[str, Any]]:
     return list(table().find(user_uid=user_uid, deleted_at=None))
 
 
+def _filled(fields: dict[str, Any]) -> dict[str, Any]:
+    filled: dict[str, Any] = {}
+    for key, value in fields.items():
+        if value is None:
+            continue
+        if isinstance(value, str):
+            value = value.strip()
+            if not value:
+                continue
+        filled[key] = value
+    return filled
+
+
+def _prefer_live(rows: list[dict[str, Any]]) -> dict[str, Any] | None:
+    if not rows:
+        return None
+    live = [row for row in rows if not row.get("deleted_at")]
+    return (live or rows)[-1]
+
+
+def _lookup(user_uid: str, provider: str, **identity: Any) -> dict[str, Any] | None:
+    return _prefer_live(
+        list(table().find(user_uid=user_uid, provider=provider, **identity))
+    )
+
+
+def _with_id(record: dict[str, Any]) -> dict[str, Any]:
+    if record.get("id"):
+        return record
+    found = table().find_one(uid=record.get("uid"))
+    return found or record
+
+
+def _retire_duplicates(
+    user_uid: str,
+    provider: str,
+    keep_id: int,
+    fields: dict[str, Any],
+    previous_token: str | None = None,
+) -> None:
+    token = fields.get("token")
+    client_id = fields.get("client_id")
+    for row in table().find(user_uid=user_uid, provider=provider, deleted_at=None):
+        if row["id"] == keep_id:
+            continue
+        if client_id and row.get("client_id") == client_id:
+            mark_dead(row["id"])
+            continue
+        if token and row.get("token") == token:
+            mark_dead(row["id"])
+            continue
+        if previous_token and row.get("token") == previous_token:
+            mark_dead(row["id"])
+
+
+def _merge(
+    existing: dict[str, Any], fields: dict[str, Any]
+) -> RegistrationWrite:
+    revived = bool(existing.get("deleted_at"))
+    patch: dict[str, Any] = {"id": existing["id"]}
+    if revived:
+        patch["deleted_at"] = None
+    changed = revived
+    for key, value in fields.items():
+        if existing.get(key) != value:
+            patch[key] = value
+            changed = True
+    if not changed:
+        return RegistrationWrite(_with_id(existing), False, False)
+    patch["registered_at"] = datetime.now(timezone.utc).isoformat()
+    previous_token = existing.get("token")
+    table().update(patch, ["id"])
+    merged = {**existing, **patch}
+    if revived:
+        merged["deleted_at"] = None
+    merged = _with_id(merged)
+    _retire_duplicates(
+        existing["user_uid"],
+        existing["provider"],
+        merged["id"],
+        fields,
+        previous_token=previous_token if previous_token != fields.get("token") else None,
+    )
+    logger.info(
+        "Updated %s push subscription for user %s%s",
+        existing.get("provider"),
+        existing.get("user_uid"),
+        " (revived)" if revived else "",
+    )
+    return RegistrationWrite(merged, False, revived)
+
+
 def register(
     user_uid: str, provider: str, fields: dict[str, Any]
-) -> tuple[dict[str, Any], bool]:
+) -> RegistrationWrite:
+    fields = _filled(fields)
     registrations = table()
-    existing = registrations.find_one(
+    client_id = fields.get("client_id")
+    token = fields.get("token")
+    endpoint = fields.get("endpoint")
+
+    if client_id:
+        existing = _lookup(user_uid, provider, client_id=client_id)
+        if existing:
+            return _merge(existing, fields)
+
+    if token:
+        existing = _lookup(user_uid, provider, token=token)
+        if existing:
+            return _merge(existing, fields)
+
+    if endpoint:
+        existing = _lookup(user_uid, provider, endpoint=endpoint)
+        if existing:
+            return _merge(existing, fields)
+
+    live = registrations.find_one(
         user_uid=user_uid, provider=provider, deleted_at=None, **fields
     )
-    if existing:
-        logger.debug("Push subscription already registered for user %s", user_uid)
-        return existing, False
+    if live:
+        return RegistrationWrite(_with_id(live), False, False)
 
+    now = datetime.now(timezone.utc).isoformat()
     record = {
         "uid": generate_uid(),
         "user_uid": user_uid,
         "provider": provider,
-        "created_at": datetime.now(timezone.utc).isoformat(),
+        "created_at": now,
+        "registered_at": now,
         "deleted_at": None,
         **fields,
     }
-    registrations.insert(record)
+    inserted = registrations.insert(record)
+    if isinstance(inserted, int):
+        record["id"] = inserted
+    record = _with_id(record)
+    if record.get("id"):
+        _retire_duplicates(user_uid, provider, record["id"], fields)
     logger.info("Registered %s push subscription for user %s", provider, user_uid)
-    return record, True
+    return RegistrationWrite(record, True, False)
 
 
-def mark_dead(registration_id: int) -> None:
+def mark_dead(registration_id: int, dead_before: str | None = None) -> None:
+    if dead_before is not None:
+        row = table().find_one(id=registration_id)
+        registered_at = row.get("registered_at") if row else None
+        if registered_at and registered_at > dead_before:
+            logger.info(
+                "Skipped marking push subscription id=%s dead: registered again at %s "
+                "after the provider confirmed it dead at %s",
+                registration_id,
+                registered_at,
+                dead_before,
+            )
+            return
     table().update(
         {"id": registration_id, "deleted_at": datetime.now(timezone.utc).isoformat()},
         ["id"],

devplacepy/routers/CLAUDE.md

@@ -11,11 +11,12 @@ Prefixes are wired in `main.py`:
 | `/auth` | auth/ package - one leaf per flow (`signup`, `login`, `logout`, `forgotpassword`, `resetpassword`) |
 | `/feed` | feed.py |
 | `/posts` | posts.py |
+| `/topics` | topics.py - crawlable per-topic category index pages (`GET /topics` hub, `GET /topics/{topic}` per-topic post listing over the same `TOPICS` set as the feed sidebar filter). Reuses `feed.py`'s `get_feed_posts`/`enrich_post_cards` so a topic page is a fully-enriched `_post_card.html` listing, not a stripped-down duplicate. Unlike `/feed?topic=X` (whose canonical strips the query string back to bare `/feed`, so it is never indexed as a distinct page), each `/topics/{topic}` page has its own canonical URL, unique title/description, breadcrumbs, and a sitemap entry - see "SEO implementation" below |
 | `/comments` | comments.py |
 | `/projects` | projects/ package - `index.py` (listing/detail/create/delete plus owner-only visibility toggles `POST /{slug}/private` and `POST /{slug}/readonly`, and async `POST /{slug}/fork`), `files.py`, and the `containers/` subpackage (below). `main.py` mounts the whole `/projects` tree from this one package. See `routers/projects/CLAUDE.md` for the deep detail on this tree |
 | `/projects/{slug}/files` | projects/files.py - per-project virtual filesystem (CRUD dirs/files, upload, inline edit, plus line-range ops: `lines` read, `replace-lines`, `insert-lines`, `delete-lines`, `append` for large text files) |
 | `/profile` | profile/ package - `index.py` (page/search/followers/update/api-key; the **Posts** tab shows only the 10 most recent posts, while the sidebar `posts_count` stat always uses the real `count()` so it is never capped to the rendered 10), `customization.py`, `notifications.py`, `ai_correction.py` (`POST /{username}/ai-correction`, owner-or-admin AI content correction toggle + prompt), `ai_modifier.py` (`POST /{username}/ai-modifier`, owner-or-admin AI modifier toggle + prompt), `interactions.py` (`POST /{username}/interactions`, owner-or-admin Devii interactive widgets preference; admin default via `devii_interactions_default`), `telegram.py` (`POST /{username}/telegram`, owner-or-admin Telegram pairing-code request/unpair), `usage.py` (`_ai_quota`, re-exported from the package) |
-| `/messages` | messages.py - real-time DM chat. `GET ""` (page; the conversation sidebar is one windowed SQL aggregate and an opened thread loads only the newest `CONVERSATION_MESSAGE_LIMIT` = 500 messages - keep these queries bounded), `GET /search`, `GET /conversations` (flat JSON conversation list, same `get_conversations` helper as the page), `POST /send` (no-JS fallback, also accepts an attachment-only empty-content message), `POST /ws-ticket` (issues a 30s single-use WS auth ticket so a browser WebSocket can authenticate cross-context, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header), and `WS /messages/ws` (NOT lock-owner gated - accepts on every worker; closes `1008` for guests). The WS adds live bidirectional delivery, typing, and read receipts on top of the existing `messages` table - presence is NOT part of this WS (it is the generic `PresenceManager`/pub/sub mechanism, see `devplacepy/services/CLAUDE.md`). Both the WS `send` and HTTP `POST /send` route through the shared `services/messaging/persist.py` `persist_message` choke point (identical audit/notification/mention) and then through `_finalize_and_broadcast`, which awaits any pending SYNC AI correction/modifier futures (DM content runs through both, so typing `@ai <instruction>` executes live), broadcasts the FINAL corrected/modified content with an additive `ai_processed` frame flag (the WS path passes `request=websocket` and awaits directly since HTTP middleware does not run for websockets; the frontend swaps the sender's optimistic bubble to the authoritative content). Same-worker delivery is instant via the per-worker `message_hub`; cross-worker delivery is filled by `services/messaging/relay.py` `message_relay` (per-worker ~1s DB poll on the `messages.id` watermark, deduped against the hub's delivered set). See `devplacepy/services/messaging/CLAUDE.md` |
+| `/messages` | messages.py - real-time DM chat. `GET ""` (page; the conversation sidebar is one windowed SQL aggregate and an opened thread loads only the newest `CONVERSATION_MESSAGE_LIMIT` = 500 messages - keep these queries bounded), `GET /search`, `GET /conversations` (flat JSON conversation list, same `get_conversations` helper as the page), `POST /send` (no-JS fallback, also accepts an attachment-only empty-content message), `POST /ws-ticket` (issues a 30s single-use WS auth ticket so a browser WebSocket can authenticate cross-context, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header), and `WS /messages/ws` (NOT lock-owner gated - accepts on every worker; closes `1008` for guests). The WS adds live bidirectional delivery, typing, and read receipts on top of the existing `messages` table - presence is NOT part of this WS (it is the generic `PresenceManager`/pub/sub mechanism, see `devplacepy/services/CLAUDE.md`). Both the WS `send` and HTTP `POST /send` route through the shared `services/messaging/persist.py` `persist_message` choke point (identical audit/notification/mention) and then through `_finalize_and_broadcast`, which broadcasts the persisted row immediately and then applies SYNC AI correction/modifier (HTTP awaits so the JSON body is final; WS schedules it so the receive loop never blocks). An in-place rewrite stamps `messages.updated_at` and emits a second `ai_processed` frame; other workers pick it up from `message_relay._tick_updates`. Same-worker delivery is instant via the per-worker `message_hub`; cross-worker delivery is filled by `services/messaging/relay.py` `message_relay` (per-worker ~1s DB poll on the `messages.id` watermark, plus `updated_at` for revisions, new rows deduped against the hub's delivered set). See `devplacepy/services/messaging/CLAUDE.md` |
 | `/notifications` | notifications.py |
 | `/votes` | votes.py |
 | `/reactions` | reactions.py - emoji reaction toggle on any target: `POST /reactions/{target_type}/{target_uid}` (optimistic `ReactionBar`) |
@@ -46,8 +47,9 @@ Prefixes are wired in `main.py`:
 | `/api` | devrant/ package - devRant-compatible REST protocol (`auth.py`, `rants.py`, `comments.py`, `notifs.py`). Translates devRant requests onto DevPlace data: rants<->posts, comments/votes onto the native engagement layer, devRant integer ids onto each table's auto-increment `id`. Token auth via `devrant_tokens`. See `routers/devrant/CLAUDE.md` for the deep detail on this tree |
 | `/dbapi` | dbapi/ package - **primary-administrator-only, strictly READ-ONLY** generic database API over `dataset` (never inserts/updates/deletes/restores; the write surface was removed because it bypassed every per-route admin safeguard). `tables.py` (`GET /tables`, `GET /{table}/schema`), `crud.py` (read only: `GET /{table}` + `GET /{table}/{key}/{value}`, `?include_deleted`), `query.py` (`POST /query` hard SELECT-only validated read-only run; `POST /query/async` + `GET /query/{uid}` + `WS /query/{uid}/ws` via `DbApiJobService` kind `dbquery`), `nl.py` (`POST /nl` natural-language->SQL via the AI gateway, returns validated SELECT, `execute=true` runs it read-only). **Auth = the PRIMARY administrator only** (the earliest-created Admin, `utils.is_primary_admin` / `database.get_primary_admin_uid` - the same identity that gates backup downloads) by session/api_key (`services/dbapi/policy.py`); every other administrator is refused like a member, and there is **no internal-key path** - the gateway `internal_gateway_key()` is NOT accepted (no service-to-service access). SQL validated by `services/dbapi/validate.py` (sqlglot classify + suspicious-flag + read-only `EXPLAIN` dry-run). Devii tools `db_*` are read-only (list/get/query/nl; no write tools) and flagged `requires_primary_admin=True`, so they are added to the LLM tool list ONLY for the primary administrator (every other session never sees them and Devii is unaware they exist). `services/pubsub/policy.py` resolves its own admin/internal actor and does NOT reuse `dbapi.policy.caller_for`, so the primary-admin restriction does not leak onto the pub/sub bus. See `devplacepy/services/dbapi/CLAUDE.md` |
 | `/quizzes` | quizzes/ package - **Quizzes**: `index.py` (hub, create, import, detail, export, edit, delete, publish, per-quiz leaderboard, cross-quiz scoreboard), `questions.py` (builder page + question CRUD + reorder), `attempts.py` (start/resume, play, answer, finish, results). The hub `GET /quizzes` is the approved three-column `.feed-layout` reused verbatim from `/feed`: filters + search left (`?filter=all|todo|done|mine|drafts`, `?search=`, `?page=`), the quiz list centre with the **New quiz** and **Create quiz with Devii** actions, and the cross-quiz **scoreboard** rail right (score per user, best attempt per quiz, 15s display cache). Publishing is terminal - every write on a published quiz is a 400 and there is no unpublish route. See `devplacepy/services/quiz/CLAUDE.md` |
+| `/battles` | battles.py - **Opinion Wars**, week-long two-faction battles attached to posts: `GET ""` (listing, quizzes-style filters active/ended/mine + search), `GET /{uid}` (JSON-only full state, runs lazy resolution), `GET /{uid}/events?after=` (durable event trail replay), `POST /{uid}/join` (join or switch faction), `POST /{uid}/fight` (spend 25 Code Farm coins, deal level-weighted damage, 24h cooldown). The battle card renders on the parent post (no battle detail HTML page). See `devplacepy/services/opinionwar/CLAUDE.md` |
 | `/game` | game/ package - the **Code Farm** idle game (`index.py` + `farm.py`): `GET /game` (page), `GET /game/state`, `GET /game/leaderboard?board=`, `POST /game/{plant,harvest,buy-plot,upgrade,fertilize,daily,grant,perk,prestige,legacy,mastery,quests/claim}`, `POST /game/{defense/upgrade,defense/downgrade,infrastructure/buy,cosmetics/buy,cosmetics/equip}`, plus social `GET /game/farm/{username}` and `POST /game/farm/{username}/{water,steal}`. See `devplacepy/services/game/CLAUDE.md` |
-| (none) | push.py - push + PWA: `GET /push.json` (VAPID public key + the providers accepting registrations), `POST /push.json` (register with any active provider; a body without `provider` is a `webpush` body, so existing clients are unchanged; an unknown, disabled or unconfigured provider is a 400), `GET /service-worker.js`, `GET /manifest.json`. Provider protocol and delivery live in `devplacepy/push/` - see `devplacepy/push/CLAUDE.md` |
+| (none) | push.py - push + PWA: `GET /push.json` (VAPID public key + the providers accepting registrations; `apns` includes `environment` when active), `POST /push.json` (register with any active provider; a body without `provider` is a `webpush` body; an APNs body is `token` plus optional `client_id` for device-stable upsert; created/revived rows are probed and the JSON may include `delivered`/`error`; an unknown, disabled or unconfigured provider is a 400), `GET /service-worker.js`, `GET /manifest.json`. Provider protocol and delivery live in `devplacepy/push/` - see `devplacepy/push/CLAUDE.md` |
 | (none) | docs.py (docs/ package) - the documentation site (prose pages + API reference). See `routers/docs/CLAUDE.md` for the deep detail on this tree (`DOCS_PAGES`, audience tiers, prose rendering pipeline) |
 | `/pubsub` | pubsub.py - **database-free** publish/subscribe bus. `WS /pubsub/ws` (subscribe/unsubscribe/publish frames, `foo.*` wildcards), `POST /pubsub/publish` + `GET /pubsub/topics` (admin/internal). Lock-owner-gated WS (`4013` retry) so subscribers converge on one worker; topic authz in `services/pubsub/policy.py` (`user.{uid}.*` private, `public.*` shared, admin/internal anywhere, guests opt-in). In-process `services.pubsub.publish(topic, data)` for backends; frontend `app.pubsub`. See `devplacepy/services/pubsub/CLAUDE.md` |
 | (none) | seo.py - `/robots.txt`, `/sitemap.xml` |
@@ -249,6 +251,14 @@ All SEO features are implemented across the following locations:
 - `devplacepy/seo.py` - JSON-LD schema generators (WebSite, BreadcrumbList, DiscussionForumPosting, ProfilePage, SoftwareApplication), meta description truncation, schema combiner, sitemap XML generator
 - `routers/seo.py` - robots.txt and sitemap.xml routes
 
+### DiscussionForumPosting nested comments
+
+`discussion_forum_posting(post, author, comment_count, star_count, base_url, comments=None)` embeds up to `MAX_SCHEMA_COMMENTS` (20) of the post's comments as nested `comment: [{"@type": "Comment", "text", "author", "datePublished"}, ...]` entities - not just the aggregate `CommentAction` `InteractionCounter`, which stays for the total count. `seo.comment_schema_list(comment_tree, base_url)` flattens the already-loaded comment tree (`content.load_detail`'s `detail["comments"]`, the same nested `{comment, author, children}` shape `_comment.html` renders) depth-first up to the cap - it does not re-query the database. `posts.py::view_post` is the only call site; a future post-like discussion surface (project/gist/news comments) can reuse `comment_schema_list` the same way once/if it gets a `DiscussionForumPosting` schema of its own.
+
+### Topic category pages (`/topics`)
+
+`routers/topics.py` gives the feed's `TOPICS` filter (`constants.py`) real, independently-crawlable pages instead of only a `?topic=` query param (whose canonical collapses back to bare `/feed` - see `base_seo_context`, `canonical = f"{base}{request.url.path}"`, which drops the query string on purpose). `GET /topics` is a hub linking every topic (with a live post count); `GET /topics/{topic}` is a full post listing for that topic, built from the exact same `get_feed_posts`/`enrich_post_cards` pair `feed.py` uses (`enrich_post_cards` was extracted out of `feed_page` specifically so this page is not a second, drifting copy of the attachments/reactions/bookmarks/poll/war enrichment loop). Each topic page gets its own canonical URL, unique title/description, breadcrumbs (Home > Topics > {label}), a `rel=next` link when paginated (`list_page_seo`/`next_page_url`, same mechanism as `/feed`/`/news`), and a real crawlable `_load_more.html` link (not JS-only infinite scroll) for reaching older posts. Both `/topics` and every `/topics/{topic}` are in `sitemap.xml`.
+
 ### SEO template context
 - Every router passes `page_title`, `meta_description`, `meta_robots`, `canonical_url`, `og_title`, `og_description`, `og_image`, `og_type`, `breadcrumbs`, `page_schema` via `base_seo_context()`
 - Auth pages: `noindex,nofollow`
@@ -265,15 +275,22 @@ All SEO features are implemented across the following locations:
 - `profile.html` - username rendered as `<h1 class="profile-name">`
 - `messages.html` - `<h1 class="sr-only">Messages</h1>`
 - `projects.html` - `<h1>Projects</h1>`
-- `post.html` - post title as `<h1>`, "Related Discussions" as `<h3>`
+- `post.html` - post title as `<h1>`; "Gists from {author}", "Projects from {author}", and "Related Discussions" are `.sidebar-heading` labels in the left column (see "Post page layout" below), not `<h3>`
 
 ### Post slugs
 - Slug generated on post creation via `slugify()` and stored in `posts.slug` column
 - Posts can be looked up by slug or UUID
 - Minimum content validation: post body >= 10 chars, comment >= 3 chars
 
-### Related posts
-- `templates/post.html` - "Related Discussions" widget at bottom of post page (queried by matching topic)
+### Post page layout (three columns, mirrors `/feed`)
+
+`post.html` reuses `/feed`'s exact layout building blocks rather than inventing new ones, wrapped in `.post-page-layout` (`static/css/post.css`, `grid-template-columns: var(--sidebar-width) minmax(0, 1fr) 280px`, collapsing to one column at 1024px):
+
+- **Left column** - `<aside class="post-page-sidebar">`, sticky, holding up to three separate `.sidebar-card` blocks (never merged into one card - each is its own bordered panel): "Gists from {author}" and "Projects from {author}" (`content.get_user_sidebar_gists`/`get_user_sidebar_projects`, up to 5 each, most-recently-modified first, private projects filtered via `can_view_project`), and "Related Discussions" (same-topic posts). Every card title is a `.sidebar-heading` div (the same class `feed.html`'s left sidebar uses for "Topics"/"Resources"/"Online now" - `.sidebar-card .sidebar-heading` in `sidebar.css`, so it only styles correctly nested inside a `.sidebar-card`, never bare). Each list item reuses the plain `.related-list`/`.related-link`/`.related-title`/`.related-meta` classes (`base.css`) with `content_preview()` for the ellipsis-truncated description.
+- **Middle column** - `.post-page` (`max-width: 720px`): the post article, comments.
+- **Right column** - `<aside class="feed-right">`, the exact same class `feed.html` uses for its Daily Topic widget (sticky, hidden below 1024px via `feed.css`'s own media query - no post-page-specific override needed). Holds the "Featured" cards (`database.get_featured_topics`, up to 3, cached pool + per-request random sample): each is a `.daily-topic-card` with its own `.daily-topic-label` ("Featured") - matching the single Daily Topic card's internal label, since there is no section-level heading here (a `.sidebar-heading` div placed directly in `.feed-right` would NOT be styled, as noted above - the fix used is a per-card label instead of a bare heading).
+
+All three columns are populated by `routers/posts.py` `post_page_context()`, the single context builder shared by the real `/posts/{slug}` route and `happy404.render()` (see the root `CLAUDE.md`), so a decoy happy-404 post page renders with the identical sidebar/featured layout as a real one.
 
 ### Performance
 - `loading="lazy"` on all avatar images

devplacepy/routers/admin/backups.py

@@ -89,6 +89,7 @@ async def admin_backups(request: Request):
             {"name": "Backups", "url": "/admin/backups"},
         ],
         schemas=[website_schema(base)],
+        robots="noindex,nofollow",
     )
     return respond(
         request,

devplacepy/routers/admin/containers.py

@@ -409,7 +409,7 @@ async def container_instance_page(request: Request, uid: str):
             "events": store.list_events(inst["uid"]),
             "schedules": store.list_schedules(inst["uid"]),
             "stats": api.instance_stats(inst["uid"]),
-            "runtime": api.instance_runtime(inst),
+            "runtime": await api.instance_runtime(inst),
             "can_manage": can_manage,
             "admin_section": "containers",
         },

devplacepy/routers/admin/devii_tasks.py

@@ -139,6 +139,7 @@ async def admin_devii_tasks(request: Request, state: str = "active"):
             {"name": "Devii tasks", "url": "/admin/devii-tasks"},
         ],
         schemas=[website_schema(base)],
+        robots="noindex,nofollow",
     )
     return respond(
         request,

devplacepy/routers/admin/gateway_configs.py

@@ -150,6 +150,16 @@ async def save_model(request: Request):
         payload = routing.ModelRouteIn(**body)
     except ValidationError as exc:
         return _validation_error(exc)
+    if payload.fallback_model:
+        fallback_route = routing.model_store.get(payload.fallback_model)
+        if fallback_route is None or fallback_route.kind != payload.kind:
+            return JSONResponse(
+                {
+                    "ok": False,
+                    "error": "Fallback model must be an existing model route of the same kind",
+                },
+                status_code=400,
+            )
     saved = routing.model_store.set(payload)
     audit.record(
         request,

devplacepy/routers/admin/issues.py

@@ -5,11 +5,12 @@ import logging
 from fastapi import APIRouter, Request
 from fastapi.responses import HTMLResponse
 
+from devplacepy.responses import respond
+from devplacepy.schemas import AdminIssuesPlanningOut
 from devplacepy.seo import base_seo_context
 from devplacepy.services.gitea import runtime
 from devplacepy.services.gitea.config import gitea_config
 from devplacepy.services.gitea.planning import collect_open_issues
-from devplacepy.templating import templates
 from devplacepy.utils import require_admin
 
 logger = logging.getLogger(__name__)
@@ -65,4 +66,4 @@ async def admin_issues_planning(request: Request):
         "tickets": tickets,
         "tickets_error": tickets_error,
     }
-    return templates.TemplateResponse(request, "admin_issues_planning.html", context)
+    return respond(request, "admin_issues_planning.html", context, model=AdminIssuesPlanningOut)

devplacepy/routers/admin/services.py

@@ -62,7 +62,7 @@ async def service_detail(request: Request, name: str):
         request,
         title=f"{info['title']} - Services",
         description=info["description"] or f"Configure the {info['title']} service.",
-        robots="noindex",
+        robots="noindex,nofollow",
         breadcrumbs=[
             {"name": "Home", "url": "/feed"},
             {"name": "Admin", "url": "/admin"},

devplacepy/routers/admin/trash.py

@@ -89,6 +89,7 @@ async def admin_trash(request: Request, table: str = "posts", page: int = 1):
             {"name": "Trash", "url": "/admin/trash"},
         ],
         schemas=[website_schema(base)],
+        robots="noindex,nofollow",
     )
     return respond(
         request,

devplacepy/routers/admin/workspaces.py

@@ -1,11 +1,12 @@
 # retoor <retoor@molodetz.nl>
 
+import asyncio
 from typing import Annotated
 
 from fastapi import APIRouter, Depends, Request
 from fastapi.responses import JSONResponse
 
-from devplacepy.database import db, get_table, get_users_by_uids
+from devplacepy.database import get_projects_by_uids, get_table, get_users_by_uids
 from devplacepy.dependencies import json_or_form
 from devplacepy.models import (
     EditorPrefsForm,
@@ -31,19 +32,14 @@ from devplacepy.utils import create_notification, generate_uid, not_found, requi
 router = APIRouter()
 
 
-def _decorate(rows: list[dict]) -> list[dict]:
+async def _decorate(rows: list[dict]) -> list[dict]:
     owner_uids = {row.get("workspace_owner_uid") for row in rows if row.get("workspace_owner_uid")}
     owners = get_users_by_uids(list(owner_uids)) if owner_uids else {}
-    projects = {}
-    if "projects" in db.tables:
-        project_uids = {row.get("project_uid") for row in rows if row.get("project_uid")}
-        for uid in project_uids:
-            found = get_table("projects").find_one(uid=uid)
-            if found:
-                projects[uid] = found
+    project_uids = {row.get("project_uid") for row in rows if row.get("project_uid")}
+    projects = get_projects_by_uids(list(project_uids)) if project_uids else {}
+    views = await asyncio.gather(*(provision.view(row) for row in rows))
     decorated = []
-    for row in rows:
-        view = provision.view(row)
+    for row, view in zip(rows, views):
         owner = owners.get(row.get("workspace_owner_uid", "")) or {}
         project = projects.get(row.get("project_uid", "")) or {}
         view["owner_username"] = owner.get("username", "")
@@ -84,7 +80,7 @@ async def admin_workspaces(request: Request):
     if not isinstance(admin, dict):
         return admin
     context = {
-        "workspaces": _decorate(_all_workspaces()),
+        "workspaces": await _decorate(_all_workspaces()),
         "flags": flags.list_flags(),
         "admin_section": "workspaces",
         "user": admin,
@@ -109,7 +105,7 @@ async def admin_workspaces_data(request: Request):
     if not isinstance(admin, dict):
         return admin
     return JSONResponse(
-        {"workspaces": _decorate(_all_workspaces()), "flags": flags.list_flags()}
+        {"workspaces": await _decorate(_all_workspaces()), "flags": flags.list_flags()}
     )
 
 

devplacepy/routers/auth/token.py

@@ -22,12 +22,6 @@ async def token(
     request: Request,
     data: Annotated[LoginForm, Depends(json_or_form(LoginForm))],
 ):
-    """Issue a DevPlace access token.
-
-    Accepts ``email`` + ``password`` (JSON or form-encoded).  Returns a JSON
-    object with ``access_token``, ``token_type``, and ``expires_in`` on success,
-    or a ``401`` error on bad credentials.
-    """
     identifier = data.email.strip().lower()
     password = data.password
 

devplacepy/routers/battles.py

@@ -0,0 +1,131 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Annotated
+from urllib.parse import quote
+
+from fastapi import APIRouter, Depends, Request
+from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
+
+from devplacepy.database import resolve_object_url
+from devplacepy.dependencies import json_or_form
+from devplacepy.models import WarJoinForm
+from devplacepy.responses import action_result, json_error, respond, wants_json
+from devplacepy.schemas import BattlesOut, WarEventsOut, WarOut
+from devplacepy.seo import base_seo_context, site_url, website_schema
+from devplacepy.services.opinionwar import WarError, rules, store
+from devplacepy.utils import get_current_user, not_found, require_user
+
+router = APIRouter()
+
+
+def _war_error(request: Request, message: str, redirect_url: str):
+    if wants_json(request):
+        return json_error(400, message)
+    separator = "&" if "?" in redirect_url else "?"
+    return RedirectResponse(
+        url=f"{redirect_url}{separator}error={quote(message)}", status_code=302
+    )
+
+
+def _post_url(war: dict) -> str:
+    return resolve_object_url("post", war["post_uid"])
+
+
+@router.get("", response_class=HTMLResponse)
+async def battles_page(
+    request: Request, filter: str = "active", search: str = "", page: int = 1
+):
+    user = get_current_user(request)
+    current_filter = filter if filter in store.FILTERS else "active"
+    battles, pagination = store.list_wars(
+        viewer=user,
+        war_filter=current_filter,
+        search=search,
+        page=max(1, page),
+    )
+    base = site_url(request)
+    seo_ctx = base_seo_context(
+        request,
+        title="Opinion Wars",
+        description=(
+            "Week-long faction battles between developers. Pick a side, fight once "
+            "a day and carry your faction to victory."
+        ),
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Battles", "url": "/battles"},
+        ],
+        schemas=[website_schema(base)],
+    )
+    return respond(
+        request,
+        "battles.html",
+        {
+            **seo_ctx,
+            "user": user,
+            "battles": battles,
+            "current_filter": current_filter,
+            "counts": store.filter_counts(user, search),
+            "search": search,
+            "pagination": pagination,
+        },
+        model=BattlesOut,
+    )
+
+
+@router.get("/{war_uid}")
+async def battle_state(request: Request, war_uid: str):
+    user = get_current_user(request)
+    serialized = store.get_war_serialized(store.get_war(war_uid), user)
+    if not serialized:
+        raise not_found("Battle not found")
+    return JSONResponse(WarOut.model_validate(serialized).model_dump())
+
+
+@router.get("/{war_uid}/events")
+async def battle_events(
+    request: Request, war_uid: str, after: int = 0, limit: int = rules.EVENT_LIMIT_DEFAULT
+):
+    war = store.resolve_if_due(store.get_war(war_uid))
+    if not war:
+        raise not_found("Battle not found")
+    events = store.events_for(war_uid, after_seq=after, limit=limit)
+    return JSONResponse(
+        WarEventsOut.model_validate(
+            {"events": events, "status": war.get("status") or "active"}
+        ).model_dump()
+    )
+
+
+@router.post("/{war_uid}/join")
+async def join_battle(
+    request: Request,
+    war_uid: str,
+    data: Annotated[WarJoinForm, Depends(json_or_form(WarJoinForm))],
+):
+    user = require_user(request)
+    war = store.get_war(war_uid)
+    if not war:
+        raise not_found("Battle not found")
+    url = _post_url(war)
+    try:
+        war = store.join_war(war, user, data.faction, request)
+    except WarError as exc:
+        return _war_error(request, str(exc), url)
+    serialized = store.get_war_serialized(war, user)
+    return action_result(request, url, data={"war": serialized})
+
+
+@router.post("/{war_uid}/fight")
+async def fight_battle(request: Request, war_uid: str):
+    user = require_user(request)
+    war = store.get_war(war_uid)
+    if not war:
+        raise not_found("Battle not found")
+    url = _post_url(war)
+    try:
+        war, damage = store.fight(war, user, request)
+    except WarError as exc:
+        return _war_error(request, str(exc), url)
+    serialized = store.get_war_serialized(war, user)
+    return action_result(request, url, data={"war": serialized, "damage": damage})

devplacepy/routers/dbapi/query.py

@@ -1,5 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
+import asyncio
 import json
 import logging
 
@@ -141,7 +142,8 @@ async def dbapi_query_result(request: Request, uid: str):
     if not path.is_relative_to(DBAPI_DIR.resolve()) or not path.is_file():
         return error(404, "Result not available")
     queue.touch_job(uid, get_int_setting("dbquery_retention_seconds", 604800))
-    return JSONResponse(json.loads(path.read_text(encoding="utf-8")))
+    text = await asyncio.to_thread(path.read_text, encoding="utf-8")
+    return JSONResponse(json.loads(text))
 
 
 @router.websocket("/query/{uid}/ws")

devplacepy/routers/devii.py

@@ -137,8 +137,6 @@ def _validate_target(value: str) -> str:
 
 @router.get("/adopt")
 async def devii_adopt(request: Request):
-    # The terminal's agent logged in; adopt the real session it minted into the browser so both
-    # share one session, then return to where the user was.
     target = _validate_target(request.query_params.get("next", "/"))
     response = RedirectResponse(target, status_code=303)
     svc = _service()
@@ -189,7 +187,9 @@ async def clippy_proxy(request: Request):
         "Content-Type": "application/json",
         "X-App-Reference": "devplace-devii-v-1-0-0",
     }
-    if cfg.get("devii_ai_key"):
+    if user.get("api_key"):
+        headers["Authorization"] = f"Bearer {user['api_key']}"
+    elif cfg.get("devii_ai_key"):
         headers["Authorization"] = f"Bearer {cfg['devii_ai_key']}"
     async with stealth.stealth_async_client(timeout=45.0) as client:
         upstream = await client.post(cfg["devii_ai_url"], content=body, headers=headers)

devplacepy/routers/docs/CLAUDE.md

@@ -66,7 +66,7 @@ The `endpoint()` factory derives `min_role` from `auth` (`public` -> Public, `us
 
 ## Admin-only pages
 
-A group with `"admin": True` (currently `services`, `admin`) is hidden from the sidebar (`docs.py` filters `visible_pages`) and returns 404 for non-admins (`page.get("admin") and not is_admin -> not_found`, same message as unknown slugs). `index.html` wraps the operator links in `{% if user and user.get('role') == 'Admin' %}`.
+A group with `"admin": True` (currently `services`, `admin`, `containers`) is hidden from the sidebar (`docs.py` filters `visible_pages`) and returns 404 for non-admins (`page.get("admin") and not is_admin -> not_found`, same message as unknown slugs). `index.html` wraps the operator links in `{% if user and user.get('role') == 'Admin' %}`.
 
 ## Dynamic Background Services page
 

devplacepy/routers/docs/pages.py

@@ -44,7 +44,6 @@ AUDIENCES = [
 ]
 
 DOCS_PAGES = [
-    # General - how to use the site and Devii (everyone)
     {"slug": "index", "title": "Overview", "kind": "prose", "section": SECTION_GENERAL},
     {
         "slug": "getting-started",
@@ -82,6 +81,12 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_GENERAL,
     },
+    {
+        "slug": "opinion-wars",
+        "title": "Opinion Wars",
+        "kind": "prose",
+        "section": SECTION_GENERAL,
+    },
     {
         "slug": "block-and-mute",
         "title": "Block and mute",
@@ -154,7 +159,6 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_GENERAL,
     },
-    # Legal - the policies the platform is operated under (everyone)
     {
         "slug": "terms",
         "title": "Terms of Service",
@@ -198,7 +202,6 @@ DOCS_PAGES = [
         "section": SECTION_LEGAL,
         "admin": True,
     },
-    # Tools - public developer tools (everyone)
     {
         "slug": "tools-seo",
         "title": "SEO Diagnostics",
@@ -223,7 +226,6 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_TOOLS,
     },
-    # Claude Code - the native subagent, command, and workflow setup under .claude/
     {
         "slug": "claude",
         "title": "Claude Code setup",
@@ -254,7 +256,6 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_CLAUDE,
     },
-    # Components - custom HTML web components with live examples (everyone)
     {
         "slug": "components",
         "title": "Components overview",
@@ -333,7 +334,6 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_COMPONENTS,
     },
-    # Styles - the design system: colors, layout, responsiveness, and hard structural rules (everyone)
     {
         "slug": "styles",
         "title": "Design system overview",
@@ -364,7 +364,6 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_STYLES,
     },
-    # API - developer reference (everyone); admin-only groups are routed to Administration below
     {
         "slug": "authentication",
         "title": "Authentication",
@@ -377,7 +376,6 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_API,
     },
-    # devRant API - legacy-compatible protocol, spread over focused pages
     {
         "slug": "devrant",
         "title": "Overview",
@@ -424,7 +422,6 @@ DOCS_PAGES = [
         {**page, "section": (SECTION_ADMIN if page.get("admin") else SECTION_API)}
         for page in api_doc_pages()
     ],
-    # Administration - operational guides (admins only)
     {
         "slug": "devii-admin",
         "title": "Devii for admins",
@@ -467,7 +464,6 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_ADMIN,
     },
-    # Devii internals - technical reference for the Devii assistant (admins only)
     {
         "slug": "audit-log",
         "title": "Audit Log",
@@ -517,7 +513,6 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_DEVII,
     },
-    # Bots internals - deep technical reference for the autonomous bot fleet (admins only)
     {
         "slug": "bots-internals",
         "title": "Bots internals",
@@ -567,7 +562,6 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_BOTS,
     },
-    # Services - the background service framework and every service in detail (admins only)
     {
         "slug": "services-overview",
         "title": "Services overview",
@@ -645,7 +639,6 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_SERVICES,
     },
-    # Architecture - platform design, structure, and development process (admins only)
     {
         "slug": "architecture",
         "title": "Architecture overview",
@@ -695,7 +688,6 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_ARCH,
     },
-    # Testing - test framework, load testing, and make targets (admins only)
     {
         "slug": "testing",
         "title": "Testing overview",
@@ -731,7 +723,6 @@ DOCS_PAGES = [
         "admin": True,
         "section": SECTION_TESTING,
     },
-    # Production - deployment and operations reference (admins only)
     {
         "slug": "production",
         "title": "Production overview",

devplacepy/routers/feed.py

@@ -14,6 +14,9 @@ from devplacepy.database import (
     get_reactions_by_targets,
     get_user_bookmarks,
     get_polls_by_post_uids,
+)
+from devplacepy.services.opinionwar import store as war_store
+from devplacepy.database import (
     paginate_diverse,
     text_search_clause,
 )
@@ -78,6 +81,27 @@ def get_feed_posts(
     return result, next_cursor
 
 
+def enrich_post_cards(posts, user):
+    post_uids_list = [item["post"]["uid"] for item in posts]
+    attachments_map = get_attachments_batch("post", post_uids_list)
+    recent_comments = get_recent_comments_by_post_uids(post_uids_list, 3, user)
+    reactions_map = get_reactions_by_targets("post", post_uids_list, user)
+    bookmark_set = (
+        get_user_bookmarks(user["uid"], "post", post_uids_list) if user else set()
+    )
+    polls_map = get_polls_by_post_uids(post_uids_list, user)
+    wars_map = war_store.get_wars_by_post_uids(post_uids_list, user)
+    for item in posts:
+        uid = item["post"]["uid"]
+        item["attachments"] = attachments_map.get(uid, [])
+        item["recent_comments"] = recent_comments.get(uid, [])
+        item["reactions"] = reactions_map.get(uid, {"counts": {}, "mine": []})
+        item["bookmarked"] = uid in bookmark_set
+        item["poll"] = polls_map.get(uid)
+        item["war"] = wars_map.get(uid)
+    return posts
+
+
 @router.get("", response_class=HTMLResponse)
 async def feed_page(
     request: Request,
@@ -93,21 +117,7 @@ async def feed_page(
     daily_topic = get_daily_topic()
     online_users = presence.online_users()
 
-    post_uids_list = [item["post"]["uid"] for item in posts]
-    attachments_map = get_attachments_batch("post", post_uids_list)
-    recent_comments = get_recent_comments_by_post_uids(post_uids_list, 3, user)
-    reactions_map = get_reactions_by_targets("post", post_uids_list, user)
-    bookmark_set = (
-        get_user_bookmarks(user["uid"], "post", post_uids_list) if user else set()
-    )
-    polls_map = get_polls_by_post_uids(post_uids_list, user)
-    for item in posts:
-        uid = item["post"]["uid"]
-        item["attachments"] = attachments_map.get(uid, [])
-        item["recent_comments"] = recent_comments.get(uid, [])
-        item["reactions"] = reactions_map.get(uid, {"counts": {}, "mine": []})
-        item["bookmarked"] = uid in bookmark_set
-        item["poll"] = polls_map.get(uid)
+    posts = enrich_post_cards(posts, user)
 
     seo_ctx = list_page_seo(
         request,

devplacepy/routers/game/farm.py

@@ -9,6 +9,7 @@ from devplacepy.models import GameSlotForm
 from devplacepy.responses import respond, wants_json
 from devplacepy.schemas import GameFarmViewOut
 from devplacepy.services.game import GameError, store
+from devplacepy.services.audit import record as audit
 from devplacepy.utils import (
     create_notification,
     get_current_user,
@@ -58,10 +59,24 @@ async def water_farm(
     if not owner:
         raise HTTPException(status_code=404, detail="Farm not found")
     try:
-        store.water(viewer, owner, data.slot)
+        result = store.water(viewer, owner, data.slot)
     except GameError as exc:
         return action_error(request, str(exc), f"/game/farm/{username}")
     track_action(viewer["uid"], "water")
+    audit.record(
+        request,
+        "game.water",
+        user=viewer,
+        target_type="user",
+        target_uid=owner["uid"],
+        target_label=owner["username"],
+        metadata=result,
+        summary=(
+            f"{viewer['username']} watered {owner['username']}'s build on plot "
+            f"{result['slot']} and earned {result['reward_coins']} coins"
+        ),
+        links=[audit.target("user", owner["uid"], owner["username"])],
+    )
     await notify_farm(owner["username"])
     if wants_json(request):
         payload = store.serialize_farm(
@@ -87,6 +102,31 @@ async def steal_farm(
     track_action(owner["uid"], "got_stolen_from")
     if result.get("underdog_triggered"):
         track_action(viewer["uid"], "underdog_raid")
+    audit.record(
+        request,
+        "game.steal",
+        user=viewer,
+        target_type="user",
+        target_uid=owner["uid"],
+        target_label=owner["username"],
+        metadata={
+            "thief_uid": viewer["uid"],
+            "thief_username": viewer["username"],
+            "owner_uid": owner["uid"],
+            "owner_username": owner["username"],
+            "slot": result["slot"],
+            "crop": result["crop"],
+            "coins": result["coins"],
+            "share": result["share"],
+            "underdog_triggered": result.get("underdog_triggered", False),
+        },
+        summary=(
+            f"{viewer['username']} raided {owner['username']}'s Code Farm and took "
+            f"{result['coins']} coins ({round(result['share'] * 100)}%) from their "
+            f"{result['crop_name']} build"
+        ),
+        links=[audit.target("user", owner["uid"], owner["username"])],
+    )
     create_notification(
         owner["uid"],
         "harvest_stolen",

devplacepy/routers/game/index.py

@@ -80,8 +80,21 @@ async def _respond_action(request: Request, user: dict, fn, on_success=None):
 @router.post("/plant")
 async def game_plant(request: Request, data: Annotated[GamePlantForm, Form()]):
     user = require_user(request)
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.plant",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} planted {result['crop']} on plot "
+                f"{result['slot']} for {result['spent']} coins"
+            ),
+        )
+
     return await _respond_action(
-        request, user, lambda: store.plant(user, data.slot, data.crop)
+        request, user, lambda: store.plant(user, data.slot, data.crop), recorded
     )
 
 
@@ -92,6 +105,16 @@ async def game_harvest(request: Request, data: Annotated[GameSlotForm, Form()]):
     def reward(result):
         track_action(user["uid"], "harvest")
         award_rewards(user["uid"], economy.site_xp_for(result.get("xp", 0)))
+        audit.record(
+            request,
+            "game.harvest",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} harvested {result['crop']} on plot "
+                f"{result['slot']} for {result['coins']} coins"
+            ),
+        )
 
     return await _respond_action(
         request, user, lambda: store.harvest(user, data.slot), reward
@@ -101,25 +124,79 @@ async def game_harvest(request: Request, data: Annotated[GameSlotForm, Form()]):
 @router.post("/buy-plot")
 async def game_buy_plot(request: Request):
     user = require_user(request)
-    return await _respond_action(request, user, lambda: store.buy_plot(user))
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.plot.buy",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} bought Code Farm plot "
+                f"{result['plot_count']} for {result['spent']} coins"
+            ),
+        )
+
+    return await _respond_action(request, user, lambda: store.buy_plot(user), recorded)
 
 
 @router.post("/upgrade")
 async def game_upgrade(request: Request):
     user = require_user(request)
-    return await _respond_action(request, user, lambda: store.upgrade_ci(user))
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.ci.upgrade",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} upgraded Code Farm CI to tier "
+                f"{result['ci_tier']} for {result['spent']} coins"
+            ),
+        )
+
+    return await _respond_action(request, user, lambda: store.upgrade_ci(user), recorded)
 
 
 @router.post("/fertilize")
 async def game_fertilize(request: Request, data: Annotated[GameSlotForm, Form()]):
     user = require_user(request)
-    return await _respond_action(request, user, lambda: store.fertilize(user, data.slot))
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.fertilize",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} fertilized plot {result['slot']} "
+                f"for {result['spent']} coins"
+            ),
+        )
+
+    return await _respond_action(
+        request, user, lambda: store.fertilize(user, data.slot), recorded
+    )
 
 
 @router.post("/daily")
 async def game_daily(request: Request):
     user = require_user(request)
-    return await _respond_action(request, user, lambda: store.claim_daily(user))
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.daily.claim",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} claimed the Code Farm daily bonus of "
+                f"{result['reward']} coins (streak {result['streak']})"
+            ),
+        )
+
+    return await _respond_action(request, user, lambda: store.claim_daily(user), recorded)
 
 
 @router.post("/grant")
@@ -141,8 +218,21 @@ async def game_claim_grant(request: Request):
 @router.post("/perk")
 async def game_perk(request: Request, data: Annotated[GamePerkForm, Form()]):
     user = require_user(request)
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.perk.upgrade",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} upgraded perk {result['perk']} to level "
+                f"{result['level']} for {result['spent']} coins"
+            ),
+        )
+
     return await _respond_action(
-        request, user, lambda: store.upgrade_perk(user, data.perk)
+        request, user, lambda: store.upgrade_perk(user, data.perk), recorded
     )
 
 
@@ -168,8 +258,21 @@ async def game_prestige(request: Request):
 @router.post("/legacy")
 async def game_legacy(request: Request, data: Annotated[GameLegacyForm, Form()]):
     user = require_user(request)
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.legacy.upgrade",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} upgraded Legacy {result['key']} to level "
+                f"{result['level']} for {result['spent']} stars"
+            ),
+        )
+
     return await _respond_action(
-        request, user, lambda: store.upgrade_legacy(user, data.key)
+        request, user, lambda: store.upgrade_legacy(user, data.key), recorded
     )
 
 
@@ -179,6 +282,16 @@ async def game_claim_quest(request: Request, data: Annotated[GameQuestForm, Form
 
     def reward(result):
         award_rewards(user["uid"], economy.site_xp_for(result.get("reward_xp", 0)))
+        audit.record(
+            request,
+            "game.quest.claim",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} claimed quest {result['kind']} for "
+                f"{result['reward_coins']} coins"
+            ),
+        )
 
     return await _respond_action(
         request, user, lambda: store.claim_quest(user, data.quest, data.scope), reward
@@ -251,8 +364,21 @@ async def game_buy_infrastructure(request: Request, data: Annotated[GameInfraFor
 @router.post("/mastery")
 async def game_upgrade_mastery(request: Request, data: Annotated[GameMasteryForm, Form()]):
     user = require_user(request)
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.mastery.upgrade",
+            user=user,
+            metadata=result,
+            summary=(
+                f"{user['username']} upgraded Mastery {result['key']} to level "
+                f"{result['level']} for {result['spent']} mastery points"
+            ),
+        )
+
     return await _respond_action(
-        request, user, lambda: store.upgrade_mastery(user, data.key)
+        request, user, lambda: store.upgrade_mastery(user, data.key), recorded
     )
 
 
@@ -281,6 +407,16 @@ async def game_buy_cosmetic(request: Request, data: Annotated[GameCosmeticForm,
 @router.post("/cosmetics/equip")
 async def game_equip_cosmetic(request: Request, data: Annotated[GameCosmeticForm, Form()]):
     user = require_user(request)
+
+    def recorded(result):
+        audit.record(
+            request,
+            "game.cosmetic.equip",
+            user=user,
+            metadata=result,
+            summary=f"{user['username']} equipped Code Farm title {result['active_title']}",
+        )
+
     return await _respond_action(
-        request, user, lambda: store.equip_title(user, data.key)
+        request, user, lambda: store.equip_title(user, data.key), recorded
     )

devplacepy/routers/issues/attachments.py

@@ -8,10 +8,12 @@ from fastapi.responses import JSONResponse
 
 from devplacepy.attachments import (
     get_attachments,
+    get_orphan_attachments_batch,
     link_attachments,
-    mirror_attachment_to_gitea,
-    remove_gitea_asset,
+    schedule_gitea_mirror,
+    schedule_gitea_removal,
     soft_delete_attachment,
+    split_attachment_uids,
 )
 from devplacepy.database import get_table
 from devplacepy.dependencies import json_or_form
@@ -28,10 +30,6 @@ logger = logging.getLogger(__name__)
 router = APIRouter()
 
 
-def _split_uids(raw) -> list[str]:
-    return [u.strip() for item in raw or [] for u in str(item).split(",") if u.strip()]
-
-
 def _can_modify(att: dict, user: dict | None, is_open: bool) -> bool:
     if not user or not is_open:
         return False
@@ -47,21 +45,6 @@ def _payload(rows: list[dict], user: dict | None, is_open: bool) -> list[dict]:
     return items
 
 
-def _claim_orphans(uids: list[str], user: dict) -> list[str]:
-    admin = is_admin(user)
-    owned = []
-    for uid in uids:
-        row = get_table("attachments").find_one(uid=uid, deleted_at=None)
-        if not row:
-            continue
-        if row.get("user_uid") and row["user_uid"] != user["uid"] and not admin:
-            continue
-        if row.get("target_uid"):
-            continue
-        owned.append(uid)
-    return owned
-
-
 async def _load_issue(number: int) -> dict:
     try:
         return await runtime.get_client().get_issue(number)
@@ -106,12 +89,14 @@ async def add_issue_attachment(
             summary=f"{user['username']} tried to attach to closed issue #{number}",
         )
         return json_error(409, "Attachments cannot be changed on a closed issue")
-    owned = _claim_orphans(_split_uids(data.attachment_uids), user)
+    owned = get_orphan_attachments_batch(
+        split_attachment_uids(data.attachment_uids), user, admin=is_admin(user)
+    )
     if not owned:
         return json_error(400, "No valid attachments to add")
     link_attachments(owned, "issue", str(number))
     for uid in owned:
-        await mirror_attachment_to_gitea(uid)
+        schedule_gitea_mirror(uid)
     audit.record(
         request,
         "issue.attachment.add",
@@ -161,7 +146,7 @@ async def delete_issue_attachment(request: Request, number: int, uid: str):
         )
         return json_error(409, "Attachments cannot be changed on a closed issue")
     soft_delete_attachment(uid, deleted_by=user["uid"])
-    await remove_gitea_asset(att)
+    schedule_gitea_removal(att)
     audit.record(
         request,
         "issue.attachment.delete",
@@ -202,12 +187,14 @@ async def add_comment_attachment(
     issue = await _load_issue(number)
     if issue.get("state") != STATE_OPEN:
         return json_error(409, "Attachments cannot be changed on a closed issue")
-    owned = _claim_orphans(_split_uids(data.attachment_uids), user)
+    owned = get_orphan_attachments_batch(
+        split_attachment_uids(data.attachment_uids), user, admin=is_admin(user)
+    )
     if not owned:
         return json_error(400, "No valid attachments to add")
     link_attachments(owned, "issue_comment", str(cid))
     for uid in owned:
-        await mirror_attachment_to_gitea(uid)
+        schedule_gitea_mirror(uid)
     audit.record(
         request,
         "issue.attachment.add",
@@ -250,7 +237,7 @@ async def delete_comment_attachment(
     if issue.get("state") != STATE_OPEN:
         return json_error(409, "Attachments cannot be changed on a closed issue")
     soft_delete_attachment(uid, deleted_by=user["uid"])
-    await remove_gitea_asset(att)
+    schedule_gitea_removal(att)
     audit.record(
         request,
         "issue.attachment.delete",

devplacepy/routers/issues/comment.py

@@ -5,7 +5,12 @@ from typing import Annotated
 
 from fastapi import Depends, APIRouter, Request
 
-from devplacepy.attachments import link_attachments, mirror_attachment_to_gitea
+from devplacepy.attachments import (
+    get_orphan_attachments_batch,
+    link_attachments,
+    schedule_gitea_mirror,
+    split_attachment_uids,
+)
 from devplacepy.database import get_table
 from devplacepy.models import IssueCommentForm
 from devplacepy.responses import action_result, json_error
@@ -21,18 +26,6 @@ logger = logging.getLogger(__name__)
 router = APIRouter()
 
 
-def _owned_orphan_uids(raw: list[str], user: dict) -> list[str]:
-    flat = [u.strip() for item in raw or [] for u in str(item).split(",") if u.strip()]
-    owned = []
-    for uid in flat:
-        row = get_table("attachments").find_one(uid=uid, deleted_at=None)
-        if not row or row.get("target_uid"):
-            continue
-        if row.get("user_uid") and row["user_uid"] != user["uid"]:
-            continue
-        owned.append(uid)
-    return owned
-
 def _notify_admins(actor: dict, number: int) -> None:
     for admin in get_table("users").find(role="Admin"):
         if admin["uid"] == actor["uid"]:
@@ -74,11 +67,13 @@ async def comment_issue(
 
     comment_id = int(comment.get("id", 0))
     store.record_comment_author(comment_id, number, user["uid"])
-    owned = _owned_orphan_uids(data.attachment_uids, user)
+    owned = get_orphan_attachments_batch(
+        split_attachment_uids(data.attachment_uids), user
+    )
     if owned:
         link_attachments(owned, "issue_comment", str(comment_id))
         for uid in owned:
-            await mirror_attachment_to_gitea(uid)
+            schedule_gitea_mirror(uid)
     background.submit(_notify_admins, user, number)
     audit.record(
         request,

devplacepy/routers/issues/create.py

@@ -6,7 +6,7 @@ from typing import Annotated
 from fastapi import Depends, APIRouter, Request
 from fastapi.responses import JSONResponse
 
-from devplacepy.database import get_table
+from devplacepy.attachments import get_orphan_attachments_batch, split_attachment_uids
 from devplacepy.models import IssueForm
 from devplacepy.responses import json_error
 from devplacepy.schemas import IssueJobOut
@@ -20,19 +20,6 @@ logger = logging.getLogger(__name__)
 router = APIRouter()
 
 
-def _owned_orphan_uids(raw: list[str], user: dict) -> list[str]:
-    flat = [u.strip() for item in raw or [] for u in str(item).split(",") if u.strip()]
-    owned = []
-    for uid in flat:
-        row = get_table("attachments").find_one(uid=uid, deleted_at=None)
-        if not row or row.get("target_uid"):
-            continue
-        if row.get("user_uid") and row["user_uid"] != user["uid"]:
-            continue
-        owned.append(uid)
-    return owned
-
-
 @router.post("/create")
 async def create_issue(request: Request, data: Annotated[IssueForm, Depends(json_or_form(IssueForm))]):
     user = require_user(request)
@@ -45,7 +32,9 @@ async def create_issue(request: Request, data: Annotated[IssueForm, Depends(json
             "author_uid": user["uid"],
             "title": title,
             "description": data.description.strip(),
-            "attachment_uids": _owned_orphan_uids(data.attachment_uids, user),
+            "attachment_uids": get_orphan_attachments_batch(
+                split_attachment_uids(data.attachment_uids), user
+            ),
         },
         "user",
         user["uid"],

devplacepy/routers/messages.py

@@ -20,7 +20,6 @@ from devplacepy.templating import clear_messages_cache
 from devplacepy.utils import (
     require_user,
     time_ago,
-    is_admin,
     _user_from_session,
     _user_from_api_key,
 )
@@ -31,6 +30,7 @@ from devplacepy.services import presence
 from devplacepy.services.audit import record as audit
 from devplacepy.services.correction import PENDING_SCOPE_KEY
 from devplacepy.dependencies import json_or_form
+from devplacepy.services.moderation.screening import ContentRefused
 from devplacepy.services.messaging import (
     issue_ticket,
     message_frame,
@@ -38,6 +38,7 @@ from devplacepy.services.messaging import (
     message_relay,
     persist_message,
     redeem_ticket,
+    stamp_content_revision,
 )
 
 logger = logging.getLogger(__name__)
@@ -113,22 +114,38 @@ def get_conversations(user_uid: str):
         conv.pop("other_uid", None)
     return conversations
 
-def get_conversation_messages(user_uid: str, other_uid: str):
+def get_conversation_messages(user_uid: str, other_uid: str, before: str = ""):
     if other_uid in get_blocked_uids(user_uid):
         return [], None
     if "messages" not in db.tables:
         return [], get_users_by_uids([other_uid]).get(other_uid)
-    msgs = list(
-        db.query(
-            "SELECT * FROM messages"
-            " WHERE (sender_uid = :me AND receiver_uid = :other)"
-            "    OR (sender_uid = :other AND receiver_uid = :me)"
-            " ORDER BY created_at DESC, id DESC LIMIT :lim",
-            me=user_uid,
-            other=other_uid,
-            lim=CONVERSATION_MESSAGE_LIMIT,
+    before = (before or "").strip()
+    if before:
+        msgs = list(
+            db.query(
+                "SELECT * FROM messages"
+                " WHERE ((sender_uid = :me AND receiver_uid = :other)"
+                "    OR (sender_uid = :other AND receiver_uid = :me))"
+                "   AND created_at < :before"
+                " ORDER BY created_at DESC, id DESC LIMIT :lim",
+                me=user_uid,
+                other=other_uid,
+                before=before,
+                lim=CONVERSATION_MESSAGE_LIMIT,
+            )
+        )
+    else:
+        msgs = list(
+            db.query(
+                "SELECT * FROM messages"
+                " WHERE (sender_uid = :me AND receiver_uid = :other)"
+                "    OR (sender_uid = :other AND receiver_uid = :me)"
+                " ORDER BY created_at DESC, id DESC LIMIT :lim",
+                me=user_uid,
+                other=other_uid,
+                lim=CONVERSATION_MESSAGE_LIMIT,
+            )
         )
-    )
     msgs.reverse()
 
     user_ids = list({m["sender_uid"] for m in msgs} | {other_uid})
@@ -158,7 +175,9 @@ def get_conversation_messages(user_uid: str, other_uid: str):
     return result, other_user
 
 @router.get("", response_class=HTMLResponse)
-async def messages_page(request: Request, with_uid: str = None, search: str = ""):
+async def messages_page(
+    request: Request, with_uid: str = None, search: str = "", before: str = ""
+):
     user = require_user(request)
     conversations = get_conversations(user["uid"])
 
@@ -174,25 +193,28 @@ async def messages_page(request: Request, with_uid: str = None, search: str = ""
     other_online = False
     other_last_seen = None
     if with_uid:
-        messages, other_user = get_conversation_messages(user["uid"], with_uid)
-        mark_conversation_read(user["uid"], with_uid)
-        mark_notifications_read_by_target(
-            user["uid"], f"/messages?with_uid={with_uid}"
+        messages, other_user = get_conversation_messages(
+            user["uid"], with_uid, before=before
         )
         current_conversation = with_uid
         other_online = presence.is_online(other_user)
         other_last_seen = other_user.get("last_seen") if other_user else None
-        audit.record(
-            request,
-            "message.read_on_view",
-            user=user,
-            target_type="user",
-            target_uid=with_uid,
-            target_label=other_user.get("username") if other_user else None,
-            metadata={"message_count": len(messages)},
-            summary=f"{user['username']} read messages from {other_user.get('username') if other_user else with_uid}",
-            links=[audit.target("user", with_uid, other_user.get("username") if other_user else None)],
-        )
+        if not before:
+            mark_conversation_read(user["uid"], with_uid)
+            mark_notifications_read_by_target(
+                user["uid"], f"/messages?with_uid={with_uid}"
+            )
+            audit.record(
+                request,
+                "message.read_on_view",
+                user=user,
+                target_type="user",
+                target_uid=with_uid,
+                target_label=other_user.get("username") if other_user else None,
+                metadata={"message_count": len(messages)},
+                summary=f"{user['username']} read messages from {other_user.get('username') if other_user else with_uid}",
+                links=[audit.target("user", with_uid, other_user.get("username") if other_user else None)],
+            )
 
     seo_ctx = base_seo_context(
         request,
@@ -259,7 +281,7 @@ async def send_message(request: Request, data: Annotated[MessageForm, Depends(js
         return action_result(request, "/messages")
 
     ai_processed = await _finalize_and_broadcast(
-        user, message, request, client_id=data.client_id
+        user, message, request, client_id=data.client_id, wait_ai=True
     )
     frame = message_frame(
         message, user.get("username", ""), data.client_id,
@@ -271,31 +293,107 @@ async def send_message(request: Request, data: Annotated[MessageForm, Depends(js
 
 async def broadcast_message(
     sender: dict, message: dict, client_id: Optional[str] = None,
-    ai_processed: bool = False,
+    ai_processed: bool = False, ai_pending: bool = False,
 ) -> None:
     frame = message_frame(
         message, sender.get("username", ""), client_id,
         sender_role=sender.get("role"), ai_processed=ai_processed,
     )
-    message_hub.mark_delivered(message["uid"])
+    frame["ai_pending"] = ai_pending
+    if not ai_processed:
+        message_hub.mark_delivered(message["uid"])
     targets = [message["sender_uid"], message["receiver_uid"]]
     await message_hub.send_to_users(targets, frame)
 
+async def _await_ai_and_push(
+    sender: dict, message: dict, pending: list, client_id: Optional[str]
+) -> bool:
+    await asyncio.gather(*pending, return_exceptions=True)
+    pending.clear()
+    row = get_table("messages").find_one(uid=message["uid"])
+    if not row:
+        return False
+    changed = row.get("content") != message.get("content")
+    if changed:
+        message["content"] = row["content"]
+        stamp_content_revision(message["uid"])
+    await broadcast_message(sender, message, client_id, ai_processed=True)
+    return changed
+
 async def _finalize_and_broadcast(
-    sender: dict, message: dict, request: object, client_id: Optional[str] = None
+    sender: dict,
+    message: dict,
+    request: object,
+    client_id: Optional[str] = None,
+    wait_ai: bool = True,
 ) -> bool:
-    message_hub.mark_delivered(message["uid"])
     scope = getattr(request, "scope", None)
     pending = scope.get(PENDING_SCOPE_KEY) if scope is not None else None
-    ai_processed = bool(pending)
-    if pending:
-        await asyncio.gather(*pending, return_exceptions=True)
-        pending.clear()
-        row = get_table("messages").find_one(uid=message["uid"])
-        if row:
-            message["content"] = row["content"]
-    await broadcast_message(sender, message, client_id, ai_processed=ai_processed)
-    return ai_processed
+    has_pending = bool(pending)
+    await broadcast_message(
+        sender, message, client_id, ai_processed=False, ai_pending=has_pending
+    )
+    if not pending:
+        return False
+    if wait_ai:
+        return await _await_ai_and_push(sender, message, pending, client_id)
+    asyncio.create_task(_await_ai_and_push(sender, message, pending, client_id))
+    return False
+
+SYNC_LIMIT = 200
+
+
+async def _sync_missed(user_uid: str, data: dict, websocket: WebSocket) -> None:
+    since = str(data.get("since") or "").strip()
+    with_uid = str(data.get("with_uid") or "").strip()
+    if not since or "messages" not in db.tables:
+        return
+    if with_uid:
+        rows = list(
+            db.query(
+                "SELECT * FROM messages"
+                " WHERE ((sender_uid = :me AND receiver_uid = :other)"
+                "    OR (sender_uid = :other AND receiver_uid = :me))"
+                "   AND (created_at > :since"
+                "        OR (updated_at IS NOT NULL AND updated_at > :since))"
+                " ORDER BY created_at ASC, id ASC LIMIT :lim",
+                me=user_uid,
+                other=with_uid,
+                since=since,
+                lim=SYNC_LIMIT,
+            )
+        )
+    else:
+        rows = list(
+            db.query(
+                "SELECT * FROM messages"
+                " WHERE (sender_uid = :me OR receiver_uid = :me)"
+                "   AND (created_at > :since"
+                "        OR (updated_at IS NOT NULL AND updated_at > :since))"
+                " ORDER BY created_at ASC, id ASC LIMIT :lim",
+                me=user_uid,
+                since=since,
+                lim=SYNC_LIMIT,
+            )
+        )
+    if not rows:
+        return
+    sender_uids = {row["sender_uid"] for row in rows}
+    senders = get_users_by_uids(list(sender_uids)) if sender_uids else {}
+    for row in rows:
+        sender = senders.get(row["sender_uid"]) or {}
+        frame = message_frame(
+            dict(row),
+            sender.get("username", ""),
+            sender_role=sender.get("role"),
+            ai_processed=bool(row.get("updated_at")),
+        )
+        try:
+            await websocket.send_json(frame)
+        except Exception:  # noqa: BLE001
+            logger.debug("sync frame dropped for %s", user_uid)
+            return
+
 
 def _resolve_ws_user(websocket: WebSocket):
     user = _user_from_session(websocket)
@@ -351,20 +449,34 @@ async def messages_ws(websocket: WebSocket):
                 attachment_uids = [str(a) for a in raw_attachments][:MAX_WS_ATTACHMENTS]
                 if not receiver_uid:
                     continue
-                message = persist_message(
-                    user,
-                    receiver_uid,
-                    content,
-                    attachment_uids,
-                    request=websocket,
-                    origin="websocket",
-                )
+                try:
+                    message = persist_message(
+                        user,
+                        receiver_uid,
+                        content,
+                        attachment_uids,
+                        request=websocket,
+                        origin="websocket",
+                    )
+                except ContentRefused as exc:
+                    await websocket.send_json(
+                        {
+                            "type": "error",
+                            "client_id": client_id,
+                            "text": exc.message,
+                        }
+                    )
+                    continue
                 if message is None:
                     await websocket.send_json(
                         {"type": "error", "client_id": client_id, "text": "Message not sent."}
                     )
                     continue
-                await _finalize_and_broadcast(user, message, websocket, client_id)
+                await _finalize_and_broadcast(
+                    user, message, websocket, client_id, wait_ai=False
+                )
+            elif kind == "sync":
+                await _sync_missed(user_uid, data, websocket)
             elif kind == "typing":
                 receiver_uid = str(data.get("receiver_uid", "")).strip()
                 if receiver_uid:

devplacepy/routers/posts.py

@@ -12,6 +12,8 @@ from devplacepy.database import (
     resolve_by_slug,
     resolve_object_url,
     mark_notifications_read_by_target,
+    get_featured_topics,
+    get_blocked_uids,
 )
 from devplacepy.utils import (
     get_current_user,
@@ -29,6 +31,8 @@ from devplacepy.content import (
     detail_context,
     canonical_redirect,
     first_image_url,
+    get_user_sidebar_gists,
+    get_user_sidebar_projects,
 )
 from devplacepy.responses import respond, action_result
 from devplacepy.schemas import PostDetailOut
@@ -37,10 +41,12 @@ from devplacepy.seo import (
     site_url,
     website_schema,
     discussion_forum_posting,
+    comment_schema_list,
 )
 from devplacepy.attachments import save_inline_image
 from devplacepy.models import PostForm, PostEditForm
 from devplacepy.services.audit import record as audit
+from devplacepy.services.opinionwar import store as war_store
 from devplacepy.dependencies import json_or_form
 
 logger = logging.getLogger(__name__)
@@ -87,6 +93,7 @@ async def create_post(request: Request, data: Annotated[PostForm, Depends(json_o
     )
 
     create_poll(uid, user, data.poll_question, data.poll_options, request)
+    war_store.create_war(uid, user, data.war_faction_a, data.war_faction_b, request)
     url = f"/posts/{post_slug}"
     return action_result(request, url, data={"uid": uid, "slug": post_slug, "url": url})
 
@@ -134,20 +141,26 @@ def create_poll(
         links=[audit.poll(poll_uid, question), audit.parent("post", post_uid)],
     )
 
-@router.get("/{post_slug}", response_class=HTMLResponse)
-async def view_post(request: Request, post_slug: str):
-    user = get_current_user(request)
-    detail = load_detail("posts", "post", post_slug, user)
-    if not detail:
-        raise not_found("Post not found")
+def _next_post_url(post: dict, viewer: dict | None) -> str | None:
+    if "posts" not in db.tables:
+        return None
+    blocked = get_blocked_uids(viewer["uid"]) if viewer else frozenset()
+    rows = db.query(
+        "SELECT slug, uid, user_uid FROM posts WHERE created_at < :created_at "
+        "AND deleted_at IS NULL ORDER BY created_at DESC LIMIT :limit",
+        created_at=post["created_at"],
+        limit=10,
+    )
+    for row in rows:
+        if row["user_uid"] not in blocked:
+            return f"/posts/{row['slug'] or row['uid']}"
+    return None
+
+
+def post_page_context(
+    request: Request, user: dict | None, detail: dict, *, robots: str | None = None
+) -> dict:
     post = detail["item"]
-    redirect = canonical_redirect("posts", post, post_slug)
-    if redirect:
-        return redirect
-    if user:
-        mark_notifications_read_by_target(
-            user["uid"], resolve_object_url("post", post["uid"])
-        )
     author = detail["author"]
     top_level = detail["comments"]
 
@@ -159,10 +172,12 @@ async def view_post(request: Request, post_slug: str):
 
     comment_count = count_all(top_level)
     base = site_url(request)
+    next_post_url = _next_post_url(post, user)
     seo_ctx = base_seo_context(
         request,
         title=post.get("title") or "Post",
         description=post.get("content", ""),
+        robots=robots or "index,follow",
         seo_target=("post", post["uid"]),
         breadcrumbs=[
             {"name": "Home", "url": "/feed"},
@@ -174,10 +189,16 @@ async def view_post(request: Request, post_slug: str):
         ],
         og_type="article",
         og_image=first_image_url(post, detail["attachments"]),
+        next_url=next_post_url,
         schemas=[
             website_schema(base),
             discussion_forum_posting(
-                post, author, comment_count, detail["star_count"], base
+                post,
+                author,
+                comment_count,
+                detail["star_count"],
+                base,
+                comments=comment_schema_list(top_level, base),
             ),
         ],
     )
@@ -200,21 +221,43 @@ async def view_post(request: Request, post_slug: str):
                 }
             )
 
+    author_uid = post["user_uid"]
+    return detail_context(
+        request,
+        user,
+        detail,
+        "post",
+        seo_ctx,
+        {
+            "comment_count": comment_count,
+            "related_posts": related_posts,
+            "topics": list(TOPICS),
+            "featured_topics": get_featured_topics(3),
+            "author_gists": get_user_sidebar_gists(author_uid, 5),
+            "author_projects": get_user_sidebar_projects(author_uid, user, 5),
+            "next_post_url": next_post_url,
+        },
+    )
+
+
+@router.get("/{post_slug}", response_class=HTMLResponse)
+async def view_post(request: Request, post_slug: str):
+    user = get_current_user(request)
+    detail = load_detail("posts", "post", post_slug, user)
+    if not detail:
+        raise not_found("Post not found")
+    post = detail["item"]
+    redirect = canonical_redirect("posts", post, post_slug)
+    if redirect:
+        return redirect
+    if user:
+        mark_notifications_read_by_target(
+            user["uid"], resolve_object_url("post", post["uid"])
+        )
     return respond(
         request,
         "post.html",
-        detail_context(
-            request,
-            user,
-            detail,
-            "post",
-            seo_ctx,
-            {
-                "comment_count": comment_count,
-                "related_posts": related_posts,
-                "topics": list(TOPICS),
-            },
-        ),
+        post_page_context(request, user, detail),
         model=PostDetailOut,
     )
 

devplacepy/routers/profile/index.py

@@ -17,6 +17,9 @@ from devplacepy.database import (
     get_reactions_by_targets,
     get_user_bookmarks,
     get_polls_by_post_uids,
+)
+from devplacepy.services.opinionwar import store as war_store
+from devplacepy.database import (
     get_activity_heatmap,
     get_activity_months,
     get_streaks,
@@ -203,11 +206,13 @@ async def profile_page(
             else set()
         )
         polls_map = get_polls_by_post_uids(post_uids, current_user)
+        wars_map = war_store.get_wars_by_post_uids(post_uids, current_user)
         for item in posts:
             uid = item["post"]["uid"]
             item["reactions"] = reactions_map.get(uid, {"counts": {}, "mine": []})
             item["bookmarked"] = uid in bookmark_set
             item["poll"] = polls_map.get(uid)
+            item["war"] = wars_map.get(uid)
 
     badges = list(get_table("badges").find(user_uid=profile_user["uid"]))
     for b in badges:

devplacepy/routers/projects/containers/instances.py

@@ -83,8 +83,6 @@ async def containers_json(request: Request, project_slug: str):
         }
     )
 
-# ---------------- instances ----------------
-
 @router.post("/{project_slug}/containers/instances")
 async def create_instance(
     request: Request, project_slug: str, data: Annotated[ContainerInstanceForm, Depends(json_or_form(ContainerInstanceForm))]
@@ -133,7 +131,7 @@ async def instance_detail(request: Request, project_slug: str, uid: str):
             "events": store.list_events(uid),
             "schedules": store.list_schedules(uid),
             "stats": api.instance_stats(uid),
-            "runtime": api.instance_runtime(inst),
+            "runtime": await api.instance_runtime(inst),
         }
     )
 

devplacepy/routers/projects/containers/workspace.py

@@ -74,11 +74,12 @@ async def workspace_page(request: Request, slug: str):
     profile = editor.resolve(user["uid"], instance)
     context = {
         "project": project,
-        "workspace": provision.view(instance) if instance else None,
+        "workspace": await provision.view(instance) if instance else None,
         "has_workspace": bool(instance),
         "viewer_can_workspace": True,
         "workspace_count": provision.count_for_owner(user["uid"]),
         "max_workspaces": limits.max_workspaces,
+        "unlimited_workspaces": limits.unlimited,
         "editor_url": (
             f"/projects/{slug}/containers/instances/{instance['uid']}/code/"
             if instance
@@ -129,7 +130,7 @@ async def workspace_open(request: Request, slug: str):
     )
     provision.write_manifest(instance)
     return action_result(
-        request, f"/projects/{slug}/workspace", data=provision.view(instance)
+        request, f"/projects/{slug}/workspace", data=await provision.view(instance)
     )
 
 
@@ -310,12 +311,18 @@ async def editor_proxy(request: Request, slug: str, uid: str, path: str = ""):
     if denial is not None:
         return denial
     if instance.get("suspended_at"):
-        return Response("this workspace is suspended", status_code=403)
+        return Response(
+            "this workspace is suspended", status_code=403, media_type="text/plain"
+        )
     if instance.get("status") != store.ST_RUNNING:
-        return Response("this workspace is not running", status_code=409)
+        return Response(
+            "this workspace is not running", status_code=409, media_type="text/plain"
+        )
     host, port = provision.editor_target(instance)
     if not host or not port:
-        return Response("the editor has no reachable port", status_code=502)
+        return Response(
+            "the editor has no reachable port", status_code=502, media_type="text/plain"
+        )
     activity.touch(instance["uid"])
     prefix = f"/projects/{slug}/containers/instances/{uid}/code"
     return await forward.proxy_http(request, host, port, path, prefix=prefix)

devplacepy/routers/projects/index.py

@@ -23,6 +23,9 @@ from devplacepy.database import (
     get_reactions_by_targets,
     get_user_bookmarks,
     get_polls_by_post_uids,
+)
+from devplacepy.services.opinionwar import store as war_store
+from devplacepy.database import (
     paginate,
     text_search_clause,
     resolve_by_slug,
@@ -184,15 +187,12 @@ async def projects_page(
         model=ProjectsOut,
     )
 
-def _editor_launch(project: dict, user: dict) -> dict:
-    from devplacepy.services.containers import store
+async def _editor_launch(project: dict, user: dict) -> dict:
     from devplacepy.services.containers.workspace import editor, provision
 
     blank = {"url": "", "mode": "tab", "width": 0, "height": 0}
     instance = provision.find_for_project(project["uid"], user["uid"])
-    if not instance or instance.get("suspended_at"):
-        return blank
-    if instance.get("status") != store.ST_RUNNING:
+    if not instance or not await provision.editor_ready(instance):
         return blank
     slug = project["slug"] or project["uid"]
     profile = editor.resolve(user["uid"], instance)
@@ -262,7 +262,7 @@ async def project_detail(request: Request, project_slug: str, before: str = None
     )
     viewer_can_workspace = can_open_workspace(project, user)
     editor_launch = (
-        _editor_launch(project, user)
+        await _editor_launch(project, user)
         if viewer_can_workspace
         else {"url": "", "mode": "tab", "width": 0, "height": 0}
     )
@@ -289,12 +289,14 @@ async def project_detail(request: Request, project_slug: str, before: str = None
             get_user_bookmarks(user["uid"], "post", post_uids) if user else set()
         )
         polls_map = get_polls_by_post_uids(post_uids, user)
+        wars_map = war_store.get_wars_by_post_uids(post_uids, user)
         for item in devlog_posts:
             uid = item["post"]["uid"]
             item["attachments"] = attachments_map.get(uid, [])
             item["reactions"] = reactions_map.get(uid, {"counts": {}, "mine": []})
             item["bookmarked"] = uid in bookmark_set
             item["poll"] = polls_map.get(uid)
+            item["war"] = wars_map.get(uid)
 
     return respond(
         request,

devplacepy/routers/push.py

@@ -49,17 +49,24 @@ async def push_register(request: Request) -> JSONResponse:
     if fields is None:
         return JSONResponse({"error": "Invalid request"}, status_code=400)
 
-    _, created = push.register(user["uid"], provider.name, fields)
+    fields = provider.stamp_registration(fields)
+    write = push.register(user["uid"], provider.name, fields)
 
-    if created:
+    delivered = None
+    detail = ""
+    if write.probe:
         try:
-            await push.notify_user(user["uid"], WELCOME_PAYLOAD)
+            outcome = await push.notify_registration(write.record, WELCOME_PAYLOAD)
+            delivered = outcome.status == providers.ACCEPTED
+            detail = outcome.detail
         except Exception as exc:
             logger.warning("Welcome push failed for %s: %s", user["uid"], exc)
+            delivered = False
+            detail = str(exc)
 
     audit.record(
         request,
-        "push.subscribe" if created else "push.update",
+        "push.subscribe" if write.created else "push.update",
         user=user,
         target_type="user",
         target_uid=user["uid"],
@@ -69,12 +76,19 @@ async def push_register(request: Request) -> JSONResponse:
             "endpoint_host": urlparse(fields["endpoint"]).hostname
             if fields.get("endpoint")
             else None,
-            "created": created,
+            "created": write.created,
+            "revived": write.revived,
+            "has_client_id": bool(fields.get("client_id")),
         },
-        summary=f"{user.get('username')} {'registered' if created else 'updated'} a push subscription",
+        summary=f"{user.get('username')} {'registered' if write.created else 'updated'} a push subscription",
         links=[audit.target("user", user["uid"], user.get("username"))],
     )
-    return JSONResponse({"registered": True})
+    payload: dict = {"registered": True}
+    if delivered is not None:
+        payload["delivered"] = delivered
+        if detail and not delivered:
+            payload["error"] = detail
+    return JSONResponse(payload)
 
 
 @router.get("/service-worker.js")

devplacepy/routers/tools/deepsearch.py

@@ -243,6 +243,7 @@ def _session_context(request: Request, uid: str, job: dict, session: dict) -> di
         "sources": report.get("sources", []),
         "findings": report.get("findings", []),
         "timeline": report.get("timeline", []),
+        "follow_up_questions": report.get("follow_up_questions", []),
         "chat_ws_url": f"/tools/deepsearch/{uid}/chat" if done else None,
         "export_md_url": f"/tools/deepsearch/{uid}/export.md" if done else None,
         "export_json_url": f"/tools/deepsearch/{uid}/export.json" if done else None,

devplacepy/routers/tools/isslop.py

@@ -1,5 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
+import asyncio
 import logging
 import re
 from urllib.parse import quote
@@ -435,7 +436,7 @@ async def isslop_source(request: Request, uid: str, path: str, line: int = 0):
     source_path = (store.media_dir_for(uid) / source_name).resolve()
     if not source_path.is_relative_to(ISSLOP_MEDIA_DIR.resolve()) or not source_path.is_file():
         raise not_found("Source not available for this file")
-    text = source_path.read_text(encoding="utf-8", errors="replace")
+    text = await asyncio.to_thread(source_path.read_text, encoding="utf-8", errors="replace")
     signals = store.decode_json(result.get("signals"), [])
     marked: dict[int, list] = {}
     for signal in signals:

devplacepy/routers/topics.py

@@ -0,0 +1,84 @@
+# retoor <retoor@molodetz.nl>
+
+from fastapi import APIRouter, Request
+from fastapi.responses import HTMLResponse
+from devplacepy.constants import TOPICS, TOPIC_LABELS
+from devplacepy.database import get_table
+from devplacepy.routers.feed import get_feed_posts, enrich_post_cards
+from devplacepy.utils import get_current_user, not_found
+from devplacepy.seo import list_page_seo, next_page_url
+from devplacepy.responses import respond
+from devplacepy.schemas import TopicOut, TopicsHubOut
+
+router = APIRouter()
+
+
+@router.get("", response_class=HTMLResponse)
+async def topics_hub(request: Request):
+    user = get_current_user(request)
+    posts_table = get_table("posts")
+    topics = [
+        {
+            "key": topic,
+            "label": TOPIC_LABELS.get(topic, topic.title()),
+            "post_count": posts_table.count(topic=topic, deleted_at=None),
+        }
+        for topic in TOPICS
+    ]
+
+    seo_ctx = list_page_seo(
+        request,
+        title="Topics",
+        description="Browse DevPlace posts by topic: devlog, showcase, questions, rants, fun, and more.",
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Topics", "url": "/topics"},
+        ],
+    )
+    return respond(
+        request,
+        "topics.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "user": user,
+            "topics": topics,
+        },
+        model=TopicsHubOut,
+    )
+
+
+@router.get("/{topic}", response_class=HTMLResponse)
+async def topic_page(request: Request, topic: str, before: str = None):
+    if topic not in TOPICS:
+        raise not_found("Topic not found")
+    user = get_current_user(request)
+    posts, next_cursor = get_feed_posts(user, "all", topic, "", before)
+    posts = enrich_post_cards(posts, user)
+    label = TOPIC_LABELS.get(topic, topic.title())
+
+    seo_ctx = list_page_seo(
+        request,
+        title=f"{label} posts",
+        description=f"Browse {label.lower()} posts from developers on DevPlace.",
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Topics", "url": "/topics"},
+            {"name": label, "url": f"/topics/{topic}"},
+        ],
+        next_url=next_page_url(request, next_cursor),
+    )
+    return respond(
+        request,
+        "topic.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "user": user,
+            "posts": posts,
+            "topic": topic,
+            "topic_label": label,
+            "next_cursor": next_cursor,
+        },
+        model=TopicOut,
+    )

devplacepy/schemas/__init__.py

@@ -52,6 +52,9 @@ from devplacepy.schemas.listings import (
     ProjectsOut,
     SavedItemOut,
     SavedOut,
+    TopicOut,
+    TopicSummaryOut,
+    TopicsHubOut,
 )
 from devplacepy.schemas.profile import (
     MediaItemOut,
@@ -59,6 +62,7 @@ from devplacepy.schemas.profile import (
     TelegramPairOut,
 )
 from devplacepy.schemas.issues import (
+    AdminIssuesPlanningOut,
     IssueAttachmentsOut,
     IssueCommentOut,
     IssueDetailOut,
@@ -151,6 +155,15 @@ from devplacepy.schemas.dbapi import (
     DbTableOut,
     NlQueryOut,
 )
+from devplacepy.schemas.battles import (
+    BattlesOut,
+    WarContributorOut,
+    WarEventOut,
+    WarEventsOut,
+    WarOut,
+    WarPersonOut,
+    WarViewerOut,
+)
 from devplacepy.schemas.quiz import (
     QuizAnswerOut,
     QuizAnswerResultOut,

devplacepy/schemas/battles.py

@@ -0,0 +1,82 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from typing import Optional
+
+from devplacepy.schemas.base import _Out
+
+
+class WarPersonOut(_Out):
+    uid: str = ""
+    username: str = ""
+    avatar_seed: Optional[str] = None
+    level: int = 1
+
+
+class WarContributorOut(WarPersonOut):
+    faction: str = ""
+    hp: int = 0
+
+
+class WarEventOut(_Out):
+    seq: int = 0
+    kind: str = ""
+    message: str = ""
+    faction: str = ""
+    created_at: str = ""
+    hp_a: Optional[int] = None
+    hp_b: Optional[int] = None
+    damage: Optional[int] = None
+    winner: Optional[str] = None
+
+
+class WarViewerOut(_Out):
+    faction: str = ""
+    hp: int = 0
+    rank: int = 0
+    fight_count: int = 0
+    last_fight_at: str = ""
+    next_fight_at: str = ""
+    can_fight: bool = False
+
+
+class WarOut(_Out):
+    uid: str = ""
+    post_uid: str = ""
+    post_url: str = ""
+    post_title: str = ""
+    author: Optional[WarPersonOut] = None
+    faction_a: str = ""
+    faction_b: str = ""
+    hp_a: int = 0
+    hp_b: int = 0
+    pct_a: int = 50
+    pct_b: int = 50
+    leader: str = ""
+    fighter_count: int = 0
+    status: str = "active"
+    winner: str = ""
+    winner_label: str = ""
+    created_at: str = ""
+    ends_at: str = ""
+    ends_in: str = ""
+    resolved_at: str = ""
+    last_seq: int = 0
+    fight_cost: int = 0
+    top_contributors: list[WarContributorOut] = []
+    recent_events: list[WarEventOut] = []
+    viewer: Optional[WarViewerOut] = None
+
+
+class BattlesOut(_Out):
+    battles: list[WarOut] = []
+    current_filter: str = "active"
+    counts: dict = {}
+    search: str = ""
+    pagination: dict = {}
+
+
+class WarEventsOut(_Out):
+    events: list[WarEventOut] = []
+    status: str = ""

devplacepy/schemas/containers.py

@@ -155,8 +155,12 @@ class EditorProfileOut(_Out):
 class WorkspaceViewOut(_Out):
     uid: str = ""
     name: str = ""
+    owner_uid: str = ""
     status: str = ""
     desired_state: str = ""
+    phase: str = ""
+    phase_label: str = ""
+    editor_ready: bool = False
     suspended: bool = False
     flag_reason: Optional[str] = ""
     tunnel_name: Optional[str] = ""
@@ -171,6 +175,7 @@ class WorkspaceViewOut(_Out):
     idle_stop_minutes: int = 0
     retention_days: int = 0
     max_tunnels: int = 0
+    unlimited: bool = False
     tunnels: list[TunnelOut] = []
     flags: list[WorkspaceFlagOut] = []
     editor: Optional[EditorProfileOut] = None
@@ -183,6 +188,7 @@ class WorkspaceOut(_Out):
     viewer_can_workspace: bool = False
     workspace_count: int = 0
     max_workspaces: int = 0
+    unlimited_workspaces: bool = False
     editor_url: str = ""
     editor_password: str = ""
     editor: Optional[EditorProfileOut] = None

devplacepy/schemas/game.py

@@ -2,6 +2,8 @@
 
 from __future__ import annotations
 
+from typing import Optional
+
 from devplacepy.schemas.base import _Out
 
 
@@ -13,8 +15,11 @@ class GameCropOut(_Out):
     reward_coins: int = 0
     reward_xp: int = 0
     min_level: int = 1
+    min_mastery: int = 0
     grow_seconds: int = 0
     locked: bool = False
+    locked_reason: str = ""
+    locked_text: str = ""
     market_state: str = "normal"
 
 
@@ -183,6 +188,7 @@ class GameFarmOut(_Out):
 class GameStateOut(_Out):
     ok: bool = True
     farm: GameFarmOut
+    game_error: Optional[str] = None
 
 
 class GameFarmViewOut(_Out):
@@ -190,6 +196,7 @@ class GameFarmViewOut(_Out):
     page_title: str = ""
     meta_description: str = ""
     stole_coins: int = 0
+    game_error: Optional[str] = None
 
 
 class GameLeaderboardEntryOut(_Out):

devplacepy/schemas/issues.py

@@ -67,3 +67,16 @@ class IssuesOut(_Out):
     state: str = "open"
     configured: bool = True
     error_message: Optional[str] = None
+    viewer_is_admin: bool = False
+
+
+class AdminPlanningTicketOut(_Out):
+    number: int = 0
+    title: str = ""
+    labels: list[str] = []
+
+
+class AdminIssuesPlanningOut(_Out):
+    configured: bool = True
+    tickets: list[AdminPlanningTicketOut] = []
+    tickets_error: bool = False

devplacepy/schemas/jobs.py

@@ -132,6 +132,7 @@ class DeepsearchSessionOut(_Out):
     sources: list = []
     findings: list = []
     timeline: list = []
+    follow_up_questions: list = []
     chat_ws_url: Optional[str] = None
     export_md_url: Optional[str] = None
     export_json_url: Optional[str] = None
@@ -203,6 +204,10 @@ class IsslopReportOut(_Out):
     detected_builder: Optional[str] = None
     dom_slop_score: Optional[float] = None
     error: Optional[str] = None
+    report_url: Optional[str] = None
+    badge_url: Optional[str] = None
+    events_url: Optional[str] = None
+    topic: Optional[str] = None
     content_hash: Optional[str] = None
     markdown: str = ""
     generator_model: str = ""
@@ -225,3 +230,7 @@ class IsslopSourceOut(_Out):
     source: str = ""
     truncated: bool = False
     signals: list = []
+    source_lines: list = []
+    marked_lines: dict = {}
+    focus_line: int = 0
+    report_url: Optional[str] = None

devplacepy/schemas/listings.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 from typing import Any, Optional
 
 from devplacepy.schemas.base import _Out
+from devplacepy.schemas.battles import WarOut
 from devplacepy.schemas.content import (
     AttachmentOut,
     CommentItemOut,
@@ -33,6 +34,7 @@ class FeedItemOut(_Out):
     reactions: ReactionsOut = ReactionsOut()
     bookmarked: bool = False
     poll: Optional[PollOut] = None
+    war: Optional[WarOut] = None
     project_link: Optional[ProjectLinkOut] = None
 
 
@@ -106,6 +108,23 @@ class SavedItemOut(_Out):
     time_ago: Optional[str] = None
 
 
+class TopicOut(_Out):
+    posts: list[FeedItemOut] = []
+    topic: str = ""
+    topic_label: str = ""
+    next_cursor: Optional[str] = None
+
+
+class TopicSummaryOut(_Out):
+    key: str = ""
+    label: str = ""
+    post_count: int = 0
+
+
+class TopicsHubOut(_Out):
+    topics: list[TopicSummaryOut] = []
+
+
 class FeedOut(_Out):
     posts: list[FeedItemOut] = []
     current_tab: Optional[str] = None
@@ -134,10 +153,15 @@ class PostDetailOut(_Out):
     reactions: ReactionsOut = ReactionsOut()
     bookmarked: bool = False
     poll: Optional[PollOut] = None
+    war: Optional[WarOut] = None
     comment_count: Optional[int] = None
     related_posts: list[FeedItemOut] = []
     topics: list[str] = []
     project_link: Optional[ProjectLinkOut] = None
+    featured_topics: list[Any] = []
+    author_gists: list[Any] = []
+    author_projects: list[Any] = []
+    next_post_url: Optional[str] = None
 
 
 class ProjectsOut(_Out):

devplacepy/schemas/quiz.py

@@ -184,6 +184,8 @@ class QuizAttemptOut(_Out):
 class QuizAttemptPageOut(_Out):
     quiz: QuizOut = QuizOut()
     attempt: QuizAttemptOut = QuizAttemptOut()
+    answer_max_chars: int = 0
+    quiz_error: Optional[str] = None
 
 
 class QuizResultOut(_Out):
@@ -225,6 +227,7 @@ class QuizBuilderOut(_Out):
     questions: list[QuizQuestionOut] = []
     kinds: list[Any] = []
     validation_errors: list[str] = []
+    quiz_error: Optional[str] = None
 
 
 class QuizFormPageOut(_Out):

devplacepy/schemas/statistics.py

@@ -39,7 +39,7 @@ class StatisticsHighlightOut(BaseModel):
     value: Any
 
 
-class StatisticsOut(BaseModel):
+class StatisticsPayloadOut(BaseModel):
     tab: str
     window_hours: int
     granularity: str
@@ -49,4 +49,18 @@ class StatisticsOut(BaseModel):
     series: list[StatisticsSeriesOut] = []
     tables: list[StatisticsTableOut] = []
     highlights: list[StatisticsHighlightOut] = []
-    notes: dict[str, Any] = {}
+    notes: dict[str, Any] = {}
+
+
+class StatisticsTabOut(BaseModel):
+    key: str
+    label: str
+    icon: str
+    active: bool
+
+
+class StatisticsOut(BaseModel):
+    active_tab: str
+    window_hours: int
+    tabs: list[StatisticsTabOut] = []
+    initial: StatisticsPayloadOut

devplacepy/seo.py

@@ -91,7 +91,41 @@ def breadcrumb_schema(items, base_url):
     }
 
 
-def discussion_forum_posting(post, author, comment_count, star_count, base_url):
+MAX_SCHEMA_COMMENTS = 20
+
+
+def comment_schema(comment_item, base_url):
+    comment = comment_item["comment"]
+    author = comment_item.get("author")
+    return {
+        "@type": "Comment",
+        "text": truncate(plain_markdown(comment.get("content", "")), 300),
+        "author": {
+            "@type": "Person",
+            "name": author["username"] if author else "Unknown",
+            "url": f"{base_url}/profile/{author['username']}" if author else "",
+        },
+        "datePublished": comment.get("created_at", ""),
+    }
+
+
+def comment_schema_list(comment_tree, base_url, limit=MAX_SCHEMA_COMMENTS):
+    flat = []
+
+    def walk(items):
+        for item in items:
+            if len(flat) >= limit:
+                return
+            flat.append(comment_schema(item, base_url))
+            walk(item.get("children", []))
+
+    walk(comment_tree)
+    return flat
+
+
+def discussion_forum_posting(
+    post, author, comment_count, star_count, base_url, comments=None
+):
     schema = {
         "@type": "DiscussionForumPosting",
         "headline": post.get("title") or "Untitled",
@@ -117,6 +151,8 @@ def discussion_forum_posting(post, author, comment_count, star_count, base_url):
             },
         ],
     }
+    if comments:
+        schema["comment"] = comments
     return schema
 
 
@@ -421,11 +457,19 @@ def _build_sitemap(base_url):
     urlset.append(url_element(f"{base_url}/", changefreq="daily", priority="1.0"))
     urlset.append(url_element(f"{base_url}/feed", changefreq="hourly", priority="0.9"))
     urlset.append(url_element(f"{base_url}/news", changefreq="hourly", priority="0.9"))
+    urlset.append(url_element(f"{base_url}/topics", changefreq="daily", priority="0.7"))
+    from devplacepy.constants import TOPICS
+
+    for topic in TOPICS:
+        urlset.append(
+            url_element(f"{base_url}/topics/{topic}", changefreq="daily", priority="0.7")
+        )
     urlset.append(
         url_element(f"{base_url}/projects", changefreq="daily", priority="0.8")
     )
     urlset.append(url_element(f"{base_url}/gists", changefreq="daily", priority="0.8"))
     urlset.append(url_element(f"{base_url}/quizzes", changefreq="daily", priority="0.8"))
+    urlset.append(url_element(f"{base_url}/battles", changefreq="daily", priority="0.7"))
     urlset.append(
         url_element(f"{base_url}/leaderboard", changefreq="daily", priority="0.7")
     )
@@ -433,6 +477,7 @@ def _build_sitemap(base_url):
     urlset.append(url_element(f"{base_url}/tools", changefreq="monthly", priority="0.5"))
     urlset.append(url_element(f"{base_url}/tools/seo", changefreq="monthly", priority="0.5"))
     urlset.append(url_element(f"{base_url}/tools/deepsearch", changefreq="monthly", priority="0.5"))
+    urlset.append(url_element(f"{base_url}/tools/isslop", changefreq="monthly", priority="0.5"))
     urlset.append(
         url_element(f"{base_url}/workspaces/index", changefreq="daily", priority="0.6")
     )

devplacepy/services/CLAUDE.md

@@ -26,7 +26,7 @@ Generic, lightweight, **fire-and-forget** offload for non-critical side-effects
 - **Field registry is the single source of truth.** `CORRECTABLE_FIELDS: dict[str, tuple[str, ...]]` maps each correctable table to its prose columns: `posts` -> `(title, content)`, `projects`/`gists` -> `(title, description)`, `comments`/`messages` -> `(content,)`, `users` -> `(bio,)`. `gists.source_code`, project files, and Gitea issues are intentionally excluded - code and external systems are never corrected.
 - **Choke entrypoint.** `schedule_correction(user, table, uid, request=None)` is the only thing handlers call (every hook passes `request`). It is a no-op unless: a user dict is present, `table` is in the registry, `user["ai_correction_enabled"]` is truthy, and the user has a non-empty `api_key`. In **sync** mode (`user["ai_correction_sync"]`) it calls `_run_inline_awaited`, which - only when a `request.scope` and a running event loop exist - submits `_run_correction` to `loop.run_in_executor(AI_APPLY_EXECUTOR, ...)` (off the loop thread, on the dedicated AI pool) and stashes the future on `request.scope[PENDING_SCOPE_KEY]` for the middleware to await; if there is no request/loop it returns False and falls back to background. In **background** mode (default) it `background.submit`s `_run_correction` (per-worker queue; also runs inline under `DEVPLACE_DISABLE_SERVICES=1`). The same `_run_correction` worker function runs in all cases.
 - **The hooks (covers UI + REST + Devii + devRant in one place):** `content.create_content_item` (posts/projects/gists), `content.edit_content_item`, `content.create_comment_record`, `content.edit_comment_record`, `services/messaging/persist.persist_message` (DMs - sender is the user), `routers/profile/index.update_profile` (bio), and the two devRant direct-update edit paths (`routers/devrant/rants.edit_rant`, `routers/devrant/comments.edit_comment`). devRant create paths route through the shared content cores, so they are already hooked. Code-only and external paths (project files, Gitea) are deliberately not hooked.
-- **The gateway call is fail-soft.** `correct_text(api_key, prompt, text)` is synchronous (runs on the background worker thread), POSTs to `INTERNAL_GATEWAY_URL` with `model=INTERNAL_MODEL` via `stealth.stealth_sync_client`, authenticated with the user's own `Bearer` api_key (per-user attribution). It returns the ORIGINAL text on any error, empty output, or suspiciously large output (`len > len(text) * MAX_GROWTH_FACTOR + 200`, rejecting hallucinated expansion). `_run_correction` reads the row back, corrects each registry field that has non-blank text, writes only changed fields via `table.update(updates, ["uid"])` without touching `updated_at` (auto-correction is not a user edit) or the slug (slugs are permanent), and `clear_user_cache(user_uid)` when `table == "users"` so the corrected bio re-caches.
+- **The gateway call is fail-soft.** `correct_text(api_key, prompt, text)` is synchronous (runs on the background worker thread), POSTs to `INTERNAL_GATEWAY_URL` with `model=correction_model()` (`get_setting("correction_model", "") or INTERNAL_MODEL` - admin-configurable at `/admin/settings`, blank falls back to the gateway default `molodetz`; the gateway URL itself is never configurable per feature, always `INTERNAL_GATEWAY_URL`) via `stealth.stealth_sync_client`, authenticated with the user's own `Bearer` api_key (per-user attribution). It returns the ORIGINAL text on any error, empty output, or suspiciously large output (`len > len(text) * MAX_GROWTH_FACTOR + 200`, rejecting hallucinated expansion). `_run_correction` reads the row back, corrects each registry field that has non-blank text, writes only changed fields via `table.update(updates, ["uid"])` without touching `updated_at` (auto-correction is not a user edit) or the slug (slugs are permanent), and `clear_user_cache(user_uid)` when `table == "users"` so the corrected bio re-caches.
 - **Per-user usage aggregation (only on success).** `correct_text` returns `(text, usage)`; `usage` is parsed from the gateway's `X-Gateway-*` response headers (`_usage_from_headers`) whenever the upstream call returned 200 (cost was incurred, even if the corrected output was rejected), else `None` on any failure. `_usage_from_headers` captures the token and cost headers PLUS the timing headers `X-Gateway-Upstream-Latency-Ms` and `X-Gateway-Total-Latency-Ms` (as `upstream_latency_ms`/`total_latency_ms`), so each call's timing is metered. `_run_correction` accumulates the per-field `usage` into one `totals` dict and, when `totals["calls"] > 0`, makes ONE call to `database.add_correction_usage(user_uid, totals)` (a single `totals` dict, not positional args) - so a 2-field content item is a single aggregated write, and a failed/empty correction records nothing. `add_correction_usage` (and `add_modifier_usage`) delegate to the shared `database._add_usage(usage_table, user_uid, totals)`: a single atomic `INSERT ... ON CONFLICT(user_uid) DO UPDATE SET col = col + excluded.col` upsert against the `correction_usage` table (per-user running SUMS: `calls`/`prompt_tokens`/`completion_tokens`/`total_tokens`/`cost_usd`/`upstream_latency_ms`/`total_latency_ms`/`updated_at`, unique index `idx_correction_usage_user`, the two latency columns REAL default 0.0, all ensured in `init_db`). It is a derived counter table (NOT in `SOFT_DELETE_TABLES`, like `gateway_usage_ledger`) and is deliberately separate from `users` so accumulating never invalidates the auth/user cache. `database.get_correction_usage(user_uid)` (via `_get_usage`) returns the stored sums PLUS computed averages: `avg_tokens` (total_tokens/calls), `avg_upstream_latency_ms`, `avg_total_latency_ms`, `avg_tokens_per_second` (completion_tokens over total upstream seconds), and `avg_cost_usd`.
 - **Profile display:** `routers/profile/usage._correction_usage(uid, include_cost)` shapes it via the shared `_usage_view(data, include_cost)` (mirrors `_ai_quota`); `profile/index.py` builds it only for `is_owner or viewer_is_admin` and passes `include_cost=viewer_is_admin`, exposed as the `correction_usage` dict on the context and `ProfileOut`. The view surfaces the sums plus averages - `avg_tokens` (avg tokens/call), `avg_latency_ms` (avg upstream latency), `avg_total_latency_ms`, `avg_tokens_per_second` (avg speed), and `total_time_s` (total upstream seconds) - rendered as extra tiles on the card. **Financial gating:** tokens/call-count and the performance tiles show to the owner and admins; the dollar `cost_usd` and `avg_cost_usd` keys are present ONLY when `viewer_is_admin`, in BOTH the HTML card (`templates/profile.html`, `.correction-usage-*`) and the `respond(..., model=ProfileOut)` JSON (same rule as `_ai_quota`'s `spent_usd` - hiding it in the template alone would leak it to a member fetching their own profile as JSON). The card renders only when `correction_usage.calls` > 0.
 - **Import-cycle discipline.** `correction.py` imports only `stealth`, `config`, `database.get_table`/`add_correction_usage`, and `services.background.background` at module top; `clear_user_cache` is imported lazily inside `_run_correction`. Never import `content` or `utils` at module top.
@@ -37,7 +37,7 @@ Generic, lightweight, **fire-and-forget** offload for non-critical side-effects
 **A sibling of AI content correction that runs only on an explicit inline directive.** The engine reuses the correction plumbing wholesale (`CORRECTABLE_FIELDS`, `PENDING_SCOPE_KEY`, `gateway_complete`, the sync/background apply-mode mechanics, the per-user usage upsert) and differs only in the trigger and the apply-mode/enabled defaults: it is **enabled by default** and **synchronous by default**, and it runs ONLY where an authored prose field contains an inline `@ai <instruction>` directive.
 
 - **The `@ai` gate is the whole difference.** `has_ai_directive(text)` matches the regex `@ai\s+\S` (case-insensitive). `schedule_modification(user, table, uid, request=None)` is a no-op unless a user is present, `table` is in `CORRECTABLE_FIELDS`, `user["ai_modifier_enabled"]` is truthy, the user has an `api_key`, AND at least one of the table's registry fields actually contains an `@ai` directive. `_run_modification` re-checks the gate per field, so untriggered fields are never sent to the gateway and never metered. Triggerless writes cost nothing. The configured prompt (default `config.DEFAULT_MODIFIER_PROMPT` = "Execute what is behind `@ai` (the prompt) and replace that part including `@ai`") tells the model to execute the instruction and replace the marked part including the `@ai` marker.
-- **Total reuse of the correction layer.** `CORRECTABLE_FIELDS`, `PENDING_SCOPE_KEY`, `gateway_complete`, the sync/background apply-mode mechanics (`_run_inline_awaited` -> `loop.run_in_executor` + `request.scope[PENDING_SCOPE_KEY]` awaited by the `await_pending_corrections` middleware in `main.py`, the same self-deadlock-avoiding path), and the per-user usage upsert pattern are all imported from / mirror `services/correction.py`. `modify_text(api_key, prompt, text, context="")` composes a modifier system message and calls the shared `gateway_complete`. The same hooks fire it: `profile/index.update_profile` calls both `schedule_correction` and `schedule_modification`, and the content/comment/messaging cores invoke it alongside correction, so it covers the web UI, REST, devRant, and Devii in one place. Code and source files are never modified (same `CORRECTABLE_FIELDS` registry, `gists.source_code`/project files/Gitea excluded). `schedule_modification` is hooked alongside `schedule_correction` at the same content/comment/messaging/profile entrypoints.
+- **Total reuse of the correction layer.** `CORRECTABLE_FIELDS`, `PENDING_SCOPE_KEY`, `gateway_complete`, the sync/background apply-mode mechanics (`_run_inline_awaited` -> `loop.run_in_executor` + `request.scope[PENDING_SCOPE_KEY]` awaited by the `await_pending_corrections` middleware in `main.py`, the same self-deadlock-avoiding path), and the per-user usage upsert pattern are all imported from / mirror `services/correction.py`. `modify_text(api_key, prompt, text, context="")` composes a modifier system message and calls the shared `gateway_complete` with `model=modifier_model()` (`get_setting("modifier_model", "") or INTERNAL_MODEL`, its own admin-configurable setting at `/admin/settings`, independent of `correction_model` so each feature can use a different model - blank falls back to the gateway default `molodetz`). The same hooks fire it: `profile/index.update_profile` calls both `schedule_correction` and `schedule_modification`, and the content/comment/messaging cores invoke it alongside correction, so it covers the web UI, REST, devRant, and Devii in one place. Code and source files are never modified (same `CORRECTABLE_FIELDS` registry, `gists.source_code`/project files/Gitea excluded). `schedule_modification` is hooked alongside `schedule_correction` at the same content/comment/messaging/profile entrypoints.
 - **Context-aware (modifier only, not correction).** Unlike correction, the modifier gives the model a grounding **context block** so an `@ai` instruction can reason about who is asking and what it is attached to. `services/ai_context.py` `build_context(table, uid, row, user_uid) -> str` assembles it; `_run_modification` builds it **lazily once per row** (only after a field is confirmed to contain `@ai`, so triggerless writes do no extra queries) and passes it to every field's `modify_text`, which appends it to the system message under a `# Context (use it to inform the result; never echo this block)` header. The block (fail-soft, length-capped, each part wrapped in try/except so a failed lookup never blocks the modification) has three parts:
   1. **date** - `Today is DD/MM/YYYY on the DevPlace developer network.`
   2. **author/stats** - the author's username, role, level, stars, post count (`get_user_post_count`), leaderboard rank (`get_user_rank`), follower count (`get_follow_counts`), member-since date, and bio (capped `MAX_BIO`).
@@ -162,8 +162,9 @@ Rules: a new hot read-path aggregate follows this exact pattern (module-level `T
 | `admin.services.{name}` | 5s | `{service}` |
 | `admin.ai-usage.{hours}` | 15s | `build_analytics(hours)` (hours parsed from the topic) |
 | `admin.backups` | 8s | `routers/admin/backups._dashboard(can_download=False)` (storage, backups, schedules, metrics) |
+| `user.{owner_uid}.workspace.{uid}` | 3s | `{workspace, editor_url}` (`provision.view`, the same shape `GET /projects/{slug}/workspace` JSON carries; `None` unless the instance is a workspace owned by `owner_uid`) |
 
-All these topics are admin-only by pub/sub policy (non-`public`, non-`user.{uid}` -> `privileged` required), matching the admin-only pages. Frontend monitors (`ContainerInstance`, `ContainerList`, `ContainerManager`, `BotMonitor`, `ServiceMonitor`, `AiUsageMonitor`, `BackupMonitor`) each `window.app.pubsub.subscribe(topic, render)` in their init and keep a **lengthened HTTP poll (15-30s) as initial-load + fallback** - the relay drives liveness at the cadence above. `AiUsageMonitor` re-subscribes (unsubscribe old, subscribe new) when the window-hours selector changes, since hours is in the topic.
+All these topics except the last are admin-only by pub/sub policy (non-`public`, non-`user.{uid}` -> `privileged` required), matching the admin-only pages. The workspace topic is the one member-facing view: it sits in the owner's private `user.{uid}.*` namespace so the owner (and admins) can subscribe and nobody else can, and the compute callable re-checks `workspace_owner_uid` against the topic so a guessed uid never leaks another member's workspace. `WorkspaceManager` subscribes to it and keeps a 2s/20s HTTP poll as fallback (see `devplacepy/services/containers/CLAUDE.md`). Frontend monitors (`ContainerInstance`, `ContainerList`, `ContainerManager`, `BotMonitor`, `ServiceMonitor`, `AiUsageMonitor`, `BackupMonitor`) each `window.app.pubsub.subscribe(topic, render)` in their init and keep a **lengthened HTTP poll (15-30s) as initial-load + fallback** - the relay drives liveness at the cadence above. `AiUsageMonitor` re-subscribes (unsubscribe old, subscribe new) when the window-hours selector changes, since hours is in the topic.
 
 **Container topics never broadcast private-project instances**: `container.list` publishes only public-project rows with `partial: true` (`ContainerList.merge` updates by uid, never removes, so private rows from the authoritative HTTP poll survive), and `project.{slug}.containers` / `container.{uid}.detail` / `container.{uid}.logs` skip private-project targets entirely (owners fall back to their HTTP polls).
 

devplacepy/services/access_tokens.py

@@ -16,11 +16,6 @@ def issue_token(
     label: str = "",
     max_age_days: Optional[int] = None,
 ) -> dict:
-    """Issue a DevPlace access token for *user*.
-
-    Returns a dict with *access_token*, *token_type*, *expires_in* (seconds),
-    *expires_at* (ISO), and *uid*.
-    """
     if max_age_days is None:
         max_age_days = max(1, get_int_setting("session_max_age_days", 7))
     max_age_seconds = max_age_days * SECONDS_PER_DAY
@@ -54,11 +49,6 @@ def issue_token(
 
 
 def resolve_token(token: str) -> Optional[dict]:
-    """Resolve a user from an access token string.
-
-    Returns the user dict or ``None`` when the token is invalid, expired, or
-    belongs to an inactive user.
-    """
     if not token:
         return None
     row = get_table("access_tokens").find_one(token=token, deleted_at=None)
@@ -83,7 +73,6 @@ def resolve_token(token: str) -> Optional[dict]:
 
 
 def revoke_token(uid: str) -> bool:
-    """Soft-delete a single access token by its uid.  Returns ``True`` on success."""
     tokens = get_table("access_tokens")
     row = tokens.find_one(uid=uid, deleted_at=None)
     if not row:
@@ -97,7 +86,6 @@ def revoke_token(uid: str) -> bool:
 
 
 def revoke_all(user_uid: str) -> int:
-    """Soft-delete every access token for *user_uid*.  Returns the count revoked."""
     tokens = get_table("access_tokens")
     stamp = datetime.now(timezone.utc).isoformat()
     count = 0
@@ -112,7 +100,6 @@ def revoke_all(user_uid: str) -> int:
 
 
 def prune_expired() -> int:
-    """Soft-delete all expired access tokens.  Returns the count pruned."""
     tokens = get_table("access_tokens")
     now = datetime.now(timezone.utc)
     stamp = now.isoformat()

devplacepy/services/ai_modifier.py

@@ -3,8 +3,8 @@
 import logging
 import re
 
-from devplacepy.config import DEFAULT_MODIFIER_PROMPT
-from devplacepy.database import add_modifier_usage, get_table
+from devplacepy.config import DEFAULT_MODIFIER_PROMPT, INTERNAL_MODEL
+from devplacepy.database import add_modifier_usage, get_setting, get_table
 from devplacepy.services.ai_context import build_context
 from devplacepy.services.background import background
 from devplacepy.services.correction import (
@@ -24,6 +24,10 @@ def has_ai_directive(text: str | None) -> bool:
     return bool(text) and AI_DIRECTIVE.search(text) is not None
 
 
+def modifier_model() -> str:
+    return get_setting("modifier_model", "") or INTERNAL_MODEL
+
+
 def modify_text(
     api_key: str, prompt: str, text: str, context: str = ""
 ) -> tuple[str, dict | None]:
@@ -38,7 +42,9 @@ def modify_text(
             "\n\n# Context (use it to inform the result; never echo this block)\n"
             + context
         )
-    return gateway_complete(api_key, system, text, MODIFIER_TIMEOUT_SECONDS, None)
+    return gateway_complete(
+        api_key, system, text, MODIFIER_TIMEOUT_SECONDS, None, model=modifier_model()
+    )
 
 
 def schedule_modification(
@@ -95,6 +101,10 @@ def _run_modification(
     if updates:
         updates["uid"] = uid
         get_table(table).update(updates, ["uid"])
+        if table == "messages":
+            from devplacepy.services.messaging.persist import push_content_revision
+
+            push_content_revision(uid, ai_processed=True)
         if table == "users" and user_uid:
             from devplacepy.utils import clear_user_cache
 

devplacepy/services/audit/CLAUDE.md

@@ -2,7 +2,7 @@ This file documents the audit log subsystem. Claude Code auto-loads it when a fi
 
 ## Audit log (`services/audit/`)
 
-**Admin-only, append-only** record of every state-changing action. The authoritative event catalogue is `events.md` (its storage model, relation vocabulary, and per-event specs are authoritative); the catalogue currently spans 288 keys across 42 domains.
+**Admin-only, append-only** record of every state-changing action. The authoritative event catalogue is `events.md` (its storage model, relation vocabulary, and per-event specs are authoritative); the catalogue currently spans 293 keys across 43 domains.
 
 ### Package layout
 
@@ -43,7 +43,7 @@ The dispatcher's **authorization guard** (before `_run`) mirrors this with `_aud
 
 ### Devii access (read-only)
 
-Admins query the same two routes conversationally via the admin-only Devii tools `audit_log` (GET `/admin/audit-log`) and `audit_event` (GET `/admin/audit-log/{uid}`) (`services/devii/actions/catalog.py`, `handler="http"`, `requires_admin=True`) - no new endpoint, since the `respond(...)` routes already serve JSON to Devii's `Accept: application/json` client (like `admin_list_users`). `audit_log` forwards every filter as a query param (`page`, `event_key`, `category`, `actor_role`, `actor_uid`, `origin`, `result`, `q`, `date_from`, `date_to`) and returns `AuditLogOut`, whose `options` object lists the valid values for each filter so the agent can discover them in one call. `audit_event` returns `AuditEventOut` (the row + related links). A system-prompt steer in `agent.py` (the AGGREGATES block) routes audit/history/"who did X" questions to these tools.
+Admins query the same two routes conversationally via the admin-only Devii tools `audit_log` (GET `/admin/audit-log`) and `audit_event` (GET `/admin/audit-log/{uid}`) (`services/devii/actions/catalog/admin.py`, `handler="http"`, `requires_admin=True`) - no new endpoint, since the `respond(...)` routes already serve JSON to Devii's `Accept: application/json` client (like `admin_list_users`). `audit_log` forwards every filter as a query param (`page`, `event_key`, `category`, `actor_role`, `actor_uid`, `origin`, `result`, `q`, `date_from`, `date_to`) and returns `AuditLogOut`, whose `options` object lists the valid values for each filter so the agent can discover them in one call. `audit_event` returns `AuditEventOut` (the row + related links). A system-prompt steer in `agent.py` (the AGGREGATES block) routes audit/history/"who did X" questions to these tools.
 
 ### Deferred persistence
 

devplacepy/services/audit/categories.py

@@ -24,6 +24,7 @@ CATEGORY_BY_PREFIX: dict[str, str] = {
     "reaction": "engagement",
     "bookmark": "engagement",
     "poll": "engagement",
+    "battle": "engagement",
     "project": "project",
     "file": "project_files",
     "dir": "project_files",

devplacepy/services/backup/CLAUDE.md

@@ -15,6 +15,20 @@ Admin-only, enterprise-grade backups built on the **same async-job pattern as zi
 - **Data model** (`store.py`, ensured in `database.init_db` via `backup_store.ensure_tables()`): `backups` (NOT soft-deletable - an archive is a reclaimable operational artifact, hard-deleted like zips) and `backup_schedules` (in `SOFT_DELETE_TABLES`, born-live `deleted_at:None`). `store` holds all CRUD plus `compute_storage_stats()` (du of every major data area + `shutil.disk_usage`, run in `asyncio.to_thread` from the route, 30s in-process TTL cache so the walk never blocks).
 - **Permanent artifact:** `cleanup(job)` only removes leftover staging, NEVER the archive. Job retention prunes the `jobs` row; the archive and `backups` row persist until an admin deletes it, a schedule rotates it out (`keep_last`), or `devplace backups clear`. Deleting a backup is a HARD delete (unlink file + delete row) - correct because backups are GC artifacts, the documented exception to the soft-delete rule.
 
+## Remote offload
+
+`devplacepy/services/backup/offload.py` ships completed archives to a Hetzner Storage Box over WebDAV via `rclone` (`config.RCLONE_BIN`/`config.RCLONE_CONFIG_FILE`, remote name `config.BACKUP_OFFLOAD_REMOTE`, default `storagebox:devplacepy-backups`) - deliberately **not** the `/backup` davfs2 mount, whose FUSE metadata cache lives on the root filesystem and breaks exactly when disk fills (the original outage cause). `BackupService._run_offload_cycle` (throttled to `backup_offload_interval_seconds`, default 300s, via `ConfigField`s in the `Offload` group) runs each cycle after `_fire_due_schedules`:
+
+1. `upload_pending` - every `done` backup with `remote_uploaded_at` unset and a live `local_path` is `rclone copyto`'d to `<remote>/<target>/<filename>`, then verified by exact byte-size match (`rclone size --json`) against `size_bytes` recorded at finalize time. Only on a verified match does `store.mark_remote_uploaded` set `remote_path`/`remote_uploaded_at`. A failed or unverified upload is silently retried next cycle - `remote_uploaded_at` is the only source of truth for "is this backup actually safe off-box."
+2. `enforce_local_retention` (`backup_offload_keep_local`, default 1) - per target, keeps the newest N **offloaded** local copies and unlinks the rest (`store.mark_local_purged`: clears `local_path`, sets `local_purged_at`, row and `remote_path` persist). A backup with no confirmed remote copy is never touched, no matter how old.
+3. `enforce_remote_retention` (`backup_offload_keep_remote`, default 30) - per target, `rclone lsjson` the remote dir and `deletefile` anything beyond the newest N, sorted by filename (safe because the `{target}-YYYYMMDD-HHMMSS-*` name is lexicographically chronological, same property `schedule.to_iso` relies on).
+
+**`rotate_schedule` is offload-aware:** it now skips any row with an empty `remote_uploaded_at` - a schedule's `keep_last` can never hard-delete a backup that was never confirmed off-box, even if offload is disabled entirely (rotation then simply stops happening, which is the safe failure direction).
+
+**`prune_orphans` (`devplace backups prune`) is offload-aware for the same reason:** a `done` row with a confirmed `remote_uploaded_at` is skipped even when its `local_path` is empty/missing - that is the normal steady state after `enforce_local_retention` purges the local copy, not an orphan. Only a `done` row with no confirmed remote copy AND a missing local file counts as truly orphaned and gets hard-deleted. Without this check, running the CLI prune after offload has done its job would delete the DB record for every successfully offloaded backup, discarding the only pointer to its `remote_path`.
+
+**Operational prerequisite (production, not automatic):** the `rclone` binary is installed in the shipped Docker image, but a working WebDAV remote still needs to exist at `config.RCLONE_CONFIG_FILE` (default `$HOME/.config/rclone/rclone.conf` inside the app container, overridable via `DEVPLACE_RCLONE_CONFIG`) with a remote named to match `config.BACKUP_OFFLOAD_REMOTE`'s prefix (default `storagebox`) pointing at the Hetzner Storage Box's WebDAV endpoint and credentials - `rclone config` (interactive) or a hand-written `rclone.conf` generates it. Until that file exists, every `upload_pending` attempt fails fast (`rclone` errors "didn't find section") and is logged and retried next cycle; local retention and rotation both stay disabled the whole time (see above), so backups simply accumulate locally with no data loss, they just never leave the box. **In Docker, `HOME=/app` (the bind-mounted repo root, `docker-compose.yml`), so the default config path resolves to `<repo>/.config/rclone/rclone.conf` on the host - `.gitignore` excludes `/.config/` precisely because this file holds live remote-storage credentials; never force-add it.**
+
 ## Schedules
 
 `backup_schedules` carry `kind` (`interval`|`cron`), `every_seconds`/`cron`, `enabled`, `keep_last`, `next_run_at`, run bookkeeping. `_fire_due_schedules` (lock-owner only, so each fires once) compares `next_run_at <= to_iso(now_utc())` and enqueues a `backup` job + a `backups` record, then advances `next_run_at` via `schedule.next_run`. **Timestamp format is load-bearing:** schedule `next_run_at` uses the devii `schedule.to_iso` format (`%Y-%m-%dT%H:%M:%S`, no tz/micros) on BOTH sides of the comparison so lexicographic compare equals chronological - do not mix it with `datetime.isoformat()`.

devplacepy/services/backup/offload.py

@@ -0,0 +1,112 @@
+# retoor <retoor@molodetz.nl>
+
+import asyncio
+import json
+from pathlib import Path
+
+from devplacepy import config
+from devplacepy.services.backup import store
+
+
+async def _run_rclone(*args: str, timeout: float = 1800.0) -> tuple[int, str, str]:
+    proc = await asyncio.create_subprocess_exec(
+        config.RCLONE_BIN,
+        "--config",
+        config.RCLONE_CONFIG_FILE,
+        *args,
+        stdout=asyncio.subprocess.PIPE,
+        stderr=asyncio.subprocess.PIPE,
+    )
+    try:
+        out, err = await asyncio.wait_for(proc.communicate(), timeout=timeout)
+    except asyncio.TimeoutError:
+        proc.kill()
+        await proc.wait()
+        return 124, "", "rclone timed out"
+    return proc.returncode, out.decode("utf-8", "replace"), err.decode("utf-8", "replace")
+
+
+def _remote_dir(target: str) -> str:
+    return f"{config.BACKUP_OFFLOAD_REMOTE}/{target}"
+
+
+async def upload_pending(log=lambda message: None) -> int:
+    uploaded = 0
+    for row in store.list_pending_offload():
+        local_path = Path(row["local_path"])
+        remote_path = f"{_remote_dir(row['target'])}/{row['filename']}"
+        code, _, err = await _run_rclone("copyto", str(local_path), remote_path)
+        if code != 0:
+            log(f"Offload failed for {row['filename']}: {err.strip()[:300]}")
+            continue
+        size_code, size_out, size_err = await _run_rclone("size", remote_path, "--json")
+        if size_code != 0:
+            log(f"Offload verify failed for {row['filename']}: {size_err.strip()[:300]}")
+            continue
+        try:
+            remote_bytes = json.loads(size_out).get("bytes", -1)
+        except (ValueError, TypeError):
+            remote_bytes = -1
+        expected_bytes = int(row.get("size_bytes") or 0)
+        if remote_bytes != expected_bytes:
+            log(
+                f"Offload size mismatch for {row['filename']} "
+                f"(local {expected_bytes}, remote {remote_bytes}), will retry"
+            )
+            continue
+        store.mark_remote_uploaded(row["uid"], remote_path)
+        uploaded += 1
+        log(f"Offloaded {row['filename']} ({store.human_bytes(expected_bytes)}) to {remote_path}")
+    return uploaded
+
+
+async def enforce_local_retention(keep_local: int, log=lambda message: None) -> int:
+    keep_local = max(1, keep_local)
+    purged = 0
+    for target in store.BACKUP_TARGETS:
+        rows = store.list_offloaded_local_by_target(target)
+        for row in rows[keep_local:]:
+            store.mark_local_purged(row["uid"])
+            purged += 1
+            log(f"Purged local copy of {row['filename']} (kept remotely at {row['remote_path']})")
+    return purged
+
+
+async def enforce_remote_retention(keep_remote: int, log=lambda message: None) -> int:
+    if keep_remote < 1:
+        return 0
+    removed = 0
+    for target in store.BACKUP_TARGETS:
+        remote_dir = _remote_dir(target)
+        code, out, _ = await _run_rclone("lsjson", remote_dir)
+        if code != 0:
+            continue
+        try:
+            entries = json.loads(out)
+        except (ValueError, TypeError):
+            continue
+        files = sorted(
+            (entry for entry in entries if not entry.get("IsDir")),
+            key=lambda entry: entry.get("Name", ""),
+            reverse=True,
+        )
+        for entry in files[keep_remote:]:
+            remote_path = f"{remote_dir}/{entry['Name']}"
+            del_code, _, del_err = await _run_rclone("deletefile", remote_path)
+            if del_code == 0:
+                removed += 1
+                log(f"Removed old remote backup {remote_path}")
+            else:
+                log(f"Could not remove remote backup {remote_path}: {del_err.strip()[:200]}")
+    return removed
+
+
+async def run_cycle(*, keep_local: int, keep_remote: int, log=lambda message: None) -> dict:
+    uploaded = await upload_pending(log=log)
+    purged_local = await enforce_local_retention(keep_local, log=log)
+    removed_remote = await enforce_remote_retention(keep_remote, log=log)
+    return {
+        "uploaded": uploaded,
+        "purged_local": purged_local,
+        "removed_remote": removed_remote,
+    }

devplacepy/services/backup/service.py

@@ -6,12 +6,14 @@ import logging
 import shutil
 import sqlite3
 import sys
+import time
 from datetime import datetime, timezone
 from pathlib import Path
 
 from devplacepy import config
 from devplacepy.attachments import _directory_for
-from devplacepy.services.backup import store
+from devplacepy.services.backup import offload, store
+from devplacepy.services.base import ConfigField
 from devplacepy.services.devii.tasks.schedule import next_run as schedule_next_run
 from devplacepy.services.devii.tasks.schedule import now_utc, to_iso
 from devplacepy.services.jobs import queue
@@ -22,6 +24,10 @@ logger = logging.getLogger(__name__)
 
 WORKER_MODULE = "devplacepy.services.jobs.backup_worker"
 
+DEFAULT_OFFLOAD_INTERVAL_SECONDS = 300
+DEFAULT_OFFLOAD_KEEP_LOCAL = 1
+DEFAULT_OFFLOAD_KEEP_REMOTE = 30
+
 
 class BackupService(JobService):
     kind = "backup"
@@ -34,6 +40,53 @@ class BackupService(JobService):
 
     def __init__(self):
         super().__init__(name="backup", interval_seconds=15)
+        self._last_offload_at = 0.0
+        self.offload_enabled_field = ConfigField(
+            "backup_offload_enabled",
+            "Offload to remote storage",
+            type="bool",
+            default=True,
+            help=(
+                "Ship completed backups to the configured remote storage and purge "
+                "older local copies once a remote copy is confirmed."
+            ),
+            group="Offload",
+        )
+        self.offload_interval_field = ConfigField(
+            "backup_offload_interval_seconds",
+            "Offload check interval (seconds)",
+            type="int",
+            default=DEFAULT_OFFLOAD_INTERVAL_SECONDS,
+            minimum=60,
+            help="How often to check for backups to offload and rotate.",
+            group="Offload",
+        )
+        self.offload_keep_local_field = ConfigField(
+            "backup_offload_keep_local",
+            "Local copies to keep per target",
+            type="int",
+            default=DEFAULT_OFFLOAD_KEEP_LOCAL,
+            minimum=1,
+            maximum=10,
+            help="Local archives beyond this count are purged per target, once offloaded.",
+            group="Offload",
+        )
+        self.offload_keep_remote_field = ConfigField(
+            "backup_offload_keep_remote",
+            "Remote copies to keep per target",
+            type="int",
+            default=DEFAULT_OFFLOAD_KEEP_REMOTE,
+            minimum=1,
+            maximum=1000,
+            help="Remote archives beyond this count are deleted per target.",
+            group="Offload",
+        )
+        self.config_fields += [
+            self.offload_enabled_field,
+            self.offload_interval_field,
+            self.offload_keep_local_field,
+            self.offload_keep_remote_field,
+        ]
 
     async def run_once(self) -> None:
         await super().run_once()
@@ -41,6 +94,30 @@ class BackupService(JobService):
             self._fire_due_schedules()
         except Exception as exc:
             self.log(f"Schedule pass failed: {exc}")
+        await self._run_offload_cycle()
+
+    async def _run_offload_cycle(self) -> None:
+        if not self.offload_enabled_field.read():
+            return
+        interval = max(60, int(self.offload_interval_field.read()))
+        now = time.monotonic()
+        if now - self._last_offload_at < interval:
+            return
+        self._last_offload_at = now
+        try:
+            result = await offload.run_cycle(
+                keep_local=int(self.offload_keep_local_field.read()),
+                keep_remote=int(self.offload_keep_remote_field.read()),
+                log=self.log,
+            )
+            if result["uploaded"] or result["purged_local"] or result["removed_remote"]:
+                self.log(
+                    f"Offload cycle: {result['uploaded']} uploaded, "
+                    f"{result['purged_local']} local purged, "
+                    f"{result['removed_remote']} remote pruned"
+                )
+        except Exception as exc:
+            self.log(f"Offload cycle failed: {exc}")
 
     async def process(self, job: dict) -> dict:
         from devplacepy.services.audit import record as audit

devplacepy/services/backup/store.py

@@ -79,6 +79,9 @@ def ensure_tables() -> None:
         ("created_at", ""),
         ("completed_at", ""),
         ("error", ""),
+        ("remote_path", ""),
+        ("remote_uploaded_at", ""),
+        ("local_purged_at", ""),
     ):
         if not backups.has_column(column):
             backups.create_column_by_example(column, example)
@@ -216,16 +219,67 @@ def rotate_schedule(schedule_uid: str, keep_last: int) -> int:
     )
     removed = 0
     for row in rows[keep_last:]:
+        if not row.get("remote_uploaded_at"):
+            continue
         delete_backup(row["uid"])
         removed += 1
     return removed
 
 
+def mark_remote_uploaded(uid: str, remote_path: str) -> None:
+    get_table("backups").update(
+        {"uid": uid, "remote_path": remote_path, "remote_uploaded_at": now_iso()},
+        ["uid"],
+    )
+
+
+def mark_local_purged(uid: str) -> None:
+    row = get_backup(uid)
+    if row:
+        _unlink_archive(row)
+    get_table("backups").update(
+        {"uid": uid, "local_path": "", "local_purged_at": now_iso()},
+        ["uid"],
+    )
+
+
+def list_pending_offload(limit: int = 50) -> list[dict]:
+    if "backups" not in db.tables:
+        return []
+    rows = get_table("backups").find(
+        status=STATUS_DONE, order_by=["created_at"], _limit=limit
+    )
+    return [
+        row
+        for row in rows
+        if not row.get("remote_uploaded_at")
+        and row.get("local_path")
+        and Path(row["local_path"]).is_file()
+    ]
+
+
+def list_offloaded_local_by_target(target: str) -> list[dict]:
+    if "backups" not in db.tables:
+        return []
+    rows = get_table("backups").find(
+        target=target, status=STATUS_DONE, order_by=["-created_at"]
+    )
+    return [
+        row
+        for row in rows
+        if row.get("remote_uploaded_at")
+        and row.get("local_path")
+        and Path(row["local_path"]).is_file()
+    ]
+
+
 def prune_orphans() -> int:
     removed = 0
     for row in list_backups(limit=100000):
         if row.get("status") != STATUS_DONE:
             continue
+        if row.get("remote_uploaded_at"):
+            continue
         local_path = row.get("local_path") or ""
         if not local_path or not Path(local_path).is_file():
             get_table("backups").delete(uid=row["uid"])

devplacepy/services/containers/CLAUDE.md

@@ -171,6 +171,23 @@ Playwright sessions and finding six terminal tabs. The fallback is now `host-${p
 extension host, which survives a browser reload and changes when the container restarts, which is
 exactly the intended semantic.
 
+**The extension's markers live in a FILE on the state mount, never in a memento (load-bearing).**
+The six-tab bug was only half fixed by the fallback above: every NEW browser session (a second tab,
+a reopen from the project page, an incognito window, a Playwright context) makes code-server start
+another extension host process while the previous one lingers for its 3h reconnection grace, and the
+new host found no marker, booted again, and the tab list grew by one shell per session - measured on
+the real image with the shipped extension (one, then two, then three `pravda@workspace` tabs across
+three sessions, three `Starting extension host process` lines in the container log). Neither
+`workspaceState` nor `globalState` can carry the marker: in code-server's web workbench both are
+proxied to the BROWSER's own storage (there is no `state.vscdb` anywhere under the state dir - only
+per-window `workspaceStorage/<hash>[-N]/vscode.lock` folders), so a memento is per browser profile,
+and a member's second device or a fresh context has never heard of the boot. The `Memory` class
+therefore keeps `devplace.bootMarker`, `devplace.panelPreset` and `devplace.welcomeShown` in
+`{DEVPLACE_WORKSPACE_STATE_DIR}/devplace-extension.json` - the one store whose lifetime is the
+workspace itself, read fresh on every `get` so concurrent hosts see each other's writes - and falls
+back to `globalState` only when the env var is absent (a non-DevPlace launch). Verified: three
+sessions, one shell, one welcome; a new container boot with a changed preset re-applies the layout.
+
 **Trust is disabled at three layers** and gated by one kill switch, `workspace_editor_trust_all`
 (default on): the `--disable-workspace-trust` flag, the seeded `security.workspace.trust.*` settings,
 and the extension's `contributes.configurationDefaults`. The third is belt and braces only -
@@ -186,6 +203,28 @@ SQLite database. Writing it from the host is rejected. The extension drives
 `workbench.action.toggleMaximizedPanel` / `increaseViewSize` instead, so the four presets are named
 honestly as presets in the UI. Do not "improve" this by writing `state.vscdb`.
 
+**The resize commands act on the FOCUSED part, and only the `ViewSize` pair does what its name
+says.** Measured on the real image with Playwright (1000px viewport, fresh state, terminal focused):
+`increaseViewSize` x4 took the panel from VS Code's default third (333px) to 573px, `decreaseViewSize`
+x24 drove it to its 77px floor, and `toggleMaximizedPanel` to 943px - while `increaseViewHeight` x4
+CRUSHED the panel to 93px and `decreaseViewHeight` x24 grew it to 873px. The first live build called
+`increaseViewSize` right after `terminal.show(true)` (`preserveFocus`), so the focused part was the
+Welcome editor and every "tall" boot shrank the terminal to its minimum: the production complaint
+that the `dpc` terminal was unusably small. `Layout.apply(terminal)` therefore takes the boot
+terminal `BootTerminals.open` returns and focuses it through its own handle
+(`terminal.show(false)`) - NOT `workbench.action.terminal.focus`, which raced the still-resolving
+boot terminals and spawned a stray default `bash` as the first tab on a fresh boot - then makes the
+size deterministic from the one known baseline: `decreaseViewSize` x`PANEL_FLOOR_STEPS` to the floor,
+then `increaseViewSize` x`PANEL_STEPS[preset]` (short 2, normal 4 = 317px, tall 7; the increment is
+60px); `maximized` toggles instead. It always normalizes, even on a brand-new state dir where VS
+Code's own default would already be a third: the panel size is persisted in the member's BROWSER, not
+in the state dir, so the only way to repair a panel crushed by the old bug on a member's existing
+browser profile is to normalize unconditionally (verified with one persistent Chromium profile: 93px
+under the shipped extension, 317px on the next boot under this one). The preset is applied when the
+recorded `devplace.panelPreset` marker differs from the profile (first boot under this extension, or
+the member changed it on DevPlace) and never otherwise, so a height the member drags themselves
+survives every restart.
+
 **The extension is a built-in, copied to
 `/usr/local/lib/code-server/lib/vscode/extensions/devplace-workspace`.** Built-ins are always
 enabled, cannot be uninstalled, need no install step and survive workspace recreation because they
@@ -196,17 +235,37 @@ code-server's Node remote extension host, so `main` applies.
 
 **`extension.js` is CommonJS, and it is the one file in this repository that may be.** The VS Code
 extension host loads CommonJS; it is not frontend code and is never served to a browser. Every other
-house rule applies unchanged. Four small classes (`Profile`, `BootTerminals`, `Layout`, `Presence`)
-and an `activate` that runs each through `stage()`, which owns the try/catch and logs to a
-`DevPlace` output channel.
+house rule applies unchanged. Seven small classes (`Profile`, `Memory`, `BootTerminals`, `Layout`,
+`Welcome`, `Presence`, `Tunnels`) and an `activate` that runs each through `stage()`, which owns the
+try/catch and logs to a `DevPlace` output channel.
+
+**The welcome page is a webview, shown once per state dir.** `Welcome.openOnBoot` opens
+`media/welcome.html` (placeholders `{{cspSource}}`, `{{nonce}}`, `{{styleUri}}`, `{{iconUri}}`,
+`{{projectTitle}}`, `{{agentStarted}}`, the two dpc URLs) in a `WebviewPanel` with `preserveFocus`,
+gated by the `devplace.welcomeShown` global-state memento, so a member sees it on the first boot of
+a workspace and never again unless they run **DevPlace: Show the welcome page** (or the walkthrough
+link). Its buttons `postMessage({action})` and the extension maps them through the
+`WELCOME_COMMANDS` allow-list to commands - a webview never names a command directly. It opens
+BEFORE `Layout` so the layout's `terminal.focus` leaves the member typing in `dpc`, not reading. The
+prose is sourced from https://dpc.app.molodetz.nl/ (context size, vision, swarms, deep research,
+safety gates, daily credits); when dpc's capabilities change, update `welcome.html`, the
+walkthrough and `/docs/workspace-editor.html` together. `welcome.css` uses only `--vscode-*`
+variables, which is what keeps it correct in both DevPlace themes.
 
 **The activation stages are awaited in order, and `Layout` never opens a panel of its own.** Firing
 them concurrently is what produced a stray third terminal in the first live build: `Layout` called
 `workbench.action.focusPanel` before `BootTerminals` had created anything, and VS Code answered by
-spawning its own default `bash`. `Layout.apply(panelIsOpen)` therefore resizes only when the boot
-terminals actually opened the panel, and `activate` awaits `terminals` before `layout`. Verified by
+spawning its own default `bash`. `Layout.apply(terminal)` therefore resizes only when
+`BootTerminals.open` actually created a terminal (it returns the one that got focus, `null` when the
+boot was skipped), and `activate` awaits `terminals`, then `welcome`, then `layout` (presence and
+the welcome commands are registered first, tunnels last). A skipped boot still calls
+`BootTerminals.reveal`, which shows the newest existing terminal (or the first one VS Code revives,
+waited for with `onDidOpenTerminal` up to `REVEAL_TIMEOUT_MS`) and never creates one, so a member
+opening the same running workspace from a second browser lands on the panel with the terminals that
+are already there instead of a closed panel or a duplicate agent. Verified by
 driving a real container with Playwright: the tab list must read exactly
-`pravda@workspace` + `DevPlace Code`.
+`pravda@workspace` + `DevPlace Code`, the editor area must hold exactly one `Welcome to DevPlace`
+tab, and `document.activeElement` must be the terminal's xterm textarea.
 
 **A workspace suppresses the editor's own AI assistant.** Recent VS Code ships a chat panel in the
 secondary sidebar, which opened by default with Microsoft branding, "AI responses may be inaccurate"
@@ -250,7 +309,7 @@ The security hotpatch that used to run per build is now baked into `ppy.Dockerfi
 - `COPY`s the **sudo superclone** (`files/sudo`) over `/usr/local/bin/sudo` (+ symlink `/usr/bin/sudo`; the real `sudo` package is not installed).
 - `COPY`s the **`aptroot` fakeroot wrapper** (`files/aptroot`, symlinked over `apt`/`apt-get`/`dpkg` in `/usr/local/bin` so pravda installs system packages without root).
 - `COPY`s **`pagent`** (`files/pagent`, the stdlib AI agent; reads `DEVPLACE_OPENAI_URL`+`DEVPLACE_API_KEY`, falling back to its public endpoint + `DEEPSEEK_API_KEY`) to `/usr/bin/pagent.py`, plus `files/.vimrc` to `/home/pravda/.vimrc` (whose AI helper - `AiEditSelection` - targets the same gateway as pagent via `DEVPLACE_OPENAI_URL`/`DEVPLACE_API_KEY`, with a public fallback, never `api.openai.com`).
-- `COPY`s **`dpc`** (`files/dpc`, DevPlace Code, the Claude-Code-class coding agent) to `/usr/bin/dpc`, and **`bot.py`** to `/usr/bin/botje.py`. **`dpc` is the one prebuilt binary in this repository**: an ELF 64-bit x86-64 position-independent executable, 10850272 bytes, sha256 `0c0f980717deebed285dcde7968c249f77638a65d76af8ae5deeda3ac2b0f042`. Its source is NOT in this repository and there is no build recipe here, so unlike every other file in `files/` it cannot be reviewed before it is installed root-owned onto `PATH` in every user container. Record a new size and checksum here whenever it is replaced - this record is the only integrity check that exists on it, so a stale entry is worse than none. (The previous entry, 3578664 bytes / sha256 `24f7fbb0...`, described a build that is no longer the file on disk.)
+- `COPY`s **`dpc`** (`files/dpc`, DevPlace Code, the Claude-Code-class coding agent) to `/usr/bin/dpc`, and **`bot.py`** to `/usr/bin/botje.py`. **`dpc` is the one prebuilt binary in this repository**: an ELF 64-bit x86-64 position-independent executable, 11345688 bytes, sha256 `f63c763196e2eddc1ab7f361c5d00fe225bd9cd1f6ce0c7ca145dabb5fd6fd82`. Its source is NOT in this repository and there is no build recipe here, so unlike every other file in `files/` it cannot be reviewed before it is installed root-owned onto `PATH` in every user container. Record a new size and checksum here whenever it is replaced - this record is the only integrity check that exists on it, so a stale entry is worse than none.
 - Evicts any pre-existing uid-1000 user, creates user **`pravda` at `1000:1000`**.
 - Hands pravda ownership of the toolchain AND the OS package trees (`chown -R pravda` over `/usr/local/lib`, `/usr/local/bin`, `/usr/lib/python3`, `/opt`, `/app`, `/home/pravda`, plus `/usr/lib`, `/usr/bin`, `/usr/sbin`, `/usr/share`, `/usr/include`, `/etc`, `/var/lib`, `/var/cache`, `/var/log`, `/srv` so `apt`/`dpkg` can write; `~/.local/bin` on `PATH`).
 - Ends on `USER pravda`.
@@ -292,11 +351,102 @@ The container manager drives the host docker daemon, which needs heavy wiring -
 
 **The DooD bind-mount gotcha:** `docker run -v <path>:/app` resolves `<path>` on the HOST, so `DEVPLACE_DATA_DIR` must be mounted at an identical host+container path (the make targets use `$(CURDIR)/data` on both sides; manual `docker compose` users get a `/srv/devplace-data` default). Build contexts ship via the docker API tarball, so the container temp dir is fine. Set `DEVPLACE_CONTAINER_PROXY_HOST=host.docker.internal` only when a containerized app cannot route to the recorded gateway. See README "Container Manager wiring".
 
-## Bidirectional newer-wins sync (load-bearing direction rule)
+## Bidirectional sync with deletion propagation (load-bearing direction rule)
 
-Sync is NOT one-directional import. `project_files.sync_dir_bidirectional(project_uid, workspace, user) -> {"exported", "imported"}` is the one helper that reconciles a project's virtual FS against an instance's `workspace_dir`: per file, the side with the newer timestamp wins (project `updated_at` epoch, via `datetime.fromisoformat(...).timestamp()`, vs filesystem `st_mtime`, with a 1s skew tolerance favouring export on ties), and a file present on only one side propagates to the other. It **NEVER deletes a file** - only creates/overwrites the older side. A **read-only** project (`is_readonly`) exports only, never imports (the read-only guard direction).
+Sync is NOT one-directional import, and it is not merely "newer wins" either - it propagates
+deletions on BOTH sides, which requires more state than comparing two live snapshots can
+ever provide. `project_files.sync_dir_bidirectional(project_uid, workspace, user) ->
+{"exported", "imported", "deleted_in_project", "deleted_in_workspace"}` reconciles a
+project's virtual FS against an instance's `workspace_dir` against a **third,
+persisted state: the manifest of what was true after the previous sync**
+(`project_file_sync_state`, one row per `(project_uid, path)` holding `db_epoch` +
+`fs_epoch` as they stood right after that prior sync wrote them - never a guess, always
+re-read from the actual post-write `stat()`/row so a later comparison is exact). Comparing
+only the two live sides can never tell "never synced here yet, please export it" apart from
+"was here, got deleted, please propagate that" - both look identical (present in the
+project, absent on disk) with no baseline. The manifest is exactly that baseline, the same
+role a `.git` index or an rsync/Unison state file plays in any real bidirectional sync.
 
-Both `api.sync_workspace` (HTTP/Devii sync action, returns `{exported, imported}`) and `api.sync_bidirectional_sync` (reconciler, non-blocking `record_event` system actor, logs only when non-zero) call the same helper. The reconciler runs it before every `_launch` AND on a ~60s wall-clock cadence over running instances (`SYNC_EVERY_SECONDS`, gated on `time.monotonic()` independent of the 5s reconcile tick). The per-instance boot-helper files (`.devplace_boot.py`/`.devplace_boot.sh`) are in `SYNC_SKIP_NAMES` so they never round-trip into the project.
+Per path, three states are possible (`db_files`/`fs_files`/`manifest`, keyed by path):
+
+- **Present both sides.** Newer wins as before (`db_epoch >= fs_epoch - 1s` skew exports,
+  otherwise imports) - a read-only project always exports here too (see below).
+- **Present in the project, absent on disk.** Not in the manifest (or read-only) -> never
+  materialized here, export it. In the manifest -> the workspace deleted it since the last
+  sync -> **propagate: soft-delete the project row** (`_delete_db_row_for_sync`, stamps
+  `deleted_at`/`deleted_by=user["uid"]`, unlinks the blob) - UNLESS the project's own copy
+  was edited after that last sync (`db_epoch` newer than the manifest's), in which case the
+  edit wins and the file is restored instead of deleted.
+- **Present on disk, absent from the project.** Not in the manifest -> a brand new file
+  created in the workspace -> import it. In the manifest -> the project deleted it since
+  the last sync -> **propagate: delete the local file** - UNLESS it was edited locally after
+  that last sync, in which case the edit wins and it is re-imported (resurrected). A failed
+  edit-wins import is left untouched rather than deleted, so a transient I/O error can never
+  destroy the only remaining copy.
+
+**A read-only project always wins, in both directions**: present-both-sides always exports
+(a locally-newer edit in a read-only workspace is discarded, never imported - it could never
+round-trip anyway); a local deletion is always restored (never propagated - read-only means
+the workspace mirrors the project verbatim, it does not get to delete from it); and when the
+*project* deletes a file, its now-stale local copy in a read-only workspace is still removed
+(that is export-direction cleanup, not an import, so it does not violate "read-only never
+imports").
+
+The manifest is written with a diff, not a full rewrite: `_save_sync_manifest` deletes only
+the rows for paths that left the merged state and upserts only the rows whose entry actually
+changed, so an idle project with nothing to sync costs zero writes on the next tick despite
+having thousands of files. It is cleared entirely (`clear_sync_state`) whenever a project's
+files are wiped (`soft_delete_all_project_files`, `delete_all_project_files` / fork
+rollback) - a later restore starts the baseline fresh, which just means the first sync after
+restore treats everything as newly seen (safe, not destructive).
+
+Both `api.sync_workspace` (HTTP/Devii sync action) and `api.sync_bidirectional_sync`
+(reconciler, non-blocking `record_event` system actor, logs only when any count is nonzero)
+call the same helper and return/record all four counts. The reconciler runs it before every
+`_launch` AND on a ~60s wall-clock cadence over running instances (`SYNC_EVERY_SECONDS`,
+gated on `time.monotonic()` independent of the 5s reconcile tick). The per-instance
+boot-helper files (`.devplace_boot.py`/`.devplace_boot.sh`) are in `SYNC_SKIP_NAMES` so they
+never round-trip into the project, and therefore never enter the manifest either.
+
+**Both call sites are unsynchronized, and that used to leak millions of orphan blobs
+(load-bearing, incident-derived).** `sync_workspace` (user-triggered) and
+`sync_bidirectional_sync` (the reconciler tick) can run for the SAME project concurrently
+whenever a sync outlasts the reconcile interval - trivial for any workspace that is
+actively compiling. `project_files.store_upload` (the import path for a changed file)
+writes a NEW blob under a fresh uuid, then updates the existing row to point at it; two
+racing imports of the same changed path each write their own blob and both update the
+same row (last writer wins), so the loser's freshly-written blob is referenced by nothing
+- an orphan on every race, not just on a crash. Combined with no build-artifact exclusion,
+an actively-compiling workspace regenerates thousands of files with fresh mtimes on every
+build, which is exactly the high-churn pattern that maximizes how often the race fires;
+sustained over time this leaked **5.9 million orphan blobs (~96GB)** in production, discovered
+only when the host disk hit 100% full. Both are now closed at their root:
+
+- **`api._sync_dir_bidirectional_locked`** wraps `project_files.sync_dir_bidirectional`
+  behind a per-`project_uid` `threading.Lock` (`api._sync_lock_for`, a lazily-created
+  registry - both `sync_workspace` and `sync_bidirectional_sync` call it instead of the
+  raw function). The acquire is **non-blocking**: a second sync for a project already mid-sync
+  is skipped outright (returns the zero-counts dict), never queued or awaited. This runs on
+  an `asyncio.to_thread` worker in both call sites, so blocking would tie up a thread pool
+  slot for no reason - a skipped project is simply picked up on the next tick or the user's
+  next explicit sync.
+- **`project_files.IMPORT_SKIP_NAMES`** gained common build-output directory names (`build`,
+  `dist`, `target`, `out`, `bin`, `obj`, `.next`, `.nuxt`, `.gradle`, `.tox`,
+  `cmake-build-debug`, `cmake-build-release`) and a NEW parallel set,
+  **`IMPORT_SKIP_EXTENSIONS`** (`.o`, `.obj`, `.pyc`, `.pyo`, `.class`, `.so`, `.dylib`,
+  `.dll`, `.a`, `.exe`), matched by suffix in `_walk_workspace_files` rather than exact
+  name (`IMPORT_SKIP_NAMES`/`SYNC_SKIP_NAMES` are exact-name-only sets, which cannot express
+  "any file ending in `.o`"). Both apply to every `_walk_workspace_files` caller (`import_from_dir`
+  AND the sync path via `SYNC_SKIP_NAMES = IMPORT_SKIP_NAMES | {...}`), so compiled/build
+  artifacts are never imported into a project regardless of entry point, not just never synced.
+
+Neither fix requires (or should ever regress into) a redesign of the manifest/reconciliation
+model above - the lock only prevents two runs from touching the same project's `project_files`
+rows at once, and the skip sets only shrink what `_walk_workspace_files` yields. Recovering
+already-leaked blobs is a separate, data-layer concern: see `devplace system prune`
+(`cli/system.py`) and `attachments.sweep_orphan_attachment_blobs`/
+`project_files.sweep_orphan_project_file_blobs`, which sweep any blob with zero database
+reference at all, regardless of what leaked it.
 
 ## Run-as user = identity + API key ONLY (load-bearing constraint)
 
@@ -322,7 +472,11 @@ Every reconciler status mutation flows through `service._set_status(inst, change
 
 ## Testing without Docker
 
-Use `FakeBackend` (its `image_exists` returns `True`) + `runtime.set_backend`, and monkeypatch `config.CONTAINER_WORKSPACES_DIR` to a tmp dir. See `tests/unit/services/containers.py` and `tests/api/containers.py` (argv, instance creation on `config.CONTAINER_IMAGE`, image-not-built guard, reconcile matrices, schedule firing, ingress validation + live HTTP proxy, HTTP admin gate).
+Use `FakeBackend` (its `image_exists` returns `True`) + `runtime.set_backend`, and monkeypatch `config.CONTAINER_WORKSPACES_DIR` to a tmp dir. See `tests/unit/services/containers.py` and `tests/api/containers.py` (argv, instance creation on `config.CONTAINER_IMAGE`, image-not-built guard, reconcile matrices, schedule firing, ingress validation + live HTTP proxy, HTTP admin gate). Editor readiness in tests is a real listening socket: bind `127.0.0.1:0`, `listen()`, and give the instance `editor_port` 8443 with a `ports_json` mapping that container port to the socket's host port (`tests/e2e/projects/workspace.py` `editor_listener`, `tests/api/projects/workspace.py` `_editor_listener`), so `phase` is exercised against the same probe production uses rather than a mock.
+
+## Testing against the real image (and the one thing never to do)
+
+The editor extension, the proxied workbench and the panel layout can only be verified on the real `ppy` image. The harness that works: a scratch code-server container started by hand (`docker run --entrypoint /bin/sh ppy:latest -c 'exec code-server ...'` with the same argv `editor.argv` builds, **no `devplace.instance` label**, the state dir and `/app` bind-mounted from a scratch directory, and the extension directory bind-mounted read-only over `/usr/local/lib/code-server/lib/vscode/extensions/devplace-workspace` so an edit is live on the next container start with no `make ppy`), plus a scratch DevPlace server (`DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` in the scratch dir, `DEVPLACE_DISABLE_SERVICES=1`, another port) seeded with an instance row whose `container_ip` is the scratch container's bridge IP, then Playwright against both. Simulate lifecycle transitions with `store.update_instance` from a script. **Never enable `ContainerService` against a scratch database on the host daemon**: its orphan sweep `rm -f`s every `devplace.instance`-labelled container its database does not know, which on this host is every production workspace. Measure VS Code layout by bounding box, never by assumption - see the `ViewSize`/`ViewHeight` table above for how wrong an assumption was.
 
 ## Vibe coding on-ramp (user-facing doc)
 
@@ -396,6 +550,22 @@ while throttled, so a second call would double the request counter). **Request b
 buffered on purpose**: they are bounded by nginx `client_max_body_size`, and streaming them would
 force chunked encoding onto arbitrary upstream apps.
 
+**`<base>` is injected into the ROOT document only (`forward.is_root_document(path)`), never into a
+nested HTML page.** A base tag exists for one case: the root document requested without a trailing
+slash (`/p/slug`, `/code`), where relative URLs would otherwise resolve one level too high. Every
+nested page already resolves its relative URLs against its own directory, so a base tag there is
+not a no-op but a corruption. The concrete victim was VS Code's webview host page
+(`.../static/out/vs/workbench/contrib/webview/browser/pre/index.html`, an iframe the Extensions
+view and every extension detail page load): it registers `service-worker.js` and probes
+`./fake.html` relative to its own directory, and with the injected base both went to
+`/code/service-worker.js` and `/code/fake.html` (observed as 404s from the proxy), so the webview's
+service worker could not register - the "Extensions view crashes" report. code-server's own pages
+already carry a `<base>` and were never touched, which is exactly why only the internal iframes
+broke. Reproduce with the real image behind the scratch proxy: fetch the webview `index.html`
+through `/code/...` and grep for `<base`. The editor route's refusals (`403`/`409`/`502`) carry
+`media_type="text/plain"`, because a bare `Response` with no content type makes a browser download
+the error as a file instead of showing it.
+
 **One keep-alive client, closed on shutdown.** `forward.client()` is a lazily-created module-level
 `httpx.AsyncClient` with `httpx.Limits` (the `ChromeStealthClient` pattern), closed by
 `forward.close_client()` in the `main.py` lifespan. A client per request meant a fresh pool and TCP
@@ -419,7 +589,16 @@ into one atomic `COALESCE` UPDATE (`store.record_activity`). Both proxy planes c
 plane B traverses DevPlace, public traffic is observed directly rather than inferred.
 
 **`workspace/` package.** `quota.py` resolves limits instance -> user rule -> setting -> default
-through ONE resolver (never read a workspace setting at a call site). `flags.py` is the abuse ledger
+through ONE resolver (never read a workspace setting at a call site). **An administrator-owned
+workspace is unlimited**, decided inside `resolve()` itself (`_is_admin_owner`, `owner_uid in
+get_admin_uids()`) and surfaced as `Limits.unlimited`, never by special-casing role at a call site.
+The three enforcement points check it: `provision.ensure` skips the `max_workspaces` count check,
+`provision.publish_tunnel` skips the `max_tunnels` count check, and
+`WorkspaceService._advance_lifecycle` `continue`s past a row entirely (no idle-stop, no idle-warn, no
+retention-delete, no delete-warn) when `limits.unlimited`. Hard docker resource allocation
+(`cpu_millicores`/`memory_mb`, the `--cpus`/`--memory` flags) and the abuse-flag evaluator
+(`_evaluate_flags`) are deliberately untouched - those protect the host itself and stay in force
+for every owner including admins. `flags.py` is the abuse ledger
 and is **idempotent per `(instance_uid, kind)` while a flag is open**, so a sustained condition is one
 row, not one per tick. `naming.py` generates faker labels with collision retry and owns the hostname
 patterns plus `is_tunnel_host`. `tunnels.py` is CRUD with revive-not-duplicate. `provision.py` is the
@@ -554,12 +733,44 @@ button in `.project-detail-actions` via `templates/_editor_open.html` (`target="
 `data-editor-*` attributes `EditorLauncher` reads) straight to the code-server proxy
 `/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url` in
 `routers/projects/index.py` and carried as `workspace_editor_url` on the context and
-`ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND their
-workspace for that project exists, is not suspended, and is `store.ST_RUNNING` - the three states the
-`editor_proxy` route itself refuses (403 suspended, 409 not running, 502 no port), so the button can
-never open a dead editor. When there is no running workspace the button is absent and the Workspace
-menu item below is the way in (create/start it there). The workspace page's own **Open editor** link
-opens in a new tab too; keep both in step.
+`ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND
+`provision.editor_ready(instance)` holds: the workspace exists, is not suspended, is
+`store.ST_RUNNING`, AND the editor port answers a TCP connect (`api.editor_reachable`, the same
+`tunnel_target` the proxy dials) - the states the `editor_proxy` route itself refuses (403 suspended,
+409 not running, 502 no port) plus the boot window in which the container is up but code-server is
+not yet listening, so the button can never open a dead editor. When there is no ready workspace the
+button is absent and the Workspace menu item below is the way in (create/start it there). The
+workspace page's own **Open editor** link opens in a new tab too; keep both in step.
+
+**The workspace page renders a PHASE, and the phase is computed once, server-side.**
+`provision.phase(instance, ready)` is a pure function of `suspended_at`, `desired_state`, `status`
+and the editor probe: `suspended`; `desired=running` -> `ready` when `running` and reachable, else
+`starting` (covers `created`, a stopped row just asked to start, and the running-but-not-listening
+boot window); `desired=stopped` -> `stopping` while the container is still `running`/`paused`/
+`restarting`, `crashed` when the exit was a crash, else `stopped`. `provision.view` adds `phase`,
+`phase_label` (`PHASE_LABELS`) and `editor_ready` (all on `WorkspaceViewOut`, so Devii's
+`workspace_status` and the admin console see them too) and reads the probe exactly once per view.
+The probe is `socket.create_connection` with a 0.3s timeout: a listening or refusing port answers in
+under a millisecond, so the page only pays the timeout on a dropped leg, which is a case the proxy
+would 502 on anyway. `workspace.html` renders EVERY phase's controls inside
+`.workspace-phase-view[data-phase-view="..."]` blocks (space-separated phases, `stopped crashed`
+share the Start form) and shows one via the `hidden` attribute, so the no-JS page is correct and
+`WorkspaceManager` only toggles `hidden`, the badge and the launch href - it never rebuilds markup.
+`starting` shows a disabled `.btn.is-loading` spinner (the `btn-spinner` pattern from
+`IssueReporter`) plus Stop; `ready` shows the `_editor_open.html` link plus Stop; `stopping` a
+spinner; `stopped`/`crashed` the Start form (its button carries a spinner span the manager turns on
+while the POST is in flight, which is what stops a member pressing Start twenty times).
+`WorkspaceManager` MUST be mounted from the template's inline module
+(`window.app.workspaceManager = WorkspaceManager.mount()`, the `containers_instance.html` pattern) -
+it shipped once as a bare `<script type="module" src>` that only defined the class, so the page had
+no JavaScript at all and the Start button was a plain form post. Liveness is two-fold: an HTTP poll
+of the page's own JSON at 2s while the phase is transitional and 20s otherwise (`Poller`, swapped
+when the phase class changes), plus a pub/sub subscription to `user.{owner_uid}.workspace.{uid}`,
+which the live view relay serves (`_workspace_detail`, 3s, owner-checked; the topic sits in the
+owner's private namespace so `pubsub/policy.py` admits the owner and admins and nobody else) with
+the same `{workspace, editor_url}` shape the page JSON carries. Only the start/stop forms
+(`data-workspace-action`) go through the manager; tunnels, editor preferences and delete keep
+their native page-reloading submit, which the e2e tests assert with `wait_for_url`.
 
 **Member entry point** is the project detail page's overflow menu (`project_detail.html`), gated by
 the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, set in