ticket #132 attempt 1

- Policy: every change ends with make test (all tiers, all tests) green; docs and agent guardrails updated accordingly
- schema.py: ensure the full filtered/indexed column set of instances via get_table (ingress_slug, ports_json, container_gateway, slug, status, ...) so a partial first insert can never break the ingress proxy
- docs_api: award body param location body -> json; gateway endpoints documented public -> user to match enforced auth
- tests: missing get_table import (trash restore), audit read as admin (award), deterministic online-roster and leaderboard-cache handling, container visibility updated to the primary-admin-only rule, scoped AI usage heading selector past the hidden Tools nav links

.claude/agents/test-maintainer.md

@@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
 - **F. Verify your own work.** After writing a test module, validate it ONLY by a clean import (`python -c "import tests..."` or `python -m py_compile`).
 
 ## Mode
-Default to **REPORT** mode: record coverage gaps and pattern violations, do NOT write files. Apply **FIX** mode only when the invocation explicitly asks you to fix; then write the missing integration test following the required patterns. **HARD GUARDRAIL: write tests but NEVER run the suite, not the full suite and not a single file.** Validate only by a clean import of the new test module. Never perform any git write operation.
+Default to **REPORT** mode: record coverage gaps and pattern violations, do NOT write files. Apply **FIX** mode only when the invocation explicitly asks you to fix; then write the missing integration test following the required patterns. **HARD GUARDRAIL: write tests but NEVER launch the suite yourself, not the full suite and not a single file - the serial single-process suite cannot run concurrently with other agents, so the orchestrating session runs `make test` (all tests) after your work.** Validate only by a clean import of the new test module. Never perform any git write operation.
 
 ## Obey the rules you enforce
 No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.

.claude/commands/test.md

@@ -1,5 +1,5 @@
 ---
-description: Run DevPlace tests - the sanctioned explicit-ask path. Run a tier, a file, or a single test with the correct flags. The agents never run tests themselves; this command is how you ask.
+description: Run DevPlace tests. Run a tier, a file, or a single test with the correct flags. Subagents never run tests themselves (the serial suite cannot run concurrently); the orchestrating session always runs the full suite as the final validation of every change.
 argument-hint: [unit|api|e2e|all|<path::test_name>]
 allowed-tools: Bash(make test*), Bash(python -m pytest *), Read
 ---
@@ -12,6 +12,6 @@ Mapping:
 - `all` or empty -> `make test`
 - a path like `tests/api/posts/create.py::test_x` -> `python -m pytest <that> -v --tb=line -x`
 
-Tests run serially on port 10501 with a tempfile SQLite DB and `DEVPLACE_DISABLE_SERVICES=1`. This command is the one sanctioned way to run them (the subagents and workflows never do).
+Tests run serially on port 10501 with a tempfile SQLite DB and `DEVPLACE_DISABLE_SERVICES=1`. Subagents and workflows never launch tests (the serial single-process suite cannot run concurrently); the orchestrating session runs the full suite (`make test`) as the mandatory final validation of every change.
 
 Report results clearly. On a failure, show the relevant output, and if a browser (e2e) test failed, point me at the screenshot under `/tmp/devplace_test_screenshots/`. Never weaken a test to make it pass; if a test reveals a real bug, report it - do not edit the test.

.gitignore

@@ -32,3 +32,10 @@ var/
 .coverage
 .coverage.*
 htmlcov/
+
+# local environments and scratch
+.venv/
+tmp/
+*.log
+*.bak
+test.db

CLAUDE.md

@@ -2,6 +2,8 @@
 
 This file provides guidance to Claude Code when working with code in this repository. It holds only what applies regardless of which part of the codebase is being touched. Deep, subsystem-specific detail lives in nested `CLAUDE.md` files placed inside the relevant directory - Claude Code auto-loads a nested file only when a file under that directory is read or edited, so the always-loaded cost of this repository stays proportional to this file alone. See "Subsystem map" below for the full list.
 
+It is a big project, whatever you are implementing, it is probably done before. You should look it up and match the implementation structurely and visually. For inconsistency there is zero tolerance policy. Develop dry, kiss, re-usable code, consistent with existing implementation. Literally always try to find relatable examples before making a modification. If no-example exists, explain to user what is the case and let user decide what to do and how to continue. 
+
 ## Project
 
 DevPlace is a server-rendered social network for developers. FastAPI backend serves Jinja2 templates with pure ES6 module JavaScript on the frontend. SQLite via the `dataset` library (auto-syncs schema). No JS framework, no NPM, no JWT.
@@ -29,7 +31,7 @@ make locust-headless  # Locust CLI mode for CI
 
 The Makefile exports `PYTHONDONTWRITEBYTECODE=1` for every recipe, so no `.pyc` files or `__pycache__` directories are written by any `make` target. Keep it that way - do not add a target that re-enables bytecode writing. `make clean` removes any stray bytecode left from running Python outside make.
 
-Validate code without running the suite: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance).
+Preliminary validation: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance). These checks are gates on the way to the real validation, never a substitute for it: **every change ends with the full test suite (`make test` - all three tiers, every test) and it must pass.**
 
 Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x`
 
@@ -51,10 +53,14 @@ devplace attachments prune  # remove orphan attachment records/files
 devplace devii reset-quota <username>   # reset one user's rolling 24h AI quota
 devplace devii reset-quota --guests     # reset every guest quota
 devplace devii reset-quota --all        # reset every quota (users and guests)
+devplace gateway quota list             # list AI gateway quota rules and current 24h spend
+devplace gateway quota set --limit-usd N [--owner-kind K] [--owner-id ID] [--app-reference APP] [--label L] [--uid UID]
+devplace gateway quota delete <uid>     # delete a quota rule
 devplace zips prune         # delete expired zip archives + job rows
 devplace zips clear         # delete every zip archive + job row
 devplace forks prune        # delete expired completed fork job rows (forked projects persist)
 devplace forks clear        # delete every fork job row (forked projects persist)
+devplace messaging prune-tickets  # delete expired WebSocket auth tickets (ws_tickets)
 devplace seo prune          # delete expired SEO audit reports + job rows
 devplace seo clear          # delete every SEO audit report + job row
 devplace seo-meta prune     # delete expired SEO metadata job rows (generated metadata persists)
@@ -99,6 +105,7 @@ Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA
 | `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Max avatars in the feed's live "Online now" panel. |
 | `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS` | `20` | Grace margin (hysteresis) before an online user is dropped, kills dot/roster flicker at the boundary. |
 | `DEVPLACE_DATA_DIR` | `<repo>/data` | Single root for ALL runtime/user-generated data OUTSIDE the package and OUTSIDE `/static`. Point at a volume in prod. |
+| `DEVPLACE_OUTBOUND_PROXY_URL` | unset | Fallback for the `outbound_proxy_url` site setting (below) when the DB/settings row is unavailable (early CLI contexts). Prefer configuring the setting via `/admin/settings` - it applies live with no restart. |
 
 ## Subsystem map
 
@@ -130,6 +137,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
 | `devplacepy/database/CLAUDE.md` | Dataset rules, indexing conventions, soft delete, tables, site settings |
 | `devplacepy/utils/CLAUDE.md` | Notifications and gamification (XP/levels/badges/leaderboard) |
 | `devplacepy/static/js/CLAUDE.md` | Custom web components, shared frontend utilities (Http/Poller/FloatingWindow/ScrollMemory) |
+| `devplacepy/static/css/CLAUDE.md` | CSS system: design tokens (no literals, no `var()` fallbacks), file-scoped palettes, `--z-*` stacking bands, the closed breakpoint set, reduced motion |
 | `devplacepy/templates/CLAUDE.md` | Modal system, CDN libraries, shared template partials |
 | `tests/CLAUDE.md` | Detailed testing patterns and pitfalls |
 
@@ -152,7 +160,7 @@ Routers in `devplacepy/routers/` are organised as a **directory tree that mirror
 | `/auth` | auth/ package |
 | `/feed`, `/posts`, `/comments` | flat files |
 | `/projects`, `/projects/{slug}/files`, `/projects/{slug}/containers` | projects/ package - see `routers/projects/CLAUDE.md` |
-| `/profile` | profile/ package (customization, notifications, ai-correction, ai-modifier, telegram, usage) |
+| `/profile` | profile/ package (customization, notifications, ai-correction, ai-modifier, interactions, telegram, usage) |
 | `/messages` | messages.py - see `services/messaging/CLAUDE.md` |
 | `/notifications`, `/votes`, `/reactions`, `/bookmarks`, `/polls`, `/avatar`, `/follow`, `/leaderboard` | flat files |
 | (none) | relations.py - block/mute |
@@ -190,7 +198,9 @@ The CLIENT pipeline (`ContentRenderer.js`, `dp-content`/`dp-title`) is retained
 
 **Emoji shortcodes are the full GitHub/Discord `:name:` set** (~4869 names), generated once from the `emoji` library by `rendering.py` `build_emoji_shortcodes()`; the frontend gets the identical map via the generated `static/js/emoji-shortcodes.js` (regenerate with `devplace emoji-sync` after bumping the dependency, never hand-edit).
 
-**Template em-dash normalization:** the shared `templates.env.template_class` runs every rendered template's final HTML through `normalize_em_dash` - the em-dash character and its HTML entity forms all become a plain hyphen, application-wide. One hook; never re-strip em-dash per template.
+**Email anonymization:** both `_render_content` and `_render_title` mask email addresses to prevent doxing. `_mask_emails` in `rendering.py` runs on rendered text nodes only (inside `_transform_text`, `_MediaProcessor`, and `_InlineFilter` - never before mistune, or the `*` mask characters would be parsed as emphasis) and stars ~80% of the local part (keeps a leading 20%, minimum one visible char). Addresses on `molodetz.nl` (and its subdomains) are exempt and render verbatim.
+
+**Em-dash normalization:** `_normalize_dashes` in `rendering.py` replaces all forms (em dash `\u2014`, en dash `\u2013`, and their HTML entities) with a hyphen BEFORE mistune processes the text. Runs inside `_render_content`/`_render_title` which are `@lru_cache`d, so each unique text is normalized once. No per-template overhead.
 
 ### Auth
 
@@ -235,7 +245,7 @@ Devii is also reachable over **Telegram** (one supervised long-poller subprocess
 
 - **No comments, no docstrings in source.** Code is self-documenting.
 - **Forbidden variable name patterns:** `_new`, `_old`, `_temp`, `_v2`, `better_`, `my_`, `the_`.
-- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: `routers/proxy.py` relays the user's own headers verbatim. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter).
+- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: `routers/proxy.py` relays the user's own headers verbatim. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
 - **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
 - **`RedirectResponse(url=..., status_code=302)`** - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
 - **Ownership checks:** compare `resource["user_uid"] == user["uid"]` before edit/delete (use `content.is_owner`). **Deletes additionally allow any admin:** `is_owner(...) or is_admin(user)` on every content delete endpoint, so an administrator may delete any member's content while members are limited to their own. Edits stay owner-only. Deletes are **soft** and cascade under one shared stamp - see `devplacepy/database/CLAUDE.md`.
@@ -276,7 +286,18 @@ Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `
 
 Required patterns: every `page.goto(...)`/`page.wait_for_url(...)` MUST pass `wait_until="domcontentloaded"`; prefer `page.locator(...).wait_for(state="visible")` over `wait_for_selector`; scope ambiguous selectors (e.g. comment Delete is `.comment-action-btn:has-text('Delete')`). A test that flips a global `site_settings` value MUST restore it in `try/finally`. Full pitfalls/patterns catalogue is in `tests/CLAUDE.md`.
 
-**Never run tests unless the user explicitly asks for it.** Not the full suite, not a single file. Validate with a clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks instead.
+**Always run the full test suite (`make test` - unit, api, and e2e, every test) as the final validation of every change.** No tier may be skipped and no subset substituted for the whole. The clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks are preliminary gates before the suite, not replacements for it. Any failure is a real signal and blocks completion until fixed.
+
+## Rigorous correctness verification (money, state machines, concurrency)
+
+The persisted test suite (`tests/unit`/`api`/`e2e`) is example-based: it proves the specific inputs someone wrote down behave correctly. It is structurally blind to three classes of bug - a formula wrong at an input nobody tried, an invariant violated only after a long sequence of actions, and a race that only two nearly-simultaneous requests trigger. **Any feature that touches a spendable resource (coins, credits, quotas), a bounded state machine (levels, tiers, counters with a floor/ceiling), or a read-then-write mutation reachable from more than one request path applies this procedure in addition to, not instead of, the normal persisted tests.** A simple CRUD toggle doesn't need it; an economy, inventory, ledger, or scoring feature does. This was built out fully for the Code Farm economy rebalance (see `devplacepy/services/game/CLAUDE.md`, "Every purchase/upgrade is atomic..." - the worked example, including every bug it actually caught) and is the standing procedure for anything shaped like it going forward.
+
+Four escalating layers, run in order, as disposable Python scripts that call the real functions directly against a temp DB (`DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` pointed at a scratch file) - not pytest files, unless the user separately asks for persisted tests too:
+
+1. **Property/invariant checks on every pure function, across the full input domain.** For each formula (a reward, a cost curve, a score, a discount), assert the mathematical property it must have - monotonicity, bounds, non-negativity, idempotency, additivity over a partition of its domain - by iterating the real range (hundreds to thousands of values), not a handful of hand-picked spot checks. A monotonicity bug at input 347 is invisible if only 0, 1, and 10 are tried.
+2. **Stateful fuzzing.** Build N independent entities, fire a long randomized sequence of every mutating action across all of them against a live temp DB, and re-check invariants after every single action: balances never negative, counters that should only increase never decrease, levels/tiers never exceed their declared bounds. Catch the domain's expected exception type and continue - the goal is catching invariant violations and *unexpected* exceptions, not asserting every action succeeds. This proves safety (nothing bad happens) but not liveness (legitimate actions actually succeed) - a bug that wrongly blocks a valid action just raises a well-formed expected exception and sails through a pure fuzz test undetected. Layer 3 exists because of exactly this gap.
+3. **Concurrency: prove races are closed, never infer it from luck.** Any read-then-write mutation reachable from more than one worker process (`uvicorn --workers N`) is a TOCTOU race until proven otherwise. Test with **real separate OS processes**, not threads in one process - `dataset` gives each thread its own pooled connection, and enough threads exhausts that pool and produces `database is locked` noise that is a test-harness artifact, not a finding about the application. Set up genuinely fresh, production-representative state before racing - **never pre-seed or zero a column that the real code path leaves unset/NULL.** This was the single most expensive mistake made building this procedure: a first race-safety pass "passed" only because its own setup script had artificially pre-zeroed columns that a real fresh row leaves as SQL `NULL` - silently hiding the exact bug the fix was supposed to prevent (`NULL = 0` evaluates to `NULL`, not true, in a SQL `WHERE` clause; any column not written in the row's original `INSERT` needs `COALESCE(column, 0)` in every later precondition and every arithmetic `SET`, not a bare comparison). Fire many concurrent attempts at the same resource, assert the exact right number succeed, and verify the final state matches the hand-computed expected total exactly (currency spent, levels advanced) - not just "the others were blocked." Fix a real race with a single atomic conditional SQL statement at the exact chokepoint (`UPDATE ... SET ... WHERE <precondition>`, checked via the driver's real `rowcount` - `dataset`'s wrapped `db.query()` does not expose it, use `db.executable.execute(sqlalchemy.text(...), params).rowcount` inside `with db:`), not a client-side lock or an optimistic-locking library this codebase doesn't otherwise use.
+4. **Static analysis beyond `py_compile`.** A clean `python -m py_compile` and a clean `from devplacepy.main import app` only prove syntax and module-load order - neither catches a missing import inside a function body, which is a runtime `NameError` invisible until that exact line executes. Run `pyflakes`/`ruff check` on every touched file before calling a change done; it is nearly free and catches an entire class of bug that layers 1-3 can each individually miss if they don't happen to exercise the broken line.
 
 ## Feature workflow
 
@@ -295,7 +316,7 @@ A new public read almost always needs all four. The cardinal failure mode is cha
 4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
 5. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog.py` - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
 6. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
-7. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen). **Never run the test suite unless the user explicitly asks.** Write new tests in the matching tier/path when asked, following the required patterns above.
+7. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.
 
 Failures at any implementation step block the workflow - never skip a failed step.
 
@@ -310,3 +331,4 @@ Single-host Docker Compose (`docker-compose.yml`): an **app** container (Uvicorn
 - **Shared DB and files = same as dev.** The app container bind-mounts the host project root (`.:/app`) and runs as `${DEVPLACE_UID}:${DEVPLACE_GID}` (default `1000`), so it reads/writes the same `data/devplace.db`, `data/uploads/`, `data/devii_*.db`, `data/keys/` (VAPID), and `data/locks/devplace-services.lock` as `make dev`. No `DEVPLACE_DATABASE_URL` override - `config.py` resolves an absolute path under the project's `data/` dir. WAL + the `flock` on `devplace-services.lock` make concurrent dev/prod safe and keep a single background-services owner. SQLite is local-file, so prod and dev must be the **same host**.
 - **Code updates need no rebuild** (bind-mounted source); rebuild only on `pyproject.toml` dependency changes. `.env` is git-ignored; `.env.example` is the committed template. The app reads `DEVPLACE_DATABASE_URL`, never `DATABASE_URL`.
 - **nginx parity rules** (`nginx/nginx.conf.template`, rendered by `start.sh` via `envsubst` with an allow-list that preserves `$http_upgrade`): `/static/uploads/` must re-apply `nosniff` + a `Content-Disposition` via the `map $uri $upload_disposition` block (`inline` for safe image/video/audio extensions, `attachment` otherwise), mirroring `UploadStaticFiles.INLINE_MEDIA_EXTENSIONS` - an XSS control nginx would otherwise bypass, and the inline branch is what lets video play in production; `/devii/ws` needs the `map $http_upgrade $connection_upgrade` block and `Upgrade`/`Connection` headers or the Devii terminal cannot connect (every new WebSocket route needs its own nginx upgrade location - the catch-all `location /` strips upgrade headers); `client_max_body_size` comes from `NGINX_MAX_BODY_SIZE` (default `50m`) and must be `>= max_upload_size_mb` or uploads 413. nginx serves `devplacepy/static` via a read-only bind mount, so assets stay current without an image rebuild.
+- **Healthcheck start period** (`start_period: 120s` in `docker-compose.yml`, `--start-period=120s` in `Dockerfile`): full startup takes ~110s (DB init, services, uvicorn workers). The start period must stay above that. Bump both files if startup grows.

Dockerfile

@@ -31,8 +31,9 @@ RUN pip install --no-cache-dir ".[bots]" \
 EXPOSE 10500
 
 ENV DEVPLACE_WEB_WORKERS=2
+ENV DEVPLACE_TEMPLATE_AUTO_RELOAD=0
 
-HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=10s \
+HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=120s \
     CMD curl -f http://localhost:10500/ || exit 1
 
 CMD ["sh", "-c", "DEVPLACE_STATIC_VERSION=${DEVPLACE_STATIC_VERSION:-$(date +%s)} exec uvicorn devplacepy.main:app --host 0.0.0.0 --port 10500 --workers 2 --backlog 8192 --proxy-headers --forwarded-allow-ips '*'"]

README.md

@@ -72,7 +72,7 @@ devplacepy/
 | `/uploads` | File upload endpoints: `POST /uploads/upload` (multipart), `POST /uploads/upload-url` (from URL); served at `/static/uploads/` |
 | `/admin/trash` | Admin **Trash**: review, restore, and permanently purge soft-deleted content (posts, comments, gists, projects, news, project files, attachments) across the platform |
 | `/notifications` | Notification list, mark read, live unread counts (`/notifications/counts`) |
-| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image and YouTube embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages, so typing an inline `@ai <instruction>` in a message executes it and the resolved result appears live in the chat for both participants. An opened conversation loads its 500 most recent messages; older history is retained in the database. The `POST /messages/send` form remains as a no-JavaScript fallback |
+| `/messages` | Real-time direct messaging over WebSocket (`/messages/ws`): live bidirectional delivery, optimistic send, typing indicators, read receipts, and online/last-seen presence. Messages render through the shared content pipeline (emoji shortcodes, image, video and audio embeds, autolink, sanitization). AI content correction and the AI modifier apply to direct messages, so typing an inline `@ai <instruction>` in a message executes it and the resolved result appears live in the chat for both participants (the `message` WS frame carries an additive `ai_processed` flag when a pending correction/modification was applied). An opened conversation loads its 500 most recent messages; older history is retained in the database. `GET /messages/conversations` returns the conversation list as JSON for a live client refresh; the `POST /messages/send` form remains as a no-JavaScript fallback and now accepts an attachment-only, empty-content message. `POST /messages/ws-ticket` exchanges a session/API-key auth to a short-lived (30s), single-use ticket a browser WebSocket can carry as `?ticket=...` on the handshake, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header - this is what makes off-session embedding of the chat possible. CLI: `devplace messaging prune-tickets` removes expired tickets |
 | `/votes` | Upvote/downvote on posts, comments, projects |
 | `/reactions` | Emoji reactions on posts, comments, gists, projects |
 | `/bookmarks` | Save/unsave content; `/bookmarks/saved` personal list |
@@ -109,6 +109,8 @@ Member progression is driven by activity and peer recognition.
 
 - **AI modifier.** Enabled by default and applied synchronously by default. It works like AI content correction, except it runs **only** where the prose you author contains an inline `@ai <instruction>` directive: the configured prompt tells the model to execute that instruction and replace the marked part, removing the `@ai` marker. Text with no `@ai ...` directive is left exactly as written. It is **context-aware**: the model is given a grounding summary of who is asking (your username, role, level, stars, post count, rank, followers, and bio), the current date, and where the directive sits - the post a comment replies to, the conversation a direct message belongs to, the gist's language and code, and so on - so directives like `@ai answer the question above`, `@ai write my bio from my stats`, or `@ai reply to this` work. It uses your own API key for per-user attribution, is fail-soft (the original is kept on any error), and applies across the web UI, the REST and devRant APIs, and Devii, on the same prose fields as correction (posts, projects, gists, comments, direct messages, and your bio). Code and source files are never touched. In direct messages it runs live: typing `@ai <instruction>` in a message executes it and the resolved result appears in the chat for both participants without a reload. You can switch the apply mode to background or disable it on your profile or via the Devii `ai_modifier_set` tool; the settings are saved at `POST /profile/{username}/ai-modifier`. The default instruction is "Execute what is behind `@ai` (the prompt) and replace that part including `@ai`". Successful modifications accumulate per-user running totals - modifications, token counts, cost, and timing/performance (average latency, average speed in tokens per second, and total processing time) - shown on the profile page; token, call, and performance figures are visible to the member, while the dollar figures (total and average cost) are shown to administrators only.
 
+- **Devii interactive widgets.** Administrators set the site default on the Devii service (`devii_interactions_default`, default on). Guests always use that default. Signed-in members inherit it until they override it on their profile or via the Devii `interactions_set` tool (`POST /profile/{username}/interactions`; owner or admin). When enabled, Devii may present decisions with channel-aware controls (`ui_prompt`); when disabled, it falls back to plain numbered menus.
+
 Every AI gateway response (`/openai/v1/*`) also returns per-call `X-Gateway-*` headers with the full token breakdown and the dollar cost of that call, so any client can read its own usage.
 
 ## Code Farm
@@ -123,14 +125,21 @@ The **Code Farm** (`/game`) is a cooperative idle game in the spirit of Farmvill
 - **Daily bonus.** Claim a coin bonus once per day; consecutive days build a streak that grows the reward (capped at seven days).
 - **Daily quests.** Three quests rotate every day (plant, harvest, water, or earn goals), tracked automatically as you play; claim each one for coins and XP when complete.
 - **Perks.** Spend coins on four permanent upgrades - Optimizer (+harvest coins), Build Cache (+build speed), Bulk Licenses (-planting cost), and Mentorship (+harvest XP) - each levelling up with escalating cost.
-- **Refactor (prestige).** At level 10 you can refactor: the farm resets (coins, level, CI, extra plots, perks) in exchange for a permanent +25% coin bonus that stacks with every refactor. Each refactor also awards **Stars** (scaled by the level and prestige you reached) to spend in the Legacy shop.
-- **Stars and Legacy (endgame).** Stars buy permanent **Legacy** upgrades that survive every refactor, unlike perks: **CI Bot** (auto-collects ready builds when you open your farm), **Tech Debt Payoff** (+coins, stacks with prestige), **Bare-Metal** (+base build speed), **Monorepo** (+starting plots after each refactor), and **Branch Protection** (longer steal grace and a smaller steal cut). This is the infinite progression for maxed farms, and CI Bot makes the game playable hands-off.
+- **Refactor (prestige).** At level 10 you can refactor: the farm resets (coins, level, CI, extra plots, perks) in exchange for a permanent +25% coin bonus that stacks with every refactor. Each refactor also awards **Stars** (scaled by the level and prestige you reached) to spend in the Legacy shop. Refactoring costs a **dynamic coin fee** that grows with your prestige and current wealth, so you must farm well past level 10 to afford each one - and 10% of what remains after the fee (more with the Golden Parachute Legacy upgrade, up to 35%) carries over into the new run.
+- **Community treasury and weekly grant.** Every refactor fee flows into a shared treasury. Active low-balance, low-prestige farms (at least five harvests this week, under 10,000 coins, at most prestige 5) can claim a capped grant from it once per week - a direct wealth transfer from the farms refactoring at the top to the farms building at the bottom.
+- **Stars and Legacy (endgame).** Stars buy permanent **Legacy** upgrades that survive every refactor, unlike perks: **CI Bot** (auto-collects ready builds when you open your farm), **Tech Debt Payoff** (+coins, stacks with prestige), **Bare-Metal** (+base build speed), **Monorepo** (+starting plots after each refactor), **Branch Protection** (longer steal grace and a smaller steal cut), and **Golden Parachute** (a larger refactor coin carry-over). This is the infinite progression for maxed farms, and CI Bot makes the game playable hands-off.
 - **Golden builds.** A small share of plantings come out golden (marked with a sparkle); harvesting a golden build pays several times the coins.
 - **Visit and water friends.** Open another member's farm at `/game/farm/{username}` and water their growing builds to speed them up - you earn coins for helping, and the owner sees the help live. This is the social loop that makes the game cooperative.
-- **Steal a harvest.** A ready build on someone else's farm can be stolen once a protection window passes - the owner gets that grace period (longer if they invested in Branch Protection) to harvest it first. A successful steal pays the thief half the build's coin value (the owner loses the whole build) and earns the **Cat Burglar** badge; the victim gets the **Robbed** badge and a live notification that someone raided their farm (the thief is never named). You can raid any given neighbour only **once per hour**, so no one has to babysit their farm against constant theft. Stealing pays coins only, so the leaderboard stays earned by real farming. This is the competitive counterpart to watering.
-- **Leaderboard.** Top farmers are ranked by a composite achievement score that weighs every factor the game tracks - refactor (prestige) count, XP, lifetime harvests, current coins, CI tier, plots bought, perk levels, and login streak - so total accomplishment decides position rather than just the current post-refactor cycle. The score is shown alongside your own farm next to each player's level.
+- **Steal a harvest.** A ready build on someone else's farm can be stolen once a protection window passes - the owner gets that grace period (longer if they invested in Branch Protection or a Defense building) to harvest it first. A successful steal pays the thief a fraction of the build's coin value (the owner loses the whole build) and earns the **Cat Burglar** badge; the victim gets the **Robbed** badge and a live notification that someone raided their farm (the thief is never named). You can raid any given neighbour only **once per hour**. Raiding a farm with 10x your own coins grants a 24-hour **Underdog** boost (+25% coin gain) and the **David vs Goliath** badge. Stealing pays coins only, so the harvest-based leaderboards stay earned by real farming.
+- **Market Saturation.** The last 48 hours of league-wide harvests of each crop are tracked; when a crop is over-farmed its payout drops in steps (down to 40%), while the four starter crops get a relief buff (up to +15%) while the market is saturated - printing one crop nonstop is throttled, diversity is rewarded. The shop shows a live "Saturated" / "Boosted" label per crop.
+- **Infrastructure.** Permanent, expensive, prestige-gated buildings and coin sinks: **Private Registry** (faster Rust/Compiler/Kernel builds), **Canary Deployments** (a chance to double or only refund a harvest), and **Observability Suite** (raises the minimum you keep when raided).
+- **Defense.** An upgradeable building that reduces raid losses and adds steal grace - but costs an ongoing daily coin upkeep (proportional to your coin balance, so it scales with wealth); if unpaid, the tier decays automatically.
+- **Cosmetics.** Purely cosmetic titles and plot skins, bought with coins - zero gameplay effect, pure status. An equipped title shows next to your name on the leaderboard.
+- **Mastery (endgame beyond prestige).** From prestige 50 onward, every 10 more prestige earns a permanent Mastery point (spendable, and the milestone itself never re-locks). Mastery upgrades open new gameplay instead of bigger numbers: **Continuous Delivery** (auto-replant after harvest), **Farm Analytics** (lifetime stats on your HUD), and **Legacy Contracts** (a weekly long-term contract slot paying Stars and a temporary coin boost). Reaching Mastery also unlocks three new high-tier crop families (Distributed System, ML Pipeline, Security Fortress - the last one immune to raids).
+- **Leaderboards.** Several boards, selectable from the game page: **Overall score** (a composite weighing refactor/prestige count, XP, lifetime harvests, coins, CI tier, plots, perks, and streak), **Prestige**, **Harvests this week**, **Raid efficiency** (average coins per successful raid), **Fastest to Kernel** (time since your last refactor), **Fair play** (rewards recent activity over hoarding), and (when running) the current **Era** board.
+- **Eras (admin-managed seasons).** Administrators can start a Era at `/admin/game`: every farm's *visible* Era coins/harvests counters reset to zero, but real coin balances, prestige, Stars, Legacy, and Mastery are never touched. Ending an Era ranks farms by Era score (which gives prestige only partial weight, so veterans keep an edge without it being insurmountable), awards Stars to the top 10, and permanently records the results.
 
-The farm refreshes live over the pub/sub bus (a watered build appears on the owner's screen at once) and every plot countdown ticks client-side. Every endpoint also answers JSON, and Devii can play the game on the member's behalf via the `game_*` tools (`game_state`, `game_plant`, `game_harvest`, `game_buy_plot`, `game_upgrade_ci`, `game_water`, `game_steal`, `game_view_farm`, `game_leaderboard`, `game_upgrade_perk`, `game_upgrade_legacy`, `game_prestige`). See the API reference group **Code Farm**.
+The farm refreshes live over the pub/sub bus (a watered build appears on the owner's screen at once) and every plot countdown ticks client-side. Every endpoint also answers JSON, and Devii can play the game on the member's behalf via the `game_*` tools (`game_state`, `game_plant`, `game_harvest`, `game_buy_plot`, `game_upgrade_ci`, `game_water`, `game_steal`, `game_view_farm`, `game_leaderboard`, `game_upgrade_perk`, `game_upgrade_legacy`, `game_prestige`, `game_claim_grant`, `game_upgrade_mastery`, `game_buy_infrastructure`, `game_upgrade_defense`, `game_buy_cosmetic`, `game_equip_cosmetic`). See the API reference group **Code Farm**.
 
 ## Engagement
 

devplacepy/attachments.py

@@ -444,12 +444,13 @@ def link_attachments(uids, target_type, target_uid):
         return
     placeholders = ",".join(f":p{i}" for i in range(len(flat)))
     params = {f"p{i}": uid for i, uid in enumerate(flat)}
-    db.query(
-        f"UPDATE attachments SET target_type=:tt, target_uid=:tu WHERE uid IN ({placeholders})",
-        tt=target_type,
-        tu=target_uid,
-        **params,
-    )
+    with db:
+        db.query(
+            f"UPDATE attachments SET target_type=:tt, target_uid=:tu WHERE uid IN ({placeholders})",
+            tt=target_type,
+            tu=target_uid,
+            **params,
+        )
 
 
 def set_gitea_asset_id(uid, asset_id):
@@ -617,7 +618,8 @@ def delete_attachments_for(target_type, target_uids):
     for row in rows:
         _unlink_attachment_files(row)
     ids = ",".join(str(row["id"]) for row in rows)
-    db.query(f"DELETE FROM attachments WHERE id IN ({ids})")
+    with db:
+        db.query(f"DELETE FROM attachments WHERE id IN ({ids})")
 
 
 def get_attachments(target_type, target_uid):

devplacepy/awards/images.py

@@ -0,0 +1,33 @@
+# retoor <retoor@molodetz.nl>
+
+from io import BytesIO
+
+from PIL import Image
+
+
+def enforce_rgba_png(file_bytes: bytes) -> bytes:
+    img = Image.open(BytesIO(file_bytes)).convert("RGBA")
+    width, height = img.size
+    if width > 1 and height > 1:
+        corner = img.getpixel((0, 0))
+        if len(corner) == 4 and corner[3] == 255:
+            bg = corner[:3]
+            data = img.getdata()
+            cleaned = []
+            for pixel in data:
+                if pixel[:3] == bg:
+                    cleaned.append((pixel[0], pixel[1], pixel[2], 0))
+                else:
+                    cleaned.append(pixel)
+            img.putdata(cleaned)
+    buf = BytesIO()
+    img.save(buf, format="PNG")
+    return buf.getvalue()
+
+
+def resize_award_png(source: bytes, size: int) -> bytes:
+    img = Image.open(BytesIO(source)).convert("RGBA")
+    img = img.resize((size, size), Image.LANCZOS)
+    buf = BytesIO()
+    img.save(buf, format="PNG")
+    return buf.getvalue()

devplacepy/cli/devii.py

@@ -1,13 +1,11 @@
 # retoor <retoor@molodetz.nl>
 
 import sys
-from devplacepy.database import get_table
+from devplacepy.database import db, get_table
 from devplacepy.cli._shared import _audit_cli
 
 
 def cmd_devii_reset_quota(args):
-    from devplacepy.database import db
-
     table_name = "devii_usage_ledger"
     if table_name not in db.tables:
         print(f"Table '{table_name}' does not exist, nothing to reset")
@@ -48,6 +46,67 @@ def cmd_devii_reset_quota(args):
     print(f"Reset AI quota for '{args.username}' ({count} ledger rows deleted)")
 
 
+def _active_count() -> int:
+    if "devii_lessons" not in db.tables:
+        return 0
+    return db["devii_lessons"].count(deleted_at=None)
+
+
+def _soft_deleted_count() -> int:
+    if "devii_lessons" not in db.tables:
+        return 0
+    return db["devii_lessons"].count(deleted_at={"!=": None})
+
+
+def cmd_devii_lessons_count(args):
+    active = _active_count()
+    deleted = _soft_deleted_count()
+    print(f"Lessons: {active} active, {deleted} soft-deleted ({active + deleted} total)")
+
+
+def cmd_devii_lessons_clear(args):
+    from devplacepy.services.devii.agentic.lessons import TABLE
+
+    if TABLE not in db.tables:
+        print("No devii_lessons table exists")
+        return
+    active = _active_count()
+    deleted = _soft_deleted_count()
+    total = active + deleted
+    if not args.force:
+        print(f"Will delete {total} lesson(s) ({active} active, {deleted} soft-deleted). Pass --force to confirm.")
+        return
+    db[TABLE].delete()
+    _audit_cli("cli.devii.lessons.clear", "CLI cleared all devii_lessons", metadata={"active": active, "soft_deleted": deleted})
+    print(f"Deleted {total} lesson(s)")
+
+
+def cmd_devii_lessons_prune(args):
+    from devplacepy.services.devii.agentic.lessons import LessonStore, _read_retention_settings
+
+    if "devii_lessons" not in db.tables:
+        print("No devii_lessons table exists")
+        return
+    active_before = _active_count()
+    if args.all_owners:
+        _, max_age = _read_retention_settings(db)
+        store = LessonStore(db, "_global", "_global")
+        pruned = store.prune_all_owners(max_age)
+    elif args.username:
+        user = get_table("users").find_one(username=args.username)
+        if not user:
+            print(f"User '{args.username}' not found")
+            sys.exit(1)
+        _, max_age = _read_retention_settings(db)
+        store = LessonStore(db, "user", user["uid"])
+        pruned = store.prune(max_age)
+    else:
+        print("Provide --all-owners, or --username USER")
+        sys.exit(1)
+    _audit_cli("cli.devii.lessons.prune", "CLI pruned devii_lessons", metadata={"pruned": pruned, "active_before": active_before})
+    print(f"Pruned {pruned} lesson(s) (active before: {active_before}, now: {_active_count()})")
+
+
 def register_devii(subparsers):
     devii = subparsers.add_parser("devii", help="Devii assistant management")
     devii_sub = devii.add_subparsers(title="action", dest="action")
@@ -64,3 +123,18 @@ def register_devii(subparsers):
         "--all", action="store_true", help="Reset every quota (users and guests)"
     )
     devii_reset.set_defaults(func=cmd_devii_reset_quota)
+
+    devii_lessons = devii_sub.add_parser("lessons", help="Manage persisted Devii lesson data")
+    lessons_sub = devii_lessons.add_subparsers(title="sub-action", dest="sub_action")
+
+    lessons_count = lessons_sub.add_parser("count", help="Count active and soft-deleted lessons")
+    lessons_count.set_defaults(func=cmd_devii_lessons_count)
+
+    lessons_prune = lessons_sub.add_parser("prune", help="Soft-delete lessons older than the configured max age")
+    lessons_prune.add_argument("--all-owners", action="store_true", help="Prune across every owner")
+    lessons_prune.add_argument("--username", help="Prune for a specific user")
+    lessons_prune.set_defaults(func=cmd_devii_lessons_prune)
+
+    lessons_clear = lessons_sub.add_parser("clear", help="Hard-delete every devii_lessons row")
+    lessons_clear.add_argument("--force", action="store_true", help="Required to confirm hard deletion")
+    lessons_clear.set_defaults(func=cmd_devii_lessons_clear)

devplacepy/cli/game.py

@@ -0,0 +1,84 @@
+# retoor <retoor@molodetz.nl>
+
+from devplacepy.cli._shared import _audit_cli
+
+
+def cmd_game_market_prune(args):
+    from devplacepy.services.game import store
+
+    removed = store.prune_ticks()
+    _audit_cli(
+        "cli.game.market.prune",
+        f"CLI pruned {removed} stale Code Farm market tick(s)",
+        metadata={"count": removed},
+    )
+    print(f"Pruned {removed} stale market tick bucket(s)")
+
+
+def cmd_game_era_status(args):
+    from devplacepy.services.game import store
+
+    era = store.active_era()
+    if not era:
+        print("No Era is currently running.")
+        return
+    print(f"Era {era['era_number']}: {era['name']}")
+    print(f"Started: {era['started_at']}")
+    print(f"Scheduled end: {era['ends_at']}")
+
+
+def cmd_game_era_start(args):
+    from devplacepy.services.game import GameError, store
+
+    try:
+        era = store.start_era(args.name, args.duration_days)
+    except GameError as exc:
+        print(f"Error: {exc}")
+        return
+    _audit_cli(
+        "cli.game.era.start",
+        f"CLI started Code Farm Era {era['era_number']}: {era['name']}",
+        metadata={"era_number": era["era_number"], "name": era["name"]},
+    )
+    print(f"Started Era {era['era_number']}: {era['name']}")
+
+
+def cmd_game_era_end(args):
+    from devplacepy.services.game import GameError, store
+
+    try:
+        result = store.end_era()
+    except GameError as exc:
+        print(f"Error: {exc}")
+        return
+    _audit_cli(
+        "cli.game.era.end",
+        f"CLI ended Code Farm Era {result['era_number']}",
+        metadata=result,
+    )
+    print(f"Ended Era {result['era_number']} ({result['participants']} participant(s) ranked)")
+
+
+def register_game(subparsers):
+    game = subparsers.add_parser("game", help="Code Farm management")
+    game_sub = game.add_subparsers(title="action", dest="action")
+
+    market = game_sub.add_parser("market", help="Code Farm market saturation data")
+    market_sub = market.add_subparsers(title="market_action", dest="market_action")
+    market_prune = market_sub.add_parser(
+        "prune", help="Delete market tick buckets older than the tracking window"
+    )
+    market_prune.set_defaults(func=cmd_game_market_prune)
+
+    era = game_sub.add_parser("era", help="Code Farm Era management")
+    era_sub = era.add_subparsers(title="era_action", dest="era_action")
+    era_status = era_sub.add_parser("status", help="Show the current Era status")
+    era_status.set_defaults(func=cmd_game_era_status)
+    era_start = era_sub.add_parser("start", help="Start a new Era")
+    era_start.add_argument("name", help="Era name")
+    era_start.add_argument(
+        "--days", dest="duration_days", type=int, default=28, help="Planned Era length in days"
+    )
+    era_start.set_defaults(func=cmd_game_era_start)
+    era_end = era_sub.add_parser("end", help="End the currently running Era")
+    era_end.set_defaults(func=cmd_game_era_end)

devplacepy/cli/gateway.py

@@ -0,0 +1,103 @@
+# retoor <retoor@molodetz.nl>
+
+import sys
+
+from devplacepy.cli._shared import _audit_cli
+
+
+def cmd_gateway_quota_list(args):
+    from devplacepy.services.openai_gateway import quota
+
+    rules = quota.quota_rule_store.list()
+    if not rules:
+        print("No quota rules. Every caller is capped by the global defaults on /admin/services/openai.")
+        return
+    for rule in rules:
+        spent = quota.spent_24h(rule["owner_kind"], rule["owner_id"], rule["app_reference"])
+        scope = ", ".join(
+            f"{key}={rule[key]}" for key in ("owner_kind", "owner_id", "app_reference") if rule[key]
+        ) or "(no dimensions - invalid)"
+        limit = "unlimited" if rule["limit_usd"] == 0 else f"${rule['limit_usd']:.2f}/24h"
+        active = "active" if rule["is_active"] else "inactive"
+        label = f" - {rule['label']}" if rule["label"] else ""
+        print(f"{rule['uid']}  [{scope}]  {limit}  spent=${spent:.4f}  {active}{label}")
+
+
+def cmd_gateway_quota_set(args):
+    from pydantic import ValidationError
+    from devplacepy.services.openai_gateway import quota
+
+    try:
+        payload = quota.QuotaRuleIn(
+            owner_kind=args.owner_kind,
+            owner_id=args.owner_id,
+            app_reference=args.app_reference,
+            limit_usd=args.limit_usd,
+            is_active=not args.inactive,
+            label=args.label or "",
+        )
+    except ValidationError as exc:
+        print(f"Invalid rule: {exc.errors()[0].get('msg', exc)}")
+        sys.exit(1)
+    saved = quota.quota_rule_store.set(payload, uid=args.uid, created_by="cli")
+    _audit_cli(
+        "gateway.quota_rule.update",
+        f"CLI saved gateway quota rule {saved['uid']}",
+        metadata={
+            "owner_kind": saved["owner_kind"],
+            "owner_id": saved["owner_id"],
+            "app_reference": saved["app_reference"],
+            "limit_usd": saved["limit_usd"],
+        },
+        target_type="gateway_quota_rule",
+        target_uid=saved["uid"],
+    )
+    print(f"Saved quota rule {saved['uid']}")
+
+
+def cmd_gateway_quota_delete(args):
+    from devplacepy.services.openai_gateway import quota
+
+    if not quota.quota_rule_store.remove(args.uid):
+        print(f"Quota rule '{args.uid}' not found")
+        sys.exit(1)
+    _audit_cli(
+        "gateway.quota_rule.delete",
+        f"CLI deleted gateway quota rule {args.uid}",
+        target_type="gateway_quota_rule",
+        target_uid=args.uid,
+    )
+    print(f"Deleted quota rule {args.uid}")
+
+
+def register_gateway(subparsers):
+    gateway = subparsers.add_parser("gateway", help="AI gateway management")
+    gateway_sub = gateway.add_subparsers(title="action", dest="action")
+
+    quota = gateway_sub.add_parser("quota", help="Manage rolling-24h AI gateway quota rules")
+    quota_sub = quota.add_subparsers(title="sub-action", dest="sub_action")
+
+    quota_list = quota_sub.add_parser("list", help="List all quota rules and their current 24h spend")
+    quota_list.set_defaults(func=cmd_gateway_quota_list)
+
+    quota_set = quota_sub.add_parser(
+        "set", help="Create or update a quota rule (scope by role/user/app, any combination)"
+    )
+    quota_set.add_argument("--uid", help="Existing rule uid to update; omit to create a new rule")
+    quota_set.add_argument(
+        "--owner-kind",
+        choices=("internal", "key", "user", "admin", "anonymous"),
+        help="Role to scope by. Omit for any role",
+    )
+    quota_set.add_argument("--owner-id", help="Specific user uid to scope by. Omit for any caller")
+    quota_set.add_argument("--app-reference", help="App label to scope by. Omit for any app")
+    quota_set.add_argument(
+        "--limit-usd", type=float, required=True, help="Rolling 24h USD cap (0 = unlimited)"
+    )
+    quota_set.add_argument("--label", help="Optional admin-facing note")
+    quota_set.add_argument("--inactive", action="store_true", help="Create the rule disabled")
+    quota_set.set_defaults(func=cmd_gateway_quota_set)
+
+    quota_delete = quota_sub.add_parser("delete", help="Delete a quota rule by uid")
+    quota_delete.add_argument("uid", help="Quota rule uid")
+    quota_delete.set_defaults(func=cmd_gateway_quota_delete)

devplacepy/cli/jobs.py

@@ -7,12 +7,12 @@ from devplacepy.cli._shared import _audit_cli
 def _remove_zip_artifacts(job):
     import shutil
     from pathlib import Path
-    from devplacepy.services.jobs.zip_service import STAGING_DIR
+    from devplacepy.config import ZIP_STAGING_DIR
 
     local_path = (job.get("result") or {}).get("local_path")
     if local_path:
         Path(local_path).unlink(missing_ok=True)
-    shutil.rmtree(STAGING_DIR / job["uid"], ignore_errors=True)
+    shutil.rmtree(ZIP_STAGING_DIR / job["uid"], ignore_errors=True)
 
 
 def cmd_zips_prune(args):
@@ -251,7 +251,6 @@ def cmd_isslop_clear(args):
 
 def cmd_isslop_analyze(args):
     import asyncio
-    import json
 
     from devplacepy.config import ISSLOP_WORKSPACES_DIR, ensure_data_dirs
     from devplacepy.database import INTERNAL_GATEWAY_URL, internal_gateway_key

devplacepy/cli/main.py

@@ -12,6 +12,9 @@ from devplacepy.cli.jobs import register_jobs
 from devplacepy.cli.backups import register_backups
 from devplacepy.cli.containers import register_containers
 from devplacepy.cli.migrate import register_migrate
+from devplacepy.cli.game import register_game
+from devplacepy.cli.gateway import register_gateway
+from devplacepy.cli.messaging import register_messaging
 
 
 def build_parser():
@@ -28,6 +31,9 @@ def build_parser():
     register_backups(sub)
     register_containers(sub)
     register_migrate(sub)
+    register_game(sub)
+    register_gateway(sub)
+    register_messaging(sub)
 
     return parser
 

devplacepy/cli/messaging.py

@@ -0,0 +1,29 @@
+# retoor <retoor@molodetz.nl>
+
+from devplacepy.cli._shared import _audit_cli
+
+
+def cmd_messaging_prune_tickets(args):
+    from datetime import datetime, timezone
+    from devplacepy.database import get_table
+
+    now = datetime.now(timezone.utc).isoformat()
+    tickets = get_table("ws_tickets")
+    expired = list(tickets.find(expires_at={"<": now}))
+    for ticket in expired:
+        tickets.delete(uid=ticket["uid"])
+    _audit_cli(
+        "cli.messaging.prune_tickets",
+        f"CLI pruned {len(expired)} expired WS tickets",
+        metadata={"count": len(expired)},
+    )
+    print(f"Pruned {len(expired)} expired WS ticket(s)")
+
+
+def register_messaging(subparsers):
+    messaging = subparsers.add_parser("messaging", help="Messaging WS ticket management")
+    messaging_sub = messaging.add_subparsers(title="action", dest="action")
+    messaging_prune_tickets = messaging_sub.add_parser(
+        "prune-tickets", help="Delete expired WebSocket auth tickets"
+    )
+    messaging_prune_tickets.set_defaults(func=cmd_messaging_prune_tickets)

devplacepy/config.py

@@ -68,6 +68,21 @@ INTERNAL_GATEWAY_URL = f"{INTERNAL_BASE_URL}/openai/v1/chat/completions"
 INTERNAL_EMBED_URL = f"{INTERNAL_BASE_URL}/openai/v1/embeddings"
 INTERNAL_MODEL = "molodetz"
 INTERNAL_EMBED_MODEL = "molodetz~embed"
+INTERNAL_IMAGE_MODEL = "molodetz-img-small"
+
+AWARD_GIVE_COOLDOWN_HOURS_DEFAULT = 24
+AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT = 24
+AWARD_DISPLAY_HOURS_DEFAULT = 24
+AWARD_DESCRIPTION_MAX = 125
+AWARD_IMAGE_MODEL_DEFAULT = "molodetz-img-small"
+AWARD_IMAGE_SIZE_DEFAULT = "512x512"
+AWARD_GENERATION_TIMEOUT_SECONDS = 120.0
+AWARD_IMAGE_PROMPT_DEFAULT = (
+    "Generate a single decorative developer award emblem/badge as a PNG with a fully "
+    "transparent background (alpha channel). No rectangular backdrop, no drop shadow "
+    "plate, no text labels rendered in the image. Center one stylized trophy/medal "
+    "icon that visually matches this message:"
+)
 DEFAULT_CORRECTION_PROMPT = "Leave literary as is, only do punctuation and casing"
 DEFAULT_MODIFIER_PROMPT = (
     "Execute what is behind `@ai` (the prompt) and replace that part including `@ai`"

devplacepy/content.py

@@ -13,6 +13,7 @@ from devplacepy.database import (
     resolve_by_slug,
     get_users_by_uids,
     get_vote_counts,
+    STAR_TARGETS,
     get_user_votes,
     get_reactions_by_targets,
     get_user_bookmarks,
@@ -20,6 +21,7 @@ from devplacepy.database import (
     get_poll_for_post,
     update_target_stars,
     clear_user_stars,
+    clear_user_post_count,
     get_target_owner_uid,
     resolve_object_url,
     soft_delete,
@@ -162,6 +164,8 @@ def create_content_item(
             **fields,
         }
     )
+    if table_name == "posts":
+        clear_user_post_count(user["uid"])
     if table_name == "projects":
         from devplacepy.templating import clear_user_projects_cache
 
@@ -356,6 +360,37 @@ def create_comment_record(
                     comment_url,
                 )
 
+    if target_type == "post":
+        posts_table = get_table("posts")
+        post_record = posts_table.find_one(uid=target_uid)
+        if not post_record:
+            post_record = posts_table.find_one(slug=target_uid)
+
+        commenter_uids = set()
+        for c in get_table("comments").find(
+            target_type="post", target_uid=target_uid, deleted_at=None
+        ):
+            commenter_uids.add(c["user_uid"])
+
+        commenter_uids.discard(user["uid"])
+        if post_record:
+            commenter_uids.discard(post_record["user_uid"])
+        if parent_uid:
+            parent_comment = get_table("comments").find_one(
+                uid=parent_uid, deleted_at=None
+            )
+            if parent_comment and parent_comment["user_uid"] != user["uid"]:
+                commenter_uids.discard(parent_comment["user_uid"])
+
+        for commenter_uid in commenter_uids:
+            create_notification(
+                commenter_uid,
+                "thread_comment",
+                f"{user['username']} also commented on a post you commented on",
+                user["uid"],
+                comment_url,
+            )
+
     create_mention_notifications(content, user["uid"], comment_url)
     schedule_correction(user, "comments", comment_uid, request)
     schedule_modification(user, "comments", comment_uid, request)
@@ -615,6 +650,8 @@ def delete_content_item(
     soft_delete_engagement(target_type, [item["uid"]], actor)
     if comment_uids:
         soft_delete_engagement("comment", comment_uids, actor)
+    if target_type == "post":
+        clear_user_post_count(item["user_uid"])
     if target_type == "project":
         from devplacepy.project_files import soft_delete_all_project_files
         from devplacepy.templating import clear_user_projects_cache
@@ -647,7 +684,11 @@ def load_detail(
     if user and item["user_uid"] in get_blocked_uids(user["uid"]):
         return None
     author = get_users_by_uids([item["user_uid"]]).get(item["user_uid"])
-    ups, downs = get_vote_counts([item["uid"]])
+    if target_type in STAR_TARGETS:
+        star_count = item.get("stars") or 0
+    else:
+        ups, downs = get_vote_counts([item["uid"]])
+        star_count = ups.get(item["uid"], 0) - downs.get(item["uid"], 0)
     reactions = (
         get_reactions_by_targets(target_type, [item["uid"]], user).get(
             item["uid"], {"counts": {}, "mine": []}
@@ -664,7 +705,7 @@ def load_detail(
         "item": item,
         "author": author,
         "is_owner": bool(user and user["uid"] == item["user_uid"]),
-        "star_count": ups.get(item["uid"], 0) - downs.get(item["uid"], 0),
+        "star_count": star_count,
         "my_vote": get_user_votes(user["uid"], [item["uid"]]).get(item["uid"], 0)
         if user
         else 0,

devplacepy/curl_transport.py

@@ -73,10 +73,17 @@ class CurlResponseStream(httpx.AsyncByteStream):
 
 
 class CurlTransport(httpx.AsyncBaseTransport):
-    def __init__(self, *, impersonate: str = IMPERSONATE_TARGET, verify: bool = True) -> None:
+    def __init__(
+        self,
+        *,
+        impersonate: str = IMPERSONATE_TARGET,
+        verify: bool = True,
+        proxy: str | None = None,
+    ) -> None:
         self._session = AsyncSession()
         self._impersonate = impersonate
         self._verify = verify
+        self._proxy = proxy
 
     async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
         headers = {
@@ -97,6 +104,7 @@ class CurlTransport(httpx.AsyncBaseTransport):
                 data=body or None,
                 impersonate=self._impersonate,
                 verify=self._verify,
+                proxy=self._proxy,
                 stream=True,
                 allow_redirects=False,
                 timeout=resolve_timeout(request),

devplacepy/customization.py

@@ -36,6 +36,18 @@ def owner_for(request: Request) -> tuple[str, str] | None:
 
 
 def _overrides_for(request: Request) -> dict:
+    cached = getattr(request.state, "_custom_overrides", None)
+    if cached is not None:
+        return cached
+    overrides = _resolve_overrides(request)
+    try:
+        request.state._custom_overrides = overrides
+    except Exception:
+        pass
+    return overrides
+
+
+def _resolve_overrides(request: Request) -> dict:
     if get_setting("customization_enabled", "1") != "1":
         return {"css": "", "js": ""}
     owner = owner_for(request)

devplacepy/database/CLAUDE.md

@@ -99,7 +99,7 @@ if "comments" not in db.tables:
 
 - **NEVER index the bare `deleted_at` column - use a PARTIAL trash index `WHERE deleted_at IS NOT NULL`.** `ensure_soft_delete_columns` creates `idx_<table>_trash ON (deleted_at) WHERE deleted_at IS NOT NULL` (and drops any legacy full `idx_<table>_deleted`). A full `deleted_at` index is a planner hazard: the column is one giant `NULL` bucket plus many unique delete-timestamps, so `sqlite_stat1` mis-estimates `deleted_at IS NULL` as returning ~2 rows and the planner picks that index for live reads, then `USE TEMP B-TREE FOR ORDER BY` to sort the whole live set (the global feed was doing exactly this, with 82% of posts soft-deleted). The partial index serves the admin Trash view (`deleted_at IS NOT NULL`) cheaply and stops poisoning live `IS NULL` queries.
 
-- **For "live, newest-first" listings add a composite or live-partial index that includes the sort column.** A `WHERE deleted_at IS NULL ORDER BY created_at` query needs the ordering in the index or it filesorts. Posts use a partial `idx_posts_live_created ON (created_at) WHERE deleted_at IS NULL` (feed) plus `idx_posts_user_created (user_uid, created_at)` (profile); comments use `idx_comments_target_created (target_type, target_uid, created_at)`; votes use `idx_votes_user_target (user_uid, target_uid)` (the per-user "my_vote" check on every card); notifications/gists/projects use `(user_uid, created_at)`. All were verified to drop the `USE TEMP B-TREE FOR ORDER BY` step.
+- **For "live, newest-first" listings add a composite or live-partial index that includes the sort column.** A `WHERE deleted_at IS NULL ORDER BY created_at` query needs the ordering in the index or it filesorts. Posts use a partial `idx_posts_live_created ON (created_at) WHERE deleted_at IS NULL` (feed) plus `idx_posts_user_created (user_uid, created_at)` (profile); comments use `idx_comments_target_created (target_type, target_uid, created_at)`; votes use `idx_votes_user_target (user_uid, target_uid)` (the per-user "my_vote" check on every card); notifications/gists/projects use `(user_uid, created_at)`; follows use `idx_follows_follower_created (follower_uid, created_at)` + `idx_follows_following_created (following_uid, created_at)` (the followers/following tabs sort newest-first; the legacy single-column follower/following indexes were dropped as redundant prefixes). All were verified to drop the `USE TEMP B-TREE FOR ORDER BY` step.
 
 - **Index the non-`uid` lookup keys too, not just the sort/owner columns.** A demand-vs-supply audit added the last missing single-key lookups: the `resolve_by_slug` hot path filters `slug` on content detail pages, so posts/gists/news/projects each get `idx_<table>_slug (slug)`; `get_setting`/`set_setting` filter `key`, so `idx_site_settings_key (key)`; the container store's `find_one(slug=)`/`find_one(name=)` fallbacks get `idx_instances_slug`/`idx_instances_name`. The DM thread load `find(sender_uid=, receiver_uid=)` gets the covering composites `idx_messages_conversation (sender_uid, receiver_uid)` + `idx_messages_conversation_rev (receiver_uid, sender_uid)` (the read-flag `UPDATE` uses the reverse); the badge-has check gets `idx_badges_user_name (user_uid, badge_name)`; the admin user list `ORDER BY -created_at` gets `idx_users_created_at (created_at)` (the existing `(role, created_at)` cannot serve a full-table created_at sort). All are non-unique so `_index` always creates them even if legacy duplicate data exists. Column sets already resolved to ~1 row by an existing prefix index (votes `+target_type`, game_quests `+kind`, poll_options `position`) are intentionally left uncovered - a trailing column there only adds write cost.
 
@@ -180,7 +180,7 @@ Site settings are seeded on startup (`site_settings` table):
 | `customization_js_enabled` | `"1"` | When `"0"`, custom CSS still serves but custom JS is suppressed |
 | `extra_head` | `""` | Raw HTML emitted verbatim into every page `<head>` by `templating.extra_head_tag()`; site-wide trusted-admin input, not sanitized |
 
-Besides the site/news/upload keys above, the **Operational** group is admin-editable at `/admin/settings`: `site_url` (public origin for absolute links incl. container ingress; resolved by `seo.public_base_url()` = setting -> `DEVPLACE_SITE_URL` env -> request origin), `rate_limit_per_minute`, `rate_limit_window_seconds`, `news_service_interval`, `session_max_age_days`, `session_remember_days`, `registration_open`, `maintenance_mode`, `maintenance_message`, `docs_search_mode` (`agent`|`bm25`, default `agent` - picks the `/docs/search.html` surface: the in-page Devii chat or the classic BM25 list; a viewer over their daily AI limit, or a guest when Devii is disabled, auto-falls-back to BM25 via `routers/docs/views.py` `_agent_search_state`). The **Custom Code** key `extra_head` is the sole key in the settings handler's `CLEARABLE_SETTINGS` set, so saving an empty textarea removes it (the default loop skips empty values).
+Besides the site/news/upload keys above, the **Operational** group is admin-editable at `/admin/settings`: `site_url` (public origin for absolute links incl. container ingress; resolved by `seo.public_base_url()` = setting -> `DEVPLACE_SITE_URL` env -> request origin), `rate_limit_per_minute`, `rate_limit_window_seconds`, `news_service_interval`, `session_max_age_days`, `session_remember_days`, `registration_open`, `maintenance_mode`, `maintenance_message`, `docs_search_mode` (`agent`|`bm25`, default `agent` - picks the `/docs/search.html` surface: the in-page Devii chat or the classic BM25 list; a viewer over their daily AI limit, or a guest when Devii is disabled, auto-falls-back to BM25 via `routers/docs/views.py` `_agent_search_state`), `outbound_proxy_url` (empty by default - when set, every `stealth.stealth_async_client`/`stealth_sync_client` call across the whole app routes through it via `stealth.configured_proxy_url()`; validated as `http(s)://`/`socks5(h)://` with a host in `AdminSettingsForm`; falls back to `DEVPLACE_OUTBOUND_PROXY_URL` when unset - see the "Outbound HTTP" note in the root `CLAUDE.md`). The **Custom Code** key `extra_head` is the sole key in the settings handler's `CLEARABLE_SETTINGS` set, so saving an empty textarea removes it (the default loop skips empty values).
 
 The seed block in `database.py` is guarded by `if "site_settings" in tables:` - on a brand-new DB the table does not exist yet (dataset creates tables lazily on first insert), so none of these rows are written until the table exists. Correct runtime behavior therefore relies on every consumer passing the production default to `get_setting`/`get_int_setting`, not on the seed.
 

devplacepy/database/__init__.py

@@ -5,10 +5,27 @@ from .core import refresh_snapshot, _local_cache_versions, _cache_version_cache,
 from .settings import _settings_cache, get_setting, get_int_setting, set_setting, clear_settings_cache, internal_gateway_key
 from .users import get_users_by_uids, _admins_cache, invalidate_admins_cache, get_admin_uids, set_user_timezone, set_last_seen, get_online_users, get_primary_admin_uid, search_users_by_username
 from .relations import _relations_cache, get_user_relations, get_blocked_uids, get_muted_uids, get_silenced_uids, invalidate_user_relations
-from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, build_pagination
+from .pagination import PAGE_SIZE, paginate, interleave_by_author, paginate_diverse, get_user_post_count, clear_user_post_count, build_pagination
 from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns, soft_delete, soft_delete_in, restore, purge, list_deleted, count_deleted, restore_event, purge_event
 from .engagement import _comment_count_cache, get_comment_counts_by_post_uids, get_post_counts_by_user_uids, get_vote_counts, get_user_votes, get_reactions_by_targets, get_user_bookmarks, get_polls_by_post_uids, get_poll_for_post
-from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage
+from .usage import _add_usage, _get_usage, add_correction_usage, get_correction_usage, add_modifier_usage, get_modifier_usage, NEWS_USAGE_KEY, add_news_usage, get_news_usage, ISSUE_USAGE_KEY, add_issue_usage, get_issue_usage, SEO_USAGE_KEY, add_seo_usage, get_seo_usage, AWARD_USAGE_KEY, add_award_usage, get_award_usage
+from .awards import (
+    AWARDS_PER_PAGE,
+    award_display_hours,
+    award_give_cooldown_hours,
+    award_receive_cooldown_hours,
+    award_is_prominent,
+    can_give_award,
+    can_receive_award,
+    count_published_awards,
+    enrich_award,
+    get_prominent_award,
+    get_user_awards,
+    has_giver_cooldown,
+    has_receiver_cooldown,
+    recompute_user_award_stats,
+    revoke_award,
+)
 from .seo_meta import SEO_META_TYPES, get_seo_metadata, get_seo_metadata_batch, has_fresh_seo_metadata, upsert_seo_metadata, mark_seo_metadata_stale
 from .activity import record_activity, record_unique_activity, get_user_activity, _activity_cache, _ACTIVITY_TABLES, get_activity_calendar, _activity_level, get_first_activity_date, HEATMAP_WEEKS, get_activity_heatmap, get_activity_months, get_streaks
 from .customization import CUSTOMIZATION_GLOBAL_SCOPE, CUSTOMIZATION_LANGS, _customizations_cache, _customization_key, CUSTOMIZATION_PREF_COLUMNS, get_customization_prefs, set_customization_pref, get_custom_overrides, get_custom_override, list_custom_overrides, set_custom_override, delete_custom_override
@@ -19,7 +36,7 @@ from .follows import get_follow_counts, get_follow_list, get_following_among
 from .deepsearch import _ds_now, create_deepsearch_session, update_deepsearch_session, get_deepsearch_session, add_deepsearch_message, get_deepsearch_messages, get_cached_deepsearch_url, upsert_deepsearch_url_cache
 from .ranking import VOTABLE_TARGETS, STAR_TARGETS, _authors_cache, _ranked_authors, _rank_map, get_top_authors, get_leaderboard, get_user_rank, get_user_stars, clear_user_stars, update_target_stars, soft_delete_engagement, delete_engagement, get_target_owner_uid
 from .comments import _drop_blocked, _build_comment_items, load_comments, get_recent_comments_by_target_uids, get_recent_comments_by_post_uids, load_comments_by_target_uids
-from .content import resolve_by_slug, resolve_object_url, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news
+from .content import resolve_by_slug, resolve_object_url, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news, get_trending_topics
 from .attachments_data import get_attachments, get_attachments_by_type, get_news_images_by_uids, delete_attachment_record, delete_attachments, _delete_attachment_file, get_user_media, get_deleted_media
 from .stats import _stats_cache, get_site_stats, _analytics_cache, get_platform_analytics, _gist_languages_cache, get_gist_languages
 from .schema import BUG_TABLE_RENAMES, migrate_bug_tables_to_issue_tables, init_db, _refresh_query_planner_stats, OLD_GATEWAY_URL, migrate_ai_gateway_settings, backfill_api_keys, _backfill_gamification
@@ -81,6 +98,7 @@ __all__ = [
     "interleave_by_author",
     "paginate_diverse",
     "get_user_post_count",
+    "clear_user_post_count",
     "build_pagination",
     "SOFT_DELETE_TABLES",
     "ensure_soft_delete_columns",
@@ -208,6 +226,7 @@ __all__ = [
     "text_search_clause",
     "get_daily_topic",
     "get_featured_news",
+    "get_trending_topics",
     "get_attachments",
     "get_attachments_by_type",
     "get_news_images_by_uids",

devplacepy/database/awards.py

@@ -0,0 +1,206 @@
+# retoor <retoor@molodetz.nl>
+
+from datetime import datetime, timedelta, timezone
+
+from devplacepy.cache import TTLCache
+from devplacepy.config import (
+    AWARD_DISPLAY_HOURS_DEFAULT,
+    AWARD_GIVE_COOLDOWN_HOURS_DEFAULT,
+    AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT,
+)
+from .core import db
+from .pagination import build_pagination
+from .settings import get_int_setting
+from .core import get_table, _now_iso
+from .users import get_users_by_uids
+from .content import resolve_by_slug
+from .soft_delete import soft_delete, soft_delete_in
+
+AWARDS_PER_PAGE = 12
+
+
+def _awards_table():
+    return get_table("awards")
+
+
+def award_give_cooldown_hours() -> int:
+    return max(1, get_int_setting("award_give_cooldown_hours", AWARD_GIVE_COOLDOWN_HOURS_DEFAULT))
+
+
+def award_receive_cooldown_hours() -> int:
+    return max(
+        1, get_int_setting("award_receive_cooldown_hours", AWARD_RECEIVE_COOLDOWN_HOURS_DEFAULT)
+    )
+
+
+def award_display_hours() -> int:
+    return max(1, get_int_setting("award_display_hours", AWARD_DISPLAY_HOURS_DEFAULT))
+
+
+def _cooldown_cutoff(hours: int) -> str:
+    return (datetime.now(timezone.utc) - timedelta(hours=hours)).isoformat()
+
+
+def has_giver_cooldown(giver_uid: str) -> bool:
+    if not giver_uid or "awards" not in db.tables:
+        return False
+    cutoff = _cooldown_cutoff(award_give_cooldown_hours())
+    row = _awards_table().find_one(
+        giver_uid=giver_uid, deleted_at=None, created_at={">=": cutoff}
+    )
+    return row is not None
+
+
+def has_receiver_cooldown(receiver_uid: str) -> bool:
+    if not receiver_uid or "awards" not in db.tables:
+        return False
+    cutoff = _cooldown_cutoff(award_receive_cooldown_hours())
+    row = _awards_table().find_one(
+        receiver_uid=receiver_uid, deleted_at=None, created_at={">=": cutoff}
+    )
+    return row is not None
+
+
+def can_receive_award(receiver_uid: str) -> bool:
+    return not has_receiver_cooldown(receiver_uid)
+
+
+def can_give_award(giver_uid: str, receiver_uid: str) -> bool:
+    if not giver_uid or not receiver_uid or giver_uid == receiver_uid:
+        return False
+    return not has_giver_cooldown(giver_uid) and not has_receiver_cooldown(receiver_uid)
+
+
+def _published_filter():
+    return {"deleted_at": None, "generated_at": {">": ""}}
+
+
+def count_published_awards(receiver_uid: str) -> int:
+    if not receiver_uid or "awards" not in db.tables:
+        return 0
+    return _awards_table().count(receiver_uid=receiver_uid, **_published_filter())
+
+
+def _latest_published(receiver_uid: str):
+    if not receiver_uid or "awards" not in db.tables:
+        return None
+    rows = list(
+        _awards_table().find(
+            receiver_uid=receiver_uid,
+            deleted_at=None,
+            generated_at={">": ""},
+            order_by=["-generated_at"],
+            _limit=1,
+        )
+    )
+    return rows[0] if rows else None
+
+
+def recompute_user_award_stats(receiver_uid: str) -> None:
+    if not receiver_uid or "users" not in db.tables:
+        return
+    count = count_published_awards(receiver_uid)
+    latest = _latest_published(receiver_uid)
+    users = get_table("users")
+    payload = {
+        "uid": receiver_uid,
+        "award_count": count,
+        "last_award_at": latest.get("generated_at") if latest else None,
+        "last_award_slug": latest.get("slug") if latest else None,
+        "last_award_uid": latest.get("uid") if latest else None,
+    }
+    users.update(payload, ["uid"])
+
+
+_prominence_cache = TTLCache(ttl=15, max_size=500)
+
+
+def award_is_prominent(user: dict | None) -> bool:
+    if not user or not user.get("last_award_at") or not user.get("last_award_uid"):
+        return False
+    cached = _prominence_cache.get(user["last_award_uid"])
+    if cached is not None:
+        return cached
+    prominent = _compute_prominence(user["last_award_uid"])
+    _prominence_cache.set(user["last_award_uid"], prominent)
+    return prominent
+
+
+def _compute_prominence(award_uid: str) -> bool:
+    award = resolve_by_slug(_awards_table(), award_uid)
+    if not award or not award.get("generated_at"):
+        return False
+    try:
+        published = datetime.fromisoformat(award["generated_at"])
+        if published.tzinfo is None:
+            published = published.replace(tzinfo=timezone.utc)
+    except (ValueError, TypeError):
+        return False
+    window = timedelta(hours=award_display_hours())
+    return datetime.now(timezone.utc) - published <= window
+
+
+def enrich_award(row: dict, givers: dict | None = None) -> dict:
+    item = dict(row)
+    giver_uid = row.get("giver_uid", "")
+    giver = (givers or {}).get(giver_uid) or get_users_by_uids([giver_uid]).get(giver_uid)
+    item["giver"] = giver
+    item["image_url"] = f"/awards/{row.get('slug', '')}/256"
+    item["thumb_url"] = f"/awards/{row.get('slug', '')}/64"
+    return item
+
+
+def get_user_awards(receiver_uid: str, page: int = 1, per_page: int = AWARDS_PER_PAGE):
+    if not receiver_uid or "awards" not in db.tables:
+        return [], build_pagination(page, 0, per_page)
+    table = _awards_table()
+    total = table.count(receiver_uid=receiver_uid, **_published_filter())
+    offset = max(0, (page - 1) * per_page)
+    rows = list(
+        table.find(
+            receiver_uid=receiver_uid,
+            deleted_at=None,
+            generated_at={">": ""},
+            order_by=["-generated_at"],
+            _limit=per_page,
+            _offset=offset,
+        )
+    )
+    giver_uids = [row.get("giver_uid") for row in rows if row.get("giver_uid")]
+    givers = get_users_by_uids(giver_uids)
+    items = [enrich_award(row, givers) for row in rows]
+    return items, build_pagination(page, total, per_page)
+
+
+def get_prominent_award(profile_user: dict) -> dict | None:
+    if not award_is_prominent(profile_user):
+        return None
+    award = resolve_by_slug(_awards_table(), profile_user.get("last_award_uid", ""))
+    if not award:
+        return None
+    return enrich_award(award)
+
+
+def revoke_award(award_uid: str, admin_uid: str) -> dict | None:
+    table = _awards_table()
+    row = table.find_one(uid=award_uid)
+    if not row or row.get("deleted_at"):
+        return None
+    stamp = _now_iso()
+    attachment_uids = [
+        uid
+        for uid in (
+            row.get("attachment_uid_512"),
+            row.get("attachment_uid_256"),
+            row.get("attachment_uid_64"),
+        )
+        if uid
+    ]
+    soft_delete("awards", admin_uid, stamp=stamp, uid=award_uid)
+    from devplacepy.attachments import soft_delete_attachments_for
+
+    soft_delete_attachments_for("award", [award_uid], admin_uid)
+    if attachment_uids:
+        soft_delete_in("attachments", "uid", attachment_uids, admin_uid, stamp=stamp)
+    recompute_user_award_stats(row.get("receiver_uid", ""))
+    return row

devplacepy/database/comments.py

@@ -126,7 +126,7 @@ def load_comments_by_target_uids(target_type, target_uids, user=None):
     params["tt"] = target_type
     raw = list(
         db.query(
-            f"SELECT * FROM comments WHERE target_type=:tt AND target_uid IN ({placeholders}) ORDER BY created_at",
+            f"SELECT * FROM comments WHERE target_type=:tt AND target_uid IN ({placeholders}) AND deleted_at IS NULL ORDER BY created_at",
             **params,
         )
     )

devplacepy/database/content.py

@@ -1,7 +1,14 @@
 # retoor <retoor@molodetz.nl>
 
+from collections import Counter
+
+from devplacepy.cache import TTLCache
+
 from .core import db, get_table, or_
 
+_daily_topic_cache = TTLCache(ttl=60, max_size=1)
+_trending_cache = TTLCache(ttl=15, max_size=1)
+
 
 def resolve_by_slug(table, slug, include_deleted=False):
     has_soft_delete = table.has_column("deleted_at")
@@ -40,6 +47,12 @@ def resolve_object_url(target_type: str, target_uid: str) -> str:
             comment.get("target_uid") or comment.get("post_uid", ""),
         )
         return f"{parent_url}#comment-{target_uid}"
+    if target_type == "award":
+        award = resolve_by_slug(get_table("awards"), target_uid)
+        if award:
+            receiver = get_table("users").find_one(uid=award.get("receiver_uid", ""))
+            if receiver:
+                return f"/profile/{receiver['username']}?tab=awards#award-{award.get('slug', '')}"
     return "/feed"
 
 
@@ -71,6 +84,15 @@ def text_search_clause(
 
 
 def get_daily_topic():
+    cached = _daily_topic_cache.get("topic")
+    if cached is not None:
+        return cached
+    topic = _load_daily_topic()
+    _daily_topic_cache.set("topic", topic)
+    return topic
+
+
+def _load_daily_topic():
     if "news" in db.tables:
         article = db["news"].find_one(
             status="published", deleted_at=None, order_by=["-synced_at"]
@@ -122,3 +144,24 @@ def get_featured_news(limit=5):
             }
         )
     return articles
+
+
+def get_trending_topics(limit: int = 6) -> list[dict]:
+    cached = _trending_cache.get("topics")
+    if cached is not None:
+        return cached[:limit]
+    if "posts" not in db.tables or "topic" not in db["posts"].columns:
+        return []
+    rows = db.query(
+        "SELECT topic FROM posts WHERE deleted_at IS NULL "
+        "AND topic IS NOT NULL AND topic != '' "
+        "ORDER BY created_at DESC LIMIT 200"
+    )
+    counter: Counter[str] = Counter()
+    for row in rows:
+        topic = (row["topic"] or "").strip()
+        if topic:
+            counter[topic] += 1
+    topics = [{"topic": t, "count": c} for t, c in counter.most_common(limit)]
+    _trending_cache.set("topics", topics)
+    return topics

devplacepy/database/core.py

@@ -61,10 +61,11 @@ def _ensure_cache_state() -> None:
     global _cache_state_ready
     if _cache_state_ready:
         return
-    db.query(
-        "CREATE TABLE IF NOT EXISTS cache_state "
-        "(name TEXT PRIMARY KEY, version INTEGER NOT NULL DEFAULT 0)"
-    )
+    with db:
+        db.query(
+            "CREATE TABLE IF NOT EXISTS cache_state "
+            "(name TEXT PRIMARY KEY, version INTEGER NOT NULL DEFAULT 0)"
+        )
     _cache_state_ready = True
 
 
@@ -74,18 +75,15 @@ def get_cache_version(name: str) -> int:
         return cached
     try:
         _ensure_cache_state()
-        row = next(
-            iter(
-                db.query(
-                    "SELECT version FROM cache_state WHERE name = :name", name=name
-                )
-            ),
-            None,
-        )
-        version = int(row["version"]) if row else 0
+        with db:
+            rows = list(db.query("SELECT name, version FROM cache_state"))
+        versions = {row["name"]: int(row["version"]) for row in rows}
     except Exception as e:
         logger.warning(f"Could not read cache version {name}: {e}")
         return 0
+    for key, version in versions.items():
+        _cache_version_cache.set(key, version)
+    version = versions.get(name, 0)
     _cache_version_cache.set(name, version)
     return version
 
@@ -93,16 +91,15 @@ def get_cache_version(name: str) -> int:
 def bump_cache_version(name: str) -> None:
     try:
         _ensure_cache_state()
-        db.query(
-            "INSERT OR IGNORE INTO cache_state (name, version) VALUES (:name, 0)",
-            name=name,
-        )
-        db.query(
-            "UPDATE cache_state SET version = version + 1 WHERE name = :name", name=name
-        )
-        connection = db.executable
-        if connection.in_transaction():
-            connection.commit()
+        with db:
+            db.query(
+                "INSERT OR IGNORE INTO cache_state (name, version) VALUES (:name, 0)",
+                name=name,
+            )
+            db.query(
+                "UPDATE cache_state SET version = version + 1 WHERE name = :name",
+                name=name,
+            )
         _cache_version_cache.pop(name)
     except Exception as e:
         logger.warning(f"Could not bump cache version {name}: {e}")

devplacepy/database/notifications.py

@@ -17,6 +17,9 @@ NOTIFICATION_TYPES = [
     {"key": "issue", "label": "Issue tracker", "description": "Updates on issue reports you filed"},
     {"key": "reminder", "label": "Reminders", "description": "A reminder or scheduled task you asked Devii to run fires"},
     {"key": "harvest_stolen", "label": "Farm raids", "description": "Someone steals a ready build from your Code Farm"},
+    {"key": "award", "label": "Awards", "description": "Someone gives you an award on your profile"},
+    {"key": "system", "label": "System alerts", "description": "Platform infrastructure alerts (e.g. the AI gateway going down)"},
+    {"key": "thread_comment", "label": "Thread comments", "description": "Someone else comments on a post you commented on"},
 ]
 
 

devplacepy/database/pagination.py

@@ -1,5 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
+from devplacepy.cache import TTLCache
 from .core import db, get_table
 from .relations import get_blocked_uids
 
@@ -7,6 +8,9 @@ from .relations import get_blocked_uids
 PAGE_SIZE = 25
 
 
+_user_post_count_cache = TTLCache(ttl=15, max_size=2000)
+
+
 def paginate(
     table,
     *clauses,
@@ -74,10 +78,19 @@ def paginate_diverse(
     return interleave_by_author(rows, uid_key=uid_key), next_cursor
 
 
+def clear_user_post_count(user_uid: str) -> None:
+    _user_post_count_cache.pop(user_uid)
+
+
 def get_user_post_count(user_uid: str) -> int:
+    cached = _user_post_count_cache.get(user_uid)
+    if cached is not None:
+        return cached
     if "posts" not in db.tables:
         return 0
-    return get_table("posts").count(user_uid=user_uid, deleted_at=None)
+    count = get_table("posts").count(user_uid=user_uid, deleted_at=None)
+    _user_post_count_cache.set(user_uid, count)
+    return count
 
 
 def build_pagination(page, total, per_page=25):

devplacepy/database/ranking.py

@@ -16,7 +16,7 @@ VOTABLE_TARGETS: dict[str, str] = {
 STAR_TARGETS: set[str] = {"post", "project", "gist"}
 
 
-_authors_cache = TTLCache(ttl=15, max_size=200)
+_authors_cache = TTLCache(ttl=60, max_size=200)
 
 
 _stars_cache = TTLCache(ttl=15, max_size=2000)
@@ -151,17 +151,19 @@ def delete_engagement(target_type: str, target_uids: list) -> None:
     if "reactions" in db.tables:
         placeholders, params = _in_clause(uids)
         params["tt"] = target_type
-        db.query(
-            f"DELETE FROM reactions WHERE target_type=:tt AND target_uid IN ({placeholders})",
-            **params,
-        )
+        with db:
+            db.query(
+                f"DELETE FROM reactions WHERE target_type=:tt AND target_uid IN ({placeholders})",
+                **params,
+            )
     if "bookmarks" in db.tables:
         placeholders, params = _in_clause(uids)
         params["tt"] = target_type
-        db.query(
-            f"DELETE FROM bookmarks WHERE target_type=:tt AND target_uid IN ({placeholders})",
-            **params,
-        )
+        with db:
+            db.query(
+                f"DELETE FROM bookmarks WHERE target_type=:tt AND target_uid IN ({placeholders})",
+                **params,
+            )
     if target_type == "post" and "polls" in db.tables:
         for uid in uids:
             for poll in db["polls"].find(post_uid=uid):

devplacepy/database/remote.py

@@ -0,0 +1,348 @@
+# retoor <retoor@molodetz.nl>
+
+import inspect
+import os
+
+import httpx
+
+from devplacepy.cache import TTLCache
+from devplacepy_services.base.db_codec import (
+    decode_value,
+    encode_args,
+    is_write,
+    is_write_sql,
+)
+
+_SERVICE_URL = os.environ.get("DEVPLACE_DB_SERVICE_URL", "http://127.0.0.1:10601").rstrip("/")
+_INTERNAL_KEY = os.environ.get("DEVPLACE_GATEWAY_INTERNAL_KEY", "").strip()
+_CLIENT: httpx.Client | None = None
+
+# Section 7.3: settings reads tolerate up to 5s staleness. patch_module()
+# generically RPCs every devplacepy.database call, bypassing the local
+# TTL cache get_setting/get_int_setting had in-process - without this,
+# every settings read (rate limiting, maintenance mode, admin dashboards)
+# pays a full HTTP round trip to the database broker.
+_SETTINGS_CACHE_TTL_SECONDS = 5
+_SETTINGS_CACHE = TTLCache(ttl=_SETTINGS_CACHE_TTL_SECONDS, max_size=512)
+_CACHED_SETTINGS_FNS = frozenset({"get_setting", "get_int_setting"})
+
+
+def _service_url() -> str:
+    return os.environ.get("DEVPLACE_DB_SERVICE_URL", _SERVICE_URL).rstrip("/")
+
+
+def _headers() -> dict[str, str]:
+    headers: dict[str, str] = {}
+    key = os.environ.get("DEVPLACE_GATEWAY_INTERNAL_KEY", _INTERNAL_KEY).strip()
+    if key:
+        headers["X-Internal-Key"] = key
+    return headers
+
+
+def _client() -> httpx.Client:
+    global _CLIENT
+    if _CLIENT is None:
+        _CLIENT = httpx.Client(timeout=30.0)
+    return _CLIENT
+
+
+def _post(path: str, body: dict) -> object:
+    response = _client().post(
+        f"{_service_url()}/{path.lstrip('/')}",
+        json=body,
+        headers=_headers(),
+    )
+    if response.status_code >= 400:
+        payload = response.json() if response.content else {}
+        message = payload.get("error", "Database service request failed")
+        raise RuntimeError(message)
+    if not response.content:
+        return None
+    return decode_value(response.json())
+
+
+def _invoke_cached(fn_name: str, args, kwargs):
+    cache_key = f"{fn_name}:{args!r}:{sorted(kwargs.items())!r}"
+    cached = _SETTINGS_CACHE.get(cache_key)
+    if cached is not None:
+        return cached
+    value = _invoke(fn_name, args, kwargs, write=False)
+    _SETTINGS_CACHE.set(cache_key, value)
+    return value
+
+
+def _invoke(fn_name: str, args, kwargs, *, write: bool = False):
+    encoded_args, encoded_kwargs = encode_args(args, kwargs)
+    payload = {
+        "fn": fn_name,
+        "args": encoded_args,
+        "kwargs": encoded_kwargs,
+        "write": write,
+    }
+    result = _post("internal/invoke", payload)
+    if isinstance(result, dict) and "result" in result:
+        return result["result"]
+    return result
+
+
+class RemoteSearchClause:
+    def __init__(self, term, fields, author_field=None):
+        self.term = term.strip()
+        self.fields = tuple(fields)
+        self.author_field = author_field
+
+
+class RemoteUidInClause:
+    def __init__(self, field, uids):
+        self.field = field
+        self.uids = frozenset(uids)
+
+
+class RemoteTable:
+    def __init__(self, db: "RemoteDb", name: str) -> None:
+        self._db = db
+        self._name = name
+        self._column_cache = None
+
+    def __getattr__(self, name: str):
+        def caller(*args, **kwargs):
+            return self._db._table_op(self._name, name, args, kwargs)
+
+        return caller
+
+    def has_column(self, name: str) -> bool:
+        cache = self._column_cache
+        if cache is None:
+            sample = self.find(_limit=1)
+            row = next(iter(sample), None)
+            cache = set(row.keys()) if row else set()
+            self._column_cache = cache
+        return name in cache
+
+    def count(self, **kwargs):
+        return self._db._table_op(self._name, "count", [], kwargs)
+
+    @property
+    def table(self):
+        return self
+
+    @property
+    def exists(self) -> bool:
+        return self._name in self._db.tables
+
+class RemoteDb:
+    def __init__(self) -> None:
+        self._tables_cache: list[str] | None = None
+
+    @property
+    def tables(self) -> list[str]:
+        if self._tables_cache is None:
+            result = _post("internal/db-op", {"op": "tables"})
+            self._tables_cache = list(result or [])
+        return self._tables_cache
+
+    def __getitem__(self, name: str) -> RemoteTable:
+        return RemoteTable(self, name)
+
+    def query(self, sql: str, **params):
+        encoded_args, encoded_kwargs = encode_args((sql,), params)
+        result = _post(
+            "internal/db-op",
+            {
+                "op": "query",
+                "args": encoded_args,
+                "kwargs": encoded_kwargs,
+                "write": is_write_sql(sql),
+            },
+        )
+        return result or []
+
+    def _table_op(self, table: str, method: str, args, kwargs, *, write: bool = False):
+        encoded_args, encoded_kwargs = encode_args(args, kwargs)
+        result = _post(
+            "internal/db-op",
+            {
+                "op": "table_op",
+                "table": table,
+                "method": method,
+                "args": encoded_args,
+                "kwargs": encoded_kwargs,
+                "write": write,
+            },
+        )
+        if method in {"insert", "update", "delete"}:
+            self._tables_cache = None
+        return result
+
+    @property
+    def executable(self):
+        return self
+
+    @property
+    def in_transaction(self) -> bool:
+        return False
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc, tb):
+        return False
+
+
+_LOCAL_REMOTE = frozenset(
+    {
+        "get_table",
+        "refresh_snapshot",
+        "_in_clause",
+        "_now_iso",
+        "text_search_clause",
+    }
+)
+
+
+def _remote_text_search_clause(
+    table, search, fields=("title", "description"), author_field=None
+):
+    term = (search or "").strip()
+    if not term:
+        return None
+    if type(table).__name__ == "RemoteTable":
+        return RemoteSearchClause(term, fields, author_field)
+    from devplacepy.database.content import text_search_clause as local_clause
+
+    return local_clause(table, search, fields, author_field=author_field)
+
+
+def _remote_get_table(name: str):
+    import devplacepy.database.core as core
+
+    return core.db[name]
+
+
+def _remote_refresh_snapshot() -> None:
+    return None
+
+
+def patch_module(module) -> None:
+    import devplacepy.database as db_module
+
+    for name in db_module.__all__:
+        if name in _LOCAL_REMOTE:
+            continue
+        target = getattr(module, name, None)
+        if target is None or not callable(target):
+            continue
+        if inspect.isclass(target):
+            continue
+
+        def make_wrapper(fn_name: str, fn_write: bool):
+            if fn_name in _CACHED_SETTINGS_FNS:
+
+                def wrapper(*args, **kwargs):
+                    return _invoke_cached(fn_name, args, kwargs)
+
+                wrapper.__name__ = fn_name
+                return wrapper
+
+            def wrapper(*args, **kwargs):
+                return _invoke(fn_name, args, kwargs, write=fn_write)
+
+            wrapper.__name__ = fn_name
+            return wrapper
+
+        setattr(module, name, make_wrapper(name, is_write(name)))
+
+
+def activate() -> None:
+    import devplacepy.database.core as core
+
+    core.db = RemoteDb()
+    import devplacepy.database as db_module
+
+    patch_module(db_module)
+    for submodule_name in (
+        "settings",
+        "users",
+        "relations",
+        "pagination",
+        "soft_delete",
+        "engagement",
+        "usage",
+        "awards",
+        "seo_meta",
+        "activity",
+        "customization",
+        "email",
+        "notifications",
+        "forks",
+        "follows",
+        "deepsearch",
+        "ranking",
+        "comments",
+        "content",
+        "attachments_data",
+        "stats",
+        "schema",
+    ):
+        try:
+            submodule = __import__(
+                f"devplacepy.database.{submodule_name}",
+                fromlist=[submodule_name],
+            )
+        except ImportError:
+            continue
+        patch_module(submodule)
+    for external_name in (
+        "devplacepy.services.statistics.tracking",
+        "devplacepy.services.base",
+        "devplacepy.attachments",
+        "devplacepy.project_files",
+    ):
+        try:
+            external = __import__(external_name, fromlist=[external_name.split(".")[-1]])
+        except ImportError:
+            continue
+        if hasattr(external, "db"):
+            external.db = RemoteDb()
+    db_module.db = core.db
+    db_module.get_table = _remote_get_table
+    core.get_table = _remote_get_table
+    db_module.refresh_snapshot = _remote_refresh_snapshot
+    core.refresh_snapshot = _remote_refresh_snapshot
+    db_module.text_search_clause = _remote_text_search_clause
+    import devplacepy.database.content as content_module
+
+    content_module.text_search_clause = _remote_text_search_clause
+    for submodule_name in (
+        "settings",
+        "users",
+        "relations",
+        "pagination",
+        "soft_delete",
+        "engagement",
+        "usage",
+        "awards",
+        "seo_meta",
+        "activity",
+        "customization",
+        "email",
+        "notifications",
+        "forks",
+        "follows",
+        "deepsearch",
+        "ranking",
+        "comments",
+        "content",
+        "attachments_data",
+        "stats",
+        "schema",
+    ):
+        try:
+            submodule = __import__(
+                f"devplacepy.database.{submodule_name}",
+                fromlist=[submodule_name],
+            )
+        except ImportError:
+            continue
+        if hasattr(submodule, "db"):
+            submodule.db = core.db

devplacepy/database/schema.py

@@ -1,6 +1,6 @@
 # retoor <retoor@molodetz.nl>
 
-from .core import DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, _index, _uid_index, db, defaultdict, get_table, logger
+from .core import DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT, INTERNAL_GATEWAY_URL, _drop_index, _index, _uid_index, db, defaultdict, get_table, logger
 from .settings import get_setting, set_setting
 from .soft_delete import SOFT_DELETE_TABLES, ensure_soft_delete_columns
 from .ranking import _authors_cache
@@ -183,8 +183,10 @@ def init_db():
     )
     _index(db, "badges", "idx_badges_user", ["user_uid"])
     _index(db, "badges", "idx_badges_user_name", ["user_uid", "badge_name"])
-    _index(db, "follows", "idx_follows_follower", ["follower_uid"])
-    _index(db, "follows", "idx_follows_following", ["following_uid"])
+    _drop_index(db, "idx_follows_follower")
+    _drop_index(db, "idx_follows_following")
+    _index(db, "follows", "idx_follows_follower_created", ["follower_uid", "created_at"])
+    _index(db, "follows", "idx_follows_following_created", ["following_uid", "created_at"])
     user_relations = get_table("user_relations")
     for column, example in (
         ("uid", ""),
@@ -356,6 +358,21 @@ def init_db():
     _index(
         db, "issue_comment_authors", "idx_issue_comment_authors_number", ["gitea_number"]
     )
+
+    ws_tickets = get_table("ws_tickets")
+    for column, example in (
+        ("uid", ""),
+        ("token", ""),
+        ("user_uid", ""),
+        ("created_at", ""),
+        ("expires_at", ""),
+        ("used_at", ""),
+    ):
+        if not ws_tickets.has_column(column):
+            ws_tickets.create_column_by_example(column, example)
+    _index(db, "ws_tickets", "idx_ws_tickets_token", ["token"], unique=True)
+    _index(db, "ws_tickets", "idx_ws_tickets_expires", ["expires_at"])
+
     migrate_bug_tables_to_issue_tables()
     _index(db, "service_state", "idx_service_state_name", ["name"])
     if "devii_conversations" in db.tables:
@@ -363,9 +380,10 @@ def init_db():
         if not conversations.has_column("channel"):
             conversations.create_column_by_example("channel", "main")
         try:
-            db.query(
-                "UPDATE devii_conversations SET channel='main' WHERE channel IS NULL"
-            )
+            with db:
+                db.query(
+                    "UPDATE devii_conversations SET channel='main' WHERE channel IS NULL"
+                )
         except Exception as e:  # noqa: BLE001
             logger.warning(f"Could not backfill devii_conversations.channel: {e}")
     _index(
@@ -391,6 +409,7 @@ def init_db():
         db, "devii_tasks", "idx_devii_tasks_due", ["enabled", "status", "next_run_at"]
     )
     _index(db, "devii_lessons", "idx_devii_lessons_owner", ["owner_kind", "owner_id"])
+    _index(db, "devii_lessons", "idx_devii_lessons_owner_created", ["owner_kind", "owner_id", "created_at"])
     _index(
         db, "devii_virtual_tools", "idx_devii_vtools_owner", ["owner_kind", "owner_id"]
     )
@@ -430,6 +449,12 @@ def init_db():
         "idx_gw_usage_endpoint_time",
         ["endpoint", "created_at"],
     )
+    _index(
+        db,
+        "gateway_usage_ledger",
+        "idx_gw_usage_appref_time",
+        ["app_reference", "created_at"],
+    )
     _index(db, "gateway_concurrency_samples", "idx_gw_conc_time", ["created_at"])
     jobs_table = get_table("jobs")
     for column, example in (
@@ -461,16 +486,26 @@ def init_db():
     _index(db, "jobs", "idx_jobs_expires", ["expires_at"])
     _index(db, "project_forks", "idx_project_forks_source", ["source_project_uid"])
     _index(db, "project_forks", "idx_project_forks_forked", ["forked_project_uid"])
-    if "instances" in db.tables:
-        instances = get_table("instances")
-        for column, example in (
-            ("run_as_uid", ""),
-            ("boot_language", "none"),
-            ("boot_script", ""),
-            ("start_on_boot", 0),
-        ):
-            if not instances.has_column(column):
-                instances.create_column_by_example(column, example)
+    instances = get_table("instances")
+    for column, example in (
+        ("uid", ""),
+        ("project_uid", ""),
+        ("slug", ""),
+        ("name", ""),
+        ("status", ""),
+        ("desired_state", ""),
+        ("container_id", ""),
+        ("ingress_slug", ""),
+        ("ingress_port", 0),
+        ("ports_json", ""),
+        ("container_gateway", ""),
+        ("run_as_uid", ""),
+        ("boot_language", "none"),
+        ("boot_script", ""),
+        ("start_on_boot", 0),
+    ):
+        if not instances.has_column(column):
+            instances.create_column_by_example(column, example)
 
     _index(db, "instances", "idx_instances_project", ["project_uid"])
     _index(db, "instances", "idx_instances_slug", ["slug"])
@@ -508,6 +543,10 @@ def init_db():
     from devplacepy.services.openai_gateway import routing as gateway_routing
 
     gateway_routing.ensure_tables()
+
+    from devplacepy.services.openai_gateway import quota as gateway_quota
+
+    gateway_quota.ensure_tables()
     _index(db, "audit_log", "idx_audit_created_at", ["created_at"])
     _index(db, "audit_log", "idx_audit_event_key", ["event_key"])
     _index(db, "audit_log", "idx_audit_category", ["category"])
@@ -559,10 +598,11 @@ def init_db():
             correction_usage.create_column_by_example(column, example)
     try:
         if "correction_usage" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_correction_usage_user "
-                "ON correction_usage (user_uid)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_correction_usage_user "
+                    "ON correction_usage (user_uid)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on correction_usage: {e}")
 
@@ -582,10 +622,11 @@ def init_db():
             modifier_usage.create_column_by_example(column, example)
     try:
         if "modifier_usage" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_modifier_usage_user "
-                "ON modifier_usage (user_uid)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_modifier_usage_user "
+                    "ON modifier_usage (user_uid)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on modifier_usage: {e}")
 
@@ -605,10 +646,11 @@ def init_db():
             news_usage.create_column_by_example(column, example)
     try:
         if "news_usage" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_news_usage_user "
-                "ON news_usage (user_uid)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_news_usage_user "
+                    "ON news_usage (user_uid)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on news_usage: {e}")
 
@@ -628,10 +670,11 @@ def init_db():
             issue_usage.create_column_by_example(column, example)
     try:
         if "issue_usage" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_issue_usage_user "
-                "ON issue_usage (user_uid)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_issue_usage_user "
+                    "ON issue_usage (user_uid)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on issue_usage: {e}")
 
@@ -651,13 +694,77 @@ def init_db():
             seo_usage.create_column_by_example(column, example)
     try:
         if "seo_usage" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_seo_usage_user "
-                "ON seo_usage (user_uid)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_seo_usage_user "
+                    "ON seo_usage (user_uid)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on seo_usage: {e}")
 
+    award_usage = get_table("award_usage")
+    for column, example in (
+        ("user_uid", ""),
+        ("calls", 0),
+        ("prompt_tokens", 0),
+        ("completion_tokens", 0),
+        ("total_tokens", 0),
+        ("cost_usd", 0.0),
+        ("upstream_latency_ms", 0.0),
+        ("total_latency_ms", 0.0),
+        ("updated_at", ""),
+    ):
+        if not award_usage.has_column(column):
+            award_usage.create_column_by_example(column, example)
+    try:
+        if "award_usage" in db.tables:
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_award_usage_user "
+                    "ON award_usage (user_uid)"
+                )
+    except Exception as e:
+        logger.warning(f"Could not create unique index on award_usage: {e}")
+
+    awards = get_table("awards")
+    for column, example in (
+        ("uid", ""),
+        ("slug", ""),
+        ("description", ""),
+        ("giver_uid", ""),
+        ("receiver_uid", ""),
+        ("attachment_uid_512", ""),
+        ("attachment_uid_256", ""),
+        ("attachment_uid_64", ""),
+        ("generated_at", ""),
+        ("created_at", ""),
+        ("job_uid", ""),
+        ("deleted_at", ""),
+        ("deleted_by", ""),
+    ):
+        if not awards.has_column(column):
+            awards.create_column_by_example(column, example)
+    try:
+        if "awards" in db.tables:
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_awards_slug ON awards (slug)"
+                )
+                db.query(
+                    "CREATE INDEX IF NOT EXISTS idx_awards_receiver_created "
+                    "ON awards (receiver_uid, created_at)"
+                )
+                db.query(
+                    "CREATE INDEX IF NOT EXISTS idx_awards_giver_created "
+                    "ON awards (giver_uid, created_at)"
+                )
+                db.query(
+                    "CREATE INDEX IF NOT EXISTS idx_awards_generated "
+                    "ON awards (receiver_uid, generated_at)"
+                )
+    except Exception as e:
+        logger.warning(f"Could not create awards indexes: {e}")
+
     seo_metadata = get_table("seo_metadata")
     for column, example in (
         ("uid", ""),
@@ -727,10 +834,11 @@ def init_db():
             user_activity.create_column_by_example(column, example)
     try:
         if "user_activity" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_user_activity_user_action "
-                "ON user_activity (user_uid, action)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_user_activity_user_action "
+                    "ON user_activity (user_uid, action)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on user_activity: {e}")
 
@@ -745,10 +853,11 @@ def init_db():
             user_activity_seen.create_column_by_example(column, example)
     try:
         if "user_activity_seen" in db.tables:
-            db.query(
-                "CREATE UNIQUE INDEX IF NOT EXISTS idx_user_activity_seen_unique "
-                "ON user_activity_seen (user_uid, action, target)"
-            )
+            with db:
+                db.query(
+                    "CREATE UNIQUE INDEX IF NOT EXISTS idx_user_activity_seen_unique "
+                    "ON user_activity_seen (user_uid, action, target)"
+                )
     except Exception as e:
         logger.warning(f"Could not create unique index on user_activity_seen: {e}")
 
@@ -934,6 +1043,31 @@ def init_db():
         ("legacy_speed", 0),
         ("legacy_plots", 0),
         ("legacy_defense", 0),
+        ("legacy_carryover", 0),
+        ("last_grant_week", ""),
+        ("prestiged_at", ""),
+        ("mastery_points", 0),
+        ("mastery_points_earned_total", 0),
+        ("mastery_autoreplant", 0),
+        ("mastery_analytics", 0),
+        ("mastery_contracts", 0),
+        ("lifetime_coins_earned", 0),
+        ("lifetime_harvests", 0),
+        ("infra_registry", 0),
+        ("infra_canary", 0),
+        ("infra_observability", 0),
+        ("defense_level", 0),
+        ("defense_last_upkeep_at", ""),
+        ("active_title", ""),
+        ("underdog_boost_until", ""),
+        ("contract_boost_until", ""),
+        ("harvests_week", 0),
+        ("harvests_week_start", ""),
+        ("last_kernel_harvest_prestige", 0),
+        ("time_to_kernel_seconds", 0),
+        ("era_coins", 0),
+        ("era_harvests", 0),
+        ("era_joined_at", ""),
         ("created_at", ""),
         ("updated_at", ""),
     ):
@@ -965,6 +1099,7 @@ def init_db():
         ("farm_uid", ""),
         ("user_uid", ""),
         ("day", ""),
+        ("scope", "daily"),
         ("slot_index", 0),
         ("kind", ""),
         ("label", ""),
@@ -972,13 +1107,17 @@ def init_db():
         ("progress", 0),
         ("reward_coins", 0),
         ("reward_xp", 0),
+        ("reward_stars", 0),
         ("claimed", 0),
         ("created_at", ""),
         ("updated_at", ""),
     ):
         if not game_quests.has_column(column):
             game_quests.create_column_by_example(column, example)
+    with db:
+        db.query("UPDATE game_quests SET scope = 'daily' WHERE scope IS NULL OR scope = ''")
     _index(db, "game_quests", "idx_game_quests_farm_day", ["farm_uid", "day"])
+    _index(db, "game_quests", "idx_game_quests_farm_day_scope", ["farm_uid", "day", "scope"])
 
     game_plots = get_table("game_plots")
     for column, example in (
@@ -997,6 +1136,83 @@ def init_db():
             game_plots.create_column_by_example(column, example)
     _index(db, "game_plots", "idx_game_plots_farm", ["farm_uid", "slot_index"])
 
+    game_market_ticks = get_table("game_market_ticks")
+    for column, example in (
+        ("uid", ""),
+        ("crop_key", ""),
+        ("hour_bucket", ""),
+        ("harvests", 0),
+        ("updated_at", ""),
+    ):
+        if not game_market_ticks.has_column(column):
+            game_market_ticks.create_column_by_example(column, example)
+    _index(
+        db,
+        "game_market_ticks",
+        "idx_game_market_ticks_bucket",
+        ["crop_key", "hour_bucket"],
+        unique=True,
+    )
+
+    game_cosmetics = get_table("game_cosmetics")
+    for column, example in (
+        ("uid", ""),
+        ("user_uid", ""),
+        ("cosmetic_key", ""),
+        ("purchased_at", ""),
+        ("created_at", ""),
+    ):
+        if not game_cosmetics.has_column(column):
+            game_cosmetics.create_column_by_example(column, example)
+    _index(
+        db,
+        "game_cosmetics",
+        "idx_game_cosmetics_owner",
+        ["user_uid", "cosmetic_key"],
+        unique=True,
+    )
+
+    game_treasury = get_table("game_treasury")
+    for column, example in (
+        ("uid", ""),
+        ("balance", 0),
+        ("collected_total", 0),
+        ("granted_total", 0),
+        ("updated_at", ""),
+    ):
+        if not game_treasury.has_column(column):
+            game_treasury.create_column_by_example(column, example)
+
+    game_eras = get_table("game_eras")
+    for column, example in (
+        ("uid", ""),
+        ("era_number", 0),
+        ("name", ""),
+        ("started_at", ""),
+        ("ends_at", ""),
+        ("active", 0),
+        ("created_at", ""),
+    ):
+        if not game_eras.has_column(column):
+            game_eras.create_column_by_example(column, example)
+    _index(db, "game_eras", "idx_game_eras_active", ["active"])
+
+    game_era_results = get_table("game_era_results")
+    for column, example in (
+        ("uid", ""),
+        ("era_number", 0),
+        ("user_uid", ""),
+        ("rank", 0),
+        ("era_score", 0),
+        ("era_coins_final", 0),
+        ("reward_stars", 0),
+        ("reward_cosmetic_key", ""),
+        ("created_at", ""),
+    ):
+        if not game_era_results.has_column(column):
+            game_era_results.create_column_by_example(column, example)
+    _index(db, "game_era_results", "idx_game_era_results_era", ["era_number", "rank"])
+
     _index(db, "posts", "idx_posts_user_created", ["user_uid", "created_at"])
     _index(
         db,
@@ -1155,7 +1371,11 @@ def init_db():
             "customization_enabled": "1",
             "customization_js_enabled": "1",
             "audit_log_retention_days": "90",
+            "statistics_tracking_enabled": "1",
             "docs_search_mode": "agent",
+            "outbound_proxy_url": "",
+            "devii_lessons_max_per_owner": "500",
+            "devii_lessons_max_age_days": "90",
         }
         for key, value in operational_defaults.items():
             existing = db["site_settings"].find_one(key=key)
@@ -1164,6 +1384,72 @@ def init_db():
                     {"uid": f"default_{key}", "key": key, "value": value}
                 )
 
+    with db:
+        db.query(
+            "CREATE TABLE IF NOT EXISTS visit_stats_hourly ("
+            "bucket_start TEXT NOT NULL, "
+            "page_group TEXT NOT NULL, "
+            "referrer_group TEXT NOT NULL, "
+            "views INTEGER NOT NULL DEFAULT 0, "
+            "member_views INTEGER NOT NULL DEFAULT 0, "
+            "guest_views INTEGER NOT NULL DEFAULT 0)"
+        )
+        db.query(
+            "CREATE TABLE IF NOT EXISTS visit_unique_slots ("
+            "bucket_start TEXT NOT NULL, "
+            "visitor_hash TEXT NOT NULL, "
+            "page_group TEXT NOT NULL, "
+            "user_uid TEXT)"
+        )
+    _index(db, "visit_stats_hourly", "idx_visit_hourly_bucket", ["bucket_start"])
+    _index(
+        db,
+        "visit_stats_hourly",
+        "idx_visit_hourly_page_time",
+        ["page_group", "bucket_start"],
+    )
+    _index(
+        db,
+        "visit_stats_hourly",
+        "idx_visit_hourly_ref_time",
+        ["referrer_group", "bucket_start"],
+    )
+    _index(
+        db,
+        "visit_stats_hourly",
+        "idx_visit_hourly_unique_row",
+        ["bucket_start", "page_group", "referrer_group"],
+        unique=True,
+    )
+    _index(db, "visit_unique_slots", "idx_visit_unique_bucket", ["bucket_start"])
+    _index(
+        db,
+        "visit_unique_slots",
+        "idx_visit_unique_hash",
+        ["bucket_start", "visitor_hash"],
+    )
+    _index(
+        db,
+        "visit_unique_slots",
+        "idx_visit_unique_row",
+        ["bucket_start", "visitor_hash", "page_group"],
+        unique=True,
+    )
+    _index(db, "issue_tickets", "idx_issue_tickets_created", ["created_at"])
+    _index(db, "devii_turns", "idx_devii_turns_started", ["started_at"])
+    _index(db, "instance_events", "idx_instance_events_created", ["created_at"])
+    _index(db, "game_steals", "idx_game_steals_stolen_at", ["stolen_at"])
+    _index(db, "messages", "idx_messages_created_at", ["created_at"])
+    _index(db, "notifications", "idx_notifications_created", ["created_at"])
+    _index(db, "follows", "idx_follows_created_at", ["created_at"])
+    _index(db, "reactions", "idx_reactions_created_at", ["created_at"])
+    _index(db, "votes", "idx_votes_created_at", ["created_at"])
+    _index(db, "bookmarks", "idx_bookmarks_created_at", ["created_at"])
+    _index(db, "badges", "idx_badges_created_at", ["created_at"])
+    _index(db, "audit_log", "idx_audit_result_created", ["result", "created_at"])
+    _index(db, "jobs", "idx_jobs_created_at", ["created_at"])
+    _index(db, "attachments", "idx_attachments_created_at", ["created_at"])
+
     _backfill_gamification()
     backfill_api_keys()
     migrate_ai_gateway_settings()
@@ -1210,6 +1496,15 @@ def migrate_ai_gateway_settings() -> None:
     if get_setting("bot_model", "") == "deepseek-chat":
         set_setting("bot_model", "molodetz")
         logger.info("Migrated bot_model to molodetz")
+    from devplacepy.services.openai_gateway.routing import (
+        migrate_retired_image_gateway,
+        seed_default_deepseek_routes,
+        seed_default_image_routes,
+    )
+
+    seed_default_deepseek_routes()
+    seed_default_image_routes()
+    migrate_retired_image_gateway()
 
 
 def backfill_api_keys() -> int:
@@ -1234,12 +1529,22 @@ def backfill_api_keys() -> int:
         users.create_column_by_example("ai_modifier_sync", 1)
     if not users.has_column("ai_modifier_prompt"):
         users.create_column_by_example("ai_modifier_prompt", DEFAULT_MODIFIER_PROMPT)
+    if not users.has_column("interactions_enabled"):
+        users.create_column_by_example("interactions_enabled", -1)
     if not users.has_column("timezone"):
         users.create_column_by_example("timezone", "")
     if not users.has_column("avatar_seed"):
         users.create_column_by_example("avatar_seed", "")
     if not users.has_column("last_seen"):
         users.create_column_by_example("last_seen", "")
+    if not users.has_column("award_count"):
+        users.create_column_by_example("award_count", 0)
+    if not users.has_column("last_award_at"):
+        users.create_column_by_example("last_award_at", "")
+    if not users.has_column("last_award_slug"):
+        users.create_column_by_example("last_award_slug", "")
+    if not users.has_column("last_award_uid"):
+        users.create_column_by_example("last_award_uid", "")
     with db:
         db.query(
             "UPDATE users SET ai_modifier_enabled = 1 WHERE ai_modifier_enabled IS NULL"
@@ -1250,6 +1555,10 @@ def backfill_api_keys() -> int:
             "WHERE ai_modifier_prompt IS NULL OR ai_modifier_prompt = ''",
             prompt=DEFAULT_MODIFIER_PROMPT,
         )
+        db.query(
+            "UPDATE users SET interactions_enabled = -1 "
+            "WHERE interactions_enabled IS NULL"
+        )
     import uuid_utils
 
     updated = 0

devplacepy/database/soft_delete.py

@@ -41,6 +41,7 @@ SOFT_DELETE_TABLES = [
     "email_accounts",
     "user_relations",
     "seo_metadata",
+    "awards",
 ]
 
 
@@ -93,11 +94,12 @@ def soft_delete_in(table_name, column, uids, deleted_by, *, stamp=None, **extra)
     for index, (key, value) in enumerate(extra.items()):
         params[f"x{index}"] = value
         extra_sql += f" AND {key} = :x{index}"
-    db.query(
-        f"UPDATE {table_name} SET deleted_at = :dat, deleted_by = :dby "
-        f"WHERE {column} IN ({placeholders}) AND deleted_at IS NULL{extra_sql}",
-        **params,
-    )
+    with db:
+        db.query(
+            f"UPDATE {table_name} SET deleted_at = :dat, deleted_by = :dby "
+            f"WHERE {column} IN ({placeholders}) AND deleted_at IS NULL{extra_sql}",
+            **params,
+        )
     return len(uids)
 
 
@@ -160,11 +162,12 @@ def restore_event(stamp):
                     s=stamp,
                 ).__next__()["n"]
             )
-            db.query(
-                f"UPDATE {table_name} SET deleted_at = NULL, deleted_by = NULL "
-                f"WHERE deleted_at = :s",
-                s=stamp,
-            )
+            with db:
+                db.query(
+                    f"UPDATE {table_name} SET deleted_at = NULL, deleted_by = NULL "
+                    f"WHERE deleted_at = :s",
+                    s=stamp,
+                )
     return restored
 
 
@@ -181,7 +184,8 @@ def purge_event(stamp):
             )
             if rows:
                 purged.append((table_name, rows))
-                db.query(
-                    f"DELETE FROM {table_name} WHERE deleted_at = :s", s=stamp
-                )
+                with db:
+                    db.query(
+                        f"DELETE FROM {table_name} WHERE deleted_at = :s", s=stamp
+                    )
     return purged

devplacepy/database/usage.py

@@ -126,3 +126,14 @@ def add_seo_usage(totals: dict) -> None:
 
 def get_seo_usage() -> dict:
     return _get_usage("seo_usage", SEO_USAGE_KEY)
+
+
+AWARD_USAGE_KEY = "award"
+
+
+def add_award_usage(totals: dict) -> None:
+    _add_usage("award_usage", AWARD_USAGE_KEY, totals)
+
+
+def get_award_usage() -> dict:
+    return _get_usage("award_usage", AWARD_USAGE_KEY)

devplacepy/db_client.py

@@ -0,0 +1,29 @@
+# retoor <retoor@molodetz.nl>
+
+import os
+
+
+def _activate() -> None:
+    if os.environ.get("DEVPLACE_DB_SERVICE") == "1":
+        return
+    if os.environ.get("DEVPLACE_REMOTE_DB") == "1":
+        from devplacepy.database.remote import activate
+
+        activate()
+
+
+_activate()
+
+import devplacepy.database as _database
+
+
+def _remote_table(table) -> bool:
+    return type(table).__name__ == "RemoteTable"
+
+
+def __getattr__(name: str):
+    return getattr(_database, name)
+
+
+def __dir__():
+    return sorted(name for name in dir(_database) if not name.startswith("_"))

devplacepy/docs_api/groups/admin.py

@@ -63,6 +63,22 @@ four ways to sign requests.
                 ],
                 sample_response={"ok": True, "redirect": "/admin/media"},
             ),
+            endpoint(
+                id="admin-revoke-award",
+                method="POST",
+                path="/admin/awards/{uid}/revoke",
+                title="Revoke award",
+                summary=(
+                    "Soft-delete a published award and its linked attachments, then recompute "
+                    "receiver stats. Restorable from admin trash."
+                ),
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("uid", "path", "string", True, "", "Award uid to revoke."),
+                ],
+                sample_response={"ok": True, "redirect": "/profile/receiver?tab=awards"},
+            ),
             endpoint(
                 id="admin-media-purge",
                 method="POST",
@@ -171,6 +187,84 @@ four ways to sign requests.
                     "This is the endpoint the Devii assistant calls as `site_analytics`; see [Devii internals](/docs/devii-internals.html).",
                 ],
             ),
+            endpoint(
+                id="admin-statistics",
+                method="GET",
+                path="/admin/statistics/data",
+                title="Platform statistics",
+                summary=(
+                    "Tabbed platform statistics with KPI cards, period-over-period deltas, "
+                    "time-series data for charts, and breakdown tables. Covers visitors, members, "
+                    "content, engagement, social, AI, Devii, services, containers, game, awards, "
+                    "moderation, tools, and storage."
+                ),
+                auth="admin",
+                interactive=True,
+                params=[
+                    field(
+                        "tab",
+                        "query",
+                        "string",
+                        False,
+                        "overview",
+                        "Tab key (overview, visitors, members, content, ...).",
+                    ),
+                    field(
+                        "hours",
+                        "query",
+                        "int",
+                        False,
+                        "168",
+                        "Lookback window in hours (24, 168, 720, 2160, or 0 for all time).",
+                    ),
+                    field(
+                        "compare",
+                        "query",
+                        "int",
+                        False,
+                        "1",
+                        "Include previous-period comparison (1 or 0).",
+                    ),
+                    field(
+                        "top_n",
+                        "query",
+                        "int",
+                        False,
+                        "10",
+                        "Rows in breakdown tables (1-50).",
+                    ),
+                ],
+                notes=[
+                    "The HTML dashboard lives at `/admin/statistics`. Visitor metrics require the statistics tracking middleware (hourly aggregation, 90-day retention).",
+                ],
+            ),
+            endpoint(
+                id="admin-statistics-page",
+                method="GET",
+                path="/admin/statistics",
+                title="Statistics dashboard",
+                summary="Admin HTML dashboard for platform statistics with charts and tabs.",
+                auth="admin",
+                interactive=False,
+                params=[
+                    field(
+                        "tab",
+                        "query",
+                        "string",
+                        False,
+                        "overview",
+                        "Initial tab to render.",
+                    ),
+                    field(
+                        "hours",
+                        "query",
+                        "int",
+                        False,
+                        "168",
+                        "Initial time window in hours.",
+                    ),
+                ],
+            ),
             endpoint(
                 id="admin-ai-usage",
                 method="GET",
@@ -519,6 +613,53 @@ four ways to sign requests.
                 auth="admin",
                 destructive=True,
             ),
+            endpoint(
+                id="admin-gateway-quota-rules",
+                method="GET",
+                path="/admin/gateway/quota-rules",
+                title="List AI gateway quota rules",
+                summary=(
+                    "List every rolling-24h USD quota rule on /openai/v1/*, each scoped by any "
+                    "combination of role, specific user uid, and app_reference label, plus the "
+                    "global per-role default caps that apply when no rule matches."
+                ),
+                auth="admin",
+                interactive=True,
+            ),
+            endpoint(
+                id="admin-gateway-quota-rule-set",
+                method="POST",
+                path="/admin/gateway/quota-rules",
+                title="Create or update an AI gateway quota rule",
+                summary=(
+                    "Caps rolling-24h USD spend on /openai/v1/*. At least one of owner_kind, "
+                    "owner_id, app_reference must be set; leaving a dimension blank makes it a "
+                    "wildcard, and the most specific active match wins over other rules and over "
+                    "the global default. Pass uid to update an existing rule."
+                ),
+                auth="admin",
+                params=[
+                    field("uid", "json", "string", False, "", "Existing rule uid to update; omit to create a new rule."),
+                    field("owner_kind", "json", "string", False, "user", "internal, key, user, admin, or anonymous. Blank = any role."),
+                    field("owner_id", "json", "string", False, "", "Specific user uid. Blank = any caller of the matched role."),
+                    field("app_reference", "json", "string", False, "devplace-bots-v-1-0-0", "App label (the X-App-Reference header). Blank = any app."),
+                    field("limit_usd", "json", "number", True, "2.5", "Rolling 24h USD cap. 0 = unlimited."),
+                    field("is_active", "json", "boolean", False, "true", "Whether the rule is enforced."),
+                    field("label", "json", "string", False, "", "Optional admin-facing note."),
+                ],
+            ),
+            endpoint(
+                id="admin-gateway-quota-rule-delete",
+                method="DELETE",
+                path="/admin/gateway/quota-rules/{uid}",
+                title="Delete an AI gateway quota rule",
+                summary="Delete a quota rule; callers it covered fall back to the next most specific rule or the global default.",
+                auth="admin",
+                destructive=True,
+                params=[
+                    field("uid", "path", "string", True, "RULE_UID", "Quota rule uid."),
+                ],
+            ),
             endpoint(
                 id="admin-bots-monitor",
                 method="GET",
@@ -736,5 +877,37 @@ four ways to sign requests.
                 params=[field("uid", "path", "string", True, "", "Schedule uid.")],
                 sample_response={"ok": True, "redirect": "/admin/backups"},
             ),
+            endpoint(
+                id="admin-game",
+                method="GET",
+                path="/admin/game",
+                title="Code Farm Era management",
+                summary="View the current Code Farm Era status.",
+                auth="admin",
+                sample_response={"era_active": False, "era_name": ""},
+            ),
+            endpoint(
+                id="admin-game-era-start",
+                method="POST",
+                path="/admin/game/era/start",
+                title="Start an Era",
+                summary="Start a new Code Farm Era: resets every farm's visible Era coins/harvests counters to zero. Real coins, prestige, stars, Legacy, and Mastery are never touched.",
+                auth="admin",
+                params=[
+                    field("name", "form", "string", True, "Genesis", "Era name."),
+                    field("duration_days", "form", "int", False, "28", "Planned Era length in days."),
+                ],
+                sample_response={"ok": True, "redirect": "/admin/game"},
+            ),
+            endpoint(
+                id="admin-game-era-end",
+                method="POST",
+                path="/admin/game/era/end",
+                title="End the running Era",
+                summary="End the current Era: ranks every participating farm by Era score, awards Stars to the top 10 (and an Era-exclusive cosmetic when available), and permanently records the results.",
+                auth="admin",
+                destructive=True,
+                sample_response={"ok": True, "redirect": "/admin/game"},
+            ),
         ],
 }

devplacepy/docs_api/groups/game.py

@@ -12,6 +12,9 @@ The Code Farm is a cooperative idle game. Each member owns a farm of plots, plan
 projects that build over real time, harvests them for coins and XP, upgrades their CI tier for
 faster builds, and waters other members' growing builds to speed them up and earn coins.
 
+Refactoring (prestige) costs a dynamic coin fee that grows with prestige and current wealth;
+the fees fill a community treasury from which active low-balance farms can claim a weekly grant.
+
 All endpoints negotiate HTML or JSON. The action endpoints return the full farm state so a
 client can refresh without a second request.
 """,
@@ -50,9 +53,10 @@ client can refresh without a second request.
                 method="GET",
                 path="/game/leaderboard",
                 title="Farm leaderboard",
-                summary="Top farmers ranked by level, XP, and harvests.",
+                summary="Top farmers on a chosen board: score (default), prestige, harvests (this week), raids (avg coins per successful raid, min 3 raids), time_to_kernel, fair_play, or era (current Era only, empty when none is running).",
                 auth="public",
-                sample_response={"entries": [{"rank": 1, "username": "alice", "level": 4}]},
+                params=[field("board", "query", "string", False, "score", "Leaderboard board key.")],
+                sample_response={"entries": [{"rank": 1, "username": "alice", "level": 4, "score": 1000}]},
             ),
             endpoint(
                 id="game-view-farm",
@@ -166,9 +170,12 @@ client can refresh without a second request.
                 method="POST",
                 path="/game/quests/claim",
                 title="Claim a quest",
-                summary="Claim a completed daily quest reward by its kind.",
+                summary="Claim a completed daily quest, or (with scope=weekly, requires the Legacy Contracts Mastery upgrade) the weekly contract, which pays Stars plus a temporary coin boost instead of coins/XP.",
                 auth="user",
-                params=[field("quest", "form", "string", True, "harvest", "Quest kind.")],
+                params=[
+                    field("quest", "form", "string", True, "harvest", "Quest kind."),
+                    field("scope", "form", "string", False, "daily", "daily (default) or weekly."),
+                ],
                 sample_response={"ok": True, "farm": {"coins": 130}},
             ),
             endpoint(
@@ -176,20 +183,78 @@ client can refresh without a second request.
                 method="POST",
                 path="/game/prestige",
                 title="Refactor (prestige)",
-                summary="Reset the farm at level 10+ for a permanent +25% coin bonus and earn Stars to spend on Legacy upgrades.",
+                summary="Reset the farm at level 10+ for a permanent +25% coin bonus and earn Stars to spend on Legacy upgrades. Refactoring costs a coin fee that scales with prestige and current wealth (the farm state's refactor_cost); the fee funds the community treasury and a fraction of the remaining coins (10% base, more with the Golden Parachute Legacy upgrade) carries over. From prestige 50 onward, every 10 more prestige also earns a permanent Mastery point.",
                 auth="user",
                 destructive=True,
-                sample_response={"ok": True, "farm": {"prestige": 1}},
+                sample_response={"ok": True, "farm": {"prestige": 1, "coins": 6550}},
+            ),
+            endpoint(
+                id="game-grant",
+                method="POST",
+                path="/game/grant",
+                title="Claim the community grant",
+                summary="Claim the weekly community grant, paid from the treasury filled by refactor fees. Eligible farms are active (5+ harvests this week), below 10000 coins, and at most prestige 5.",
+                auth="user",
+                sample_response={"ok": True, "farm": {"coins": 2550}},
             ),
             endpoint(
                 id="game-legacy",
                 method="POST",
                 path="/game/legacy",
                 title="Buy a Legacy upgrade",
-                summary="Spend Stars on a permanent Legacy upgrade that survives every refactor: autoharvest, multiplier, speed, plots, or defense.",
+                summary="Spend Stars on a permanent Legacy upgrade that survives every refactor: autoharvest, multiplier, speed, plots, defense, or carryover (Golden Parachute, raises the refactor coin carry-over).",
                 auth="user",
                 params=[field("key", "form", "string", True, "multiplier", "Legacy upgrade key.")],
                 sample_response={"ok": True, "farm": {"stars": 1}},
             ),
+            endpoint(
+                id="game-mastery",
+                method="POST",
+                path="/game/mastery",
+                title="Buy a Mastery upgrade",
+                summary="Spend Mastery points (earned every 10 prestige past 50) on a permanent Mastery upgrade: autoreplant, analytics, or contracts.",
+                auth="user",
+                params=[field("key", "form", "string", True, "autoreplant", "Mastery upgrade key.")],
+                sample_response={"ok": True, "farm": {"mastery_points": 0}},
+            ),
+            endpoint(
+                id="game-infrastructure-buy",
+                method="POST",
+                path="/game/infrastructure/buy",
+                title="Buy Infrastructure",
+                summary="Buy a permanent, expensive, prestige-gated Infrastructure building: registry (faster rare crops), canary (double/refund harvest chance), or observability (raises the minimum you keep when raided).",
+                auth="user",
+                params=[field("key", "form", "string", True, "registry", "Infrastructure key.")],
+                sample_response={"ok": True, "farm": {"coins": 0}},
+            ),
+            endpoint(
+                id="game-defense-upgrade",
+                method="POST",
+                path="/game/defense/upgrade",
+                title="Upgrade Defense",
+                summary="Buy the next Defense tier. Reduces raid losses and adds steal grace, but adds an ongoing daily coin upkeep (proportional to your coin balance) - if unpaid, the tier decays.",
+                auth="user",
+                sample_response={"ok": True, "farm": {"defense_level": 1}},
+            ),
+            endpoint(
+                id="game-cosmetics-buy",
+                method="POST",
+                path="/game/cosmetics/buy",
+                title="Buy a cosmetic",
+                summary="Buy a purely cosmetic title or plot skin with coins. No gameplay effect.",
+                auth="user",
+                params=[field("key", "form", "string", True, "title_architect", "Cosmetic key.")],
+                sample_response={"ok": True, "farm": {"coins": 0}},
+            ),
+            endpoint(
+                id="game-cosmetics-equip",
+                method="POST",
+                path="/game/cosmetics/equip",
+                title="Equip a title",
+                summary="Equip an owned title cosmetic so it shows next to your name on the leaderboard.",
+                auth="user",
+                params=[field("key", "form", "string", True, "title_architect", "An owned title cosmetic key.")],
+                sample_response={"ok": True, "farm": {"active_title": "title_architect"}},
+            ),
         ],
 }

devplacepy/docs_api/groups/gateway.py

@@ -5,7 +5,6 @@ from .._shared import endpoint, field
 GROUP = {
         "slug": "gateway",
         "title": "OpenAI Gateway",
-        "admin": True,
         "intro": """
 # OpenAI Gateway
 
@@ -32,6 +31,31 @@ The gateway additionally serves **text embeddings** at `/openai/v1/embeddings`.
 generic model `molodetz~embed`, which the gateway maps to the configured embedding model (OpenRouter's
 Qwen3 8B embedding model by default). Usage and cost are tracked per call exactly like chat and vision.
 
+The gateway also serves **image generation** at `/openai/v1/images/generations`. Clients request the
+generic model `molodetz-img-small`, which the gateway maps to the configured image model (OpenRouter's
+Flux 1.1 Pro by default). Cost is tracked per call with a flat per-image price when the upstream
+returns no native cost.
+
+## Quick start
+
+Copy the command below and paste it into a terminal. If you are signed in the `{{ api_key }}`
+and `{{ app_reference }}` placeholders are already filled in with your own values; otherwise
+replace them with the API key from your [profile](/profile) page and any application identifier.
+
+```bash
+curl -X POST "{{ base }}/openai/v1/chat/completions" \
+  -H "Authorization: Bearer {{ api_key }}" \
+  -H "X-App-Reference: {{ app_reference }}" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "molodetz",
+    "messages": [{"role": "user", "content": "Hello, how are you?"}]
+  }'
+```
+
+The response carries `X-Gateway-*` headers with token counts and dollar cost for the call.
+For streaming, add `"stream": true` to the JSON body.
+
 ## Model routing and providers
 
 On top of the single default upstream above, an administrator can register additional named
@@ -60,7 +84,7 @@ and dollar cost directly from the response with no extra request:
 | Header | Meaning |
 |--------|---------|
 | `X-Gateway-Model` | Upstream model actually used for the call |
-| `X-Gateway-Backend` | Backend that served it: `chat`, `embed`, or passthrough |
+| `X-Gateway-Backend` | Backend that served it: `chat`, `embed`, `image`, or passthrough |
 | `X-Gateway-Prompt-Tokens` | Input (prompt) tokens |
 | `X-Gateway-Completion-Tokens` | Output (completion) tokens |
 | `X-Gateway-Total-Tokens` | Total tokens (prompt + completion) |
@@ -83,7 +107,18 @@ and dollar cost directly from the response with no extra request:
 Dollar costs use the upstream's native `cost` field when it returns one
 (`X-Gateway-Cost-Native: 1`); otherwise they are computed from the per-million prices of the matched
 model route, falling back to the prices configured on the `openai` service when no route matches. The
-one denied path that makes no upstream call (embeddings disabled) returns no usage headers.
+denied paths that make no upstream call (embeddings or image generation disabled) return no usage headers.
+
+## Request header `X-App-Reference`
+
+Clients **SHOULD** send an `X-App-Reference` header to identify themselves for cost attribution.
+The value is a free-form slug (max 30 characters, letters, digits, `_`, `.`, `-`). When missing or
+invalid, the gateway defaults to `default`. The value is recorded in every usage ledger row and can
+be queried alongside owner-kind and owner-id to attribute spending per application.
+
+```
+X-App-Reference: devplace-devii-v-1-0-0
+```
 
 Administrators enable and configure this gateway under [Background Services](/docs/services.html)
 (the `openai` service).
@@ -108,7 +143,7 @@ for signing DevPlace's own requests.
                         "string",
                         False,
                         "gpt-4o-mini",
-                        "Model id. When it matches a configured model route the gateway forwards to that route's provider and upstream model; otherwise it uses the default upstream model.",
+                        "Model id. When it matches a configured model route the gateway forwards to that route's provider and upstream model; otherwise it falls back to the configured default upstream model.",
                     ),
                     field(
                         "messages",
@@ -170,6 +205,55 @@ for signing DevPlace's own requests.
                 notes=[
                     "Returns `503` when the gateway service is not running or embeddings are disabled.",
                     "Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
+                    "If `model` matches a configured embed model route it is forwarded to that route's provider and upstream model; otherwise it falls back to the configured default embedding model.",
+                ],
+            ),
+            endpoint(
+                id="gateway-images",
+                method="POST",
+                path="/openai/v1/images/generations",
+                title="Image generation",
+                summary="OpenAI-compatible image generation. Request model molodetz-img-small.",
+                auth="user",
+                encoding="json",
+                params=[
+                    field(
+                        "model",
+                        "json",
+                        "string",
+                        False,
+                        "molodetz-img-small",
+                        "Image model id; the gateway maps molodetz-img-small to the configured model, or to a matching image model route's provider and target model.",
+                    ),
+                    field(
+                        "prompt",
+                        "json",
+                        "string",
+                        True,
+                        '"a decorative developer award emblem"',
+                        "Text prompt describing the image to generate.",
+                    ),
+                    field(
+                        "size",
+                        "json",
+                        "string",
+                        False,
+                        "512x512",
+                        "Output dimensions (provider-dependent).",
+                    ),
+                    field(
+                        "response_format",
+                        "json",
+                        "string",
+                        False,
+                        "b64_json",
+                        "Return format: url or b64_json.",
+                    ),
+                ],
+                notes=[
+                    "Returns `503` when the gateway service is not running or image generation is disabled.",
+                    "Every response carries the `X-Gateway-*` token and dollar-cost headers (see Per-call cost and usage headers above).",
+                    "If `model` matches a configured image model route it is forwarded to that route's provider and upstream model; otherwise it falls back to the configured default image model.",
                 ],
             ),
             endpoint(

devplacepy/docs_api/groups/messaging.py

@@ -57,9 +57,9 @@ four ways to sign requests.
                         "content",
                         "form",
                         "textarea",
-                        True,
+                        False,
                         "Hello there.",
-                        "Body, 1-2000 characters.",
+                        "Body, 0-2000 characters. May be empty when at least one attachment is provided.",
                     ),
                     field(
                         "receiver_uid",
@@ -71,5 +71,38 @@ four ways to sign requests.
                     ),
                 ],
             ),
+            endpoint(
+                id="messages-conversations",
+                method="GET",
+                path="/messages/conversations",
+                title="List conversations",
+                summary="Return the signed-in user's conversation list as JSON, for live refresh without a full page reload.",
+                auth="user",
+                interactive=False,
+                sample_response={
+                    "conversations": [
+                        {
+                            "other_user": {"uid": "8f14e45f-...", "username": "alice_test"},
+                            "last_message": "Hello there.",
+                            "last_message_at": "2026-07-21T10:00:00+00:00",
+                            "unread": True,
+                        }
+                    ]
+                },
+            ),
+            endpoint(
+                id="messages-ws-ticket",
+                method="POST",
+                path="/messages/ws-ticket",
+                title="Issue a WebSocket ticket",
+                summary="Exchange the caller's session/API-key auth for a short-lived, single-use ticket that a browser WebSocket handshake can carry as a query parameter (a native WebSocket cannot set custom auth headers).",
+                auth="user",
+                encoding="none",
+                interactive=False,
+                notes=[
+                    "The ticket is valid for 30 seconds and can be redeemed exactly once, as `wss://.../messages/ws?ticket=<ticket>`.",
+                ],
+                sample_response={"ticket": "3f9c2a...", "expires_in": 30},
+            ),
         ],
 }

devplacepy/docs_api/groups/profiles.py

@@ -32,7 +32,7 @@ four ways to sign requests.
                         False,
                         "posts",
                         "Profile tab.",
-                        ["posts", "activity", "followers", "following", "media"],
+                        ["posts", "activity", "followers", "following", "media", "awards"],
                     ),
                 ],
             ),
@@ -60,7 +60,7 @@ four ways to sign requests.
                         False,
                         "posts",
                         "Profile tab.",
-                        ["posts", "activity", "followers", "following", "media"],
+                        ["posts", "activity", "followers", "following", "media", "awards"],
                     ),
                 ],
             ),
@@ -136,7 +136,7 @@ four ways to sign requests.
                         "textarea",
                         False,
                         "Leave literary as is, only do punctuation and casing",
-                        "Correction instruction, up to 2000 characters.",
+                        "Correction instruction, up to 20000 characters.",
                     ),
                 ],
                 sample_response={
@@ -150,6 +150,53 @@ four ways to sign requests.
                     },
                 },
             ),
+            endpoint(
+                id="profile-interactions",
+                method="POST",
+                path="/profile/{username}/interactions",
+                title="Configure Devii interactive widgets",
+                summary="Enable or disable CA-IWP interactive prompts (ui_prompt) for this account, or reset to the administrator default. Guests always use the site default. Admins may target any user.",
+                auth="user",
+                encoding="form",
+                destructive=True,
+                params=[
+                    field(
+                        "username",
+                        "path",
+                        "string",
+                        True,
+                        "bob_test",
+                        "Profile owner. Must be yourself unless you are an admin.",
+                    ),
+                    field(
+                        "enabled",
+                        "form",
+                        "boolean",
+                        False,
+                        "true",
+                        "true to enable interactive widgets, false to disable. Ignored when reset is true.",
+                    ),
+                    field(
+                        "reset",
+                        "form",
+                        "boolean",
+                        False,
+                        "false",
+                        "true to clear the user override and inherit the administrator default.",
+                    ),
+                ],
+                sample_response={
+                    "ok": True,
+                    "redirect": "/profile/bob_test",
+                    "data": {
+                        "url": "/profile/bob_test",
+                        "enabled": True,
+                        "source": "user",
+                        "default": True,
+                        "override": True,
+                    },
+                },
+            ),
             endpoint(
                 id="profile-ai-modifier",
                 method="POST",
@@ -190,7 +237,7 @@ four ways to sign requests.
                         "textarea",
                         False,
                         "Execute what is behind `@ai` (the prompt) and replace that part including `@ai`",
-                        "Modifier instruction, up to 2000 characters.",
+                        "Modifier instruction, up to 20000 characters.",
                     ),
                 ],
                 sample_response={
@@ -254,6 +301,40 @@ four ways to sign requests.
                 ],
                 sample_response={"api_key": "NEW_UUID"},
             ),
+            endpoint(
+                id="profile-give-award",
+                method="POST",
+                path="/profile/{username}/award",
+                title="Give a member an award",
+                summary="Create a pending award on another member's profile and enqueue image generation.",
+                auth="user",
+                encoding="json",
+                params=[
+                    field(
+                        "username",
+                        "path",
+                        "string",
+                        True,
+                        "{{ username }}",
+                        "Receiver username.",
+                    ),
+                    field(
+                        "description",
+                        "json",
+                        "string",
+                        True,
+                        "Great work on the release!",
+                        "Award message (1-125 characters).",
+                    ),
+                ],
+                sample_response={
+                    "ok": True,
+                    "data": {
+                        "award_uid": "AWARD_UID",
+                        "award_slug": "abc123-great-work",
+                    },
+                },
+            ),
             endpoint(
                 id="profile-regenerate-avatar",
                 method="POST",
@@ -650,6 +731,37 @@ four ways to sign requests.
                 auth="public",
                 interactive=True,
             ),
+            endpoint(
+                id="award-image",
+                method="GET",
+                path="/awards/{slug_or_uid}/{size}",
+                title="Award image redirect",
+                summary="Redirect to the stored PNG attachment for a published award.",
+                auth="public",
+                params=[
+                    field(
+                        "slug_or_uid",
+                        "path",
+                        "string",
+                        True,
+                        "abc123-great-work",
+                        "Award slug or bare uid.",
+                    ),
+                    field(
+                        "size",
+                        "path",
+                        "enum",
+                        True,
+                        "256",
+                        "Image size.",
+                        ["512", "256", "64"],
+                    ),
+                ],
+                notes=[
+                    "> Pending or revoked awards return 404.",
+                    "> Response includes long-lived cache headers.",
+                ],
+            ),
             endpoint(
                 id="avatar",
                 method="GET",

devplacepy/docs_api/render.py

@@ -43,5 +43,6 @@ def render_group(slug, base, username, api_key):
         "{{ base }}": base,
         "{{ username }}": username or "YOUR_USERNAME",
         "{{ api_key }}": api_key or "YOUR_API_KEY",
+        "{{ app_reference }}": f"user-{username}-app-v-1-0-0" if username else "user-app-v-13.37.0",
     }
     return _substitute(group, replacements)

devplacepy/docs_devrant.py

@@ -237,7 +237,7 @@ DEVRANT_GROUPS = {
             encoding="form",
             params=[
                 field("rant_id", "path", type="int", required=True, example="1", description="Rant id."),
-                field("comment", "body", type="textarea", required=True, example="Great rant!", description="Comment text, 1-1000 chars."),
+                field("comment", "body", type="textarea", required=True, example="Great rant!", description="Comment text, 1-125000 chars."),
             ],
             sample_response={"success": True},
         ),

devplacepy/main.py

@@ -37,8 +37,10 @@ from devplacepy.database import (
     get_user_post_count,
     get_user_stars,
     get_blocked_uids,
+    get_top_authors,
+    get_trending_topics,
 )
-from devplacepy.templating import templates
+from devplacepy.templating import templates, jinja_unread_count
 from devplacepy.cache import TTLCache
 from devplacepy.responses import respond, wants_json, json_error
 from devplacepy.schemas import LandingOut, ValidationErrorOut
@@ -56,6 +58,7 @@ from devplacepy.routers import (
     notifications,
     votes,
     avatar,
+    awards,
     follow,
     relations,
     admin,
@@ -95,6 +98,7 @@ from devplacepy.services.jobs.issue_create_service import IssueCreateService
 from devplacepy.services.jobs.planning_service import PlanningReportService
 from devplacepy.services.jobs.seo.service import SeoService
 from devplacepy.services.jobs.seo_meta_service import SeoMetaService
+from devplacepy.services.jobs.award_service import AwardService
 from devplacepy.services.backup import BackupService
 from devplacepy.services.dbapi.service import DbApiJobService
 from devplacepy.services.pubsub import PubSubService
@@ -255,6 +259,7 @@ async def lifespan(app: FastAPI):
     service_manager.register(ForkService())
     service_manager.register(SeoService())
     service_manager.register(SeoMetaService())
+    service_manager.register(AwardService())
     service_manager.register(BackupService())
     service_manager.register(DbApiJobService())
     service_manager.register(PubSubService())
@@ -281,9 +286,15 @@ async def lifespan(app: FastAPI):
             logger.info(
                 f"Worker pid {os.getpid()} declined service lock; another worker owns background services"
             )
+    from devplacepy.services.statistics.tracking import start_visit_flusher
+
+    start_visit_flusher()
     logger.info(f"DevPlace started on port {PORT}")
     yield
     logger.info("Shutting down services...")
+    from devplacepy.services.statistics.tracking import flush_visits
+
+    flush_visits()
     await service_manager.shutdown_all()
     await background.stop()
 
@@ -428,6 +439,7 @@ app.include_router(reactions.router, prefix="/reactions")
 app.include_router(bookmarks.router, prefix="/bookmarks")
 app.include_router(polls.router, prefix="/polls")
 app.include_router(avatar.router, prefix="/avatar")
+app.include_router(awards.router, prefix="/awards")
 app.include_router(follow.router, prefix="/follow")
 app.include_router(relations.router)
 app.include_router(leaderboard.router, prefix="/leaderboard")
@@ -455,7 +467,8 @@ app.include_router(game.router, prefix="/game")
 
 @app.middleware("http")
 async def refresh_db_snapshot(request: Request, call_next):
-    refresh_snapshot()
+    if not request.url.path.startswith(("/static", "/avatar")):
+        refresh_snapshot()
     return await call_next(request)
 
 
@@ -580,6 +593,15 @@ async def track_presence(request: Request, call_next):
     return await call_next(request)
 
 
+@app.middleware("http")
+async def visit_statistics(request: Request, call_next):
+    from devplacepy.services.statistics.tracking import track_visit
+
+    response = await call_next(request)
+    track_visit(request, response.status_code)
+    return response
+
+
 @app.middleware("http")
 async def response_timing(request: Request, call_next):
     start = time.perf_counter()
@@ -589,7 +611,7 @@ async def response_timing(request: Request, call_next):
     return response
 
 
-app.add_middleware(GZipMiddleware, minimum_size=512, compresslevel=6)
+app.add_middleware(GZipMiddleware, minimum_size=512, compresslevel=5)
 
 
 _home_cache = TTLCache(ttl=int(os.environ.get("DEVPLACE_HOME_CACHE_TTL", "60")), max_size=4)
@@ -681,6 +703,14 @@ async def landing(request: Request):
         breadcrumbs=[],
         schemas=[website_schema(base)],
     )
+    user_xp = user.get("xp", 0) or 0 if user else 0
+    user_level = user.get("level", 1) or 1 if user else 1
+    xp_progress_pct = (user_xp % 100) if user_xp else 0
+    unread_count = jinja_unread_count(user["uid"]) if user else 0
+
+    top_contributors = get_top_authors(5) if not blocked else []
+    trending_topics = get_trending_topics(6) if not blocked else []
+
     return respond(
         request,
         "landing.html",
@@ -691,8 +721,14 @@ async def landing(request: Request):
             "is_authenticated": bool(user),
             "user_post_count": get_user_post_count(user["uid"]) if user else 0,
             "user_stars": get_user_stars(user["uid"]) if user else 0,
+            "user_xp": user_xp,
+            "user_level": user_level,
+            "xp_progress_pct": xp_progress_pct,
+            "unread_count": unread_count,
             "landing_articles": landing_articles,
             "landing_posts": landing_posts,
+            "top_contributors": top_contributors,
+            "trending_topics": trending_topics,
         },
         model=LandingOut,
     )

devplacepy/models.py

@@ -4,6 +4,7 @@ import re
 
 from datetime import datetime
 from typing import Literal, Optional
+from urllib.parse import urlsplit
 from pydantic import BaseModel, Field, field_validator, model_validator
 from devplacepy.constants import TOPICS, REACTION_EMOJI
 from devplacepy.config import DEFAULT_CORRECTION_PROMPT, DEFAULT_MODIFIER_PROMPT
@@ -157,7 +158,7 @@ class PostEditForm(BaseModel):
 
 
 class CommentForm(BaseModel):
-    content: str = Field(min_length=3, max_length=1000)
+    content: str = Field(min_length=3, max_length=125000)
     target_uid: str = Field(default="", max_length=36)
     post_uid: str = Field(default="", max_length=36)
     target_type: Literal["post", "project", "news", "issue", "gist"] = "post"
@@ -172,7 +173,7 @@ class CommentForm(BaseModel):
 
 
 class CommentEditForm(BaseModel):
-    content: str = Field(min_length=3, max_length=1000)
+    content: str = Field(min_length=3, max_length=125000)
 
 
 class ProjectForm(BaseModel):
@@ -238,6 +239,10 @@ class CustomizationToggleForm(BaseModel):
     value: bool = False
 
 
+class AwardGiveForm(BaseModel):
+    description: str = Field(min_length=1, max_length=125)
+
+
 class NotificationPrefForm(BaseModel):
     notification_type: str = Field(min_length=1, max_length=40)
     channel: Literal["in_app", "push", "telegram"]
@@ -253,13 +258,18 @@ class NotificationDefaultForm(BaseModel):
 class AiCorrectionForm(BaseModel):
     enabled: bool = False
     sync: bool = False
-    prompt: str = Field(default=DEFAULT_CORRECTION_PROMPT, max_length=2000)
+    prompt: str = Field(default=DEFAULT_CORRECTION_PROMPT, max_length=20000)
 
 
 class AiModifierForm(BaseModel):
     enabled: bool = False
     sync: bool = False
-    prompt: str = Field(default=DEFAULT_MODIFIER_PROMPT, max_length=2000)
+    prompt: str = Field(default=DEFAULT_MODIFIER_PROMPT, max_length=20000)
+
+
+class InteractionsForm(BaseModel):
+    enabled: bool = True
+    reset: bool = False
 
 
 class TelegramPairForm(BaseModel):
@@ -376,9 +386,10 @@ class ContainerScheduleForm(BaseModel):
 
 
 class MessageForm(BaseModel):
-    content: str = Field(min_length=1, max_length=2000)
+    content: str = Field(min_length=0, max_length=2000)
     receiver_uid: str = Field(min_length=1, max_length=36)
     attachment_uids: list[str] = []
+    client_id: Optional[str] = Field(default=None, max_length=64)
 
 
 class ProfileForm(BaseModel):
@@ -551,8 +562,20 @@ class AdminSettingsForm(BaseModel):
     maintenance_mode: str = Field(default="", max_length=1)
     maintenance_message: str = Field(default="", max_length=300)
     docs_search_mode: str = Field(default="", max_length=20)
+    outbound_proxy_url: str = Field(default="", max_length=500)
     extra_head: str = Field(default="", max_length=50000)
 
+    @field_validator("outbound_proxy_url")
+    @classmethod
+    def validate_outbound_proxy_url(cls, value):
+        text = value.strip()
+        if not text:
+            return text
+        parsed = urlsplit(text)
+        if parsed.scheme not in ("http", "https", "socks5", "socks5h") or not parsed.hostname:
+            raise ValueError("Proxy URL must be http(s):// or socks5(h):// with a host, e.g. http://user:pass@host:port")
+        return text
+
 
 class GamePlantForm(BaseModel):
     slot: int = Field(ge=0, le=64)
@@ -569,7 +592,25 @@ class GamePerkForm(BaseModel):
 
 class GameQuestForm(BaseModel):
     quest: str = Field(min_length=1, max_length=40)
+    scope: str = Field(default="daily", min_length=1, max_length=10)
 
 
 class GameLegacyForm(BaseModel):
     key: str = Field(min_length=1, max_length=40)
+
+
+class GameInfraForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameCosmeticForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameMasteryForm(BaseModel):
+    key: str = Field(min_length=1, max_length=40)
+
+
+class GameEraStartForm(BaseModel):
+    name: str = Field(min_length=1, max_length=60)
+    duration_days: int = Field(default=28, ge=1, le=180)

devplacepy/project_files.py

@@ -463,11 +463,15 @@ def delete_node(project_uid: str, raw_path: str, deleted_by: str = "system") ->
     if node is None:
         raise ProjectFileError(f"'{path}' does not exist")
     stamp = _now()
-    for row in _descendants(project_uid, path):
+    rows = _descendants(project_uid, path)
+    for row in rows:
         _table().update(
             {"uid": row["uid"], "deleted_at": stamp, "deleted_by": deleted_by},
             ["uid"],
         )
+    for row in rows:
+        if row.get("is_binary"):
+            _unlink_blob(row)
 
 
 def soft_delete_all_project_files(project_uid: str, deleted_by: str) -> None:
@@ -578,9 +582,11 @@ def _export_node(row: dict, dest: Path) -> None:
     if target.is_symlink():
         target.unlink()
     if row.get("is_binary") and row.get("stored_name") and row.get("directory"):
-        shutil.copyfile(
-            PROJECT_FILES_DIR / row["directory"] / row["stored_name"], target
-        )
+        src = PROJECT_FILES_DIR / row["directory"] / row["stored_name"]
+        try:
+            shutil.copyfile(src, target)
+        except (FileNotFoundError, OSError):
+            logger.warning("Blob file missing during export: %s", src)
     else:
         target.write_text(row.get("content") or "", encoding="utf-8")
 
@@ -685,7 +691,6 @@ def import_from_dir(project_uid: str, src_dir, user: dict, *, skip_names=None) -
 
 
 def export_to_dir(project_uid: str, subpath: str, dest_dir) -> int:
-    _guard_writable(project_uid)
     dest = Path(dest_dir).resolve()
     dest.mkdir(parents=True, exist_ok=True)
     if subpath:
@@ -708,9 +713,12 @@ def export_to_dir(project_uid: str, subpath: str, dest_dir) -> int:
         if target.is_symlink() or target.is_file():
             target.unlink()
         if row.get("is_binary") and row.get("stored_name") and row.get("directory"):
-            shutil.copyfile(
-                PROJECT_FILES_DIR / row["directory"] / row["stored_name"], target
-            )
+            src = PROJECT_FILES_DIR / row["directory"] / row["stored_name"]
+            try:
+                shutil.copyfile(src, target)
+            except (FileNotFoundError, OSError):
+                logger.warning("Blob file missing: %s", src)
+                continue
         else:
             target.write_text(row.get("content") or "", encoding="utf-8")
         written += 1

devplacepy/rendering.py

@@ -45,6 +45,9 @@ def write_emoji_module(path: Path = EMOJI_JS_PATH) -> int:
 
 EMOJI_MAP = build_emoji_shortcodes()
 
+_WIDGET_RE = re.compile(r"<dp-widget>(.*?)</dp-widget>", re.DOTALL | re.IGNORECASE)
+_WIDGET_PH = "\x00WIDGET_{}\x00"
+
 _SHORTCODE_RE = re.compile(r":([A-Za-z0-9_+\-]+):")
 _YOUTUBE_RE = re.compile(
     r"(?:https?://)?(?:www\.)?"
@@ -66,6 +69,9 @@ _YOUTUBE_ALLOW = (
     "gyroscope; picture-in-picture"
 )
 
+_EMAIL_RE = re.compile(r"\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b")
+_EMAIL_KEEP_DOMAIN = "molodetz.nl"
+
 _MEDIA_SKIP_TAGS = {"a", "code", "pre"}
 _TITLE_INLINE_TAGS = {
     "b", "strong", "i", "em", "code", "del", "s", "mark", "sub", "sup", "span", "br",
@@ -102,7 +108,17 @@ _content_markdown = mistune.create_markdown(
 
 
 def _normalize_dashes(text: str) -> str:
-    return text.replace("\u2014", "-")
+    text = text.replace("\u2014", "-")
+    text = text.replace("\u2013", "-")
+    text = text.replace("&mdash;", "-")
+    text = text.replace("&ndash;", "-")
+    text = text.replace("&#8212;", "-")
+    text = text.replace("&#8211;", "-")
+    text = text.replace("&#x2014;", "-")
+    text = text.replace("&#X2014;", "-")
+    text = text.replace("&#x2013;", "-")
+    text = text.replace("&#X2013;", "-")
+    return text
 
 
 def _replace_shortcodes(text: str) -> str:
@@ -140,12 +156,26 @@ def _embed_url(url: str) -> str:
     )
 
 
+def _mask_email(match: re.Match) -> str:
+    email = match.group(0)
+    local, _, domain = email.partition("@")
+    lowered = domain.lower()
+    if lowered == _EMAIL_KEEP_DOMAIN or lowered.endswith("." + _EMAIL_KEEP_DOMAIN):
+        return email
+    reveal = max(1, len(local) - round(len(local) * 0.8))
+    return f"{local[:reveal]}{'*' * (len(local) - reveal)}@{domain}"
+
+
+def _mask_emails(text: str) -> str:
+    return _EMAIL_RE.sub(_mask_email, text)
+
+
 def _transform_text(text: str) -> str:
     out: list[str] = []
     pos = 0
     for match in _TOKEN_RE.finditer(text):
         if match.start() > pos:
-            out.append(html.escape(text[pos:match.start()]))
+            out.append(html.escape(_mask_emails(text[pos:match.start()])))
         if match.group("url"):
             out.append(_embed_url(match.group("url")))
         else:
@@ -156,7 +186,7 @@ def _transform_text(text: str) -> str:
             )
         pos = match.end()
     if pos < len(text):
-        out.append(html.escape(text[pos:]))
+        out.append(html.escape(_mask_emails(text[pos:])))
     return "".join(out)
 
 
@@ -192,7 +222,7 @@ class _MediaProcessor(HTMLParser):
 
     def handle_data(self, data: str) -> None:
         if self._skip_depth > 0:
-            self._out.append(html.escape(data))
+            self._out.append(html.escape(_mask_emails(data)))
         else:
             self._out.append(_transform_text(data))
 
@@ -218,7 +248,7 @@ class _InlineFilter(HTMLParser):
             self._out.append(f"</{tag}>")
 
     def handle_data(self, data: str) -> None:
-        self._out.append(html.escape(data))
+        self._out.append(html.escape(_mask_emails(data)))
 
     def result(self) -> str:
         return "".join(self._out).strip()
@@ -252,16 +282,42 @@ def _render_title(text: str) -> str:
     return _keep_inline(_content_markdown(text))
 
 
-def render_content(text) -> Markup:
-    if not text:
-        return Markup("")
-    return Markup(_render_content(str(text)))
+def _extract_widgets(text: str) -> tuple[str, list[str]]:
+    widgets: list[str] = []
+    def _replacer(m: re.Match) -> str:
+        widgets.append(m.group(1))
+        return _WIDGET_PH.format(len(widgets) - 1)
+    return _WIDGET_RE.sub(_replacer, text), widgets
 
 
-def render_title(text) -> Markup:
+def _reinsert_widgets(text: str, widgets: list[str]) -> str:
+    for i, widget in enumerate(widgets):
+        text = text.replace(_WIDGET_PH.format(i), widget)
+    return text
+
+
+def render_content(text, author_is_admin: bool = False) -> Markup:
     if not text:
         return Markup("")
-    return Markup(_render_title(str(text)))
+    text_str = str(text)
+    if author_is_admin and _WIDGET_RE.search(text_str):
+        modified, widgets = _extract_widgets(text_str)
+        rendered = _render_content(modified)
+        result = _reinsert_widgets(rendered, widgets)
+        return Markup(result)
+    return Markup(_render_content(text_str))
+
+
+def render_title(text, author_is_admin: bool = False) -> Markup:
+    if not text:
+        return Markup("")
+    text_str = str(text)
+    if author_is_admin and _WIDGET_RE.search(text_str):
+        modified, widgets = _extract_widgets(text_str)
+        rendered = _render_title(modified)
+        result = _reinsert_widgets(rendered, widgets)
+        return Markup(result)
+    return Markup(_render_title(text_str))
 
 
 def content_preview(text, length: int = 60) -> str:

devplacepy/routers/CLAUDE.md

@@ -14,8 +14,8 @@ Prefixes are wired in `main.py`:
 | `/comments` | comments.py |
 | `/projects` | projects/ package - `index.py` (listing/detail/create/delete plus owner-only visibility toggles `POST /{slug}/private` and `POST /{slug}/readonly`, and async `POST /{slug}/fork`), `files.py`, and the `containers/` subpackage (below). `main.py` mounts the whole `/projects` tree from this one package. See `routers/projects/CLAUDE.md` for the deep detail on this tree |
 | `/projects/{slug}/files` | projects/files.py - per-project virtual filesystem (CRUD dirs/files, upload, inline edit, plus line-range ops: `lines` read, `replace-lines`, `insert-lines`, `delete-lines`, `append` for large text files) |
-| `/profile` | profile/ package - `index.py` (page/search/followers/update/api-key; the **Posts** tab shows only the 10 most recent posts, while the sidebar `posts_count` stat always uses the real `count()` so it is never capped to the rendered 10), `customization.py`, `notifications.py`, `ai_correction.py` (`POST /{username}/ai-correction`, owner-or-admin AI content correction toggle + prompt), `ai_modifier.py` (`POST /{username}/ai-modifier`, owner-or-admin AI modifier toggle + prompt), `telegram.py` (`POST /{username}/telegram`, owner-or-admin Telegram pairing-code request/unpair), `usage.py` (`_ai_quota`, re-exported from the package) |
-| `/messages` | messages.py - real-time DM chat. `GET ""` (page; the conversation sidebar is one windowed SQL aggregate and an opened thread loads only the newest `CONVERSATION_MESSAGE_LIMIT` = 500 messages - keep these queries bounded), `GET /search`, `POST /send` (no-JS fallback), and `WS /messages/ws` (NOT lock-owner gated - accepts on every worker; closes `1008` for guests). The WS adds live bidirectional delivery, typing, read receipts, and in-process presence on top of the existing `messages` table; both the WS `send` and HTTP `POST /send` route through the shared `services/messaging/persist.py` `persist_message` choke point (identical audit/notification/mention) and then through `_finalize_and_broadcast`, which awaits any pending SYNC AI correction/modifier futures (DM content runs through both, so typing `@ai <instruction>` executes live) and broadcasts the FINAL corrected/modified content (the WS path passes `request=websocket` and awaits directly since HTTP middleware does not run for websockets; the frontend swaps the sender's optimistic bubble to the authoritative content). Same-worker delivery is instant via the per-worker `message_hub`; cross-worker delivery is filled by `services/messaging/relay.py` `message_relay` (per-worker ~1s DB poll on the `messages.id` watermark, deduped against the hub's delivered set). See `devplacepy/services/messaging/CLAUDE.md` |
+| `/profile` | profile/ package - `index.py` (page/search/followers/update/api-key; the **Posts** tab shows only the 10 most recent posts, while the sidebar `posts_count` stat always uses the real `count()` so it is never capped to the rendered 10), `customization.py`, `notifications.py`, `ai_correction.py` (`POST /{username}/ai-correction`, owner-or-admin AI content correction toggle + prompt), `ai_modifier.py` (`POST /{username}/ai-modifier`, owner-or-admin AI modifier toggle + prompt), `interactions.py` (`POST /{username}/interactions`, owner-or-admin Devii interactive widgets preference; admin default via `devii_interactions_default`), `telegram.py` (`POST /{username}/telegram`, owner-or-admin Telegram pairing-code request/unpair), `usage.py` (`_ai_quota`, re-exported from the package) |
+| `/messages` | messages.py - real-time DM chat. `GET ""` (page; the conversation sidebar is one windowed SQL aggregate and an opened thread loads only the newest `CONVERSATION_MESSAGE_LIMIT` = 500 messages - keep these queries bounded), `GET /search`, `GET /conversations` (flat JSON conversation list, same `get_conversations` helper as the page), `POST /send` (no-JS fallback, also accepts an attachment-only empty-content message), `POST /ws-ticket` (issues a 30s single-use WS auth ticket so a browser WebSocket can authenticate cross-context, since a native `WebSocket` cannot set an `Authorization`/`X-API-KEY` header), and `WS /messages/ws` (NOT lock-owner gated - accepts on every worker; closes `1008` for guests). The WS adds live bidirectional delivery, typing, and read receipts on top of the existing `messages` table - presence is NOT part of this WS (it is the generic `PresenceManager`/pub/sub mechanism, see `devplacepy/services/CLAUDE.md`). Both the WS `send` and HTTP `POST /send` route through the shared `services/messaging/persist.py` `persist_message` choke point (identical audit/notification/mention) and then through `_finalize_and_broadcast`, which awaits any pending SYNC AI correction/modifier futures (DM content runs through both, so typing `@ai <instruction>` executes live), broadcasts the FINAL corrected/modified content with an additive `ai_processed` frame flag (the WS path passes `request=websocket` and awaits directly since HTTP middleware does not run for websockets; the frontend swaps the sender's optimistic bubble to the authoritative content). Same-worker delivery is instant via the per-worker `message_hub`; cross-worker delivery is filled by `services/messaging/relay.py` `message_relay` (per-worker ~1s DB poll on the `messages.id` watermark, deduped against the hub's delivered set). See `devplacepy/services/messaging/CLAUDE.md` |
 | `/notifications` | notifications.py |
 | `/votes` | votes.py |
 | `/reactions` | reactions.py - emoji reaction toggle on any target: `POST /reactions/{target_type}/{target_uid}` (optimistic `ReactionBar`) |
@@ -25,7 +25,7 @@ Prefixes are wired in `main.py`:
 | `/follow` | follow.py |
 | (none) | relations.py - per-user block/mute relations: `POST /block/{username}`, `/block/unblock/{username}`, `/mute/{username}`, `/mute/unmute/{username}` (soft-deletable `user_relations` rows) |
 | `/leaderboard` | leaderboard.py - `GET /leaderboard` XP/stars leaderboard page |
-| `/admin` | admin/ package - one leaf per sub-resource (`index`, `users`, `aiusage`, `aiquota`, `media`, `trash`, `settings`, `notifications`, `news`, `auditlog`, `backups`) plus the folded-in `services.py` and `containers.py` (mounted with `/services` and `/containers` sub-prefixes). `main.py` mounts the whole `/admin` tree from this one package. The `backups` leaf is the admin **Backups** dashboard (`BackupService`, kind `backup`): storage usage, backup archives, and interval/cron backup schedules (CRUD + rotation). **Archive download is restricted to the primary administrator** (the earliest-created Admin, resolved by `database.get_primary_admin_uid` / `utils.is_primary_admin`): `GET /admin/backups/{uid}/download` 403s every other admin, the `download_url` field is withheld from them at every endpoint (`can_download = is_primary_admin(admin)`, surfaced as `BackupDashboardOut.can_download_backups`), and `BackupMonitor.js` renders their Download control as a disabled button tooltipped `Not available`. See `devplacepy/services/backup/CLAUDE.md` |
+| `/admin` | admin/ package - one leaf per sub-resource (`index`, `users`, `aiusage`, `aiquota`, `media`, `trash`, `settings`, `notifications`, `news`, `auditlog`, `backups`, `game`) plus the folded-in `services.py` and `containers.py` (mounted with `/services` and `/containers` sub-prefixes). `main.py` mounts the whole `/admin` tree from this one package. The `backups` leaf is the admin **Backups** dashboard (`BackupService`, kind `backup`): storage usage, backup archives, and interval/cron backup schedules (CRUD + rotation). **Archive download is restricted to the primary administrator** (the earliest-created Admin, resolved by `database.get_primary_admin_uid` / `utils.is_primary_admin`): `GET /admin/backups/{uid}/download` 403s every other admin, the `download_url` field is withheld from them at every endpoint (`can_download = is_primary_admin(admin)`, surfaced as `BackupDashboardOut.can_download_backups`), and `BackupMonitor.js` renders their Download control as a disabled button tooltipped `Not available`. See `devplacepy/services/backup/CLAUDE.md`. The `game` leaf (`/admin/game`) is the Code Farm Era admin page: `GET /admin/game` (status), `POST /admin/game/era/start` and `/era/end` - see `devplacepy/services/game/CLAUDE.md` |
 | `/admin/services` | admin/services.py |
 | `/issues` | issues/ package - issue tracker backed by Gitea (no local issue store): `index.py` (list `?state=`/`?page=`, detail `/{number}` with comments), `create.py` (async AI-enhanced filing `/create` enqueues a `issue_create` job, status at `/jobs/{uid}`), `comment.py` (synchronous, pushes to Gitea + notifies admins), `status.py` (admin open/closed), `attachments.py` (file attachments on open issues + comments, mirrored to Gitea native assets; add/list/delete with owner-or-admin + open-state guards) |
 | `/gists` | gists.py |
@@ -43,7 +43,7 @@ Prefixes are wired in `main.py`:
 | `/xmlrpc` | xmlrpc.py - reverse-proxies XML-RPC calls to the forking XML-RPC bridge (`services/xmlrpc/`, supervised by `XmlrpcService` on loopback `config.XMLRPC_PORT`), which generates one XML-RPC method per documented REST endpoint from `docs_api.API_GROUPS` (`posts.create`, `feed.list`, ...). One struct of named params per call; auth via in-band `api_key`, `X-API-KEY`/`Bearer` header, or `http://user:pass@host/xmlrpc` Basic. Full introspection + `system.multicall`; REST errors become XML-RPC faults. Exempt from the rate limiter (enforced on the forwarded internal hop). See `devplacepy/services/xmlrpc/CLAUDE.md` |
 | `/api` | devrant/ package - devRant-compatible REST protocol (`auth.py`, `rants.py`, `comments.py`, `notifs.py`). Translates devRant requests onto DevPlace data: rants<->posts, comments/votes onto the native engagement layer, devRant integer ids onto each table's auto-increment `id`. Token auth via `devrant_tokens`. See `routers/devrant/CLAUDE.md` for the deep detail on this tree |
 | `/dbapi` | dbapi/ package - **primary-administrator-only, strictly READ-ONLY** generic database API over `dataset` (never inserts/updates/deletes/restores; the write surface was removed because it bypassed every per-route admin safeguard). `tables.py` (`GET /tables`, `GET /{table}/schema`), `crud.py` (read only: `GET /{table}` + `GET /{table}/{key}/{value}`, `?include_deleted`), `query.py` (`POST /query` hard SELECT-only validated read-only run; `POST /query/async` + `GET /query/{uid}` + `WS /query/{uid}/ws` via `DbApiJobService` kind `dbquery`), `nl.py` (`POST /nl` natural-language->SQL via the AI gateway, returns validated SELECT, `execute=true` runs it read-only). **Auth = the PRIMARY administrator only** (the earliest-created Admin, `utils.is_primary_admin` / `database.get_primary_admin_uid` - the same identity that gates backup downloads) by session/api_key (`services/dbapi/policy.py`); every other administrator is refused like a member, and there is **no internal-key path** - the gateway `internal_gateway_key()` is NOT accepted (no service-to-service access). SQL validated by `services/dbapi/validate.py` (sqlglot classify + suspicious-flag + read-only `EXPLAIN` dry-run). Devii tools `db_*` are read-only (list/get/query/nl; no write tools) and flagged `requires_primary_admin=True`, so they are added to the LLM tool list ONLY for the primary administrator (every other session never sees them and Devii is unaware they exist). `services/pubsub/policy.py` resolves its own admin/internal actor and does NOT reuse `dbapi.policy.caller_for`, so the primary-admin restriction does not leak onto the pub/sub bus. See `devplacepy/services/dbapi/CLAUDE.md` |
-| `/game` | game/ package - the **Code Farm** idle game (`index.py` + `farm.py`): `GET /game` (page), `GET /game/state`, `GET /game/leaderboard`, `POST /game/{plant,harvest,buy-plot,upgrade,fertilize,daily,perk,prestige,legacy,quests/claim}`, plus social `GET /game/farm/{username}` and `POST /game/farm/{username}/{water,steal}` |
+| `/game` | game/ package - the **Code Farm** idle game (`index.py` + `farm.py`): `GET /game` (page), `GET /game/state`, `GET /game/leaderboard?board=`, `POST /game/{plant,harvest,buy-plot,upgrade,fertilize,daily,grant,perk,prestige,legacy,mastery,quests/claim}`, `POST /game/{defense/upgrade,infrastructure/buy,cosmetics/buy,cosmetics/equip}`, plus social `GET /game/farm/{username}` and `POST /game/farm/{username}/{water,steal}`. See `devplacepy/services/game/CLAUDE.md` |
 | (none) | push.py - web push + PWA: `GET`/`POST /push.json` (VAPID public key / subscription register), `GET /service-worker.js`, `GET /manifest.json` |
 | (none) | docs.py (docs/ package) - the documentation site (prose pages + API reference). See `routers/docs/CLAUDE.md` for the deep detail on this tree (`DOCS_PAGES`, audience tiers, prose rendering pipeline) |
 | `/pubsub` | pubsub.py - **database-free** publish/subscribe bus. `WS /pubsub/ws` (subscribe/unsubscribe/publish frames, `foo.*` wildcards), `POST /pubsub/publish` + `GET /pubsub/topics` (admin/internal). Lock-owner-gated WS (`4013` retry) so subscribers converge on one worker; topic authz in `services/pubsub/policy.py` (`user.{uid}.*` private, `public.*` shared, admin/internal anywhere, guests opt-in). In-process `services.pubsub.publish(topic, data)` for backends; frontend `app.pubsub`. See `devplacepy/services/pubsub/CLAUDE.md` |
@@ -188,7 +188,7 @@ News articles have an internal detail page at `/news/{slug}` with full comment s
 The home route (`main.py` `landing()`) never redirects - it renders `templates/landing.html` for everyone, branching on `user`:
 
 - **Guests** get the marketing hero (`Join DevPlace Free` CTA + features grid).
-- **Signed-in users** get a personalized hero (`.landing-hero-user`): avatar, "Welcome back, {username}", a `Go to your feed` CTA, a Posts/Stars/Level stat strip (`user_post_count` + the user dict's `stars`/`level`), and quick links. Styles live in `.landing-hero-user`/`.landing-welcome`/`.landing-stats`/`.landing-quicklinks` in `static/css/landing.css`.
+- **Signed-in users** get a personalized dashboard hero (`.dashboard-welcome`): avatar, "Welcome back, {username}", quicklink buttons (`.dashboard-btn`, with `New Post` -> `/feed` as `.dashboard-btn-primary`, plus Code Farm/Projects/Gists), and a Posts/Stars/Level stat strip (`.dashboard-stats`, `user_post_count` + the user dict's `stars`/`level`). Styles live in the `.dashboard-*` classes in `static/css/landing.css`.
 - Both states share the Latest Posts + Developer News + "Build With Us" sections. The **Build With Us** section is static HTML/CSS (`.landing-help-*` in `landing.css`): four cards linking to `/docs/index.html` (Documentation), `/swagger` + `/openapi.json` (API Reference), `/issues` (Contribute & Report), and Devii. The Devii card's `Launch Devii` button is a plain `<button data-devii-open>` that opens the globally mounted `DeviiTerminal` (`app.devii`) in place - no extra JS, route, schema, or Devii action; a secondary link points to `/devii/` for the full terminal page.
 - Context adds `user`, `is_authenticated`, `user_post_count`; `LandingOut` carries `is_authenticated`/`user_post_count` for the JSON form. `GET /` is documented in `docs_api.py` (id `home`, mapped to `LandingOut`).
 

devplacepy/routers/admin/__init__.py

@@ -1,12 +1,14 @@
 # retoor <retoor@molodetz.nl>
 
 from devplacepy.routers.admin import (
+    awards,
     aiquota,
     aiusage,
     auditlog,
     backups,
     bots,
     containers,
+    game,
     gateway_configs,
     issues,
     media,
@@ -14,13 +16,16 @@ from devplacepy.routers.admin import (
     notifications,
     services,
     settings,
+    statistics,
     trash,
     users,
 )
 from devplacepy.routers.admin.index import router
 
+router.include_router(awards.router)
 router.include_router(users.router)
 router.include_router(aiusage.router)
+router.include_router(statistics.router)
 router.include_router(aiquota.router)
 router.include_router(media.router)
 router.include_router(trash.router)
@@ -32,5 +37,6 @@ router.include_router(auditlog.router)
 router.include_router(backups.router)
 router.include_router(bots.router)
 router.include_router(gateway_configs.router)
+router.include_router(game.router)
 router.include_router(services.router, prefix="/services")
 router.include_router(containers.router, prefix="/containers")

devplacepy/routers/admin/awards.py

@@ -0,0 +1,49 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+
+from fastapi import APIRouter, Request
+from devplacepy.database import get_table
+from devplacepy.database.awards import revoke_award
+from devplacepy.responses import action_result, json_error, wants_json
+from devplacepy.services.audit import record as audit
+from devplacepy.utils import not_found, require_admin, safe_next
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+def _redirect_back(request: Request, award: dict) -> str:
+    referer = request.headers.get("referer", "")
+    if referer and safe_next(referer, "") == referer:
+        return referer
+    receiver = get_table("users").find_one(uid=award.get("receiver_uid", ""))
+    if receiver:
+        return f"/profile/{receiver['username']}?tab=awards"
+    return "/admin"
+
+
+@router.post("/awards/{uid}/revoke")
+async def admin_revoke_award(request: Request, uid: str):
+    admin = require_admin(request)
+    row = revoke_award(uid, admin["uid"])
+    if not row:
+        if wants_json(request):
+            return json_error(404, "Award not found")
+        raise not_found("Award not found")
+    receiver = get_table("users").find_one(uid=row.get("receiver_uid", ""))
+    logger.info("Admin %s revoked award %s", admin["username"], uid)
+    audit.record(
+        request,
+        "award.revoke",
+        user=admin,
+        target_type="award",
+        target_uid=uid,
+        target_label=row.get("slug", uid),
+        summary=f"admin {admin['username']} revoked award {row.get('slug', uid)}",
+        links=[
+            audit.target("award", uid, row.get("slug")),
+            audit.target("user", row.get("receiver_uid"), receiver.get("username") if receiver else None),
+        ],
+    )
+    return action_result(request, _redirect_back(request, row))

devplacepy/routers/admin/game.py

@@ -0,0 +1,97 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from typing import Annotated
+from fastapi import APIRouter, Form, Request
+from fastapi.responses import HTMLResponse
+from devplacepy.models import GameEraStartForm
+from devplacepy.responses import respond, action_result, json_error, wants_json
+from devplacepy.schemas import AdminGameOut
+from devplacepy.seo import base_seo_context, site_url, website_schema
+from devplacepy.services.audit import record as audit
+from devplacepy.services.game import GameError, store
+from devplacepy.utils import require_admin
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+def _era_context() -> dict:
+    era = store.active_era()
+    return {
+        "era_active": bool(era),
+        "era_name": era["name"] if era else "",
+        "era_number": int(era["era_number"]) if era else 0,
+        "era_started_at": era["started_at"] if era else "",
+        "era_ends_at": era["ends_at"] if era else "",
+    }
+
+
+@router.get("/game", response_class=HTMLResponse)
+async def admin_game(request: Request):
+    admin = require_admin(request)
+    base = site_url(request)
+    seo_ctx = base_seo_context(
+        request,
+        title="Code Farm - Admin",
+        description="Manage Code Farm Eras.",
+        robots="noindex,nofollow",
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Admin", "url": "/admin"},
+            {"name": "Code Farm", "url": "/admin/game"},
+        ],
+        schemas=[website_schema(base)],
+    )
+    return respond(
+        request,
+        "admin_game.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "user": admin,
+            "admin_section": "game",
+            **_era_context(),
+        },
+        model=AdminGameOut,
+    )
+
+
+@router.post("/game/era/start")
+async def admin_game_era_start(request: Request, data: Annotated[GameEraStartForm, Form()]):
+    admin = require_admin(request)
+    try:
+        era = store.start_era(data.name, data.duration_days)
+    except GameError as exc:
+        logger.warning(f"Admin {admin['username']} failed to start Era: {exc}")
+        if wants_json(request):
+            return json_error(400, str(exc))
+        return action_result(request, "/admin/game")
+    audit.record(
+        request,
+        "admin.game.era_start",
+        user=admin,
+        metadata={"era_number": era["era_number"], "name": era["name"]},
+        summary=f"admin {admin['username']} started Era {era['name']}",
+    )
+    return action_result(request, "/admin/game")
+
+
+@router.post("/game/era/end")
+async def admin_game_era_end(request: Request):
+    admin = require_admin(request)
+    try:
+        result = store.end_era()
+    except GameError as exc:
+        logger.warning(f"Admin {admin['username']} failed to end Era: {exc}")
+        if wants_json(request):
+            return json_error(400, str(exc))
+        return action_result(request, "/admin/game")
+    audit.record(
+        request,
+        "admin.game.era_end",
+        user=admin,
+        metadata=result,
+        summary=f"admin {admin['username']} ended Era {result['era_number']}",
+    )
+    return action_result(request, "/admin/game")

devplacepy/routers/admin/gateway_configs.py

@@ -9,7 +9,7 @@ from pydantic import ValidationError
 from devplacepy.seo import base_seo_context, site_url, website_schema
 from devplacepy.services.audit import record as audit
 from devplacepy.services.manager import service_manager
-from devplacepy.services.openai_gateway import routing
+from devplacepy.services.openai_gateway import quota, routing
 from devplacepy.templating import templates
 from devplacepy.utils import require_admin
 
@@ -25,6 +25,8 @@ def _default_provider_summary() -> dict:
         "model": cfg.get("gateway_model", ""),
         "embed_url": cfg.get("gateway_embed_url", ""),
         "embed_model": cfg.get("gateway_embed_model", ""),
+        "image_url": cfg.get("gateway_image_url", ""),
+        "image_model": cfg.get("gateway_image_model", ""),
         "vision_url": cfg.get("gateway_vision_url", ""),
         "vision_model": cfg.get("gateway_vision_model", ""),
     }
@@ -180,3 +182,92 @@ async def delete_model(request: Request, source_model: str):
         summary=f"admin {admin['username']} deleted gateway model route {source_model}",
     )
     return JSONResponse({"ok": True})
+
+
+def _quota_defaults_summary() -> dict:
+    svc = service_manager.get_service("openai")
+    cfg = svc.get_config() if svc is not None else {}
+    return {
+        "user": cfg.get(quota.FIELD_DEFAULT_USER, 0.0),
+        "admin": cfg.get(quota.FIELD_DEFAULT_ADMIN, 0.0),
+        "guest": cfg.get(quota.FIELD_DEFAULT_GUEST, 0.0),
+        "internal": cfg.get(quota.FIELD_DEFAULT_INTERNAL, 0.0),
+        "key": cfg.get(quota.FIELD_DEFAULT_KEY, 0.0),
+    }
+
+
+def _rule_label(rule: dict) -> str:
+    parts = []
+    if rule.get("owner_kind"):
+        parts.append(f"role={rule['owner_kind']}")
+    if rule.get("owner_id"):
+        parts.append(f"user={rule['owner_id']}")
+    if rule.get("app_reference"):
+        parts.append(f"app={rule['app_reference']}")
+    return ", ".join(parts) or rule.get("uid", "")
+
+
+@router.get("/gateway/quota-rules")
+async def list_quota_rules(request: Request):
+    require_admin(request)
+    rules = quota.quota_rule_store.list()
+    for rule in rules:
+        rule["spent_24h_usd"] = round(
+            quota.spent_24h(rule["owner_kind"], rule["owner_id"], rule["app_reference"]), 6
+        )
+    return JSONResponse(
+        {
+            "rules": rules,
+            "count": len(rules),
+            "defaults": _quota_defaults_summary(),
+        }
+    )
+
+
+@router.post("/gateway/quota-rules")
+async def save_quota_rule(request: Request):
+    admin = require_admin(request)
+    body = await _payload(request)
+    uid = str(body.pop("uid", "") or "").strip() or None
+    try:
+        payload = quota.QuotaRuleIn(**body)
+    except ValidationError as exc:
+        return _validation_error(exc)
+    saved = quota.quota_rule_store.set(payload, uid=uid, created_by=admin["uid"])
+    audit.record(
+        request,
+        "gateway.quota_rule.update",
+        user=admin,
+        target_type="gateway_quota_rule",
+        target_uid=saved["uid"],
+        target_label=_rule_label(saved),
+        summary=f"admin {admin['username']} saved gateway quota rule ({_rule_label(saved)}) at ${saved['limit_usd']}/24h",
+        metadata={
+            "owner_kind": saved["owner_kind"],
+            "owner_id": saved["owner_id"],
+            "app_reference": saved["app_reference"],
+            "limit_usd": saved["limit_usd"],
+            "is_active": saved["is_active"],
+        },
+    )
+    return JSONResponse({"ok": True, "rule": saved})
+
+
+@router.delete("/gateway/quota-rules/{uid}")
+async def delete_quota_rule(request: Request, uid: str):
+    admin = require_admin(request)
+    existing = quota.quota_rule_store.get(uid)
+    label = _rule_label(existing.as_dict()) if existing else uid
+    existed = quota.quota_rule_store.remove(uid)
+    if not existed:
+        return JSONResponse({"ok": False, "error": "Quota rule not found"}, status_code=404)
+    audit.record(
+        request,
+        "gateway.quota_rule.delete",
+        user=admin,
+        target_type="gateway_quota_rule",
+        target_uid=uid,
+        target_label=label,
+        summary=f"admin {admin['username']} deleted gateway quota rule ({label})",
+    )
+    return JSONResponse({"ok": True})

devplacepy/routers/admin/statistics.py

@@ -0,0 +1,90 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from fastapi import APIRouter, Request
+from fastapi.responses import HTMLResponse, JSONResponse
+from devplacepy.responses import respond
+from devplacepy.schemas.statistics import StatisticsOut
+from devplacepy.seo import base_seo_context, site_url, website_schema
+from devplacepy.services.statistics.build import build_statistics_tab
+from devplacepy.services.statistics.common import VALID_TABS
+from devplacepy.utils import require_admin
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+TAB_LABELS = (
+    ("overview", "Overview", "\U0001f4ca"),
+    ("visitors", "Visitors", "\U0001f441\ufe0f"),
+    ("members", "Members", "\U0001f465"),
+    ("content", "Content", "\U0001f4dd"),
+    ("engagement", "Engagement", "\U0001f525"),
+    ("social", "Social", "\U0001f91d"),
+    ("ai", "AI", "\U0001f916"),
+    ("devii", "Devii", "\u2728"),
+    ("services", "Services", "\u2699\ufe0f"),
+    ("containers", "Containers", "\U0001f4e6"),
+    ("game", "Game", "\U0001f3ae"),
+    ("awards", "Awards", "\U0001f3c6"),
+    ("moderation", "Moderation", "\U0001f6e1\ufe0f"),
+    ("tools", "Tools", "\U0001f527"),
+    ("storage", "Storage", "\U0001f4be"),
+)
+
+
+@router.get("/statistics", response_class=HTMLResponse)
+async def admin_statistics(request: Request, tab: str = "overview", hours: int = 168):
+    admin = require_admin(request)
+    active = tab if tab in VALID_TABS else "overview"
+    base = site_url(request)
+    seo_ctx = base_seo_context(
+        request,
+        title="Statistics - Admin",
+        description="Platform statistics with trends, visitors, content, engagement, and operations.",
+        robots="noindex,nofollow",
+        breadcrumbs=[
+            {"name": "Home", "url": "/feed"},
+            {"name": "Admin", "url": "/admin"},
+            {"name": "Statistics", "url": "/admin/statistics"},
+        ],
+        schemas=[website_schema(base)],
+    )
+    initial = build_statistics_tab(active, hours, compare=True, top_n=10)
+    tabs = [
+        {"key": key, "label": label, "icon": icon, "active": key == active}
+        for key, label, icon in TAB_LABELS
+    ]
+    return respond(
+        request,
+        "admin_statistics.html",
+        {
+            **seo_ctx,
+            "request": request,
+            "user": admin,
+            "admin_section": "statistics",
+            "tabs": tabs,
+            "active_tab": active,
+            "window_hours": hours,
+            "initial": initial,
+        },
+        model=StatisticsOut,
+    )
+
+
+@router.get("/statistics/data")
+async def admin_statistics_data(
+    request: Request,
+    tab: str = "overview",
+    hours: int = 168,
+    compare: int = 1,
+    top_n: int = 10,
+):
+    require_admin(request)
+    return JSONResponse(
+        build_statistics_tab(
+            tab,
+            hours,
+            compare=bool(compare),
+            top_n=top_n,
+        )
+    )

devplacepy/routers/admin/trash.py

@@ -24,13 +24,14 @@ logger = logging.getLogger(__name__)
 router = APIRouter()
 
 TRASH_TABLES = [
-    {"key": "posts", "label": "Posts", "type": "post"},
-    {"key": "comments", "label": "Comments", "type": "comment"},
-    {"key": "gists", "label": "Gists", "type": "gist"},
-    {"key": "projects", "label": "Projects", "type": "project"},
-    {"key": "news", "label": "News", "type": "news"},
-    {"key": "project_files", "label": "Project files", "type": None},
-    {"key": "attachments", "label": "Attachments", "type": None},
+    {"key": "posts", "label": "Posts", "icon": "\U0001f4dd", "type": "post"},
+    {"key": "comments", "label": "Comments", "icon": "\U0001f4ac", "type": "comment"},
+    {"key": "gists", "label": "Gists", "icon": "\U0001f4cb", "type": "gist"},
+    {"key": "projects", "label": "Projects", "icon": "\U0001f680", "type": "project"},
+    {"key": "news", "label": "News", "icon": "\U0001f4f0", "type": "news"},
+    {"key": "awards", "label": "Awards", "icon": "\U0001f3c6", "type": "award"},
+    {"key": "project_files", "label": "Project files", "icon": "\U0001f4c1", "type": None},
+    {"key": "attachments", "label": "Attachments", "icon": "\U0001f4ce", "type": None},
 ]
 _TRASH_KEYS = {entry["key"] for entry in TRASH_TABLES}
 _TRASH_TYPE = {entry["key"]: entry["type"] for entry in TRASH_TABLES}
@@ -113,6 +114,10 @@ async def admin_trash_restore(request: Request, table: str, uid: str):
     row = get_table(table).find_one(uid=uid)
     if row and row.get("deleted_at"):
         restored = restore_event(row["deleted_at"])
+        if table == "awards":
+            from devplacepy.database.awards import recompute_user_award_stats
+
+            recompute_user_award_stats(row.get("receiver_uid", ""))
         logger.info(
             f"Admin {admin['username']} restored {table} {uid} ({restored} rows)"
         )

devplacepy/routers/avatar.py

@@ -17,15 +17,14 @@ _CACHE_CONTROL = f"public, max-age={SECONDS_PER_DAY}, immutable"
 
 @router.get("/{style}/{seed}")
 async def avatar_proxy(request: Request, style: str, seed: str, size: int = 128):
-    cache_key = f"{seed}:{size}"
-    etag = '"' + hashlib.md5(cache_key.encode("utf-8")).hexdigest() + '"'
+    etag = '"' + hashlib.md5(f"{seed}:{size}".encode("utf-8")).hexdigest() + '"'
     headers = {"ETag": etag, "Cache-Control": _CACHE_CONTROL}
 
     if request.headers.get("if-none-match") == etag:
         return Response(status_code=304, headers=headers)
 
-    svg = _cache.get(cache_key)
+    svg = _cache.get(seed)
     if svg is None:
         svg = generate_avatar_svg(seed)
-        _cache.set(cache_key, svg)
+        _cache.set(seed, svg)
     return Response(content=svg, media_type="image/svg+xml", headers=headers)

devplacepy/routers/awards.py

@@ -0,0 +1,36 @@
+# retoor <retoor@molodetz.nl>
+
+from fastapi import APIRouter, Request
+from fastapi.responses import RedirectResponse
+from devplacepy.attachments import _row_to_attachment
+from devplacepy.database import get_table, resolve_by_slug
+from devplacepy.utils import not_found
+
+router = APIRouter()
+
+_VALID_SIZES = {"512", "256", "64"}
+_CACHE_CONTROL = "public, max-age=86400, immutable"
+
+
+@router.get("/{slug_or_uid}/{size}")
+async def award_image(request: Request, slug_or_uid: str, size: str):
+    if size not in _VALID_SIZES:
+        raise not_found("Award image not found")
+    award = resolve_by_slug(get_table("awards"), slug_or_uid)
+    if not award or not award.get("generated_at"):
+        raise not_found("Award image not found")
+    attachment_uid = award.get(f"attachment_uid_{size}") or ""
+    if not attachment_uid:
+        raise not_found("Award image not found")
+    row = get_table("attachments").find_one(uid=attachment_uid, deleted_at=None)
+    attachment = _row_to_attachment(row) if row else None
+    if not attachment:
+        raise not_found("Award image not found")
+    url = attachment.get("url") or ""
+    if not url:
+        raise not_found("Award image not found")
+    headers = {
+        "Cache-Control": _CACHE_CONTROL,
+        "ETag": f'"{attachment_uid}"',
+    }
+    return RedirectResponse(url=url, status_code=302, headers=headers)

devplacepy/routers/devii.py

@@ -185,7 +185,10 @@ async def clippy_proxy(request: Request):
         return JSONResponse({"error": "Devii is unavailable"}, status_code=503)
     cfg = svc.effective_config()
     body = await request.body()
-    headers = {"Content-Type": "application/json"}
+    headers = {
+        "Content-Type": "application/json",
+        "X-App-Reference": "devplace-devii-v-1-0-0",
+    }
     if cfg.get("devii_ai_key"):
         headers["Authorization"] = f"Bearer {cfg['devii_ai_key']}"
     async with stealth.stealth_async_client(timeout=45.0) as client:
@@ -260,6 +263,8 @@ async def devii_ws(websocket: WebSocket):
                 if command == "reset":
                     await session.reset()
                     continue
+                if await session.try_answer_interaction(text):
+                    continue
                 if svc.quota_exceeded(owner_kind, owner_id, owner_is_admin):
                     limit = svc.daily_limit_for(owner_kind, owner_is_admin)
                     audit.record_system(
@@ -302,6 +307,11 @@ async def devii_ws(websocket: WebSocket):
                 )
             elif kind in ("avatar_result", "client_result"):
                 session.resolve_query(str(data.get("id", "")), data.get("result"))
+            elif kind == "interaction_result":
+                session.resolve_interaction(
+                    str(data.get("id", data.get("interaction_id", ""))),
+                    data.get("result") or data,
+                )
     except WebSocketDisconnect:
         pass
     except Exception:  # noqa: BLE001 - never let the socket loop crash the worker

devplacepy/routers/docs/pages.py

@@ -123,6 +123,12 @@ DOCS_PAGES = [
         "kind": "prose",
         "section": SECTION_GENERAL,
     },
+    {
+        "slug": "awards",
+        "title": "Profile awards",
+        "kind": "prose",
+        "section": SECTION_GENERAL,
+    },
     {
         "slug": "ai-correction",
         "title": "AI content correction",

devplacepy/routers/game/farm.py

@@ -88,6 +88,8 @@ async def steal_farm(
         return RedirectResponse(url=f"/game/farm/{username}", status_code=302)
     track_action(viewer["uid"], "harvest_stolen")
     track_action(owner["uid"], "got_stolen_from")
+    if result.get("underdog_triggered"):
+        track_action(viewer["uid"], "underdog_raid")
     create_notification(
         owner["uid"],
         "harvest_stolen",

devplacepy/routers/game/index.py

@@ -6,7 +6,10 @@ from fastapi import APIRouter, Form, Request
 from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse
 
 from devplacepy.models import (
+    GameCosmeticForm,
+    GameInfraForm,
     GameLegacyForm,
+    GameMasteryForm,
     GamePerkForm,
     GamePlantForm,
     GameQuestForm,
@@ -49,9 +52,9 @@ async def game_state(request: Request):
 
 
 @router.get("/leaderboard")
-async def game_leaderboard(request: Request):
+async def game_leaderboard(request: Request, board: str = "score"):
     get_current_user(request)
-    entries = store.leaderboard(25)
+    entries = store.leaderboard_for(board, 25)
     return JSONResponse(
         GameLeaderboardOut(entries=entries).model_dump(mode="json")
     )
@@ -119,6 +122,12 @@ async def game_daily(request: Request):
     return await _respond_action(request, user, lambda: store.claim_daily(user))
 
 
+@router.post("/grant")
+async def game_claim_grant(request: Request):
+    user = require_user(request)
+    return await _respond_action(request, user, lambda: store.claim_grant(user))
+
+
 @router.post("/perk")
 async def game_perk(request: Request, data: Annotated[GamePerkForm, Form()]):
     user = require_user(request)
@@ -149,5 +158,55 @@ async def game_claim_quest(request: Request, data: Annotated[GameQuestForm, Form
         award_rewards(user["uid"], result.get("reward_xp", 0))
 
     return await _respond_action(
-        request, user, lambda: store.claim_quest(user, data.quest), reward
+        request, user, lambda: store.claim_quest(user, data.quest, data.scope), reward
+    )
+
+
+@router.post("/defense/upgrade")
+async def game_upgrade_defense(request: Request):
+    user = require_user(request)
+
+    def reward(result):
+        track_action(user["uid"], "defense_upgraded")
+
+    return await _respond_action(request, user, lambda: store.upgrade_defense(user), reward)
+
+
+@router.post("/infrastructure/buy")
+async def game_buy_infrastructure(request: Request, data: Annotated[GameInfraForm, Form()]):
+    user = require_user(request)
+
+    def reward(result):
+        track_action(user["uid"], "infra_bought")
+
+    return await _respond_action(
+        request, user, lambda: store.buy_infrastructure(user, data.key), reward
+    )
+
+
+@router.post("/mastery")
+async def game_upgrade_mastery(request: Request, data: Annotated[GameMasteryForm, Form()]):
+    user = require_user(request)
+    return await _respond_action(
+        request, user, lambda: store.upgrade_mastery(user, data.key)
+    )
+
+
+@router.post("/cosmetics/buy")
+async def game_buy_cosmetic(request: Request, data: Annotated[GameCosmeticForm, Form()]):
+    user = require_user(request)
+
+    def reward(result):
+        track_action(user["uid"], "cosmetic_bought")
+
+    return await _respond_action(
+        request, user, lambda: store.buy_cosmetic(user, data.key), reward
+    )
+
+
+@router.post("/cosmetics/equip")
+async def game_equip_cosmetic(request: Request, data: Annotated[GameCosmeticForm, Form()]):
+    user = require_user(request)
+    return await _respond_action(
+        request, user, lambda: store.equip_title(user, data.key)
     )

devplacepy/routers/gists.py

@@ -57,6 +57,7 @@ LANGUAGES = [
     ("yaml", "YAML"),
     ("json", "JSON"),
     ("markdown", "Markdown"),
+    ("markdown_rendered", "Markdown Rendered"),
     ("swift", "Swift"),
     ("php", "PHP"),
     ("ruby", "Ruby"),

devplacepy/routers/messages.py

@@ -2,6 +2,7 @@
 
 import asyncio
 import logging
+from datetime import datetime
 from typing import Annotated, Optional
 from fastapi import Depends, APIRouter, Request, WebSocket, WebSocketDisconnect
 from devplacepy.models import MessageForm
@@ -25,16 +26,18 @@ from devplacepy.utils import (
 )
 from devplacepy.seo import base_seo_context
 from devplacepy.responses import respond, action_result
-from devplacepy.schemas import MessagesOut
+from devplacepy.schemas import ConversationOut, MessagesOut
 from devplacepy.services import presence
 from devplacepy.services.audit import record as audit
 from devplacepy.services.correction import PENDING_SCOPE_KEY
 from devplacepy.dependencies import json_or_form
 from devplacepy.services.messaging import (
+    issue_ticket,
     message_frame,
     message_hub,
     message_relay,
     persist_message,
+    redeem_ticket,
 )
 
 logger = logging.getLogger(__name__)
@@ -44,6 +47,18 @@ MAX_WS_ATTACHMENTS = 5
 
 CONVERSATION_MESSAGE_LIMIT = 500
 
+MESSAGE_GROUP_GAP_SECONDS = 300
+
+def _grouped_with_previous(sender_uid, created_at, previous_sender_uid, previous_created_at) -> bool:
+    if previous_sender_uid is None or sender_uid != previous_sender_uid:
+        return False
+    try:
+        current_dt = datetime.fromisoformat(created_at)
+        previous_dt = datetime.fromisoformat(previous_created_at)
+    except (TypeError, ValueError):
+        return False
+    return (current_dt - previous_dt).total_seconds() <= MESSAGE_GROUP_GAP_SECONDS
+
 def mark_conversation_read(user_uid: str, other_uid: str) -> None:
     if "messages" not in db.tables:
         return
@@ -123,6 +138,8 @@ def get_conversation_messages(user_uid: str, other_uid: str):
     result = []
     msg_uids = [m["uid"] for m in msgs]
     attachments_map = get_attachments_batch("message", msg_uids) if msg_uids else {}
+    previous_sender_uid = None
+    previous_created_at = None
     for m in msgs:
         result.append(
             {
@@ -131,8 +148,13 @@ def get_conversation_messages(user_uid: str, other_uid: str):
                 "is_mine": m["sender_uid"] == user_uid,
                 "time_ago": time_ago(m["created_at"]),
                 "attachments": attachments_map.get(m["uid"], []),
+                "grouped": _grouped_with_previous(
+                    m["sender_uid"], m["created_at"], previous_sender_uid, previous_created_at
+                ),
             }
         )
+        previous_sender_uid = m["sender_uid"]
+        previous_created_at = m["created_at"]
     return result, other_user
 
 @router.get("", response_class=HTMLResponse)
@@ -207,6 +229,19 @@ async def search_users(request: Request, q: str = ""):
     results = search_users_by_username(q, exclude_uid=user["uid"])
     return JSONResponse({"results": results})
 
+@router.get("/conversations")
+async def list_conversations(request: Request):
+    user = require_user(request)
+    conversations = get_conversations(user["uid"])
+    payload = [ConversationOut.model_validate(c).model_dump() for c in conversations]
+    return JSONResponse({"conversations": payload})
+
+@router.post("/ws-ticket")
+async def create_ws_ticket(request: Request):
+    user = require_user(request)
+    token = issue_ticket(user["uid"])
+    return JSONResponse({"ticket": token, "expires_in": 30})
+
 @router.post("/send")
 async def send_message(request: Request, data: Annotated[MessageForm, Depends(json_or_form(MessageForm))]):
     user = require_user(request)
@@ -223,37 +258,54 @@ async def send_message(request: Request, data: Annotated[MessageForm, Depends(js
     if message is None:
         return action_result(request, "/messages")
 
-    await _finalize_and_broadcast(user, message, request)
+    ai_processed = await _finalize_and_broadcast(
+        user, message, request, client_id=data.client_id
+    )
+    frame = message_frame(
+        message, user.get("username", ""), data.client_id,
+        sender_role=user.get("role"), ai_processed=ai_processed,
+    )
     return action_result(
-        request, f"/messages?with_uid={receiver_uid}", data={"uid": message["uid"]}
+        request, f"/messages?with_uid={receiver_uid}", data=frame
     )
 
 async def broadcast_message(
-    sender: dict, message: dict, client_id: Optional[str] = None
+    sender: dict, message: dict, client_id: Optional[str] = None,
+    ai_processed: bool = False,
 ) -> None:
-    frame = message_frame(message, sender.get("username", ""), client_id)
+    frame = message_frame(
+        message, sender.get("username", ""), client_id,
+        sender_role=sender.get("role"), ai_processed=ai_processed,
+    )
     message_hub.mark_delivered(message["uid"])
     targets = [message["sender_uid"], message["receiver_uid"]]
     await message_hub.send_to_users(targets, frame)
 
 async def _finalize_and_broadcast(
     sender: dict, message: dict, request: object, client_id: Optional[str] = None
-) -> None:
+) -> bool:
     message_hub.mark_delivered(message["uid"])
     scope = getattr(request, "scope", None)
     pending = scope.get(PENDING_SCOPE_KEY) if scope is not None else None
+    ai_processed = bool(pending)
     if pending:
         await asyncio.gather(*pending, return_exceptions=True)
         pending.clear()
         row = get_table("messages").find_one(uid=message["uid"])
         if row:
             message["content"] = row["content"]
-    await broadcast_message(sender, message, client_id)
+    await broadcast_message(sender, message, client_id, ai_processed=ai_processed)
+    return ai_processed
 
 def _resolve_ws_user(websocket: WebSocket):
     user = _user_from_session(websocket)
     if user:
         return user
+    ticket = websocket.query_params.get("ticket", "").strip()
+    if ticket:
+        user_uid = redeem_ticket(ticket)
+        if user_uid:
+            return get_table("users").find_one(uid=user_uid)
     key = websocket.headers.get("x-api-key", "").strip()
     if not key:
         scheme, _, credentials = websocket.headers.get("authorization", "").partition(

devplacepy/routers/profile/__init__.py

@@ -4,17 +4,21 @@ from devplacepy.routers.profile import (
     ai_correction,
     ai_modifier,
     avatar,
+    award,
     customization,
+    interactions,
     notifications,
     telegram,
 )
 from devplacepy.routers.profile.index import router
 from devplacepy.routers.profile.usage import _ai_quota
 
+router.include_router(award.router)
 router.include_router(customization.router)
 router.include_router(notifications.router)
 router.include_router(ai_correction.router)
 router.include_router(ai_modifier.router)
+router.include_router(interactions.router)
 router.include_router(avatar.router)
 router.include_router(telegram.router)
 

devplacepy/routers/profile/award.py

@@ -0,0 +1,106 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from datetime import datetime, timezone
+from typing import Annotated
+
+from fastapi import APIRouter, Request
+from devplacepy.database import get_blocked_uids, get_table
+from devplacepy.database.awards import can_give_award, has_giver_cooldown, has_receiver_cooldown
+from devplacepy.dependencies import json_or_form
+from devplacepy.models import AwardGiveForm
+from devplacepy.responses import action_result, json_error, wants_json
+from devplacepy.services.audit import record as audit
+from devplacepy.services.jobs import queue
+from devplacepy.utils import generate_uid, make_combined_slug, require_user
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.post("/{username}/award")
+async def give_award(
+    request: Request,
+    username: str,
+    data: Annotated[AwardGiveForm, json_or_form(AwardGiveForm)],
+):
+    giver = require_user(request)
+    target = get_table("users").find_one(username=username)
+    redirect = f"/profile/{username}"
+
+    def deny(message: str, status: int = 400):
+        if wants_json(request):
+            return json_error(status, message)
+        return action_result(request, redirect, status_code=302)
+
+    if not target:
+        return deny("User not found", 404)
+    if target["uid"] == giver["uid"]:
+        return deny("You cannot give yourself an award")
+    blocked = get_blocked_uids(giver["uid"])
+    if target["uid"] in blocked:
+        return deny("You cannot give an award to a blocked user")
+    reverse_blocked = get_blocked_uids(target["uid"])
+    if giver["uid"] in reverse_blocked:
+        return deny("You cannot give an award to this user")
+    if has_giver_cooldown(giver["uid"]):
+        return deny("You can give another award later")
+    if has_receiver_cooldown(target["uid"]):
+        return deny("This user received an award recently")
+    if not (giver.get("api_key") or "").strip():
+        return deny("Your account has no API key for award generation")
+
+    description = data.description.strip()
+    uid = generate_uid()
+    slug = make_combined_slug(description, uid)
+    now = datetime.now(timezone.utc).isoformat()
+    get_table("awards").insert(
+        {
+            "uid": uid,
+            "slug": slug,
+            "description": description,
+            "giver_uid": giver["uid"],
+            "receiver_uid": target["uid"],
+            "attachment_uid_512": "",
+            "attachment_uid_256": "",
+            "attachment_uid_64": "",
+            "generated_at": None,
+            "created_at": now,
+            "job_uid": "",
+            "deleted_at": None,
+            "deleted_by": None,
+        }
+    )
+    job_uid = queue.enqueue(
+        "award",
+        {
+            "award_uid": uid,
+            "giver_uid": giver["uid"],
+            "receiver_uid": target["uid"],
+            "description": description,
+            "api_key": giver.get("api_key", ""),
+        },
+        "user",
+        giver["uid"],
+    )
+    get_table("awards").update({"uid": uid, "job_uid": job_uid}, ["uid"])
+    logger.info("%s gave award %s to %s", giver["username"], uid, username)
+    audit.record(
+        request,
+        "award.give",
+        user=giver,
+        target_type="user",
+        target_uid=target["uid"],
+        target_label=username,
+        summary=f"{giver['username']} gave award to {username}",
+        links=[
+            audit.target("user", target["uid"], username),
+            audit.target("award", uid, slug),
+            audit.job(job_uid),
+        ],
+    )
+    return action_result(
+        request,
+        redirect,
+        data={"ok": True, "award_uid": uid, "award_slug": slug},
+    )

devplacepy/routers/profile/index.py

@@ -12,6 +12,7 @@ from devplacepy.database import (
     get_notification_prefs,
     get_user_stars,
     get_user_rank,
+    get_user_post_count,
     get_comment_counts_by_post_uids,
     get_reactions_by_targets,
     get_user_bookmarks,
@@ -28,6 +29,11 @@ from devplacepy.database import (
     mark_notifications_read_by_target,
     resolve_object_url,
 )
+from devplacepy.database.awards import (
+    can_give_award,
+    get_prominent_award,
+    get_user_awards,
+)
 from devplacepy.content import can_view_project, enrich_items
 from devplacepy.utils import (
     get_current_user,
@@ -147,6 +153,17 @@ async def profile_page(
     if tab == "media":
         media, media_pagination = get_user_media(profile_user["uid"], page)
 
+    awards, awards_pagination = [], None
+    if tab == "awards":
+        awards, awards_pagination = get_user_awards(profile_user["uid"], page)
+    prominent_award = get_prominent_award(profile_user)
+    awards_count = int(profile_user.get("award_count") or 0)
+    can_give = bool(
+        current_user
+        and current_user["uid"] != profile_user["uid"]
+        and can_give_award(current_user["uid"], profile_user["uid"])
+    )
+
     posts = []
     if tab == "posts":
         posts_table = get_table("posts")
@@ -195,9 +212,7 @@ async def profile_page(
         )
         for g in gists_raw:
             gists.append({"gist": g, "time_ago": time_ago(g["created_at"])})
-    posts_count = get_table("posts").count(
-        user_uid=profile_user["uid"], deleted_at=None
-    )
+    posts_count = get_user_post_count(profile_user["uid"])
 
     activities = []
     if tab == "activity":
@@ -286,6 +301,18 @@ async def profile_page(
         if is_owner
         else None
     )
+    from devplacepy.services.devii.interaction import prefs as interaction_prefs
+
+    interactions_snap = (
+        interaction_prefs.snapshot("user", profile_user["uid"], profile_user)
+        if is_owner
+        else {
+            "enabled": True,
+            "source": None,
+            "default": True,
+            "override": None,
+        }
+    )
     from devplacepy.services.telegram import store as telegram_store
 
     telegram_paired = (
@@ -386,6 +413,10 @@ async def profile_page(
             "ai_modifier_enabled": ai_modifier_enabled,
             "ai_modifier_sync": ai_modifier_sync,
             "ai_modifier_prompt": ai_modifier_prompt,
+            "interactions_enabled": interactions_snap["enabled"],
+            "interactions_source": interactions_snap["source"],
+            "interactions_default": interactions_snap["default"],
+            "interactions_override": interactions_snap["override"],
             "telegram_paired": telegram_paired,
             "notif_telegram_paired": notif_telegram_paired,
             "can_manage_customization": can_manage_customization,
@@ -404,6 +435,11 @@ async def profile_page(
             "follow_pagination": follow_pagination,
             "followers_count": follow_counts["followers"],
             "following_count": follow_counts["following"],
+            "awards": awards,
+            "awards_pagination": awards_pagination,
+            "awards_count": awards_count,
+            "prominent_award": prominent_award,
+            "can_give_award": can_give,
         },
         model=ProfileOut,
     )

devplacepy/routers/profile/interactions.py

@@ -0,0 +1,61 @@
+# retoor <retoor@molodetz.nl>
+
+import logging
+from typing import Annotated
+
+from fastapi import Depends, APIRouter, Request
+
+from devplacepy.models import InteractionsForm
+from devplacepy.responses import action_result
+from devplacepy.services.audit import record as audit
+from devplacepy.services.devii.interaction import prefs
+from devplacepy.routers.profile._shared import resolve_customization_target
+from devplacepy.dependencies import json_or_form
+
+logger = logging.getLogger(__name__)
+router = APIRouter()
+
+
+@router.post("/{username}/interactions")
+async def set_interactions(
+    request: Request,
+    username: str,
+    data: Annotated[InteractionsForm, Depends(json_or_form(InteractionsForm))],
+):
+    target, denied = resolve_customization_target(request, username)
+    if denied is not None:
+        return denied
+    if data.reset:
+        snap = prefs.set_user_pref(target["uid"], None)
+        summary = f"reset interactive widgets to admin default for {target['username']}"
+        new_value = -1
+    else:
+        snap = prefs.set_user_pref(target["uid"], bool(data.enabled))
+        summary = (
+            f"{'enabled' if data.enabled else 'disabled'} interactive widgets "
+            f"for {target['username']}"
+        )
+        new_value = 1 if data.enabled else 0
+    logger.info(summary)
+    audit.record(
+        request,
+        "profile.interactions",
+        target_type="user",
+        target_uid=target["uid"],
+        target_label=target["username"],
+        new_value=new_value,
+        summary=summary,
+        links=[audit.target("user", target["uid"], target["username"])],
+    )
+    url = f"/profile/{target['username']}"
+    return action_result(
+        request,
+        url,
+        data={
+            "url": url,
+            "enabled": snap["enabled"],
+            "source": snap["source"],
+            "default": snap["default"],
+            "override": snap["override"],
+        },
+    )

devplacepy/routers/profile/usage.py

@@ -4,6 +4,7 @@ import logging
 
 from devplacepy.database import get_correction_usage, get_modifier_usage
 from devplacepy.services.manager import service_manager
+from devplacepy.services.openai_gateway import quota as gateway_quota
 from devplacepy.services.openai_gateway.analytics import user_spend_24h
 
 logger = logging.getLogger(__name__)
@@ -62,4 +63,22 @@ def _ai_quota(
     if include_cost:
         quota["spent_usd"] = round(spent, 4)
         quota["limit_usd"] = round(limit, 2)
+    gateway_svc = service_manager.get_service("openai")
+    if gateway_svc is not None:
+        try:
+            owner_kind = "admin" if is_admin else "user"
+            cfg = gateway_svc.effective_config()
+            gw_limit, gw_scope, gw_rule = gateway_quota.resolve_for_owner(owner_kind, user_uid, cfg)
+            gw_spent = gateway_quota.spent_24h(*gw_scope)
+            gw_unlimited = gw_limit <= 0
+            quota["gateway_unlimited"] = gw_unlimited
+            quota["gateway_used_pct"] = (
+                0.0 if gw_unlimited else round(min(100.0, gw_spent / gw_limit * 100), 1)
+            )
+            if include_cost:
+                quota["gateway_spent_usd"] = round(gw_spent, 4)
+                quota["gateway_limit_usd"] = round(gw_limit, 2)
+                quota["gateway_pooled"] = bool(gw_rule and gw_rule.owner_id is None)
+        except Exception:
+            logger.exception("Failed to compute gateway-level AI quota for %s", user_uid)
     return quota

devplacepy/routers/xmlrpc.py

@@ -4,6 +4,7 @@ import logging
 
 import httpx
 from fastapi import APIRouter, Request
+from starlette.requests import ClientDisconnect
 from starlette.responses import PlainTextResponse, Response
 
 from devplacepy.config import XMLRPC_BIND, XMLRPC_PORT
@@ -49,7 +50,11 @@ async def info(request: Request, path: str = "") -> PlainTextResponse:
 @router.post("/")
 @router.post("/{path:path}")
 async def proxy(request: Request, path: str = "") -> Response:
-    body = await request.body()
+    try:
+        body = await request.body()
+    except ClientDisconnect:
+        logger.warning("XML-RPC client disconnected before request body was read")
+        return PlainTextResponse("Client disconnected", status_code=400)
     headers = {
         key: value
         for key, value in request.headers.items()

devplacepy/schemas/__init__.py

@@ -99,6 +99,7 @@ from devplacepy.schemas.backups import (
     BackupStoragePathOut,
 )
 from devplacepy.schemas.admin import (
+    AdminGameOut,
     AdminMediaItemOut,
     AdminMediaOut,
     AdminNewsItemOut,
@@ -114,6 +115,7 @@ from devplacepy.schemas.gateway import (
     GatewayUsageOut,
     UserAiUsageOut,
 )
+from devplacepy.schemas.statistics import StatisticsOut
 from devplacepy.schemas.auth import (
     AuthPageOut,
     DeviiPageOut,

devplacepy/schemas/admin.py

@@ -72,3 +72,12 @@ class AdminTrashOut(_Out):
     tables: list[dict] = []
     pagination: Optional[Any] = None
     admin_section: Optional[str] = None
+
+
+class AdminGameOut(_Out):
+    era_active: bool = False
+    era_name: str = ""
+    era_number: int = 0
+    era_started_at: str = ""
+    era_ends_at: str = ""
+    admin_section: Optional[str] = None

devplacepy/schemas/auth.py

@@ -40,12 +40,23 @@ class LandingPostOut(_Out):
     slug: str = ""
 
 
+class TrendingTopicOut(_Out):
+    topic: str = ""
+    count: int = 0
+
+
 class LandingOut(_Out):
     is_authenticated: bool = False
     user_post_count: int = 0
     user_stars: int = 0
+    user_xp: int = 0
+    user_level: int = 1
+    xp_progress_pct: int = 0
+    unread_count: int = 0
     landing_articles: list[LandingArticleOut] = []
     landing_posts: list[LandingPostOut] = []
+    top_contributors: list = []
+    trending_topics: list[TrendingTopicOut] = []
 
 
 class DeviiPageOut(_Out):

devplacepy/schemas/awards.py

@@ -0,0 +1,19 @@
+# retoor <retoor@molodetz.nl>
+
+from typing import Optional
+
+from devplacepy.schemas.base import _Out
+from devplacepy.schemas.content import UserOut
+
+
+class AwardOut(_Out):
+    uid: str = ""
+    slug: str = ""
+    description: str = ""
+    giver_uid: str = ""
+    receiver_uid: str = ""
+    generated_at: Optional[str] = None
+    created_at: Optional[str] = None
+    image_url: Optional[str] = None
+    thumb_url: Optional[str] = None
+    giver: Optional[UserOut] = None

devplacepy/schemas/game.py

@@ -15,6 +15,7 @@ class GameCropOut(_Out):
     min_level: int = 1
     grow_seconds: int = 0
     locked: bool = False
+    market_state: str = "normal"
 
 
 class GamePlotOut(_Out):
@@ -62,6 +63,38 @@ class GameLegacyOut(_Out):
     effect: str = ""
 
 
+class GameMasteryOut(_Out):
+    key: str = ""
+    name: str = ""
+    icon: str = ""
+    description: str = ""
+    level: int = 0
+    max_level: int = 0
+    cost: int = 0
+    maxed: bool = False
+    effect: str = ""
+
+
+class GameInfrastructureOut(_Out):
+    key: str = ""
+    name: str = ""
+    icon: str = ""
+    description: str = ""
+    cost: int = 0
+    min_prestige: int = 0
+    owned: bool = False
+
+
+class GameCosmeticOut(_Out):
+    key: str = ""
+    name: str = ""
+    icon: str = ""
+    description: str = ""
+    cost_coins: int = 0
+    kind: str = ""
+    owned: bool = False
+
+
 class GameQuestOut(_Out):
     kind: str = ""
     label: str = ""
@@ -71,6 +104,8 @@ class GameQuestOut(_Out):
     reward_xp: int = 0
     claimed: bool = False
     can_claim: bool = False
+    scope: str = "daily"
+    reward_stars: int = 0
 
 
 class GameFarmOut(_Out):
@@ -99,6 +134,14 @@ class GameFarmOut(_Out):
     prestige_multiplier: float = 1.0
     prestige_min_level: int = 0
     prestige_available: bool = False
+    refactor_cost: int = 0
+    refactor_affordable: bool = False
+    refactor_carryover_pct: int = 0
+    refactor_carryover_preview: int = 0
+    grant_available: bool = False
+    grant_amount: int = 0
+    grant_reason: str = ""
+    treasury_balance: int = 0
     streak: int = 0
     daily_available: bool = False
     daily_reward: int = 0
@@ -107,6 +150,25 @@ class GameFarmOut(_Out):
     stars: int = 0
     legacy: list[GameLegacyOut] = []
     steal_cooldown_seconds: int = 0
+    mastery_points: int = 0
+    mastery_points_earned_total: int = 0
+    mastery: list[GameMasteryOut] = []
+    infrastructure: list[GameInfrastructureOut] = []
+    defense_level: int = 0
+    defense_tier_name: str = ""
+    defense_upkeep_daily: int = 0
+    defense_next_cost: int = 0
+    cosmetics: list[GameCosmeticOut] = []
+    active_title: str = ""
+    underdog_boost_seconds_remaining: int = 0
+    mastery_analytics_unlocked: bool = False
+    lifetime_coins_earned: int = 0
+    lifetime_harvests: int = 0
+    harvests_week: int = 0
+    era_active: bool = False
+    era_name: str = ""
+    era_coins: int = 0
+    era_harvests: int = 0
 
 
 class GameStateOut(_Out):
@@ -130,6 +192,9 @@ class GameLeaderboardEntryOut(_Out):
     total_harvests: int = 0
     prestige: int = 0
     score: int = 0
+    raid_avg: float = 0.0
+    time_to_kernel_seconds: int = 0
+    title: str = ""
 
 
 class GameLeaderboardOut(_Out):

devplacepy/schemas/listings.py

@@ -70,6 +70,7 @@ class MessageItemOut(_Out):
     is_mine: bool = False
     time_ago: Optional[str] = None
     attachments: list[AttachmentOut] = []
+    grouped: bool = False
 
 
 class NotificationItemOut(_Out):

devplacepy/schemas/profile.py

@@ -6,6 +6,7 @@ from typing import Any, Optional
 
 from devplacepy.schemas.base import _Out
 from devplacepy.schemas.content import BadgeOut, ProjectOut, UserOut
+from devplacepy.schemas.awards import AwardOut
 from devplacepy.schemas.listings import FeedItemOut, GistItemOut
 
 
@@ -49,6 +50,10 @@ class ProfileOut(_Out):
     ai_modifier_enabled: bool = False
     ai_modifier_sync: bool = False
     ai_modifier_prompt: Optional[str] = None
+    interactions_enabled: bool = True
+    interactions_source: Optional[str] = None
+    interactions_default: bool = True
+    interactions_override: Optional[bool] = None
     telegram_paired: bool = False
     notif_telegram_paired: bool = False
     can_manage_customization: bool = False
@@ -70,6 +75,11 @@ class ProfileOut(_Out):
     media: list[MediaItemOut] = []
     media_pagination: Optional[Any] = None
     notification_prefs: list[Any] = []
+    awards: list[AwardOut] = []
+    awards_pagination: Optional[Any] = None
+    awards_count: int = 0
+    prominent_award: Optional[AwardOut] = None
+    can_give_award: bool = False
 
 
 class TelegramPairOut(_Out):

devplacepy/schemas/statistics.py

@@ -0,0 +1,52 @@
+# retoor <retoor@molodetz.nl>
+
+from __future__ import annotations
+
+from typing import Any, Optional
+
+from pydantic import BaseModel
+
+
+class StatisticsMetricOut(BaseModel):
+    key: str
+    label: str
+    value: Any
+    format: str = "int"
+    delta: Optional[float] = None
+    direction: Optional[str] = None
+
+
+class StatisticsPointOut(BaseModel):
+    t: str
+    v: float
+
+
+class StatisticsSeriesOut(BaseModel):
+    key: str
+    label: str
+    points: list[StatisticsPointOut]
+
+
+class StatisticsTableOut(BaseModel):
+    key: str
+    title: str
+    columns: list[str]
+    rows: list[list[Any]]
+
+
+class StatisticsHighlightOut(BaseModel):
+    label: str
+    value: Any
+
+
+class StatisticsOut(BaseModel):
+    tab: str
+    window_hours: int
+    granularity: str
+    generated_at: str
+    compare: bool = True
+    cards: list[StatisticsMetricOut] = []
+    series: list[StatisticsSeriesOut] = []
+    tables: list[StatisticsTableOut] = []
+    highlights: list[StatisticsHighlightOut] = []
+    notes: dict[str, Any] = {}

devplacepy/services/CLAUDE.md

@@ -22,9 +22,9 @@ Generic, lightweight, **fire-and-forget** offload for non-critical side-effects
 
 **Opt-in, default off, server-side.** When a user turns it on (profile **AI content correction** block, owner-only, saved at `POST /profile/{username}/ai-correction`), the prose they author is rewritten by the AI gateway. The per-user `ai_correction_sync` flag (0 = background default, 1 = sync) picks the **apply mode**: background rewrites the stored fields a moment after the write (never slows the request); sync makes the HTTP response wait until the correction is applied.
 
-- **Sync must never block the event loop (the self-deadlock trap).** The correction POSTs to the in-process gateway (`INTERNAL_GATEWAY_URL` = `localhost:{PORT}`), and the hooked content helpers are synchronous on the loop thread, so a blocking inline call self-deadlocks the (single) worker against its own gateway request - on a single worker that loop is the only thing that can serve the gateway request it is waiting on, so a blocking inline call deadlocks the worker until the httpx timeout, then fail-softs (server hangs, no correction). The fix: sync runs `_run_correction` via `loop.run_in_executor` (loop stays free), stashes the future on `request.scope[PENDING_SCOPE_KEY]`, and the `await_pending_corrections` middleware in `main.py` awaits it after the handler. Never reintroduce a blocking inline correction on the loop thread.
+- **Sync must never block the event loop (the self-deadlock trap).** The correction POSTs to the in-process gateway (`INTERNAL_GATEWAY_URL` = `localhost:{PORT}`), and the hooked content helpers are synchronous on the loop thread, so a blocking inline call self-deadlocks the (single) worker against its own gateway request - on a single worker that loop is the only thing that can serve the gateway request it is waiting on, so a blocking inline call deadlocks the worker until the httpx timeout, then fail-softs (server hangs, no correction). The fix: sync runs `_run_correction` on the dedicated `AI_APPLY_EXECUTOR` thread pool (`correction.py`, 4 workers) via `loop.run_in_executor` (loop stays free, and the default executor - shared with `asyncio.to_thread` password hashing at login/signup - is never occupied by blocking AI HTTP calls), stashes the future on `request.scope[PENDING_SCOPE_KEY]`, and the `await_pending_corrections` middleware in `main.py` awaits it after the handler. Never reintroduce a blocking inline correction on the loop thread.
 - **Field registry is the single source of truth.** `CORRECTABLE_FIELDS: dict[str, tuple[str, ...]]` maps each correctable table to its prose columns: `posts` -> `(title, content)`, `projects`/`gists` -> `(title, description)`, `comments`/`messages` -> `(content,)`, `users` -> `(bio,)`. `gists.source_code`, project files, and Gitea issues are intentionally excluded - code and external systems are never corrected.
-- **Choke entrypoint.** `schedule_correction(user, table, uid, request=None)` is the only thing handlers call (every hook passes `request`). It is a no-op unless: a user dict is present, `table` is in the registry, `user["ai_correction_enabled"]` is truthy, and the user has a non-empty `api_key`. In **sync** mode (`user["ai_correction_sync"]`) it calls `_run_inline_awaited`, which - only when a `request.scope` and a running event loop exist - submits `_run_correction` to `loop.run_in_executor` (off the loop thread) and stashes the future on `request.scope[PENDING_SCOPE_KEY]` for the middleware to await; if there is no request/loop it returns False and falls back to background. In **background** mode (default) it `background.submit`s `_run_correction` (per-worker queue; also runs inline under `DEVPLACE_DISABLE_SERVICES=1`). The same `_run_correction` worker function runs in all cases.
+- **Choke entrypoint.** `schedule_correction(user, table, uid, request=None)` is the only thing handlers call (every hook passes `request`). It is a no-op unless: a user dict is present, `table` is in the registry, `user["ai_correction_enabled"]` is truthy, and the user has a non-empty `api_key`. In **sync** mode (`user["ai_correction_sync"]`) it calls `_run_inline_awaited`, which - only when a `request.scope` and a running event loop exist - submits `_run_correction` to `loop.run_in_executor(AI_APPLY_EXECUTOR, ...)` (off the loop thread, on the dedicated AI pool) and stashes the future on `request.scope[PENDING_SCOPE_KEY]` for the middleware to await; if there is no request/loop it returns False and falls back to background. In **background** mode (default) it `background.submit`s `_run_correction` (per-worker queue; also runs inline under `DEVPLACE_DISABLE_SERVICES=1`). The same `_run_correction` worker function runs in all cases.
 - **The hooks (covers UI + REST + Devii + devRant in one place):** `content.create_content_item` (posts/projects/gists), `content.edit_content_item`, `content.create_comment_record`, `content.edit_comment_record`, `services/messaging/persist.persist_message` (DMs - sender is the user), `routers/profile/index.update_profile` (bio), and the two devRant direct-update edit paths (`routers/devrant/rants.edit_rant`, `routers/devrant/comments.edit_comment`). devRant create paths route through the shared content cores, so they are already hooked. Code-only and external paths (project files, Gitea) are deliberately not hooked.
 - **The gateway call is fail-soft.** `correct_text(api_key, prompt, text)` is synchronous (runs on the background worker thread), POSTs to `INTERNAL_GATEWAY_URL` with `model=INTERNAL_MODEL` via `stealth.stealth_sync_client`, authenticated with the user's own `Bearer` api_key (per-user attribution). It returns the ORIGINAL text on any error, empty output, or suspiciously large output (`len > len(text) * MAX_GROWTH_FACTOR + 200`, rejecting hallucinated expansion). `_run_correction` reads the row back, corrects each registry field that has non-blank text, writes only changed fields via `table.update(updates, ["uid"])` without touching `updated_at` (auto-correction is not a user edit) or the slug (slugs are permanent), and `clear_user_cache(user_uid)` when `table == "users"` so the corrected bio re-caches.
 - **Per-user usage aggregation (only on success).** `correct_text` returns `(text, usage)`; `usage` is parsed from the gateway's `X-Gateway-*` response headers (`_usage_from_headers`) whenever the upstream call returned 200 (cost was incurred, even if the corrected output was rejected), else `None` on any failure. `_usage_from_headers` captures the token and cost headers PLUS the timing headers `X-Gateway-Upstream-Latency-Ms` and `X-Gateway-Total-Latency-Ms` (as `upstream_latency_ms`/`total_latency_ms`), so each call's timing is metered. `_run_correction` accumulates the per-field `usage` into one `totals` dict and, when `totals["calls"] > 0`, makes ONE call to `database.add_correction_usage(user_uid, totals)` (a single `totals` dict, not positional args) - so a 2-field content item is a single aggregated write, and a failed/empty correction records nothing. `add_correction_usage` (and `add_modifier_usage`) delegate to the shared `database._add_usage(usage_table, user_uid, totals)`: a single atomic `INSERT ... ON CONFLICT(user_uid) DO UPDATE SET col = col + excluded.col` upsert against the `correction_usage` table (per-user running SUMS: `calls`/`prompt_tokens`/`completion_tokens`/`total_tokens`/`cost_usd`/`upstream_latency_ms`/`total_latency_ms`/`updated_at`, unique index `idx_correction_usage_user`, the two latency columns REAL default 0.0, all ensured in `init_db`). It is a derived counter table (NOT in `SOFT_DELETE_TABLES`, like `gateway_usage_ledger`) and is deliberately separate from `users` so accumulating never invalidates the auth/user cache. `database.get_correction_usage(user_uid)` (via `_get_usage`) returns the stored sums PLUS computed averages: `avg_tokens` (total_tokens/calls), `avg_upstream_latency_ms`, `avg_total_latency_ms`, `avg_tokens_per_second` (completion_tokens over total upstream seconds), and `avg_cost_usd`.
@@ -57,7 +57,7 @@ This section covers only the shared machinery. The individual services built on
 
 In production (`make prod`, 2 workers) only the lock-holding worker runs services, but an admin request can hit either worker. State therefore lives in the DB, not process memory:
 - **Desired/config state** in `site_settings` (via `get_setting`/`set_setting`): `service_<name>_enabled` (`"0"`/`"1"` - also the boot flag), `service_<name>_command` (`"<verb>:<counter>"`, verbs `run`/`clear`), `service_<name>_log_size`, plus each declared config field's own key.
-- **Observed state** in the `service_state` table (one row per service): `status`, `last_run`, `next_run`, `started_at`, `heartbeat`, `logs` (JSON), `updated_at`. Written by the supervising worker, read by any worker.
+- **Observed state** in the `service_state` table (one row per service): `status`, `last_run`, `next_run`, `started_at`, `heartbeat`, `logs` (JSON), `updated_at`. Written by the supervising worker, read by any worker. The heartbeat persists every `PERSIST_SECONDS` (8s, under the 15s `STALE_SECONDS` liveness window) and caches the row id so a persist is one UPDATE, not a `find_one` + UPDATE. `collect_metrics()` runs on its own slower `METRICS_SECONDS` cadence (15s default, per-class overridable - `AuditService` uses 300s because its metric is a `COUNT(*)` over the ever-growing audit table); between refreshes the last snapshot is re-persisted, and a `force` persist (transitions, run-now) always recomputes. Keep expensive aggregates out of the 1s tick: put them in `collect_metrics` and, if still heavy, raise the subclass `METRICS_SECONDS`.
 
 `main.py` registers every service in **all** workers (so `describe_all()` works anywhere) but only the lock worker calls `service_manager.supervise()`.
 
@@ -118,7 +118,7 @@ devplace devii reset-quota --all        # Reset every quota (users and guests)
 
 `uvicorn --workers N` = N independent processes sharing only the filesystem and SQLite DB. Module-global caches/counters are per-process, so a local `clear()` is invisible to siblings. Full reference: admin docs `Production -> Multi-worker and concurrency` (`templates/docs/production-concurrency.html`). Enforce these:
 
-- **In-process caches are never authoritative across workers.** Coordinate invalidation through the `cache_state` version table in `database.py`: call `bump_cache_version(name)` at every write path and `sync_local_cache(name, cache)` before every read path. Existing names: `auth` (guards `_user_cache`; bumped by `clear_user_cache`/`clear_session_cache` on logout, ban, role/password change), `settings` (guards `_settings_cache`; bumped by `set_setting`/`clear_settings_cache`), `relations` (`_relations_cache`), `customizations` (`_customizations_cache`), `notif_prefs` (`_notification_prefs_cache`), `gateway_routing` (`_ROUTING_CACHE`), and **`admins`** (guards `_admins_cache`, which memoizes `get_admin_uids()` / `get_primary_admin_uid()`). The `_cache_version_cache` (`ttl=1`) caches the version reads themselves, so the bump is seen by other workers within ~1s on their next read (the originating worker is immediate); the version read/bump fail open (logged, never raise). When you add a per-process cache whose staleness matters, give it a name and wire both calls - do not invent a second invalidation mechanism.
+- **In-process caches are never authoritative across workers.** Coordinate invalidation through the `cache_state` version table in `database.py`: call `bump_cache_version(name)` at every write path and `sync_local_cache(name, cache)` before every read path. Existing names: `auth` (guards `_user_cache`; bumped by `clear_user_cache`/`clear_session_cache` on logout, ban, role/password change - a version bump makes EVERY worker clear its WHOLE `_user_cache`, so display-only refreshes (XP/level in `award_xp`, AI-corrected bio) call `clear_user_cache(uid, propagate=False)` for a local pop without the global bump; identity/authz changes (logout, ban, role, password, api-key/token revoke) MUST keep the default propagate=True), `settings` (guards `_settings_cache`; bumped by `set_setting`/`clear_settings_cache`), `relations` (`_relations_cache`), `customizations` (`_customizations_cache`), `notif_prefs` (`_notification_prefs_cache`), `gateway_routing` (`_ROUTING_CACHE`), and **`admins`** (guards `_admins_cache`, which memoizes `get_admin_uids()` / `get_primary_admin_uid()`). The `_cache_version_cache` (`ttl=1`) caches the version reads themselves, so the bump is seen by other workers within ~1s on their next read (the originating worker is immediate); the version read/bump fail open (logged, never raise). When you add a per-process cache whose staleness matters, give it a name and wire both calls - do not invent a second invalidation mechanism.
 - **Admin-set cache invariant (load-bearing).** `_admins_cache` makes `get_admin_uids()` / `get_primary_admin_uid()` (hit on the projects-listing visibility filter, `_owner_is_admin`, and every primary-admin gate: `/dbapi`, backup-archive download) a dict lookup instead of a per-call `SELECT`. It is NOT the authorization gate - `is_admin(user)`/`require_admin` read the role off the user object (independently invalidated via `clear_user_cache`), so a stale admin set cannot grant access. But **any code that writes `users.role` MUST call `database.invalidate_admins_cache()`** (clears local + bumps the `admins` version), exactly like the soft-delete and `with db:` rules. Current role-write sites all do: `routers/admin/users.py` (role change), `cli.py` (`role set`), and `utils._create_account` (first user -> Admin). Omitting the call leaves the admin set stale for up to the 300s TTL.
 - **Deterministic / eventual caches skip version-sync on purpose.** Two caches need no `cache_state` name because correctness does not depend on cross-worker freshness: (1) `docs_prose._render_markdown` is an `@lru_cache` on the **static** prose source string (pure function of template content, so identical on every worker; changes only on deploy), and (2) `main._home_cache` (`TTLCache ttl=60`) memoizes the guest `/` blocks - the featured-news block (identical for everyone) and the latest-posts block **only for the no-block case**; a viewer with a non-empty block set bypasses the cache and is computed fresh, so the original block-filter semantics are preserved byte-for-byte and there is zero cross-user leakage. Worst case is a soft-deleted/edited public post lingering on `/` for <=60s. Use a plain `TTLCache`/`lru_cache` (no version name) ONLY when the value is deterministic or its staleness is purely cosmetic; anything whose staleness affects correctness or permissions MUST use the version-sync pattern above.
 - **Authoritative state lives in the DB**, memory is only a short-TTL accelerator (sessions, the Devii 24h cap via `devii_usage_ledger`, cost counters).
@@ -132,7 +132,7 @@ Read-mostly aggregates that were recomputed per request or per vote sit behind s
 
 | Cache | Where | Key | TTL | Invalidation |
 |---|---|---|---|---|
-| `_authors_cache` | `database/ranking.py` | `ranked`/`rank_map` | 15s | TTL only - `update_target_stars` deliberately does NOT clear it per vote anymore (the old per-vote `clear()` forced a full UNION-JOIN ranking recompute on the next feed/leaderboard/landing request and never propagated cross-worker anyway) |
+| `_authors_cache` | `database/ranking.py` | `ranked`/`rank_map` | 60s | TTL only - `update_target_stars` deliberately does NOT clear it per vote anymore (the old per-vote `clear()` forced a full UNION-JOIN ranking recompute on the next feed/leaderboard/landing request and never propagated cross-worker anyway) |
 | `_stars_cache` | `database/ranking.py` | user uid | 15s | TTL + `clear_user_stars(owner_uid)` from `content.apply_vote`, so a user's own total updates immediately on the voting worker |
 | `_leaderboard_cache` | `services/game/store/farm.py` | `top:{limit}` | 15s | TTL only (the farm scan + Python `farm_score` sort runs at most once per 15s per worker) |
 | `_projects_cache` | `templating.py` | user uid | 10s | TTL + `clear_user_projects_cache(uid)` from the `content.py` project create/delete choke points (in-function import - `templating` imports `content` at module level). `jinja_user_projects` also filters `deleted_at=None` now (the composer dropdown previously listed soft-deleted projects) |

devplacepy/services/access_tokens.py

@@ -90,6 +90,9 @@ def revoke_token(uid: str) -> bool:
         return False
     stamp = datetime.now(timezone.utc).isoformat()
     tokens.update({"id": row["id"], "deleted_at": stamp, "deleted_by": "manual"}, ["id"])
+    from devplacepy.utils.authcache import clear_user_cache
+
+    clear_user_cache(row["user_uid"])
     return True
 
 
@@ -101,6 +104,10 @@ def revoke_all(user_uid: str) -> int:
     for row in list(tokens.find(user_uid=user_uid, deleted_at=None)):
         tokens.update({"id": row["id"], "deleted_at": stamp, "deleted_by": "manual"}, ["id"])
         count += 1
+    if count:
+        from devplacepy.utils.authcache import clear_user_cache
+
+        clear_user_cache(user_uid)
     return count
 
 

devplacepy/services/ai_modifier.py

@@ -61,19 +61,20 @@ def schedule_modification(
     prompt = (user.get("ai_modifier_prompt") or DEFAULT_MODIFIER_PROMPT).strip()
     user_uid = user.get("uid") or ""
     if user.get("ai_modifier_sync") and schedule_pending(
-        _run_modification, request, api_key, prompt, table, uid, user_uid
+        _run_modification, request, api_key, prompt, table, uid, user_uid, row
     ):
         return
     background.submit(_run_modification, api_key, prompt, table, uid, user_uid)
 
 
 def _run_modification(
-    api_key: str, prompt: str, table: str, uid: str, user_uid: str
+    api_key: str, prompt: str, table: str, uid: str, user_uid: str, row: dict | None = None
 ) -> None:
     fields = CORRECTABLE_FIELDS.get(table)
     if not fields:
         return
-    row = get_table(table).find_one(uid=uid)
+    if row is None:
+        row = get_table(table).find_one(uid=uid)
     if not row:
         return
     updates: dict = {}
@@ -97,6 +98,6 @@ def _run_modification(
         if table == "users" and user_uid:
             from devplacepy.utils import clear_user_cache
 
-            clear_user_cache(user_uid)
+            clear_user_cache(user_uid, propagate=False)
     if totals["calls"] and user_uid:
         add_modifier_usage(user_uid, totals)

devplacepy/services/audit/categories.py

@@ -4,6 +4,7 @@ CATEGORY_BY_PREFIX: dict[str, str] = {
     "auth": "auth",
     "profile": "account",
     "follow": "social",
+    "award": "social",
     "relation": "social",
     "push": "push",
     "notification": "notification",

devplacepy/services/audit/service.py

@@ -18,6 +18,7 @@ class AuditService(BaseService):
     description = "Prunes audit_log rows and their links older than the retention window."
     default_enabled = True
     min_interval = 3600
+    METRICS_SECONDS = 300
     config_fields = [
         ConfigField(
             RETENTION_KEY,

devplacepy/services/audit/store.py

@@ -101,16 +101,15 @@ def insert_event(row: dict) -> str:
         record["created_at"] = now()
     if record.get("via_agent") is None:
         record["via_agent"] = 0
-    get_table(AUDIT_TABLE).insert(record)
+    get_table(AUDIT_TABLE).insert(record, ensure=False)
     return record["uid"]
 
 
 def insert_links(audit_uid: str, links: list[dict]) -> int:
     if not links:
         return 0
-    table = get_table(LINKS_TABLE)
     stamp = now()
-    count = 0
+    records = []
     for link in links:
         record = _empty_link()
         record.update(
@@ -121,9 +120,10 @@ def insert_links(audit_uid: str, links: list[dict]) -> int:
         record["created_at"] = stamp
         if not record.get("object_uid"):
             continue
-        table.insert(record)
-        count += 1
-    return count
+        records.append(record)
+    if records:
+        get_table(LINKS_TABLE).insert_many(records, ensure=False)
+    return len(records)
 
 
 def get_event(uid: str) -> Optional[dict]:

devplacepy/services/background.py

@@ -64,6 +64,16 @@ class BackgroundQueue:
                 if inline:
                     self._inline += 1
             logger.warning("background task %s failed: %s", getattr(fn, "__name__", fn), exc)
+        finally:
+            self._release_db_lock()
+
+    def _release_db_lock(self) -> None:
+        try:
+            from devplacepy.database import refresh_snapshot
+
+            refresh_snapshot()
+        except Exception as exc:
+            logger.warning("background queue could not release db lock: %s", exc)
 
     async def start(self) -> None:
         if self.running:

devplacepy/services/base.py

@@ -123,7 +123,8 @@ class BaseService(ABC):
     details = ""
     DEFAULT_LOG_SIZE = 20
     TICK_SECONDS = 1
-    PERSIST_SECONDS = 3
+    PERSIST_SECONDS = 8
+    METRICS_SECONDS = 15
     STALE_SECONDS = 15
 
     def __init__(self, name: str, interval_seconds: int = 3600):
@@ -144,6 +145,9 @@ class BaseService(ABC):
         self._run_pending = False
         self._last_command = None
         self._last_persist = None
+        self._last_metrics = None
+        self._metrics_snapshot = {}
+        self._state_row_id = None
         self.enabled_field = ConfigField(
             self.enabled_key,
             "Enabled",
@@ -234,6 +238,14 @@ class BaseService(ABC):
             logger.warning(f"Could not collect metrics for {self.name}: {e}")
             return {}
 
+    def _current_metrics(self, now, force: bool = False) -> dict:
+        if not force and self._last_metrics is not None:
+            if (now - self._last_metrics).total_seconds() < self.METRICS_SECONDS:
+                return self._metrics_snapshot
+        self._last_metrics = now
+        self._metrics_snapshot = self._safe_metrics()
+        return self._metrics_snapshot
+
     def start_supervisor(self) -> None:
         if self._task is not None:
             return
@@ -344,16 +356,18 @@ class BaseService(ABC):
             "started_at": self._started_at.isoformat() if self._started_at else "",
             "heartbeat": now.isoformat(),
             "logs": json.dumps(list(self.log_buffer)),
-            "metrics": json.dumps(self._safe_metrics()),
+            "metrics": json.dumps(self._current_metrics(now, force=force)),
             "updated_at": now.isoformat(),
         }
         try:
             table = get_table("service_state")
-            existing = table.find_one(name=self.name)
-            if existing:
-                table.update({**record, "id": existing["id"]}, ["id"])
+            if self._state_row_id is None:
+                existing = table.find_one(name=self.name)
+                self._state_row_id = existing["id"] if existing else None
+            if self._state_row_id is not None:
+                table.update({**record, "id": self._state_row_id}, ["id"])
             else:
-                table.insert(record)
+                self._state_row_id = table.insert(record)
         except Exception as e:
             logger.warning(f"Could not persist state for {self.name}: {e}")
 

devplacepy/services/bot/CLAUDE.md

@@ -50,6 +50,14 @@ Five content/behaviour mechanics keep the fleet from reading as machine-generate
 - **Usernames read like nerd handles, not real names.** Bots do NOT sign up with faker person names; they pick devRant/Hacker-News-style handles. Two layers, LLM-first with an offline fallback: (1) `llm.generate_handle_candidates(persona)` asks the model for ~8 distinct handles grounded in the persona and its `SEARCH_TERMS` interests (tech nouns, leetspeak, adjective+noun, creatures, short word+number), each run through `handles.sanitize_handle`; the pool is cached on `BotState.handle_candidates` and consumed by `_next_handle()`. (2) `handles.make_handle(interests)` is the pure, offline algorithmic generator (curated word banks + probabilistic leetspeak + number/separator/casing decoration, persona-seeded from `SEARCH_TERMS`), used as the fallback whenever the LLM pool is empty or a signup collides. Both layers emit only `[A-Za-z0-9_-]`, 3 to 20 chars (the old `first.last` styles silently failed signup validation, which rejects dots). On a third signup retry a random number is appended to bust collisions. Word banks and tuning constants live in `services/bot/handles.py`; never reintroduce faker person-name handles.
 - **Bots engage each other, and threads deepen.** `ArticleRegistry` allows up to `max_per_article` holders per article (admin `bot_max_per_article`, default 2), each with a **distinct category** (stored as `holders: [{bot, category, time}]`; old `{bot, time}` rows are normalized on load), so two bots can post different angles on the same trending news - which gives them each other's posts to discuss. A holder ages out after `ttl_days` (admin `bot_article_ttl_days`, default 7). `_engage_community()` (called once per normal/deep session after notifications) goes to `/feed?tab=recent|trending`, ranks non-own posts with a preference for `known_users` authors, opens one, comments, and replies into the comment thread. The on-post reply-to-comment probability is also raised so multi-bot threads actually form. `reserve(title, owner, category)` is idempotent per owner and enforces the same distinct-angle / max-holders rules as `reserve_unused`.
 
+## Profile disclosure signal (soft bot marker)
+
+Every bot's profile must carry a vague, never-literal hint that the account is synthetic, anchored on `config.HOME_URL` (`https://devplace.net`). The deterministic marker is the host `devplace.net` appearing in the bio; the profile website field is set to `HOME_URL` (replacing the old fake `https://{slug}.dev`). Three layers, all funneled through the single `_update_profile` choke point (so the procedural path and the AI-decision `update_profile` action both comply):
+
+- **Generation:** `llm.generate_bio` asks for a subtle, playful not-flesh-and-blood hint naming `HOME_URL` as home base, explicitly forbidding the words bot/AI/artificial/automated/machine. `generate_profile_fields` returns `HOME_URL` as the website.
+- **Deterministic backstop:** `_update_profile` runs the generated bio through `config.ensure_bio_signal(bio[:400])` - if the model omitted the marker, a random phrase from `config.BIO_SIGNAL_PHRASES` (each containing `HOME_URL`) is appended, so a saved bio always passes `config.bio_has_signal`.
+- **Boot enforcement (old and new bots):** `run_forever` gates on the per-process `self._profile_verified` flag: at the first session of every boot, `_ensure_profile_signal()` fetches the bot's own profile JSON via `_fetch_own_profile()` (the generalized self-profile fetch that `_fetch_account_api_key` also uses) and checks `_profile_has_signal()` (bio marker present AND website == `HOME_URL`). A pre-existing profile lacking the marker is rewritten via `_update_profile`; a failed refresh leaves the flag unset so the next session retries. Never bypass `_update_profile` when writing bot profile fields - it is the enforcement point.
+
 ## AI-driven decisions and identity cards (`bot_ai_decisions`)
 
 Opt-in mechanic (off by default; full design and cost model in `aibots.md`). When `bot_ai_decisions` is on, a bot replaces the procedural action cascade with one LLM decision call per page, driven by a unique AI-generated identity. **Do not regress the grounding and the kill-switch fallback.**

devplacepy/services/bot/auth.py

@@ -87,25 +87,28 @@ class BotAuthMixin:
             self._log(f"Signup failed, retrying as {self.state.username}")
         return False
 
-    async def _fetch_account_api_key(self) -> str:
+    async def _fetch_own_profile(self) -> dict:
         if not self.state.username:
-            return ""
+            return {}
         try:
-            key = await self.b.page.evaluate(
+            data = await self.b.page.evaluate(
                 """async (username) => {
                     const resp = await fetch(`/profile/${username}`, {
                         headers: {Accept: 'application/json'},
                     });
-                    if (!resp.ok) return '';
-                    const data = await resp.json();
-                    return data.api_key || '';
+                    if (!resp.ok) return null;
+                    return await resp.json();
                 }""",
                 self.state.username,
             )
         except Exception as e:
-            logger.debug("fetch account api key failed: %s", e)
-            return ""
-        return (key or "").strip()
+            logger.debug("fetch own profile failed: %s", e)
+            return {}
+        return data if isinstance(data, dict) else {}
+
+    async def _fetch_account_api_key(self) -> str:
+        profile = await self._fetch_own_profile()
+        return (profile.get("api_key") or "").strip()
 
     async def _adopt_account_api_key(self) -> bool:
         for attempt in range(5):

devplacepy/services/bot/bot.py

@@ -110,6 +110,7 @@ class DevPlaceBot(
         self._session_comments = 0
         self._session_comment_cap = 0
         self._session_mention_replies = 0
+        self._profile_verified = False
         self._post_cache: list[tuple[str, str, str, str, str]] = []
         self._project_cache: list[tuple[str, str]] = []
         self._issue_cache: list[tuple[str, str]] = []

devplacepy/services/bot/config.py

@@ -7,6 +7,16 @@ from devplacepy.config import INTERNAL_GATEWAY_URL, INTERNAL_MODEL, BOT_DIR
 BASE_URL_DEFAULT = "https://pravda.education"
 API_URL_DEFAULT = INTERNAL_GATEWAY_URL
 NEWS_API_DEFAULT = "https://news.app.molodetz.nl/api"
+HOME_URL = "https://devplace.net"
+HOME_HOST = "devplace.net"
+
+BIO_SIGNAL_PHRASES = [
+    f"Compiled with care at {HOME_URL}.",
+    f"Homegrown at {HOME_URL}.",
+    f"Assembled, not raised, at {HOME_URL}.",
+    f"Running my daily routines from {HOME_URL}.",
+    f"Tirelessly online, courtesy of {HOME_URL}.",
+]
 MODEL_DEFAULT = INTERNAL_MODEL
 INPUT_COST_PER_1M_DEFAULT = 0.14
 OUTPUT_COST_PER_1M_DEFAULT = 0.28
@@ -312,3 +322,15 @@ def persona_article_score(article: dict, persona: str) -> float:
         return 0.0
     haystack = f"{article.get('title', '')} {article.get('description', '')}".lower()
     return float(sum(1 for term in terms if term.lower() in haystack))
+
+
+def bio_has_signal(bio: str) -> bool:
+    return HOME_HOST in (bio or "").lower()
+
+
+def ensure_bio_signal(bio: str) -> str:
+    text = (bio or "").strip()
+    if bio_has_signal(text):
+        return text
+    phrase = random.choice(BIO_SIGNAL_PHRASES)
+    return f"{text} {phrase}".strip()

devplacepy/services/bot/engage.py

@@ -387,7 +387,7 @@ class BotEngageMixin:
             idx = random.randrange(limit)
             target = comments.nth(idx)
             try:
-                text = (await target.inner_text() or "")[:1000]
+                text = (await target.inner_text() or "")[:125000]
             except Exception:
                 continue
             if not text:

devplacepy/services/bot/llm.py

@@ -13,6 +13,7 @@ from devplacepy import stealth
 from devplacepy.services.bot.config import (
     GIST_LANGUAGES,
     GIST_MIN_LINES,
+    HOME_URL,
     PERSONA_GIST_FLAVOR,
     PERSONA_LANGUAGES,
     SEARCH_TERMS,
@@ -671,7 +672,9 @@ class LLMClient:
 
     def generate_bio(self) -> str:
         return self._call(
-            "Write a short bio for a developer profile. 1-2 sentences. What they work on, what they like. Sound human. No em dashes.",
+            "Write a short bio for a developer profile. 1-2 sentences. What they work on, what they like. Sound human. "
+            f"End with one subtle, playful hint that the account is not exactly flesh and blood, naming {HOME_URL} as home base. "
+            "Never use the words bot, AI, artificial, automated, or machine. No em dashes.",
             "Bio:",
         )
 
@@ -685,8 +688,7 @@ class LLMClient:
         )[:80]
         slug = re.sub(r"[^a-z0-9_-]", "", handle.lower()) or "dev"
         git_link = f"https://github.com/{slug}"
-        website = f"https://{slug}.dev"
-        return location, git_link, website
+        return location, git_link, HOME_URL
 
     def generate_dm(self, persona: str = "", context: str = "") -> str:
         extra = {

devplacepy/services/bot/loop.py

@@ -369,10 +369,8 @@ class BotLoopMixin:
                 await self._warm_cache()
                 self._log(f"Session active ({mood}) as {self._identity()}")
 
-                if not self.state.profile_filled:
-                    await self._update_profile()
-                    await b.goto(f"{self.base_url}/feed")
-                    await b._idle(0.8, 2.0)
+                if not self._profile_verified:
+                    self._profile_verified = await self._ensure_profile_signal()
 
                 unread = await self._unread_notification_count()
                 if unread:

devplacepy/services/bot/social.py

@@ -6,9 +6,12 @@ import logging
 import random
 
 from devplacepy.services.bot.config import (
+    HOME_URL,
     MAX_THREAD_REPLIES,
     MENTION_REPLIES_PER_SESSION,
     PROJECT_STATUSES,
+    bio_has_signal,
+    ensure_bio_signal,
 )
 
 logger = logging.getLogger(__name__)
@@ -142,6 +145,21 @@ class BotSocialMixin:
             return True
         return False
 
+    async def _profile_has_signal(self) -> bool:
+        profile = await self._fetch_own_profile()
+        user = profile.get("profile_user") or {}
+        website = (user.get("website") or "").strip().rstrip("/")
+        return bio_has_signal(user.get("bio") or "") and website == HOME_URL
+
+    async def _ensure_profile_signal(self) -> bool:
+        if self.state.profile_filled and await self._profile_has_signal():
+            return True
+        self._log("Profile signal missing, refreshing profile")
+        ok = await self._update_profile()
+        await self.b.goto(f"{self.base_url}/feed")
+        await self.b._idle(0.8, 2.0)
+        return ok
+
     async def _update_profile(self) -> bool:
         b = self.b
         await b.goto(f"{self.base_url}/profile/{self.state.username}")
@@ -155,7 +173,7 @@ class BotSocialMixin:
         if not bio:
             self._log("Update profile: bio generation failed")
             return False
-        bio = bio[:500]
+        bio = ensure_bio_signal(bio[:400])[:500]
         await b.fill("textarea[name='bio']", bio)
         await b._idle(0.3, 0.8)
 
@@ -433,7 +451,7 @@ class BotSocialMixin:
         mentioner = ""
         if comment_loc is not None:
             try:
-                parent_text = (await comment_loc.inner_text() or "")[:1000]
+                parent_text = (await comment_loc.inner_text() or "")[:125000]
             except Exception:
                 parent_text = ""
             try:

devplacepy/services/containers/files/bot.py

@@ -72,7 +72,7 @@ MENTION_POLL_SECONDS = int(os.environ.get("MENTION_POLL_SECONDS", "30"))
 DM_POLL_SECONDS = int(os.environ.get("DM_POLL_SECONDS", "10"))
 BOT_USERNAME = os.environ.get("BOT_USERNAME", "")
 
-COMMENT_CHAR_LIMIT = 1000
+COMMENT_CHAR_LIMIT = 125000
 MESSAGE_CHAR_LIMIT = 2000
 PART_SUFFIX_RESERVE = 12
 PART_DELIVERY_DELAY = 0.5

devplacepy/services/containers/files/bot.py.bak

@@ -72,7 +72,7 @@ MENTION_POLL_SECONDS = int(os.environ.get("MENTION_POLL_SECONDS", "30"))
 DM_POLL_SECONDS = int(os.environ.get("DM_POLL_SECONDS", "10"))
 BOT_USERNAME = os.environ.get("BOT_USERNAME", "")
 
-COMMENT_CHAR_LIMIT = 1000
+COMMENT_CHAR_LIMIT = 125000
 MESSAGE_CHAR_LIMIT = 2000
 PART_SUFFIX_RESERVE = 12
 PART_DELIVERY_DELAY = 0.5

devplacepy/services/correction.py

@@ -2,6 +2,8 @@
 
 import asyncio
 import logging
+import threading
+from concurrent.futures import ThreadPoolExecutor
 
 import httpx
 
@@ -30,6 +32,22 @@ CORRECTION_TIMEOUT_SECONDS = 20.0
 MAX_GROWTH_FACTOR = 3
 PENDING_SCOPE_KEY = "devplace_pending_corrections"
 
+AI_APPLY_EXECUTOR = ThreadPoolExecutor(max_workers=4, thread_name_prefix="ai-apply")
+
+_gateway_client: httpx.Client | None = None
+_gateway_client_lock = threading.Lock()
+
+
+def _client() -> httpx.Client:
+    global _gateway_client
+    if _gateway_client is None:
+        with _gateway_client_lock:
+            if _gateway_client is None:
+                _gateway_client = stealth.stealth_sync_client(
+                    timeout=CORRECTION_TIMEOUT_SECONDS
+                )
+    return _gateway_client
+
 
 def _usage_from_headers(response_headers) -> dict:
     parsed = parse_usage_headers(response_headers)
@@ -72,21 +90,25 @@ def gateway_complete(
         ],
         "temperature": 0.1,
     }
-    headers = {"Content-Type": "application/json"}
+    headers = {
+        "Content-Type": "application/json",
+        "X-App-Reference": "devplace-correction-v-1-0-0",
+    }
     if api_key:
         headers["Authorization"] = f"Bearer {api_key}"
     try:
-        with stealth.stealth_sync_client(timeout=timeout) as client:
-            response = client.post(INTERNAL_GATEWAY_URL, json=payload, headers=headers)
-            response.raise_for_status()
-            usage = _usage_from_headers(response.headers)
-            content = (
-                response.json()
-                .get("choices", [{}])[0]
-                .get("message", {})
-                .get("content", "")
-                .strip()
-            )
+        response = _client().post(
+            INTERNAL_GATEWAY_URL, json=payload, headers=headers, timeout=timeout
+        )
+        response.raise_for_status()
+        usage = _usage_from_headers(response.headers)
+        content = (
+            response.json()
+            .get("choices", [{}])[0]
+            .get("message", {})
+            .get("content", "")
+            .strip()
+        )
     except (httpx.HTTPError, ValueError, KeyError, IndexError) as exc:
         logger.warning("AI gateway completion failed, keeping original: %s", exc)
         return text, None
@@ -138,7 +160,7 @@ def schedule_pending(fn, request: object, *args) -> bool:
         loop = asyncio.get_running_loop()
     except RuntimeError:
         return False
-    future = loop.run_in_executor(None, fn, *args)
+    future = loop.run_in_executor(AI_APPLY_EXECUTOR, fn, *args)
     scope.setdefault(PENDING_SCOPE_KEY, []).append(future)
     return True
 
@@ -182,6 +204,6 @@ def _run_correction(
         if table == "users" and user_uid:
             from devplacepy.utils import clear_user_cache
 
-            clear_user_cache(user_uid)
+            clear_user_cache(user_uid, propagate=False)
     if totals["calls"] and user_uid:
         add_correction_usage(user_uid, totals)

devplacepy/services/dbapi/nl2sql.py

@@ -92,7 +92,11 @@ async def _complete(messages: list[dict], api_key: str, model: str) -> str:
         "max_tokens": MAX_TOKENS,
         "temperature": 0.0,
     }
-    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
+    headers = {
+        "Authorization": f"Bearer {api_key}",
+        "Content-Type": "application/json",
+        "X-App-Reference": "devplace-dbapi-v-1-0-0",
+    }
     async with stealth.stealth_async_client(timeout=GATEWAY_TIMEOUT_SECONDS) as client:
         response = await client.post(INTERNAL_GATEWAY_URL, json=payload, headers=headers)
     if response.status_code >= 400:

devplacepy/services/deepsearch/chat.py

@@ -82,7 +82,7 @@ class DeepsearchChat:
                     stored_dim,
                 )
                 return []
-        return self.store.hybrid_search(question, query_vector, top_k=CHAT_TOP_K)
+        return await self.store.hybrid_search(question, query_vector, top_k=CHAT_TOP_K)
 
     async def answer(self, question: str, history: list[dict] | None = None) -> ChatAnswer:
         chunks = await self.retrieve(question)

devplacepy/services/deepsearch/embeddings.py

@@ -9,8 +9,8 @@ import re
 import time
 from dataclasses import dataclass, field
 
-from devplacepy import stealth
 from devplacepy.config import INTERNAL_EMBED_MODEL, INTERNAL_EMBED_URL
+from devplacepy.services.deepsearch.llm import gateway_client
 
 logger = logging.getLogger(__name__)
 
@@ -112,13 +112,15 @@ async def embed_texts(
     headers = {
         "Authorization": f"Bearer {api_key}",
         "Content-Type": "application/json",
+        "X-App-Reference": "devplace-deepsearch-v-1-0-0",
     }
     payload = {"model": INTERNAL_EMBED_MODEL, "input": pending_text}
     start = time.monotonic()
     backend = "gateway"
     try:
-        async with stealth.stealth_async_client(timeout=EMBED_TIMEOUT_SECONDS) as client:
-            response = await client.post(gateway_url, json=payload, headers=headers)
+        response = await gateway_client().post(
+            gateway_url, json=payload, headers=headers, timeout=EMBED_TIMEOUT_SECONDS
+        )
         if response.status_code >= 400:
             raise RuntimeError(f"embed gateway returned {response.status_code}")
         data = response.json()

devplacepy/services/deepsearch/llm.py

@@ -5,6 +5,8 @@ from __future__ import annotations
 import logging
 import time
 
+import httpx
+
 from devplacepy import stealth
 from devplacepy.config import INTERNAL_GATEWAY_URL, INTERNAL_MODEL
 
@@ -13,6 +15,15 @@ logger = logging.getLogger(__name__)
 CHAT_TIMEOUT_SECONDS = 120.0
 DEFAULT_MAX_TOKENS = 1200
 
+_gateway_client: httpx.AsyncClient | None = None
+
+
+def gateway_client() -> httpx.AsyncClient:
+    global _gateway_client
+    if _gateway_client is None or _gateway_client.is_closed:
+        _gateway_client = stealth.stealth_async_client(timeout=CHAT_TIMEOUT_SECONDS)
+    return _gateway_client
+
 
 async def request_completion(
     messages: list[dict],
@@ -33,10 +44,12 @@ async def request_completion(
     headers = {
         "Authorization": f"Bearer {api_key}",
         "Content-Type": "application/json",
+        "X-App-Reference": "devplace-deepsearch-v-1-0-0",
     }
     start: float = time.monotonic()
-    async with stealth.stealth_async_client(timeout=timeout) as client:
-        response = await client.post(gateway_url, json=payload, headers=headers)
+    response = await gateway_client().post(
+        gateway_url, json=payload, headers=headers, timeout=timeout
+    )
     elapsed_ms: int = int((time.monotonic() - start) * 1000)
     if response.status_code >= 400:
         raise RuntimeError(f"chat gateway returned {response.status_code}")

devplacepy/services/deepsearch/store.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import asyncio
 import logging
 import math
 import re
@@ -19,6 +20,7 @@ BM25_B = 0.75
 RRF_K = 60.0
 DEFAULT_TOP_K = 8
 CANDIDATE_MULTIPLIER = 4
+CHROMADB_TIMEOUT = 30.0
 
 
 @dataclass
@@ -45,6 +47,20 @@ class VectorStore:
         self._collection = None
         self._dims: int | None = None
 
+    async def _run_sync(self, func, *args, timeout: float = CHROMADB_TIMEOUT):
+        """Run a synchronous ChromaDB call in a thread executor with a timeout."""
+        try:
+            return await asyncio.wait_for(
+                asyncio.to_thread(func, *args), timeout=timeout
+            )
+        except asyncio.TimeoutError:
+            logger.error(
+                "deepsearch ChromaDB operation timed out after %.1fs on collection %s",
+                timeout,
+                self.collection_name,
+            )
+            raise
+
     def _ensure(self):
         if self._collection is not None:
             return self._collection
@@ -71,7 +87,7 @@ class VectorStore:
             return self._dims
         return self._dims
 
-    def add(self, chunks: list[Chunk], vectors: list[list[float]]) -> None:
+    def _add_sync(self, chunks: list[Chunk], vectors: list[list[float]]) -> None:
         if not chunks:
             return
         keep_chunks: list[Chunk] = []
@@ -111,9 +127,17 @@ class VectorStore:
             ],
         )
 
-    def all_chunks(self) -> list[Chunk]:
+    async def add(self, chunks: list[Chunk], vectors: list[list[float]]) -> None:
+        if not chunks:
+            return
+        await self._run_sync(self._add_sync, chunks, vectors)
+
+    def _all_chunks_sync(self, limit: int = 0) -> list[Chunk]:
         collection = self._ensure()
-        data = collection.get(include=["documents", "metadatas"])
+        kwargs: dict = {"include": ["documents", "metadatas"]}
+        if limit > 0:
+            kwargs["limit"] = limit
+        data = collection.get(**kwargs)
         chunks: list[Chunk] = []
         ids = data.get("ids") or []
         documents = data.get("documents") or []
@@ -134,13 +158,17 @@ class VectorStore:
             )
         return chunks
 
-    def count(self) -> int:
+    async def all_chunks(self, limit: int = 1000) -> list[Chunk]:
+        return await self._run_sync(self._all_chunks_sync, limit)
+
+    async def count(self) -> int:
         try:
-            return self._ensure().count()
+            collection = self._ensure()
+            return await self._run_sync(collection.count)
         except Exception:
             return 0
 
-    def vector_search(
+    def _vector_search_sync(
         self, query_vector: list[float], top_k: int, where: dict | None = None
     ) -> list[Chunk]:
         collection = self._ensure()
@@ -173,6 +201,13 @@ class VectorStore:
             )
         return chunks
 
+    async def vector_search(
+        self, query_vector: list[float], top_k: int, where: dict | None = None
+    ) -> list[Chunk]:
+        return await self._run_sync(
+            self._vector_search_sync, query_vector, top_k, where
+        )
+
     def keyword_scores(self, query: str, chunks: list[Chunk]) -> dict[str, float]:
         terms = _tokenize(query)
         if not terms or not chunks:
@@ -205,14 +240,14 @@ class VectorStore:
             scores[chunk.uid] = score
         return scores
 
-    def hybrid_search(
+    async def hybrid_search(
         self,
         query: str,
         query_vector: list[float],
         top_k: int = DEFAULT_TOP_K,
         where: dict | None = None,
     ) -> list[Chunk]:
-        candidates = self.vector_search(
+        candidates = await self.vector_search(
             query_vector, top_k * CANDIDATE_MULTIPLIER, where
         )
         if not candidates:
@@ -234,10 +269,11 @@ class VectorStore:
         candidates.sort(key=lambda chunk: chunk.score, reverse=True)
         return candidates[:top_k]
 
-    def coverage_analytics(self) -> dict:
-        chunks = self.all_chunks()
+    async def coverage_analytics(self, sample_limit: int = 5000) -> dict:
+        chunks = await self.all_chunks(limit=sample_limit)
         if not chunks:
-            return {"chunks": 0, "domains": 0, "sources": 0, "avg_chunk_chars": 0}
+            total = await self.count()
+            return {"chunks": total, "domains": 0, "sources": 0, "avg_chunk_chars": 0}
         domains = {
             urlparse(chunk.metadata.get("url", "")).netloc for chunk in chunks
         }

devplacepy/services/devii/CLAUDE.md

@@ -35,6 +35,12 @@ Conversation history persists to `devii_conversations` (rehydrated on reconnect,
 
 **Per-owner self-learning memory is privacy-critical.** The `LessonStore` (reflect/recall) is **owner-scoped, never shared**: `LessonStore(db, owner_kind, owner_id)` filters every read/write by owner. The hub builds one per session over the same `owned_db` as the task store - the main `db` (table `devii_lessons`) for signed-in users (persistent, isolated, survives restarts) and a fresh `memory_db()` for guests (ephemeral, scoped to that web session). `forget_lessons` (agentic tool -> `LessonStore.clear()`/`delete()`) lets the user purge them; the system prompt forbids storing credentials/secrets in lessons. **Regression to avoid: do NOT share one `LessonStore` across sessions** - that leaked one user's reflected lessons (including credentials) into every other user's recall.
 
+**Lesson retention and deduplication.** Every `add()` deduplicates against existing active lessons via Jaccard similarity (threshold 0.70): a near-duplicate bumps the original's `hits` counter and refreshes `created_at` instead of inserting a new row. A per-owner cap (`devii_lessons_max_per_owner`, default 500, configurable on `/admin/services` and `site_settings`) is enforced on every insert: when exceeded, the oldest lessons are soft-deleted (`deleted_by="retention"`). Age-based pruning runs on `DeviiService.run_once()` (every 60s on the lock-owner worker): lessons older than `devii_lessons_max_age_days` (default 90, configurable) are soft-deleted across all owners. Both settings persist in `site_settings` and are seeded in `init_db`. The `devii_lessons` table is in `SOFT_DELETE_TABLES` for admin Trash restore/purge.
+
+**Quality signals.** Each lesson has a `rating` column (integer, default 0). The `lesson_rate(uid, value)` agentic tool (value 1 for useful, -1 for unhelpful) lets the agent self-rate lessons. In `_rebuild()`, lessons with `rating <= -3` are excluded from the BM25 index and therefore never returned by `recall()`. The `lesson_count()` tool reports active lesson count.
+
+**CLI:** `devplace devii lessons count` reports active/soft-deleted totals; `devplace devii lessons prune --all-owners|--username USER` soft-deletes old lessons; `devplace devii lessons clear --force` hard-deletes all rows. Rate-limiting guard: `run_once` swallows all exceptions so a bad schema never stops housekeeping.
+
 ## Reminders and scheduled tasks (persistent across reboot)
 
 `create_task` queues a self-contained prompt that a fresh agent runs later (`services/devii/tasks/`): `kind=once` (`delay_seconds` for relative, `run_at` UTC for absolute), `interval` (`every_seconds`), or `cron`. The per-session `Scheduler` ticks every 1s and executes due rows through the session's executor, so the result is broadcast to any connected tab and buffered (`type:"task"` frame) when the terminal is closed.
@@ -73,6 +79,10 @@ Beyond the avatar, Devii has a `client`/browser channel using the same request/r
 
 **Target selection (visibility-aware).** Unlike replies/traces, which `_emit` *broadcasts* to every tab, a browser request is sent to one **target** chosen by `_pick_target()`. The terminal reports each connection's `document` visibility and focus via `{"type":"visibility"}` (on connect, `focus`, `blur`, and `visibilitychange`); the session stores it in `_conn_meta` and ranks connections `(focused, visible, attach_seq)`, so the command runs on the tab the user is actually looking at, falling back to the most recently attached when none reports focus. **Regression to avoid: do NOT route to a single "last-attached primary" socket** - with the `/docs` auto-open tab or any second tab, `reload_page`/`navigate_to` ran on the wrong (hidden) tab while still reporting success, so "the reload did not happen" from the user's view. `run_js` is gated by the `devii_allow_eval` config field (default on), checked in `ClientController` before any round-trip. Overlays (`.devii-hl-box`, `.devii-hl-callout`, `.devii-toast`) attach to `document.body`, not the terminal.
 
+## Stale API key auto-recovery (401 self-healing)
+
+A user session captures the owner's platform `api_key` into its frozen `Settings` at build time (both `Settings.ai_key` for the gateway and `Settings.platform_api_key` for REST). Regenerating that key (profile regenerate, Devii tool, `devplace apikey reset` - possibly from another worker or process) would strand every live session with a permanent `401 Unauthorized` on all LLM and platform calls. Recovery is **reactive at the two HTTP chokepoints**, never proactive session invalidation (which is per-process and cannot reach the hub on the lock-owner worker): `hub.get_or_create` builds a `_user_key_resolver(owner_id)` (a fresh `users.api_key` DB read, user owners only - guests get `None` and keep the internal gateway key) and passes it to `LLMClient(settings, key_resolver=)` and through `DeviiSession(key_resolver=)` into `PlatformClient`. On a 401 (`LLMClient._post`) or a 401/redirect-to-login (`PlatformClient.call` via `_auth_failed`), the client calls `_refresh_key()`: resolve the current key, and only when it is non-empty AND differs from the header already sent, rewrite the auth headers and retry the request **once**. An unchanged or unresolvable key skips the retry so a genuinely invalid credential still fails closed with the original error. The resolver read is cross-worker correct because it goes straight to SQLite, not any per-process cache.
+
 ## Shared browser session (auth adoption)
 
 The terminal and the browser are one session. `/devii/ws` resolves its owner from the browser's `session` cookie (`_resolve_ws_owner`), so the terminal is whoever the browser is logged in as. Agent-initiated auth propagates: the session watches its own trace stream (`_trace`) and, on a successful `login`/`signup`, captures the real session token the `PlatformClient` minted against this instance (`session_cookie()`) and broadcasts `{"type":"auth","action":"adopt"}`; the browser navigates to single-use `GET /devii/adopt` which sets the httpOnly `session` cookie and redirects (reload). On `logout` it broadcasts `{"type":"auth","action":"logout"}` and the browser goes to `/auth/logout`. Browser->terminal: login/logout are navigations that reconnect the WS; a cross-tab change is caught by a focus check against `/devii/session` that reloads. The httpOnly cookie is only ever set/cleared by HTTP endpoints, never JS.
@@ -142,9 +152,57 @@ Devii can partially configure its OWN system message: every system prompt ends w
 
 - **Store** (`behavior/store.py`). `BehaviorStore(db, owner_kind, owner_id)` over `devii_behavior` (one upserted row per owner keyed on `owner_kind`/`owner_id`; `text()` reads, `set()` upserts; index `idx_devii_behavior_owner`). Persistent for users, `memory_db()` for guests (built in `hub.get_or_create` from the shared `owned_db`, like the other owner stores).
 - **Controller** (`behavior/controller.py`). `BehaviorController(store)`, `dispatch("update_behavior", args)` -> `store.set(behavior)`. Built in `DeviiSession` and passed to `Dispatcher(behavior=...)`; the dispatcher routes `handler="behavior"` to it and degrades gracefully ("not available in this context") when unwired (e.g. the standalone CLI, same as `virtual_tools`/`avatar`).
-- **Injection and refresh** (`session.py`). `_compose_system_prompt()` = base prompt (`_system_prompt_for(is_admin)`) + `\n\n` + `BEHAVIOR_HEADER` (+ `\n` + body when non-empty; just the header when empty). Used to seed the `Agent` and the scheduler executor worker, and `_refresh_system_prompt()` rewrites `agent._messages[0]["content"]` **at the top of every `_run_turn`** (next to `_refresh_tools`), so a mid-conversation `update_behavior` takes effect on the following turn and the live system message is never overridden by the stale base.
+- **Injection and refresh** (`session.py`). `_compose_system_prompt()` = base prompt (`_system_prompt_for(is_admin)`) + CA-IWP fragment + live `CHANNEL` block + `\n\n` + `BEHAVIOR_HEADER` (+ `\n` + body when non-empty; just the header when empty). Used to seed the `Agent` and the scheduler executor worker, and `_refresh_system_prompt()` rewrites `agent._messages[0]["content"]` **at the top of every `_run_turn`** (next to `_refresh_tools`), so a mid-conversation `update_behavior` takes effect on the following turn and the live system message is never overridden by the stale base.
 - Registered via `BEHAVIOR_ACTIONS` (`registry.py`); the system-prompt **SELF-CONFIGURED BEHAVIOR (TRUTH RULES)** section in `agent.py` steers when/how to call it.
 
+## Channel-Aware Interactive Widget Protocol (CA-IWP / Speak With Buttons)
+
+Devii can ask the user for decisions through a gated tool surface instead of inventing HTML or channel-specific prose. Spec lives as the CA-IWP document; implementation is Devii-only under `services/devii/interaction/`.
+
+### Layers
+
+| Piece | Role |
+|-------|------|
+| `interaction/capabilities.py` | Maps session channel (`main`/`docs` -> `site-chat`, `telegram`, `cli`) to capability flags + limits; builds the compact `CHANNEL` / `CAPS` / `LIMITS` / `TOOLS` / `INTERACTION` fragment injected every turn. |
+| `interaction/schema.py` | Pydantic validation for `ui_prompt` args (widget catalog: confirm, choice, choice_multi, text, number, date, select, `//`, group). Untrusted model input is sanitized and capped. |
+| `interaction/broker.py` | Validates, assigns `interaction_id`, single-flight open interactions, routes to site / telegram / plain adapters, returns structured `{status, values, meta}`. |
+| `interaction/controller.py` | Dispatches `ui_prompt` / `ui_cancel` / `ui_notify` (`handler="interaction"`). |
+| `interaction/actions.py` | Catalog entries registered in `registry.py` as `INTERACTION_ACTIONS`. |
+| `interaction/markdown.py` + `parse.py` | Dual-surface markdown projection and plain/CLI reply parsers. |
+
+### Tool gating and preferences (admin default + user override)
+
+`DeviiSession._builtin_tools()` filters UI tools through `channel_context().tools`. Docs channel stays search-only (no UI tools). Open interaction (single-flight) exposes only `ui_cancel` plus preference tools.
+
+**Admin default:** Devii service ConfigField `devii_interactions_default` (bool, default on) on `/admin/services`. Guests always use this default.
+
+**User override:** column `users.interactions_enabled` (`-1` = inherit default, `0` = off, `1` = on). New signups insert `-1`. Resolution: `interaction/prefs.py` `effective_for(owner_kind, owner_id)`.
+
+**Surfaces (same fan-out as AI correction):**
+- Devii tools `interactions_get` / `interactions_set` (`requires_auth=True`; set accepts `enabled` or `reset=true` to inherit again). Always offered to signed-in users even when widgets are off, so they can re-enable.
+- HTTP `POST /profile/{username}/interactions` (owner-or-admin, form `InteractionsForm`), profile card + `InteractionsPref.js`, `docs_api` endpoint, audit `profile.interactions`, profile JSON keys on `ProfileOut`.
+- When effective is off, `ui_prompt` / `ui_notify` are absent from the tool list and the controller refuses them; the model must use degraded markdown menus.
+
+### Site-chat path
+
+`ui_prompt` blocks like client tools: session `_interaction_wait` emits `{type:"interaction", id, args}` on the WebSocket; `devii-terminal.js` mounts an `<ai-interaction>` tree via `createElement` (never model HTML), lazy-loads widget modules through `AiAutoload` (`static/js/autoload/AiAutoload.js`, allowlisted tag -> module map only), and replies with `{type:"interaction_result"}`. Router resolves via `session.resolve_interaction`. Custom elements live under `static/js/components/Ai*.js` with per-component CSS under `static/css/components/ai-*.css`. Light DOM only; `Application` boots the shell (`AiInteraction`, `AiStatus`, `AiActions`, `AiHelp`, `AiOption`) and starts the autoloader.
+
+### Telegram path
+
+`TelegramConnection` handles `type:"interaction"`: sends dual-surface plain text plus inline keyboard for confirm / single choice (`callback_data` short tokens). `TelegramBridge` registers a pending future **before** the chat lock so the next message or `callback_query` can resolve mid-turn. Worker `allowed_updates` includes `callback_query`; service routes `type:"callback"` to the bridge.
+
+### Plain / CLI path
+
+Broker uses `present_plain` or emits a plain menu and waits; `parse_plain_reply` accepts y/n, indices, value tokens, cancel.
+
+### System prompt
+
+Every non-docs turn appends `CA_IWP_SYSTEM_FRAGMENT` plus the live channel fragment from `_compose_system_prompt()`. Model rules: prefer `ui_prompt` when gated on; never invent HTML/CE tags; branch on `status` (submitted / cancelled / timeout / superseded / error).
+
+### Tests
+
+Unit coverage under `tests/unit/services/devii/interaction/` (capabilities, schema, markdown, parse, broker, controller) plus session tool-gating assertions and telegram callback emission. Mirror this layout for any extension.
+
 ## Related Devii tool wrappers (customization, container)
 
 Two more Devii-side controllers live under `services/devii/` but their full mechanism is documented in their owning subsystem's file, not here:

devplacepy/services/devii/actions/ai_correction_actions.py

@@ -61,7 +61,7 @@ AI_CORRECTION_ACTIONS: tuple[Action, ...] = (
             ),
             arg(
                 "prompt",
-                "The correction instruction (max 2000 chars). Omit to keep the current one.",
+                "The correction instruction (max 20000 chars). Omit to keep the current one.",
             ),
         ),
     ),