# CLAUDE.md

This file provides guidance to Claude Code when working with code in this repository. It holds only what applies regardless of which part of the codebase is being touched. Deep, subsystem-specific detail lives in nested `CLAUDE.md` files placed inside the relevant directory - Claude Code auto-loads a nested file only when a file under that directory is read or edited, so the always-loaded cost of this repository stays proportional to this file alone. See "Subsystem map" below for the full list.

It is a big project, whatever you are implementing, it is probably done before. You should look it up and match the implementation structurely and visually. For inconsistency there is zero tolerance policy. Develop dry, kiss, re-usable code, consistent with existing implementation. Literally always try to find relatable examples before making a modification. If no-example exists, explain to user what is the case and let user decide what to do and how to continue. 

## Project

DevPlace is a server-rendered social network for developers. FastAPI backend serves Jinja2 templates with pure ES6 module JavaScript on the frontend. SQLite via the `dataset` library (auto-syncs schema). No JS framework, no NPM, no JWT.

- **Database:** `dataset` (auto-syncs schema, uses `uid` for PKs). SQLite.
- **Auth:** `session` cookie, plus `X-API-KEY` / `Authorization: Bearer <api_key>` / HTTP Basic (username-or-email:password) - all resolved in `get_current_user`. PBKDF2-SHA256 via passlib. No JWT. Every user has an `api_key` (uuid7), set at signup and backfilled in `init_db`/`devplace apikey backfill`. Docs site at `/docs` (`routers/docs/` package, `DOCS_PAGES` registry; FastAPI's Swagger is moved to `/swagger` so `/docs` is free).
- **Static:** `devplacepy/static/` mounted at `/static`; URLs are boot-versioned (`/static/v<ts>/...`) via `static_url`/`assetUrl` and served immutable for a year.
- **Templates:** `devplacepy/templates/`. Shared `templates` instance from `devplacepy.templating` - all routers import from there, never instantiate their own.
- **Ports:** 10500 (dev), 10501 (tests; the serial suite uses a single uvicorn subprocess).
- **Username:** letters, numbers, hyphens, underscores only, 3-32 chars. **Password:** minimum 6 chars.
- **Avatars:** Multiavatar SVG generated locally from a seed in <5ms, in-memory cache cleared on restart, fallback to initial-based SVG on error. URL `/avatar/multiavatar/{seed}?size={size}`. The seed is per-user: the nullable `users.avatar_seed` column overrides the username. Always resolve it through the single null-safe choke point `avatar.avatar_seed(user)` (a Jinja global) - `user.get("avatar_seed") or user.get("username")` - never read `avatar_seed` or pass `username` to `avatar_url(...)` directly; every render site (the `_avatar_link.html` partial, `og_image`, the devRant avatar payload, etc.) goes through it. Regenerate is owner-or-admin at `POST /profile/{username}/regenerate-avatar` (writes a fresh `generate_uid()`, invalidates the user cache, audits `profile.avatar.regenerate`); the old seed is never stored, so the old avatar cannot return. Devii tool `regenerate_avatar` (`CONFIRM_REQUIRED`).

## Commands

```bash
make install        # pip install -e . + playwright install chromium
make ppy            # build the single shared container image (ppy:latest); run once before launching instances
make dev            # uvicorn --reload on port 10500, backlog 4096
make prod           # uvicorn --workers $(WEB_WORKERS) (defaults to nproc), port 10500 (backlog 8192)
make test           # full suite (unit + api + e2e), headless, serial; one pass reports EVERY failure
make test-fast      # unit + api only, no browser - the quickest triage pass (~3 min)
make test-failed    # re-run only the tests that failed in the previous run
make test-first-failure  # full suite with -x, stops at the first failure
make test-slowest   # full suite plus the 40 slowest tests, to find what costs wall-clock
make test-headed    # same tests in a visible Chromium window (single process)
make locust         # Locust load test, interactive web UI
make locust-headless  # Locust CLI mode for CI
```

The Makefile exports `PYTHONDONTWRITEBYTECODE=1` for every recipe, so no `.pyc` files or `__pycache__` directories are written by any `make` target. Keep it that way - do not add a target that re-enables bytecode writing. `make clean` removes any stray bytecode left from running Python outside make.

Preliminary validation: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance). These checks are gates on the way to the real validation, never a substitute for it: **every change ends with the full test suite (`make test` - all three tiers, every test) and it must pass.**

Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x`

**Finding failures fast (the triage order).** The suite no longer stops at the first failure - `-rf` is in `pyproject.toml` `addopts`, so every run (make target or bare `pytest`) prints one `FAILED <nodeid>` line per failure at the end, giving the complete list from a single pass instead of one pass per bug. Triage cheapest-first: `make test-fast` (unit + api, no browser, ~3 min) covers most regressions; only then pay for the browser tier with `make test` or `make test-e2e`. After a run, `make test-failed` re-runs just the failures from pytest's cache (`--last-failed`), which is the loop to iterate in until it is empty. `make test-first-failure` keeps the old `-x` behaviour for the rare case where a single early failure poisons everything after it.

CLI (installed as `devplace`):
```bash
devplace role get <username>
devplace role set <username> <member|admin>
devplace apikey get <username>      # print a user's API key
devplace apikey reset <username>    # regenerate a user's API key
devplace apikey backfill            # assign API keys to users that lack one
devplace token issue <username> [--label L]  # issue a DevPlace access token
devplace token list <username>      # list a user's active access tokens
devplace token revoke <token_uid>   # revoke a single access token by uid
devplace token revoke-all <username>  # revoke all access tokens for a user
devplace token prune                # soft-delete all expired access tokens
devplace news clear         # delete all news rows
devplace news sanitize      # strip HTML from news descriptions/content
devplace attachments prune  # remove orphan attachment records/files
devplace devii reset-quota <username>   # reset one user's rolling 24h AI quota
devplace devii reset-quota --guests     # reset every guest quota
devplace devii reset-quota --all        # reset every quota (users and guests)
devplace devii tasks list [--all]       # list scheduled Devii tasks and their owners
devplace devii tasks disable <uid>      # disable one scheduled task
devplace devii tasks prune              # disable every task whose owner may not schedule
devplace gateway quota list             # list AI gateway quota rules and current 24h spend
devplace gateway quota set --limit-usd N [--owner-kind K] [--owner-id ID] [--app-reference APP] [--label L] [--uid UID]
devplace gateway quota delete <uid>     # delete a quota rule
devplace gateway quota reset [--owner-kind K] [--owner-id ID] [--app-reference APP]  # clear the counted 24h spend (keeps the usage history)
devplace zips prune         # delete expired zip archives + job rows
devplace zips clear         # delete every zip archive + job row
devplace forks prune        # delete expired completed fork job rows (forked projects persist)
devplace forks clear        # delete every fork job row (forked projects persist)
devplace messaging prune-tickets  # delete expired WebSocket auth tickets (ws_tickets)
devplace seo prune          # delete expired SEO audit reports + job rows
devplace seo clear          # delete every SEO audit report + job row
devplace seo-meta prune     # delete expired SEO metadata job rows (generated metadata persists)
devplace seo-meta clear     # delete every SEO metadata job row (generated metadata persists)
devplace deepsearch prune   # delete expired DeepSearch sessions + job rows + collections
devplace deepsearch clear   # delete every DeepSearch session + job row + collection
devplace isslop analyze <url>  # run a AI usage analysis from the terminal (report persists)
devplace isslop prune       # delete expired AI usage analysis job rows (analyses + reports persist)
devplace isslop clear       # delete every AI usage analysis, its report and job rows
devplace quiz prune         # delete abandoned/expired quiz attempts older than the retention window
devplace game market prune  # delete Code Farm market tick buckets older than the tracking window
devplace game steals prune  # delete Code Farm raid records older than the raid-efficiency window
devplace game era status    # show the current Code Farm Era
devplace game era start <name> [--days N]  # start a Code Farm Era (default 28 days)
devplace game era end       # end the running Code Farm Era (ranks, awards Stars, records results)
devplace accounts pending   # list deleted accounts awaiting their purge
devplace accounts prune     # permanently purge accounts past the deletion grace window (--dry-run to preview)
devplace backups list       # list recorded backups
devplace backups run <database|uploads|keys|full>  # enqueue a backup (processed by the running server)
devplace backups prune      # remove backup records whose archive file is missing
devplace backups clear      # delete every backup archive + record
devplace containers list    # list container instances
devplace containers reconcile  # run one reconcile pass (desired vs docker ps)
devplace containers prune   # reap orphan containers + dangling images
devplace containers prune-builds  # remove legacy per-project images + clear dockerfiles/builds tables (one-time)
devplace containers gc-workspaces  # remove workspace dirs with no instances
devplace emoji-sync         # regenerate static/js/emoji-shortcodes.js from the emoji library (run after an emoji dep bump)
devplace migrate-data       # relocate legacy runtime files into data/ (idempotent; --dry-run to preview)
```

### Runtime data layout (single source of truth)

Every runtime/user-generated artifact lives under one root, `config.DATA_DIR` (`DEVPLACE_DATA_DIR`, default `<repo>/data`). `config.py` derives every runtime path from it and lists them in the `DATA_PATHS` registry; `ensure_data_dirs()` creates the whole tree before anything is written. **No module computes a runtime path from scratch** - import the constants (`UPLOADS_DIR`, `ATTACHMENTS_DIR`, `PROJECT_FILES_DIR`, `ZIPS_DIR`, `ZIP_STAGING_DIR`, `FORK_STAGING_DIR`, `KEYS_DIR`, `BOT_DIR`, `LOCKS_DIR`, `CONTAINER_WORKSPACES_DIR`, `DEVII_TASKS_DB`, `DEVII_LESSONS_DB`, `VAPID_*_FILE`, `SERVICE_LOCK_FILE`, `INIT_LOCK_FILE`). Uploads are physically under `data/uploads/` but still served at the unchanged `/static/uploads/` URL. Stored DB values for attachments/project files are relative (`directory` + `stored_name`), so no DB rewrite is ever needed for a layout change.

**Blob sharding (`attachments._directory_for`, reused by `attachments`, `project_files`, `zip_service`):** blobs are spread into a two-level `xx/yy` tree keyed on the **random tail** of the uuid7 (`tail[-2:]/tail[-4:-2]`), NEVER the leading bytes - a uuid7's first 48 bits are a millisecond timestamp, so prefix-sharding a time-ordered id funnels every contemporaneous write into one bucket. Any new shard helper MUST shard on a high-entropy field (uuid tail or a hash), never the head. Forward-only: pre-existing rows keep resolving from their stored `directory`/`local_path`.

Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA_DIR`, `DEVPLACE_DISABLE_SERVICES=1`. `make test` runs **serially, one test at a time, in a single process**, enforced centrally in `pyproject.toml` (pytest-xdist is not a dependency, `-n` is rejected).

## Environment variables

| Var | Default | Purpose |
|-----|---------|---------|
| `DEVPLACE_DATABASE_URL` | `sqlite:///<repo>/data/devplace.db` | Override DB path (tests use this) |
| `SECRET_KEY` | hardcoded fallback | Session signing |
| `DEVPLACE_DISABLE_SERVICES` | unset | When `1`, NewsService and other background services skip start (set by test conftest) |
| `PLAYWRIGHT_HEADLESS` | `1` in tests | Toggle headed mode |
| `DEVPLACE_TEMPLATE_AUTO_RELOAD` | `1` (on) | Jinja template auto-reload. `1` stat-checks every template per render (dev hot-reload); set `0` in production so compiled templates stay cached in memory. |
| `DEVPLACE_WEB_WORKERS` / `--workers` | `nproc` (prod) | Uvicorn worker count; `make prod WEB_WORKERS=N` to override. |
| `DEVPLACE_PRESENCE_TIMEOUT_SECONDS` | `60` | How long after a user's last activity they still count as online. `config.PRESENCE_WRITE_SECONDS` (half of it) throttles `last_seen` writes per worker. |
| `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Max avatars in the feed's live "Online now" panel. |
| `DEVPLACE_PRESENCE_TRACK_LIMIT` | `500` | Size of the online set the presence relay tracks and publishes as the authority for every avatar dot. `PRESENCE_ONLINE_LIMIT` only caps how many of them the feed panel *displays*. |
| `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS` | `20` | Grace margin (hysteresis) before an online user is dropped, kills dot/roster flicker at the boundary. |
| `DEVPLACE_DATA_DIR` | `<repo>/data` | Single root for ALL runtime/user-generated data OUTSIDE the package and OUTSIDE `/static`. Point at a volume in prod. |
| `DEVPLACE_OUTBOUND_PROXY_URL` | unset | Fallback for the `outbound_proxy_url` site setting (below) when the DB/settings row is unavailable (early CLI contexts). Prefer configuring the setting via `/admin/settings` - it applies live with no restart. |

## Subsystem map

Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file in that directory is touched):

| Path | Covers |
|------|--------|
| `devplacepy/routers/CLAUDE.md` | Full URL prefix map, route aggregation rules, HTML/JSON negotiation, FastAPI patterns, gists, feed/listing features, engagement (reactions/bookmarks/polls/heatmap/follow/block-mute), SEO implementation, polymorphic comments/votes |
| `devplacepy/routers/projects/CLAUDE.md` | Project detail page, virtual filesystem routes, visibility/read-only UI, deletion confirmation |
| `devplacepy/routers/docs/CLAUDE.md` | The `/docs` documentation site: `DOCS_PAGES`, audience tiers, prose rendering, API tester |
| `devplacepy/routers/devrant/CLAUDE.md` | devRant-compatible REST API (`/api`) |
| `devplacepy/services/containers/CLAUDE.md` | Container manager: backend, security, `ppy` image, ingress, terminals, sync |
| `devplacepy/services/devii/CLAUDE.md` | Devii assistant: sessions/channels, scheduler, virtual tools, self-configured behavior, client browser tools |
| `devplacepy/services/openai_gateway/CLAUDE.md` | AI gateway: `/openai/v1/*`, usage ledger, provider/model routing |
| `devplacepy/services/jobs/CLAUDE.md` | Async job services: zip, fork, SEO diagnostics, SEO metadata, DeepSearch, AI Usage Analyzer |
| `devplacepy/services/moderation/CLAUDE.md` | Trust and safety: the reportable-target registry, the content filter and its five choke points, the report queue and its atomic resolution, enforcement, consent, maturity, account deletion |
| `devplacepy/services/acceptance/CLAUDE.md` | Acceptance convergence: the opt-in service that grants every policy agreement to every account that has not declined it, its invisibility contract and the ledger-as-decline-register rule |
| `devplacepy/services/audit/CLAUDE.md` | Audit log: recorders, categories, retention |
| `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download |
| `devplacepy/services/telegram/CLAUDE.md` | Telegram bot bridge |
| `devplacepy/services/email/CLAUDE.md` | Devii IMAP/SMTP email tools |
| `devplacepy/services/gitea/CLAUDE.md` | Issue tracker (Gitea-backed, no local issue store) |
| `devplacepy/services/messaging/CLAUDE.md` | Real-time DM chat (WS + relay) |
| `devplacepy/services/xmlrpc/CLAUDE.md` | XML-RPC bridge |
| `devplacepy/services/news/CLAUDE.md` | `NewsService` import pipeline |
| `devplacepy/services/bot/CLAUDE.md` | `BotsService` fleet |
| `devplacepy/services/dbapi/CLAUDE.md` | `/dbapi` primary-admin-only read-only database API |
| `devplacepy/services/pubsub/CLAUDE.md` | Database-free pub/sub bus |
| `devplacepy/push/CLAUDE.md` | Push notification providers: the `PushProvider` protocol, the registry, Web Push and APNs, registration storage |
| `devplacepy/services/game/CLAUDE.md` | Code Farm idle game (economy invariants, raids, the one-pure-function rule) |
| `devplacepy/services/opinionwar/CLAUDE.md` | Opinion Wars (week-long faction battles on posts: atomic fight/resolve transitions, cooldown-before-coins compensation, event seq allocation, relay-on-lock-owner) |
| `devplacepy/services/quiz/CLAUDE.md` | Quizzes (the terminal publish lock, attempt atomicity, answer-key withholding, AI free-text grading, the best-attempt scoreboard) |
| `devplacepy/services/CLAUDE.md` | Background task queue, AI correction/modifier, presence, live view relay, `BaseService`/`ServiceManager` |
| `devplacepy/database/CLAUDE.md` | Dataset rules, indexing conventions, soft delete, tables, site settings |
| `devplacepy/utils/CLAUDE.md` | Notifications and gamification (XP/levels/badges/leaderboard) |
| `devplacepy/static/js/CLAUDE.md` | Custom web components, shared frontend utilities (Http/Poller/FloatingWindow/ScrollMemory) |
| `devplacepy/static/css/CLAUDE.md` | CSS system: design tokens (no literals, no `var()` fallbacks), file-scoped palettes, `--z-*` stacking bands, the closed breakpoint set, reduced motion |
| `devplacepy/templates/CLAUDE.md` | Modal system, CDN libraries, shared template partials |
| `tests/CLAUDE.md` | Detailed testing patterns and pitfalls |

The AI usage analyzer (`devplace isslop analyze`, the `/tools/isslop` job service) lives entirely inside the package at `devplacepy/services/jobs/isslop/` and is documented in `devplacepy/services/jobs/CLAUDE.md`. There is no repo-root `isslop/` project.

## Architecture

### Request pipeline

`main.py` mounts `/static`, registers every router with its prefix, installs middlewares (security headers, a per-IP rate limit in an in-process `defaultdict`, a maintenance gate), and registers global 404/500 handlers. Rate limiting reads `rate_limit_per_minute`/`rate_limit_window_seconds` from `site_settings` (default 60/60s) and **applies only to mutating methods** (`POST`/`PUT`/`DELETE`/`PATCH`); reads and the whole `/openai` gateway are exempt. Bucket key is `X-Real-IP` falling back to `request.client.host`; an over-limit request gets `429` with `Retry-After`. The maintenance middleware short-circuits non-admin requests with a 503 when `maintenance_mode="1"`, but always allows `/static`, `/avatar`, `/auth`, `/admin`, and admin users. The outermost middleware is `response_timing`: stamps `request.state.request_start` and sets `X-Response-Time` on every response; the `response_time_ms(request)` Jinja global renders it as a fixed bottom-left badge in `base.html`. This is the single generic timing mechanism - never re-time per route.

`@app.on_event("startup")` calls `init_db()` and, unless `DEVPLACE_DISABLE_SERVICES` is set, registers `NewsService` and kicks off `start_all()`. `GET /` never redirects: guests get the marketing splash, authenticated users get a personalized home, both sharing Latest Posts + Developer News. The feed and Latest Posts enforce **author diversity** by interleaving authors (never dropping posts) via `interleave_by_author`/`paginate_diverse` in `database/`.

### Routing layout

Routers in `devplacepy/routers/` are organised as a **directory tree that mirrors the endpoint (URL) path**, exactly like `tests/`. A domain with a single resource stays one flat file (`feed.py`, `posts.py`, ...); a domain with several sub-resources is a **package directory** split one file per sub-resource. Each leaf declares its own `router = APIRouter()`; the package `__init__.py` aggregates with `router.include_router(...)`. A leaf owning the domain's collection-root (`""`) route must be the package's base router (FastAPI rejects an empty path under an empty prefix). Full elaboration, per-domain detail, and the complete deep-dive live in `devplacepy/routers/CLAUDE.md` - read it before touching routing. Compact map:

| Prefix | Router |
|--------|--------|
| `/auth` | auth/ package |
| `/feed`, `/posts`, `/comments` | flat files |
| `/projects`, `/projects/{slug}/files`, `/projects/{slug}/containers` | projects/ package - see `routers/projects/CLAUDE.md` |
| `/profile` | profile/ package (customization, notifications, ai-correction, ai-modifier, interactions, telegram, usage) |
| `/messages` | messages.py - see `services/messaging/CLAUDE.md` |
| `/notifications`, `/votes`, `/reactions`, `/bookmarks`, `/polls`, `/avatar`, `/follow`, `/leaderboard` | flat files |
| (none) | relations.py - block/mute |
| `/admin`, `/admin/services`, `/admin/containers` | admin/ package |
| `/issues` | issues/ package - see `services/gitea/CLAUDE.md` |
| `/gists`, `/news`, `/uploads`, `/media` | flat files |
| `/openai` | openai_gateway.py - see `services/openai_gateway/CLAUDE.md` |
| `/devii` | devii.py - see `services/devii/CLAUDE.md` |
| `/zips`, `/forks` | see `services/jobs/CLAUDE.md` |
| `/tools` | tools/ package (SEO diagnostics, DeepSearch, AI Usage Analyzer) - see `services/jobs/CLAUDE.md` |
| `/p/{slug}` | proxy.py - container ingress reverse proxy |
| `/xmlrpc` | xmlrpc.py - see `services/xmlrpc/CLAUDE.md` |
| `/api` | devrant/ package - see `routers/devrant/CLAUDE.md` |
| `/dbapi` | dbapi/ package, **primary-administrator-only, strictly READ-ONLY** - see `services/dbapi/CLAUDE.md` |
| `/game` | game/ package - see `services/game/CLAUDE.md` |
| `/reports`, `/admin/moderation`, `/workspaces` | reports.py, admin/moderation.py, workspaces.py - see `services/moderation/CLAUDE.md` |
| `/quizzes` | quizzes/ package - see `services/quiz/CLAUDE.md` |
| `/battles` | battles.py - see `services/opinionwar/CLAUDE.md` |
| (none) | push.py (web push/PWA), docs.py (see `routers/docs/CLAUDE.md`) |
| `/pubsub` | pubsub.py - see `services/pubsub/CLAUDE.md` |
| (none) | seo.py - `/robots.txt`, `/sitemap.xml` |

### Templates and frontend

- All routers MUST import the shared `templates` from `devplacepy.templating`. Do NOT instantiate `Jinja2Templates` per router.
- Templates extend `base.html`; page CSS via `{% block extra_head %}`, page JS via `{% block extra_js %}`.
- **Nav active state is the `nav_active` Jinja global**, never an inline path check.
- **Static asset URLs are boot-versioned:** never hardcode a bare `/static/...` href/src - use `static_url(path)` (Jinja) / `assetUrl(path)` (JS).
- All JS is ES6 modules, one class per file, instantiated as `app`. Custom web components (`dp-*` prefix) and shared frontend utilities (`Http`, `Poller`, `JobPoller`, `OptimisticAction`, `FloatingWindow`, `ScrollMemory`) are documented in `devplacepy/static/js/CLAUDE.md` - reuse them, never re-implement.
- CDN scripts in `base.html` MUST use `defer` or `type="module"`.
- Modal system, shared template partials (`_avatar_link.html`, `_user_link.html`, `_sidebar_search.html`) are documented in `devplacepy/templates/CLAUDE.md`.

### Content rendering pipeline

**Server-rendered content and titles are rendered on the BACKEND for SEO** (`devplacepy/rendering.py`, Jinja globals `render_content(text)`/`render_title(text)`). This is the default for all content that exists at request time (post bodies, comments, news, project/gist descriptions, DM history, every content title). It mirrors the client pipeline using **mistune**: em-dash normalization, emoji shortcodes, GFM markdown (`escape=True` - the server XSS control), then a media pass turning bare URLs into embeds and `@mentions` into links. Both are `@lru_cache`d. Server-rendered code blocks get syntax highlighting + Copy button client-side via `ContentEnhancer`.

The CLIENT pipeline (`ContentRenderer.js`, `dp-content`/`dp-title`) is retained ONLY for genuinely live/dynamic content that does not exist at request time (live comments, DM bubbles, Devii/DeepSearch/Docs chat, planning report). Runs `marked` -> **`DOMPurify.sanitize`** (the client XSS control, fail-closed) -> highlight.js -> media/autolink. **Do NOT add `data-render` to server-rendered content - call `render_content`/`render_title` instead.**

**Emoji shortcodes are the full GitHub/Discord `:name:` set** (~4869 names), generated once from the `emoji` library by `rendering.py` `build_emoji_shortcodes()`; the frontend gets the identical map via the generated `static/js/emoji-shortcodes.js` (regenerate with `devplace emoji-sync` after bumping the dependency, never hand-edit).

**Email anonymization:** both `_render_content` and `_render_title` mask email addresses to prevent doxing. `_mask_emails` in `rendering.py` runs on rendered text nodes only (inside `_transform_text`, `_MediaProcessor`, and `_InlineFilter` - never before mistune, or the `*` mask characters would be parsed as emphasis) and stars ~80% of the local part (keeps a leading 20%, minimum one visible char). Addresses on `molodetz.nl` (and its subdomains) are exempt and render verbatim.

**Em-dash normalization:** `_normalize_dashes` in `rendering.py` replaces all forms (em dash `\u2014`, en dash `\u2013`, and their HTML entities) with a hyphen BEFORE mistune processes the text. Runs inside `_render_content`/`_render_title` which are `@lru_cache`d, so each unique text is normalized once. No per-template overhead.

### Auth

Session cookies named `session`, value a 64-char hex token. Passwords hashed with `pbkdf2_sha256` via passlib. `get_current_user(request)` returns user dict or `None`, per-process TTL cache (300s) keyed by token. `require_user(request)` raises a 303 redirect to `/` for guests; `require_admin(request)` redirects to `/feed` for non-admins. Public pages use `get_current_user` so guests can browse - POSTs are all guarded by `require_user`.

### Database

SQLite via `dataset.connect` with WAL + 30s busy timeout + 256MB mmap. **SQLite is synchronous by design and will never be made async (hard rule).** The `dataset`/SQLite layer is called directly inside async route handlers - the database is a local file tuned with WAL/`synchronous=NORMAL`/8MB page cache/256MB mmap, so queries are sub-millisecond and never meaningfully stall the event loop. Do NOT convert the database layer to async, wrap calls in a threadpool/`run_in_executor`/`to_thread`, or raise synchronous DB access as a blocking-I/O concern - this is a settled design decision, not open for revisiting.

`init_db()` is idempotent and **ensures the full column set of any table code filters on** before creating indexes - `dataset` gives a lazily-created table only the columns of its first insert, so a partial insert elsewhere would leave a reduced schema that later 500s with `no such column`. Full dataset rules, indexing conventions (the soft-delete planner trap), site settings, and the complete table list live in `devplacepy/database/CLAUDE.md`.

Runtime config lives in `site_settings`, read via `get_setting(key, default)`/`get_int_setting(key, default)` (60s TTL cache, cross-worker invalidated via `cache_state`). Batch helpers (`get_users_by_uids`, `get_comment_counts_by_post_uids`, `get_vote_counts`, `load_comments`) exist specifically to avoid N+1 - use them instead of per-row lookups.

### Per-user site customization (CSS/JS)

Users and guests inject their own CSS and JS, scoped to a page type or globally, configured **conversationally through Devii only** (no HTTP routes - persistence is owner-scoped from the trusted Devii session, like `LessonStore`). It runs solely in that owner's own browser sessions (self-XSS, like a userscript) - never in anyone else's view.

- **Page type** = the matched route template (`page_type_for(request)`, e.g. `/posts/{slug}`) - one value covers all instances of a type. **Owner** = `owner_for(request)`: user uid, else the `DEVII_GUEST_COOKIE`, else none.
- **Storage:** table `user_customizations` (per `owner_kind`/`owner_id`/`scope`/`lang`); `get_custom_overrides` merges global-then-page-type, cached under the `"customizations"` cache-version name.
- **Per-user suppression toggles** (distinct from the site-wide `customization_enabled` kill-switch): `cust_disable_global`/`cust_disable_pagetype` columns let a user hide their own customizations without deleting rows, edited at `POST /profile/{username}/customization/{global|pagetype}` (owner or admin).
- **Robustness against hostile CSS:** app chrome (`.topnav`, overlays, modals, context menu, toast host, FAB, lightbox, devii window) is layer-promoted with `will-change: transform` so a user's `position: fixed`/`background-attachment: fixed` CSS cannot trigger a Chromium compositing bug that wipes fixed UI. Any new always-on fixed UI element must join one of these groups.
- **Injection:** `custom_css_tag(request)` emits a `<style>` after `extra_head`; `custom_js_tag(request)` emits a JSON island run via `new Function(JSON.parse(...))` after `Application.js`. Both fail closed to empty Markup - a customization bug must never break page render. Two kill-switches: `customization_enabled`, `customization_js_enabled`.
- **Devii** (`services/devii/customization/`, `handler="customization"`): `customize_list/get/set_css/set_js/reset` (`requires_auth=False`) plus `customize_set_enabled` (the suppression toggles). Set/reset are in `CONFIRM_REQUIRED`, forcing Devii to ask page-type vs global before `confirm=true`. Devii previews live with `run_js`, then `reload_page`.

### Background task queue, AI correction/modifier, background services

`services/background.py` `background.submit(fn, *args)` is a generic fire-and-forget offload onto one per-worker `asyncio.Queue`; when the consumer isn't running (tests, full queue) it runs the callable **inline**, so audit/notification/XP writes stay deterministic for the suite while production defers them. It is the choke point for every audit-log write, every XP award, and every notification - handlers call `award_rewards`/`create_notification` directly (never wrap them in `background.submit`, that double-queues). AI content correction (opt-in, off by default) and the AI modifier (`@ai <instruction>` inline directive, on by default) rewrite user prose via the internal gateway; sync apply mode never blocks the event loop (`run_in_executor` + the `await_pending_corrections` middleware). `BaseService`/`ServiceManager` provide the async run loop and singleton registry for background services (`NewsService`, `GatewayService`, `DeviiService`, `BotsService`, container/audit/telegram reconcilers). Full detail on all of this, plus presence and the live view relay, is in `devplacepy/services/CLAUDE.md`.

### Container manager, Devii assistant, AI gateway, async jobs, audit log

DevPlace ships a Docker-backed **Container manager** (admin-only, one shared `ppy:latest` image, security-hardened rootless workflow via a `sudo` superclone and `aptroot`), the **Devii** in-platform agentic assistant (WebSocket terminal, per-owner-channel sessions, scheduler/reminders, self-learning lessons, user-defined virtual tools, self-configured behavior, browser-automation client tools), the **AI gateway** (`/openai/v1/*`, single point of truth for every AI call, per-user cost attribution, provider/model routing overlay), **async job services** (`JobService` base pattern: zip, fork, SEO diagnostics, SEO metadata generation, DeepSearch, AI Usage Analyzer), and an admin-only append-only **audit log** (`record`/`record_system` entrypoints, `events.md` catalogue of 288 keys, never raises into the caller). These are among the largest subsystems in the codebase - read their dedicated nested `CLAUDE.md` files (see Subsystem map) before working in any of them; do not assume prior knowledge from this summary.

### Telegram bot, email, devRant compatibility API, issue tracker

Devii is also reachable over **Telegram** (one supervised long-poller subprocess, `channel="telegram"` isolated conversation thread), can drive a user's own **external mailbox** over IMAP/SMTP (stdlib only, credentials in a soft-deletable table, SSRF-guarded), and DevPlace exposes a **devRant-compatible REST API** at `/api` (translates devRant requests onto native posts/comments/votes via reused audited cores, ID mapping is `posts.id`/`comments.id` directly). The **issue tracker** at `/issues` has no local store - it reads/writes Gitea live via one shared bot token, filing is an async AI-enhanced job. Full detail in the respective nested `CLAUDE.md` files.

### SEO

`devplacepy/seo.py` generates JSON-LD schemas (WebSite, BreadcrumbList, DiscussionForumPosting, ProfilePage, SoftwareApplication). Every router builds context via `base_seo_context(request, ...)`. Auth/messages/notifications are `noindex,nofollow`; profiles with fewer than 2 posts are `noindex,follow`. `/robots.txt` and `/sitemap.xml` are served by `routers/seo.py`. Full implementation map (template layer, heading hierarchy, slugs, related posts, performance, default OG image, SEO tests) is in `devplacepy/routers/CLAUDE.md`.

## The production database is never touched without explicit confirmation (hard rule)

`data/devplace.db` is the live production database, and `make dev`, `make prod` and the Docker stack all share it (see "Production deployment"). No agent-initiated command may read or write it, or anything else under `data/`, without the user's explicit, stated confirmation - not a one-click approval, a confirmation they wrote themselves after being told exactly what the command would do.

This is enforced, not remembered. `.claude/hooks/guard_production_db.py` runs as a `PreToolUse` hook on every Bash call and **denies** the command outright when it reaches production, naming the reason. The interesting case is the one that motivated the rule: a script that never mentions a path at all but imports `devplacepy` and therefore resolves `config.DATA_DIR` to the real database. The hook reads the script and decides on its content, so a scratch-database script passes and an unguarded one does not.

What the guard blocks: any command naming `data/devplace.db` or a production data directory, the `devplace` management CLI, `python -m devplacepy...`, and any inline `-c` or script file that imports `devplacepy` without a `DEVPLACE_DATABASE_URL` override. What stays free: `make test` and `pytest` (the suite runs on its own temp database), `make dev`/`make prod`/`make docker-*`, the mandated import gate `python -c "from devplacepy.main import app"`, and anything that sets `DEVPLACE_DATABASE_URL` to a scratch file. `permissions.deny` in `.claude/settings.json` additionally refuses `Write`/`Edit` anywhere under `data/`, which the Bash hook cannot see.

The escape hatch is deliberately two-factor and must never be self-served: after the user has confirmed in their own words, the command may carry the literal token `I-HAVE-CONFIRMED-PRODUCTION-DB-ACCESS`, which downgrades the denial to a permission prompt the user still has to approve. **Never add that token on your own initiative.** Write disposable scripts against a temp database via `DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` instead, exactly as "Rigorous correctness verification" already requires.

## Conventions (project-specific)

- **No comments, no docstrings in source.** Code is self-documenting.
- **Forbidden variable name patterns:** `_new`, `_old`, `_temp`, `_v2`, `better_`, `my_`, `the_`.
- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: the reverse-proxy forwarding core `services/containers/forward.py` (and the three routers that call it - `routers/proxy.py`, `routers/tunnel.py`, the workspace editor route), which relays the user's own headers verbatim and therefore keeps a plain `httpx.AsyncClient`; bolting the Chrome identity onto it would overwrite the very headers it exists to forward. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
- **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
- **`RedirectResponse(url=..., status_code=302)`** - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
- **Ownership checks:** compare `resource["user_uid"] == user["uid"]` before edit/delete (use `content.is_owner`). **Deletes additionally allow any admin:** `is_owner(...) or is_admin(user)` on every content delete endpoint, so an administrator may delete any member's content while members are limited to their own. Edits stay owner-only. Deletes are **soft** and cascade under one shared stamp - see `devplacepy/database/CLAUDE.md`.
- **All dates shown to users are DD/MM/YYYY** (European), rendered in the viewer's own timezone client-side. Timestamps are stored/emitted as UTC ISO. Use the `local_dt(iso, mode)`/`dt_ago(iso)` Jinja globals for any user-facing instant - they emit `<time data-dt>` and `static/js/LocalTime.js` reformats to local timezone with a `MutationObserver` for dynamic content. `format_date()`/`time_ago()` stay as plain-text helpers for JSON responses, no-JS fallbacks, and non-timestamp date fields (e.g. project `release_date`) - do NOT wrap those in `local_dt`.
- **Slug + UUID lookup:** resources with slugs accept either the slug or the bare UUID via `resolve_by_slug()`. Slugs are `make_combined_slug(title, uid)`, prefixed with the **random tail** of the UUID (never the leading bytes - same timestamp-collision reasoning as blob sharding).
- **Roles are stored capitalized:** `users.role` is exactly `"Admin"` or `"Member"`. Always test admin-ness through the `is_admin(user)` global (case-sensitive `== "Admin"`) - never hand-roll a lowercase compare. **Any write to `users.role` MUST call `database.invalidate_admins_cache()`.**
- **Admin seniority: a junior admin cannot manage a more senior admin.** Every per-user mutation in `routers/admin/users.py` and `routers/admin/moderation.py` is gated by `is_senior_admin(actor, target)` (`routers/admin/_shared.py`) - blocks (audits `result="denied"`) when the target is an Admin who registered earlier. Server-side, so it also covers Devii's admin tools.
- **Never pass a `respond()` context key that collides with a Jinja global.** `respond(request, template, ctx, model=XOut)` feeds the same `ctx` to both the Pydantic model (JSON) and the template. A key like `is_admin`/`avatar_url`/`is_self` holding a non-callable value shadows the global across the whole inheritance chain, turning `{% if is_admin(user) %}` into `False(user)` -> `TypeError`, a 500 that fires only for the branch that calls the global. Name viewer/permission flags distinctly (`viewer_is_admin`, not `is_admin`) in both schema and context.
- **Project visibility (`is_private`) and read-only (`read_only`)** are owner-controlled flags on `projects`. Read access is gated by the single `content.can_view_project(project, user)` predicate at EVERY read surface - never re-implement the check inline. Predicate: `not is_private OR is_owner OR (is_admin AND owner is not an admin)` - a project hidden by a member stays visible to any admin, but one hidden by an admin is visible only to that owner admin. **Containers have their own, stricter isolation predicates** (`owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance`) - the primary administrator sees/manages every container; any other admin can VIEW others' containers only on public projects and can MANAGE only instances they own. Read-only is enforced as a single data-layer guard `project_files._guard_writable(project_uid)` at the top of every mutation entrypoint - add it to any NEW file-mutating function. Devii may flip read-only/visibility only after explicit confirmation (`CONFIRM_REQUIRED`). Full UI-level detail in `devplacepy/routers/projects/CLAUDE.md`.
- **Deletions ALWAYS require confirmation:** `confirmation_error` gates EVERY content delete tool via `CONFIRM_REQUIRED` (`delete_post`, `delete_comment`, `delete_gist`, `delete_project`, `project_delete_file`, `delete_media`, `delete_attachment`, `admin_delete_news`, container delete, and any `container_exec` matching `dispatcher.DESTRUCTIVE_COMMAND`). The first call is refused; the agent must show the exact target then pass `confirm=true`. **Load-bearing: every confirmation-gated tool MUST also declare a `confirm` boolean param in its catalog spec** - schemas set `additionalProperties: false`, so a gated tool without a declared `confirm` param can never receive it and loops forever.

## Every user-generated surface is reportable by construction (hard rule)

A new content table added to `SOFT_DELETE_TABLES` that is visible to anyone other than its author MUST be registered in `database/moderation.py` `REPORTABLE_TARGETS`, MUST resolve in `resolve_object_url`, and MUST include `_report_button.html` in its action bar. A table that is genuinely private to its owner goes in `UNREPORTABLE_TABLES` **with its reason** instead. `tests/unit/database/moderation.py` computes the difference and fails the suite on anything unclassified, so report coverage is closed under future additions rather than remembered; the e2e coverage test enforces the third requirement.

The same rule keeps the untriggered app-store conditionals untriggered: **no social login, no payment path, no purchasable randomness, no advertising, and no cross-app tracking** may be introduced without also implementing the obligations each of them creates (Sign in with Apple, in-app purchase, odds disclosure, ad reporting, App Tracking Transparency). Full detail in `devplacepy/services/moderation/CLAUDE.md`.

## Modal pattern

`Application.js` `initModals()` toggles a `.visible` CSS class on `.modal-overlay`; the CSS rule `.modal-overlay.visible { display: flex; }` handles visibility. Triggers usually have `href="#"`, so call `e.preventDefault()`. `.modal-close` is wired generically - no inline JS needed. Full modal/partial/CDN detail in `devplacepy/templates/CLAUDE.md`.

## Polymorphic comments and votes

The `comments` table uses `(target_type, target_uid)` so `_comment_section.html` works for `post`, `project`, `gist`, and `news`. Votes follow the same shape via `/votes/{target_type}/{uid}`. `resolve_target_redirect()` in `comments.py` maps target_type back to the correct detail URL. Full detail (reactions/bookmarks/polls/heatmap/follow/block-mute reuse the same target-type pattern) in `devplacepy/routers/CLAUDE.md`.

## Project-wide soft delete (hard rule)

**Every removal is a soft delete; only garbage collection is a hard delete.** Removable rows carry `deleted_at` (ISO timestamp) + `deleted_by` (actor uid or `system`); a live row has `deleted_at = NULL` and every list/count read filters `deleted_at IS NULL`. The table set is `database.SOFT_DELETE_TABLES`; `init_db` ensures both columns + a partial index per table. Core primitives in `database/`: `soft_delete`, `soft_delete_in`, `restore`, `purge`, `list_deleted`/`count_deleted`, `restore_event`/`purge_event`.

- **Every INSERT into a soft-deletable table MUST write `deleted_at: None, deleted_by: None`** - `dataset.find(deleted_at=None)` on a table missing the column matches NOTHING, silently hiding all rows on a fresh DB.
- **Any new read of a soft-deletable table MUST filter `deleted_at IS NULL`.**
- **Toggles revive, never duplicate:** look up the physical row ignoring `deleted_at`, stamp on toggle-off, clear on re-toggle.
- **Cascades share one `stamp`** so the event restores/purges atomically.
- **Delete authz is owner-OR-admin on the endpoint** - one check covers the UI and Devii.
- **GC stays HARD** (job sweep, metrics ring, usage-ledger prune/reset, expired-session cleanup, fork rollback). Logout is soft (auditable); only expiry GC is hard.

Admin **Trash** at `/admin/trash` restores/purges by event. Full table list, dataset rules, indexing conventions (the soft-delete planner trap), and site settings are in `devplacepy/database/CLAUDE.md`.

## Testing

Playwright (NOT pytest-playwright). Around 2882 tests in `tests/`, split into three category directories by *what they exercise*: `tests/api/` (HTTP integration, no browser), `tests/e2e/` (Playwright browser), `tests/unit/` (pure in-process). **The directory tree mirrors the path** - one segment per directory, last segment is the file. `api`/`e2e` mirror the URL path (`GET /admin/ai-usage` -> `tests/e2e/admin/aiusage.py`); `unit` mirrors the source module path (`devplacepy/utils.py` -> `tests/unit/utils.py`). A test's tier is decided by its fixtures: `page`/`alice`/`bob` = e2e; `app_server`/`seeded_db` or any HTTP call = api; `local_db`-only or no fixture = unit. Run a tier with `make test-unit`/`make test-api`/`make test-e2e`; `make test` runs all three.

Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `browser_context` (session-scoped Playwright context), `page` (function-scoped, fresh cookies), `alice`/`bob` (seeded logged-in users, `bob` gets its own context for multi-user tests).

Required patterns: every `page.goto(...)`/`page.wait_for_url(...)` MUST pass `wait_until="domcontentloaded"`; prefer `page.locator(...).wait_for(state="visible")` over `wait_for_selector`; scope ambiguous selectors (e.g. comment Delete is `.comment-action-btn:has-text('Delete')`). A test that flips a global `site_settings` value MUST restore it in `try/finally`. Full pitfalls/patterns catalogue is in `tests/CLAUDE.md`.

**Always run the full test suite (`make test` - unit, api, and e2e, every test) as the final validation of every change.** No tier may be skipped and no subset substituted for the whole. The clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks are preliminary gates before the suite, not replacements for it. Any failure is a real signal and blocks completion until fixed.

## Rigorous correctness verification (money, state machines, concurrency)

The persisted test suite (`tests/unit`/`api`/`e2e`) is example-based: it proves the specific inputs someone wrote down behave correctly. It is structurally blind to three classes of bug - a formula wrong at an input nobody tried, an invariant violated only after a long sequence of actions, and a race that only two nearly-simultaneous requests trigger. **Any feature that touches a spendable resource (coins, credits, quotas), a bounded state machine (levels, tiers, counters with a floor/ceiling), or a read-then-write mutation reachable from more than one request path applies this procedure in addition to, not instead of, the normal persisted tests.** A simple CRUD toggle doesn't need it; an economy, inventory, ledger, or scoring feature does. This was built out fully for the Code Farm economy rebalance (see `devplacepy/services/game/CLAUDE.md`, "Every purchase/upgrade is atomic..." - the worked example, including every bug it actually caught) and is the standing procedure for anything shaped like it going forward.

Four escalating layers, run in order, as disposable Python scripts that call the real functions directly against a temp DB (`DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` pointed at a scratch file) - not pytest files, unless the user separately asks for persisted tests too:

1. **Property/invariant checks on every pure function, across the full input domain.** For each formula (a reward, a cost curve, a score, a discount), assert the mathematical property it must have - monotonicity, bounds, non-negativity, idempotency, additivity over a partition of its domain - by iterating the real range (hundreds to thousands of values), not a handful of hand-picked spot checks. A monotonicity bug at input 347 is invisible if only 0, 1, and 10 are tried.
2. **Stateful fuzzing.** Build N independent entities, fire a long randomized sequence of every mutating action across all of them against a live temp DB, and re-check invariants after every single action: balances never negative, counters that should only increase never decrease, levels/tiers never exceed their declared bounds. Catch the domain's expected exception type and continue - the goal is catching invariant violations and *unexpected* exceptions, not asserting every action succeeds. This proves safety (nothing bad happens) but not liveness (legitimate actions actually succeed) - a bug that wrongly blocks a valid action just raises a well-formed expected exception and sails through a pure fuzz test undetected. Layer 3 exists because of exactly this gap.
3. **Concurrency: prove races are closed, never infer it from luck.** Any read-then-write mutation reachable from more than one worker process (`uvicorn --workers N`) is a TOCTOU race until proven otherwise. Test with **real separate OS processes**, not threads in one process - `dataset` gives each thread its own pooled connection, and enough threads exhausts that pool and produces `database is locked` noise that is a test-harness artifact, not a finding about the application. Set up genuinely fresh, production-representative state before racing - **never pre-seed or zero a column that the real code path leaves unset/NULL.** This was the single most expensive mistake made building this procedure: a first race-safety pass "passed" only because its own setup script had artificially pre-zeroed columns that a real fresh row leaves as SQL `NULL` - silently hiding the exact bug the fix was supposed to prevent (`NULL = 0` evaluates to `NULL`, not true, in a SQL `WHERE` clause; any column not written in the row's original `INSERT` needs `COALESCE(column, 0)` in every later precondition and every arithmetic `SET`, not a bare comparison). Fire many concurrent attempts at the same resource, assert the exact right number succeed, and verify the final state matches the hand-computed expected total exactly (currency spent, levels advanced) - not just "the others were blocked." Fix a real race with a single atomic conditional SQL statement at the exact chokepoint (`UPDATE ... SET ... WHERE <precondition>`, checked via the driver's real `rowcount` - `dataset`'s wrapped `db.query()` does not expose it, use `db.executable.execute(sqlalchemy.text(...), params).rowcount` inside `with db:`), not a client-side lock or an optimistic-locking library this codebase doesn't otherwise use.
4. **Static analysis beyond `py_compile`.** A clean `python -m py_compile` and a clean `from devplacepy.main import app` only prove syntax and module-load order - neither catches a missing import inside a function body, which is a runtime `NameError` invisible until that exact line executes. Run `pyflakes`/`ruff check` on every touched file before calling a change done; it is nearly free and catches an entire class of bug that layers 1-3 can each individually miss if they don't happen to exercise the broken line.

## Feature workflow

Every feature in DevPlace is **one data source fanning out into several consumers**. DevPlace has no isolated change - internalize this before editing. A single handler in `routers/{area}.py` is simultaneously the **four faces of one route**:

1. **HTML** - `respond()` returns a rendered template for browsers.
2. **JSON** - the SAME `respond(..., model=XOut)` returns JSON when `Accept: application/json`. The `*Out` schema is the gate: a context key not declared on `*Out` is silently dropped from JSON even though the template still sees it.
3. **Agent tool** - `services/devii/actions/catalog.py` exposes the route as an `Action` so Devii can call it on the user's behalf. `requires_auth` mirrors the route guard.
4. **Documented endpoint** - `docs_api.py` `endpoint()` describes it (params, auth, `sample_response`) in the right group, rendered at `/docs/{group}.html`.

A new public read almost always needs all four. The cardinal failure mode is changing one face and forgetting a connected one - a real worked example (followers/following listing) touched nine files across data helpers, output schema, route, view, agent tool, API docs, and the docs trio below for one conceptual feature; that count is the norm, not the exception. If a change touches only one file, confirm none of the four faces were missed. Checklist, ordered by data flow:

1. **Understand.** Read the router, template, matching tests (per the tier naming rule above), and the matching nested `CLAUDE.md`. Trace input model -> router -> data helper -> response (HTML and JSON).
2. **Data layer.** `database/` for query/batch helpers (never inline N+1 loops - use `get_users_by_uids`, `build_pagination`, `_in_clause`; guard raw SQL with `if "table" in db.tables`). `models.py` for the Pydantic `Form` input model. `schemas/` for the `*Out` JSON response model - every context key a JSON route exposes via `respond(..., model=XOut)` MUST exist on `XOut` or it is silently dropped. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
3. **Server layer.** Handler with the right auth guard (`get_current_user` public read, `require_user` member POST, `require_admin` admin). Specific paths (`/{username}/followers`) declared before catch-alls (`/{username}`). Return HTML+JSON via `respond(request, template, ctx, model=XOut)` or pure JSON via `JSONResponse`. Register any NEW router in `main.py`.
4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
5. **Safety layer (skip only when the feature stores nothing and shows nothing).** If the feature adds a surface that carries user-generated content, register it in `database/moderation.py` `REPORTABLE_TARGETS` (or in `UNREPORTABLE_TABLES` with a reason), resolve it in `resolve_object_url`, and include `_report_button.html` in its action bar - the registry test and the e2e coverage test both fail otherwise. If it collects or transmits a new *category* of personal data, or sends anything new to a third party, update `/docs/privacy.html` and the app-store privacy declarations in the same change; that disclosure is part of the feature, not a follow-up.
6. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog.py` - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
7. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
8. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.

Failures at any implementation step block the workflow - never skip a failed step.

## CI/CD

Gitea Actions workflow at `.gitea/workflows/test.yaml` runs on every push/PR to `master`: installs dependencies + Playwright Chromium, runs the full suite serially under coverage, publishes coverage HTML as an artifact, uploads failure screenshots. CI must be green before merging. Changes move through DTAP: Development (`make dev`) -> Test (CI suite + coverage on `master`) -> Acceptance (`master` to `production` promotion via `make deploy`) -> Production (Docker Compose stack). Only CI-green `master` commits are promoted to `production`.

## Diagnosing a production failure (the order that finds it fastest)

This procedure exists because a single "the editor is down" report turned out to be **three unrelated faults stacked on each other** (a stale URL, a firewalled network leg, and a corrupt database), and the investigation wasted hours by guessing before measuring. Work the layers outward from the browser; each step is cheap and each one eliminates a whole class of cause. **Never skip to a hypothesis, and never repair anything before the layer above it is proven healthy.**

**Layer 0 - is the request even arriving here?** Fetch the hostname over the public internet exactly as it resolves (`curl -sS -o /dev/null -w '%{http_code} %{remote_ip}' https://host/`). Compare the answering IP against this machine's own addresses (`ip -6 addr`, `curl https://api.ipify.org`). Two hostnames serve this platform by different routes - see the topology section above. **Never use `curl --resolve` to force a hostname onto an IP it does not resolve to**; that fabricates a path that no real traffic takes and produces confident, wrong conclusions.

**Layer 1 - which edge answered?** The error body identifies it. `application/problem+json` with `No site configured for host` is molohttp. A DevPlace HTML error page is the application. An nginx error page is the nginx container. A browser `ERR_*` with no body means nothing well-formed was returned at all.

**Layer 2 - same failure on both hostnames?** Run the identical authenticated request against `pravda.education` and `devplace.net`. Failing on **both** means the application or the database; failing on **one** means that host's edge. This single comparison is the highest-value measurement available and costs one command.

**Layer 3 - the application log, before any theory.** `docker logs --since 5m devplace-app-1`. Count error classes rather than reading prose (`grep -c malformed`). A recurring service-loop error is a systemic fault even when it looks unrelated to the symptom.

**Layer 4 - reproduce the failing hop in isolation.** Point the real code at the real upstream from a scratch harness rather than reasoning about it. Running `forward.proxy_http` against a live code-server is what exposed the duplicate `Date` header; reading the function had not. Use a scratch database (`DEVPLACE_DATABASE_URL`) so the harness never reaches production.

**Layer 5 - test from where the code actually runs.** The app runs **inside a container**; `127.0.0.1` there is not the host. `docker exec devplace-app-1 curl ...` is the only honest reachability test for a container-to-container hop. A hang with zero bytes means a packet was **DROPped** (firewall), a refusal means nothing is listening, and a slow error means the upstream answered badly - three different causes with three different fixes.

**Layer 6 - confirm the object exists before blaming the plumbing.** A 404 from a guard is not a proxy failure. Resolve the identifier through the application's own read surface (the workspace page, an admin JSON endpoint) with the affected account's session. A stale instance uid in a bookmarked URL looks exactly like an outage.

### Rules learned the hard way

- **State what a command will read or write before running it against production, and keep production access read-only until the diagnosis is complete.** The one write in a repair is the final swap, and it comes after verification, not before.
- **Copy before repairing, and copy the whole set.** A WAL-mode SQLite database is `.db` **plus** `-wal` **plus** `-shm`; a `.db`-only copy silently discards every transaction still in the WAL. Stop writes first, or the snapshot is inconsistent. Never leave a stale `-wal` beside a recovered file - SQLite will replay it and re-corrupt the result.
- **Repair on a copy, verify on the copy, and prove what was preserved.** `PRAGMA integrity_check` names the damaged objects; index damage is derived data and costs nothing (`REINDEX`, or `.recover`), while a table b-tree fault is the only kind that can lose rows. Diff row counts table by table between the original and the recovered file and report the delta - "it says ok" is not evidence that data survived.
- **Verify the fix through the user's own path, with their account, in a real browser.** A green unit test and a 200 from `curl` did not prove the editor worked; driving Playwright through login, the code-server password prompt and a `.monaco-workbench` selector did.
- **A measurement recorded in these files can go stale.** `services/containers/CLAUDE.md` recorded that `container_ip:port` times out from the app container while `gateway:published_host_port` connects. A later change (`make docker-attach`) inverted it, and a host firewall closed the documented leg entirely. Re-measure before trusting a recorded measurement, and update the record when it turns out to be false.
- **Report each fault separately and correct yourself explicitly.** Three stacked faults produce a symptom that no single explanation covers, and an early wrong theory is worse than no theory once it is repeated as fact.

## Production hostnames and the devplace.net SSH tunnel (verified topology, do not re-derive)

**The platform answers on two public hostnames, and they reach the same application by two completely different paths.** This has already cost one debugging session; the failure mode is that a `curl --resolve devplace.net:443:<production ip>` "test" reports `No site configured for host: devplace.net` and looks like a total outage, when in fact devplace.net never touches the production edge at all.

| | `pravda.education` | `devplace.net` |
|---|---|---|
| DNS | `95.216.15.238`, `2a01:4f9:2a:100e::2` | `88.198.21.243`, `2a01:4f8:222:2c45::2` |
| Machine | the production host itself | a separate front host (Hetzner, PTR `static.88-198-21-243.clients.your-server.de`) |
| Path in | molohttp on `:443` -> `127.0.0.1:10500` | its own proxy -> **SSH tunnel** -> `127.0.0.1:10500` on production |
| Reaches molohttp | yes | **no, never** |

**`devplace.net` is a front host that forwards over SSH.** It holds a persistent SSH session into the production host (visible there as an established inbound connection from `88.198.21.243` to port 22) and forwards through it to `127.0.0.1:10500`, which is the `docker-proxy` for the `devplace-nginx` container. The listening socket lives on the **front** host (an `ssh -L` style local forward), so the production host shows **no** sshd-owned listener - that absence is expected and is not evidence against the tunnel.

Two consequences that must not be forgotten:

- **molohttp has no `devplace.net` site, and that is correct.** Its site list is `mail`/`smtp`/`imap.molodetz.nl`, `pravda.education` and `*.tunnel.pravda.education`. devplace.net traffic enters below molohttp, straight into `127.0.0.1:10500`, so it needs no site. **Never "fix" this by adding a devplace.net site to molohttp** - devplace.net does not resolve to the production host, so such a site could never match, and its absence is not a bug.
- **Both hostnames land on the same nginx and the same app**, so a request that fails on both is failing in the application, not in either edge. That comparison is the fastest triage available here: run the same authenticated request against both hostnames. Same failure on both means look at the app or the database; a failure only on devplace.net means look at the front host's proxy (WebSocket `Upgrade` headers are the usual culprit, exactly as for the production nginx locations below).

**Testing rule.** Never point a hostname at an IP it does not resolve to in order to "test" it. Fetch each hostname over the public internet as it really resolves (`curl https://devplace.net/...` and `curl https://pravda.education/...`), because forcing devplace.net onto the production IP tests molohttp with a `Host` it deliberately does not serve and proves nothing about the real path.

## Production deployment

Single-host Docker Compose (`docker-compose.yml`): an **app** container (Uvicorn, 2 workers) behind an **nginx** container. Full operator reference is the admin-only `Production` docs section (`templates/docs/production*.html`); conventions that must not regress:

- **Shared DB and files = same as dev.** The app container bind-mounts the host project root (`.:/app`) and runs as `${DEVPLACE_UID}:${DEVPLACE_GID}` (default `1000`), so it reads/writes the same `data/devplace.db`, `data/uploads/`, `data/devii_*.db`, `data/keys/` (VAPID), and `data/locks/devplace-services.lock` as `make dev`. No `DEVPLACE_DATABASE_URL` override - `config.py` resolves an absolute path under the project's `data/` dir. WAL + the `flock` on `devplace-services.lock` make concurrent dev/prod safe and keep a single background-services owner. SQLite is local-file, so prod and dev must be the **same host**.
- **Code updates need no rebuild** (bind-mounted source); rebuild only on `pyproject.toml` dependency changes. Use **`make docker-reload`** (`restart app` + `up -d --wait`) to pick up new source - a bare `make docker-up` does **not** restart an unchanged container, so the running uvicorn keeps serving the code it imported at boot. `.env` is git-ignored; `.env.example` is the committed template. The app reads `DEVPLACE_DATABASE_URL`, never `DATABASE_URL`.
- **Dockerfile layer order is load-bearing for build time.** The dependency layer (`pip install ".[bots]"` + `playwright install --with-deps chromium`, ~3GB and ~2.5 min) must depend on `pyproject.toml` **only**. `COPY devplacepy/` therefore comes *after* it, and the project itself is installed last with `pip install --no-deps --force-reinstall .`. hatchling needs the package directory to exist to build a wheel, so the dependency layer creates a placeholder `devplacepy/__init__.py` that the real `COPY` overwrites (verified: site-packages holds the full 39-entry package, not the stub). Copying source before the install inverts this and makes **every source edit** reinstall every dependency and re-download Chromium - measured 2m36s per source-only rebuild versus 7.4s with the correct order. Never move `COPY devplacepy/` above the dependency layer.
- **nginx parity rules** (`nginx/nginx.conf.template`, rendered by `start.sh` via `envsubst` with an allow-list that preserves `$http_upgrade`): `/static/uploads/` must re-apply `nosniff` + a `Content-Disposition` via the `map $uri $upload_disposition` block (`inline` for safe image/video/audio extensions, `attachment` otherwise), mirroring `UploadStaticFiles.INLINE_MEDIA_EXTENSIONS` - an XSS control nginx would otherwise bypass, and the inline branch is what lets video play in production; `/devii/ws` needs the `map $http_upgrade $connection_upgrade` block and `Upgrade`/`Connection` headers or the Devii terminal cannot connect (every new WebSocket route needs its own nginx upgrade location - the catch-all `location /` strips upgrade headers); `client_max_body_size` comes from `NGINX_MAX_BODY_SIZE` (default `50m`) and must be `>= max_upload_size_mb` or uploads 413. nginx serves `devplacepy/static` via a read-only bind mount, so assets stay current without an image rebuild.
- **Healthcheck cadence** (`docker-compose.yml` + `Dockerfile`, keep both in step): `start_period: 120s` is the grace window in which a failing probe does not count against `retries`; `start_interval: 2s` is how often the probe runs *inside* that window. Without `start_interval` the first probe only fires after the full `interval: 30s`, so a container ready in 5s still reports healthy at 30s and `depends_on: service_healthy` holds nginx back for no reason. The generous 120s start period is deliberate headroom for a cold page cache on a multi-GB database, not a measure of normal startup - normal startup is a few seconds. **Startup work is a per-worker, lock-serialized cost:** `lifespan` runs `init_db()` under an exclusive `init_lock()`, so every uvicorn worker pays it end to end, one after another, and total time-to-serving is `workers x init_db`. Never put a per-user or per-row scan in `init_db` - see the backfill convergence rule in `devplacepy/database/CLAUDE.md`.
