# CLAUDE.md This file provides guidance to Claude Code when working with code in this repository. It holds only what applies regardless of which part of the codebase is being touched. Deep, subsystem-specific detail lives in nested `CLAUDE.md` files placed inside the relevant directory - Claude Code auto-loads a nested file only when a file under that directory is read or edited, so the always-loaded cost of this repository stays proportional to this file alone. See "Subsystem map" below for the full list. It is a big project, whatever you are implementing, it is probably done before. You should look it up and match the implementation structurely and visually. For inconsistency there is zero tolerance policy. Develop dry, kiss, re-usable code, consistent with existing implementation. Literally always try to find relatable examples before making a modification. If no-example exists, explain to user what is the case and let user decide what to do and how to continue. The completion contract for every change is [DEFINITION_OF_DONE.md](./DEFINITION_OF_DONE.md): universal gates, test obligations, the four-layer rigor procedure for spendable resources and races, per-subsystem rules, and the final checklist. A change is done only when every applicable gate in it passes. ## Project DevPlace is a server-rendered social network for developers. FastAPI backend serves Jinja2 templates with pure ES6 module JavaScript on the frontend. SQLite via the `dataset` library (auto-syncs schema). No JS framework, no NPM, no JWT. - **Database:** `dataset` (auto-syncs schema, uses `uid` for PKs). SQLite. - **Auth:** three account credentials: username+password, email+password, personal `api_key`. All password exchanges (`POST /auth/login` form+JSON, `POST /auth/token`, HTTP Basic, devRant `POST /api/users/auth-token`) take username-or-email as identifier (username case-sensitive, email lowered); the `api_key` rides `X-API-KEY` / `Authorization: Bearer` (client-side `DEVPLACE_API_KEY` env var holds its value, injected into containers) plus `session` cookie - all resolved in `get_current_user` (order: session, X-API-KEY, Bearer, Basic; headers also try DevRant/access tokens). Every JSON return of the personal key also returns `username`+`email` (login/signup `data`, gated `ProfileOut.email` beside `api_key`, regenerate), so `GET /profile` with an API key is the whoami/account fetch; never add `email` to public `UserOut`. PBKDF2-SHA256 via passlib. No JWT. Every user has an `api_key` (uuid7), set at signup and backfilled in `init_db`/`devplace apikey backfill`. Docs site at `/docs` (`routers/docs/` package, `DOCS_PAGES` registry; FastAPI's Swagger is moved to `/swagger` so `/docs` is free). - **Static:** `devplacepy/static/` mounted at `/static`; URLs are boot-versioned (`/static/v/...`) via `static_url`/`assetUrl` and served immutable for a year. - **Templates:** `devplacepy/templates/`. Shared `templates` instance from `devplacepy.templating` - all routers import from there, never instantiate their own. - **Ports:** 10500 (dev), 10501 (tests; the serial suite uses a single uvicorn subprocess). - **Username:** letters, numbers, hyphens, underscores only, 3-32 chars. **Password:** minimum 6 chars. - **Avatars:** Multiavatar SVG generated locally at `/avatar/multiavatar/{seed}?size={size}`. Seed resolution, regenerate flow, and the owner-or-admin rule live in `devplacepy/templates/CLAUDE.md` ("Avatar seed choke point"). ## Commands ```bash make install # create .venv if needed, pip install -e ".[dev]", playwright chromium make ppy # build the single shared container image (ppy:latest); run once before launching instances make dev # uvicorn --reload on port 10500, backlog 4096 make prod # uvicorn --workers $(WEB_WORKERS) (defaults to nproc), port 10500 (backlog 8192) make test # full suite (unit + api + e2e), headless, serial; one pass reports EVERY failure make test-fast # unit + api only, no browser - the quickest triage pass (~3 min) make test-failed # re-run only the tests that failed in the previous run make test-first-failure # full suite with -x, stops at the first failure make test-slowest # full suite plus the 40 slowest tests, to find what costs wall-clock make test-headed # same tests in a visible Chromium window (single process) make locust # Locust load test, interactive web UI make locust-headless # Locust CLI mode for CI ``` Every Python make target (`install`, `dev`, `prod`, `test*`, `coverage*`, `locust*`) uses `.venv/bin/python`. If `.venv` is missing, make creates it from `python3`, installs `-e ".[dev]"`, and installs Playwright Chromium before running the target. `make install` refreshes that environment. `make clean` removes `.venv` as well as stray bytecode. The Makefile exports `PYTHONDONTWRITEBYTECODE=1` for every recipe, so no `.pyc` files or `__pycache__` directories are written by any `make` target. Keep it that way - do not add a target that re-enables bytecode writing. `make clean` removes any stray bytecode left from running Python outside make. Preliminary validation: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance). These checks are gates on the way to the real validation, never a substitute for it: **every change ends with the full test suite (`make test` - all three tiers, every test) and it must pass.** Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x` **Finding failures fast (the triage order).** The suite no longer stops at the first failure - `-rf` is in `pyproject.toml` `addopts`, so every run (make target or bare `pytest`) prints one `FAILED ` line per failure at the end, giving the complete list from a single pass instead of one pass per bug. Triage cheapest-first: `make test-fast` (unit + api, no browser, ~3 min) covers most regressions; only then pay for the browser tier with `make test` or `make test-e2e`. After a run, `make test-failed` re-runs just the failures from pytest's cache (`--last-failed`), which is the loop to iterate in until it is empty. `make test-first-failure` keeps the old `-x` behaviour for the rare case where a single early failure poisons everything after it. CLI (installed as `devplace`): ```bash devplace role get devplace role set devplace apikey get # print a user's API key devplace apikey reset # regenerate a user's API key devplace apikey backfill # assign API keys to users that lack one devplace token issue [--label L] # issue a DevPlace access token devplace token list # list a user's active access tokens devplace token revoke # revoke a single access token by uid devplace token revoke-all # revoke all access tokens for a user devplace token prune # soft-delete all expired access tokens devplace migrate-data # relocate legacy runtime files into data/ (idempotent; --dry-run to preview) ``` Per-subsystem commands live in the nested file: `devplace devii ...` (Devii), `devplace gateway quota ...` (AI gateway), `devplace backups ...` (backup service), `devplace containers ...` (container manager), `devplace news ...` (news service), `devplace community ...` (community discovery), `devplace messaging prune-tickets` (messaging), `devplace {zips|forks|seo|seo-meta|deepsearch|isslop} ...` (async jobs), `devplace quiz prune` (quizzes), `devplace game ...` (Code Farm), `devplace accounts ...` (moderation), `devplace apikey backfill`, `devplace attachments prune`, `devplace system prune` (database), `devplace emoji-sync` (frontend JS), `devplace embeds-sync` (templates).devplace migrate-data # relocate legacy runtime files into data/ (idempotent; --dry-run to preview) ``` ### Runtime data layout (single source of truth) Every runtime/user-generated artifact lives under one root, `config.DATA_DIR` (`DEVPLACE_DATA_DIR`, default `/data`). `config.py` derives every runtime path from it and lists them in the `DATA_PATHS` registry; `ensure_data_dirs()` creates the whole tree before anything is written. **No module computes a runtime path from scratch** - import the constants (`UPLOADS_DIR`, `ATTACHMENTS_DIR`, `PROJECT_FILES_DIR`, `ZIPS_DIR`, `ZIP_STAGING_DIR`, `FORK_STAGING_DIR`, `KEYS_DIR`, `BOT_DIR`, `LOCKS_DIR`, `CONTAINER_WORKSPACES_DIR`, `DEVII_TASKS_DB`, `DEVII_LESSONS_DB`, `VAPID_*_FILE`, `SERVICE_LOCK_FILE`, `INIT_LOCK_FILE`). Uploads are physically under `data/uploads/` but still served at the unchanged `/static/uploads/` URL. Stored DB values for attachments/project files are relative (`directory` + `stored_name`), so no DB rewrite is ever needed for a layout change. **Blob sharding (`attachments._directory_for`, reused by `attachments`, `project_files`, `zip_service`):** blobs are spread into a two-level `xx/yy` tree keyed on the **random tail** of the uuid7 (`tail[-2:]/tail[-4:-2]`), NEVER the leading bytes - a uuid7's first 48 bits are a millisecond timestamp, so prefix-sharding a time-ordered id funnels every contemporaneous write into one bucket. Any new shard helper MUST shard on a high-entropy field (uuid tail or a hash), never the head. Forward-only: pre-existing rows keep resolving from their stored `directory`/`local_path`. Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA_DIR`, `DEVPLACE_DISABLE_SERVICES=1`. `make test` runs **serially, one test at a time, in a single process**, enforced centrally in `pyproject.toml` (pytest-xdist is not a dependency, `-n` is rejected). ## Environment variables | Var | Default | Purpose | |-----|---------|---------| | `DEVPLACE_DATABASE_URL` | `sqlite:////data/devplace.db` | Override DB path (tests use this) | | `DEVPLACE_PORT` | `10500` | Port the app itself binds to bare-metal (`make dev`/`make prod`, wired to uvicorn's `--port`; tests use their own `DEVPLACE_TEST_PORT`, default `10501`). Also `config.PORT`'s source, so `DEVPLACE_INTERNAL_BASE_URL`'s default and Devii's self-dial URL (`DEFAULT_BASE_URL`/`INSTANCE_ORIGIN_DEFAULT`) follow it automatically. The Docker app container pins it to `10500` in `docker-compose.yml` regardless of `.env` - Docker's externally reachable port is the unrelated `PORT` var (nginx's host mapping), never this one. | | `SECRET_KEY` | hardcoded fallback | Session signing | | `DEVPLACE_DISABLE_SERVICES` | unset | When `1`, NewsService and other background services skip start (set by test conftest) | | `PLAYWRIGHT_HEADLESS` | `1` in tests | Toggle headed mode | | `DEVPLACE_TEMPLATE_AUTO_RELOAD` | `1` (on) | Jinja template auto-reload. `1` stat-checks every template per render (dev hot-reload); set `0` in production so compiled templates stay cached in memory. | | `DEVPLACE_WEB_WORKERS` / `--workers` | `nproc` (prod) | Uvicorn worker count; `make prod WEB_WORKERS=N` to override. | | `DEVPLACE_PRESENCE_TIMEOUT_SECONDS` | `60` | How long after a user's last activity they still count as online. `config.PRESENCE_WRITE_SECONDS` (half of it) throttles `last_seen` writes per worker. | | `DEVPLACE_PRESENCE_ONLINE_LIMIT` | `30` | Max avatars in the feed's live "Online now" panel. | | `DEVPLACE_PRESENCE_TRACK_LIMIT` | `500` | Size of the online set the presence relay tracks and publishes as the authority for every avatar dot. `PRESENCE_ONLINE_LIMIT` only caps how many of them the feed panel *displays*. | | `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS` | `20` | Grace margin (hysteresis) before an online user is dropped, kills dot/roster flicker at the boundary. | | `DEVPLACE_DATA_DIR` | `/data` | Single root for ALL runtime/user-generated data OUTSIDE the package and OUTSIDE `/static`. Point at a volume in prod. | | `DEVPLACE_OUTBOUND_PROXY_URL` | unset | Fallback for the `outbound_proxy_url` site setting (below) when the DB/settings row is unavailable (early CLI contexts). Prefer configuring the setting via `/admin/settings` - it applies live with no restart. | ## Subsystem map Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file in that directory is touched): | Path | Covers | |------|--------| | `devplacepy/routers/CLAUDE.md` | Full URL prefix map, route aggregation rules, HTML/JSON negotiation, FastAPI patterns, gists, feed/listing features, engagement (reactions/bookmarks/polls/heatmap/follow/block-mute), SEO implementation, polymorphic comments/votes | | `devplacepy/routers/projects/CLAUDE.md` | Project detail page, virtual filesystem routes, visibility/read-only UI, deletion confirmation | | `devplacepy/routers/docs/CLAUDE.md` | The `/docs` documentation site: `DOCS_PAGES`, audience tiers, prose rendering, API tester | | `devplacepy/routers/devrant/CLAUDE.md` | devRant-compatible REST API (`/api`) | | `devplacepy/services/containers/CLAUDE.md` | Container manager: backend, security, `ppy` image, ingress, terminals, sync | | `devplacepy/services/devii/CLAUDE.md` | Devii assistant: sessions/channels, scheduler, virtual tools, self-configured behavior, client browser tools | | `devplacepy/services/openai_gateway/CLAUDE.md` | AI gateway: `/openai/v1/*`, usage ledger, provider/model routing | | `devplacepy/services/jobs/CLAUDE.md` | Async job services: zip, fork, SEO diagnostics, SEO metadata, DeepSearch, AI Usage Analyzer | | `devplacepy/services/moderation/CLAUDE.md` | Trust and safety: the reportable-target registry, the content filter and its five choke points, the report queue and its atomic resolution, enforcement, consent, maturity, account deletion | | `devplacepy/services/acceptance/CLAUDE.md` | Acceptance convergence: the opt-in service that grants every policy agreement to every account that has not declined it, its invisibility contract and the ledger-as-decline-register rule | | `devplacepy/services/audit/CLAUDE.md` | Audit log: recorders, categories, retention | | `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download, remote offload to Hetzner Storage Box | | `devplacepy/services/telegram/CLAUDE.md` | Telegram bot bridge | | `devplacepy/services/email/CLAUDE.md` | Devii IMAP/SMTP email tools | | `devplacepy/services/gitea/CLAUDE.md` | Issue tracker (Gitea-backed, no local issue store) | | `devplacepy/services/messaging/CLAUDE.md` | Real-time DM chat (WS + relay) | | `devplacepy/services/xmlrpc/CLAUDE.md` | XML-RPC bridge | | `devplacepy/services/news/CLAUDE.md` | `NewsService` import pipeline | | `devplacepy/services/bot/CLAUDE.md` | `BotsService` fleet | | `devplacepy/services/dbapi/CLAUDE.md` | `/dbapi` primary-admin-only read-only database API | | `devplacepy/services/pubsub/CLAUDE.md` | Database-free pub/sub bus | | `devplacepy/push/CLAUDE.md` | Push notification providers: the `PushProvider` protocol, the registry, Web Push and APNs, registration storage | | `devplacepy/services/game/CLAUDE.md` | Code Farm idle game (economy invariants, raids, the one-pure-function rule) | | `devplacepy/services/opinionwar/CLAUDE.md` | Opinion Wars (week-long faction battles on posts: atomic fight/resolve transitions, cooldown-before-coins compensation, event seq allocation, relay-on-lock-owner) | | `devplacepy/services/quiz/CLAUDE.md` | Quizzes (the terminal publish lock, attempt atomicity, answer-key withholding, AI free-text grading, the best-attempt scoreboard) | | `devplacepy/services/giphy/CLAUDE.md` | GIF picker: GIPHY search through the server, admin-held API key, GIFs stored as ordinary attachments, cache and per-user caps | | `devplacepy/services/skills/CLAUDE.md` | Agent Skills: strict `SKILL.md` parser, SSRF-guarded fetcher, the bounded prompt index, per-user privacy, soft delete and account-deletion cascade, why a skill is data and never code | | `devplacepy/services/nodes/CLAUDE.md` | Community node network: the protocol specification (single source of truth), the no-open-proxy rule, the two transports and their trust levels, the decisions taken, and the gaps per phase | | `devplacepy/services/community/CLAUDE.md` | Community discovery: rsearch + feed detection, one-time structure learning, generated collectors, dry run, the failure floor, admin-only visibility | | `devplacepy/services/sync/CLAUDE.md` | Community sync: the origin link table, idempotent batch upserts, normalization and rejection rules, what is deliberately not imported, the race that was measured | | `devplacepy/services/CLAUDE.md` | Background task queue, AI correction/modifier, presence, live view relay, `BaseService`/`ServiceManager` | | `devplacepy/database/CLAUDE.md` | Dataset rules, indexing conventions, soft delete, tables, site settings | | `devplacepy/utils/CLAUDE.md` | Notifications and gamification (XP/levels/badges/leaderboard) | | `devplacepy/static/js/CLAUDE.md` | Custom web components, shared frontend utilities (Http/Poller/FloatingWindow/ScrollMemory) | | `devplacepy/static/css/CLAUDE.md` | CSS system: design tokens (no literals, no `var()` fallbacks), file-scoped palettes, `--z-*` stacking bands, the closed breakpoint set, reduced motion | | `devplacepy/templates/CLAUDE.md` | Modal system, CDN libraries, shared template partials | | `tests/CLAUDE.md` | Detailed testing patterns and pitfalls | The AI usage analyzer (`devplace isslop analyze`, the `/tools/isslop` job service) lives entirely inside the package at `devplacepy/services/jobs/isslop/` and is documented in `devplacepy/services/jobs/CLAUDE.md`. There is no repo-root `isslop/` project. ## Architecture ### Request pipeline `main.py` mounts `/static`, registers every router with its prefix, installs middlewares from `middleware/` (`register()` in addition order: snapshot, corrections, security headers, rate limit, maintenance, terms, presence, visits, timing, tunnel dispatch, gzip), and registers the global handlers from `errors.py` (404/500/`ContentRefused`/`RequestValidationError`). The per-IP rate limit state lives in `middleware/state.py`; app lifespan (locks, `SERVICE_CLASSES`, startup/shutdown) lives in `lifespan.py`; the home route lives in `routers/landing.py`; the static-file classes live in `static_files.py`. Rate limiting reads `rate_limit_per_minute`/`rate_limit_window_seconds` from `site_settings` (default 60/60s) and **applies only to mutating methods** (`POST`/`PUT`/`DELETE`/`PATCH`); reads and the whole `/openai` gateway are exempt. Bucket key is `X-Real-IP` falling back to `request.client.host`; an over-limit request gets `429` with `Retry-After`. The maintenance middleware short-circuits non-admin requests with a 503 when `maintenance_mode="1"`, but always allows `/static`, `/avatar`, `/auth`, `/admin`, and admin users. The outermost middleware is `response_timing`: stamps `request.state.request_start` and sets `X-Response-Time` on every response; the `response_time_ms(request)` Jinja global renders it as a fixed bottom-left badge in `base.html`. This is the single generic timing mechanism - never re-time per route. `lifespan.py` `lifespan()` calls `init_db()` and, unless `DEVPLACE_DISABLE_SERVICES` is set, registers every service in `SERVICE_CLASSES` and supervises them in the lock owner. `GET /` never redirects: guests get the marketing splash, authenticated users get a personalized home, both sharing Latest Posts + Developer News. The feed and Latest Posts enforce **author diversity** by interleaving authors (never dropping posts) via `interleave_by_author`/`paginate_diverse` in `database/`. ### Routing layout Routers in `devplacepy/routers/` are organised as a **directory tree that mirrors the endpoint (URL) path**, exactly like `tests/` (one segment per directory, last segment is the file; packages aggregate with `include_router`). A leaf owning the domain's collection-root (`""`) route must be the package's base router. Full URL prefix map, aggregation rules, HTML/JSON negotiation, FastAPI patterns, and per-domain detail live in `devplacepy/routers/CLAUDE.md` - read it before touching routing. Sub-domain pages: `routers/projects/`, `routers/docs/`, `routers/devrant/` (each with its own nested file, see Subsystem map). ### Templates and frontend All routers import the shared `templates` from `devplacepy.templating` (never instantiate per router); templates extend `base.html` (`extra_head`/`extra_js` blocks, `nav_active` global, boot-versioned `static_url`/`assetUrl`, deferred CDN scripts). JS is ES6 modules, one class per file, `dp-*` web components plus shared utilities (see `devplacepy/static/js/CLAUDE.md`). Modal system and shared partials: see `devplacepy/templates/CLAUDE.md`. ### Content rendering pipeline Server content renders on the backend for SEO (`render_content`/`render_title` in `devplacepy/rendering.py`); the client pipeline is only for live content (`DOMPurify` fail-closed). Full pipeline rules (emoji map, YouTube spec, email masking, dash normalization) live in `devplacepy/templates/CLAUDE.md` ("Content rendering pipeline"). ### Auth Session cookies named `session`, value a 64-char hex token. Passwords hashed with `pbkdf2_sha256` via passlib. `get_current_user(request)` returns user dict or `None`, per-process TTL cache (300s) keyed by token. `require_user(request)` raises a 303 redirect to `/` for guests; `require_admin(request)` redirects to `/feed` for non-admins. Public pages use `get_current_user` so guests can browse - POSTs are all guarded by `require_user`. Password exchanges (`routers/auth/login.py`, `routers/auth/token.py`) reuse the `LoginForm.email` field as username-or-email: look up `email=identifier.lower()` first, then `username=identifier` un-lowered (mirroring `_user_from_basic` semantics); never lower the identifier before the username lookup. ### Database **SQLite is synchronous by design and will never be made async (hard rule).** The `dataset`/SQLite layer is called directly inside async route handlers; never wrap it in a threadpool/`run_in_executor`/`to_thread` or raise sync DB access as a blocking-I/O concern. Dataset rules, indexing, soft delete, visibility spec, `site_settings`, batch helpers (use instead of N+1), and the table list live in `devplacepy/database/CLAUDE.md`. ### Per-user site customization (CSS/JS) Users and guests inject their own CSS and JS, scoped to a page type or globally, configured **conversationally through Devii only** (no HTTP routes - persistence is owner-scoped from the trusted Devii session, like `LessonStore`). It runs solely in that owner's own browser sessions (self-XSS, like a userscript) - never in anyone else's view. - **Page type** = the matched route template (`page_type_for(request)`, e.g. `/posts/{slug}`) - one value covers all instances of a type. **Owner** = `owner_for(request)`: user uid, else the `DEVII_GUEST_COOKIE`, else none. - **Storage:** table `user_customizations` (per `owner_kind`/`owner_id`/`scope`/`lang`); `get_custom_overrides` merges global-then-page-type, cached under the `"customizations"` cache-version name. - **Per-user suppression toggles** (distinct from the site-wide `customization_enabled` kill-switch): `cust_disable_global`/`cust_disable_pagetype` columns let a user hide their own customizations without deleting rows, edited at `POST /profile/{username}/customization/{global|pagetype}` (owner or admin). - **Robustness against hostile CSS:** app chrome (`.topnav`, overlays, modals, context menu, toast host, FAB, lightbox, devii window) is layer-promoted with `will-change: transform` so a user's `position: fixed`/`background-attachment: fixed` CSS cannot trigger a Chromium compositing bug that wipes fixed UI. Any new always-on fixed UI element must join one of these groups. - **Injection:** `custom_css_tag(request)` emits a `