This file documents detailed testing patterns, fixtures, and pitfalls for devplacepy/tests/ (Playwright + unit tests across api/e2e/unit tiers). Claude Code loads it automatically whenever a file under this directory is read or edited. Every change ends with the full suite: `make test` runs all tests across all three tiers, and it must pass.
**Every change is validated by running the full test suite - ALL tests, all three tiers, via `make test`.** No tier skipped, no subset substituted for the whole. Preliminary per-language checks (Python compile + import, JS parse, CSS brace matching, HTML tag matching) and a clean `python -c "from devplacepy.main import app"` import come first, but they never replace the suite. A failure is a real signal and blocks completion until fixed.
During iteration a single test may be run for a fast feedback loop: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x` - but the change is finished only when the complete suite is green.
The suite does NOT stop at the first failure. `-rf` lives in `pyproject.toml``addopts` (one source of truth, so a bare `python -m pytest ...` gets it too), and every run prints a `FAILED <nodeid>` line per failure at the end - the complete failure list from a single pass instead of one full pass per bug. Triage cheapest tier first:
| Command | Covers | Cost |
|---|---|---|
| `make test-fast` | unit + api, no browser | ~3 min |
| `make test-unit` | unit only | ~50 s |
| `make test-api` | api only | ~2 min |
| `make test-e2e` | Playwright browser tier | the expensive one |
| `make test` | all three tiers | full |
| `make test-failed` | only what failed last run (`--last-failed`) | proportional to the failure count |
| `make test-first-failure` | all three tiers with `-x` | stops at failure 1 |
| `make test-slowest` | all three tiers + the 40 slowest tests | full |
**Server-side display caches are disabled for the suite, never waited out.** A test that changes state the server serves from a TTL cache must not poll until the TTL expires - that burns the TTL in wall-clock on every run (the leaderboard ranking test cost 60s and the Code Farm saturation test 30s exactly this way). Each such cache reads its TTL from an env var with the production value as the default, and `conftest.py` sets it to `0` before the server subprocess starts: `DEVPLACE_SITEMAP_TTL` (`seo.SITEMAP_TTL`), `DEVPLACE_HOME_CACHE_TTL` (`main._home_cache`), `DEVPLACE_RANKING_TTL` (`database/ranking.py` `_authors_cache`), `DEVPLACE_MARKET_SATURATION_TTL` (`services/game/store/market.py` `_saturation_cache`). Add a new display cache to that list rather than sleeping in the test. **Correctness caches are different**: the cross-worker `cache_state` version read (`_cache_version_cache`, 1s) is load-bearing and stays on, so a test that flips the admin set from the test process must wait for the server to agree before asserting (see `tests/e2e/admin/containers/manage.py``_await_instance_visible`), never assume the change is instant.
The loop: `make test-fast` -> fix -> `make test-failed` until it is empty -> `make test` as the final gate. `make test-first-failure` is only for the rare case where one early failure poisons every test after it (a corrupted session fixture, a stopped service). Pytest's cache lives in `.pytest_cache/` (git-ignored), so `--last-failed` works with no extra setup; do not pass `-p no:cacheprovider` in a run whose failures you intend to re-run. **If `make test-failed` runs the whole suite instead of just the failures, the cache is polluted**: pytest keeps `lastfailed` entries for tests it did not collect, so node ids from deleted files (or from a stray `pytest` invocation that walked `data/`/`var/`) linger forever and make `--last-failed` give up and select everything. `make test-cache-clean` drops `.pytest_cache/` and fixes it; `make clean` does it too.
-`app_server` (session-scoped): spawns uvicorn as a subprocess on port 10501 (the `PORT` constant) with a tempfile DB.
-`browser_context` (session-scoped): one Playwright context shared by all tests.
-`page` (function-scoped): `clear_cookies()` on the session context, then a fresh page.
-`alice` / `bob`: seeded users `alice_test` / `bob_test` logged in via the login form. `bob` gets its own context for multi-user tests. Returns `(page, user_dict)`.
Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA_DIR`, and `DEVPLACE_DISABLE_SERVICES=1` so background services don't start. `make test` runs **serially, one test at a time, in a single process**: one isolated DB + data dir + uvicorn subprocess + Chromium, shared across the whole session, so session fixtures (`seeded_db`) and intra-file ordering hold. Serial execution is enforced centrally in `pyproject.toml` (`[tool.pytest.ini_options]` `addopts = "--tb=line -rf -p no:xdist"`); pytest-xdist is no longer a dependency and `-n` is rejected, so the suite can never run concurrently.
Opening a URL with a `#comment-<uid>` fragment scrolls the comment into view and adds the
`comment-highlight` class (`NotificationManager.initHashScroll`). Tests assert on
`#comment-<uid>.comment-highlight` after the page loads.
## Browser context
- **`browser_context` is session-scoped** (one per test session, shared by all tests in all files).
- **Cookies are cleared per test via `browser_context.clear_cookies()`** in the `page` fixture.
- **Each test gets a fresh `page`** from the shared context.
- **`bob` fixture creates its own context** from the session `browser` - necessary for multi-user tests.
- **Never share a page between two logged-in users** in the same test - use separate contexts.
## Test users
- **`alice_test` / `bob_test` are seeded once at session level** via HTTP POST to `/auth/signup`.
- **`alice` fixture logs in alice_test** via the login form.
- **`bob` fixture logs in bob_test** in a separate Playwright context.
- **Use `alice` for single-user tests.** It returns `(page, user_dict)`.
## Required test patterns (summary)
- Every `page.goto(...)` and `page.wait_for_url(...)` MUST pass `wait_until="domcontentloaded"`. CDN libs and avatars cause the default `"load"` to time out.
- Prefer `page.locator(...).wait_for(state="visible")` over `wait_for_selector`.
- Scope ambiguous selectors: comment Delete is `.comment-action-btn:has-text('Delete')`; create-post Post button is `#create-post-modal button.btn-primary:has-text('Post')`; comment textarea is `.comment-form textarea[name='content']`.
- Failure screenshots auto-save to `/tmp/devplace_test_screenshots/` via the `pytest_runtest_makereport` hook in `conftest.py`.
## Global `site_settings` tests
- **The uvicorn server is shared for the whole session.** A test that flips a global setting (maintenance mode, registration, rate limit, session length) changes behavior for every later test, so **restore the value in a `try/finally`** - see `tests/e2e/operational.py`.
- **Saving through the admin form is the reliable mutation path**, not a direct DB write: the form handler runs in the server process and calls `clear_settings_cache()`, so the change takes effect immediately (a direct DB write waits out the 60s server-side cache).
- **`conftest.py`'s `app_server` seeds `rate_limit_per_minute="1000000"`** so the suite is never throttled and the settings form round-trips a safe value instead of the template's `"60"` default.
- **Rate-limit unit tests patch `m.get_int_setting`**, not the `RATE_LIMIT` constant - the constant is now only the fallback default passed to `get_int_setting`.
- **Service enabled/disabled state is global state too - restore it.** Stopping a service through `POST /admin/services/{name}/stop` sets its `service_{name}_enabled` setting to `"0"` for the rest of the session. A test that stops a service (e.g. `test_gateway_disabled_returns_503` stops `openai` to assert the gateway returns 503) MUST restart it in a `try/finally` (`POST /admin/services/{name}/start`), and any `finally` that ends in `stop` should end in `start` instead - the default, expected state is enabled. Leaving the gateway stopped makes a later file's request hit `is_enabled()` first and get a `503` where it expected a `401`/`200`. The `openai` gateway's `handle()` checks `is_enabled()` (503) BEFORE `authorize()` (401), so a stopped gateway masks the auth result entirely.
- **The 1,000,000/min rate-limit seed only applies after the server's 60s settings cache expires.** `conftest.py` seeds `rate_limit_per_minute` with a raw DB insert (no `clear_settings_cache()`), so for the first ~60s of server uptime the server still uses the `60`/min default. A high-volume request loop (e.g. `test_auth_matrix`) run in the FIRST minute can therefore trip a spurious `429`; in the full suite that test runs well past the window so it never does. If you reproduce a `429` only in a tiny isolated run, it is this warm-up artifact, not a real auth/rate-limit issue.
- Operational tests live in `tests/e2e/operational.py` plus the `tests/api/admin/settings.py` family; rate-limit tests sit with the endpoint they probe (`tests/api/root.py`, `tests/api/auth/login.py`, `tests/api/robotstxt.py`).
## Failure handling
- **Failure screenshots auto-save** to `/tmp/devplace_test_screenshots/`.
## Never assert against a formula's base value when the real call has probabilistic or global inputs
Two whole-suite flakes came from tests that pinned an exact number the production formula does not actually guarantee. Both were invisible under `-x` because they sit late in the run.
- **Golden crops.** `economy.is_golden(plot_uid, planted_at)` is a deterministic hash, but its inputs are freshly generated per run, so ~5% of harvests pay `GOLDEN_MULTIPLIER` (5x). `assert result["coins"] == crop.reward_coins` is therefore a 5%-per-harvest flake. Every harvest-reward assertion must go through `economy.realizable_harvest_coins(crop, ..., golden=result["golden"])` - `store.harvest` returns `golden` precisely so the test can.
- **Global market state.** `economy.supply_days` divides by `market_store.active_farms()`, a **count over every farm in the database**, so a saturation fixture computed as `worst_days * 86400 / crop.grow_seconds` is only correct when exactly one farm is active. After the api/e2e tiers have created farms it lands two tiers milder and the assertion fails. Scale the seeded harvest count by `market_store.active_farms()` (see `_worst_tier_harvests` in `tests/unit/services/game/store.py`).
The rule generalizes: if the production function reads a random-ish input or a table-wide aggregate, assert against **that same function's output for the observed state**, never against the bare constant.
| Modal not opening/closing | Use `style.display`, not `classList.add/remove` |
| Dual Delete buttons match | Scope to `.comment-action-btn` in tests |
| Dual Post buttons match (feed inline comment) | Scope to `#create-post-modal button.btn-primary:has-text('Post')` in tests |
| Edit modal textarea conflicts with comment textarea | Scope to `.comment-form textarea[name='content']` for comments |
| Tests fail in sequence | Session-scoped context + `clear_cookies()` per test |
| Double messages in chat | Deduplicate by message UID with `seen` set |
| Avatar generation fails | Falls back to initial-based SVG - check multiavatar import |
| `Http.getJson` returns HTML / `JSON.parse` throws | The helper sends `Accept: application/json`; a content-negotiated route (`respond(..., model=...)`) needs it. Don't hand-roll `fetch()` without that header against a negotiated route |
| Page renders but data empty / 500 only after a CLI test ran earlier | A partial insert created the table with a reduced schema. Ensure the full column set in `init_db()` (see "Dataset rules -> `init_db()` MUST create every queried column") |
| `no such column: X` 500 in the server log | Same root cause: a queried/indexed column is missing because the table was created lazily by a partial insert. Add the column to the `init_db()` ensure-block |
| Action button click times out ("element is not visible") | The button moved into the `.project-actions-overflow` menu. Open it first: click `.project-actions-more`, then `.context-menu-item:has-text('<Label>')` (see `tests/e2e/projects/index.py`) |
| `ImportError: cannot import name X from devplacepy.utils` in a test | The util was renamed (e.g. `badge_info` -> `get_badge`). Update the test import to the current name; grep `templating.py` globals for the new name |
| Admin page heading assertion fails | Admin pages use `<h2>` inside `.admin-toolbar`, not `<h1>` - assert `text=` or `.admin-toolbar h2:has-text(...)` |
| `test_auth_matrix` reports `422` instead of `401`/`403` | A POST route's form params are undocumented in `docs_api.py`, so the matrix sends an empty body and FastAPI validation (422) fires before the auth guard. Document every required form param so auth is reached and tested |
| `test_auth_matrix` reports `503` on a gateway endpoint | A prior test left the `openai` service stopped. Restore it in `finally` (see "Global `site_settings` tests -> Service enabled/disabled state") |
