# Test framework and rules The structure of the correctness suite, the fixtures that back it, and the mandatory patterns every test must follow. See also [Testing overview](/docs/testing.html) and [Running tests via make](/docs/testing-make.html). ## Playwright, not pytest-playwright The suite uses **raw Playwright** driven by plain `pytest`, not the `pytest-playwright` plugin, because the two conflict over fixture names. If `pytest-playwright` is installed, uninstall it before running the suite. ## Layout: three tiers, a directory tree that mirrors the path Tests are split into three category directories by *what each test exercises*, and within each tier the **directory structure mirrors the path** (one segment per directory, the last segment is the file): - **`tests/api/`** - HTTP integration tests (`requests`/`httpx` vs the live `app_server`), no browser. Organised by endpoint path. - **`tests/e2e/`** - Playwright browser tests (`page`/`alice`/`bob`). Organised by endpoint path. - **`tests/unit/`** - pure in-process tests (the `local_db` fixture or no fixture) over functions and data helpers, no running server. Organised by the source module path of the code under test. A test's tier is decided by the fixtures it uses: a browser fixture (`page`/`alice`/`bob`) makes it `e2e`; `app_server`/`seeded_db` or any HTTP call makes it `api`; `local_db`-only or no fixture makes it `unit`. **api and e2e directories follow the endpoint path:** drop `{param}` segments, lowercase each remaining segment and strip non-alphanumeric characters, then make every segment but the last a directory and the last the file. Examples: `GET /admin/ai-usage` → `tests/e2e/admin/aiusage.py`, `POST /auth/login` → `tests/api/auth/login.py`, `GET /projects/{slug}/files/lines` → `tests/api/projects/files/lines.py`. A bare collection path that is also a parent uses `index.py` inside its own directory (`/posts` → `tests/e2e/posts/index.py`, `/posts/create` → `tests/e2e/posts/create.py`); `GET /` → `tests/<tier>/root.py`. Multiple verbs of one path share one file. **unit directories mirror the source module:** tests for `devplacepy/utils.py` live in `tests/unit/utils.py`, and tests for `devplacepy/services/audit/store.py` live in `tests/unit/services/audit/store.py`. Every directory is a Python package (`__init__.py`). Files are **not** `test_`-prefixed, so `pyproject.toml` widens discovery with `python_files = ["*.py"]` and points `testpaths` at the three subdirectories; test *functions* still start with `test_`. The single `tests/conftest.py` stays at the `tests/` root so its fixtures inherit into every tier. Helpers shared across files import from the canonical module path (for example `from tests.e2e.post import create_post`). ## Fixture stack All fixtures live in `tests/conftest.py`. They layer from a shared server up to per-test isolation: | Fixture | Scope | Purpose | |---------|-------|---------| | `app_server` | session | Spawns Uvicorn as a subprocess on port 10501 with a tempfile SQLite database, then waits for it to answer. | | `playwright_instance` / `browser` | session | One Playwright runtime and one Chromium browser for the whole run. | | `browser_context` | module | One shared browser context. | | `page` | function | Clears cookies on the shared context, then yields a fresh page. | | `seeded_db` | session | Seeds baseline users and content once. | | `alice` / `bob` | function | Logged-in seeded users (`alice_test` / `bob_test`); `bob` gets its own context for multi-user tests. Each returns `(page, user_dict)`. | In-process tests skip the browser entirely and call a FastAPI `TestClient` or run coroutines through the `run_async` helper, which executes them on a worker thread and refreshes the database snapshot afterward. ## Serial execution, isolation, and ports The suite runs **serially, one test at a time, in a single process**. There is one server on port `10501`, backed by one tempfile database and one `DEVPLACE_DATA_DIR`, shared for the whole session. Serial execution is enforced centrally in `pyproject.toml` (`[tool.pytest.ini_options]` sets `addopts = "--tb=line -p no:xdist"`), so passing `-n` is rejected and the suite can never run concurrently. Background services are disabled in tests via `DEVPLACE_DISABLE_SERVICES=1`, and the rate limit is seeded to `1000000` so the suite is never throttled. Because every test shares one server and one database, tests that mutate global or seeded state MUST clean up after themselves (see the `try/finally` rule below), and a test that changes a cached user's row through a direct database write should invalidate the auth cache (`clear_user_cache(uid)`) so the running server observes the change. ## Mandatory patterns These are not style preferences; violating them produces flaky or hanging tests: - **Always pass `wait_until="domcontentloaded"`** to every `page.goto(...)` and `page.wait_for_url(...)`. CDN libraries and avatars make the default `"load"` event time out. - **Prefer `page.locator(...).wait_for(state="visible")`** over `wait_for_selector`. - **Scope ambiguous selectors.** Examples: comment Delete is `.comment-action-btn:has-text('Delete')`; the create-post Post button is `#create-post-modal button.btn-primary:has-text('Post')`; the comment textarea is `.comment-form textarea[name='content']`. - **Restore global state in `try/finally`.** The server is shared session-wide, so a test that flips a `site_settings` value (maintenance mode, registration, rate limit, session length) MUST restore it. Rate-limit tests patch `get_int_setting` rather than the fallback constant. ## Failure diagnostics A `pytest_runtest_makereport` hook saves a screenshot to `/tmp/devplace_test_screenshots/` whenever a Playwright test fails. In CI those screenshots are uploaded as a `failure-screenshots` artifact. ## When to write which test - **UI or browser behavior** (rendering, clicks, forms, navigation, JS): a Playwright test in `tests/e2e/<endpoint>.py`. This is required for every user-facing feature. - **JSON response shape, auth guards, status codes** against the live server: an HTTP test in `tests/api/<endpoint>.py`. - **Data helpers and pure logic** with no server: a unit test in `tests/unit/<module>.py`. Many features need all three: the unit test pins helper logic, the api test pins the JSON contract and guards, and the e2e test proves the page renders. ## Running a single test ```bash python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x ``` The full suite is run through the `make` targets documented in [Running tests via make](/docs/testing-make.html).