forked from retoor/devplacepy
chore: reorganize test files into domain-specific subdirectories under tests/
Split the monolithic test directory into three tiers (unit, api, e2e) with a path-mirroring directory structure. Added corresponding Makefile targets (test-unit, test-api, test-e2e) and updated all documentation references (CLAUDE.md, README.md, testing-cicd.html, testing-framework.html, testing-make.html) to reflect the new layout and naming conventions.
This commit is contained in:
@@ -45,7 +45,7 @@ DevPlace promotes a change through automated Development, Test, Acceptance, and
|
||||
| Street | Where | What happens |
|
||||
|--------|-------|--------------|
|
||||
| **Development** | a developer's machine, `make dev` | Code is written and run against the shared local SQLite database with live reload. Locally, `make test` and `make coverage` are available before pushing. |
|
||||
| **Test** | Gitea Actions on `master` | Every push and pull request to `master` triggers the full Playwright + unit suite under coverage. This is the automated quality gate: the coverage report is published as an artifact and failure screenshots are captured. |
|
||||
| **Test** | Gitea Actions on `master` | Every push and pull request to `master` triggers the full suite (all three tiers - `tests/unit`, `tests/api`, `tests/e2e`) under coverage. This is the automated quality gate: the coverage report is published as an artifact and failure screenshots are captured. |
|
||||
| **Acceptance** | the `master` to `production` promotion | `make deploy` fast-forwards the release branch: `git checkout production && git merge master && git push origin production`. Only commits that are already green on `master` reach `production`, so the promotion is the controlled release gate. |
|
||||
| **Production** | the host running Docker Compose | The production host pulls the `production` branch and runs the nginx + app stack. Most updates are a `git pull` plus a restart because code is bind-mounted; the image is rebuilt only when dependencies change. See the [Production overview](/docs/production.html) section. |
|
||||
|
||||
|
||||
@@ -7,6 +7,22 @@ The structure of the correctness suite, the fixtures that back it, and the manda
|
||||
|
||||
The suite uses **raw Playwright** driven by plain `pytest`, not the `pytest-playwright` plugin. The two conflict over fixture names. If `pytest-playwright` is installed, uninstall it before running the suite.
|
||||
|
||||
## Layout: three tiers, a directory tree that mirrors the path
|
||||
|
||||
Tests are split into three category directories by *what each test exercises*, and within each tier the **directory structure mirrors the path** (one segment per directory, the last segment is the file):
|
||||
|
||||
- **`tests/api/`** - HTTP integration tests (`requests`/`httpx` vs the live `app_server`), no browser. Organised by endpoint path.
|
||||
- **`tests/e2e/`** - Playwright browser tests (`page`/`alice`/`bob`). Organised by endpoint path.
|
||||
- **`tests/unit/`** - pure in-process tests (the `local_db` fixture or no fixture) over functions and data helpers, no running server. Organised by the source module path of the code under test.
|
||||
|
||||
A test's tier is decided by the fixtures it uses: a browser fixture (`page`/`alice`/`bob`) makes it `e2e`; `app_server`/`seeded_db` or any HTTP call makes it `api`; `local_db`-only or no fixture makes it `unit`.
|
||||
|
||||
**api and e2e directories follow the endpoint path:** drop `{param}` segments, lowercase each remaining segment and strip non-alphanumeric characters, then make every segment but the last a directory and the last the file. Examples: `GET /admin/ai-usage` → `tests/e2e/admin/aiusage.py`, `POST /auth/login` → `tests/api/auth/login.py`, `GET /projects/{slug}/files/lines` → `tests/api/projects/files/lines.py`. A bare collection path that is also a parent uses `index.py` inside its own directory (`/posts` → `tests/e2e/posts/index.py`, `/posts/create` → `tests/e2e/posts/create.py`); `GET /` → `tests/<tier>/root.py`. Multiple verbs of one path share one file.
|
||||
|
||||
**unit directories mirror the source module:** tests for `devplacepy/utils.py` live in `tests/unit/utils.py`, and tests for `devplacepy/services/audit/store.py` live in `tests/unit/services/audit/store.py`.
|
||||
|
||||
Every directory is a Python package (`__init__.py`). Files are **not** `test_`-prefixed, so `pyproject.toml` widens discovery with `python_files = ["*.py"]` and points `testpaths` at the three subdirectories; test *functions* still start with `test_`. The single `tests/conftest.py` stays at the `tests/` root so its fixtures inherit into every tier. Helpers shared across files import from the canonical new module path (for example `from tests.e2e.post import create_post`).
|
||||
|
||||
## Fixture stack
|
||||
|
||||
All fixtures live in `tests/conftest.py`. They layer from a shared server up to per-test isolation:
|
||||
@@ -35,7 +51,7 @@ These are not style preferences. Violating them produces flaky or hanging tests:
|
||||
- **Always pass `wait_until="domcontentloaded"`** to every `page.goto(...)` and `page.wait_for_url(...)`. CDN libraries and avatars make the default `"load"` event time out.
|
||||
- **Prefer `page.locator(...).wait_for(state="visible")`** over `wait_for_selector`.
|
||||
- **Scope ambiguous selectors.** Examples: comment Delete is `.comment-action-btn:has-text('Delete')`; the create-post Post button is `#create-post-modal button.btn-primary:has-text('Post')`; the comment textarea is `.comment-form textarea[name='content']`.
|
||||
- **Restore global state in `try/finally`.** The server is shared session-wide, so a test that flips a `site_settings` value (maintenance mode, registration, rate limit, session length) MUST restore it. Rate-limit unit tests patch `get_int_setting` rather than the fallback constant.
|
||||
- **Restore global state in `try/finally`.** The server is shared session-wide, so a test that flips a `site_settings` value (maintenance mode, registration, rate limit, session length) MUST restore it. Rate-limit tests patch `get_int_setting` rather than the fallback constant.
|
||||
|
||||
## Failure diagnostics
|
||||
|
||||
@@ -43,15 +59,16 @@ A `pytest_runtest_makereport` hook saves a screenshot to `/tmp/devplace_test_scr
|
||||
|
||||
## When to write which test
|
||||
|
||||
- **UI or browser behavior** (rendering, clicks, forms, navigation, JS): end-to-end Playwright test. This is required for every user-facing feature.
|
||||
- **JSON response shape, auth guards, data helpers, pure logic**: in-process `TestClient` or `asyncio` test.
|
||||
- **UI or browser behavior** (rendering, clicks, forms, navigation, JS): a Playwright test in `tests/e2e/<endpoint>.py`. This is required for every user-facing feature.
|
||||
- **JSON response shape, auth guards, status codes** against the live server: an HTTP test in `tests/api/<endpoint>.py`.
|
||||
- **Data helpers and pure logic** with no server: a unit test in `tests/unit/<module>.py`.
|
||||
|
||||
Many features need both: the in-process test pins the JSON contract and the E2E test proves the page works.
|
||||
Many features need all three: the unit test pins helper logic, the api test pins the JSON contract and guards, and the e2e test proves the page works.
|
||||
|
||||
## Running a single test
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_feed.py::test_name -v --tb=line -x
|
||||
python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x
|
||||
```
|
||||
|
||||
The full suite is run through the `make` targets documented in [Running tests via make](/docs/testing-make.html).
|
||||
|
||||
@@ -19,16 +19,19 @@ python -m playwright install chromium # browser used by the E2E suite
|
||||
|--------|--------------|
|
||||
| `make test` | Full suite headless, serial (one test at a time), fail-fast (`-x`). The everyday command. |
|
||||
| `make test-headed` | The same suite in a visible browser, for watching or debugging an E2E test. |
|
||||
| `make test-unit` | Only `tests/unit/` - pure in-process tests, no server, fastest. |
|
||||
| `make test-api` | Only `tests/api/` - HTTP integration tests against the live server, no browser. |
|
||||
| `make test-e2e` | Only `tests/e2e/` - the Playwright browser tests, headless. |
|
||||
| `make coverage` | Full suite under coverage, then prints a coverage report. |
|
||||
| `make coverage-headed` | Coverage run with a visible browser. |
|
||||
| `make coverage-html` | Runs `coverage`, then writes the browsable HTML report to `htmlcov/index.html`. |
|
||||
|
||||
`make test` sets `PLAYWRIGHT_HEADLESS=1`; `make test-headed` sets it to `0`. The whole suite runs in a single process: serial execution is enforced in `pyproject.toml` (`addopts = "--tb=line -p no:xdist"`), so there are no parallel workers and the coverage targets collect everything in one process, exactly how CI runs.
|
||||
|
||||
To run one test instead of the suite, call pytest directly:
|
||||
Tests are split into `tests/unit/`, `tests/api/`, and `tests/e2e/` as a directory tree that mirrors the path (api/e2e by endpoint, for example `tests/e2e/admin/aiusage.py`; unit by source module); see [Test framework and rules](/docs/testing-framework.html) for the naming convention. To run one test instead of the suite, call pytest directly:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_feed.py::test_name -v --tb=line -x
|
||||
python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x
|
||||
```
|
||||
|
||||
## Load testing
|
||||
|
||||
@@ -3,14 +3,21 @@
|
||||
|
||||
How DevPlace verifies itself: an end-to-end browser suite, in-process unit tests, and a load-testing rig, all driven through the project `Makefile`, measured for coverage, and promoted through automated DTAP streets. See also [Test framework and rules](/docs/testing-framework.html), [Load testing with Locust](/docs/testing-locust.html), [Running tests via make](/docs/testing-make.html), and [Coverage and CI/CD](/docs/testing-cicd.html).
|
||||
|
||||
## Two layers, one suite
|
||||
## Three tiers, a directory tree that mirrors the path
|
||||
|
||||
The `tests/` directory holds the correctness suite: over 600 tests across 48 `test_*.py` files. Two styles live side by side and run under the same `pytest` invocation:
|
||||
The `tests/` directory holds the correctness suite: 932 tests organised into three category directories by *what each test exercises*. Within each tier the **directory structure mirrors the path** - one segment per directory, the last segment is the file:
|
||||
|
||||
- **End-to-end (Playwright).** A real Chromium browser drives a real Uvicorn server. These cover anything a user sees or clicks: rendering, forms, navigation, modals, and JavaScript behavior.
|
||||
- **In-process unit and API tests.** A FastAPI `TestClient` (or direct `asyncio` calls) exercises routers, data helpers, and JSON responses without a browser, for logic that has no UI surface.
|
||||
- **`tests/api/` - HTTP integration tests.** `requests`/`httpx` drive a real Uvicorn server (`app_server`) and assert on status, JSON, and headers without a browser. **Organised by endpoint path.**
|
||||
- **`tests/e2e/` - end-to-end (Playwright).** A real Chromium browser drives the same Uvicorn server, covering anything a user sees or clicks. **Organised by endpoint path.**
|
||||
- **`tests/unit/` - in-process unit tests.** Pure tests over functions and data helpers (the `local_db` fixture or none), no running server. **Organised by the source module path** of the code under test.
|
||||
|
||||
Both styles share one fixture stack and one ephemeral SQLite database, so a single `make test` runs everything.
|
||||
A test's tier is chosen by its fixtures: a browser fixture (`page`/`alice`/`bob`) makes it e2e; `app_server`/`seeded_db` or any HTTP call makes it api; `local_db`-only or no fixture makes it unit.
|
||||
|
||||
**api and e2e tests follow the endpoint's path:** each route segment becomes a directory and the final segment becomes the file, with `{param}` segments dropped and each segment lowercased + stripped of non-alphanumerics. So `GET /admin/ai-usage` becomes `tests/e2e/admin/aiusage.py`, `POST /auth/login` becomes `tests/api/auth/login.py`, and `GET /projects/{slug}/files/lines` becomes `tests/api/projects/files/lines.py`. A bare collection path that is also a parent of deeper paths uses `index.py` inside its own directory (`GET /posts` → `tests/e2e/posts/index.py`, beside `POST /posts/create` → `tests/e2e/posts/create.py`); `GET /` becomes `tests/<tier>/root.py`. Multiple HTTP verbs for one path share a single file.
|
||||
|
||||
**unit tests mirror the source module path:** tests for `devplacepy/utils.py` live in `tests/unit/utils.py`, and tests for `devplacepy/services/audit/store.py` live in `tests/unit/services/audit/store.py`.
|
||||
|
||||
Every directory is a Python package (`__init__.py`). Files are not `test_`-prefixed (the discovery glob is widened to `*.py`); test *functions* still start with `test_`. All three tiers share one fixture stack (the root `tests/conftest.py`) and one ephemeral SQLite database, so a single `make test` runs everything, while `make test-unit`, `make test-api`, and `make test-e2e` run a single tier.
|
||||
|
||||
## The load layer
|
||||
|
||||
|
||||
Reference in New Issue
Block a user