forked from retoor/devplacepy
Report every test failure in one pass and fix the whole suite
The suite ran with -x, so a run stopped at the first failure and finding N failures cost N full runs. Move -rf into the pytest addopts so every run lists each failure, and add the triage targets test-fast (unit + api, no browser), test-failed (--last-failed), test-first-failure (the old -x), test-slowest and test-cache-clean. A stale .pytest_cache holding node ids from deleted files made --last-failed select everything; make clean and test-cache-clean drop it. Fix the fifteen failures this surfaced. DeepSearch crawling raised AttributeError in its finally block on every run: async_playwright().__aenter__() returns a Playwright, which has no __aexit__. Use start()/stop() at both call sites. Update the tests left behind by changed signatures: VectorStore is async now, fetch_page takes a browser, _summary_payload takes dom_evidence, and the messages/notifications page compounds take a user_uid. Close four real flakes that fail-fast had been hiding, all of them late in the run. Harvest assertions pinned crop.reward_coins while is_golden pays five times on about five percent of harvests, so they now assert through realizable_harvest_coins with the observed golden flag. Market saturation fixtures assumed a single active farm and landed two tiers milder once the api and e2e tiers had created farms, so they scale by active_farms(). The primary-administrator container test raced the one second cross-worker cache version window and now waits for the server to agree. The isslop tools test matched the collapsed nav dropdown link instead of the tools grid card. Stop burning ninety seconds waiting out server-side display caches: DEVPLACE_RANKING_TTL and DEVPLACE_MARKET_SATURATION_TTL follow the existing sitemap and home cache precedent and are zero for the suite, taking the leaderboard test from 60.6s to 4.4s and the saturation test from 30.1s to under a second. 2881 passed, 1 skipped in 15:13.
This commit is contained in:
@@ -23,7 +23,11 @@ make install # pip install -e . + playwright install chromium
|
||||
make ppy # build the single shared container image (ppy:latest); run once before launching instances
|
||||
make dev # uvicorn --reload on port 10500, backlog 4096
|
||||
make prod # uvicorn --workers $(WEB_WORKERS) (defaults to nproc), port 10500 (backlog 8192)
|
||||
make test # Playwright + unit tests, headless, serial (one at a time), -x fail-fast
|
||||
make test # full suite (unit + api + e2e), headless, serial; one pass reports EVERY failure
|
||||
make test-fast # unit + api only, no browser - the quickest triage pass (~3 min)
|
||||
make test-failed # re-run only the tests that failed in the previous run
|
||||
make test-first-failure # full suite with -x, stops at the first failure
|
||||
make test-slowest # full suite plus the 40 slowest tests, to find what costs wall-clock
|
||||
make test-headed # same tests in a visible Chromium window (single process)
|
||||
make locust # Locust load test, interactive web UI
|
||||
make locust-headless # Locust CLI mode for CI
|
||||
@@ -35,6 +39,8 @@ Preliminary validation: confirm `python -c "from devplacepy.main import app"` im
|
||||
|
||||
Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x`
|
||||
|
||||
**Finding failures fast (the triage order).** The suite no longer stops at the first failure - `-rf` is in `pyproject.toml` `addopts`, so every run (make target or bare `pytest`) prints one `FAILED <nodeid>` line per failure at the end, giving the complete list from a single pass instead of one pass per bug. Triage cheapest-first: `make test-fast` (unit + api, no browser, ~3 min) covers most regressions; only then pay for the browser tier with `make test` or `make test-e2e`. After a run, `make test-failed` re-runs just the failures from pytest's cache (`--last-failed`), which is the loop to iterate in until it is empty. `make test-first-failure` keeps the old `-x` behaviour for the rare case where a single early failure poisons everything after it.
|
||||
|
||||
CLI (installed as `devplace`):
|
||||
```bash
|
||||
devplace role get <username>
|
||||
@@ -291,7 +297,7 @@ Admin **Trash** at `/admin/trash` restores/purges by event. Full table list, dat
|
||||
|
||||
## Testing
|
||||
|
||||
Playwright (NOT pytest-playwright). Around 1959 tests in `tests/`, split into three category directories by *what they exercise*: `tests/api/` (HTTP integration, no browser), `tests/e2e/` (Playwright browser), `tests/unit/` (pure in-process). **The directory tree mirrors the path** - one segment per directory, last segment is the file. `api`/`e2e` mirror the URL path (`GET /admin/ai-usage` -> `tests/e2e/admin/aiusage.py`); `unit` mirrors the source module path (`devplacepy/utils.py` -> `tests/unit/utils.py`). A test's tier is decided by its fixtures: `page`/`alice`/`bob` = e2e; `app_server`/`seeded_db` or any HTTP call = api; `local_db`-only or no fixture = unit. Run a tier with `make test-unit`/`make test-api`/`make test-e2e`; `make test` runs all three.
|
||||
Playwright (NOT pytest-playwright). Around 2882 tests in `tests/`, split into three category directories by *what they exercise*: `tests/api/` (HTTP integration, no browser), `tests/e2e/` (Playwright browser), `tests/unit/` (pure in-process). **The directory tree mirrors the path** - one segment per directory, last segment is the file. `api`/`e2e` mirror the URL path (`GET /admin/ai-usage` -> `tests/e2e/admin/aiusage.py`); `unit` mirrors the source module path (`devplacepy/utils.py` -> `tests/unit/utils.py`). A test's tier is decided by its fixtures: `page`/`alice`/`bob` = e2e; `app_server`/`seeded_db` or any HTTP call = api; `local_db`-only or no fixture = unit. Run a tier with `make test-unit`/`make test-api`/`make test-e2e`; `make test` runs all three.
|
||||
|
||||
Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `browser_context` (session-scoped Playwright context), `page` (function-scoped, fresh cookies), `alice`/`bob` (seeded logged-in users, `bob` gets its own context for multi-user tests).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user