Report every test failure in one pass and fix the whole suite

The suite ran with -x, so a run stopped at the first failure and finding N
failures cost N full runs. Move -rf into the pytest addopts so every run
lists each failure, and add the triage targets test-fast (unit + api, no
browser), test-failed (--last-failed), test-first-failure (the old -x),
test-slowest and test-cache-clean. A stale .pytest_cache holding node ids
from deleted files made --last-failed select everything; make clean and
test-cache-clean drop it.

Fix the fifteen failures this surfaced.

DeepSearch crawling raised AttributeError in its finally block on every run:
async_playwright().__aenter__() returns a Playwright, which has no __aexit__.
Use start()/stop() at both call sites.

Update the tests left behind by changed signatures: VectorStore is async now,
fetch_page takes a browser, _summary_payload takes dom_evidence, and the
messages/notifications page compounds take a user_uid.

Close four real flakes that fail-fast had been hiding, all of them late in
the run. Harvest assertions pinned crop.reward_coins while is_golden pays
five times on about five percent of harvests, so they now assert through
realizable_harvest_coins with the observed golden flag. Market saturation
fixtures assumed a single active farm and landed two tiers milder once the
api and e2e tiers had created farms, so they scale by active_farms(). The
primary-administrator container test raced the one second cross-worker cache
version window and now waits for the server to agree. The isslop tools test
matched the collapsed nav dropdown link instead of the tools grid card.

Stop burning ninety seconds waiting out server-side display caches:
DEVPLACE_RANKING_TTL and DEVPLACE_MARKET_SATURATION_TTL follow the existing
sitemap and home cache precedent and are zero for the suite, taking the
leaderboard test from 60.6s to 4.4s and the saturation test from 30.1s to
under a second.

2881 passed, 1 skipped in 15:13.
This commit is contained in:
retoor 2026-07-26 16:02:36 +02:00
parent 4780016980
commit f996336afb
17 changed files with 149 additions and 49 deletions

View File

@ -23,7 +23,11 @@ make install # pip install -e . + playwright install chromium
make ppy # build the single shared container image (ppy:latest); run once before launching instances make ppy # build the single shared container image (ppy:latest); run once before launching instances
make dev # uvicorn --reload on port 10500, backlog 4096 make dev # uvicorn --reload on port 10500, backlog 4096
make prod # uvicorn --workers $(WEB_WORKERS) (defaults to nproc), port 10500 (backlog 8192) make prod # uvicorn --workers $(WEB_WORKERS) (defaults to nproc), port 10500 (backlog 8192)
make test # Playwright + unit tests, headless, serial (one at a time), -x fail-fast make test # full suite (unit + api + e2e), headless, serial; one pass reports EVERY failure
make test-fast # unit + api only, no browser - the quickest triage pass (~3 min)
make test-failed # re-run only the tests that failed in the previous run
make test-first-failure # full suite with -x, stops at the first failure
make test-slowest # full suite plus the 40 slowest tests, to find what costs wall-clock
make test-headed # same tests in a visible Chromium window (single process) make test-headed # same tests in a visible Chromium window (single process)
make locust # Locust load test, interactive web UI make locust # Locust load test, interactive web UI
make locust-headless # Locust CLI mode for CI make locust-headless # Locust CLI mode for CI
@ -35,6 +39,8 @@ Preliminary validation: confirm `python -c "from devplacepy.main import app"` im
Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x` Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x`
**Finding failures fast (the triage order).** The suite no longer stops at the first failure - `-rf` is in `pyproject.toml` `addopts`, so every run (make target or bare `pytest`) prints one `FAILED <nodeid>` line per failure at the end, giving the complete list from a single pass instead of one pass per bug. Triage cheapest-first: `make test-fast` (unit + api, no browser, ~3 min) covers most regressions; only then pay for the browser tier with `make test` or `make test-e2e`. After a run, `make test-failed` re-runs just the failures from pytest's cache (`--last-failed`), which is the loop to iterate in until it is empty. `make test-first-failure` keeps the old `-x` behaviour for the rare case where a single early failure poisons everything after it.
CLI (installed as `devplace`): CLI (installed as `devplace`):
```bash ```bash
devplace role get <username> devplace role get <username>
@ -291,7 +297,7 @@ Admin **Trash** at `/admin/trash` restores/purges by event. Full table list, dat
## Testing ## Testing
Playwright (NOT pytest-playwright). Around 1959 tests in `tests/`, split into three category directories by *what they exercise*: `tests/api/` (HTTP integration, no browser), `tests/e2e/` (Playwright browser), `tests/unit/` (pure in-process). **The directory tree mirrors the path** - one segment per directory, last segment is the file. `api`/`e2e` mirror the URL path (`GET /admin/ai-usage` -> `tests/e2e/admin/aiusage.py`); `unit` mirrors the source module path (`devplacepy/utils.py` -> `tests/unit/utils.py`). A test's tier is decided by its fixtures: `page`/`alice`/`bob` = e2e; `app_server`/`seeded_db` or any HTTP call = api; `local_db`-only or no fixture = unit. Run a tier with `make test-unit`/`make test-api`/`make test-e2e`; `make test` runs all three. Playwright (NOT pytest-playwright). Around 2882 tests in `tests/`, split into three category directories by *what they exercise*: `tests/api/` (HTTP integration, no browser), `tests/e2e/` (Playwright browser), `tests/unit/` (pure in-process). **The directory tree mirrors the path** - one segment per directory, last segment is the file. `api`/`e2e` mirror the URL path (`GET /admin/ai-usage` -> `tests/e2e/admin/aiusage.py`); `unit` mirrors the source module path (`devplacepy/utils.py` -> `tests/unit/utils.py`). A test's tier is decided by its fixtures: `page`/`alice`/`bob` = e2e; `app_server`/`seeded_db` or any HTTP call = api; `local_db`-only or no fixture = unit. Run a tier with `make test-unit`/`make test-api`/`make test-e2e`; `make test` runs all three.
Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `browser_context` (session-scoped Playwright context), `page` (function-scoped, fresh cookies), `alice`/`bob` (seeded logged-in users, `bob` gets its own context for multi-user tests). Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `browser_context` (session-scoped Playwright context), `page` (function-scoped, fresh cookies), `alice`/`bob` (seeded logged-in users, `bob` gets its own context for multi-user tests).

View File

@ -12,7 +12,7 @@ DEVPLACE_RATE_LIMIT ?= 1000000
PYTHONDONTWRITEBYTECODE := 1 PYTHONDONTWRITEBYTECODE := 1
export PYTHONDONTWRITEBYTECODE export PYTHONDONTWRITEBYTECODE
.PHONY: install dev clean tree tree-loc zip test test-headed coverage coverage-headed coverage-html locust locust-headless .PHONY: install dev clean tree tree-loc zip test test-headed test-unit test-api test-e2e test-fast test-failed test-first-failure test-slowest test-cache-clean coverage coverage-headed coverage-html locust locust-headless
install: install:
pip install -e . pip install -e .
@ -43,19 +43,31 @@ zip:
@printf 'Wrote %s (%s files)\n' "$(notdir $(CURDIR)).zip" "$$(git ls-files | wc -l)" @printf 'Wrote %s (%s files)\n' "$(notdir $(CURDIR)).zip" "$$(git ls-files | wc -l)"
test: test:
PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ -x PLAYWRIGHT_HEADLESS=1 python -m pytest tests/
test-headed: test-headed:
PLAYWRIGHT_HEADLESS=0 python -m pytest tests/ -x PLAYWRIGHT_HEADLESS=0 python -m pytest tests/
test-unit: test-unit:
python -m pytest tests/unit -x python -m pytest tests/unit
test-api: test-api:
python -m pytest tests/api -x python -m pytest tests/api
test-e2e: test-e2e:
PLAYWRIGHT_HEADLESS=1 python -m pytest tests/e2e -x PLAYWRIGHT_HEADLESS=1 python -m pytest tests/e2e
test-fast:
python -m pytest tests/unit tests/api
test-failed:
PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ --last-failed --last-failed-no-failures none
test-first-failure:
PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ -x
test-slowest:
PLAYWRIGHT_HEADLESS=1 python -m pytest tests/ --durations=40
coverage: coverage:
rm -f .coverage .coverage.* rm -f .coverage .coverage.*
@ -109,8 +121,12 @@ clean:
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
find . -type f -name '*.pyc' -delete find . -type f -name '*.pyc' -delete
rm -rf devplacepy.egg-info rm -rf devplacepy.egg-info
rm -rf .pytest_cache
rm -rf .venv rm -rf .venv
test-cache-clean:
rm -rf .pytest_cache
# Container Manager works out of the box: the overlay installs the docker CLI in # Container Manager works out of the box: the overlay installs the docker CLI in
# the image and mounts the host socket. DOCKER_GID is read straight from the # the image and mounts the host socket. DOCKER_GID is read straight from the
# socket so the UID-1000 app can use it; the data dir is the project's own data/ # socket so the UID-1000 app can use it; the data dir is the project's own data/

View File

@ -1,5 +1,7 @@
# retoor <retoor@molodetz.nl> # retoor <retoor@molodetz.nl>
import os
from .core import TTLCache, _in_clause, _now_iso, db, get_table from .core import TTLCache, _in_clause, _now_iso, db, get_table
from .users import get_users_by_uids from .users import get_users_by_uids
from .soft_delete import soft_delete, soft_delete_in from .soft_delete import soft_delete, soft_delete_in
@ -17,7 +19,10 @@ VOTABLE_TARGETS: dict[str, str] = {
STAR_TARGETS: set[str] = {"post", "project", "gist", "quiz"} STAR_TARGETS: set[str] = {"post", "project", "gist", "quiz"}
_authors_cache = TTLCache(ttl=60, max_size=200) RANKING_TTL = int(os.environ.get("DEVPLACE_RANKING_TTL", "60"))
_authors_cache = TTLCache(ttl=RANKING_TTL, max_size=200)
_stars_cache = TTLCache(ttl=15, max_size=2000) _stars_cache = TTLCache(ttl=15, max_size=2000)

View File

@ -2,6 +2,7 @@
from __future__ import annotations from __future__ import annotations
import os
from datetime import datetime, timedelta from datetime import datetime, timedelta
from devplacepy.cache import TTLCache from devplacepy.cache import TTLCache
@ -12,7 +13,10 @@ from .. import economy
from .common import _iso from .common import _iso
_saturation_cache = TTLCache(ttl=30, max_size=32) SATURATION_TTL = int(os.environ.get("DEVPLACE_MARKET_SATURATION_TTL", "30"))
_saturation_cache = TTLCache(ttl=SATURATION_TTL, max_size=32)
def _ticks(): def _ticks():

View File

@ -53,7 +53,7 @@ build-backend = "hatchling.build"
[tool.pytest.ini_options] [tool.pytest.ini_options]
testpaths = ["tests/unit", "tests/api", "tests/e2e"] testpaths = ["tests/unit", "tests/api", "tests/e2e"]
python_files = ["*.py"] python_files = ["*.py"]
addopts = "--tb=line -p no:xdist" addopts = "--tb=line -rf -p no:xdist"
filterwarnings = [ filterwarnings = [
"ignore:'crypt' is deprecated", "ignore:'crypt' is deprecated",
"ignore:Using `httpx` with `starlette.testclient` is deprecated", "ignore:Using `httpx` with `starlette.testclient` is deprecated",

View File

@ -8,6 +8,26 @@ This file documents detailed testing patterns, fixtures, and pitfalls for devpla
During iteration a single test may be run for a fast feedback loop: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x` - but the change is finished only when the complete suite is green. During iteration a single test may be run for a fast feedback loop: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x` - but the change is finished only when the complete suite is green.
## Finding failures fast
The suite does NOT stop at the first failure. `-rf` lives in `pyproject.toml` `addopts` (one source of truth, so a bare `python -m pytest ...` gets it too), and every run prints a `FAILED <nodeid>` line per failure at the end - the complete failure list from a single pass instead of one full pass per bug. Triage cheapest tier first:
| Command | Covers | Cost |
|---|---|---|
| `make test-fast` | unit + api, no browser | ~3 min |
| `make test-unit` | unit only | ~50 s |
| `make test-api` | api only | ~2 min |
| `make test-e2e` | Playwright browser tier | the expensive one |
| `make test` | all three tiers | full |
| `make test-failed` | only what failed last run (`--last-failed`) | proportional to the failure count |
| `make test-first-failure` | all three tiers with `-x` | stops at failure 1 |
| `make test-slowest` | all three tiers + the 40 slowest tests | full |
| `make test-cache-clean` | drops `.pytest_cache/` when `--last-failed` misbehaves | instant |
**Server-side display caches are disabled for the suite, never waited out.** A test that changes state the server serves from a TTL cache must not poll until the TTL expires - that burns the TTL in wall-clock on every run (the leaderboard ranking test cost 60s and the Code Farm saturation test 30s exactly this way). Each such cache reads its TTL from an env var with the production value as the default, and `conftest.py` sets it to `0` before the server subprocess starts: `DEVPLACE_SITEMAP_TTL` (`seo.SITEMAP_TTL`), `DEVPLACE_HOME_CACHE_TTL` (`main._home_cache`), `DEVPLACE_RANKING_TTL` (`database/ranking.py` `_authors_cache`), `DEVPLACE_MARKET_SATURATION_TTL` (`services/game/store/market.py` `_saturation_cache`). Add a new display cache to that list rather than sleeping in the test. **Correctness caches are different**: the cross-worker `cache_state` version read (`_cache_version_cache`, 1s) is load-bearing and stays on, so a test that flips the admin set from the test process must wait for the server to agree before asserting (see `tests/e2e/admin/containers/manage.py` `_await_instance_visible`), never assume the change is instant.
The loop: `make test-fast` -> fix -> `make test-failed` until it is empty -> `make test` as the final gate. `make test-first-failure` is only for the rare case where one early failure poisons every test after it (a corrupted session fixture, a stopped service). Pytest's cache lives in `.pytest_cache/` (git-ignored), so `--last-failed` works with no extra setup; do not pass `-p no:cacheprovider` in a run whose failures you intend to re-run. **If `make test-failed` runs the whole suite instead of just the failures, the cache is polluted**: pytest keeps `lastfailed` entries for tests it did not collect, so node ids from deleted files (or from a stray `pytest` invocation that walked `data/`/`var/`) linger forever and make `--last-failed` give up and select everything. `make test-cache-clean` drops `.pytest_cache/` and fixes it; `make clean` does it too.
## Fixture stack ## Fixture stack
- `app_server` (session-scoped): spawns uvicorn as a subprocess on port 10501 (the `PORT` constant) with a tempfile DB. - `app_server` (session-scoped): spawns uvicorn as a subprocess on port 10501 (the `PORT` constant) with a tempfile DB.
@ -15,12 +35,12 @@ During iteration a single test may be run for a fast feedback loop: `python -m p
- `page` (function-scoped): `clear_cookies()` on the session context, then a fresh page. - `page` (function-scoped): `clear_cookies()` on the session context, then a fresh page.
- `alice` / `bob`: seeded users `alice_test` / `bob_test` logged in via the login form. `bob` gets its own context for multi-user tests. Returns `(page, user_dict)`. - `alice` / `bob`: seeded users `alice_test` / `bob_test` logged in via the login form. `bob` gets its own context for multi-user tests. Returns `(page, user_dict)`.
Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA_DIR`, and `DEVPLACE_DISABLE_SERVICES=1` so background services don't start. `make test` runs **serially, one test at a time, in a single process**: one isolated DB + data dir + uvicorn subprocess + Chromium, shared across the whole session, so session fixtures (`seeded_db`) and intra-file ordering hold. Serial execution is enforced centrally in `pyproject.toml` (`[tool.pytest.ini_options]` `addopts = "--tb=line -p no:xdist"`); pytest-xdist is no longer a dependency and `-n` is rejected, so the suite can never run concurrently. Tests run on port 10501 with a tempfile SQLite DB and a dedicated `DEVPLACE_DATA_DIR`, and `DEVPLACE_DISABLE_SERVICES=1` so background services don't start. `make test` runs **serially, one test at a time, in a single process**: one isolated DB + data dir + uvicorn subprocess + Chromium, shared across the whole session, so session fixtures (`seeded_db`) and intra-file ordering hold. Serial execution is enforced centrally in `pyproject.toml` (`[tool.pytest.ini_options]` `addopts = "--tb=line -rf -p no:xdist"`); pytest-xdist is no longer a dependency and `-n` is rejected, so the suite can never run concurrently.
## General ## General
- **Around 1959 tests across `tests/{unit,api,e2e}/`.** Playwright integration + HTTP + unit tests. All must pass before any merge. - **Around 2882 tests across `tests/{unit,api,e2e}/`.** Playwright integration + HTTP + unit tests. All must pass before any merge.
- **Tests use `-x` (fail-fast).** The suite stops at the first failure. Fix that test, then re-run. - **The suite runs to completion and reports every failure** (`-rf`), it does not stop at the first one. See "Finding failures fast" above.
- **Validate each touched language manually:** Python (compile + import), JS (parse / bracket matching), CSS (brace matching), HTML (tag matching). Zero tolerance. - **Validate each touched language manually:** Python (compile + import), JS (parse / bracket matching), CSS (brace matching), HTML (tag matching). Zero tolerance.
- Playwright is used directly, NOT pytest-playwright (uninstall it if present - it conflicts). - Playwright is used directly, NOT pytest-playwright (uninstall it if present - it conflicts).
@ -100,9 +120,18 @@ Opening a URL with a `#comment-<uid>` fragment scrolls the comment into view and
## Failure handling ## Failure handling
- **Failure screenshots auto-save** to `/tmp/devplace_test_screenshots/`. - **Failure screenshots auto-save** to `/tmp/devplace_test_screenshots/`.
- **Tests stop at first failure** (`-x` flag in Makefile). No cascading failures. - **Every failure is reported in one pass** (`-rf` in the Makefile); re-run just those with `make test-failed`.
- **If the server won't start, kill leftover processes:** `kill -9 $(pgrep -f "uvicorn")` - **If the server won't start, kill leftover processes:** `kill -9 $(pgrep -f "uvicorn")`
## Never assert against a formula's base value when the real call has probabilistic or global inputs
Two whole-suite flakes came from tests that pinned an exact number the production formula does not actually guarantee. Both were invisible under `-x` because they sit late in the run.
- **Golden crops.** `economy.is_golden(plot_uid, planted_at)` is a deterministic hash, but its inputs are freshly generated per run, so ~5% of harvests pay `GOLDEN_MULTIPLIER` (5x). `assert result["coins"] == crop.reward_coins` is therefore a 5%-per-harvest flake. Every harvest-reward assertion must go through `economy.realizable_harvest_coins(crop, ..., golden=result["golden"])` - `store.harvest` returns `golden` precisely so the test can.
- **Global market state.** `economy.supply_days` divides by `market_store.active_farms()`, a **count over every farm in the database**, so a saturation fixture computed as `worst_days * 86400 / crop.grow_seconds` is only correct when exactly one farm is active. After the api/e2e tiers have created farms it lands two tiers milder and the assertion fails. Scale the seeded harvest count by `market_store.active_farms()` (see `_worst_tier_harvests` in `tests/unit/services/game/store.py`).
The rule generalizes: if the production function reads a random-ish input or a table-wide aggregate, assert against **that same function's output for the observed state**, never against the bare constant.
## Common pitfalls ## Common pitfalls
| Pitfall | Fix | | Pitfall | Fix |

View File

@ -33,6 +33,8 @@ os.environ["DEVPLACE_WEB_WORKERS"] = "1"
os.environ["DEVPLACE_DISABLE_RATE_LIMIT"] = "1" os.environ["DEVPLACE_DISABLE_RATE_LIMIT"] = "1"
os.environ["DEVPLACE_SITEMAP_TTL"] = "0" os.environ["DEVPLACE_SITEMAP_TTL"] = "0"
os.environ["DEVPLACE_HOME_CACHE_TTL"] = "0" os.environ["DEVPLACE_HOME_CACHE_TTL"] = "0"
os.environ["DEVPLACE_RANKING_TTL"] = "0"
os.environ["DEVPLACE_MARKET_SATURATION_TTL"] = "0"
def run_async(coro): def run_async(coro):

View File

@ -100,6 +100,25 @@ def _cleanup(inst_uid):
refresh_snapshot() refresh_snapshot()
ADMIN_CACHE_PROPAGATION_SECONDS = 5.0
ADMIN_CACHE_POLL_SECONDS = 0.2
def _await_instance_visible(api_key, inst_uid):
deadline = time.time() + ADMIN_CACHE_PROPAGATION_SECONDS
while time.time() < deadline:
response = requests.get(
f"{BASE_URL}/admin/containers/data",
headers={"Accept": "application/json", "X-API-KEY": api_key},
)
if response.status_code == 200 and any(
inst["uid"] == inst_uid for inst in response.json()["instances"]
):
return
time.sleep(ADMIN_CACHE_POLL_SECONDS)
raise AssertionError(f"instance {inst_uid} never became visible to the viewer")
def test_admin_list_shows_view_only_for_non_owned_public_instance(page, app_server): def test_admin_list_shows_view_only_for_non_owned_public_instance(page, app_server):
owner = _make_admin() owner = _make_admin()
other = _make_admin() other = _make_admin()
@ -212,6 +231,7 @@ def test_primary_admin_sees_and_manages_others_private_instance(page, app_server
) )
inst_uid = _insert_instance(project["uid"], owner["uid"]) inst_uid = _insert_instance(project["uid"], owner["uid"])
try: try:
_await_instance_visible(primary["api_key"], inst_uid)
login_user(page, primary) login_user(page, primary)
page.goto(f"{BASE_URL}/admin/containers", wait_until="domcontentloaded") page.goto(f"{BASE_URL}/admin/containers", wait_until="domcontentloaded")
row = page.locator(f"tr.cm-row[data-uid='{inst_uid}']") row = page.locator(f"tr.cm-row[data-uid='{inst_uid}']")

View File

@ -463,21 +463,22 @@ def test_crop_shows_saturated_label_when_overfarmed(alice):
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
crop = economy.crop_for("shell") crop = economy.crop_for("shell")
worst_days = economy.MARKET_SATURATION_TIERS[-1][0] worst_days = economy.MARKET_SATURATION_TIERS[-1][0]
supply_per_harvest = crop.grow_seconds / 86400.0 / market_store.active_farms()
get_table("game_market_ticks").insert( get_table("game_market_ticks").insert(
{ {
"uid": generate_uid(), "uid": generate_uid(),
"crop_key": "shell", "crop_key": "shell",
"hour_bucket": market_store._hour_bucket(now), "hour_bucket": market_store._hour_bucket(now),
"harvests": int(worst_days * 86400 / crop.grow_seconds) + 1, "harvests": int(worst_days / supply_per_harvest) + 1,
"updated_at": now.isoformat(), "updated_at": now.isoformat(),
} }
) )
open_game(page) open_game(page)
option = page.locator("form[data-game-action='plant'] select option[value='shell']").first option = page.locator("form[data-game-action='plant'] select option[value='shell']").first
option.wait_for(state="attached") option.wait_for(state="attached")
deadline = datetime.now(timezone.utc) + timedelta(seconds=40) deadline = datetime.now(timezone.utc) + timedelta(seconds=10)
while "saturated" not in option.inner_text() and datetime.now(timezone.utc) < deadline: while "saturated" not in option.inner_text() and datetime.now(timezone.utc) < deadline:
page.wait_for_timeout(2000) page.wait_for_timeout(500)
page.reload(wait_until="domcontentloaded") page.reload(wait_until="domcontentloaded")
page.locator("[data-game-grid]").first.wait_for(state="visible") page.locator("[data-game-grid]").first.wait_for(state="visible")
option = page.locator( option = page.locator(

View File

@ -45,15 +45,15 @@ def test_leaderboard_ranks_after_upvote(app_server, browser, seeded_db):
pa.wait_for_timeout(1500) pa.wait_for_timeout(1500)
ranked = False ranked = False
for _ in range(15): for _ in range(10):
pa.goto(f"{BASE_URL}/leaderboard", wait_until="domcontentloaded") pa.goto(f"{BASE_URL}/leaderboard", wait_until="domcontentloaded")
body = pa.locator("body").text_content() body = pa.locator("body").text_content()
assert "Internal Server Error" not in body, f"Got 500 on leaderboard: {body[:300]}" assert "Internal Server Error" not in body, f"Got 500 on leaderboard: {body[:300]}"
if pa.is_visible("a.leaderboard-name:has-text('bob_test')"): if pa.is_visible("a.leaderboard-name:has-text('bob_test')"):
ranked = True ranked = True
break break
pa.wait_for_timeout(5000) pa.wait_for_timeout(500)
assert ranked, "bob_test never appeared on the leaderboard within the 60s ranking cache TTL" assert ranked, "bob_test never appeared on the leaderboard"
finally: finally:
ctx_a.close() ctx_a.close()
ctx_b.close() ctx_b.close()

View File

@ -24,8 +24,9 @@ def test_isslop_page_loads(page, app_server):
def test_isslop_tools_menu_links_to_page(page, app_server): def test_isslop_tools_menu_links_to_page(page, app_server):
page.goto(f"{BASE_URL}/tools", wait_until="domcontentloaded") page.goto(f"{BASE_URL}/tools", wait_until="domcontentloaded")
card = page.locator("a[href='/tools/isslop']").first card = page.locator(".tools-grid a.tool-card[href='/tools/isslop']")
card.wait_for(state="visible") card.wait_for(state="visible")
assert "AI Usage Analyzer" in card.inner_text()
def test_isslop_submit_opens_live_report(alice): def test_isslop_submit_opens_live_report(alice):

View File

@ -14,7 +14,7 @@ def _seed(collection):
Chunk(uid="c1", text="Silicon wafers are used to make chips.", url="https://b.example", title="B"), Chunk(uid="c1", text="Silicon wafers are used to make chips.", url="https://b.example", title="B"),
] ]
vectors = local_embed([c.text for c in chunks]).vectors vectors = local_embed([c.text for c in chunks]).vectors
store.add(chunks, vectors) run_async(store.add(chunks, vectors))
def test_answer_is_grounded_and_cited(monkeypatch): def test_answer_is_grounded_and_cited(monkeypatch):

View File

@ -2,6 +2,7 @@
from devplacepy.services.deepsearch.embeddings import local_embed from devplacepy.services.deepsearch.embeddings import local_embed
from devplacepy.services.deepsearch.store import Chunk, VectorStore from devplacepy.services.deepsearch.store import Chunk, VectorStore
from tests.conftest import run_async
def _chunks(): def _chunks():
@ -22,8 +23,8 @@ def test_add_and_count():
try: try:
chunks = _chunks() chunks = _chunks()
vectors = local_embed([c.text for c in chunks]).vectors vectors = local_embed([c.text for c in chunks]).vectors
store.add(chunks, vectors) run_async(store.add(chunks, vectors))
assert store.count() == 3 assert run_async(store.count()) == 3
finally: finally:
store.drop() store.drop()
@ -33,10 +34,10 @@ def test_hybrid_search_returns_relevant_chunk():
try: try:
chunks = _chunks() chunks = _chunks()
vectors = local_embed([c.text for c in chunks]).vectors vectors = local_embed([c.text for c in chunks]).vectors
store.add(chunks, vectors) run_async(store.add(chunks, vectors))
query = "where was the transistor invented" query = "where was the transistor invented"
query_vector = local_embed([query]).vectors[0] query_vector = local_embed([query]).vectors[0]
results = store.hybrid_search(query, query_vector, top_k=2) results = run_async(store.hybrid_search(query, query_vector, top_k=2))
assert results assert results
assert any("transistor" in r.text.lower() for r in results) assert any("transistor" in r.text.lower() for r in results)
finally: finally:
@ -57,7 +58,7 @@ def test_dims_reflects_embedding_width():
try: try:
chunks = _chunks() chunks = _chunks()
vectors = local_embed([c.text for c in chunks]).vectors vectors = local_embed([c.text for c in chunks]).vectors
store.add(chunks, vectors) run_async(store.add(chunks, vectors))
assert store.dims == len(vectors[0]) assert store.dims == len(vectors[0])
finally: finally:
store.drop() store.drop()
@ -66,7 +67,7 @@ def test_dims_reflects_embedding_width():
def test_coverage_analytics_empty_collection(): def test_coverage_analytics_empty_collection():
store = VectorStore("ds_store_test_cov_empty") store = VectorStore("ds_store_test_cov_empty")
try: try:
analytics = store.coverage_analytics() analytics = run_async(store.coverage_analytics())
assert analytics == { assert analytics == {
"chunks": 0, "chunks": 0,
"domains": 0, "domains": 0,
@ -84,8 +85,8 @@ def test_coverage_analytics_counts_chunks_and_sources():
for index, chunk in enumerate(chunks): for index, chunk in enumerate(chunks):
chunk.source = "httpx" if index == 0 else "playwright" chunk.source = "httpx" if index == 0 else "playwright"
vectors = local_embed([c.text for c in chunks]).vectors vectors = local_embed([c.text for c in chunks]).vectors
store.add(chunks, vectors) run_async(store.add(chunks, vectors))
analytics = store.coverage_analytics() analytics = run_async(store.coverage_analytics())
assert analytics["chunks"] == 3 assert analytics["chunks"] == 3
assert analytics["sources"] == 2 assert analytics["sources"] == 2
assert analytics["avg_chunk_chars"] > 0 assert analytics["avg_chunk_chars"] > 0

View File

@ -169,9 +169,10 @@ def test_harvest_success_awards_coins_and_xp(local_db):
before = _coins(user) before = _coins(user)
result = store.harvest(user, 0) result = store.harvest(user, 0)
crop = economy.crop_for("shell") crop = economy.crop_for("shell")
assert result["coins"] == crop.reward_coins expected = economy.realizable_harvest_coins(crop, golden=result["golden"])
assert result["coins"] == expected
assert result["xp"] == crop.reward_xp assert result["xp"] == crop.reward_xp
assert _coins(user) == before + crop.reward_coins assert _coins(user) == before + expected
assert store.get_farm(user["uid"])["total_harvests"] == 1 assert store.get_farm(user["uid"])["total_harvests"] == 1
@ -778,21 +779,21 @@ def test_harvest_records_market_tick_and_saturates_reward(local_db):
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
crop = economy.crop_for("shell") crop = economy.crop_for("shell")
worst_days = economy.MARKET_SATURATION_TIERS[-1][0]
worst_tier_count = int(worst_days * 86400 / crop.grow_seconds) + 1
get_table("game_market_ticks").insert( get_table("game_market_ticks").insert(
{ {
"uid": generate_uid(), "uid": generate_uid(),
"crop_key": "shell", "crop_key": "shell",
"hour_bucket": market_store._hour_bucket(now), "hour_bucket": market_store._hour_bucket(now),
"harvests": worst_tier_count, "harvests": _worst_tier_harvests("shell"),
"updated_at": now.isoformat(), "updated_at": now.isoformat(),
} }
) )
market_store._saturation_cache.clear()
store.plant(user, 0, "shell") store.plant(user, 0, "shell")
_warp(user) _warp(user)
result = store.harvest(user, 0) result = store.harvest(user, 0)
assert result["coins"] < crop.reward_coins unsaturated = economy.realizable_harvest_coins(crop, golden=result["golden"])
assert result["coins"] < unsaturated
def _seed_market_ticks(crop_key, harvests): def _seed_market_ticks(crop_key, harvests):
@ -813,9 +814,12 @@ def _seed_market_ticks(crop_key, harvests):
def _worst_tier_harvests(crop_key): def _worst_tier_harvests(crop_key):
from devplacepy.services.game.store import market as market_store
crop = economy.crop_for(crop_key) crop = economy.crop_for(crop_key)
worst_days = economy.MARKET_SATURATION_TIERS[-1][0] worst_days = economy.MARKET_SATURATION_TIERS[-1][0]
return int(worst_days * 86400 / crop.grow_seconds) + 1 supply_per_harvest = crop.grow_seconds / 86400.0 / market_store.active_farms()
return int(worst_days / supply_per_harvest) + 1
def test_starter_crop_boosted_when_market_saturated(local_db): def test_starter_crop_boosted_when_market_saturated(local_db):

View File

@ -52,7 +52,7 @@ def test_snippet_page_none_when_too_thin():
def test_crawl_uses_snippet_for_hostile_and_skips_fetch(monkeypatch): def test_crawl_uses_snippet_for_hostile_and_skips_fetch(monkeypatch):
fetched = [] fetched = []
async def fake_fetch(url, depth): async def fake_fetch(url, depth, browser=None):
fetched.append(url) fetched.append(url)
return CrawledPage(url=url, title="T", text="x " * 200, source="httpx", status=200, depth=depth) return CrawledPage(url=url, title="T", text="x " * 200, source="httpx", status=200, depth=depth)
@ -74,7 +74,7 @@ def test_crawl_uses_snippet_for_hostile_and_skips_fetch(monkeypatch):
def test_crawl_prefers_richer_crawl_over_snippet(monkeypatch): def test_crawl_prefers_richer_crawl_over_snippet(monkeypatch):
async def fake_fetch(url, depth): async def fake_fetch(url, depth, browser=None):
return CrawledPage(url=url, title="Article", text="Deep article body. " * 60, source="httpx", status=200, depth=depth) return CrawledPage(url=url, title="Article", text="Deep article body. " * 60, source="httpx", status=200, depth=depth)
monkeypatch.setattr(crawl_module, "fetch_page", fake_fetch) monkeypatch.setattr(crawl_module, "fetch_page", fake_fetch)
@ -93,7 +93,7 @@ def test_crawl_prefers_richer_crawl_over_snippet(monkeypatch):
def test_crawl_falls_back_to_snippet_when_fetch_empty(monkeypatch): def test_crawl_falls_back_to_snippet_when_fetch_empty(monkeypatch):
async def fake_fetch(url, depth): async def fake_fetch(url, depth, browser=None):
return None return None
monkeypatch.setattr(crawl_module, "fetch_page", fake_fetch) monkeypatch.setattr(crawl_module, "fetch_page", fake_fetch)

View File

@ -61,7 +61,18 @@ def test_report_payload_grade_matches_final_scores():
verdicts = [_verdict("a.py", 90.0)] verdicts = [_verdict("a.py", 90.0)]
adjusted = apply_ai_verdicts(scores, verdicts) adjusted = apply_ai_verdicts(scores, verdicts)
final = aggregate(adjusted) final = aggregate(adjusted)
payload = _summary_payload("https://x.example/repo", "git", final, adjusted, verdicts, 0, [], {}, TemplateEvidence()) payload = _summary_payload(
"https://x.example/repo",
"git",
final,
adjusted,
verdicts,
0,
[],
{},
TemplateEvidence(),
None,
)
assert payload["repo_scores"]["grade"] == final.grade assert payload["repo_scores"]["grade"] == final.grade
assert payload["repo_scores"]["human_percent"] == final.human_percent assert payload["repo_scores"]["human_percent"] == final.human_percent
markdown = fallback_report(payload) markdown = fallback_report(payload)

View File

@ -192,19 +192,19 @@ def test_build_gist_detail_shape(local_db):
@pytest.mark.parametrize( @pytest.mark.parametrize(
"fn_name,expected_keys", "fn_name,args,expected_keys",
[ [
("build_messages_page", {"threads", "unread", "partners"}), ("build_messages_page", ("u1",), {"threads", "unread", "partners"}),
("build_notifications_page", {"notifications", "actors", "unread_count"}), ("build_notifications_page", ("u1",), {"notifications", "actors", "unread_count"}),
("build_game_state", {"store", "leaderboard"}), ("build_game_state", (), {"store", "leaderboard"}),
("build_admin_dashboard", {"stats", "service_states"}), ("build_admin_dashboard", (), {"stats", "service_states"}),
], ],
) )
def test_stub_compounds_documented(local_db, fn_name, expected_keys): def test_stub_compounds_documented(local_db, fn_name, args, expected_keys):
import devplacepy_services.database.compounds_page as compounds_page import devplacepy_services.database.compounds_page as compounds_page
fn = getattr(compounds_page, fn_name) fn = getattr(compounds_page, fn_name)
result = fn() if fn_name != "build_admin_dashboard" else fn() result = fn(*args)
assert set(result.keys()) == expected_keys, ( assert set(result.keys()) == expected_keys, (
f"{fn_name} shape changed - if this is a real implementation now, " f"{fn_name} shape changed - if this is a real implementation now, "
"replace this stub-tracking test with a real assertion and update " "replace this stub-tracking test with a real assertion and update "