CLAUDE.md

This file documents detailed testing patterns, fixtures, and pitfalls for devplacepy/tests/ (Playwright + unit tests across api/e2e/unit tiers). Claude Code loads it automatically whenever a file under this directory is read or edited. Every change ends with the full suite: make test runs all tests across all three tiers, and it must pass.

Always run the full suite (critical guardrail)

Every change is validated by running the full test suite - ALL tests, all three tiers, via make test. No tier skipped, no subset substituted for the whole. Preliminary per-language checks (Python compile + import, JS parse, CSS brace matching, HTML tag matching) and a clean python -c "from devplacepy.main import app" import come first, but they never replace the suite. A failure is a real signal and blocks completion until fixed.

During iteration a single test may be run for a fast feedback loop: python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x - but the change is finished only when the complete suite is green.

Finding failures fast

The suite does NOT stop at the first failure. -rf lives in pyproject.toml addopts (one source of truth, so a bare python -m pytest ... gets it too), and every run prints a FAILED <nodeid> line per failure at the end - the complete failure list from a single pass instead of one full pass per bug. Triage cheapest tier first:

Command Covers Cost
make test-fast unit + api, no browser ~3 min
make test-unit unit only ~50 s
make test-api api only ~2 min
make test-e2e Playwright browser tier the expensive one
make test all three tiers full
make test-failed only what failed last run (--last-failed) proportional to the failure count
make test-first-failure all three tiers with -x stops at failure 1
make test-slowest all three tiers + the 40 slowest tests full
make test-cache-clean drops .pytest_cache/ when --last-failed misbehaves instant

Server-side display caches are disabled for the suite, never waited out. A test that changes state the server serves from a TTL cache must not poll until the TTL expires - that burns the TTL in wall-clock on every run (the leaderboard ranking test cost 60s and the Code Farm saturation test 30s exactly this way). Each such cache reads its TTL from an env var with the production value as the default, and conftest.py sets it to 0 before the server subprocess starts: DEVPLACE_SITEMAP_TTL (seo.SITEMAP_TTL), DEVPLACE_HOME_CACHE_TTL (main._home_cache), DEVPLACE_RANKING_TTL (database/ranking.py _authors_cache), DEVPLACE_MARKET_SATURATION_TTL (services/game/store/market.py _saturation_cache). Add a new display cache to that list rather than sleeping in the test. Correctness caches are different: the cross-worker cache_state version read (_cache_version_cache, 1s) is load-bearing and stays on, so a test that flips the admin set from the test process must wait for the server to agree before asserting (see tests/e2e/admin/containers/manage.py _await_instance_visible), never assume the change is instant.

The loop: make test-fast -> fix -> make test-failed until it is empty -> make test as the final gate. make test-first-failure is only for the rare case where one early failure poisons every test after it (a corrupted session fixture, a stopped service). Pytest's cache lives in .pytest_cache/ (git-ignored), so --last-failed works with no extra setup; do not pass -p no:cacheprovider in a run whose failures you intend to re-run. If make test-failed runs the whole suite instead of just the failures, the cache is polluted: pytest keeps lastfailed entries for tests it did not collect, so node ids from deleted files (or from a stray pytest invocation that walked data//var/) linger forever and make --last-failed give up and select everything. make test-cache-clean drops .pytest_cache/ and fixes it; make clean does it too.

Fixture stack

  • app_server (session-scoped): spawns uvicorn as a subprocess on port 10501 (the PORT constant) with a tempfile DB.
  • browser_context (session-scoped): one Playwright context shared by all tests.
  • page (function-scoped): clear_cookies() on the session context, then a fresh page.
  • alice / bob: seeded users alice_test / bob_test logged in via the login form. bob gets its own context for multi-user tests. Returns (page, user_dict).

Tests run on port 10501 with a tempfile SQLite DB and a dedicated DEVPLACE_DATA_DIR, and DEVPLACE_DISABLE_SERVICES=1 so background services don't start. make test runs serially, one test at a time, in a single process: one isolated DB + data dir + uvicorn subprocess + Chromium, shared across the whole session, so session fixtures (seeded_db) and intra-file ordering hold. Serial execution is enforced centrally in pyproject.toml ([tool.pytest.ini_options] addopts = "--tb=line -rf -p no:xdist"); pytest-xdist is no longer a dependency and -n is rejected, so the suite can never run concurrently.

General

  • Around 2882 tests across tests/{unit,api,e2e}/. Playwright integration + HTTP + unit tests. All must pass before any merge.
  • The suite runs to completion and reports every failure (-rf), it does not stop at the first one. See "Finding failures fast" above.
  • Validate each touched language manually: Python (compile + import), JS (parse / bracket matching), CSS (brace matching), HTML (tag matching). Zero tolerance.
  • Playwright is used directly, NOT pytest-playwright (uninstall it if present - it conflicts).

Playwright navigation

  • Every page.goto() must use wait_until="domcontentloaded", never the default "load". CDN scripts and avatar images cause load to timeout.
  • Every page.wait_for_url() must also use wait_until="domcontentloaded" for the same reason.
  • Prefer page.locator(...).wait_for(state="visible") over bare wait_for_selector - it gives better error messages.
  • Default timeout is 15 seconds (increased from 10s for CDN script loading).
page.goto(f"{BASE_URL}/feed", wait_until="domcontentloaded")
page.locator(".feed-fab").first.wait_for(state="visible", timeout=10000)

Delete button locator scoping

When both a post Delete and comment Delete button exist, always scope to the comment:

# CORRECT - scoped to comment:
page.locator(".comment-action-btn:has-text('Delete')")

# WRONG - matches both post and comment Delete:
page.locator("button:has-text('Delete')")

Confirm dialogs

Delete buttons carry data-confirm, which ModalManager turns into a native confirm() dialog. Playwright auto-dismisses dialogs (treated as Cancel), so the form never submits. Accept the dialog before clicking, or the delete silently does nothing:

page.once("dialog", lambda d: d.accept())
page.locator(".comment-action-btn:has-text('Delete')").click()

Comment hash navigation

Opening a URL with a #comment-<uid> fragment scrolls the comment into view and adds the comment-highlight class (NotificationManager.initHashScroll). Tests assert on #comment-<uid>.comment-highlight after the page loads.

Browser context

  • browser_context is session-scoped (one per test session, shared by all tests in all files).
  • Cookies are cleared per test via browser_context.clear_cookies() in the page fixture.
  • Each test gets a fresh page from the shared context.
  • bob fixture creates its own context from the session browser - necessary for multi-user tests.
  • Never share a page between two logged-in users in the same test - use separate contexts.

Test users

  • alice_test / bob_test are seeded once at session level via HTTP POST to /auth/signup.
  • alice fixture logs in alice_test via the login form.
  • bob fixture logs in bob_test in a separate Playwright context.
  • Use alice for single-user tests. It returns (page, user_dict).

Required test patterns (summary)

  • Every page.goto(...) and page.wait_for_url(...) MUST pass wait_until="domcontentloaded". CDN libs and avatars cause the default "load" to time out.
  • Prefer page.locator(...).wait_for(state="visible") over wait_for_selector.
  • Scope ambiguous selectors: comment Delete is .comment-action-btn:has-text('Delete'); create-post Post button is #create-post-modal button.btn-primary:has-text('Post'); comment textarea is .comment-form textarea[name='content'].
  • Failure screenshots auto-save to /tmp/devplace_test_screenshots/ via the pytest_runtest_makereport hook in conftest.py.

Global site_settings tests

  • The uvicorn server is shared for the whole session. A test that flips a global setting (maintenance mode, registration, rate limit, session length) changes behavior for every later test, so restore the value in a try/finally - see tests/e2e/operational.py.
  • Saving through the admin form is the reliable mutation path, not a direct DB write: the form handler runs in the server process and calls clear_settings_cache(), so the change takes effect immediately (a direct DB write waits out the 60s server-side cache).
  • conftest.py's app_server seeds rate_limit_per_minute="1000000" so the suite is never throttled and the settings form round-trips a safe value instead of the template's "60" default.
  • Rate-limit unit tests patch m.get_int_setting, not the RATE_LIMIT constant - the constant is now only the fallback default passed to get_int_setting.
  • Service enabled/disabled state is global state too - restore it. Stopping a service through POST /admin/services/{name}/stop sets its service_{name}_enabled setting to "0" for the rest of the session. A test that stops a service (e.g. test_gateway_disabled_returns_503 stops openai to assert the gateway returns 503) MUST restart it in a try/finally (POST /admin/services/{name}/start), and any finally that ends in stop should end in start instead - the default, expected state is enabled. Leaving the gateway stopped makes a later file's request hit is_enabled() first and get a 503 where it expected a 401/200. The openai gateway's handle() checks is_enabled() (503) BEFORE authorize() (401), so a stopped gateway masks the auth result entirely.
  • The 1,000,000/min rate-limit seed only applies after the server's 60s settings cache expires. conftest.py seeds rate_limit_per_minute with a raw DB insert (no clear_settings_cache()), so for the first ~60s of server uptime the server still uses the 60/min default. A high-volume request loop (e.g. test_auth_matrix) run in the FIRST minute can therefore trip a spurious 429; in the full suite that test runs well past the window so it never does. If you reproduce a 429 only in a tiny isolated run, it is this warm-up artifact, not a real auth/rate-limit issue.
  • Operational tests live in tests/e2e/operational.py plus the tests/api/admin/settings.py family; rate-limit tests sit with the endpoint they probe (tests/api/root.py, tests/api/auth/login.py, tests/api/robotstxt.py).

Failure handling

  • Failure screenshots auto-save to /tmp/devplace_test_screenshots/.
  • Every failure is reported in one pass (-rf in the Makefile); re-run just those with make test-failed.
  • If the server won't start, kill leftover processes: kill -9 $(pgrep -f "uvicorn")

Never assert against a formula's base value when the real call has probabilistic or global inputs

Two whole-suite flakes came from tests that pinned an exact number the production formula does not actually guarantee. Both were invisible under -x because they sit late in the run.

  • Golden crops. economy.is_golden(plot_uid, planted_at) is a deterministic hash, but its inputs are freshly generated per run, so ~5% of harvests pay GOLDEN_MULTIPLIER (5x). assert result["coins"] == crop.reward_coins is therefore a 5%-per-harvest flake. Every harvest-reward assertion must go through economy.realizable_harvest_coins(crop, ..., golden=result["golden"]) - store.harvest returns golden precisely so the test can.
  • Global market state. economy.supply_days divides by market_store.active_farms(), a count over every farm in the database, so a saturation fixture computed as worst_days * 86400 / crop.grow_seconds is only correct when exactly one farm is active. After the api/e2e tiers have created farms it lands two tiers milder and the assertion fails. Scale the seeded harvest count by market_store.active_farms() (see _worst_tier_harvests in tests/unit/services/game/store.py).

The rule generalizes: if the production function reads a random-ish input or a table-wide aggregate, assert against that same function's output for the observed state, never against the bare constant.

Common pitfalls

Pitfall Fix
goto/wait_for_url times out Add wait_until="domcontentloaded"
CDN scripts block page load Use defer on all <script> tags
500 on dataset find() Use dict comparison syntax, not raw SQL
N+1 query slowness Use batch helpers: get_users_by_uids(), get_comment_counts_by_post_uids()
Modal not opening/closing Use style.display, not classList.add/remove
Dual Delete buttons match Scope to .comment-action-btn in tests
Dual Post buttons match (feed inline comment) Scope to #create-post-modal button.btn-primary:has-text('Post') in tests
Edit modal textarea conflicts with comment textarea Scope to .comment-form textarea[name='content'] for comments
Tests fail in sequence Session-scoped context + clear_cookies() per test
Double messages in chat Deduplicate by message UID with seen set
Avatar generation fails Falls back to initial-based SVG - check multiavatar import
Http.getJson returns HTML / JSON.parse throws The helper sends Accept: application/json; a content-negotiated route (respond(..., model=...)) needs it. Don't hand-roll fetch() without that header against a negotiated route
Page renders but data empty / 500 only after a CLI test ran earlier A partial insert created the table with a reduced schema. Ensure the full column set in init_db() (see "Dataset rules -> init_db() MUST create every queried column")
no such column: X 500 in the server log Same root cause: a queried/indexed column is missing because the table was created lazily by a partial insert. Add the column to the init_db() ensure-block
Action button click times out ("element is not visible") The button moved into the .project-actions-overflow menu. Open it first: click .project-actions-more, then .context-menu-item:has-text('<Label>') (see tests/e2e/projects/index.py)
ImportError: cannot import name X from devplacepy.utils in a test The util was renamed (e.g. badge_info -> get_badge). Update the test import to the current name; grep templating.py globals for the new name
Admin page heading assertion fails Admin pages use <h2> inside .admin-toolbar, not <h1> - assert text= or .admin-toolbar h2:has-text(...)
test_auth_matrix reports 422 instead of 401/403 A POST route's form params are undocumented in docs_api.py, so the matrix sends an empty body and FastAPI validation (422) fires before the auth guard. Document every required form param so auth is reached and tested
test_auth_matrix reports 503 on a gateway endpoint A prior test left the openai service stopped. Restore it in finally (see "Global site_settings tests -> Service enabled/disabled state")