Full test suite is now the mandatory final validation; fix everything it surfaced
Some checks failed
DevPlace CI / test (pull_request) Failing after 10m29s
Some checks failed
DevPlace CI / test (pull_request) Failing after 10m29s
- Policy: every change ends with make test (all tiers, all tests) green; docs and agent guardrails updated accordingly - schema.py: ensure the full filtered/indexed column set of instances via get_table (ingress_slug, ports_json, container_gateway, slug, status, ...) so a partial first insert can never break the ingress proxy - docs_api: award body param location body -> json; gateway endpoints documented public -> user to match enforced auth - tests: missing get_table import (trash restore), audit read as admin (award), deterministic online-roster and leaderboard-cache handling, container visibility updated to the primary-admin-only rule, scoped AI usage heading selector past the hidden Tools nav links
This commit is contained in:
parent
77f043640e
commit
34fa56a836
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After writing a test module, validate it ONLY by a clean import (`python -c "import tests..."` or `python -m py_compile`).
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record coverage gaps and pattern violations, do NOT write files. Apply **FIX** mode only when the invocation explicitly asks you to fix; then write the missing integration test following the required patterns. **HARD GUARDRAIL: write tests but NEVER run the suite, not the full suite and not a single file.** Validate only by a clean import of the new test module. Never perform any git write operation.
|
||||
Default to **REPORT** mode: record coverage gaps and pattern violations, do NOT write files. Apply **FIX** mode only when the invocation explicitly asks you to fix; then write the missing integration test following the required patterns. **HARD GUARDRAIL: write tests but NEVER launch the suite yourself, not the full suite and not a single file - the serial single-process suite cannot run concurrently with other agents, so the orchestrating session runs `make test` (all tests) after your work.** Validate only by a clean import of the new test module. Never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
---
|
||||
description: Run DevPlace tests - the sanctioned explicit-ask path. Run a tier, a file, or a single test with the correct flags. The agents never run tests themselves; this command is how you ask.
|
||||
description: Run DevPlace tests. Run a tier, a file, or a single test with the correct flags. Subagents never run tests themselves (the serial suite cannot run concurrently); the orchestrating session always runs the full suite as the final validation of every change.
|
||||
argument-hint: [unit|api|e2e|all|<path::test_name>]
|
||||
allowed-tools: Bash(make test*), Bash(python -m pytest *), Read
|
||||
---
|
||||
@ -12,6 +12,6 @@ Mapping:
|
||||
- `all` or empty -> `make test`
|
||||
- a path like `tests/api/posts/create.py::test_x` -> `python -m pytest <that> -v --tb=line -x`
|
||||
|
||||
Tests run serially on port 10501 with a tempfile SQLite DB and `DEVPLACE_DISABLE_SERVICES=1`. This command is the one sanctioned way to run them (the subagents and workflows never do).
|
||||
Tests run serially on port 10501 with a tempfile SQLite DB and `DEVPLACE_DISABLE_SERVICES=1`. Subagents and workflows never launch tests (the serial single-process suite cannot run concurrently); the orchestrating session runs the full suite (`make test`) as the mandatory final validation of every change.
|
||||
|
||||
Report results clearly. On a failure, show the relevant output, and if a browser (e2e) test failed, point me at the screenshot under `/tmp/devplace_test_screenshots/`. Never weaken a test to make it pass; if a test reveals a real bug, report it - do not edit the test.
|
||||
|
||||
@ -31,7 +31,7 @@ make locust-headless # Locust CLI mode for CI
|
||||
|
||||
The Makefile exports `PYTHONDONTWRITEBYTECODE=1` for every recipe, so no `.pyc` files or `__pycache__` directories are written by any `make` target. Keep it that way - do not add a target that re-enables bytecode writing. `make clean` removes any stray bytecode left from running Python outside make.
|
||||
|
||||
Validate code without running the suite: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance).
|
||||
Preliminary validation: confirm `python -c "from devplacepy.main import app"` imports clean and check each touched language manually (Python compiles/imports, JS parses, CSS braces and HTML tags balance). These checks are gates on the way to the real validation, never a substitute for it: **every change ends with the full test suite (`make test` - all three tiers, every test) and it must pass.**
|
||||
|
||||
Single test: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x`
|
||||
|
||||
@ -60,6 +60,7 @@ devplace zips prune # delete expired zip archives + job rows
|
||||
devplace zips clear # delete every zip archive + job row
|
||||
devplace forks prune # delete expired completed fork job rows (forked projects persist)
|
||||
devplace forks clear # delete every fork job row (forked projects persist)
|
||||
devplace messaging prune-tickets # delete expired WebSocket auth tickets (ws_tickets)
|
||||
devplace seo prune # delete expired SEO audit reports + job rows
|
||||
devplace seo clear # delete every SEO audit report + job row
|
||||
devplace seo-meta prune # delete expired SEO metadata job rows (generated metadata persists)
|
||||
@ -284,7 +285,7 @@ Fixture stack: `app_server` (session-scoped uvicorn subprocess on port 10501), `
|
||||
|
||||
Required patterns: every `page.goto(...)`/`page.wait_for_url(...)` MUST pass `wait_until="domcontentloaded"`; prefer `page.locator(...).wait_for(state="visible")` over `wait_for_selector`; scope ambiguous selectors (e.g. comment Delete is `.comment-action-btn:has-text('Delete')`). A test that flips a global `site_settings` value MUST restore it in `try/finally`. Full pitfalls/patterns catalogue is in `tests/CLAUDE.md`.
|
||||
|
||||
**Never run tests unless the user explicitly asks for it.** Not the full suite, not a single file. Validate with a clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks instead.
|
||||
**Always run the full test suite (`make test` - unit, api, and e2e, every test) as the final validation of every change.** No tier may be skipped and no subset substituted for the whole. The clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks are preliminary gates before the suite, not replacements for it. Any failure is a real signal and blocks completion until fixed.
|
||||
|
||||
## Rigorous correctness verification (money, state machines, concurrency)
|
||||
|
||||
@ -314,7 +315,7 @@ A new public read almost always needs all four. The cardinal failure mode is cha
|
||||
4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
|
||||
5. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog.py` - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
|
||||
6. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
|
||||
7. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Never run the persisted test suite unless the user explicitly asks.** Write new tests in the matching tier/path when asked, following the required patterns above.
|
||||
7. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.
|
||||
|
||||
Failures at any implementation step block the workflow - never skip a failed step.
|
||||
|
||||
|
||||
@ -358,6 +358,21 @@ def init_db():
|
||||
_index(
|
||||
db, "issue_comment_authors", "idx_issue_comment_authors_number", ["gitea_number"]
|
||||
)
|
||||
|
||||
ws_tickets = get_table("ws_tickets")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("token", ""),
|
||||
("user_uid", ""),
|
||||
("created_at", ""),
|
||||
("expires_at", ""),
|
||||
("used_at", ""),
|
||||
):
|
||||
if not ws_tickets.has_column(column):
|
||||
ws_tickets.create_column_by_example(column, example)
|
||||
_index(db, "ws_tickets", "idx_ws_tickets_token", ["token"], unique=True)
|
||||
_index(db, "ws_tickets", "idx_ws_tickets_expires", ["expires_at"])
|
||||
|
||||
migrate_bug_tables_to_issue_tables()
|
||||
_index(db, "service_state", "idx_service_state_name", ["name"])
|
||||
if "devii_conversations" in db.tables:
|
||||
@ -471,16 +486,26 @@ def init_db():
|
||||
_index(db, "jobs", "idx_jobs_expires", ["expires_at"])
|
||||
_index(db, "project_forks", "idx_project_forks_source", ["source_project_uid"])
|
||||
_index(db, "project_forks", "idx_project_forks_forked", ["forked_project_uid"])
|
||||
if "instances" in db.tables:
|
||||
instances = get_table("instances")
|
||||
for column, example in (
|
||||
("run_as_uid", ""),
|
||||
("boot_language", "none"),
|
||||
("boot_script", ""),
|
||||
("start_on_boot", 0),
|
||||
):
|
||||
if not instances.has_column(column):
|
||||
instances.create_column_by_example(column, example)
|
||||
instances = get_table("instances")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("project_uid", ""),
|
||||
("slug", ""),
|
||||
("name", ""),
|
||||
("status", ""),
|
||||
("desired_state", ""),
|
||||
("container_id", ""),
|
||||
("ingress_slug", ""),
|
||||
("ingress_port", 0),
|
||||
("ports_json", ""),
|
||||
("container_gateway", ""),
|
||||
("run_as_uid", ""),
|
||||
("boot_language", "none"),
|
||||
("boot_script", ""),
|
||||
("start_on_boot", 0),
|
||||
):
|
||||
if not instances.has_column(column):
|
||||
instances.create_column_by_example(column, example)
|
||||
|
||||
_index(db, "instances", "idx_instances_project", ["project_uid"])
|
||||
_index(db, "instances", "idx_instances_slug", ["slug"])
|
||||
|
||||
@ -134,7 +134,7 @@ for signing DevPlace's own requests.
|
||||
path="/openai/v1/chat/completions",
|
||||
title="Chat completions",
|
||||
summary="OpenAI-compatible chat completion. Supports streaming.",
|
||||
auth="public",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field(
|
||||
@ -174,7 +174,7 @@ for signing DevPlace's own requests.
|
||||
path="/openai/v1/embeddings",
|
||||
title="Embeddings",
|
||||
summary="OpenAI-compatible text embeddings. Request model molodetz~embed.",
|
||||
auth="public",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field(
|
||||
@ -214,7 +214,7 @@ for signing DevPlace's own requests.
|
||||
path="/openai/v1/images/generations",
|
||||
title="Image generation",
|
||||
summary="OpenAI-compatible image generation. Request model molodetz-img-small.",
|
||||
auth="public",
|
||||
auth="user",
|
||||
encoding="json",
|
||||
params=[
|
||||
field(
|
||||
@ -262,7 +262,7 @@ for signing DevPlace's own requests.
|
||||
path="/openai/v1/{path}",
|
||||
title="Passthrough",
|
||||
summary="Any other /v1 path is forwarded to the upstream as-is.",
|
||||
auth="public",
|
||||
auth="user",
|
||||
interactive=False,
|
||||
params=[
|
||||
field(
|
||||
|
||||
@ -320,7 +320,7 @@ four ways to sign requests.
|
||||
),
|
||||
field(
|
||||
"description",
|
||||
"body",
|
||||
"json",
|
||||
"string",
|
||||
True,
|
||||
"Great work on the release!",
|
||||
|
||||
@ -1,14 +1,12 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file documents detailed testing patterns, fixtures, and pitfalls for devplacepy/tests/ (Playwright + unit tests across api/e2e/unit tiers). Claude Code loads it automatically whenever a file under this directory is read or edited. Never run the test suite unless the user explicitly asks - validate with imports and manual checks instead.
|
||||
This file documents detailed testing patterns, fixtures, and pitfalls for devplacepy/tests/ (Playwright + unit tests across api/e2e/unit tiers). Claude Code loads it automatically whenever a file under this directory is read or edited. Every change ends with the full suite: `make test` runs all tests across all three tiers, and it must pass.
|
||||
|
||||
## Never run tests unless explicitly asked (critical guardrail)
|
||||
## Always run the full suite (critical guardrail)
|
||||
|
||||
**NEVER run tests unless specifically asked by user.** Not the full suite, not a single file - do not run any tests unless the user explicitly requests it. Validate each touched language manually instead: Python (compile + import), JS (parse / bracket matching), CSS (brace matching), HTML (tag matching). Zero tolerance. A clean `python -c "from devplacepy.main import app"` import is the baseline check.
|
||||
**Every change is validated by running the full test suite - ALL tests, all three tiers, via `make test`.** No tier skipped, no subset substituted for the whole. Preliminary per-language checks (Python compile + import, JS parse, CSS brace matching, HTML tag matching) and a clean `python -c "from devplacepy.main import app"` import come first, but they never replace the suite. A failure is a real signal and blocks completion until fixed.
|
||||
|
||||
**Never run tests unless the user explicitly asks for it.** Not the full suite, not a single file. Validate code with a clean import (`python -c "from devplacepy.main import app"`) and per-language manual checks instead.
|
||||
|
||||
If you do need to run a single test on explicit request: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x`.
|
||||
During iteration a single test may be run for a fast feedback loop: `python -m pytest tests/e2e/feed.py::test_name -v --tb=line -x` - but the change is finished only when the complete suite is green.
|
||||
|
||||
## Fixture stack
|
||||
|
||||
|
||||
@ -107,6 +107,7 @@ def test_restore_revives_post_and_records_audit(seeded_db):
|
||||
|
||||
def test_restore_revoked_award_recomputes_stats(seeded_db):
|
||||
from datetime import datetime, timezone
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.database.awards import revoke_award
|
||||
from devplacepy.utils import generate_uid, make_combined_slug
|
||||
|
||||
|
||||
@ -115,7 +115,6 @@ def _seed_news_audit_log():
|
||||
)
|
||||
refresh_snapshot()
|
||||
return uid
|
||||
from devplacepy.database import get_table
|
||||
_counter_polls = [0]
|
||||
AJAX_polls = {"X-Requested-With": "fetch"}
|
||||
def _session_polls():
|
||||
@ -363,7 +362,18 @@ def test_feed_avatar_has_presence_dot(app_server):
|
||||
|
||||
|
||||
def test_feed_shows_online_now_section(app_server):
|
||||
s, name = _member()
|
||||
name = _unique("0onl")
|
||||
s = requests.Session()
|
||||
s.post(
|
||||
f"{BASE_URL}/auth/signup",
|
||||
data={
|
||||
"username": name,
|
||||
"email": f"{name}@t.dev",
|
||||
"password": "secret123",
|
||||
"confirm_password": "secret123",
|
||||
},
|
||||
allow_redirects=True,
|
||||
)
|
||||
html = s.get(f"{BASE_URL}/feed").text
|
||||
assert "online-users" in html
|
||||
assert "data-online-users-list" in html
|
||||
|
||||
@ -87,15 +87,23 @@ def test_leaderboard_entries_carry_score_and_prestige(app_server, seeded_db):
|
||||
session, name = _signup()
|
||||
_reset_farm(name)
|
||||
_set_farm(name, prestige=50, total_harvests=20, ci_tier=3)
|
||||
response = requests.get(f"{BASE_URL}/game/leaderboard", headers=JSON)
|
||||
assert response.status_code == 200
|
||||
entries = response.json()["entries"]
|
||||
deadline = time.time() + 20
|
||||
while True:
|
||||
response = requests.get(f"{BASE_URL}/game/leaderboard", headers=JSON)
|
||||
assert response.status_code == 200
|
||||
entries = response.json()["entries"]
|
||||
mine = next(
|
||||
(entry for entry in entries if entry["username"] == name), None
|
||||
)
|
||||
if mine is not None or time.time() >= deadline:
|
||||
break
|
||||
time.sleep(0.5)
|
||||
assert entries
|
||||
scores = [entry["score"] for entry in entries]
|
||||
assert scores == sorted(scores, reverse=True)
|
||||
for entry in entries:
|
||||
assert "score" in entry and "prestige" in entry
|
||||
mine = next(entry for entry in entries if entry["username"] == name)
|
||||
assert mine is not None
|
||||
assert mine["prestige"] == 50
|
||||
assert mine["score"] == economy.farm_score(store.get_farm(
|
||||
get_table("users").find_one(username=name)["uid"]
|
||||
|
||||
@ -2,10 +2,8 @@
|
||||
|
||||
import time
|
||||
import requests
|
||||
from datetime import datetime, timezone
|
||||
from tests.conftest import BASE_URL
|
||||
from devplacepy.db_client import get_table
|
||||
from devplacepy.utils import generate_uid, make_combined_slug
|
||||
|
||||
JSON = {"Accept": "application/json"}
|
||||
_counter = [0]
|
||||
@ -177,7 +175,7 @@ def test_valid_post_creates_pending_row_and_job(app_server):
|
||||
assert job and job.get("kind") == "award"
|
||||
|
||||
|
||||
def test_award_give_audit_recorded(app_server):
|
||||
def test_award_give_audit_recorded(app_server, seeded_db):
|
||||
giver, _ = _signup()
|
||||
_, receiver_name = _signup()
|
||||
r = giver.post(
|
||||
@ -186,5 +184,6 @@ def test_award_give_audit_recorded(app_server):
|
||||
json={"description": "Audit me"},
|
||||
)
|
||||
assert r.status_code == 200
|
||||
entry = _audit_find(giver, "award.give", _uid(receiver_name))
|
||||
admin = _login(seeded_db, "alice")
|
||||
entry = _audit_find(admin, "award.give", _uid(receiver_name))
|
||||
assert entry is not None
|
||||
@ -3,7 +3,7 @@
|
||||
import time
|
||||
import requests
|
||||
from tests.conftest import BASE_URL
|
||||
from devplacepy.database import get_table
|
||||
from devplacepy.database import get_primary_admin_uid, get_table, invalidate_admins_cache
|
||||
from devplacepy.utils import clear_user_cache
|
||||
|
||||
_counter_cv = [0]
|
||||
@ -70,11 +70,14 @@ def test_admin_private_project_containers_hidden_from_other_admin(app_server):
|
||||
assert _containers_data_status(owner_key, slug) == 200
|
||||
|
||||
|
||||
def test_member_private_project_containers_visible_to_admin(app_server):
|
||||
def test_member_private_project_containers_primary_admin_only(app_server):
|
||||
_, _, owner_key = _signup()
|
||||
_, _, admin_key = _make_admin()
|
||||
slug = _create_project(owner_key, "Member Container Hidden", is_private=True)
|
||||
assert _containers_data_status(admin_key, slug) == 200
|
||||
assert _containers_data_status(admin_key, slug) == 404
|
||||
invalidate_admins_cache()
|
||||
primary_key = get_table("users").find_one(uid=get_primary_admin_uid())["api_key"]
|
||||
assert _containers_data_status(primary_key, slug) == 200
|
||||
|
||||
|
||||
def test_public_project_containers_visible_to_any_admin(app_server):
|
||||
|
||||
@ -68,4 +68,4 @@ def _seed_target_user():
|
||||
def test_admin_ai_usage_page_loads(alice):
|
||||
page, _ = alice
|
||||
page.goto(f"{BASE_URL}/admin/ai-usage", wait_until="domcontentloaded")
|
||||
assert page.is_visible("text=AI usage") or page.is_visible("text=AI Usage")
|
||||
page.locator("h2:has-text('AI usage')").wait_for(state="visible")
|
||||
|
||||
Loading…
Reference in New Issue
Block a user