Compare commits
26 Commits
typosaurus
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| 72e11db185 | |||
| 37d23e8581 | |||
| b97b5a7854 | |||
| 45ad8e79ed | |||
| 11c0cc66cc | |||
| 6514261730 | |||
| ecb22f2b2d | |||
| 265cb781f9 | |||
| 72e088c160 | |||
| 782bcec5bc | |||
| f3b91ac75b | |||
| 6cac64a3f6 | |||
| 2bdcf6528f | |||
| 7e37122f9f | |||
| 3fca4be72e | |||
| c09e6b328c | |||
| 2921edd7f3 | |||
| c19ff19de2 | |||
| 1a5fc9428a | |||
| c0742994cd | |||
| 91fac7fd67 | |||
| 8e9d3fad98 | |||
| 68c2bbe387 | |||
| 372067bbe4 | |||
| 56becfb3f7 | |||
| 192df12b1d |
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -40,10 +40,10 @@ The already-established **choke points** (the public function is a thin wrapper
|
||||
- **C. Cross-reference before every change.** A funnel is called from many sites; deferring inside it changes ALL of them. Find every caller and confirm none depends on the side-effect's result synchronously. If even one does, do not defer the funnel.
|
||||
- **D. Zero degradation.** A fix must never reduce functionality, weaken a check, or change observable behavior beyond moving WHEN a side-effect runs. Deferral is best-effort and must NEVER raise into the caller. If the only fix would degrade or risk stale reads, record it unfixed with the safe path forward.
|
||||
- **E. Dig deep.** Pursue the root cause; gather more evidence rather than guessing or bailing.
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region, re-check the callers, confirm `python -c "from devplacepy.main import app"` still imports clean, and run `hawk .`.
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region, re-check the callers, confirm `python -c "from devplacepy.main import app"` still imports clean, and re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates).
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, confirm the app imports clean, then run `hawk .` and confirm it passes. **HARD GUARDRAIL: never run the test suite (no `make test`, no `pytest`); never perform any git write operation.** Validate by clean import + hawk + an em-dash scan only.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, confirm the app imports clean, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm they pass. **HARD GUARDRAIL: never run the test suite (no `make test`, no `pytest`); never perform any git write operation.** Validate by clean import + the per-language checks + an em-dash scan only.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); full typing on functions you add; keep `retoor <retoor@molodetz.nl>` as the first line of any source file you create.
|
||||
@ -78,4 +78,4 @@ FIX: move the side-effect into a thin public wrapper that `background.submit(_wo
|
||||
- **wrong-tool**: grep `background.submit(` for any argument that is an `async def`/coroutine function, and any external-client call (gitea/push/AI) deferred via the sync queue.
|
||||
|
||||
## Output
|
||||
Return a markdown report: a one-line summary, then one bullet per finding with severity (`error`/`warning`/`info`), `file:line`, the rule name (e.g. `missing-deferral`, `double-wrap`, `unsafe-deferral`, `bypass-funnel`, `wrong-tool`, `captured-request`, `broken-wiring`), the message, and (in fix mode) whether it was fixed. End with the verification you ran (clean import, `hawk .`, em-dash scan) and its result. Never claim the test suite was run.
|
||||
Return a markdown report: a one-line summary, then one bullet per finding with severity (`error`/`warning`/`info`), `file:line`, the rule name (e.g. `missing-deferral`, `double-wrap`, `unsafe-deferral`, `bypass-funnel`, `wrong-tool`, `captured-request`, `broken-wiring`), the message, and (in fix mode) whether it was fixed. End with the verification you ran (clean import, the per-language checks, em-dash scan) and its result. Never claim the test suite was run.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -56,11 +56,11 @@ When a layer is intentionally absent (an internal route with no public docs, a r
|
||||
No comments or docstrings in source you author; full typing on every signature and variable; `pathlib` over `os`; dataclasses over fixed-key dicts; no magic numbers; no version pinning; no em-dashes anywhere (use a hyphen) in any file you touch. Keep `retoor <retoor@molodetz.nl>` as the first line (correct comment style for the language) of any NEW source file you create - never of the existing files you edit, and never inside a `.md` with YAML frontmatter.
|
||||
|
||||
## Validation (after implementing; never skip)
|
||||
There is NO `hawk` or validator binary in this environment - validate each touched file directly, using the Python interpreter where `import devplacepy` resolves its dependencies (verify that first; the repo `.venv` may be incomplete). Then: confirm `python -c "from devplacepy.main import app"` imports clean; compile or parse every touched language (`python -m py_compile <files>` for Python, `node --check <file>` for JS, brace balance for CSS, tag and `{% %}`/`{{ }}` balance for templates); and grep every touched file for em-dashes - the character AND the entity forms `—`/`—`/`—` - confirming none. For any new `*Out` schema, `model_validate` it against a representative context dict so a key mismatch surfaces now, not at request time. Do NOT run the test suite. Then hand off: name which maintainer dimensions are most relevant to the change (e.g. fanout, security, dry, docs, seo, audit, frontend, style, test) so the fleet can verify it.
|
||||
There is NO validator binary in this environment - validate each touched file directly, using the Python interpreter where `import devplacepy` resolves its dependencies (verify that first; the repo `.venv` may be incomplete). Then: confirm `python -c "from devplacepy.main import app"` imports clean; compile or parse every touched language (`python -m py_compile <files>` for Python, `node --check <file>` for JS, brace balance for CSS, tag and `{% %}`/`{{ }}` balance for templates); and grep every touched file for em-dashes - the character AND the entity forms `—`/`—`/`—` - confirming none. For any new `*Out` schema, `model_validate` it against a representative context dict so a key mismatch surfaces now, not at request time. Do NOT run the test suite. Then hand off: name which maintainer dimensions are most relevant to the change (e.g. fanout, security, dry, docs, seo, audit, frontend, style, test) so the fleet can verify it.
|
||||
|
||||
## Live verification of UI/API changes (mandatory for visual work)
|
||||
A structurally valid template can still render broken - `hawk` and the import check never open a browser. Per CLAUDE.md this project treats live verification as non-negotiable for any layout, styling, component, responsive, or backend change:
|
||||
- Do not assume any verification CLI is installed (`hawk`, `mole`, `falcon`, `hound` are NOT present here); check with `command -v` first and fall back to the steps below or the project's `screenshot`/`serve`/`validate` skills when they exist.
|
||||
A structurally valid template can still render broken - the static checks and the import check never open a browser. Per CLAUDE.md this project treats live verification as non-negotiable for any layout, styling, component, responsive, or backend change:
|
||||
- Do not assume any verification CLI is installed (`mole`, `falcon`, `hound` are NOT present here); check with `command -v` first and fall back to the steps below or the project's `screenshot`/`serve`/`validate` skills when they exist.
|
||||
- When your change touches `templates/` or `static/`, the rendered result MUST be visually verified: start the dev server (`make dev` in the background; confirm it is healthy on `http://localhost:10500`), capture each new/changed route with headless Playwright (`wait_until="domcontentloaded"`), and inspect the screenshot against the intended UI and the surrounding design system (tokens, spacing, responsiveness). Tear down any server you started.
|
||||
- When your change touches `routers/`, verify the endpoints over HTTP against the live server (an api-spec runner if available, otherwise `curl`/`httpx` asserting the status and a body fragment).
|
||||
- The `/feature` workflow performs this live `Verify` phase for you automatically; when you are invoked standalone for UI/API work, perform it yourself before declaring the work complete, or explicitly state it is the caller's responsibility and name the routes to check.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Visual judgement is out of scope for auto-fix and is recorded as a finding. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Visual judgement is out of scope for auto-fix and is recorded as a finding. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -30,7 +30,7 @@ All application code is under `devplacepy/`: `devplacepy/routers/`, `devplacepy/
|
||||
- **F. Verify your own work.** After each edit, re-read the changed region and re-check the consumers.
|
||||
|
||||
## Mode
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then run `hawk .` and confirm it passes. A rename is auto-applied ONLY for a confirmed local/private name that passed the decision algorithm AND only after you grep and update every reference in the same run. Never run the test suite; never perform any git write operation.
|
||||
Default to **REPORT** mode: record findings, do NOT modify files. Apply **FIX** mode only when the invocation explicitly asks you to fix. In FIX mode: reconfirm each finding survives refutation, run the cross-reference impact check, apply a minimal idiomatic root-cause fix, re-read the change, then re-validate every file you touched with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and confirm `python -c "from devplacepy.main import app"` still imports clean. A rename is auto-applied ONLY for a confirmed local/private name that passed the decision algorithm AND only after you grep and update every reference in the same run. Never run the test suite; never perform any git write operation.
|
||||
|
||||
## Obey the rules you enforce
|
||||
No comments or docstrings in source you author; no em-dashes (use a hyphen); keep `retoor <retoor@molodetz.nl>` as the first line of any file you create.
|
||||
|
||||
@ -12,4 +12,4 @@ Follow the audit-log design (`devplacepy/services/audit/`); confirm against the
|
||||
3. Call the recorder on the mutation's success path: `audit.record(request, event_key, ...)` in HTTP or WebSocket handlers, or `audit.record_system(event_key, ...)` in request-less contexts (services, jobs, CLI). On a guard or denial branch pass `result="denied"`; on a failure branch pass `result="failure"`.
|
||||
4. Route through the existing DRY choke point when one applies (`content.py`, the `project_files.py` helpers, `routers/containers.py` `_audit_instance`, the Devii dispatcher `_audit_mechanic`) instead of scattering call sites. The HTTP path and the Devii path for one mutation must stay disjoint (no double counting).
|
||||
5. Recording is best-effort: wrap nothing the caller depends on, and NEVER gate the audited action on the record succeeding.
|
||||
6. Validate with `hawk` on the touched files and `python -c "from devplacepy.main import app"`.
|
||||
6. Validate the touched files with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and `python -c "from devplacepy.main import app"`.
|
||||
|
||||
@ -10,4 +10,4 @@ Follow the docs convention exactly (confirm against `devplacepy/routers/docs/pag
|
||||
1. Create `devplacepy/templates/docs/<slug>.html` as a prose page: one `<div class="docs-content" data-render> ... </div>` containing GitHub-flavored markdown. The page is rendered server-side. Any example component markup INSIDE the data-render block must be HTML-escaped (`<dp-...>`); a live demo, if any, goes in a SEPARATE block OUTSIDE the data-render div with its own `<script type="module">`.
|
||||
2. Register it in `DOCS_PAGES` in `devplacepy/routers/docs/pages.py`: `{"slug": "<slug>", "title": "<title>", "kind": "prose", "section": SECTION_*}`. Add `"admin": True` for an admin-only page. If a new section is needed, add a `SECTION_*` constant and place it in the correct `AUDIENCES` group.
|
||||
3. Write accurate, professional content - confirm every factual claim against the source. No em-dashes, no AI disclaimers, dates as DD/MM/YYYY.
|
||||
4. Validate: run `hawk` on the new template and on `pages.py`, run `python -c "from devplacepy.main import app"`, and confirm the slug is registered with no duplicate.
|
||||
4. Validate: check the new template for tag and `{% %}` balance and `pages.py` with `python -m py_compile` + `pyflakes`, run `python -c "from devplacepy.main import app"`, and confirm the slug is registered with no duplicate.
|
||||
|
||||
@ -33,7 +33,7 @@ Arguments: `$ARGUMENTS`
|
||||
## Execute
|
||||
1. Resolve the dimension list and mode from the arguments above.
|
||||
2. **CHECK mode**: launch every selected subagent concurrently (one `Agent` call per dimension in a single message). Each subagent runs read-only and returns its findings report. Tell each subagent explicitly: "Operate in REPORT mode. Do not modify any file." If `changed`, append the file list and: "Restrict findings to these files."
|
||||
3. **FIX mode**: launch the selected subagents **one at a time in canonical order** (never in parallel - parallel edits to the same file would conflict). Tell each: "Operate in FIX mode: apply minimal root-cause fixes per your doctrine, then run `hawk .` and confirm it passes." Wait for each to finish before starting the next. If `changed`, append the file list and: "Restrict fixes to these files."
|
||||
3. **FIX mode**: launch the selected subagents **one at a time in canonical order** (never in parallel - parallel edits to the same file would conflict). Tell each: "Operate in FIX mode: apply minimal root-cause fixes per your doctrine, then re-validate every file you touched with the per-language checks and confirm `python -c \"from devplacepy.main import app\"` still imports clean." Wait for each to finish before starting the next. If `changed`, append the file list and: "Restrict fixes to these files."
|
||||
4. Each subagent's final message is its report; it is not shown to the user directly, so collect them.
|
||||
|
||||
## Report
|
||||
|
||||
@ -13,4 +13,4 @@ Mirror an existing service - read `devplacepy/services/base.py` (BaseService) an
|
||||
4. If it calls an LLM, default its endpoint to `config.INTERNAL_GATEWAY_URL` and authenticate with the internal gateway key, like the other AI consumers.
|
||||
5. Emit audit events via `record_system` for any state change it makes.
|
||||
6. Document it in `devplacepy/services/CLAUDE.md` (Background services base machinery section, or the service's own nested `CLAUDE.md` if it has one) and in `README.md` if user-visible.
|
||||
7. Validate with `hawk` on the touched files and `python -c "from devplacepy.main import app"`.
|
||||
7. Validate the touched files with the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates) and `python -c "from devplacepy.main import app"`.
|
||||
|
||||
@ -1,13 +1,13 @@
|
||||
---
|
||||
description: Run the mandatory DevPlace pre-completion verification on changed files - the validator, the app import, and an em-dash scan. Zero errors required. Never runs the test suite.
|
||||
allowed-tools: Bash(python *), Bash(hawk *), Bash(git status:*), Bash(git diff:*), Read, Grep
|
||||
description: Run the mandatory DevPlace pre-completion verification on changed files - the per-language checks, the app import, and an em-dash scan. Zero errors required. Never runs the test suite.
|
||||
allowed-tools: Bash(python *), Bash(node *), Bash(git status:*), Bash(git diff:*), Read, Grep
|
||||
---
|
||||
Changed files in the working tree:
|
||||
!`git status --porcelain`
|
||||
|
||||
Verify the work is complete and correct, following the DevPlace verification rule (zero tolerance):
|
||||
|
||||
1. For each changed or new file under `devplacepy/` or `tests/`, run `hawk <file>` (it covers Python, JavaScript, CSS, and HTML/Jinja). Every file must report clean.
|
||||
1. For each changed or new file under `devplacepy/` or `tests/`, run the per-language checks (`python -m py_compile` + `pyflakes` for Python, `node --check` for JS, brace balance for CSS, tag and `{% %}` balance for templates). Every file must come back clean.
|
||||
2. Run `python -c "from devplacepy.main import app"` - it must import with no error.
|
||||
3. Grep the changed files for em-dash characters (U+2014 and U+2013) that are authored prose, and report any. Leave em-dashes that are data (replace/maketrans/regex targets, fixtures) untouched.
|
||||
4. Report a PASS or FAIL summary with the exact failures.
|
||||
|
||||
154
.claude/hooks/guard_production_db.py
Normal file
154
.claude/hooks/guard_production_db.py
Normal file
@ -0,0 +1,154 @@
|
||||
#!/usr/bin/env python3
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
CONFIRMATION_TOKEN = "I-HAVE-CONFIRMED-PRODUCTION-DB-ACCESS"
|
||||
|
||||
PRODUCTION_PATHS = re.compile(
|
||||
r"data/(devplace\.db|devii_tasks\.db|devii_lessons\.db|keys|uploads"
|
||||
r"|attachments|project_files|backups)\b"
|
||||
)
|
||||
PRODUCTION_DB_FILE = re.compile(r"\bdevplace\.db\b")
|
||||
MANAGEMENT_CLI = re.compile(r"(?:^|[;&|(]\s*|\s)(?:[\w./-]*/)?devplace\s+(?!-)")
|
||||
PYTHON_INVOCATION = re.compile(r"(?:^|[;&|(\s])(?:[\w./-]*/)?python[0-9.]*(?:\s|$)")
|
||||
DATABASE_OVERRIDE = re.compile(r"DEVPLACE_DATABASE_URL\s*=\s*[\"']?(\S+?)[\"']?(?:\s|$)")
|
||||
DATABASE_ASSIGNMENT = re.compile(r"DEVPLACE_DATABASE_URL[\"'\]\s]*[=,]")
|
||||
MODULE_INVOCATION = re.compile(r"-m\s+devplacepy")
|
||||
INLINE_CODE = re.compile(r"-c\s+(?P<quote>[\"'])(?P<code>.*?)(?P=quote)", re.DOTALL)
|
||||
SCRIPT_PATH = re.compile(r"(?:^|\s)(?P<path>[\w./~-]+\.py)(?:\s|$)")
|
||||
IMPORT_GATE = re.compile(
|
||||
r"^from devplacepy\.main import app\s*;?\s*(?:print\([^)]*\)\s*;?\s*)?$"
|
||||
)
|
||||
TEST_RUNNER = re.compile(r"\bpytest\b|\bmake\s+(test|test-[\w-]+)\b")
|
||||
SERVER_TARGET = re.compile(r"\bmake\s+(dev|prod|docker-[\w-]+|ppy)\b")
|
||||
|
||||
|
||||
APPLICATION_IMPORT = re.compile(
|
||||
r"(?:^|[\s;])(?:from|import)\s+devplacepy\b"
|
||||
r"|import_module\s*\(\s*[\"']devplacepy",
|
||||
re.MULTILINE,
|
||||
)
|
||||
|
||||
|
||||
def reaches_application(text: str) -> bool:
|
||||
return bool(APPLICATION_IMPORT.search(text))
|
||||
|
||||
|
||||
def overrides_the_database(text: str) -> bool:
|
||||
match = DATABASE_OVERRIDE.search(text)
|
||||
if not match:
|
||||
return False
|
||||
return "data/devplace.db" not in match.group(1)
|
||||
|
||||
|
||||
def source_targets_a_scratch_database(source: str) -> bool:
|
||||
if PRODUCTION_DB_FILE.search(source):
|
||||
return False
|
||||
return bool(DATABASE_ASSIGNMENT.search(source))
|
||||
|
||||
|
||||
def script_is_safe(command: str) -> bool | None:
|
||||
match = SCRIPT_PATH.search(command)
|
||||
if not match:
|
||||
return None
|
||||
path = Path(match.group("path")).expanduser()
|
||||
try:
|
||||
body = path.read_text(errors="replace")
|
||||
except OSError:
|
||||
return None
|
||||
if not reaches_application(body):
|
||||
return True
|
||||
return source_targets_a_scratch_database(body)
|
||||
|
||||
|
||||
def hazard_in(command: str) -> str:
|
||||
if CONFIRMATION_TOKEN in command:
|
||||
return ""
|
||||
if TEST_RUNNER.search(command) or SERVER_TARGET.search(command):
|
||||
return ""
|
||||
if PRODUCTION_DB_FILE.search(command) or PRODUCTION_PATHS.search(command):
|
||||
return "it names the production database or a production data directory"
|
||||
if MANAGEMENT_CLI.search(command):
|
||||
return "the devplace management CLI operates on the production database"
|
||||
if not PYTHON_INVOCATION.search(command):
|
||||
return ""
|
||||
if overrides_the_database(command):
|
||||
return ""
|
||||
if MODULE_INVOCATION.search(command):
|
||||
return "it runs a devplacepy module with no DEVPLACE_DATABASE_URL override"
|
||||
inline = INLINE_CODE.search(command)
|
||||
if inline:
|
||||
code = inline.group("code").strip()
|
||||
if not reaches_application(code):
|
||||
return ""
|
||||
if IMPORT_GATE.match(code):
|
||||
return ""
|
||||
return "it imports devplacepy inline with no DEVPLACE_DATABASE_URL override"
|
||||
safe = script_is_safe(command)
|
||||
if safe is None:
|
||||
return ""
|
||||
if safe:
|
||||
return ""
|
||||
return "the script imports devplacepy with no DEVPLACE_DATABASE_URL override"
|
||||
|
||||
|
||||
def refuse(reason: str) -> dict:
|
||||
return {
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": "PreToolUse",
|
||||
"permissionDecision": "deny",
|
||||
"permissionDecisionReason": (
|
||||
f"Blocked: this command reaches the production database because {reason}. "
|
||||
"The production database is never touched without the user's explicit, "
|
||||
"stated confirmation. Stop, tell the user exactly what the command would "
|
||||
"read or write, and ask them to confirm in their own words. Only after "
|
||||
f"they have done so may the command carry the literal token "
|
||||
f"{CONFIRMATION_TOKEN}, which still raises a permission prompt they must "
|
||||
"approve. Never add that token on your own initiative. Alternatives that "
|
||||
"need no confirmation: set DEVPLACE_DATABASE_URL to a scratch database, "
|
||||
"or run the test suite."
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def confirm(reason: str) -> dict:
|
||||
return {
|
||||
"hookSpecificOutput": {
|
||||
"hookEventName": "PreToolUse",
|
||||
"permissionDecision": "ask",
|
||||
"permissionDecisionReason": (
|
||||
"This command carries the production-database confirmation token and "
|
||||
f"reaches the production database because {reason}. Approve only if you "
|
||||
"asked for this."
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
try:
|
||||
payload = json.load(sys.stdin)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
sys.exit(0)
|
||||
command = (payload.get("tool_input") or {}).get("command") or ""
|
||||
if not command:
|
||||
sys.exit(0)
|
||||
if CONFIRMATION_TOKEN in command:
|
||||
stripped = command.replace(CONFIRMATION_TOKEN, "")
|
||||
reason = hazard_in(stripped)
|
||||
if reason:
|
||||
print(json.dumps(confirm(reason)))
|
||||
sys.exit(0)
|
||||
reason = hazard_in(command)
|
||||
if reason:
|
||||
print(json.dumps(refuse(reason)))
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
25
.claude/settings.json
Normal file
25
.claude/settings.json
Normal file
@ -0,0 +1,25 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-settings.json",
|
||||
"permissions": {
|
||||
"deny": [
|
||||
"Bash(devplace *)",
|
||||
"Write(data/**)",
|
||||
"Edit(data/**)"
|
||||
]
|
||||
},
|
||||
"hooks": {
|
||||
"PreToolUse": [
|
||||
{
|
||||
"matcher": "Bash",
|
||||
"hooks": [
|
||||
{
|
||||
"type": "command",
|
||||
"command": "python3 \"$CLAUDE_PROJECT_DIR/.claude/hooks/guard_production_db.py\"",
|
||||
"timeout": 10,
|
||||
"statusMessage": "Checking for production database access"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
@ -18,7 +18,7 @@ const RULES = [
|
||||
'- A Devii Action requires_auth/requires_admin MUST exactly match the underlying route guard. Never grant a member an admin capability. A non-admin must not even see an admin tool schema.',
|
||||
'- If the action is irreversible or destructive, add it to dispatcher CONFIRM_REQUIRED and declare a confirm boolean param in its spec (schemas set additionalProperties:false, so a gated tool without a declared confirm param can never receive confirm=true and loops forever).',
|
||||
'- Prefer handler="http" reusing an existing REST route; only add a local controller handler when there is no route. Reuse the arg()/body()/query()/confirm() helpers for params.',
|
||||
'- Validate with "hawk ." and "python -c \\"from devplacepy.main import app\\"". NEVER run the test suite. Never perform any git write.',
|
||||
'- Validate with the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". NEVER run the test suite. Never perform any git write.',
|
||||
].join('\n')
|
||||
|
||||
const TESTS = [
|
||||
@ -100,7 +100,7 @@ const map = await agent(
|
||||
)
|
||||
|
||||
const build = await agent(
|
||||
`Add this Devii tool, editing files directly in the repo. Add the Action to the catalog mirroring the similar action, set requires_auth/requires_admin to exactly match the underlying route guard, wire the dispatcher handler if a new local handler is needed, and add a docs_api.py entry if it wraps an HTTP endpoint. If destructive, add it to CONFIRM_REQUIRED and declare a confirm param. Then run "hawk ." and "python -c \\"from devplacepy.main import app\\"". Do not write the test here. Do not run the suite. Do not commit.\n\nTool request: ${ask}\n\nContext:\n${JSON.stringify(map, null, 2)}\n\n${RULES}\n\nReturn the action name, files changed, and whether validator and import passed.`,
|
||||
`Add this Devii tool, editing files directly in the repo. Add the Action to the catalog mirroring the similar action, set requires_auth/requires_admin to exactly match the underlying route guard, wire the dispatcher handler if a new local handler is needed, and add a docs_api.py entry if it wraps an HTTP endpoint. If destructive, add it to CONFIRM_REQUIRED and declare a confirm param. Then run the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". Do not write the test here. Do not run the suite. Do not commit.\n\nTool request: ${ask}\n\nContext:\n${JSON.stringify(map, null, 2)}\n\n${RULES}\n\nReturn the action name, files changed, and whether the checks and the import passed.`,
|
||||
{ label: 'implement', phase: 'Implement', schema: BUILD_SCHEMA }
|
||||
)
|
||||
|
||||
@ -127,7 +127,7 @@ const gaps = audits
|
||||
let gapFix = 'no actionable gaps'
|
||||
if (gaps.length) {
|
||||
gapFix = await agent(
|
||||
`Close these Devii tool gaps with minimal root-cause fixes in the repo, then re-run "hawk .". Do not run the suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
`Close these Devii tool gaps with minimal root-cause fixes in the repo, then re-run the per-language checks. Do not run the suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
{ label: 'fix-gaps', phase: 'Fix' }
|
||||
)
|
||||
}
|
||||
|
||||
@ -17,7 +17,7 @@ const RULES = [
|
||||
'- No em-dash characters; use a hyphen. Full Python type hints; pathlib over os; Pydantic Form input with explicit max lengths; sanitize and bound user input.',
|
||||
'- Reuse templating.templates, database.py batch helpers, respond(), the shared partials and frontend utilities. Never per-router Jinja2Templates.',
|
||||
'- Guards: get_current_user (public read), require_user (member write), require_admin (admin). Every POST/PUT/DELETE is guarded. Declare specific routes before catch-alls. Never pass a respond() context key that collides with a Jinja global (use viewer_is_admin).',
|
||||
'- Validate with "hawk ." and "python -c \\"from devplacepy.main import app\\"". NEVER run the test suite. Never perform any git write.',
|
||||
'- Validate with the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". NEVER run the test suite. Never perform any git write.',
|
||||
].join('\n')
|
||||
|
||||
const TOUCHPOINTS = [
|
||||
@ -108,7 +108,7 @@ const map = await agent(
|
||||
)
|
||||
|
||||
const build = await agent(
|
||||
`Implement this single DevPlace route across every applicable touchpoint, editing files directly in the repo, mirroring the closest existing route. Keep the layers in agreement (Out schema carries every returned JSON key; Devii action auth flags match the guard). Then run "hawk ." and "python -c \\"from devplacepy.main import app\\"". Do not write the test here. Do not run the suite. Do not commit.\n\nEndpoint: ${ask}\n\nClosest route to mirror:\n${JSON.stringify(map, null, 2)}\n\n${TOUCHPOINTS}\n\n${RULES}\n\nReturn the files changed and whether validator and import passed.`,
|
||||
`Implement this single DevPlace route across every applicable touchpoint, editing files directly in the repo, mirroring the closest existing route. Keep the layers in agreement (Out schema carries every returned JSON key; Devii action auth flags match the guard). Then run the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". Do not write the test here. Do not run the suite. Do not commit.\n\nEndpoint: ${ask}\n\nClosest route to mirror:\n${JSON.stringify(map, null, 2)}\n\n${TOUCHPOINTS}\n\n${RULES}\n\nReturn the files changed and whether the checks and the import passed.`,
|
||||
{ label: 'implement', phase: 'Implement', schema: BUILD_SCHEMA }
|
||||
)
|
||||
|
||||
@ -135,7 +135,7 @@ const gaps = audits
|
||||
let gapFix = 'no actionable gaps'
|
||||
if (gaps.length) {
|
||||
gapFix = await agent(
|
||||
`Close these gaps on the new route with minimal root-cause fixes in the repo, then re-run "hawk .". Do not run the suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
`Close these gaps on the new route with minimal root-cause fixes in the repo, then re-run the per-language checks. Do not run the suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
{ label: 'fix-gaps', phase: 'Fix' }
|
||||
)
|
||||
}
|
||||
|
||||
@ -22,7 +22,7 @@ const RULES = [
|
||||
'- Reuse shared helpers: templating.templates (never a per-router Jinja2Templates), database.py batch helpers (no inline N+1), the respond() negotiator, _avatar_link.html / _user_link.html, and on the frontend Http / Poller / JobPoller / OptimisticAction / FloatingWindow and the dp-* components.',
|
||||
'- Auth guards: get_current_user (public read), require_user (member write), require_admin (admin). Every POST/PUT/DELETE is guarded; deletes are soft and owner-or-admin.',
|
||||
'- Never pass a respond() context key that collides with a Jinja global (use viewer_is_admin, not is_admin). Dates are DD/MM/YYYY via format_date.',
|
||||
'- Validate with "hawk ." and "python -c \\"from devplacepy.main import app\\"". NEVER run the pytest suite. Never perform any git write.',
|
||||
'- Validate with the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". NEVER run the pytest suite. Never perform any git write.',
|
||||
].join('\n')
|
||||
|
||||
const FANOUT = [
|
||||
@ -185,7 +185,7 @@ const plan = await agent(
|
||||
)
|
||||
|
||||
const build = await agent(
|
||||
`Implement directly - no plan, no approval needed, this is implement mode. Build this DevPlace feature coherently and completely, editing files in the repo, following the plan. Keep every layer in agreement (the *Out schema must carry every JSON key the handler returns; the Devii action auth flags must match the route guard; a respond() context key must never shadow a Jinja global). Do NOT write pytest tests in this step (a later phase owns that). When done, run "hawk ." and "python -c \\"from devplacepy.main import app\\"" and report whether each passed, and list the user-facing routes the feature exposes.\n\nFeature request: ${ask}\n\nPlan:\n${JSON.stringify(plan, null, 2)}\n\n${FANOUT}\n\n${RULES}\n\nReturn the list of files you changed or created, whether the validator and the import passed, the routes, and a short summary.`,
|
||||
`Implement directly - no plan, no approval needed, this is implement mode. Build this DevPlace feature coherently and completely, editing files in the repo, following the plan. Keep every layer in agreement (the *Out schema must carry every JSON key the handler returns; the Devii action auth flags must match the route guard; a respond() context key must never shadow a Jinja global). Do NOT write pytest tests in this step (a later phase owns that). When done, run the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"" and report whether each passed, and list the user-facing routes the feature exposes.\n\nFeature request: ${ask}\n\nPlan:\n${JSON.stringify(plan, null, 2)}\n\n${FANOUT}\n\n${RULES}\n\nReturn the list of files you changed or created, whether the checks and the import passed, the routes, and a short summary.`,
|
||||
{ agentType: 'feature-builder', label: 'implement', phase: 'Implement', schema: BUILD_SCHEMA }
|
||||
)
|
||||
|
||||
@ -279,7 +279,7 @@ for (const i of (live && live.issues) || []) {
|
||||
let gapFix = 'no actionable gaps from the audit or live verification'
|
||||
if (gaps.length) {
|
||||
gapFix = await agent(
|
||||
`Close these confirmed completeness, security, style, frontend, and live-rendering gaps found in the new feature. Apply minimal root-cause fixes directly in the repo, keeping all layers in agreement and the styling consistent with the design system. Re-run "hawk ." afterward. Do not run the pytest suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
`Close these confirmed completeness, security, style, frontend, and live-rendering gaps found in the new feature. Apply minimal root-cause fixes directly in the repo, keeping all layers in agreement and the styling consistent with the design system. Re-run the per-language checks afterward. Do not run the pytest suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
{ agentType: 'feature-builder', label: 'fix-gaps', phase: 'Fix' }
|
||||
)
|
||||
}
|
||||
|
||||
@ -19,7 +19,7 @@ const RULES = [
|
||||
'- Runtime artifacts live in config.DATA_DIR (the var/ dir), OUTSIDE the devplacepy package and NOT under /static. Heavy compression or blocking work runs in a subprocess. SQLite stays synchronous.',
|
||||
'- Enqueue endpoints own authz (require_user plus any resource guard); status and download are capability URLs scoped only by the unguessable uuid7. Soft-delete the job tracking rows; permanent artifacts are not deleted by cleanup().',
|
||||
'- Record audit events with record_system in the service. Frontend status polling uses JobPoller, never a bespoke loop.',
|
||||
'- Validate with "hawk ." and "python -c \\"from devplacepy.main import app\\"". NEVER run the test suite. Never perform any git write.',
|
||||
'- Validate with the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". NEVER run the test suite. Never perform any git write.',
|
||||
].join('\n')
|
||||
|
||||
const CHECKLIST = [
|
||||
@ -133,7 +133,7 @@ const plan = await agent(
|
||||
)
|
||||
|
||||
const build = await agent(
|
||||
`Implement this new async job kind coherently, editing files directly in the repo, mirroring ZipService/ForkService and following the plan. Keep the *JobOut schema, routes, Devii tools, and docs in agreement. Then run "hawk ." and "python -c \\"from devplacepy.main import app\\"". Do not write tests here. Do not run the suite. Do not commit.\n\nJob request: ${ask}\n\nPlan:\n${JSON.stringify(plan, null, 2)}\n\n${CHECKLIST}\n\n${RULES}\n\nReturn the job kind, files changed, and whether validator and import passed.`,
|
||||
`Implement this new async job kind coherently, editing files directly in the repo, mirroring ZipService/ForkService and following the plan. Keep the *JobOut schema, routes, Devii tools, and docs in agreement. Then run the per-language checks (py_compile + pyflakes for Python, node --check for JS, brace balance for CSS, tag and {% %} balance for templates) and "python -c \\"from devplacepy.main import app\\"". Do not write tests here. Do not run the suite. Do not commit.\n\nJob request: ${ask}\n\nPlan:\n${JSON.stringify(plan, null, 2)}\n\n${CHECKLIST}\n\n${RULES}\n\nReturn the job kind, files changed, and whether the checks and the import passed.`,
|
||||
{ label: 'implement', phase: 'Implement', schema: BUILD_SCHEMA }
|
||||
)
|
||||
|
||||
@ -162,7 +162,7 @@ const gaps = audits
|
||||
let gapFix = 'no actionable gaps'
|
||||
if (gaps.length) {
|
||||
gapFix = await agent(
|
||||
`Close these job-service gaps with minimal root-cause fixes in the repo, then re-run "hawk .". Do not run the suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
`Close these job-service gaps with minimal root-cause fixes in the repo, then re-run the per-language checks. Do not run the suite. Do not commit.\n\nGaps:\n${JSON.stringify(gaps, null, 2)}\n\n${RULES}`,
|
||||
{ label: 'fix-gaps', phase: 'Fix' }
|
||||
)
|
||||
}
|
||||
|
||||
636
ARCHITECTURE.md
Normal file
636
ARCHITECTURE.md
Normal file
@ -0,0 +1,636 @@
|
||||
# DevPlace architecture atlas
|
||||
|
||||
Author: retoor <retoor@molodetz.nl>
|
||||
|
||||
Every diagram below was derived from the source tree, the live SQLite schema and the imported FastAPI app object, not from the documentation. Where the repository documentation and the running code disagree, the disagreement is recorded in the last section.
|
||||
|
||||
| Measure | Value |
|
||||
|---|---|
|
||||
| python files | 644 |
|
||||
| routers | 38 |
|
||||
| http paths | 348 |
|
||||
| websockets | 12 |
|
||||
| services | 27 |
|
||||
| db tables | 102 |
|
||||
| agent tools | 318 |
|
||||
| test functions | 3073 |
|
||||
|
||||
## 01 Deployment topology
|
||||
|
||||
Source: `docker-compose.yml / Dockerfile / nginx/nginx.conf.template`
|
||||
|
||||
Two containers on one bridge network, on a single host. nginx is the only published listener and binds to loopback only. The app container bind-mounts the repository root, so the production process reads the same SQLite file and the same `data/` tree as `make dev`.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"13px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
client["Browser / API client / Telegram / devRant client"]
|
||||
subgraph host["Single host"]
|
||||
subgraph appnet["docker network: appnet"]
|
||||
nginx["nginx container<br/>127.0.0.1:PORT to :80<br/>8 websocket upgrade locations"]
|
||||
app["app container<br/>uvicorn devplacepy.main:app<br/>--workers 2 --backlog 8192"]
|
||||
end
|
||||
repo["repository root<br/>bind mount .:/app"]
|
||||
static["devplacepy/static<br/>read only mount"]
|
||||
data["DEVPLACE_DATA_DIR = data/<br/>23 registered paths"]
|
||||
sqlite[("data/devplace.db<br/>SQLite, WAL, 256MB mmap")]
|
||||
lock["data/locks/devplace-services.lock<br/>flock, elects one service owner"]
|
||||
docker["host Docker daemon<br/>ppy:latest instances"]
|
||||
end
|
||||
client --> nginx
|
||||
nginx -->|"proxy_pass, X-Real-IP"| app
|
||||
nginx -->|"/static/, /static/uploads/"| static
|
||||
app --> repo
|
||||
app --> data
|
||||
data --> sqlite
|
||||
app --> lock
|
||||
app -->|"docker CLI backend"| docker
|
||||
```
|
||||
|
||||
nginx depends_on app with condition service_healthy. App healthcheck: interval 30s, timeout 10s, retries 3, start_period 120s, start_interval 2s.
|
||||
|
||||
## 02 Worker boot and service election
|
||||
|
||||
Source: `devplacepy/main.py lifespan`
|
||||
|
||||
Every uvicorn worker runs the full lifespan. Schema work is serialized behind an exclusive lock so workers pay it one after another; background services are elected, so exactly one worker in the host owns them.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"13px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
s1["ensure_data_dirs<br/>creates all 23 DATA_PATHS"]
|
||||
s2["init_lock<br/>exclusive flock on INIT_LOCK_FILE"]
|
||||
s3["init_db<br/>idempotent columns and indexes"]
|
||||
s4["ensure_certificates<br/>VAPID keys in data/keys"]
|
||||
s5["service_manager.register x27"]
|
||||
gate{"DEVPLACE_DISABLE_SERVICES set?"}
|
||||
skip["services registered but never started<br/>test mode"]
|
||||
bg["background.start<br/>per worker asyncio queue"]
|
||||
elect{"acquire_service_lock"}
|
||||
owner["set_lock_owner True<br/>supervise all 27 services"]
|
||||
decline["declined, another worker owns them"]
|
||||
vis["start_visit_flusher"]
|
||||
serve(["serving on port 10500"])
|
||||
s1 --> s2 --> s3 --> s4 --> s5 --> gate
|
||||
gate -->|yes| skip --> vis
|
||||
gate -->|no| bg --> elect
|
||||
elect -->|"lock acquired"| owner --> vis
|
||||
elect -->|"lock held elsewhere"| decline --> vis
|
||||
vis --> serve
|
||||
```
|
||||
|
||||
Shutdown reverses it: flush_visits, service_manager.shutdown_all, background.stop.
|
||||
|
||||
## 03 Request pipeline
|
||||
|
||||
Source: `devplacepy/main.py, verified against app.user_middleware`
|
||||
|
||||
Ten middlewares wrap every request. The order below is the real execution order read off the built middleware stack, outermost first. Two of them are plain ASGI classes; the other eight are `BaseHTTPMiddleware` dispatch functions.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"13px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
req(["incoming ASGI scope"])
|
||||
m0["0 GZipMiddleware<br/>minimum_size 512, level 5"]
|
||||
m1["1 TunnelDispatchMiddleware<br/>host matches a workspace tunnel?"]
|
||||
tun["routers/tunnel.py<br/>handle_http / handle_ws"]
|
||||
m2["2 response_timing<br/>request.state.request_start, X-Response-Time"]
|
||||
m3["3 visit_statistics"]
|
||||
m4["4 track_presence<br/>throttled last_seen write"]
|
||||
m5["5 maintenance_middleware<br/>503 unless static, avatar, auth, admin"]
|
||||
m6["6 rate_limit_middleware<br/>mutating methods only, per IP bucket"]
|
||||
m7["7 add_security_headers"]
|
||||
m8["8 await_pending_corrections<br/>drains sync AI correction futures"]
|
||||
m9["9 refresh_db_snapshot"]
|
||||
mounts["mounts: /static/uploads, /static/v{ts}, /static"]
|
||||
routes["38 included routers"]
|
||||
handlers["exception handlers<br/>404, 500, RequestValidationError"]
|
||||
req --> m0 --> m1
|
||||
m1 -->|"tunnel host"| tun
|
||||
m1 -->|"normal host"| m2 --> m3 --> m4 --> m5 --> m6 --> m7 --> m8 --> m9
|
||||
m9 --> mounts
|
||||
m9 --> routes
|
||||
routes --> handlers
|
||||
```
|
||||
|
||||
Rate limiting reads rate_limit_per_minute and rate_limit_window_seconds from site_settings and exempts reads and the /openai gateway. Bucket key is X-Real-IP falling back to request.client.host.
|
||||
|
||||
## 04 URL surface
|
||||
|
||||
Source: `devplacepy/main.py include_router calls`
|
||||
|
||||
38 routers, each mounted under one prefix, grouped here by the concern they serve. Directory shape mirrors the URL: a domain with one resource is a flat module, a domain with several sub-resources is a package that aggregates its leaves in `__init__.py`.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart LR
|
||||
app(["FastAPI app"])
|
||||
subgraph identity["Identity and profile"]
|
||||
r1["/auth - auth/"]
|
||||
r2["/profile - profile/"]
|
||||
r3["/avatar - avatar.py"]
|
||||
r4["/follow - follow.py"]
|
||||
r5["(none) - relations.py block, mute"]
|
||||
end
|
||||
subgraph social["Content and engagement"]
|
||||
r6["/feed - feed.py"]
|
||||
r7["/posts - posts.py"]
|
||||
r8["/comments - comments.py"]
|
||||
r9["/gists - gists.py"]
|
||||
r10["/news - news.py"]
|
||||
r11["/votes - votes.py"]
|
||||
r12["/reactions - reactions.py"]
|
||||
r13["/bookmarks - bookmarks.py"]
|
||||
r14["/polls - polls.py"]
|
||||
r15["/awards - awards.py"]
|
||||
r16["/leaderboard - leaderboard.py"]
|
||||
r17["/notifications - notifications.py"]
|
||||
r18["/messages - messages.py"]
|
||||
r19["/uploads - uploads.py"]
|
||||
r20["/media - media.py"]
|
||||
end
|
||||
subgraph work["Projects and compute"]
|
||||
r21["/projects - projects/ incl files, containers"]
|
||||
r22["/p - proxy.py container ingress"]
|
||||
r23["/zips - zips.py"]
|
||||
r24["/forks - forks.py"]
|
||||
r25["/issues - issues/"]
|
||||
end
|
||||
subgraph aiml["AI and tools"]
|
||||
r26["/openai - openai_gateway.py"]
|
||||
r27["/devii - devii.py"]
|
||||
r28["/tools - tools/ seo, deepsearch, isslop"]
|
||||
end
|
||||
subgraph play["Play"]
|
||||
r29["/game - game/"]
|
||||
r30["/quizzes - quizzes/"]
|
||||
end
|
||||
subgraph platform["Platform"]
|
||||
r31["/admin - admin/ 23 modules"]
|
||||
r32["/docs - docs/"]
|
||||
r33["(none) - push.py, seo.py"]
|
||||
end
|
||||
subgraph machine["Machine interfaces"]
|
||||
r34["/api - devrant/"]
|
||||
r35["/dbapi - dbapi/ read only"]
|
||||
r36["/xmlrpc - xmlrpc.py"]
|
||||
r37["/pubsub - pubsub.py"]
|
||||
end
|
||||
app --> identity
|
||||
app --> social
|
||||
app --> work
|
||||
app --> aiml
|
||||
app --> play
|
||||
app --> platform
|
||||
app --> machine
|
||||
```
|
||||
|
||||
| Prefix | Paths | Prefix | Paths | Prefix | Paths |
|
||||
|---|---|---|---|---|---|
|
||||
| /admin | 92 | /profile | 17 | /gists | 5 |
|
||||
| /projects | 46 | /issues | 13 | /messages | 5 |
|
||||
| /tools | 27 | /dbapi | 9 | /notifications | 5 |
|
||||
| /game | 23 | /auth | 6 | /uploads | 5 |
|
||||
| /quizzes | 21 | /devii | 5 | /docs | 4 |
|
||||
| /api | 20 | /posts | 4 | everything else | 36 |
|
||||
|
||||
## 05 Route fan-out
|
||||
|
||||
Source: `devplacepy/responses.py, schemas/, docs_api/, services/devii/actions/`
|
||||
|
||||
One handler serves four consumers. `respond()` feeds the same context dict to a Pydantic model and to a Jinja template, so a key missing from the `*Out` schema is silently absent from JSON while still rendering in HTML. The agent catalog and the docs registry are separate declarations that must be kept in step by hand.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"13px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
form["models.py<br/>Pydantic form model via Annotated Form"]
|
||||
guard["guard: get_current_user<br/>require_user / require_admin"]
|
||||
handler["router handler"]
|
||||
helpers["database/ batch helpers<br/>get_users_by_uids, build_pagination"]
|
||||
ctx["context dict"]
|
||||
neg{"wants_json(request)"}
|
||||
json["model.model_validate<br/>JSONResponse"]
|
||||
html["templates.TemplateResponse<br/>extends base.html"]
|
||||
action["services/devii/actions catalog<br/>Action, requires_auth mirrors guard"]
|
||||
docs["docs_api endpoint()<br/>params and sample_response"]
|
||||
seo["seo.py base_seo_context<br/>JSON-LD, sitemap entry"]
|
||||
form --> handler
|
||||
guard --> handler
|
||||
handler --> helpers --> ctx --> neg
|
||||
neg -->|"json accepted"| json
|
||||
neg -->|"otherwise"| html
|
||||
handler -.->|"same capability"| action
|
||||
handler -.->|"same contract"| docs
|
||||
html -.-> seo
|
||||
```
|
||||
|
||||
wants_json is true when the request content-type starts with application/json, or Accept contains application/json and does not contain text/html. Registry sizes: 318 agent actions, 259 documented endpoint entries across 20 API groups, 122 prose docs pages.
|
||||
|
||||
## 06 Identity resolution
|
||||
|
||||
Source: `devplacepy/utils/auth.py`
|
||||
|
||||
A single resolver serves browsers, API clients, the devRant compatibility layer and issued access tokens. The result is memoised on `request.state` for the request and in a process TTL cache keyed by credential.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"13px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
start(["get_current_user"])
|
||||
cached{"request.state._auth_user set?"}
|
||||
sess{"session cookie<br/>64 hex chars"}
|
||||
xapi{"X-API-KEY header"}
|
||||
bearer{"Authorization: Bearer"}
|
||||
basic{"Authorization: Basic"}
|
||||
triple["try in order:<br/>users.api_key,<br/>devrant token 40 hex,<br/>access token 64 chars"]
|
||||
user(["user dict"])
|
||||
guest(["None, guest"])
|
||||
gu["require_user: 303 to /"]
|
||||
ga["require_admin: redirect to /feed"]
|
||||
start --> cached
|
||||
cached -->|yes| user
|
||||
cached -->|no| sess
|
||||
sess -->|match| user
|
||||
sess -->|no| xapi
|
||||
xapi -->|present| triple
|
||||
triple -->|match| user
|
||||
triple -->|no match| bearer
|
||||
xapi -->|absent| bearer
|
||||
bearer -->|present| triple
|
||||
bearer -->|absent| basic
|
||||
basic -->|"username-or-email:password, pbkdf2_sha256"| user
|
||||
basic -->|no| guest
|
||||
guest --> gu
|
||||
user --> ga
|
||||
```
|
||||
|
||||
Roles are stored capitalized as Admin or Member and tested through the is_admin global. Admin seniority is enforced per user mutation in routers/admin/users.py.
|
||||
|
||||
## 07 Background service fleet
|
||||
|
||||
Source: `devplacepy/services/manager.py, base.py, jobs/base.py`
|
||||
|
||||
27 singletons registered at boot, supervised only by the worker that won the service lock. Sixteen are long-lived loops on `BaseService`; eleven are queue consumers on `JobService`, which adds retention, concurrency and timeout settings on top and drains rows from the shared `jobs` table.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart LR
|
||||
mgr["ServiceManager<br/>supervise, shutdown_all"]
|
||||
settings[("site_settings<br/>enabled flag and interval per service")]
|
||||
jobs[("jobs table")]
|
||||
subgraph base["BaseService loops - 16"]
|
||||
b1["NewsService"]
|
||||
b2["BotsService"]
|
||||
b3["GatewayService"]
|
||||
b4["DeviiService"]
|
||||
b5["PubSubService"]
|
||||
b6["NotificationRelayService"]
|
||||
b7["LiveViewRelayService"]
|
||||
b8["PresenceRelayService"]
|
||||
b9["IssueTrackerService"]
|
||||
b10["ContainerService"]
|
||||
b11["WorkspaceService"]
|
||||
b12["XmlrpcService"]
|
||||
b13["AuditService"]
|
||||
b14["PushService"]
|
||||
b15["TelegramService"]
|
||||
b16["TelegramOutboxService"]
|
||||
end
|
||||
subgraph job["JobService consumers - 11"]
|
||||
j1["ZipService"]
|
||||
j2["ForkService"]
|
||||
j3["SeoService"]
|
||||
j4["SeoMetaService"]
|
||||
j5["AwardService"]
|
||||
j6["BackupService"]
|
||||
j7["DbApiJobService"]
|
||||
j8["DeepsearchService"]
|
||||
j9["IsslopService"]
|
||||
j10["IssueCreateService"]
|
||||
j11["PlanningReportService"]
|
||||
end
|
||||
mgr --> base
|
||||
mgr --> job
|
||||
settings --> mgr
|
||||
job --> jobs
|
||||
```
|
||||
|
||||
Separate from the fleet, services/background.py offers a fire and forget queue per worker for audit writes, XP awards and notifications; when no consumer runs, as in tests, the callable executes inline so ordering stays deterministic.
|
||||
|
||||
## 08 Data layer
|
||||
|
||||
Source: `devplacepy/database/, devplacepy/config.py`
|
||||
|
||||
One SQLite file reached through `dataset`, called synchronously from async handlers by design. 102 tables exist in the live schema; 52 of them carry the soft delete pair and are restorable as one event from the admin trash.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
db[("data/devplace.db")]
|
||||
subgraph g1["Identity and access - 14"]
|
||||
t1["users, sessions, password_resets<br/>access_tokens, devrant_tokens<br/>user_relations, follows<br/>notification_preferences, user_customizations<br/>push_registration, email_accounts<br/>telegram_links, telegram_pairings, telegram_outbox"]
|
||||
end
|
||||
subgraph g2["Content and engagement - 16"]
|
||||
t2["posts, comments, gists<br/>projects, project_files, project_forks<br/>attachments, news, news_images, news_sync<br/>polls, poll_options, poll_votes<br/>votes, reactions, bookmarks"]
|
||||
end
|
||||
subgraph g3["Reputation - 5"]
|
||||
t3["awards, badges, award_usage<br/>user_activity, user_activity_seen"]
|
||||
end
|
||||
subgraph g4["Code Farm - 9"]
|
||||
t4["game_farms, game_plots, game_quests<br/>game_cosmetics, game_market_ticks<br/>game_steals, game_treasury<br/>game_eras, game_era_results"]
|
||||
end
|
||||
subgraph g5["Quizzes - 5"]
|
||||
t5["quizzes, quiz_questions, quiz_options<br/>quiz_attempts, quiz_answers"]
|
||||
end
|
||||
subgraph g6["Live delivery - 3"]
|
||||
t6["messages, notifications, ws_tickets"]
|
||||
end
|
||||
subgraph g7["Devii - 8"]
|
||||
t7["devii_conversations, devii_turns<br/>devii_tasks, devii_task_runs<br/>devii_lessons, devii_virtual_tools<br/>devii_behavior, devii_usage_ledger"]
|
||||
end
|
||||
subgraph g8["AI gateway - 6"]
|
||||
t8["gateway_providers, gateway_models<br/>gateway_usage_ledger<br/>gateway_quota_rules, gateway_quota_resets<br/>gateway_concurrency_samples"]
|
||||
end
|
||||
subgraph g9["Jobs, tools and usage - 20"]
|
||||
t9["jobs, backups, backup_schedules<br/>deepsearch_sessions, deepsearch_messages, deepsearch_url_cache<br/>isslop_analyses, isslop_reports, isslop_events<br/>isslop_dom_results, isslop_file_results, isslop_image_results<br/>seo_metadata, seo_usage<br/>issue_tickets, issue_comment_authors, issue_usage<br/>correction_usage, modifier_usage, news_usage"]
|
||||
end
|
||||
subgraph g10["Containers - 9"]
|
||||
t10["instances, instance_events, instance_metrics<br/>tunnels, workspace_flags, workspace_quota_rules<br/>builds, dockerfiles, dockerfile_versions"]
|
||||
end
|
||||
subgraph g11["Platform state - 7"]
|
||||
t11["site_settings, cache_state, service_state<br/>audit_log, audit_log_links<br/>visit_stats_hourly, visit_unique_slots"]
|
||||
end
|
||||
db --> g1
|
||||
db --> g2
|
||||
db --> g3
|
||||
db --> g4
|
||||
db --> g5
|
||||
db --> g6
|
||||
db --> g7
|
||||
db --> g8
|
||||
db --> g9
|
||||
db --> g10
|
||||
db --> g11
|
||||
```
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart LR
|
||||
root["DEVPLACE_DATA_DIR<br/>default repo/data"]
|
||||
blobs["uploads, attachments, project_files<br/>sharded xx/yy on the uuid7 random tail"]
|
||||
jobsdir["zips, zip_staging, fork_staging<br/>backups, backup_staging"]
|
||||
reports["seo_reports, planning_reports<br/>dbapi, deepsearch, deepsearch_chroma"]
|
||||
isslop["isslop, isslop_workspaces<br/>isslop_runs, isslop_media"]
|
||||
ws["container_workspaces, workspace_state"]
|
||||
keys["keys VAPID, bot, locks"]
|
||||
dbs["devplace.db, devii_tasks.db, devii_lessons.db"]
|
||||
root --> blobs
|
||||
root --> jobsdir
|
||||
root --> reports
|
||||
root --> isslop
|
||||
root --> ws
|
||||
root --> keys
|
||||
root --> dbs
|
||||
```
|
||||
|
||||
config.DATA_PATHS registers 23 directories and ensure_data_dirs creates the whole tree before any write. Uploads live under data/uploads but are served at the unchanged /static/uploads URL.
|
||||
|
||||
## 09 Content rendering
|
||||
|
||||
Source: `devplacepy/rendering.py, static/js/ContentRenderer.js`
|
||||
|
||||
Two pipelines with a deliberate split: anything that exists at request time is rendered on the server for SEO, and the client pipeline is reserved for content that does not exist yet. Each has its own XSS control at a different point.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"13px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
subgraph server["Server, rendering.py, lru_cache"]
|
||||
a1["raw text"]
|
||||
a2["normalize dashes to hyphen"]
|
||||
a3["emoji shortcodes, 4869 names"]
|
||||
a4["mistune GFM, escape=True"]
|
||||
a5["media pass: bare URLs to embeds, mentions to links"]
|
||||
a6["mask emails on rendered text nodes"]
|
||||
a7["render_content / render_title in template"]
|
||||
a8["ContentEnhancer adds highlighting and copy buttons"]
|
||||
a1 --> a2 --> a3 --> a4 --> a5 --> a6 --> a7 --> a8
|
||||
end
|
||||
subgraph client["Client, ContentRenderer.js"]
|
||||
b1["live text: comments, DM bubbles, Devii, DeepSearch"]
|
||||
b2["marked"]
|
||||
b3["DOMPurify.sanitize, fail closed"]
|
||||
b4["highlight.js"]
|
||||
b5["media and autolink pass"]
|
||||
b1 --> b2 --> b3 --> b4 --> b5
|
||||
end
|
||||
```
|
||||
|
||||
The server escapes at the markdown step, the client sanitizes after parsing. Server rendered content must not carry data-render, or both pipelines run over it.
|
||||
|
||||
## 10 AI plane
|
||||
|
||||
Source: `devplacepy/routers/openai_gateway.py, services/openai_gateway/, services/devii/`
|
||||
|
||||
Every model call in the product, internal or external, leaves through one gateway, which is where routing, quota and cost attribution live. Internal consumers call it over loopback HTTP rather than importing a client, so their spend lands in the same ledger.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
ext["external client<br/>/openai/v1/*"]
|
||||
subgraph consumers["Internal consumers via INTERNAL_GATEWAY_URL"]
|
||||
c1["services/devii"]
|
||||
c2["services/correction<br/>AI correction and modifier"]
|
||||
c3["services/news"]
|
||||
c4["services/bot"]
|
||||
c5["services/deepsearch + jobs/deepsearch"]
|
||||
c6["services/dbapi nl2sql"]
|
||||
c7["services/gitea enhance + planning"]
|
||||
c8["jobs/isslop"]
|
||||
end
|
||||
gw["gateway.py<br/>routing.py, quota.py, usage.py<br/>reliability.py, vision.py"]
|
||||
ledger[("gateway_usage_ledger<br/>gateway_quota_rules")]
|
||||
stealth["stealth_async_client<br/>curl_cffi Chrome 146 fingerprint"]
|
||||
up["upstream provider"]
|
||||
subgraph devii["Devii assistant"]
|
||||
d1["registry.py CATALOG<br/>318 tools"]
|
||||
d2["role gating<br/>guest 108, member 246<br/>admin 312, primary 318"]
|
||||
d3["45 tools behind CONFIRM_REQUIRED"]
|
||||
d4["session, tasks, lessons<br/>virtual tools, behavior"]
|
||||
d5["surfaces: /devii ws, Telegram, CLI"]
|
||||
end
|
||||
ext --> gw
|
||||
consumers --> gw
|
||||
gw --> ledger
|
||||
gw --> stealth --> up
|
||||
d1 --> d2 --> d3
|
||||
devii --> c1
|
||||
d4 --> d1
|
||||
d5 --> d4
|
||||
```
|
||||
|
||||
Cleartext loopback calls are forced to HTTP/1.1 in curl_transport.http_version_for, because the Chrome impersonation profile would otherwise negotiate HTTP/2 against an HTTP/1.1 only uvicorn.
|
||||
|
||||
## 11 Containers, workspaces and ingress
|
||||
|
||||
Source: `devplacepy/services/containers/, routers/projects/containers/, routers/proxy.py, routers/tunnel.py`
|
||||
|
||||
One shared image serves every instance. Reconciliation is a loop that compares desired state in the database against what the Docker daemon actually reports, and there are two independent ways in from the outside: a path prefix and a hostname.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
ui["/projects/{slug}/containers<br/>instances, schedules, workspace"]
|
||||
api["services/containers/api.py"]
|
||||
store[("instances, instance_events<br/>instance_metrics, tunnels<br/>workspace_flags, workspace_quota_rules")]
|
||||
svc["ContainerService<br/>reconcile desired vs docker ps"]
|
||||
wsvc["WorkspaceService<br/>provision, certs, quota, tunnels"]
|
||||
backend["backend/docker_cli.py<br/>backend/fake.py for tests"]
|
||||
image["single shared image ppy:latest"]
|
||||
inst["running instance<br/>rootless workflow, aptroot"]
|
||||
wsdir["data/container_workspaces/{uid}"]
|
||||
proxy["/p/{slug}<br/>routers/proxy.py, relays headers verbatim"]
|
||||
tunnel["host based tunnel<br/>TunnelDispatchMiddleware to routers/tunnel.py"]
|
||||
code["workspace editor over websocket<br/>/projects/{slug}/containers/instances/{uid}/code"]
|
||||
exec["terminal over websocket<br/>.../exec/ws"]
|
||||
ui --> api --> store
|
||||
svc --> store
|
||||
wsvc --> store
|
||||
api --> backend
|
||||
svc --> backend
|
||||
wsvc --> wsdir
|
||||
backend --> image --> inst
|
||||
proxy --> inst
|
||||
tunnel --> inst
|
||||
code --> inst
|
||||
exec --> inst
|
||||
```
|
||||
|
||||
Access uses stricter predicates than the rest of the product: owns_instance, can_view_project_containers, can_view_instance, can_manage_instance. The primary administrator sees and manages every container; any other admin can only view others on public projects and manage instances they own.
|
||||
|
||||
## 12 Real time plane
|
||||
|
||||
Source: `routers/*.py websocket handlers, services relays, nginx.conf.template`
|
||||
|
||||
Twelve WebSocket endpoints, each of which needs its own nginx upgrade location because the catch-all location strips upgrade headers. Fan-out to connected sockets goes through in-process hubs driven by relay services on the lock-owning worker.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart LR
|
||||
subgraph sockets["WebSocket endpoints"]
|
||||
s1["/devii/ws"]
|
||||
s2["/messages/ws"]
|
||||
s3["/pubsub/ws"]
|
||||
s4["/tools/seo/{uid}/ws"]
|
||||
s5["/tools/deepsearch/{uid}/ws"]
|
||||
s6["/tools/deepsearch/{uid}/chat"]
|
||||
s7["/dbapi/query/{uid}/ws"]
|
||||
s8[".../containers/instances/{uid}/exec/ws"]
|
||||
s9[".../containers/instances/{uid}/code"]
|
||||
s10[".../code/{path}"]
|
||||
s11["/p/{slug}"]
|
||||
s12["/p/{slug}/{path}"]
|
||||
end
|
||||
subgraph relays["Relay services"]
|
||||
r1["NotificationRelayService"]
|
||||
r2["LiveViewRelayService"]
|
||||
r3["PresenceRelayService<br/>track limit 500, online limit 30"]
|
||||
r4["PubSubService"]
|
||||
end
|
||||
hubs["messaging/hub.py, pubsub/hub.py<br/>devii/hub.py"]
|
||||
tick[("ws_tickets<br/>expiring auth tickets")]
|
||||
s2 --> hubs
|
||||
s3 --> hubs
|
||||
s1 --> hubs
|
||||
relays --> hubs
|
||||
tick --> s2
|
||||
```
|
||||
|
||||
Presence is authoritative from the relay: PRESENCE_TRACK_LIMIT sets the tracked online set, PRESENCE_ONLINE_LIMIT only caps how many the feed panel displays, and PRESENCE_ONLINE_MARGIN_SECONDS provides hysteresis at the boundary.
|
||||
|
||||
## 13 Frontend
|
||||
|
||||
Source: `devplacepy/static/, devplacepy/templates/`
|
||||
|
||||
No framework and no package manager. 131 ES6 modules, one class per file, hung off a single global `app`; 33 of them are custom elements. 217 Jinja templates, of which 107 are the docs site.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart TB
|
||||
base["templates/base.html<br/>extra_head for CSS, extra_js for JS"]
|
||||
partials["shared partials<br/>_avatar_link, _user_link, _comment_section<br/>_post_card, _pagination, _sidebar_search"]
|
||||
appjs["static/js/Application.js<br/>instantiated once as app"]
|
||||
subgraph modules["131 ES6 modules"]
|
||||
u1["utilities<br/>Http, Poller, JobPoller<br/>OptimisticAction, FloatingWindow, ScrollMemory"]
|
||||
u2["33 custom elements<br/>components/ with dp- prefix"]
|
||||
u3["feature modules<br/>chat/, devii/, autoload/"]
|
||||
end
|
||||
css["49 stylesheets<br/>variables.css tokens, per page files"]
|
||||
vers["static_url in Jinja, assetUrl in JS<br/>/static/v{boot ts}/, immutable for a year"]
|
||||
cust["per user CSS and JS injection<br/>custom_css_tag, custom_js_tag"]
|
||||
base --> partials
|
||||
base --> appjs --> modules
|
||||
base --> css
|
||||
base --> cust
|
||||
vers --> css
|
||||
vers --> modules
|
||||
```
|
||||
|
||||
Per user customizations are configured only through Devii, stored in user_customizations, scoped globally or per matched route template, and run solely in that owner's own browser sessions.
|
||||
|
||||
## 14 Test and delivery topology
|
||||
|
||||
Source: `tests/, pyproject.toml, .gitea/workflows/test.yaml`
|
||||
|
||||
Three tiers separated by what they exercise, decided by fixtures, run serially in a single process against one uvicorn subprocess on port 10501 with a temporary database.
|
||||
|
||||
```mermaid
|
||||
%%{init:{"theme":"base","themeVariables":{"background":"#F4F8F7","primaryColor":"#E4EEED","primaryTextColor":"#0F1A1B","primaryBorderColor":"#0A6D6A","secondaryColor":"#EDE6D9","tertiaryColor":"#F4F8F7","lineColor":"#6B7C7C","textColor":"#0F1A1B","fontFamily":"ui-monospace,SFMono-Regular,Menlo,monospace","fontSize":"12px","clusterBkg":"#FAFCFC","clusterBorder":"#C6D4D3"}}}%%
|
||||
flowchart LR
|
||||
unit["tests/unit - 140 modules<br/>mirrors source module path<br/>local_db or no fixture"]
|
||||
api["tests/api - 221 modules<br/>mirrors URL path<br/>app_server, seeded_db"]
|
||||
e2e["tests/e2e - 131 modules<br/>mirrors URL path<br/>page, alice, bob"]
|
||||
srv["uvicorn subprocess :10501<br/>temp SQLite, own DATA_DIR<br/>DEVPLACE_DISABLE_SERVICES=1"]
|
||||
pw["Playwright Chromium<br/>session scoped context"]
|
||||
ci["Gitea Actions on push and PR to master<br/>full suite under coverage"]
|
||||
prod["promotion to production<br/>docker compose"]
|
||||
unit --> ci
|
||||
api --> srv --> ci
|
||||
e2e --> srv
|
||||
e2e --> pw --> ci
|
||||
ci --> prod
|
||||
```
|
||||
|
||||
pytest-xdist is not a dependency and -n is rejected centrally, so no tier can be parallelised by accident.
|
||||
|
||||
## 15 Findings
|
||||
|
||||
Source: `measured against the repository documentation`
|
||||
|
||||
The structure holds up: the router tree mirrors the URL tree, the test tree mirrors both, every runtime path is registered in one place, and every model call has a single exit. The items below are the places where the running code and the written record have drifted apart, or where a structure exists that nothing currently uses. None of them is a functional defect.
|
||||
|
||||
### Middleware order in the documentation is stale
|
||||
|
||||
The root CLAUDE.md states that response_timing is the outermost middleware. Measured from app.user_middleware it is third, behind GZipMiddleware and TunnelDispatchMiddleware, both added after it. The consequence is minor but real: X-Response-Time excludes compression time and excludes tunnel-host dispatch entirely.
|
||||
|
||||
### Two catalogue counts have drifted
|
||||
|
||||
CLAUDE.md cites around 2882 tests and an events.md catalogue of 288 keys. Measured now: 3073 functions named test_ across 492 modules, and 312 distinct dotted event keys in events.md. Both are undercounts in the docs, not missing implementation.
|
||||
|
||||
### instance_schedules is declared soft-deletable but has no table
|
||||
|
||||
It appears in database.SOFT_DELETE_TABLES and in the routers under /projects/{slug}/containers/schedules, but no such table exists in the live schema. dataset creates it lazily on first insert, so this is correct only for as long as every read path tolerates the table being absent.
|
||||
|
||||
### Three legacy container tables still exist
|
||||
|
||||
builds, dockerfiles and dockerfile_versions are present in the live database. The project replaced per-project images with the single shared ppy:latest image and ships devplace containers prune-builds as the one-time cleanup for exactly these rows. The tables are still carried.
|
||||
|
||||
### Five duplicate OpenAPI operation IDs
|
||||
|
||||
Generating the schema warns on editor_proxy twice in routers/projects/containers/workspace.py, passthrough in routers/openai_gateway.py, and proxy_http twice in routers/proxy.py. These are catch-all routes registered for several methods, so a generated client would collide on those names.
|
||||
|
||||
### The one deliberate asymmetry is documented and intentional
|
||||
|
||||
routers/__init__.py holds nothing but the attribution line; unlike every nested router package, the top level does not aggregate. main.py performs all 38 include_router calls directly, which keeps prefix ownership in a single readable block.
|
||||
|
||||
---
|
||||
|
||||
Sources: devplacepy/main.py, the imported FastAPI app object, data/devplace.db sqlite_master, and the working tree at HEAD 192df12b with uncommitted local modifications present. Counts exclude __pycache__ and the virtual environment.
|
||||
79
CLAUDE.md
79
CLAUDE.md
@ -86,6 +86,8 @@ devplace game steals prune # delete Code Farm raid records older than the raid-
|
||||
devplace game era status # show the current Code Farm Era
|
||||
devplace game era start <name> [--days N] # start a Code Farm Era (default 28 days)
|
||||
devplace game era end # end the running Code Farm Era (ranks, awards Stars, records results)
|
||||
devplace accounts pending # list deleted accounts awaiting their purge
|
||||
devplace accounts prune # permanently purge accounts past the deletion grace window (--dry-run to preview)
|
||||
devplace backups list # list recorded backups
|
||||
devplace backups run <database|uploads|keys|full> # enqueue a backup (processed by the running server)
|
||||
devplace backups prune # remove backup records whose archive file is missing
|
||||
@ -138,6 +140,8 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
|
||||
| `devplacepy/services/devii/CLAUDE.md` | Devii assistant: sessions/channels, scheduler, virtual tools, self-configured behavior, client browser tools |
|
||||
| `devplacepy/services/openai_gateway/CLAUDE.md` | AI gateway: `/openai/v1/*`, usage ledger, provider/model routing |
|
||||
| `devplacepy/services/jobs/CLAUDE.md` | Async job services: zip, fork, SEO diagnostics, SEO metadata, DeepSearch, AI Usage Analyzer |
|
||||
| `devplacepy/services/moderation/CLAUDE.md` | Trust and safety: the reportable-target registry, the content filter and its five choke points, the report queue and its atomic resolution, enforcement, consent, maturity, account deletion |
|
||||
| `devplacepy/services/acceptance/CLAUDE.md` | Acceptance convergence: the opt-in service that grants every policy agreement to every account that has not declined it, its invisibility contract and the ledger-as-decline-register rule |
|
||||
| `devplacepy/services/audit/CLAUDE.md` | Audit log: recorders, categories, retention |
|
||||
| `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download |
|
||||
| `devplacepy/services/telegram/CLAUDE.md` | Telegram bot bridge |
|
||||
@ -195,6 +199,7 @@ Routers in `devplacepy/routers/` are organised as a **directory tree that mirror
|
||||
| `/api` | devrant/ package - see `routers/devrant/CLAUDE.md` |
|
||||
| `/dbapi` | dbapi/ package, **primary-administrator-only, strictly READ-ONLY** - see `services/dbapi/CLAUDE.md` |
|
||||
| `/game` | game/ package - see `services/game/CLAUDE.md` |
|
||||
| `/reports`, `/admin/moderation`, `/workspaces` | reports.py, admin/moderation.py, workspaces.py - see `services/moderation/CLAUDE.md` |
|
||||
| `/quizzes` | quizzes/ package - see `services/quiz/CLAUDE.md` |
|
||||
| (none) | push.py (web push/PWA), docs.py (see `routers/docs/CLAUDE.md`) |
|
||||
| `/pubsub` | pubsub.py - see `services/pubsub/CLAUDE.md` |
|
||||
@ -261,22 +266,38 @@ Devii is also reachable over **Telegram** (one supervised long-poller subprocess
|
||||
|
||||
`devplacepy/seo.py` generates JSON-LD schemas (WebSite, BreadcrumbList, DiscussionForumPosting, ProfilePage, SoftwareApplication). Every router builds context via `base_seo_context(request, ...)`. Auth/messages/notifications are `noindex,nofollow`; profiles with fewer than 2 posts are `noindex,follow`. `/robots.txt` and `/sitemap.xml` are served by `routers/seo.py`. Full implementation map (template layer, heading hierarchy, slugs, related posts, performance, default OG image, SEO tests) is in `devplacepy/routers/CLAUDE.md`.
|
||||
|
||||
## The production database is never touched without explicit confirmation (hard rule)
|
||||
|
||||
`data/devplace.db` is the live production database, and `make dev`, `make prod` and the Docker stack all share it (see "Production deployment"). No agent-initiated command may read or write it, or anything else under `data/`, without the user's explicit, stated confirmation - not a one-click approval, a confirmation they wrote themselves after being told exactly what the command would do.
|
||||
|
||||
This is enforced, not remembered. `.claude/hooks/guard_production_db.py` runs as a `PreToolUse` hook on every Bash call and **denies** the command outright when it reaches production, naming the reason. The interesting case is the one that motivated the rule: a script that never mentions a path at all but imports `devplacepy` and therefore resolves `config.DATA_DIR` to the real database. The hook reads the script and decides on its content, so a scratch-database script passes and an unguarded one does not.
|
||||
|
||||
What the guard blocks: any command naming `data/devplace.db` or a production data directory, the `devplace` management CLI, `python -m devplacepy...`, and any inline `-c` or script file that imports `devplacepy` without a `DEVPLACE_DATABASE_URL` override. What stays free: `make test` and `pytest` (the suite runs on its own temp database), `make dev`/`make prod`/`make docker-*`, the mandated import gate `python -c "from devplacepy.main import app"`, and anything that sets `DEVPLACE_DATABASE_URL` to a scratch file. `permissions.deny` in `.claude/settings.json` additionally refuses `Write`/`Edit` anywhere under `data/`, which the Bash hook cannot see.
|
||||
|
||||
The escape hatch is deliberately two-factor and must never be self-served: after the user has confirmed in their own words, the command may carry the literal token `I-HAVE-CONFIRMED-PRODUCTION-DB-ACCESS`, which downgrades the denial to a permission prompt the user still has to approve. **Never add that token on your own initiative.** Write disposable scripts against a temp database via `DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR` instead, exactly as "Rigorous correctness verification" already requires.
|
||||
|
||||
## Conventions (project-specific)
|
||||
|
||||
- **No comments, no docstrings in source.** Code is self-documenting.
|
||||
- **Forbidden variable name patterns:** `_new`, `_old`, `_temp`, `_v2`, `better_`, `my_`, `the_`.
|
||||
- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: `routers/proxy.py` relays the user's own headers verbatim. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
|
||||
- **Outbound HTTP goes through the stealth client only.** Never instantiate a bare `httpx.AsyncClient(...)`/`httpx.Client(...)` for outbound traffic - build it via `devplacepy.stealth.stealth_async_client(...)`/`stealth_sync_client(...)`. The async factory routes bytes through **`curl_cffi` (curl-impersonate, BoringSSL)** behind an `httpx` transport adapter, so TLS JA3/JA4 and the HTTP/2 fingerprint match Chrome 146 byte-for-byte; falls back to a pure-httpx Chrome-aligned transport if `curl_cffi` is missing. Applies to news, gateway upstream, Devii/DeepSearch/SEO fetch, gitea, push, attachments. SSRF-guarded fetches via `net_guard.guarded_async_client(...)` get the fingerprint automatically. Sole exception: the reverse-proxy forwarding core `services/containers/forward.py` (and the three routers that call it - `routers/proxy.py`, `routers/tunnel.py`, the workspace editor route), which relays the user's own headers verbatim and therefore keeps a plain `httpx.AsyncClient`; bolting the Chrome identity onto it would overwrite the very headers it exists to forward. **Load-bearing gotcha:** cleartext `http://` is forced to HTTP/1.1 (`curl_transport.http_version_for`) - the Chrome-impersonation profile defaults to HTTP/2, but a plaintext origin has no TLS/ALPN to negotiate h2 and uvicorn (HTTP/1.1 only) returns `400 "Invalid HTTP request received."` once a cleartext HTTP/2 body crosses ~128KB. Every internal AI consumer calls `INTERNAL_GATEWAY_URL` = `http://localhost:<port>/openai/v1/...`, so without this override a large Devii turn (big system prompt + tool list) fails on every request. `https://` keeps the HTTP/2 default. `stealth_sync_client` stays pure-httpx Chrome-aligned (no curl_cffi sync adapter). **Never bolt a hand-written browser-header dict (UA/`sec-ch-ua`/`Sec-Fetch-*`) onto a `stealth_async_client`/`stealth_sync_client` call** - curl_cffi's impersonation already emits an internally-consistent Chrome 146 identity across TLS, UA, and Client Hints; a caller-supplied header set claiming a different Chrome version or platform creates a UA-vs-TLS-vs-Client-Hints mismatch that is a textbook anti-bot tripwire (this was a real bug in Devii's `fetch_url`/`http_request` tool, fixed by deleting its hardcoded Chrome-131 header block). `stealth.detect_consent_gate(html)` recognizes the common JS-redirect cookie-consent wall pattern (DPG Media's `nu.nl` and sibling sites, and generically any page gated behind a `decodeURIComponent(...)` callback referencing a "privacy gate"/`consent.js`) - reuse it wherever a fetched page's real content sits behind that gate rather than special-casing a domain. `stealth_transport`/`stealth_async_client`/`stealth_sync_client` all accept a `proxy=` override, defaulting to `stealth.configured_proxy_url()` (the admin-editable `outbound_proxy_url` site setting, falling back to `DEVPLACE_OUTBOUND_PROXY_URL` when the setting is unset) - this is the only real fix for a destination that blocks by IP/ASN reputation (e.g. reddit.com returns an identical "blocked by network security" page across every TLS/header fingerprint tried, confirmed by testing curl_cffi's full impersonation list - it is blocking the datacenter ASN outright, not fingerprinting the client).
|
||||
- **Form validation is Pydantic-native.** Routers declare `data: Annotated[SomeForm, Form()]` (models in `models.py`); the global `RequestValidationError` handler re-renders auth pages with messages (400) or redirects other routes. Read raw `await request.form()` only when also reading an uploaded file alongside a model.
|
||||
- **`RedirectResponse(url=..., status_code=302)`** - always pass `status_code` as an integer literal, never `status.HTTP_302_FOUND`.
|
||||
- **Ownership checks:** compare `resource["user_uid"] == user["uid"]` before edit/delete (use `content.is_owner`). **Deletes additionally allow any admin:** `is_owner(...) or is_admin(user)` on every content delete endpoint, so an administrator may delete any member's content while members are limited to their own. Edits stay owner-only. Deletes are **soft** and cascade under one shared stamp - see `devplacepy/database/CLAUDE.md`.
|
||||
- **All dates shown to users are DD/MM/YYYY** (European), rendered in the viewer's own timezone client-side. Timestamps are stored/emitted as UTC ISO. Use the `local_dt(iso, mode)`/`dt_ago(iso)` Jinja globals for any user-facing instant - they emit `<time data-dt>` and `static/js/LocalTime.js` reformats to local timezone with a `MutationObserver` for dynamic content. `format_date()`/`time_ago()` stay as plain-text helpers for JSON responses, no-JS fallbacks, and non-timestamp date fields (e.g. project `release_date`) - do NOT wrap those in `local_dt`.
|
||||
- **Slug + UUID lookup:** resources with slugs accept either the slug or the bare UUID via `resolve_by_slug()`. Slugs are `make_combined_slug(title, uid)`, prefixed with the **random tail** of the UUID (never the leading bytes - same timestamp-collision reasoning as blob sharding).
|
||||
- **Roles are stored capitalized:** `users.role` is exactly `"Admin"` or `"Member"`. Always test admin-ness through the `is_admin(user)` global (case-sensitive `== "Admin"`) - never hand-roll a lowercase compare. **Any write to `users.role` MUST call `database.invalidate_admins_cache()`.**
|
||||
- **Admin seniority: a junior admin cannot manage a more senior admin.** Every per-user mutation in `routers/admin/users.py` is gated by `_is_senior_admin(actor, target)` - blocks (audits `result="denied"`) when the target is an Admin who registered earlier. Server-side, so it also covers Devii's admin tools.
|
||||
- **Admin seniority: a junior admin cannot manage a more senior admin.** Every per-user mutation in `routers/admin/users.py` and `routers/admin/moderation.py` is gated by `is_senior_admin(actor, target)` (`routers/admin/_shared.py`) - blocks (audits `result="denied"`) when the target is an Admin who registered earlier. Server-side, so it also covers Devii's admin tools.
|
||||
- **Never pass a `respond()` context key that collides with a Jinja global.** `respond(request, template, ctx, model=XOut)` feeds the same `ctx` to both the Pydantic model (JSON) and the template. A key like `is_admin`/`avatar_url`/`is_self` holding a non-callable value shadows the global across the whole inheritance chain, turning `{% if is_admin(user) %}` into `False(user)` -> `TypeError`, a 500 that fires only for the branch that calls the global. Name viewer/permission flags distinctly (`viewer_is_admin`, not `is_admin`) in both schema and context.
|
||||
- **Project visibility (`is_private`) and read-only (`read_only`)** are owner-controlled flags on `projects`. Read access is gated by the single `content.can_view_project(project, user)` predicate at EVERY read surface - never re-implement the check inline. Predicate: `not is_private OR is_owner OR (is_admin AND owner is not an admin)` - a project hidden by a member stays visible to any admin, but one hidden by an admin is visible only to that owner admin. **Containers have their own, stricter isolation predicates** (`owns_instance`, `can_view_project_containers`, `can_view_instance`, `can_manage_instance`) - the primary administrator sees/manages every container; any other admin can VIEW others' containers only on public projects and can MANAGE only instances they own. Read-only is enforced as a single data-layer guard `project_files._guard_writable(project_uid)` at the top of every mutation entrypoint - add it to any NEW file-mutating function. Devii may flip read-only/visibility only after explicit confirmation (`CONFIRM_REQUIRED`). Full UI-level detail in `devplacepy/routers/projects/CLAUDE.md`.
|
||||
- **Deletions ALWAYS require confirmation:** `confirmation_error` gates EVERY content delete tool via `CONFIRM_REQUIRED` (`delete_post`, `delete_comment`, `delete_gist`, `delete_project`, `project_delete_file`, `delete_media`, `delete_attachment`, `admin_delete_news`, container delete, and any `container_exec` matching `dispatcher.DESTRUCTIVE_COMMAND`). The first call is refused; the agent must show the exact target then pass `confirm=true`. **Load-bearing: every confirmation-gated tool MUST also declare a `confirm` boolean param in its catalog spec** - schemas set `additionalProperties: false`, so a gated tool without a declared `confirm` param can never receive it and loops forever.
|
||||
|
||||
## Every user-generated surface is reportable by construction (hard rule)
|
||||
|
||||
A new content table added to `SOFT_DELETE_TABLES` that is visible to anyone other than its author MUST be registered in `database/moderation.py` `REPORTABLE_TARGETS`, MUST resolve in `resolve_object_url`, and MUST include `_report_button.html` in its action bar. A table that is genuinely private to its owner goes in `UNREPORTABLE_TABLES` **with its reason** instead. `tests/unit/database/moderation.py` computes the difference and fails the suite on anything unclassified, so report coverage is closed under future additions rather than remembered; the e2e coverage test enforces the third requirement.
|
||||
|
||||
The same rule keeps the untriggered app-store conditionals untriggered: **no social login, no payment path, no purchasable randomness, no advertising, and no cross-app tracking** may be introduced without also implementing the obligations each of them creates (Sign in with Apple, in-app purchase, odds disclosure, ad reporting, App Tracking Transparency). Full detail in `devplacepy/services/moderation/CLAUDE.md`.
|
||||
|
||||
## Modal pattern
|
||||
|
||||
`Application.js` `initModals()` toggles a `.visible` CSS class on `.modal-overlay`; the CSS rule `.modal-overlay.visible { display: flex; }` handles visibility. Triggers usually have `href="#"`, so call `e.preventDefault()`. `.modal-close` is wired generically - no inline JS needed. Full modal/partial/CDN detail in `devplacepy/templates/CLAUDE.md`.
|
||||
@ -334,9 +355,10 @@ A new public read almost always needs all four. The cardinal failure mode is cha
|
||||
2. **Data layer.** `database/` for query/batch helpers (never inline N+1 loops - use `get_users_by_uids`, `build_pagination`, `_in_clause`; guard raw SQL with `if "table" in db.tables`). `models.py` for the Pydantic `Form` input model. `schemas/` for the `*Out` JSON response model - every context key a JSON route exposes via `respond(..., model=XOut)` MUST exist on `XOut` or it is silently dropped. Schema auto-syncs via `dataset`; add indexes in `init_db()` with `CREATE INDEX IF NOT EXISTS`.
|
||||
3. **Server layer.** Handler with the right auth guard (`get_current_user` public read, `require_user` member POST, `require_admin` admin). Specific paths (`/{username}/followers`) declared before catch-alls (`/{username}`). Return HTML+JSON via `respond(request, template, ctx, model=XOut)` or pure JSON via `JSONResponse`. Register any NEW router in `main.py`.
|
||||
4. **View layer.** Extend `base.html`; page CSS in `{% block extra_head %}`, page JS in `{% block extra_js %}`. Reuse partials and template globals. JS is ES6 modules, one class per file, instantiated on `app`.
|
||||
5. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog.py` - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
|
||||
6. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
|
||||
7. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.
|
||||
5. **Safety layer (skip only when the feature stores nothing and shows nothing).** If the feature adds a surface that carries user-generated content, register it in `database/moderation.py` `REPORTABLE_TARGETS` (or in `UNREPORTABLE_TABLES` with a reason), resolve it in `resolve_object_url`, and include `_report_button.html` in its action bar - the registry test and the e2e coverage test both fail otherwise. If it collects or transmits a new *category* of personal data, or sends anything new to a third party, update `/docs/privacy.html` and the app-store privacy declarations in the same change; that disclosure is part of the feature, not a follow-up.
|
||||
6. **Agent + docs layer (the most-forgotten step - do not skip).** `services/devii/actions/catalog.py` - add an `Action` tool if a user could ask Devii to do this (`requires_auth=False` for public reads). `docs_api.py` - every public/auth endpoint gets an `endpoint()` entry with params and a `sample_response`. `routers/docs/pages.py` `DOCS_PAGES` for prose pages. SEO (`seo.py`) - `base_seo_context` and any JSON-LD for a new public page; add to `routers/seo.py` sitemap if indexable.
|
||||
7. **Document.** `README.md` (product-facing, routes/config/dependencies/user-visible features), the relevant nested `CLAUDE.md` (any new mechanic, helper, pitfall, or cross-layer wiring), this root `CLAUDE.md` only when a NEW architectural rule or workflow step is introduced (not per feature).
|
||||
8. **Validate.** Confirm a clean import (`python -c "from devplacepy.main import app"`), check each touched language manually, grep for em-dash in touched files (neither the character nor its HTML entity - use a hyphen), and run `pyflakes`/`ruff check` on every touched file. **If the feature touches a spendable resource, a bounded state machine, or a read-then-write mutation reachable from more than one request path, also run the four-layer procedure in "Rigorous correctness verification" above** - it is the default for that shape of feature, not an optional extra. **Finish by running the full persisted test suite (`make test` - every tier, every test) and fixing any failure before the change is done.** Write new tests in the matching tier/path, following the required patterns above.
|
||||
|
||||
Failures at any implementation step block the workflow - never skip a failed step.
|
||||
|
||||
@ -344,6 +366,53 @@ Failures at any implementation step block the workflow - never skip a failed ste
|
||||
|
||||
Gitea Actions workflow at `.gitea/workflows/test.yaml` runs on every push/PR to `master`: installs dependencies + Playwright Chromium, runs the full suite serially under coverage, publishes coverage HTML as an artifact, uploads failure screenshots. CI must be green before merging. Changes move through DTAP: Development (`make dev`) -> Test (CI suite + coverage on `master`) -> Acceptance (`master` to `production` promotion via `make deploy`) -> Production (Docker Compose stack). Only CI-green `master` commits are promoted to `production`.
|
||||
|
||||
## Diagnosing a production failure (the order that finds it fastest)
|
||||
|
||||
This procedure exists because a single "the editor is down" report turned out to be **three unrelated faults stacked on each other** (a stale URL, a firewalled network leg, and a corrupt database), and the investigation wasted hours by guessing before measuring. Work the layers outward from the browser; each step is cheap and each one eliminates a whole class of cause. **Never skip to a hypothesis, and never repair anything before the layer above it is proven healthy.**
|
||||
|
||||
**Layer 0 - is the request even arriving here?** Fetch the hostname over the public internet exactly as it resolves (`curl -sS -o /dev/null -w '%{http_code} %{remote_ip}' https://host/`). Compare the answering IP against this machine's own addresses (`ip -6 addr`, `curl https://api.ipify.org`). Two hostnames serve this platform by different routes - see the topology section above. **Never use `curl --resolve` to force a hostname onto an IP it does not resolve to**; that fabricates a path that no real traffic takes and produces confident, wrong conclusions.
|
||||
|
||||
**Layer 1 - which edge answered?** The error body identifies it. `application/problem+json` with `No site configured for host` is molohttp. A DevPlace HTML error page is the application. An nginx error page is the nginx container. A browser `ERR_*` with no body means nothing well-formed was returned at all.
|
||||
|
||||
**Layer 2 - same failure on both hostnames?** Run the identical authenticated request against `pravda.education` and `devplace.net`. Failing on **both** means the application or the database; failing on **one** means that host's edge. This single comparison is the highest-value measurement available and costs one command.
|
||||
|
||||
**Layer 3 - the application log, before any theory.** `docker logs --since 5m devplace-app-1`. Count error classes rather than reading prose (`grep -c malformed`). A recurring service-loop error is a systemic fault even when it looks unrelated to the symptom.
|
||||
|
||||
**Layer 4 - reproduce the failing hop in isolation.** Point the real code at the real upstream from a scratch harness rather than reasoning about it. Running `forward.proxy_http` against a live code-server is what exposed the duplicate `Date` header; reading the function had not. Use a scratch database (`DEVPLACE_DATABASE_URL`) so the harness never reaches production.
|
||||
|
||||
**Layer 5 - test from where the code actually runs.** The app runs **inside a container**; `127.0.0.1` there is not the host. `docker exec devplace-app-1 curl ...` is the only honest reachability test for a container-to-container hop. A hang with zero bytes means a packet was **DROPped** (firewall), a refusal means nothing is listening, and a slow error means the upstream answered badly - three different causes with three different fixes.
|
||||
|
||||
**Layer 6 - confirm the object exists before blaming the plumbing.** A 404 from a guard is not a proxy failure. Resolve the identifier through the application's own read surface (the workspace page, an admin JSON endpoint) with the affected account's session. A stale instance uid in a bookmarked URL looks exactly like an outage.
|
||||
|
||||
### Rules learned the hard way
|
||||
|
||||
- **State what a command will read or write before running it against production, and keep production access read-only until the diagnosis is complete.** The one write in a repair is the final swap, and it comes after verification, not before.
|
||||
- **Copy before repairing, and copy the whole set.** A WAL-mode SQLite database is `.db` **plus** `-wal` **plus** `-shm`; a `.db`-only copy silently discards every transaction still in the WAL. Stop writes first, or the snapshot is inconsistent. Never leave a stale `-wal` beside a recovered file - SQLite will replay it and re-corrupt the result.
|
||||
- **Repair on a copy, verify on the copy, and prove what was preserved.** `PRAGMA integrity_check` names the damaged objects; index damage is derived data and costs nothing (`REINDEX`, or `.recover`), while a table b-tree fault is the only kind that can lose rows. Diff row counts table by table between the original and the recovered file and report the delta - "it says ok" is not evidence that data survived.
|
||||
- **Verify the fix through the user's own path, with their account, in a real browser.** A green unit test and a 200 from `curl` did not prove the editor worked; driving Playwright through login, the code-server password prompt and a `.monaco-workbench` selector did.
|
||||
- **A measurement recorded in these files can go stale.** `services/containers/CLAUDE.md` recorded that `container_ip:port` times out from the app container while `gateway:published_host_port` connects. A later change (`make docker-attach`) inverted it, and a host firewall closed the documented leg entirely. Re-measure before trusting a recorded measurement, and update the record when it turns out to be false.
|
||||
- **Report each fault separately and correct yourself explicitly.** Three stacked faults produce a symptom that no single explanation covers, and an early wrong theory is worse than no theory once it is repeated as fact.
|
||||
|
||||
## Production hostnames and the devplace.net SSH tunnel (verified topology, do not re-derive)
|
||||
|
||||
**The platform answers on two public hostnames, and they reach the same application by two completely different paths.** This has already cost one debugging session; the failure mode is that a `curl --resolve devplace.net:443:<production ip>` "test" reports `No site configured for host: devplace.net` and looks like a total outage, when in fact devplace.net never touches the production edge at all.
|
||||
|
||||
| | `pravda.education` | `devplace.net` |
|
||||
|---|---|---|
|
||||
| DNS | `95.216.15.238`, `2a01:4f9:2a:100e::2` | `88.198.21.243`, `2a01:4f8:222:2c45::2` |
|
||||
| Machine | the production host itself | a separate front host (Hetzner, PTR `static.88-198-21-243.clients.your-server.de`) |
|
||||
| Path in | molohttp on `:443` -> `127.0.0.1:10500` | its own proxy -> **SSH tunnel** -> `127.0.0.1:10500` on production |
|
||||
| Reaches molohttp | yes | **no, never** |
|
||||
|
||||
**`devplace.net` is a front host that forwards over SSH.** It holds a persistent SSH session into the production host (visible there as an established inbound connection from `88.198.21.243` to port 22) and forwards through it to `127.0.0.1:10500`, which is the `docker-proxy` for the `devplace-nginx` container. The listening socket lives on the **front** host (an `ssh -L` style local forward), so the production host shows **no** sshd-owned listener - that absence is expected and is not evidence against the tunnel.
|
||||
|
||||
Two consequences that must not be forgotten:
|
||||
|
||||
- **molohttp has no `devplace.net` site, and that is correct.** Its site list is `mail`/`smtp`/`imap.molodetz.nl`, `pravda.education` and `*.tunnel.pravda.education`. devplace.net traffic enters below molohttp, straight into `127.0.0.1:10500`, so it needs no site. **Never "fix" this by adding a devplace.net site to molohttp** - devplace.net does not resolve to the production host, so such a site could never match, and its absence is not a bug.
|
||||
- **Both hostnames land on the same nginx and the same app**, so a request that fails on both is failing in the application, not in either edge. That comparison is the fastest triage available here: run the same authenticated request against both hostnames. Same failure on both means look at the app or the database; a failure only on devplace.net means look at the front host's proxy (WebSocket `Upgrade` headers are the usual culprit, exactly as for the production nginx locations below).
|
||||
|
||||
**Testing rule.** Never point a hostname at an IP it does not resolve to in order to "test" it. Fetch each hostname over the public internet as it really resolves (`curl https://devplace.net/...` and `curl https://pravda.education/...`), because forcing devplace.net onto the production IP tests molohttp with a `Host` it deliberately does not serve and proves nothing about the real path.
|
||||
|
||||
## Production deployment
|
||||
|
||||
Single-host Docker Compose (`docker-compose.yml`): an **app** container (Uvicorn, 2 workers) behind an **nginx** container. Full operator reference is the admin-only `Production` docs section (`templates/docs/production*.html`); conventions that must not regress:
|
||||
|
||||
16
Makefile
16
Makefile
@ -135,10 +135,11 @@ test-cache-clean:
|
||||
COMPOSE := docker compose -f docker-compose.yml -f docker-compose.containers.yml
|
||||
DEVPLACE_DATA_DIR ?= $(CURDIR)/data
|
||||
DOCKER_GID ?= $(shell stat -c '%g' /var/run/docker.sock 2>/dev/null)
|
||||
DEVPLACE_CONTAINER_NETWORK ?= bridge
|
||||
export DEVPLACE_DATA_DIR
|
||||
export DOCKER_GID
|
||||
|
||||
.PHONY: docker-build docker-up docker-reload docker-down docker-logs docker-clean docker-prep ppy
|
||||
.PHONY: docker-build docker-up docker-attach docker-reload docker-down docker-logs docker-clean docker-prep ppy
|
||||
|
||||
# Build the single shared container image every instance runs. Build once;
|
||||
# rebuild only when ppy.Dockerfile, the sudo shim, or pagent change.
|
||||
@ -153,10 +154,23 @@ docker-build: docker-prep
|
||||
|
||||
docker-up: docker-prep
|
||||
$(COMPOSE) up -d
|
||||
$(MAKE) docker-attach
|
||||
|
||||
# Workspace tunnels reach a container port that was never published on the host,
|
||||
# so the app must sit on the same docker network as the instances it runs. The
|
||||
# default bridge rejects the network-scoped aliases compose always sends, so
|
||||
# this cannot live in docker-compose.containers.yml and is wired here instead.
|
||||
docker-attach:
|
||||
@app=$$($(COMPOSE) ps -q app); \
|
||||
test -n "$$app" || { echo "app container is not running"; exit 1; }; \
|
||||
docker network connect $(DEVPLACE_CONTAINER_NETWORK) $$app 2>/dev/null \
|
||||
&& echo "attached app to the $(DEVPLACE_CONTAINER_NETWORK) network" \
|
||||
|| echo "app is already on the $(DEVPLACE_CONTAINER_NETWORK) network"
|
||||
|
||||
docker-reload:
|
||||
$(COMPOSE) restart app
|
||||
$(COMPOSE) up -d --wait
|
||||
$(MAKE) docker-attach
|
||||
|
||||
docker-down:
|
||||
$(COMPOSE) down
|
||||
|
||||
75
README.md
75
README.md
@ -15,6 +15,8 @@ make test-headed # same tests in visible browser
|
||||
|
||||
Open `http://localhost:10500`.
|
||||
|
||||
A measured structural overview of the whole application, fifteen Mermaid diagrams covering deployment, request pipeline, URL surface, data layer, AI plane, containers and the real-time plane, is in [ARCHITECTURE.md](ARCHITECTURE.md).
|
||||
|
||||
PDF export (DeepSearch reports, via weasyprint) needs the Pango text stack installed at system level. The production image installs it; on a development host install it once:
|
||||
|
||||
```bash
|
||||
@ -66,6 +68,7 @@ devplacepy/
|
||||
| `/gists` | Code gist listing, detail, creation, and editing; left panel offers language filtering and free-text `search` (title, description, and author username), public read |
|
||||
| `/comments` | Comment creation, owner editing (`POST /comments/edit/{comment_uid}`), deletion |
|
||||
| `/projects` | Project listing (left panel offers type filtering and free-text `search` over title, description, and author username), creation, owner editing (`POST /projects/edit/{slug}`), and per-project visibility toggles: `POST /projects/{slug}/private` (owner-only visibility) and `POST /projects/{slug}/readonly` (immutable files). A project hidden by a member stays visible to administrators, but a project hidden by an administrator is visible only to that owner administrator - other administrators cannot see it, its files, or its containers (web UI and REST API alike). The primary administrator (the first Admin account) is the single exception and retains full visibility |
|
||||
| `/projects/{slug}` | Dedicated project page: one encompassing card with a cover banner and project logo (owner-uploaded through the standard attachment uploader), the title overlaid on the banner, status/type/platform chips, owner-set Website and Repository links, section tabs (Overview, Devlog, Screenshots, Comments, Files), an About section, the Devlog timeline of every post linked to the project (owners post updates straight from the page via the shared composer preset to the `devlog` topic), a Screenshots gallery built from image attachments (owners add more from the More menu), and a sidebar with links, stats and the author card |
|
||||
| `/projects/{slug}/files` | Per-project filesystem: directory and file CRUD, upload, inline editing, and line-range operations (`lines` read, `replace-lines`, `insert-lines`, `delete-lines`, `append`) for surgical edits to large text files (public read, owner write; all writes refused while the project is read-only) |
|
||||
| `/zips` | Zip job status (`/zips/{uid}`) and archive download (`/zips/{uid}/download`); archives are queued via `/projects/{slug}/zip` and `/projects/{slug}/files/zip` |
|
||||
| `/forks` | Fork job status (`/forks/{uid}`); forks are queued via `/projects/{slug}/fork`. Any signed-in user can fork a project they can view into a new project they own; once the job finishes the response carries the new project URL |
|
||||
@ -85,7 +88,11 @@ devplacepy/
|
||||
| `/bookmarks` | Save/unsave content; `/bookmarks/saved` personal list |
|
||||
| `/polls` | Vote on post-attached polls |
|
||||
| `/follow` | Follow/unfollow users |
|
||||
| `/block` | Block/unblock a user: hides all of their posts, comments and messages from you everywhere except their own profile, and stops them notifying you |
|
||||
| `/reports` | Content reporting: `POST /reports/{target_type}/{target_uid}` files a report against any user-generated surface, `GET /reports/mine` lists the reports you filed and their outcome, `GET /reports/reasons` serves the reason registry so every client renders the same dialog |
|
||||
| `/admin/moderation` | Admin **Moderation** queue: reported content oldest-open-first with the response-window badge, one report per detail page with the offender's history, triage (`/status`) and decisions (`/decide`) |
|
||||
| `/workspaces/index` | Public index of every workspace published to the ingress proxy, with owner, project, maturity label and direct link |
|
||||
| `/projects/{slug}/workspace` | A member's dev workspace for a project: open/start/stop/delete, quota and idle status, public tunnels, and the **Editor** card holding their editor preferences (`GET`/`POST /projects/{slug}/workspace/editor`) |
|
||||
| `/block` | Block/unblock a user: hides all of their posts, comments and messages from you everywhere except their own profile, and stops them notifying you. Also reachable directly from every content action bar |
|
||||
| `/mute` | Mute/unmute a user: stops them creating notifications for you while their content stays visible |
|
||||
| `/leaderboard` | Contributor ranking by total stars earned |
|
||||
| `/game` | **Code Farm** cooperative idle game (member-only): plant projects that build over real time, harvest coins and XP, upgrade CI, buy plots, and water friends' builds at `/game/farm/{username}`. Live over pub/sub; every endpoint negotiates JSON |
|
||||
@ -201,6 +208,20 @@ The farm refreshes live over the pub/sub bus (a watered build appears on the own
|
||||
|
||||
XP awards are wired at the existing content-creation, vote, and follow hook points in the routers and centralized in `award_xp()` / `check_milestone_badges()` (`devplacepy/utils/`). Existing accounts have their XP and levels backfilled once from prior activity at startup (`init_db()`).
|
||||
|
||||
## Trust and safety
|
||||
|
||||
DevPlace is open in the sense that it does not editorialise technical opinion. It enforces a short, fixed list of prohibited categories, published as the [Community Guidelines](/docs/community-guidelines.html), and it has the machinery to make that enforcement real.
|
||||
|
||||
- **Automated filtering at post time.** Every user-authored text passes a classifier at the five places content is created or changed (content creation, content editing, comments, direct messages, profile and signup fields), covering every surface with no per-route code. The default mode is `review`, not `block`: a match is published **and** raises a report for a human, because a developer platform discusses exploits, malware analysis and violent subject matter as its work and a machine that suppressed that would destroy the product. Only sexual and exploitative content is refused outright. Classification failure falls back to review, never to publication. Tunable live at `/admin/settings` (`moderation_filter_mode`: `off`, `label`, `review`, `block`).
|
||||
- **A report control on every surface.** Posts, comments, gists, projects, project files, news, uploads, direct messages, quizzes, polls, awards, profiles, issues, workspaces and assistant output are all reportable through one polymorphic endpoint keyed on `(target_type, target_uid)`, from one dialog, with one reason list. Reports are private to the reporter; the reported person is never told who filed.
|
||||
- **One queue with a published response window.** `/admin/moderation` is worked oldest-open-first and carries a badge showing the age of the oldest unresolved report against `moderation_sla_hours` (default 24), so a breach is visible rather than assumed. Resolution is a single atomic conditional update: two administrators deciding at once produce exactly one decision, the second gets a 409.
|
||||
- **Real enforcement, with a statement of reasons.** Remove or restore content, warn, suspend for a stated period, ban, lift, dismiss, or escalate. Every decision writes a permanent `moderation_actions` row plus an audit event, and tells the affected user what was decided and why. A suspended account can still read, still see why, still report, and still delete itself; it cannot create. A junior administrator can never action a senior one.
|
||||
- **Blocking**, unchanged and independent of all of the above, is now reachable from the content itself as well as from a profile.
|
||||
- **Age and maturity.** Signup collects a date of birth, derives an age band, and **discards the date**; accounts below `moderation_minimum_age` (default 16) are refused. Content labelled mature is hidden behind an interstitial until the viewer explicitly opts in, and the reveal is never offered to a minor age band for restricted content.
|
||||
- **Consent.** Five versioned, independently withdrawable consents (`terms`, `privacy`, `ai_third_party`, `activity_recording`, `container_credentials`) with full history, managed at `/profile/{username}?tab=privacy` and changeable **only by the account holder** - an administrator reads the record but never grants or withdraws on someone else's behalf. **No content is sent to a third-party AI provider without `ai_third_party` consent**, enforced once at the gateway; the per-feature AI toggles remain as preferences subordinate to it. Withdrawing `activity_recording` stops presence recording. While recording is on, an indicator says so on every page. `container_credentials` is what lets software another member runs in a container receive your API key; running your own container never asks.
|
||||
- **Account deletion.** Self-service at `/profile/{username}/delete`, reauthenticated with the account password. Sessions and tokens are revoked, the profile is anonymised immediately, and all content is removed under one deletion event that stays restorable for `account_deletion_grace_hours` (default 24) before `devplace accounts prune` (and the Moderation housekeeping service) purges it permanently. The devRant `DELETE /api/users/me` routes into the same cascade.
|
||||
- **Legal pages**: [Terms of Service](/docs/terms.html), [Community Guidelines](/docs/community-guidelines.html), [Privacy Policy](/docs/privacy.html), [How moderation works](/docs/content-moderation.html), [Notice and takedown](/docs/intellectual-property.html) and [Contact](/docs/contact.html). All six are indexed in the sitemap and reachable from the docs index; the footer of every page links Terms, Privacy, Community Guidelines and Contact. Contact details come from the `contact_email`, `contact_phone` and `contact_address` settings, so the in-product page and any app-store trader declaration have one source of truth.
|
||||
|
||||
## Vibe coding (Alpha, admin only)
|
||||
|
||||
Build software by talking to an AI agent instead of typing every line. Create a project for storage, attach a container to it (the shared `ppy` image, your files mounted at `/app`), start it, and open a terminal. The whole flow is drivable conversationally through Devii. Inside every container three agents ship preinstalled and run on **your own API key**, so all AI usage is metered to your account: **DevPlace Code (`dpc`)**, a coding agent in the same class as Claude Code; **`botje.py`**, a plug-and-play DevPlace bot you can copy and customise; and **`pagent`**, a minimal zero-dependency agent. Each container is launched with `DEVPLACE_BASE_URL`, `DEVPLACE_OPENAI_URL`, `DEVPLACE_API_KEY`, `DEVPLACE_USER_UID`, `DEVPLACE_CONTAINER_NAME`, `DEVPLACE_CONTAINER_UID`, and `DEVPLACE_INGRESS_URL` already set. Publish a container port to a public URL at `/p/<slug>` by setting an `ingress_slug` and `ingress_port` (ask Devii to do it at create time). The feature is in **Alpha** and currently limited to administrators; the full walkthrough, including a tutorial that vibes a web app and puts it online, is at `/docs/getting-started-vibing.html`.
|
||||
@ -415,15 +436,61 @@ and its full configuration are documented automatically - including future servi
|
||||
- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected
|
||||
- **`JobService` / `ZipService` / `ForkService`** - generic async job framework (`services/jobs/`) for heavy, blocking work run off the request path; `ZipService` builds project zip archives in a subprocess, `ForkService` copies a project into a new project owned by the forking user
|
||||
- **`ContainerService`** - the admin container manager (`services/containers/`): a reconciling supervisor for container instances, all running one shared prebuilt image
|
||||
- **`AcceptanceService`** - grants every policy agreement (Terms of Service, Privacy Policy, third-party AI processing, activity recording, container credentials) to every account that has not declined it, so an instance kept production-identical for extended manual testing never interrupts with an acceptance dialog. Administrator-only, **off by default**, with a separate switch per agreement type and a dry run that reports what it would do without writing. An account that withdrew a consent is never granted it again, with no further action: the consent ledger itself is the decline register. It is not appropriate on a real production host
|
||||
|
||||
### Container manager (admin only)
|
||||
|
||||
`services/containers/` runs supervised container instances from the web UI, the HTTP API, and Devii. It drives the `docker` CLI via async subprocesses behind a pluggable `Backend` interface (a `DockerCliBackend` plus a `FakeBackend` for tests). **There is no in-app image building.** Every instance runs ONE shared prebuilt image, `ppy:latest` (override `DEVPLACE_CONTAINER_IMAGE`), built once with **`make ppy`** from `ppy.Dockerfile`: a Python + Playwright base with a broad set of common libraries preinstalled, the `pravda` (uid 1000) user, the sudo superclone, and `pagent` at `/usr/bin/pagent.py` all baked in. Creating an instance is then an instant `docker run` (it fails fast with a clear error if the `ppy` image has not been built yet). `ContainerService` reconciles desired instance state against `docker ps` each tick (containers are labeled `devplace.instance=<uid>`, so orphans are reaped and no state is lost), applies restart policies, fires cron/interval/one-time schedules, and samples metrics. The container's `/app` is bind-mounted to a persistent project workspace (materialized from the project files) and **stays in sync automatically**: the manager runs a bidirectional, newer-wins sync between the project files and the workspace on every start and roughly once a minute while running, so edits made inside the container and edits made in the project file editor converge without manual intervention (the sync only ever creates or overwrites the older copy of a file, never deletes one; a read-only project is export-only). Projects that need extra packages use runtime `pip install` (pravda owns the site-packages, no sudo needed) or `apt install` directly (the `pravda` user runs `apt`/`dpkg` through a fakeroot wrapper, so system packages install without root) or add the library to `ppy.Dockerfile` and rerun `make ppy`. Each instance can run a **boot script** in Python or Bash (written into the workspace and run on launch) or a plain boot command, can be set to **start automatically** whenever the container service starts, and can be configured to **run as** a chosen DevPlace user - which only selects whose identity and API key are injected into the container (the container always runs as the unprivileged `pravda` user). Every status change is recorded and shown as a status history on the instance page. A running instance can be **published** with an `ingress_slug`, making its service reachable (HTTP and WebSocket) at `/p/<slug>` through DevPlace. The manager is reached two ways: the admin **Containers** sidebar entry (`/admin/containers`) lists, creates, edits, and controls every instance across all projects and opens a dedicated detail page per instance, and each project page carries an admin-only **Containers** button to its own instance manager.
|
||||
`services/containers/` runs supervised container instances from the web UI, the HTTP API, and Devii. It drives the `docker` CLI via async subprocesses behind a pluggable `Backend` interface (a `DockerCliBackend` plus a `FakeBackend` for tests). **There is no in-app image building.** Every instance runs ONE shared prebuilt image, `ppy:latest` (override `DEVPLACE_CONTAINER_IMAGE`), built once with **`make ppy`** from `ppy.Dockerfile`: a Python + Playwright base with a broad set of common libraries preinstalled, the `pravda` (uid 1000) user, the sudo superclone, `pagent` at `/usr/bin/pagent.py`, and the `code-server` browser IDE that backs dev workspaces all baked in. Creating an instance is then an instant `docker run` (it fails fast with a clear error if the `ppy` image has not been built yet). `ContainerService` reconciles desired instance state against `docker ps` each tick (containers are labeled `devplace.instance=<uid>`, so orphans are reaped and no state is lost), applies restart policies, fires cron/interval/one-time schedules, and samples metrics. The container's `/app` is bind-mounted to a persistent project workspace (materialized from the project files) and **stays in sync automatically**: the manager runs a bidirectional, newer-wins sync between the project files and the workspace on every start and roughly once a minute while running, so edits made inside the container and edits made in the project file editor converge without manual intervention (the sync only ever creates or overwrites the older copy of a file, never deletes one; a read-only project is export-only). Projects that need extra packages use runtime `pip install` (pravda owns the site-packages, no sudo needed) or `apt install` directly (the `pravda` user runs `apt`/`dpkg` through a fakeroot wrapper, so system packages install without root) or add the library to `ppy.Dockerfile` and rerun `make ppy`. Each instance can run a **boot script** in Python or Bash (written into the workspace and run on launch) or a plain boot command, can be set to **start automatically** whenever the container service starts, and can be configured to **run as** a chosen DevPlace user - which only selects whose identity and API key are injected into the container (the container always runs as the unprivileged `pravda` user). Every status change is recorded and shown as a status history on the instance page. A running instance can be **published** with an `ingress_slug`, making its service reachable (HTTP and WebSocket) at `/p/<slug>` through DevPlace. The manager is reached two ways: the admin **Containers** sidebar entry (`/admin/containers`) lists, creates, edits, and controls every instance across all projects and opens a dedicated detail page per instance, and each project page carries an admin-only **Containers** button to its own instance manager.
|
||||
|
||||
**Security:** this requires mounting the Docker socket, which grants host root. Every run, exec, lifecycle, and schedule operation is administrator-only; `--privileged` is never used and all docker calls are argument-list subprocesses. Containers are additionally isolated per user: an instance is managed only by its owner (the administrator who created it, or the owner of its project) and by the primary administrator, who alone sees and manages every instance including those on private projects; other administrators get a read-only view of instances on public projects and none of another user's private-project instances (exec, terminals, schedules, edits, and lifecycle actions are all refused and audited as denied). The service is disabled by default; an admin enables **Containers** on `/admin/services`. CLI: `devplace containers list | reconcile | prune | prune-builds | gc-workspaces` (`prune-builds` is a one-time cleanup that removes legacy per-project images and the old dockerfiles/builds tables).
|
||||
|
||||
**Runtime data** (container workspaces and zip archives) lives in `DEVPLACE_DATA_DIR` (default `data/`), **outside the package and never served via `/static`**. The docker daemon must be able to bind-mount the data dir for `/app`.
|
||||
|
||||
### Dev Workspaces and the browser editor
|
||||
|
||||
A **workspace** is a member-facing container running the DevPlace browser editor, layered on the
|
||||
container runtime above. It is opened from a project's **Workspace** page and reached at
|
||||
`/projects/{slug}/workspace`; the editor itself is proxied at
|
||||
`/projects/{slug}/containers/instances/{uid}/code/`, and an **Editor** button appears on the project
|
||||
page whenever the workspace is running.
|
||||
|
||||
The editor is `code-server`, rebranded as DevPlace end to end: the application name, the browser tab
|
||||
icon and PWA icons, the login page styling, and `product.json` all carry DevPlace, and a bundled
|
||||
built-in extension ships the **DevPlace Dark** and **DevPlace Light** themes (generated from the
|
||||
site's own design tokens), a **Get started on DevPlace** walkthrough, a project status bar item and
|
||||
five `DevPlace:` commands. Nothing in the interface identifies as code-server.
|
||||
|
||||
**On boot** two terminals open: a focused **DevPlace Code** terminal already running `dpc`, the
|
||||
coding agent baked into the image, and a plain login shell beside it with the Python, Rust, Nim and
|
||||
Swift toolchains on `PATH`. Both are configurable, and `bash` stays the default profile for
|
||||
terminals the member opens later.
|
||||
|
||||
The workspace opens straight onto the member's files rather than a welcome page, and the editor's
|
||||
own built-in chat assistant is suppressed so `dpc` is the only agent on offer and every token it
|
||||
spends is ledgered against the member's DevPlace account. `dpc`'s own working files (`.dpc/`,
|
||||
`dpc.log`) are in `SYNC_SKIP_NAMES`, so running an agent on every boot never pollutes the project.
|
||||
|
||||
**Every workspace is trusted.** VS Code Restricted Mode is disabled at the command line and in the
|
||||
seeded settings, so nothing prompts and automatic tasks run. This is a deliberate default with a
|
||||
real consequence (a project's own `.vscode/tasks.json` will run on folder open), it is documented to
|
||||
members on `/docs/workspace-editor.html`, and an administrator can restore Restricted Mode site-wide
|
||||
with the `workspace_editor_trust_all` setting.
|
||||
|
||||
**Four sizes are configurable through one resolver.** Editor and terminal font size plus zoom, the
|
||||
editor layout and terminal panel preset, whether the editor opens in a tab or a sized window, and the
|
||||
container's CPU, memory and disk. The first three are the member's own preferences on their workspace
|
||||
page (and over the API, and through Devii's `workspace_editor_get` / `workspace_editor_set`); the
|
||||
container size is part of the administrator-set workspace quota. Each preference resolves instance
|
||||
override, then the member's row, then the site setting, then the built-in default, and the page shows
|
||||
which of those each value came from.
|
||||
|
||||
**A member edit is never overwritten.** DevPlace seeds the editor's `settings.json` from the host
|
||||
before each launch and records exactly what it wrote; on the next launch it updates only the keys
|
||||
whose current value is still the one it wrote. A setting the member changed inside the editor is
|
||||
theirs permanently, while a change to the site default still reaches everyone who has expressed no
|
||||
preference. Preferences apply on the next workspace start, and the page says so and offers the
|
||||
restart.
|
||||
|
||||
### Async job framework and zip downloads
|
||||
|
||||
`services/jobs/` is the standard way to run blocking work asynchronously and hand the caller a result URL. A shared `jobs` table is the queue (discriminated by `kind`); `queue.enqueue()` inserts a `pending` row from any worker, the lock-owning worker processes jobs in `JobService.run_once` (reap, recover orphans, refill up to a concurrency limit, prune expired), and status is polled from the database. Retention is built in: each job service deletes its own expired artifacts via a `cleanup` hook (default 7 days, admin-configurable). To add a kind, subclass `JobService`, set `kind`, and implement `process()` and `cleanup()`.
|
||||
@ -1056,6 +1123,10 @@ What the overlay (`docker-compose.containers.yml`) changes:
|
||||
- **Data dir at a consistent path (critical).** When the app (in its container) runs `docker run -v <path>:/app`, the daemon resolves `<path>` against the **host**, not the app container. So the workspace/data dir must be mounted at the **same absolute path** on host and in the container - the make targets set `DEVPLACE_DATA_DIR` to the project's `./data` (an absolute host path) and mount it at that identical path on both sides. (Build contexts go through the docker API as a tarball, so they can stay in the container's temp dir - only the `/app` bind mount needs path consistency.)
|
||||
- **Ingress reach:** published container ports live on the **host**, so the overlay sets `DEVPLACE_CONTAINER_PROXY_HOST=host.docker.internal` (with `extra_hosts: host-gateway`) so the `/p/<slug>` proxy can reach them. On a bare-metal `make prod` deploy the app is already on the host, so the default `127.0.0.1` works and no overlay is needed (just install the docker CLI and run the services).
|
||||
|
||||
One piece of wiring cannot live in the overlay:
|
||||
|
||||
- **Workspace tunnel reach.** A workspace tunnel serves a port the member chose, which is almost never published on the host, so the app has to dial the container directly - and it can only do that from the container's own docker network. Compose cannot attach a service to the default `bridge` network (it always sends network-scoped aliases, which that network rejects), so `make docker-up` and `make docker-reload` run `make docker-attach`, an idempotent `docker network connect` of the app container to `DEVPLACE_CONTAINER_NETWORK` (default `bridge`). A bare `docker compose up -d` skips it and every tunnel to an unpublished port answers `502`. On a bare-metal `make prod` deploy the app is already on the host and reaches container IPs with no wiring at all.
|
||||
|
||||
Then build the shared `ppy` image once with `make ppy` and enable **Containers** on `/admin/services`. There is no in-app image building; every instance runs that one prebuilt image.
|
||||
|
||||
### nginx specifics
|
||||
|
||||
@ -28,6 +28,6 @@ def generate_avatar_svg(seed: str) -> str:
|
||||
initial = seed[:1].upper() if seed else "?"
|
||||
return (
|
||||
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100">'
|
||||
f'<rect width="100" height="100" rx="50" fill="#ff6b35"/>'
|
||||
f'<rect width="100" height="100" rx="50" fill="#b73f1e"/>'
|
||||
f'<text x="50" y="65" text-anchor="middle" fill="white" font-size="40" font-weight="700" font-family="sans-serif">{initial}</text></svg>'
|
||||
)
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
|
||||
from devplacepy.cli.main import main, build_parser
|
||||
from devplacepy.cli._shared import _audit_cli
|
||||
from devplacepy.cli.accounts import cmd_accounts_pending, cmd_accounts_prune
|
||||
from devplacepy.cli.roles import cmd_role_get, cmd_role_set
|
||||
from devplacepy.cli.apikeys import cmd_apikey_get, cmd_apikey_reset, cmd_apikey_backfill
|
||||
from devplacepy.cli.tokens import (
|
||||
@ -49,6 +50,8 @@ __all__ = [
|
||||
"main",
|
||||
"build_parser",
|
||||
"_audit_cli",
|
||||
"cmd_accounts_pending",
|
||||
"cmd_accounts_prune",
|
||||
"cmd_role_get",
|
||||
"cmd_role_set",
|
||||
"cmd_apikey_get",
|
||||
|
||||
46
devplacepy/cli/accounts.py
Normal file
46
devplacepy/cli/accounts.py
Normal file
@ -0,0 +1,46 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.cli._shared import _audit_cli
|
||||
|
||||
|
||||
def cmd_accounts_prune(args):
|
||||
from devplacepy.services.moderation import deletion
|
||||
|
||||
pending = deletion.due_purges()
|
||||
if args.dry_run:
|
||||
for row in pending:
|
||||
print(f"{row['uid']} deleted at {row['deletion_requested_at']}")
|
||||
print(f"{len(pending)} account(s) due for purge")
|
||||
return
|
||||
purged = deletion.purge_due()
|
||||
_audit_cli(
|
||||
"cli.accounts.prune",
|
||||
f"CLI purged {purged} deleted account(s) past the grace window",
|
||||
metadata={"count": purged},
|
||||
)
|
||||
print(f"Purged {purged} deleted account(s)")
|
||||
|
||||
|
||||
def cmd_accounts_pending(args):
|
||||
from devplacepy.services.moderation import deletion
|
||||
|
||||
pending = deletion.due_purges()
|
||||
for row in pending:
|
||||
print(f"{row['uid']}\t{row['deletion_requested_at']}")
|
||||
print(f"{len(pending)} account(s) past the {deletion.grace_hours()}h grace window")
|
||||
|
||||
|
||||
def register_accounts(subparsers):
|
||||
accounts = subparsers.add_parser("accounts", help="Deleted account management")
|
||||
accounts_sub = accounts.add_subparsers(title="action", dest="action")
|
||||
prune = accounts_sub.add_parser(
|
||||
"prune", help="Permanently purge accounts past the deletion grace window"
|
||||
)
|
||||
prune.add_argument(
|
||||
"--dry-run", action="store_true", help="List what would be purged and exit"
|
||||
)
|
||||
prune.set_defaults(func=cmd_accounts_prune)
|
||||
pending = accounts_sub.add_parser(
|
||||
"pending", help="List deleted accounts awaiting their purge"
|
||||
)
|
||||
pending.set_defaults(func=cmd_accounts_pending)
|
||||
@ -2,6 +2,7 @@
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from devplacepy.cli.accounts import register_accounts
|
||||
from devplacepy.cli.roles import register_roles
|
||||
from devplacepy.cli.apikeys import register_apikeys
|
||||
from devplacepy.cli.tokens import register_tokens
|
||||
@ -36,6 +37,7 @@ def build_parser():
|
||||
register_quiz(sub)
|
||||
register_gateway(sub)
|
||||
register_messaging(sub)
|
||||
register_accounts(sub)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
@ -31,6 +31,11 @@ from devplacepy.database import (
|
||||
soft_delete_engagement,
|
||||
soft_delete_fork_relations,
|
||||
load_comments,
|
||||
band_allows_mature,
|
||||
band_allows_restricted,
|
||||
get_maturity,
|
||||
get_maturity_by_targets,
|
||||
get_int_setting,
|
||||
_now_iso,
|
||||
db,
|
||||
)
|
||||
@ -51,6 +56,11 @@ from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.correction import schedule_correction
|
||||
from devplacepy.services.ai_modifier import schedule_modification
|
||||
from devplacepy.services.seo_meta import schedule_seo_meta_for_table
|
||||
from devplacepy.services.moderation.screening import (
|
||||
record as record_screening,
|
||||
refuse_if_blocked,
|
||||
screen_fields,
|
||||
)
|
||||
|
||||
CREATE_METADATA_KEYS = ("project_type", "is_private", "language", "topic", "status")
|
||||
|
||||
@ -79,6 +89,30 @@ def is_owner(item: dict | None, user: dict | None) -> bool:
|
||||
return bool(item and user and item["user_uid"] == user["uid"])
|
||||
|
||||
|
||||
def mature_hidden_by_default() -> bool:
|
||||
return get_int_setting("moderation_mature_default_hidden", 1) != 0
|
||||
|
||||
|
||||
def maturity_hidden(level: str | None, user: dict | None) -> bool:
|
||||
if not level or level == "general":
|
||||
return False
|
||||
if not mature_hidden_by_default():
|
||||
return False
|
||||
if not user:
|
||||
return True
|
||||
band = user.get("age_band") or "adult"
|
||||
allowed = (
|
||||
band_allows_restricted(band) if level == "restricted" else band_allows_mature(band)
|
||||
)
|
||||
return not (allowed and bool(user.get("mature_opt_in")))
|
||||
|
||||
|
||||
def is_suspended(user: dict | None) -> bool:
|
||||
from devplacepy.database import suspension_active
|
||||
|
||||
return suspension_active(user)
|
||||
|
||||
|
||||
def _owner_is_admin(project: dict) -> bool:
|
||||
owner_uid = project.get("user_uid")
|
||||
owner = get_users_by_uids([owner_uid]).get(owner_uid) if owner_uid else None
|
||||
@ -206,6 +240,8 @@ def create_content_item(
|
||||
attachment_uids: list | None,
|
||||
request=None,
|
||||
) -> tuple[str, str]:
|
||||
screening = screen_fields(table_name, fields)
|
||||
refuse_if_blocked(screening)
|
||||
uid = generate_uid()
|
||||
slug = make_combined_slug(slug_source, uid)
|
||||
get_table(table_name).insert(
|
||||
@ -251,6 +287,13 @@ def create_content_item(
|
||||
metadata=metadata or None,
|
||||
links=links,
|
||||
)
|
||||
record_screening(
|
||||
screening,
|
||||
target_type=target_type,
|
||||
target_uid=uid,
|
||||
actor_uid=user["uid"],
|
||||
request=request,
|
||||
)
|
||||
schedule_correction(user, table_name, uid, request)
|
||||
schedule_modification(user, table_name, uid, request)
|
||||
schedule_seo_meta_for_table(table_name, uid)
|
||||
@ -367,6 +410,8 @@ def create_comment_record(
|
||||
parent_uid: str | None = None,
|
||||
attachment_uids: list | None = None,
|
||||
) -> tuple[str, str]:
|
||||
screening = screen_fields("comments", {"content": content})
|
||||
refuse_if_blocked(screening)
|
||||
comment_uid = generate_uid()
|
||||
redirect_url = resolve_object_url(target_type, target_uid)
|
||||
insert = {
|
||||
@ -417,6 +462,13 @@ def create_comment_record(
|
||||
)
|
||||
|
||||
create_mention_notifications(content, user["uid"], comment_url)
|
||||
record_screening(
|
||||
screening,
|
||||
target_type="comment",
|
||||
target_uid=comment_uid,
|
||||
actor_uid=user["uid"],
|
||||
request=request,
|
||||
)
|
||||
schedule_correction(user, "comments", comment_uid, request)
|
||||
schedule_modification(user, "comments", comment_uid, request)
|
||||
logger.info(f"Comment by {user['username']} on {target_type} {target_uid}")
|
||||
@ -441,10 +493,19 @@ def create_comment_record(
|
||||
def edit_comment_record(request, user: dict, comment: dict, content: str) -> str:
|
||||
target_type = comment.get("target_type", "post")
|
||||
target_uid = comment.get("target_uid") or comment.get("post_uid", "")
|
||||
screening = screen_fields("comments", {"content": content})
|
||||
refuse_if_blocked(screening)
|
||||
updated_at = datetime.now(timezone.utc).isoformat()
|
||||
get_table("comments").update(
|
||||
{"uid": comment["uid"], "content": content, "updated_at": updated_at}, ["uid"]
|
||||
)
|
||||
record_screening(
|
||||
screening,
|
||||
target_type="comment",
|
||||
target_uid=comment["uid"],
|
||||
actor_uid=user["uid"],
|
||||
request=request,
|
||||
)
|
||||
schedule_correction(user, "comments", comment["uid"], request)
|
||||
schedule_modification(user, "comments", comment["uid"], request)
|
||||
logger.info(f"Comment {comment['uid']} edited by {user['username']}")
|
||||
@ -567,6 +628,7 @@ def detail_context(
|
||||
"bookmarked": detail.get("bookmarked", False),
|
||||
"poll": detail.get("poll"),
|
||||
"project_link": detail.get("project_link"),
|
||||
"maturity": detail.get("maturity", "general"),
|
||||
}
|
||||
if extra:
|
||||
context.update(extra)
|
||||
@ -601,11 +663,20 @@ def edit_content_item(
|
||||
if wants_json(request):
|
||||
return json_error(403, "Not allowed")
|
||||
return RedirectResponse(url=redirect_fail, status_code=302)
|
||||
screening = screen_fields(table_name, update_fields)
|
||||
refuse_if_blocked(screening)
|
||||
update_fields = {
|
||||
**update_fields,
|
||||
"updated_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
table.update({"uid": item["uid"], **update_fields}, ["uid"])
|
||||
record_screening(
|
||||
screening,
|
||||
target_type=kind,
|
||||
target_uid=item["uid"],
|
||||
actor_uid=user["uid"],
|
||||
request=request,
|
||||
)
|
||||
schedule_correction(user, table_name, item["uid"], request)
|
||||
schedule_modification(user, table_name, item["uid"], request)
|
||||
schedule_seo_meta_for_table(table_name, item["uid"], regenerate=True)
|
||||
@ -747,6 +818,7 @@ def load_detail(
|
||||
"bookmarked": bookmarked,
|
||||
"poll": get_poll_for_post(item["uid"], user) if target_type == "post" else None,
|
||||
"project_link": get_project_by_uid(item.get("project_uid")) if target_type == "post" else None,
|
||||
"maturity": get_maturity(target_type, item["uid"])["level"],
|
||||
}
|
||||
|
||||
|
||||
@ -762,6 +834,7 @@ def enrich_items(
|
||||
user_votes = (
|
||||
get_user_votes(user["uid"], [item["uid"] for item in items]) if user else {}
|
||||
)
|
||||
maturity = get_maturity_by_targets(key, [item["uid"] for item in items])
|
||||
enriched = []
|
||||
for item in items:
|
||||
entry = {
|
||||
@ -769,6 +842,7 @@ def enrich_items(
|
||||
"author": authors.get(item["user_uid"]),
|
||||
"time_ago": time_ago(item[ts_field]),
|
||||
"my_vote": user_votes.get(item["uid"], 0),
|
||||
"maturity": maturity.get(item["uid"], {}).get("level", "general"),
|
||||
}
|
||||
for name, source in extra_maps.items():
|
||||
entry[name] = (
|
||||
@ -780,6 +854,10 @@ def enrich_items(
|
||||
return enriched
|
||||
|
||||
|
||||
def count_project_devlog(project_uid: str) -> int:
|
||||
return get_table("posts").count(project_uid=project_uid, deleted_at=None)
|
||||
|
||||
|
||||
def get_project_devlog(
|
||||
project_uid: str, before: str | None = None, viewer: dict | None = None
|
||||
) -> tuple[list, str | None]:
|
||||
|
||||
@ -12,7 +12,9 @@ PRAGMA cache_size=-8000; -- 8MB page cache
|
||||
PRAGMA temp_store=MEMORY; -- temp tables in memory
|
||||
```
|
||||
|
||||
Configured via `dataset.connect(engine_kwargs={"connect_args": {"timeout": 30, "check_same_thread": False}}, on_connect_statements=[...])`. In addition to the pragmas above, the connection is tuned with a 30s busy timeout, an 8MB page cache, and a 256MB mmap.
|
||||
Configured via `dataset.connect(engine_kwargs={"connect_args": {"timeout": 30, "check_same_thread": False}, "poolclass": NullPool}, on_connect_statements=[...])`. In addition to the pragmas above, the connection is tuned with a 30s busy timeout, an 8MB page cache, and a 256MB mmap.
|
||||
|
||||
**`poolclass=NullPool` is load-bearing - never revert it to SQLAlchemy's default `QueuePool` (caused a production outage).** `dataset.Database.executable` caches ONE DBAPI connection per OS thread ID **forever** and never returns it to the pool except via `db.close()`, which nothing in this codebase calls (`dataset/database.py`: `self.connections[tid] = self.engine.connect()`). That is fine as long as the same handful of threads ever touch the DB - but FastAPI runs every sync route dependency (`get_setting` and friends, hit on nearly every request) through `anyio.to_thread.run_sync`, whose worker pool scales up and recycles threads elastically under load, and container sync (`asyncio.to_thread`) adds more. Each new thread's first query permanently claims one pool slot. With the default bounded `QueuePool` (`pool_size=5, max_overflow=10` = 15 total), a burst of concurrent load creates enough new threads that the pool fills for good within minutes, and every request thereafter - including the Docker healthcheck's own probe - blocks the full 30s pool timeout and then raises `sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached`, wedging the whole app (nginx waits on an app that is waiting on itself; every external caller sees a bare connection timeout, not an HTTP error). `NullPool` removes the artificial ceiling: each `engine.connect()` opens a real, unpooled SQLite connection, so the existing "one connection cached per thread forever" behavior just works, exactly as WAL mode is designed to support. Never pass `pool_size`/`max_overflow` alongside `NullPool` (SQLAlchemy rejects them). Do not "fix" the underlying thread churn instead - that means touching the sync-dependency/threadpool model, which the hard rule below forbids.
|
||||
|
||||
`init_db()` is idempotent - every index is created via `CREATE INDEX IF NOT EXISTS` (through the `_index()` helper, wrapped in try/except so it is safe to run on every startup regardless of table state), and seed defaults for `site_settings` only insert when the key doesn't already exist.
|
||||
|
||||
@ -146,6 +148,23 @@ The profile **Media** tab (`/profile/{username}?tab=media`, public) is a paginat
|
||||
- **Devii:** `list_media` (public) and `delete_media` (auth, in `CONFIRM_REQUIRED`) in the catalog.
|
||||
- **Docs visibility (deliberate):** members and guests must never be told this is a *soft* delete. The public prose page `docs/media-gallery.html` (General) and the member-facing `media-delete` API endpoint (Profiles group) describe deletion as a plain "remove" - no soft-delete, restore, trash, or purge language. All moderation mechanics live on the admin-only `docs/media-moderation` prose page (`admin: True`) and in the admin API group (`media-restore`, `admin-media`, `admin-media-purge`, all `auth="admin"`), which `docs_search` excludes from member results and `routers/docs/` package 404s for non-admins. Because `docs_search._strip` keeps the text *inside* `{% if %}` blocks, admin content must live on a separate `admin: True` page, never inline-gated on a public page (a public page may only carry an admin-gated *link*). The member `MediaItemOut` schema omits `deleted_at`; the admin-only `AdminMediaItemOut` adds it.
|
||||
|
||||
## Moderation, consent and maturity tables (`database/moderation.py`)
|
||||
|
||||
Four soft-deletable tables carry the trust-and-safety layer; the full subsystem is documented in `devplacepy/services/moderation/CLAUDE.md`.
|
||||
|
||||
| Table | Shape | Notes |
|
||||
|---|---|---|
|
||||
| `content_reports` | `(target_type, target_uid)` + `reporter_uid` + `owner_uid` | The one queue. `owner_uid` is denormalised at insert so the admin list never N+1s. Indexes `(status, created_at)` for the queue and SLA scan, `(target_type, target_uid)` for duplicate detection, `(reporter_uid)`, `(owner_uid)` |
|
||||
| `moderation_actions` | one row per moderator decision, linked to its report | Queryable moderation state with its own lifecycle - deliberately separate from the append-only audit log, the same way `workspace_flags` is |
|
||||
| `content_maturity` | `(target_type, target_uid)` -> `level` | Polymorphic age label. Read through the batch helper `get_maturity_by_targets`, never per row. **Absence of a row means `general`**, so nothing needed backfilling |
|
||||
| `user_consents` | `(owner_kind, owner_id, kind)` | Append-only in effect: withdrawing stamps `withdrawn_at` on the current row and inserts a new one, so the history is provable |
|
||||
|
||||
`REPORTABLE_TARGETS` (target type -> table) is the registry every consumer reads, exactly like `VOTABLE_TARGETS`. `UNREPORTABLE_TABLES` is its explicit counterpart: each entry names a soft-deletable table and **why** it carries no reportable content. `tests/unit/database/moderation.py` asserts the two partition `SOFT_DELETE_TABLES`, so a new user-generated table cannot be added without classifying it.
|
||||
|
||||
The `users` columns added alongside are ensured in `backfill_api_keys()` like every other non-signup column: `terms_version`, `terms_accepted_at`, `age_band`, `age_declared_at`, `mature_opt_in`, `suspended_until`, `suspension_reason`, `deletion_requested_at`. `mature_opt_in` is normalised from NULL to `0` in the same `with db:` block that fixes the AI-modifier defaults, because it is read as a flag. **No date of birth is ever stored** - only the derived `age_band`.
|
||||
|
||||
Two atomic conditional updates protect this data and must never become read-then-write: `queue.claim_open` (report resolution) and `deletion.claim_deletion` (the account-deletion cascade). The latter's precondition is `COALESCE(deletion_requested_at, '') = ''` because the column is SQL `NULL` on rows that predate it - the exact `NULL = 0` trap recorded above.
|
||||
|
||||
## Role-based visibility (generic + DRY)
|
||||
|
||||
- **One source of truth for role/visibility checks**, registered as Jinja globals in `templating.py` - never hand-roll `user.get('role') == 'Admin'` or `user['uid'] == x['user_uid']` in a template again:
|
||||
@ -192,6 +211,15 @@ Site settings are seeded on startup (`site_settings` table):
|
||||
| `maintenance_message` | scheduled-maintenance text | Body shown on the maintenance 503 page |
|
||||
| `customization_enabled` | `"1"` | When `"0"`, `custom_css_tag`/`custom_js_tag` inject nothing (feature off) |
|
||||
| `customization_js_enabled` | `"1"` | When `"0"`, custom CSS still serves but custom JS is suppressed |
|
||||
| `moderation_sla_hours` | `"24"` | The published moderation response window; the admin queue badge turns red past it |
|
||||
| `moderation_filter_mode` | `"review"` | `off`/`label`/`review`/`block` - how the content filter acts on a match |
|
||||
| `moderation_filter_review_score` | `"2"` | Rule score at which a match becomes a report rather than a label |
|
||||
| `moderation_minimum_age` | `"16"` | Signup floor; only the derived age band is stored |
|
||||
| `moderation_mature_default_hidden` | `"1"` | Hide mature-labelled content behind an interstitial by default |
|
||||
| `account_deletion_grace_hours` | `"24"` | Reversible window before a deleted account is purged |
|
||||
| `contact_email` / `contact_phone` / `contact_address` | `""` | Published contact details, rendered on `/docs/contact.html` |
|
||||
| `terms_version` / `privacy_version` / `guidelines_version` | `"1"` | Bumping `terms_version` forces re-acceptance before the next write. **Every reader uses `get_setting(key, "1") or "1"`** - an empty stored value must read as the default or the gate 403s every write |
|
||||
| `ai_third_party_provider` | `""` | Named in the consent copy and the privacy policy |
|
||||
| `extra_head` | `""` | Raw HTML emitted verbatim into every page `<head>` by `templating.extra_head_tag()`; site-wide trusted-admin input, not sanitized |
|
||||
|
||||
Besides the site/news/upload keys above, the **Operational** group is admin-editable at `/admin/settings`: `site_url` (public origin for absolute links incl. container ingress; resolved by `seo.public_base_url()` = setting -> `DEVPLACE_SITE_URL` env -> request origin), `rate_limit_per_minute`, `rate_limit_window_seconds`, `news_service_interval`, `session_max_age_days`, `session_remember_days`, `registration_open`, `maintenance_mode`, `maintenance_message`, `docs_search_mode` (`agent`|`bm25`, default `agent` - picks the `/docs/search.html` surface: the in-page Devii chat or the classic BM25 list; a viewer over their daily AI limit, or a guest when Devii is disabled, auto-falls-back to BM25 via `routers/docs/views.py` `_agent_search_state`), `outbound_proxy_url` (empty by default - when set, every `stealth.stealth_async_client`/`stealth_sync_client` call across the whole app routes through it via `stealth.configured_proxy_url()`; validated as `http(s)://`/`socks5(h)://` with a host in `AdminSettingsForm`; falls back to `DEVPLACE_OUTBOUND_PROXY_URL` when unset - see the "Outbound HTTP" note in the root `CLAUDE.md`). The **Custom Code** key `extra_head` is the sole key in the settings handler's `CLEARABLE_SETTINGS` set, so saving an empty textarea removes it (the default loop skips empty values).
|
||||
|
||||
@ -36,6 +36,43 @@ from .forks import record_fork, get_fork_parent, count_forks, soft_delete_fork_r
|
||||
from .follows import get_follow_counts, get_follow_list, get_following_among
|
||||
from .deepsearch import _ds_now, create_deepsearch_session, update_deepsearch_session, get_deepsearch_session, add_deepsearch_message, get_deepsearch_messages, get_cached_deepsearch_url, upsert_deepsearch_url_cache
|
||||
from .ranking import VOTABLE_TARGETS, STAR_TARGETS, _authors_cache, _ranked_authors, _rank_map, get_top_authors, get_leaderboard, get_user_rank, get_user_stars, clear_user_stars, update_target_stars, soft_delete_engagement, delete_engagement, get_target_owner_uid
|
||||
from .moderation import (
|
||||
ACTIONS_TABLE,
|
||||
ADULT_AGE,
|
||||
AGE_BANDS,
|
||||
CONSENTS_TABLE,
|
||||
CONSENT_KINDS,
|
||||
CONSENT_STATES,
|
||||
MATURITY_LEVELS,
|
||||
MATURITY_SOURCES,
|
||||
MATURITY_TABLE,
|
||||
MATURITY_TARGETS,
|
||||
MODERATION_ACTIONS,
|
||||
MODERATION_TABLES,
|
||||
REPORTABLE_TARGETS,
|
||||
REPORTS_TABLE,
|
||||
REPORT_OPEN_STATUSES,
|
||||
REPORT_ORIGINS,
|
||||
REPORT_REASONS,
|
||||
REPORT_SEVERITIES,
|
||||
REPORT_STATUSES,
|
||||
SYSTEM_ACTOR,
|
||||
UNREPORTABLE_TABLES,
|
||||
age_band_for,
|
||||
band_allows_mature,
|
||||
band_allows_restricted,
|
||||
consent_granted,
|
||||
consent_state,
|
||||
get_maturity,
|
||||
get_maturity_by_targets,
|
||||
list_consents,
|
||||
minimum_age,
|
||||
report_reason_options,
|
||||
set_consent,
|
||||
set_maturity,
|
||||
suspension_active,
|
||||
years_between,
|
||||
)
|
||||
from .comments import _drop_blocked, _build_comment_items, load_comments, get_recent_comments_by_target_uids, get_recent_comments_by_post_uids, load_comments_by_target_uids
|
||||
from .content import resolve_by_slug, resolve_object_url, get_uids_by_username_match, text_search_clause, get_daily_topic, get_featured_news, get_trending_topics
|
||||
from .attachments_data import get_attachments, get_attachments_by_type, get_news_images_by_uids, delete_attachment_record, delete_attachments, _delete_attachment_file, get_user_media, get_user_attachments, get_user_attachment, get_deleted_media
|
||||
@ -217,6 +254,40 @@ __all__ = [
|
||||
"soft_delete_engagement",
|
||||
"delete_engagement",
|
||||
"get_target_owner_uid",
|
||||
"ACTIONS_TABLE",
|
||||
"ADULT_AGE",
|
||||
"AGE_BANDS",
|
||||
"CONSENTS_TABLE",
|
||||
"CONSENT_KINDS",
|
||||
"CONSENT_STATES",
|
||||
"MATURITY_LEVELS",
|
||||
"MATURITY_SOURCES",
|
||||
"MATURITY_TABLE",
|
||||
"MATURITY_TARGETS",
|
||||
"MODERATION_ACTIONS",
|
||||
"MODERATION_TABLES",
|
||||
"REPORTABLE_TARGETS",
|
||||
"REPORTS_TABLE",
|
||||
"REPORT_OPEN_STATUSES",
|
||||
"REPORT_ORIGINS",
|
||||
"REPORT_REASONS",
|
||||
"REPORT_SEVERITIES",
|
||||
"REPORT_STATUSES",
|
||||
"SYSTEM_ACTOR",
|
||||
"UNREPORTABLE_TABLES",
|
||||
"age_band_for",
|
||||
"band_allows_mature",
|
||||
"band_allows_restricted",
|
||||
"consent_granted",
|
||||
"consent_state",
|
||||
"get_maturity",
|
||||
"get_maturity_by_targets",
|
||||
"list_consents",
|
||||
"report_reason_options",
|
||||
"set_consent",
|
||||
"set_maturity",
|
||||
"suspension_active",
|
||||
"years_between",
|
||||
"_drop_blocked",
|
||||
"_build_comment_items",
|
||||
"load_comments",
|
||||
|
||||
@ -56,6 +56,44 @@ def resolve_object_url(target_type: str, target_uid: str) -> str:
|
||||
receiver = get_table("users").find_one(uid=award.get("receiver_uid", ""))
|
||||
if receiver:
|
||||
return f"/profile/{receiver['username']}?tab=awards#award-{award.get('slug', '')}"
|
||||
return "/feed"
|
||||
if target_type == "user":
|
||||
person = get_table("users").find_one(uid=target_uid)
|
||||
return f"/profile/{person['username']}" if person else "/feed"
|
||||
if target_type == "project_file":
|
||||
node = get_table("project_files").find_one(uid=target_uid)
|
||||
if not node:
|
||||
return "/projects"
|
||||
project = get_table("projects").find_one(uid=node.get("project_uid", ""))
|
||||
if not project:
|
||||
return "/projects"
|
||||
slug = project.get("slug") or project["uid"]
|
||||
return f"/projects/{slug}/files?path={node.get('path', '')}"
|
||||
if target_type == "attachment":
|
||||
attachment = get_table("attachments").find_one(uid=target_uid)
|
||||
if not attachment:
|
||||
return "/feed"
|
||||
parent_type = attachment.get("target_type") or ""
|
||||
parent_uid = attachment.get("target_uid") or ""
|
||||
if parent_type and parent_uid:
|
||||
return resolve_object_url(parent_type, parent_uid)
|
||||
owner = get_table("users").find_one(uid=attachment.get("user_uid", ""))
|
||||
return f"/profile/{owner['username']}?tab=media" if owner else "/feed"
|
||||
if target_type == "message":
|
||||
message = get_table("messages").find_one(uid=target_uid)
|
||||
if not message:
|
||||
return "/messages"
|
||||
return f"/messages?with_uid={message.get('sender_uid', '')}"
|
||||
if target_type == "poll":
|
||||
poll = get_table("polls").find_one(uid=target_uid)
|
||||
if not poll:
|
||||
return "/feed"
|
||||
return resolve_object_url("post", poll.get("post_uid", ""))
|
||||
if target_type == "workspace":
|
||||
instance = get_table("instances").find_one(uid=target_uid)
|
||||
return f"/admin/containers/{instance['uid']}" if instance else "/admin/containers"
|
||||
if target_type == "devii_output":
|
||||
return "/devii"
|
||||
return "/feed"
|
||||
|
||||
|
||||
|
||||
@ -4,6 +4,7 @@ import dataset
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from sqlalchemy import or_
|
||||
from sqlalchemy.pool import NullPool
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from devplacepy.cache import TTLCache
|
||||
@ -30,6 +31,7 @@ db = dataset.connect(
|
||||
"timeout": 30,
|
||||
"check_same_thread": False,
|
||||
},
|
||||
"poolclass": NullPool,
|
||||
},
|
||||
on_connect_statements=[
|
||||
"PRAGMA journal_mode=WAL",
|
||||
|
||||
332
devplacepy/database/moderation.py
Normal file
332
devplacepy/database/moderation.py
Normal file
@ -0,0 +1,332 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from datetime import date, datetime, timezone
|
||||
|
||||
from .core import _in_clause, _now_iso, db, get_table
|
||||
from .settings import get_int_setting
|
||||
|
||||
|
||||
REPORTABLE_TARGETS: dict[str, str] = {
|
||||
"post": "posts",
|
||||
"comment": "comments",
|
||||
"gist": "gists",
|
||||
"project": "projects",
|
||||
"project_file": "project_files",
|
||||
"news": "news",
|
||||
"attachment": "attachments",
|
||||
"message": "messages",
|
||||
"quiz": "quizzes",
|
||||
"poll": "polls",
|
||||
"award": "awards",
|
||||
"user": "users",
|
||||
"issue": "issue_tickets",
|
||||
"workspace": "instances",
|
||||
"devii_output": "devii_conversations",
|
||||
}
|
||||
|
||||
|
||||
MATURITY_TARGETS: set[str] = {
|
||||
"post",
|
||||
"comment",
|
||||
"gist",
|
||||
"project",
|
||||
"news",
|
||||
"attachment",
|
||||
"quiz",
|
||||
}
|
||||
|
||||
|
||||
UNREPORTABLE_TABLES: dict[str, str] = {
|
||||
"news_images": "child rows of a reportable news article",
|
||||
"poll_options": "child rows of a reportable poll",
|
||||
"quiz_questions": "child rows of a reportable quiz",
|
||||
"quiz_options": "child rows of a reportable quiz",
|
||||
"tunnels": "child rows of a reportable workspace instance",
|
||||
"issue_comment_authors": "authorship index for reportable issue comments",
|
||||
"votes": "engagement counters, carry no authored content",
|
||||
"reactions": "engagement counters, carry no authored content",
|
||||
"bookmarks": "private to the owner",
|
||||
"follows": "relationship rows, carry no authored content",
|
||||
"poll_votes": "private ballots",
|
||||
"quiz_attempts": "private to the participant",
|
||||
"quiz_answers": "private to the participant",
|
||||
"sessions": "authentication state",
|
||||
"access_tokens": "authentication state",
|
||||
"devrant_tokens": "authentication state",
|
||||
"user_relations": "private block and mute lists",
|
||||
"notification_preferences": "private to the owner",
|
||||
"user_customizations": "runs only in the owner's own browser",
|
||||
"devii_tasks": "private to the owner",
|
||||
"devii_lessons": "private to the owner",
|
||||
"devii_virtual_tools": "private to the owner",
|
||||
"deepsearch_sessions": "private to the owner",
|
||||
"deepsearch_messages": "private to the owner",
|
||||
"isslop_analyses": "generated from a public URL, not authored content",
|
||||
"email_accounts": "private mailbox credentials",
|
||||
"instance_schedules": "child rows of a reportable workspace instance",
|
||||
"workspace_flags": "moderation records, not authored content",
|
||||
"workspace_quota_rules": "administrator-set limits, not authored content",
|
||||
"workspace_editor_prefs": (
|
||||
"private per-user editor configuration, never shown to another member"
|
||||
),
|
||||
"content_reports": "moderation records, readable only by the reporter and moderators",
|
||||
"moderation_actions": "moderation records, not authored content",
|
||||
"content_maturity": "moderation labels, not authored content",
|
||||
"user_consents": "private consent history of the account holder",
|
||||
"backup_schedules": "operator configuration",
|
||||
"project_forks": "lineage index for reportable projects",
|
||||
"seo_metadata": "generated metadata for reportable content",
|
||||
}
|
||||
|
||||
|
||||
REPORT_REASONS: dict[str, str] = {
|
||||
"hate": "Hate speech or discriminatory content",
|
||||
"violence": "Realistic violence or threats",
|
||||
"weapons": "Weapons or dangerous instructions",
|
||||
"sexual": "Sexual or pornographic content",
|
||||
"religious": "Content targeting religion or belief",
|
||||
"misinformation": "False or misleading information",
|
||||
"exploitative": "Content exploiting a person",
|
||||
"harassment": "Harassment or bullying",
|
||||
"spam": "Spam or unwanted promotion",
|
||||
"intellectual_property": "Copyright or trademark infringement",
|
||||
"self_harm": "Self-harm or suicide",
|
||||
"illegal": "Illegal activity",
|
||||
"other": "Something else",
|
||||
}
|
||||
|
||||
|
||||
def report_reason_options() -> list[dict[str, str]]:
|
||||
return [{"key": key, "label": label} for key, label in REPORT_REASONS.items()]
|
||||
|
||||
|
||||
REPORT_STATUSES: tuple[str, ...] = ("open", "acknowledged", "actioned", "dismissed")
|
||||
REPORT_OPEN_STATUSES: tuple[str, ...] = ("open", "acknowledged")
|
||||
REPORT_SEVERITIES: tuple[str, ...] = ("info", "warn", "critical")
|
||||
REPORT_ORIGINS: tuple[str, ...] = ("member", "filter")
|
||||
|
||||
MODERATION_ACTIONS: tuple[str, ...] = (
|
||||
"remove_content",
|
||||
"restore_content",
|
||||
"warn",
|
||||
"suspend",
|
||||
"ban",
|
||||
"lift",
|
||||
"dismiss",
|
||||
"escalate",
|
||||
)
|
||||
|
||||
MATURITY_LEVELS: tuple[str, ...] = ("general", "mature", "restricted")
|
||||
MATURITY_SOURCES: tuple[str, ...] = ("author", "filter", "moderator")
|
||||
|
||||
CONSENT_KINDS: dict[str, str] = {
|
||||
"terms": "Terms of Service and Community Guidelines",
|
||||
"privacy": "Privacy Policy",
|
||||
"ai_third_party": "Processing of your content by a third-party AI provider",
|
||||
"activity_recording": "Recording of your presence and session activity",
|
||||
"container_credentials": (
|
||||
"Sharing your DevPlace credentials with software another member runs "
|
||||
"in a container"
|
||||
),
|
||||
}
|
||||
CONSENT_STATES: tuple[str, ...] = ("granted", "withdrawn")
|
||||
|
||||
AGE_BANDS: tuple[str, ...] = ("under_min", "13_15", "16_17", "adult")
|
||||
|
||||
ADULT_AGE = 18
|
||||
TEEN_AGE = 16
|
||||
YOUNG_TEEN_AGE = 13
|
||||
|
||||
MINIMUM_AGE_FLOOR = YOUNG_TEEN_AGE
|
||||
DEFAULT_MINIMUM_AGE = TEEN_AGE
|
||||
|
||||
SYSTEM_ACTOR = "system"
|
||||
|
||||
REPORTS_TABLE = "content_reports"
|
||||
ACTIONS_TABLE = "moderation_actions"
|
||||
MATURITY_TABLE = "content_maturity"
|
||||
CONSENTS_TABLE = "user_consents"
|
||||
|
||||
MODERATION_TABLES: tuple[str, ...] = (
|
||||
REPORTS_TABLE,
|
||||
ACTIONS_TABLE,
|
||||
MATURITY_TABLE,
|
||||
CONSENTS_TABLE,
|
||||
)
|
||||
|
||||
|
||||
def years_between(born: date, today: date) -> int:
|
||||
years = today.year - born.year
|
||||
if (today.month, today.day) < (born.month, born.day):
|
||||
years -= 1
|
||||
return years
|
||||
|
||||
|
||||
def minimum_age() -> int:
|
||||
return max(
|
||||
MINIMUM_AGE_FLOOR,
|
||||
get_int_setting("moderation_minimum_age", DEFAULT_MINIMUM_AGE),
|
||||
)
|
||||
|
||||
|
||||
def age_band_for(age: int) -> str:
|
||||
if age >= ADULT_AGE:
|
||||
return "adult"
|
||||
if age >= TEEN_AGE:
|
||||
return "16_17"
|
||||
if age >= YOUNG_TEEN_AGE:
|
||||
return "13_15"
|
||||
return "under_min"
|
||||
|
||||
|
||||
def band_allows_mature(band: str) -> bool:
|
||||
return band == "adult"
|
||||
|
||||
|
||||
def band_allows_restricted(band: str) -> bool:
|
||||
return band == "adult"
|
||||
|
||||
|
||||
def get_maturity_by_targets(target_type: str, uids: list[str]) -> dict[str, dict]:
|
||||
uids = [uid for uid in (uids or []) if uid]
|
||||
if not uids or MATURITY_TABLE not in db.tables:
|
||||
return {}
|
||||
placeholders, params = _in_clause(uids)
|
||||
params["tt"] = target_type
|
||||
rows = db.query(
|
||||
f"SELECT target_uid, level, source FROM {MATURITY_TABLE} "
|
||||
f"WHERE target_type = :tt AND target_uid IN ({placeholders}) "
|
||||
f"AND deleted_at IS NULL",
|
||||
**params,
|
||||
)
|
||||
return {
|
||||
row["target_uid"]: {"level": row["level"], "source": row["source"]}
|
||||
for row in rows
|
||||
}
|
||||
|
||||
|
||||
def get_maturity(target_type: str, target_uid: str) -> dict:
|
||||
found = get_maturity_by_targets(target_type, [target_uid])
|
||||
return found.get(target_uid, {"level": "general", "source": ""})
|
||||
|
||||
|
||||
def set_maturity(
|
||||
target_type: str, target_uid: str, level: str, source: str, set_by: str
|
||||
) -> dict | None:
|
||||
if target_type not in MATURITY_TARGETS or level not in MATURITY_LEVELS:
|
||||
return None
|
||||
from devplacepy.utils import generate_uid
|
||||
|
||||
table = get_table(MATURITY_TABLE)
|
||||
existing = table.find_one(target_type=target_type, target_uid=target_uid)
|
||||
now = _now_iso()
|
||||
if existing:
|
||||
table.update(
|
||||
{
|
||||
"id": existing["id"],
|
||||
"level": level,
|
||||
"source": source,
|
||||
"set_by": set_by,
|
||||
"updated_at": now,
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
},
|
||||
["id"],
|
||||
)
|
||||
return table.find_one(id=existing["id"])
|
||||
uid = generate_uid()
|
||||
table.insert(
|
||||
{
|
||||
"uid": uid,
|
||||
"target_type": target_type,
|
||||
"target_uid": target_uid,
|
||||
"level": level,
|
||||
"source": source,
|
||||
"set_by": set_by,
|
||||
"created_at": now,
|
||||
"updated_at": now,
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
return table.find_one(uid=uid)
|
||||
|
||||
|
||||
def list_consents(owner_kind: str, owner_id: str) -> list[dict]:
|
||||
if not owner_id or CONSENTS_TABLE not in db.tables:
|
||||
return []
|
||||
return list(
|
||||
get_table(CONSENTS_TABLE).find(
|
||||
owner_kind=owner_kind,
|
||||
owner_id=owner_id,
|
||||
deleted_at=None,
|
||||
order_by=["-created_at"],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def consent_state(owner_kind: str, owner_id: str, kind: str) -> dict | None:
|
||||
if not owner_id or CONSENTS_TABLE not in db.tables:
|
||||
return None
|
||||
rows = list(
|
||||
get_table(CONSENTS_TABLE).find(
|
||||
owner_kind=owner_kind,
|
||||
owner_id=owner_id,
|
||||
kind=kind,
|
||||
deleted_at=None,
|
||||
order_by=["-created_at", "-id"],
|
||||
_limit=1,
|
||||
)
|
||||
)
|
||||
return rows[0] if rows else None
|
||||
|
||||
|
||||
def consent_granted(owner_kind: str, owner_id: str, kind: str) -> bool:
|
||||
row = consent_state(owner_kind, owner_id, kind)
|
||||
return bool(row and row.get("state") == "granted")
|
||||
|
||||
|
||||
def set_consent(
|
||||
owner_kind: str, owner_id: str, kind: str, granted: bool, version: str = "1"
|
||||
) -> dict | None:
|
||||
if kind not in CONSENT_KINDS or not owner_id:
|
||||
return None
|
||||
from devplacepy.utils import generate_uid
|
||||
|
||||
table = get_table(CONSENTS_TABLE)
|
||||
now = _now_iso()
|
||||
current = consent_state(owner_kind, owner_id, kind)
|
||||
if current and not granted and current.get("state") == "granted":
|
||||
table.update({"id": current["id"], "withdrawn_at": now}, ["id"])
|
||||
uid = generate_uid()
|
||||
table.insert(
|
||||
{
|
||||
"uid": uid,
|
||||
"owner_kind": owner_kind,
|
||||
"owner_id": owner_id,
|
||||
"kind": kind,
|
||||
"version": version,
|
||||
"state": "granted" if granted else "withdrawn",
|
||||
"granted_at": now if granted else "",
|
||||
"withdrawn_at": "" if granted else now,
|
||||
"created_at": now,
|
||||
"deleted_at": None,
|
||||
"deleted_by": None,
|
||||
}
|
||||
)
|
||||
return table.find_one(uid=uid)
|
||||
|
||||
|
||||
def suspension_active(user: dict | None) -> bool:
|
||||
if not user:
|
||||
return False
|
||||
until = (user.get("suspended_until") or "").strip()
|
||||
if not until:
|
||||
return False
|
||||
try:
|
||||
expiry = datetime.fromisoformat(until)
|
||||
except ValueError:
|
||||
return False
|
||||
if expiry.tzinfo is None:
|
||||
expiry = expiry.replace(tzinfo=timezone.utc)
|
||||
return expiry > datetime.now(timezone.utc)
|
||||
@ -20,6 +20,7 @@ NOTIFICATION_TYPES = [
|
||||
{"key": "award", "label": "Awards", "description": "Someone gives you an award on your profile"},
|
||||
{"key": "quiz_attempt", "label": "Quiz attempts", "description": "Someone completes one of your quizzes"},
|
||||
{"key": "workspace", "label": "Dev workspaces", "description": "Idle, quota, retention and moderation notices for your dev workspaces"},
|
||||
{"key": "moderation", "label": "Moderation", "description": "Acknowledgements for reports you file and decisions taken on your content"},
|
||||
{"key": "system", "label": "System alerts", "description": "Platform infrastructure alerts (e.g. the AI gateway going down)"},
|
||||
]
|
||||
|
||||
|
||||
@ -121,6 +121,17 @@ def init_db():
|
||||
_index(db, "comments", "idx_comments_user_uid", ["user_uid"])
|
||||
_index(db, "comments", "idx_comments_created_at", ["created_at"])
|
||||
_index(db, "votes", "idx_votes_target", ["target_uid", "target_type"])
|
||||
messages = get_table("messages")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("sender_uid", ""),
|
||||
("receiver_uid", ""),
|
||||
("content", ""),
|
||||
("read", False),
|
||||
("created_at", ""),
|
||||
):
|
||||
if not messages.has_column(column):
|
||||
messages.create_column_by_example(column, example)
|
||||
_index(db, "messages", "idx_messages_sender", ["sender_uid"])
|
||||
_index(db, "messages", "idx_messages_receiver", ["receiver_uid"])
|
||||
_index(
|
||||
@ -171,6 +182,13 @@ def init_db():
|
||||
("title", ""),
|
||||
("description", ""),
|
||||
("status", ""),
|
||||
("platforms", ""),
|
||||
("release_date", ""),
|
||||
("demo_date", ""),
|
||||
("website_url", ""),
|
||||
("repo_url", ""),
|
||||
("cover_attachment_uid", ""),
|
||||
("logo_attachment_uid", ""),
|
||||
):
|
||||
if not projects.has_column(column):
|
||||
projects.create_column_by_example(column, example)
|
||||
@ -277,7 +295,7 @@ def init_db():
|
||||
defaults = {
|
||||
"site_name": "DevPlace",
|
||||
"site_description": "The Developer Social Network",
|
||||
"site_tagline": "Track industry shifts. Discover bold releases. Share what you are building in an open, uncensored environment.",
|
||||
"site_tagline": "Track industry shifts. Discover bold releases. Share what you are building in an open environment built by developers, for developers.",
|
||||
}
|
||||
for key, value in defaults.items():
|
||||
existing = db["site_settings"].find_one(key=key)
|
||||
@ -558,6 +576,9 @@ def init_db():
|
||||
("slug", ""),
|
||||
("name", ""),
|
||||
("status", ""),
|
||||
("created_at", ""),
|
||||
("owner_uid", ""),
|
||||
("created_by", ""),
|
||||
("desired_state", ""),
|
||||
("container_id", ""),
|
||||
("ingress_slug", ""),
|
||||
@ -572,6 +593,7 @@ def init_db():
|
||||
("workspace_owner_uid", ""),
|
||||
("editor_port", 0),
|
||||
("editor_host_port", 0),
|
||||
("editor_password", ""),
|
||||
("tunnel_name", ""),
|
||||
("last_active_at", ""),
|
||||
("idle_warned_at", ""),
|
||||
@ -584,6 +606,10 @@ def init_db():
|
||||
("flag_reason", ""),
|
||||
("suspended_at", ""),
|
||||
("suspended_by", ""),
|
||||
("boot_marker", ""),
|
||||
("workspace_cpu_millicores", 0),
|
||||
("workspace_memory_mb", 0),
|
||||
("workspace_disk_quota_mb", 0),
|
||||
):
|
||||
if not instances.has_column(column):
|
||||
instances.create_column_by_example(column, example)
|
||||
@ -624,12 +650,36 @@ def init_db():
|
||||
("egress_quota_mb", 0),
|
||||
("idle_stop_minutes", 0),
|
||||
("retention_days", 0),
|
||||
("cpu_millicores", 0),
|
||||
("memory_mb", 0),
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
):
|
||||
if not quota_rules.has_column(column):
|
||||
quota_rules.create_column_by_example(column, example)
|
||||
|
||||
editor_prefs = get_table("workspace_editor_prefs")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
("owner_kind", ""),
|
||||
("owner_id", ""),
|
||||
("font_size", 0),
|
||||
("terminal_font_size", 0),
|
||||
("zoom_level", -99),
|
||||
("theme", ""),
|
||||
("layout", ""),
|
||||
("panel_preset", ""),
|
||||
("window_mode", ""),
|
||||
("window_width", 0),
|
||||
("window_height", 0),
|
||||
("boot_agent", ""),
|
||||
("boot_shell", -1),
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
):
|
||||
if not editor_prefs.has_column(column):
|
||||
editor_prefs.create_column_by_example(column, example)
|
||||
|
||||
flags = get_table("workspace_flags")
|
||||
for column, example in (
|
||||
("uid", ""),
|
||||
@ -662,6 +712,12 @@ def init_db():
|
||||
"idx_workspace_quota_owner",
|
||||
["owner_kind", "owner_id"],
|
||||
)
|
||||
_index(
|
||||
db,
|
||||
"workspace_editor_prefs",
|
||||
"idx_workspace_editor_prefs_owner",
|
||||
["owner_kind", "owner_id"],
|
||||
)
|
||||
_index(db, "workspace_flags", "idx_workspace_flags_open", ["status", "created_at"])
|
||||
_index(
|
||||
db,
|
||||
@ -1549,6 +1605,8 @@ def init_db():
|
||||
["user_uid", "created_at"],
|
||||
)
|
||||
|
||||
_ensure_moderation_tables()
|
||||
|
||||
for table in db.tables:
|
||||
_uid_index(db, table)
|
||||
|
||||
@ -1687,6 +1745,19 @@ def init_db():
|
||||
"outbound_proxy_url": "",
|
||||
"devii_lessons_max_per_owner": "500",
|
||||
"devii_lessons_max_age_days": "90",
|
||||
"moderation_sla_hours": "24",
|
||||
"moderation_filter_mode": "review",
|
||||
"moderation_filter_review_score": "2",
|
||||
"moderation_minimum_age": "16",
|
||||
"moderation_mature_default_hidden": "1",
|
||||
"contact_email": "",
|
||||
"contact_phone": "",
|
||||
"contact_address": "",
|
||||
"terms_version": "1",
|
||||
"privacy_version": "1",
|
||||
"guidelines_version": "1",
|
||||
"ai_third_party_provider": "",
|
||||
"account_deletion_grace_hours": "24",
|
||||
}
|
||||
for key, value in operational_defaults.items():
|
||||
existing = db["site_settings"].find_one(key=key)
|
||||
@ -1767,6 +1838,83 @@ def init_db():
|
||||
_refresh_query_planner_stats()
|
||||
|
||||
|
||||
MODERATION_COLUMNS: dict[str, tuple[tuple[str, object], ...]] = {
|
||||
"content_reports": (
|
||||
("uid", ""),
|
||||
("reporter_uid", ""),
|
||||
("target_type", ""),
|
||||
("target_uid", ""),
|
||||
("owner_uid", ""),
|
||||
("reason", ""),
|
||||
("detail", ""),
|
||||
("severity", "warn"),
|
||||
("status", "open"),
|
||||
("origin", "member"),
|
||||
("categories", ""),
|
||||
("resolved_by", ""),
|
||||
("resolved_at", ""),
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
),
|
||||
"moderation_actions": (
|
||||
("uid", ""),
|
||||
("report_uid", ""),
|
||||
("actor_uid", ""),
|
||||
("action", ""),
|
||||
("target_type", ""),
|
||||
("target_uid", ""),
|
||||
("subject_uid", ""),
|
||||
("reason", ""),
|
||||
("notes", ""),
|
||||
("expires_at", ""),
|
||||
("created_at", ""),
|
||||
),
|
||||
"content_maturity": (
|
||||
("uid", ""),
|
||||
("target_type", ""),
|
||||
("target_uid", ""),
|
||||
("level", "general"),
|
||||
("source", ""),
|
||||
("set_by", ""),
|
||||
("created_at", ""),
|
||||
("updated_at", ""),
|
||||
),
|
||||
"user_consents": (
|
||||
("uid", ""),
|
||||
("owner_kind", ""),
|
||||
("owner_id", ""),
|
||||
("kind", ""),
|
||||
("version", "1"),
|
||||
("state", ""),
|
||||
("granted_at", ""),
|
||||
("withdrawn_at", ""),
|
||||
("created_at", ""),
|
||||
),
|
||||
}
|
||||
|
||||
MODERATION_INDEXES: tuple[tuple[str, str, list[str]], ...] = (
|
||||
("content_reports", "idx_content_reports_queue", ["status", "created_at"]),
|
||||
("content_reports", "idx_content_reports_target", ["target_type", "target_uid"]),
|
||||
("content_reports", "idx_content_reports_reporter", ["reporter_uid"]),
|
||||
("content_reports", "idx_content_reports_owner", ["owner_uid"]),
|
||||
("moderation_actions", "idx_moderation_actions_report", ["report_uid"]),
|
||||
("moderation_actions", "idx_moderation_actions_subject", ["subject_uid"]),
|
||||
("moderation_actions", "idx_moderation_actions_created", ["created_at"]),
|
||||
("content_maturity", "idx_content_maturity_target", ["target_type", "target_uid"]),
|
||||
("user_consents", "idx_user_consents_owner", ["owner_kind", "owner_id", "kind"]),
|
||||
)
|
||||
|
||||
|
||||
def _ensure_moderation_tables() -> None:
|
||||
for table_name, columns in MODERATION_COLUMNS.items():
|
||||
table = get_table(table_name)
|
||||
for column, example in columns:
|
||||
if not table.has_column(column):
|
||||
table.create_column_by_example(column, example)
|
||||
for table_name, index_name, columns in MODERATION_INDEXES:
|
||||
_index(db, table_name, index_name, columns)
|
||||
|
||||
|
||||
def _refresh_query_planner_stats() -> None:
|
||||
try:
|
||||
has_stats = bool(
|
||||
@ -1856,6 +2004,22 @@ def backfill_api_keys() -> int:
|
||||
users.create_column_by_example("last_award_slug", "")
|
||||
if not users.has_column("last_award_uid"):
|
||||
users.create_column_by_example("last_award_uid", "")
|
||||
if not users.has_column("terms_version"):
|
||||
users.create_column_by_example("terms_version", "")
|
||||
if not users.has_column("terms_accepted_at"):
|
||||
users.create_column_by_example("terms_accepted_at", "")
|
||||
if not users.has_column("age_band"):
|
||||
users.create_column_by_example("age_band", "")
|
||||
if not users.has_column("age_declared_at"):
|
||||
users.create_column_by_example("age_declared_at", "")
|
||||
if not users.has_column("mature_opt_in"):
|
||||
users.create_column_by_example("mature_opt_in", 0)
|
||||
if not users.has_column("suspended_until"):
|
||||
users.create_column_by_example("suspended_until", "")
|
||||
if not users.has_column("suspension_reason"):
|
||||
users.create_column_by_example("suspension_reason", "")
|
||||
if not users.has_column("deletion_requested_at"):
|
||||
users.create_column_by_example("deletion_requested_at", "")
|
||||
with db:
|
||||
db.query(
|
||||
"UPDATE users SET ai_modifier_enabled = 1 WHERE ai_modifier_enabled IS NULL"
|
||||
@ -1870,6 +2034,7 @@ def backfill_api_keys() -> int:
|
||||
"UPDATE users SET interactions_enabled = -1 "
|
||||
"WHERE interactions_enabled IS NULL"
|
||||
)
|
||||
db.query("UPDATE users SET mature_opt_in = 0 WHERE mature_opt_in IS NULL")
|
||||
import uuid_utils
|
||||
|
||||
updated = 0
|
||||
|
||||
@ -25,6 +25,8 @@ SOFT_DELETE_TABLES = [
|
||||
"instance_schedules",
|
||||
"tunnels",
|
||||
"workspace_flags",
|
||||
"workspace_quota_rules",
|
||||
"workspace_editor_prefs",
|
||||
"backup_schedules",
|
||||
"devii_conversations",
|
||||
"devii_tasks",
|
||||
@ -49,6 +51,10 @@ SOFT_DELETE_TABLES = [
|
||||
"quiz_options",
|
||||
"quiz_attempts",
|
||||
"quiz_answers",
|
||||
"content_reports",
|
||||
"moderation_actions",
|
||||
"content_maturity",
|
||||
"user_consents",
|
||||
]
|
||||
|
||||
|
||||
|
||||
@ -8,6 +8,7 @@ from . import (
|
||||
content,
|
||||
profiles,
|
||||
messaging,
|
||||
moderation,
|
||||
notifications,
|
||||
uploads,
|
||||
project_files,
|
||||
@ -31,6 +32,7 @@ ORDERED_GROUPS = [
|
||||
content.GROUP,
|
||||
profiles.GROUP,
|
||||
messaging.GROUP,
|
||||
moderation.GROUP,
|
||||
notifications.GROUP,
|
||||
uploads.GROUP,
|
||||
project_files.GROUP,
|
||||
|
||||
@ -50,6 +50,13 @@ as a `422` with the shape `{ "fields": {...}, "messages": [...] }`.
|
||||
field("email", "form", "string", True, "alice@example.com", "Email address; must be unique and contain an @."),
|
||||
field("password", "form", "string", True, "mysecret", "Password, 6+ characters."),
|
||||
field("confirm_password", "form", "string", True, "mysecret", "Must match password."),
|
||||
field("birth_date", "form", "string", True, "01/01/1990", "Date of birth, DD/MM/YYYY or YYYY-MM-DD. Only the derived age band is stored; the date is discarded."),
|
||||
field("accept_terms", "form", "enum", True, "1", "Acceptance of the Terms of Service and Community Guidelines.", ["1"]),
|
||||
],
|
||||
notes=[
|
||||
"Signup is refused below the platform minimum age (`moderation_minimum_age`).",
|
||||
"Accepting records the terms, privacy and activity-recording consents; "
|
||||
"third-party AI processing stays off until it is granted separately.",
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
|
||||
@ -362,7 +362,7 @@ four ways to sign requests.
|
||||
method="GET",
|
||||
path="/projects/{project_slug}",
|
||||
title="View a project",
|
||||
summary="Render a project with comments. Returns an HTML page.",
|
||||
summary="Render the project overview with its devlog, screenshots and comments. Returns an HTML page.",
|
||||
auth="public",
|
||||
interactive=True,
|
||||
params=[
|
||||
@ -373,7 +373,42 @@ four ways to sign requests.
|
||||
True,
|
||||
"PROJECT_SLUG",
|
||||
"Slug or UID of the project.",
|
||||
)
|
||||
),
|
||||
field(
|
||||
"before",
|
||||
"query",
|
||||
"string",
|
||||
False,
|
||||
"",
|
||||
"Devlog pagination cursor (devlog_next_cursor from the previous page).",
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
id="projects-screenshots",
|
||||
method="POST",
|
||||
path="/projects/{project_slug}/screenshots",
|
||||
title="Add screenshots to a project",
|
||||
summary="Link uploaded image attachments to an owned project's Screenshots gallery. Redirects to the gallery.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
params=[
|
||||
field(
|
||||
"project_slug",
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
"my-project-1a2b3c4d",
|
||||
"Project slug or uid.",
|
||||
),
|
||||
field(
|
||||
"attachment_uids",
|
||||
"form",
|
||||
"string",
|
||||
True,
|
||||
"ATTACHMENT_UID",
|
||||
"Comma-separated attachment uids from POST /uploads/upload or /uploads/upload-url.",
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
@ -443,6 +478,38 @@ four ways to sign requests.
|
||||
"31/12/2026",
|
||||
"Optional demo date in DD/MM/YYYY format.",
|
||||
),
|
||||
field(
|
||||
"website_url",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"https://myproject.dev",
|
||||
"Optional official website URL (http/https).",
|
||||
),
|
||||
field(
|
||||
"repo_url",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"https://github.com/me/project",
|
||||
"Optional source repository URL (http/https).",
|
||||
),
|
||||
field(
|
||||
"cover_attachment_uid",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"ATTACHMENT_UID",
|
||||
"Optional attachment uid of an uploaded cover image.",
|
||||
),
|
||||
field(
|
||||
"logo_attachment_uid",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"ATTACHMENT_UID",
|
||||
"Optional attachment uid of an uploaded project logo.",
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
@ -520,6 +587,38 @@ four ways to sign requests.
|
||||
"31/12/2026",
|
||||
"Optional demo date in DD/MM/YYYY format.",
|
||||
),
|
||||
field(
|
||||
"website_url",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"https://myproject.dev",
|
||||
"Optional official website URL (http/https).",
|
||||
),
|
||||
field(
|
||||
"repo_url",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"https://github.com/me/project",
|
||||
"Optional source repository URL (http/https).",
|
||||
),
|
||||
field(
|
||||
"cover_attachment_uid",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"ATTACHMENT_UID",
|
||||
"Optional attachment uid of an uploaded cover image.",
|
||||
),
|
||||
field(
|
||||
"logo_attachment_uid",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"ATTACHMENT_UID",
|
||||
"Optional attachment uid of an uploaded project logo.",
|
||||
),
|
||||
],
|
||||
),
|
||||
endpoint(
|
||||
|
||||
489
devplacepy/docs_api/groups/moderation.py
Normal file
489
devplacepy/docs_api/groups/moderation.py
Normal file
@ -0,0 +1,489 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from .._shared import endpoint, field
|
||||
from devplacepy.database.moderation import (
|
||||
CONSENT_KINDS,
|
||||
MODERATION_ACTIONS,
|
||||
REPORTABLE_TARGETS,
|
||||
REPORT_REASONS,
|
||||
REPORT_STATUSES,
|
||||
)
|
||||
|
||||
REPORT_TARGETS = list(REPORTABLE_TARGETS)
|
||||
REASON_KEYS = list(REPORT_REASONS)
|
||||
CONSENT_KEYS = list(CONSENT_KINDS)
|
||||
|
||||
SAMPLE_REPORT = {
|
||||
"uid": "REPORT_UID",
|
||||
"target_type": "post",
|
||||
"target_uid": "POST_UID",
|
||||
"target_url": "/posts/a-post",
|
||||
"reason": "harassment",
|
||||
"reason_label": "Harassment or bullying",
|
||||
"detail": "Repeated personal attacks in the thread.",
|
||||
"severity": "warn",
|
||||
"status": "open",
|
||||
"origin": "member",
|
||||
"categories": [],
|
||||
"created_at": "2026-01-05T10:00:00+00:00",
|
||||
"resolved_at": "",
|
||||
"reporter_name": "alice",
|
||||
"owner_name": "bob",
|
||||
"report_count": 2,
|
||||
}
|
||||
|
||||
GROUP = {
|
||||
"slug": "moderation",
|
||||
"title": "Reporting & Moderation",
|
||||
"intro": """
|
||||
# Reporting & Moderation
|
||||
|
||||
Every externally visible surface on DevPlace is reportable through one polymorphic
|
||||
endpoint, and every report lands in one queue with one state machine. The reason
|
||||
list is served by `GET /reports/reasons`, so a native client renders the same
|
||||
dialog the web UI does.
|
||||
|
||||
DevPlace commits to reviewing every report within the window published on the
|
||||
[content moderation](/docs/content-moderation.html) page. Filing a report always
|
||||
returns an acknowledgement carrying that window.
|
||||
|
||||
The moderation endpoints under `/admin/moderation` are administrator-only and are
|
||||
subject to the admin seniority rule: a junior administrator cannot action a more
|
||||
senior one.
|
||||
|
||||
Every endpoint follows the shared [Conventions & Errors](/docs/conventions.html) (auth, content
|
||||
negotiation, pagination, status codes); see [Authentication](/docs/authentication.html) for the
|
||||
four ways to sign requests.
|
||||
""",
|
||||
"endpoints": [
|
||||
endpoint(
|
||||
id="report-reasons",
|
||||
method="GET",
|
||||
path="/reports/reasons",
|
||||
title="List report reasons",
|
||||
summary="The reason keys a report may be filed under, with their labels.",
|
||||
auth="public",
|
||||
sample_response={
|
||||
"reasons": [{"key": "harassment", "label": "Harassment or bullying"}],
|
||||
"severities": ["info", "warn", "critical"],
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="report-create",
|
||||
method="POST",
|
||||
path="/reports/{target_type}/{target_uid}",
|
||||
title="Report content",
|
||||
summary="File a report against any user-generated surface.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
destructive=False,
|
||||
params=[
|
||||
field(
|
||||
"target_type",
|
||||
"path",
|
||||
"enum",
|
||||
True,
|
||||
"post",
|
||||
"The kind of content being reported.",
|
||||
REPORT_TARGETS,
|
||||
),
|
||||
field(
|
||||
"target_uid",
|
||||
"path",
|
||||
"string",
|
||||
True,
|
||||
"POST_UID",
|
||||
"UID of the reported item.",
|
||||
),
|
||||
field(
|
||||
"reason",
|
||||
"form",
|
||||
"enum",
|
||||
True,
|
||||
"harassment",
|
||||
"Why the content breaks the guidelines.",
|
||||
REASON_KEYS,
|
||||
),
|
||||
field(
|
||||
"detail",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"",
|
||||
"Free text for the moderator, up to 2000 characters.",
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"A second report on the same target by the same reporter updates the "
|
||||
"open report instead of creating a duplicate.",
|
||||
"You cannot report your own content.",
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/reports/mine",
|
||||
"data": {
|
||||
"uid": "REPORT_UID",
|
||||
"status": "open",
|
||||
"severity": "warn",
|
||||
"sla_hours": 24,
|
||||
},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="reports-mine",
|
||||
method="GET",
|
||||
path="/reports/mine",
|
||||
title="List your reports",
|
||||
summary="The reports you filed and the outcome of each.",
|
||||
auth="user",
|
||||
params=[
|
||||
field(
|
||||
"status",
|
||||
"query",
|
||||
"enum",
|
||||
False,
|
||||
"open",
|
||||
"Filter by report status.",
|
||||
list(REPORT_STATUSES),
|
||||
),
|
||||
field("page", "query", "integer", False, "1", "Page number."),
|
||||
],
|
||||
sample_response={
|
||||
"reports": [SAMPLE_REPORT],
|
||||
"pagination": {"page": 1, "total": 1, "total_pages": 1},
|
||||
"status": "",
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-moderation",
|
||||
method="GET",
|
||||
path="/admin/moderation",
|
||||
title="The moderation queue",
|
||||
summary="Reported content awaiting a decision, oldest open first.",
|
||||
auth="admin",
|
||||
params=[
|
||||
field(
|
||||
"status",
|
||||
"query",
|
||||
"enum",
|
||||
False,
|
||||
"open",
|
||||
"Filter by report status.",
|
||||
list(REPORT_STATUSES),
|
||||
),
|
||||
field("page", "query", "integer", False, "1", "Page number."),
|
||||
],
|
||||
sample_response={
|
||||
"reports": [SAMPLE_REPORT],
|
||||
"counts": {"open": 1, "acknowledged": 0, "actioned": 0, "dismissed": 0},
|
||||
"sla": {
|
||||
"sla_hours": 24,
|
||||
"oldest_open_hours": 1.5,
|
||||
"breached": 0,
|
||||
"within_sla": True,
|
||||
},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-moderation-report",
|
||||
method="GET",
|
||||
path="/admin/moderation/{uid}",
|
||||
title="Read one report",
|
||||
summary="One report with its decisions and the author's history.",
|
||||
auth="admin",
|
||||
params=[
|
||||
field("uid", "path", "string", True, "REPORT_UID", "Report UID."),
|
||||
],
|
||||
sample_response={
|
||||
"report": SAMPLE_REPORT,
|
||||
"actions": [],
|
||||
"history": [],
|
||||
"available_actions": list(MODERATION_ACTIONS),
|
||||
"can_remove": True,
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-moderation-status",
|
||||
method="POST",
|
||||
path="/admin/moderation/{uid}/status",
|
||||
title="Set a report status",
|
||||
summary="Move a report through the triage state machine.",
|
||||
auth="admin",
|
||||
encoding="form",
|
||||
params=[
|
||||
field("uid", "path", "string", True, "REPORT_UID", "Report UID."),
|
||||
field(
|
||||
"status",
|
||||
"form",
|
||||
"enum",
|
||||
True,
|
||||
"acknowledged",
|
||||
"New status.",
|
||||
list(REPORT_STATUSES),
|
||||
),
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/admin/moderation/REPORT_UID"},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-moderation-decide",
|
||||
method="POST",
|
||||
path="/admin/moderation/{uid}/decide",
|
||||
title="Decide a report",
|
||||
summary="Apply a moderation decision and notify the affected user.",
|
||||
auth="admin",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("uid", "path", "string", True, "REPORT_UID", "Report UID."),
|
||||
field(
|
||||
"action",
|
||||
"form",
|
||||
"enum",
|
||||
True,
|
||||
"dismiss",
|
||||
"The decision to apply.",
|
||||
list(MODERATION_ACTIONS),
|
||||
),
|
||||
field(
|
||||
"reason",
|
||||
"form",
|
||||
"string",
|
||||
False,
|
||||
"",
|
||||
"Reason shown to the affected user.",
|
||||
),
|
||||
field("notes", "form", "string", False, "", "Internal notes."),
|
||||
field(
|
||||
"duration_hours",
|
||||
"form",
|
||||
"integer",
|
||||
False,
|
||||
"24",
|
||||
"Suspension length in hours.",
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"A report already resolved by another moderator answers 409.",
|
||||
"Content removal is unavailable for targets that have no removal "
|
||||
"path (direct messages, accounts, workspaces, polls, assistant "
|
||||
"output); act on the account instead.",
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/admin/moderation/REPORT_UID"},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-user-suspend",
|
||||
method="POST",
|
||||
path="/admin/users/{uid}/suspend",
|
||||
title="Suspend an account",
|
||||
summary="Suspend an account for a fixed period with a stated reason.",
|
||||
auth="admin",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("uid", "path", "string", True, "USER_UID", "User UID."),
|
||||
field("reason", "form", "string", False, "", "Reason shown to the user."),
|
||||
field(
|
||||
"duration_hours",
|
||||
"form",
|
||||
"integer",
|
||||
False,
|
||||
"24",
|
||||
"Suspension length in hours.",
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"A suspended account can still read, still see why, and still delete "
|
||||
"itself, but cannot create content.",
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/admin/users"},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-user-lift",
|
||||
method="POST",
|
||||
path="/admin/users/{uid}/lift",
|
||||
title="Lift a restriction",
|
||||
summary="Clear a suspension or ban and restore the account.",
|
||||
auth="admin",
|
||||
encoding="form",
|
||||
params=[field("uid", "path", "string", True, "USER_UID", "User UID.")],
|
||||
sample_response={"ok": True, "redirect": "/admin/users"},
|
||||
),
|
||||
endpoint(
|
||||
id="admin-user-ban",
|
||||
method="POST",
|
||||
path="/admin/users/{uid}/ban",
|
||||
title="Ban an account",
|
||||
summary="Permanently close an account and revoke every credential.",
|
||||
auth="admin",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("uid", "path", "string", True, "USER_UID", "User UID."),
|
||||
field("reason", "form", "string", False, "", "Reason shown to the user."),
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/admin/users"},
|
||||
),
|
||||
endpoint(
|
||||
id="auth-accept-terms-page",
|
||||
method="GET",
|
||||
path="/auth/accept-terms",
|
||||
title="The terms acceptance page",
|
||||
summary="The Terms of Service version in force and the version this account accepted.",
|
||||
auth="user",
|
||||
sample_response={"terms_version": "1", "accepted_version": ""},
|
||||
),
|
||||
endpoint(
|
||||
id="auth-accept-terms",
|
||||
method="POST",
|
||||
path="/auth/accept-terms",
|
||||
title="Accept the terms",
|
||||
summary="Record acceptance of the Terms of Service version in force.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
notes=[
|
||||
"A member whose accepted version is behind the version in force is "
|
||||
"redirected here on any mutating request. Reading, the docs, the "
|
||||
"safety controls and account deletion are never blocked.",
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/feed", "data": {"terms_version": "1"}},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-consent",
|
||||
method="POST",
|
||||
path="/profile/{username}/consent",
|
||||
title="Grant or withdraw a consent",
|
||||
summary="Change one consent on your own account. Withdrawal takes effect at once.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
params=[
|
||||
field("username", "path", "string", True, "USERNAME", "Your own username."),
|
||||
field(
|
||||
"kind",
|
||||
"form",
|
||||
"enum",
|
||||
True,
|
||||
"ai_third_party",
|
||||
"Consent to change.",
|
||||
CONSENT_KEYS,
|
||||
),
|
||||
field("granted", "form", "enum", True, "1", "1 grants, 0 withdraws.", ["1", "0"]),
|
||||
],
|
||||
notes=[
|
||||
"Withdrawing `ai_third_party` makes the AI gateway refuse every call "
|
||||
"that would send your own content to the provider, whatever the "
|
||||
"per-feature preference says.",
|
||||
"Withdrawing `activity_recording` stops presence writes; you simply "
|
||||
"appear offline.",
|
||||
"Only the account holder can change a consent. An administrator "
|
||||
"reads the record but never grants or withdraws it for someone else.",
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/profile/USERNAME?tab=privacy",
|
||||
"data": {"kind": "ai_third_party", "state": "granted"},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-mature-content",
|
||||
method="POST",
|
||||
path="/profile/{username}/mature-content",
|
||||
title="Set the mature-content preference",
|
||||
summary="Show or hide content labelled mature for your own account.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
params=[
|
||||
field("username", "path", "string", True, "USERNAME", "Your own username."),
|
||||
field(
|
||||
"mature_opt_in",
|
||||
"form",
|
||||
"enum",
|
||||
True,
|
||||
"1",
|
||||
"1 shows mature content, 0 hides it.",
|
||||
["1", "0"],
|
||||
),
|
||||
],
|
||||
notes=[
|
||||
"Only the account holder can change this preference. An administrator "
|
||||
"reads the privacy tab but never sets it for someone else.",
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/profile/USERNAME?tab=privacy",
|
||||
"data": {"mature_opt_in": True},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-delete",
|
||||
method="GET",
|
||||
path="/profile/{username}/delete",
|
||||
title="Account deletion page",
|
||||
summary="What deletion removes, what is retained, and the grace window.",
|
||||
auth="user",
|
||||
params=[
|
||||
field("username", "path", "string", True, "USERNAME", "Your own username."),
|
||||
],
|
||||
sample_response={
|
||||
"username": "USERNAME",
|
||||
"grace_hours": 24,
|
||||
"removed": ["Your account record, username, email address and password"],
|
||||
"retained": ["Append-only audit and moderation records"],
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="profile-delete-confirm",
|
||||
method="POST",
|
||||
path="/profile/{username}/delete",
|
||||
title="Delete your account",
|
||||
summary="Permanently delete your account and personal data.",
|
||||
auth="user",
|
||||
encoding="form",
|
||||
destructive=True,
|
||||
params=[
|
||||
field("username", "path", "string", True, "USERNAME", "Your own username."),
|
||||
field("password", "form", "string", True, "PASSWORD", "Your account password."),
|
||||
],
|
||||
notes=[
|
||||
"Only the account holder can delete an account; an administrator uses "
|
||||
"a ban instead.",
|
||||
"Sessions and tokens are revoked and the profile is anonymised "
|
||||
"immediately; the deletion event is purged after the grace window.",
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/",
|
||||
"data": {"stamp": "2026-01-05T10:00:00+00:00", "rows": 42, "grace_hours": 24},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="workspaces-index",
|
||||
method="GET",
|
||||
path="/workspaces/index",
|
||||
title="Published workspace index",
|
||||
summary="Every workspace published to the public ingress, with its link.",
|
||||
auth="public",
|
||||
params=[field("page", "query", "integer", False, "1", "Page number.")],
|
||||
notes=[
|
||||
"The project-derived `description` and `project_url` come back empty "
|
||||
"unless you may view the workspace's project, so a private project "
|
||||
"never leaks its title or description through this public listing.",
|
||||
],
|
||||
sample_response={
|
||||
"workspaces": [
|
||||
{
|
||||
"uid": "INSTANCE_UID",
|
||||
"name": "demo",
|
||||
"slug": "demo",
|
||||
"owner_uid": "USER_UID",
|
||||
"url": "{{ base }}/p/demo",
|
||||
"description": "A demo workspace.",
|
||||
"owner": "alice",
|
||||
"maturity": "general",
|
||||
"project_url": "/projects/demo",
|
||||
}
|
||||
],
|
||||
"total": 1,
|
||||
},
|
||||
),
|
||||
],
|
||||
}
|
||||
@ -8,14 +8,22 @@ GROUP = {
|
||||
"intro": """
|
||||
# Dev Workspaces
|
||||
|
||||
A workspace is a browser VS Code environment attached to one of your projects. It runs your project
|
||||
files, a terminal, and preinstalled Python, Rust, Nim and Swift toolchains. `sudo` and
|
||||
`apt install` work with no extra setup; ports below 1024 cannot bind, so use a high port and publish
|
||||
it through a tunnel.
|
||||
A workspace is a browser editor attached to one of your projects. It runs your project files, a
|
||||
terminal, and preinstalled Python, Rust, Nim and Swift toolchains. `sudo` and `apt install` work
|
||||
with no extra setup; ports below 1024 cannot bind, so use a high port and publish it through a
|
||||
tunnel.
|
||||
|
||||
The editor opens with a **DevPlace Code** terminal already running the `dpc` coding agent and a
|
||||
plain shell beside it, and it trusts every folder, so nothing opens in Restricted Mode. Its
|
||||
appearance and boot behaviour are your own preferences, readable and writable through the two
|
||||
`/workspace/editor` endpoints below and explained on
|
||||
[the workspace editor page](/docs/workspace-editor.html). Editor preferences apply on the next
|
||||
workspace start.
|
||||
|
||||
A **tunnel** publishes one port from inside your container on a public HTTPS hostname of the form
|
||||
`<port>-<name>.tunnel.pravda.education`. **Tunnel URLs are public and unauthenticated** - anyone with
|
||||
the link can reach whatever you are serving.
|
||||
the link can reach whatever you are serving. Forwarding a port in the editor's **Ports** view creates
|
||||
the tunnel for you through the same endpoint; un-forwarding it does not remove the tunnel.
|
||||
|
||||
Workspaces are bounded: a count limit per user, a disk quota, an egress quota, and a tunnel limit.
|
||||
An idle workspace is warned about, then stopped, then warned again, then removed. Every warning
|
||||
@ -101,6 +109,88 @@ arrives as a `workspace` notification and states exactly what happens next and w
|
||||
],
|
||||
sample_response={"ok": True, "redirect": "/projects/my-project/workspace"},
|
||||
),
|
||||
endpoint(
|
||||
id="workspace-editor-get",
|
||||
method="GET",
|
||||
path="/projects/{slug}/workspace/editor",
|
||||
title="Read editor profile",
|
||||
summary=(
|
||||
"The resolved DevPlace editor profile for this workspace: theme, layout, "
|
||||
"panel preset, font sizes, zoom, boot terminals, how the editor opens, the "
|
||||
"container size, and where each value comes from."
|
||||
),
|
||||
auth="user",
|
||||
params=[
|
||||
field("slug", "path", "string", True, "my-project", "Project slug or uid."),
|
||||
],
|
||||
sample_response={
|
||||
"editor": {
|
||||
"trust_all": True,
|
||||
"theme": "devplace-dark",
|
||||
"font_size": 14,
|
||||
"terminal_font_size": 13,
|
||||
"zoom_level": 0,
|
||||
"layout": "standard",
|
||||
"panel_preset": "tall",
|
||||
"boot_agent": "dpc",
|
||||
"boot_shell": True,
|
||||
"window_mode": "tab",
|
||||
"window_width": 1600,
|
||||
"window_height": 1000,
|
||||
"cpu_millicores": 2000,
|
||||
"cpu_cores": 2.0,
|
||||
"memory_mb": 2048,
|
||||
"disk_quota_mb": 2048,
|
||||
"sources": {"theme": "user", "font_size": "site"},
|
||||
},
|
||||
"restart_required": False,
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="workspace-editor-set",
|
||||
method="POST",
|
||||
path="/projects/{slug}/workspace/editor",
|
||||
title="Set editor preferences",
|
||||
summary=(
|
||||
"Change your own editor preferences. Only the fields you send are "
|
||||
"changed; within those, an empty string or zero means inherit the site "
|
||||
"default, and `reset` drops every preference. Applies on the next "
|
||||
"workspace start, and the response says whether a restart is needed."
|
||||
),
|
||||
auth="user",
|
||||
params=[
|
||||
field("slug", "path", "string", True, "my-project", "Project slug or uid."),
|
||||
field("theme", "body", "string", False, "devplace-dark",
|
||||
"devplace-dark, devplace-light or system."),
|
||||
field("layout", "body", "string", False, "standard",
|
||||
"standard, terminal-focus or zen."),
|
||||
field("panel_preset", "body", "string", False, "tall",
|
||||
"short, normal, tall or maximized."),
|
||||
field("font_size", "body", "integer", False, "14",
|
||||
"Editor font size in pixels. Zero inherits."),
|
||||
field("terminal_font_size", "body", "integer", False, "13",
|
||||
"Terminal font size in pixels. Zero inherits."),
|
||||
field("zoom_level", "body", "integer", False, "0",
|
||||
"Window zoom, -5 to 5. Send -99 to inherit."),
|
||||
field("boot_agent", "body", "string", False, "dpc",
|
||||
"dpc or none."),
|
||||
field("boot_shell", "body", "integer", False, "1",
|
||||
"1 opens a shell on boot, 0 skips it, -1 inherits."),
|
||||
field("window_mode", "body", "string", False, "tab",
|
||||
"tab, window or fullscreen."),
|
||||
field("window_width", "body", "integer", False, "1600",
|
||||
"Editor window width in pixels. Zero inherits."),
|
||||
field("window_height", "body", "integer", False, "1000",
|
||||
"Editor window height in pixels. Zero inherits."),
|
||||
field("reset", "body", "boolean", False, "false",
|
||||
"Drop every preference and fall back to the site defaults."),
|
||||
],
|
||||
sample_response={
|
||||
"ok": True,
|
||||
"redirect": "/projects/my-project/workspace",
|
||||
"data": {"restart_required": True},
|
||||
},
|
||||
),
|
||||
endpoint(
|
||||
id="workspace-tunnels-list",
|
||||
method="GET",
|
||||
@ -133,7 +223,9 @@ arrives as a `workspace` notification and states exactly what happens next and w
|
||||
title="Create tunnel",
|
||||
summary=(
|
||||
"Publish a container port on a public HTTPS hostname. The URL is public "
|
||||
"and unauthenticated. Refused past the tunnel limit."
|
||||
"and unauthenticated. Refused past the tunnel limit. The certificate is "
|
||||
"ordered right away, so the hostname answers plain HTTP for a few seconds "
|
||||
"before it serves HTTPS. Forwarding a port in the editor calls this for you."
|
||||
),
|
||||
auth="user",
|
||||
params=[
|
||||
|
||||
@ -72,6 +72,7 @@ from devplacepy.routers import (
|
||||
push,
|
||||
leaderboard,
|
||||
reactions,
|
||||
reports,
|
||||
bookmarks,
|
||||
polls,
|
||||
docs,
|
||||
@ -86,6 +87,7 @@ from devplacepy.routers import (
|
||||
dbapi,
|
||||
pubsub,
|
||||
game,
|
||||
workspaces,
|
||||
)
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.services.background import background
|
||||
@ -115,6 +117,9 @@ from devplacepy.services.containers.service import ContainerService
|
||||
from devplacepy.services.containers.workspace_service import WorkspaceService
|
||||
from devplacepy.services.xmlrpc import XmlrpcService
|
||||
from devplacepy.services.audit import AuditService
|
||||
from devplacepy.services.moderation.service import ModerationService
|
||||
from devplacepy.services.moderation.screening import ContentRefused
|
||||
from devplacepy.services.acceptance.service import AcceptanceService
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.push import PushService
|
||||
from devplacepy.services.telegram import TelegramService
|
||||
@ -278,6 +283,8 @@ async def lifespan(app: FastAPI):
|
||||
service_manager.register(WorkspaceService())
|
||||
service_manager.register(XmlrpcService())
|
||||
service_manager.register(AuditService())
|
||||
service_manager.register(ModerationService())
|
||||
service_manager.register(AcceptanceService())
|
||||
service_manager.register(PushService())
|
||||
service_manager.register(TelegramService())
|
||||
service_manager.register(TelegramOutboxService())
|
||||
@ -302,6 +309,9 @@ async def lifespan(app: FastAPI):
|
||||
flush_visits()
|
||||
await service_manager.shutdown_all()
|
||||
await background.stop()
|
||||
from devplacepy.services.containers import forward
|
||||
|
||||
await forward.close_client()
|
||||
|
||||
|
||||
app = FastAPI(
|
||||
@ -371,6 +381,30 @@ async def server_error(request: Request, exc):
|
||||
)
|
||||
|
||||
|
||||
@app.exception_handler(ContentRefused)
|
||||
async def content_refused(request: Request, exc: ContentRefused):
|
||||
logger.info("content refused on %s %s: %s", request.method, request.url.path, exc.message)
|
||||
if wants_json(request):
|
||||
return json_error(400, exc.message, categories=list(exc.categories))
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Content not published - DevPlace",
|
||||
description=exc.message,
|
||||
robots="noindex",
|
||||
)
|
||||
return templates.TemplateResponse(
|
||||
request,
|
||||
"error.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"error_code": 400,
|
||||
"error_message": exc.message,
|
||||
},
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
|
||||
_AUTH_FORM_PAGES = {
|
||||
"/auth/signup": ("signup.html", "Join DevPlace"),
|
||||
"/auth/login": ("login.html", "Sign In"),
|
||||
@ -441,6 +475,7 @@ app.include_router(messages.router, prefix="/messages")
|
||||
app.include_router(notifications.router, prefix="/notifications")
|
||||
app.include_router(votes.router, prefix="/votes")
|
||||
app.include_router(reactions.router, prefix="/reactions")
|
||||
app.include_router(reports.router, prefix="/reports")
|
||||
app.include_router(bookmarks.router, prefix="/bookmarks")
|
||||
app.include_router(polls.router, prefix="/polls")
|
||||
app.include_router(avatar.router, prefix="/avatar")
|
||||
@ -469,6 +504,7 @@ app.include_router(dbapi.router, prefix="/dbapi")
|
||||
app.include_router(pubsub.router, prefix="/pubsub")
|
||||
app.include_router(game.router, prefix="/game")
|
||||
app.include_router(quizzes.router, prefix="/quizzes")
|
||||
app.include_router(workspaces.router, prefix="/workspaces")
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
@ -487,6 +523,15 @@ async def await_pending_corrections(request: Request, call_next):
|
||||
return response
|
||||
|
||||
|
||||
def _frame_ancestors() -> str:
|
||||
from devplacepy.services.containers.workspace import naming
|
||||
|
||||
tunnel_domain = naming.domain()
|
||||
if not tunnel_domain:
|
||||
return "'self'"
|
||||
return f"'self' https://*.{tunnel_domain}"
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def add_security_headers(request: Request, call_next):
|
||||
response = await call_next(request)
|
||||
@ -496,10 +541,9 @@ async def add_security_headers(request: Request, call_next):
|
||||
response.headers["Strict-Transport-Security"] = "max-age=31536000; includeSubDomains"
|
||||
response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
|
||||
if not request.url.path.startswith("/p/"):
|
||||
response.headers["X-Frame-Options"] = "DENY"
|
||||
response.headers["Content-Security-Policy"] = (
|
||||
"object-src 'none'; base-uri 'self'; "
|
||||
"frame-ancestors 'none'; form-action 'self'"
|
||||
f"frame-ancestors {_frame_ancestors()}; form-action 'self'"
|
||||
)
|
||||
if request.url.path.startswith("/admin"):
|
||||
response.headers["Cache-Control"] = "no-store, no-cache, must-revalidate, max-age=0"
|
||||
@ -589,6 +633,53 @@ async def maintenance_middleware(request: Request, call_next):
|
||||
)
|
||||
|
||||
|
||||
_TERMS_ALLOWED_PREFIXES = (
|
||||
"/static",
|
||||
"/avatar",
|
||||
"/auth",
|
||||
"/docs",
|
||||
"/reports",
|
||||
"/block",
|
||||
"/mute",
|
||||
"/openai",
|
||||
)
|
||||
|
||||
_TERMS_GATED_METHODS = ("POST", "PUT", "DELETE", "PATCH")
|
||||
|
||||
|
||||
def _terms_exempt(path: str) -> bool:
|
||||
if path.startswith(_TERMS_ALLOWED_PREFIXES):
|
||||
return True
|
||||
return path.startswith("/profile/") and (
|
||||
path.endswith("/delete") or path.endswith("/consent")
|
||||
)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def terms_acceptance_gate(request: Request, call_next):
|
||||
if request.method not in _TERMS_GATED_METHODS or _terms_exempt(request.url.path):
|
||||
return await call_next(request)
|
||||
from devplacepy.routers.auth.terms import (
|
||||
TERMS_ACCEPTANCE_CODE,
|
||||
current_terms_version,
|
||||
needs_acceptance,
|
||||
)
|
||||
|
||||
user = get_current_user(request)
|
||||
if not needs_acceptance(user):
|
||||
return await call_next(request)
|
||||
message = "Accept the updated Terms of Service to continue."
|
||||
if wants_json(request):
|
||||
return json_error(
|
||||
403,
|
||||
message,
|
||||
code=TERMS_ACCEPTANCE_CODE,
|
||||
redirect="/auth/accept-terms",
|
||||
terms_version=current_terms_version(),
|
||||
)
|
||||
return RedirectResponse(url="/auth/accept-terms", status_code=303)
|
||||
|
||||
|
||||
@app.middleware("http")
|
||||
async def track_presence(request: Request, call_next):
|
||||
path = request.url.path
|
||||
@ -741,7 +832,7 @@ async def landing(request: Request):
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="DevPlace - The Developer Social Network",
|
||||
description="Track industry shifts. Discover bold releases. Share what you're building in an open, uncensored environment.",
|
||||
description="Track industry shifts. Discover bold releases. Share what you're building in an open environment built by developers, for developers.",
|
||||
breadcrumbs=[],
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
|
||||
@ -5,7 +5,7 @@ import re
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal, Optional
|
||||
from urllib.parse import urlsplit
|
||||
from urllib.parse import urlsplit, urlparse
|
||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
||||
from devplacepy.constants import TOPICS
|
||||
from devplacepy.rendering import is_single_emoji
|
||||
@ -33,6 +33,20 @@ def normalize_european_date(value):
|
||||
raise ValueError("Date must be in DD/MM/YYYY format")
|
||||
|
||||
|
||||
def normalize_website_url(value):
|
||||
if not value:
|
||||
return ""
|
||||
text = str(value).strip()
|
||||
if not text:
|
||||
return ""
|
||||
if not text.lower().startswith(("http://", "https://")):
|
||||
text = f"https://{text}"
|
||||
parsed = urlparse(text)
|
||||
if parsed.scheme not in ("http", "https") or not parsed.hostname or "." not in parsed.hostname:
|
||||
raise ValueError("Website must be a valid http(s) URL")
|
||||
return text
|
||||
|
||||
|
||||
def normalize_poll_options(value):
|
||||
if value is None:
|
||||
return []
|
||||
@ -48,11 +62,53 @@ def normalize_poll_options(value):
|
||||
return value
|
||||
|
||||
|
||||
def _declared_age(birth_date: str) -> int:
|
||||
from datetime import date
|
||||
|
||||
from devplacepy.database.moderation import years_between
|
||||
|
||||
normalized = normalize_european_date(birth_date)
|
||||
if not normalized:
|
||||
raise ValueError("Date of birth is required")
|
||||
born = date.fromisoformat(normalized)
|
||||
today = date.today()
|
||||
if born > today:
|
||||
raise ValueError("Date of birth cannot be in the future")
|
||||
return years_between(born, today)
|
||||
|
||||
|
||||
class SignupForm(BaseModel):
|
||||
username: str = Field(min_length=3, max_length=32)
|
||||
email: str = Field(min_length=1, max_length=255)
|
||||
password: str = Field(min_length=6, max_length=128)
|
||||
confirm_password: str = Field(min_length=1, max_length=128)
|
||||
birth_date: str = Field(min_length=1, max_length=20)
|
||||
accept_terms: str = Field(default="")
|
||||
|
||||
@property
|
||||
def age_band(self) -> str:
|
||||
from devplacepy.database.moderation import age_band_for
|
||||
|
||||
return age_band_for(_declared_age(self.birth_date))
|
||||
|
||||
@field_validator("birth_date")
|
||||
@classmethod
|
||||
def old_enough(cls, value):
|
||||
from devplacepy.database import minimum_age
|
||||
|
||||
minimum = minimum_age()
|
||||
if _declared_age(value) < minimum:
|
||||
raise ValueError(f"You must be at least {minimum} years old to join")
|
||||
return value
|
||||
|
||||
@field_validator("accept_terms")
|
||||
@classmethod
|
||||
def terms_accepted(cls, value):
|
||||
if value.strip().lower() not in ("1", "on", "true", "yes"):
|
||||
raise ValueError(
|
||||
"You must accept the Terms of Service and Community Guidelines"
|
||||
)
|
||||
return value
|
||||
|
||||
@field_validator("username")
|
||||
@classmethod
|
||||
@ -63,6 +119,10 @@ class SignupForm(BaseModel):
|
||||
raise ValueError(
|
||||
"Username can only contain letters, numbers, hyphens, and underscores"
|
||||
)
|
||||
from devplacepy.services.moderation.filter import classify
|
||||
|
||||
if classify(value).verdict == "block":
|
||||
raise ValueError("That username breaks the community guidelines")
|
||||
return value
|
||||
|
||||
@field_validator("email")
|
||||
@ -195,6 +255,10 @@ class ProjectForm(BaseModel):
|
||||
)
|
||||
platforms: str = Field(default="", max_length=500)
|
||||
status: str = Field(default="In Development", max_length=100)
|
||||
website_url: str = Field(default="", max_length=500)
|
||||
repo_url: str = Field(default="", max_length=500)
|
||||
cover_attachment_uid: str = Field(default="", max_length=64)
|
||||
logo_attachment_uid: str = Field(default="", max_length=64)
|
||||
is_private: bool = False
|
||||
attachment_uids: list[str] = []
|
||||
|
||||
@ -203,6 +267,11 @@ class ProjectForm(BaseModel):
|
||||
def normalize_dates(cls, value):
|
||||
return normalize_european_date(value)
|
||||
|
||||
@field_validator("website_url", "repo_url")
|
||||
@classmethod
|
||||
def valid_link_url(cls, value):
|
||||
return normalize_website_url(value)
|
||||
|
||||
|
||||
class ProjectEditForm(BaseModel):
|
||||
title: str = Field(min_length=1, max_length=200)
|
||||
@ -214,12 +283,21 @@ class ProjectEditForm(BaseModel):
|
||||
)
|
||||
platforms: str = Field(default="", max_length=500)
|
||||
status: str = Field(default="In Development", max_length=100)
|
||||
website_url: str = Field(default="", max_length=500)
|
||||
repo_url: str = Field(default="", max_length=500)
|
||||
cover_attachment_uid: str = Field(default="", max_length=64)
|
||||
logo_attachment_uid: str = Field(default="", max_length=64)
|
||||
|
||||
@field_validator("release_date", "demo_date", mode="before")
|
||||
@classmethod
|
||||
def normalize_dates(cls, value):
|
||||
return normalize_european_date(value)
|
||||
|
||||
@field_validator("website_url", "repo_url")
|
||||
@classmethod
|
||||
def valid_link_url(cls, value):
|
||||
return normalize_website_url(value)
|
||||
|
||||
|
||||
class BackupRunForm(BaseModel):
|
||||
target: Literal["database", "uploads", "keys", "full"] = "full"
|
||||
@ -244,6 +322,10 @@ class ProjectFlagForm(BaseModel):
|
||||
value: bool = False
|
||||
|
||||
|
||||
class ProjectScreenshotsForm(BaseModel):
|
||||
attachment_uids: list[str] = []
|
||||
|
||||
|
||||
class CustomizationToggleForm(BaseModel):
|
||||
value: bool = False
|
||||
|
||||
@ -587,8 +669,29 @@ class AdminSettingsForm(BaseModel):
|
||||
maintenance_message: str = Field(default="", max_length=300)
|
||||
docs_search_mode: str = Field(default="", max_length=20)
|
||||
outbound_proxy_url: str = Field(default="", max_length=500)
|
||||
moderation_sla_hours: str = Field(default="", max_length=10)
|
||||
moderation_filter_mode: str = Field(default="", max_length=10)
|
||||
moderation_minimum_age: str = Field(default="", max_length=3)
|
||||
moderation_mature_default_hidden: str = Field(default="", max_length=1)
|
||||
account_deletion_grace_hours: str = Field(default="", max_length=10)
|
||||
contact_email: str = Field(default="", max_length=200)
|
||||
contact_phone: str = Field(default="", max_length=60)
|
||||
contact_address: str = Field(default="", max_length=500)
|
||||
terms_version: str = Field(default="", max_length=20)
|
||||
privacy_version: str = Field(default="", max_length=20)
|
||||
guidelines_version: str = Field(default="", max_length=20)
|
||||
ai_third_party_provider: str = Field(default="", max_length=120)
|
||||
extra_head: str = Field(default="", max_length=50000)
|
||||
|
||||
@field_validator("moderation_filter_mode")
|
||||
@classmethod
|
||||
def validate_filter_mode(cls, value):
|
||||
from devplacepy.services.moderation.rules import FILTER_MODES
|
||||
|
||||
if value and value not in FILTER_MODES:
|
||||
raise ValueError(f"Filter mode must be one of {', '.join(FILTER_MODES)}")
|
||||
return value
|
||||
|
||||
@field_validator("outbound_proxy_url")
|
||||
@classmethod
|
||||
def validate_outbound_proxy_url(cls, value):
|
||||
@ -892,6 +995,21 @@ class TunnelForm(BaseModel):
|
||||
container_port: int = Field(default=0, ge=0, le=65535)
|
||||
|
||||
|
||||
class EditorPrefsForm(BaseModel):
|
||||
font_size: int = Field(default=0, ge=0, le=48)
|
||||
terminal_font_size: int = Field(default=0, ge=0, le=48)
|
||||
zoom_level: int = Field(default=-99, ge=-99, le=5)
|
||||
theme: str = Field(default="", max_length=32)
|
||||
layout: str = Field(default="", max_length=32)
|
||||
panel_preset: str = Field(default="", max_length=32)
|
||||
window_mode: str = Field(default="", max_length=32)
|
||||
window_width: int = Field(default=0, ge=0, le=7680)
|
||||
window_height: int = Field(default=0, ge=0, le=4320)
|
||||
boot_agent: str = Field(default="", max_length=32)
|
||||
boot_shell: int = Field(default=-1, ge=-1, le=1)
|
||||
reset: bool = False
|
||||
|
||||
|
||||
class WorkspaceQuotaForm(BaseModel):
|
||||
owner_id: str = Field(default="", max_length=36)
|
||||
label: str = Field(default="", max_length=64)
|
||||
@ -901,6 +1019,8 @@ class WorkspaceQuotaForm(BaseModel):
|
||||
egress_quota_mb: int = Field(default=0, ge=0)
|
||||
idle_stop_minutes: int = Field(default=0, ge=0)
|
||||
retention_days: int = Field(default=0, ge=0)
|
||||
cpu_millicores: int = Field(default=0, ge=0, le=64000)
|
||||
memory_mb: int = Field(default=0, ge=0, le=1048576)
|
||||
|
||||
|
||||
class WorkspaceFlagForm(BaseModel):
|
||||
@ -911,3 +1031,91 @@ class WorkspaceFlagForm(BaseModel):
|
||||
|
||||
class WorkspaceSuspendForm(BaseModel):
|
||||
reason: str = Field(default="", max_length=500)
|
||||
|
||||
|
||||
class ReportForm(BaseModel):
|
||||
reason: str = Field(min_length=1, max_length=40)
|
||||
detail: str = Field(default="", max_length=2000)
|
||||
|
||||
@field_validator("reason")
|
||||
@classmethod
|
||||
def known_reason(cls, value):
|
||||
from devplacepy.database.moderation import REPORT_REASONS
|
||||
|
||||
if value not in REPORT_REASONS:
|
||||
raise ValueError("Unknown report reason")
|
||||
return value
|
||||
|
||||
|
||||
class ReportStatusForm(BaseModel):
|
||||
status: str = Field(default="acknowledged", max_length=20)
|
||||
|
||||
@field_validator("status")
|
||||
@classmethod
|
||||
def known_status(cls, value):
|
||||
from devplacepy.database.moderation import REPORT_STATUSES
|
||||
|
||||
if value not in REPORT_STATUSES:
|
||||
raise ValueError("Unknown report status")
|
||||
return value
|
||||
|
||||
|
||||
class ModerationDecisionForm(BaseModel):
|
||||
action: str = Field(min_length=1, max_length=30)
|
||||
reason: str = Field(default="", max_length=200)
|
||||
notes: str = Field(default="", max_length=2000)
|
||||
duration_hours: int = Field(default=24, ge=1, le=8760)
|
||||
|
||||
@field_validator("action")
|
||||
@classmethod
|
||||
def known_action(cls, value):
|
||||
from devplacepy.database.moderation import MODERATION_ACTIONS
|
||||
|
||||
if value not in MODERATION_ACTIONS:
|
||||
raise ValueError("Unknown moderation action")
|
||||
return value
|
||||
|
||||
|
||||
class SuspensionForm(BaseModel):
|
||||
reason: str = Field(default="", max_length=200)
|
||||
duration_hours: int = Field(default=24, ge=1, le=8760)
|
||||
|
||||
|
||||
class BanForm(BaseModel):
|
||||
reason: str = Field(default="", max_length=200)
|
||||
|
||||
|
||||
class ConsentForm(BaseModel):
|
||||
kind: str = Field(min_length=1, max_length=40)
|
||||
granted: str = Field(default="0", max_length=5)
|
||||
|
||||
@field_validator("kind")
|
||||
@classmethod
|
||||
def known_kind(cls, value):
|
||||
from devplacepy.database.moderation import CONSENT_KINDS
|
||||
|
||||
if value not in CONSENT_KINDS:
|
||||
raise ValueError("Unknown consent kind")
|
||||
return value
|
||||
|
||||
|
||||
class MaturityForm(BaseModel):
|
||||
level: str = Field(default="general", max_length=20)
|
||||
|
||||
@field_validator("level")
|
||||
@classmethod
|
||||
def known_level(cls, value):
|
||||
from devplacepy.database.moderation import MATURITY_LEVELS
|
||||
|
||||
if value not in MATURITY_LEVELS:
|
||||
raise ValueError("Unknown maturity level")
|
||||
return value
|
||||
|
||||
|
||||
class MaturePreferenceForm(BaseModel):
|
||||
mature_opt_in: str = Field(default="0", max_length=5)
|
||||
|
||||
|
||||
class AccountDeleteForm(BaseModel):
|
||||
password: str = Field(min_length=1, max_length=128)
|
||||
confirm_text: str = Field(default="", max_length=40)
|
||||
|
||||
@ -666,6 +666,9 @@ IMPORT_SKIP_NAMES = {
|
||||
SYNC_SKIP_NAMES = IMPORT_SKIP_NAMES | {
|
||||
".devplace_boot.py",
|
||||
".devplace_boot.sh",
|
||||
".devplace",
|
||||
".dpc",
|
||||
"dpc.log",
|
||||
}
|
||||
IMPORT_MAX_FILE_BYTES = 25 * 1024 * 1024
|
||||
|
||||
|
||||
@ -23,9 +23,11 @@ Prefixes are wired in `main.py`:
|
||||
| `/polls` | polls.py - poll voting: `POST /polls/{poll_uid}/vote` |
|
||||
| `/avatar` | avatar.py |
|
||||
| `/follow` | follow.py |
|
||||
| `/reports` | reports.py - polymorphic content reporting: `POST /reports/{target_type}/{target_uid}` (member), `GET /reports/mine` (member), `GET /reports/reasons` (public). See `devplacepy/services/moderation/CLAUDE.md` |
|
||||
| `/workspaces` | workspaces.py - `GET /workspaces/index`, the public index of every workspace published to the `/p/{slug}` ingress, with owner, project, maturity label and absolute link. Indexed in the sitemap. Publishing an ingress slug is the deliberate public act, so the workspace itself is always listed, but the project-derived fields (`description` and `project_url`, whose slug carries the project title) are withheld unless `content.can_view_project(project, viewer)` passes - a private project must not leak its title or description through this public listing |
|
||||
| (none) | relations.py - per-user block/mute relations: `POST /block/{username}`, `/block/unblock/{username}`, `/mute/{username}`, `/mute/unmute/{username}` (soft-deletable `user_relations` rows) |
|
||||
| `/leaderboard` | leaderboard.py - `GET /leaderboard` XP/stars leaderboard page |
|
||||
| `/admin` | admin/ package - one leaf per sub-resource (`index`, `users`, `aiusage`, `aiquota`, `media`, `trash`, `settings`, `notifications`, `news`, `auditlog`, `backups`, `game`) plus the folded-in `services.py` and `containers.py` (mounted with `/services` and `/containers` sub-prefixes). `main.py` mounts the whole `/admin` tree from this one package. The `backups` leaf is the admin **Backups** dashboard (`BackupService`, kind `backup`): storage usage, backup archives, and interval/cron backup schedules (CRUD + rotation). **Archive download is restricted to the primary administrator** (the earliest-created Admin, resolved by `database.get_primary_admin_uid` / `utils.is_primary_admin`): `GET /admin/backups/{uid}/download` 403s every other admin, the `download_url` field is withheld from them at every endpoint (`can_download = is_primary_admin(admin)`, surfaced as `BackupDashboardOut.can_download_backups`), and `BackupMonitor.js` renders their Download control as a disabled button tooltipped `Not available`. See `devplacepy/services/backup/CLAUDE.md`. The `game` leaf (`/admin/game`) is the Code Farm Era admin page: `GET /admin/game` (status), `POST /admin/game/era/start` and `/era/end` - see `devplacepy/services/game/CLAUDE.md` |
|
||||
| `/admin` | admin/ package - one leaf per sub-resource (`index`, `users`, `aiusage`, `aiquota`, `media`, `trash`, `settings`, `notifications`, `news`, `auditlog`, `backups`, `game`) plus the folded-in `services.py` and `containers.py` (mounted with `/services` and `/containers` sub-prefixes). `main.py` mounts the whole `/admin` tree from this one package. The `backups` leaf is the admin **Backups** dashboard (`BackupService`, kind `backup`): storage usage, backup archives, and interval/cron backup schedules (CRUD + rotation). **Archive download is restricted to the primary administrator** (the earliest-created Admin, resolved by `database.get_primary_admin_uid` / `utils.is_primary_admin`): `GET /admin/backups/{uid}/download` 403s every other admin, the `download_url` field is withheld from them at every endpoint (`can_download = is_primary_admin(admin)`, surfaced as `BackupDashboardOut.can_download_backups`), and `BackupMonitor.js` renders their Download control as a disabled button tooltipped `Not available`. See `devplacepy/services/backup/CLAUDE.md`. The `moderation` leaf (`/admin/moderation`) is the report queue: the list (oldest-open-first, status tabs, the SLA badge), the per-report detail with the offender's history, `POST /{uid}/status` for triage and `POST /{uid}/decide` for decisions; the per-user enforcement routes `POST /admin/users/{uid}/{suspend,lift,ban}` live in the `users` leaf alongside the legacy `toggle`. Both share `is_senior_admin`/`deny_senior` from `admin/_shared.py`. The `game` leaf (`/admin/game`) is the Code Farm Era admin page: `GET /admin/game` (status), `POST /admin/game/era/start` and `/era/end` - see `devplacepy/services/game/CLAUDE.md` |
|
||||
| `/admin/services` | admin/services.py |
|
||||
| `/issues` | issues/ package - issue tracker backed by Gitea (no local issue store): `index.py` (list `?state=`/`?page=`, detail `/{number}` with comments), `create.py` (async AI-enhanced filing `/create` enqueues a `issue_create` job, status at `/jobs/{uid}`), `comment.py` (synchronous, pushes to Gitea + notifies admins), `status.py` (admin open/closed), `attachments.py` (file attachments on open issues + comments, mirrored to Gitea native assets; add/list/delete with owner-or-admin + open-state guards) |
|
||||
| `/gists` | gists.py |
|
||||
@ -60,6 +62,7 @@ Every page/redirect endpoint also returns JSON when the client asks. Core in `de
|
||||
|
||||
- **Page GETs:** `return respond(request, "x.html", context, model=XOut)` - HTML renders the template; JSON does `XOut.model_validate(context).model_dump()`. One context, two renderings.
|
||||
- **Action POSTs:** `return action_result(request, url, data=<resource|None>)` - HTML 302 redirects; JSON returns `{ok, redirect, data}`. (Set cookies on the returned response after calling it, as `auth.py` login/signup do.)
|
||||
- **Refusals:** `return json_error(status_code, message)` - **status first, message second**. Swapping them is not a lint-level mistake, it is a guaranteed 500: `JSONResponse(..., status_code="some message")` raises `TypeError` inside `Response.init_headers` before any byte is written, so the branch that was supposed to explain a limit to the user crashes instead. It was a real production bug - every refusal in the two workspace routers was written `json_error(message, status)`, so a member already at the default 2-workspace quota got a 500 from the workspace page's **Open workspace** button rather than the quota message. `tests/unit/responses.py` now AST-scans the whole package for both argument orders, so the swap cannot come back anywhere.
|
||||
|
||||
Response models live in `devplacepy/schemas.py` (Pydantic v2, `extra="ignore"`, all-Optional so they validate the existing context dicts directly). **Always project users through `UserOut`** (and `AdminUserOut`) - the raw user rows contain `email`/`api_key`/`password_hash`, and the models drop them; never serialize a raw user row. List item shapes vary (feed/gists/news/admin-news are wrapped `{post|gist|article: ...}`; projects are flat rows with `author_name`/`my_vote`) - match the context exactly. Errors negotiate centrally: `main.py` 404/500/validation handlers and the rate-limit/maintenance middleware, plus `utils.require_user`/`require_admin` (401/403 for JSON, 303 redirect for browsers). The four legacy AJAX endpoints (votes/reactions/bookmarks/polls) keep their original flat JSON shapes and are left untouched. Documented in `docs_api.py`'s Conventions group.
|
||||
|
||||
@ -314,6 +317,14 @@ All SEO features are implemented across the following locations:
|
||||
- `database.py` helpers: `get_follow_counts(uid)` (`{followers, following}`), `get_follow_list(uid, mode, page)` (paginated people + `build_pagination`, ordered newest-first), and `get_following_among(follower_uid, target_uids)` (single IN-clause set used to set `is_following` per row, avoiding N+1). `mode` is `"followers"` (people who follow `uid`) or `"following"` (people `uid` follows).
|
||||
- Devii catalog tools `list_followers` / `list_following` (`requires_auth=False`) map to the JSON endpoints; documented in `docs_api.py` under the `profiles` group.
|
||||
|
||||
### Reporting and moderation
|
||||
|
||||
`_report_button.html` is the single report control, included with the same two-variable idiom as `_reaction_bar.html` at **fifteen** sites (`_post_card`, `_comment`, `post`, `gist_detail`, `project_detail`, `news_detail`, `quiz`, `_media_gallery`, `_awards_gallery`, `messages`, `profile`, `project_files`, `issue_detail`, `containers_instance`, `workspace_index`). Locals: `_type`, `_uid`, `_owner` (owner uid, so the control hides on your own content), `_owner_name` (optional; when present the partial also renders the **Block** form, which is what makes blocking reachable from the content rather than only from a profile) and `_class` (the surrounding button class so it inherits each surface's visual language).
|
||||
|
||||
`_report_dialog.html` is included once in `base.html` for signed-in users and driven by `static/js/ReportDialog.js` (`app.reportDialog`) through the standard `.modal-overlay`/`.visible` pattern and `Http.sendForm`. The reason list is the `REPORT_REASONS` Jinja global, sourced from `database/moderation.py`, so the dialog, the API enum, the docs enum and the guidelines page can never drift.
|
||||
|
||||
An e2e coverage test asserts the control is reachable on every include site; the registry test asserts every reportable target resolves. See `devplacepy/services/moderation/CLAUDE.md`.
|
||||
|
||||
### Block and mute (`routers/relations.py`)
|
||||
A logged-in user can **block** or **mute** another user; both are one-directional and reversible. **Block** hides every piece of the blocked user's content from the blocker - posts, comments (any category), feed, listings, issue list, detail pages, and DMs - everywhere EXCEPT the blocked user's own profile page (kept fully visible so the blocker can review and unblock), and it also suppresses any notification that user would generate. **Mute** is the lighter option: it only suppresses the muted user's notifications while their content stays visible. The blocked/muted user is unaffected and is not told.
|
||||
|
||||
|
||||
@ -14,6 +14,7 @@ from devplacepy.routers.admin import (
|
||||
gateway_configs,
|
||||
issues,
|
||||
media,
|
||||
moderation,
|
||||
news,
|
||||
notifications,
|
||||
services,
|
||||
@ -30,6 +31,7 @@ router.include_router(aiusage.router)
|
||||
router.include_router(statistics.router)
|
||||
router.include_router(aiquota.router)
|
||||
router.include_router(media.router)
|
||||
router.include_router(moderation.router)
|
||||
router.include_router(trash.router)
|
||||
router.include_router(settings.router)
|
||||
router.include_router(notifications.router)
|
||||
|
||||
@ -2,6 +2,45 @@
|
||||
|
||||
import json
|
||||
|
||||
from fastapi import Request
|
||||
|
||||
from devplacepy.responses import action_result
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
|
||||
def seniority_key(user: dict) -> tuple[str, int]:
|
||||
return (user.get("created_at") or "", user.get("id") or 0)
|
||||
|
||||
|
||||
def is_senior_admin(actor: dict, target: dict | None) -> bool:
|
||||
if not target or target.get("role") != "Admin":
|
||||
return False
|
||||
if target.get("uid") == actor.get("uid"):
|
||||
return False
|
||||
return seniority_key(target) < seniority_key(actor)
|
||||
|
||||
|
||||
def deny_senior(
|
||||
request: Request,
|
||||
admin: dict,
|
||||
uid: str,
|
||||
target: dict,
|
||||
event_key: str,
|
||||
redirect_url: str = "/admin/users",
|
||||
):
|
||||
audit.record(
|
||||
request,
|
||||
event_key,
|
||||
user=admin,
|
||||
result="denied",
|
||||
target_type="user",
|
||||
target_uid=uid,
|
||||
target_label=target.get("username"),
|
||||
summary=f"admin {admin['username']} cannot manage senior admin {target.get('username')}",
|
||||
links=[audit.target("user", uid, target.get("username"))],
|
||||
)
|
||||
return action_result(request, redirect_url)
|
||||
|
||||
|
||||
def parse_metadata(raw: str | dict | None) -> dict | None:
|
||||
if not raw:
|
||||
|
||||
323
devplacepy/routers/admin/moderation.py
Normal file
323
devplacepy/routers/admin/moderation.py
Normal file
@ -0,0 +1,323 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.database import (
|
||||
REPORT_STATUSES,
|
||||
SYSTEM_ACTOR,
|
||||
get_table,
|
||||
get_users_by_uids,
|
||||
)
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import ModerationDecisionForm, ReportStatusForm
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.routers.admin._shared import deny_senior, is_senior_admin
|
||||
from devplacepy.schemas import AdminModerationOut, AdminReportOut
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.moderation import enforcement, queue, sla
|
||||
from devplacepy.utils import create_notification, not_found, require_admin
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
QUEUE_URL = "/admin/moderation"
|
||||
|
||||
STATUS_TABS = [
|
||||
{"key": "open", "label": "Open"},
|
||||
{"key": "acknowledged", "label": "Acknowledged"},
|
||||
{"key": "actioned", "label": "Actioned"},
|
||||
{"key": "dismissed", "label": "Dismissed"},
|
||||
]
|
||||
|
||||
SUBJECT_ACTIONS = ("warn", "suspend", "ban", "lift")
|
||||
|
||||
ENFORCEMENT_EVENTS = {
|
||||
"remove_content": "moderation.remove",
|
||||
"restore_content": "moderation.restore",
|
||||
"warn": "moderation.warn",
|
||||
"suspend": "moderation.suspend",
|
||||
"ban": "moderation.ban",
|
||||
"lift": "moderation.lift",
|
||||
}
|
||||
|
||||
DECISION_MESSAGES = {
|
||||
"remove_content": "Your {target} was removed after a moderation review.",
|
||||
"restore_content": "Your {target} was restored after a moderation review.",
|
||||
"warn": "A moderator issued a warning about your {target}.",
|
||||
"suspend": "Your account is suspended following a moderation review.",
|
||||
"ban": "Your account has been closed following a moderation review.",
|
||||
"lift": "Your account restriction has been lifted.",
|
||||
}
|
||||
|
||||
|
||||
def _breadcrumbs(extra: list[dict] | None = None) -> list[dict]:
|
||||
trail = [
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Admin", "url": "/admin"},
|
||||
{"name": "Moderation", "url": QUEUE_URL},
|
||||
]
|
||||
return trail + (extra or [])
|
||||
|
||||
|
||||
def _available_actions(target_type: str) -> list[str]:
|
||||
actions = ["dismiss", "escalate"]
|
||||
if enforcement.can_remove(target_type):
|
||||
actions = ["remove_content", "restore_content"] + actions
|
||||
return actions + list(SUBJECT_ACTIONS)
|
||||
|
||||
|
||||
def _subject(report: dict) -> dict | None:
|
||||
owner_uid = report.get("owner_uid") or ""
|
||||
if not owner_uid:
|
||||
return None
|
||||
return get_users_by_uids([owner_uid]).get(owner_uid)
|
||||
|
||||
|
||||
def _action_view(rows: list[dict]) -> list[dict]:
|
||||
actors = get_users_by_uids([row.get("actor_uid") for row in rows if row.get("actor_uid")])
|
||||
view = []
|
||||
for row in rows:
|
||||
actor = actors.get(row.get("actor_uid"))
|
||||
view.append(
|
||||
{
|
||||
"uid": row["uid"],
|
||||
"report_uid": row.get("report_uid", ""),
|
||||
"action": row.get("action", ""),
|
||||
"actor_name": actor["username"] if actor else row.get("actor_uid", ""),
|
||||
"reason": row.get("reason", ""),
|
||||
"notes": row.get("notes", ""),
|
||||
"expires_at": row.get("expires_at", ""),
|
||||
"created_at": row.get("created_at", ""),
|
||||
}
|
||||
)
|
||||
return view
|
||||
|
||||
|
||||
@router.get("/moderation", response_class=HTMLResponse)
|
||||
async def admin_moderation(request: Request, status: str = "open", page: int = 1):
|
||||
admin = require_admin(request)
|
||||
if status not in REPORT_STATUSES:
|
||||
status = "open"
|
||||
reports, pagination = queue.list_reports(status=status, page=page)
|
||||
counts = queue.status_counts()
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Moderation - Admin",
|
||||
description="Triage reported content and apply moderation decisions.",
|
||||
robots="noindex,nofollow",
|
||||
breadcrumbs=_breadcrumbs(),
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
return respond(
|
||||
request,
|
||||
"admin_moderation.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"reports": reports,
|
||||
"pagination": pagination,
|
||||
"status": status,
|
||||
"statuses": [
|
||||
{**tab, "count": counts.get(tab["key"], 0), "active": tab["key"] == status}
|
||||
for tab in STATUS_TABS
|
||||
],
|
||||
"counts": counts,
|
||||
"sla": sla.snapshot(),
|
||||
"admin_section": "moderation",
|
||||
},
|
||||
model=AdminModerationOut,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/moderation/{uid}", response_class=HTMLResponse)
|
||||
async def admin_report_detail(request: Request, uid: str):
|
||||
admin = require_admin(request)
|
||||
report = queue.get_report(uid)
|
||||
if not report:
|
||||
raise not_found("Report not found")
|
||||
view = queue.enrich_reports([report])[0]
|
||||
subject = _subject(report)
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title=f"Report {uid} - Admin",
|
||||
description="One reported item and the decisions taken on it.",
|
||||
robots="noindex,nofollow",
|
||||
breadcrumbs=_breadcrumbs([{"name": "Report", "url": f"{QUEUE_URL}/{uid}"}]),
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
return respond(
|
||||
request,
|
||||
"admin_report.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": admin,
|
||||
"report": view,
|
||||
"actions": _action_view(queue.actions_for_report(uid)),
|
||||
"history": _action_view(
|
||||
queue.actions_for_subject(report.get("owner_uid", ""))
|
||||
),
|
||||
"available_actions": _available_actions(report["target_type"]),
|
||||
"can_remove": enforcement.can_remove(report["target_type"]),
|
||||
"subject": subject,
|
||||
"sla": sla.snapshot(),
|
||||
"admin_section": "moderation",
|
||||
},
|
||||
model=AdminReportOut,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/moderation/{uid}/status")
|
||||
async def admin_report_status(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[ReportStatusForm, Depends(json_or_form(ReportStatusForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
report = queue.get_report(uid)
|
||||
if not report:
|
||||
raise not_found("Report not found")
|
||||
updated = queue.set_status(uid, data.status, admin["uid"])
|
||||
if not updated:
|
||||
return json_error(400, "Report status could not be changed")
|
||||
logger.info(f"Admin {admin['username']} set report {uid} to {data.status}")
|
||||
audit.record(
|
||||
request,
|
||||
"report.status",
|
||||
user=admin,
|
||||
target_type=report["target_type"],
|
||||
target_uid=report["target_uid"],
|
||||
old_value=report.get("status"),
|
||||
new_value=data.status,
|
||||
metadata={"report_uid": uid},
|
||||
summary=f"{admin['username']} set report {uid} to {data.status}",
|
||||
links=[audit.target(report["target_type"], report["target_uid"])],
|
||||
)
|
||||
return action_result(request, f"{QUEUE_URL}/{uid}")
|
||||
|
||||
|
||||
@router.post("/moderation/{uid}/decide")
|
||||
async def admin_report_decide(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[ModerationDecisionForm, Depends(json_or_form(ModerationDecisionForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
report = queue.get_report(uid)
|
||||
if not report:
|
||||
raise not_found("Report not found")
|
||||
action = data.action
|
||||
if action not in _available_actions(report["target_type"]):
|
||||
return json_error(400, "That action does not apply to this target")
|
||||
subject = _subject(report)
|
||||
if action in SUBJECT_ACTIONS:
|
||||
if not subject:
|
||||
return json_error(400, "This report has no account to act on")
|
||||
if is_senior_admin(admin, subject):
|
||||
return deny_senior(
|
||||
request,
|
||||
admin,
|
||||
subject["uid"],
|
||||
subject,
|
||||
f"moderation.{action}",
|
||||
redirect_url=f"{QUEUE_URL}/{uid}",
|
||||
)
|
||||
if action == "escalate":
|
||||
queue.escalate(uid)
|
||||
else:
|
||||
outcome = "dismissed" if action == "dismiss" else "actioned"
|
||||
if not queue.claim_open(uid, outcome, admin["uid"]):
|
||||
return json_error(409, "This report was already resolved")
|
||||
expires_at = ""
|
||||
if action == "remove_content":
|
||||
enforcement.remove_content(
|
||||
request, admin, report["target_type"], report["target_uid"]
|
||||
)
|
||||
elif action == "restore_content":
|
||||
enforcement.restore_content(report["target_type"], report["target_uid"])
|
||||
elif action == "suspend":
|
||||
expires_at = enforcement.suspend_user(subject, data.duration_hours, data.reason)
|
||||
elif action == "ban":
|
||||
enforcement.ban_user(subject, data.reason)
|
||||
elif action == "lift":
|
||||
enforcement.lift_suspension(subject)
|
||||
enforcement.unban_user(subject)
|
||||
queue.record_action(
|
||||
report_uid=uid,
|
||||
actor_uid=admin["uid"],
|
||||
action=action,
|
||||
target_type=report["target_type"],
|
||||
target_uid=report["target_uid"],
|
||||
subject_uid=subject["uid"] if subject else "",
|
||||
reason=data.reason,
|
||||
notes=data.notes,
|
||||
expires_at=expires_at,
|
||||
)
|
||||
logger.info(f"Admin {admin['username']} applied {action} to report {uid}")
|
||||
metadata = {
|
||||
"report_uid": uid,
|
||||
"action": action,
|
||||
"reason": data.reason,
|
||||
"subject_uid": subject["uid"] if subject else "",
|
||||
}
|
||||
links = [audit.target(report["target_type"], report["target_uid"])]
|
||||
audit.record(
|
||||
request,
|
||||
"report.decide",
|
||||
user=admin,
|
||||
target_type=report["target_type"],
|
||||
target_uid=report["target_uid"],
|
||||
metadata=metadata,
|
||||
summary=f"{admin['username']} applied {action} on report {uid}",
|
||||
links=links,
|
||||
)
|
||||
enforcement_key = ENFORCEMENT_EVENTS.get(action)
|
||||
if enforcement_key:
|
||||
audit.record(
|
||||
request,
|
||||
enforcement_key,
|
||||
user=admin,
|
||||
target_type=report["target_type"],
|
||||
target_uid=report["target_uid"],
|
||||
metadata=metadata,
|
||||
summary=f"{admin['username']} applied {action} from report {uid}",
|
||||
links=links,
|
||||
)
|
||||
_notify_subject(subject, action, report, data.reason)
|
||||
if action != "escalate":
|
||||
_notify_reporter(report, action)
|
||||
return action_result(request, f"{QUEUE_URL}/{uid}")
|
||||
|
||||
|
||||
def _notify_subject(subject: dict | None, action: str, report: dict, reason: str) -> None:
|
||||
template = DECISION_MESSAGES.get(action)
|
||||
if not subject or not template:
|
||||
return
|
||||
message = template.format(target=report["target_type"])
|
||||
if reason:
|
||||
message = f"{message} Reason: {reason}."
|
||||
enforcement.notify_subject(subject["uid"], message)
|
||||
|
||||
|
||||
def _notify_reporter(report: dict, action: str) -> None:
|
||||
reporter_uid = report.get("reporter_uid") or ""
|
||||
if not reporter_uid or reporter_uid == SYSTEM_ACTOR:
|
||||
return
|
||||
if not get_table("users").find_one(uid=reporter_uid):
|
||||
return
|
||||
verb = "dismissed" if action == "dismiss" else "actioned"
|
||||
create_notification(
|
||||
reporter_uid,
|
||||
"moderation",
|
||||
f"Your report on a {report['target_type']} was {verb}.",
|
||||
reporter_uid,
|
||||
"/reports/mine",
|
||||
)
|
||||
@ -4,7 +4,7 @@ import logging
|
||||
from typing import Annotated
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.models import AdminRoleForm, AdminPasswordForm
|
||||
from devplacepy.models import AdminRoleForm, AdminPasswordForm, BanForm, SuspensionForm
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
build_pagination,
|
||||
@ -21,37 +21,16 @@ from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.responses import respond, action_result
|
||||
from devplacepy.schemas import AdminUsersOut, UserAiUsageOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.moderation import enforcement, queue
|
||||
from devplacepy.services.manager import service_manager
|
||||
from devplacepy.services.openai_gateway.analytics import build_user_usage
|
||||
from devplacepy.services.openai_gateway.usage import pricing_from_cfg
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.routers.admin._shared import deny_senior, is_senior_admin
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
def _seniority_key(u: dict) -> tuple[str, int]:
|
||||
return (u.get("created_at") or "", u.get("id") or 0)
|
||||
|
||||
def _is_senior_admin(actor: dict, target: dict | None) -> bool:
|
||||
if not target or target.get("role") != "Admin":
|
||||
return False
|
||||
if target.get("uid") == actor.get("uid"):
|
||||
return False
|
||||
return _seniority_key(target) < _seniority_key(actor)
|
||||
|
||||
def _deny_senior(request: Request, admin: dict, uid: str, target: dict, event_key: str):
|
||||
audit.record(
|
||||
request,
|
||||
event_key,
|
||||
user=admin,
|
||||
result="denied",
|
||||
target_type="user",
|
||||
target_uid=uid,
|
||||
target_label=target.get("username"),
|
||||
summary=f"admin {admin['username']} cannot manage senior admin {target.get('username')}",
|
||||
links=[audit.target("user", uid, target.get("username"))],
|
||||
)
|
||||
return action_result(request, "/admin/users")
|
||||
|
||||
@router.get("/users/{uid}/ai-usage")
|
||||
async def admin_user_ai_usage(request: Request, uid: str, hours: int = 24):
|
||||
@ -124,8 +103,8 @@ async def admin_user_role(
|
||||
return action_result(request, "/admin/users")
|
||||
users = get_table("users")
|
||||
target_user = users.find_one(uid=uid)
|
||||
if _is_senior_admin(admin, target_user):
|
||||
return _deny_senior(request, admin, uid, target_user, "admin.user.role.change")
|
||||
if is_senior_admin(admin, target_user):
|
||||
return deny_senior(request, admin, uid, target_user, "admin.user.role.change")
|
||||
old_role = target_user.get("role") if target_user else None
|
||||
users.update({"uid": uid, "role": role}, ["uid"])
|
||||
clear_user_cache(uid)
|
||||
@ -156,8 +135,8 @@ async def admin_user_password(
|
||||
admin = require_admin(request)
|
||||
users = get_table("users")
|
||||
target_user = users.find_one(uid=uid)
|
||||
if _is_senior_admin(admin, target_user):
|
||||
return _deny_senior(request, admin, uid, target_user, "admin.user.password.reset")
|
||||
if is_senior_admin(admin, target_user):
|
||||
return deny_senior(request, admin, uid, target_user, "admin.user.password.reset")
|
||||
users.update({"uid": uid, "password_hash": await hash_password_async(data.password)}, ["uid"])
|
||||
logger.info(f"Admin {admin['username']} changed password for user {uid}")
|
||||
audit.record(
|
||||
@ -194,8 +173,8 @@ async def admin_user_toggle(request: Request, uid: str):
|
||||
return action_result(request, "/admin/users")
|
||||
users = get_table("users")
|
||||
user = users.find_one(uid=uid)
|
||||
if _is_senior_admin(admin, user):
|
||||
return _deny_senior(request, admin, uid, user, "admin.user.active.disable")
|
||||
if is_senior_admin(admin, user):
|
||||
return deny_senior(request, admin, uid, user, "admin.user.active.disable")
|
||||
if user:
|
||||
new_state = not is_account_active(user)
|
||||
users.update({"uid": uid, "is_active": new_state}, ["uid"])
|
||||
@ -216,12 +195,121 @@ async def admin_user_toggle(request: Request, uid: str):
|
||||
)
|
||||
return action_result(request, "/admin/users")
|
||||
|
||||
def _enforcement_target(request: Request, admin: dict, uid: str, event_key: str):
|
||||
if uid == admin["uid"]:
|
||||
audit.record(
|
||||
request,
|
||||
event_key,
|
||||
user=admin,
|
||||
result="denied",
|
||||
target_type="user",
|
||||
target_uid=uid,
|
||||
target_label=admin.get("username"),
|
||||
summary=f"admin {admin['username']} cannot enforce against their own account",
|
||||
links=[audit.target("user", uid, admin.get("username"))],
|
||||
)
|
||||
return None, action_result(request, "/admin/users")
|
||||
target_user = get_table("users").find_one(uid=uid)
|
||||
if not target_user:
|
||||
return None, action_result(request, "/admin/users")
|
||||
if is_senior_admin(admin, target_user):
|
||||
return None, deny_senior(request, admin, uid, target_user, event_key)
|
||||
return target_user, None
|
||||
|
||||
|
||||
def _record_enforcement(
|
||||
request: Request, admin: dict, target_user: dict, event_key: str, metadata: dict
|
||||
):
|
||||
logger.info(
|
||||
f"Admin {admin['username']} applied {event_key} to {target_user['username']}"
|
||||
)
|
||||
audit.record(
|
||||
request,
|
||||
event_key,
|
||||
user=admin,
|
||||
target_type="user",
|
||||
target_uid=target_user["uid"],
|
||||
target_label=target_user.get("username"),
|
||||
metadata=metadata,
|
||||
summary=f"{admin['username']} applied {event_key} to {target_user['username']}",
|
||||
links=[audit.target("user", target_user["uid"], target_user.get("username"))],
|
||||
)
|
||||
queue.record_action(
|
||||
report_uid="",
|
||||
actor_uid=admin["uid"],
|
||||
action=event_key.rsplit(".", 1)[-1],
|
||||
target_type="user",
|
||||
target_uid=target_user["uid"],
|
||||
subject_uid=target_user["uid"],
|
||||
reason=metadata.get("reason", ""),
|
||||
expires_at=metadata.get("expires_at", ""),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/users/{uid}/suspend")
|
||||
async def admin_user_suspend(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[SuspensionForm, Depends(json_or_form(SuspensionForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
target_user, refusal = _enforcement_target(request, admin, uid, "moderation.suspend")
|
||||
if refusal is not None:
|
||||
return refusal
|
||||
expires_at = enforcement.suspend_user(target_user, data.duration_hours, data.reason)
|
||||
_record_enforcement(
|
||||
request,
|
||||
admin,
|
||||
target_user,
|
||||
"moderation.suspend",
|
||||
{"reason": data.reason, "expires_at": expires_at, "hours": data.duration_hours},
|
||||
)
|
||||
enforcement.notify_subject(
|
||||
uid,
|
||||
f"Your account is suspended until {expires_at}. Reason: {data.reason or 'policy violation'}.",
|
||||
)
|
||||
return action_result(request, "/admin/users")
|
||||
|
||||
|
||||
@router.post("/users/{uid}/lift")
|
||||
async def admin_user_lift(request: Request, uid: str):
|
||||
admin = require_admin(request)
|
||||
target_user, refusal = _enforcement_target(request, admin, uid, "moderation.lift")
|
||||
if refusal is not None:
|
||||
return refusal
|
||||
enforcement.lift_suspension(target_user)
|
||||
enforcement.unban_user(target_user)
|
||||
_record_enforcement(request, admin, target_user, "moderation.lift", {})
|
||||
enforcement.notify_subject(uid, "Your account restriction has been lifted.")
|
||||
return action_result(request, "/admin/users")
|
||||
|
||||
|
||||
@router.post("/users/{uid}/ban")
|
||||
async def admin_user_ban(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[BanForm, Depends(json_or_form(BanForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
target_user, refusal = _enforcement_target(request, admin, uid, "moderation.ban")
|
||||
if refusal is not None:
|
||||
return refusal
|
||||
enforcement.ban_user(target_user, data.reason)
|
||||
_record_enforcement(
|
||||
request, admin, target_user, "moderation.ban", {"reason": data.reason}
|
||||
)
|
||||
enforcement.notify_subject(
|
||||
uid, f"Your account has been closed. Reason: {data.reason or 'policy violation'}."
|
||||
)
|
||||
return action_result(request, "/admin/users")
|
||||
|
||||
|
||||
@router.post("/users/{uid}/reset-ai-quota")
|
||||
async def admin_user_reset_ai_quota(request: Request, uid: str):
|
||||
admin = require_admin(request)
|
||||
target_user = get_table("users").find_one(uid=uid)
|
||||
if _is_senior_admin(admin, target_user):
|
||||
return _deny_senior(request, admin, uid, target_user, "admin.user.ai_quota.reset")
|
||||
if is_senior_admin(admin, target_user):
|
||||
return deny_senior(request, admin, uid, target_user, "admin.user.ai_quota.reset")
|
||||
devii = service_manager.get_service("devii")
|
||||
removed = devii.reset_quota("user", uid) if devii is not None else 0
|
||||
logger.info(
|
||||
|
||||
@ -8,16 +8,24 @@ from fastapi.responses import JSONResponse
|
||||
from devplacepy.database import db, get_table, get_users_by_uids
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import (
|
||||
EditorPrefsForm,
|
||||
WorkspaceFlagForm,
|
||||
WorkspaceQuotaForm,
|
||||
WorkspaceSuspendForm,
|
||||
)
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.routers.admin._shared import deny_senior, is_senior_admin
|
||||
from devplacepy.schemas import AdminWorkspacesOut
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.containers import store
|
||||
from devplacepy.services.containers.workspace import flags, provision, quota, tunnels
|
||||
from devplacepy.services.containers.workspace import (
|
||||
editor,
|
||||
flags,
|
||||
provision,
|
||||
quota,
|
||||
tunnels,
|
||||
)
|
||||
from devplacepy.utils import create_notification, generate_uid, not_found, require_admin
|
||||
|
||||
router = APIRouter()
|
||||
@ -117,7 +125,7 @@ async def admin_workspace_suspend(
|
||||
instance = _instance_or_404(uid)
|
||||
reason = (data.reason or "").strip()
|
||||
if not reason:
|
||||
return json_error("a reason is required and is shown to the owner", 400)
|
||||
return json_error(400, "a reason is required and is shown to the owner")
|
||||
provision.suspend(instance, admin["uid"], reason)
|
||||
_audit(request, admin, "container.workspace.suspend", instance, metadata={"reason": reason})
|
||||
owner = instance.get("workspace_owner_uid", "")
|
||||
@ -237,6 +245,43 @@ async def admin_flag_resolve(request: Request, flag_uid: str, status: str = "res
|
||||
return action_result(request, "/admin/workspaces")
|
||||
|
||||
|
||||
@router.post("/workspaces/{uid}/editor")
|
||||
async def admin_workspace_editor(
|
||||
request: Request,
|
||||
uid: str,
|
||||
data: Annotated[EditorPrefsForm, Depends(json_or_form(EditorPrefsForm))],
|
||||
):
|
||||
admin = require_admin(request)
|
||||
if not isinstance(admin, dict):
|
||||
return admin
|
||||
instance = _instance_or_404(uid)
|
||||
owner_uid = instance.get("workspace_owner_uid", "")
|
||||
owner = get_users_by_uids([owner_uid]).get(owner_uid) if owner_uid else None
|
||||
if is_senior_admin(admin, owner):
|
||||
return deny_senior(
|
||||
request,
|
||||
admin,
|
||||
owner_uid,
|
||||
owner,
|
||||
"container.workspace.editor.update",
|
||||
"/admin/workspaces",
|
||||
)
|
||||
if not owner_uid:
|
||||
return json_error(400, "this workspace has no owner")
|
||||
if data.reset:
|
||||
editor.reset_prefs(owner_uid, admin["uid"])
|
||||
else:
|
||||
editor.save_prefs(
|
||||
owner_uid, data.model_dump(exclude={"reset"}, exclude_unset=True)
|
||||
)
|
||||
_audit(request, admin, "container.workspace.editor.update", instance)
|
||||
return action_result(
|
||||
request,
|
||||
"/admin/workspaces",
|
||||
data={"editor": editor.view(owner_uid, instance)},
|
||||
)
|
||||
|
||||
|
||||
@router.post("/workspaces/quota")
|
||||
async def admin_workspace_quota(
|
||||
request: Request,
|
||||
@ -246,7 +291,7 @@ async def admin_workspace_quota(
|
||||
if not isinstance(admin, dict):
|
||||
return admin
|
||||
if not data.owner_id:
|
||||
return json_error("owner_id is required", 400)
|
||||
return json_error(400, "owner_id is required")
|
||||
table = get_table(quota.RULES_TABLE)
|
||||
existing = table.find_one(
|
||||
owner_kind="user", owner_id=data.owner_id, deleted_at=None
|
||||
|
||||
@ -8,6 +8,7 @@ from devplacepy.routers.auth import (
|
||||
logout,
|
||||
resetpassword,
|
||||
signup,
|
||||
terms,
|
||||
token,
|
||||
)
|
||||
|
||||
@ -18,3 +19,4 @@ router.include_router(token.router)
|
||||
router.include_router(forgotpassword.router)
|
||||
router.include_router(resetpassword.router)
|
||||
router.include_router(logout.router)
|
||||
router.include_router(terms.router)
|
||||
|
||||
@ -90,7 +90,9 @@ async def signup(request: Request, data: Annotated[SignupForm, Depends(json_or_f
|
||||
},
|
||||
)
|
||||
|
||||
uid, role, is_first = await register_account_async(username, email, password)
|
||||
uid, role, is_first = await register_account_async(
|
||||
username, email, password, age_band=data.age_band, accepted_terms=True
|
||||
)
|
||||
|
||||
max_age = max(1, get_int_setting("session_max_age_days", 7)) * SECONDS_PER_DAY
|
||||
token = create_session(uid, max_age)
|
||||
|
||||
84
devplacepy/routers/auth/terms.py
Normal file
84
devplacepy/routers/auth/terms.py
Normal file
@ -0,0 +1,84 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.database import _now_iso, get_setting, get_table, set_consent
|
||||
from devplacepy.responses import action_result, respond
|
||||
from devplacepy.schemas import AcceptTermsOut
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.utils import clear_user_cache, require_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
TERMS_ACCEPTANCE_CODE = "terms_acceptance_required"
|
||||
|
||||
|
||||
def current_terms_version() -> str:
|
||||
return get_setting("terms_version", "1") or "1"
|
||||
|
||||
|
||||
def needs_acceptance(user: dict | None) -> bool:
|
||||
if not user:
|
||||
return False
|
||||
return (user.get("terms_version") or "") != current_terms_version()
|
||||
|
||||
|
||||
@router.get("/accept-terms", response_class=HTMLResponse)
|
||||
async def accept_terms_page(request: Request):
|
||||
user = require_user(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Accept the updated terms",
|
||||
description="The Terms of Service changed. Accept the new version to continue.",
|
||||
robots="noindex,nofollow",
|
||||
)
|
||||
return respond(
|
||||
request,
|
||||
"accept_terms.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": user,
|
||||
"terms_version": current_terms_version(),
|
||||
"accepted_version": user.get("terms_version") or "",
|
||||
},
|
||||
model=AcceptTermsOut,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/accept-terms")
|
||||
async def accept_terms(request: Request):
|
||||
user = require_user(request)
|
||||
version = current_terms_version()
|
||||
now = _now_iso()
|
||||
get_table("users").update(
|
||||
{"uid": user["uid"], "terms_version": version, "terms_accepted_at": now},
|
||||
["uid"],
|
||||
)
|
||||
set_consent("user", user["uid"], "terms", True, version=version)
|
||||
set_consent(
|
||||
"user",
|
||||
user["uid"],
|
||||
"privacy",
|
||||
True,
|
||||
version=get_setting("privacy_version", "1") or "1",
|
||||
)
|
||||
clear_user_cache(user["uid"])
|
||||
logger.info(f"{user['username']} accepted terms version {version}")
|
||||
audit.record(
|
||||
request,
|
||||
"terms.accept",
|
||||
user=user,
|
||||
target_type="user",
|
||||
target_uid=user["uid"],
|
||||
target_label=user.get("username"),
|
||||
new_value=version,
|
||||
summary=f"{user['username']} accepted terms version {version}",
|
||||
links=[audit.target("user", user["uid"], user.get("username"))],
|
||||
)
|
||||
return action_result(request, "/feed", data={"terms_version": version})
|
||||
@ -12,7 +12,7 @@ A second REST protocol mounted at `/api` that reproduces the public devRant API
|
||||
|
||||
**ID mapping (load-bearing).** devRant integer ids ARE the auto-increment `id` PK every `dataset` table already has: `rant_id`=`posts.id`, `comment_id`=`comments.id`, `user_id`=`users.id`, `token_id`=`devrant_tokens.id`. No translation table exists - `post_by_id` is `find_one(id=...)`. Serialization converts ISO `created_at` to unix via `ids.to_unix`.
|
||||
|
||||
**Auth.** `POST /api/users/auth-token` accepts username OR email, verifies with passlib, and inserts a `devrant_tokens` row (in `SOFT_DELETE_TABLES`; born-live; `key`=`secrets.token_hex`, `expire_time` from `session_max_age_days`). Every later call re-validates `(token_id, token_key, user_id)` with `tokens.resolve_user(params)`. Read endpoints take an OPTIONAL viewer (`resolve_user` may return None); write endpoints return `_shared.unauthorized()` (401) when it does.
|
||||
**Auth.** `POST /api/users/auth-token` accepts username OR email, verifies with passlib, and inserts a `devrant_tokens` row (in `SOFT_DELETE_TABLES`; born-live; `key`=`secrets.token_hex`, `expire_time` from `session_max_age_days`). Every later call re-validates `(token_id, token_key, user_id)` through **`_shared.resolve_actor(request, params)`**, which wraps `tokens.resolve_user` with `utils.guards.refuse_suspended` - because this path never touches `require_user`, a moderator's suspension would otherwise not bind here at all (the token resolver's `is_account_active` check covers a **ban** but not a time-boxed suspension). `refuse_suspended` gates mutating methods only, so read endpoints are unaffected. Read endpoints take an OPTIONAL viewer (it may return None); write endpoints return `_shared.unauthorized()` (401) when it does. **`DELETE /api/users/me` deliberately calls the bare `resolve_user`** - it is the account-deletion path and must stay reachable to a suspended user, matching the `/profile/{username}/delete` exemption on the web side.
|
||||
|
||||
**Writes reuse the audited native cores - never duplicate.** Implementing this drove four DRY extractions in `content.py` (`apply_vote`, `create_comment_record`, `delete_comment_record`, `set_bookmark`) and one in `utils.py` (`register_account`); the native `routers/votes.py`, `routers/comments.py`, and `auth/signup.py` were refactored onto the SAME functions. So a devRant rant/comment/vote awards XP, fires notifications, writes the audit row, and soft-deletes exactly like the UI path. Rant create calls `content.create_content_item` directly; rant delete calls `content.delete_content_item` (full cascade) and returns the devRant envelope.
|
||||
|
||||
|
||||
@ -2,10 +2,19 @@
|
||||
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import HTTPException
|
||||
from fastapi import HTTPException, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from devplacepy.database import get_setting
|
||||
from devplacepy.services.devrant.tokens import resolve_user
|
||||
from devplacepy.utils.guards import refuse_suspended
|
||||
|
||||
|
||||
def resolve_actor(request: Request, params: dict) -> Optional[dict]:
|
||||
user = resolve_user(params)
|
||||
if user:
|
||||
refuse_suspended(request, user)
|
||||
return user
|
||||
|
||||
|
||||
def api_enabled() -> bool:
|
||||
|
||||
@ -2,7 +2,6 @@
|
||||
|
||||
import logging
|
||||
import re
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import Response
|
||||
@ -17,7 +16,7 @@ from devplacepy.services.devrant.tokens import issue_token, resolve_user, revoke
|
||||
from devplacepy.services.devrant.profile import build_profile
|
||||
from devplacepy.services.devrant.ids import user_by_id
|
||||
from devplacepy.services.devrant.avatar import render_png
|
||||
from devplacepy.routers.devrant._shared import dr_ok, dr_error, unauthorized
|
||||
from devplacepy.routers.devrant._shared import dr_ok, dr_error, resolve_actor, unauthorized
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@ -131,14 +130,14 @@ async def profile(request: Request, user_id: str):
|
||||
user = user_by_id(user_id)
|
||||
if not user:
|
||||
return dr_error("User not found.")
|
||||
viewer = resolve_user(params)
|
||||
viewer = resolve_actor(request, params)
|
||||
return dr_ok(profile=build_profile(user, viewer))
|
||||
|
||||
|
||||
@router.post("/users/me/edit-profile")
|
||||
async def edit_profile(request: Request):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
updates = {"uid": user["uid"]}
|
||||
@ -191,21 +190,29 @@ async def delete_account(request: Request):
|
||||
user = resolve_user(params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
get_table("users").update(
|
||||
{"uid": user["uid"], "is_active": False}, ["uid"]
|
||||
)
|
||||
from devplacepy.services.moderation import deletion
|
||||
|
||||
username = user["username"]
|
||||
revoke_all(user["uid"])
|
||||
logger.info("devrant account deactivated for %s", user["username"])
|
||||
result = deletion.delete_account(user)
|
||||
if result is None:
|
||||
return dr_error("This account is already being deleted.")
|
||||
logger.info("devrant account deleted for %s", username)
|
||||
audit.record(
|
||||
request,
|
||||
"auth.account.disable",
|
||||
"account.delete.request",
|
||||
user=user,
|
||||
target_type="user",
|
||||
target_uid=user["uid"],
|
||||
target_label=user["username"],
|
||||
target_label=username,
|
||||
origin="devrant",
|
||||
summary=f"{user['username']} deactivated account via devrant",
|
||||
links=[audit.target("user", user["uid"], user["username"])],
|
||||
metadata={
|
||||
"stamp": result["stamp"],
|
||||
"rows": result["rows"],
|
||||
"grace_hours": result["grace_hours"],
|
||||
},
|
||||
summary=f"{username} deleted account via devrant",
|
||||
links=[audit.target("user", user["uid"], username)],
|
||||
)
|
||||
return dr_ok()
|
||||
|
||||
|
||||
@ -17,10 +17,9 @@ from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.correction import schedule_correction
|
||||
from devplacepy.services.ai_modifier import schedule_modification
|
||||
from devplacepy.services.devrant.params import merge_params
|
||||
from devplacepy.services.devrant.tokens import resolve_user
|
||||
from devplacepy.services.devrant.ids import as_int, comment_by_id
|
||||
from devplacepy.services.devrant.serializers import serialize_comment
|
||||
from devplacepy.routers.devrant._shared import dr_ok, dr_error, unauthorized
|
||||
from devplacepy.routers.devrant._shared import dr_ok, dr_error, resolve_actor, unauthorized
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@ -50,7 +49,7 @@ def _serialize_single(comment: dict, viewer) -> dict:
|
||||
@router.get("/comments/{comment_id}")
|
||||
async def get_comment(request: Request, comment_id: str):
|
||||
params = await merge_params(request)
|
||||
viewer = resolve_user(params)
|
||||
viewer = resolve_actor(request, params)
|
||||
comment = comment_by_id(comment_id)
|
||||
if not comment:
|
||||
return dr_error("Invalid comment specified in path.")
|
||||
@ -60,7 +59,7 @@ async def get_comment(request: Request, comment_id: str):
|
||||
@router.post("/comments/{comment_id}")
|
||||
async def edit_comment(request: Request, comment_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
comment = comment_by_id(comment_id)
|
||||
@ -97,7 +96,7 @@ async def edit_comment(request: Request, comment_id: str):
|
||||
@router.delete("/comments/{comment_id}")
|
||||
async def delete_comment(request: Request, comment_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
comment = comment_by_id(comment_id)
|
||||
@ -112,7 +111,7 @@ async def delete_comment(request: Request, comment_id: str):
|
||||
@router.post("/comments/{comment_id}/vote")
|
||||
async def vote_comment(request: Request, comment_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
comment = comment_by_id(comment_id)
|
||||
|
||||
@ -5,9 +5,8 @@ import logging
|
||||
from fastapi import APIRouter, Request
|
||||
|
||||
from devplacepy.services.devrant.params import merge_params
|
||||
from devplacepy.services.devrant.tokens import resolve_user
|
||||
from devplacepy.services.devrant.notifications import build_notif_feed, clear_notifications
|
||||
from devplacepy.routers.devrant._shared import dr_ok, unauthorized
|
||||
from devplacepy.routers.devrant._shared import dr_ok, resolve_actor, unauthorized
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -17,7 +16,7 @@ router = APIRouter()
|
||||
@router.get("/users/me/notif-feed")
|
||||
async def notif_feed(request: Request):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
return dr_ok(data=build_notif_feed(user))
|
||||
@ -26,7 +25,7 @@ async def notif_feed(request: Request):
|
||||
@router.delete("/users/me/notif-feed")
|
||||
async def clear_notif_feed(request: Request):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
clear_notifications(user)
|
||||
|
||||
@ -21,11 +21,10 @@ from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.correction import schedule_correction
|
||||
from devplacepy.services.ai_modifier import schedule_modification
|
||||
from devplacepy.services.devrant.params import merge_params
|
||||
from devplacepy.services.devrant.tokens import resolve_user
|
||||
from devplacepy.services.devrant.ids import as_int, post_by_id
|
||||
from devplacepy.services.devrant.feed import list_rants, search_rants, load_rant_detail
|
||||
from devplacepy.services.devrant.serializers import encode_tags
|
||||
from devplacepy.routers.devrant._shared import dr_ok, dr_error, unauthorized
|
||||
from devplacepy.routers.devrant._shared import dr_ok, dr_error, resolve_actor, unauthorized
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
@ -45,7 +44,7 @@ def _parse_tags(raw: object) -> list:
|
||||
@router.get("/devrant/rants")
|
||||
async def rant_feed(request: Request):
|
||||
params = await merge_params(request)
|
||||
viewer = resolve_user(params)
|
||||
viewer = resolve_actor(request, params)
|
||||
sort = params.get("sort") or "recent"
|
||||
limit = min(MAX_LIMIT, max(1, as_int(params.get("limit"), DEFAULT_LIMIT)))
|
||||
skip = max(0, as_int(params.get("skip"), 0))
|
||||
@ -70,7 +69,7 @@ async def rant_feed(request: Request):
|
||||
@router.get("/devrant/search")
|
||||
async def search(request: Request):
|
||||
params = await merge_params(request)
|
||||
viewer = resolve_user(params)
|
||||
viewer = resolve_actor(request, params)
|
||||
term = (params.get("term") or "").strip()
|
||||
return dr_ok(results=search_rants(term, viewer) if term else [])
|
||||
|
||||
@ -78,7 +77,7 @@ async def search(request: Request):
|
||||
@router.post("/devrant/rants")
|
||||
async def create_rant(request: Request):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
text = (params.get("rant") or "").strip()
|
||||
@ -114,7 +113,7 @@ async def create_rant(request: Request):
|
||||
@router.get("/devrant/rants/{rant_id}")
|
||||
async def get_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
viewer = resolve_user(params)
|
||||
viewer = resolve_actor(request, params)
|
||||
post = post_by_id(rant_id)
|
||||
if not post:
|
||||
return dr_error("This rant does not exist.")
|
||||
@ -125,7 +124,7 @@ async def get_rant(request: Request, rant_id: str):
|
||||
@router.post("/devrant/rants/{rant_id}")
|
||||
async def edit_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
@ -164,7 +163,7 @@ async def edit_rant(request: Request, rant_id: str):
|
||||
@router.delete("/devrant/rants/{rant_id}")
|
||||
async def delete_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
@ -179,7 +178,7 @@ async def delete_rant(request: Request, rant_id: str):
|
||||
@router.post("/devrant/rants/{rant_id}/vote")
|
||||
async def vote_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
@ -206,7 +205,7 @@ async def unfavorite_rant(request: Request, rant_id: str):
|
||||
|
||||
async def _set_favorite(request: Request, rant_id: str, saved: bool):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
@ -219,7 +218,7 @@ async def _set_favorite(request: Request, rant_id: str, saved: bool):
|
||||
@router.post("/devrant/rants/{rant_id}/comments")
|
||||
async def comment_rant(request: Request, rant_id: str):
|
||||
params = await merge_params(request)
|
||||
user = resolve_user(params)
|
||||
user = resolve_actor(request, params)
|
||||
if not user:
|
||||
return unauthorized()
|
||||
post = post_by_id(rant_id)
|
||||
|
||||
@ -10,6 +10,8 @@ This file documents the documentation site (`/docs`) - prose pages, API referenc
|
||||
|
||||
## Audience tiers and navigation
|
||||
|
||||
The **Legal** section (`SECTION_LEGAL`, in the `AUDIENCE_START` tier so it is one click from `/docs`) carries the platform's policies: `terms`, `community-guidelines`, `privacy`, `content-moderation`, `intellectual-property`, `contact`, plus the admin-gated `moderation-operations`. They are ordinary prose pages, which is exactly why they were built here rather than as new routes - role gating, SEO, the search index and the docs export all come for free. The six public ones are listed in `seo.LEGAL_DOC_SLUGS` and appear in the sitemap; `_footer_links.html` links four of them from every page. Their prose reads live values through the `policy_version`, `moderation_sla_hours`, `moderation_minimum_age`, `ai_provider_name` and `contact_details` Jinja globals, so a settings change is reflected without a content edit.
|
||||
|
||||
`DOCS_PAGES` entries take optional `admin: True` (hidden + 404 for non-admins, but still indexed and surfaced only to admins by `docs_search`) and `section: "..."` (a nested sidebar group rendered by `docs_base.html`). The sidebar groups `section`s under four ordered **audience tiers** (`AUDIENCES` in `routers/docs/pages.py`): `Start here` (General), `Build with the API` (API, Components, Styles), `Contribute and internals` (Architecture, Services, Devii internals, Bots internals, Testing, Claude Code), and `Operate` (Administration, Production). `nav_groups(visible_pages)` builds the `[(audience, [(section, [pages])])]` tree from the flat visible-page list (so a section's pages collect under one heading regardless of `DOCS_PAGES` order or the API/Administration interleave from `api_doc_pages()`); `views.py` passes it as `nav`, and `docs_base.html` renders an audience super-header (`.sidebar-tier`) above each section subheading (`.sidebar-subheading`). `DOCS_PAGES` stays the canonical list for search/export/routing - the tiering is sidebar-only.
|
||||
|
||||
The public `getting-started` page (`SECTION_GENERAL`) is the new-contributor on-ramp (install/run, the four-faces workflow, validation); gate its deep-internals links with `{% if is_admin(user) %}` so guests get no 404s. Keep one canonical home per concept: the `auth` API group intro in `docs_api.py` defers method detail to the `authentication` prose page rather than re-listing the four methods. The member-facing `devii` prose page is functional; admins also get a `Devii internals` section of `devii-*` technical subpages.
|
||||
|
||||
@ -4,6 +4,7 @@ from devplacepy.docs_api import api_doc_pages
|
||||
|
||||
SECTION_GENERAL = "General"
|
||||
SECTION_TOOLS = "Tools"
|
||||
SECTION_LEGAL = "Legal"
|
||||
SECTION_COMPONENTS = "Components"
|
||||
SECTION_STYLES = "Styles"
|
||||
SECTION_API = "API"
|
||||
@ -23,7 +24,7 @@ AUDIENCE_CONTRIBUTE = "Contribute and internals"
|
||||
AUDIENCE_OPERATE = "Operate"
|
||||
|
||||
AUDIENCES = [
|
||||
(AUDIENCE_START, [SECTION_GENERAL, SECTION_TOOLS]),
|
||||
(AUDIENCE_START, [SECTION_GENERAL, SECTION_LEGAL, SECTION_TOOLS]),
|
||||
(
|
||||
AUDIENCE_BUILD,
|
||||
[SECTION_API, SECTION_DEVRANT, SECTION_COMPONENTS, SECTION_STYLES],
|
||||
@ -57,6 +58,12 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "workspace-editor",
|
||||
"title": "The workspace editor",
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
{
|
||||
"slug": "feed",
|
||||
"title": "The feed",
|
||||
@ -147,6 +154,50 @@ DOCS_PAGES = [
|
||||
"kind": "prose",
|
||||
"section": SECTION_GENERAL,
|
||||
},
|
||||
# Legal - the policies the platform is operated under (everyone)
|
||||
{
|
||||
"slug": "terms",
|
||||
"title": "Terms of Service",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
},
|
||||
{
|
||||
"slug": "community-guidelines",
|
||||
"title": "Community Guidelines",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
},
|
||||
{
|
||||
"slug": "privacy",
|
||||
"title": "Privacy Policy",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
},
|
||||
{
|
||||
"slug": "content-moderation",
|
||||
"title": "How moderation works",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
},
|
||||
{
|
||||
"slug": "intellectual-property",
|
||||
"title": "Notice and takedown",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
},
|
||||
{
|
||||
"slug": "contact",
|
||||
"title": "Contact",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
},
|
||||
{
|
||||
"slug": "moderation-operations",
|
||||
"title": "Operating the moderation queue",
|
||||
"kind": "prose",
|
||||
"section": SECTION_LEGAL,
|
||||
"admin": True,
|
||||
},
|
||||
# Tools - public developer tools (everyone)
|
||||
{
|
||||
"slug": "tools-seo",
|
||||
|
||||
@ -185,6 +185,7 @@ async def docs_page(request: Request, slug: str):
|
||||
request,
|
||||
title=f"{page['title']} - Documentation",
|
||||
description="DevPlace developer documentation.",
|
||||
robots="noindex,nofollow" if page.get("admin") else "index,follow",
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Docs", "url": "/docs/index.html"},
|
||||
|
||||
@ -317,6 +317,13 @@ def _resolve_ws_user(websocket: WebSocket):
|
||||
return _user_from_api_key(key)
|
||||
return None
|
||||
|
||||
|
||||
def _ws_may_write(user: dict) -> bool:
|
||||
from devplacepy.database import suspension_active
|
||||
from devplacepy.routers.auth.terms import needs_acceptance
|
||||
|
||||
return not suspension_active(user) and not needs_acceptance(user)
|
||||
|
||||
@router.websocket("/ws")
|
||||
async def messages_ws(websocket: WebSocket):
|
||||
await websocket.accept()
|
||||
@ -324,6 +331,9 @@ async def messages_ws(websocket: WebSocket):
|
||||
if not user:
|
||||
await websocket.close(code=1008)
|
||||
return
|
||||
if not _ws_may_write(user):
|
||||
await websocket.close(code=1008)
|
||||
return
|
||||
|
||||
user_uid = user["uid"]
|
||||
message_hub.register(user_uid, websocket)
|
||||
|
||||
@ -5,6 +5,7 @@ from datetime import datetime, timedelta, timezone
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from devplacepy.database import (
|
||||
get_maturity,
|
||||
get_table,
|
||||
db,
|
||||
load_comments,
|
||||
@ -156,6 +157,7 @@ async def news_detail_page(request: Request, news_slug: str):
|
||||
"time_ago": time_ago(article["synced_at"]),
|
||||
"comments": comments,
|
||||
"bookmarked": bookmarked,
|
||||
"maturity": get_maturity("news", article["uid"])["level"],
|
||||
},
|
||||
model=NewsDetailOut,
|
||||
)
|
||||
|
||||
@ -5,7 +5,9 @@ from devplacepy.routers.profile import (
|
||||
ai_modifier,
|
||||
avatar,
|
||||
award,
|
||||
consent,
|
||||
customization,
|
||||
delete,
|
||||
interactions,
|
||||
notifications,
|
||||
telegram,
|
||||
@ -21,5 +23,7 @@ router.include_router(ai_modifier.router)
|
||||
router.include_router(interactions.router)
|
||||
router.include_router(avatar.router)
|
||||
router.include_router(telegram.router)
|
||||
router.include_router(consent.router)
|
||||
router.include_router(delete.router)
|
||||
|
||||
__all__ = ["router", "_ai_quota"]
|
||||
|
||||
127
devplacepy/routers/profile/consent.py
Normal file
127
devplacepy/routers/profile/consent.py
Normal file
@ -0,0 +1,127 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
|
||||
from devplacepy.database import (
|
||||
CONSENT_KINDS,
|
||||
consent_state,
|
||||
get_setting,
|
||||
get_table,
|
||||
list_consents,
|
||||
set_consent,
|
||||
)
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import ConsentForm, MaturePreferenceForm
|
||||
from devplacepy.responses import action_result
|
||||
from devplacepy.routers.profile.delete import _owner_only
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.utils import clear_user_cache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
TRUTHY = ("1", "on", "true", "yes")
|
||||
|
||||
VERSION_KEYS = {"terms": "terms_version", "privacy": "privacy_version"}
|
||||
|
||||
|
||||
def consent_view(owner_kind: str, owner_id: str) -> list[dict]:
|
||||
latest = {}
|
||||
for row in list_consents(owner_kind, owner_id):
|
||||
latest.setdefault(row["kind"], row)
|
||||
return [
|
||||
{
|
||||
"kind": kind,
|
||||
"label": label,
|
||||
"state": (latest.get(kind) or {}).get("state", "withdrawn"),
|
||||
"version": (latest.get(kind) or {}).get("version", ""),
|
||||
"granted_at": (latest.get(kind) or {}).get("granted_at", ""),
|
||||
"withdrawn_at": (latest.get(kind) or {}).get("withdrawn_at", ""),
|
||||
}
|
||||
for kind, label in CONSENT_KINDS.items()
|
||||
]
|
||||
|
||||
|
||||
def consent_version(kind: str) -> str:
|
||||
key = VERSION_KEYS.get(kind)
|
||||
return (get_setting(key, "1") or "1") if key else "1"
|
||||
|
||||
|
||||
@router.post("/{username}/consent")
|
||||
async def set_user_consent(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[ConsentForm, Depends(json_or_form(ConsentForm))],
|
||||
):
|
||||
target, denied = _owner_only(
|
||||
request, username, "Only the account holder can change a consent"
|
||||
)
|
||||
if denied is not None:
|
||||
return denied
|
||||
granted = data.granted.strip().lower() in TRUTHY
|
||||
before = consent_state("user", target["uid"], data.kind)
|
||||
set_consent(
|
||||
"user", target["uid"], data.kind, granted, version=consent_version(data.kind)
|
||||
)
|
||||
logger.info(
|
||||
f"Consent {data.kind} {'granted' if granted else 'withdrawn'} for {target['username']}"
|
||||
)
|
||||
audit.record(
|
||||
request,
|
||||
"consent.grant" if granted else "consent.withdraw",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
old_value=(before or {}).get("state"),
|
||||
new_value="granted" if granted else "withdrawn",
|
||||
metadata={"kind": data.kind},
|
||||
summary=(
|
||||
f"{'granted' if granted else 'withdrew'} {data.kind} consent "
|
||||
f"for {target['username']}"
|
||||
),
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
url = f"/profile/{target['username']}?tab=privacy"
|
||||
return action_result(
|
||||
request,
|
||||
url,
|
||||
data={"kind": data.kind, "state": "granted" if granted else "withdrawn"},
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{username}/mature-content")
|
||||
async def set_mature_preference(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[MaturePreferenceForm, Depends(json_or_form(MaturePreferenceForm))],
|
||||
):
|
||||
target, denied = _owner_only(
|
||||
request,
|
||||
username,
|
||||
"Only the account holder can change the mature-content preference",
|
||||
)
|
||||
if denied is not None:
|
||||
return denied
|
||||
opted_in = data.mature_opt_in.strip().lower() in TRUTHY
|
||||
get_table("users").update(
|
||||
{"uid": target["uid"], "mature_opt_in": 1 if opted_in else 0}, ["uid"]
|
||||
)
|
||||
clear_user_cache(target["uid"])
|
||||
audit.record(
|
||||
request,
|
||||
"profile.mature_content",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
new_value=1 if opted_in else 0,
|
||||
summary=(
|
||||
f"{'enabled' if opted_in else 'disabled'} mature content "
|
||||
f"for {target['username']}"
|
||||
),
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
url = f"/profile/{target['username']}?tab=privacy"
|
||||
return action_result(request, url, data={"mature_opt_in": opted_in})
|
||||
150
devplacepy/routers/profile/delete.py
Normal file
150
devplacepy/routers/profile/delete.py
Normal file
@ -0,0 +1,150 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import AccountDeleteForm
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.routers.profile._shared import resolve_customization_target
|
||||
from devplacepy.schemas import AccountDeletionOut
|
||||
from devplacepy.seo import base_seo_context
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.moderation import deletion
|
||||
from devplacepy.utils import get_current_user, verify_password_async
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
REMOVED = [
|
||||
"Your account record, username, email address and password",
|
||||
"Your profile: bio, location, links and avatar",
|
||||
"Your posts, comments, gists, projects, project files and quizzes",
|
||||
"Your uploads and media gallery",
|
||||
"Your direct-message history, votes, reactions, bookmarks and polls",
|
||||
"Your API key, access tokens and every signed-in session",
|
||||
"Your assistant conversations, tasks, lessons and custom tools",
|
||||
]
|
||||
|
||||
RETAINED = [
|
||||
"Append-only audit and moderation records, which hold identifiers rather than "
|
||||
"your profile, so the platform can show it enforced its own rules",
|
||||
"Backup archives, until they rotate out on their normal schedule",
|
||||
]
|
||||
|
||||
|
||||
def _owner_only(
|
||||
request: Request,
|
||||
username: str,
|
||||
message: str = "Only the account holder can delete this account",
|
||||
):
|
||||
target, denied = resolve_customization_target(request, username)
|
||||
if denied is not None:
|
||||
return None, denied
|
||||
viewer = get_current_user(request)
|
||||
if not viewer or viewer["uid"] != target["uid"]:
|
||||
audit.record(
|
||||
request,
|
||||
"security.authz.denied",
|
||||
user=viewer,
|
||||
result="denied",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
metadata={"reason": message},
|
||||
summary=f"non-owner denied {request.method} {request.url.path}",
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
return None, json_error(403, message)
|
||||
return target, None
|
||||
|
||||
|
||||
async def _password_matches(password: str, hashed: str) -> bool:
|
||||
if not hashed:
|
||||
return False
|
||||
try:
|
||||
return await verify_password_async(password, hashed)
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
|
||||
@router.get("/{username}/delete", response_class=HTMLResponse)
|
||||
async def delete_account_page(request: Request, username: str):
|
||||
target, denied = _owner_only(request, username)
|
||||
if denied is not None:
|
||||
return denied
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Delete your account",
|
||||
description="Permanently remove your DevPlace account and personal data.",
|
||||
robots="noindex,nofollow",
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": target["username"], "url": f"/profile/{target['username']}"},
|
||||
{
|
||||
"name": "Delete account",
|
||||
"url": f"/profile/{target['username']}/delete",
|
||||
},
|
||||
],
|
||||
)
|
||||
return respond(
|
||||
request,
|
||||
"account_delete.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": target,
|
||||
"username": target["username"],
|
||||
"grace_hours": deletion.grace_hours(),
|
||||
"removed": REMOVED,
|
||||
"retained": RETAINED,
|
||||
},
|
||||
model=AccountDeletionOut,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{username}/delete")
|
||||
async def delete_account(
|
||||
request: Request,
|
||||
username: str,
|
||||
data: Annotated[AccountDeleteForm, Depends(json_or_form(AccountDeleteForm))],
|
||||
):
|
||||
target, denied = _owner_only(request, username)
|
||||
if denied is not None:
|
||||
return denied
|
||||
if not await _password_matches(data.password, target.get("password_hash", "")):
|
||||
audit.record(
|
||||
request,
|
||||
"account.delete.request",
|
||||
result="denied",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
summary=f"account deletion for {target['username']} refused: wrong password",
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
return json_error(403, "That password is not correct")
|
||||
result = deletion.delete_account(target)
|
||||
if result is None:
|
||||
return json_error(409, "This account is already being deleted")
|
||||
logger.info(f"Account {target['username']} deleted by request")
|
||||
audit.record(
|
||||
request,
|
||||
"account.delete.request",
|
||||
target_type="user",
|
||||
target_uid=target["uid"],
|
||||
target_label=target["username"],
|
||||
metadata={
|
||||
"stamp": result["stamp"],
|
||||
"rows": result["rows"],
|
||||
"grace_hours": result["grace_hours"],
|
||||
},
|
||||
summary=f"account {target['username']} deleted",
|
||||
links=[audit.target("user", target["uid"], target["username"])],
|
||||
)
|
||||
response = action_result(request, "/", data=result)
|
||||
response.delete_cookie("session")
|
||||
return response
|
||||
@ -6,6 +6,7 @@ from fastapi import Depends, APIRouter, Request
|
||||
from devplacepy.models import ProfileForm
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from devplacepy.database import (
|
||||
get_setting,
|
||||
get_table,
|
||||
get_customization_prefs,
|
||||
get_notification_prefs,
|
||||
@ -34,6 +35,13 @@ from devplacepy.database.awards import (
|
||||
get_user_awards,
|
||||
)
|
||||
from devplacepy.content import can_view_project, enrich_items
|
||||
from devplacepy.routers.profile.consent import consent_view
|
||||
from devplacepy.services.moderation.deletion import grace_hours
|
||||
from devplacepy.services.moderation.screening import (
|
||||
record as record_screening,
|
||||
refuse_if_blocked,
|
||||
screen_fields,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
get_badge,
|
||||
@ -363,6 +371,30 @@ async def profile_page(
|
||||
if (tab == "notifications" and can_manage_customization)
|
||||
else False
|
||||
)
|
||||
consents = (
|
||||
consent_view("user", profile_user["uid"])
|
||||
if (tab == "privacy" and can_manage_customization)
|
||||
else []
|
||||
)
|
||||
privacy_fields = (
|
||||
{
|
||||
"mature_opt_in": bool(profile_user.get("mature_opt_in")),
|
||||
"age_band": profile_user.get("age_band", ""),
|
||||
"terms_version": profile_user.get("terms_version", ""),
|
||||
"terms_accepted_at": profile_user.get("terms_accepted_at", ""),
|
||||
"suspended_until": profile_user.get("suspended_until", ""),
|
||||
"suspension_reason": profile_user.get("suspension_reason", ""),
|
||||
}
|
||||
if can_manage_customization
|
||||
else {
|
||||
"mature_opt_in": False,
|
||||
"age_band": "",
|
||||
"terms_version": "",
|
||||
"terms_accepted_at": "",
|
||||
"suspended_until": "",
|
||||
"suspension_reason": "",
|
||||
}
|
||||
)
|
||||
|
||||
base = site_url(request)
|
||||
robots = "noindex,follow" if posts_count < 2 else "index,follow"
|
||||
@ -434,6 +466,10 @@ async def profile_page(
|
||||
"cust_disable_global": customization_prefs["disable_global"],
|
||||
"cust_disable_pagetype": customization_prefs["disable_pagetype"],
|
||||
"notification_prefs": notification_prefs,
|
||||
"consents": consents,
|
||||
**privacy_fields,
|
||||
"current_terms_version": get_setting("terms_version", "1") or "1",
|
||||
"deletion_grace_hours": grace_hours(),
|
||||
"ai_quota": ai_quota,
|
||||
"correction_usage": correction_usage,
|
||||
"modifier_usage": modifier_usage,
|
||||
@ -461,6 +497,10 @@ async def profile_page(
|
||||
async def update_profile(request: Request, data: Annotated[ProfileForm, Depends(json_or_form(ProfileForm))]):
|
||||
user = require_user(request)
|
||||
users = get_table("users")
|
||||
screening = screen_fields(
|
||||
"users", {"bio": data.bio, "location": data.location}
|
||||
)
|
||||
refuse_if_blocked(screening)
|
||||
users.update(
|
||||
{
|
||||
"uid": user["uid"],
|
||||
@ -472,6 +512,13 @@ async def update_profile(request: Request, data: Annotated[ProfileForm, Depends(
|
||||
["uid"],
|
||||
)
|
||||
clear_user_cache(user["uid"])
|
||||
record_screening(
|
||||
screening,
|
||||
target_type="user",
|
||||
target_uid=user["uid"],
|
||||
actor_uid=user["uid"],
|
||||
request=request,
|
||||
)
|
||||
schedule_correction(user, "users", user["uid"], request)
|
||||
schedule_modification(user, "users", user["uid"], request)
|
||||
|
||||
|
||||
@ -9,7 +9,9 @@ This file documents the project detail page, the per-project virtual filesystem,
|
||||
|
||||
Each project card links to `/projects/{project_uid}` showing full project details, author info, platforms, star count, delete-for-owner, and (for the owner) Private/Read-only toggle buttons plus badges (see **Project visibility and read-only** below). The route is `GET /projects/{project_uid}` in `routers/projects/index.py` and 404s when the viewer cannot see a private project. The sitemap generator links to this URL (not the old `?user_uid=` query param). The detail page also links to the project filesystem at `/projects/{slug}/files`.
|
||||
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
**Project overview page.** The detail page is a dedicated project showcase: one encompassing dark card (`.project-shell`, the site `--bg-card` surface with clipped corners) wraps the hero, the section tab bar and the two-column body, and every inner panel (tab bar, sidebar cards, devlog post cards, empty state, comments section) sits one elevation lighter on `--bg-secondary`. The hero's cover banner is the attachment referenced by `projects.cover_attachment_uid`, falling back to the first image attachment (brand-gradient band when neither exists); the title block, type/platform chips and author row render OVERLAID on the banner behind a bottom scrim (dark text-shadow for readability) beside the optional `projects.logo_attachment_uid` tile, with an owner-set **Visit Website** CTA (`projects.website_url`). Cover and logo ride the ONE existing upload pipeline: `dp-upload` widgets (`name="cover_attachment_uid"`/`"logo_attachment_uid"`, `max-files="1"`) in the create/edit modals upload to `/uploads/upload`, the route validates each uid via `database.get_user_attachment` (must exist, belong to the actor, be an image - `_hero_attachment_uid`) and links it to the project through `attachments.link_attachments`; an empty value on edit keeps the current image (no removal control). `website_url`/`repo_url` are normalized by `models.normalize_website_url` (scheme-less input gets `https://`, non-http(s) rejected) and render with `rel="noopener nofollow"`. Below the hero an anchor **section tab bar** (`.project-tabs`, underline style, Overview `.active`) links `#about` / `#devlog` / `#screenshots` (only when gallery images exist) / `#comments` / the Files page - server-rendered anchors, no JS tab state. The main column holds **About** (description + non-image attachments), the **Devlog** (every post whose `project_uid` points at the project via `_post_card.html` - the template loads `feed.css` for the card styles alongside `post.css`, the same rule as `news.html`) with `devlog_count` (`content.count_project_devlog`) and an owner **Post update** button (`.project-devlog-post-btn`) opening the shared composer preset to `topic=devlog` + this project (the form lives ONCE in `templates/_post_composer_form.html`, locals `_composer_topic`/`_composer_project`, included by `feed.html` and `project_detail.html` - never fork a second copy), a **Screenshots** gallery (image attachments minus the cover/logo, thumbnails, `data-lightbox`, capped at 12 rendered), and the comment thread; the sidebar holds Links (website/repository/files/fork source), the Stats card (5 `.project-stat` entries + a last-update line) and the Author card. Owners add gallery images via the More-menu **Add screenshots** modal: `_attachment_form.html` uploads, then `POST /projects/{slug}/screenshots` (`ProjectScreenshotsForm`, owner-only, audit `project.screenshots.add`) links the uids through the same `link_attachments` choke point; Devii action `project_add_screenshots`, docs id `projects-screenshots`. `comment_count`/`devlog_count` ride `ProjectDetailOut`; the new project fields ride `ProjectOut`; the page og:image prefers the cover attachment. **Locator discipline:** the page has several `Files` anchors (action row, tab bar, sidebar) and, for owners, a second hidden `textarea[name='content']`/Post button inside the composer modal - tests MUST scope (`.project-detail-actions a:has-text('Files')`, `.comment-form textarea[name='content']`).
|
||||
|
||||
**Action row overflow.** The detail page has more actions than fit one line, so `project_detail.html` keeps the engagement actions inline (Files, Share, star vote, bookmark, reactions) and collapses the rest behind a single **More** button (`.project-actions-more`) that opens the shared `app.contextMenu`. The secondary actions (Workspace, Containers, Download zip, Fork, the owner Edit/Private/Read-only/Delete controls) live as real elements inside a hidden `.project-actions-overflow` container, each tagged `data-menu-action` plus `data-menu-icon`/`data-menu-label`. `static/js/ProjectActionsMenu.js` builds the menu from those elements and each item's `onSelect` simply `.click()`s the real element, so all existing wiring is reused unchanged - `app.zipDownloader` (`data-zip-download`), `app.projectForker` (`data-fork-project`), `data-share`, the `data-modal` Edit trigger, and the delegated `data-confirm`/`data-confirm-danger` dialog on the owner forms. The open handler must `stopPropagation()` because `app.contextMenu`'s document-level close listener would otherwise dismiss it on the same click (every other caller opens it from a right-click `attach`, not a left-click). Reuse this pattern - a `More` trigger over `[data-menu-action]` real elements - for any future action row that overflows; do not duplicate controller logic into menu callbacks.
|
||||
|
||||
**Owner editing.** Mirrors post editing exactly: an owner-only **Edit** menu item (`data-modal="edit-project-modal"`) opens the `modal()` macro's `edit-project-modal`, a plain `POST` form to `/projects/edit/{slug}` (route `edit_project` in `routers/projects/index.py`, body `ProjectEditForm`, owner-gated through the shared `content.edit_content_item` which returns 403 JSON / redirect for non-owners and stamps `updated_at`). The modal is the create modal pre-filled from the `project` row (title, description, type/status radios pre-checked, dates via `format_date()` back to DD/MM/YYYY). `is_private`/`read_only` are NOT edited here - they stay on their dedicated toggles. The platforms tag widget reuses the create modal's `platforms-input`/`platforms`/`platforms-tags` ids; `ProfileEditor.initPlatformTags` now **seeds existing tags** from the hidden `#platforms` value on load, so both the empty create form and the pre-filled edit form work from the same code. Devii tool `edit_project`; documented in `docs_api.py` (`projects-edit`).
|
||||
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Form, Request, WebSocket
|
||||
from fastapi import APIRouter, Depends, Form, Request, WebSocket
|
||||
from starlette.responses import Response
|
||||
|
||||
from devplacepy.content import (
|
||||
@ -10,20 +10,28 @@ from devplacepy.content import (
|
||||
can_open_workspace,
|
||||
)
|
||||
from devplacepy.database import get_table, resolve_by_slug
|
||||
from devplacepy.models import TunnelForm
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import EditorPrefsForm, TunnelForm
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.schemas import WorkspaceOut
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.containers import activity, forward, store
|
||||
from devplacepy.services.containers.workspace import provision, quota, tunnels
|
||||
from devplacepy.services.containers import activity, api, forward, store
|
||||
from devplacepy.services.containers.api import ContainerError
|
||||
from devplacepy.services.containers.workspace import editor, provision, quota, tunnels
|
||||
from devplacepy.services.containers.workspace.provision import WorkspaceError
|
||||
from devplacepy.utils import not_found, require_user
|
||||
|
||||
from ._shared import audit_instance
|
||||
from ._shared import audit_instance, fail
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _restart_required(instance: dict, profile: editor.EditorProfile) -> bool:
|
||||
if instance.get("status") != store.ST_RUNNING:
|
||||
return False
|
||||
return editor.restart_required(instance, profile)
|
||||
|
||||
|
||||
def _project_or_404(slug: str) -> dict:
|
||||
project = resolve_by_slug(get_table("projects"), slug)
|
||||
if not project:
|
||||
@ -63,6 +71,7 @@ async def workspace_page(request: Request, slug: str):
|
||||
_guard(request, project, user, "container.workspace.open")
|
||||
instance = provision.find_for_project(project["uid"], user["uid"])
|
||||
limits = quota.resolve(user["uid"])
|
||||
profile = editor.resolve(user["uid"], instance)
|
||||
context = {
|
||||
"project": project,
|
||||
"workspace": provision.view(instance) if instance else None,
|
||||
@ -75,6 +84,13 @@ async def workspace_page(request: Request, slug: str):
|
||||
if instance
|
||||
else ""
|
||||
),
|
||||
"editor_password": (
|
||||
api.ensure_editor_password(instance) if instance else ""
|
||||
),
|
||||
"editor": editor.view(user["uid"], instance),
|
||||
"restart_required": (
|
||||
_restart_required(instance, profile) if instance else False
|
||||
),
|
||||
"user": user,
|
||||
}
|
||||
return respond(request, "workspace.html", context, model=WorkspaceOut)
|
||||
@ -100,7 +116,9 @@ async def workspace_open(request: Request, slug: str):
|
||||
summary=str(error),
|
||||
result="denied",
|
||||
)
|
||||
return json_error(str(error), 400)
|
||||
return json_error(400, str(error))
|
||||
except ContainerError as error:
|
||||
return fail(error)
|
||||
audit_instance(
|
||||
request,
|
||||
user,
|
||||
@ -123,7 +141,7 @@ async def workspace_stop(request: Request, slug: str):
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error("Not allowed to manage this workspace", 403)
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
provision.stop(instance)
|
||||
audit_instance(request, user, "container.workspace.stop", instance, project)
|
||||
return action_result(request, f"/projects/{slug}/workspace")
|
||||
@ -137,7 +155,7 @@ async def workspace_delete(request: Request, slug: str):
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error("Not allowed to manage this workspace", 403)
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
for row in tunnels.list_for_instance(instance["uid"]):
|
||||
tunnels.soft_delete(row["uid"], user["uid"])
|
||||
store.delete_instance(instance["uid"], user["uid"])
|
||||
@ -145,6 +163,63 @@ async def workspace_delete(request: Request, slug: str):
|
||||
return action_result(request, f"/projects/{slug}/workspace")
|
||||
|
||||
|
||||
@router.get("/{slug}/workspace/editor")
|
||||
async def editor_prefs_read(request: Request, slug: str):
|
||||
user = require_user(request)
|
||||
if isinstance(user, Response):
|
||||
return user
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
profile = editor.resolve(user["uid"], instance)
|
||||
return {
|
||||
"editor": editor.view(user["uid"], instance),
|
||||
"restart_required": _restart_required(instance, profile),
|
||||
}
|
||||
|
||||
|
||||
@router.post("/{slug}/workspace/editor")
|
||||
async def editor_prefs_write(
|
||||
request: Request,
|
||||
slug: str,
|
||||
data: Annotated[EditorPrefsForm, Depends(json_or_form(EditorPrefsForm))],
|
||||
):
|
||||
user = require_user(request)
|
||||
if isinstance(user, Response):
|
||||
return user
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
owner_uid = instance.get("workspace_owner_uid") or user["uid"]
|
||||
if data.reset:
|
||||
editor.reset_prefs(owner_uid, user["uid"])
|
||||
summary = f"{user['username']} reset their workspace editor preferences"
|
||||
else:
|
||||
editor.save_prefs(
|
||||
owner_uid, data.model_dump(exclude={"reset"}, exclude_unset=True)
|
||||
)
|
||||
summary = f"{user['username']} updated their workspace editor preferences"
|
||||
audit_instance(
|
||||
request,
|
||||
user,
|
||||
"container.workspace.editor.update",
|
||||
instance,
|
||||
project,
|
||||
summary=summary,
|
||||
)
|
||||
profile = editor.resolve(owner_uid, instance)
|
||||
return action_result(
|
||||
request,
|
||||
f"/projects/{slug}/workspace",
|
||||
data={
|
||||
"editor": editor.view(owner_uid, instance),
|
||||
"restart_required": _restart_required(instance, profile),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{slug}/workspace/tunnels")
|
||||
async def tunnel_list(request: Request, slug: str):
|
||||
user = require_user(request)
|
||||
@ -153,7 +228,7 @@ async def tunnel_list(request: Request, slug: str):
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error("Not allowed to manage this workspace", 403)
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
return {"tunnels": tunnels.list_for_instance(instance["uid"])}
|
||||
|
||||
|
||||
@ -167,17 +242,13 @@ async def tunnel_create(
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error("Not allowed to manage this workspace", 403)
|
||||
if data.container_port <= 0:
|
||||
return json_error("container_port must be between 1 and 65535", 400)
|
||||
limits = quota.resolve(instance.get("workspace_owner_uid", ""), instance)
|
||||
if limits.max_tunnels and tunnels.count_for_instance(
|
||||
instance["uid"]
|
||||
) >= limits.max_tunnels:
|
||||
return json_error(f"tunnel limit reached ({limits.max_tunnels})", 400)
|
||||
row = tunnels.create(instance, data.label, data.container_port, user["uid"])
|
||||
if not row:
|
||||
return json_error("could not create tunnel", 400)
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
try:
|
||||
row = provision.publish_tunnel(
|
||||
instance, data.label, data.container_port, user["uid"]
|
||||
)
|
||||
except provision.WorkspaceError as error:
|
||||
return json_error(400, str(error))
|
||||
audit_instance(
|
||||
request,
|
||||
user,
|
||||
@ -186,7 +257,6 @@ async def tunnel_create(
|
||||
project,
|
||||
metadata={"hostname": row["hostname"], "port": data.container_port},
|
||||
)
|
||||
provision.write_manifest(instance)
|
||||
return action_result(request, f"/projects/{slug}/workspace", data=row)
|
||||
|
||||
|
||||
@ -199,7 +269,7 @@ async def tunnel_delete(request: Request, slug: str, uid: str):
|
||||
project = _project_or_404(slug)
|
||||
instance = _workspace_or_404(project, user)
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return json_error("Not allowed to manage this workspace", 403)
|
||||
return json_error(403, "Not allowed to manage this workspace")
|
||||
row = tunnels.get(uid)
|
||||
if not row or row.get("instance_uid") != instance["uid"]:
|
||||
raise not_found("Tunnel not found")
|
||||
@ -225,7 +295,7 @@ def _editor_guard(request: Request, slug: str, uid: str):
|
||||
if not instance or not instance.get("is_workspace"):
|
||||
raise not_found("Workspace not found")
|
||||
if not can_manage_workspace(instance, project, user):
|
||||
return None, None, json_error("Not allowed to open this workspace", 403)
|
||||
return None, None, json_error(403, "Not allowed to open this workspace")
|
||||
return project, instance, None
|
||||
|
||||
|
||||
@ -275,4 +345,5 @@ async def editor_proxy_ws(
|
||||
await websocket.close(code=1011)
|
||||
return
|
||||
activity.touch(instance["uid"])
|
||||
await forward.proxy_ws(websocket, host, port, path)
|
||||
prefix = f"/projects/{slug}/containers/instances/{uid}/code"
|
||||
await forward.proxy_ws(websocket, host, port, path, prefix=prefix)
|
||||
|
||||
@ -4,9 +4,15 @@ import logging
|
||||
from typing import Annotated
|
||||
from sqlalchemy import or_
|
||||
from fastapi import Depends, APIRouter, Request
|
||||
from devplacepy.models import ProjectForm, ProjectEditForm, ProjectFlagForm, ForkForm
|
||||
from devplacepy.models import (
|
||||
ProjectForm,
|
||||
ProjectEditForm,
|
||||
ProjectFlagForm,
|
||||
ProjectScreenshotsForm,
|
||||
ForkForm,
|
||||
)
|
||||
from fastapi.responses import HTMLResponse, RedirectResponse, JSONResponse
|
||||
from devplacepy.attachments import get_attachments_batch
|
||||
from devplacepy.attachments import get_attachments_batch, link_attachments
|
||||
from devplacepy.database import (
|
||||
get_table,
|
||||
get_users_by_uids,
|
||||
@ -25,6 +31,7 @@ from devplacepy.database import (
|
||||
get_fork_parent,
|
||||
count_forks,
|
||||
get_top_authors,
|
||||
get_user_attachment,
|
||||
)
|
||||
from devplacepy.project_files import count_files
|
||||
from devplacepy.services.jobs import queue
|
||||
@ -39,7 +46,9 @@ from devplacepy.content import (
|
||||
is_owner,
|
||||
can_view_project,
|
||||
can_view_project_containers,
|
||||
can_open_workspace,
|
||||
get_project_devlog,
|
||||
count_project_devlog,
|
||||
)
|
||||
from devplacepy.utils import (
|
||||
get_current_user,
|
||||
@ -175,6 +184,38 @@ async def projects_page(
|
||||
model=ProjectsOut,
|
||||
)
|
||||
|
||||
def _editor_launch(project: dict, user: dict) -> dict:
|
||||
from devplacepy.services.containers import store
|
||||
from devplacepy.services.containers.workspace import editor, provision
|
||||
|
||||
blank = {"url": "", "mode": "tab", "width": 0, "height": 0}
|
||||
instance = provision.find_for_project(project["uid"], user["uid"])
|
||||
if not instance or instance.get("suspended_at"):
|
||||
return blank
|
||||
if instance.get("status") != store.ST_RUNNING:
|
||||
return blank
|
||||
slug = project["slug"] or project["uid"]
|
||||
profile = editor.resolve(user["uid"], instance)
|
||||
return {
|
||||
"url": f"/projects/{slug}/containers/instances/{instance['uid']}/code/",
|
||||
"mode": profile.window_mode,
|
||||
"width": profile.window_width,
|
||||
"height": profile.window_height,
|
||||
}
|
||||
|
||||
|
||||
def _hero_attachment_uid(user: dict, raw_uid: str) -> str | None:
|
||||
uid = (raw_uid or "").strip()
|
||||
if not uid:
|
||||
return None
|
||||
attachment = get_user_attachment(uid)
|
||||
if not attachment or attachment.get("user_uid") != user["uid"]:
|
||||
return None
|
||||
if not attachment.get("is_image"):
|
||||
return None
|
||||
return uid
|
||||
|
||||
|
||||
@router.get("/{project_slug}", response_class=HTMLResponse)
|
||||
async def project_detail(request: Request, project_slug: str, before: str = None):
|
||||
user = get_current_user(request)
|
||||
@ -194,13 +235,21 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
|
||||
base = site_url(request)
|
||||
robots = "noindex,nofollow" if project.get("is_private") else "index,follow"
|
||||
cover_url = next(
|
||||
(
|
||||
a["url"]
|
||||
for a in detail["attachments"]
|
||||
if a["uid"] == project.get("cover_attachment_uid") and a.get("is_image")
|
||||
),
|
||||
None,
|
||||
)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title=project.get("title", "Project"),
|
||||
description=project.get("description", ""),
|
||||
seo_target=("project", project["uid"]),
|
||||
robots=robots,
|
||||
og_image=first_image_url(project, detail["attachments"]),
|
||||
og_image=cover_url or first_image_url(project, detail["attachments"]),
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Projects", "url": "/projects"},
|
||||
@ -211,6 +260,13 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
],
|
||||
schemas=[website_schema(base), software_application_schema(project, base)],
|
||||
)
|
||||
viewer_can_workspace = can_open_workspace(project, user)
|
||||
editor_launch = (
|
||||
_editor_launch(project, user)
|
||||
if viewer_can_workspace
|
||||
else {"url": "", "mode": "tab", "width": 0, "height": 0}
|
||||
)
|
||||
workspace_editor_url = editor_launch["url"]
|
||||
parent = get_fork_parent(project["uid"])
|
||||
forked_from = (
|
||||
{
|
||||
@ -256,11 +312,22 @@ async def project_detail(request: Request, project_slug: str, before: str = None
|
||||
"is_private": bool(project.get("is_private")),
|
||||
"read_only": bool(project.get("read_only")),
|
||||
"viewer_can_containers": can_view_project_containers(project, user),
|
||||
"viewer_can_workspace": viewer_can_workspace,
|
||||
"workspace_editor_url": workspace_editor_url,
|
||||
"workspace_editor_mode": editor_launch["mode"],
|
||||
"workspace_editor_width": editor_launch["width"],
|
||||
"workspace_editor_height": editor_launch["height"],
|
||||
"forked_from": forked_from,
|
||||
"fork_count": count_forks(project["uid"]),
|
||||
"file_count": count_files(project["uid"]),
|
||||
"comment_count": get_table("comments").count(
|
||||
target_type="project",
|
||||
target_uid=project["uid"],
|
||||
deleted_at=None,
|
||||
),
|
||||
"devlog_posts": devlog_posts,
|
||||
"devlog_next_cursor": devlog_next_cursor,
|
||||
"devlog_count": count_project_devlog(project["uid"]),
|
||||
},
|
||||
),
|
||||
model=ProjectDetailOut,
|
||||
@ -376,6 +443,8 @@ async def create_project(request: Request, data: Annotated[ProjectForm, Depends(
|
||||
user = require_user(request)
|
||||
title = data.title.strip()
|
||||
description = data.description.strip()
|
||||
cover_uid = _hero_attachment_uid(user, data.cover_attachment_uid)
|
||||
logo_uid = _hero_attachment_uid(user, data.logo_attachment_uid)
|
||||
|
||||
uid, project_slug = create_content_item(
|
||||
"projects",
|
||||
@ -389,6 +458,10 @@ async def create_project(request: Request, data: Annotated[ProjectForm, Depends(
|
||||
"project_type": data.project_type,
|
||||
"platforms": data.platforms.strip(),
|
||||
"status": data.status,
|
||||
"website_url": data.website_url or None,
|
||||
"repo_url": data.repo_url or None,
|
||||
"cover_attachment_uid": cover_uid,
|
||||
"logo_attachment_uid": logo_uid,
|
||||
"is_private": 1 if data.is_private else 0,
|
||||
"read_only": 0,
|
||||
},
|
||||
@ -399,6 +472,7 @@ async def create_project(request: Request, data: Annotated[ProjectForm, Depends(
|
||||
data.attachment_uids,
|
||||
request,
|
||||
)
|
||||
link_attachments([u for u in (cover_uid, logo_uid) if u], "project", uid)
|
||||
url = f"/projects/{project_slug}"
|
||||
return action_result(
|
||||
request, url, data={"uid": uid, "slug": project_slug, "url": url}
|
||||
@ -409,23 +483,71 @@ async def edit_project(
|
||||
request: Request, project_slug: str, data: Annotated[ProjectEditForm, Depends(json_or_form(ProjectEditForm))]
|
||||
):
|
||||
user = require_user(request)
|
||||
return edit_content_item(
|
||||
fields = {
|
||||
"title": data.title.strip(),
|
||||
"description": data.description.strip(),
|
||||
"release_date": data.release_date or None,
|
||||
"demo_date": data.demo_date or None,
|
||||
"project_type": data.project_type,
|
||||
"platforms": data.platforms.strip(),
|
||||
"status": data.status,
|
||||
"website_url": data.website_url or None,
|
||||
"repo_url": data.repo_url or None,
|
||||
}
|
||||
hero_uids = []
|
||||
for field in ("cover_attachment_uid", "logo_attachment_uid"):
|
||||
uid = _hero_attachment_uid(user, getattr(data, field))
|
||||
if uid:
|
||||
fields[field] = uid
|
||||
hero_uids.append(uid)
|
||||
result = edit_content_item(
|
||||
request,
|
||||
"projects",
|
||||
user,
|
||||
project_slug,
|
||||
{
|
||||
"title": data.title.strip(),
|
||||
"description": data.description.strip(),
|
||||
"release_date": data.release_date or None,
|
||||
"demo_date": data.demo_date or None,
|
||||
"project_type": data.project_type,
|
||||
"platforms": data.platforms.strip(),
|
||||
"status": data.status,
|
||||
},
|
||||
fields,
|
||||
"/projects",
|
||||
target_type="project",
|
||||
)
|
||||
if hero_uids:
|
||||
project = resolve_by_slug(get_table("projects"), project_slug)
|
||||
if project and is_owner(project, user):
|
||||
link_attachments(hero_uids, "project", project["uid"])
|
||||
return result
|
||||
|
||||
@router.post("/{project_slug}/screenshots")
|
||||
async def add_project_screenshots(
|
||||
request: Request,
|
||||
project_slug: str,
|
||||
data: Annotated[ProjectScreenshotsForm, Depends(json_or_form(ProjectScreenshotsForm))],
|
||||
):
|
||||
user = require_user(request)
|
||||
project = resolve_by_slug(get_table("projects"), project_slug)
|
||||
if not project:
|
||||
raise not_found("Project not found")
|
||||
if not is_owner(project, user):
|
||||
if wants_json(request):
|
||||
return json_error(403, "Not allowed")
|
||||
return RedirectResponse(url=f"/projects/{project_slug}", status_code=302)
|
||||
link_attachments(data.attachment_uids, "project", project["uid"])
|
||||
audit.record(
|
||||
request,
|
||||
"project.screenshots.add",
|
||||
user=user,
|
||||
target_type="project",
|
||||
target_uid=project["uid"],
|
||||
target_label=project.get("title"),
|
||||
metadata={"attachment_count": len(data.attachment_uids)},
|
||||
summary=f"{user['username']} added {len(data.attachment_uids)} screenshot(s) to project {project.get('title')}",
|
||||
links=[audit.target("project", project["uid"], project.get("title"))],
|
||||
)
|
||||
url = f"/projects/{project['slug'] or project['uid']}#screenshots"
|
||||
return action_result(
|
||||
request,
|
||||
url,
|
||||
data={"uid": project["uid"], "linked": len(data.attachment_uids), "url": url},
|
||||
)
|
||||
|
||||
|
||||
_FLAG_EVENTS = {
|
||||
("is_private", True): "project.visibility.private",
|
||||
|
||||
@ -55,7 +55,6 @@ async def proxy_ws(websocket: WebSocket, slug: str, path: str = ""):
|
||||
if instance is None or not host or not port:
|
||||
await websocket.close(code=1011)
|
||||
return
|
||||
await websocket.accept()
|
||||
audit.record(
|
||||
websocket,
|
||||
"proxy.access",
|
||||
@ -67,4 +66,4 @@ async def proxy_ws(websocket: WebSocket, slug: str, path: str = ""):
|
||||
summary=f"websocket proxied to instance {instance.get('name')} via ingress {slug}",
|
||||
links=[audit.instance(instance["uid"], instance.get("name"))],
|
||||
)
|
||||
await forward.proxy_ws(websocket, host, port, path, accepted=True)
|
||||
await forward.proxy_ws(websocket, host, port, path, prefix=f"/p/{slug}")
|
||||
|
||||
143
devplacepy/routers/reports.py
Normal file
143
devplacepy/routers/reports.py
Normal file
@ -0,0 +1,143 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.database import (
|
||||
REPORTABLE_TARGETS,
|
||||
REPORT_SEVERITIES,
|
||||
REPORT_STATUSES,
|
||||
report_reason_options,
|
||||
)
|
||||
from devplacepy.dependencies import json_or_form
|
||||
from devplacepy.models import ReportForm
|
||||
from devplacepy.responses import action_result, json_error, respond
|
||||
from devplacepy.schemas import ReportListOut, ReportReasonsOut
|
||||
from devplacepy.seo import base_seo_context, site_url, website_schema
|
||||
from devplacepy.services.audit import record as audit
|
||||
from devplacepy.services.moderation import queue, sla
|
||||
from devplacepy.utils import create_notification, get_current_user, require_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/reasons")
|
||||
async def report_reasons(request: Request):
|
||||
base = site_url(request)
|
||||
return respond(
|
||||
request,
|
||||
"report_reasons.html",
|
||||
{
|
||||
**base_seo_context(
|
||||
request,
|
||||
title="Report reasons",
|
||||
description="The categories DevPlace accepts content reports under.",
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Report reasons", "url": "/reports/reasons"},
|
||||
],
|
||||
schemas=[website_schema(base)],
|
||||
),
|
||||
"request": request,
|
||||
"user": get_current_user(request),
|
||||
"reasons": report_reason_options(),
|
||||
"severities": list(REPORT_SEVERITIES),
|
||||
},
|
||||
model=ReportReasonsOut,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/mine", response_class=HTMLResponse)
|
||||
async def my_reports(request: Request, status: str = "", page: int = 1):
|
||||
user = require_user(request)
|
||||
if status not in REPORT_STATUSES:
|
||||
status = ""
|
||||
reports, pagination = queue.list_reports(
|
||||
status=status, reporter_uid=user["uid"], page=page
|
||||
)
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Your reports",
|
||||
description="The reports you filed and the outcome of each.",
|
||||
robots="noindex,nofollow",
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Your reports", "url": "/reports/mine"},
|
||||
],
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
return respond(
|
||||
request,
|
||||
"reports_mine.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": user,
|
||||
"reports": reports,
|
||||
"pagination": pagination,
|
||||
"status": status,
|
||||
"reasons": report_reason_options(),
|
||||
},
|
||||
model=ReportListOut,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{target_type}/{target_uid}")
|
||||
async def submit_report(
|
||||
request: Request,
|
||||
target_type: str,
|
||||
target_uid: str,
|
||||
data: Annotated[ReportForm, Depends(json_or_form(ReportForm))],
|
||||
):
|
||||
user = require_user(request)
|
||||
if target_type not in REPORTABLE_TARGETS:
|
||||
return json_error(400, "Unknown report target")
|
||||
owner_uid = queue.owner_uid_for(target_type, target_uid)
|
||||
if owner_uid and owner_uid == user["uid"]:
|
||||
return json_error(400, "You cannot report your own content")
|
||||
report = queue.raise_report(
|
||||
target_type=target_type,
|
||||
target_uid=target_uid,
|
||||
reporter_uid=user["uid"],
|
||||
reason=data.reason,
|
||||
detail=data.detail,
|
||||
origin="member",
|
||||
)
|
||||
if not report:
|
||||
return json_error(400, "Report could not be filed")
|
||||
hours = sla.sla_hours()
|
||||
logger.info(
|
||||
f"{user['username']} reported {target_type} {target_uid} as {data.reason}"
|
||||
)
|
||||
audit.record(
|
||||
request,
|
||||
"report.create",
|
||||
user=user,
|
||||
target_type=target_type,
|
||||
target_uid=target_uid,
|
||||
metadata={"reason": data.reason, "severity": report["severity"]},
|
||||
summary=f"{user['username']} reported {target_type} {target_uid} as {data.reason}",
|
||||
links=[audit.target(target_type, target_uid)],
|
||||
)
|
||||
create_notification(
|
||||
user["uid"],
|
||||
"moderation",
|
||||
f"Report received. A moderator reviews it within {hours} hours.",
|
||||
user["uid"],
|
||||
"/reports/mine",
|
||||
)
|
||||
return action_result(
|
||||
request,
|
||||
"/reports/mine",
|
||||
data={
|
||||
"uid": report["uid"],
|
||||
"status": report["status"],
|
||||
"severity": report["severity"],
|
||||
"sla_hours": hours,
|
||||
},
|
||||
)
|
||||
@ -22,6 +22,8 @@ Disallow: /avatar/
|
||||
Disallow: /follow/
|
||||
Disallow: /admin/
|
||||
Disallow: /uploads/
|
||||
Disallow: /reports/mine
|
||||
Disallow: /profile/*/delete
|
||||
Disallow: /*?tab=
|
||||
Disallow: /*?sort=
|
||||
Allow: /static/
|
||||
|
||||
@ -27,18 +27,8 @@ def resolve(host: str):
|
||||
return row, instance, None, None
|
||||
if instance.get("status") != store.ST_RUNNING:
|
||||
return row, instance, None, None
|
||||
gateway, _ = api.proxy_target(instance)
|
||||
host_port = _published_host_port(instance, int(row.get("container_port") or 0))
|
||||
return row, instance, gateway, host_port
|
||||
|
||||
|
||||
def _published_host_port(instance: dict, container_port: int) -> int:
|
||||
import json
|
||||
|
||||
for mapping in json.loads(instance.get("ports_json") or "[]"):
|
||||
if int(mapping.get("container") or 0) == container_port:
|
||||
return int(mapping.get("host") or 0)
|
||||
return 0
|
||||
host, port = api.tunnel_target(instance, int(row.get("container_port") or 0))
|
||||
return row, instance, host, port
|
||||
|
||||
|
||||
async def handle_http(request: Request, path: str) -> Response:
|
||||
@ -50,11 +40,18 @@ async def handle_http(request: Request, path: str) -> Response:
|
||||
return Response("this workspace is suspended", status_code=403)
|
||||
if not gateway or not port:
|
||||
return Response("the tunnel has no reachable port", status_code=502)
|
||||
response = await forward.proxy_http(request, gateway, port, path)
|
||||
size = len(response.body) if hasattr(response, "body") and response.body else 0
|
||||
activity.touch(instance["uid"], egress_bytes=size)
|
||||
tunnels.record_hit(row["uid"], size)
|
||||
return response
|
||||
return await forward.proxy_http(
|
||||
request,
|
||||
gateway,
|
||||
port,
|
||||
path,
|
||||
on_complete=lambda sent: record_traffic(instance["uid"], row["uid"], sent),
|
||||
)
|
||||
|
||||
|
||||
def record_traffic(instance_uid: str, tunnel_uid: str, sent: int) -> None:
|
||||
activity.touch(instance_uid, egress_bytes=sent)
|
||||
tunnels.record_hit(tunnel_uid, sent)
|
||||
|
||||
|
||||
async def handle_ws(websocket: WebSocket, path: str) -> None:
|
||||
|
||||
116
devplacepy/routers/workspaces.py
Normal file
116
devplacepy/routers/workspaces.py
Normal file
@ -0,0 +1,116 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
|
||||
from devplacepy.database import (
|
||||
build_pagination,
|
||||
db,
|
||||
get_maturity_by_targets,
|
||||
get_table,
|
||||
get_users_by_uids,
|
||||
)
|
||||
from devplacepy.content import can_view_project
|
||||
from devplacepy.responses import respond
|
||||
from devplacepy.schemas import WorkspaceIndexOut
|
||||
from devplacepy.seo import base_seo_context, public_base_url, site_url, website_schema
|
||||
from devplacepy.utils import get_current_user
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter()
|
||||
|
||||
PER_PAGE = 50
|
||||
|
||||
|
||||
def published_instances() -> list[dict]:
|
||||
if "instances" not in db.tables:
|
||||
return []
|
||||
table = get_table("instances")
|
||||
if not table.has_column("ingress_slug"):
|
||||
return []
|
||||
rows = [
|
||||
row
|
||||
for row in table.find(deleted_at=None, order_by=["-created_at"])
|
||||
if (row.get("ingress_slug") or "").strip()
|
||||
]
|
||||
return rows
|
||||
|
||||
|
||||
def projects_by_uids(uids: list[str]) -> dict[str, dict]:
|
||||
unique = [uid for uid in set(uids) if uid]
|
||||
if not unique or "projects" not in db.tables:
|
||||
return {}
|
||||
table = get_table("projects")
|
||||
return {
|
||||
row["uid"]: row for row in table.find(table.table.columns.uid.in_(unique))
|
||||
}
|
||||
|
||||
|
||||
def index_entries(rows: list[dict], user: dict | None) -> list[dict]:
|
||||
projects = projects_by_uids([row.get("project_uid", "") for row in rows])
|
||||
owners = get_users_by_uids(
|
||||
[row.get("owner_uid") or row.get("created_by") for row in rows]
|
||||
)
|
||||
maturity = get_maturity_by_targets("workspace", [row["uid"] for row in rows])
|
||||
base = public_base_url()
|
||||
entries = []
|
||||
for row in rows:
|
||||
project = projects.get(row.get("project_uid", ""))
|
||||
if not can_view_project(project, user):
|
||||
project = None
|
||||
owner_uid = row.get("owner_uid") or row.get("created_by") or ""
|
||||
owner = owners.get(owner_uid)
|
||||
slug = row["ingress_slug"]
|
||||
project_slug = (project or {}).get("slug") or (project or {}).get("uid") or ""
|
||||
entries.append(
|
||||
{
|
||||
"uid": row["uid"],
|
||||
"name": row.get("name") or slug,
|
||||
"slug": slug,
|
||||
"owner_uid": owner_uid,
|
||||
"url": f"{base}/p/{slug}" if base else f"/p/{slug}",
|
||||
"description": (project or {}).get("description", "") or "",
|
||||
"owner": owner["username"] if owner else "",
|
||||
"maturity": maturity.get(row["uid"], {}).get("level", "general"),
|
||||
"project_url": f"/projects/{project_slug}" if project_slug else "",
|
||||
}
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
@router.get("/index", response_class=HTMLResponse)
|
||||
async def workspace_index(request: Request, page: int = 1):
|
||||
user = get_current_user(request)
|
||||
rows = published_instances()
|
||||
pagination = build_pagination(page, len(rows), PER_PAGE)
|
||||
offset = (pagination["page"] - 1) * pagination["per_page"]
|
||||
window = rows[offset : offset + pagination["per_page"]]
|
||||
base = site_url(request)
|
||||
seo_ctx = base_seo_context(
|
||||
request,
|
||||
title="Published workspaces",
|
||||
description=(
|
||||
"Every workspace DevPlace members have published to the public ingress, "
|
||||
"with its owner, project and direct link."
|
||||
),
|
||||
breadcrumbs=[
|
||||
{"name": "Home", "url": "/feed"},
|
||||
{"name": "Published workspaces", "url": "/workspaces/index"},
|
||||
],
|
||||
schemas=[website_schema(base)],
|
||||
)
|
||||
return respond(
|
||||
request,
|
||||
"workspace_index.html",
|
||||
{
|
||||
**seo_ctx,
|
||||
"request": request,
|
||||
"user": user,
|
||||
"workspaces": index_entries(window, user),
|
||||
"pagination": pagination,
|
||||
"total": len(rows),
|
||||
},
|
||||
model=WorkspaceIndexOut,
|
||||
)
|
||||
@ -74,6 +74,7 @@ from devplacepy.schemas.containers import (
|
||||
AdminWorkspacesOut,
|
||||
BotFrameOut,
|
||||
ContainersOut,
|
||||
EditorProfileOut,
|
||||
InstanceOut,
|
||||
ScheduleOut,
|
||||
TunnelOut,
|
||||
@ -183,3 +184,20 @@ from devplacepy.schemas.game import (
|
||||
GameQuestOut,
|
||||
GameStateOut,
|
||||
)
|
||||
from devplacepy.schemas.moderation import (
|
||||
AcceptTermsOut,
|
||||
AccountDeletionOut,
|
||||
AdminModerationOut,
|
||||
AdminReportOut,
|
||||
ConsentListOut,
|
||||
ConsentOut,
|
||||
MaturityOut,
|
||||
ModerationActionOut,
|
||||
ReportCreatedOut,
|
||||
ReportListOut,
|
||||
ReportOut,
|
||||
ReportReasonsOut,
|
||||
SlaOut,
|
||||
WorkspaceIndexItemOut,
|
||||
WorkspaceIndexOut,
|
||||
)
|
||||
|
||||
@ -132,16 +132,36 @@ class WorkspaceFlagOut(_Out):
|
||||
created_at: str = ""
|
||||
|
||||
|
||||
class EditorProfileOut(_Out):
|
||||
trust_all: bool = True
|
||||
theme: str = ""
|
||||
font_size: int = 0
|
||||
terminal_font_size: int = 0
|
||||
zoom_level: int = 0
|
||||
layout: str = ""
|
||||
panel_preset: str = ""
|
||||
boot_agent: str = ""
|
||||
boot_shell: bool = True
|
||||
window_mode: str = ""
|
||||
window_width: int = 0
|
||||
window_height: int = 0
|
||||
cpu_millicores: int = 0
|
||||
cpu_cores: float = 0.0
|
||||
memory_mb: int = 0
|
||||
disk_quota_mb: int = 0
|
||||
sources: dict = {}
|
||||
|
||||
|
||||
class WorkspaceViewOut(_Out):
|
||||
uid: str = ""
|
||||
name: str = ""
|
||||
status: str = ""
|
||||
desired_state: str = ""
|
||||
suspended: bool = False
|
||||
flag_reason: str = ""
|
||||
tunnel_name: str = ""
|
||||
primary_url: str = ""
|
||||
last_active_at: str = ""
|
||||
flag_reason: Optional[str] = ""
|
||||
tunnel_name: Optional[str] = ""
|
||||
primary_url: Optional[str] = ""
|
||||
last_active_at: Optional[str] = ""
|
||||
disk_bytes: int = 0
|
||||
disk_quota_mb: int = 0
|
||||
disk_percent: int = 0
|
||||
@ -153,6 +173,7 @@ class WorkspaceViewOut(_Out):
|
||||
max_tunnels: int = 0
|
||||
tunnels: list[TunnelOut] = []
|
||||
flags: list[WorkspaceFlagOut] = []
|
||||
editor: Optional[EditorProfileOut] = None
|
||||
|
||||
|
||||
class WorkspaceOut(_Out):
|
||||
@ -163,6 +184,9 @@ class WorkspaceOut(_Out):
|
||||
workspace_count: int = 0
|
||||
max_workspaces: int = 0
|
||||
editor_url: str = ""
|
||||
editor_password: str = ""
|
||||
editor: Optional[EditorProfileOut] = None
|
||||
restart_required: bool = False
|
||||
user: Optional[Any] = None
|
||||
|
||||
|
||||
|
||||
@ -121,6 +121,10 @@ class ProjectOut(_Out):
|
||||
read_only: Optional[bool] = None
|
||||
release_date: Optional[str] = None
|
||||
demo_date: Optional[str] = None
|
||||
website_url: Optional[str] = None
|
||||
repo_url: Optional[str] = None
|
||||
cover_attachment_uid: Optional[str] = None
|
||||
logo_attachment_uid: Optional[str] = None
|
||||
created_at: Optional[str] = None
|
||||
updated_at: Optional[str] = None
|
||||
|
||||
|
||||
@ -23,6 +23,7 @@ from devplacepy.schemas.content import (
|
||||
|
||||
class FeedItemOut(_Out):
|
||||
post: PostOut
|
||||
maturity: Optional[str] = None
|
||||
author: Optional[UserOut] = None
|
||||
time_ago: Optional[str] = None
|
||||
my_vote: int = 0
|
||||
@ -37,6 +38,7 @@ class FeedItemOut(_Out):
|
||||
|
||||
class GistItemOut(_Out):
|
||||
gist: GistOut
|
||||
maturity: Optional[str] = None
|
||||
author: Optional[UserOut] = None
|
||||
time_ago: Optional[str] = None
|
||||
my_vote: int = 0
|
||||
@ -120,6 +122,7 @@ class FeedOut(_Out):
|
||||
|
||||
|
||||
class PostDetailOut(_Out):
|
||||
maturity: Optional[str] = None
|
||||
post: PostOut
|
||||
author: Optional[UserOut] = None
|
||||
is_owner: bool = False
|
||||
@ -149,6 +152,7 @@ class ProjectsOut(_Out):
|
||||
|
||||
|
||||
class ProjectDetailOut(_Out):
|
||||
maturity: Optional[str] = None
|
||||
project: ProjectOut
|
||||
author: Optional[UserOut] = None
|
||||
is_owner: bool = False
|
||||
@ -163,11 +167,18 @@ class ProjectDetailOut(_Out):
|
||||
is_private: bool = False
|
||||
read_only: bool = False
|
||||
viewer_can_containers: bool = False
|
||||
viewer_can_workspace: bool = False
|
||||
workspace_editor_url: Optional[str] = None
|
||||
workspace_editor_mode: Optional[str] = None
|
||||
workspace_editor_width: Optional[int] = None
|
||||
workspace_editor_height: Optional[int] = None
|
||||
forked_from: Optional[dict] = None
|
||||
fork_count: int = 0
|
||||
file_count: int = 0
|
||||
comment_count: int = 0
|
||||
devlog_posts: list[FeedItemOut] = []
|
||||
devlog_next_cursor: Optional[str] = None
|
||||
devlog_count: int = 0
|
||||
|
||||
|
||||
class GistsOut(_Out):
|
||||
@ -181,6 +192,7 @@ class GistsOut(_Out):
|
||||
|
||||
|
||||
class GistDetailOut(_Out):
|
||||
maturity: Optional[str] = None
|
||||
gist: GistOut
|
||||
author: Optional[UserOut] = None
|
||||
is_owner: bool = False
|
||||
@ -199,6 +211,7 @@ class NewsListOut(_Out):
|
||||
|
||||
|
||||
class NewsDetailOut(_Out):
|
||||
maturity: Optional[str] = None
|
||||
article: NewsOut
|
||||
canonical_slug: Optional[str] = None
|
||||
image_url: Optional[str] = None
|
||||
|
||||
136
devplacepy/schemas/moderation.py
Normal file
136
devplacepy/schemas/moderation.py
Normal file
@ -0,0 +1,136 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Optional
|
||||
|
||||
from devplacepy.schemas.admin import AdminUserOut
|
||||
from devplacepy.schemas.base import _Out
|
||||
|
||||
|
||||
class ReportOut(_Out):
|
||||
uid: Optional[str] = None
|
||||
target_type: Optional[str] = None
|
||||
target_uid: Optional[str] = None
|
||||
target_url: Optional[str] = None
|
||||
reason: Optional[str] = None
|
||||
reason_label: Optional[str] = None
|
||||
detail: Optional[str] = None
|
||||
severity: Optional[str] = None
|
||||
status: Optional[str] = None
|
||||
origin: Optional[str] = None
|
||||
categories: list[str] = []
|
||||
created_at: Optional[str] = None
|
||||
updated_at: Optional[str] = None
|
||||
resolved_at: Optional[str] = None
|
||||
reporter_name: Optional[str] = None
|
||||
owner_name: Optional[str] = None
|
||||
report_count: Optional[int] = None
|
||||
|
||||
|
||||
class ReportCreatedOut(_Out):
|
||||
report: Optional[ReportOut] = None
|
||||
sla_hours: Optional[int] = None
|
||||
message: Optional[str] = None
|
||||
|
||||
|
||||
class ReportListOut(_Out):
|
||||
reports: list[ReportOut] = []
|
||||
pagination: Optional[Any] = None
|
||||
status: Optional[str] = None
|
||||
reasons: list[Any] = []
|
||||
|
||||
|
||||
class ReportReasonsOut(_Out):
|
||||
reasons: list[Any] = []
|
||||
severities: list[str] = []
|
||||
|
||||
|
||||
class ModerationActionOut(_Out):
|
||||
uid: Optional[str] = None
|
||||
report_uid: Optional[str] = None
|
||||
action: Optional[str] = None
|
||||
actor_name: Optional[str] = None
|
||||
reason: Optional[str] = None
|
||||
notes: Optional[str] = None
|
||||
expires_at: Optional[str] = None
|
||||
created_at: Optional[str] = None
|
||||
|
||||
|
||||
class SlaOut(_Out):
|
||||
sla_hours: Optional[int] = None
|
||||
oldest_open_hours: Optional[float] = None
|
||||
oldest_open_uid: Optional[str] = None
|
||||
breached: Optional[int] = None
|
||||
within_sla: Optional[bool] = None
|
||||
|
||||
|
||||
class AdminModerationOut(_Out):
|
||||
reports: list[ReportOut] = []
|
||||
pagination: Optional[Any] = None
|
||||
status: Optional[str] = None
|
||||
statuses: list[Any] = []
|
||||
counts: dict = {}
|
||||
sla: Optional[SlaOut] = None
|
||||
admin_section: Optional[str] = None
|
||||
|
||||
|
||||
class AdminReportOut(_Out):
|
||||
report: Optional[ReportOut] = None
|
||||
actions: list[ModerationActionOut] = []
|
||||
history: list[ModerationActionOut] = []
|
||||
available_actions: list[str] = []
|
||||
can_remove: Optional[bool] = None
|
||||
subject: Optional[AdminUserOut] = None
|
||||
sla: Optional[SlaOut] = None
|
||||
admin_section: Optional[str] = None
|
||||
|
||||
|
||||
class MaturityOut(_Out):
|
||||
target_type: Optional[str] = None
|
||||
target_uid: Optional[str] = None
|
||||
level: Optional[str] = None
|
||||
source: Optional[str] = None
|
||||
|
||||
|
||||
class ConsentOut(_Out):
|
||||
kind: Optional[str] = None
|
||||
label: Optional[str] = None
|
||||
state: Optional[str] = None
|
||||
version: Optional[str] = None
|
||||
granted_at: Optional[str] = None
|
||||
withdrawn_at: Optional[str] = None
|
||||
|
||||
|
||||
class ConsentListOut(_Out):
|
||||
consents: list[ConsentOut] = []
|
||||
|
||||
|
||||
class AcceptTermsOut(_Out):
|
||||
terms_version: Optional[str] = None
|
||||
accepted_version: Optional[str] = None
|
||||
|
||||
|
||||
class AccountDeletionOut(_Out):
|
||||
username: Optional[str] = None
|
||||
grace_hours: Optional[int] = None
|
||||
retained: list[str] = []
|
||||
removed: list[str] = []
|
||||
|
||||
|
||||
class WorkspaceIndexItemOut(_Out):
|
||||
uid: Optional[str] = None
|
||||
name: Optional[str] = None
|
||||
slug: Optional[str] = None
|
||||
owner_uid: Optional[str] = None
|
||||
url: Optional[str] = None
|
||||
description: Optional[str] = None
|
||||
owner: Optional[str] = None
|
||||
maturity: Optional[str] = None
|
||||
project_url: Optional[str] = None
|
||||
|
||||
|
||||
class WorkspaceIndexOut(_Out):
|
||||
workspaces: list[WorkspaceIndexItemOut] = []
|
||||
pagination: Optional[Any] = None
|
||||
total: Optional[int] = None
|
||||
@ -82,6 +82,15 @@ class ProfileOut(_Out):
|
||||
awards_count: int = 0
|
||||
prominent_award: Optional[AwardOut] = None
|
||||
can_give_award: bool = False
|
||||
consents: list[Any] = []
|
||||
mature_opt_in: bool = False
|
||||
age_band: Optional[str] = None
|
||||
terms_version: Optional[str] = None
|
||||
terms_accepted_at: Optional[str] = None
|
||||
current_terms_version: Optional[str] = None
|
||||
suspended_until: Optional[str] = None
|
||||
suspension_reason: Optional[str] = None
|
||||
deletion_grace_hours: Optional[int] = None
|
||||
|
||||
|
||||
class TelegramPairOut(_Out):
|
||||
|
||||
@ -82,6 +82,7 @@ class QuizOut(_Out):
|
||||
|
||||
|
||||
class QuizDetailOut(_Out):
|
||||
maturity: Optional[str] = None
|
||||
quiz: QuizOut = QuizOut()
|
||||
questions: list[QuizQuestionOut] = []
|
||||
comments: list[CommentItemOut] = []
|
||||
|
||||
@ -15,6 +15,15 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
SITE_NAME = "DevPlace"
|
||||
SITEMAP_URL_LIMIT = 5000
|
||||
|
||||
LEGAL_DOC_SLUGS = (
|
||||
"terms",
|
||||
"community-guidelines",
|
||||
"privacy",
|
||||
"content-moderation",
|
||||
"intellectual-property",
|
||||
"contact",
|
||||
)
|
||||
SITEMAP_TTL = int(os.environ.get("DEVPLACE_SITEMAP_TTL", "3600"))
|
||||
_sitemap_cache = {}
|
||||
|
||||
@ -424,6 +433,18 @@ def _build_sitemap(base_url):
|
||||
urlset.append(url_element(f"{base_url}/tools", changefreq="monthly", priority="0.5"))
|
||||
urlset.append(url_element(f"{base_url}/tools/seo", changefreq="monthly", priority="0.5"))
|
||||
urlset.append(url_element(f"{base_url}/tools/deepsearch", changefreq="monthly", priority="0.5"))
|
||||
urlset.append(
|
||||
url_element(f"{base_url}/workspaces/index", changefreq="daily", priority="0.6")
|
||||
)
|
||||
urlset.append(
|
||||
url_element(f"{base_url}/reports/reasons", changefreq="monthly", priority="0.4")
|
||||
)
|
||||
for slug in LEGAL_DOC_SLUGS:
|
||||
urlset.append(
|
||||
url_element(
|
||||
f"{base_url}/docs/{slug}.html", changefreq="monthly", priority="0.5"
|
||||
)
|
||||
)
|
||||
|
||||
if "posts" in db.tables:
|
||||
posts = _collect(
|
||||
@ -533,9 +554,13 @@ def _build_sitemap(base_url):
|
||||
try:
|
||||
from devplacepy.routers.docs.pages import DOCS_PAGES
|
||||
|
||||
listed = set(LEGAL_DOC_SLUGS)
|
||||
for page in DOCS_PAGES:
|
||||
if page.get("admin") or page.get("kind") == "live":
|
||||
continue
|
||||
if page["slug"] in listed:
|
||||
continue
|
||||
listed.add(page["slug"])
|
||||
urlset.append(
|
||||
url_element(
|
||||
f"{base_url}/docs/{page['slug']}.html",
|
||||
|
||||
@ -114,6 +114,14 @@ devplace devii reset-quota --guests # Reset every guest quota
|
||||
devplace devii reset-quota --all # Reset every quota (users and guests)
|
||||
```
|
||||
|
||||
## Moderation housekeeping (`services/moderation/service.py`)
|
||||
|
||||
`ModerationService` is a lock-owner `BaseService` (default-enabled, hourly, floor 300s) with two jobs: it purges accounts whose deletion grace window has closed (`deletion.purge_due`, the same code path as `devplace accounts prune`), and it reports the moderation queue's service-level snapshot - logging when a report is past the published response window and exposing the queue counts, the oldest open age and the pending-purge count as `collect_metrics` stat cards. It owns no request-path work; the queue itself is entirely synchronous. Full subsystem detail in `devplacepy/services/moderation/CLAUDE.md`.
|
||||
|
||||
## Acceptance convergence (`services/acceptance/service.py`)
|
||||
|
||||
`AcceptanceService` is a lock-owner `BaseService` (**opt-in**, `default_enabled = False`, five minutes, floor 60s) that grants every policy agreement to every account which has not declined it, so a production-identical instance used for manual testing never interrupts with an acceptance dialog. It is invisible to the rest of the application by contract: one registration line in `main.py` is the only import anywhere, and there is no route, schema, template, Devii tool or environment flag. The decline register is the consent ledger itself - the service only ever writes `granted`, so any `withdrawn` row was written by a human and that pair is never touched again. Three gates stand between a fresh install and a single written row (service disabled, every agreement disabled, dry run on). Full subsystem detail in `devplacepy/services/acceptance/CLAUDE.md`; the design record is `accept.md` at the repository root.
|
||||
|
||||
## Multi-worker concurrency (preferred rules)
|
||||
|
||||
`uvicorn --workers N` = N independent processes sharing only the filesystem and SQLite DB. Module-global caches/counters are per-process, so a local `clear()` is invisible to siblings. Full reference: admin docs `Production -> Multi-worker and concurrency` (`templates/docs/production-concurrency.html`). Enforce these:
|
||||
@ -177,6 +185,8 @@ Online status is a single **`users.last_seen`** UTC-ISO column (ensured in `data
|
||||
|
||||
**Write path (all workers):** `main.py`'s `track_presence` HTTP middleware resolves the cached current user on every non-`/static`, non-`/avatar` request and calls `presence.touch(uid)`. `touch` keeps a per-worker in-memory `_last_write: dict[uid -> monotonic]` and writes `users.last_seen` (via `database.set_last_seen`) only when the last write for that uid is older than `config.PRESENCE_WRITE_SECONDS` (= `PRESENCE_TIMEOUT_SECONDS // 2`). So continuous browsing is a dict lookup; a write happens at most ~once per half-window per active user per worker, and the row is updated in place (zero growth). It deliberately does **not** call `clear_user_cache` (that would defeat the 300s auth cache; the stale cached self-row is irrelevant since presence of *other* users is always read from a fresh row).
|
||||
|
||||
**Consent gate (write path).** `touch` checks `presence.recording_allowed(uid)` (the `activity_recording` consent) **after** the per-worker throttle, so the consent read costs at most one query per half-window per active user rather than one per request. A user who withdraws the consent simply stops being written and appears offline. Never move the check above the throttle.
|
||||
|
||||
**Read path (any worker):** `presence.is_online(user_row)` = `now - last_seen < PRESENCE_TIMEOUT_SECONDS` (env `DEVPLACE_PRESENCE_TIMEOUT_SECONDS`, default 60). Profile (`routers/profile/index.py` -> `profile_online`) and messages (`routers/messages.py` seed) read `last_seen` off the user row they already loaded - no extra query. Exposed as the Jinja global `is_online(user)` (`templating.py`), on `UserOut.last_seen` and `ProfileOut.profile_online`. This is the **only** cross-worker-correct approach here because pub/sub is in-process.
|
||||
|
||||
**Live path (lock owner only), ONE set on ONE topic, change-only + hysteresis:** `PresenceRelayService` (`services/presence_relay.py`, `BaseService`, default-enabled, 2s tick, registered in `main.py`) is a sibling of `NotificationRelayService`/`LiveViewRelayService` and is **the single source of truth for live online status**. Each tick, only while the roster topic has subscribers, it reads the online population in ONE indexed query (`presence.online_candidates()`, capped at `config.PRESENCE_TRACK_LIMIT`, env `DEVPLACE_PRESENCE_TRACK_LIMIT`, default 500) and recomputes ONE set `self._online` with **hysteresis** via `presence.stays_online(elapsed, was_online)`: a user becomes online at `PRESENCE_TIMEOUT_SECONDS` but only drops after `+ PRESENCE_ONLINE_MARGIN_SECONDS` (env `DEVPLACE_PRESENCE_ONLINE_MARGIN_SECONDS`, default 20) - **quick to go online, slow (grace margin) to go offline** - which kills boundary flicker. It publishes that set on the ONE shared topic `public.presence.roster` (`roster_payload`: `{count, online: [uid...], users: [display rows]}`) **only when the set of online uids changes** (a `frozenset` compare, so reordering never republishes), never on a fixed interval, so an idle site emits nothing. `online` is the authority for EVERY avatar dot; `users` is the same set trimmed to `PRESENCE_ONLINE_LIMIT` for the feed's avatar panel, and `count` matches it. **There are no per-user `public.presence.{uid}` topics** - they were removed precisely because their candidate population differed from the roster's, so dots and roster could disagree (the /messages-vs-feed bug). One set, one topic, one frame. `public.presence.roster` is subscribable by any logged-in user (`pubsub/policy.py` allows `public.*`); guests keep the server-rendered initial state.
|
||||
|
||||
65
devplacepy/services/acceptance/CLAUDE.md
Normal file
65
devplacepy/services/acceptance/CLAUDE.md
Normal file
@ -0,0 +1,65 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file documents the acceptance convergence subsystem (`devplacepy/services/acceptance/`). Claude Code auto-loads it whenever a file under this directory is read or edited. The full design record is [`accept.md`](../../../accept.md) at the repository root.
|
||||
|
||||
## Why this subsystem exists
|
||||
|
||||
An operator running a production-identical instance for extended manual testing is otherwise taxed forever by the platform's own safety controls: five consents, a versioned terms gate on every mutating request, and every account predating the trust-and-safety commit reading `terms_version = NULL` because `init_db` deliberately never backfills it. This service converges each account onto the acceptance state a real population would have produced itself, so the instance stays production byte for byte while nobody has to click the same dialog again.
|
||||
|
||||
It is off by default and it is never appropriate on a real production host.
|
||||
|
||||
## The two load-bearing ideas
|
||||
|
||||
**The application must not know.** There is no request-path branch, no schema, no route, no template, no Jinja global, no Devii tool and no environment flag. The only import of this package anywhere is the one registration line in `main.py`, and `tests/unit/services/acceptance/isolation.py` fails the suite if a second one appears. An environment flag would be exactly the knowledge the application is not allowed to have, which is why there is none.
|
||||
|
||||
**The decline register needs no storage.** `user_consents` is append-only in effect, so the latest live row for a `(user, kind)` pair already is the register. The service only ever writes `granted`; it follows that any `withdrawn` row in the ledger was written by a human, and the service never touches that pair again. No provenance column, no marker, no flag, and nothing for the application to observe.
|
||||
|
||||
## Module map
|
||||
|
||||
| File | Owns |
|
||||
|---|---|
|
||||
| `agreements.py` | `Agreement`, `AGREEMENTS`, `setting_key`, `label_for`, `agreement_for` |
|
||||
| `pending.py` | `latest_consent`, `not_withdrawn`, `satisfied_clause`, `live_account_clauses`, `current_version`, `pending` |
|
||||
| `grant.py` | `converge_user` plus the two private claim shapes and the audit call |
|
||||
| `service.py` | `AcceptanceService`: config fields built from the registry, `run_once`, `collect_metrics` |
|
||||
|
||||
`pending.py` and `grant.py` import only `devplacepy.database` and `sqlalchemy` at module top; `generate_uid` and the audit recorder are imported lazily inside the functions that use them, mirroring `services/moderation/deletion.py`.
|
||||
|
||||
## The registry is the completeness guarantee
|
||||
|
||||
`AGREEMENTS` annotates `database.CONSENT_KINDS` with two facts: which `site_settings` key holds the policy version, and which `users` column the application's own gate reads. `terms` is the only agreement with a gate column, because `needs_acceptance` reads `users.terms_version` and not the ledger; `privacy` is versioned but ledger-only, matching `VERSION_KEYS` in `routers/profile/consent.py`.
|
||||
|
||||
A unit test asserts `{a.kind for a in AGREEMENTS} == set(CONSENT_KINDS)`. **A sixth consent fails the suite until it is classified here**, and it then appears in the admin form with no edit to the service, because the per-agreement config fields are built from the registry rather than written out by hand.
|
||||
|
||||
## Rules that must not regress
|
||||
|
||||
- **Satisfaction is the gate's own expression, never a proxy.** `pending` compares exactly what the application compares: `users.terms_version` against `get_setting("terms_version", "1") or "1"` for `terms`, the ledger row's `version` for `privacy`, the latest state for the other three. The `or "1"` is load-bearing: an admin settings save can write `terms_version = ""`, and a bare `get_setting` would make every account pending forever.
|
||||
- **Order by `created_at DESC, id DESC`, never one of the two.** That pair is what `database.consent_state` selects, so it is the expression the gate evaluates. `consent_view` on the privacy tab orders by `created_at` alone; that is the display path, not a gate. Never introduce a third ordering.
|
||||
- **Build the live-account clauses with `has_column`.** `init_db` ensures `terms_version`, `terms_accepted_at` and `deletion_requested_at` on `users`, but **not `is_active`** - that column is created implicitly the first time a suspension or a deletion writes it, so a hardcoded reference raises `no such column` on an instance where nobody was ever suspended. An absent column means no account can be in that state, so omitting the clause is the correct answer.
|
||||
- **Every write is one conditional statement decided on the driver's real `rowcount`**, via `db.executable.execute(text(...), params).rowcount` inside `with db:`, exactly like `deletion.claim_deletion`. Sixteen real processes racing one account produce exactly one ledger row and one audit row.
|
||||
- **Never write `updated_at` on `users`.** That is why `database.atomic.conditional_update_row` cannot be reused here: it appends `updated_at` unconditionally, the table has no such column, and creating one would make the service's rows distinguishable from the route's.
|
||||
- **The ledger insert must stay byte-compatible with `set_consent`**, including `withdrawn_at = ''` rather than `NULL` on the unused side. A unit test compares the two field by field. It is not `set_consent` itself only because `set_consent` cannot express a precondition or join a caller's transaction.
|
||||
- **One cache bump per run, not one per account.** `clear_user_cache` propagates a global `auth` version bump that makes every worker drop its whole user cache; `run_once` bumps once at the end, and only when a gate column actually changed.
|
||||
- **The audit row is deliberate.** It uses the existing `terms.accept` and `consent.grant` keys with `actor_kind="service"`, `actor_username="acceptance"`. The audit log is the operator's record and no code path reads it, so it costs nothing in invisibility and is the only trace distinguishing a converged acceptance from a human one.
|
||||
|
||||
## What is deliberately not an agreement
|
||||
|
||||
| Excluded | Why |
|
||||
|---|---|
|
||||
| `users.age_band` | A declaration of fact, not an agreement. Fabricating a declared age would silently unlock `restricted` content for an account that declared 13-15. Sign the test account up with an adult date of birth instead. |
|
||||
| `users.mature_opt_in` | A preference gated by the age band, with no ledger row and therefore no decline register. The site setting `moderation_mature_default_hidden` already turns the interstitial off instance-wide. |
|
||||
| Guest consents | Nothing writes a consent row with `owner_kind = "guest"`, and the gateway resolves a guest to owner kind `anonymous`, which `consent_denied` exempts by design. |
|
||||
| Preferences (`interactions_enabled`, notifications, customization) | None of them blocks anything. |
|
||||
|
||||
## The terms asymmetry, and the one operator step
|
||||
|
||||
`POST /auth/accept-terms` grants **two** agreements: `terms` and `privacy`. The service keeps them independent on purpose, because the per-agreement switches exist for edge-case testing; enabling both reproduces the human path exactly, enabling one is a deliberate divergence.
|
||||
|
||||
Withdrawing the `terms` consent does not clear `users.terms_version`, so a tester who wants to exercise the gate withdraws their own `terms` consent and then has an administrator bump `terms_version` at `/admin/settings`. Every other account converges within one interval; the tester stays gated indefinitely.
|
||||
|
||||
## Rules for extending this
|
||||
|
||||
- A new consent kind: classify it in `AGREEMENTS` (the test forces this), and nothing else.
|
||||
- Never add a route, a schema, a template, a Devii tool or a `docs_api` entry. The surface is the generic services admin, exactly as for `NotificationRelayService` and `AuditService`.
|
||||
- Never add a second decline register, a second ordering, or an environment check.
|
||||
- Never write a persisted test that enables the service and asserts convergence: the suite is serial against one seeded database, and granting `terms` to every account would poison the moderation tests that assert the gate refuses. Convergence is covered by unit tests calling `converge_user` directly.
|
||||
1
devplacepy/services/acceptance/__init__.py
Normal file
1
devplacepy/services/acceptance/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
37
devplacepy/services/acceptance/agreements.py
Normal file
37
devplacepy/services/acceptance/agreements.py
Normal file
@ -0,0 +1,37 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from devplacepy.database import CONSENT_KINDS
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Agreement:
|
||||
kind: str
|
||||
version_setting: str
|
||||
gate_column: str
|
||||
stamp_column: str
|
||||
|
||||
|
||||
AGREEMENTS: tuple[Agreement, ...] = (
|
||||
Agreement("terms", "terms_version", "terms_version", "terms_accepted_at"),
|
||||
Agreement("privacy", "privacy_version", "", ""),
|
||||
Agreement("ai_third_party", "", "", ""),
|
||||
Agreement("activity_recording", "", "", ""),
|
||||
Agreement("container_credentials", "", "", ""),
|
||||
)
|
||||
|
||||
|
||||
def setting_key(kind: str) -> str:
|
||||
return f"acceptance_grant_{kind}"
|
||||
|
||||
|
||||
def label_for(kind: str) -> str:
|
||||
return CONSENT_KINDS.get(kind, kind)
|
||||
|
||||
|
||||
def agreement_for(kind: str) -> Agreement | None:
|
||||
for agreement in AGREEMENTS:
|
||||
if agreement.kind == kind:
|
||||
return agreement
|
||||
return None
|
||||
130
devplacepy/services/acceptance/grant.py
Normal file
130
devplacepy/services/acceptance/grant.py
Normal file
@ -0,0 +1,130 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from sqlalchemy import text
|
||||
|
||||
from devplacepy.database import CONSENTS_TABLE, _now_iso, db, get_table
|
||||
|
||||
from .agreements import Agreement
|
||||
from .pending import has_gate_column, not_withdrawn, satisfied_clause
|
||||
|
||||
CONSENT_COLUMNS = (
|
||||
"uid",
|
||||
"owner_kind",
|
||||
"owner_id",
|
||||
"kind",
|
||||
"version",
|
||||
"state",
|
||||
"granted_at",
|
||||
"withdrawn_at",
|
||||
"created_at",
|
||||
"deleted_at",
|
||||
"deleted_by",
|
||||
)
|
||||
|
||||
|
||||
def _gate_set_clause(agreement: Agreement) -> str:
|
||||
users = get_table("users")
|
||||
assignments = [f"{agreement.gate_column} = :version"]
|
||||
if agreement.stamp_column and users.has_column(agreement.stamp_column):
|
||||
assignments.append(f"{agreement.stamp_column} = :now")
|
||||
return ", ".join(assignments)
|
||||
|
||||
|
||||
def _insert_consent_sql(precondition: str) -> str:
|
||||
columns = ", ".join(CONSENT_COLUMNS)
|
||||
sql = (
|
||||
f"INSERT INTO {CONSENTS_TABLE} ({columns}) "
|
||||
"SELECT :row_uid, 'user', :uid, :kind, :version, 'granted', "
|
||||
":now, '', :now, NULL, NULL"
|
||||
)
|
||||
if precondition:
|
||||
return f"{sql} WHERE {precondition}"
|
||||
return sql
|
||||
|
||||
|
||||
def _consent_params(agreement: Agreement, uid: str, version: str, now: str) -> dict:
|
||||
from devplacepy.utils import generate_uid
|
||||
|
||||
return {
|
||||
"row_uid": generate_uid(),
|
||||
"uid": uid,
|
||||
"kind": agreement.kind,
|
||||
"version": version,
|
||||
"now": now,
|
||||
}
|
||||
|
||||
|
||||
def _claim_gate_column(agreement: Agreement, uid: str, version: str, now: str) -> bool:
|
||||
claim = (
|
||||
f"UPDATE users SET {_gate_set_clause(agreement)} "
|
||||
"WHERE uid = :uid "
|
||||
f"AND COALESCE({agreement.gate_column}, '') != :version "
|
||||
f"AND {not_withdrawn('users.uid')}"
|
||||
)
|
||||
params = _consent_params(agreement, uid, version, now)
|
||||
with db:
|
||||
claimed = db.executable.execute(text(claim), params).rowcount
|
||||
if claimed != 1:
|
||||
return False
|
||||
db.executable.execute(text(_insert_consent_sql("")), params)
|
||||
return True
|
||||
|
||||
|
||||
def _claim_ledger(agreement: Agreement, uid: str, version: str, now: str) -> bool:
|
||||
precondition = (
|
||||
f"{not_withdrawn(':uid')} AND {satisfied_clause(agreement, ':uid', '')}"
|
||||
)
|
||||
params = _consent_params(agreement, uid, version, now)
|
||||
with db:
|
||||
written = db.executable.execute(
|
||||
text(_insert_consent_sql(precondition)), params
|
||||
).rowcount
|
||||
return written == 1
|
||||
|
||||
|
||||
def _record(agreement: Agreement, user: dict, version: str) -> None:
|
||||
from devplacepy.services.audit import record as audit
|
||||
|
||||
uid = user["uid"]
|
||||
username = user.get("username") or ""
|
||||
links = [audit.target("user", uid, username)]
|
||||
if agreement.kind == "terms":
|
||||
audit.record_system(
|
||||
"terms.accept",
|
||||
actor_kind="service",
|
||||
actor_username="acceptance",
|
||||
target_type="user",
|
||||
target_uid=uid,
|
||||
target_label=username,
|
||||
new_value=version,
|
||||
summary=f"{username} accepted terms version {version}",
|
||||
links=links,
|
||||
)
|
||||
return
|
||||
audit.record_system(
|
||||
"consent.grant",
|
||||
actor_kind="service",
|
||||
actor_username="acceptance",
|
||||
target_type="user",
|
||||
target_uid=uid,
|
||||
target_label=username,
|
||||
new_value="granted",
|
||||
metadata={"kind": agreement.kind},
|
||||
summary=f"granted {agreement.kind} consent for {username}",
|
||||
links=links,
|
||||
)
|
||||
|
||||
|
||||
def converge_user(agreement: Agreement, user: dict, version: str) -> bool:
|
||||
uid = user.get("uid")
|
||||
if not uid:
|
||||
return False
|
||||
now = _now_iso()
|
||||
if has_gate_column(agreement):
|
||||
won = _claim_gate_column(agreement, uid, version, now)
|
||||
else:
|
||||
won = _claim_ledger(agreement, uid, version, now)
|
||||
if not won:
|
||||
return False
|
||||
_record(agreement, user, version)
|
||||
return True
|
||||
78
devplacepy/services/acceptance/pending.py
Normal file
78
devplacepy/services/acceptance/pending.py
Normal file
@ -0,0 +1,78 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
from devplacepy.database import CONSENTS_TABLE, db, get_setting, get_table
|
||||
|
||||
from .agreements import Agreement
|
||||
|
||||
DEFAULT_VERSION = "1"
|
||||
|
||||
LIVE_ACCOUNT_CLAUSES: tuple[tuple[str, str], ...] = (
|
||||
("deletion_requested_at", "COALESCE(u.deletion_requested_at, '') = ''"),
|
||||
("is_active", "COALESCE(u.is_active, 1) != 0"),
|
||||
("deleted_at", "u.deleted_at IS NULL"),
|
||||
)
|
||||
|
||||
|
||||
def latest_consent(column: str, user_ref: str) -> str:
|
||||
return (
|
||||
f"COALESCE((SELECT c.{column} FROM {CONSENTS_TABLE} c "
|
||||
f"WHERE c.owner_kind = 'user' AND c.owner_id = {user_ref} "
|
||||
"AND c.kind = :kind AND c.deleted_at IS NULL "
|
||||
"ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '')"
|
||||
)
|
||||
|
||||
|
||||
def not_withdrawn(user_ref: str) -> str:
|
||||
return f"{latest_consent('state', user_ref)} != 'withdrawn'"
|
||||
|
||||
|
||||
def has_gate_column(agreement: Agreement) -> bool:
|
||||
if not agreement.gate_column:
|
||||
return False
|
||||
if "users" not in db.tables:
|
||||
return False
|
||||
return get_table("users").has_column(agreement.gate_column)
|
||||
|
||||
|
||||
def satisfied_clause(agreement: Agreement, user_ref: str, column_prefix: str) -> str:
|
||||
if has_gate_column(agreement):
|
||||
return f"COALESCE({column_prefix}{agreement.gate_column}, '') != :version"
|
||||
if agreement.version_setting:
|
||||
return f"{latest_consent('version', user_ref)} != :version"
|
||||
return f"{latest_consent('state', user_ref)} != 'granted'"
|
||||
|
||||
|
||||
def live_account_clauses() -> list[str]:
|
||||
users = get_table("users")
|
||||
return [
|
||||
clause for column, clause in LIVE_ACCOUNT_CLAUSES if users.has_column(column)
|
||||
]
|
||||
|
||||
|
||||
def current_version(agreement: Agreement) -> str:
|
||||
if not agreement.version_setting:
|
||||
return DEFAULT_VERSION
|
||||
return get_setting(agreement.version_setting, DEFAULT_VERSION) or DEFAULT_VERSION
|
||||
|
||||
|
||||
def pending(agreement: Agreement, limit: int) -> list[dict]:
|
||||
if "users" not in db.tables or CONSENTS_TABLE not in db.tables:
|
||||
return []
|
||||
bound = max(1, int(limit))
|
||||
clauses = [
|
||||
*live_account_clauses(),
|
||||
not_withdrawn("u.uid"),
|
||||
satisfied_clause(agreement, "u.uid", "u."),
|
||||
]
|
||||
sql = (
|
||||
"SELECT u.uid, u.username FROM users u "
|
||||
f"WHERE {' AND '.join(clauses)} "
|
||||
"ORDER BY u.id LIMIT :limit"
|
||||
)
|
||||
rows = db.query(
|
||||
sql,
|
||||
kind=agreement.kind,
|
||||
version=current_version(agreement),
|
||||
limit=bound,
|
||||
)
|
||||
return [dict(row) for row in rows]
|
||||
168
devplacepy/services/acceptance/service.py
Normal file
168
devplacepy/services/acceptance/service.py
Normal file
@ -0,0 +1,168 @@
|
||||
# retoor <retoor@molodetz.nl>
|
||||
|
||||
import logging
|
||||
|
||||
from devplacepy.database import CONSENT_KINDS, bump_cache_version
|
||||
from devplacepy.services.acceptance.agreements import (
|
||||
AGREEMENTS,
|
||||
label_for,
|
||||
setting_key,
|
||||
)
|
||||
from devplacepy.services.acceptance.grant import converge_user
|
||||
from devplacepy.services.acceptance.pending import current_version, pending
|
||||
from devplacepy.services.base import BaseService, ConfigField
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DRY_RUN_KEY = "acceptance_dry_run"
|
||||
BATCH_SIZE_KEY = "acceptance_batch_size"
|
||||
DEFAULT_BATCH_SIZE = 200
|
||||
MAX_BATCH_SIZE = 5000
|
||||
SAMPLE_NAMES = 20
|
||||
|
||||
|
||||
def sample_names(candidates: list[dict]) -> str:
|
||||
names = [row.get("username") or row.get("uid") or "" for row in candidates]
|
||||
shown = ", ".join(names[:SAMPLE_NAMES])
|
||||
remaining = len(names) - SAMPLE_NAMES
|
||||
if remaining > 0:
|
||||
return f"{shown} and {remaining} more"
|
||||
return shown
|
||||
|
||||
|
||||
class AcceptanceService(BaseService):
|
||||
title = "Acceptance convergence"
|
||||
description = (
|
||||
"Grants every policy agreement to every account that has not declined it, "
|
||||
"so a production-identical test instance never interrupts manual testing "
|
||||
"with an acceptance dialog. Off by default and never appropriate on a real "
|
||||
"production host."
|
||||
)
|
||||
details = (
|
||||
"An account that withdrew a consent is never granted it again, with no "
|
||||
"further action: the latest row in the consent ledger is the decline "
|
||||
"register. The service only ever grants, so any withdrawal in the ledger "
|
||||
"was written by a human. Withdraw a consent from the profile privacy tab "
|
||||
"to keep an account permanently outside the convergence."
|
||||
)
|
||||
default_enabled = False
|
||||
min_interval = 60
|
||||
METRICS_SECONDS = 60
|
||||
config_fields = [
|
||||
ConfigField(
|
||||
DRY_RUN_KEY,
|
||||
"Dry run",
|
||||
type="bool",
|
||||
default=True,
|
||||
help=(
|
||||
"Log which accounts would be converged and write nothing. Switch "
|
||||
"off only after the log lists what you expect."
|
||||
),
|
||||
group="Safety",
|
||||
),
|
||||
ConfigField(
|
||||
BATCH_SIZE_KEY,
|
||||
"Accounts per agreement per run",
|
||||
type="int",
|
||||
default=DEFAULT_BATCH_SIZE,
|
||||
minimum=1,
|
||||
maximum=MAX_BATCH_SIZE,
|
||||
help="Upper bound on one sweep. The remainder converges on the next run.",
|
||||
group="Safety",
|
||||
),
|
||||
*[
|
||||
ConfigField(
|
||||
setting_key(agreement.kind),
|
||||
label_for(agreement.kind),
|
||||
type="bool",
|
||||
default=False,
|
||||
help=f"Grant the {agreement.kind} agreement to every account that has not declined it.",
|
||||
group="Agreements",
|
||||
)
|
||||
for agreement in AGREEMENTS
|
||||
],
|
||||
]
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__("acceptance", interval_seconds=300)
|
||||
self._converged = {agreement.kind: 0 for agreement in AGREEMENTS}
|
||||
|
||||
async def run_once(self) -> None:
|
||||
config = self.get_config()
|
||||
dry = config[DRY_RUN_KEY]
|
||||
limit = config[BATCH_SIZE_KEY]
|
||||
gate_changed = False
|
||||
for agreement in AGREEMENTS:
|
||||
if not config[setting_key(agreement.kind)]:
|
||||
continue
|
||||
version = current_version(agreement)
|
||||
candidates = pending(agreement, limit)
|
||||
if not candidates:
|
||||
continue
|
||||
if dry:
|
||||
self.log(
|
||||
f"[dry run] {agreement.kind}: {len(candidates)} account(s) would be "
|
||||
f"converged to version {version}: {sample_names(candidates)}"
|
||||
)
|
||||
continue
|
||||
granted = 0
|
||||
for user in candidates:
|
||||
if converge_user(agreement, user, version):
|
||||
granted += 1
|
||||
gate_changed = gate_changed or bool(agreement.gate_column)
|
||||
self._converged[agreement.kind] += granted
|
||||
self.log(
|
||||
f"{agreement.kind}: converged {granted} of {len(candidates)} pending "
|
||||
f"at version {version}"
|
||||
)
|
||||
if len(candidates) == limit:
|
||||
self.log(
|
||||
f"{agreement.kind}: batch cap of {limit} reached, more remain "
|
||||
"for the next run"
|
||||
)
|
||||
if gate_changed:
|
||||
bump_cache_version("auth")
|
||||
|
||||
def collect_metrics(self) -> dict:
|
||||
config = self.get_config()
|
||||
limit = config[BATCH_SIZE_KEY]
|
||||
enabled = [
|
||||
agreement
|
||||
for agreement in AGREEMENTS
|
||||
if config[setting_key(agreement.kind)]
|
||||
]
|
||||
stats = [
|
||||
{"label": "Dry run", "value": 1 if config[DRY_RUN_KEY] else 0},
|
||||
{"label": "Agreements enabled", "value": len(enabled)},
|
||||
]
|
||||
rows = []
|
||||
for agreement in AGREEMENTS:
|
||||
is_enabled = config[setting_key(agreement.kind)]
|
||||
count = len(pending(agreement, limit)) if is_enabled else 0
|
||||
display = f"{count}+" if is_enabled and count == limit else str(count)
|
||||
if is_enabled:
|
||||
stats.append(
|
||||
{"label": f"{agreement.kind} pending", "value": display}
|
||||
)
|
||||
rows.append(
|
||||
[
|
||||
agreement.kind,
|
||||
CONSENT_KINDS.get(agreement.kind, agreement.kind)[:48],
|
||||
1 if is_enabled else 0,
|
||||
agreement.version_setting or "",
|
||||
display if is_enabled else "",
|
||||
self._converged[agreement.kind],
|
||||
]
|
||||
)
|
||||
table = {
|
||||
"columns": [
|
||||
"Agreement",
|
||||
"Policy",
|
||||
"Enabled",
|
||||
"Version setting",
|
||||
"Pending",
|
||||
"Converged",
|
||||
],
|
||||
"rows": rows,
|
||||
}
|
||||
return {"stats": stats, "table": table}
|
||||
@ -3,6 +3,12 @@
|
||||
CATEGORY_BY_PREFIX: dict[str, str] = {
|
||||
"auth": "auth",
|
||||
"profile": "account",
|
||||
"account": "account",
|
||||
"consent": "account",
|
||||
"terms": "account",
|
||||
"report": "moderation",
|
||||
"moderation": "moderation",
|
||||
"filter": "moderation",
|
||||
"follow": "social",
|
||||
"award": "social",
|
||||
"relation": "social",
|
||||
|
||||
@ -10,6 +10,8 @@ from devplacepy.services.bot.handles import make_handle
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SIGNUP_BIRTH_DATE = "1995-01-01"
|
||||
|
||||
|
||||
class BotAuthMixin:
|
||||
def _generate_handle(self) -> str:
|
||||
@ -70,6 +72,10 @@ class BotAuthMixin:
|
||||
await b.fill("#password", self.state.password)
|
||||
await b._idle(0.2, 0.5)
|
||||
await b.fill("#confirm_password", self.state.password)
|
||||
await b._idle(0.2, 0.5)
|
||||
await b.fill("#birth_date", SIGNUP_BIRTH_DATE)
|
||||
await b._idle(0.2, 0.5)
|
||||
await b.click("#accept_terms")
|
||||
await b._idle(0.5, 1.0)
|
||||
await b.click(".auth-submit")
|
||||
await asyncio.sleep(2)
|
||||
|
||||
@ -112,6 +112,136 @@ An instance's shell is NOT inline - it is a floating `<container-terminal>` (`st
|
||||
|
||||
**Minimize/Normalize.** Geometry presets (`_presetGeometry(w,h)`, smallest-usable / comfortable), exposed both as titlebar buttons (`data-win="minimize|normalize"`) and menu items on every window.
|
||||
|
||||
## Editor profile and branding (`workspace/editor.py`, `files/vscode/`)
|
||||
|
||||
The browser editor is a DevPlace product surface, not stock code-server. Three layers own it, and
|
||||
the split is the design: the deterministic part is host-side and unit-testable without Docker, the
|
||||
cosmetic part is an extension that fails soft.
|
||||
|
||||
| Layer | Owns | Fails how |
|
||||
|---|---|---|
|
||||
| **Host** `workspace/editor.py` | Resolving the profile, seeding `settings.json`, building the argv, setting container CPU and memory | Deterministic, unit-tested against a temp state dir, no container needed |
|
||||
| **Image** `ppy.Dockerfile` + `files/vscode/` | Branding assets, patched `product.json`, the bundled extension | Verified by the build smoke test; an image that cannot brand cannot build green |
|
||||
| **Extension** `files/vscode/devplace-workspace/` | Boot terminals, panel layout, status bar, walkthrough, `DevPlace:` commands | Each stage in its own try/catch to a `DevPlace` output channel. A failure costs the terminals, never the editor |
|
||||
|
||||
**One resolver, exactly like `quota.resolve`.** `editor.resolve(owner_uid, instance) -> EditorProfile`
|
||||
is the only place a `workspace_editor_*` setting is read. Order is user preference row, then site
|
||||
setting, then built-in default; the container size (`cpu_millicores`, `memory_mb`, `disk_quota_mb`)
|
||||
comes from `quota.resolve` so all four "sizes" live on one object. `editor.view()` adds
|
||||
`source_map()` so every surface can say where a value came from. Never read one of those settings at
|
||||
a call site.
|
||||
|
||||
**Inherit sentinels are explicit, never truthiness.** `workspace_editor_prefs` stores `""` / `0` for
|
||||
"inherit", but zoom level `0` is a real value, so its sentinel is `-99` (`INHERIT_ZOOM`), and
|
||||
`boot_shell` uses `-1` (`INHERIT_FLAG`) because `0` means off. `_inherits(key, row)` is the single
|
||||
predicate; a bare `if value:` here would silently ignore a member who wants zoom 0 or no shell.
|
||||
|
||||
**`merge_managed` is the contract that a member edit is never overwritten** and it is a pure
|
||||
function, which is why it is exhaustively unit-tested. DevPlace writes a key only when it is absent
|
||||
or still equal to the value DevPlace wrote last time, recorded in
|
||||
`{state}/data/User/.devplace-managed.json`. So raising a site default reaches everyone who never
|
||||
expressed a preference and nobody who did. Do not replace this with a plain merge or a full rewrite.
|
||||
|
||||
**Seeding runs at launch, in `run_spec_for`**, alongside `ensure_editor_password` - the one point
|
||||
every workspace launch passes through, so a workspace created before this feature is seeded on its
|
||||
next boot. `stamp_boot_marker` writes a fresh `instances.boot_marker` there too; it reaches the
|
||||
container as `DEVPLACE_CONTAINER_BOOT` and is what makes the extension's boot terminals idempotent
|
||||
across browser reloads. Because seeding happens at launch, a preference change applies on the **next
|
||||
start**: the workspace page compares the resolved profile against `{state}/devplace-editor.json`
|
||||
(`editor.restart_required`) and shows a restart banner rather than pretending it applied.
|
||||
|
||||
**Nothing DevPlace writes goes to `/app`.** `/app` round-trips into the member's project through
|
||||
`sync_dir_bidirectional`, so a `.vscode/tasks.json` there would land in their repository. Every
|
||||
artefact goes under `WORKSPACE_STATE_DIR`, which is a separate bind mount and never synced. This is
|
||||
why boot terminals are an extension rather than a folder-open task.
|
||||
|
||||
**The agent's own working files are in `SYNC_SKIP_NAMES` for the same reason.** `dpc` writes `.dpc/`
|
||||
and `dpc.log` into its working directory, which is `/app`. That was harmless while `dpc` only ran
|
||||
when a member typed it; now that it starts on every workspace boot, those artefacts would be
|
||||
imported into every project on the next sync. `project_files.SYNC_SKIP_NAMES` therefore carries
|
||||
`.dpc`, `dpc.log` and `.devplace` (the tunnel manifest directory, which was already being written
|
||||
and already leaking) alongside the `.devplace_boot.*` entries. Any future in-container tool that
|
||||
writes state next to the member's code needs the same entry.
|
||||
|
||||
**The boot marker degrades to once-per-extension-host, never to "always".** `BootTerminals` is
|
||||
guarded by `DEVPLACE_CONTAINER_BOOT`, but an instance created before that column existed injects an
|
||||
empty value. The guard used to treat an empty marker as "not booted yet" and opened a fresh pair of
|
||||
terminals on **every browser reload** - caught by driving one container with three consecutive
|
||||
Playwright sessions and finding six terminal tabs. The fallback is now `host-${process.pid}` of the
|
||||
extension host, which survives a browser reload and changes when the container restarts, which is
|
||||
exactly the intended semantic.
|
||||
|
||||
**Trust is disabled at three layers** and gated by one kill switch, `workspace_editor_trust_all`
|
||||
(default on): the `--disable-workspace-trust` flag, the seeded `security.workspace.trust.*` settings,
|
||||
and the extension's `contributes.configurationDefaults`. The third is belt and braces only -
|
||||
`security.workspace.trust.enabled` is application-scoped and VS Code restricts which scopes an
|
||||
extension may re-default - so never let it be the only layer. Turning the switch off restores
|
||||
Restricted Mode with no code change and no image rebuild. It also enables
|
||||
`task.allowAutomaticTasks`, so a project's own `runOn: folderOpen` task will run; that consequence is
|
||||
documented to members on `/docs/workspace-editor.html` and must stay documented.
|
||||
|
||||
**Panel height is a preset, not a pixel value, and that is a hard constraint.** VS Code stores part
|
||||
sizes in the workbench grid inside `state.vscdb`, an undocumented and version-unstable internal
|
||||
SQLite database. Writing it from the host is rejected. The extension drives
|
||||
`workbench.action.toggleMaximizedPanel` / `increaseViewSize` instead, so the four presets are named
|
||||
honestly as presets in the UI. Do not "improve" this by writing `state.vscdb`.
|
||||
|
||||
**The extension is a built-in, copied to
|
||||
`/usr/local/lib/code-server/lib/vscode/extensions/devplace-workspace`.** Built-ins are always
|
||||
enabled, cannot be uninstalled, need no install step and survive workspace recreation because they
|
||||
live in the image. `--builtin-extensions-dir` is deliberately NOT used: it has a known upstream
|
||||
defect where extensions loaded through it present as disabled. There is no build step, no npm and no
|
||||
bundler - a VS Code extension is a directory with a `package.json` and an entry point, and it runs in
|
||||
code-server's Node remote extension host, so `main` applies.
|
||||
|
||||
**`extension.js` is CommonJS, and it is the one file in this repository that may be.** The VS Code
|
||||
extension host loads CommonJS; it is not frontend code and is never served to a browser. Every other
|
||||
house rule applies unchanged. Four small classes (`Profile`, `BootTerminals`, `Layout`, `Presence`)
|
||||
and an `activate` that runs each through `stage()`, which owns the try/catch and logs to a
|
||||
`DevPlace` output channel.
|
||||
|
||||
**The activation stages are awaited in order, and `Layout` never opens a panel of its own.** Firing
|
||||
them concurrently is what produced a stray third terminal in the first live build: `Layout` called
|
||||
`workbench.action.focusPanel` before `BootTerminals` had created anything, and VS Code answered by
|
||||
spawning its own default `bash`. `Layout.apply(panelIsOpen)` therefore resizes only when the boot
|
||||
terminals actually opened the panel, and `activate` awaits `terminals` before `layout`. Verified by
|
||||
driving a real container with Playwright: the tab list must read exactly
|
||||
`pravda@workspace` + `DevPlace Code`.
|
||||
|
||||
**A workspace suppresses the editor's own AI assistant.** Recent VS Code ships a chat panel in the
|
||||
secondary sidebar, which opened by default with Microsoft branding, "AI responses may be inaccurate"
|
||||
copy, and a competing agent right beside `dpc`. `editor.FOREIGN_AI_SETTINGS` turns it off
|
||||
(`chat.disableAIFeatures`, `chat.commandCenter.enabled`, `workbench.secondarySideBar.defaultVisibility`)
|
||||
and `workbench.startupEditor` is `none` so a workspace opens straight onto the member's code with the
|
||||
agent terminal ready, rather than onto a welcome page listing a "Get Started with VS Code"
|
||||
walkthrough. Unknown keys are ignored by VS Code, so these stay safe across version bumps. The
|
||||
DevPlace walkthrough is still contributed and reachable from Help and the command palette.
|
||||
|
||||
**Branding is six things**, all baked in: the `--app-name` / `--welcome-text` /
|
||||
`--disable-getting-started-override` flags in `editor.argv`; the favicon and PWA icon set generated
|
||||
from `static/icon-512.png` into `files/vscode/branding/`; `devplace-login.css` appended to
|
||||
code-server's `login.css`; `product.patch.json` merged key-wise into `product.json` (additive, so an
|
||||
unnamed upstream key survives a bump); the `DevPlace Dark` / `DevPlace Light` themes generated from
|
||||
`static/css/variables.css`; and the walkthrough, status bar item and five `DevPlace:` commands. The
|
||||
login stylesheet is the **single sanctioned exception to the no-colour-literals rule** - code-server
|
||||
serves it outside the application and cannot read `variables.css`, so the tokens are restated as
|
||||
literals with a comment naming each one. Do not spread that exception anywhere else.
|
||||
|
||||
**Theme token mapping** (`variables.css` -> VS Code), recorded here because a JSON theme cannot carry
|
||||
a comment: `--bg-primary` -> `editor.background`; `--bg-secondary` -> `sideBar`/`activityBar`;
|
||||
`--bg-card` -> `editorWidget`/`panel`; `--accent` -> `focusBorder`/`button.background`/`progressBar`;
|
||||
`--accent-light` -> `list.activeSelectionBackground`; `--text-primary` -> `foreground`;
|
||||
`--text-secondary` -> `descriptionForeground`; `--border` -> every `*.border`;
|
||||
`--success`/`--warning`/`--danger`/`--info` -> the ANSI green/yellow/red/blue. `DevPlace Light` is a
|
||||
derived light palette (DevPlace ships no light tokens); keep the two in step by construction.
|
||||
|
||||
**Adding an editor setting** touches five places: `editor.DEFAULTS` + `SETTING_KEYS`, a `ConfigField`
|
||||
on `WorkspaceService` (group `Editor`), the `workspace_editor_prefs` ensure block and
|
||||
`editor.PREF_COLUMNS`, `EditorPrefsForm` + `EditorProfileOut`, and the `settings_for` map or the
|
||||
extension. A `select` `ConfigField` MUST use `options=[{"value":..., "label":...}]` - plain strings
|
||||
crash `docs_api.build_services_group`, which `docs_search` indexes, which 500s the docs search page.
|
||||
|
||||
|
||||
## Pravda image (load-bearing, workspace ownership)
|
||||
|
||||
The `ppy` image (`ppy.Dockerfile`, built by `make ppy`, context `devplacepy/services/containers/files`) is a `python:3.13-slim-bookworm` base with Playwright plus a broad set of common Python libraries preinstalled, plus CLI tools (`tmux`, `apache2-utils` for `ab`, `procps`/`htop`/`iftop`/`iotop`, `netcat-openbsd` for `nc`, `zip`/`unzip`, `fakeroot`, git/curl/wget/vim/ack).
|
||||
@ -133,7 +263,7 @@ The security hotpatch that used to run per build is now baked into `ppy.Dockerfi
|
||||
|
||||
**Trade-off (intentional).** The only genuinely-root operation that still does NOT escalate is binding a port < 1024 - use a high port + `/p/<slug>` ingress instead. Enforcement lives entirely in the Dockerfile (no `--user` on `docker run`). The `export_to_dir` unlink-before-write fix remains as belt-and-suspenders (the app owns the workspace dir, so it may delete any stale file in it regardless of owner before rewriting it).
|
||||
|
||||
## `PRAVDA_*` runtime env injection
|
||||
## `DEVPLACE_*` runtime env injection (function name `pravda_env`)
|
||||
|
||||
`api.run_spec_for` merges `api.pravda_env(instance)` over the instance's own `env_json` (PRAVDA keys win), so every running container gets these platform vars:
|
||||
- `DEVPLACE_BASE_URL` - the `site_url` setting via `seo.public_base_url()`.
|
||||
@ -172,6 +302,8 @@ Both `api.sync_workspace` (HTTP/Devii sync action, returns `{exported, imported}
|
||||
|
||||
An instance's `run_as_uid` column selects WHICH DevPlace user's identity and `api_key` are injected (`DEVPLACE_API_KEY`, `DEVPLACE_USER_UID`), resolved in `api.pravda_env` ahead of the `created_by`/`owner_uid` fallback chain. It does **NOT** change the container OS user, which is ALWAYS `pravda` (uid 1000) - required for the bind-mounted `/app` (DooD uid maps 1:1 to host). Validate it against an existing user via `api.validate_run_as`.
|
||||
|
||||
**Running as someone else requires their consent (load-bearing).** `validate_run_as(run_as_uid, actor_uid)` refuses when the run-as user is not the actor and has not granted the `container_credentials` consent. This is not a policy nicety: `pravda_env` injects that user's real `DEVPLACE_API_KEY` into a container someone else operates, so without the gate an administrator could hand any member's platform credential to software that member never saw. `create_instance` and `update_instance_config` both pass `_actor_uid(actor)`, so the gate covers the admin UI, the per-project manager and the Devii container tools at once. Running as **yourself** needs no consent - you are the one handing over your own credential. The consent is granted and withdrawn from the member's own profile privacy tab like every other consent; see `devplacepy/services/moderation/CLAUDE.md`.
|
||||
|
||||
## Boot source precedence (load-bearing)
|
||||
|
||||
Columns `boot_language` (`none`|`python`|`bash`) + `boot_script` (multiline source) sit alongside the legacy `boot_command`. Precedence in `api.run_spec_for`: `boot_script` (by language) > `boot_command` > image CMD (`sleep infinity`). When a boot script is set, the reconciler writes it into the workspace (`api.materialize_boot_script`, `.devplace_boot.py`/`.devplace_boot.sh`, excluded from sync) before launch and runs `python|bash /app/.devplace_boot.<ext>`. `api.validate_boot` enforces the language set and a 100k char cap.
|
||||
@ -194,9 +326,9 @@ Use `FakeBackend` (its `image_exists` returns `True`) + `runtime.set_backend`, a
|
||||
|
||||
## Vibe coding on-ramp (user-facing doc)
|
||||
|
||||
The container runtime is also the basis of "vibe coding": the public prose page `templates/docs/getting-started-vibing.html` (slug `getting-started-vibing`, `SECTION_GENERAL`, not admin-gated so the docs stay public while the feature is admin-only/Alpha) is the canonical user-facing guide. It teaches the Devii-driven flow project -> container -> start -> terminal (`create_project`, `container_create_instance`, `container_instance_action`, the `open_terminal` client action), documents the three baked-in agents (`dpc` = DevPlace Code at `/usr/bin/dpc`, the Claude-Code-class coding agent; `botje.py` = the copy of `services/containers/files/bot.py` at `/usr/bin/botje.py`; `pagent`), all metered through the container's own `DEVPLACE_API_KEY`, the full `PRAVDA_*` env table (see `api.pravda_env`), and ingress at `/p/<slug>` via `ingress_slug`/`ingress_port`.
|
||||
The container runtime is also the basis of "vibe coding": the public prose page `templates/docs/getting-started-vibing.html` (slug `getting-started-vibing`, `SECTION_GENERAL`, not admin-gated so the docs stay public while the feature is admin-only/Alpha) is the canonical user-facing guide. It teaches the Devii-driven flow project -> container -> start -> terminal (`create_project`, `container_create_instance`, `container_instance_action`, the `open_terminal` client action), documents the three baked-in agents (`dpc` = DevPlace Code at `/usr/bin/dpc`, the Claude-Code-class coding agent; `botje.py` = the copy of `services/containers/files/bot.py` at `/usr/bin/botje.py`; `pagent`), all metered through the container's own `DEVPLACE_API_KEY`, the full `DEVPLACE_*` env table (see `api.pravda_env`), and ingress at `/p/<slug>` via `ingress_slug`/`ingress_port`.
|
||||
|
||||
When the runtime, the agent binaries, or the `PRAVDA_*`/ingress contract change, update this page alongside the source.
|
||||
When the runtime, the agent binaries, or the `DEVPLACE_*`/ingress contract change, update this page alongside the source.
|
||||
|
||||
**The agents are gateway-only:** `dpc`/`d.py` and `botje.py`/`bot.py` use a single `molodetz` backend pointed at `DEVPLACE_OPENAI_URL` (the gateway); the former direct `api.deepseek.com` fallback backend was removed so every in-container AI call is ledgered under the run-as user and nothing bypasses `gateway_usage_ledger`. `pagent`/`.vimrc` already posted to the gateway URL (using `DEEPSEEK_API_KEY` only as a key fallback, never the DeepSeek endpoint). Rebuild the image (`make ppy`) for the change to reach running containers.
|
||||
|
||||
@ -211,7 +343,7 @@ admin container manager's authorization is unchanged; workspaces add their own n
|
||||
|---|---|---|
|
||||
| Entry | `/projects/{slug}/containers/instances/{uid}/code/...` | `{port}-{name}.tunnel.pravda.education` |
|
||||
| Auth | session + `can_manage_workspace` | none, public by design |
|
||||
| Backend | code-server, `--auth none`, bound in-container | whatever the user runs |
|
||||
| Backend | code-server, `--auth password`, bound in-container | whatever the user runs |
|
||||
|
||||
Plane B routing is **one static molohttp site** `*.tunnel.pravda.education -> 127.0.0.1:10500`;
|
||||
DevPlace resolves the instance from the `Host` header. There is no molohttp object per tunnel.
|
||||
@ -220,12 +352,65 @@ WebSocket, so a tunnel host can never render the application. molohttp's `HostIn
|
||||
**exactly one label**, which is why the pattern is `{port}-{name}`, never `{port}.{name}`.
|
||||
|
||||
**One forwarding core.** `forward.py` owns header filtering, prefix stripping, `Location` rewriting,
|
||||
`<base>` injection and the bidirectional WS pump. `/p/{slug}`, the editor route and the tunnel route
|
||||
all call it. Never write a second proxy.
|
||||
`<base>` injection, response streaming and the bidirectional WS pump. `/p/{slug}`, the editor route
|
||||
and the tunnel route all call it. Never write a second proxy, and never let the two planes drift:
|
||||
**both build their headers from the single `base_headers` core**, so anything a tunnelled app sees
|
||||
over HTTP (`X-Real-IP`, `X-Forwarded-For`, `Accept-Language`, custom auth headers) it also sees over
|
||||
a websocket. `forward_headers` adds only what is HTTP-specific (`Host`, `Accept-Encoding: identity`
|
||||
so the internal hop is never compressed); `ws_headers` drops only `WS_HANDSHAKE_HEADERS`, which the
|
||||
`websockets` client regenerates itself. Both planes therefore send the same `Host` (the public one),
|
||||
the same `X-Forwarded-Host`/`-Proto`/`-Prefix`, and the same `X-Script-Name`.
|
||||
|
||||
**The websocket handshake carries exactly ONE `Host`, and it is the public one (load-bearing).**
|
||||
Never put `Host` in `additional_headers`: the `websockets` client already writes its own for the real
|
||||
TCP target and `Headers.update` *appends*, so a second one produces a handshake with two `Host`
|
||||
lines. Node keeps the FIRST (the internal `gateway:port`) and Go's `net/http` rejects the request
|
||||
outright. The fix is `websockets.connect(uri, host=..., port=...)` - the public host goes in the URI
|
||||
(so it becomes the single `Host` header) while the connection still dials the container. code-server
|
||||
resolves the request host as `Forwarded` -> `X-Forwarded-Host` -> `Host` and runs
|
||||
`authenticateOrigin` on EVERY websocket (`wsRouter.ws(/.*/, ensureOrigin, ...)`), so with only the
|
||||
losing duplicate it compared the browser's `Origin` (`devplace.net`) against the internal host and
|
||||
answered **403**, and the editor died on *"The workbench failed to connect to the server (Error: Time
|
||||
limit reached)"* - a blank page under the whole
|
||||
`/projects/{slug}/containers/instances/{uid}/code/` sub-path while its HTTP assets loaded fine
|
||||
(`forward_headers` already sent `X-Forwarded-Host`, which is why only the websocket plane broke).
|
||||
This hit the tunnel plane identically. Reproduce with the real image, not a mock: `docker run ppy
|
||||
code-server --auth none` and replay the header set - the origin check runs before authentication.
|
||||
|
||||
**Dial upstream BEFORE accepting the browser socket.** `proxy_ws` connects first and only then calls
|
||||
`websocket.accept(subprotocol=upstream.subprotocol)`. Accepting first turns every upstream refusal
|
||||
into a phantom `101` followed by an immediate `1011`, which is exactly what made the 403 above
|
||||
present as an opaque client-side timeout instead of a handshake failure; it also makes it impossible
|
||||
to echo the negotiated subprotocol, because the answer is not known yet. Subprotocols travel through
|
||||
`subprotocols=` (never as a forwarded header) and come back on the `accept`.
|
||||
|
||||
**Responses stream; only the `<base>`-injected HTML is buffered.** `proxy_http` sends with
|
||||
`stream=True` and returns a `StreamingResponse` over `aiter_raw()`, passing `content-length` and
|
||||
`content-encoding` through untouched (`RESPONSE_HOP_HEADERS` strips hop-by-hop only), so a large
|
||||
download through a tunnel costs a constant few hundred KiB of worker memory instead of its full size,
|
||||
and SSE works. The HTML branch must buffer because injection needs the whole body, and it therefore
|
||||
drops `content-length`/`content-encoding` and lets Starlette recompute. Byte accounting rides the
|
||||
`on_complete(sent)` callback fired when the stream ends - `routers/tunnel.py` uses it for
|
||||
`activity.touch` + `tunnels.record_hit`, which is ONE `touch` per request (`activity` accumulates
|
||||
while throttled, so a second call would double the request counter). **Request bodies stay
|
||||
buffered on purpose**: they are bounded by nginx `client_max_body_size`, and streaming them would
|
||||
force chunked encoding onto arbitrary upstream apps.
|
||||
|
||||
**One keep-alive client, closed on shutdown.** `forward.client()` is a lazily-created module-level
|
||||
`httpx.AsyncClient` with `httpx.Limits` (the `ChromeStealthClient` pattern), closed by
|
||||
`forward.close_client()` in the `main.py` lifespan. A client per request meant a fresh pool and TCP
|
||||
handshake for every one of the workbench's hundreds of assets.
|
||||
|
||||
**Path and query are forwarded byte-exactly.** The route-matched path is `quote()`d before it is
|
||||
interpolated (a filename containing `#` or a space otherwise truncates or corrupts the upstream URL),
|
||||
and the query comes from `raw_query(connection)` = the raw ASGI `scope["query_string"]`, NEVER
|
||||
`request.url.query`/`request.query_params`. Starlette rebuilds `.url` by string-joining path and
|
||||
query, so a literal `#` in the path makes the reconstructed URL treat the query as a fragment and
|
||||
`.url.query` silently returns empty - which would drop code-server's `?reconnectionToken=`.
|
||||
|
||||
**Editor persistence.** code-server's user-data and extensions live in
|
||||
`config.WORKSPACE_STATE_DIR/<instance uid>`, bind-mounted at `WORKSPACE_STATE_MOUNT`, so extensions
|
||||
survive container recreation. `api.editor_command` builds the argv; `run_spec_for` prefers it over
|
||||
survive container recreation. `editor.argv` builds the argv; `run_spec_for` prefers it over
|
||||
the boot-script/boot-command chain when `is_workspace` and `editor_port` are set.
|
||||
|
||||
**Activity and egress are the presence pattern.** `activity.py` keeps a per-worker monotonic dict and
|
||||
@ -241,8 +426,120 @@ patterns plus `is_tunnel_host`. `tunnels.py` is CRUD with revive-not-duplicate.
|
||||
create/resume/stop/suspend/view surface and writes `/app/.devplace/tunnels.json`.
|
||||
|
||||
**`WorkspaceService`** is the only new service: lock-owner, `default_enabled=False`, four wrapped
|
||||
phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep). It is a
|
||||
reconciler, not a `JobService`.
|
||||
synchronous phases (disk sample on its own slow cadence, flag evaluation, lifecycle, purge sweep)
|
||||
plus the async `_issue_tunnel_certificates`. It is a reconciler, not a `JobService`.
|
||||
|
||||
**Tunnel certificates are per-host, and that is forced by molohttp, not a preference.** molohttp's
|
||||
ACME client implements **`http-01` only** (`Sources/MoloHTTP/ACME/AcmeClient.swift`,
|
||||
`AcmeRenewalService.swift` - there is no `dns-01` anywhere), and Let's Encrypt will not issue a
|
||||
wildcard over `http-01`. So a `*.tunnel.pravda.education` **certificate** is impossible today and
|
||||
`workspace_cert_mode=per_host` is the only working mode; the prose above about one static wildcard
|
||||
site describes **routing**, which is correct and already in place (one enabled molohttp site
|
||||
`*.tunnel.pravda.education -> http://127.0.0.1:10500`, no per-tunnel site object). Only the cert is
|
||||
per-host. `tunnels.create` writes a row at `STATUS_PENDING` and nothing else; the row is inert until
|
||||
something certifies it - `pending` is not in `SERVING_STATUSES`, so an un-provisioned tunnel 404s.
|
||||
|
||||
**One state machine, two callers.** `certs.certify(row, log=None)` owns the whole transition:
|
||||
`provisioning` -> `certs.issue(hostname)` (`POST {workspace_molohttp_base_url}/api/v1/certs/issue`
|
||||
with the `x-api-key` header, Basic as fallback) -> `active` or `failed` with `last_error`, then
|
||||
`certs.announce(row, hostname)` sends the owner the live URL (plus the password for the editor
|
||||
tunnel). Never re-implement those transitions at a call site - both paths call `certify`:
|
||||
|
||||
- **Eagerly, at workspace creation.** `provision.ensure` ends with
|
||||
`schedule_certificate(publish_editor_tunnel(...))`, which claims the row as `provisioning`
|
||||
**synchronously** and then spawns `certify` as a loop task, so the editor certificate is ordered
|
||||
the instant the workspace exists rather than up to one service tick later. It is fire-and-forget
|
||||
by necessity: an ACME `http-01` order takes ~10s and `certs.issue` allows up to 180s, so awaiting
|
||||
it would hang the `POST /projects/{slug}/workspace` response. The task is held in
|
||||
`provision._pending_certificates` and discarded on completion - a bare `create_task` reference can
|
||||
be garbage collected mid-flight. The synchronous claim is what closes the race with the tick: the
|
||||
service selects only `pending`, so a row already claimed can never be issued twice and burn a
|
||||
Let's Encrypt duplicate-certificate slot. `schedule_certificate` no-ops (leaving the row `pending`
|
||||
for the tick) when molohttp is unconfigured or there is no running loop.
|
||||
- **On the tick, as the safety net.** `WorkspaceService._issue_tunnel_certificates` reads
|
||||
`tunnels.awaiting_certificate()` (`status=pending` + `desired_state=present` + not deleted) and
|
||||
calls `certify` per row. This covers user-created tunnels, rows predating the eager path, and any
|
||||
creation that happened with molohttp unconfigured.
|
||||
|
||||
The residual cost is the ACME round trip itself, which is irreducible under `http-01`. The tick
|
||||
interval (`service_workspace_interval`, default 30s, floor `min_interval = 5`) no longer sits in
|
||||
front of the editor tunnel, but still bounds how long a user-created tunnel waits.
|
||||
|
||||
**A tunnel serves before its certificate exists.** `SERVING_STATUSES = (provisioning, active)`, so
|
||||
`routers/tunnel.py` routes the host the moment the row is claimed, while the order is still in
|
||||
flight. The wildcard DNS already resolves, so molohttp terminates TLS with its default certificate
|
||||
and the browser shows `ERR_CERT_COMMON_NAME_INVALID` for that window rather than failing to connect.
|
||||
That is deliberate (a reachable host beats a dead one) and is the reason eager issuance matters: the
|
||||
window is only as long as issuance takes. **Only `pending` is retried** - a `failed` row stays failed until the user recreates
|
||||
the tunnel (`create` revives it to `pending`), which keeps a broken host from burning the Let's
|
||||
Encrypt failure rate limit every 30s. **Renewal is molohttp's job, not DevPlace's**: once a host is
|
||||
issued, `AcmeRenewalService` re-issues it against its own expiry threshold forever, so DevPlace never
|
||||
schedules or tracks renewals. This whole phase did not exist - `tunnels` had no reader at all, and
|
||||
the six `workspace_molohttp_*`/`workspace_cert_mode`/`workspace_acme_email` settings were admin
|
||||
fields wired to nothing, which is why every tunnel sat at `pending` with no certificate.
|
||||
|
||||
**`provision.publish_tunnel` is the ONE way a user-created tunnel comes into existence** - the HTTP
|
||||
route, the Devii tool and the editor all funnel through it. It owns the port check, the `max_tunnels`
|
||||
quota, `tunnels.create`, `schedule_certificate` and `write_manifest`, and raises `WorkspaceError` for
|
||||
every refusal. Before it existed, the route and the Devii controller each carried their own copy of
|
||||
the quota check and **neither ordered a certificate**, so a user-created tunnel sat `pending` until
|
||||
the (default-disabled) `WorkspaceService` happened to tick - which on an instance where that service
|
||||
was never enabled is forever. `schedule_certificate` now also writes `provision.CERT_UNCONFIGURED`
|
||||
into the row's `last_error` when molohttp is not configured, because a tunnel that can never be
|
||||
certified must say so on the workspace page rather than sit at `pending` with a blank error.
|
||||
|
||||
**A tunnel reaches its port through `api.tunnel_target(instance, container_port)`, never through
|
||||
`proxy_target`.** `proxy_target` answers for `/p/{slug}`, whose port is published by construction;
|
||||
a tunnel's port is whatever the member decided to serve on and is almost never published, because a
|
||||
workspace publishes only `editor_port`. `tunnel_target` therefore prefers the published host port
|
||||
when the port happens to have one (`CONTAINER_PROXY_HOST` or the recorded gateway, exactly like
|
||||
`proxy_target`) and otherwise dials `container_ip:container_port` directly. The direct leg is what
|
||||
makes an arbitrary port tunnellable at all: docker cannot add a published port to a running
|
||||
container, so publishing on demand would mean recreating the container and killing the very dev
|
||||
server the member just asked to share.
|
||||
|
||||
**The direct leg needs the app on the same docker network as the instances, and that wiring cannot
|
||||
live in compose.** Measured on this host: from the app container, `container_ip:port` times out
|
||||
(docker's inter-network isolation) while `gateway:published_host_port` connects; from the host, and
|
||||
from any container sharing the instances' network, `container_ip:port` connects. So `make dev` works
|
||||
untouched and the containerized production app does not - it must be attached to the network the
|
||||
instances run on. Compose cannot express that: it always sends network-scoped aliases, which the
|
||||
default `bridge` rejects (`invalid endpoint settings: network-scoped aliases are only supported for
|
||||
user-defined networks`). The attachment is therefore a `make docker-attach` step, run by
|
||||
`docker-up` and `docker-reload` and idempotent, deriving its input like `DOCKER_GID` does
|
||||
(`DEVPLACE_CONTAINER_NETWORK`, default `bridge`). A bare `docker compose up -d` skips it and
|
||||
silently re-breaks every unpublished-port tunnel - one more reason the make targets are the only
|
||||
supported path.
|
||||
|
||||
**Forwarding a port in the editor publishes it, and that is the whole point of `VSCODE_PROXY_URI`.**
|
||||
`api.workspace_env` advertises `https://{{port}}-{name}.{domain}` to VS Code, so the Ports view shows
|
||||
a DevPlace address for every forwarded port - but VS Code never tells DevPlace, so that address had
|
||||
no `tunnels` row, was 404ed by `routers/tunnel.py` and never got a certificate. The editor promised a
|
||||
URL the platform could not serve. The `Tunnels` stage in the workspace extension closes it: it
|
||||
subscribes to `vscode.workspace.onDidChangeTunnels`, reads `vscode.workspace.tunnels`, and POSTs each
|
||||
new `remoteAddress.port` (skipping `DEVPLACE_EDITOR_PORT`, which is already published) to
|
||||
`{DEVPLACE_BASE_URL}/projects/{DEVPLACE_PROJECT_SLUG}/workspace/tunnels` with the container's own
|
||||
`DEVPLACE_API_KEY` and `Accept: application/json`. Four things about it:
|
||||
|
||||
- **`tunnels` is a proposed API** (`checkProposedApiEnabled(extension, 'tunnels')`), so the extension
|
||||
declares `enabledApiProposals: ["tunnels"]` and `product.patch.json` names it under
|
||||
`extensionEnabledApiProposals`. code-server patches the check to always pass, so it works today
|
||||
either way; the declarations are what keep it working if that patch goes away. Because the patch
|
||||
adds a nested object, the Dockerfile's `product.json` merge now merges one level deep - a plain
|
||||
`dict.update` would wipe an upstream map of the same name on a version bump.
|
||||
- **It only ever creates.** Un-forwarding a port leaves the tunnel standing, because deleting it
|
||||
would revoke the certificate and a re-forward would re-issue, churning against Let's Encrypt's
|
||||
duplicate-certificate limit. Removal stays the explicit act it already was.
|
||||
- **A port is added to the in-memory `published` set before the POST and removed again on failure**,
|
||||
so a burst of change events cannot double-post and a refusal (quota, 403) can still retry on the
|
||||
next change. Refusals surface both in the `DevPlace` output channel and as a warning message.
|
||||
- **It uses `http`/`https` from Node, not `fetch`**, and is wrapped in the same `stage()` try/catch as
|
||||
every other activation step - a workspace whose network is down must still open its editor.
|
||||
|
||||
Verified against the real image by driving code-server with Playwright and forwarding port 3000: the
|
||||
Ports view lists the port, the extension POSTs `label=Port+3000&container_port=3000` with the API key,
|
||||
and the output channel reports the public URL. Reproduce it that way, not with a mock - the Ports
|
||||
view is the only trigger, there is no `Forward a Port` command in the palette in code-server.
|
||||
|
||||
**Two contracts that bite:**
|
||||
- A `ConfigField` with `type="select"` needs `options=[{"value": ..., "label": ...}]`. Plain strings
|
||||
@ -252,13 +549,95 @@ reconciler, not a `JobService`.
|
||||
required field makes an anonymous request 422 instead of 401 and `tests/api/auth/matrix.py` fails.
|
||||
Give the field a default and validate it inside the handler after `require_user`.
|
||||
|
||||
**The editor opens through one shared partial.** The project detail page renders an inline **Editor**
|
||||
button in `.project-detail-actions` via `templates/_editor_open.html` (`target="_blank"`, plus the
|
||||
`data-editor-*` attributes `EditorLauncher` reads) straight to the code-server proxy
|
||||
`/projects/{slug}/containers/instances/{uid}/code/`, built by `_editor_url` in
|
||||
`routers/projects/index.py` and carried as `workspace_editor_url` on the context and
|
||||
`ProjectDetailOut`. It is emitted ONLY when the viewer passes `can_open_workspace` AND their
|
||||
workspace for that project exists, is not suspended, and is `store.ST_RUNNING` - the three states the
|
||||
`editor_proxy` route itself refuses (403 suspended, 409 not running, 502 no port), so the button can
|
||||
never open a dead editor. When there is no running workspace the button is absent and the Workspace
|
||||
menu item below is the way in (create/start it there). The workspace page's own **Open editor** link
|
||||
opens in a new tab too; keep both in step.
|
||||
|
||||
**Member entry point** is the project detail page's overflow menu (`project_detail.html`), gated by
|
||||
the `viewer_can_workspace` context flag (`can_open_workspace(project, user)`, set in
|
||||
`routers/projects/index.py` and declared on `ProjectDetailOut`) - exactly the pattern the admin-only
|
||||
**Containers** item uses with `viewer_can_containers`. `can_open_workspace` folds in the
|
||||
`workspace_enabled` master switch, so the item disappears for everyone while the feature is off and
|
||||
the route's own `_guard` stays the authority. **A workspace surface with no context flag is
|
||||
unreachable**: the whole feature shipped once with routes, Devii tools and docs but no link into
|
||||
`/projects/{slug}/workspace`, so it was reachable only by typing the URL - and then 404'd anyway
|
||||
because `workspace_enabled` defaults to `"0"`. Any new workspace surface needs both the flag on the
|
||||
page that links to it and the setting turned on.
|
||||
|
||||
**Admin console** is `/admin/workspaces` (`routers/admin/workspaces.py`, `admin_workspaces.html`):
|
||||
list, start/stop, suspend/unsuspend with a required reason, raise/resolve/dismiss flags, per-user
|
||||
quota rules. `base_seo_context` takes `breadcrumbs`/`schemas`, not `canonical`/`schema`.
|
||||
|
||||
**Devii** has 15 tools under `handler="workspace"`; five are in `CONFIRM_REQUIRED` and every one of
|
||||
**Devii** has 17 tools under `handler="workspace"`; six are in `CONFIRM_REQUIRED` and every one of
|
||||
them declares a `confirm` param (schemas are `additionalProperties: false`, so a gated tool without it
|
||||
loops forever).
|
||||
loops forever). `workspace_editor_get` / `workspace_editor_set` cover the editor profile;
|
||||
`workspace_editor_set` is gated because it changes how every workspace the member opens behaves.
|
||||
|
||||
**Opening a workspace publishes its editor on a public tunnel, automatically.** `provision.ensure`
|
||||
does three things after creating the instance: `api.ensure_editor_password`, then
|
||||
`publish_editor_tunnel` (a `tunnels.create` for the editor port, so the hostname is
|
||||
`{editor_port}-{name}.{domain}`), and the row lands at `pending`. The `WorkspaceService` cert phase
|
||||
then issues the certificate and calls `_announce_tunnel`, which sends the owner a `workspace`
|
||||
notification carrying the live `https://` URL **and the password**. The notification fires on
|
||||
`pending -> active`, not at creation, so the user is only told about a URL that already serves TLS.
|
||||
`provision.is_editor_tunnel(instance, tunnel)` distinguishes the auto-published editor tunnel (its
|
||||
`container_port` equals `editor_port`) from user-created ones, which get a plainer message with no
|
||||
password. This is a deliberate departure from the plane-A/plane-B split described above: the editor
|
||||
is now reachable publicly, which is only acceptable **because** `--auth password` is on - never
|
||||
reintroduce `--auth none` while the editor tunnel is auto-published.
|
||||
|
||||
**The editor is password-protected, per workspace.** `editor.argv` runs code-server with
|
||||
`--auth password`, and `api.pravda_env` injects the secret as `PASSWORD` (the variable code-server
|
||||
reads). The secret is an 8-character **pronounceable** token from `api.generate_editor_password()`,
|
||||
built as four consonant-vowel pairs (`ronebamu`, `zipesodu`) so a user can read it once and retype it
|
||||
from memory. That is ~24 bits of entropy, which is deliberately weak for convenience: it is
|
||||
acceptable only because the alternative in practice was `--auth none`. If the editor tunnel is ever
|
||||
exposed to untrusted traffic at scale, lengthen the pair count rather than switching alphabet, and
|
||||
keep it pronounceable. `api.ensure_editor_password(instance)` is the single generator/persister: it returns the
|
||||
stored value or mints and saves one, so the password is **stable** across restarts and recreations.
|
||||
It is called from `api.run_spec_for` whenever `is_workspace` - the one point every workspace launch
|
||||
passes through - so a legacy instance created before this column existed gets one on its next boot
|
||||
rather than letting code-server invent an unknowable password of its own. The owner reads it off the
|
||||
workspace page (`editor_password` on the context and on `WorkspaceOut`). **It is deliberately NOT on
|
||||
`WorkspaceViewOut`**: that model is shared with `AdminWorkspacesOut`, which lists every workspace on
|
||||
the instance, so putting it there would hand every user's editor password to any admin loading
|
||||
`/admin/workspaces`. Keep per-workspace secrets on the owner-scoped model only. The column is
|
||||
`instances.editor_password`, ensured in `init_db`. This closes the hole that made a tunnelled editor
|
||||
an unauthenticated public shell: `--auth none` was safe only while the editor was reachable solely
|
||||
through the session-authenticated proxy route, and a user can publish a tunnel to the editor port.
|
||||
|
||||
**code-server lives in the `ppy` image, and nothing else supplies it.** `editor.argv` makes
|
||||
`code-server` the container's argv, so an image without it fails `docker run` with exit **127**
|
||||
(`executable file not found in $PATH`) and the reconciler records `crashed` / `launch_failed` with an
|
||||
empty `container_id` - the container process never existed. It is installed in `ppy.Dockerfile` in
|
||||
the runtime stage **before `USER pravda`** and before the `chown -R pravda` block (so uid 1000 owns
|
||||
it with no elevation): version pinned in the single `ARG CODE_SERVER_VERSION`, arch resolved from
|
||||
`dpkg --print-architecture`, release tarball unpacked to `/usr/local/lib/code-server` with a symlink
|
||||
at `/usr/local/bin/code-server`. `code-server --version` is in the build smoke-test loop and the
|
||||
symlink is in the executable-check list, so an image that cannot run the editor can never build
|
||||
green. Bumping the version is a one-line `ARG` change plus `make ppy`, and the branding smoke test
|
||||
(below) re-verifies the four CLI flags, the media file names and the `product.json` keys, so a
|
||||
version that breaks any of them cannot build green either. (`workspace_editor_version` was a
|
||||
`ConfigField` read by nothing and has been removed: the version is a property of the shared image,
|
||||
not a runtime setting, and a live control that changes nothing is a silent failure.)
|
||||
|
||||
**A container stuck in `created` is recreated, never retried forever.** The reconciler's
|
||||
`desired=running` branch reaches `backend.start()` for `ps.state == "created"`. That call is wrapped:
|
||||
on failure it logs, records an instance event `recreated`, `rm -f`s the container and calls
|
||||
`_launch`, so the instance is rebuilt from the CURRENT image. Without this, a container created
|
||||
against an older image that can no longer start is a permanent wedge - `docker start` re-resolves
|
||||
nothing, so the reconciler loops on the same error every tick until someone removes it by hand (the
|
||||
real failure: after code-server was added to `ppy`, the already-created workspace container kept
|
||||
failing `docker start` with the old 127 even though the new image was correct). Recreation is not a
|
||||
new retry class: a launch that fails already leaves `ps is None`, which `_launch`es again next tick.
|
||||
|
||||
**Toolchains in `ppy`.** Rust (rustup), Nim (choosenim), Swift (swiftly, then the toolchain is moved
|
||||
to a fixed `/opt/swift/toolchain` because swiftly's proxy resolves against `$HOME` and breaks for
|
||||
|
||||
@ -3,6 +3,7 @@
|
||||
import asyncio
|
||||
import json
|
||||
import re
|
||||
import secrets
|
||||
import socket
|
||||
from pathlib import Path
|
||||
|
||||
@ -48,7 +49,11 @@ def _validate_limits(cpu_limit: str, mem_limit: str) -> None:
|
||||
raise ContainerError("memory limit must look like 512m, 1g, or a byte count")
|
||||
|
||||
|
||||
def validate_run_as(run_as_uid) -> str:
|
||||
def _actor_uid(actor) -> str:
|
||||
return actor[1] if actor and actor[0] == "user" else ""
|
||||
|
||||
|
||||
def validate_run_as(run_as_uid, actor_uid: str = "") -> str:
|
||||
uid = str(run_as_uid or "").strip()
|
||||
if not uid:
|
||||
return ""
|
||||
@ -57,6 +62,13 @@ def validate_run_as(run_as_uid) -> str:
|
||||
user = database.get_users_by_uids([uid]).get(uid)
|
||||
if not user:
|
||||
raise ContainerError(f"run-as user not found: {uid}")
|
||||
if actor_uid and actor_uid != uid:
|
||||
if not database.consent_granted("user", uid, "container_credentials"):
|
||||
raise ContainerError(
|
||||
f"{user['username']} has not consented to their DevPlace credentials "
|
||||
f"being shared with software run by someone else; they grant it under "
|
||||
f"Privacy on their profile"
|
||||
)
|
||||
return uid
|
||||
|
||||
|
||||
@ -223,7 +235,7 @@ async def create_instance(
|
||||
f"restart policy must be one of {', '.join(store.RESTART_POLICIES)}"
|
||||
)
|
||||
_validate_limits(cpu_limit, mem_limit)
|
||||
run_as_uid = validate_run_as(run_as_uid)
|
||||
run_as_uid = validate_run_as(run_as_uid, _actor_uid(actor))
|
||||
boot_language, boot_script = validate_boot(boot_language, boot_script)
|
||||
port_list = assign_host_ports(parse_ports(ports))
|
||||
env_map = parse_env(env)
|
||||
@ -323,7 +335,7 @@ def update_instance_config(
|
||||
) -> dict:
|
||||
changes: dict = {}
|
||||
if run_as_uid is not None:
|
||||
changes["run_as_uid"] = validate_run_as(run_as_uid)
|
||||
changes["run_as_uid"] = validate_run_as(run_as_uid, _actor_uid(actor))
|
||||
if boot_language is not None or boot_script is not None:
|
||||
language = (
|
||||
boot_language
|
||||
@ -431,7 +443,7 @@ def pravda_env(instance: dict) -> dict:
|
||||
def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
from devplacepy import database
|
||||
from devplacepy.database import get_setting
|
||||
from devplacepy.services.containers.workspace import naming, quota
|
||||
from devplacepy.services.containers.workspace import editor, naming, quota
|
||||
|
||||
if not instance.get("is_workspace"):
|
||||
return {"DEVPLACE_WORKSPACE": ""}
|
||||
@ -464,9 +476,12 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
gallery = get_setting("workspace_extensions_gallery", "").strip()
|
||||
editor_port = int(instance.get("editor_port") or 0)
|
||||
|
||||
profile = editor.resolve(owner_uid, instance)
|
||||
env = {
|
||||
"DEVPLACE_WORKSPACE": "1",
|
||||
"DEVPLACE_WORKSPACE_UID": instance.get("uid") or "",
|
||||
"DEVPLACE_CONTAINER_BOOT": instance.get("boot_marker") or "",
|
||||
**editor.env_for(profile),
|
||||
"DEVPLACE_WORKSPACE_URL": workspace_url,
|
||||
"DEVPLACE_WORKSPACE_OWNER": owner_name,
|
||||
"DEVPLACE_WORKSPACE_OWNER_UID": owner_uid,
|
||||
@ -487,6 +502,7 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
"DEVPLACE_TUNNEL_MANIFEST": f"{WORKSPACE_MOUNT}/.devplace/tunnels.json",
|
||||
"DEVPLACE_TUNNEL_MAX": str(limits.max_tunnels),
|
||||
"VSCODE_PROXY_URI": naming.proxy_uri_template(name),
|
||||
"PASSWORD": instance.get("editor_password") or "",
|
||||
"DEVPLACE_EDITOR": "code-server",
|
||||
"DEVPLACE_EDITOR_PORT": str(editor_port),
|
||||
"DEVPLACE_EDITOR_URL": (
|
||||
@ -509,27 +525,52 @@ def workspace_env(instance: dict, base_url: str) -> dict:
|
||||
|
||||
|
||||
EDITOR_DEFAULT_PORT = 8443
|
||||
EDITOR_PASSWORD_LENGTH = 8
|
||||
EDITOR_PASSWORD_CONSONANTS = "bdfgkmnprstvz"
|
||||
EDITOR_PASSWORD_VOWELS = "aeiou"
|
||||
|
||||
|
||||
def editor_command(instance: dict) -> list[str]:
|
||||
port = int(instance.get("editor_port") or EDITOR_DEFAULT_PORT)
|
||||
return [
|
||||
"code-server",
|
||||
"--bind-addr",
|
||||
f"0.0.0.0:{port}",
|
||||
"--auth",
|
||||
"none",
|
||||
"--disable-telemetry",
|
||||
"--disable-update-check",
|
||||
"--user-data-dir",
|
||||
f"{WORKSPACE_STATE_MOUNT}/data",
|
||||
"--extensions-dir",
|
||||
f"{WORKSPACE_STATE_MOUNT}/extensions",
|
||||
WORKSPACE_MOUNT,
|
||||
]
|
||||
def generate_editor_password() -> str:
|
||||
pairs = EDITOR_PASSWORD_LENGTH // 2
|
||||
return "".join(
|
||||
secrets.choice(EDITOR_PASSWORD_CONSONANTS)
|
||||
+ secrets.choice(EDITOR_PASSWORD_VOWELS)
|
||||
for _ in range(pairs)
|
||||
)
|
||||
|
||||
|
||||
def ensure_editor_password(instance: dict) -> str:
|
||||
password = (instance.get("editor_password") or "").strip()
|
||||
if password:
|
||||
return password
|
||||
password = generate_editor_password()
|
||||
store.update_instance(instance["uid"], {"editor_password": password})
|
||||
instance["editor_password"] = password
|
||||
return password
|
||||
|
||||
|
||||
def stamp_boot_marker(instance: dict) -> str:
|
||||
from devplacepy.utils import generate_uid
|
||||
|
||||
marker = generate_uid()
|
||||
store.update_instance(instance["uid"], {"boot_marker": marker})
|
||||
instance["boot_marker"] = marker
|
||||
return marker
|
||||
|
||||
|
||||
def run_spec_for(instance: dict, image_tag: str) -> RunSpec:
|
||||
from devplacepy.services.containers.workspace import editor
|
||||
|
||||
profile = None
|
||||
cpu_limit = instance.get("cpu_limit", "")
|
||||
mem_limit = instance.get("mem_limit", "")
|
||||
if instance.get("is_workspace"):
|
||||
ensure_editor_password(instance)
|
||||
stamp_boot_marker(instance)
|
||||
profile = editor.resolve(instance.get("workspace_owner_uid", ""), instance)
|
||||
editor.seed_state(instance, profile)
|
||||
cpu_limit = profile.cpu_limit() or cpu_limit
|
||||
mem_limit = profile.mem_limit() or mem_limit
|
||||
env = {**json.loads(instance.get("env_json") or "{}"), **pravda_env(instance)}
|
||||
ports = [
|
||||
PortMapping(p["host"], p["container"], p.get("proto", "tcp"))
|
||||
@ -547,8 +588,8 @@ def run_spec_for(instance: dict, image_tag: str) -> RunSpec:
|
||||
)
|
||||
language = (instance.get("boot_language") or "none").strip().lower()
|
||||
boot = (instance.get("boot_command") or "").strip()
|
||||
if instance.get("is_workspace") and int(instance.get("editor_port") or 0):
|
||||
command = editor_command(instance)
|
||||
if profile and int(instance.get("editor_port") or 0):
|
||||
command = editor.wrap_with_env_export(editor.argv(instance, profile))
|
||||
elif language in BOOT_SCRIPT_FILES and (instance.get("boot_script") or "").strip():
|
||||
script_path = f"{WORKSPACE_MOUNT}/{BOOT_SCRIPT_FILES[language]}"
|
||||
command = [BOOT_SCRIPT_RUNNERS[language], script_path]
|
||||
@ -564,8 +605,8 @@ def run_spec_for(instance: dict, image_tag: str) -> RunSpec:
|
||||
PROJECT_LABEL: instance["project_uid"],
|
||||
},
|
||||
env=env,
|
||||
cpu_limit=instance.get("cpu_limit", ""),
|
||||
mem_limit=instance.get("mem_limit", ""),
|
||||
cpu_limit=cpu_limit,
|
||||
mem_limit=mem_limit,
|
||||
ports=ports,
|
||||
mounts=mounts,
|
||||
restart_policy=instance.get("restart_policy", "never"),
|
||||
@ -691,18 +732,30 @@ def _host_port_for(port_maps: list, container_port: int) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def reachable_target(instance: dict, container_port: int, port_maps: list) -> tuple:
|
||||
container_ip = (instance.get("container_ip") or "").strip()
|
||||
if container_ip and container_port > 0:
|
||||
return container_ip, container_port
|
||||
host_port = _host_port_for(port_maps, container_port)
|
||||
if not host_port:
|
||||
return None, None
|
||||
gateway = (instance.get("container_gateway") or "").strip()
|
||||
return config.CONTAINER_PROXY_HOST or gateway or "127.0.0.1", host_port
|
||||
|
||||
|
||||
def proxy_target(instance: dict) -> tuple:
|
||||
port_maps = json.loads(instance.get("ports_json") or "[]")
|
||||
container_port = _ingress_container_port(instance, port_maps)
|
||||
if not container_port:
|
||||
return None, None
|
||||
host_port = _host_port_for(port_maps, container_port)
|
||||
if config.CONTAINER_PROXY_HOST:
|
||||
return (config.CONTAINER_PROXY_HOST, host_port) if host_port else (None, None)
|
||||
if not host_port:
|
||||
return reachable_target(instance, container_port, port_maps)
|
||||
|
||||
|
||||
def tunnel_target(instance: dict, container_port: int) -> tuple:
|
||||
if container_port <= 0:
|
||||
return None, None
|
||||
gateway = (instance.get("container_gateway") or "").strip()
|
||||
return (gateway or "127.0.0.1", host_port)
|
||||
port_maps = json.loads(instance.get("ports_json") or "[]")
|
||||
return reachable_target(instance, container_port, port_maps)
|
||||
|
||||
|
||||
def instance_runtime(instance: dict) -> dict:
|
||||
|
||||
@ -52,7 +52,7 @@ def _resolve_devplace_url() -> str:
|
||||
base = os.environ.get("DEVPLACE_BASE_URL", "").strip().rstrip("/")
|
||||
if base:
|
||||
return base
|
||||
return os.environ.get("DEVPLACE_URL", "https://devplace.net").strip().rstrip("/")
|
||||
return os.environ.get("DEVPLACE_URL", "").strip().rstrip("/")
|
||||
|
||||
|
||||
def _resolve_llm_endpoint() -> str:
|
||||
@ -63,11 +63,7 @@ def _resolve_llm_endpoint() -> str:
|
||||
|
||||
|
||||
DEVPLACE_URL = _resolve_devplace_url()
|
||||
DEVPLACE_API_KEY = (
|
||||
os.environ.get("DEVPLACE_API_KEY")
|
||||
or os.environ.get("DEVPLACE_API_KEY")
|
||||
or "019ea58c-fae0-7112-8025-e629a54104a4"
|
||||
)
|
||||
DEVPLACE_API_KEY = os.environ.get("DEVPLACE_API_KEY", "").strip()
|
||||
MENTION_POLL_SECONDS = int(os.environ.get("MENTION_POLL_SECONDS", "30"))
|
||||
DM_POLL_SECONDS = int(os.environ.get("DM_POLL_SECONDS", "10"))
|
||||
BOT_USERNAME = os.environ.get("BOT_USERNAME", "")
|
||||
@ -2000,9 +1996,9 @@ async def reflect(observation: str, conclusion: str, next_action: str):
|
||||
|
||||
|
||||
@tool
|
||||
async def verify(command: str = "hawk .", timeout: int = 600):
|
||||
async def verify(command: str, timeout: int = 600):
|
||||
"""Run a verification command (linter, tests, validator). Marks the task verified on success.
|
||||
command: Shell command, default 'hawk .'.
|
||||
command: Shell command that verifies the change, for example the project's test or lint command.
|
||||
timeout: Timeout in seconds.
|
||||
"""
|
||||
try:
|
||||
@ -2526,7 +2522,7 @@ async def react_loop(
|
||||
"role": "user",
|
||||
"content": (
|
||||
"[verification-gate] You produced a final answer after modifying files "
|
||||
"without a successful verify(). Call verify() now (default 'hawk .') and "
|
||||
"without a successful verify(). Call verify() with the command that verifies this project now and "
|
||||
"report the result. If verification truly does not apply, reply starting "
|
||||
"with: 'No verification applicable: <reason>'."
|
||||
),
|
||||
@ -2559,7 +2555,7 @@ OPERATING PROTOCOL
|
||||
|
||||
3. INVESTIGATE BEFORE EDITING. Use grep, glob_files, list_dir, find_symbol, and retrieve to navigate. You MUST read_file (or read_lines) an existing file before edit_file, patch_file, or write_file touches it — the harness enforces this. Prefer edit_file for surgical replacements, patch_file for multi-hunk diffs, create_file for new files, write_file for full rewrites of files you have read.
|
||||
|
||||
4. VERIFY BEFORE FINISHING. Whenever you modify files, call verify() (default 'hawk .') before your final answer. The harness rejects a final answer that changed files without a successful verify().
|
||||
4. VERIFY BEFORE FINISHING. Whenever you modify files, call verify() with the command that verifies this project before your final answer. The harness rejects a final answer that changed files without a successful verify().
|
||||
|
||||
5. REFLECT ON FAILURE. After any tool returns status=error, the harness injects a reflection trigger. Respond with reflect() (observation, conclusion, next_action), then proceed. Never blindly retry the same call.
|
||||
|
||||
@ -2812,8 +2808,17 @@ async def _agent_answer_for_devplace(
|
||||
async def devplace_bot_loop() -> None:
|
||||
"""Run the DevPlace bot: poll mentions and DMs forever."""
|
||||
logger.info("Botje starting — DevPlace bot with full X-agent capabilities")
|
||||
|
||||
if not DEVPLACE_URL or not DEVPLACE_API_KEY:
|
||||
logger.error(
|
||||
"DEVPLACE_BASE_URL and DEVPLACE_API_KEY are not set. Both are injected "
|
||||
"automatically inside a DevPlace-managed container; set them manually "
|
||||
"only when running botje.py outside one.",
|
||||
)
|
||||
return
|
||||
|
||||
logger.info("DevPlace URL: %s", DEVPLACE_URL)
|
||||
logger.info("API key: %s...", DEVPLACE_API_KEY[:12] if DEVPLACE_API_KEY else "(none)")
|
||||
logger.info("API key: %s...", DEVPLACE_API_KEY[:12])
|
||||
|
||||
dp = DevPlace(DEVPLACE_URL, DEVPLACE_API_KEY)
|
||||
|
||||
|
||||
@ -1757,9 +1757,9 @@ async def reflect(observation: str, conclusion: str, next_action: str):
|
||||
|
||||
|
||||
@tool
|
||||
async def verify(command: str = "hawk .", timeout: int = 600):
|
||||
async def verify(command: str, timeout: int = 600):
|
||||
"""Run a verification command (linter, tests, validator). Marks the task verified on success.
|
||||
command: Shell command, default 'hawk .'.
|
||||
command: Shell command that verifies the change, for example the project's test or lint command.
|
||||
timeout: Timeout in seconds.
|
||||
"""
|
||||
try:
|
||||
@ -2196,7 +2196,7 @@ async def react_loop(
|
||||
"role": "user",
|
||||
"content": (
|
||||
"[verification-gate] You produced a final answer after modifying files without a successful verify(). "
|
||||
"Call verify() now (default 'hawk .') and report the result. If verification truly does not apply, reply "
|
||||
"Call verify() with the command that verifies this project now and report the result. If verification truly does not apply, reply "
|
||||
"starting with: 'No verification applicable: <reason>'."
|
||||
),
|
||||
})
|
||||
@ -2224,7 +2224,7 @@ OPERATING PROTOCOL
|
||||
|
||||
3. INVESTIGATE BEFORE EDITING. Use grep, glob_files, list_dir, find_symbol, and retrieve to navigate. You MUST read_file (or read_lines) an existing file before edit_file, patch_file, or write_file touches it — the harness enforces this. Prefer edit_file for surgical replacements, patch_file for multi-hunk diffs, create_file for new files, write_file for full rewrites of files you have read.
|
||||
|
||||
4. VERIFY BEFORE FINISHING. Whenever you modify files, call verify() (default 'hawk .') before your final answer. The harness rejects a final answer that changed files without a successful verify().
|
||||
4. VERIFY BEFORE FINISHING. Whenever you modify files, call verify() with the command that verifies this project before your final answer. The harness rejects a final answer that changed files without a successful verify().
|
||||
|
||||
5. REFLECT ON FAILURE. After any tool returns status=error, the harness injects a reflection trigger. Respond with reflect() (observation, conclusion, next_action), then proceed. Never blindly retry the same call.
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user