Converge every account onto every policy agreement it has not declined

An instance kept production-identical for extended manual testing is
otherwise taxed forever by its own safety controls: five consents, a
versioned terms gate on every mutating request, and every account
predating the trust and safety commit reading terms_version NULL because
init_db deliberately never backfills it. AcceptanceService grants each
agreement to each account that has not declined it, so the instance stays
production byte for byte while nobody clicks the same dialog again. It is
opt-in, dry run by default, and one switch per agreement type.

The application is not allowed to know it exists. One registration line
in main.py is the only import anywhere, there is no route, schema,
template, Devii tool or environment flag, and a unit test greps the tree
and fails the suite if a second importer appears. The decline register
needs no storage: the ledger is append-only in effect, the service only
ever grants, so any withdrawn row was written by a human and that pair is
never touched again. No provenance column, nothing to observe.

Satisfaction is the gate's own expression, never a proxy, which is why
the ordering is created_at then id exactly as consent_state selects, and
why the live-account clauses are built with has_column: init_db ensures
terms_version and deletion_requested_at but not is_active, so a hardcoded
reference raises no such column on an instance where nobody was ever
suspended. Every write is one conditional statement decided on the real
rowcount, proven with sixteen processes racing one account to exactly one
ledger row and one audit row. The two existing audit keys carry it, with
actor kind service, because a service that silently mutated consent state
would be the worst possible exception to the append-only rule.

lensfl.md is the source brief accept.md records the design against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
retoor 2026-08-10 00:17:47 +02:00
parent 1a5fc9428a
commit 7e37122f9f
20 changed files with 1834 additions and 2 deletions

View File

@ -141,6 +141,7 @@ Nested `CLAUDE.md` files (loaded automatically by Claude Code only when a file i
| `devplacepy/services/openai_gateway/CLAUDE.md` | AI gateway: `/openai/v1/*`, usage ledger, provider/model routing | | `devplacepy/services/openai_gateway/CLAUDE.md` | AI gateway: `/openai/v1/*`, usage ledger, provider/model routing |
| `devplacepy/services/jobs/CLAUDE.md` | Async job services: zip, fork, SEO diagnostics, SEO metadata, DeepSearch, AI Usage Analyzer | | `devplacepy/services/jobs/CLAUDE.md` | Async job services: zip, fork, SEO diagnostics, SEO metadata, DeepSearch, AI Usage Analyzer |
| `devplacepy/services/moderation/CLAUDE.md` | Trust and safety: the reportable-target registry, the content filter and its five choke points, the report queue and its atomic resolution, enforcement, consent, maturity, account deletion | | `devplacepy/services/moderation/CLAUDE.md` | Trust and safety: the reportable-target registry, the content filter and its five choke points, the report queue and its atomic resolution, enforcement, consent, maturity, account deletion |
| `devplacepy/services/acceptance/CLAUDE.md` | Acceptance convergence: the opt-in service that grants every policy agreement to every account that has not declined it, its invisibility contract and the ledger-as-decline-register rule |
| `devplacepy/services/audit/CLAUDE.md` | Audit log: recorders, categories, retention | | `devplacepy/services/audit/CLAUDE.md` | Audit log: recorders, categories, retention |
| `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download | | `devplacepy/services/backup/CLAUDE.md` | Backup service: targets, worker, schedules, primary-admin-only download |
| `devplacepy/services/telegram/CLAUDE.md` | Telegram bot bridge | | `devplacepy/services/telegram/CLAUDE.md` | Telegram bot bridge |

View File

@ -434,6 +434,7 @@ and its full configuration are documented automatically - including future servi
- **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected - **`GatewayService`** - OpenAI-compatible LLM gateway at `/openai/v1/*`, forwarding to DeepSeek (default) with optional vision augmentation; the single point of truth for AI that every other service routes through (enabled by default). Every chat request is made date-aware by injecting the current date in EU `DD/MM/YYYY` format into the system message when it contains no date already (date only, never time, to keep upstream prompt caching effective), and an admin-configurable system preamble (`gateway_system_preamble`) can be prepended ahead of the client's system message on every call. **Provider and model routing** (admin **Gateway** page, `/admin/gateway`) maps any number of requested model names onto named upstream providers and target models, each with its own pricing economy (input, output and cache-hit/cache-miss rates per million tokens) and an optional vision model that describes image content before forwarding, so text and vision models are merged transparently. Unmapped requests fall through to the default upstream unchanged, so existing clients are unaffected
- **`JobService` / `ZipService` / `ForkService`** - generic async job framework (`services/jobs/`) for heavy, blocking work run off the request path; `ZipService` builds project zip archives in a subprocess, `ForkService` copies a project into a new project owned by the forking user - **`JobService` / `ZipService` / `ForkService`** - generic async job framework (`services/jobs/`) for heavy, blocking work run off the request path; `ZipService` builds project zip archives in a subprocess, `ForkService` copies a project into a new project owned by the forking user
- **`ContainerService`** - the admin container manager (`services/containers/`): a reconciling supervisor for container instances, all running one shared prebuilt image - **`ContainerService`** - the admin container manager (`services/containers/`): a reconciling supervisor for container instances, all running one shared prebuilt image
- **`AcceptanceService`** - grants every policy agreement (Terms of Service, Privacy Policy, third-party AI processing, activity recording, container credentials) to every account that has not declined it, so an instance kept production-identical for extended manual testing never interrupts with an acceptance dialog. Administrator-only, **off by default**, with a separate switch per agreement type and a dry run that reports what it would do without writing. An account that withdrew a consent is never granted it again, with no further action: the consent ledger itself is the decline register. It is not appropriate on a real production host
### Container manager (admin only) ### Container manager (admin only)

769
accept.md Normal file
View File

@ -0,0 +1,769 @@
# DevPlace: acceptance convergence implementation design
Author: retoor <retoor@molodetz.nl>
Design input is [`lensfl.md`](lensfl.md). This document specifies a background service that converges every
account onto every policy agreement the platform can ask for, so that an operator running a
production-identical instance for manual testing is never interrupted by an acceptance dialog, a consent
refusal, or the terms gate.
Nothing here is implemented. This is the specification.
Every claim below about existing behaviour was read out of the source at the time of writing, and the
citations are exact. The subsystem this service operates on is documented in
[`devplacepy/services/moderation/CLAUDE.md`](devplacepy/services/moderation/CLAUDE.md); the framework it is
built on is documented in [`devplacepy/services/CLAUDE.md`](devplacepy/services/CLAUDE.md). Read both before
implementing.
---
## 1. The problem, stated exactly
DevPlace enforces five consents, a versioned terms gate on every mutating request, and an age-banded
maturity gate. Each is correct and each must stay. On a production-identical instance used for extended
manual testing they produce a permanent tax: every new account, every bot in the fleet, and every account
that predates the trust-and-safety commit (whose `users.terms_version` is deliberately left `NULL`, never
backfilled) is blocked or degraded until a human clicks through the same dialogs again.
The instance must remain production. Not production-like with a flag threaded through the request path;
production, byte for byte, with one background process quietly converging the acceptance state that a real
population of users would have produced themselves.
Three constraints follow, and they are the whole design:
1. **The application must not know.** No request-path branch, no schema, no template, no route, no response
field, no Jinja global, no Devii tool. If any consumer could observe the feature, the instance is no
longer a production simulation and the exercise is worthless.
2. **A manual decline must be permanent.** Testing the refusal path is the reason the operator is here. An
account that has declined an agreement must never be converged, ever, with no further operator action.
3. **It must be off, and stay off, unless deliberately switched on.** Two independent switches, both
defaulting to off, plus a dry run that defaults to on.
---
## 2. The invisibility contract
"The system must not know" is vague. These are the testable clauses that replace it.
| # | Clause | How it is enforced |
|---|--------|--------------------|
| **I1** | No new table, no new column, anywhere. | The service writes only through existing columns of `users` and existing columns of `user_consents`. A unit test asserts the column sets it writes are subsets of what `database.moderation.set_consent` and `routers/auth/terms.py` already write. |
| **I2** | No new value in any existing registry or enum. `CONSENT_KINDS`, `CONSENT_STATES`, `AGE_BANDS`, `MATURITY_LEVELS`, `REPORTABLE_TARGETS`, `UNREPORTABLE_TABLES`, `SOFT_DELETE_TABLES` are untouched. | Reviewed at implementation; the moderation registry test already fails on an unclassified table, and no table is added. |
| **I3** | No new audit event key. The service records the same two keys the human paths record. | `events.md` gains no key, the header's stated key count does not change and no category count changes; only the "Recorded in" column of `terms.accept` and `consent.grant` gains a file. |
| **I4** | The rows produced are indistinguishable in shape from the rows a real acceptance produces. | A unit test writes one row through `set_consent` and one through the service and compares column sets and value shapes field by field. |
| **I5** | Nothing imports the package except one registration line in `main.py`. | A unit test greps the source tree for `services.acceptance` and asserts exactly one importing module, `devplacepy/main.py`. |
| **I6** | The feature has no route, no schema, no template, no Devii action and no `docs_api` entry. | It inherits the generic services admin surface, exactly as `NotificationRelayService`, `PresenceRelayService`, `LiveViewRelayService` and `AuditService` do. See section 14 for why this does not violate the four-faces rule. |
| **I7** | The test suite is unaffected. | `DEVPLACE_DISABLE_SERVICES=1` in `tests/conftest.py` stops `supervise()` from ever being called, and `default_enabled = False` keeps the service off even when services do run. The moderation tests that assert a `403` from the terms gate stay green by construction. |
The one thing that is deliberately visible is the audit log. Every state change in DevPlace records one
append-only row, and a service that silently mutated user consent state would be the single worst possible
exception to that rule. The audit log is the operator's record, not application state: no code path reads it
to decide behaviour, so recording there costs nothing in invisibility and buys full traceability. This is
the position `devplacepy/services/audit/CLAUDE.md` already takes.
---
## 3. Design axioms
Every axiom is an existing DevPlace pattern, named with its precedent. None is invented for this feature.
| # | Axiom | Precedent |
|---|-------|-----------|
| **A1** | A background subsystem is a `BaseService` with declarative `config_fields`, and gets start/stop, enable-on-boot, run-now, a config form, logs, live status and metrics for free. | `services/base.py`; `ModerationService`, `PushService` |
| **A2** | An opt-in service sets `default_enabled = False` so `is_enabled()` reads `get_setting(enabled_key, "0")` and it never auto-starts on boot. | `services/base.py:203`; `BotsService` |
| **A3** | Runtime policy lives in `site_settings`, read through the field's own `ConfigField.read()`, live-editable, never a code constant. | `services/base.py:218` (`get_config`) |
| **A4** | The target set is a registry, not a literal. Every consumer reads the same structure; adding a type is one entry. | `REPORTABLE_TARGETS`, `CONSENT_KINDS`, `DATA_PATHS`; `PushService.config_fields` splices `*providers.admin_fields()` |
| **A5** | A read-then-write mutation reachable from more than one path is a race until it is one conditional SQL statement decided on the driver's real `rowcount`. | `services/moderation/deletion.py:140` (`claim_deletion`), `queue.claim_open`, `database/atomic.py` |
| **A6** | Lock-owner-only background work; never per worker. | `main.py:289-293`: services are registered in every worker, `supervise()` runs only in the worker that won `acquire_service_lock()` |
| **A7** | Every state change records an audit row, and the recorder never raises into the caller. | `services/audit/record.py:288` (`record_system`) |
| **A8** | A control that cannot evaluate its own precondition fails closed, never open. | `moderation/filter.classify` fails to `review`, never to `allow` |
| **A9** | Legacy rows carry `NULL` in any column added after they were written, and a column added after the table may not exist at all. Every precondition uses `COALESCE`, and every raw-SQL reference to an optional column is guarded by `has_column`. | `claim_deletion`'s `COALESCE(deletion_requested_at, '')`; `deletion.due_purges`'s `if not table.has_column(...)`; the root `CLAUDE.md` race-safety rule |
| **A10** | No silent caps. When a bound truncates the work, say so in the log. | Root `CLAUDE.md`, "Rigorous correctness verification" |
---
## 4. The agreement registry
An **agreement** is a policy statement the platform asks a member to accept, recorded in `user_consents`.
There are exactly five and they are already enumerated by `database.CONSENT_KINDS`
(`database/moderation.py:122`). The registry does not invent a parallel list; it annotates the existing one
with the two facts the service needs.
`devplacepy/services/acceptance/agreements.py`:
```python
@dataclass(frozen=True)
class Agreement:
kind: str
version_setting: str
gate_column: str
stamp_column: str
AGREEMENTS: tuple[Agreement, ...] = (
Agreement("terms", "terms_version", "terms_version", "terms_accepted_at"),
Agreement("privacy", "privacy_version", "", ""),
Agreement("ai_third_party", "", "", ""),
Agreement("activity_recording", "", "", ""),
Agreement("container_credentials", "", "", ""),
)
```
`kind` is a key of `CONSENT_KINDS`. `version_setting` is the `site_settings` key holding the policy version,
`""` when the agreement is unversioned. `gate_column` is the `users` column the application's own gate reads,
`""` when the agreement is ledger-only. `stamp_column` holds the acceptance instant beside it.
`terms` is the only agreement with a gate column, because `routers/auth/terms.needs_acceptance` reads
`users.terms_version` and not the ledger. `privacy` is versioned but ledger-only: `VERSION_KEYS` in
`routers/profile/consent.py:28` maps exactly `terms` and `privacy` to a version setting, and every other kind
resolves to the literal `"1"` through `consent_version`. That asymmetry is in the application, not in this
design; the registry records it rather than papering over it.
**Completeness is enforced, not remembered.** A unit test asserts
`{a.kind for a in AGREEMENTS} == set(CONSENT_KINDS)`. A sixth consent added to `CONSENT_KINDS` in the future
fails the suite until it is classified here, exactly as `tests/unit/database/moderation.py` does for
reportable tables. This is the single most valuable line in the feature: it is what keeps the convergence
complete as the platform grows.
The label rendered in the admin form is `CONSENT_KINDS[kind]`, read at field-construction time. The service
never carries its own copy of the wording.
### 4.1 What is deliberately not an agreement
| Excluded | Why | The supported answer instead |
|---|---|---|
| `users.age_band` | A declaration of fact, not an agreement. Fabricating a declared age is not a production simulation, it is a falsification, and it would silently unlock `restricted` content for accounts that declared 13-15. | Sign the test account up with an adult date of birth. |
| `users.mature_opt_in` | A content preference gated by the age band, with no entry in the consent ledger and therefore no decline register that outlives the 90-day `audit_log_retention_days` window. Adding a second, weaker decline mechanism for one field would violate DRY and clause I2. | The existing site setting `moderation_mature_default_hidden`, editable at `/admin/settings`, already turns the interstitial off instance-wide. |
| Guest consents | Nothing writes a consent row with `owner_kind = "guest"` and no guest surface is gated by one. The gateway resolves a guest to owner kind `anonymous`, which `consent_denied` exempts by design. | None needed. |
| `interactions_enabled`, notification preferences, customization toggles | Per-user preferences, not policy acceptances. None of them blocks anything. | None needed. |
Each exclusion is a boundary, not an oversight, and each is restated in the nested `CLAUDE.md`.
---
## 5. The decline register
This is the load-bearing idea and it needs no storage at all.
`user_consents` is append-only in effect: `set_consent` stamps `withdrawn_at` on the current row when it is
granted and then inserts a new row either way (`database/moderation.py:289`), so the full history is provable.
The **latest live row for a `(user, kind)` pair therefore already is the decline register**:
| Latest row state | Meaning | Service behaviour |
|---|---|---|
| no row | never asked | converge |
| `granted`, current version | satisfied | nothing to do |
| `granted`, stale version | policy changed since acceptance | converge |
| `withdrawn` | **declined by a human** | never touch, permanently |
The service only ever writes `granted`. It follows that any `withdrawn` row in the ledger was written by a
human action: the owner-only `POST /profile/{username}/consent`, the Devii consent tool acting on the owner's
behalf, or the CLI. No provenance column, no marker, no flag, and nothing for the application to observe.
Clause I1 is satisfied not by discipline but by the shape of the existing data.
**One SQL fragment expresses it**, defined once in `pending.py` and used by every read and every write:
```sql
COALESCE((SELECT c.state FROM user_consents c
WHERE c.owner_kind = 'user' AND c.owner_id = <user>
AND c.kind = :kind AND c.deleted_at IS NULL
ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'withdrawn'
```
**The ordering is not a detail and must not be simplified.** `database.consent_state` selects
`order_by=["-created_at", "-id"]` (`database/moderation.py:274`) and `consent_granted` is a thin wrapper over
it, so that pair *is* the expression the application evaluates when it asks whether a consent stands.
Ordering by `id` alone, or by `created_at` alone, would make the service disagree with the gate in exactly the
tie case that matters: two rows written in the same millisecond, which is precisely what a converge-then-
withdraw race produces. `deleted_at IS NULL` is required because `user_consents` is in `SOFT_DELETE_TABLES`.
The index `idx_user_consents_owner` (`owner_kind`, `owner_id`, `kind`) already exists
(`database/schema.py:1897`) and serves this; no index is added.
`consent_view` on the profile privacy tab orders by `-created_at` only and takes the first row per kind. That
is a looser tiebreak than `consent_state`, and it is the display path, not a gate. Follow `consent_state`;
never introduce a third ordering.
### 5.1 Terms is the one case that needs an operator step
Withdrawing the `terms` consent does not clear `users.terms_version`, so a tester who withdraws it is not
immediately gated. The recipe is two actions, both of which already exist:
1. The tester withdraws their own `terms` consent once, at `/profile/{username}?tab=privacy`.
2. An administrator bumps `terms_version` at `/admin/settings`.
Every other account converges within one interval; the tester stays gated indefinitely and can exercise
`/auth/accept-terms`, the `TermsGate.js` in-place dialog, and the `terms_acceptance_required` JSON refusal as
often as they like. This is the workflow section 13 documents for the operator.
---
## 6. Module map
New package `devplacepy/services/acceptance/`, laid out like `services/moderation/`: registry, read side,
write side, service.
| File | Owns |
|---|---|
| `__init__.py` | Nothing but the package. No re-exports; the service is imported by path from `main.py`. |
| `agreements.py` | `Agreement`, `AGREEMENTS`, `setting_key(kind)`, `label_for(kind)` |
| `pending.py` | `NOT_WITHDRAWN`, `satisfied_clause(agreement)`, `pending(agreement, limit) -> list[dict]`, `current_version(agreement) -> str` |
| `grant.py` | `converge_user(agreement, user, version) -> bool`, `_claim_gate_column`, `_insert_consent_row`, the audit calls |
| `service.py` | `AcceptanceService(BaseService)`: config fields, `run_once`, `collect_metrics` |
| `CLAUDE.md` | The nested subsystem documentation (section 14) |
Import discipline, mirroring `services/moderation/deletion.py`: `pending.py` and `grant.py` import only from
`devplacepy.database` and `sqlalchemy`; `devplacepy.services.audit`, `devplacepy.utils.generate_uid` and
`devplacepy.database.bump_cache_version` are imported lazily inside the functions that use them, exactly as
`deletion.py` imports its audit recorder at line 187. Nothing in the package imports `content`, a router, or a
template global.
---
## 7. The read side: what is pending
`pending(agreement, limit)` returns the bounded list of accounts that are neither satisfied nor declined.
One statement, three shapes of one clause.
```sql
SELECT u.uid, u.username
FROM users u
WHERE <live account clauses>
AND <not_withdrawn>
AND <not_satisfied>
ORDER BY u.id
LIMIT :limit
```
`<not_withdrawn>` is the fragment from section 5, correlated on `u.uid`.
`<not_satisfied>` is derived from the registry, and the derivation rule is the principle that keeps this
honest: **satisfaction is defined by the exact expression the application's own gate evaluates, never by a
proxy.**
| Agreement shape | Application gate | `<not_satisfied>` |
|---|---|---|
| gate column declared (`terms`) | `needs_acceptance` compares `users.terms_version` to `get_setting("terms_version", "1") or "1"` (`routers/auth/terms.py:21-28`) | `COALESCE(u.terms_version, '') != :version` |
| versioned, ledger only (`privacy`) | the privacy tab compares the latest row's `version` against `consent_version(kind)` | `COALESCE((SELECT c.version FROM user_consents c WHERE ... ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != :version` |
| unversioned (the other three) | `consent_granted` asks whether the latest state is `granted` | `COALESCE((SELECT c.state FROM user_consents c WHERE ... ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'granted'` |
For the versioned ledger-only shape the state does not appear in the clause: a `withdrawn` row is already
excluded by `<not_withdrawn>`, so comparing the version alone is complete. Keeping the two concerns in two
independent clauses is what makes that provable rather than lucky.
`current_version(agreement)` is `get_setting(agreement.version_setting, "1") or "1"` for a versioned agreement
and the literal `"1"` for an unversioned one, which is byte-identical to `consent_version` in
`routers/profile/consent.py:47`. **The `or "1"` is load-bearing and is not cosmetic:** on a fresh database an
admin settings save can insert `terms_version = ""`, and a bare `get_setting` would then make every account
pending forever and re-write the same rows on every tick. This is the trap documented in
`services/moderation/CLAUDE.md`, and every reader of a policy version in this codebase carries the guard.
### 7.1 The live-account clauses are built, not hardcoded
This is the correction that backwards compatibility actually turns on, and it is stricter than a `COALESCE`.
`init_db` ensures `terms_version`, `terms_accepted_at` and `deletion_requested_at` on `users`
(`database/schema.py:2000-2015`). **It does not ensure `is_active`.** That column is created implicitly by
`dataset` the first time `enforcement.suspend_user` or `deletion.anonymise` writes it, so on an instance where
nobody has ever been suspended or deleted the column does not exist and a raw-SQL reference to it raises
`OperationalError: no such column`. A background service that dies on a clean database is worse than no
service.
So the `WHERE` clauses are assembled from what the table actually has, the same way `deletion.due_purges`
guards with `table.has_column` before querying:
```python
users = get_table("users")
clauses = []
if users.has_column("deletion_requested_at"):
clauses.append("COALESCE(u.deletion_requested_at, '') = ''")
if users.has_column("is_active"):
clauses.append("COALESCE(u.is_active, 1) != 0")
if users.has_column("deleted_at"):
clauses.append("u.deleted_at IS NULL")
```
An absent column means no account can be in that state, so omitting the clause is the correct answer and not
a relaxation. `COALESCE(u.is_active, 1) != 0` matches `database.users.is_account_active` exactly, which treats
`NULL` as active (`database/users.py:77-79`). `users` is not in `SOFT_DELETE_TABLES` - deletion anonymises and
stamps `deletion_requested_at` instead - so the `deleted_at` clause is defensive and will normally be omitted.
The same rule applies to the gate column: if `agreement.gate_column` is declared but the column is missing,
`pending` treats the agreement as ledger-only for that database rather than raising.
Guards, in order, at the top of `pending`:
- `if "users" not in db.tables or CONSENTS_TABLE not in db.tables: return []`. On a database so fresh that the
consent table has not been created, no account can be shown to have declined, so per axiom A8 the answer is
"converge nobody", not "converge everybody".
- `limit = max(1, limit)`.
`ORDER BY u.id` gives a stable, resumable sweep: a capped run always resumes from the same place, and a large
population converges deterministically over consecutive intervals rather than randomly.
---
## 8. The write side: one conditional statement per grant
Every write re-evaluates its own precondition inside the write transaction and is decided on the driver's real
`rowcount`. This closes the race between the service's `SELECT` and a member withdrawing a consent from the
profile page a moment later; SQLite serialises writers, so a withdrawal committed before ours is visible to
our `WHERE` clause and the grant simply does not happen.
`dataset`'s wrapped `db.query()` does not expose `rowcount`. Use
`db.executable.execute(sqlalchemy.text(...), params).rowcount` inside `with db:`, exactly as
`deletion.claim_deletion` does at `services/moderation/deletion.py:140`.
### 8.1 With a gate column (`terms`)
The users update is the atomic claim; the ledger row is the follow-through, in the same transaction so the two
can never diverge.
```sql
UPDATE users
SET terms_version = :version, terms_accepted_at = :now
WHERE uid = :uid
AND COALESCE(terms_version, '') != :version
AND COALESCE((SELECT c.state FROM user_consents c
WHERE c.owner_kind = 'user' AND c.owner_id = users.uid
AND c.kind = 'terms' AND c.deleted_at IS NULL
ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'withdrawn'
```
`rowcount == 1` wins the claim; `0` means another worker, a real acceptance, or a withdrawal got there first,
and the function returns `False` without writing anything else.
The set clause writes exactly the two columns `POST /auth/accept-terms` writes and nothing more
(`routers/auth/terms.py:57-61`). **`updated_at` is not written**, which is also why
`database.atomic.conditional_update_row` cannot be reused here: it appends `updated_at = :updated_at`
unconditionally (`database/atomic.py:16`), the `users` table has no such column, and creating one would make
the service's rows distinguishable from the route's. The hand-rolled statement is the same shape
`claim_deletion` uses on the same table, for the same reason.
**The human path grants two agreements, not one.** `POST /auth/accept-terms` writes the users row, then calls
`set_consent` for `terms` **and** for `privacy` at `get_setting("privacy_version", "1") or "1"`. The service
deliberately keeps the two agreements independent, because the whole point of the per-agreement switches is
edge-case testing: an operator must be able to converge `terms` while leaving `privacy` pending. Enabling both
agreements reproduces the human path exactly; enabling one is a deliberate, documented divergence and the
nested `CLAUDE.md` says so.
### 8.2 Ledger only (the other four)
```sql
INSERT INTO user_consents
(uid, owner_kind, owner_id, kind, version, state,
granted_at, withdrawn_at, created_at, deleted_at, deleted_by)
SELECT :row_uid, 'user', :uid, :kind, :version, 'granted',
:now, '', :now, NULL, NULL
WHERE COALESCE((SELECT c.state FROM user_consents c
WHERE c.owner_kind = 'user' AND c.owner_id = :uid
AND c.kind = :kind AND c.deleted_at IS NULL
ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'withdrawn'
AND <not_satisfied, the same clause pending used>
```
`row_uid` is `utils.generate_uid()`. The column list, the values, and the explicit `deleted_at: NULL,
deleted_by: NULL` are copied from `database.moderation.set_consent` (`database/moderation.py:300-315`) and must
stay in step with it; the test in clause I4 is what enforces that. Note `granted_at = :now` and
`withdrawn_at = ''` and not `NULL`: `set_consent` writes the empty string on the unused side, and clause I4
means matching it.
**Why this is not `set_consent`.** `set_consent` cannot express a precondition and cannot join a caller's
transaction, so reusing it would reintroduce exactly the race this statement closes and would break the
atomicity of the terms path. The duplication is one INSERT, it is confined to one private function
`_insert_consent_row(uid, kind, version, now)` used by both write shapes, and it is covered by a test that
compares it against `set_consent` field by field. This is a deliberate, documented divergence of the same kind
`claim_deletion` makes against a plain `table.update`.
### 8.3 Audit
Only on a won claim, and only with keys that already exist:
| Agreement | Event key | Fields |
|---|---|---|
| `terms` | `terms.accept` | `target_type="user"`, `target_uid`, `target_label=username`, `new_value=version` |
| the other four | `consent.grant` | as above, `new_value="granted"`, `metadata={"kind": kind}` |
Both carry `links=[audit.target("user", uid, username)]`, matching the human paths at
`routers/auth/terms.py:82` and `routers/profile/consent.py:84`.
Recorded through the lazily imported recorder, following `deletion.py:187`:
```python
from devplacepy.services.audit import record as audit
audit.record_system(
"terms.accept",
actor_kind="service",
actor_username="acceptance",
target_type="user",
...
)
```
`actor_role="system"` and `origin="service"` are already the defaults of `record_system`
(`services/audit/record.py:288-307`); `service` is a legal actor kind and a legal origin
(`services/audit/categories.py:57-58`). Both keys already exist in `events.md` (lines 69 and 72) and already
map to the `account` category. The recorder routes through `background.submit` and never raises into the
caller. Nothing is added to `events.md` except the new file name in the "Recorded in" column of those two
rows; the key count stated in the file header is unchanged.
The audit row is the only thing that distinguishes a converged acceptance from a human one, and it lives where
an operator looks and no code path reads.
### 8.4 Cache invalidation
`users.terms_version` is on the cached user row and the row is cached per worker for 300s
(`utils/authcache.py:6`), so a converged account would otherwise stay gated on the workers that already cached
it.
- A gate-column write requires the cross-worker `auth` version bump.
- A ledger-only write does not: `consent_granted` reads the table fresh on every call.
`clear_user_cache(uid)` bumps the global `auth` version by default (`utils/authcache.py:13-18`), and a version
bump makes **every** worker drop its **whole** user cache. Calling it per account would do that once per
converged user; a run of 200 accounts would do it 200 times. Instead: `run_once` performs exactly **one**
`database.bump_cache_version("auth")` at the end of the run, and only when at least one gate column actually
changed. One bump invalidates every worker's cache including the lock owner's, which is precisely the required
effect, at 1/200th of the cost. Terms acceptance gates writes, so it is an authorization change and must
propagate; it just does not need to propagate 200 times.
---
## 9. The service
`devplacepy/services/acceptance/service.py`.
```
name "acceptance"
title "Acceptance convergence"
default_enabled False
interval_seconds 300 (five minutes, as specified)
min_interval 60
METRICS_SECONDS 60
```
`description`, shown on `/admin/services` and its detail page, states plainly what it does and that it is for
a production-identical test instance, so an operator who finds it enabled on a real production host knows
immediately that it is wrong. `details` carries the decline-register rule, so the one thing an operator must
understand is on the page they are already looking at.
### 9.1 Configuration fields
| Key | Type | Default | Group | Purpose |
|---|---|---|---|---|
| `service_acceptance_enabled` | bool | `0` | General | Framework field. The master switch, off by boot default via `default_enabled = False`. |
| `service_acceptance_interval` | int | `300` | General | Framework field, floored at `min_interval`. |
| `acceptance_dry_run` | bool | `1` | Safety | Compute and log what would be converged; write nothing, record nothing. |
| `acceptance_batch_size` | int | `200` (min 1, max 5000) | Safety | Maximum accounts converged per agreement per run. |
| `acceptance_grant_terms` | bool | `0` | Agreements | One field per registry entry. |
| `acceptance_grant_privacy` | bool | `0` | Agreements | |
| `acceptance_grant_ai_third_party` | bool | `0` | Agreements | |
| `acceptance_grant_activity_recording` | bool | `0` | Agreements | |
| `acceptance_grant_container_credentials` | bool | `0` | Agreements | |
`setting_key(kind)` is `f"acceptance_grant_{kind}"`, defined once in `agreements.py` and used by the field
list, `run_once` and the tests. The per-agreement fields are **built from the registry**, not written out by
hand, the same way `PushService.config_fields` splices `*providers.admin_fields()`:
```python
config_fields = [
ConfigField(
setting_key(agreement.kind),
label_for(agreement.kind),
type="bool",
default=False,
help=CONSENT_KINDS[agreement.kind],
group="Agreements",
)
for agreement in AGREEMENTS
] + [ ...the two safety fields... ]
```
`ConfigField.read()` returns the typed default whenever the setting is unset or unparseable
(`services/base.py:81-89`), so a fresh install reads `False` for every agreement with no seeding in
`init_db` and no change to `operational_defaults`.
A sixth consent added to `CONSENT_KINDS` therefore appears in the admin form with no edit to the service. This
is what the request asked for: one option in the admin, per type of agreement.
Three gates stand between a fresh install and a single written row: the service is disabled, every agreement
is disabled, and dry run is on. All three must be changed deliberately.
### 9.2 `run_once`
```
config = self.get_config()
dry = config["acceptance_dry_run"]
limit = config["acceptance_batch_size"]
gate_changed = False
for agreement in AGREEMENTS:
if not config[setting_key(agreement.kind)]:
continue
version = current_version(agreement)
candidates = pending(agreement, limit)
if not candidates:
continue
if dry:
self.log(f"[dry run] {agreement.kind}: {len(candidates)} account(s) would be converged "
f"to version {version}: {sample_names(candidates)}")
continue
granted = 0
for user in candidates:
if converge_user(agreement, user, version):
granted += 1
gate_changed = gate_changed or bool(agreement.gate_column)
self.log(f"{agreement.kind}: converged {granted} of {len(candidates)} pending at version {version}")
if len(candidates) == limit:
self.log(f"{agreement.kind}: batch cap of {limit} reached, more remain for the next run")
if gate_changed:
bump_cache_version("auth")
```
Notes that are not incidental:
- **There is no `owns_lock()` guard and there must not be one.** `main.py:289-293` calls
`service_manager.supervise()` only in the worker that won `acquire_service_lock()`, so `run_once` is
physically unreachable on any other worker. `ModerationService` has no such guard either. Adding one would
be cargo cult; the routers that call `owns_lock()` do so because a *request* can land on any worker.
- The batch-cap log line is required by axiom A10. A silently truncated sweep reads as "everyone is converged"
when they are not.
- `converge_user` returning `False` is normal, not an error: it means the account was claimed by a real
acceptance or a withdrawal between the `SELECT` and the `UPDATE`. The count difference in the log is the
operator's visibility into that.
- `sample_names(candidates)` caps the dry-run listing at 20 usernames and appends the remaining count.
- No exception is swallowed here. `BaseService._execute_run` already wraps `run_once`, logs
`Error in run_once: {e}` to the service's buffer, and keeps the supervisor loop alive
(`services/base.py:307-313`).
### 9.3 `collect_metrics`
The detail page renders `{"stats": [...], "table": {"columns": [...], "rows": [...]}}`, the shape
`ContainerService` and `BotsService` already return.
| Stat | Source |
|---|---|
| `Dry run` | the config field, as `1`/`0` |
| `Agreements enabled` | count of registry entries whose field reads true |
| `<label> pending` | `len(pending(agreement, limit))`, suffixed `+` when it equals the limit |
| `<label> converged` | an in-memory counter on the service instance, since process start |
The table has one row per agreement: kind, label, enabled, versioned, pending, converged since start.
The pending counts reuse the same bounded `pending()` the run uses, deliberately: an unbounded `COUNT(*)` over
`users` with a correlated subquery per row is the kind of aggregate the framework tells you to keep out of the
tick. Reporting a saturated count as `200+` is honest and costs one already written query.
`METRICS_SECONDS = 60` keeps even that off the one-second tick, and `_safe_metrics` already swallows and logs
any exception so a metrics bug can never take the service down (`services/base.py:227-233`).
### 9.4 Registration
One line in the existing `lifespan` register block in `main.py`, and nothing else anywhere:
```python
service_manager.register(AcceptanceService())
```
Registered in every worker so `describe_all()` works on any of them, supervised only on the lock owner, like
every other service. This is the single import permitted by clause I5.
---
## 10. Backwards compatibility
"Works against a database that predates this feature" is a hard requirement. It has two halves: columns that
exist but hold `NULL`, and columns that do not exist at all.
| Situation | Rule |
|---|---|
| `users.terms_version` is `NULL` on every account created before the trust-and-safety commit; `init_db` ensures the column but deliberately never backfills it. | `COALESCE(u.terms_version, '') != :version`. A bare `!=` against `NULL` yields `NULL`, not true, and would silently converge nobody. |
| `users.is_active` **is not ensured by `init_db` at all** and only exists once a suspension or a deletion has written it. | Build the clause with `has_column`, per section 7.1. A raw reference to a missing column is an `OperationalError`, not a `NULL`. |
| `users.is_active` exists but is `NULL` on rows written before the first suspension. | `COALESCE(u.is_active, 1) != 0`, matching `is_account_active` exactly. |
| `users.deletion_requested_at` is `NULL` on every row that predates account deletion. | `COALESCE(u.deletion_requested_at, '') = ''`, the guard `claim_deletion` uses. |
| `user_consents` does not exist on a database that has never run the trust-and-safety `init_db`. | `if CONSENTS_TABLE not in db.tables: return []`. Fail closed. |
| A consent row written before a version was recorded carries `version = NULL`. | `COALESCE(c.version, '') != :version`, which correctly treats it as stale and re-grants at the current version. |
| `site_settings` has no `terms_version` row, or has one holding `""`. | `get_setting(key, "1") or "1"`. |
No migration, no backfill, no `init_db` change, no `operational_defaults` entry and no schema change of any
kind. The feature is additive to an untouched database, which is the same property that lets `/admin/trash`
restore an event written before the trash existed.
---
## 11. Correctness verification
This feature is a read-then-write mutation reachable from more than one request path, so the four-layer
procedure in the root `CLAUDE.md` applies in full, in addition to the persisted tests in section 12. All four
are disposable scripts run against a temp database via `DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR`, not pytest
files.
**The invariant, stated once:** *for any `(user, kind)` pair, a row written by the service never follows a
`withdrawn` row.* Everything below tests that one sentence.
### Layer 1: properties over the full input domain
`pending` and `satisfied_clause` are pure with respect to a given database state, so enumerate that state
rather than sampling it. Build the full matrix: ledger state in `{absent, granted, withdrawn}` x row version
in `{"", "1", "2"}` x gate column in `{NULL, "", "1", "2"}` x current version in `{"1", "2"}`, for each of the
five agreements. That is a few hundred cases and every one has a hand-derivable answer.
Assert:
- `pending` returns the account if and only if it is neither satisfied nor declined;
- **decline monotonicity**: once the latest row is `withdrawn`, no subsequent version bump, in either
direction, makes the account pending again;
- **version monotonicity**: for a non-declined account, changing the current version to a value different from
the recorded one always makes it pending, and back again always makes it satisfied;
- **gate agreement**: for every state in the matrix, `pending`'s verdict for `terms` equals
`routers.auth.terms.needs_acceptance` on the same row, and for the other four it equals
`database.consent_granted` combined with the version comparison. This is the clause that proves "never a
proxy" rather than asserting it.
Run the matrix twice: once on a table with an `is_active` column and once on a table without one, to cover
both branches of section 7.1.
### Layer 2: stateful fuzzing
Fifty accounts, a few thousand randomized actions drawn from: bump `terms_version`, bump `privacy_version`,
human grant, human withdraw, run the service, toggle an agreement field, toggle the master switch, toggle dry
run. Re-check after **every** action:
- no `granted` row written by the service follows a `withdrawn` row for the same pair;
- with the master switch off, with the agreement off, or with dry run on, the ledger and the `users` row are
byte-identical before and after the run;
- every row the service wrote has exactly the column set `set_consent` produces, with the same empty-string
convention on `granted_at`/`withdrawn_at`;
- the number of `terms.accept` and `consent.grant` audit rows with `actor_kind = "service"` equals the number
of rows the service actually wrote, never more.
Catch the expected exception types and continue; the goal is invariant violations and unexpected exceptions.
### Layer 3: concurrency, with real processes
Sixteen real OS processes, never threads: `dataset` gives each thread its own pooled connection and enough
threads produce `database is locked` noise that is a harness artifact, not a finding.
Two scenarios, each on a genuinely fresh account whose columns are left exactly as a real signup leaves them.
**Do not pre-seed or zero `terms_version`, `is_active` or `deletion_requested_at`.** Pre-zeroing a column that
production leaves `NULL` is precisely the mistake that made an earlier race-safety pass in this codebase pass
while hiding the bug it was written to catch.
1. **Convergence storm.** Sixteen processes call `converge_user` for the same account and agreement. Exactly
one wins. Exactly one ledger row is written. Exactly one audit row is recorded.
2. **Grant against withdrawal.** Eight processes converge, eight call `set_consent(..., granted=False)`. Scan
the resulting ledger ordered by `created_at, id` and assert no service-written `granted` row follows a
`withdrawn` row. Since only the service grants in this scenario, the invariant is exact.
### Layer 4: static analysis
`pyflakes` and `ruff check` on every touched file. `py_compile` and a clean
`python -c "from devplacepy.main import app"` prove syntax and import order only; neither catches a missing
import inside a function body, and this design uses lazy imports in three functions.
---
## 12. Persisted tests
Tier is decided by fixtures, and the paths mirror the source module path. `tests/unit/services/moderation/`
and `tests/api/admin/services/` already exist and set the pattern.
| File | Tier | Covers |
|---|---|---|
| `tests/unit/services/acceptance/__init__.py` | - | package |
| `tests/unit/services/acceptance/agreements.py` | unit | registry completeness against `CONSENT_KINDS` (the enforcement test), unique setting keys, every agreement has a config field, labels resolve |
| `tests/unit/services/acceptance/pending.py` | unit | `local_db`: the satisfied and declined matrix per shape, `NULL` legacy rows, the missing-`is_active` branch, missing-table fail-closed, batch bound, stable ordering |
| `tests/unit/services/acceptance/grant.py` | unit | `local_db`: a won claim writes the gate column and the ledger row; a lost claim writes nothing; a withdrawn ledger is never converged; the written row matches `set_consent` field for field (clause I4); the audit row carries `actor_kind="service"` and an existing event key |
| `tests/unit/services/acceptance/isolation.py` | unit | greps the source tree and asserts `devplacepy/main.py` is the only importer (clause I5), and that no template, schema, router or Devii catalog file mentions the package |
| `tests/api/admin/services/acceptance.py` | api | the service is listed at `/admin/services`, reports `stopped` on a fresh instance, its detail page renders the per-agreement fields, saving the config validates and persists, and a non-admin is redirected |
`tests/unit/services/acceptance/isolation.py` is the test that makes the invisibility contract survive future
edits. Without it, clause I5 is a promise; with it, it is a build failure.
**Do not write a test that enables the service and asserts convergence in the shared suite.** The suite runs
serially against one seeded database, and a service that grants `terms` to every account would poison the
moderation tests that assert the gate refuses. Convergence behaviour is covered by the unit tests calling
`converge_user` directly against `local_db`, which is where it belongs. The api test asserts the admin
surface only, and must leave `service_acceptance_enabled` untouched.
---
## 13. Operator procedures
To be written into the admin-only docs page, section 14.
**Enable it.**
1. `/admin/services/acceptance`, Configuration tab: switch on only the agreements the test run needs.
2. Leave dry run on. Start the service. Wait one interval, or use Run now. Read the Logs tab and confirm the
accounts listed are the ones expected.
3. Switch dry run off. The next run converges them.
**Test the refusal path for a consent.** From the tester's own profile, privacy tab, withdraw the consent. It
is never granted again while that withdrawal stands, regardless of how many intervals pass.
**Test the terms gate.** The tester withdraws their own `terms` consent, then an administrator bumps
`terms_version` at `/admin/settings`. Every other account converges; the tester stays gated.
**Test a partial state.** Enable `terms` and leave `privacy` off. Accounts pass the write gate but still show
the privacy policy as unaccepted on their privacy tab, which is the edge case the per-agreement switches
exist for.
**Undo a decline.** Grant the consent from the profile page. The ledger's latest row is `granted` again and
the account leaves the pending set; nothing to reset and no operator action on the service.
**Turn it off.** Stop the service, or switch off the individual agreement. Nothing already written is
reverted, and nothing should be: the accounts are in exactly the state a real population would have reached.
**Never enable it on production.** The description on the service page says so, and this is the only control
that matters. There is no environment detection anywhere in the design, deliberately: an environment flag
would be exactly the knowledge the application is not allowed to have.
---
## 14. Documentation
| File | Change |
|---|---|
| `devplacepy/services/acceptance/CLAUDE.md` | New. The nested subsystem doc: why the subsystem exists, the invisibility contract, the decline register and its ordering rule, the module map, the exclusions from section 4.1, the `has_column` rule from section 7.1, the single cache bump, the rules for extending it. |
| `CLAUDE.md` (root) | One row in the Subsystem map table. No architectural rule is introduced, so nothing else changes. |
| `devplacepy/services/CLAUDE.md` | One short section, sized and placed like the "Moderation housekeeping" section it sits beside. |
| `events.md` | Add `services/acceptance/grant.py` to the "Recorded in" column of `terms.accept` (line 72) and `consent.grant` (line 69). No new key, no category count change, no change to the header's key count. |
| `README.md` | One subsection under the background services material, stating that it is admin-only, off by default, and what it is for. |
| `devplacepy/templates/docs/moderation-operations.html` | One section carrying the procedures from section 13. The page is already `"admin": True` in `DOCS_PAGES` (`routers/docs/pages.py:195-199`), so it is invisible to members with no gating work. |
No `docs_api.py` entry and no Devii action, because there is no endpoint and no user-facing capability. This
is the one deliberate exemption from the four-faces rule in the root `CLAUDE.md`, and it is the exemption
every pure `BaseService` already takes: `NotificationRelayService`, `PresenceRelayService`,
`LiveViewRelayService` and `AuditService` have no routes, no schemas and no agent tools either. Their surface
is the generic services admin, which this service inherits without writing a line of template or JavaScript.
Exposing a Devii tool for it would additionally hand the assistant, which runs inside the application, direct
knowledge of the feature, breaking clause I5 for no benefit.
---
## 15. Definition of done
In order. A failure at any step blocks the next.
1. `python -c "from devplacepy.main import app"` imports clean.
2. `pyflakes` and `ruff check` clean on every touched file.
3. No em dash, neither the character nor the HTML entity, in any touched file.
4. The four verification layers of section 11 run and pass, including the sixteen-process race and both
branches of the missing-column matrix.
5. `make test` passes in full: unit, api and e2e, every test, no tier skipped and no subset substituted.
6. A manual check on a dev instance: with everything default, `/admin/services/acceptance` shows `stopped` and
the database is untouched after ten minutes. With the service started, dry run on and `terms` enabled, the
log lists the pending accounts and the database is still untouched. With dry run off, the accounts converge
within one interval, the audit log shows `terms.accept` rows with actor kind `service`, and an account
whose `terms` consent was withdrawn is still gated.
---
## 16. Non-goals
- No environment variable, no build flag, no `ACCEPTANCE_MODE` of any kind. The service is the entire
mechanism.
- No fabrication of declared facts: age band and date of birth are never written.
- No new consent kind, no new agreement outside `CONSENT_KINDS`, and no second decline register.
- No convergence of guests, of deleted or anonymised accounts, or of deactivated accounts.
- No reversal path. The service grants; only a human withdraws.
- No route, no JSON schema, no template, no Devii tool, no API documentation entry.

View File

@ -119,6 +119,7 @@ from devplacepy.services.xmlrpc import XmlrpcService
from devplacepy.services.audit import AuditService from devplacepy.services.audit import AuditService
from devplacepy.services.moderation.service import ModerationService from devplacepy.services.moderation.service import ModerationService
from devplacepy.services.moderation.screening import ContentRefused from devplacepy.services.moderation.screening import ContentRefused
from devplacepy.services.acceptance.service import AcceptanceService
from devplacepy.services.audit import record as audit from devplacepy.services.audit import record as audit
from devplacepy.services.push import PushService from devplacepy.services.push import PushService
from devplacepy.services.telegram import TelegramService from devplacepy.services.telegram import TelegramService
@ -283,6 +284,7 @@ async def lifespan(app: FastAPI):
service_manager.register(XmlrpcService()) service_manager.register(XmlrpcService())
service_manager.register(AuditService()) service_manager.register(AuditService())
service_manager.register(ModerationService()) service_manager.register(ModerationService())
service_manager.register(AcceptanceService())
service_manager.register(PushService()) service_manager.register(PushService())
service_manager.register(TelegramService()) service_manager.register(TelegramService())
service_manager.register(TelegramOutboxService()) service_manager.register(TelegramOutboxService())

View File

@ -118,6 +118,10 @@ devplace devii reset-quota --all # Reset every quota (users and guests)
`ModerationService` is a lock-owner `BaseService` (default-enabled, hourly, floor 300s) with two jobs: it purges accounts whose deletion grace window has closed (`deletion.purge_due`, the same code path as `devplace accounts prune`), and it reports the moderation queue's service-level snapshot - logging when a report is past the published response window and exposing the queue counts, the oldest open age and the pending-purge count as `collect_metrics` stat cards. It owns no request-path work; the queue itself is entirely synchronous. Full subsystem detail in `devplacepy/services/moderation/CLAUDE.md`. `ModerationService` is a lock-owner `BaseService` (default-enabled, hourly, floor 300s) with two jobs: it purges accounts whose deletion grace window has closed (`deletion.purge_due`, the same code path as `devplace accounts prune`), and it reports the moderation queue's service-level snapshot - logging when a report is past the published response window and exposing the queue counts, the oldest open age and the pending-purge count as `collect_metrics` stat cards. It owns no request-path work; the queue itself is entirely synchronous. Full subsystem detail in `devplacepy/services/moderation/CLAUDE.md`.
## Acceptance convergence (`services/acceptance/service.py`)
`AcceptanceService` is a lock-owner `BaseService` (**opt-in**, `default_enabled = False`, five minutes, floor 60s) that grants every policy agreement to every account which has not declined it, so a production-identical instance used for manual testing never interrupts with an acceptance dialog. It is invisible to the rest of the application by contract: one registration line in `main.py` is the only import anywhere, and there is no route, schema, template, Devii tool or environment flag. The decline register is the consent ledger itself - the service only ever writes `granted`, so any `withdrawn` row was written by a human and that pair is never touched again. Three gates stand between a fresh install and a single written row (service disabled, every agreement disabled, dry run on). Full subsystem detail in `devplacepy/services/acceptance/CLAUDE.md`; the design record is `accept.md` at the repository root.
## Multi-worker concurrency (preferred rules) ## Multi-worker concurrency (preferred rules)
`uvicorn --workers N` = N independent processes sharing only the filesystem and SQLite DB. Module-global caches/counters are per-process, so a local `clear()` is invisible to siblings. Full reference: admin docs `Production -> Multi-worker and concurrency` (`templates/docs/production-concurrency.html`). Enforce these: `uvicorn --workers N` = N independent processes sharing only the filesystem and SQLite DB. Module-global caches/counters are per-process, so a local `clear()` is invisible to siblings. Full reference: admin docs `Production -> Multi-worker and concurrency` (`templates/docs/production-concurrency.html`). Enforce these:

View File

@ -0,0 +1,65 @@
# CLAUDE.md
This file documents the acceptance convergence subsystem (`devplacepy/services/acceptance/`). Claude Code auto-loads it whenever a file under this directory is read or edited. The full design record is [`accept.md`](../../../accept.md) at the repository root.
## Why this subsystem exists
An operator running a production-identical instance for extended manual testing is otherwise taxed forever by the platform's own safety controls: five consents, a versioned terms gate on every mutating request, and every account predating the trust-and-safety commit reading `terms_version = NULL` because `init_db` deliberately never backfills it. This service converges each account onto the acceptance state a real population would have produced itself, so the instance stays production byte for byte while nobody has to click the same dialog again.
It is off by default and it is never appropriate on a real production host.
## The two load-bearing ideas
**The application must not know.** There is no request-path branch, no schema, no route, no template, no Jinja global, no Devii tool and no environment flag. The only import of this package anywhere is the one registration line in `main.py`, and `tests/unit/services/acceptance/isolation.py` fails the suite if a second one appears. An environment flag would be exactly the knowledge the application is not allowed to have, which is why there is none.
**The decline register needs no storage.** `user_consents` is append-only in effect, so the latest live row for a `(user, kind)` pair already is the register. The service only ever writes `granted`; it follows that any `withdrawn` row in the ledger was written by a human, and the service never touches that pair again. No provenance column, no marker, no flag, and nothing for the application to observe.
## Module map
| File | Owns |
|---|---|
| `agreements.py` | `Agreement`, `AGREEMENTS`, `setting_key`, `label_for`, `agreement_for` |
| `pending.py` | `latest_consent`, `not_withdrawn`, `satisfied_clause`, `live_account_clauses`, `current_version`, `pending` |
| `grant.py` | `converge_user` plus the two private claim shapes and the audit call |
| `service.py` | `AcceptanceService`: config fields built from the registry, `run_once`, `collect_metrics` |
`pending.py` and `grant.py` import only `devplacepy.database` and `sqlalchemy` at module top; `generate_uid` and the audit recorder are imported lazily inside the functions that use them, mirroring `services/moderation/deletion.py`.
## The registry is the completeness guarantee
`AGREEMENTS` annotates `database.CONSENT_KINDS` with two facts: which `site_settings` key holds the policy version, and which `users` column the application's own gate reads. `terms` is the only agreement with a gate column, because `needs_acceptance` reads `users.terms_version` and not the ledger; `privacy` is versioned but ledger-only, matching `VERSION_KEYS` in `routers/profile/consent.py`.
A unit test asserts `{a.kind for a in AGREEMENTS} == set(CONSENT_KINDS)`. **A sixth consent fails the suite until it is classified here**, and it then appears in the admin form with no edit to the service, because the per-agreement config fields are built from the registry rather than written out by hand.
## Rules that must not regress
- **Satisfaction is the gate's own expression, never a proxy.** `pending` compares exactly what the application compares: `users.terms_version` against `get_setting("terms_version", "1") or "1"` for `terms`, the ledger row's `version` for `privacy`, the latest state for the other three. The `or "1"` is load-bearing: an admin settings save can write `terms_version = ""`, and a bare `get_setting` would make every account pending forever.
- **Order by `created_at DESC, id DESC`, never one of the two.** That pair is what `database.consent_state` selects, so it is the expression the gate evaluates. `consent_view` on the privacy tab orders by `created_at` alone; that is the display path, not a gate. Never introduce a third ordering.
- **Build the live-account clauses with `has_column`.** `init_db` ensures `terms_version`, `terms_accepted_at` and `deletion_requested_at` on `users`, but **not `is_active`** - that column is created implicitly the first time a suspension or a deletion writes it, so a hardcoded reference raises `no such column` on an instance where nobody was ever suspended. An absent column means no account can be in that state, so omitting the clause is the correct answer.
- **Every write is one conditional statement decided on the driver's real `rowcount`**, via `db.executable.execute(text(...), params).rowcount` inside `with db:`, exactly like `deletion.claim_deletion`. Sixteen real processes racing one account produce exactly one ledger row and one audit row.
- **Never write `updated_at` on `users`.** That is why `database.atomic.conditional_update_row` cannot be reused here: it appends `updated_at` unconditionally, the table has no such column, and creating one would make the service's rows distinguishable from the route's.
- **The ledger insert must stay byte-compatible with `set_consent`**, including `withdrawn_at = ''` rather than `NULL` on the unused side. A unit test compares the two field by field. It is not `set_consent` itself only because `set_consent` cannot express a precondition or join a caller's transaction.
- **One cache bump per run, not one per account.** `clear_user_cache` propagates a global `auth` version bump that makes every worker drop its whole user cache; `run_once` bumps once at the end, and only when a gate column actually changed.
- **The audit row is deliberate.** It uses the existing `terms.accept` and `consent.grant` keys with `actor_kind="service"`, `actor_username="acceptance"`. The audit log is the operator's record and no code path reads it, so it costs nothing in invisibility and is the only trace distinguishing a converged acceptance from a human one.
## What is deliberately not an agreement
| Excluded | Why |
|---|---|
| `users.age_band` | A declaration of fact, not an agreement. Fabricating a declared age would silently unlock `restricted` content for an account that declared 13-15. Sign the test account up with an adult date of birth instead. |
| `users.mature_opt_in` | A preference gated by the age band, with no ledger row and therefore no decline register. The site setting `moderation_mature_default_hidden` already turns the interstitial off instance-wide. |
| Guest consents | Nothing writes a consent row with `owner_kind = "guest"`, and the gateway resolves a guest to owner kind `anonymous`, which `consent_denied` exempts by design. |
| Preferences (`interactions_enabled`, notifications, customization) | None of them blocks anything. |
## The terms asymmetry, and the one operator step
`POST /auth/accept-terms` grants **two** agreements: `terms` and `privacy`. The service keeps them independent on purpose, because the per-agreement switches exist for edge-case testing; enabling both reproduces the human path exactly, enabling one is a deliberate divergence.
Withdrawing the `terms` consent does not clear `users.terms_version`, so a tester who wants to exercise the gate withdraws their own `terms` consent and then has an administrator bump `terms_version` at `/admin/settings`. Every other account converges within one interval; the tester stays gated indefinitely.
## Rules for extending this
- A new consent kind: classify it in `AGREEMENTS` (the test forces this), and nothing else.
- Never add a route, a schema, a template, a Devii tool or a `docs_api` entry. The surface is the generic services admin, exactly as for `NotificationRelayService` and `AuditService`.
- Never add a second decline register, a second ordering, or an environment check.
- Never write a persisted test that enables the service and asserts convergence: the suite is serial against one seeded database, and granting `terms` to every account would poison the moderation tests that assert the gate refuses. Convergence is covered by unit tests calling `converge_user` directly.

View File

@ -0,0 +1 @@
# retoor <retoor@molodetz.nl>

View File

@ -0,0 +1,37 @@
# retoor <retoor@molodetz.nl>
from dataclasses import dataclass
from devplacepy.database import CONSENT_KINDS
@dataclass(frozen=True)
class Agreement:
kind: str
version_setting: str
gate_column: str
stamp_column: str
AGREEMENTS: tuple[Agreement, ...] = (
Agreement("terms", "terms_version", "terms_version", "terms_accepted_at"),
Agreement("privacy", "privacy_version", "", ""),
Agreement("ai_third_party", "", "", ""),
Agreement("activity_recording", "", "", ""),
Agreement("container_credentials", "", "", ""),
)
def setting_key(kind: str) -> str:
return f"acceptance_grant_{kind}"
def label_for(kind: str) -> str:
return CONSENT_KINDS.get(kind, kind)
def agreement_for(kind: str) -> Agreement | None:
for agreement in AGREEMENTS:
if agreement.kind == kind:
return agreement
return None

View File

@ -0,0 +1,130 @@
# retoor <retoor@molodetz.nl>
from sqlalchemy import text
from devplacepy.database import CONSENTS_TABLE, _now_iso, db, get_table
from .agreements import Agreement
from .pending import has_gate_column, not_withdrawn, satisfied_clause
CONSENT_COLUMNS = (
"uid",
"owner_kind",
"owner_id",
"kind",
"version",
"state",
"granted_at",
"withdrawn_at",
"created_at",
"deleted_at",
"deleted_by",
)
def _gate_set_clause(agreement: Agreement) -> str:
users = get_table("users")
assignments = [f"{agreement.gate_column} = :version"]
if agreement.stamp_column and users.has_column(agreement.stamp_column):
assignments.append(f"{agreement.stamp_column} = :now")
return ", ".join(assignments)
def _insert_consent_sql(precondition: str) -> str:
columns = ", ".join(CONSENT_COLUMNS)
sql = (
f"INSERT INTO {CONSENTS_TABLE} ({columns}) "
"SELECT :row_uid, 'user', :uid, :kind, :version, 'granted', "
":now, '', :now, NULL, NULL"
)
if precondition:
return f"{sql} WHERE {precondition}"
return sql
def _consent_params(agreement: Agreement, uid: str, version: str, now: str) -> dict:
from devplacepy.utils import generate_uid
return {
"row_uid": generate_uid(),
"uid": uid,
"kind": agreement.kind,
"version": version,
"now": now,
}
def _claim_gate_column(agreement: Agreement, uid: str, version: str, now: str) -> bool:
claim = (
f"UPDATE users SET {_gate_set_clause(agreement)} "
"WHERE uid = :uid "
f"AND COALESCE({agreement.gate_column}, '') != :version "
f"AND {not_withdrawn('users.uid')}"
)
params = _consent_params(agreement, uid, version, now)
with db:
claimed = db.executable.execute(text(claim), params).rowcount
if claimed != 1:
return False
db.executable.execute(text(_insert_consent_sql("")), params)
return True
def _claim_ledger(agreement: Agreement, uid: str, version: str, now: str) -> bool:
precondition = (
f"{not_withdrawn(':uid')} AND {satisfied_clause(agreement, ':uid', '')}"
)
params = _consent_params(agreement, uid, version, now)
with db:
written = db.executable.execute(
text(_insert_consent_sql(precondition)), params
).rowcount
return written == 1
def _record(agreement: Agreement, user: dict, version: str) -> None:
from devplacepy.services.audit import record as audit
uid = user["uid"]
username = user.get("username") or ""
links = [audit.target("user", uid, username)]
if agreement.kind == "terms":
audit.record_system(
"terms.accept",
actor_kind="service",
actor_username="acceptance",
target_type="user",
target_uid=uid,
target_label=username,
new_value=version,
summary=f"{username} accepted terms version {version}",
links=links,
)
return
audit.record_system(
"consent.grant",
actor_kind="service",
actor_username="acceptance",
target_type="user",
target_uid=uid,
target_label=username,
new_value="granted",
metadata={"kind": agreement.kind},
summary=f"granted {agreement.kind} consent for {username}",
links=links,
)
def converge_user(agreement: Agreement, user: dict, version: str) -> bool:
uid = user.get("uid")
if not uid:
return False
now = _now_iso()
if has_gate_column(agreement):
won = _claim_gate_column(agreement, uid, version, now)
else:
won = _claim_ledger(agreement, uid, version, now)
if not won:
return False
_record(agreement, user, version)
return True

View File

@ -0,0 +1,78 @@
# retoor <retoor@molodetz.nl>
from devplacepy.database import CONSENTS_TABLE, db, get_setting, get_table
from .agreements import Agreement
DEFAULT_VERSION = "1"
LIVE_ACCOUNT_CLAUSES: tuple[tuple[str, str], ...] = (
("deletion_requested_at", "COALESCE(u.deletion_requested_at, '') = ''"),
("is_active", "COALESCE(u.is_active, 1) != 0"),
("deleted_at", "u.deleted_at IS NULL"),
)
def latest_consent(column: str, user_ref: str) -> str:
return (
f"COALESCE((SELECT c.{column} FROM {CONSENTS_TABLE} c "
f"WHERE c.owner_kind = 'user' AND c.owner_id = {user_ref} "
"AND c.kind = :kind AND c.deleted_at IS NULL "
"ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '')"
)
def not_withdrawn(user_ref: str) -> str:
return f"{latest_consent('state', user_ref)} != 'withdrawn'"
def has_gate_column(agreement: Agreement) -> bool:
if not agreement.gate_column:
return False
if "users" not in db.tables:
return False
return get_table("users").has_column(agreement.gate_column)
def satisfied_clause(agreement: Agreement, user_ref: str, column_prefix: str) -> str:
if has_gate_column(agreement):
return f"COALESCE({column_prefix}{agreement.gate_column}, '') != :version"
if agreement.version_setting:
return f"{latest_consent('version', user_ref)} != :version"
return f"{latest_consent('state', user_ref)} != 'granted'"
def live_account_clauses() -> list[str]:
users = get_table("users")
return [
clause for column, clause in LIVE_ACCOUNT_CLAUSES if users.has_column(column)
]
def current_version(agreement: Agreement) -> str:
if not agreement.version_setting:
return DEFAULT_VERSION
return get_setting(agreement.version_setting, DEFAULT_VERSION) or DEFAULT_VERSION
def pending(agreement: Agreement, limit: int) -> list[dict]:
if "users" not in db.tables or CONSENTS_TABLE not in db.tables:
return []
bound = max(1, int(limit))
clauses = [
*live_account_clauses(),
not_withdrawn("u.uid"),
satisfied_clause(agreement, "u.uid", "u."),
]
sql = (
"SELECT u.uid, u.username FROM users u "
f"WHERE {' AND '.join(clauses)} "
"ORDER BY u.id LIMIT :limit"
)
rows = db.query(
sql,
kind=agreement.kind,
version=current_version(agreement),
limit=bound,
)
return [dict(row) for row in rows]

View File

@ -0,0 +1,168 @@
# retoor <retoor@molodetz.nl>
import logging
from devplacepy.database import CONSENT_KINDS, bump_cache_version
from devplacepy.services.acceptance.agreements import (
AGREEMENTS,
label_for,
setting_key,
)
from devplacepy.services.acceptance.grant import converge_user
from devplacepy.services.acceptance.pending import current_version, pending
from devplacepy.services.base import BaseService, ConfigField
logger = logging.getLogger(__name__)
DRY_RUN_KEY = "acceptance_dry_run"
BATCH_SIZE_KEY = "acceptance_batch_size"
DEFAULT_BATCH_SIZE = 200
MAX_BATCH_SIZE = 5000
SAMPLE_NAMES = 20
def sample_names(candidates: list[dict]) -> str:
names = [row.get("username") or row.get("uid") or "" for row in candidates]
shown = ", ".join(names[:SAMPLE_NAMES])
remaining = len(names) - SAMPLE_NAMES
if remaining > 0:
return f"{shown} and {remaining} more"
return shown
class AcceptanceService(BaseService):
title = "Acceptance convergence"
description = (
"Grants every policy agreement to every account that has not declined it, "
"so a production-identical test instance never interrupts manual testing "
"with an acceptance dialog. Off by default and never appropriate on a real "
"production host."
)
details = (
"An account that withdrew a consent is never granted it again, with no "
"further action: the latest row in the consent ledger is the decline "
"register. The service only ever grants, so any withdrawal in the ledger "
"was written by a human. Withdraw a consent from the profile privacy tab "
"to keep an account permanently outside the convergence."
)
default_enabled = False
min_interval = 60
METRICS_SECONDS = 60
config_fields = [
ConfigField(
DRY_RUN_KEY,
"Dry run",
type="bool",
default=True,
help=(
"Log which accounts would be converged and write nothing. Switch "
"off only after the log lists what you expect."
),
group="Safety",
),
ConfigField(
BATCH_SIZE_KEY,
"Accounts per agreement per run",
type="int",
default=DEFAULT_BATCH_SIZE,
minimum=1,
maximum=MAX_BATCH_SIZE,
help="Upper bound on one sweep. The remainder converges on the next run.",
group="Safety",
),
*[
ConfigField(
setting_key(agreement.kind),
label_for(agreement.kind),
type="bool",
default=False,
help=f"Grant the {agreement.kind} agreement to every account that has not declined it.",
group="Agreements",
)
for agreement in AGREEMENTS
],
]
def __init__(self) -> None:
super().__init__("acceptance", interval_seconds=300)
self._converged = {agreement.kind: 0 for agreement in AGREEMENTS}
async def run_once(self) -> None:
config = self.get_config()
dry = config[DRY_RUN_KEY]
limit = config[BATCH_SIZE_KEY]
gate_changed = False
for agreement in AGREEMENTS:
if not config[setting_key(agreement.kind)]:
continue
version = current_version(agreement)
candidates = pending(agreement, limit)
if not candidates:
continue
if dry:
self.log(
f"[dry run] {agreement.kind}: {len(candidates)} account(s) would be "
f"converged to version {version}: {sample_names(candidates)}"
)
continue
granted = 0
for user in candidates:
if converge_user(agreement, user, version):
granted += 1
gate_changed = gate_changed or bool(agreement.gate_column)
self._converged[agreement.kind] += granted
self.log(
f"{agreement.kind}: converged {granted} of {len(candidates)} pending "
f"at version {version}"
)
if len(candidates) == limit:
self.log(
f"{agreement.kind}: batch cap of {limit} reached, more remain "
"for the next run"
)
if gate_changed:
bump_cache_version("auth")
def collect_metrics(self) -> dict:
config = self.get_config()
limit = config[BATCH_SIZE_KEY]
enabled = [
agreement
for agreement in AGREEMENTS
if config[setting_key(agreement.kind)]
]
stats = [
{"label": "Dry run", "value": 1 if config[DRY_RUN_KEY] else 0},
{"label": "Agreements enabled", "value": len(enabled)},
]
rows = []
for agreement in AGREEMENTS:
is_enabled = config[setting_key(agreement.kind)]
count = len(pending(agreement, limit)) if is_enabled else 0
display = f"{count}+" if is_enabled and count == limit else str(count)
if is_enabled:
stats.append(
{"label": f"{agreement.kind} pending", "value": display}
)
rows.append(
[
agreement.kind,
CONSENT_KINDS.get(agreement.kind, agreement.kind)[:48],
1 if is_enabled else 0,
agreement.version_setting or "",
display if is_enabled else "",
self._converged[agreement.kind],
]
)
table = {
"columns": [
"Agreement",
"Policy",
"Enabled",
"Version setting",
"Pending",
"Converged",
],
"rows": rows,
}
return {"stats": stats, "table": table}

View File

@ -100,4 +100,38 @@ The operator transcribes these into the store listing; every answer is a fact ab
A wrong removal is undone with the **Restore content** decision, or from `/admin/trash`, which A wrong removal is undone with the **Restore content** decision, or from `/admin/trash`, which
restores the whole deletion event under one stamp. Nothing a moderator removes is destroyed until it restores the whole deletion event under one stamp. Nothing a moderator removes is destroyed until it
is purged. is purged.
## Acceptance convergence on a test instance
An instance kept production-identical for extended manual testing otherwise stops the tester at the
same acceptance dialogs forever. The **Acceptance convergence** background service grants every
policy agreement to every account that has not declined it. It is administrator-only, **off by
default**, and **never appropriate on a real production host** - there is no environment detection
anywhere in it, deliberately, so the only control is the operator's own judgement.
**Enable it.**
1. Open `/admin/services/acceptance`, Configuration tab, and switch on only the agreements the test
run needs. There is one switch per agreement type, so a single policy can be converged while the
rest stay pending.
2. Leave **Dry run** on. Start the service, then use **Run now**. Read the Logs tab and confirm the
accounts listed are the ones expected. Nothing has been written yet.
3. Switch Dry run off. The next run converges them, within five minutes or immediately with
**Run now**.
**Test a refusal path.** Withdraw the consent from your own profile's privacy tab. The consent
ledger is the decline register: the service only ever grants, so a withdrawal is permanent and no
number of intervals will undo it. Grant it again from the same tab to rejoin the convergence.
**Test the terms gate.** Withdraw your own Terms of Service consent, then bump `terms_version` at
`/admin/settings`. Every other account converges; you stay gated and can exercise
`/auth/accept-terms`, the in-place acceptance dialog and the JSON refusal as often as you like.
**Test a partial state.** Enable Terms of Service and leave Privacy Policy off. Accounts pass the
write gate but still show the privacy policy as unaccepted on their privacy tab.
**Turn it off.** Stop the service, or switch off the individual agreement. Nothing already written
is reverted, and nothing should be: those accounts are in exactly the state a real population would
have reached. Every convergence is in the audit log under `terms.accept` and `consent.grant` with
actor kind `service`, which is how you tell a converged acceptance from a human one.
</div> </div>

View File

@ -66,10 +66,10 @@ Every state-changing action in DevPlace records one append-only row through `dev
| `profile.update` | `routers/devrant/auth.py`, `routers/profile/index.py` | | `profile.update` | `routers/devrant/auth.py`, `routers/profile/index.py` |
| `account.delete.purge` | `services/moderation/deletion.py` | | `account.delete.purge` | `services/moderation/deletion.py` |
| `account.delete.request` | `routers/devrant/auth.py`, `routers/profile/delete.py` | | `account.delete.request` | `routers/devrant/auth.py`, `routers/profile/delete.py` |
| `consent.grant` | `routers/profile/consent.py` | | `consent.grant` | `routers/profile/consent.py`, `services/acceptance/grant.py` |
| `profile.mature_content` | `routers/profile/consent.py` | | `profile.mature_content` | `routers/profile/consent.py` |
| `consent.withdraw` | `routers/profile/consent.py` | | `consent.withdraw` | `routers/profile/consent.py` |
| `terms.accept` | `routers/auth/terms.py` | | `terms.accept` | `routers/auth/terms.py`, `services/acceptance/grant.py` |
## Administration (`admin`) ## Administration (`admin`)

6
lensfl.md Normal file
View File

@ -0,0 +1,6 @@
Oke, we do have a system that has a lot of privacy and terms and ccondition options. That is perffect. But while extensiive manually testing, do not want to be bottered ever. Especially not on accceptence mode. Acceptence mode is like producton, but every terms and conditon will be applied automatically using a typical devplacce sercive with backwards compattebility, it shouuld run every five minutes and ensure that all termss and conditon are agreed to by literally every user unless manually declined.
To be profressonal, this feature is not allowed to be traced to the database because of custom fields sor whatever, remember, the system is in acceptnce mode what should literally be production mode but with that side effect. Of coursse, by default it must be turned off. It has to be put manually enabled because it would be a nightmre if it was triggered on production.
The only way to satissfy this is a isolated sservice in our appliction like the rest, but the 0ther applcation is not allowed to know anything about it. It just ensures severy five minutes that all users did comply to every type of thiingy for real produuction simuulaton.
It has to be implemented as single option in the admin but per type of `agreemenet` to be abble to test edge cases. Please do create a design that would fit fine like other servicess implemented defaults and servicecs.
Your only task for now is to creatae a full implementaton document of this acceptence mode. But again, the system is not allowed to know that it is running in acceptance mode, it would canccel the whole point and principle.
Now, generate the docuument called accept.md in detail conform our appliaton guidelines spread everywhere. Consistency and dry is key to success. Youre the best young man. Like if Lensflare would be an LLM and shit.

View File

@ -0,0 +1,89 @@
# retoor <retoor@molodetz.nl>
import requests
from tests.conftest import BASE_URL
from devplacepy.database import (
clear_settings_cache,
get_setting,
get_table,
refresh_snapshot,
set_setting,
)
from devplacepy.services.acceptance.agreements import AGREEMENTS, setting_key
JSON = {"Accept": "application/json"}
SERVICE_URL = f"{BASE_URL}/admin/services/acceptance"
def _admin(seeded_db):
refresh_snapshot()
key = get_table("users").find_one(username="alice_test")["api_key"]
session = requests.Session()
session.headers.update({"X-API-KEY": key, **JSON})
return session
def test_the_service_is_listed_and_stopped_by_default(seeded_db):
admin = _admin(seeded_db)
r = admin.get(f"{BASE_URL}/admin/services/data")
assert r.status_code == 200, r.text[:300]
entry = next(
item for item in r.json()["services"] if item["name"] == "acceptance"
)
assert entry["enabled"] is False
assert entry["status"] == "stopped"
assert entry["title"] == "Acceptance convergence"
def test_the_detail_page_renders_one_field_per_agreement(seeded_db):
admin = _admin(seeded_db)
r = admin.get(f"{SERVICE_URL}/data")
assert r.status_code == 200, r.text[:300]
fields = {field["key"]: field for field in r.json()["service"]["fields"]}
for agreement in AGREEMENTS:
field = fields[setting_key(agreement.kind)]
assert field["type"] == "bool"
assert field["value"] == "0"
assert field["group"] == "Agreements"
assert fields["acceptance_dry_run"]["value"] == "1"
assert fields["service_acceptance_enabled"]["value"] == "0"
def test_the_detail_page_renders_html_for_an_admin(seeded_db):
admin = _admin(seeded_db)
r = requests.get(SERVICE_URL, headers={"X-API-KEY": admin.headers["X-API-KEY"]})
assert r.status_code == 200
assert "Acceptance convergence" in r.text
def test_an_admin_saves_an_agreement_toggle(seeded_db):
admin = _admin(seeded_db)
key = setting_key("container_credentials")
original = get_setting(key, "0")
try:
r = admin.post(f"{SERVICE_URL}/config", data={key: "1"})
assert r.status_code == 200, r.text[:300]
assert r.json().get("ok") is True
refresh_snapshot()
clear_settings_cache()
assert get_setting(key, "0") == "1"
finally:
set_setting(key, original)
def test_an_invalid_batch_size_is_refused(seeded_db):
admin = _admin(seeded_db)
r = admin.post(f"{SERVICE_URL}/config", data={"acceptance_batch_size": "banana"})
assert r.status_code == 400
assert r.json()["errors"]["acceptance_batch_size"]
def test_a_guest_cannot_read_the_service(app_server):
r = requests.get(f"{SERVICE_URL}/data", headers=JSON, allow_redirects=False)
assert r.status_code in (302, 303, 401, 403)
def test_the_master_switch_stays_off_for_the_suite(seeded_db):
refresh_snapshot()
clear_settings_cache()
assert get_setting("service_acceptance_enabled", "0") != "1"

View File

@ -0,0 +1 @@
# retoor <retoor@molodetz.nl>

View File

@ -0,0 +1,65 @@
# retoor <retoor@molodetz.nl>
from devplacepy.database import CONSENT_KINDS
from devplacepy.services.acceptance.agreements import (
AGREEMENTS,
agreement_for,
label_for,
setting_key,
)
from devplacepy.services.acceptance.service import AcceptanceService
def test_every_consent_kind_is_classified_as_an_agreement():
assert {agreement.kind for agreement in AGREEMENTS} == set(CONSENT_KINDS)
def test_setting_keys_are_unique_and_namespaced():
keys = [setting_key(agreement.kind) for agreement in AGREEMENTS]
assert len(keys) == len(set(keys))
for key in keys:
assert key.startswith("acceptance_grant_")
def test_every_agreement_has_a_config_field_in_the_agreements_group():
service = AcceptanceService()
fields = {field.key: field for field in service.config_fields}
for agreement in AGREEMENTS:
field = fields[setting_key(agreement.kind)]
assert field.type == "bool"
assert field.default is False
assert field.group == "Agreements"
def test_labels_come_from_the_consent_registry():
for agreement in AGREEMENTS:
assert label_for(agreement.kind) == CONSENT_KINDS[agreement.kind]
def test_only_terms_declares_a_gate_column():
gated = [agreement.kind for agreement in AGREEMENTS if agreement.gate_column]
assert gated == ["terms"]
def test_versioned_agreements_match_the_application_version_keys():
from devplacepy.routers.profile.consent import VERSION_KEYS
versioned = {
agreement.kind: agreement.version_setting
for agreement in AGREEMENTS
if agreement.version_setting
}
assert versioned == VERSION_KEYS
def test_agreement_lookup_is_total_over_the_registry():
for agreement in AGREEMENTS:
assert agreement_for(agreement.kind) is agreement
assert agreement_for("not_a_consent") is None
def test_the_service_is_opt_in_and_runs_every_five_minutes():
service = AcceptanceService()
assert service.default_enabled is False
assert service.interval_seconds == 300
assert service.is_enabled() is False

View File

@ -0,0 +1,147 @@
# retoor <retoor@molodetz.nl>
import pytest
from devplacepy.database import (
CONSENTS_TABLE,
consent_granted,
get_setting,
get_table,
set_consent,
set_setting,
)
from devplacepy.services.acceptance.agreements import agreement_for
from devplacepy.services.acceptance.grant import converge_user
from devplacepy.services.audit.store import AUDIT_TABLE
from devplacepy.utils import generate_uid
TERMS = agreement_for("terms")
AI = agreement_for("ai_third_party")
@pytest.fixture
def account(local_db):
uid = generate_uid()
get_table("users").insert(
{
"uid": uid,
"username": f"grantprobe_{uid[:8]}",
"email": f"grantprobe_{uid[:8]}@example.com",
"password_hash": "x",
"role": "Member",
"terms_version": "",
"terms_accepted_at": "",
"deletion_requested_at": "",
"created_at": "2020-01-01T00:00:00",
}
)
yield get_table("users").find_one(uid=uid)
get_table("users").delete(uid=uid)
get_table(CONSENTS_TABLE).delete(owner_id=uid)
get_table(AUDIT_TABLE).delete(target_uid=uid)
@pytest.fixture
def versions():
before = get_setting("terms_version", "1")
yield
set_setting("terms_version", before)
def consent_rows(uid, kind):
return list(
get_table(CONSENTS_TABLE).find(
owner_kind="user", owner_id=uid, kind=kind, deleted_at=None
)
)
def audit_rows(uid, event_key):
return list(
get_table(AUDIT_TABLE).find(
target_uid=uid, event_key=event_key, actor_kind="service"
)
)
def test_a_won_terms_claim_writes_the_gate_column_and_the_ledger(account, versions):
set_setting("terms_version", "5")
assert converge_user(TERMS, account, "5") is True
row = get_table("users").find_one(uid=account["uid"])
assert row["terms_version"] == "5"
assert row["terms_accepted_at"]
written = consent_rows(account["uid"], "terms")
assert len(written) == 1
assert written[0]["state"] == "granted"
assert written[0]["version"] == "5"
def test_a_second_claim_at_the_same_version_writes_nothing(account, versions):
set_setting("terms_version", "5")
assert converge_user(TERMS, account, "5") is True
assert converge_user(TERMS, account, "5") is False
assert len(consent_rows(account["uid"], "terms")) == 1
assert len(audit_rows(account["uid"], "terms.accept")) == 1
def test_a_withdrawn_terms_consent_is_never_converged(account, versions):
set_setting("terms_version", "5")
set_consent("user", account["uid"], "terms", False, version="5")
assert converge_user(TERMS, account, "5") is False
assert get_table("users").find_one(uid=account["uid"])["terms_version"] == ""
def test_a_withdrawn_ledger_agreement_is_never_converged(account):
set_consent("user", account["uid"], "ai_third_party", False)
assert converge_user(AI, account, "1") is False
assert not consent_granted("user", account["uid"], "ai_third_party")
def test_a_ledger_agreement_converges_once(account):
assert converge_user(AI, account, "1") is True
assert consent_granted("user", account["uid"], "ai_third_party")
assert converge_user(AI, account, "1") is False
granted = [
row for row in consent_rows(account["uid"], "ai_third_party")
if row["state"] == "granted"
]
assert len(granted) == 1
def test_the_written_row_matches_the_human_path_field_for_field(account):
reference = set_consent("user", account["uid"], "container_credentials", True)
get_table(CONSENTS_TABLE).delete(uid=reference["uid"])
assert converge_user(AI, account, "1") is True
written = consent_rows(account["uid"], "ai_third_party")[0]
assert set(written.keys()) == set(reference.keys())
assert written["owner_kind"] == reference["owner_kind"]
assert written["state"] == reference["state"]
assert written["withdrawn_at"] == reference["withdrawn_at"]
assert written["deleted_at"] is None
assert written["deleted_by"] is None
assert written["granted_at"] == written["created_at"]
def test_a_won_claim_records_an_existing_event_key_as_a_service_actor(account, versions):
set_setting("terms_version", "5")
converge_user(TERMS, account, "5")
converge_user(AI, account, "1")
accepted = audit_rows(account["uid"], "terms.accept")
granted = audit_rows(account["uid"], "consent.grant")
assert len(accepted) == 1
assert len(granted) == 1
assert accepted[0]["actor_username"] == "acceptance"
assert accepted[0]["origin"] == "service"
assert accepted[0]["new_value"] == "5"
assert granted[0]["new_value"] == "granted"
def test_a_lost_claim_records_nothing(account, versions):
set_setting("terms_version", "5")
converge_user(TERMS, account, "5")
converge_user(TERMS, account, "5")
assert len(audit_rows(account["uid"], "terms.accept")) == 1
def test_an_account_without_a_uid_is_refused():
assert converge_user(AI, {}, "1") is False

View File

@ -0,0 +1,68 @@
# retoor <retoor@molodetz.nl>
from pathlib import Path
PACKAGE_ROOT = Path(__file__).resolve().parents[4] / "devplacepy"
PACKAGE_DIR = PACKAGE_ROOT / "services" / "acceptance"
NEEDLES = (
"services.acceptance",
"devplacepy/services/acceptance",
"AcceptanceService",
)
def source_files(suffixes):
return [
path
for path in PACKAGE_ROOT.rglob("*")
if path.suffix in suffixes
and path.is_file()
and PACKAGE_DIR not in path.parents
and "__pycache__" not in path.parts
]
def references(path):
body = path.read_text()
return any(needle in body for needle in NEEDLES)
def test_only_main_imports_the_acceptance_package():
importers = [
path.relative_to(PACKAGE_ROOT).as_posix()
for path in source_files({".py"})
if references(path)
]
assert importers == ["main.py"]
def test_no_template_or_static_asset_mentions_the_package():
mentions = [
path.relative_to(PACKAGE_ROOT).as_posix()
for path in source_files({".html", ".js", ".css"})
if references(path)
]
assert mentions == []
def test_the_feature_has_no_route_schema_or_agent_tool():
watched = ("routers", "schemas", "docs_api")
surfaced = [
path.relative_to(PACKAGE_ROOT).as_posix()
for path in source_files({".py"})
if path.parts[len(PACKAGE_ROOT.parts)] in watched and references(path)
]
assert surfaced == []
def test_the_devii_catalog_does_not_expose_the_service():
catalog = PACKAGE_ROOT / "services" / "devii" / "actions"
mentions = [path.name for path in catalog.rglob("*.py") if references(path)]
assert mentions == []
def test_the_package_imports_no_router_and_no_template_global():
for path in PACKAGE_DIR.glob("*.py"):
body = path.read_text()
assert "devplacepy.routers" not in body
assert "devplacepy.templating" not in body

View File

@ -0,0 +1,166 @@
# retoor <retoor@molodetz.nl>
import pytest
from devplacepy.database import CONSENTS_TABLE, get_setting, get_table, set_setting
from devplacepy.services.acceptance.agreements import agreement_for
from devplacepy.services.acceptance.pending import (
LIVE_ACCOUNT_CLAUSES,
current_version,
live_account_clauses,
pending,
)
from devplacepy.utils import generate_uid
TERMS = agreement_for("terms")
PRIVACY = agreement_for("privacy")
AI = agreement_for("ai_third_party")
@pytest.fixture
def account(local_db):
uid = generate_uid()
get_table("users").insert(
{
"uid": uid,
"username": f"pendingprobe_{uid[:8]}",
"email": f"pendingprobe_{uid[:8]}@example.com",
"password_hash": "x",
"role": "Member",
"terms_version": "",
"terms_accepted_at": "",
"deletion_requested_at": "",
"created_at": "2020-01-01T00:00:00",
}
)
yield uid
get_table("users").delete(uid=uid)
get_table(CONSENTS_TABLE).delete(owner_id=uid)
@pytest.fixture
def versions():
before = {
"terms_version": get_setting("terms_version", "1"),
"privacy_version": get_setting("privacy_version", "1"),
}
yield
for key, value in before.items():
set_setting(key, value)
def write_consent(uid, kind, state, version):
get_table(CONSENTS_TABLE).insert(
{
"uid": generate_uid(),
"owner_kind": "user",
"owner_id": uid,
"kind": kind,
"version": version,
"state": state,
"granted_at": "2021-01-01T00:00:00" if state == "granted" else "",
"withdrawn_at": "" if state == "granted" else "2021-01-01T00:00:00",
"created_at": "2021-01-01T00:00:00",
"deleted_at": None,
"deleted_by": None,
}
)
def is_pending(agreement, uid):
return uid in {row["uid"] for row in pending(agreement, 5000)}
def test_an_account_that_never_accepted_is_pending(account):
assert is_pending(TERMS, account)
assert is_pending(AI, account)
def test_a_null_gate_column_is_pending_not_invisible(account, versions):
set_setting("terms_version", "1")
get_table("users").update({"uid": account, "terms_version": None}, ["uid"])
assert is_pending(TERMS, account)
def test_a_satisfied_gate_column_is_not_pending(account, versions):
set_setting("terms_version", "7")
get_table("users").update({"uid": account, "terms_version": "7"}, ["uid"])
assert not is_pending(TERMS, account)
def test_a_stale_gate_column_is_pending_again(account, versions):
set_setting("terms_version", "7")
get_table("users").update({"uid": account, "terms_version": "7"}, ["uid"])
set_setting("terms_version", "8")
assert is_pending(TERMS, account)
def test_a_withdrawal_removes_the_account_permanently(account, versions):
set_setting("terms_version", "1")
write_consent(account, "terms", "withdrawn", "1")
assert not is_pending(TERMS, account)
set_setting("terms_version", "2")
assert not is_pending(TERMS, account)
get_table("users").update({"uid": account, "terms_version": None}, ["uid"])
assert not is_pending(TERMS, account)
def test_an_unversioned_agreement_is_satisfied_by_a_granted_row(account):
write_consent(account, "ai_third_party", "granted", "1")
assert not is_pending(AI, account)
def test_a_versioned_ledger_agreement_tracks_the_policy_version(account, versions):
set_setting("privacy_version", "3")
write_consent(account, "privacy", "granted", "3")
assert not is_pending(PRIVACY, account)
set_setting("privacy_version", "4")
assert is_pending(PRIVACY, account)
def test_a_legacy_consent_row_without_a_version_is_stale(account, versions):
set_setting("privacy_version", "1")
write_consent(account, "privacy", "granted", None)
assert is_pending(PRIVACY, account)
def test_a_deleted_account_is_never_pending(account):
get_table("users").update(
{"uid": account, "deletion_requested_at": "2024-01-01T00:00:00"}, ["uid"]
)
assert not is_pending(TERMS, account)
def test_a_deactivated_account_is_never_pending(account):
get_table("users").update({"uid": account, "is_active": False}, ["uid"])
assert not is_pending(TERMS, account)
def test_the_batch_size_bounds_the_sweep_and_the_order_is_stable(account):
first = [row["uid"] for row in pending(TERMS, 1)]
second = [row["uid"] for row in pending(TERMS, 1)]
assert len(first) <= 1
assert first == second
def test_a_zero_batch_size_still_returns_at_most_one_row(account):
assert len(pending(TERMS, 0)) <= 1
def test_the_live_account_clauses_only_name_existing_columns(local_db):
users = get_table("users")
expected = [
clause
for column, clause in LIVE_ACCOUNT_CLAUSES
if users.has_column(column)
]
assert live_account_clauses() == expected
def test_an_empty_version_setting_falls_back_to_one(versions):
set_setting("terms_version", "")
assert current_version(TERMS) == "1"
def test_an_unversioned_agreement_reports_version_one():
assert current_version(AI) == "1"