# DevPlace: acceptance convergence implementation design
Author: retoor <retoor@molodetz.nl>
Design input is [`lensfl.md`](lensfl.md). This document specifies a background service that converges every
account onto every policy agreement the platform can ask for, so that an operator running a
production-identical instance for manual testing is never interrupted by an acceptance dialog, a consent
refusal, or the terms gate.
Nothing here is implemented. This is the specification.
Every claim below about existing behaviour was read out of the source at the time of writing, and the
citations are exact. The subsystem this service operates on is documented in
[`devplacepy/services/moderation/CLAUDE.md`](devplacepy/services/moderation/CLAUDE.md); the framework it is
built on is documented in [`devplacepy/services/CLAUDE.md`](devplacepy/services/CLAUDE.md). Read both before
implementing.
---
## 1. The problem, stated exactly
DevPlace enforces five consents, a versioned terms gate on every mutating request, and an age-banded
maturity gate. Each is correct and each must stay. On a production-identical instance used for extended
manual testing they produce a permanent tax: every new account, every bot in the fleet, and every account
that predates the trust-and-safety commit (whose `users.terms_version` is deliberately left `NULL`, never
backfilled) is blocked or degraded until a human clicks through the same dialogs again.
The instance must remain production. Not production-like with a flag threaded through the request path;
production, byte for byte, with one background process quietly converging the acceptance state that a real
population of users would have produced themselves.
Three constraints follow, and they are the whole design:
1. **The application must not know.** No request-path branch, no schema, no template, no route, no response
field, no Jinja global, no Devii tool. If any consumer could observe the feature, the instance is no
longer a production simulation and the exercise is worthless.
2. **A manual decline must be permanent.** Testing the refusal path is the reason the operator is here. An
account that has declined an agreement must never be converged, ever, with no further operator action.
3. **It must be off, and stay off, unless deliberately switched on.** Two independent switches, both
defaulting to off, plus a dry run that defaults to on.
---
## 2. The invisibility contract
"The system must not know" is vague. These are the testable clauses that replace it.
| # | Clause | How it is enforced |
|---|--------|--------------------|
| **I1** | No new table, no new column, anywhere. | The service writes only through existing columns of `users` and existing columns of `user_consents`. A unit test asserts the column sets it writes are subsets of what `database.moderation.set_consent` and `routers/auth/terms.py` already write. |
| **I2** | No new value in any existing registry or enum. `CONSENT_KINDS`, `CONSENT_STATES`, `AGE_BANDS`, `MATURITY_LEVELS`, `REPORTABLE_TARGETS`, `UNREPORTABLE_TABLES`, `SOFT_DELETE_TABLES` are untouched. | Reviewed at implementation; the moderation registry test already fails on an unclassified table, and no table is added. |
| **I3** | No new audit event key. The service records the same two keys the human paths record. | `events.md` gains no key, the header's stated key count does not change and no category count changes; only the "Recorded in" column of `terms.accept` and `consent.grant` gains a file. |
| **I4** | The rows produced are indistinguishable in shape from the rows a real acceptance produces. | A unit test writes one row through `set_consent` and one through the service and compares column sets and value shapes field by field. |
| **I5** | Nothing imports the package except one registration line in `main.py`. | A unit test greps the source tree for `services.acceptance` and asserts exactly one importing module, `devplacepy/main.py`. |
| **I6** | The feature has no route, no schema, no template, no Devii action and no `docs_api` entry. | It inherits the generic services admin surface, exactly as `NotificationRelayService`, `PresenceRelayService`, `LiveViewRelayService` and `AuditService` do. See section 14 for why this does not violate the four-faces rule. |
| **I7** | The test suite is unaffected. | `DEVPLACE_DISABLE_SERVICES=1` in `tests/conftest.py` stops `supervise()` from ever being called, and `default_enabled = False` keeps the service off even when services do run. The moderation tests that assert a `403` from the terms gate stay green by construction. |
The one thing that is deliberately visible is the audit log. Every state change in DevPlace records one
append-only row, and a service that silently mutated user consent state would be the single worst possible
exception to that rule. The audit log is the operator's record, not application state: no code path reads it
to decide behaviour, so recording there costs nothing in invisibility and buys full traceability. This is
the position `devplacepy/services/audit/CLAUDE.md` already takes.
---
## 3. Design axioms
Every axiom is an existing DevPlace pattern, named with its precedent. None is invented for this feature.
| # | Axiom | Precedent |
|---|-------|-----------|
| **A1** | A background subsystem is a `BaseService` with declarative `config_fields`, and gets start/stop, enable-on-boot, run-now, a config form, logs, live status and metrics for free. | `services/base.py`; `ModerationService`, `PushService` |
| **A2** | An opt-in service sets `default_enabled = False` so `is_enabled()` reads `get_setting(enabled_key, "0")` and it never auto-starts on boot. | `services/base.py:203`; `BotsService` |
| **A3** | Runtime policy lives in `site_settings`, read through the field's own `ConfigField.read()`, live-editable, never a code constant. | `services/base.py:218` (`get_config`) |
| **A4** | The target set is a registry, not a literal. Every consumer reads the same structure; adding a type is one entry. | `REPORTABLE_TARGETS`, `CONSENT_KINDS`, `DATA_PATHS`; `PushService.config_fields` splices `*providers.admin_fields()` |
| **A5** | A read-then-write mutation reachable from more than one path is a race until it is one conditional SQL statement decided on the driver's real `rowcount`. | `services/moderation/deletion.py:140` (`claim_deletion`), `queue.claim_open`, `database/atomic.py` |
| **A6** | Lock-owner-only background work; never per worker. | `main.py:289-293`: services are registered in every worker, `supervise()` runs only in the worker that won `acquire_service_lock()` |
| **A7** | Every state change records an audit row, and the recorder never raises into the caller. | `services/audit/record.py:288` (`record_system`) |
| **A8** | A control that cannot evaluate its own precondition fails closed, never open. | `moderation/filter.classify` fails to `review`, never to `allow` |
| **A9** | Legacy rows carry `NULL` in any column added after they were written, and a column added after the table may not exist at all. Every precondition uses `COALESCE`, and every raw-SQL reference to an optional column is guarded by `has_column`. | `claim_deletion`'s `COALESCE(deletion_requested_at, '')`; `deletion.due_purges`'s `if not table.has_column(...)`; the root `CLAUDE.md` race-safety rule |
| **A10** | No silent caps. When a bound truncates the work, say so in the log. | Root `CLAUDE.md`, "Rigorous correctness verification" |
---
## 4. The agreement registry
An **agreement** is a policy statement the platform asks a member to accept, recorded in `user_consents`.
There are exactly five and they are already enumerated by `database.CONSENT_KINDS`
(`database/moderation.py:122`). The registry does not invent a parallel list; it annotates the existing one
with the two facts the service needs.
`devplacepy/services/acceptance/agreements.py`:
```python
@dataclass(frozen=True)
class Agreement:
kind: str
version_setting: str
gate_column: str
stamp_column: str
AGREEMENTS: tuple[Agreement, ...] = (
Agreement("terms", "terms_version", "terms_version", "terms_accepted_at"),
Agreement("privacy", "privacy_version", "", ""),
Agreement("ai_third_party", "", "", ""),
Agreement("activity_recording", "", "", ""),
Agreement("container_credentials", "", "", ""),
)
```
`kind` is a key of `CONSENT_KINDS`. `version_setting` is the `site_settings` key holding the policy version,
`""` when the agreement is unversioned. `gate_column` is the `users` column the application's own gate reads,
`""` when the agreement is ledger-only. `stamp_column` holds the acceptance instant beside it.
`terms` is the only agreement with a gate column, because `routers/auth/terms.needs_acceptance` reads
`users.terms_version` and not the ledger. `privacy` is versioned but ledger-only: `VERSION_KEYS` in
`routers/profile/consent.py:28` maps exactly `terms` and `privacy` to a version setting, and every other kind
resolves to the literal `"1"` through `consent_version`. That asymmetry is in the application, not in this
design; the registry records it rather than papering over it.
**Completeness is enforced, not remembered.** A unit test asserts
`{a.kind for a in AGREEMENTS} == set(CONSENT_KINDS)`. A sixth consent added to `CONSENT_KINDS` in the future
fails the suite until it is classified here, exactly as `tests/unit/database/moderation.py` does for
reportable tables. This is the single most valuable line in the feature: it is what keeps the convergence
complete as the platform grows.
The label rendered in the admin form is `CONSENT_KINDS[kind]`, read at field-construction time. The service
never carries its own copy of the wording.
### 4.1 What is deliberately not an agreement
| Excluded | Why | The supported answer instead |
|---|---|---|
| `users.age_band` | A declaration of fact, not an agreement. Fabricating a declared age is not a production simulation, it is a falsification, and it would silently unlock `restricted` content for accounts that declared 13-15. | Sign the test account up with an adult date of birth. |
| `users.mature_opt_in` | A content preference gated by the age band, with no entry in the consent ledger and therefore no decline register that outlives the 90-day `audit_log_retention_days` window. Adding a second, weaker decline mechanism for one field would violate DRY and clause I2. | The existing site setting `moderation_mature_default_hidden`, editable at `/admin/settings`, already turns the interstitial off instance-wide. |
| Guest consents | Nothing writes a consent row with `owner_kind = "guest"` and no guest surface is gated by one. The gateway resolves a guest to owner kind `anonymous`, which `consent_denied` exempts by design. | None needed. |
| `interactions_enabled`, notification preferences, customization toggles | Per-user preferences, not policy acceptances. None of them blocks anything. | None needed. |
Each exclusion is a boundary, not an oversight, and each is restated in the nested `CLAUDE.md`.
---
## 5. The decline register
This is the load-bearing idea and it needs no storage at all.
`user_consents` is append-only in effect: `set_consent` stamps `withdrawn_at` on the current row when it is
granted and then inserts a new row either way (`database/moderation.py:289`), so the full history is provable.
The **latest live row for a `(user, kind)` pair therefore already is the decline register**:
| Latest row state | Meaning | Service behaviour |
|---|---|---|
| no row | never asked | converge |
| `granted`, current version | satisfied | nothing to do |
| `granted`, stale version | policy changed since acceptance | converge |
| `withdrawn` | **declined by a human** | never touch, permanently |
The service only ever writes `granted`. It follows that any `withdrawn` row in the ledger was written by a
human action: the owner-only `POST /profile/{username}/consent`, the Devii consent tool acting on the owner's
behalf, or the CLI. No provenance column, no marker, no flag, and nothing for the application to observe.
Clause I1 is satisfied not by discipline but by the shape of the existing data.
**One SQL fragment expresses it**, defined once in `pending.py` and used by every read and every write:
```sql
COALESCE((SELECT c.state FROM user_consents c
WHERE c.owner_kind = 'user' AND c.owner_id = <user>
AND c.kind = :kind AND c.deleted_at IS NULL
ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'withdrawn'
```
**The ordering is not a detail and must not be simplified.** `database.consent_state` selects
`order_by=["-created_at", "-id"]` (`database/moderation.py:274`) and `consent_granted` is a thin wrapper over
it, so that pair *is* the expression the application evaluates when it asks whether a consent stands.
Ordering by `id` alone, or by `created_at` alone, would make the service disagree with the gate in exactly the
tie case that matters: two rows written in the same millisecond, which is precisely what a converge-then-
withdraw race produces. `deleted_at IS NULL` is required because `user_consents` is in `SOFT_DELETE_TABLES`.
The index `idx_user_consents_owner` (`owner_kind`, `owner_id`, `kind`) already exists
(`database/schema.py:1897`) and serves this; no index is added.
`consent_view` on the profile privacy tab orders by `-created_at` only and takes the first row per kind. That
is a looser tiebreak than `consent_state`, and it is the display path, not a gate. Follow `consent_state`;
never introduce a third ordering.
### 5.1 Terms is the one case that needs an operator step
Withdrawing the `terms` consent does not clear `users.terms_version`, so a tester who withdraws it is not
immediately gated. The recipe is two actions, both of which already exist:
1. The tester withdraws their own `terms` consent once, at `/profile/{username}?tab=privacy`.
2. An administrator bumps `terms_version` at `/admin/settings`.
Every other account converges within one interval; the tester stays gated indefinitely and can exercise
`/auth/accept-terms`, the `TermsGate.js` in-place dialog, and the `terms_acceptance_required` JSON refusal as
often as they like. This is the workflow section 13 documents for the operator.
---
## 6. Module map
New package `devplacepy/services/acceptance/`, laid out like `services/moderation/`: registry, read side,
write side, service.
| File | Owns |
|---|---|
| `__init__.py` | Nothing but the package. No re-exports; the service is imported by path from `main.py`. |
| `agreements.py` | `Agreement`, `AGREEMENTS`, `setting_key(kind)`, `label_for(kind)` |
| `pending.py` | `NOT_WITHDRAWN`, `satisfied_clause(agreement)`, `pending(agreement, limit) -> list[dict]`, `current_version(agreement) -> str` |
| `grant.py` | `converge_user(agreement, user, version) -> bool`, `_claim_gate_column`, `_insert_consent_row`, the audit calls |
| `service.py` | `AcceptanceService(BaseService)`: config fields, `run_once`, `collect_metrics` |
| `CLAUDE.md` | The nested subsystem documentation (section 14) |
Import discipline, mirroring `services/moderation/deletion.py`: `pending.py` and `grant.py` import only from
`devplacepy.database` and `sqlalchemy`; `devplacepy.services.audit`, `devplacepy.utils.generate_uid` and
`devplacepy.database.bump_cache_version` are imported lazily inside the functions that use them, exactly as
`deletion.py` imports its audit recorder at line 187. Nothing in the package imports `content`, a router, or a
template global.
---
## 7. The read side: what is pending
`pending(agreement, limit)` returns the bounded list of accounts that are neither satisfied nor declined.
One statement, three shapes of one clause.
```sql
SELECT u.uid, u.username
FROM users u
WHERE <live account clauses>
AND <not_withdrawn>
AND <not_satisfied>
ORDER BY u.id
LIMIT :limit
```
`<not_withdrawn>` is the fragment from section 5, correlated on `u.uid`.
`<not_satisfied>` is derived from the registry, and the derivation rule is the principle that keeps this
honest: **satisfaction is defined by the exact expression the application's own gate evaluates, never by a
proxy.**
| Agreement shape | Application gate | `<not_satisfied>` |
|---|---|---|
| gate column declared (`terms`) | `needs_acceptance` compares `users.terms_version` to `get_setting("terms_version", "1") or "1"` (`routers/auth/terms.py:21-28`) | `COALESCE(u.terms_version, '') != :version` |
| versioned, ledger only (`privacy`) | the privacy tab compares the latest row's `version` against `consent_version(kind)` | `COALESCE((SELECT c.version FROM user_consents c WHERE ... ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != :version` |
| unversioned (the other three) | `consent_granted` asks whether the latest state is `granted` | `COALESCE((SELECT c.state FROM user_consents c WHERE ... ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'granted'` |
For the versioned ledger-only shape the state does not appear in the clause: a `withdrawn` row is already
excluded by `<not_withdrawn>`, so comparing the version alone is complete. Keeping the two concerns in two
independent clauses is what makes that provable rather than lucky.
`current_version(agreement)` is `get_setting(agreement.version_setting, "1") or "1"` for a versioned agreement
and the literal `"1"` for an unversioned one, which is byte-identical to `consent_version` in
`routers/profile/consent.py:47`. **The `or "1"` is load-bearing and is not cosmetic:** on a fresh database an
admin settings save can insert `terms_version = ""`, and a bare `get_setting` would then make every account
pending forever and re-write the same rows on every tick. This is the trap documented in
`services/moderation/CLAUDE.md`, and every reader of a policy version in this codebase carries the guard.
### 7.1 The live-account clauses are built, not hardcoded
This is the correction that backwards compatibility actually turns on, and it is stricter than a `COALESCE`.
`init_db` ensures `terms_version`, `terms_accepted_at` and `deletion_requested_at` on `users`
(`database/schema.py:2000-2015`). **It does not ensure `is_active`.** That column is created implicitly by
`dataset` the first time `enforcement.suspend_user` or `deletion.anonymise` writes it, so on an instance where
nobody has ever been suspended or deleted the column does not exist and a raw-SQL reference to it raises
`OperationalError: no such column`. A background service that dies on a clean database is worse than no
service.
So the `WHERE` clauses are assembled from what the table actually has, the same way `deletion.due_purges`
guards with `table.has_column` before querying:
```python
users = get_table("users")
clauses = []
if users.has_column("deletion_requested_at"):
clauses.append("COALESCE(u.deletion_requested_at, '') = ''")
if users.has_column("is_active"):
clauses.append("COALESCE(u.is_active, 1) != 0")
if users.has_column("deleted_at"):
clauses.append("u.deleted_at IS NULL")
```
An absent column means no account can be in that state, so omitting the clause is the correct answer and not
a relaxation. `COALESCE(u.is_active, 1) != 0` matches `database.users.is_account_active` exactly, which treats
`NULL` as active (`database/users.py:77-79`). `users` is not in `SOFT_DELETE_TABLES` - deletion anonymises and
stamps `deletion_requested_at` instead - so the `deleted_at` clause is defensive and will normally be omitted.
The same rule applies to the gate column: if `agreement.gate_column` is declared but the column is missing,
`pending` treats the agreement as ledger-only for that database rather than raising.
Guards, in order, at the top of `pending`:
- `if "users" not in db.tables or CONSENTS_TABLE not in db.tables: return []`. On a database so fresh that the
consent table has not been created, no account can be shown to have declined, so per axiom A8 the answer is
"converge nobody", not "converge everybody".
- `limit = max(1, limit)`.
`ORDER BY u.id` gives a stable, resumable sweep: a capped run always resumes from the same place, and a large
population converges deterministically over consecutive intervals rather than randomly.
---
## 8. The write side: one conditional statement per grant
Every write re-evaluates its own precondition inside the write transaction and is decided on the driver's real
`rowcount`. This closes the race between the service's `SELECT` and a member withdrawing a consent from the
profile page a moment later; SQLite serialises writers, so a withdrawal committed before ours is visible to
our `WHERE` clause and the grant simply does not happen.
`dataset`'s wrapped `db.query()` does not expose `rowcount`. Use
`db.executable.execute(sqlalchemy.text(...), params).rowcount` inside `with db:`, exactly as
`deletion.claim_deletion` does at `services/moderation/deletion.py:140`.
### 8.1 With a gate column (`terms`)
The users update is the atomic claim; the ledger row is the follow-through, in the same transaction so the two
can never diverge.
```sql
UPDATE users
SET terms_version = :version, terms_accepted_at = :now
WHERE uid = :uid
AND COALESCE(terms_version, '') != :version
AND COALESCE((SELECT c.state FROM user_consents c
WHERE c.owner_kind = 'user' AND c.owner_id = users.uid
AND c.kind = 'terms' AND c.deleted_at IS NULL
ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'withdrawn'
```
`rowcount == 1` wins the claim; `0` means another worker, a real acceptance, or a withdrawal got there first,
and the function returns `False` without writing anything else.
The set clause writes exactly the two columns `POST /auth/accept-terms` writes and nothing more
(`routers/auth/terms.py:57-61`). **`updated_at` is not written**, which is also why
`database.atomic.conditional_update_row` cannot be reused here: it appends `updated_at = :updated_at`
unconditionally (`database/atomic.py:16`), the `users` table has no such column, and creating one would make
the service's rows distinguishable from the route's. The hand-rolled statement is the same shape
`claim_deletion` uses on the same table, for the same reason.
**The human path grants two agreements, not one.** `POST /auth/accept-terms` writes the users row, then calls
`set_consent` for `terms` **and** for `privacy` at `get_setting("privacy_version", "1") or "1"`. The service
deliberately keeps the two agreements independent, because the whole point of the per-agreement switches is
edge-case testing: an operator must be able to converge `terms` while leaving `privacy` pending. Enabling both
agreements reproduces the human path exactly; enabling one is a deliberate, documented divergence and the
nested `CLAUDE.md` says so.
### 8.2 Ledger only (the other four)
```sql
INSERT INTO user_consents
(uid, owner_kind, owner_id, kind, version, state,
granted_at, withdrawn_at, created_at, deleted_at, deleted_by)
SELECT :row_uid, 'user', :uid, :kind, :version, 'granted',
:now, '', :now, NULL, NULL
WHERE COALESCE((SELECT c.state FROM user_consents c
WHERE c.owner_kind = 'user' AND c.owner_id = :uid
AND c.kind = :kind AND c.deleted_at IS NULL
ORDER BY c.created_at DESC, c.id DESC LIMIT 1), '') != 'withdrawn'
AND <not_satisfied, the same clause pending used>
```
`row_uid` is `utils.generate_uid()`. The column list, the values, and the explicit `deleted_at: NULL,
deleted_by: NULL` are copied from `database.moderation.set_consent` (`database/moderation.py:300-315`) and must
stay in step with it; the test in clause I4 is what enforces that. Note `granted_at = :now` and
`withdrawn_at = ''` and not `NULL`: `set_consent` writes the empty string on the unused side, and clause I4
means matching it.
**Why this is not `set_consent`.** `set_consent` cannot express a precondition and cannot join a caller's
transaction, so reusing it would reintroduce exactly the race this statement closes and would break the
atomicity of the terms path. The duplication is one INSERT, it is confined to one private function
`_insert_consent_row(uid, kind, version, now)` used by both write shapes, and it is covered by a test that
compares it against `set_consent` field by field. This is a deliberate, documented divergence of the same kind
`claim_deletion` makes against a plain `table.update`.
### 8.3 Audit
Only on a won claim, and only with keys that already exist:
| Agreement | Event key | Fields |
|---|---|---|
| `terms` | `terms.accept` | `target_type="user"`, `target_uid`, `target_label=username`, `new_value=version` |
| the other four | `consent.grant` | as above, `new_value="granted"`, `metadata={"kind": kind}` |
Both carry `links=[audit.target("user", uid, username)]`, matching the human paths at
`routers/auth/terms.py:82` and `routers/profile/consent.py:84`.
Recorded through the lazily imported recorder, following `deletion.py:187`:
```python
from devplacepy.services.audit import record as audit
audit.record_system(
"terms.accept",
actor_kind="service",
actor_username="acceptance",
target_type="user",
...
)
```
`actor_role="system"` and `origin="service"` are already the defaults of `record_system`
(`services/audit/record.py:288-307`); `service` is a legal actor kind and a legal origin
(`services/audit/categories.py:57-58`). Both keys already exist in `events.md` (lines 69 and 72) and already
map to the `account` category. The recorder routes through `background.submit` and never raises into the
caller. Nothing is added to `events.md` except the new file name in the "Recorded in" column of those two
rows; the key count stated in the file header is unchanged.
The audit row is the only thing that distinguishes a converged acceptance from a human one, and it lives where
an operator looks and no code path reads.
### 8.4 Cache invalidation
`users.terms_version` is on the cached user row and the row is cached per worker for 300s
(`utils/authcache.py:6`), so a converged account would otherwise stay gated on the workers that already cached
it.
- A gate-column write requires the cross-worker `auth` version bump.
- A ledger-only write does not: `consent_granted` reads the table fresh on every call.
`clear_user_cache(uid)` bumps the global `auth` version by default (`utils/authcache.py:13-18`), and a version
bump makes **every** worker drop its **whole** user cache. Calling it per account would do that once per
converged user; a run of 200 accounts would do it 200 times. Instead: `run_once` performs exactly **one**
`database.bump_cache_version("auth")` at the end of the run, and only when at least one gate column actually
changed. One bump invalidates every worker's cache including the lock owner's, which is precisely the required
effect, at 1/200th of the cost. Terms acceptance gates writes, so it is an authorization change and must
propagate; it just does not need to propagate 200 times.
---
## 9. The service
`devplacepy/services/acceptance/service.py`.
```
name "acceptance"
title "Acceptance convergence"
default_enabled False
interval_seconds 300 (five minutes, as specified)
min_interval 60
METRICS_SECONDS 60
```
`description`, shown on `/admin/services` and its detail page, states plainly what it does and that it is for
a production-identical test instance, so an operator who finds it enabled on a real production host knows
immediately that it is wrong. `details` carries the decline-register rule, so the one thing an operator must
understand is on the page they are already looking at.
### 9.1 Configuration fields
| Key | Type | Default | Group | Purpose |
|---|---|---|---|---|
| `service_acceptance_enabled` | bool | `0` | General | Framework field. The master switch, off by boot default via `default_enabled = False`. |
| `service_acceptance_interval` | int | `300` | General | Framework field, floored at `min_interval`. |
| `acceptance_dry_run` | bool | `1` | Safety | Compute and log what would be converged; write nothing, record nothing. |
| `acceptance_batch_size` | int | `200` (min 1, max 5000) | Safety | Maximum accounts converged per agreement per run. |
| `acceptance_grant_terms` | bool | `0` | Agreements | One field per registry entry. |
| `acceptance_grant_privacy` | bool | `0` | Agreements | |
| `acceptance_grant_ai_third_party` | bool | `0` | Agreements | |
| `acceptance_grant_activity_recording` | bool | `0` | Agreements | |
| `acceptance_grant_container_credentials` | bool | `0` | Agreements | |
`setting_key(kind)` is `f"acceptance_grant_{kind}"`, defined once in `agreements.py` and used by the field
list, `run_once` and the tests. The per-agreement fields are **built from the registry**, not written out by
hand, the same way `PushService.config_fields` splices `*providers.admin_fields()`:
```python
config_fields = [
ConfigField(
setting_key(agreement.kind),
label_for(agreement.kind),
type="bool",
default=False,
help=CONSENT_KINDS[agreement.kind],
group="Agreements",
)
for agreement in AGREEMENTS
] + [ ...the two safety fields... ]
```
`ConfigField.read()` returns the typed default whenever the setting is unset or unparseable
(`services/base.py:81-89`), so a fresh install reads `False` for every agreement with no seeding in
`init_db` and no change to `operational_defaults`.
A sixth consent added to `CONSENT_KINDS` therefore appears in the admin form with no edit to the service. This
is what the request asked for: one option in the admin, per type of agreement.
Three gates stand between a fresh install and a single written row: the service is disabled, every agreement
is disabled, and dry run is on. All three must be changed deliberately.
### 9.2 `run_once`
```
config = self.get_config()
dry = config["acceptance_dry_run"]
limit = config["acceptance_batch_size"]
gate_changed = False
for agreement in AGREEMENTS:
if not config[setting_key(agreement.kind)]:
continue
version = current_version(agreement)
candidates = pending(agreement, limit)
if not candidates:
continue
if dry:
self.log(f"[dry run] {agreement.kind}: {len(candidates)} account(s) would be converged "
f"to version {version}: {sample_names(candidates)}")
continue
granted = 0
for user in candidates:
if converge_user(agreement, user, version):
granted += 1
gate_changed = gate_changed or bool(agreement.gate_column)
self.log(f"{agreement.kind}: converged {granted} of {len(candidates)} pending at version {version}")
if len(candidates) == limit:
self.log(f"{agreement.kind}: batch cap of {limit} reached, more remain for the next run")
if gate_changed:
bump_cache_version("auth")
```
Notes that are not incidental:
- **There is no `owns_lock()` guard and there must not be one.** `main.py:289-293` calls
`service_manager.supervise()` only in the worker that won `acquire_service_lock()`, so `run_once` is
physically unreachable on any other worker. `ModerationService` has no such guard either. Adding one would
be cargo cult; the routers that call `owns_lock()` do so because a *request* can land on any worker.
- The batch-cap log line is required by axiom A10. A silently truncated sweep reads as "everyone is converged"
when they are not.
- `converge_user` returning `False` is normal, not an error: it means the account was claimed by a real
acceptance or a withdrawal between the `SELECT` and the `UPDATE`. The count difference in the log is the
operator's visibility into that.
- `sample_names(candidates)` caps the dry-run listing at 20 usernames and appends the remaining count.
- No exception is swallowed here. `BaseService._execute_run` already wraps `run_once`, logs
`Error in run_once: {e}` to the service's buffer, and keeps the supervisor loop alive
(`services/base.py:307-313`).
### 9.3 `collect_metrics`
The detail page renders `{"stats": [...], "table": {"columns": [...], "rows": [...]}}`, the shape
`ContainerService` and `BotsService` already return.
| Stat | Source |
|---|---|
| `Dry run` | the config field, as `1`/`0` |
| `Agreements enabled` | count of registry entries whose field reads true |
| `<label> pending` | `len(pending(agreement, limit))`, suffixed `+` when it equals the limit |
| `<label> converged` | an in-memory counter on the service instance, since process start |
The table has one row per agreement: kind, label, enabled, versioned, pending, converged since start.
The pending counts reuse the same bounded `pending()` the run uses, deliberately: an unbounded `COUNT(*)` over
`users` with a correlated subquery per row is the kind of aggregate the framework tells you to keep out of the
tick. Reporting a saturated count as `200+` is honest and costs one already written query.
`METRICS_SECONDS = 60` keeps even that off the one-second tick, and `_safe_metrics` already swallows and logs
any exception so a metrics bug can never take the service down (`services/base.py:227-233`).
### 9.4 Registration
One line in the existing `lifespan` register block in `main.py`, and nothing else anywhere:
```python
service_manager.register(AcceptanceService())
```
Registered in every worker so `describe_all()` works on any of them, supervised only on the lock owner, like
every other service. This is the single import permitted by clause I5.
---
## 10. Backwards compatibility
"Works against a database that predates this feature" is a hard requirement. It has two halves: columns that
exist but hold `NULL`, and columns that do not exist at all.
| Situation | Rule |
|---|---|
| `users.terms_version` is `NULL` on every account created before the trust-and-safety commit; `init_db` ensures the column but deliberately never backfills it. | `COALESCE(u.terms_version, '') != :version`. A bare `!=` against `NULL` yields `NULL`, not true, and would silently converge nobody. |
| `users.is_active` **is not ensured by `init_db` at all** and only exists once a suspension or a deletion has written it. | Build the clause with `has_column`, per section 7.1. A raw reference to a missing column is an `OperationalError`, not a `NULL`. |
| `users.is_active` exists but is `NULL` on rows written before the first suspension. | `COALESCE(u.is_active, 1) != 0`, matching `is_account_active` exactly. |
| `users.deletion_requested_at` is `NULL` on every row that predates account deletion. | `COALESCE(u.deletion_requested_at, '') = ''`, the guard `claim_deletion` uses. |
| `user_consents` does not exist on a database that has never run the trust-and-safety `init_db`. | `if CONSENTS_TABLE not in db.tables: return []`. Fail closed. |
| A consent row written before a version was recorded carries `version = NULL`. | `COALESCE(c.version, '') != :version`, which correctly treats it as stale and re-grants at the current version. |
| `site_settings` has no `terms_version` row, or has one holding `""`. | `get_setting(key, "1") or "1"`. |
No migration, no backfill, no `init_db` change, no `operational_defaults` entry and no schema change of any
kind. The feature is additive to an untouched database, which is the same property that lets `/admin/trash`
restore an event written before the trash existed.
---
## 11. Correctness verification
This feature is a read-then-write mutation reachable from more than one request path, so the four-layer
procedure in the root `CLAUDE.md` applies in full, in addition to the persisted tests in section 12. All four
are disposable scripts run against a temp database via `DEVPLACE_DATABASE_URL`/`DEVPLACE_DATA_DIR`, not pytest
files.
**The invariant, stated once:** *for any `(user, kind)` pair, a row written by the service never follows a
`withdrawn` row.* Everything below tests that one sentence.
### Layer 1: properties over the full input domain
`pending` and `satisfied_clause` are pure with respect to a given database state, so enumerate that state
rather than sampling it. Build the full matrix: ledger state in `{absent, granted, withdrawn}` x row version
in `{"", "1", "2"}` x gate column in `{NULL, "", "1", "2"}` x current version in `{"1", "2"}`, for each of the
five agreements. That is a few hundred cases and every one has a hand-derivable answer.
Assert:
- `pending` returns the account if and only if it is neither satisfied nor declined;
- **decline monotonicity**: once the latest row is `withdrawn`, no subsequent version bump, in either
direction, makes the account pending again;
- **version monotonicity**: for a non-declined account, changing the current version to a value different from
the recorded one always makes it pending, and back again always makes it satisfied;
- **gate agreement**: for every state in the matrix, `pending`'s verdict for `terms` equals
`routers.auth.terms.needs_acceptance` on the same row, and for the other four it equals
`database.consent_granted` combined with the version comparison. This is the clause that proves "never a
proxy" rather than asserting it.
Run the matrix twice: once on a table with an `is_active` column and once on a table without one, to cover
both branches of section 7.1.
### Layer 2: stateful fuzzing
Fifty accounts, a few thousand randomized actions drawn from: bump `terms_version`, bump `privacy_version`,
human grant, human withdraw, run the service, toggle an agreement field, toggle the master switch, toggle dry
run. Re-check after **every** action:
- no `granted` row written by the service follows a `withdrawn` row for the same pair;
- with the master switch off, with the agreement off, or with dry run on, the ledger and the `users` row are
byte-identical before and after the run;
- every row the service wrote has exactly the column set `set_consent` produces, with the same empty-string
convention on `granted_at`/`withdrawn_at`;
- the number of `terms.accept` and `consent.grant` audit rows with `actor_kind = "service"` equals the number
of rows the service actually wrote, never more.
Catch the expected exception types and continue; the goal is invariant violations and unexpected exceptions.
### Layer 3: concurrency, with real processes
Sixteen real OS processes, never threads: `dataset` gives each thread its own pooled connection and enough
threads produce `database is locked` noise that is a harness artifact, not a finding.
Two scenarios, each on a genuinely fresh account whose columns are left exactly as a real signup leaves them.
**Do not pre-seed or zero `terms_version`, `is_active` or `deletion_requested_at`.** Pre-zeroing a column that
production leaves `NULL` is precisely the mistake that made an earlier race-safety pass in this codebase pass
while hiding the bug it was written to catch.
1. **Convergence storm.** Sixteen processes call `converge_user` for the same account and agreement. Exactly
one wins. Exactly one ledger row is written. Exactly one audit row is recorded.
2. **Grant against withdrawal.** Eight processes converge, eight call `set_consent(..., granted=False)`. Scan
the resulting ledger ordered by `created_at, id` and assert no service-written `granted` row follows a
`withdrawn` row. Since only the service grants in this scenario, the invariant is exact.
### Layer 4: static analysis
`pyflakes` and `ruff check` on every touched file. `py_compile` and a clean
`python -c "from devplacepy.main import app"` prove syntax and import order only; neither catches a missing
import inside a function body, and this design uses lazy imports in three functions.
---
## 12. Persisted tests
Tier is decided by fixtures, and the paths mirror the source module path. `tests/unit/services/moderation/`
and `tests/api/admin/services/` already exist and set the pattern.
| File | Tier | Covers |
|---|---|---|
| `tests/unit/services/acceptance/__init__.py` | - | package |
| `tests/unit/services/acceptance/agreements.py` | unit | registry completeness against `CONSENT_KINDS` (the enforcement test), unique setting keys, every agreement has a config field, labels resolve |
| `tests/unit/services/acceptance/pending.py` | unit | `local_db`: the satisfied and declined matrix per shape, `NULL` legacy rows, the missing-`is_active` branch, missing-table fail-closed, batch bound, stable ordering |
| `tests/unit/services/acceptance/grant.py` | unit | `local_db`: a won claim writes the gate column and the ledger row; a lost claim writes nothing; a withdrawn ledger is never converged; the written row matches `set_consent` field for field (clause I4); the audit row carries `actor_kind="service"` and an existing event key |
| `tests/unit/services/acceptance/isolation.py` | unit | greps the source tree and asserts `devplacepy/main.py` is the only importer (clause I5), and that no template, schema, router or Devii catalog file mentions the package |
| `tests/api/admin/services/acceptance.py` | api | the service is listed at `/admin/services`, reports `stopped` on a fresh instance, its detail page renders the per-agreement fields, saving the config validates and persists, and a non-admin is redirected |
`tests/unit/services/acceptance/isolation.py` is the test that makes the invisibility contract survive future
edits. Without it, clause I5 is a promise; with it, it is a build failure.
**Do not write a test that enables the service and asserts convergence in the shared suite.** The suite runs
serially against one seeded database, and a service that grants `terms` to every account would poison the
moderation tests that assert the gate refuses. Convergence behaviour is covered by the unit tests calling
`converge_user` directly against `local_db`, which is where it belongs. The api test asserts the admin
surface only, and must leave `service_acceptance_enabled` untouched.
---
## 13. Operator procedures
To be written into the admin-only docs page, section 14.
**Enable it.**
1. `/admin/services/acceptance`, Configuration tab: switch on only the agreements the test run needs.
2. Leave dry run on. Start the service. Wait one interval, or use Run now. Read the Logs tab and confirm the
accounts listed are the ones expected.
3. Switch dry run off. The next run converges them.
**Test the refusal path for a consent.** From the tester's own profile, privacy tab, withdraw the consent. It
is never granted again while that withdrawal stands, regardless of how many intervals pass.
**Test the terms gate.** The tester withdraws their own `terms` consent, then an administrator bumps
`terms_version` at `/admin/settings`. Every other account converges; the tester stays gated.
**Test a partial state.** Enable `terms` and leave `privacy` off. Accounts pass the write gate but still show
the privacy policy as unaccepted on their privacy tab, which is the edge case the per-agreement switches
exist for.
**Undo a decline.** Grant the consent from the profile page. The ledger's latest row is `granted` again and
the account leaves the pending set; nothing to reset and no operator action on the service.
**Turn it off.** Stop the service, or switch off the individual agreement. Nothing already written is
reverted, and nothing should be: the accounts are in exactly the state a real population would have reached.
**Never enable it on production.** The description on the service page says so, and this is the only control
that matters. There is no environment detection anywhere in the design, deliberately: an environment flag
would be exactly the knowledge the application is not allowed to have.
---
## 14. Documentation
| File | Change |
|---|---|
| `devplacepy/services/acceptance/CLAUDE.md` | New. The nested subsystem doc: why the subsystem exists, the invisibility contract, the decline register and its ordering rule, the module map, the exclusions from section 4.1, the `has_column` rule from section 7.1, the single cache bump, the rules for extending it. |
| `CLAUDE.md` (root) | One row in the Subsystem map table. No architectural rule is introduced, so nothing else changes. |
| `devplacepy/services/CLAUDE.md` | One short section, sized and placed like the "Moderation housekeeping" section it sits beside. |
| `events.md` | Add `services/acceptance/grant.py` to the "Recorded in" column of `terms.accept` (line 72) and `consent.grant` (line 69). No new key, no category count change, no change to the header's key count. |
| `README.md` | One subsection under the background services material, stating that it is admin-only, off by default, and what it is for. |
| `devplacepy/templates/docs/moderation-operations.html` | One section carrying the procedures from section 13. The page is already `"admin": True` in `DOCS_PAGES` (`routers/docs/pages.py:195-199`), so it is invisible to members with no gating work. |
No `docs_api.py` entry and no Devii action, because there is no endpoint and no user-facing capability. This
is the one deliberate exemption from the four-faces rule in the root `CLAUDE.md`, and it is the exemption
every pure `BaseService` already takes: `NotificationRelayService`, `PresenceRelayService`,
`LiveViewRelayService` and `AuditService` have no routes, no schemas and no agent tools either. Their surface
is the generic services admin, which this service inherits without writing a line of template or JavaScript.
Exposing a Devii tool for it would additionally hand the assistant, which runs inside the application, direct
knowledge of the feature, breaking clause I5 for no benefit.
---
## 15. Definition of done
In order. A failure at any step blocks the next.
1. `python -c "from devplacepy.main import app"` imports clean.
2. `pyflakes` and `ruff check` clean on every touched file.
3. No em dash, neither the character nor the HTML entity, in any touched file.
4. The four verification layers of section 11 run and pass, including the sixteen-process race and both
branches of the missing-column matrix.
5. `make test` passes in full: unit, api and e2e, every test, no tier skipped and no subset substituted.
6. A manual check on a dev instance: with everything default, `/admin/services/acceptance` shows `stopped` and
the database is untouched after ten minutes. With the service started, dry run on and `terms` enabled, the
log lists the pending accounts and the database is still untouched. With dry run off, the accounts converge
within one interval, the audit log shows `terms.accept` rows with actor kind `service`, and an account
whose `terms` consent was withdrawn is still gated.
---
## 16. Non-goals
- No environment variable, no build flag, no `ACCEPTANCE_MODE` of any kind. The service is the entire
mechanism.
- No fabrication of declared facts: age band and date of birth are never written.
- No new consent kind, no new agreement outside `CONSENT_KINDS`, and no second decline register.
- No convergence of guests, of deleted or anonymised accounts, or of deactivated accounts.
- No reversal path. The service grants; only a human withdraws.
- No route, no JSON schema, no template, no Devii tool, no API documentation entry.
Reference in New Issue View Git Blame Copy Permalink