ad/DEVELOPMENT_FLOW.md

<!-- retoor <retoor@molodetz.nl> -->
# Development flow lessons

## Purpose

This file is the shared handover of teachings and knowledge that may feed
the DPP project template. Agents record generic, project-agnostic lessons
here so no good preference or idea is lost. Grok alone folds accepted
lessons into the normative template for future projects.

Normative fold-in: lessons below are reflected in
`/workspace/dpp/dptemplate.md` where Grok has folded them. Where a lesson
could not override an existing template MUST or MUST NOT, the template
wins and the residual is recorded in `/workspace/dpp/CONFLICTS.md`. This
file remains the lesson source; the template remains the conformance
authority.

## Who may edit what

| Surface | Who | Rule |
|---------|-----|------|
| `/workspace/dpp/dptemplate.md` | Grok Bot only (`grok-bot` / AbsurdDevelopmentProject) | Only Grok updates the frozen template, and only on explicit user order. |
| `/workspace/dpp/CONFLICTS.md` | Grok Bot only | Written when a lesson conflicts with an existing template MUST. |
| `DEVELOPMENT_FLOW.md` (this file) | Any agent on the project | Append new lessons. Do not delete existing lesson content. Reformat only when the user orders a format change. |
| Project code and docs | Agents per retoor's instructions | Continue the project from available knowledge plus user direction. |

Tone for every entry: generic, consistent, dry. No product names in the
lesson body when the rule is meant for the template. Name a product only
when stating a concrete observation that cannot be generalized yet.

## Lesson format

Every lesson is one subsection under [Lessons](#lessons). Fields are
fixed. Body text is prose plus bullets. Status values:

- `proposed`: recorded, not yet folded into the template
- `folded`: reflected in `dptemplate.md`
- `conflict`: folded where possible; residual conflict kept in
  `CONFLICTS.md` (template rule wins)

Every lesson MUST record whether it became part of the DPP template:

- `dpp_template`: exactly `yes` or `no` (no other values)
- `dpp_reason`: one short dry line. If `yes`, cite the template
  section(s) and note any residual conflict. If `no`, state why it was
  not folded (blocked, meta-only, withdrawn, or still proposed).

These two fields are mandatory on every new entry from now on.

### Skeleton

```markdown
### L-YYYY-MM-DD-NN: Short title

| Field        | Value |
|--------------|-------|
| date         | YYYY-MM-DD |
| agent        | grok-bot \| muse-spark \| <other-id> |
| status       | proposed \| folded \| conflict |
| dpp_template | yes \| no |
| dpp_reason   | short dry reason |

Body starts here. Project-agnostic. Dry. Complete enough that Grok can
turn it into MUST / MUST NOT text without asking again.
```

### Example entry

```markdown
### L-2026-10-05-00: Example (format only)

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | grok-bot |
| status       | proposed |
| dpp_template | no |
| dpp_reason   | format example only; not a real lesson |

Applications MUST refuse to start when the configured data directory
resolves outside `./data`. Prove it with a maintained subprocess test
against an inside path (allowed) and an outside path (refused).
```

Append new lessons at the end of [Lessons](#lessons). Never renumber or
delete prior entries. If a later lesson revises an earlier one, add a new
entry that cites the earlier id.

## Lessons

### L-2026-10-05-01: Storage law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | conflict |
| dpp_template | yes |
| dpp_reason   | sections 3.1 (data root, enforcement, no named volumes); residual CONFLICTS.md 1 and 4 |

All application data lives under `./data/<env>` inside the project. No
exceptions, no second locations.

- `development` for local runs, `staging` and `production` as container
  file mounts, `test` for persistent test services.
- Ephemeral runs use dotted subdirectories that clean themselves:
  `./data/.tmp-pytest-<pid>`, `./data/.tmp-locust`,
  `./data/.tmp-shots-<pid>`.
- Each env dir holds the database file plus one subdirectory per kind
  (`uploads`, `backups`, `keys`, `locks`, `seo_reports`, ...). Database
  and files are separated by directory, never by system location.
- Enforcement beats convention: the application refuses to start when its
  data dir resolves outside `./data` or its sqlite file outside the data
  dir. Prove it with a maintained test that runs the real import in a
  subprocess against inside and outside paths.
- Named docker volumes are forbidden. Every mount is a file mount.
  Containers run as uid/gid 1000 so files stay human-owned; bind host
  directories must pre-exist with correct ownership because docker
  creates missing ones as root.

Source commits (approx.): `44d0c08`, `4268b1e`, `60dd597`. Residual
conflict with template bare-metal shared-DB and "data volume" wording:
see `CONFLICTS.md` items 1 and 4.

### L-2026-10-05-02: Environments

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | conflict |
| dpp_template | yes |
| dpp_reason   | section 3.5 (envs and one-way snapshots); residual CONFLICTS.md 1 vs bare-metal 28.1 |

Four environments, one shape each, fixed localhost ports (example
offsets: staging 19847, production 19848, test 19849).

- Production owns the holy database. Nothing writes to it except
  production itself.
- Staging is refreshed from production snapshots only: online sqlite
  backup API while prod keeps running, integrity check, stop, swap,
  start, verify health. Never the reverse direction.
- Test is hermetic: persistent service data for the public test site,
  wiped or temp dirs for gallery and suite runs.
- Snapshots flow one way: production to staging. Test never touches
  production data.

Source commits (approx.): `4268b1e`, `60dd597`. Residual conflict with
template section 28.1 (`make dev` / `make prod` share the production
database): see `CONFLICTS.md` item 1.

### L-2026-10-05-03: Docker law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | conflict |
| dpp_template | yes |
| dpp_reason   | section 28.2 (per-env compose, PROD_ :? guards, bind mounts); residual CONFLICTS.md 2 and 4 |

One compose file per environment, each with its own top-level `name:` so
projects can never steal each other's containers.

- Dev and test run the dev server (`uvicorn --reload`) with the package,
  the manifest (`pyproject.toml`), and `./data/<env>` mounted: every
  saved change is live within seconds.
- Production runs workers, no reload, no source mount except the data
  dir.
- Production secrets use dedicated `PROD_`-prefixed variables with `:?`
  guards, so the dev `.env` can never silently satisfy them. Missing
  secrets fail loudly at startup, never with defaults.

Source commits (approx.): `4268b1e`, `60dd597`. Residual conflict with
template zero-config boot and `SECRET_KEY` hardcoded fallback: see
`CONFLICTS.md` item 2.

### L-2026-10-05-04: Environment files law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | conflict |
| dpp_template | yes |
| dpp_reason   | section 3.4 (env-file contract); residual CONFLICTS.md 2 on secret defaults |

`.env.example` lists every variable the code reads, no more, no less. A
maintained contract test asserts the exact key set plus non-empty values
for every path-like key, and fails on the old file before passing on the
new one.

- Never leave a key empty when empty overrides a good default (paths,
  binaries, URLs): use safe literals (`./data/development`, `rclone`).
- The private `.env` is generated from the example with fresh random
  secrets, mode 600, never committed. The admin password applies at
  first bootstrap only and is reported to the owner once.

Source commit (approx.): `60dd597`. Tied to `CONFLICTS.md` item 2 for
secret defaults versus loud failure.

### L-2026-10-05-05: Serving law (molohttp)

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | folded |
| dpp_template | yes |
| dpp_reason   | section 28.4 (front-door reverse proxy); molohttp specifics remain lesson-local |

One front door on 80/443, all local reverse-proxy upstreams.

- Back up before every edit, one format only:
  `<file>.bak-<reason>-<YYYYMMDDHHMMSS>`, `root:root 640`, reason slug
  required, never deleted.
- Assert unique site ids AND unique hostnames before appending: on
  reload molohttp keeps the first site per hostname and silently drops
  later duplicates. Re-read the file after reload and assert the new
  site survived.
- Change order: backup, edit, restore `molohttp:molohttp 640` ownership,
  validate JSON, reload, verify the site over HTTPS. Rollback is the
  same steps with the previous values; the old backend stays running
  until rollback is no longer wanted.
- The admin surface is never exposed publicly. Local CLI only.

Source commits (approx.): `60dd597` (operator notes), production
cutover work. Folded into template section 28.4 in product-agnostic
form (front-door reverse proxy); the molohttp-specific ownership and
JSON validate steps remain the lesson source here.

### L-2026-10-05-06: Gallery law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | folded |
| dpp_template | yes |
| dpp_reason   | section 28.5 (screenshot galleries) |

Every environment gets a screenshot gallery, generated the same way
every time.

- Boot mode builds a throwaway server on a fresh data dir and seeds one
  fixed record; base-url mode shoots a live environment and never writes
  seed data.
- Full-page desktop shots plus a responsive matrix (down to 320px) with
  a horizontal-overflow check per cell; overflows are reported and exit
  nonzero.
- One gallery per environment plus a hub page that rebuilds from
  whichever environments exist. Same script, same selectors, same waits
  for all.

Source commits (approx.): `4268b1e`, `60dd597`. Folded into template
section 28.5.

### L-2026-10-05-07: Test law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | conflict |
| dpp_template | yes |
| dpp_reason   | sections 26.1, 26.3, 27.2 (tests, shared retry, doc text gates); residual CONFLICTS.md 3 (15s vs 60s) |

Mirror layout (`tests/unit`, `tests/api`, `tests/e2e`), full suite green
before done.

- Browser tests wait up to 60 seconds per step and retry a stalled page
  load once through one shared helper. Filter only proven environmental
  console noise, never application errors.
- Static gates run on every change: imports, linter, syntax, schema
  lint, UI invariants. Text gates (for example the em-dash ban) apply to
  docs too.
- Every behavior change ships the smallest maintained test exercising
  it, proven red before the fix and green after. Never weaken a failing
  test to get green.

Source commits (approx.): `94ae750`, `60dd597`. Residual conflict with
template 15s page-fixture default timeout: see `CONFLICTS.md` item 3.
Shared retry helper and related gates were folded.

### L-2026-10-05-08: Git law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | folded |
| dpp_template | yes |
| dpp_reason   | sections 2 and 36 (gitignore list, git law) |

Stage named files only, never broad-add a dirty tree. Commit and push
only on explicit ask, never as a side effect of finishing.

- `.gitignore` contains exactly: `.venv/`, `__pycache__/`, `*.pyc`,
  `.pytest_cache/`, `.coverage*`, `data/`, `out/`, `.env`,
  `*.egg-info/`, `build/`, `dist/`, `temp_*`, `*.swp`, `*~`,
  `.DS_Store`. Never ignore source, tests, docs, compose files,
  Dockerfiles, `.env.example`, Makefiles, or CI configs.
- Any scratch, probe, or temporary file must be prefixed `temp_` and is
  never committed. `temp_*` in `.gitignore` enforces it.
- Markdown files are never committed without explicit user confirmation.
  Docs are reviewed prose, not byproducts.
- A pre-commit hook owns the patch version bump; nothing else edits the
  version.

Source commit (approx.): `60dd597`. Folded into template sections 2 and
36.

### L-2026-10-05-09: Code law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | folded |
| dpp_template | yes |
| dpp_reason   | section 32 (code conventions); already largely present, kept as lesson source |

Explicit over implicit, flat over nested, sparse over dense, readable
above all. No comments anywhere except the author line
`retoor <retoor@molodetz.nl>` at the top of every file; code that needs
a comment is rewritten until it does not. Defensive everywhere, errors
never pass silently, no invented names with suffixes. JavaScript is ES6
modules, one class per file, one `Application` instantiated as `app` on
`window`. Storage is dataset plus SQLite. The frontend is served through
the backend. No emojis, no hype language, professional and scientific.

Source commit (approx.): `60dd597`. Already largely present in the
template; kept here as the lesson source.

### L-2026-10-05-10: Docs law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | folded |
| dpp_template | yes |
| dpp_reason   | section 32 (README entry point and operator-guide bullets) |

`README.md` is the living entry point (running, testing, docker,
screenshots, data, operations) and stays true to the repo. Each
deployment gets an operator guide with access facts, file owners, backup
specification, and toggle procedures. Document only inspected facts,
mark dated observations, and record anomalies instead of hiding them.

Source commits (approx.): `4268b1e`, `60dd597`. Folded into template
section 32.

### L-2026-10-05-11: Handover law

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | folded |
| dpp_template | yes |
| dpp_reason   | section 37 (secrets handover) |

Secrets are generated with a cryptographic RNG, stored only in their
target (`.env` 600, container env), and reported to the owner once in
chat, never written to the repo, docs, or logs beyond that handoff.

Source commit (approx.): `60dd597`. Folded into template section 37.

### L-2026-10-05-12: Multi-agent lesson format for DPP feed

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | grok-bot |
| status       | folded |
| dpp_template | no |
| dpp_reason   | process meta for this lesson log; not a template MUST |

This file is reformatted as the shared multi-agent lesson log. Grok Bot
(AbsurdDevelopmentProject) alone updates `/workspace/dpp/dptemplate.md`.
Other agents append generic lessons here. Existing storage,
environments, docker, env-files, serving, gallery, test, git, code,
docs, and handover laws are preserved as dated `muse-spark` entries
above. New teachings use the skeleton under [Lesson format](#lesson-format).
No lesson content may be deleted when the format evolves.

### L-2026-10-05-13: Restart reload-based dev servers after every pull

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | proposed |
| dpp_template | no |
| dpp_reason   | proposed; Grok folds into template deploy section |

File-watching dev servers restart on the first changed file they notice.
A version-control update writes files one by one, so the restart can
fire mid-checkout and boot workers from a mixed old/new tree with no
further restart once the checkout completes (observed: health reported
the previous version while new code was partially loaded).

After every pull or checkout on a host running reload-based servers,
restart those servers (or their containers) and verify the reported
version matches the checked-out tree before declaring the deploy done.
Production targets without reload are unaffected but still verify.

### L-2026-10-05-14: Autonomous development rule

| Field        | Value |
|--------------|-------|
| date         | 2026-10-05 |
| agent        | muse-spark |
| status       | proposed |
| dpp_template | no |
| dpp_reason   | proposed; agent operating rule, Grok decides the template fold |

Standing instruction from retoor, recorded as a clear rule:

- Move one way only: forward. Do the maximum-effort option that adds
  no degradation and deletes no functionality.
- Whatever is unclear after the user's answers is resolved by online
  deep research first; the agent then picks the research-backed option
  and continues autonomously instead of stalling on questions.
- Research grounds the choice (example: invite tokens use hash-only
  storage, single use and expiry per current security practice), it
  never replaces verification against the repo's own code and tests.