# 10 - Workflow, Testing Discipline, Living Example

Scope: plan.md sections 8-9, 10 step 7. Topics: specs, AGENTS.md, autonomous git, integration testing, living example.

## 1. Purpose and philosophy

This file defines how MoloApp is built, not what it does. Feature scope lives in siblings 01-runtime.md through 09-cli-ops.md; this file owns the process that keeps those features consistent: structure-driven development (every component has a written spec), an agent-executable workflow (`AGENTS.md`), autonomous versioning and release, integration-only testing through the real stack, and one living example project that proves every feature works end to end.

Philosophy, from plan.md section 9:

- Structure first: no code without a spec declaration somewhere; docs state what the system does today, gaps listed next to the code that closes them.
- Isolation with DRY integration: components depend only on declared interfaces, shared behavior lives in exactly one base class or component.
- Autonomous by default, human on floors: the AI agent builds, tests, versions, commits, tags, deploys and verifies in one session; humans intervene only on conflicts, protected paths, retention windows, destructive ops, and release failures.
- A change that is not running is not done: commit, deploy, verify, same session.
- User-facing simplicity is a release gate; responsive-by-default is a ship blocker for every template and example UI.

## 2. Best practices (from research)

1. AGENTS.md is a README for agents: a dedicated predictable place for build steps, tests, and conventions, kept separate so READMEs stay human-focused (https://agents.md/, retrieved 2026-10-23). MoloApp adopts this split literally: root `AGENTS.md` is normative for the agent; `README.md` is human marketing plus quickstart.
2. Conventional Commits gives the subject line a machine-parseable structure `<type>[optional scope]: <description>` with optional body and footers, where `fix:` maps to PATCH, `feat:` to MINOR, and `BREAKING CHANGE:` footer or `!` marker to MAJOR (https://www.conventionalcommits.org/en/v1.0.0/, retrieved 2026-10-23). MoloApp uses exactly this mapping for autonomous bumps.
3. Semantic Versioning 2.0.0 defines MAJOR for incompatible API changes, MINOR for backward-compatible additions, PATCH for backward-compatible fixes (https://semver.org/, retrieved 2026-10-23). Triangulated with (2) and (5): commit type is the sole version signal, no manual version choice.
4. Keep a Changelog requires one `CHANGELOG.md`, Unreleased section at top, `Added/Changed/Fixed/Removed` groups per release, ISO 8601 dates, and commit-log dumps explicitly rejected (https://keepachangelog.com/en/1.0.0/, retrieved 2026-10-23). MoloApp generates the changelog from parsed commits but edits entries into human-readable form before tagging.
5. semantic-release automates determining the next version, generating release notes, and publishing, removing the connection between human emotions and version numbers and running only after successful CI (https://semantic-release.gitbook.io/semantic-release/, retrieved 2026-10-23; canonical docs now at https://semantic-release.org/). Triangulated with (2)+(3): MoloApp release pipeline runs only on a green suite and derives version purely from commits since last tag.
6. FastAPI documents testing via `TestClient(app)` exercising real routes with `client.get/post` and status plus body assertions (https://fastapi.tiangolo.com/tutorial/testing/, retrieved 2026-10-23). Django documents simulating requests through the full stack (URL, view, middleware, form, template) with its test Client (https://docs.djangoproject.com/en/5.0/topics/testing/, retrieved 2026-10-23). Triangulated: MoloApp integration tests follow the same black-box route-level pattern, but stricter (no isolated unit tests at all; every test hits a URL end to end, see 08-clients-openapi.md for the test client itself).

## 3. Structure

Deep structure, one class or concern per file, namespaces mirror directories.

```
MoloApp/
  AGENTS.md                  # normative agent workflow (this file specifies its content)
  README.md                  # human quickstart (no agent instructions)
  CHANGELOG.md               # Keep a Changelog format, agent-maintained
  VERSION                    # single line MAJOR.MINOR.PATCH, agent-maintained
  Makefile                   # canonical targets: build, test, lint, docs, release, verify
  specs/
    SPEC-TEMPLATE.md         # normative template (see section 6.1)
    runtime.md               # one spec per sibling area, mirrors 01-runtime.md through 09-cli-ops.md
    observability.md
    security.md
    data.md
    request-routing.md
    realtime.md
    supervisor-replication.md
    clients-openapi.md
    cli-ops.md
    workflow-example.md      # this plan, once accepted
  tools/
    release/                 # autonomous git/version/changelog pipeline (behavior here, CLI UX in 09-cli-ops.md)
      ReleasePlanner.hpp     # molo::workflow::ReleasePlanner: commits since tag -> bump decision
      ChangelogWriter.hpp    # molo::workflow::ChangelogWriter: commits -> CHANGELOG entries
      ReleaseVerifier.hpp    # molo::workflow::ReleaseVerifier: green-suite + live checks before tag
  tests/
    integration/             # ONLY test location; mirrors app routes (see 6.3)
      CASE-TEMPLATE.md       # normative test-case template
      health_test.cpp
      <router>/
        <route>_test.cpp
  examples/
    living/                  # the big living example project (see 6.4)
      App.cpp                # mounts every demo sub-app (application chaining demo)
      routers/
      views/
      templates/
      static/
      tests/                 # example suite, same templates as tests/
      README.md              # feature coverage matrix with links to routes
```

Namespaces: `molo::workflow` for ReleasePlanner, ChangelogWriter, ReleaseVerifier. No other framework namespace may contain workflow logic. Test files use `molo::test` helpers only. Example app uses `molo::example::` sub-namespaces per demo area.

## 4. Rules (hard, numbered)

1. No feature is done until: spec updated, docs updated, living example demonstrates it, full suite green, all in the same session.
2. Integration tests only: every test exercises a URL end to end through real HTTP, routing, middleware, and DB. Isolated unit tests are forbidden.
3. Test placement mirrors the app: test file path and name derive from the URL/router under test; exactly one structured place per route.
4. Conventional Commits format is mandatory for every commit; malformed commits are rejected by the pipeline before version analysis.
5. Version numbers are never hand-chosen; the bump derives solely from commit types since the last tag (feat -> minor, fix -> patch, BREAKING CHANGE or `!` -> major).
6. Tagging requires a green suite plus live verification after deploy, same session; failed verification blocks the tag.
7. Destructive git ops (history rewrite, force push, hard reset) are never autonomous: dry-run plus explicit human confirmation, no exceptions.
8. Protected floors the agent must never cross alone: `specs/`, `CHANGELOG.md`, `VERSION`, tags, retention windows, destructive file deletes; on a floor hit, stop and page a human.
9. Every shipped template, example UI, and generated page works on small phones; big button groups collapse to dropdowns or equivalent, verified in the example suite.
10. Specs are normative: on any conflict between code and spec, the spec wins until formally amended in the same change.

## 5. Development guidelines

- Write the spec delta first, then code, then example coverage, then tests, then changelog entry, then commit. Never commit code whose spec is stale.
- Keep changes small and reviewable: one router or one behavior per commit; squash agent thrash before tagging.
- Prefer static ceilings over autotuning everywhere (timeouts, caps, intervals are constants in specs, visible in `AGENTS.md`).
- Loud skips: if input, fixture, or environment cannot be verified, fail the test with a clear reason and counter, never silently pass.
- Reproduce bugs through the real stack first (failing route-level test), then fix at root cause; the reproduction test stays in the suite.
- Docs use fixed templates; every doc names what the system does today and lists gaps next to the code that will close them.
- `AGENTS.md` is the single entry point for autonomous work: any workflow an agent must run (build, test, lint, docs, bump, commit, tag, deploy, verify) is invocable from there with exact commands.

## 6. Specifications

### 6.1 Spec template (specs/SPEC-TEMPLATE.md)

Every spec file contains, in order: Status (draft/accepted/deprecated) and date; Behaviors (numbered, testable); Caps (every rate, size, count, interval, memory ceiling as literal constants); Formats (wire, file, log, error shapes); Errors (exception type per failure, status mapping); Isolation (declared interfaces it may depend on, explicit non-dependencies); Gaps (what is not true yet, with owning file). Max 500 lines per spec; split by sub-concern beyond that.

### 6.2 AGENTS.md content and autonomous git pipeline

`AGENTS.md` sections, in order: Setup commands; Build; Test; Lint; Docs update; Version bump; Commit; Tag; Deploy; Verify live; Floors and escalation. Each section gives one copy-paste command (Makefile target) plus expected output and failure action.

Commit format: `<type>[(scope)][!]: <description>` plus optional body plus `BREAKING CHANGE:` footer, per Conventional Commits 1.0.0. Allowed types: feat, fix, docs, style, refactor, perf, test, build, ci, chore. Scope is the sibling area slug (runtime, security, data, routing, realtime, supervisor, clients, cli, workflow).

Bump mapping: any `!` or `BREAKING CHANGE` -> MAJOR; else any `feat` -> MINOR; else any `fix` -> PATCH; else no release. `VERSION` file and annotated git tag `vMAJOR.MINOR.PATCH` update atomically with the changelog entry.

Changelog: Keep a Changelog groups (Added, Changed, Fixed, Removed) under `## [x.y.z] - YYYY-MM-DD`, Unreleased section always present, ISO dates only.

Pipeline order: lint -> full integration suite -> bump decision -> changelog -> commit -> tag -> deploy -> live verify. Any red step halts the pipeline; the agent reports, never force-pushes past red. CLI command surface for these steps is specified in 09-cli-ops.md; this file owns the ordering, gating, and format rules.

Caps: max 1 release per green suite run; changelog entry max 20 lines per version; commit subject max 100 chars; pipeline timeout 30 min, then human escalation.

### 6.3 Integration test discipline

Location: `tests/integration/<router-path>/<route>_test.cpp`. Naming: route `/users/:id` -> `users/id_test.cpp`. Fixtures: DB/ORM fixtures plus frozen time via the test client from 08-clients-openapi.md; no real network except loopback server under test.

Case template (CASE-TEMPLATE.md): Arrange (seed via app API, never direct DB writes unless testing self-healing); Act (one HTTP request through the test client); Assert (status code, content-type, body shape, plus one side-effect check: DB row, metric emitted, log line). Every route needs at minimum: happy path, auth failure, validation failure, and one limit case (body too large, rate limited, or not found).

Caps: full suite max 10 min on reference hardware; single test max 30 s; fixture DBs reset per file, never shared across files; flaky test policy is zero tolerance (quarantine file plus spec gap entry, fix within one session).

### 6.4 Living example project (examples/living/)

Must demonstrate, each reachable via a documented URL and covered by `examples/living/tests/`: routing, views, routers, ORM both backends, auth, sessions, forms, validation, templates, all response formats, streaming/passthrough, WebSocket plus Socket.IO, pubsub, events, metrics, telemetry, KV, cache, uploads, WebDAV, replication, supervisor plus background tasks, REST client self-call, OpenAPI page, embedded static assets, AI harness CLI sample, structured integration tests.

Coverage matrix in `examples/living/README.md`: table of feature -> route -> test file -> spec link. Matrix row count equals feature count; CI fails on missing row. Example must compile warning-free and stay green as part of the normal suite; it is also the manual post-deploy verification target (open index, click through matrix top 10 routes on a small-phone viewport).

## 7. Execution steps (in build order)

1. Write `specs/SPEC-TEMPLATE.md` and stub specs for all areas.
2. Write root `AGENTS.md` with build/test/lint/docs/bump/commit/tag/deploy/verify commands wired to the Makefile.
3. Implement `molo::workflow` release pipeline (ReleasePlanner, ChangelogWriter, ReleaseVerifier) with dry-run mode.
4. Write `tests/integration/CASE-TEMPLATE.md` plus first health-route test proving the harness works.
5. Scaffold `examples/living/` with index, coverage matrix, and one route per area stub.
6. Fill example coverage feature by feature alongside siblings 01-runtime.md through 09-cli-ops.md; each feature lands with spec, example route, and tests in one session.
7. Harden pipeline: floor guards, protected paths, destructive-op blocks, timeout and escalation paths.
8. First autonomous end-to-end run: green suite -> bump -> changelog -> tag -> deploy -> live verify -> matrix check.

## 8. Validation (how to verify each step)

1. Template check: every spec file parses against the section order in 6.1; reject on missing Caps or Gaps.
2. Agent replay: a fresh agent following only `AGENTS.md` reaches a green build plus green suite with zero questions.
3. Pipeline dry-run: on a fixture repo with known commits, bump decision and changelog output match expected version and groups exactly.
4. Harness proof: health-route test passes and fails correctly when the route is broken (mutation check).
5. Matrix audit: every matrix row resolves to a live route, a test file, and a spec anchor; script fails on dangling links.
6. Per-feature gate: rule 1 checklist enforced by CI (spec diff plus example diff plus tests diff present in the same release).
7. Floor drill: attempt a protected-path write and a force push in dry-run; both must refuse and request human confirmation.
8. Live verify: after deploy, automated health plus metrics plus trace checks pass and the example index loads on a 360px viewport.

## 9. References

Depends on: 09-cli-ops.md (CLI command surface the pipeline invokes, dev server, Makefile targets); 08-clients-openapi.md (test client, fixtures, time freezing used by every test); 02-observability.md (log format, metrics, trace linkage asserted in tests). Feeds: all siblings 01-runtime.md through 09-cli-ops.md (spec template, test template, release gates, and example coverage apply to every area). Never duplicates: CLI flags and UX (09-cli-ops.md), test-client API (08-clients-openapi.md), metric/log schemas (02-observability.md).

## 10. Open questions

1. Reference hardware for the 10-minute suite cap: which CI runner SKU is normative?
2. Deploy target for live verification: single binary on loopback, or staged multi-instance per 07-supervisor-replication.md?
3. Should `BREAKING CHANGE` commits require a human co-sign before MAJOR tagging, or stay fully autonomous?
4. Example backend default: SQLite for speed with Postgres nightly, or both on every run?

## 11. Sources

- AGENTS.md open format, https://agents.md/ (retrieved 2026-10-23)
- Conventional Commits 1.0.0, https://www.conventionalcommits.org/en/v1.0.0/ (retrieved 2026-10-23)
- Semantic Versioning 2.0.0, https://semver.org/ (retrieved 2026-10-23)
- Keep a Changelog 1.0.0, https://keepachangelog.com/en/1.0.0/ (retrieved 2026-10-23)
- semantic-release docs, https://semantic-release.gitbook.io/semantic-release/ (retrieved 2026-10-23)
- FastAPI testing tutorial, https://fastapi.tiangolo.com/tutorial/testing/ (retrieved 2026-10-23)
- Django testing docs, https://docs.djangoproject.com/en/5.0/topics/testing/ (retrieved 2026-10-23)