Files
moloapp/specs/example-site.md
retoor 0595284fe3 feat(doos): add container runtime, audit cases, example demo and tests
- New rootless doos container runtime: sandbox, cgroup, network, images, hardening, davmount, CLI plus molo integration
- 18 CVE audit reproductions with READMEs and Makefiles
- Example DoosDemo routes, IndexCatalog, static assets, spec update
- CLI serve on 0.0.0.0:8000, arg parser, dev server improvements
- Runtime HTTP fixes, supervisor isolation backend
- Tests for doos, example doos routes, CLI serve; suite count 261 to 265
2026-11-06 04:36:00 +01:00

25 KiB

Example-Site Specification Draft

Status: draft for review. Source: synthesis of workflow evidence P1-P6 (structure, tests, OpenAPI drift, reference-site reuse, error/UX audit, falsification gaps). No live probes were run; counts and bodies cited below are source/grep-level and need the falsification reruns in Section 11 before this becomes normative.

Conventions: MUST/SHOULD/MAY per RFC 2119. This spec wins over code on conflict until amended in the same change.

1. Scope, goals, non-goals

Goals:

  • One example site that demonstrates every framework module through a consistent shell, nav, page, form, error, and JS contract.
  • Commercial and scientific tone: direct, factual, user friendly; no complexity pushed to the visitor.
  • Responsive at 360px viewport minimum; narrow screens get dropdowns/collapse, never hidden nav with no fallback.
  • Fully specified: every shared component, envelope, tag, and automation gate is declared here.

Non-goals:

  • Production auth hardening and threat modeling (out of scope for now).
  • Live vendor calls in default suite (paid/live routes stay dry-run or skipped without keys/network).
  • Rewriting framework modules; the example consumes them via the documented demo mounts.

2. Quality standards, coverage targets, caps, floors

2.1 Test standard

  • Example suite is integration-only; isolated unit tests are forbidden. Arrange/Act/Assert with Fixture plus TestClient, PASS:/FAIL: protocol (tests/README.md).
  • Target: example suite: 265 passed, 0 failed (AGENTS.md). Current inventory: 265 files under tests/example, 285 under tests/src. Any count change MUST update this spec plus AGENTS.md in the same change.
  • Per-route minimum (4 cases): happy path, auth-fail or 404/405, validation/422, limit/edge. Applies to every mounted route including /readyz, /startupz (currently misplaced in probes_test.cpp; MUST move to tests/example/readyz and tests/example/startupz).
  • Matrix test: GET every index href, require 2xx/3xx (401 allowed only for /auth/me), minimum row count declared (today >=37; index catalog carries ~40 features in 16 groups).
  • Loud skips: unverifiable input is rejected with reason plus counter, never silently stored.

2.2 Build and lint gates

  • Warning-free build with -std=c++23 -Wall -Wextra everywhere. -Werror MUST be universal; today it is missing in example, app, data, cli, observability, realtime, tenancy, audit, governance, supplychain .mk files. Closing that gap is required.
  • make lint = warning-free build plus repo-wide em-dash grep. No tidy/sanitizer/format/coverage gate exists today; until one lands, the lint target is the gate and MUST stay green.
  • Pipeline order: lint -> suite -> bump -> changelog -> commit -> tag -> deploy -> verify. Timeout 30 min, then escalate.

2.3 Coverage targets

  • Route coverage: 100 percent of mounted router registrations reachable from index matrix or docs table, or explicitly listed as excluded (paid/live prefixes /voice, /google, /agent, /ai excluded from locust by design).
  • OpenAPI coverage: 100 percent of mounted routes appear in served spec (today 3 manual adds vs ~490 registrations; see Section 9).
  • Component coverage: every named component in Section 5 has at least one route-level test rendering it.
  • JS coverage: every shipped asset JS file has a route test that loads its page plus a simulated POST exercising its err/out path.

2.4 Caps (static ceilings, no autotuning)

  • Suite: 10 min total, 30 s single test, zero flakes (tests/README.md).
  • HTTP demo payloads: 64 KB request body, 256 KB response body, 100 items per list page default.
  • Rate: 60 req/min per demo write endpoint when loopback deployed; 429 with Retry-After on breach.
  • Embedded bundle: 5 MB total under example/assets; single asset 512 KB max.
  • Gate output panels: 200 lines max with truncation notice; stats tiles: 12 max.
  • Locust headless run: 60 s, 10 users default.

2.5 Floors (never crossed alone; dry-run, stop, page human)

  • Protected: specs/, CHANGELOG.md, VERSION, git tags, retention windows, destructive file deletes, history rewrite, force push, hard reset.
  • Docs/off behavior: doc routes disabled by default in prod (prod 404 test required).
  • Destructive/paid demo calls: dry-run by default, explicit confirm to execute; no auto-spend.
  • Release: max 1 release per green run; failed live verification blocks tag.

3. Whole-site structure

3.1 Information architecture

  • Single shell (base.html) with sticky masthead, main, footer. One shell template only; merge page.html, ent_page.html, platform_page.html into one.
  • Global topbar MUST cover all 16 index groups (today only 7 links: Index/Templates/Forms/Stats/Voice/Docs/Health). Groups: routing, requests, data, security, static, realtime, observability, operations, doos, tenancy, audit, governance, google(+voice), agent, vendors(/ai), tools, plus Other fallback. Narrow screens use the existing details-dropdown pattern (keep; it beats the reference-site nav-hide).
  • Index page: grouped matrix via IndexCatalog render, filter input (no-JS safe), needs-API-key badges on google/agent/vendors groups.
  • Every detail page: section-head (numeral + lede), breadcrumbs (Index / Group / Page), back-link, section nav where page has 2+ sections.
  • Docs page: filterable register table (see Section 5), not a plain list.
  • Static vs embedded: example/static holds shell CSS/JS (app.css, app.js, index.js); example/assets holds per-demo JS bundled to build/example/gen/assets.cpp and served under /embedded/{name}.

3.2 Route map (mount contract)

  • Core: / (index), /health, /readyz, /startupz, /metrics, /telemetry, /routing*, /orm/users, /kv/items, /cache/lookup, /pubsub/publish, /auth/, /sessions/, /security/, /security-extra/, /formats/data, /templates/page, /forms/contact, /validate/user, /ws, /socketio, /sse, /streaming, /webdav, /expose, /supervisor/, /restclient, /clients/, /operability/, /cli/, /workflow/release, /supplychain/sbom.json, /openapi.json, /docs, /stats, /stats/data, /voice, /voice/say|hear|turn|reset, /google/status, /doos, /doos/status, /doos/create, /doos/delete, /tenancy/, /audit/, /governance/*, /v1|v2/items.
  • Agent (9, /agent/): models, runtime, sessions, sandbox, registry, armor, optimize, adk, live.
  • AI (20, /ai/): xai, xaiplatform, anthropic, anthropicadmin, meta, deepseek, openai, openaiplatform, openaiadmin, openrouter, compat, mistral, cohere, perplexity, bedrock, azure, huggingface, audio, media, retrieval.
  • Google (7, /google/): discovery, cloudai, enterprise, data, maps, workspace, ops.
  • Contract for agent/ai/google: GET page plus status returns 2xx always; POST/DELETE without key returns 503 with machine-readable not_configured/google_not_configured; validation failures return 422; paid/destructive runs dry-run unless confirm:true.
  • Doos: GET page, GET status JSON, POST create, POST delete with confirm=yes (else dry-run). Demo store: DOOS_HOME else build/doos-demo.

3.3 Build, deploy, verify-live

  • Standalone: make -f example/example.mk example|example-test|example-clean|example-locust. Whole repo: make, make test, make lint via root Makefile aggregation.
  • Deploy: loopback living-server on EXAMPLE_PORT (default 8000), pid build/example/run/server.pid, health-gated; verify via tools/verify_deploy.py.
  • Verify-live gate (manual today, MUST stay manual until wired): health probe, /metrics scrape, index 360px markup checks (viewport meta, .nav-narrow collapse), crawl first 10 matrix links. Any red step halts the pipeline.

4. Per-module look (page contract)

Every demo page MUST follow this skeleton in order:

  1. Breadcrumbs plus section-head (numeral, title, one-line lede, status pill if the module has state).
  2. Config/status strip: molo-config-status element (configured/not-configured/live/dry-run) for key-gated modules.
  3. How-it-works block: steps or tabs plus callout for paid/dry-run notes.
  4. Interactive form(s): shared FormView builder only (Section 7); one primary submit; confirm control per Section 7 for paid/destructive.
  5. Output area: molo-json-out (codeblock in panel-dark) plus molo-alert region; run-button triggers fetch; output capped at 200 lines.
  6. Reference row: link to /docs entry plus OpenAPI operation id.

Look rules:

  • One section class vocabulary from Section 5; no per-vendor invented section classes or id prefixes.
  • No page-local <style> blocks and no inline style attributes; all styling lives in app.css tokens/components.
  • Global .primary button token works on every page (today dead except voice page; MUST fix in app.css).
  • Vendor pages share one JS module plus per-page config; no more ~30 copy-paste fetch/err/out files (see Section 8).
  • Delete or generate example/assets/app.css stub from the same source as example/static/app.css; two diverging stylesheets are forbidden.
  • CSP page MUST use the shared shell (nav + stylesheet), not a standalone document.

5. Reference-site component minimum map

Reference: seriousdevelopment single-page brief (static.molodetz.nl/seriousdevelopment). Minimum portable set, mapped to example targets:

  • REUSE-01 tokens + type trio: paper/wash/ink/hairline/oxblood/navy/panel vars; serif display, system sans body 17px/1.65, mono labels. Target: :root in example/static/app.css.
  • REUSE-02 dark masthead/footer bars with 3px accent rule. Target: kBase header/footer in Templates.cpp. Keep details-dropdown narrow nav.
  • REUSE-03 section-head numeral + lede. Target: index group heads and demo page heads.
  • REUSE-04 register table (mono ID / rule / check). Target: /docs route table and index group tables.
  • REUSE-05 status pills + stamp badges. Target: /health, /doos/status, supervisor/worker states, config strips.
  • REUSE-06 tabs + steps + callout. Target: how-it-works/mode blocks. ARIA tabs with arrow-key step, portable as IIFE.
  • REUSE-07 panel-dark + tree/codeblock + gate-output + run-button. Target: scaffold/API/code display and run-a-gate demos (/docs, /formats, /workflow/release). Timed line reveal with reduced-motion instant fallback is the model for live demo output.
  • REUSE-08 hero-stats strip. Target: stats tiles on index hero and /stats.
  • REUSE-09 data-component one-class-per-file JS with Application register/start. Target: organization of example/assets plus static JS; no-JS-safe defaults required.
  • REUSE-10 single-column collapse + overflow-x:auto code panels at <=760px. Target: app.css responsive rules checklist.

Named class vocabulary to adopt (minimum): masthead, nav, hero, hero-stats, figure-frame, titleblock, section, section-head, section-numeral, finding, stamp, pillar, tabs, tablist, tabpanel, steps, callout, terminal-head, run-button, gate-output, track-controls, status + modifiers, stack-row, register, panel-dark, tree, codeblock, split, footer, footer-grid, footer-base.

Explicit non-reuse: reference nav-hide at <=760px (display:none, no fallback) is rejected; example keeps its dropdown.

6. Visual bar: rich and fancy standard

Floor (must pass):

  • 360px viewport: no horizontal scroll; single-column collapse per REUSE-10; code panels scroll internally.
  • Serif display headline with clamp(), mono kickers/IDs, hairline rules, squared corners, 3px accent bars.
  • Dark masthead/hero-figure/footer-panels for contrast rhythm; light paper body.

Rich bar (expected on index, /docs, /stats, /voice, and one page per agent/ai/google family):

  • Hero with kicker, headline, standfirst, dual buttons, 4-stat strip, framed figure with mono caption.
  • Status pills that reflect live state (not static text); stamp badges for HALT/DRIFT/dry-run states.
  • Tabs or steps with keyboard support; callouts for paid/key-gated notes.
  • Panel-dark code/output blocks with run-button and reveal animation (instant when prefers-reduced-motion).
  • Stats tiles fed by live /stats/data with empty-state and error-state designs (no bare hint spans).

Proof: every visual-bar claim needs a route-level test asserting the markup plus a 360px markup check in verify_deploy.py. Rendered browser screenshots are recommended but the markup gate is the normative one until a screenshot tool is granted.

7. Error, message, dialog, form standards

7.1 Error envelope (one standard)

  • Canonical JSON: {error, message} for all API errors (SPEC.md contract). Validation: {error: validation_error, failures: [{field, message}]} with 422.
  • Retire or map: bare {error: string}, {errors: [...]}, 3-key {error, field, message} variant, and all plain-text error branches (405 text, static-file text, HTTP-layer text, platform text). HTML clients get error.html rendered with the same fields; error.html MUST actually be rendered (today registered but never used).
  • problem+json {type, title, status, detail, instance}: everywhere or nowhere. Today enterprise-only; decide in implementation plan and apply uniformly.
  • Status codes: bad credentials uniformly 401 (today 403 in SecurityDemo vs 401 in SecurityExtraDemo; SPEC pins 403 and MUST be amended to 401 in the same change if 401 is chosen). Missing key: 503 with not_configured code. Unknown route: 404 JSON for API, error.html for browsers. Wrong method: 405 JSON with Allow header.
  • Content negotiation: one wants_html() helper everywhere; retire the Accept-only fork in /validate/user.

7.2 Messages and notices

  • One pattern: notice/error regions with role=status/alert plus aria-live on every page including /stats. Retire the three divergent success patterns (done-flag hide-form, lab-page notice paragraph, PRG 303 flash) in favor of: inline molo-alert region update for fetch flows; PRG flash only for full-page form posts, rendered through the same molo-alert component.
  • Async JS results use one toast component (stacked, dismissible, role=status/alert) replacing per-file err/out drift.

7.3 Dialogs and confirms

  • Zero native dialogs today; standard is -based molo-confirm: two-step destructive flow (arm + confirm), focus trap, Esc to cancel, return focus to invoker.
  • One confirm vocabulary: checkbox label Confirm (run paid or destructive calls; otherwise dry-run) plus dialog copy naming the spend/destruction. Retire the five divergent labels (spend money / really send non-read calls / dry run if paid / confirm=yes variants).
  • Dry-run default holds: unconfirmed paid/destructive POST returns the dry-run plan with 200, never 4xx, so clients can preview.

7.4 Forms

  • One builder: FormView plus form_page.html partial, novalidate, summary box plus inline per-field errors. Retire form_html() (no error display) and lab toolbar inline-style forms.
  • Invalid input re-renders the same template (no lab_page to form_page layout jump).
  • Binding: shared Form binder (query merged) everywhere; retire hand-parsing in /validate/user.
  • Every form has error summary region, per-field messages, and preserved values on failure; pinned by contact_test pattern extended to all forms.

7.5 JS error handling

  • Shared client module reads message with fallback chain: j.message else j.error (string) else j.detail else j.title else HTTP status. No more j.message-only degradation to HTTP .
  • No generic Request failed discards; no errors written into output
     without an alert region (ADK page MUST gain one; today it is the only aria-live in the repo and still misplaces errors).
  • Uniform output: JSON codeblock plus optional text preamble; xAI text+JSON vs siblings JSON-only divergence is resolved in favor of the shared renderer with a text slot.

8. ECMAScript custom-element tag strategy

Strategy: molo-* autonomous custom elements defined in one shared module (molo.js), used directly in server-rendered templates. Progressive enhancement: every tag has useful server-rendered fallback content; JS upgrades behavior.

Required tags (minimum set):

  • molo-topnav: global nav with wide links plus narrow dropdown; data-groups JSON; current-group highlight.
  • molo-breadcrumbs: Index / Group / Page trail from data attributes.
  • molo-alert: notice/error region; attrs type (ok/error/hint), role auto (status/alert); method show(text)/clear().
  • molo-field-errors: summary plus inline error binding; attrs errors JSON.
  • molo-confirm: -based confirm; attrs action-label, dry-run-note; events molo:confirmed/molo:cancelled.
  • molo-config-status: key/config pill; attrs state (configured/not-configured/live/dry-run), service label.
  • molo-json-out: codeblock output with line cap, copy button, truncation notice; attrs max-lines default 200.
  • molo-toast-stack: toast host; method push({type, text}).
  • molo-tabs: ARIA tabs wrapper (arrow-key nav) for how-it-works blocks.
  • molo-gate-output: reveal-into-output panel honoring prefers-reduced-motion.

Rules:

  • Tag names kebab-case with molo- prefix; one class per tag in molo.js (or one file per tag under static/components/ if molo.js exceeds 500 lines).
  • Templates use the tags as plain HTML; no framework-specific template syntax beyond existing ctx substitution.
  • No customElements polyfill required for evergreen; fallback content MUST remain readable with JS off.
  • Migration: replace ~30 copy-paste asset files with molo.js plus small per-page config scripts (data attributes preferred over new files).

9. openapi.json auto-sync

Problem (DRIFT-1..8): no committed artifact; served spec is a 3-operation paths-only template at 3.0.0 while the real builder mandates 3.1.0; tests assert both versions against different generators; ~490 routes unrepresented; DELETE/methods/schemas/status/security absent; /openapi.yaml and /redoc declared but never mounted; governance/S1-S10 and version wiring dead.

Target contract:

  • Served spec is OpenAPI 3.1.0, derived at startup from mounted routes, never hand-maintained.
  • One generator: the real Builder (src/clients/openapi) fed by a Route to RouteDecl adapter. The example/fw/Net.cpp template emitter is retired or reduced to a thin delegate.
  • Coverage: every router.get/post/del registration appears with path, method, summary, tags, parameters (query/header/cookie from route models), requestBody/response schemas where route declares models, and status codes observed from handlers (minimum: 200/201 success plus documented 400/401/404/422/503/500 where applicable).
  • Routes: /openapi.json (JSON), /openapi.yaml (YAML), /docs (Swagger UI), /redoc (ReDoc). Real UI shells replace the static placeholder divs; the part-09 embedder packs them or they load from vendored assets.
  • Versioning: info.version reads VERSION file at startup (today hardcoded in 3 places: Net.cpp, document.h ApiInfo default, caps.h kFrameworkVersion). No hardcoded doc versions.
  • Prod default: doc routes return 404 in production unless explicitly enabled; covered by a prod-404 test.

Automation:

  • Startup derivation: from_app-style walk over the mounted app router (build the missing adapter; plan name from_app does not exist yet).
  • Determinism/snapshot test: canonical JSON snapshot of the derived spec checked into tests/example/openapi/; any route change without snapshot update fails with a diff.
  • Lint: RouteLinter S1-S10 runs over real example routes in the suite (today unit-tested only, never executed over real routes). Example toy linter (path casing/verbs textarea) is retired or relabeled as a teaching demo that delegates to S1-S10.
  • Governance: Builder::use_governance_declarations wired to route gov_* fields; dead-code grep becomes a passing integration test.

10. Whole-site auto-maintenance

Beyond OpenAPI, these artifacts MUST be derived or checked, never hand-synced:

  • Index matrix: rendered from the same route registry that feeds OpenAPI; matrix_test cross-checks that every registry route is linked or explicitly excluded.
  • Docs table: same registry; columns route/methods/group/summary/status.
  • Locust coverage: check_locust_coverage.py stays a gate (static route-vs-locust check; fails on gaps, paid-route leaks, missing methods) and MUST be wired into test (today standalone only).
  • Embedded bundle: gen/assets.cpp regenerated on asset change; bundle-list test fails on unlisted files.
  • Nav groups: topnav data-groups generated from IndexCatalog groups; test asserts topbar link set equals catalog group set.
  • Confirm/error vocabulary: grep-based consistency test over templates and demo .cpps asserting only the canonical labels/envelopes exist (with an allowlist file for intentional exceptions).
  • Release: tools/release.py bump flow extended to regenerate the OpenAPI snapshot and fail on uncommitted spec drift before tagging.

11. Refactor automation plan

Order of work (each step: spec delta first if behavior changes, then code, then example route, then tests, then changelog):

  1. Falsification reruns (no code): make -n all/test/lint; find/grep reruns for counts (~490 routes, 265/285 files, ~35 .primary hits, 40 assets); live GET /openapi.json plus POST/DELETE 503 matrix plus HTML-vs-JSON error matrix; run matrix_test, check_locust_coverage.py, verify_deploy.py, cve-audit; 360px render checks. Close every gap named in the critic evidence before normative edits.
  2. Envelope unification: single renderer + wants_html everywhere; map legacy shapes; wire error.html; fix 403-vs-401 with SPEC amendment; add content-negotiated error matrix tests.
  3. Shell/nav consolidation: merge 3 shell templates; expand topbar to 16 groups; add breadcrumbs; make /docs filterable; fix doos column misuse; bring CSP page into shell.
  4. Token/component CSS: global .primary, dialog/toast/empty-state/skeleton/confirm styles; adopt REUSE-01..10 vocabulary; resolve assets/app.css stub; page-local styles deleted.
  5. molo-* elements + molo.js: implement Section 8 tag set; migrate vendor pages in batches (agent 9, google 7, ai 20) with per-batch route tests; delete migrated copy-paste files in the same change (delete reclaims; no orphan assets).
  6. OpenAPI auto-sync: Route to RouteDecl adapter; 3.1.0 builder wiring; yaml/redoc mounts; VERSION read; snapshot + determinism + prod-404 + S1-S10 tests; retire toy linter/manual adds.
  7. Whole-site maintenance wiring: registry-driven index/docs/nav; locust gate into test; bundle-list, vocabulary-grep, release-drift checks.
  8. -Werror universal + missing placements: add -Werror to the ~10 .mk files lacking it; move readyz/startupz tests; wire zero_cost_probe.cpp or delete it; remove stray empty tests/src/security subdirs; resolve doos SUP_LIB standalone skip.
  9. Visual-bar pass: hero/stats/tabs/panels per Section 6 on the required pages; 360px gate green; per-component render tests.
  10. Docs/version close-out: CHANGELOG entry without lossy ...and N more summaries; VERSION bump per Conventional-Commit rule (any ! or BREAKING CHANGE -> MAJOR; else feat -> MINOR; else fix -> PATCH; else no release); annotated tag only after green suite plus live verification.

Each step MUST keep make, make test, make lint green or stop and report; never force-push past red.

12. Spec storage path, versioning, docs discipline

  • Logical path: specs/example-site.md is the normative home for this draft (specs/ is a protected floor; changes need the spec-delta-first workflow). Until specs/ landing is approved, this draft lives as the review artifact at /tmp/example-site-spec-draft.md and the working reference at example/SPEC.md stays descriptive of today only.
  • Supporting docs: tests/README.md (suite contract), plans/08-clients-openapi.md (builder rules; rule 8 pins 3.1.0), audit/README.md (30-check CVE audit), docs/ for runbooks. Docs state what the system does today; gaps list the owning file next to the code.
  • Versioning: VERSION plus annotated tag vMAJOR.MINOR.PATCH update atomically with the changelog entry. Demo: POST /workflow/release. Malformed commits (non-Conventional) are rejected before version analysis.
  • Commit scope for this work: workflow(example-site) for spec, feat(example) for site changes, fix(example) for drift closures, test(example) for gate additions.

Appendix A. Section list (for workflow evidence)

1 Scope-goals-non-goals; 2 Quality-standards-coverage-caps-floors; 3 Whole-site-structure; 4 Per-module-look; 5 Reference-component-minimum-map; 6 Visual-bar; 7 Error-message-dialog-form-standards; 8 Custom-element-tag-strategy; 9 OpenAPI-auto-sync; 10 Whole-site-auto-maintenance; 11 Refactor-automation-plan; 12 Spec-storage-versioning-docs.

Appendix B. Open falsification gaps carried forward

  • Live probes never executed (spec bodies, 503 matrix, error matrix, suite runs, 360px renders, make -n reruns).
  • Full /ai|/google|/agent sub-route table beyond grep head/tail; Ops/Platform/Resilience/SecurityExtra/Formats/Realtime/Routing/Security .cpp bodies; 39 of 40 asset JS bodies; tools/release.py body beyond bump header; audit/ and plans/*.md bodies beyond cited excerpts.
  • Full 4-case-per-route audit beyond 3 sampled test files; release.py tag-side gating below line 163.
  • Vendor check_spec.py beyond OpenAI header; matrix_test OpenAPI cross-check; prod app-tree mount check beyond grep.
  • Rendered 360px browser evidence; hero.svg artwork; IndexCatalog.cpp render body; /embedded/app.css reference trace.