moloapp/plans/05-request-routing.md

# 05 - Request/Response, Routing, Views, Templates

Scope: plan.md sections 2, 6 partial, 10 step 5 partial.
Topics: app, routers, views, validation/forms, negotiation, templates, middleware, DI, errors.

## 1. Purpose and philosophy

This file owns everything between "bytes parsed by the runtime" and "value returned by user code": the
`Application` composition root, FastAPI-style routers, aiohttp-style class-based views, the single
validation framework (Pydantic-style models plus HTML/JSON forms), the response hierarchy with content
negotiation, the Jinja-like template engine, the middleware onion, dependency injection without globals,
and the typed error hierarchy.

Philosophy:

- One request object, one response hierarchy. Handlers never touch sockets; they take a `Request` and
  return a `Response`. Streaming, SSE and realtime build on the same pair (see 06-realtime.md).
- Declarative over imperative. Routes, validation models, negotiation and DI bindings are declared;
  the framework binds, validates, negotiates and injects before user code runs.
- A single validation framework feeds path, query, header, cookie and body binding, forms, and OpenAPI
  schema generation (see 08-clients-openapi.md). No second ad-hoc parser anywhere.
- Templates are logic-light text renderers with secure defaults: auto-escape on for HTML/XML, explicit
  `safe` opt-out only.
- Errors are typed values, not strings. Every framework exception maps to exactly one HTTP status and
  renders as JSON for API clients or HTML for browsers, chosen by negotiation.

## 2. Best practices (from research)

- Routers group related endpoints and mount them on the app with a shared prefix, tags and dependencies.
  FastAPI documents `APIRouter` plus `app.include_router(router)` as the scaling pattern for bigger
  applications, where the include call applies prefix, tags, dependencies and responses to every route
  in the router (Source: https://fastapi.tiangolo.com/tutorial/bigger-applications/, retrieved 2026-10-23).
- Class-based views derive from a base `View` holding the request and implement one coroutine per HTTP
  method (`get`, `post`, ...). aiohttp documents `class MyView(web.View)` with `async def get(self)`
  returning a response, registered on the router like a function handler
  (Source: https://docs.aiohttp.org/en/stable/web_quickstart.html, retrieved 2026-10-23).
- Validation guarantees the output, not the input: untrusted data is parsed and coerced into a model
  instance whose fields conform to declared types and constraints, else a single error listing all
  failures is raised. Pydantic v2 states models this way: fields are annotated attributes on a
  `BaseModel` subclass, and a `ValidationError` is raised when data cannot be parsed into an instance
  (Source: https://pydantic.dev/docs/validation/latest/concepts/models/, retrieved 2026-10-23).
- Templates center on one reusable `Environment` holding loaders, filters, tests and globals; syntax is
  `{{ ... }}` for expressions, `{% ... %}` for statements, `{# ... #}` for comments, with inheritance
  (`extends`, `blocks`), includes, `for`/`if`, and autoescaping selectable per template name/extension
  (Source: https://jinja.palletsprojects.com/en/stable/templates/, retrieved 2026-10-23).
- Content negotiation is server-driven: the client sends `Accept` with media ranges and optional `q`
  weights, the server picks one proposal and answers with `Content-Type`, or `406 Not Acceptable`
  when nothing matches (Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation,
  retrieved 2026-10-23; triangulated by RFC 9110, Section 12, which obsoletes RFC 7231,
  Source: https://www.rfc-editor.org/rfc/rfc9110.html, retrieved 2026-10-23).
- Middleware forms an onion: each layer sees the request before the endpoint and the response after it,
  and may short-circuit. Ordering rule derived from plan.md layering: exception mapping outermost,
  then observability, security, session/auth, routing-adjacent concerns innermost.

## 3. Structure

Directories (one class or concern per file, deep over flat):

```
src/molo/app/          Application, sub-app mounting, lifespan, plugin registry
src/molo/routing/      Router, Route, path compiler, matchers, url_for
src/molo/http_msg/     Request, Response hierarchy, Headers, Query, Cookies view
src/molo/views/        View base, method dispatch, handler adapters
src/molo/validate/     Model, Field, validators, ValidationError, coercions
src/molo/forms/        Form, JSON binding, urlencoded/multipart binding
src/molo/negotiate/    Accept parser, q-sorting, format selection
src/molo/templates/    Environment, Loader, Lexer, Parser, Renderer, Filters, Tests
src/molo/middleware/   Middleware base, Chain, built-in middlewares owned here
src/molo/di/           Container, Scope, Provider, LuckilyNoGlobals
src/molo/errors/       HttpError hierarchy, handler, renderers (JSON, HTML)
```

Namespaces and key classes:

- `molo::app::Application`: owns routers, middleware chain, DI container, template env, error handler.
  `include_router(prefix, router)`, `mount(path, sub_app)`, `on_startup/on_shutdown` hooks.
- `molo::routing::Router`: `get/post/put/patch/delete/head/options(path, handler)`; nested
  `include_router`; `Route` record (methods, compiled path, handler, name, validation models, tags).
- `molo::http_msg::Request`: method, path, query, headers, cookies, `get<T>(key)` for query/form,
  `json<T>()`, `form<T>()`, body access with caps from 01-runtime.md/03-security.md; `Response` base plus `HtmlResponse`,
  `JsonResponse`, `XmlResponse`, `TextResponse`, `RedirectResponse`, `FileResponse`,
  `StreamingResponse` (streaming mechanics live in 06-realtime.md, the class shell lives here).
- `molo::views::View`: constructed with `Request`; virtual `get/post/...` default to 405; `dispatch()`
  selects by method. Function handlers adapt to the same interface.
- `molo::validate::Model`: field descriptors with type, constraints, default, required; `validate()`
  returns model or throws `ValidationError` listing every field failure. Used for path/query/body.
- `molo::forms::Form`: binds `application/json`, `application/x-www-form-urlencoded`,
  `multipart/form-data` (file parts delegate to upload handling in 03-security.md) into a `Model`.
- `molo::negotiate::Negotiator`: parses `Accept`, sorts by q, matches handler-declared formats.
- `molo::templates::Environment`: loaders, filters, tests, globals, autoescape predicate, cache.
- `molo::middleware::Middleware`: `handle(Request, Next) -> Response`; `Chain` composes in order.
- `molo::di::Container`: request-scoped and app-scoped providers, constructor injection for views.
- `molo::errors::HttpError(status, code, message, details)`; `ValidationHttpError` (422);
  `ErrorHandler` maps exception to status plus negotiated JSON/HTML body.

## 4. Rules (hard, numbered)

- R1: Every route handler is a coroutine taking `Request` (or a `View` holding one) and returning a
  `Response`. No handler touches sockets, the event loop, or globals.
- R2: All input binding (path, query, header, cookie, JSON body, form body) goes through
  `molo::validate`. A second parser for the same input is a defect.
- R3: `ValidationError` lists ALL field failures in one response, never fail-fast on the first field.
- R4: Responses are chosen from the `Response` hierarchy only. Raw string returns from handlers are
  rejected at compile time (no implicit conversion to 200 text).
- R5: Template autoescaping is ON for `.html`, `.htm`, `.xml` and for all streamed template output.
  `safe` bypass requires naming the value safe at the call site; no global disable flag.
- R6: Content negotiation runs for every response with more than one declared format. Explicit
  `?format=` or path suffix overrides `Accept` only when the route declares `allow_format_override`.
  No match yields `406` with a JSON or HTML body listing available types.
- R7: Middleware order is fixed by category (errors outer, then observability, security, session/auth,
  user middleware, router innermost). Reordering across categories is a compile-time or startup error.
- R8: DI provides DB, cache, auth, templates and settings. No service singletons, no file-static
  mutable state reachable from handlers. Request scope ends at response completion.
- R9: Every framework exception maps to exactly one HTTP status via the error table in section 6.
  Unmapped exceptions become 500 and are logged with correlation ID; stack details never leak to clients.
- R10: Route names are unique per application (including mounted sub-apps after path merge). Duplicate
  names fail at startup with the colliding paths quoted.
- R11: All caps in section 6 are static constants with documented defaults. No autotuning, no silent
  growth.
- R12: No feature in this file duplicates scope owned by a sibling: no TLS/auth/session logic (03-security.md),
  no ORM/KV/cache (04-data.md), no WebSocket/SSE/Socket.IO mechanics (06-realtime.md), no OpenAPI emission (08-clients-openapi.md), no CLI
  (09-cli-ops.md). This file exposes the hooks those siblings consume.

## 5. Development guidelines

- Build in the order of section 7; each step ends green with integration tests through real HTTP.
- Prefer small composable classes: one concern per file, behavior in exactly one base class, features
  composed through declared interfaces (plan.md section 9).
- Headers follow the runtime: case-insensitive lookup, size and count caps enforced in 01-runtime.md before this
  layer sees them.
- Keep the happy path allocation-light: route match without heap allocation per request where feasible;
  template rendering reuses the compiled template cache.
- Every public API gets a usage example in the living example project (see 10-workflow-example.md)
  in the same change that introduces it.
- Docs state current behavior with gaps listed next to the code that closes them; no aspirational APIs
  in headers.

## 6. Specifications (concrete behaviors, caps, formats)

Route matching:

- Path syntax: `/users/{id:int}` with converters `int`, `float`, `str` (single segment), `path`
  (remainder, must be last). Unknown converter is a startup error naming the route.
- Matching order: static segments first, then converters in registration order, then `path` catch-alls.
  Method mismatch on an existing path yields 405 with an `Allow` header, not 404.
- Trailing slash: strict. `/users` and `/users/` are different routes unless the route sets
  `slash_insensitive=true`, which registers both and canonical-redirects (308) to the declared form.
- Sub-app mount merges paths: mounting app B at `/app1` inside A prefixes all B routes with `/app1`;
  triple nesting yields `/app1/app1/app1` automatically. Mounted apps keep their own DI scope but share
  the parent middleware chain plus their own appended layers.

Validation and forms:

- Field constraints: `min/max` (numbers), `min_len/max_len` (strings/collections), `pattern` (regex),
  `choices`, `gt/ge/lt/le`, `required`, `default`, `nullable`. Coercion: strict for path params
  (no `"1abc"` to int), lenient-but-documented for query/form strings.
- `ValidationError` JSON shape: `{"detail":[{"loc":["query","age"],"msg":"...","type":"..."}]}`,
  status 422. HTML rendering shows the same list in a 422 page.
- Form caps: max fields 1000, max field name 256 chars, max non-file value 1 MiB, max JSON depth 32,
  max JSON document 10 MiB default (overridable per route down to 0 = reject bodies). File parts use
  the upload quotas from 03-security.md.

Negotiation:

- Parse `Accept` into (range, q, params); sort by q desc, then specificity (exact beat type star beat
  star star), then handler declaration order. Missing `Accept` means `*/*`.
- Handler declares `produces = {HTML, JSON}` etc. First negotiated match wins. No match: 406 with
  `{"detail":{"available":["text/html","application/json"]}}` or the HTML equivalent.
- `Content-Type` of requests must match a route-declared `consumes` set when declared, else 415.

Templates (Jinja-compatible subset, in-house engine):

- Delimiters: `{{ }}`, `{% %}`, `{# #}`. Constructs: `extends`, `block`/`endblock`, `include`, `if`/
  `elif`/`else`/`endif`, `for`/`else`/`endfor` with `loop` variable, `set`, `macro`, filters with `|`,
  tests with `is`, `autoescape` blocks. Undefined variables render per policy: `chain` default renders
  empty string and logs at debug; `strict` mode (dev default) throws `UndefinedError` (500 in prod
  rendering, startup warning listing the template).
- Filters shipped: `escape`, `safe`, `upper`, `lower`, `trim`, `striptags`, `urlencode`, `length`,
  `default`, `join`, `date`, `tojson`. Tests shipped: `defined`, `none`, `string`, `number`,
  `sequence`, `mapping`, `even`, `odd`.
- Caps: max template size 1 MiB, max include depth 16, max extends chain 8, max loop iterations 100000,
  max render output 32 MiB, compiled-template cache max 512 entries with LRU eviction. Recursion beyond
  depth aborts with 500 and a metric increment, never a stack overflow.
- Loaders: filesystem directory, embedded assets (from 09-cli-ops.md embedder), in-memory dict. Dev reloads from
  disk per render when `dev_reload=true`; prod serves compiled cache only.

Middleware and DI:

- Signature: `Task<Response> handle(Request req, Next next)`. `Next` invokes the remainder exactly once;
  calling twice is a runtime error. Middleware may replace request/response but must preserve the
  correlation ID (owned by 02-observability.md).
- Built-ins owned here: `ExceptionMappingMiddleware` (outermost), `NegotiationMiddleware`,
  `CORSPassthrough` is NOT here (03-security.md owns CORS policy; this file only preserves its headers),
  `RequestIdEcho` helpers defer to 02-observability.md. User middleware registers with an explicit category tag.
- DI scopes: `app` (one per Application), `request` (one per request, destroyed after response).
  Providers are factories or values; circular dependencies fail at startup with the cycle quoted.

Errors:

- Mapping table: 400 `BadRequest`, 401 `Unauthorized`, 403 `Forbidden`, 404 `NotFound` (includes
  `url_for` miss at render time), 405 `MethodNotAllowed` (sets `Allow`), 406 `NotAcceptable`,
  408/413/414/431 from runtime caps pass through, 415 `UnsupportedMediaType`, 422 `ValidationFailed`,
  429 from 03-security.md passes through, 500 `InternalError`, 503 `Unavailable`.
- JSON error shape: `{"error":{"code":"not_found","message":"...","request_id":"..."}}`.
  HTML error shape: responsive error page with the same fields, no internals.
- Redirects: `RedirectResponse(url, permanent)` emits 308/307 (method-preserving). Flash messages ride
  the session (03-security.md) via `flash(request, level, message)` and render once.

Global caps (static, overridable by config, never autotuned):

- max routes per app 100000; max path length 2048 chars; max path params per route 32;
  max middleware layers 64; max sub-app nesting depth 8; max DI providers 4096.

## 7. Execution steps (in build order)

- E1: `Request`, `Headers`, `Query`, cookie view, `Response` base plus Text/JSON/HTML/XML/Redirect.
  No routing yet; handlers invoked directly in tests.
- E2: Path compiler plus `Router` (static, converters, 404/405, Allow header), `url_for` by name.
- E3: `Application`, `include_router` with prefix/tags/dependencies, sub-app `mount` with path merge,
  startup validation (duplicate names, unknown converters, cycles).
- E4: `molo::validate` (Model, Field, coercions, full-error `ValidationError`) bound to path, query,
  headers, cookies.
- E5: `molo::errors` (hierarchy, mapping table, JSON plus HTML renderers, global handler).
- E6: `molo::negotiate` plus `NegotiationMiddleware`, `consumes/produces` declarations, 406/415 paths.
- E7: `molo::views::View` with method dispatch, function-handler adapter, redirects, flash helpers.
- E8: Middleware `Chain` with category ordering enforcement and `Next`-once guard.
- E9: `molo::di` (app and request scopes, cycle detection) wired into views and handlers.
- E10: `molo::forms` (JSON, urlencoded, multipart field parts) on top of validation.
- E11: Template engine: lexer, parser, renderer, Environment, loaders, filters/tests, autoescape,
  caps and compiled cache. `TemplateResponse(template, context)` as a `Response` subclass.
- E12: Plugin/extension registry on `Application` (named hooks: route_added, app_started, error_render)
  with capability declarations so plugins cannot reach undeclared internals.

## 8. Validation (how to verify each step)

Integration tests only, through the real stack (real HTTP, routing, middleware), per plan.md section 8.
Test files mirror routes: `tests/routes/<router>/<route>_test.cpp`.

- V1: GET/POST round-trips return correct status, headers and bodies for each response subclass.
- V2: static, converter and catch-all matches; 404 body shape; 405 with exact `Allow` value;
  `url_for` reverses every named route including converted params.
- V3: prefixed includes; triple-nested mount produces `/app1/app1/app1`; duplicate names and unknown
  converters fail startup with quoted diagnostics.
- V4: valid input binds; invalid input yields one 422 listing every bad field (path, query, header,
  cookie cases each covered).
- V5: each mapped exception renders the exact JSON shape and the HTML page; unknown exception yields
  500 JSON/HTML without internals; `request_id` present in both.
- V6: q-weighted `Accept` fixtures pick the expected format; no-match yields 406 listing available
  types; wrong `Content-Type` yields 415.
- V7: View dispatches each method; unimplemented method yields 405; redirects preserve method (307/308);
  flash message appears exactly once across two requests.
- V8: middleware order test asserts onion sequence; double-`Next` call fails loudly; short-circuit
  returns without reaching the handler.
- V9: request scope isolation across two concurrent requests; app scope shared; cycle fixture fails
  startup quoting the cycle.
- V10: JSON, urlencoded and multipart forms bind to the same model; cap fixtures (1001 fields, depth
  33, oversized value) are rejected with the documented status.
- V11: template fixtures cover extends/blocks, include, if/for/else, filters, tests, macros; XSS
  fixture asserts escaping; `safe` fixture asserts opt-out; cap fixtures (depth 17, 100001 iterations)
  abort safely; dev reload picks up an edited file, prod serves cache.
- V12: sample plugin receives hooks in order and is denied undeclared access.

## 9. References (sibling files this depends on or feeds)

- Depends on 01-runtime.md: timeouts, body/header caps, cookies, compression, static serving. This
  file never re-enforces byte-level caps.
- Depends on 02-observability.md: correlation IDs, log format, per-route metrics, tracing spans.
- Depends on 03-security.md: CSRF, CORS policy, rate limits, session/flash storage, upload quotas,
  auth guards consumed by views and DI.
- Depends on 04-data.md: ORM/KV/cache handles provided through DI; explicit invalidation hooks.
- Feeds 06-realtime.md: `Request`/`Response` shells, router mounting and middleware/DI participation
  for WebSocket/SSE/Socket.IO views and streaming responses.
- Feeds 07-supervisor-replication.md: request scope teardown points safe for background handoff.
- Feeds 08-clients-openapi.md: routers plus validation models as the OpenAPI source; response formats
  as client expectations; test client drives `Application` without network.
- Feeds 09-cli-ops.md: dev server hosts `Application`; embedder output plugs into template loaders;
  health endpoints are plain routes defined here.
- Follows 10-workflow-example.md: specs, test layout, living example coverage for every API above.

## 10. Open questions

- Q1: Exact lenient coercion table for query/form strings (e.g. `"yes"` to bool, empty string to
  null). Proposal: strict booleans (`true/false/1/0` only), empty string never coerces to null.
- Q2: Whether `?format=` override defaults on or off. Proposal: off; routes opt in.
- Q3: Template `strict` undefined policy in production: render empty plus warn, or 500. Proposal:
  empty plus warn and metric, 500 only in dev/test.
- Q4: Plugin capability model granularity: coarse hook names now, fine-grained review before v1.

## 11. Sources (exact URLs with retrieval dates)

- FastAPI tutorial, Bigger Applications (APIRouter, include_router):
  https://fastapi.tiangolo.com/tutorial/bigger-applications/ (retrieved 2026-10-23)
- aiohttp Web Server Quickstart (Application, handlers, class-based views):
  https://docs.aiohttp.org/en/stable/web_quickstart.html (retrieved 2026-10-23)
- Pydantic v2 concepts, Models (validation guarantees, ValidationError):
  https://pydantic.dev/docs/validation/latest/concepts/models/ (retrieved 2026-10-23)
- Jinja Template Designer Documentation (syntax, inheritance, autoescape):
  https://jinja.palletsprojects.com/en/stable/templates/ (retrieved 2026-10-23)
- MDN, Content negotiation (Accept, q weights, 406):
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation (retrieved 2026-10-23)
- RFC 9110, HTTP Semantics, Section 12 (content negotiation, obsoletes RFC 7231):
  https://www.rfc-editor.org/rfc/rfc9110.html (retrieved 2026-10-23)