moloapp/plans/05-request-routing.md

05 - Request/Response, Routing, Views, Templates

Scope: plan.md sections 2, 6 partial, 10 step 5 partial. Topics: app, routers, views, validation/forms, negotiation, templates, middleware, DI, errors.

1. Purpose and philosophy

This file owns everything between "bytes parsed by the runtime" and "value returned by user code": the Application composition root, FastAPI-style routers, aiohttp-style class-based views, the single validation framework (Pydantic-style models plus HTML/JSON forms), the response hierarchy with content negotiation, the Jinja-like template engine, the middleware onion, dependency injection without globals, and the typed error hierarchy.

Philosophy:

• One request object, one response hierarchy. Handlers never touch sockets; they take a Request and return a Response. Streaming, SSE and realtime build on the same pair (see 06-realtime.md).
• Declarative over imperative. Routes, validation models, negotiation and DI bindings are declared; the framework binds, validates, negotiates and injects before user code runs.
• A single validation framework feeds path, query, header, cookie and body binding, forms, and OpenAPI schema generation (see 08-clients-openapi.md). No second ad-hoc parser anywhere.
• Templates are logic-light text renderers with secure defaults: auto-escape on for HTML/XML, explicit safe opt-out only.
• Errors are typed values, not strings. Every framework exception maps to exactly one HTTP status and renders as JSON for API clients or HTML for browsers, chosen by negotiation.

2. Best practices (from research)

• Routers group related endpoints and mount them on the app with a shared prefix, tags and dependencies. FastAPI documents APIRouter plus app.include_router(router) as the scaling pattern for bigger applications, where the include call applies prefix, tags, dependencies and responses to every route in the router (Source: https://fastapi.tiangolo.com/tutorial/bigger-applications/, retrieved 2026-10-23).
• Class-based views derive from a base View holding the request and implement one coroutine per HTTP method (get, post, ...). aiohttp documents class MyView(web.View) with async def get(self) returning a response, registered on the router like a function handler (Source: https://docs.aiohttp.org/en/stable/web_quickstart.html, retrieved 2026-10-23).
• Validation guarantees the output, not the input: untrusted data is parsed and coerced into a model instance whose fields conform to declared types and constraints, else a single error listing all failures is raised. Pydantic v2 states models this way: fields are annotated attributes on a BaseModel subclass, and a ValidationError is raised when data cannot be parsed into an instance (Source: https://pydantic.dev/docs/validation/latest/concepts/models/, retrieved 2026-10-23).
• Templates center on one reusable Environment holding loaders, filters, tests and globals; syntax is {{ ... }} for expressions, {% ... %} for statements, {# ... #} for comments, with inheritance (extends, blocks), includes, for/if, and autoescaping selectable per template name/extension (Source: https://jinja.palletsprojects.com/en/stable/templates/, retrieved 2026-10-23).
• Content negotiation is server-driven: the client sends Accept with media ranges and optional q weights, the server picks one proposal and answers with Content-Type, or 406 Not Acceptable when nothing matches (Source: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation, retrieved 2026-10-23; triangulated by RFC 9110, Section 12, which obsoletes RFC 7231, Source: https://www.rfc-editor.org/rfc/rfc9110.html, retrieved 2026-10-23).
• Middleware forms an onion: each layer sees the request before the endpoint and the response after it, and may short-circuit. Ordering rule derived from plan.md layering: exception mapping outermost, then observability, security, session/auth, routing-adjacent concerns innermost.

3. Structure

Directories (one class or concern per file, deep over flat):

src/molo/app/          Application, sub-app mounting, lifespan, plugin registry
src/molo/routing/      Router, Route, path compiler, matchers, url_for
src/molo/http_msg/     Request, Response hierarchy, Headers, Query, Cookies view
src/molo/views/        View base, method dispatch, handler adapters
src/molo/validate/     Model, Field, validators, ValidationError, coercions
src/molo/forms/        Form, JSON binding, urlencoded/multipart binding
src/molo/negotiate/    Accept parser, q-sorting, format selection
src/molo/templates/    Environment, Loader, Lexer, Parser, Renderer, Filters, Tests
src/molo/middleware/   Middleware base, Chain, built-in middlewares owned here
src/molo/di/           Container, Scope, Provider, LuckilyNoGlobals
src/molo/errors/       HttpError hierarchy, handler, renderers (JSON, HTML)

Namespaces and key classes:

• molo::app::Application: owns routers, middleware chain, DI container, template env, error handler. include_router(prefix, router), mount(path, sub_app), on_startup/on_shutdown hooks.
• molo::routing::Router: get/post/put/patch/delete/head/options(path, handler); nested include_router; Route record (methods, compiled path, handler, name, validation models, tags).
• molo::http_msg::Request: method, path, query, headers, cookies, get<T>(key) for query/form, json<T>(), form<T>(), body access with caps from 01-runtime.md/03-security.md; Response base plus HtmlResponse, JsonResponse, XmlResponse, TextResponse, RedirectResponse, FileResponse, StreamingResponse (streaming mechanics live in 06-realtime.md, the class shell lives here).
• molo::views::View: constructed with Request; virtual get/post/... default to 405; dispatch() selects by method. Function handlers adapt to the same interface.
• molo::validate::Model: field descriptors with type, constraints, default, required; validate() returns model or throws ValidationError listing every field failure. Used for path/query/body.
• molo::forms::Form: binds application/json, application/x-www-form-urlencoded, multipart/form-data (file parts delegate to upload handling in 03-security.md) into a Model.
• molo::negotiate::Negotiator: parses Accept, sorts by q, matches handler-declared formats.
• molo::templates::Environment: loaders, filters, tests, globals, autoescape predicate, cache.
• molo::middleware::Middleware: handle(Request, Next) -> Response; Chain composes in order.
• molo::di::Container: request-scoped and app-scoped providers, constructor injection for views.
• molo::errors::HttpError(status, code, message, details); ValidationHttpError (422); ErrorHandler maps exception to status plus negotiated JSON/HTML body.

4. Rules (hard, numbered)

• R1: Every route handler is a coroutine taking Request (or a View holding one) and returning a Response. No handler touches sockets, the event loop, or globals.
• R2: All input binding (path, query, header, cookie, JSON body, form body) goes through molo::validate. A second parser for the same input is a defect.
• R3: ValidationError lists ALL field failures in one response, never fail-fast on the first field.
• R4: Responses are chosen from the Response hierarchy only. Raw string returns from handlers are rejected at compile time (no implicit conversion to 200 text).
• R5: Template autoescaping is ON for .html, .htm, .xml and for all streamed template output. safe bypass requires naming the value safe at the call site; no global disable flag.
• R6: Content negotiation runs for every response with more than one declared format. Explicit ?format= or path suffix overrides Accept only when the route declares allow_format_override. No match yields 406 with a JSON or HTML body listing available types.
• R7: Middleware order is fixed by category (errors outer, then observability, security, session/auth, user middleware, router innermost). Reordering across categories is a compile-time or startup error.
• R8: DI provides DB, cache, auth, templates and settings. No service singletons, no file-static mutable state reachable from handlers. Request scope ends at response completion.
• R9: Every framework exception maps to exactly one HTTP status via the error table in section 6. Unmapped exceptions become 500 and are logged with correlation ID; stack details never leak to clients.
• R10: Route names are unique per application (including mounted sub-apps after path merge). Duplicate names fail at startup with the colliding paths quoted.
• R11: All caps in section 6 are static constants with documented defaults. No autotuning, no silent growth.
• R12: No feature in this file duplicates scope owned by a sibling: no TLS/auth/session logic (03-security.md), no ORM/KV/cache (04-data.md), no WebSocket/SSE/Socket.IO mechanics (06-realtime.md), no OpenAPI emission (08-clients-openapi.md), no CLI (09-cli-ops.md). This file exposes the hooks those siblings consume.

5. Development guidelines

• Build in the order of section 7; each step ends green with integration tests through real HTTP.
• Prefer small composable classes: one concern per file, behavior in exactly one base class, features composed through declared interfaces (plan.md section 9).
• Headers follow the runtime: case-insensitive lookup, size and count caps enforced in 01-runtime.md before this layer sees them.
• Keep the happy path allocation-light: route match without heap allocation per request where feasible; template rendering reuses the compiled template cache.
• Every public API gets a usage example in the living example project (see 10-workflow-example.md) in the same change that introduces it.
• Docs state current behavior with gaps listed next to the code that closes them; no aspirational APIs in headers.

6. Specifications (concrete behaviors, caps, formats)

Route matching:

• Path syntax: /users/{id:int} with converters int, float, str (single segment), path (remainder, must be last). Unknown converter is a startup error naming the route.
• Matching order: static segments first, then converters in registration order, then path catch-alls. Method mismatch on an existing path yields 405 with an Allow header, not 404.
• Trailing slash: strict. /users and /users/ are different routes unless the route sets slash_insensitive=true, which registers both and canonical-redirects (308) to the declared form.
• Sub-app mount merges paths: mounting app B at /app1 inside A prefixes all B routes with /app1; triple nesting yields /app1/app1/app1 automatically. Mounted apps keep their own DI scope but share the parent middleware chain plus their own appended layers.

Validation and forms:

• Field constraints: min/max (numbers), min_len/max_len (strings/collections), pattern (regex), choices, gt/ge/lt/le, required, default, nullable. Coercion: strict for path params (no "1abc" to int), lenient-but-documented for query/form strings.
• ValidationError JSON shape: {"detail":[{"loc":["query","age"],"msg":"...","type":"..."}]}, status 422. HTML rendering shows the same list in a 422 page.
• Form caps: max fields 1000, max field name 256 chars, max non-file value 1 MiB, max JSON depth 32, max JSON document 10 MiB default (overridable per route down to 0 = reject bodies). File parts use the upload quotas from 03-security.md.

Negotiation:

• Parse Accept into (range, q, params); sort by q desc, then specificity (exact beat type star beat star star), then handler declaration order. Missing Accept means */*.
• Handler declares produces = {HTML, JSON} etc. First negotiated match wins. No match: 406 with {"detail":{"available":["text/html","application/json"]}} or the HTML equivalent.
• Content-Type of requests must match a route-declared consumes set when declared, else 415.

Templates (Jinja-compatible subset, in-house engine):

• Delimiters: {{ }}, {% %}, {# #}. Constructs: extends, block/endblock, include, if/ elif/else/endif, for/else/endfor with loop variable, set, macro, filters with |, tests with is, autoescape blocks. Undefined variables render per policy: chain default renders empty string and logs at debug; strict mode (dev default) throws UndefinedError (500 in prod rendering, startup warning listing the template).
• Filters shipped: escape, safe, upper, lower, trim, striptags, urlencode, length, default, join, date, tojson. Tests shipped: defined, none, string, number, sequence, mapping, even, odd.
• Caps: max template size 1 MiB, max include depth 16, max extends chain 8, max loop iterations 100000, max render output 32 MiB, compiled-template cache max 512 entries with LRU eviction. Recursion beyond depth aborts with 500 and a metric increment, never a stack overflow.
• Loaders: filesystem directory, embedded assets (from 09-cli-ops.md embedder), in-memory dict. Dev reloads from disk per render when dev_reload=true; prod serves compiled cache only.

Middleware and DI:

• Signature: Task<Response> handle(Request req, Next next). Next invokes the remainder exactly once; calling twice is a runtime error. Middleware may replace request/response but must preserve the correlation ID (owned by 02-observability.md).
• Built-ins owned here: ExceptionMappingMiddleware (outermost), NegotiationMiddleware, CORSPassthrough is NOT here (03-security.md owns CORS policy; this file only preserves its headers), RequestIdEcho helpers defer to 02-observability.md. User middleware registers with an explicit category tag.
• DI scopes: app (one per Application), request (one per request, destroyed after response). Providers are factories or values; circular dependencies fail at startup with the cycle quoted.

Errors:

• Mapping table: 400 BadRequest, 401 Unauthorized, 403 Forbidden, 404 NotFound (includes url_for miss at render time), 405 MethodNotAllowed (sets Allow), 406 NotAcceptable, 408/413/414/431 from runtime caps pass through, 415 UnsupportedMediaType, 422 ValidationFailed, 429 from 03-security.md passes through, 500 InternalError, 503 Unavailable.
• JSON error shape: {"error":{"code":"not_found","message":"...","request_id":"..."}}. HTML error shape: responsive error page with the same fields, no internals.
• Redirects: RedirectResponse(url, permanent) emits 308/307 (method-preserving). Flash messages ride the session (03-security.md) via flash(request, level, message) and render once.

Global caps (static, overridable by config, never autotuned):

• max routes per app 100000; max path length 2048 chars; max path params per route 32; max middleware layers 64; max sub-app nesting depth 8; max DI providers 4096.

7. Execution steps (in build order)

• E1: Request, Headers, Query, cookie view, Response base plus Text/JSON/HTML/XML/Redirect. No routing yet; handlers invoked directly in tests.
• E2: Path compiler plus Router (static, converters, 404/405, Allow header), url_for by name.
• E3: Application, include_router with prefix/tags/dependencies, sub-app mount with path merge, startup validation (duplicate names, unknown converters, cycles).
• E4: molo::validate (Model, Field, coercions, full-error ValidationError) bound to path, query, headers, cookies.
• E5: molo::errors (hierarchy, mapping table, JSON plus HTML renderers, global handler).
• E6: molo::negotiate plus NegotiationMiddleware, consumes/produces declarations, 406/415 paths.
• E7: molo::views::View with method dispatch, function-handler adapter, redirects, flash helpers.
• E8: Middleware Chain with category ordering enforcement and Next-once guard.
• E9: molo::di (app and request scopes, cycle detection) wired into views and handlers.
• E10: molo::forms (JSON, urlencoded, multipart field parts) on top of validation.
• E11: Template engine: lexer, parser, renderer, Environment, loaders, filters/tests, autoescape, caps and compiled cache. TemplateResponse(template, context) as a Response subclass.
• E12: Plugin/extension registry on Application (named hooks: route_added, app_started, error_render) with capability declarations so plugins cannot reach undeclared internals.

8. Validation (how to verify each step)

Integration tests only, through the real stack (real HTTP, routing, middleware), per plan.md section 8. Test files mirror routes: tests/routes/<router>/<route>_test.cpp.

• V1: GET/POST round-trips return correct status, headers and bodies for each response subclass.
• V2: static, converter and catch-all matches; 404 body shape; 405 with exact Allow value; url_for reverses every named route including converted params.
• V3: prefixed includes; triple-nested mount produces /app1/app1/app1; duplicate names and unknown converters fail startup with quoted diagnostics.
• V4: valid input binds; invalid input yields one 422 listing every bad field (path, query, header, cookie cases each covered).
• V5: each mapped exception renders the exact JSON shape and the HTML page; unknown exception yields 500 JSON/HTML without internals; request_id present in both.
• V6: q-weighted Accept fixtures pick the expected format; no-match yields 406 listing available types; wrong Content-Type yields 415.
• V7: View dispatches each method; unimplemented method yields 405; redirects preserve method (307/308); flash message appears exactly once across two requests.
• V8: middleware order test asserts onion sequence; double-Next call fails loudly; short-circuit returns without reaching the handler.
• V9: request scope isolation across two concurrent requests; app scope shared; cycle fixture fails startup quoting the cycle.
• V10: JSON, urlencoded and multipart forms bind to the same model; cap fixtures (1001 fields, depth 33, oversized value) are rejected with the documented status.
• V11: template fixtures cover extends/blocks, include, if/for/else, filters, tests, macros; XSS fixture asserts escaping; safe fixture asserts opt-out; cap fixtures (depth 17, 100001 iterations) abort safely; dev reload picks up an edited file, prod serves cache.
• V12: sample plugin receives hooks in order and is denied undeclared access.

9. References (sibling files this depends on or feeds)

• Depends on 01-runtime.md: timeouts, body/header caps, cookies, compression, static serving. This file never re-enforces byte-level caps.
• Depends on 02-observability.md: correlation IDs, log format, per-route metrics, tracing spans.
• Depends on 03-security.md: CSRF, CORS policy, rate limits, session/flash storage, upload quotas, auth guards consumed by views and DI.
• Depends on 04-data.md: ORM/KV/cache handles provided through DI; explicit invalidation hooks.
• Feeds 06-realtime.md: Request/Response shells, router mounting and middleware/DI participation for WebSocket/SSE/Socket.IO views and streaming responses.
• Feeds 07-supervisor-replication.md: request scope teardown points safe for background handoff.
• Feeds 08-clients-openapi.md: routers plus validation models as the OpenAPI source; response formats as client expectations; test client drives Application without network.
• Feeds 09-cli-ops.md: dev server hosts Application; embedder output plugs into template loaders; health endpoints are plain routes defined here.
• Follows 10-workflow-example.md: specs, test layout, living example coverage for every API above.

10. Open questions

• Q1: Exact lenient coercion table for query/form strings (e.g. "yes" to bool, empty string to null). Proposal: strict booleans (true/false/1/0 only), empty string never coerces to null.
• Q2: Whether ?format= override defaults on or off. Proposal: off; routes opt in.
• Q3: Template strict undefined policy in production: render empty plus warn, or 500. Proposal: empty plus warn and metric, 500 only in dev/test.
• Q4: Plugin capability model granularity: coarse hook names now, fine-grained review before v1.

11. Sources (exact URLs with retrieval dates)

• FastAPI tutorial, Bigger Applications (APIRouter, include_router): https://fastapi.tiangolo.com/tutorial/bigger-applications/ (retrieved 2026-10-23)
• aiohttp Web Server Quickstart (Application, handlers, class-based views): https://docs.aiohttp.org/en/stable/web_quickstart.html (retrieved 2026-10-23)
• Pydantic v2 concepts, Models (validation guarantees, ValidationError): https://pydantic.dev/docs/validation/latest/concepts/models/ (retrieved 2026-10-23)
• Jinja Template Designer Documentation (syntax, inheritance, autoescape): https://jinja.palletsprojects.com/en/stable/templates/ (retrieved 2026-10-23)
• MDN, Content negotiation (Accept, q weights, 406): https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Content_negotiation (retrieved 2026-10-23)
• RFC 9110, HTTP Semantics, Section 12 (content negotiation, obsoletes RFC 7231): https://www.rfc-editor.org/rfc/rfc9110.html (retrieved 2026-10-23)