moloapp/plans/01-runtime.md

Core Runtime and HTTP Fundamentals

File: plans/01-runtime.md Scope: plan.md sections 2-3, 10 step 1 partial. Topics: event loop, lifecycle, HTTP/1.1 + HTTP/2, compression, cookies, static.

1. Purpose and philosophy

Provide the single non-blocking foundation every other component builds on: one in-house reactor, one coroutine model, one connection lifecycle, one HTTP/1.1 plus HTTP/2 stack, one compression layer, one cookie transport, one static file path. No feature reimplements I/O, parsing, framing, or file delivery.

Philosophy:

• Boring core, strict caps. Every knob has a static ceiling. No autotuning.
• Correctness from standards, not folklore. HTTP behavior follows RFC 9110/9112/9113, cookies follow RFC 6265, compression formats follow RFC 1952/7932.
• Transport only here. Sessions, auth, guards live in 03-security.md. Routing, views, middleware, errors live in 05-request-routing.md. Metrics and tracing hooks live in 02-observability.md but cost near zero when disabled.
• Deep structure: one class or concern per file, namespaces mirror directories.

2. Best practices (from research)

1. HTTP/1.1: persistent connections are the default; framing comes from Content-Length or chunked transfer coding; a recipient MUST parse chunked; unknown transfer codings SHOULD get 501; keep-alive plus correct length framing enables connection reuse (Source: https://www.rfc-editor.org/rfc/rfc9112, retrieved 2026-10-23).
2. HTTP/1.1 request smuggling defense: never accept both Content-Length and Transfer-Encoding on one message; treat conflicts as 400 and close. This triangulates with RFC 9112 framing rules and current CVE practice around chunked plus keep-alive handling.
3. HTTP/2: binary framing, interleaved multiplexed streams on one connection, HPACK field compression, connection-level plus stream-level flow control, request prioritization (Source: https://www.rfc-editor.org/rfc/rfc9113, retrieved 2026-10-23). Independent H2 implementations confirm the same trio: multiplexing, HPACK, dual flow control.
4. HTTP/2 server push is gone in RFC 9113 (obsoletes RFC 7540/8740). Do not implement push. Use 103 Early Hints, normal preload, or SSE from 06-realtime.md instead (Source: https://www.rfc-editor.org/rfc/rfc9113, retrieved 2026-10-23).
5. HTTP semantics are version independent and defined once in RFC 9110: methods, status codes, fields, content negotiation, validators, ranges, caching (Source: https://www.rfc-editor.org/rfc/rfc9110, retrieved 2026-10-23). MoloApp therefore shares one Request/Response semantic layer across H1 and H2.
6. Static validators: send both ETag and Last-Modified on 200 when cheap; If-None-Match takes precedence over If-Modified-Since; support If-Range, single byte ranges with 206, and 416 for unsatisfiable ranges (Source: https://www.rfc-editor.org/rfc/rfc9110, retrieved 2026-10-23; corroborated by https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Caching, retrieved 2026-10-23).
7. Cookies: servers store state via Set-Cookie and Cookie fields; RFC 6265 defines syntax, domain/path scoping, and expiry semantics (Source: https://www.rfc-editor.org/rfc/rfc6265, retrieved 2026-10-23). H2 MAY split one logical Cookie into several fields to help HPACK; a server MUST join them back with "; " before parsing (Source: https://www.rfc-editor.org/rfc/rfc9113, retrieved 2026-10-23). Set-Cookie MUST never be joined.
8. Compression: Brotli combines LZ77 plus Huffman coding with quality comparable to best general-purpose methods (Source: https://www.rfc-editor.org/rfc/rfc7932, retrieved 2026-10-23). gzip wraps DEFLATE with header, CRC32, and original size (Source: https://www.rfc-editor.org/rfc/rfc1952, retrieved 2026-10-23). Both are negotiated via Accept-Encoding/Content-Encoding; Vary: Accept-Encoding is required on negotiated responses.
9. Reactor: epoll scales to many FDs via epoll_create/ctl/wait and supports level-triggered (default) plus edge-triggered (EPOLLET) modes (Source: https://man7.org/linux/man-pages/man7/epoll.7.html, retrieved 2026-10-23). kqueue (BSD/macOS) and IOCP with GetQueuedCompletionStatus/PostQueuedCompletionStatus (Windows, see https://learn.microsoft.com/en-us/windows/win32/fileio/synchronous-and-asynchronous-i-o, retrieved 2026-10-23) are the native peers. MoloApp hides all three behind one Poller interface; default is level-triggered for safety.
10. Coroutines: a C++20/23 coroutine is promise plus handle plus awaitable; initial_suspend decides lazy vs eager start; co_await suspends without blocking the thread (Source: https://en.cppreference.com/cpp/language/coroutines, retrieved 2026-10-23). All handlers are coroutines returning Task; blocking work never runs on loop threads.

3. Structure

Directories mirror namespaces. One concern per file. Tests mirror routes per 10-workflow-example.md.

src/molo/runtime/
  event_loop.h/.cpp        molo::runtime::EventLoop
  poller.h                 molo::runtime::IPoller (interface)
  poller_epoll.h/.cpp      molo::runtime::EpollPoller (Linux)
  poller_kqueue.h/.cpp     molo::runtime::KqueuePoller (BSD/macOS)
  poller_iocp.h/.cpp       molo::runtime::IocpPoller (Windows)
  task.h                   molo::runtime::Task<T>, molo::runtime::Generator<T>
  executor.h/.cpp          molo::runtime::ThreadPool (bounded blocking pool)
  timer_wheel.h/.cpp       molo::runtime::TimerWheel
  signal_handler.h/.cpp    molo::runtime::SignalHandler
  strand.h                 molo::runtime::Strand (per-connection serial queue)
  channel.h                molo::runtime::Channel<T> (inter-loop messages)
src/molo/http/
  message.h/.cpp           molo::http::RequestHead, ResponseHead, Headers
  connection.h/.cpp        molo::http::Connection (H1/H2 dispatch, lifecycle)
  listener.h/.cpp          molo::http::Listener (accept, SO_REUSEPORT)
  protocol_detect.h/.cpp   molo::http::ProtocolDetector (ALPN, prior knowledge)
  v1/parser.h/.cpp         molo::http::v1::Parser (strict RFC 9112)
  v1/serializer.h/.cpp     molo::http::v1::Serializer
  v2/connection.h/.cpp     molo::http::v2::H2Connection (framing, settings)
  v2/stream.h/.cpp         molo::http::v2::H2Stream (state machine)
  v2/hpack.h/.cpp          molo::http::v2::HpackEncoder, HpackDecoder
  v2/flow_control.h/.cpp   molo::http::v2::FlowController
  v2/settings.h            molo::http::v2::Settings (caps negotiation)
src/molo/compress/
  compressor.h             molo::compress::ICompressor (interface)
  gzip.h/.cpp              molo::compress::GzipCompressor (RFC 1952)
  brotli.h/.cpp            molo::compress::BrotliCompressor (RFC 7932)
  negotiate.h/.cpp         molo::compress::Negotiator (Accept-Encoding)
src/molo/cookie/
  cookie.h/.cpp            molo::cookie::Cookie (parse, serialize)
  jar.h/.cpp               molo::cookie::CookieJar (request join, response list)
  signer.h                 molo::cookie::ISigner (HMAC interface, impl in 03-security.md)
src/molo/static/
  file_server.h/.cpp       molo::static_::FileServer (conditional, ranges)
  asset_provider.h         molo::static_::IAssetProvider (disk + embedded)
  etag.h/.cpp              molo::static_::Etag (strong/weak generation)

Namespace rule: molo::runtime, molo::http, molo::http::v1, molo::http::v2, molo::compress, molo::cookie, molo::static_. No cross-namespace includes except through declared interfaces.

4. Rules (hard, numbered)

• R1. No external loop, HTTP, compression, or cookie library. Only stdlib, OS syscalls, and wolfSSL (TLS, owned by 03-security.md).
• R2. Non-blocking everywhere on loop threads. Blocking syscalls, DNS, file hashing, or compression above threshold go to ThreadPool or use async FDs.
• R3. One EventLoop per thread; connections are pinned to their loop. Cross-loop contact only via Channel. No shared mutable connection state.
• R4. Every limit in section 6 is a compile-time default plus runtime config with a static ceiling. No unbounded queue, header, body, stream, or decompressed size.
• R5. H1 parser is strict: reject ambiguous framing with 400 and close; require Host; accept chunked; answer unknown transfer coding with 501.
• R6. H2 MUST implement settings, flow control, headers/data/priority (ignored)/rst/ping/goaway/window-update; MUST NOT implement server push.
• R7. Compression never changes semantics: always send Vary: Accept-Encoding when negotiating; never compress 204/304, HEAD, or already-encoded bodies.
• R8. Static serving defaults to safe: directory indexing off, dotfiles denied, symlinks not followed unless explicitly allowed, traversal outside root impossible.
• R9. Cookies: parse per RFC 6265, join split H2 Cookie fields, never join Set-Cookie, default SameSite=Lax plus HttpOnly; Secure forced on https.
• R10. Graceful shutdown is mandatory: stop accept, drain with deadline, GOAWAY on H2, Connection: close on H1, then hard close and report.
• R11. No H2 or H1 knowledge inside handlers. Handlers see one semantic Request/Response; version differences stay inside src/molo/http.
• R12. HTTP/3 is a designed path, not v1 scope: keep a Transport interface QUIC can implement later; ship H1 + H2 only.

5. Development guidelines

• Write the interface header first, then the strictest parser or state machine, then the serializer, then integration. Each file gets a spec comment naming its RFC section.
• Prefer state machines with explicit states and transition tables for H1 parsing, H2 streams, and HPACK. Fuzz each with truncated, oversized, and interleaved inputs.
• Coroutine style: Task<Response> handle(Request); co_await I/O, timers, pool work; honor CancellationToken; never hold a lock across co_await.
• Threading: N loops equal min(CPU count, configured workers), each with own Poller; Listener uses SO_REUSEPORT where available, else single acceptor handing FDs round-robin.
• Backpressure: stop reading when the write buffer passes high watermark; resume at low watermark; H2 additionally obeys stream and connection windows.
• Errors are structured: parser errors map to 400/413/431/501 with connection action (keep or close); internal errors emit 500 via 05-request-routing.md error hierarchy.
• Observability hooks (02-observability.md) wrap accept, parse, compress, static hit/miss; disabled builds compile them out.
• Keep gzip and Brotli streaming: bounded input chunks, bounded output chunks, flush points; never buffer a full passthrough body to compress it.

6. Specifications

6.1 Event loop and threading

• Pollers: epoll (Linux), kqueue (BSD/macOS), IOCP (Windows). Default level-triggered; edge-triggered only inside Poller with drain-to-EAGAIN.
• Task: lazy start (initial_suspend_always), executor affinity to owning loop, cancellation via token checked at each await.
• ThreadPool for blocking work: default 4 threads, queue cap 1024 jobs, overflow returns 503 with Retry-After, never grows silently.
• TimerWheel: 10 ms granularity, cap 65536 timers per loop.

6.2 Connection and request lifecycle

• Stages: accept, TLS handshake (03-security.md), protocol detect, head parse, body read, dispatch, handler, serialize, keep-alive or close.
• Timeouts (defaults, all configurable down to floor, up to ceiling): header 10 s, body 30 s, idle keep-alive 75 s, handler soft budget 30 s then 503, shutdown drain 30 s.
• Caps: max connections 16384 per process (ceiling 65536); max requests per H1 connection 1000; H2 max concurrent streams 100 per connection; header block max 64 KB and 100 fields; URI max 8 KB; default body max 10 MB (routes may lower; raising above 100 MB needs explicit config and streams to disk).
• Keep-alive: H1 persistent by default; Connection: close or protocol error closes. H2 uses GOAWAY with last-stream-id on drain.
• Signals: SIGINT/SIGTERM start drain; SIGHUP reloads config where safe; SIGTERM during drain shortens deadline to 5 s.

6.3 HTTP/1.1

• Strict start-line and field parsing per RFC 9112; obs-fold rejected; Host required; 400 on bad version, bad chunk, or Content-Length plus Transfer-Encoding conflict.
• Transfer codings: must parse chunked including extensions (ignored); unknown coding gets 501. Trailers parsed and exposed, capped by same field limits.
• Pipelining: accepted but processed sequentially in order; no parallel handler execution per connection.
• Responses: automatic Content-Length or chunked; HEAD sends head only; 204/304 never carry a body.

6.4 HTTP/2

• Entry: h2 over TLS via ALPN h2; h2c via prior knowledge (PRI preface) or Upgrade from H1. Cleartext default in dev, TLS default in prod (TLS owned by 03-security.md).
• Preface and Settings handshake mandatory; bad preface is connection error PROTOCOL_ERROR.
• HPACK: static plus dynamic table, decoder cap 4096 bytes default (SETTINGS_HEADER_TABLE_SIZE ceiling 16384); oversized header list is COMPRESSION_ERROR.
• Flow control: initial window 65535 both directions; max frame 16384 (ceiling 16777215); WINDOW_UPDATE honored; writes block on window via backpressure, never buffer unbounded.
• Streams: states idle, reserved, open, half-closed, closed per RFC 9113; max 100 concurrent; RST_STREAM on handler cancel or caps; PRIORITY frames accepted and ignored.
• No server push. No CONNECT except plain tunnels if later specified; v1 returns 405 for exotic methods unless a sibling spec claims them.

6.5 Compression

• Encodings: gzip and br, both in-house. Negotiation picks br over gzip when q-values tie and type is compressible.
• Threshold: only bodies >= 1024 bytes and content-type in allowlist (html, json, xml, text, css, js, svg, wasm except precompressed). Never compress images, video, audio, zip, pdf, or responses with Content-Encoding already set.
• Levels: gzip 6 dynamic; brotli quality 4 dynamic, 11 for precompressed static assets at build time.
• Static precompression: build may store .gz and .br siblings; server selects by Accept-Encoding and sends Vary plus correct Content-Encoding and ETag variant suffix (-gz, -br).
• Bomb guard: decompression of request bodies capped at 10x compressed size and 10 MB absolute; overflow is 413.

6.6 Cookies (transport only)

• Request: parse Cookie per RFC 6265; join H2 split fields with "; "; cap 50 cookies and 8 KB total; overflow is 431.
• Response: emit one Set-Cookie per cookie; attributes Domain, Path, Max-Age/Expires, Secure, HttpOnly, SameSite (Lax default, None requires Secure), Partitioned passthrough.
• Signed and encrypted values: this file defines Cookie and ISigner interfaces; crypto and key hierarchy live in 03-security.md. Format: v1.<base64url(payload)>.<base64url(tag)>.
• Size: single Set-Cookie max 4 KB; larger is a handler error, never silently truncated.

6.7 Static files

• Providers: DiskProvider (root jailed) and EmbeddedProvider (from 09-cli-ops.md embedder). Same FileServer logic over both.
• Safety: lexical normalize then verify prefix is inside root; reject NUL, backslash on Windows, .. escapes; symlinks not followed unless follow_symlinks=true and target stays inside root; dotfiles denied by default.
• Validators: strong ETag "size-mtimehex" for disk, "len-hashhex" for embedded; Last-Modified from mtime; send both; honor If-None-Match then If-Modified-Since; 304 with validators and Cache-Control.
• Ranges: Accept-Ranges: bytes; single range yields 206 with Content-Range; multiple ranges yield 416 in v1 (documented gap, multipart later); If-Range honored; bad range yields 416 with Content-Range: bytes */len.
• Defaults: Cache-Control public, max-age=3600 for versioned assets, no-cache for index HTML (overridable); directory index off (403); only GET, HEAD, OPTIONS; others 405.
• Performance: sendfile or TransmitFile on disk when no compression or encryption transform; otherwise bounded streaming (64 KB chunks).

6.8 HTTP/3 path (design only)

• Connection owns a Transport abstraction (TCP today). H2 logic never touches sockets directly. A future QUIC transport can reuse semantic Request/Response, HPACK successor QPACK stays isolated in a new v3 directory. No QUIC code in v1.

7. Execution steps (in build order)

1. Reactor skeleton: IPoller, EpollPoller, Task, EventLoop with timers, ThreadPool, echo TCP server, signal handling.
2. Lifecycle core: Listener, Connection, timeouts, watermarks, graceful drain, structured accept/close logging hooks.
3. HTTP/1.1: strict parser, serializer, keep-alive, chunked, pipelining queue, 400/413/431/501 mapping.
4. H2 framing: preface, settings, HPACK codec, stream states, headers/data flow, flow control, GOAWAY/RST/PING.
5. Protocol detection: ALPN wiring point, h2c prior knowledge and Upgrade, version-independent dispatch.
6. Compression: gzip encoder/decoder, brotli encoder/decoder, negotiator, thresholds, Vary, bomb guards.
7. Cookies: parser, serializer, jar join/split, signer interface with stub, attribute defaults.
8. Static: providers, traversal guard, ETag/Last-Modified, conditional, single range, cache headers, sendfile path.
9. Hardening pass: all caps enforced, fuzz harnesses for H1/H2/HPACK/cookies, slowloris and large-body tests.
10. Integration seam: hand semantic Request/Response to 05-request-routing.md, export counters to 02-observability.md, accept TLS handle from 03-security.md.

8. Validation (how to verify each step)

• Step 1: echo server holds 10k idle connections under 512 MB; kill -INT drains in under 30 s; blocking job overflow returns 503 not hang.
• Step 2: header/body/idle timeouts fire within 10 percent; high/low watermarks pause and resume reads; drain closes listeners first then connections.
• Step 3: RFC 9112 vectors pass (chunked, trailers, bad framing gives 400+close, unknown TE gives 501); keep-alive reuses connection for 1000 requests; pipelined requests answer in order.
• Step 4: h2spec-style checks pass for settings, window updates, stream reset, GOAWAY; HPACK round-trips interop vectors; oversized header list triggers COMPRESSION_ERROR.
• Step 5: ALPN h2 negotiates H2, http/1.1 negotiates H1; prior-knowledge h2c works; H1 Upgrade to h2c works; wrong preface closes with PROTOCOL_ERROR.
• Step 6: Accept-Encoding br/gzip/identity matrix returns right encoding plus Vary; bodies under 1024 bytes stay identity; bomb input stops at cap with 413.
• Step 7: RFC 6265 cookie vectors parse; H2 split Cookie joins correctly; Set-Cookie never merged; oversized jar returns 431.
• Step 8: traversal attempts (.., encoded, NUL, symlink escape) all fail closed; conditional requests return 304; single range returns 206 and bad range 416; embedded and disk providers return byte-identical bodies.
• Step 9: fuzzers run 1M mutated inputs with zero crashes or hangs; slowloris with 1000 trickling connections stays under caps.
• Step 10: integration suite from 10-workflow-example.md passes end to end through real sockets for H1 and H2; disabled observability changes throughput by under 2 percent.

9. References

• Depends on: 02-observability.md (event, metric, trace hooks and zero-cost macros); 03-security.md (TLS handle, cookie signing and encryption, body and header ceilings alignment).
• Feeds: 05-request-routing.md (semantic Request/Response, app and router dispatch); 06-realtime.md (H1 Upgrade path for WebSocket, H2 streams for SSE); 08-clients-openapi.md (framing and HPACK reuse for test and REST clients); 09-cli-ops.md (embedded asset format, dev server listener reuse, health endpoints).
• Parallel: 04-data.md and 07-supervisor-replication.md consume lifecycle and Task but define their own pooling and durability.

10. Open questions

1. Default loop count: thread-per-core with SO_REUSEPORT, or single acceptor plus worker loops for simpler Windows parity?
2. H2 PRIORITY: fully ignore, or keep a minimal scheduling hint for SSE vs API multiplexing?
3. Multipart ranges in v1 or documented 416 gap until demand appears?
4. Brotli static dictionary: ship full default dictionary or trimmed web subset for binary size?
5. Cookie chunking for >4 KB signed values: reject, or support name_0..N reassembly?

11. Sources

• RFC 9112 HTTP/1.1, R. Fielding et al., Internet Standard, https://www.rfc-editor.org/rfc/rfc9112 (retrieved 2026-10-23).
• RFC 9113 HTTP/2, M. Thomson et al., Proposed Standard obsoleting RFC 7540/8740, https://www.rfc-editor.org/rfc/rfc9113 (retrieved 2026-10-23).
• RFC 9110 HTTP Semantics, R. Fielding et al., Internet Standard, https://www.rfc-editor.org/rfc/rfc9110 (retrieved 2026-10-23).
• RFC 6265 HTTP State Management Mechanism, A. Barth, https://www.rfc-editor.org/rfc/rfc6265 (retrieved 2026-10-23).
• RFC 7932 Brotli Compressed Data Format, https://www.rfc-editor.org/rfc/rfc7932 (retrieved 2026-10-23); RFC 1952 GZIP File Format 4.3, https://www.rfc-editor.org/rfc/rfc1952 (retrieved 2026-10-23).
• C++ coroutines reference, https://en.cppreference.com/cpp/language/coroutines (retrieved 2026-10-23).
• epoll man page man7/epoll.7, https://man7.org/linux/man-pages/man7/epoll.7.html (retrieved 2026-10-23); Windows async I/O and IOCP, https://learn.microsoft.com/en-us/windows/win32/fileio/synchronous-and-asynchronous-i-o (retrieved 2026-10-23).