retoor/moloapp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
moloapp/plans/06-realtime.md
T
retoor 54068f1953 docs: split plan into numbered detailed files with research 
Ten sequential spec files plus index covering runtime,
observability, security, data, routing, realtime, supervisor,
clients, CLI ops, and workflow, each with best practices,
structure, rules, guidelines, specs, and cross-references.
2026-10-23 14:22:29 +02:00

19 KiB
Raw Permalink Blame History

06 - Realtime

Title: Realtime (WebSocket, SSE, Socket.IO, Streaming, WebDAV, Client Parity) Scope: plan.md sections 2, 6-7 partial, 10 step 4 partial.

Purpose and philosophy

Realtime turns MoloApp from a request/response framework into a live system: bidirectional WebSocket channels, one-way server push over SSE, multiplexed Socket.IO namespaces with rooms, unbounded streaming responses, and WebDAV resource management, every one of them usable through the same middleware, auth, session, logging and metrics path as plain HTTP handlers.

Philosophy:

  • One connection model. A realtime connection is a long-lived request context, not a parallel universe. It enters through routing, middleware, auth and sessions exactly like any handler, then upgrades.
  • Protocol correctness first. Wire formats follow the governing standard byte for byte (RFC 6455 framing, WHATWG event-stream grammar, Socket.IO v5 packet encoding, RFC 4918 method semantics), verified against third-party clients, not only our own.
  • Client parity is a release gate. Per plan.md section 2, everything served ships a matching client (WebSocket, SSE, Socket.IO, WebDAV), sharing config, auth, reconnect and logging behavior with the server side. A protocol without its client is not done.
  • Backpressure everywhere. Slow consumers must slow producers or be shed by rule, never buffer unboundedly until the process dies.
  • Layered, not duplicated. Socket.IO sits on Engine.IO, Engine.IO sits on the WebSocket/long-polling transports, streaming sits on the response hierarchy from 05-request-routing.md. Shared behavior lives once in a base class.

Best practices (from research)

  1. WebSocket handshake: validate Upgrade: websocket and Connection: Upgrade, require a Sec-WebSocket-Key, answer 101 Switching Protocols with Sec-WebSocket-Accept computed as base64(SHA-1(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11")). (RFC 6455)
  2. WebSocket framing: opcodes 0x0 continuation, 0x1 text (must be valid UTF-8), 0x2 binary, 0x8 close, 0x9 ping, 0xA pong; control frames max 125 bytes payload and never fragmented; server MUST fail connections whose client frames are unmasked, and MUST NOT mask server-to-client frames. (RFC 6455)
  3. Liveness: answer every ping with a pong carrying identical payload, run an application heartbeat (ping interval plus pong timeout) and close dead peers with a close frame and status code. (RFC 6455)
  4. SSE wire format: Content-Type: text/event-stream, UTF-8 only, fields event:, data:, id:, retry:, blank line dispatches; default event type is message; multiline data joins with LF. (WHATWG HTML 9.2)
  5. SSE resumability: server assigns monotonically increasing id: per stream; on reconnect the client sends Last-Event-ID and the server replays missed events; HTTP 204 tells a client to stop reconnecting; initial retry: sets reconnect delay. (WHATWG HTML)
  6. Engine.IO v4.1: transports polling then WebSocket upgrade (WebTransport out of scope for v1); packet types open 0, close 1, ping 2, pong 3, message 4, upgrade 5, noop 6; handshake returns JSON {sid, upgrades, pingInterval, pingTimeout, maxPayload}; default path /engine.io/ with EIO=4 query; at most one active polling GET and one WebSocket per session. (Engine.IO protocol)
  7. Socket.IO v5: packet types CONNECT 0, DISCONNECT 1, EVENT 2, ACK 3, CONNECT_ERROR 4, BINARY_EVENT 5, BINARY_ACK 6; text encoding <type>[<attachments>-][<nsp>,][<ackId>]<json>, each packet wrapped as one Engine.IO message (wire prefix 4); namespace omitted when /; CONNECT handshake carries session id or refusal reason; binary payloads extracted as attachments with placeholder objects. (Socket.IO protocol)
  8. WebDAV: implement PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK over the RFC 4918 data model; multistatus 207 with DAV: XML bodies; honor Depth: 0,1,infinity (infinity capped, see Specifications); advertise compliance via the DAV: header; locks are advisory with tokens, timeouts and refresh. (RFC 4918)
  9. Streaming: chunked transfer with explicit flush points, SSE and file/upstream passthrough stream with bounded buffers and pause signals; compression never buffers a whole stream. Corroborated by HTTP semantics practice and the WHATWG line-by-line processing model.

Structure

Directories (one class or concern per file, deep structure):

  • molo/realtime/websocket/: server/ (WebSocketView, WebSocketResponse, WebSocketHandler, FrameParser, FrameSerializer, HandshakeValidator, HeartbeatManager), client/ (WebSocketClient, ClientHandshake, ReconnectPolicy shared with all clients).
  • molo/realtime/sse/: SseResponse, EventEncoder, EventStore (replay buffer interface), SseClient (EventSource-compatible: readyState CONNECTING/OPEN/CLOSED, Last-Event-ID tracking).
  • molo/realtime/engineio/: EngineSession, PollingTransport, WebSocketTransport, UpgradeManager, HeartbeatScheduler, PacketCodec (types 0-6, polling payload framing).
  • molo/realtime/socketio/: SocketIoServer, Namespace, RoomManager, EventDispatcher, AckTracker, PacketCodec (types 0-6, binary placeholder extraction), client/ (SocketIoClient with identical namespace/room/event API).
  • molo/realtime/streaming/: ChunkedBody, PassthroughBody, BackpressureValve, FlushPolicy (used by SSE, streaming responses, WebDAV passthrough).
  • molo/realtime/webdav/: server/ (DavView, PropfindHandler, ProppatchHandler, MkcolHandler, CopyMoveHandler, LockManager, MultistatusBuilder, QuotaGuard), client/ (WebDavClient with PROPFIND/GET/PUT/DELETE passthrough, response cache, quota accounting).
  • molo/realtime/shared/: ConnectionContext (auth/session/logging handle shared by every realtime connection), SendQueue (bounded per-connection queue), RateLimiter hook.

Namespaces: molo::realtime::websocket, ::sse, ::engineio, ::socketio, ::streaming, ::webdav. All connection types derive from ConnectionContext; all clients derive from BaseRealtimeClient (auth, reconnect, backoff, logging in one place).

Rules (hard, numbered)

  1. Every realtime endpoint mounts on a router and passes through the full middleware, auth and session chain before upgrade. No side-channel listeners.
  2. Server MUST reject client WebSocket frames without the mask bit by failing the connection; server MUST NOT mask its own frames.
  3. Text frames MUST contain valid UTF-8; invalid UTF-8 fails the connection with close code 1007.
  4. Control frames MUST have payload length at most 125 bytes and FIN set; fragmented control frames are a protocol error (1002).
  5. Max message size, max frame count per message, max connections per IP/user/route, send-queue depth and idle timeout are static caps from config; exceeding them closes or sheds by rule, never grows.
  6. SSE responses MUST send Content-Type: text/event-stream, Cache-Control: no-store, and MUST flush headers immediately so intermediaries do not buffer the stream.
  7. Socket.IO and Engine.IO packet codecs MUST round-trip the official test vectors (packet type tables in Specifications) before any feature work builds on them.
  8. Client parity: no realtime server feature merges without its matching client supporting the same auth, reconnect/backoff and logging behavior in the same change.
  9. Slow consumers MUST be shed: when a send queue is full, apply the route policy (drop oldest, drop newest, or close with code) and count it in metrics. Unbounded queues are forbidden.
  10. WebDAV Depth: infinity MUST be rewritten to the configured max depth; recursive operations MUST enforce quotas and per-request resource ceilings.
  11. Locks MUST have finite timeouts and MUST auto-expire; lock tokens MUST be unguessable random values.
  12. Streaming and passthrough paths MUST propagate backpressure to the producer (pause reads) and MUST support client disconnect cleanup within one heartbeat interval.

Development guidelines

  • Build bottom-up: handshake plus framing codecs first with byte-level tests against captured third-party traffic (browsers, websocat equivalents, official Socket.IO client), then views, then rooms and namespaces, then clients, then WebDAV last.
  • Every codec gets a fuzzer-fed unit harness plus golden vectors from the official protocol test suites; every view gets integration tests through the real stack per 10-workflow-example.md.
  • Reuse, never fork: heartbeat, reconnect with jittered exponential backoff, send queues and logging context are implemented once in shared/ and composed into each protocol.
  • Coroutine handlers (co_await) for message loops; blocking work (XML parsing for large PROPFIND, disk-backed event replay) goes to the off-loop executor defined in 01-runtime.md.
  • Log connection lifecycle (open, upgrade, heartbeat loss, close code and reason) at fixed levels with correlation IDs per 02-observability.md; per-connection debug logging is gated and capped.
  • Document every wire-visible behavior in the component spec before coding it; docs state what the system does today, gaps listed next to the code that closes them.

Specifications

  • WebSocket handshake: GET with Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Key (16 random bytes, base64), optional Sec-WebSocket-Protocol subprotocol negotiation (server selects at most one, echoes it, else fails); response 101 with Sec-WebSocket-Accept. Origin checked against CORS policy from 03-security.md.
  • WebSocket opcodes: 0x0 cont, 0x1 text, 0x2 binary, 0x8 close (0-125 byte body: 2-byte code plus UTF-8 reason), 0x9 ping, 0xA pong. Close codes used: 1000 normal, 1001 going away, 1002 protocol error, 1003 unsupported data, 1007 bad UTF-8, 1008 policy violation, 1009 message too big, 1011 internal error. Fragmented messages reassembled up to max message size.
  • Caps (defaults, all overridable): max message 1 MiB, max frame payload 256 KiB, max frames per message 64, send queue 256 messages, connections per IP 128, per route 100k, idle timeout 60 s, ping interval 25 s, pong timeout 20 s (Engine.IO defaults aligned), close handshake timeout 5 s.
  • SSE event grammar: lines field: value, fields event, data, id, retry, : comment (keep-alive); blank line dispatches; retry: is milliseconds, clamped to 1 s..300 s; server sends comment keep-alive every 15 s; Last-Event-ID replay window holds last 1000 events or 5 minutes per stream, whichever is smaller.
  • SseClient states CONNECTING 0, OPEN 1, CLOSED 2; fails the connection on non-200 status or wrong content type; honors 301/307 redirect and 204 stop; sends stored last event id on reconnect.
  • Engine.IO: path /engine.io/, query EIO=4, transport=polling|websocket, sid after handshake; open packet JSON keys exactly sid, upgrades, pingInterval, pingTimeout, maxPayload; polling GET long-holds up to 25 s; POST bodies carry length-prefixed packet payloads per the polling encoding; upgrade probe ping/probe then upgrade.
  • Socket.IO: namespaces start with /, no commas; rooms are server-side sets with join/leave, broadcast excludes sender by default with explicit include flag; EVENT payload is a JSON array whose first element is the event name; ack id is a monotonically increasing integer per connection with 30 s default ack timeout; binary attachments use {"_placeholder":true,"num":N} markers, max 16 attachments and 8 MiB total per packet by default.
  • Rooms and broadcast fan-out MUST complete in O(subscribers) with per-recipient queue caps; broadcast to 0 subscribers is a no-op counted in metrics, not an error.
  • Streaming: chunked bodies flush at least every 64 KiB or 100 ms during activity; passthrough (proxy/file/WebDAV GET) uses a 256 KiB ring buffer with producer pause; range and conditional requests supported on file passthrough.
  • WebDAV server: resource tree rooted at a configured collection; methods GET, PUT, DELETE plus PROPFIND (allprop, propname, prop selections), PROPPATCH (set/remove with 207 per-property status), MKCOL, COPY/MOVE (with Overwrite, Destination, Depth headers), LOCK (exclusive and shared, Timeout, Depth: 0), UNLOCK (matching Lock-Token); DAV: 1 header on OPTIONS and responses; XML bodies use DAV: namespace, max XML body 1 MiB, max depth 32.
  • WebDAV client: streams GET directly into a caller response body (no full buffering), conditional GET with ETag cache (max 128 entries, 64 MiB), quota accounting hooks per transfer.
  • Replication hooks: connection open/close, room join/leave and broadcast envelopes are emitted as typed events for 07-supervisor-replication.md; v1 guarantees at-most-once delivery across instances, at-least-once only inside one process via the send queue.

Execution steps (in build order)

  1. shared/: ConnectionContext, SendQueue, BaseRealtimeClient (auth, jittered backoff, logging), BackpressureValve.
  2. WebSocket: HandshakeValidator, FrameParser/Serializer, WebSocketView/Handler/Response, HeartbeatManager.
  3. WebSocketClient plus conformance tests against a third-party peer.
  4. SSE: EventEncoder, SseResponse, EventStore replay, SseClient.
  5. Streaming: ChunkedBody, PassthroughBody, FlushPolicy; wire through SSE and file download.
  6. Engine.IO: PacketCodec, EngineSession, PollingTransport, WebSocketTransport, UpgradeManager, HeartbeatScheduler.
  7. Socket.IO: PacketCodec (text plus binary), Namespace, RoomManager, EventDispatcher, AckTracker, SocketIoServer.
  8. SocketIoClient with namespace/room/event parity and long-polling fallback.
  9. WebDAV server: MultistatusBuilder, PROPFIND/PROPPATCH/MKCOL, COPY/MOVE, LockManager, QuotaGuard, DavView.
  10. WebDavClient with streaming GET passthrough, cache, quota.
  11. Cross-cutting: metrics per connection/room/route, trace spans for upgrade and broadcast, living-example realtime pages.

Validation (how to verify each step)

  1. Shared: unit tests prove queue shed policies and backoff timings; no unbounded growth under a blocked consumer test.
  2. WebSocket: golden handshake vectors (key/accept pairs); byte-level frame round-trips including fragmentation, masking rejection, UTF-8 rejection, control-frame limits; interop test against an independent client implementation.
  3. Client: connects to a third-party echo server, survives restart with backoff, exchanges text plus binary.
  4. SSE: byte-exact event encoding tests; browser/EventSource interop including reconnect with Last-Event-ID replay and 204 stop; wrong content-type fails cleanly.
  5. Streaming: 1 GiB passthrough with flat memory profile; disconnect mid-stream releases all resources within one heartbeat interval; flush latency within policy.
  6. Engine.IO: open/handshake JSON shape, ping/pong heartbeat timing, polling-to-WebSocket upgrade sequence, duplicate transport rejection, unknown sid returns 400.
  7. Socket.IO: official packet vectors for all 7 types including binary placeholders; namespace connect/refuse; rooms join/leave/broadcast; ack round-trip and timeout.
  8. Client parity: scripted session where our client drives the official server behaviors (namespaces, rooms, acks, binary) and vice versa; long-polling-only mode passes with WebSocket disabled.
  9. WebDAV: PROPFIND/MKCOL/COPY/MOVE/LOCK/UNLOCK sequences against litmus-style assertions; Depth infinity capped; quota exceeded returns 507; lock expiry releases.
  10. Client: PROPFIND listings, streaming GET into a response with bounded memory, ETag cache hits, quota accounting.
  11. Observability: connection counts, message rates, shed counts, room sizes and broadcast fan-out exported per 02-observability.md; full integration suite green through the real stack.

References

  • Depends on 01-runtime.md: event loop, lifecycle, timeouts, HTTP/1.1 framing and upgrade path, compression interplay.
  • Depends on 02-observability.md: log format, correlation IDs, metrics and tracing primitives, zero-cost gating.
  • Depends on 03-security.md: origin/CORS checks, rate limits, body caps, auth/session resolution before upgrade.
  • Depends on 05-request-routing.md: routers, view base classes, response hierarchy, middleware ordering, DI access to shared services.
  • Feeds 04-data.md: pubsub bridge for cross-instance broadcast and event replay storage interfaces.
  • Feeds 07-supervisor-replication.md: connection/room events, broadcast envelopes, delivery guarantees, multi-instance session sharing.
  • Feeds 08-clients-openapi.md: shared client transport behavior (timeouts, retries, TLS, test doubles) reused by realtime clients.
  • Feeds 10-workflow-example.md: integration test templates for realtime routes and living-example chat, feed and DAV pages.

Open questions

  1. WebTransport (Engine.IO transport) needs HTTP/3, which 01-runtime.md lists as a future path. Stay out of v1: yes, unless runtime lands H3 early.
  2. Should Socket.IO v2/v3 protocol compat be offered? Proposal: no, v5 plus Engine.IO v4 only; document the refusal clearly.
  3. Cross-instance room broadcast transport: reuse pubsub from 04-data.md or a dedicated replication channel in 07-supervisor-replication.md? Needs a joint decision.
  4. SSE replay durability: in-memory ring per process for v1, or persistent store from the start? Proposal: in-memory with a documented interface for a persistent EventStore later.
  5. WebDAV ACL (RFC 3744) and versioning (DeltaV): explicitly out of v1; confirm no v1 consumer needs them.

Sources

  1. IETF RFC 6455, The WebSocket Protocol (Fette, Melnikov, Dec 2011, Standards Track). Handshake, framing, opcodes, masking, close codes. URL: https://www.rfc-editor.org/rfc/rfc6455 (retrieved 2026-10-23).
  2. WHATWG HTML Living Standard, section 9.2 Server-sent events (EventSource interface, processing model, event-stream parsing and interpretation). Wire grammar, reconnect, Last-Event-ID, 204 stop. URL: https://html.spec.whatwg.org/multipage/server-sent-events.html (retrieved 2026-10-23).
  3. Socket.IO Protocol v5, official docs (packet types 0-6, encoding format, binary placeholders, sample session). URL: https://socket.io/docs/v4/socket-io-protocol/ (retrieved 2026-10-23).
  4. Engine.IO Protocol v4.1, official docs (transports, handshake, heartbeat, upgrade, packet encoding). URL: https://socket.io/docs/v4/engine-io-protocol/ (retrieved 2026-10-23).
  5. IETF RFC 4918, HTTP Extensions for WebDAV (Dusseault, ed., Jun 2007, Standards Track, obsoletes RFC 2518). Methods PROPFIND/PROPPATCH/MKCOL/COPY/MOVE/LOCK/UNLOCK, 207 Multistatus, Depth/DAV headers. URL: https://www.rfc-editor.org/rfc/rfc4918 (retrieved 2026-10-23).
Reference in New Issue View Git Blame Copy Permalink