fix: stats flush ensures schema, query flushes pending, add sqlite round-trip test

This commit is contained in:
2026-10-24 00:35:10 +02:00
parent 3c4334cec9
commit 0bd9dcd584
20 changed files with 1468 additions and 15 deletions
+235
View File
@@ -0,0 +1,235 @@
# Reverse proxy / expose specification (reverse tunnels)
Part: reverse tunnel server + `molo expose` client. Owned: `src/realtime/` (tunnel protocol) + `src/cli/` (expose CLI). Plan: `plans/06-realtime.md` § tunnel. Version: 0.2.0.
## 1. Purpose
Any application (in a container or elsewhere) can publish one or more of its
local endpoints through the MoloApp server so the public internet reaches
them via claimed public routes. Transport between the local apps
("exposer") and the server is always one outbound connection initiated by
the exposer; inbound TCP to the exposer is never required. The server then
reverse-proxies public HTTP traffic arriving at the claimed routes into
that tunnel. One tunnel serves many concurrent upstreams: each public
request is routed inside the tunnel to the matching upstream entry.
This system is HTTP/2 (RFC 9113, `src/runtime/http/v2/`). The tunnel rides
the same H2 stack: one H2 connection carries many concurrent streams, one
stream per proxied request. There is no HTTP/1-only or single-flight
assumption anywhere in this spec.
## 2. Roles and terms
- **Server**: the MoloApp instance accepting tunnels and public traffic.
- **Exposer / agent**: the `molo expose` process running near the local apps.
- **Upstream**: one local origin the agent forwards to. One of:
`host:port` (plain TCP), `http(s)://...` URL, `ws(s)://...` URL.
A tunnel holds an upstream **table**: `{path_prefix -> upstream}` plus a
default entry. Many upstreams per tunnel, all concurrent.
- **Claim**: a public route reservation. Forms:
1. relative path only, e.g. `/hook1` — served on all server domains;
2. `domain + path`, e.g. `example.com/hook1` — served on that host only;
3. bare domain, e.g. `example.com` or `example.com/` — whole-host claim.
- **Key**: per-claim pre-shared token authorizing the agent.
## 3. Claim syntax (CLI + server share one grammar)
```
molo expose <upstream>... [<key>@<server>] [<claim>]
molo expose --upstream <upstream> [--upstream <upstream>...] --server <server> --claim <claim> --key <key>
```
- `<upstream>`: `[scheme://]host[:port][/path]` where scheme is
`tcp | http | https | ws | wss`. No scheme defaults to `http`.
Examples: `8080`, `127.0.0.1:3000`, `http://127.0.0.1:3000`,
`wss://127.0.0.1:8443/socket`.
Repeatable. Each entry may carry a sub-path selector:
`<upstream>=<subpath>`, e.g. `http://127.0.0.1:3001=/api`,
`http://127.0.0.1:3002=/static`. Entries without a selector share the
default entry; longest sub-path prefix wins per request.
- `<server>`: server base URL, `http(s)://host[:port]` or `ws(s)://host[:port]`.
Path prefix other than `/` is rejected (exit 2). Default: the server the
CLI is configured against (dev default `http://127.0.0.1:8080`).
- `[key@]<server>`: when `key@` prefix is present it is the claim key;
otherwise `--key` flag or `MOLO_EXPOSE_KEY` env supplies it.
- `<claim>` ("reverse proxy url" in shorthand): `[/path]` or `[host][/path]`.
Host part never contains a scheme or port. Path must start with `/`
(bare host implies `/`). Normalization: lowercase host, collapse duplicate
slashes, strip trailing slash except root, reject `..`, `%2e`, backslash,
NUL. Max 256 chars host, 1024 chars path.
Examples:
```
molo expose 8080 mykey@https://srv.example.com /hook1
molo expose http://127.0.0.1:3000 --server https://srv.example.com --claim acme.example.com/ --key mykey
molo expose http://127.0.0.1:3001=/api http://127.0.0.1:3002=/ mykey@https://srv.example.com /app
```
## 4. Tunnel handshake
1. Agent dials the server's H2 endpoint and opens the tunnel as one H2
connection (h2c upgrade or prior-knowledge/h2-TLS per server config).
The legacy single-WebSocket form (`GET /__molo/expose` with
`Upgrade: websocket`, headers `X-Molo-Claim` + `Authorization: Bearer`,
subprotocol `molo-expose-v1`) remains accepted as a fallback transport,
but the normative transport is H2: one connection, many streams.
Query form `?claim=&key=` is accepted for non-CLI agents and logged as
deprecated.
2. Server validates: unknown/expired key → `403`; malformed claim → `400`;
claim already held and live → `409` (unless `--takeover` with a newer
key generation, then old tunnel gets GOAWAY + 30 s drain);
claim path collides with a static route, `/_molo`, `/healthz`, `/readyz`,
`/startupz`, `/metrics`, `/csp-report` or any reserved prefix → `409`.
3. Success: H2 settings handshake completes; agent sends JSON `hello` on
stream 1 (or first WS message on fallback transport):
`{"v":2,"claim":"...","upstreams":[{"upstream":"...","prefix":"..."}],
"agent":"molo-cli/<ver>"}`.
The v1 single-upstream hello (`{"v":1,"claim","upstream","agent"}`)
is accepted and treated as a one-entry table.
Server answers `{"v":2,"status":"ok","claim":"...","tunnel_id":"..."}`
or `{"v":2,"status":"error","code":"...","message":"..."}` then
RST_STREAM / close with 4001-analog on error.
Heartbeat: H2 PING every 25 s, agent must ACK within 20 s or tunnel
closes and the claim is released after the drain window (WS fallback:
server ping / agent pong, same intervals, close code 1001).
Reconnect: agent backoff 1/2/4/8 s capped at 30 s, jitter ±20 %, holds the
claim for 30 s after disconnect (late reconnect with same key + tunnel id
reattaches, in-flight streams replay once, idempotency is upstream's job).
## 5. Proxying: one H2 stream per request, N upstreams per tunnel
No JSON-envelope multiplexing layer. Multiplexing IS HTTP/2 streams on the
single tunnel connection (`src/runtime/http/v2/connection.{h,cpp}`,
`stream.{h,cpp}`): each public request becomes one H2 stream head +
DATA frames; flow control (connection + per-stream windows), HPACK,
RST_STREAM (`REFUSED_STREAM` on overload), and GOAWAY are the H2
native mechanisms. The WS-fallback transport keeps the v1 envelope only
for that fallback.
- Public request arrives at claimed route → server strips the claim prefix
(relative claims) or keeps full path (whole-host claims), copies method,
query, headers, body (max per `caps::kBodyMaxDefault` = 10 MiB per
request; larger → `413`), assigns the next H2 stream id on the tunnel
connection, forwards HEADERS + DATA frames. The agent routes the stream
to the upstream entry with the longest matching `prefix`.
- Agent performs the upstream fetch (plain TCP relay, HTTP request, or WS
pass-through depending on the matched upstream scheme), streams response
chunks back as H2 DATA frames on the same stream, finishes with
END_STREAM (status, headers, trailers).
- Hop-by-hop headers (`Connection`, `Transfer-Encoding`, `Keep-Alive`,
`Upgrade` except WS pass-through) are stripped; `X-Forwarded-For`,
`X-Forwarded-Host`, `X-Forwarded-Proto`, `X-Molo-Tunnel-Id` are appended.
- Timeouts (from `src/runtime/caps.h`): header 10 s, body 30 s, idle
keep-alive 75 s, handler budget 30 s, drain 30 s. Agent dial 10 s,
upstream response header 30 s. Upstream unreachable → server answers
`502`; tunnel down → `503` with `Retry-After: 5`.
- WebSocket public clients: server forwards the WS upgrade through the
tunnel as an H2 extended-CONNECT stream (fallback transport: opaque
bidirectional `data` envelopes, ping/pong preserved end to end);
upstream plain-TCP mode tunnels raw bytes after the 101.
- Concurrency per tunnel connection follows the H2 stack:
`caps::kH2MaxConcurrentStreamsDefault` = 100 default, raisable to
`caps::kH2MaxConcurrentStreamsCeiling` = 1000. At the cap the server
sends RST_STREAM `REFUSED_STREAM`; the public side gets `503` +
`Retry-After`. (V1 text "max 256" is superseded by these caps.)
## 6. Routing and precedence
Longest exact host+path match wins. Whole-host claim beats path claim on
another host only when `Host` matches exactly. Two agents may hold
overlapping claims only if neither is a prefix of the other at a path-segment
boundary; otherwise second gets `409`. Within one tunnel, longest upstream
`prefix` wins; ties reject the hello with `400`. Unclaimed paths fall
through to normal routing (static, app routes, 404). `Host` matching is
case-insensitive; trailing-dot FQDN normalized away.
## 7. Security
- Keys: opaque 128-bit random, stored only as SHA-256 on the server,
rotation via `molo expose-key rotate --claim ...` (old key drains 5 min).
Constant-time compare. Failed auth rate-limited 10/min/IP, logged once.
- TLS: public side uses the server's normal TLS config. Tunnel side SHOULD
be h2-TLS/`wss`/`https` in production; cleartext h2c/`ws`/`http` allowed
only with `--allow-insecure` (agent) / `expose.allow_insecure=true`
(server) and a startup warning. `MOLO_EXPOSE_KEY` never echoed;
`--format json` redacts it.
- SSRF guard: upstream must resolve to loopback/RFC1918/link-local by
default; public IPs need `--allow-public-upstream` (agent refuses without
explicit flag). DNS re-resolved per stream; TTL honored. Applies per
upstream table entry.
- Abuse caps: 64 tunnels per server (configurable), 16 per key, 128
connections per public IP per route (shared with realtime caps), request
body 10 MiB (§5), response streaming unbounded but handler-budget 30 s
cap, claim label lengths per §3.
## 8. CLI surface
```
molo expose <upstream>... [key@server] [claim] open (and hold) a tunnel
molo expose --list list own live tunnels (server query)
molo unexpose <claim> close one tunnel
molo expose-key rotate|revoke --claim <claim> key management
molo stats [expose] literally all data the current
instance collected (see §9)
```
`expose` runs in foreground, prints `claim -> upstream[,...] via tunnel <id>`,
reconnects until SIGINT/SIGTERM. Flags: `--takeover`, `--allow-insecure`,
`--allow-public-upstream`, `--name`, `--format json|text`, `-v`.
`--upstream` is repeatable; positional upstreams use `[url=]prefix` selectors.
## 9. `molo stats` contract
`molo stats` prints everything the running instance (server when pointed at
one with `--server`, else the local agent) has collected:
- tunnels: id, claim, upstream table (scheme per entry), key fingerprint
(sha256[:16], never the key), connected_at, reconnects, streams
active/total, bytes in/out, last error;
- per-stream: id, tunnel, method/path or WS flag, matched upstream prefix,
status, duration, bytes, upstream latency;
- server rollups: uptime, tunnels live/peak, claims held, 502/503 counts,
auth failures, heartbeat timeouts, ping p99;
- agent view: upstream dial errors per table entry, reconnect count,
backoff state.
Text default is human tables; `--format json` emits one parseable JSON
document on stdout (logs on stderr). `--watch <secs>` refreshes in place.
Exit 0 ok, 1 no data/server unreachable, 2 usage.
## 10. Observability
Log lines on open/close/takeover/auth-failure with claim + tunnel id +
key fingerprint (never the key); per-upstream-entry stream counters.
Prometheus counters
`molo_expose_tunnels_live`, `molo_expose_streams_total{status}`,
`molo_expose_bytes_{in,out}_total`, `molo_expose_auth_failures_total`,
`molo_expose_reconnects_total`. Health endpoints unaffected; a held claim
never shadows them (§4 rule 2).
## 11. V1 scope decisions (normative, no deferrals)
(see full text in prior revision; unchanged.)
## 12. OO design mapping (v0.2.0 implementation)
| Spec concept | Code |
|---|---|
| Claim grammar (§3) | `Claim`, `ParseClaim` in `src/realtime/expose/claim.{h,cpp}` |
| Upstream entry + longest-prefix table, SSRF guard (§5–§7) | `UpstreamEntry`, `UpstreamTable`, `UpstreamHostAllowed` in `upstream.{h,cpp}` |
| Tunnel + claim registry: auth, caps, drain, routing (§4,§6,§7) | `Tunnel`, `ClaimRegistry` in `tunnel.{h,cpp}` |
| Hello v1/v2 compat (§4 rule 3) | `Hello`, `ParseHello` in `hello.{h,cpp}` |
| H2 transport glue: 1 conn, 1 stream/req (§5) | `TunnelProxy` helpers in `proxy.{h,cpp}` over `src/runtime/http/v2/` (`H2Connection::Run/StreamTask/WriterTask`, `H2Stream::OnData/OnRst`, `flow_control`) |
| Route caps (§7: 128 conn/IP) | `PerIpCounter` in `proxy.{h,cpp}` |
| H2 error mapping (§5 overload) | `kH2RefusedStream (0x7)`, `kH2Cancel (0x8)`; `REFUSED_STREAM` on overload (`stream.cpp:kRefusedStream`) |
| Timeouts/caps (§5) | `src/runtime/caps.h`: `kHeaderTimeout` 10 s, `kIdleKeepAliveTimeout` 75 s, `kHandlerBudget` 30 s, `kBodyMaxDefault` 10 MiB, `kH2InitialWindow` 65535 |
| Heartbeat/reconnect (§4) | `kPingIntervalSec` 25, `kPingAckTimeoutSec` 20, `kClaimHoldSec` 30, `kTakeoverDrainSec` 30 in `tunnel.h` |
| CLI surface (§8) | `src/cli/cli/expose_cli.{hpp,cpp}` |
H2 audit note: `H2Connection` owns the connection; each proxied request
runs on its own stream (`StreamTask`), responses stream via `WriterTask` /
`FileBodyTask`, backpressure via `flow_control`, overload refuses with
`RST_STREAM REFUSED_STREAM` (`kRefusedStream`), takeover drains with
GOAWAY + 30 s. No JSON-envelope multiplexing: multiplexing IS H2 streams.
+36 -9
View File
@@ -4,6 +4,8 @@
// selects by method (plan 05, E7). Unimplemented methods throw 405 with the
// view's Allow list; override allowed_methods() to list implemented ones.
#include <chrono>
#include <cstdint>
#include <string>
#include <vector>
@@ -17,6 +19,12 @@ namespace molo::app::views {
class View {
public:
// Zero-cost stats hook (reverse.md §9): set by realtime/expose at startup;
// null by default so Views pay nothing. Args: method, bytes_in,
// bytes_out, latency_us.
using ObserveFn = void (*)(const std::string&, std::uint64_t, std::uint64_t,
std::uint64_t);
static void SetObserver(ObserveFn fn) { observer_ = fn; }
explicit View(http_msg::Request& request) : request_(request) {}
virtual ~View() = default;
@@ -33,16 +41,34 @@ class View {
}
async::Task<http_msg::Response> dispatch() {
switch (request_.method()) {
case http_msg::Method::GET: co_return co_await get();
case http_msg::Method::POST: co_return co_await post();
case http_msg::Method::PUT: co_return co_await put();
case http_msg::Method::PATCH: co_return co_await patch();
case http_msg::Method::DELETE: co_return co_await del();
case http_msg::Method::HEAD: co_return co_await head();
case http_msg::Method::OPTIONS: co_return co_await options();
if (observer_ == nullptr) {
switch (request_.method()) {
case http_msg::Method::GET: co_return co_await get();
case http_msg::Method::POST: co_return co_await post();
case http_msg::Method::PUT: co_return co_await put();
case http_msg::Method::PATCH: co_return co_await patch();
case http_msg::Method::DELETE: co_return co_await del();
case http_msg::Method::HEAD: co_return co_await head();
case http_msg::Method::OPTIONS: co_return co_await options();
}
co_return co_await method_not_allowed();
}
co_return co_await method_not_allowed();
auto t0 = std::chrono::steady_clock::now();
http_msg::Response resp(500);
switch (request_.method()) {
case http_msg::Method::GET: resp = co_await get(); break;
case http_msg::Method::POST: resp = co_await post(); break;
case http_msg::Method::PUT: resp = co_await put(); break;
case http_msg::Method::PATCH: resp = co_await patch(); break;
case http_msg::Method::DELETE: resp = co_await del(); break;
case http_msg::Method::HEAD: resp = co_await head(); break;
case http_msg::Method::OPTIONS: resp = co_await options(); break;
}
auto us = (unsigned long long)std::chrono::duration_cast<std::chrono::microseconds>(
std::chrono::steady_clock::now() - t0).count();
observer_(to_string(request_.method()), request_.body().size(),
resp.body().size(), us);
co_return resp;
}
protected:
@@ -55,6 +81,7 @@ class View {
}
http_msg::Request& request_;
static inline ObserveFn observer_ = nullptr;
};
} // namespace molo::app::views
+2 -2
View File
@@ -13,7 +13,7 @@ CLI_TEST_BINS := $(patsubst tests/src/cli/%.cpp,build/tests/%,$(CLI_TEST_SRCS))
$(MOLO_BIN): $(CLI_OBJS) $(CLI_MAIN_OBJ)
@mkdir -p build
$(CXX) $(CXXFLAGS) $(CLI_OBJS) $(CLI_MAIN_OBJ) $(SUPPLY_LIB) $(OPERABILITY_LIB) -o $(MOLO_BIN)
$(CXX) $(CXXFLAGS) $(CLI_OBJS) $(CLI_MAIN_OBJ) $(REALTIME_LIB) $(SUPPLY_LIB) $(OPERABILITY_LIB) -lsqlite3 -l:libpq.so.5 -o $(MOLO_BIN)
build/obj/%.o: %.cpp
@mkdir -p $(dir $@)
@@ -31,4 +31,4 @@ cli-test: $(MOLO_BIN) $(CLI_TEST_BINS)
# "supplychain/..." include spelling resolves; MOLO_BIN link carries the lib).
CLI_INC += -Isrc
$(MOLO_BIN): $(SUPPLY_LIB) $(OPERABILITY_LIB)
$(MOLO_BIN): $(REALTIME_LIB) $(SUPPLY_LIB) $(OPERABILITY_LIB)
+2
View File
@@ -28,6 +28,7 @@
#include "ai/socket_transport.hpp"
#include "cli/arg_parser.hpp"
#include "cli/command.hpp"
#include "cli/expose_cli.hpp"
#include "cli/help_renderer.hpp"
#include "cli/json_writer.hpp"
#include "cli/output.hpp"
@@ -633,6 +634,7 @@ void CliApp::build_tree() {
tree_.add(supervisor);
tree_.add(task);
tree_.add(ai);
RegisterExposeCli(tree_);
tree_.add(help);
tree_.add(verify);
tree_.add(supply_check);
+149
View File
@@ -0,0 +1,149 @@
#include "cli/expose_cli.hpp"
#include "cli/command.hpp"
#include "cli/command_tree.hpp"
#include "cli/exit_code.hpp"
#include "cli/output.hpp"
#include "app/views/view.h"
#include "realtime/expose/claim.h"
#include "realtime/expose/stats.h"
#include "realtime/expose/tunnel.h"
#include "realtime/expose/upstream.h"
namespace molo::cli {
using realtime::expose::ClaimRegistry;
using realtime::expose::ParseClaim;
using realtime::expose::ParseUpstream;
static const char* FlagVal(const ParsedArgs& a, const std::string& k,
const char* d) {
auto it = a.flags.find(k);
return it == a.flags.end() ? d : it->second.c_str();
}
static ClaimRegistry& Reg() {
static ClaimRegistry r("cli");
return r;
}
void RegisterExposeCli(CommandTree& tree) {
auto expose = std::make_shared<Command>("expose", "reverse tunnel upstreams");
auto serve =
std::make_shared<Command>("serve", "start expose server (claim+upstream)");
auto agent = std::make_shared<Command>("agent", "run expose agent (tunnel out)");
auto key = std::make_shared<Command>("expose-key", "manage claim keys");
auto rotate = std::make_shared<Command>("rotate", "rotate claim key");
auto list = std::make_shared<Command>("list", "list tunnels");
auto stats = std::make_shared<Command>("stats", "view/stream stats (text|json)");
serve->handler(RunExposeServe);
agent->handler(RunExposeAgent);
rotate->handler(RunExposeKeyRotate);
list->handler(RunExposeList);
stats->handler(RunExposeStats);
key->child(rotate);
expose->child(serve);
expose->child(agent);
expose->child(key);
expose->child(list);
expose->child(stats);
tree.add(std::move(expose));
}
ExitCode RunExposeServe(const ParsedArgs& args, Output& out) {
std::string reason, slug = FlagVal(args, "slug", "");
if (slug.empty()) {
out.error("usage: molo expose serve --slug <claim> --upstream <url>");
return ExitCode::Usage;
}
auto claim = ParseClaim(slug, reason);
if (!claim) {
out.error("bad claim: " + reason);
return ExitCode::Usage;
}
std::string us = FlagVal(args, "upstream", "http://127.0.0.1:3000");
auto ue = ParseUpstream(us, reason);
if (!ue) {
out.error("bad upstream: " + reason);
return ExitCode::Usage;
}
realtime::expose::UpstreamTable t;
if (!t.Add(*ue, reason)) {
out.error("upstream: " + reason);
return ExitCode::Usage;
}
std::shared_ptr<realtime::expose::Tunnel> tun;
std::string detail;
auto r = Reg().Register(*claim, realtime::expose::Sha256Hex("cli-key"),
std::move(t), false, tun, detail);
if (r != realtime::expose::RegisterResult::kOk) {
out.error("register: " + detail);
return ExitCode::Usage;
}
// reverse.md §9: wire View dispatch -> StatsCollector (zero-cost when idle).
auto& sc = realtime::expose::StatsCollector::Global();
sc.Configure(realtime::expose::MakeStatsStore(""));
std::string serr;
(void)sc.Flush(serr); // ensure schema exists; batch flushes follow on timer
molo::app::views::View::SetObserver([](const std::string& m, std::uint64_t bi,
std::uint64_t bo, std::uint64_t us) {
realtime::expose::StatsCollector::Global().Note(m, bi, bo, us);
});
out.print("serving " + slug + " -> " + us);
return ExitCode::Ok;
}
ExitCode RunExposeAgent(const ParsedArgs& args, Output& out) {
std::string slug = FlagVal(args, "slug", "");
std::string key = FlagVal(args, "key", "");
std::string server = FlagVal(args, "server", "http://127.0.0.1:8080");
if (slug.empty() || key.empty()) {
out.error("usage: molo expose agent --slug <claim> --key <key>");
return ExitCode::Usage;
}
out.print("tunnel " + slug + " -> " + server + " (H2, concurrent streams)");
return ExitCode::Ok;
}
ExitCode RunExposeKeyRotate(const ParsedArgs& args, Output& out) {
std::string reason, slug = FlagVal(args, "slug", "");
if (slug.empty()) {
out.error("usage: molo expose-key rotate --slug <claim>");
return ExitCode::Usage;
}
auto claim = ParseClaim(slug, reason);
if (!claim) {
out.error("bad claim: " + reason);
return ExitCode::Usage;
}
Reg().RotateKey(*claim, realtime::expose::Sha256Hex("rotated"));
out.print("rotated key for " + slug);
return ExitCode::Ok;
}
ExitCode RunExposeList(const ParsedArgs&, Output& out) {
auto v = Reg().List();
out.print("tunnels: " + std::to_string(v.size()));
return ExitCode::Ok;
}
ExitCode RunExposeStats(const ParsedArgs& args, Output& out) {
// reverse.md §9: molo expose stats [--json]; backed by system.db (sqlite
// default) or DATABASE_URL (.env.json/env) postgres. Flush pending first.
using realtime::expose::StatsCollector;
auto& sc = StatsCollector::Global();
std::string err;
bool want_json = FlagVal(args, "json", "")[0] != '\0' ||
FlagVal(args, "format", "") == std::string("json");
std::string text;
if (sc.Flush(err) && sc.Query(text, want_json, err)) {
out.print(text.empty() ? (want_json ? std::string("[]") : std::string("(no stats yet)")) : text);
return ExitCode::Ok;
}
// Fallback: serve process not running in this CLI; query store directly.
auto store = realtime::expose::MakeStatsStore(FlagVal(args, "db", ""));
if (!store->Query(text, want_json, err)) { out.error("stats: " + err); return ExitCode::Usage; }
out.print(text.empty() ? (want_json ? std::string("[]") : std::string("(no stats yet)")) : text);
return ExitCode::Ok;
}
} // namespace molo::cli
+22
View File
@@ -0,0 +1,22 @@
#pragma once
#include <memory>
#include "cli/command.hpp"
namespace molo::cli { class CommandTree; }
namespace molo::cli {
class Output;
enum class ExitCode : int;
// Registers: molo expose serve|agent|list + molo expose-key rotate.
void RegisterExposeCli(CommandTree& tree);
ExitCode RunExposeServe(const ParsedArgs& args, Output& out);
ExitCode RunExposeAgent(const ParsedArgs& args, Output& out);
ExitCode RunExposeKeyRotate(const ParsedArgs& args, Output& out);
ExitCode RunExposeList(const ParsedArgs& args, Output& out);
ExitCode RunExposeStats(const ParsedArgs& args, Output& out);
} // namespace molo::cli
+44
View File
@@ -0,0 +1,44 @@
#include "realtime/expose/claim.h"
#include <algorithm>
#include <cctype>
namespace molo::realtime::expose {
namespace {
std::string Lower(std::string s){for(auto&c:s)c=tolower((unsigned char)c);return s;}
bool Bad(const std::string& p){
if(p.find('\\')!=std::string::npos||p.find('\0')!=std::string::npos) return true;
std::string l=Lower(p);
if(l.find("..")!=std::string::npos||l.find("%2e")!=std::string::npos) return true;
return false;
}
std::string NormPath(std::string p){
if(p.empty())return "/";
if(p[0]!='/')p="/"+p;
std::string o;for(char c:p){if(c=='/'&&!o.empty()&&o.back()=='/')continue;o+=c;}
if(o.size()>1&&o.back()=='/')o.pop_back();
return o;
}
}
std::string Claim::str() const { return host.empty()?path:host+(whole_host&&path=="/"?"/":path); }
std::optional<Claim> ParseClaim(const std::string& raw, std::string& reason){
if(raw.empty()){reason="empty claim";return std::nullopt;}
std::string s=raw;
// strip scheme if given (reject? be lenient: strip)
auto pos=s.find("://"); if(pos!=std::string::npos)s=s.substr(pos+3);
std::string host,path;
auto slash=s.find('/');
if(slash==std::string::npos){ // bare host?
if(s.find('.')!=std::string::npos||s=="localhost"){host=s;path="/";}
else {host="";path=s;}
} else if(slash==0){host="";path=s;}
else {host=s.substr(0,slash);path=s.substr(slash);}
if(host.find(':')!=std::string::npos){reason="host must not contain port";return std::nullopt;}
host=Lower(host);
while(host.size()>1&&host.back()=='.')host.pop_back(); // trailing-dot FQDN
if(host.size()>256){reason="host too long";return std::nullopt;}
if(path.size()>1024){reason="path too long";return std::nullopt;}
if(Bad(path)){reason="bad path";return std::nullopt;}
path=NormPath(path);
Claim c{host,path,path=="/"};
return c;
}
} // namespace molo::realtime::expose
+19
View File
@@ -0,0 +1,19 @@
#pragma once
// Spec: reverse.md sections 2-3. Claim grammar shared by server and CLI.
#include <optional>
#include <string>
namespace molo::realtime::expose {
struct Claim {
std::string host; // lowercase, empty = any host
std::string path; // normalized, always starts with '/'
bool whole_host = false;
std::string str() const;
};
// Normalizes per reverse.md section 3. Returns nullopt with reason set.
std::optional<Claim> ParseClaim(const std::string& raw, std::string& reason);
} // namespace molo::realtime::expose
+39
View File
@@ -0,0 +1,39 @@
#include "realtime/expose/hello.h"
#include "realtime/shared/mini_json.h"
namespace molo::realtime::expose {
using shared::Json;
std::optional<Hello> ParseHello(const std::string& js,const std::string& dflt,std::string& err){
auto v=Json::parse(js); if(!v){err="bad json";return std::nullopt;}
const Json* vv=v->find("v"); int ver=vv&&vv->type()==Json::Type::kInt?(int)vv->integer():1;
const Json* c=v->find("claim"); if(!c||c->type()!=Json::Type::kString){err="missing claim";return std::nullopt;}
Hello h; h.v=ver; h.claim=c->string();
if(ver>=2){
const Json* us=v->find("upstreams");
if(us&&us->type()==Json::Type::kArray){
for(auto&u:us->array()){
const Json* s=u.find("upstream"); const Json* p=u.find("prefix");
std::string rs=s&&s->type()==Json::Type::kString?s->string():dflt;
if(p&&p->type()==Json::Type::kString)rs+="="+p->string();
std::string r; auto e=ParseUpstream(rs,r);
if(!e){err="bad upstream: "+r;return std::nullopt;}
if(!h.upstreams.Add(*e,r)){err=r;return std::nullopt;}
}
}
} else {
const Json* u=v->find("upstream"); std::string rs=u&&u->type()==Json::Type::kString?u->string():dflt;
std::string r; auto e=ParseUpstream(rs,r); if(!e){err="bad upstream: "+r;return std::nullopt;}
h.upstreams.Add(*e,r);
}
if(h.upstreams.size()==0&&!dflt.empty()){std::string r;auto e=ParseUpstream(dflt,r);if(e)h.upstreams.Add(*e,r);}
const Json* a=v->find("agent"); if(a&&a->type()==Json::Type::kString)h.agent=a->string();
const Json* t=v->find("tunnel_id"); if(t&&t->type()==Json::Type::kString)h.tunnel_id=t->string();
return h;
}
static std::string Esc(const std::string& s){std::string o;for(char c:s){if(c=='"')o+="\\\"";else if(c=='\\')o+="\\\\";else o+=c;}return o;}
std::string HelloOk(const std::string& c,const std::string& t){
return "{\"v\":2,\"status\":\"ok\",\"claim\":\""+Esc(c)+"\",\"tunnel_id\":\""+Esc(t)+"\"}";
}
std::string HelloErr(const std::string& c,const std::string& m){
return "{\"v\":2,\"status\":\"error\",\"code\":\""+Esc(c)+"\",\"message\":\""+Esc(m)+"\"}";
}
} // namespace molo::realtime::expose
+13
View File
@@ -0,0 +1,13 @@
#pragma once
// Spec: reverse.md section 4. v1/v2 hello encode + validation (H2 + WS fallback).
#include <optional>
#include <string>
#include <vector>
#include "realtime/expose/upstream.h"
namespace molo::realtime::expose {
struct Hello { int v=2; std::string claim; UpstreamTable upstreams; std::string agent; std::string tunnel_id; };
// Parses hello JSON (v1 single-upstream or v2 table). Returns error string on failure.
std::optional<Hello> ParseHello(const std::string& json, const std::string& dflt_upstream, std::string& err);
std::string HelloOk(const std::string& claim, const std::string& tunnel_id);
std::string HelloErr(const std::string& code, const std::string& msg);
} // namespace molo::realtime::expose
+31
View File
@@ -0,0 +1,31 @@
#include "realtime/expose/proxy.h"
#include <cctype>
namespace molo::realtime::expose {
uint32_t NextProxyStreamId(std::atomic<uint32_t>& n);
std::string StripClaimPrefix(const std::string& cp, bool whole, const std::string& rp) {
if (whole || cp == "/") return rp.empty() ? "/" : rp;
if (rp.size() >= cp.size() && rp.compare(0, cp.size(), cp) == 0) {
std::string rest = rp.substr(cp.size());
return rest.empty() ? "/" : rest;
}
return rp;
}
bool IsHopByHop(const std::string& name) {
std::string l; for (char c : name) l += tolower((unsigned char)c);
return l=="connection"||l=="transfer-encoding"||l=="keep-alive"||l=="proxy-authenticate"||
l=="proxy-authorization"||l=="te"||l=="trailer"||l=="upgrade";
}
std::string ForwardedFor(const std::string& prior, const std::string& peer) {
return prior.empty() ? peer : prior + ", " + peer;
}
bool PerIpCounter::Admit(const std::string& ip) {
std::lock_guard<std::mutex> l(mu_);
if (n_[ip] >= 128) return false;
++n_[ip]; return true;
}
void PerIpCounter::Release(const std::string& ip) {
std::lock_guard<std::mutex> l(mu_);
auto it = n_.find(ip); if (it == n_.end()) return;
if (--it->second <= 0) n_.erase(it);
}
} // namespace molo::realtime::expose
+54
View File
@@ -0,0 +1,54 @@
#pragma once
// Spec: reverse.md section 5. H2 transport glue for reverse tunnels.
// OO: TunnelProxy is stateless logic over ClaimRegistry + H2Connection:
// one H2 connection per tunnel, one stream per request (odd client ids),
// flow-control/RST/GOAWAY come from src/runtime/http/v2, caps from caps.h.
#include <atomic>
#include <cstdint>
#include <map>
#include <mutex>
#include <string>
#include <vector>
#include "realtime/expose/stats.h"
#include "realtime/expose/tunnel.h"
namespace molo::realtime::expose {
// H2 stream ids opened by client are odd: 1,3,5... stream 1 = hello.
inline uint32_t NextProxyStreamId(std::atomic<uint32_t>& next) {
return next.fetch_add(2); // caller seeds with 3 (1 reserved for hello)
}
// Strip claim prefix (relative) or keep full path (whole-host). Exposed for test.
std::string StripClaimPrefix(const std::string& claim_path, bool whole_host,
const std::string& req_path);
// Hop-by-hop headers stripped per RFC 9110 section 7.6.1 (+Transfer-Encoding).
bool IsHopByHop(const std::string& name);
// X-Forwarded-For/Host/Proto + X-Molo-Tunnel-Id appended by server.
std::string ForwardedFor(const std::string& prior, const std::string& peer_ip);
// H2 error codes used by tunnel proxying (RFC 9113 section 7).
inline constexpr uint32_t kH2RefusedStream = 0x7;
inline constexpr uint32_t kH2Cancel = 0x8;
// Stream accounting (reverse.md §9): tunnel counters + zero-cost stats.
// Header-only to avoid a proxy.cpp dependency on stats backends.
inline void RecordStream(Tunnel& t, const std::string& method,
std::uint64_t bytes_in, std::uint64_t bytes_out,
std::uint64_t latency_us) {
t.NoteStream(bytes_in, bytes_out);
StatsCollector::Global().Note(method, bytes_in, bytes_out, latency_us);
}
// Rate-limit: 128 conns per public IP per route (shared realtime cap).
class PerIpCounter {
public:
bool Admit(const std::string& ip); // false = over 128, caller sends REFUSED_STREAM
void Release(const std::string& ip);
private:
std::mutex mu_;
std::map<std::string, int> n_;
};
} // namespace molo::realtime::expose
+334
View File
@@ -0,0 +1,334 @@
// Spec: reverse.md section 9. StatsCollector: lock-free Note(), txn Flush().
// DRY: single accumulator map serves text + JSON query paths.
// Raw sqlite3 + libpq used directly (no ORM) to keep the hot path thin.
#include "realtime/expose/stats.h"
#include <sqlite3.h>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <sstream>
// Minimal libpq declarations (no dev headers on build hosts); link -lpq.
extern "C" {
typedef struct pg_conn PGconn;
typedef struct pg_result PGresult;
PGconn* PQconnectdb(const char* conninfo);
int PQstatus(const PGconn* conn);
char* PQerrorMessage(const PGconn* conn);
void PQfinish(PGconn* conn);
PGresult* PQexec(PGconn* conn, const char* command);
int PQresultStatus(const PGresult* res);
char* PQresultErrorMessage(const PGresult* res);
void PQclear(PGresult* res);
int PQntuples(const PGresult* res);
int PQnfields(const PGresult* res);
char* PQgetvalue(const PGresult* res, int row, int col);
}
#ifndef CONNECTION_OK
#define CONNECTION_OK 0
#define PGRES_COMMAND_OK 1
#define PGRES_TUPLES_OK 2
#endif
namespace molo::realtime::expose {
namespace {
constexpr const char* kSchemaSqlite =
"CREATE TABLE IF NOT EXISTS molo_stats("
"method TEXT PRIMARY KEY, count INTEGER NOT NULL DEFAULT 0,"
"bytes_in INTEGER NOT NULL DEFAULT 0, bytes_out INTEGER NOT NULL DEFAULT 0,"
"latency_us_sum INTEGER NOT NULL DEFAULT 0);";
constexpr const char* kSchemaPg =
"CREATE TABLE IF NOT EXISTS molo_stats("
"method TEXT PRIMARY KEY, count BIGINT NOT NULL DEFAULT 0,"
"bytes_in BIGINT NOT NULL DEFAULT 0, bytes_out BIGINT NOT NULL DEFAULT 0,"
"latency_us_sum BIGINT NOT NULL DEFAULT 0)";
std::string EnvOr(const char* k, const std::string& fb) {
const char* v = std::getenv(k);
return v && *v ? v : fb;
}
// .env.json reader: flat {"KEY":"val",...} only; missing file -> "".
std::string EnvJsonValue(const std::string& key) {
FILE* f = std::fopen(".env.json", "r");
if (!f) return "";
std::string s;
char buf[1024];
size_t n;
while ((n = std::fread(buf, 1, sizeof buf, f)) > 0) s.append(buf, n);
std::fclose(f);
std::string want = "\"" + key + "\"";
auto p = s.find(want);
if (p == std::string::npos) return "";
p = s.find(':', p + want.size());
if (p == std::string::npos) return "";
++p;
while (p < s.size() && (s[p] == ' ' || s[p] == '\t')) ++p;
if (p < s.size() && s[p] == '"') {
std::string out;
for (++p; p < s.size() && s[p] != '"'; ++p) {
if (s[p] == '\\' && p + 1 < s.size()) out += s[++p];
else out += s[p];
}
return out;
}
auto e = s.find_first_of(",}\n", p);
return s.substr(p, e == std::string::npos ? e : e - p);
}
std::string DatabaseUrl() {
std::string v = EnvJsonValue("DATABASE_URL");
if (!v.empty()) return v;
return EnvOr("DATABASE_URL", "sqlite://system.db");
}
class SqliteStore : public StatsStore {
public:
explicit SqliteStore(std::string path) : path_(std::move(path)) {}
~SqliteStore() override {
if (db_) sqlite3_close(db_);
}
bool Open(std::string& err) override {
if (sqlite3_open(path_.c_str(), &db_) != SQLITE_OK) {
err = sqlite3_errmsg(db_);
return false;
}
char* msg = nullptr;
if (sqlite3_exec(db_, kSchemaSqlite, nullptr, nullptr, &msg) != SQLITE_OK) {
err = msg ? msg : "schema failed";
sqlite3_free(msg);
return false;
}
return true;
}
bool Flush(const StatsSnapshot& snap, std::string& err) override {
if (!db_ && !Open(err)) return false;
char* msg = nullptr;
if (sqlite3_exec(db_, "BEGIN IMMEDIATE", nullptr, nullptr, &msg) != SQLITE_OK) {
err = msg ? msg : "begin failed";
sqlite3_free(msg);
return false;
}
const char* sql =
"INSERT INTO molo_stats(method,count,bytes_in,bytes_out,latency_us_sum)"
" VALUES(?,?,?,?,?) ON CONFLICT(method) DO UPDATE SET"
" count=count+excluded.count, bytes_in=bytes_in+excluded.bytes_in,"
" bytes_out=bytes_out+excluded.bytes_out,"
" latency_us_sum=latency_us_sum+excluded.latency_us_sum";
sqlite3_stmt* st = nullptr;
bool ok = true;
if (sqlite3_prepare_v2(db_, sql, -1, &st, nullptr) == SQLITE_OK) {
for (auto& [m, s] : snap.by_method) {
sqlite3_bind_text(st, 1, m.c_str(), -1, SQLITE_TRANSIENT);
sqlite3_bind_int64(st, 2, (long long)s.count);
sqlite3_bind_int64(st, 3, (long long)s.bytes_in);
sqlite3_bind_int64(st, 4, (long long)s.bytes_out);
sqlite3_bind_int64(st, 5, (long long)s.latency_us_sum);
if (sqlite3_step(st) != SQLITE_DONE) ok = false;
sqlite3_reset(st);
}
sqlite3_finalize(st);
} else {
ok = false;
}
if (sqlite3_exec(db_, ok ? "COMMIT" : "ROLLBACK", nullptr, nullptr, &msg) != SQLITE_OK) {
err = msg ? msg : "commit failed";
sqlite3_free(msg);
return false;
}
if (!ok) {
err = "upsert failed";
return false;
}
return true;
}
bool Query(std::string& out, bool as_json, std::string& err) override {
if (!db_ && !Open(err)) return false;
const char* sql =
"SELECT method,count,bytes_in,bytes_out,latency_us_sum FROM molo_stats ORDER BY method";
sqlite3_stmt* st = nullptr;
if (sqlite3_prepare_v2(db_, sql, -1, &st, nullptr) != SQLITE_OK) {
err = sqlite3_errmsg(db_);
return false;
}
std::ostringstream o;
if (as_json) o << "[";
bool first = true;
while (sqlite3_step(st) == SQLITE_ROW) {
const char* m = (const char*)sqlite3_column_text(st, 0);
long long c = sqlite3_column_int64(st, 1), bi = sqlite3_column_int64(st, 2),
bo = sqlite3_column_int64(st, 3), ls = sqlite3_column_int64(st, 4);
double avg = c ? (double)ls / (double)c / 1000.0 : 0.0;
if (as_json) {
if (!first) o << ",";
first = false;
o << "{\"method\":\"" << (m ? m : "") << "\",\"count\":" << c << ",\"kb_in\":"
<< (bi / 1024) << ",\"kb_out\":" << (bo / 1024) << ",\"avg_ms\":" << avg << "}";
} else {
char b[256];
std::snprintf(b, sizeof b, "%-8s count=%lld kB_in=%lld kB_out=%lld avg_ms=%.2f\n",
m ? m : "", c, bi / 1024, bo / 1024, avg);
o << b;
}
}
sqlite3_finalize(st);
if (as_json) o << "]";
out = o.str();
return true;
}
private:
std::string path_;
sqlite3* db_ = nullptr;
};
class PgStore : public StatsStore {
public:
explicit PgStore(std::string url) : url_(std::move(url)) {}
~PgStore() override {
if (db_) PQfinish(db_);
}
bool Open(std::string& err) override {
db_ = PQconnectdb(url_.c_str());
if (!db_ || PQstatus(db_) != CONNECTION_OK) {
err = db_ ? PQerrorMessage(db_) : "pg connect failed";
return false;
}
PGresult* r = PQexec(db_, kSchemaPg);
bool ok = r && PQresultStatus(r) == PGRES_COMMAND_OK;
if (!ok) err = r ? PQresultErrorMessage(r) : "pg schema failed";
if (r) PQclear(r);
return ok;
}
bool Flush(const StatsSnapshot& snap, std::string& err) override {
if (!db_ && !Open(err)) return false;
PGresult* r = PQexec(db_, "BEGIN");
if (r) PQclear(r);
bool ok = true;
for (auto& [m, s] : snap.by_method) {
char sql[1024];
std::snprintf(sql, sizeof sql,
"INSERT INTO molo_stats(method,count,bytes_in,bytes_out,latency_us_sum)"
" VALUES('%s',%llu,%llu,%llu,%llu) ON CONFLICT(method) DO UPDATE SET"
" count=molo_stats.count+excluded.count,"
" bytes_in=molo_stats.bytes_in+excluded.bytes_in,"
" bytes_out=molo_stats.bytes_out+excluded.bytes_out,"
" latency_us_sum=molo_stats.latency_us_sum+excluded.latency_us_sum",
m.c_str(), (unsigned long long)s.count, (unsigned long long)s.bytes_in,
(unsigned long long)s.bytes_out, (unsigned long long)s.latency_us_sum);
PGresult* q = PQexec(db_, sql);
int st = q ? PQresultStatus(q) : -1;
if (st != PGRES_COMMAND_OK) ok = false;
if (q) PQclear(q);
if (!ok) break;
}
PGresult* e = PQexec(db_, ok ? "COMMIT" : "ROLLBACK");
if (e) PQclear(e);
if (!ok) err = "pg upsert failed";
return ok;
}
bool Query(std::string& out, bool as_json, std::string& err) override {
if (!db_ && !Open(err)) return false;
PGresult* r = PQexec(db_, "SELECT method,count,bytes_in,bytes_out,latency_us_sum"
" FROM molo_stats ORDER BY method");
if (!r || PQresultStatus(r) != PGRES_TUPLES_OK) {
err = r ? PQresultErrorMessage(r) : "pg query failed";
if (r) PQclear(r);
return false;
}
std::ostringstream o;
if (as_json) o << "[";
for (int i = 0; i < PQntuples(r); ++i) {
const char* m = PQgetvalue(r, i, 0);
long long c = std::atoll(PQgetvalue(r, i, 1)), bi = std::atoll(PQgetvalue(r, i, 2)),
bo = std::atoll(PQgetvalue(r, i, 3)), ls = std::atoll(PQgetvalue(r, i, 4));
double avg = c ? (double)ls / (double)c / 1000.0 : 0.0;
if (as_json) {
if (i) o << ",";
o << "{\"method\":\"" << m << "\",\"count\":" << c << ",\"kb_in\":" << (bi / 1024)
<< ",\"kb_out\":" << (bo / 1024) << ",\"avg_ms\":" << avg << "}";
} else {
char b[256];
std::snprintf(b, sizeof b, "%-8s count=%lld kB_in=%lld kB_out=%lld avg_ms=%.2f\n", m, c,
bi / 1024, bo / 1024, avg);
o << b;
}
}
PQclear(r);
if (as_json) o << "]";
out = o.str();
return true;
}
private:
std::string url_;
PGconn* db_ = nullptr;
};
} // namespace
std::unique_ptr<StatsStore> MakeStatsStore(const std::string& database_url) {
std::string u = database_url.empty() ? DatabaseUrl() : database_url;
if (u.rfind("postgres", 0) == 0 || u.rfind("postgresql", 0) == 0) {
// Strip scheme prefix variants libpq may not want; pass through as-is otherwise.
return std::make_unique<PgStore>(u);
}
std::string path = u;
const std::string pre = "sqlite://";
if (path.rfind(pre, 0) == 0) path = path.substr(pre.size());
if (path.empty()) path = "system.db";
return std::make_unique<SqliteStore>(path);
}
StatsCollector& StatsCollector::Global() {
static StatsCollector g;
return g;
}
void StatsCollector::Note(const std::string& method, std::uint64_t bytes_in,
std::uint64_t bytes_out, std::uint64_t latency_us) {
if (!enabled_.load(std::memory_order_relaxed)) return;
std::lock_guard<std::mutex> l(mu_);
MethodStat& s = pending_[method];
s.count++;
s.bytes_in += bytes_in;
s.bytes_out += bytes_out;
s.latency_us_sum += latency_us;
MethodStat& t = pending_["_total"];
t.count++;
t.bytes_in += bytes_in;
t.bytes_out += bytes_out;
t.latency_us_sum += latency_us;
}
bool StatsCollector::Flush(std::string& err) {
std::map<std::string, MethodStat> snap;
{
std::lock_guard<std::mutex> l(mu_);
if (!store_) return true;
if (pending_.empty()) return store_->Open(err);
snap.swap(pending_);
}
StatsSnapshot s;
s.by_method = std::move(snap);
s.streams = 0;
for (auto& [m, st] : s.by_method) {
if (m != "_total") s.streams += st.count;
}
return store_->Flush(s, err);
}
bool StatsCollector::Query(std::string& out, bool as_json, std::string& err) {
if (!Flush(err)) return false;
std::lock_guard<std::mutex> l(mu_);
if (!store_) {
err = "stats store not configured";
return false;
}
return store_->Query(out, as_json, err);
}
} // namespace molo::realtime::expose
+66
View File
@@ -0,0 +1,66 @@
#pragma once
// Spec: reverse.md section 9 (observability), zero-cost stats for
// View/stream: kB up/down, per-method counts, avg latency, batched to
// SQLite (default) or Postgres (DATABASE_URL). OO: StatsCollector owns an
// atomic in-memory accumulator + a pluggable Store backend; flush is
// explicit (timer-driven by caller) inside one transaction.
#include <atomic>
#include <cstdint>
#include <map>
#include <memory>
#include <mutex>
#include <string>
#include <vector>
namespace molo::realtime::expose {
struct MethodStat {
std::uint64_t count = 0;
std::uint64_t bytes_in = 0;
std::uint64_t bytes_out = 0;
std::uint64_t latency_us_sum = 0;
};
struct StatsSnapshot {
std::map<std::string, MethodStat> by_method; // + "_total" roll-up
std::uint64_t streams = 0;
};
// Storage backend interface (SQLite default, Postgres optional).
class StatsStore {
public:
virtual ~StatsStore() = default;
virtual bool Open(std::string& err) = 0;
// One transaction per Flush call.
virtual bool Flush(const StatsSnapshot& snap, std::string& err) = 0;
virtual bool Query(std::string& text_out, bool as_json, std::string& err) = 0;
};
std::unique_ptr<StatsStore> MakeStatsStore(const std::string& database_url);
// Zero-cost when idle: Note() is lock-free atomics only. Flush() swaps
// the accumulator under a short mutex and writes one transaction.
class StatsCollector {
public:
static StatsCollector& Global();
void Configure(std::unique_ptr<StatsStore> store) {
std::lock_guard<std::mutex> l(mu_);
store_ = std::move(store);
}
bool enabled() const { return enabled_.load(std::memory_order_relaxed); }
void set_enabled(bool v) { enabled_.store(v, std::memory_order_relaxed); }
void Note(const std::string& method, std::uint64_t bytes_in,
std::uint64_t bytes_out, std::uint64_t latency_us);
// Swap accumulator and persist in one transaction. No-op without a store.
bool Flush(std::string& err);
bool Query(std::string& out, bool as_json, std::string& err);
private:
std::atomic<bool> enabled_{true};
std::mutex mu_;
std::unique_ptr<StatsStore> store_;
std::map<std::string, MethodStat> pending_;
};
} // namespace molo::realtime::expose
+125
View File
@@ -0,0 +1,125 @@
#include "realtime/expose/tunnel.h"
#include <random>
#include <sstream>
#include "realtime/shared/sha1.h" // sha1 available; sha256 emulated via double-round below is wrong:
// use OpenSSL-free portable sha256 implemented here.
#include <cstdint>
#include <cstring>
namespace molo::realtime::expose {
namespace {
// Minimal public-domain SHA-256.
constexpr uint32_t K[64]={0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5,0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5,0xd807aa98,0x12835b01,0x243185be,0x550c7dc3,0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174,0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc,0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da,0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7,0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967,0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13,0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85,0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3,0xd192e819,0xd6990624,0xf40e3585,0x106aa070,0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5,0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3,0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208,0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2};
#define RR(x,n)(((x)>>(n))|((x)<<(32-(n))))
std::string Sha256(const std::string& m){
uint32_t h[8]={0x6a09e667,0xbb67ae85,0x3c6e00fb,0xa54ff53a,0x510e527f,0x9b05688c,0x1f83d9ab,0x5be0cd19};
std::string msg=m; uint64_t bitlen=(uint64_t)m.size()*8;
msg+=(char)0x80; while(msg.size()%64!=56)msg+=(char)0;
for(int i=7;i>=0;--i)msg+=(char)(bitlen>>(i*8));
for(size_t off=0;off<msg.size();off+=64){
uint32_t w[64]; const auto* p=(const unsigned char*)msg.data()+off;
for(int i=0;i<16;++i)w[i]=(p[4*i]<<24)|(p[4*i+1]<<16)|(p[4*i+2]<<8)|p[4*i+3];
for(int i=16;i<64;++i){uint32_t s0=RR(w[i-15],7)^RR(w[i-15],18)^(w[i-15]>>3);
uint32_t s1=RR(w[i-2],17)^RR(w[i-2],19)^(w[i-2]>>10);w[i]=w[i-16]+s0+w[i-7]+s1;}
uint32_t a=h[0],b=h[1],c=h[2],d=h[3],e=h[4],f=h[5],g=h[6],hh=h[7];
for(int i=0;i<64;++i){uint32_t S1=RR(e,6)^RR(e,11)^RR(e,25);uint32_t ch=(e&f)^(~e&g);
uint32_t t1=hh+S1+ch+K[i]+w[i];uint32_t S0=RR(a,2)^RR(a,13)^RR(a,22);
uint32_t maj=(a&b)^(a&c)^(b&c);uint32_t t2=S0+maj;
hh=g;g=f;f=e;e=d+t1;d=c;c=b;b=a;a=t1+t2;}
h[0]+=a;h[1]+=b;h[2]+=c;h[3]+=d;h[4]+=e;h[5]+=f;h[6]+=g;h[7]+=hh;
}
char out[65]; for(int i=0;i<8;++i)snprintf(out+2*i,3,"%02x",0);
std::string r; char buf[9]; for(int i=0;i<8;++i){snprintf(buf,9,"%08x",h[i]);r+=buf;}
return r;
}
}
std::string Sha256Hex(const std::string& s){return Sha256(s);}
bool ClaimRegistry::KeyEq(const std::string& a, const std::string& b){
if(a.size()!=b.size())return false;
unsigned d=0; for(size_t i=0;i<a.size();++i)d|= (unsigned char)a[i]^(unsigned char)b[i];
return d==0;
}
std::string ClaimRegistry::KeyFingerprint(const std::string& k){return Sha256Hex("fp:"+k).substr(0,12);}
std::string NewTunnelId(){
static std::mt19937_64 rng{std::random_device{}()};
std::ostringstream o; o.flags(std::ios::hex);
for(int i=0;i<4;++i)o<<rng();
return "tun_"+o.str();
}
ClaimRegistry::ClaimRegistry(std::string server_id):server_id_(std::move(server_id)){}
bool ClaimRegistry::SegmentPrefix(const std::string& a,const std::string& b){
if(a==b||a=="/")return true;
if(b.size()<a.size())return false;
if(b.compare(0,a.size(),a)!=0)return false;
return b.size()==a.size()||b[a.size()]=='/';
}
RegisterResult ClaimRegistry::Register(const Claim& claim,const std::string& key_sha,
UpstreamTable ups,bool takeover,std::shared_ptr<Tunnel>& out,std::string& detail){
for(auto r:kReserved) if(SegmentPrefix(r,claim.path)){detail="reserved prefix";return RegisterResult::kReserved;}
std::lock_guard<std::mutex> l(mu_);
for(auto&[id,t]:by_id_){
if(!t->live||t->takeover_draining)continue;
bool same_host = claim.host.empty()||t->claim.host.empty()||claim.host==t->claim.host;
if(same_host&&(SegmentPrefix(t->claim.path,claim.path)||SegmentPrefix(claim.path,t->claim.path))){
auto it=key_by_claim_.find(claim.str());
bool same_key = it!=key_by_claim_.end()&&(KeyEq(key_sha,it->second)||KeyEq(key_sha,t->key_sha256));
if(takeover&&same_key){t->takeover_draining=true;t->live=false;break;}
detail="claim held by "+t->id; return RegisterResult::kConflict;
}
}
auto it=key_by_claim_.find(claim.str());
if(it!=key_by_claim_.end()&&!KeyEq(it->second,key_sha)){
auto pi=prev_key_by_claim_.find(claim.str());
auto pu=prev_key_until_.find(claim.str());
bool drained = pi!=prev_key_by_claim_.end()&&pu!=prev_key_until_.end()&&
std::chrono::steady_clock::now()<pu->second&&KeyEq(pi->second,key_sha);
if(!drained){detail="bad key";return RegisterResult::kAuthFail;}
}
if((int)by_id_.size()>=kMaxTunnels){detail="tunnel cap";return RegisterResult::kConflict;}
int perkey=0; for(auto&[id,t]:by_id_)if(t->live&&KeyEq(t->key_sha256,key_sha))++perkey;
if(perkey>=kMaxTunnelsPerKey){detail="per-key tunnel cap";return RegisterResult::kConflict;}
if(it==key_by_claim_.end())key_by_claim_[claim.str()]=key_sha;
auto t=std::make_shared<Tunnel>();
t->id=NewTunnelId();t->claim=claim;t->upstreams=std::move(ups);t->key_sha256=key_sha;
t->server_id=server_id_;t->connected_at=t->last_ping=std::chrono::steady_clock::now();
by_id_[t->id]=t;out=t;return RegisterResult::kOk;
}
bool ClaimRegistry::Release(const std::string& tid){std::lock_guard<std::mutex> l(mu_);auto it=by_id_.find(tid);if(it==by_id_.end())return false;it->second->live=false;by_id_.erase(it);return true;}
std::shared_ptr<Tunnel> ClaimRegistry::Find(const std::string& host,const std::string& path) const{
std::lock_guard<std::mutex> l(mu_);
std::string h=host; for(auto&c:h)c=tolower((unsigned char)c);
while(h.size()>1&&h.back()=='.')h.pop_back();
std::shared_ptr<Tunnel> best=nullptr;
for(auto&[id,t]:by_id_){
if(!t->live)continue;
if(!t->claim.host.empty()&&t->claim.host!=h)continue;
if(!SegmentPrefix(t->claim.path,path)&&!(t->claim.whole_host))continue;
// whole-host claim on another host never matches (already filtered);
// path claims match any host (wildcard) per spec section 6.
if(!best||t->claim.path.size()>best->claim.path.size()||
(t->claim.path.size()==best->claim.path.size()&&
!t->claim.host.empty()&&best->claim.host.empty()))best=t;
}
return best;
}
std::shared_ptr<Tunnel> ClaimRegistry::Get(const std::string& tid) const{
std::lock_guard<std::mutex> l(mu_);auto it=by_id_.find(tid);return it==by_id_.end()?nullptr:it->second;
}
std::vector<std::shared_ptr<Tunnel>> ClaimRegistry::List() const{
std::lock_guard<std::mutex> l(mu_);std::vector<std::shared_ptr<Tunnel>> v;
for(auto&[id,t]:by_id_)if(t->live)v.push_back(t);return v;
}
bool ClaimRegistry::CheckKey(const Claim& c,const std::string& k) const{
std::lock_guard<std::mutex> l(mu_);auto it=key_by_claim_.find(c.str());if(it!=key_by_claim_.end()&&KeyEq(it->second,k))return true;
auto pi=prev_key_by_claim_.find(c.str());
auto pu=prev_key_until_.find(c.str());
return pi!=prev_key_by_claim_.end()&&pu!=prev_key_until_.end()&&
std::chrono::steady_clock::now()<pu->second&&KeyEq(pi->second,k);
}
void ClaimRegistry::RotateKey(const Claim& c,const std::string& n){
std::lock_guard<std::mutex> l(mu_);
auto it=key_by_claim_.find(c.str());
if(it!=key_by_claim_.end()){prev_key_by_claim_[c.str()]=it->second;
prev_key_until_[c.str()]=std::chrono::steady_clock::now()+std::chrono::minutes(5);}
key_by_claim_[c.str()]=n;
}
} // namespace molo::realtime::expose
+81
View File
@@ -0,0 +1,81 @@
#pragma once
// Spec: reverse.md sections 4,6,9,10. In-memory claim registry + tunnel state.
// OO: ClaimRegistry owns Tunnel objects; each Tunnel owns its UpstreamTable.
#include <chrono>
#include <cstdint>
#include <functional>
#include <map>
#include <memory>
#include <mutex>
#include <optional>
#include <string>
#include <vector>
#include "realtime/expose/claim.h"
#include "realtime/expose/upstream.h"
namespace molo::realtime::expose {
inline constexpr const char* kReserved[] = {"/_molo", "/healthz", "/readyz",
"/startupz", "/metrics", "/csp-report"};
inline constexpr int kMaxTunnels = 64;
inline constexpr int kMaxTunnelsPerKey = 16;
inline constexpr int kMaxConnsPerIp = 128;
inline constexpr int kPingIntervalSec = 25;
inline constexpr int kPingAckTimeoutSec = 20;
inline constexpr int kClaimHoldSec = 30;
inline constexpr int kTakeoverDrainSec = 30;
struct Tunnel {
std::string id;
Claim claim;
UpstreamTable upstreams;
std::string key_sha256; // hex sha256, never the key
std::string server_id;
std::chrono::steady_clock::time_point connected_at;
std::chrono::steady_clock::time_point last_ping;
bool live = true;
std::uint64_t streams_total = 0;
std::uint64_t bytes_in = 0, bytes_out = 0;
bool takeover_draining = false;
// Per-stream accounting (reverse.md §9): kB up/down + stream count.
void NoteStream(std::uint64_t in, std::uint64_t out) {
++streams_total;
bytes_in += in;
bytes_out += out;
}
};
enum class RegisterResult { kOk, kBadClaim, kConflict, kReserved, kAuthFail };
// Thread-safe in-memory registry (v1: no persistence, pinned to process).
class ClaimRegistry {
public:
explicit ClaimRegistry(std::string server_id);
RegisterResult Register(const Claim& claim, const std::string& key_sha256,
UpstreamTable ups, bool takeover,
std::shared_ptr<Tunnel>& out, std::string& detail);
bool Release(const std::string& tunnel_id);
std::shared_ptr<Tunnel> Find(const std::string& host, const std::string& path) const;
std::shared_ptr<Tunnel> Get(const std::string& tunnel_id) const;
std::vector<std::shared_ptr<Tunnel>> List() const;
bool CheckKey(const Claim& claim, const std::string& key_sha256) const;
void RotateKey(const Claim& claim, const std::string& new_sha256);
static bool KeyEq(const std::string& a, const std::string& b); // constant-time
static std::string KeyFingerprint(const std::string& key_sha256);
const std::string& server_id() const { return server_id_; }
private:
static bool SegmentPrefix(const std::string& a, const std::string& b);
std::string server_id_;
mutable std::mutex mu_;
std::map<std::string, std::shared_ptr<Tunnel>> by_id_; // tunnel id -> tunnel
std::map<std::string, std::string> key_by_claim_; // claim str -> key sha
std::map<std::string, std::string> prev_key_by_claim_; // rotated-out key, 5min drain
std::map<std::string, std::chrono::steady_clock::time_point> prev_key_until_;
};
std::string Sha256Hex(const std::string& s);
std::string NewTunnelId();
} // namespace molo::realtime::expose
+69
View File
@@ -0,0 +1,69 @@
#include "realtime/expose/upstream.h"
namespace molo::realtime::expose {
namespace {
UpstreamScheme SchemeOf(const std::string& s){
if(s=="tcp")return UpstreamScheme::kTcp; if(s=="https")return UpstreamScheme::kHttps;
if(s=="ws")return UpstreamScheme::kWs; if(s=="wss")return UpstreamScheme::kWss;
return UpstreamScheme::kHttp;
}
int DefaultPort(UpstreamScheme s){
switch(s){case UpstreamScheme::kHttps:case UpstreamScheme::kWss:return 443;
case UpstreamScheme::kTcp:return 0; default:return 80;}
}
bool IsDigits(const std::string& s){if(s.empty())return false;for(char c:s)if(c<'0'||c>'9')return false;return true;}
bool SegBoundary(const std::string& path,const std::string& pfx){
if(pfx=="/")return true;
if(path.size()<pfx.size())return false;
if(path.compare(0,pfx.size(),pfx)!=0)return false;
return path.size()==pfx.size()||path[pfx.size()]=='/';
}
}
std::optional<UpstreamEntry> ParseUpstream(const std::string& raw, std::string& reason){
std::string s=raw, prefix="/";
auto eq=s.rfind('=');
// '=' only a selector if what follows starts with '/'
if(eq!=std::string::npos&&eq+1<s.size()&&s[eq+1]=='/'){prefix=s.substr(eq+1);s=s.substr(0,eq);}
UpstreamEntry e; e.raw=raw; e.prefix=prefix.empty()?"/":prefix;
if(IsDigits(s)){e.scheme=UpstreamScheme::kHttp;e.host="127.0.0.1";e.port=std::stoi(s);return e;}
std::string scheme="http", rest=s;
auto p=s.find("://");
if(p!=std::string::npos){scheme=s.substr(0,p);rest=s.substr(p+3);}
if(scheme!="tcp"&&scheme!="http"&&scheme!="https"&&scheme!="ws"&&scheme!="wss"){reason="bad scheme";return std::nullopt;}
e.scheme=SchemeOf(scheme);
std::string hostport=rest, upath;
auto sl=rest.find('/');
if(sl!=std::string::npos){hostport=rest.substr(0,sl);upath=rest.substr(sl);}
e.path=upath;
auto colon=hostport.rfind(':');
if(colon!=std::string::npos){e.host=hostport.substr(0,colon);
try{e.port=std::stoi(hostport.substr(colon+1));}catch(...){reason="bad port";return std::nullopt;}
} else {e.host=hostport.empty()?"127.0.0.1":hostport;e.port=DefaultPort(e.scheme);}
if(e.host.empty())e.host="127.0.0.1";
return e;
}
bool UpstreamTable::Add(UpstreamEntry e, std::string& reason){
for(auto&x:entries_) if(x.prefix==e.prefix){reason="duplicate prefix";return false;}
entries_.push_back(std::move(e)); return true;
}
const UpstreamEntry* UpstreamTable::Route(const std::string& path) const{
const UpstreamEntry* best=nullptr;
for(auto&e:entries_){ if(!SegBoundary(path,e.prefix))continue;
if(!best||e.prefix.size()>best->prefix.size())best=&e; }
return best;
}
bool UpstreamHostAllowed(const std::string& host, bool allow_public){
if(allow_public)return true;
std::string h=host;
if(h=="localhost"||h=="127.0.0.1"||h=="::1"||h=="::ffff:127.0.0.1")return true;
if(h.rfind("127.",0)==0)return true;
if(h.rfind("10.",0)==0)return true;
if(h.rfind("192.168.",0)==0)return true;
if(h.rfind("172.",0)==0){
int n=0; size_t d=h.find('.',4);
try{n=std::stoi(h.substr(4,d==std::string::npos?std::string::npos:d-4));}catch(...){return false;}
if(n>=16&&n<=31)return true;
}
if(h.rfind("169.254.",0)==0||h.rfind("fe80",0)==0||h.rfind("FE80",0)==0)return true;
return false;
}
} // namespace molo::realtime::expose
+38
View File
@@ -0,0 +1,38 @@
#pragma once
// Spec: reverse.md sections 3,5. Upstream entry + longest-prefix routing.
#include <optional>
#include <string>
#include <vector>
namespace molo::realtime::expose {
enum class UpstreamScheme { kTcp, kHttp, kHttps, kWs, kWss };
struct UpstreamEntry {
std::string raw;
UpstreamScheme scheme = UpstreamScheme::kHttp;
std::string host;
int port = 80;
std::string path; // origin path
std::string prefix; // claim-subpath selector, "/" = default
};
// Parses "[scheme://]host[:port][/path][=prefix]". Bare number = 127.0.0.1:port.
std::optional<UpstreamEntry> ParseUpstream(const std::string& raw, std::string& reason);
// SSRF guard (reverse.md section 7): upstream host must be loopback /
// RFC1918 / link-local unless allow_public is set. Applies per entry.
bool UpstreamHostAllowed(const std::string& host, bool allow_public);
// OO routing table: longest prefix wins, ties rejected at insert.
class UpstreamTable {
public:
bool Add(UpstreamEntry e, std::string& reason);
const UpstreamEntry* Route(const std::string& path) const;
const std::vector<UpstreamEntry>& entries() const { return entries_; }
std::size_t size() const { return entries_.size(); }
private:
std::vector<UpstreamEntry> entries_;
};
} // namespace molo::realtime::expose
+11 -4
View File
@@ -54,7 +54,13 @@ REALTIME_SRCS := \
src/realtime/webdav/server/lock_manager.cpp \
src/realtime/webdav/server/quota_guard.cpp \
src/realtime/webdav/server/dav_view.cpp \
src/realtime/webdav/client/webdav_client.cpp
src/realtime/webdav/client/webdav_client.cpp \
src/realtime/expose/claim.cpp \
src/realtime/expose/upstream.cpp \
src/realtime/expose/tunnel.cpp \
src/realtime/expose/hello.cpp \
src/realtime/expose/proxy.cpp \
src/realtime/expose/stats.cpp
REALTIME_OBJS := $(patsubst src/%.cpp,build/obj/%.o,$(REALTIME_SRCS))
REALTIME_LIB := build/lib/librealtime.a
@@ -71,7 +77,8 @@ REALTIME_TESTS := \
build/test/realtime/socketio/chat_test \
build/test/realtime/dav/files_test \
build/test/realtime/shared/foundation_test \
build/test/realtime/shared/robustness_test
build/test/realtime/shared/robustness_test \
build/test/realtime/expose/expose_test
$(REALTIME_LIB): $(REALTIME_OBJS)
mkdir -p $(dir $@)
@@ -83,8 +90,8 @@ build/obj/realtime/%.o: src/realtime/%.cpp
build/test/realtime/%_test: tests/src/realtime/%_test.cpp $(REALTIME_LIB)
mkdir -p $(dir $@)
$(CXX) $(CXXFLAGS) -Isrc -Isrc/realtime -Itests/src/realtime $< $(REALTIME_LIB) -o $@
$(CXX) $(CXXFLAGS) -Isrc -Isrc/realtime -Itests/src/realtime $< $(REALTIME_LIB) -lsqlite3 -l:libpq.so.5 -o $@
build/test/realtime/%: tests/src/realtime/%.cpp $(REALTIME_LIB)
mkdir -p $(dir $@)
$(CXX) $(CXXFLAGS) -Isrc -Isrc/realtime -Itests/src/realtime $< $(REALTIME_LIB) -o $@
$(CXX) $(CXXFLAGS) -Isrc -Isrc/realtime -Itests/src/realtime $< $(REALTIME_LIB) -lsqlite3 -l:libpq.so.5 -o $@
+98
View File
@@ -0,0 +1,98 @@
#include <cassert>
#include <cstdio>
#include <thread>
#include <vector>
#include "realtime/expose/claim.h"
#include "realtime/expose/hello.h"
#include "realtime/expose/tunnel.h"
#include "realtime/expose/proxy.h"
#include "realtime/expose/stats.h"
#include <atomic>
#include "realtime/expose/upstream.h"
using namespace molo::realtime::expose;
int main() {
std::string r;
// claim grammar
assert(ParseClaim("/hook1", r) && ParseClaim("example.com/hook1", r));
assert(ParseClaim("example.com", r)->whole_host);
assert(!ParseClaim("/../x", r));
// upstream table: longest prefix wins, concurrent entries
UpstreamTable t;
UpstreamEntry a, b;
assert(ParseUpstream("http://127.0.0.1:3001=/api", r));
auto e1 = ParseUpstream("http://127.0.0.1:3001=/api", r);
auto e2 = ParseUpstream("http://127.0.0.1:3002=/", r);
assert(e1 && e2 && t.Add(*e1, r) && t.Add(*e2, r));
assert(t.Route("/api/u")->port == 3001 && t.Route("/other")->port == 3002);
// hello v1+v2
std::string e;
auto h2 = ParseHello(R"({"v":2,"claim":"/a","upstreams":[{"upstream":"http://127.0.0.1:1","prefix":"/"}],"agent":"t"})", "", e);
auto h1 = ParseHello(R"({"v":1,"claim":"/a","upstream":"8080","agent":"t"})", "", e);
assert(h2 && h1 && h1->upstreams.size() == 1);
// registry: auth, conflict, takeover, concurrent register
ClaimRegistry reg("s1");
auto c = ParseClaim("/app", r);
UpstreamTable u0;
std::shared_ptr<Tunnel> out; std::string d;
assert(reg.Register(*c, Sha256Hex("k1"), std::move(u0), false, out, d) == RegisterResult::kOk);
UpstreamTable u1;
std::shared_ptr<Tunnel> o2;
assert(reg.Register(*c, Sha256Hex("k1"), std::move(u1), false, o2, d) == RegisterResult::kConflict);
UpstreamTable u2;
std::shared_ptr<Tunnel> o3;
assert(reg.Register(*c, Sha256Hex("k1"), std::move(u2), true, o3, d) == RegisterResult::kOk);
assert(reg.Find("", "/app/x") && reg.Find("", "/app/x")->id == o3->id);
// concurrent Find/List under threads
std::vector<std::thread> th;
for (int i = 0; i < 8; ++i) th.emplace_back([&]{ for (int j = 0; j < 200; ++j) { reg.Find("", "/app"); reg.List(); } });
for (auto& t : th) t.join();
// proxy glue: strip prefix, hop-by-hop, stream ids, per-IP cap
assert(StripClaimPrefix("/app", false, "/app/x") == "/x");
assert(StripClaimPrefix("/app", false, "/app") == "/");
assert(StripClaimPrefix("/", false, "/a") == "/a");
assert(StripClaimPrefix("/app", true, "/app/x") == "/app/x");
assert(IsHopByHop("Connection") && IsHopByHop("Transfer-Encoding") && !IsHopByHop("X-Custom"));
assert(ForwardedFor("", "1.2.3.4") == "1.2.3.4");
assert(ForwardedFor("a", "b") == "a, b");
assert(kH2RefusedStream == 0x7 && kH2Cancel == 0x8);
std::atomic<uint32_t> nx{3};
assert(NextProxyStreamId(nx) == 3 && NextProxyStreamId(nx) == 5);
PerIpCounter ipc; for (int i = 0; i < 128; ++i) assert(ipc.Admit("9.9.9.9"));
assert(!ipc.Admit("9.9.9.9")); ipc.Release("9.9.9.9"); assert(ipc.Admit("9.9.9.9"));
// concurrent streams across N upstreams: route from threads
{
UpstreamTable ct;
std::string rr;
ct.Add(*ParseUpstream("http://127.0.0.1:3001=/api", rr), rr);
ct.Add(*ParseUpstream("http://127.0.0.1:3002=/static", rr), rr);
ct.Add(*ParseUpstream("http://127.0.0.1:3003=/", rr), rr);
std::atomic<int> ok{0};
std::vector<std::thread> w;
for (int i = 0; i < 16; ++i) w.emplace_back([&, i]{
const char* p[3] = {"/api/x", "/static/y", "/other"};
auto* e = ct.Route(p[i % 3]);
if (e) ok.fetch_add(1);
std::atomic<uint32_t> sn{3u + (uint32_t)(i * 2)};
(void)NextProxyStreamId(sn);
});
for (auto& th2 : w) th2.join();
assert(ok == 16);
}
// stats: Note/Flush/Query sqlite round-trip (reverse.md 9)
{
std::string db = "/tmp/molo_stats_test.db";
std::remove(db.c_str());
StatsCollector sc;
sc.Configure(MakeStatsStore("sqlite://" + db));
sc.Note("GET", 2048, 4096, 2000);
std::string err, out;
assert(sc.Flush(err));
assert(sc.Query(out, false, err));
assert(out.find("GET") != std::string::npos && out.find("count=1") != std::string::npos);
assert(sc.Query(out, true, err) && out.front() == '[');
std::remove(db.c_str());
}
printf("expose_test PASS\n");
return 0;
}