From 23cdad896606f090a9df48b1e5df4873a32b2ebe Mon Sep 17 00:00:00 2001 From: retoor Date: Wed, 9 Sep 2026 14:27:44 +0200 Subject: [PATCH] Add PERFORMANCE.md and link it plus STEALTH_PATCHES.md from README Documents measured single-request and concurrency (5x/20x) overhead of the context.request fast path against aiohttp: ~15-20ms/request steady state, no serialization bottleneck through the shared context up to at least 20 concurrent requests, browser launch (~1s) paid once at startup. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_0116i7dYLNZTXNR7Lqf439bV --- PERFORMANCE.md | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 10 +++++- 2 files changed, 93 insertions(+), 1 deletion(-) create mode 100644 PERFORMANCE.md diff --git a/PERFORMANCE.md b/PERFORMANCE.md new file mode 100644 index 0000000..a84afa6 --- /dev/null +++ b/PERFORMANCE.md @@ -0,0 +1,84 @@ +# Performance + +`retoor ` + +This document records measured overhead of the `context.request`-backed +fast path (`get`/`post`/`put`/`patch`/`delete`/`head`/`download`) against +a plain `aiohttp.ClientSession`, and the fast path's behaviour under +concurrency. All numbers below were measured against public, internet-hosted +endpoints (`httpbin.org`), so absolute times include normal internet +latency and third-party server jitter — the comparison between the two +clients on the same run is the meaningful signal, not the absolute +seconds. + +## Single-request latency + +| | One-time | Per request (steady state, n=10) | +|---|---|---| +| Browser launch (`start()`) | ~0.96s | — | +| First request (context warm-up) | ~0.47s | — | +| stealthii `get()` | — | avg 0.184s (min 0.101s, max 0.791s) | +| aiohttp `get()` | — | avg 0.168s (min 0.102s, max 0.441s) | + +Browser launch and the shared context's warm-up are each paid exactly +once per process — in a host application, at `start()` during startup, +not per request. Steady-state, the fast path costs roughly 15-20ms more +per request than `aiohttp` on average, which is within normal network +jitter for a single request (the observed max-min spread on both clients +individually is far larger than the 15-20ms difference between them). + +## Concurrency + +Five and twenty concurrent requests against an endpoint with a fixed +1-second server-side delay (`httpbin.org/delay/1`), so any serialization +in either client shows up directly as total time approaching `N × 1s` +rather than approaching `1s`: + +| Concurrency | stealthii | aiohttp | +|---|---|---| +| 5× | 1.449s – 2.311s across 3 rounds | 1.907s – 3.152s across 3 rounds | +| 20× | 2.033s, 20/20 succeeded | 1.867s, 20/20 succeeded | + +Both clients complete 20 concurrent 1-second-delay requests in +~2 seconds total, not 20 seconds — confirming the fast path's requests +against the one shared `BrowserContext` are not serialized through some +hidden single-flight point (the underlying CDP connection, the Python +process, or the browser's own request handling). The 5× round-to-round +spread on both clients is larger than the difference between the two +clients, again pointing at server/network variance rather than a +client-side bottleneck. + +## Practical takeaway + +For the request volumes and concurrency levels a scraping/search +aggregator workload produces (single digits to low tens of concurrent +requests), the fast path's overhead relative to `aiohttp` is negligible +next to ordinary network latency, and it does not degrade under +concurrency up to at least 20 simultaneous requests on one shared +context. The one meaningfully different cost is the one-time browser +launch (~1s) — pay it once at application startup (`await +stealth.start()` in an `on_startup` hook, or accept it landing on +whichever request happens to be first) rather than per request. + +`render()`/`screenshot()`/`page()` (full page navigation) were not +benchmarked here — they are inherently much heavier than the fast path +(loading a real document, executing its JS, subject to `max_tabs`), by +design and by necessity for the targets that require them. Reach for the +fast path by default; escalate to page rendering only for targets that +actually need JS execution. + +## Methodology notes + +- Measured with Python's `time.monotonic()` around `asyncio.gather()` + batches, one process, one machine, one network path — not a controlled + lab benchmark. Re-running these will not reproduce the exact numbers, + only the same qualitative shape (comparable per-request cost, no + concurrency bottleneck). +- `httpbin.org` is a shared public service; individual requests can be + slow or occasionally rate-limited independent of either client. An + earlier attempt at this same measurement against + `html.duckduckgo.com/html/` was discarded after DuckDuckGo started + returning `202` instead of `200` after a handful of identical rapid + requests — a server-side response to request pattern, not a + client-timing artifact, but one that would have corrupted a latency + comparison. diff --git a/README.md b/README.md index 40cad3e..04f78ba 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,12 @@ persistent, stealth-patched Chromium instance. Every request goes out through the real browser's own network stack, so it carries a genuine Chrome TLS/HTTP2/UA fingerprint instead of Python's OpenSSL-based one. +This README is the complete documentation. Two companion documents go +deeper on specific topics: [`STEALTH_PATCHES.md`](./STEALTH_PATCHES.md) +(every anti-automation patch applied, and why) and +[`PERFORMANCE.md`](./PERFORMANCE.md) (measured latency and concurrency +overhead against `aiohttp`). + ## Why this exists `requests`, `httpx`, and `aiohttp` all negotiate TLS through Python's @@ -33,7 +39,9 @@ endpoint: Distinct hashes, produced by the same physical request path our `get`/`post`/etc. use — proof the fast path is not just claiming to be -Chrome, it is Chrome's own network stack. +Chrome, it is Chrome's own network stack. Measured latency and +concurrency overhead of that same fast path against `aiohttp` — steady +state, and 5×/20× concurrent — are in [`PERFORMANCE.md`](./PERFORMANCE.md). On top of that, `stealthii` removes the JS-level automation tells a stock headless Chromium exposes — `navigator.webdriver`, a missing