Files

T

retoorandClaude Sonnet 5 423170e1bb Fail fast with a clear error on event loop reuse instead of hanging

A Stealthii instance binds its lock, browser, and contexts to whichever
asyncio event loop is running the first time it is used. Using it again
from a different loop previously hung forever (the lock/semaphore/
Playwright transport are all bound to the dead first loop and never
wake the waiting coroutine). Detected in rsearch's test suite, where
pytest-asyncio's default per-test-function event loop broke the
module-level Stealthii singleton shared across tests.

Now raises StealthUnavailableError immediately with an actionable
message. README documents the constraint and the pytest-asyncio config
fix (asyncio_default_test_loop_scope = session).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0116i7dYLNZTXNR7Lqf439bV

2026-09-10 01:58:55 +02:00

4.0 KiB

Raw Blame History

Performance

retoor <retoor@molodetz.nl>

This document records measured overhead of the context.request-backed fast path (get/post/put/patch/delete/head/download) against a plain aiohttp.ClientSession, and the fast path's behaviour under concurrency. All numbers below were measured against public, internet-hosted endpoints (httpbin.org), so absolute times include normal internet latency and third-party server jitter - the comparison between the two clients on the same run is the meaningful signal, not the absolute seconds.

Single-request latency

	One-time	Per request (steady state, n=10)
Browser launch (`start()`)	~0.96s	-
First request (context warm-up)	~0.47s	-
stealthii `get()`	-	avg 0.184s (min 0.101s, max 0.791s)
aiohttp `get()`	-	avg 0.168s (min 0.102s, max 0.441s)

Browser launch and the shared context's warm-up are each paid exactly once per process - in a host application, at start() during startup, not per request. Steady-state, the fast path costs roughly 15-20ms more per request than aiohttp on average, which is within normal network jitter for a single request (the observed max-min spread on both clients individually is far larger than the 15-20ms difference between them).

Concurrency

Five and twenty concurrent requests against an endpoint with a fixed 1-second server-side delay (httpbin.org/delay/1), so any serialization in either client shows up directly as total time approaching N × 1s rather than approaching 1s:

Concurrency	stealthii	aiohttp
5×	1.449s – 2.311s across 3 rounds	1.907s – 3.152s across 3 rounds
20×	2.033s, 20/20 succeeded	1.867s, 20/20 succeeded

Both clients complete 20 concurrent 1-second-delay requests in ~2 seconds total, not 20 seconds - confirming the fast path's requests against the one shared BrowserContext are not serialized through some hidden single-flight point (the underlying CDP connection, the Python process, or the browser's own request handling). The 5× round-to-round spread on both clients is larger than the difference between the two clients, again pointing at server/network variance rather than a client-side bottleneck.

Practical takeaway

For the request volumes and concurrency levels a scraping/search aggregator workload produces (single digits to low tens of concurrent requests), the fast path's overhead relative to aiohttp is negligible next to ordinary network latency, and it does not degrade under concurrency up to at least 20 simultaneous requests on one shared context. The one meaningfully different cost is the one-time browser launch (~1s) - pay it once at application startup (await stealth.start() in an on_startup hook, or accept it landing on whichever request happens to be first) rather than per request.

render()/screenshot()/page() (full page navigation) were not benchmarked here - they are inherently much heavier than the fast path (loading a real document, executing its JS, subject to max_tabs), by design and by necessity for the targets that require them. Reach for the fast path by default; escalate to page rendering only for targets that actually need JS execution.

Methodology notes

Measured with Python's time.monotonic() around asyncio.gather() batches, one process, one machine, one network path - not a controlled lab benchmark. Re-running these will not reproduce the exact numbers, only the same qualitative shape (comparable per-request cost, no concurrency bottleneck).
httpbin.org is a shared public service; individual requests can be slow or occasionally rate-limited independent of either client. An earlier attempt at this same measurement against html.duckduckgo.com/html/ was discarded after DuckDuckGo started returning 202 instead of 200 after a handful of identical rapid requests - a server-side response to request pattern, not a client-timing artifact, but one that would have corrupted a latency comparison.