Fail fast with a clear error on event loop reuse instead of hanging
A Stealthii instance binds its lock, browser, and contexts to whichever asyncio event loop is running the first time it is used. Using it again from a different loop previously hung forever (the lock/semaphore/ Playwright transport are all bound to the dead first loop and never wake the waiting coroutine). Detected in rsearch's test suite, where pytest-asyncio's default per-test-function event loop broke the module-level Stealthii singleton shared across tests. Now raises StealthUnavailableError immediately with an actionable message. README documents the constraint and the pytest-asyncio config fix (asyncio_default_test_loop_scope = session). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0116i7dYLNZTXNR7Lqf439bV
This commit is contained in:
+11
-11
@@ -7,7 +7,7 @@ fast path (`get`/`post`/`put`/`patch`/`delete`/`head`/`download`) against
|
||||
a plain `aiohttp.ClientSession`, and the fast path's behaviour under
|
||||
concurrency. All numbers below were measured against public, internet-hosted
|
||||
endpoints (`httpbin.org`), so absolute times include normal internet
|
||||
latency and third-party server jitter — the comparison between the two
|
||||
latency and third-party server jitter - the comparison between the two
|
||||
clients on the same run is the meaningful signal, not the absolute
|
||||
seconds.
|
||||
|
||||
@@ -15,13 +15,13 @@ seconds.
|
||||
|
||||
| | One-time | Per request (steady state, n=10) |
|
||||
|---|---|---|
|
||||
| Browser launch (`start()`) | ~0.96s | — |
|
||||
| First request (context warm-up) | ~0.47s | — |
|
||||
| stealthii `get()` | — | avg 0.184s (min 0.101s, max 0.791s) |
|
||||
| aiohttp `get()` | — | avg 0.168s (min 0.102s, max 0.441s) |
|
||||
| Browser launch (`start()`) | ~0.96s | - |
|
||||
| First request (context warm-up) | ~0.47s | - |
|
||||
| stealthii `get()` | - | avg 0.184s (min 0.101s, max 0.791s) |
|
||||
| aiohttp `get()` | - | avg 0.168s (min 0.102s, max 0.441s) |
|
||||
|
||||
Browser launch and the shared context's warm-up are each paid exactly
|
||||
once per process — in a host application, at `start()` during startup,
|
||||
once per process - in a host application, at `start()` during startup,
|
||||
not per request. Steady-state, the fast path costs roughly 15-20ms more
|
||||
per request than `aiohttp` on average, which is within normal network
|
||||
jitter for a single request (the observed max-min spread on both clients
|
||||
@@ -40,7 +40,7 @@ rather than approaching `1s`:
|
||||
| 20× | 2.033s, 20/20 succeeded | 1.867s, 20/20 succeeded |
|
||||
|
||||
Both clients complete 20 concurrent 1-second-delay requests in
|
||||
~2 seconds total, not 20 seconds — confirming the fast path's requests
|
||||
~2 seconds total, not 20 seconds - confirming the fast path's requests
|
||||
against the one shared `BrowserContext` are not serialized through some
|
||||
hidden single-flight point (the underlying CDP connection, the Python
|
||||
process, or the browser's own request handling). The 5× round-to-round
|
||||
@@ -56,12 +56,12 @@ requests), the fast path's overhead relative to `aiohttp` is negligible
|
||||
next to ordinary network latency, and it does not degrade under
|
||||
concurrency up to at least 20 simultaneous requests on one shared
|
||||
context. The one meaningfully different cost is the one-time browser
|
||||
launch (~1s) — pay it once at application startup (`await
|
||||
launch (~1s) - pay it once at application startup (`await
|
||||
stealth.start()` in an `on_startup` hook, or accept it landing on
|
||||
whichever request happens to be first) rather than per request.
|
||||
|
||||
`render()`/`screenshot()`/`page()` (full page navigation) were not
|
||||
benchmarked here — they are inherently much heavier than the fast path
|
||||
benchmarked here - they are inherently much heavier than the fast path
|
||||
(loading a real document, executing its JS, subject to `max_tabs`), by
|
||||
design and by necessity for the targets that require them. Reach for the
|
||||
fast path by default; escalate to page rendering only for targets that
|
||||
@@ -70,7 +70,7 @@ actually need JS execution.
|
||||
## Methodology notes
|
||||
|
||||
- Measured with Python's `time.monotonic()` around `asyncio.gather()`
|
||||
batches, one process, one machine, one network path — not a controlled
|
||||
batches, one process, one machine, one network path - not a controlled
|
||||
lab benchmark. Re-running these will not reproduce the exact numbers,
|
||||
only the same qualitative shape (comparable per-request cost, no
|
||||
concurrency bottleneck).
|
||||
@@ -79,6 +79,6 @@ actually need JS execution.
|
||||
earlier attempt at this same measurement against
|
||||
`html.duckduckgo.com/html/` was discarded after DuckDuckGo started
|
||||
returning `202` instead of `200` after a handful of identical rapid
|
||||
requests — a server-side response to request pattern, not a
|
||||
requests - a server-side response to request pattern, not a
|
||||
client-timing artifact, but one that would have corrupted a latency
|
||||
comparison.
|
||||
|
||||
Reference in New Issue
Block a user