# stealthii

`retoor <retoor@molodetz.nl>`

An `aiohttp`/`httpx`-shaped async HTTP client backed by a single, persistent,
stealth-patched Chromium instance. Every request goes out through the real
browser's own network stack, so it carries a genuine Chrome TLS/HTTP2/UA
fingerprint instead of Python's OpenSSL-based one — the signal most
anti-bot systems key on first, before they ever look at a header.

## Why

`requests`, `httpx`, and `aiohttp` all negotiate TLS through Python's `ssl`
module (OpenSSL). Real browsers negotiate through their own TLS stack
(Chrome: BoringSSL). The resulting ClientHello — cipher suites, extensions,
curves, and their order — is a fingerprint (JA3/JA4) that anti-bot systems
read directly off the handshake, independent of whatever `User-Agent`
header the client claims. A Python client cannot fake this from the
application layer; the only way to get a real Chrome fingerprint is to
make the request from an actual Chrome process.

Playwright's `BrowserContext.request` API does exactly that: it issues
plain HTTP requests through the same browser process backing a real page,
without needing a tab or JS execution — as fast as a normal HTTP call, but
with the browser's authentic identity. `stealthii` wraps that fast path
(`get`/`post`/`put`/`patch`/`delete`/`head`/`download`) behind a familiar
API, and adds full page rendering (`render`/`screenshot`/`page`) as a
heavier fallback for targets that require solving a JS challenge.

On top of that, `stealthii` removes the JS-level automation tells a stock
headless Chromium exposes (`navigator.webdriver`, missing `window.chrome`,
an empty plugin list, a SwiftShader WebGL renderer, a permissions-query
mismatch) — verified clean against `bot.sannysoft.com`'s full check suite.

## What it does not do

TLS/HTTP2 impersonation defeats fingerprint-based detection. It does not
defeat IP-reputation-based rate limiting, nor the behavioural layer of a
system like DataDome or a Cloudflare managed challenge — those need a
clean IP and, for the hardest targets, genuine human-like interaction.
Layer `stealthii` with residential proxies and session warm-up
(`session=`, see below) for those targets; don't expect it alone to be a
universal bypass.

## Install

```bash
pip install -e .
playwright install chromium
```

## Usage

```python
from stealthii import Stealthii

stealth = Stealthii()          # cheap, lazy — launches nothing yet
resp = await stealth.get("https://example.com/api", params={"q": "x"})
resp.status
await resp.json()
await resp.text()

await stealth.shutdown()       # call once at host process shutdown
```

`get`/`post`/`put`/`patch`/`delete`/`head` accept the same shapes as
`aiohttp`/`httpx`: `params`, `headers`, `data` (dict → form-urlencoded,
bytes/str → raw body), `json`, `files` (multipart), `cookies` (per-call,
layered on the context's own jar), `auth` as an `(user, password)` Basic
tuple, `allow_redirects` / `follow_redirects` (both spellings accepted),
`max_redirects`, and `timeout`.

```python
data = await stealth.download("https://example.com/photo.png")  # raw bytes

png_b64 = await stealth.screenshot("https://example.com")        # base64 PNG
await stealth.screenshot("https://example.com", path="shot.png")  # saved to disk

rendered = await stealth.render("https://example.com", collect_links=True)
rendered["text"]; rendered["links"]; rendered["status"]
```

### Named sessions (cookie persistence / warm-up)

Every call is anonymous by default (a shared, periodically recycled
context). Pass `session="name"` to use a dedicated, reused context instead
— cookies set by one call under that name are seen by the next:

```python
await stealth.render("https://example.com/", session="acme")       # picks up cookies
resp = await stealth.get("https://example.com/api", session="acme") # reuses them
```

### Raw page access

For anything the high-level API doesn't cover, `page()` yields a real
Playwright `Page` (already stealth-patched), guaranteed to close even on
exception:

```python
async with stealth.page() as pg:
    await pg.goto("https://example.com")
    await pg.click("#accept-cookies")
```

## Configuration

All configuration is passed once, at construction — `Stealthii(...)` does
no I/O itself; the browser launches lazily on first real use:

```python
stealth = Stealthii(
    max_tabs=4,                            # concurrent render()/page() tabs
    fingerprint="ja4",                     # "ja3" or "ja4" — see fingerprint_info()
    request_timeout=15.0,
    max_retries=3,
    disk_cache_bytes=1,                    # keep Chromium's own on-disk cache tiny
    recycle_context_after=(500, 1800.0),   # (max requests, max seconds) per context
    recycle_browser_after_seconds=6*3600,  # full relaunch cadence, or None to disable
)
```

`fingerprint` selects which hash `fingerprint_info()` reports (JA3 or
JA4) — it does not change what's sent on the wire; both describe the same
genuine Chromium handshake.

### Why the recycle settings exist

A browser that runs for the process's entire lifetime otherwise
accumulates cookies and an on-disk HTTP cache without bound. `stealthii`
closes and recreates any context — anonymous or named — once it crosses
`recycle_context_after`, and fully relaunches the browser (discarding its
whole profile directory) every `recycle_browser_after_seconds`. Named
sessions meant to stay warm longer should get a larger
`recycle_context_after` at construction time.

## Testing

```bash
make test-unit          # no network required
make test-integration   # hits real sites; requires Chromium installed
```