diff --git a/README.md b/README.md index 4962825..dedb375 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,37 @@ Six install targets exist side by side: `binary` (`~/.local/bin/tai.py`), `bash- ## Backends -The primary backend is `model.cloud.pravda.education`, an OpenAI-compatible gateway that needs no API key and selects a free model per request. If a request fails, tai retries it on `devplace.net/openai/v1`, which requires `DEVPLACE_API_KEY`. Both endpoints speak `/chat/completions`, including native tool calls and streaming. +tai never picks one backend and hopes. `ChatClient.complete()` builds an ordered list of every backend it can find any evidence of (`candidates()`), tries each one in a round, and on total failure sleeps on a capped backoff (`RETRY_BACKOFF = (2, 5, 10, 20, 30, 60)` seconds) and tries the *entire* list again — forever, by default. `retry_seconds` (env `TAI_RETRY_SECONDS`) defaults to `0`, which means no give-up deadline at all: a one-shot `tai.py "prompt"` with every backend down will sit there retrying every backend, every round, until one works or you kill it. This is deliberate, not an oversight: next model, next provider, next everything, until the bitter end. + +**What's actually in the candidate pool**, roughly in the order a fresh round tries them: + +- `openrouter` — `OPENROUTER_BASE`, `:free`-suffixed models, works with or without `OPENROUTER_API_KEY` for most models but **some free-tier models require one**; a missing key surfaces as `401: No cookie auth credentials found`, not a quieter failure. +- `pollinations` — `POLLINATIONS_BASE`, no key needed, rate-limited per IP (`429: Queue full for IP: ...: 1 requests already queued`) — on a shared/NAT'd network this can mean genuinely never succeeding, not flakiness, since the queue slot stays permanently occupied by other traffic. +- `ollama` — `localhost:11434`, only if something's actually listening there (3-second probe timeout, so a missing local Ollama costs almost nothing). +- `devplace` — `DEVPLACE_API_KEY` against `devplace.net/openai/v1`, opt-in. +- `zen` — `opencode.ai/zen/v1`, opt-in via `TAI_ZEN=1`, spoofs the real `opencode` CLI's client headers to reach its free tier without the CLI installed. **Currently dead**: the server now returns `403 FreeTierError: "OpenCode's free tier can only be used from within OpenCode"` for every request, confirmed by direct test — it checks for more than headers now. Left in place (off by default) rather than ripped out, in case that check ever loosens again. +- `muse`, `grok`, `claude`, `codex`, `gemini` — each wraps a locally installed CLI tool (`shutil.which()` or an env override: `MUSE_BIN`, `GROK_BIN`, `CLAUDE_BIN`, `CODEX_BIN`, `GEMINI_BIN`) and shells out to it per turn rather than speaking HTTP. `grok`/`claude`/`codex`/`gemini` are only added to the pool if their binary is actually found; `muse` is tai's own default model and is always added regardless, so a machine without the `muse` CLI at all will see `muse/muse: muse CLI not found` fail every single round — harmless (the retry loop just moves on), but not a sign anything is broken. +- `opencode` — same idea, but if there's no local `opencode` CLI *and* the sandbox box (see "Sandbox" below) is already running, this is the one candidate that doesn't just get skipped: see "Last-resort box backend" below. + +### What the startup banner's status column actually means + +Every backend prints a one-line health status at startup and in `/models`. These are not all the same kind of claim, and conflating them was a real bug (fixed in this codebase, not theoretical): `codex_health()`/`gemini_health()`/`claude_health()`/`grok_health()`/`muse_health()` used to run nothing but ` --version` and print `"ok, subscription"` on any zero exit — which proves the binary runs, and proves nothing about whether it's logged in. A `codex` binary that was installed once and never authenticated reported `ok, subscription` right alongside the ones that actually worked. + +Now the status line only claims what it actually checked: + +| Backend | What's checked | What `ok` means | +|---|---|---| +| `claude` | `claude auth status` (real JSON, `loggedIn` field) | genuinely logged in | +| `codex` | `codex login status` (real exit code) | genuinely logged in; a missing login shows as `not logged in: Not logged in`, not `ok` | +| `grok`, `gemini`, `muse` | `--version` only — no CLI exposes a cheaper, read-only login check (checked `gemini --help` / `muse --help`, no `status`/`whoami`-style command exists for either) | `cli found (), login unverified` — explicitly *not* a subscription claim, because this check cannot back one up | + +A CLI being installed and a CLI being logged in are two separate facts. `codex`'s own `login status` is the only thing that can tell them apart; `--version` can't, and the status line doesn't pretend otherwise anymore. + +### Last-resort box backend: installing `opencode` where it's missing + +If there's no local `opencode` CLI at all, `ChatClient.box_opencode_candidates()` checks whether the sandbox box (see "Sandbox" below) is already running, and if so, installs the real `opencode` CLI *inside that box* — official Node.js tarball from nodejs.org, not `apt`'s `nodejs`/`npm` packages, which pull in `webpack`/`babel`/`jest` as transitive dependencies for no reason relevant here (measured: 1.49GB vs. 971MB for the identical end result) — then routes completions through it via `box_exec()`. The real `opencode` CLI needs no API key or login at all for its own `opencode/*` free-tier models (confirmed: `opencode auth list` reports zero stored credentials even on a machine where it works), unlike spoofing its HTTP headers (see `zen` above, which the server now blocks). + +This only ever activates as a genuine last resort: it's wired into `all_candidates()`, which `complete()` only reaches once the first, fast round (`ordered_candidates()`) has already failed everything — so a normal turn that succeeds through a real backend never touches the box, never pays an install cost, and never even calls `container_engine()`. It never creates the box itself; that stays the explicit, approval-gated action it already was (`ensure_box()`, via `tai.py --env sandbox` or the `install` tool). A successful reply through this path is labeled `opencode (box)`, not plain `opencode`, so it's visible exactly when this path fired. The install, when it happens, streams real `apt`/`curl`/`npm` output live rather than running silently. ---