retoor/tai
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document the real backend candidate pool, the honest status check, and the box fallback

Browse Source 
The Backends section was two lines about the primary HTTP gateway and
didn't mention the five CLI-wrapped backends, the box last-resort
fallback, the infinite-retry design, or the codex/claude/gemini/muse/grok
status distinction from the previous commit. Document all of it with the
actual measured facts (pollinations' queue cap, zen's current 403, the
Node-tarball-vs-apt size difference, opencode's zero-credential free
tier) rather than a restated feature list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
retoor
2026-10-10 06:06:10 +02:00
co-authored by Claude Sonnet 5
parent 622e2fecf8
commit c46000d7e5
1 changed files with 31 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+31 -1
View File
@@ -83,7 +83,37 @@ Six install targets exist side by side: `binary` (`~/.local/bin/tai.py`), `bash-
## Backends
The primary backend is `model.cloud.pravda.education`, an OpenAI-compatible gateway that needs no API key and selects a free model per request. If a request fails, tai retries it on `devplace.net/openai/v1`, which requires `DEVPLACE_API_KEY`. Both endpoints speak `/chat/completions`, including native tool calls and streaming.
tai never picks one backend and hopes. `ChatClient.complete()` builds an ordered list of every backend it can find any evidence of (`candidates()`), tries each one in a round, and on total failure sleeps on a capped backoff (`RETRY_BACKOFF = (2, 5, 10, 20, 30, 60)` seconds) and tries the *entire* list again — forever, by default. `retry_seconds` (env `TAI_RETRY_SECONDS`) defaults to `0`, which means no give-up deadline at all: a one-shot `tai.py "prompt"` with every backend down will sit there retrying every backend, every round, until one works or you kill it. This is deliberate, not an oversight: next model, next provider, next everything, until the bitter end.
**What's actually in the candidate pool**, roughly in the order a fresh round tries them:
- `openrouter` — `OPENROUTER_BASE`, `:free`-suffixed models, works with or without `OPENROUTER_API_KEY` for most models but **some free-tier models require one**; a missing key surfaces as `401: No cookie auth credentials found`, not a quieter failure.
- `pollinations` — `POLLINATIONS_BASE`, no key needed, rate-limited per IP (`429: Queue full for IP: ...: 1 requests already queued`) — on a shared/NAT'd network this can mean genuinely never succeeding, not flakiness, since the queue slot stays permanently occupied by other traffic.
- `ollama` — `localhost:11434`, only if something's actually listening there (3-second probe timeout, so a missing local Ollama costs almost nothing).
- `devplace` — `DEVPLACE_API_KEY` against `devplace.net/openai/v1`, opt-in.
- `zen` — `opencode.ai/zen/v1`, opt-in via `TAI_ZEN=1`, spoofs the real `opencode` CLI's client headers to reach its free tier without the CLI installed. **Currently dead**: the server now returns `403 FreeTierError: "OpenCode's free tier can only be used from within OpenCode"` for every request, confirmed by direct test — it checks for more than headers now. Left in place (off by default) rather than ripped out, in case that check ever loosens again.
- `muse`, `grok`, `claude`, `codex`, `gemini` — each wraps a locally installed CLI tool (`shutil.which()` or an env override: `MUSE_BIN`, `GROK_BIN`, `CLAUDE_BIN`, `CODEX_BIN`, `GEMINI_BIN`) and shells out to it per turn rather than speaking HTTP. `grok`/`claude`/`codex`/`gemini` are only added to the pool if their binary is actually found; `muse` is tai's own default model and is always added regardless, so a machine without the `muse` CLI at all will see `muse/muse: muse CLI not found` fail every single round — harmless (the retry loop just moves on), but not a sign anything is broken.
- `opencode` — same idea, but if there's no local `opencode` CLI *and* the sandbox box (see "Sandbox" below) is already running, this is the one candidate that doesn't just get skipped: see "Last-resort box backend" below.
### What the startup banner's status column actually means
Every backend prints a one-line health status at startup and in `/models`. These are not all the same kind of claim, and conflating them was a real bug (fixed in this codebase, not theoretical): `codex_health()`/`gemini_health()`/`claude_health()`/`grok_health()`/`muse_health()` used to run nothing but `<cli> --version` and print `"ok, subscription"` on any zero exit — which proves the binary runs, and proves nothing about whether it's logged in. A `codex` binary that was installed once and never authenticated reported `ok, subscription` right alongside the ones that actually worked.
Now the status line only claims what it actually checked:
| Backend | What's checked | What `ok` means |
|---|---|---|
| `claude` | `claude auth status` (real JSON, `loggedIn` field) | genuinely logged in |
| `codex` | `codex login status` (real exit code) | genuinely logged in; a missing login shows as `not logged in: Not logged in`, not `ok` |
| `grok`, `gemini`, `muse` | `--version` only — no CLI exposes a cheaper, read-only login check (checked `gemini --help` / `muse --help`, no `status`/`whoami`-style command exists for either) | `cli found (<version>), login unverified` — explicitly *not* a subscription claim, because this check cannot back one up |
A CLI being installed and a CLI being logged in are two separate facts. `codex`'s own `login status` is the only thing that can tell them apart; `--version` can't, and the status line doesn't pretend otherwise anymore.
### Last-resort box backend: installing `opencode` where it's missing
If there's no local `opencode` CLI at all, `ChatClient.box_opencode_candidates()` checks whether the sandbox box (see "Sandbox" below) is already running, and if so, installs the real `opencode` CLI *inside that box* — official Node.js tarball from nodejs.org, not `apt`'s `nodejs`/`npm` packages, which pull in `webpack`/`babel`/`jest` as transitive dependencies for no reason relevant here (measured: 1.49GB vs. 971MB for the identical end result) — then routes completions through it via `box_exec()`. The real `opencode` CLI needs no API key or login at all for its own `opencode/*` free-tier models (confirmed: `opencode auth list` reports zero stored credentials even on a machine where it works), unlike spoofing its HTTP headers (see `zen` above, which the server now blocks).
This only ever activates as a genuine last resort: it's wired into `all_candidates()`, which `complete()` only reaches once the first, fast round (`ordered_candidates()`) has already failed everything — so a normal turn that succeeds through a real backend never touches the box, never pays an install cost, and never even calls `container_engine()`. It never creates the box itself; that stays the explicit, approval-gated action it already was (`ensure_box()`, via `tai.py --env sandbox` or the `install` tool). A successful reply through this path is labeled `opencode (box)`, not plain `opencode`, so it's visible exactly when this path fired. The install, when it happens, streams real `apt`/`curl`/`npm` output live rather than running silently.
---