Docs audit: fix stale claims, cross-link hub docs

- README: 42 tools (was 38), /demo row, TUI section, testing plus
  integration line, full layout map
- tui.md: research link, tests pointer, soften shot-tested claim
- AGENTS.md/tests/README.md/research hubs: cross-links between docs
- Research: forward-correct tui conclusion plus INDEX activity row
  (19 failures fixed, tai_pkg deleted, 320/320 green); audit PASS
This commit is contained in:
2026-10-10 23:51:06 +02:00
parent 133eb5ef63
commit 871f3e5ae6
7 changed files with 38 additions and 20 deletions
+3 -2
View File
@@ -6,12 +6,13 @@
- Use the `.py` extension for Python source and test files.
- `tai.py` is the entire product in one standalone stdlib-only file: CLI entry plus the stable `import tai` surface, grouped into classes (instance classes for stateful clusters, static utility classes for stateless groups). It must import nothing outside the standard library; a bare copy of the file has to import and run anywhere. Canonical patch targets are `tai.<Class>.<name>` (or the module attribute for module-level data).
- There are two kinds of test, in two different places, on purpose — see "Testing" below for the full split. In short: unit tests are top-level `test_*.py`; integration tests are `tests/<topic>.py` (no `test_` prefix) built on `tai_test.py`.
- The in-`tai.py` TUI framework is documented in [tui.md](tui.md) — read it before touching any `Tui*` code.
## Testing
**Unit tests**: `test_tai.py`, `test_seal.py`, at the repository top level, named `test_*.py`, `unittest`-based, import `tai.py` as a module directly, no subprocess, fast. `python3 test_tai.py` / `python3 test_seal.py`, or a single test via `python3 -m unittest test_tai.SomeClass.test_method`.
**Integration tests**: `tests/<topic>.py` (e.g. `tests/rag.py`), deliberately *not* named `test_*.py` and *not* at the top level, so the two kinds are never confused. These drive the real `tai.py` binary as a subprocess (one-shot or REPL) and check actual CLI behavior, using `tai_test.py` (the framework, at the repository root) as the one and only base to build on — never a one-off `subprocess`/`asyncio` harness per file. **They must exercise a real LLM call through the real binary — never a mocked model or a database write standing in for what the agent would have done; the resulting flakiness from a free, rotating backend is accepted, not engineered away.** Full conventions, that rule's full rationale, the required isolation rule (always a throwaway `TAI_HOME`), and why verification must check vault state rather than scrape CLI rendering, are in `tests/README.md` — read it before adding a new one. Run with `python3 tests/<topic>.py` (`--filter`, `--interactive`, `--timeout` all supported).
**Integration tests**: `tests/<topic>.py` (e.g. `tests/rag.py`), deliberately *not* named `test_*.py` and *not* at the top level, so the two kinds are never confused. These drive the real `tai.py` binary as a subprocess (one-shot or REPL) and check actual CLI behavior, using `tai_test.py` (the framework, at the repository root) as the one and only base to build on — never a one-off `subprocess`/`asyncio` harness per file. **They must exercise a real LLM call through the real binary — never a mocked model or a database write standing in for what the agent would have done; the resulting flakiness from a free, rotating backend is accepted, not engineered away.** Full conventions, that rule's full rationale, the required isolation rule (always a throwaway `TAI_HOME`), and why verification must check vault state rather than scrape CLI rendering, are in [tests/README.md](tests/README.md) — read it before adding a new one. Run with `python3 tests/<topic>.py` (`--filter`, `--interactive`, `--timeout` all supported).
## Agent Overview
@@ -83,6 +84,6 @@
## Research Rule
All external research conducted for this project (web searches, reading documentation to settle a design question, competitive/prior-art analysis) is archived under `research/`, maintained under the auspices of the **Molodetz Institute — Research & Retrieval Division**. The full specification — folder layout, the three required document types (Source Dossier, Source Citation Card, Conclusion Document), required metadata, and the quality bar — lives in `research/README.md`; `research/INDEX.md` is the living registry and statistics page. Read `research/README.md` in full before writing into this archive.
All external research conducted for this project (web searches, reading documentation to settle a design question, competitive/prior-art analysis) is archived under `research/`, maintained under the auspices of the **Molodetz Institute — Research & Retrieval Division**. The full specification — folder layout, the three required document types (Source Dossier, Source Citation Card, Conclusion Document), required metadata, and the quality bar — lives in [research/README.md](research/README.md); [research/INDEX.md](research/INDEX.md) is the living registry and statistics page. Read [research/README.md](research/README.md) in full before writing into this archive.
The binding rule, in short: **a research task is not done until it is written into `research/`.** Every source actually consulted gets its own document (Tier 1 if read and synthesized, Tier 2 if only seen in a result listing — never skipped, never inflated past what's actually known), every decision the research produced gets its own Conclusion document in that subject's `conclusions/` folder, and `research/INDEX.md` gets updated in the same pass. Check `research/INDEX.md` for an existing subject before opening a new one.
+17 -3
View File
@@ -4,7 +4,7 @@ retoor <retoor@molodetz.nl>
**Zero dependencies. A real agent.**
`tai` is an autonomous AI agent written in Python — the entire product lives in the standalone `tai.py` file (grouped into classes) and imports **only the Python standard library**. No `pip install`, no `requirements.txt`, no build step, no lockfile to rot. It runs as an interactive REPL or a one-shot command, reasons through an OpenAI-compatible backend, acts through 38 tools, keeps per-profile memory in a self-healing SQLite vault, seals that vault at rest, ranks every search with real BM25 (no vector database, no embeddings bill, no GPU), chunks and indexes anything worth remembering for retrieval, and can wall shell/file work off inside a disposable container sandbox.
`tai` is an autonomous AI agent written in Python — the entire product lives in the standalone `tai.py` file (grouped into classes) and imports **only the Python standard library**. No `pip install`, no `requirements.txt`, no build step, no lockfile to rot. It runs as an interactive REPL or a one-shot command, reasons through an OpenAI-compatible backend, acts through 42 tools, keeps per-profile memory in a self-healing SQLite vault, seals that vault at rest, ranks every search with real BM25 (no vector database, no embeddings bill, no GPU), chunks and indexes anything worth remembering for retrieval, and can wall shell/file work off inside a disposable container sandbox.
Nothing here is decorative. Every mechanism below exists because something specific was expensive — context tokens, API calls, attacker surface, disk writes, or your attention — and got engineered out.
@@ -68,6 +68,7 @@ Trailing arguments form a one-shot prompt: the agent answers once and exits with
| `/agent clear` | Purge finished subagents |
| `/compact` | Compress history into a summary |
| `/clear` | Drop history, keep the system message |
| `/demo` | Fullscreen TUI framework showcase |
| `/help` | Show the command overview |
| `/quit` | Exit |
@@ -134,7 +135,7 @@ This only ever activates as a genuine last resort: it's wired into `all_candidat
Every tool schema you hand an LLM costs real tokens on every single turn, whether or not it gets used, and a model drowning in fifty tool definitions picks worse than one offered the five that actually matter. tai treats this as a first-class engineering problem, not an afterthought, with three independent layers stacked on top of each other:
**1. Core vs. lazy.** Of the 38 tools, only `CORE_TOOLS` — `shell`, `read_file`, `write_file`, `remember`, `recall`, `load_skill`, `fork`, `poll` — are sent to the model on every turn. The other 30 are *lazy*: their full JSON schema only enters the request payload when they're judged relevant to what's actually being discussed.
**1. Core vs. lazy.** Of the 42 tools, only `CORE_TOOLS` — `shell`, `read_file`, `write_file`, `remember`, `recall`, `load_skill`, `fork`, `poll` — are sent to the model on every turn. The other 34 are *lazy*: their full JSON schema only enters the request payload when they're judged relevant to what's actually being discussed.
**2. Relevance by three independent signals, not a fixed list.** `select_tools()` scans the last 6000 characters of conversation (user, assistant, and tool-result turns — `conversation_text()`) and pulls in a lazy tool's full schema the moment any one of three things matches:
@@ -148,7 +149,7 @@ Tool *results* feed this loop too: a spill pointer that mentions `record_read` l
A compact name-plus-one-line-summary catalog of every lazy tool (`tool_catalog()`) still rides in the system prompt at all times, so nothing is ever *hidden* — only its verbose, multi-field JSON schema is deferred. `/tools` shows the core/lazy split live (respecting the same environment gates a real turn would see), `/tools <name>` shows one tool with its full description and trigger tags.
Net effect: the model sees 8 schemas by default instead of 38, picks up exactly the ones a conversation actually needs, and never has a tool pushed on it that can't possibly apply in the current environment.
Net effect: the model sees 8 schemas by default instead of 42, picks up exactly the ones a conversation actually needs, and never has a tool pushed on it that can't possibly apply in the current environment.
## Context budget and compaction
@@ -316,6 +317,10 @@ Shell output streams live while a command runs — a rolling 4-line window insid
File changes render as a unified diff right inside the call box: line numbers, green `+`/red `-` rows on tinted backgrounds, Python syntax coloring, and `···` separators between hunks, capped at 120 rows with an explicit hidden-line note rather than a wall of scroll. New files show all green, deletes show all red, restores diff against whatever's currently on disk.
## TUI framework
The interactive UI toolkit lives in `tai.py` itself (stdlib-only, no dependencies): a retained, DOM-like panel tree with rows/cols flow, percent/`fr`/`auto` sizing, docking, named z-layers, a cuts/chops occlusion compositor, SGR mouse, bubbling events, Tab/Shift+Tab focus with modal trap, tick animations, and live pty sub-terminals behind a reimplemented VT screen. `/demo` (or `./tai.py --demo`) tours every widget; `./tai.py --demo-shot` prints one composed frame as plain text. Full reference: [tui.md](tui.md); design background: [research/tui-framework-design/conclusions/retained-dom-compositor-tui.md](research/tui-framework-design/conclusions/retained-dom-compositor-tui.md).
## Configuration
| Variable | Purpose | Default |
@@ -343,6 +348,10 @@ Install asks for the bot token up front, verifies it against `getMe`, stores it
python3 -m unittest test_tai.CreateSkillTests
python3 -m unittest test_tai.CreateSkillTests.test_creates_and_refreshes
python3 tests/<topic>.py # integration suite: real binary, real LLM calls
Unit tests live at the top level (`test_tai.py`, `test_seal.py`); integration tests live in `tests/` on the `tai_test.py` framework and must drive real LLM calls through the real binary. Conventions: [AGENTS.md](AGENTS.md#testing), [tests/README.md](tests/README.md).
## Layout
tai.py the entire agent in one standalone file: CLI entry + stable
@@ -352,3 +361,8 @@ Install asks for the bot token up front, verifies it against `getMe`, stores it
config, rendering — grouped into classes
test_seal.py seal regression tests
test_tai.py agent, tools, records, graph, search/RAG, skills, scheduler tests
tai_test.py integration-test framework (process spawn, pty, cleanup)
tests/ integration tests, one file per topic (see tests/README.md)
tui.md TUI framework reference (see "TUI framework" above)
research/ archived design research, registry at research/INDEX.md
AGENTS.md contributor conventions (tests, research rule)
+5 -4
View File
@@ -1,6 +1,6 @@
# Research Archive Index
**Molodetz Institute — Research & Retrieval Division.** This is the living registry of every subject in `research/`. It is updated in the same pass as any new research, per the standing rule in `README.md` §4. Never edit a subject's documents without also updating its row here.
**Molodetz Institute — Research & Retrieval Division.** This is the living registry of every subject in `research/`. It is updated in the same pass as any new research, per the standing rule in [README.md](README.md#4-the-standing-rule-also-stated-in-agentsmd) §4. Never edit a subject's documents without also updating its row here.
## Global statistics
@@ -22,7 +22,7 @@
| [`agent-memory-recency-tracking`](agent-memory-recency-tracking/) | 63 (16 / 47) | 1 | 2026-10-09 | 2026-10-09 | Designing an LRU-style last-used/use-count tracker for vault records and chunks |
| [`async-pty-process-testing`](async-pty-process-testing/) | 59 (20 / 39) | 1 | 2026-10-09 | 2026-10-09 | Designing tai_test.py (pty vs. pipes, non-blocking stdin, process-group cleanup, test-framework shape) — backfilled after it was initially skipped, see activity log |
| [`shell-multiplexer-auto-attach`](shell-multiplexer-auto-attach/) | 138 (48 / 90) | 2 | 2026-10-09 | 2026-10-10 | Designing a tai-managed terminal multiplexer + bashrc auto-attach: tmux-as-substrate recommendation, interactive-vs-automation shell detection; extended 2026-10-10 to cover zsh/fish hook syntax |
| [`tui-framework-design`](tui-framework-design/) | 54 (9 / 45) | 1 | 2026-10-10 | 2026-10-10 | Designing tai's in-house TUI framework (retained DOM, cuts/chops compositor, SGR mouse, bubbling events, VT sub-terminals); extended 2026-10-10 with synchronized-output verification (DEC 2026); implemented in working-tree tai.py and verified the same session |
| [`tui-framework-design`](tui-framework-design/) | 54 (9 / 45) | 1 | 2026-10-10 | 2026-10-10 | Designing tai's in-house TUI framework (retained DOM, cuts/chops compositor, SGR mouse, bubbling events, VT sub-terminals); extended 2026-10-10 with synchronized-output verification (DEC 2026); implemented in working-tree tai.py and verified the same session; same-day follow-up fixed the 19 suite failures natively, deleted tai_pkg/, suite 320/320 green, committed |
## Recent activity log
@@ -38,9 +38,10 @@
| 2026-10-10 | `tui-framework-design` | New subject: 44 sources (7 Tier 1 / 37 Tier 2) + 1 conclusion (`conclusions/retained-dom-compositor-tui.md`), covering 4 WebSearch calls plus 7 direct page fetches on TUI architecture (Textual events/layout/animation guides, pyte repo, SGR mouse protocol, rabbitui Textual synthesis, tmux popup-removal commit). Decision: retained-DOM framework in tai.py with cuts/chops compositor, named layers, bubbling events, SGR mouse, tick animations, reimplemented VT sub-terminals. Nothing built yet; implementation plan presented, awaiting go. Counts hand-verified (shell unavailable for `research/audit.py` in this session). |
| 2026-10-10 | `tui-framework-design` | Extended with 10 more sources (2 Tier 1 / 8 Tier 2) verifying synchronized-output framing (DEC mode 2026, `CSI ? 2026 h/l`, harmless no-op where unsupported). Subject total now 54 sources (9 Tier 1 / 45 Tier 2), 1 conclusion. Implementation approved by retoor the same session; framework being built into tai.py. |
| 2026-10-10 | `tui-framework-design` | Implementation finished and verified (no new sources): framework in working-tree `tai.py`, `tui.md` docs, `/demo` + `--demo` + `--demo-shot` wiring, 49 TUI unit tests green, 21/21 headless interaction checks green, 10/10 live pty e2e checks green, `test_seal.py` green. The 19 remaining `test_tai.py` failures are the pre-existing monolith/facade working-tree mismatch, proven unrelated via stash comparison. Conclusion document updated. |
| 2026-10-10 | `tui-framework-design` | Follow-up (no new sources): the 19 failures from the previous row were fixed natively — five classes (`Consts`, `Install`, `Sandbox`, `Opencode`, `SysInfo`) defined directly in standalone `tai.py` with module functions delegating, four test patch-targets corrected to the real `tai.*` seams (assertions unchanged) — then `tai_pkg/` deleted outright per retoor's call. Suite now 320/320 green (`test_tai.py`) + 16/16 (`test_seal.py`); `tui.md` focus rules updated (Tab always cycles, auto-focus, focus memory); committed. Conclusion document forward-corrected. |
## How to use this index
- Before starting new research, search this table for an existing subject before opening a new one (see `README.md` §2, "one subject, one home").
- Before starting new research, search this table for an existing subject before opening a new one (see [README.md](README.md#2-directory-layout) §2, "one subject, one home").
- Every subject folder is self-contained: its sources live flat inside it, its conclusions live in its `conclusions/` subfolder.
- Tier 1 vs Tier 2 is a statement about how deeply a source was actually read, not about its importance — see `README.md` §3 for the exact distinction.
- Tier 1 vs Tier 2 is a statement about how deeply a source was actually read, not about its importance — see [README.md](README.md#3-the-three-document-types) §3 for the exact distinction.
+2 -2
View File
@@ -175,14 +175,14 @@ What actually happened as a result (a feature shipped, a design rule adopted, a
| <YYYY-MM-DD> | Initial conclusion recorded. |
```
## 4. The standing rule (also stated in `AGENTS.md`)
## 4. The standing rule (also stated in [AGENTS.md](../AGENTS.md#research-rule))
**Every time research is conducted for this project — a web search, a round of searches, reading documentation to settle a design question — it is written into this archive before the research is considered "done," not as an afterthought.** Concretely, every research task produces:
1. a subject folder (new, or an existing one reused);
2. one document per source actually consulted, Tier 1 if it was read and synthesized, Tier 2 if it was only seen in a result listing — no source gets skipped just because it turned out to be low-value; a low-value source is itself a finding, recorded honestly;
3. a conclusion document for every decision the research actually produced, if any was reached;
4. an updated `INDEX.md` — the subject's row, the global counts, and the "Recent activity" log all get updated in the same pass, never left for later.
4. an updated [INDEX.md](INDEX.md) — the subject's row, the global counts, and the "Recent activity" log all get updated in the same pass, never left for later.
A research task that produces no written archive entries did not happen, as far as this project is concerned — if it isn't here, it gets redone and re-verified the next time someone needs it, which is the exact waste this archive exists to prevent.
@@ -51,7 +51,7 @@ Build a small retained-DOM TUI framework directly in `tai.py`, borrowing Textual
## Action taken
Built and verified the same session: framework implemented in working-tree `tai.py` only (per retoor's explicit call), documented in `tui.md`, wired to `/demo`, `--demo`, `--demo-shot`. Verification: 49/49 new TUI unit tests pass, 21/21 headless interaction checks pass (focus, modal trap, clicks, wheel, resize, spawn/close/ask, quit paths, 60-frame animation glide at ~15ms/frame, real pty spawn/stream/reap), 10/10 live pty end-to-end checks pass (dialog, options, answer toast, stream ticks, alt-screen/mouse/sync markers, clean teardown, exit 0), `test_seal.py` 16/16 pass. Full `test_tai.py` still reports the 19 pre-existing facade-world failures that exist with and without this work (working-tree `tai.py` is an uncommitted monolith while the committed suite targets the facade; verified via stash run and zero removed definitions).
Built and verified the same session: framework implemented in working-tree `tai.py` only (per retoor's explicit call), documented in `tui.md`, wired to `/demo`, `--demo`, `--demo-shot`. Verification: 49/49 new TUI unit tests pass, 21/21 headless interaction checks pass (focus, modal trap, clicks, wheel, resize, spawn/close/ask, quit paths, 60-frame animation glide at ~15ms/frame, real pty spawn/stream/reap), 10/10 live pty end-to-end checks pass (dialog, options, answer toast, stream ticks, alt-screen/mouse/sync markers, clean teardown, exit 0), `test_seal.py` 16/16 pass. Full `test_tai.py` still reports the 19 pre-existing facade-world failures that exist with and without this work (working-tree `tai.py` is an uncommitted monolith while the committed suite targets the facade; verified via stash run and zero removed definitions). Update later the same day: the 19 were fixed natively — the five classes defined directly in standalone `tai.py` with module functions delegating, four test patch-targets corrected to the real `tai.*` seams with assertions unchanged — then `tai_pkg/` was deleted outright per retoor's call. Suite is 320/320 green (`test_tai.py`) plus 16/16 (`test_seal.py`), `tui.md` focus rules updated (Tab always cycles, auto-focus prefers non-terminals, dialogs restore focus on close), all committed.
## Changelog
@@ -59,3 +59,4 @@ Built and verified the same session: framework implemented in working-tree `tai.
|---|---|
| 2026-10-10 | Initial conclusion recorded. |
| 2026-10-10 | Implementation built, verified, and recorded; research extended to 54 sources with synchronized-output verification. |
| 2026-10-10 | Follow-up: 19 suite failures fixed natively in `tai.py`, `tai_pkg/` deleted, suite 320/320 green; Action taken updated. |
+6 -6
View File
@@ -4,11 +4,11 @@ This is where **integration tests** live — tests that drive the real `tai.py`
subprocess and check its actual behavior. They are a different thing from the project's
**unit tests** (`test_tai.py`, `test_seal.py`, at the repository top level, `unittest`-based,
import `tai.py` as a module, fast, no subprocess). Both exist on purpose, side by side; see
`AGENTS.md`'s "Testing" section for which convention applies to which kind of test, and do
[AGENTS.md](../AGENTS.md#testing)'s "Testing" section for which convention applies to which kind of test, and do
not move one style into the other's location or naming.
**This is the only way integration tests get written for this project from now on.** A new
integration test is a new file under `tests/`, built on `tai_test.py` (the framework, at the
integration test is a new file under `tests/`, built on [tai_test.py](../tai_test.py) (the framework, at the
repository root — read its module docstring before using it). Don't hand-roll a fresh
`subprocess`/`asyncio` harness per test file; extend `tai_test.TestCase` instead, so every
integration test behaves consistently (same timeout handling, same process cleanup, same
@@ -43,7 +43,7 @@ end to end, while building `tests/rag.py`:
fidelity score of 1.00 against a value the test itself inserted would prove nothing.
So: **we do not write integration tests that fake the AI's part to get a clean, deterministic
result. We accept that free, rotating backends (see root `README.md`'s "Backends" section)
result. We accept that free, rotating backends (see root [README.md](../README.md#backends)'s "Backends" section)
make LLM-driven steps genuinely flaky, and we treat that flakiness as the cost of actually
testing the thing we built, not as a defect to engineer around by testing something easier
instead.** The practical consequences of this — report ratios instead of demanding
@@ -55,12 +55,12 @@ the harness itself works, never as a replacement for the AI-driven test of the a
1. Create `tests/<topic>.py` (lowercase, no `test_` prefix — that prefix is reserved for the
top-level unit-test files so the two kinds are never confused by name alone).
2. `sys.path.insert(0, str(Path(__file__).resolve().parent.parent))` then `from tai_test import TestCase, ...` — see `tests/rag.py` for the exact boilerplate.
2. `sys.path.insert(0, str(Path(__file__).resolve().parent.parent))` then `from tai_test import TestCase, ...` — see [tests/rag.py](rag.py) for the exact boilerplate.
3. Subclass `TestCase` for each distinct scenario. Required: `name` (a class attribute),
`async def run(self)` (drive the process via `self.spawn(...)`, return whatever `verify()`
needs), `async def verify(self, result)` (raise `TestAssertionFailed` with a clear message
on failure). Optional: `async def setup(self)` / `async def teardown(self)`, `timeout`,
`mode` ("pipe" or "pty" — see `tai_test.py`'s docstring for when each applies).
`mode` ("pipe" or "pty" — see [tai_test.py](../tai_test.py)'s docstring for when each applies).
4. Collect your cases into a module-level `ALL_CASES = [...]` list and end the file with:
```python
if __name__ == "__main__":
@@ -107,7 +107,7 @@ step is never faked out for convenience. What *can* and should vary is how much
"is the pty transport broken" from "did the model behave today" when something fails. It is
a smoke check *alongside* the AI-driven test, never a *replacement* for it when the thing
under test is the agent's own tool-use behavior.
- tai's primary backend selects a free model per request (see the root `README.md`'s
- tai's primary backend selects a free model per request (see the root [README.md](../README.md#backends)'s
"Backends" section) — the exact model, and therefore exact tool-calling reliability, can
differ between runs. Say this plainly in the test's docstring wherever a step is LLM-driven.
- Write `verify()` to fail only on a genuinely empty/broken result, not on "fewer than 100% of
+3 -2
View File
@@ -8,10 +8,11 @@ animations, and rendering.
Run the live tour with `/demo` in the REPL or `./tai.py --demo`.
`./tai.py --demo-shot` prints one composed frame as plain text (the
headless smoke path, also used by tests).
headless smoke path). Covered by `test_tai.py`'s `Tui*Tests` classes
plus live pty checks; user-facing summary in [README.md](README.md#tui-framework).
Design background and rejected alternatives:
`research/tui-framework-design/conclusions/retained-dom-compositor-tui.md`.
[research/tui-framework-design/conclusions/retained-dom-compositor-tui.md](research/tui-framework-design/conclusions/retained-dom-compositor-tui.md).
## Quick start