Docs audit: fix stale claims, cross-link hub docs

- README: 42 tools (was 38), /demo row, TUI section, testing plus
  integration line, full layout map
- tui.md: research link, tests pointer, soften shot-tested claim
- AGENTS.md/tests/README.md/research hubs: cross-links between docs
- Research: forward-correct tui conclusion plus INDEX activity row
  (19 failures fixed, tai_pkg deleted, 320/320 green); audit PASS
This commit is contained in:
2026-10-10 23:51:06 +02:00
parent 133eb5ef63
commit 871f3e5ae6
7 changed files with 38 additions and 20 deletions
+6 -6
View File
@@ -4,11 +4,11 @@ This is where **integration tests** live — tests that drive the real `tai.py`
subprocess and check its actual behavior. They are a different thing from the project's
**unit tests** (`test_tai.py`, `test_seal.py`, at the repository top level, `unittest`-based,
import `tai.py` as a module, fast, no subprocess). Both exist on purpose, side by side; see
`AGENTS.md`'s "Testing" section for which convention applies to which kind of test, and do
[AGENTS.md](../AGENTS.md#testing)'s "Testing" section for which convention applies to which kind of test, and do
not move one style into the other's location or naming.
**This is the only way integration tests get written for this project from now on.** A new
integration test is a new file under `tests/`, built on `tai_test.py` (the framework, at the
integration test is a new file under `tests/`, built on [tai_test.py](../tai_test.py) (the framework, at the
repository root — read its module docstring before using it). Don't hand-roll a fresh
`subprocess`/`asyncio` harness per test file; extend `tai_test.TestCase` instead, so every
integration test behaves consistently (same timeout handling, same process cleanup, same
@@ -43,7 +43,7 @@ end to end, while building `tests/rag.py`:
fidelity score of 1.00 against a value the test itself inserted would prove nothing.
So: **we do not write integration tests that fake the AI's part to get a clean, deterministic
result. We accept that free, rotating backends (see root `README.md`'s "Backends" section)
result. We accept that free, rotating backends (see root [README.md](../README.md#backends)'s "Backends" section)
make LLM-driven steps genuinely flaky, and we treat that flakiness as the cost of actually
testing the thing we built, not as a defect to engineer around by testing something easier
instead.** The practical consequences of this — report ratios instead of demanding
@@ -55,12 +55,12 @@ the harness itself works, never as a replacement for the AI-driven test of the a
1. Create `tests/<topic>.py` (lowercase, no `test_` prefix — that prefix is reserved for the
top-level unit-test files so the two kinds are never confused by name alone).
2. `sys.path.insert(0, str(Path(__file__).resolve().parent.parent))` then `from tai_test import TestCase, ...` — see `tests/rag.py` for the exact boilerplate.
2. `sys.path.insert(0, str(Path(__file__).resolve().parent.parent))` then `from tai_test import TestCase, ...` — see [tests/rag.py](rag.py) for the exact boilerplate.
3. Subclass `TestCase` for each distinct scenario. Required: `name` (a class attribute),
`async def run(self)` (drive the process via `self.spawn(...)`, return whatever `verify()`
needs), `async def verify(self, result)` (raise `TestAssertionFailed` with a clear message
on failure). Optional: `async def setup(self)` / `async def teardown(self)`, `timeout`,
`mode` ("pipe" or "pty" — see `tai_test.py`'s docstring for when each applies).
`mode` ("pipe" or "pty" — see [tai_test.py](../tai_test.py)'s docstring for when each applies).
4. Collect your cases into a module-level `ALL_CASES = [...]` list and end the file with:
```python
if __name__ == "__main__":
@@ -107,7 +107,7 @@ step is never faked out for convenience. What *can* and should vary is how much
"is the pty transport broken" from "did the model behave today" when something fails. It is
a smoke check *alongside* the AI-driven test, never a *replacement* for it when the thing
under test is the agent's own tool-use behavior.
- tai's primary backend selects a free model per request (see the root `README.md`'s
- tai's primary backend selects a free model per request (see the root [README.md](../README.md#backends)'s
"Backends" section) — the exact model, and therefore exact tool-calling reliability, can
differ between runs. Say this plainly in the test's docstring wherever a step is LLM-driven.
- Write `verify()` to fail only on a genuinely empty/broken result, not on "fewer than 100% of