Add plexer subsystem, container test infra, and last-resort opencode box backend
- Plexer: always-active terminal multiplexing. NativePlexerController/ TmuxPlexerController share one interface, chosen by which env var is already set (TAI_PLEXER_SOCKET vs TMUX_PANE); NativePlexerBackend is the pty.fork()-based wrapper (raw mode, SIGWINCH resize forwarding, a Unix-socket control protocol, continuous session logging, a Ctrl+A prefix key distinct from tmux's own Ctrl+B). Auto-wraps new interactive shells via a .bashrc hook mirroring the existing command-not-found install pattern, gated on $TMUX/$TAI_PLEXER_ACTIVE/ a $HOME/.tai-no-plexer bypass. New plexer_send/read/type/status tools, env-gated on a detected controller. - tests/container.py + docker-compose.yml + tests/docker/Dockerfile: a disposable podman/docker image for the tests that must never touch a real machine (installing the plexer hook into a real ~/.bashrc, a live tmux session). Required suite (test_seal.py, test_tai.py, tests/plexer.py, tests/bashrc.py) gates the result; tests/rag.py runs as best-effort since its real-LLM dependency can't be satisfied inside a disposable container with no nested engine. - tests/bashrc.py: real end-to-end hook install + bash -i auto-wrap, anti-recursion, tmux-active skip, non-interactive skip, a live TmuxPlexerController test, and an install/uninstall round-trip. - tests/plexer.py: NativePlexerBackend under a real pty, plus a dedicated robustness pass (file permissions, >64KB payload framing, a malformed control request, a genuine self-deadlock fix in the write-to-child path). - tai_test.py: the shared async process-driving framework (pipe/pty transport, group-wide kill-and-reap, a StdinBridge for --interactive) both test files build on. - ChatClient last-resort backend: when there's no local opencode CLI but the sandbox box is already running, install opencode into it (official Node tarball, not apt -- apt's nodejs/npm pull in webpack/babel/jest as transitive deps) and route through it instead of giving up. Only reachable after the first retry round already failed everything else; never creates the box itself. TmuxPlexerController.status() fixed to use `list-panes` instead of `display-message -p`, which silently fell back to the default pane instead of erroring against a dead target. - Fix 4 pre-existing test_tai.py bugs surfaced by running it in a clean environment for the first time: two stale mock signatures (subprocess.run's stdout= kwarg, Chat.complete's deadline= kwarg), one $TMUX-dependent catalog assertion, one tty-dependent color assertion. - research/: a sourced research archive (Source Dossier/Citation Card/Conclusion tiers) backing the above design decisions, with its own audit.py consistency checker. - .gitignore: fix a blanket *.md rule that was about to silently drop the entire research/ archive from version control; add __pycache__/ and the one-off debug transcript this session started from. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,53 @@
|
||||
# Conclusion: tai_test.py's Transport, Cleanup, and Framework Design
|
||||
|
||||
## Metadata
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Institute | Molodetz Institute — Research & Retrieval Division |
|
||||
| Subject | async-pty-process-testing |
|
||||
| Document type | Conclusion |
|
||||
| Tier | N/A |
|
||||
| Date decided | 2026-10-09 |
|
||||
| Decided by | retoor, advised by tai (agent, Claude Sonnet 5) |
|
||||
| Status | Active |
|
||||
| Confidence | High |
|
||||
|
||||
## Question / problem statement
|
||||
|
||||
How should `tai_test.py` — a reusable integration-test framework for driving `tai.py` both as a one-shot subprocess and as a persistent interactive REPL (including scripting input and detecting turn boundaries) — be built so that it's safe (no leaked zombie processes on timeout, crash, or cancellation) and extensible (a clean base class future tests can subclass), using only the Python standard library, consistent with `tai.py`'s own zero-dependency rule?
|
||||
|
||||
## Conclusion
|
||||
|
||||
The framework uses two unified transports behind one `asyncio.StreamReader`-based interface: plain `asyncio.create_subprocess_exec` with `PIPE` streams for one-shot calls, and a real pty (stdlib `pty.openpty()`, the slave fd given to the child as stdin/stdout/stderr, the master fd wrapped into a `StreamReader` via `loop.connect_read_pipe`) for anything that needs the child to behave as if a human were typing — because `tai.py`'s own code branches on `sys.stdin.isatty()`. Optional live human stdin passthrough is bridged via `loop.add_reader` + raw `os.read` on the fd (never buffered `sys.stdin`, which blocks), with no background thread, mirroring how the child's own output is already consumed. Every process is started with `start_new_session=True` and torn down through an escalating SIGTERM-then-SIGKILL `os.killpg` call that always finishes with `proc.wait()` to reap it, so a timeout or crash never leaves an orphan. The `TestCase` base class (`setup`/`run`/`verify`/`teardown` hooks, typed exceptions carrying the full transcript) follows the declarative-base-class-plus-override-hooks pattern used by real prior art (CERN's Gaudi, gem5, OpenStack's oslotest) rather than being invented from scratch.
|
||||
|
||||
## Supporting sources
|
||||
|
||||
- `../ptyprocess-docs.md`, `../pypi-shellous.md`, `../github-agents-issue1-controlling-terminal.md` — established why a real pty (not a pipe) was needed, and the documented, accepted limitation that `tai_test.py`'s pty mode does not call `ioctl(TIOCSCTTY)` and so doesn't guarantee job-control signal semantics.
|
||||
- `../github-gist-nonblocking-stdin-os-read.md`, `../python-asyncio-issue499-add-reader-eof.md`, `../glama-mcp-remote-py-stdio-jsonl.md` — the exact techniques `StdinBridge` (raw `os.read` via `add_reader`) and `ManagedProcess`'s pty-mode `StreamReader` wrapping (`loop.connect_read_pipe`) implement.
|
||||
- `../pypi-pexpect.md`, `../pyte-terminal-emulator.md`, `../pypi-stitch-pty.md` — the alternatives surveyed and not adopted.
|
||||
- `../iannuccisolutions-timeout-process-group.md`, `../python-docs-asyncio-subprocess.md`, `../alexandra-zaharia-kill-subprocess-children-timeout.md`, `../execa-termination-docs.md` — the direct justification for `start_new_session` + `killpg` + always-reap.
|
||||
- `../discourse-merge-two-streams.md` — the reason a generic multi-stream-merge utility was deliberately **not** built.
|
||||
- `../gaudi-subprocess-base-test.md`, `../gem5-test-infrastructure-commit.md`, `../oslotest-timeout-fixture.md` — the test-framework architecture `TestCase` is modeled on.
|
||||
- `../discourse-subprocess-process-gc.md`, `../slingacademy-stop-kill-child-process.md`, `../bpo-wait-for-timeout-leaves-process-alive.md` — confirmation that relying on timeout or garbage collection alone to clean up a process is documented, known-unreliable, which is why `execute()`'s `finally` block always calls `terminate()` explicitly regardless of how the test ended.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **pexpect** — rejected because it's a blocking/synchronous library and `tai_test.py` is asyncio-native throughout; mixing a blocking library in would have meant either wrapping every call in a thread executor or accepting blocking the event loop, neither of which fit.
|
||||
- **stitch-pty** — rejected as too young/unverified to depend on for core framework plumbing, and as a compiled Rust-backed dependency it would also have violated the project's stdlib-only constraint.
|
||||
- **A generic tagged-queue multi-stream-merge utility** — rejected because the Discourse thread's own participants described the resulting async-generator plumbing as hard to maintain, and `tai_test.py`'s actual need (merging stdout+stderr in pipe mode) is fully satisfied by the simpler `stderr=STDOUT` subprocess argument, making the general utility unnecessary complexity.
|
||||
- **A background thread for bridging live stdin** — not rejected as wrong, but passed over in favor of `loop.add_reader` + `os.read` specifically so stdin-bridging follows the same no-thread, fd-based pattern as the child's own output handling, rather than mixing two different concurrency models in one framework.
|
||||
|
||||
## Risks / open questions
|
||||
|
||||
The pty mode does not explicitly call `ioctl(TIOCSCTTY)`, so there is no guarantee the child's session becomes the pty's controlling terminal in the full job-control sense. `isatty()`-based behavior (all that `tai.py`'s own code checks) is unaffected, since that's a property of the fd itself, not of session leadership — but SIGWINCH-based resize forwarding and tty-line-discipline-generated Ctrl-C-as-SIGINT are not guaranteed to work. This is accepted because no test built so far needs either capability; every test built on `tai_test.py` so far only scripts line-based input/output.
|
||||
|
||||
## Action taken
|
||||
|
||||
Implemented the same day as this research: `tai_test.py` (`ManagedProcess`, `StdinBridge`, `TeeLogger`, `TestCase`, `TestRunner`, `main_cli`) at the repository root, and `tests/rag.py` built on it — both already in use and passing against the real `tai.py` binary.
|
||||
|
||||
## Changelog
|
||||
|
||||
| Date | Change |
|
||||
|---|---|
|
||||
| 2026-10-09 | Initial conclusion recorded. |
|
||||
Reference in New Issue
Block a user