Files
tai/research/async-pty-process-testing/gem5-test-infrastructure-commit.md
T
retoorandClaude Sonnet 5 9dbd1fe667 Add plexer subsystem, container test infra, and last-resort opencode box backend
- Plexer: always-active terminal multiplexing. NativePlexerController/
  TmuxPlexerController share one interface, chosen by which env var is
  already set (TAI_PLEXER_SOCKET vs TMUX_PANE); NativePlexerBackend is
  the pty.fork()-based wrapper (raw mode, SIGWINCH resize forwarding,
  a Unix-socket control protocol, continuous session logging, a
  Ctrl+A prefix key distinct from tmux's own Ctrl+B). Auto-wraps new
  interactive shells via a .bashrc hook mirroring the existing
  command-not-found install pattern, gated on $TMUX/$TAI_PLEXER_ACTIVE/
  a $HOME/.tai-no-plexer bypass. New plexer_send/read/type/status
  tools, env-gated on a detected controller.
- tests/container.py + docker-compose.yml + tests/docker/Dockerfile:
  a disposable podman/docker image for the tests that must never touch
  a real machine (installing the plexer hook into a real ~/.bashrc, a
  live tmux session). Required suite (test_seal.py, test_tai.py,
  tests/plexer.py, tests/bashrc.py) gates the result; tests/rag.py
  runs as best-effort since its real-LLM dependency can't be satisfied
  inside a disposable container with no nested engine.
- tests/bashrc.py: real end-to-end hook install + bash -i auto-wrap,
  anti-recursion, tmux-active skip, non-interactive skip, a live
  TmuxPlexerController test, and an install/uninstall round-trip.
- tests/plexer.py: NativePlexerBackend under a real pty, plus a
  dedicated robustness pass (file permissions, >64KB payload framing,
  a malformed control request, a genuine self-deadlock fix in the
  write-to-child path).
- tai_test.py: the shared async process-driving framework (pipe/pty
  transport, group-wide kill-and-reap, a StdinBridge for --interactive)
  both test files build on.
- ChatClient last-resort backend: when there's no local opencode CLI
  but the sandbox box is already running, install opencode into it
  (official Node tarball, not apt -- apt's nodejs/npm pull in
  webpack/babel/jest as transitive deps) and route through it instead
  of giving up. Only reachable after the first retry round already
  failed everything else; never creates the box itself. TmuxPlexerController.status()
  fixed to use `list-panes` instead of `display-message -p`, which
  silently fell back to the default pane instead of erroring against
  a dead target.
- Fix 4 pre-existing test_tai.py bugs surfaced by running it in a
  clean environment for the first time: two stale mock signatures
  (subprocess.run's stdout= kwarg, Chat.complete's deadline= kwarg),
  one $TMUX-dependent catalog assertion, one tty-dependent color
  assertion.
- research/: a sourced research archive (Source Dossier/Citation
  Card/Conclusion tiers) backing the above design decisions, with its
  own audit.py consistency checker.
- .gitignore: fix a blanket *.md rule that was about to silently drop
  the entire research/ archive from version control; add __pycache__/
  and the one-off debug transcript this session started from.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-10 04:36:05 +02:00

3.2 KiB

gem5 test infrastructure — Timer-based timeout + typed exceptions

Metadata

Field Value
Institute Molodetz Institute — Research & Retrieval Division
Subject async-pty-process-testing
Document type Source Dossier
Tier 1
Source title tests: Add test infrastructure as a Python module (commit), and companion helpers.py
Source URL https://git.sanchayanmaity.net/sanchayanmaity/gem5/commit/f385adc8af744e7f6b78ff9adafd9c3ecf65ce68
Source publisher / author gem5 project (computer architecture simulator)
Source type repository (commit + source file)
Source date (published) Unknown — not stated
Date researched 2026-10-09
Researched by tai (agent), model: Claude Sonnet 5, on behalf of retoor
Research method web search synthesis
Confidence High
Status Active

Summary

A commit adding Python test infrastructure to the gem5 simulator project, together with its companion tests/testing/helpers.py. The helper uses a threading.Timer to enforce a timeout on a running subprocess: the timer's callback terminates the child process, then the surrounding code raises a dedicated exception carrying both the captured exit status and the process's output.

Key findings

  • Timeout enforcement pattern: Timer(timeout, on_timeout_handler) started before communicate()-ing with the process, cancelled afterward if the process finished in time.
  • On timeout, the handler terminates the child and a dedicated exception type is raised carrying the captured status and output together — the caller doesn't have to separately go fetch output after catching a generic timeout error.
  • gem5's newer test framework (noted in the same research pass) further splits each test into a "run" stage and a "verify" stage, where verify units are automatically skipped if their corresponding run stage didn't execute.

Pros / strengths

  • The "exception carries the captured output" design is directly actionable and immediately reusable — it solves the exact problem of "how do you debug a timed-out test" by attaching the evidence to the failure itself rather than requiring separate log inspection.
  • The run/verify stage split is a clean idea for keeping a test's "did it crash" question separate from "did it produce the right result."

Cons / limitations / caveats

  • Uses threading.Timer, a synchronous/thread-based mechanism, not asyncio.wait_for — tai_test.py adapted the idea (typed exception carrying output) to an asyncio-native mechanism (asyncio.wait_for plus try/except) rather than reusing gem5's thread-based implementation directly.
  • Reviewed via commit metadata and the helper file's described behavior, not gem5's full test suite or its actual usage patterns in practice.

Relevance to tai

The direct model for tai_test.py's exception hierarchy: TestTimeout, TestCrashed, and TestAssertionFailed all subclass TaiTestError, which carries a transcript list — so a failing test's exception is the debugging evidence, exactly matching gem5's "exception carries status and output" pattern, just built on asyncio.wait_for/try-except instead of threading.Timer.

Changelog

Date Change
2026-10-09 Initial dossier created.