tai/tui.md

tai TUI framework

A retained, DOM-like terminal UI toolkit living entirely in tai.py (standard library only, no dependencies). Components are plain Python objects in a tree; a compositor merges them bottom-to-top by layer and z-order every frame; one asyncio loop drives input, timers, animations, and rendering.

Run the live tour with /demo in the REPL or ./tai.py --demo. ./tai.py --demo-shot prints one composed frame as plain text (the headless smoke path). Covered by test_tai.py's Tui*Tests classes plus live pty checks; user-facing summary in README.md.

Design background and rejected alternatives: research/tui-framework-design/conclusions/retained-dom-compositor-tui.md.

Quick start

import tai

app = tai.TuiApp()
root = tai.TuiPanel(flow=None, style=tai.TuiStyle(bg="black"))

def answered(value):
    print("picked:", value)

root.add(tai.TuiQuestion(
    "Deploy to production?",
    ["Ship it", "Staging first", "Abort"],
    on_done=answered, title="Confirm", w=44,
))
app.mount(root)
app.run()

Keys: Tab / Shift+Tab move focus, arrows + Enter drive lists and buttons, Esc closes dialogs, q quits demo apps, Ctrl+C always quits.

Architecture

• Tree. TuiPanel is the base of everything: geometry, children, paint, event dispatch. TuiRows / TuiCols preset flow direction. add() / remove() / find(id) mutate the tree; Mount/Unmount events fire on bind.
• Layout. Two phases: measure() (intrinsic content size, chrome excluded) then arrange() (absolute rects plus clip rects). Docked children carve edges first, flow children fill the remainder, and positioned children (x/y given) overlay out of flow. See sizes below.
• Compositor. Lazy, see "Lazy rendering" below: painted cells are cached per panel, a frame is built front to back so every cell is written once, and only damaged rects are recomposed and diffed against the screen the compositor believes is shown. One single-write update wrapped in synchronized-output markers (CSI ? 2026 h/l, ignored where unsupported) keeps output atomic and flicker-free; a frame that changes nothing writes nothing.
• Hit-testing. Panels index into an 8x4-cell spatial grid. A layout only hands over its stack; the grid is built by the first click or wheel event after it, so frames without mouse input never pay for it. Clicks resolve topmost-first inside clip rects.
• Loop. One asyncio loop. stdin and sub-terminal ptys are loop readers (and writers while input is queued) that dispatch as bytes arrive; a 30 fps frame clock (asyncio.sleep) advances timers, animations and painting. offload(work, done) runs blocking work on its own daemon thread and calls done(result, error) back on the loop via call_soon_threadsafe; without a running loop (headless tests) it runs inline. An exception in a loop callback is kept, the terminal restores, then it is re-raised. Dirty flags skip idle frames entirely. Timers (after/every), animations, widget on_tick, and terminal polling all run on the same clock. Terminal state (raw mode, alt screen, mouse, cursor, SIGWINCH handler, child ptys) restores in a finally.

Lazy rendering

Rule: nothing is computed that is not visible, and nothing is recomputed that did not change. Four layers, each with a hard cap:

1. Per-panel cell cache. paint() output is converted once to interned (char, style key) cells and kept on the panel (_cells, at most the panel's own area). A panel repaints only after touch(). Writing any attribute of a mounted panel is the change signal: TuiPanel.__setattr__ calls touch() (repaint this panel) or, for geometry names (x y w h flow dock layer z offset align valign padding gap border visible), dirty_layout(paint=False). A scalar or an equal TuiStyle written back unchanged is not a change. In-place mutation (list.append, set.add) must call self.touch(); every built-in widget does. The anonymous app.dirty_paint() / app.dirty_layout() still mean "everything may have changed" and drop every cache, so an unaudited caller is slow, never wrong. set_focus touches the old and new focus panel (border and cursor style depend on focus). Cache invalidation also covers resize (aw/ah change), re-attach (_bind_app), and a broken widget (its paint() raising covers nothing that frame and is retried).
2. Front-to-back coverage. TuiCompositor.fill walks the stack from the topmost panel down, keeping per-row covered intervals. Each panel copies only its uncovered intervals (list slice copies, no per-cell Python), and a panel with no uncovered cell is never painted at all: the root background under a full-screen layout costs nothing. No overdraw, no sort per frame (the stack is built once per layout).
3. Damage rects. touch() records the panel; a layout pass diffs each panel's (ax, ay, aw, ah, clip) against the previous pass and records old and new rects of what moved or resized; a different stack (panel added, removed, reordered, shown, hidden) means the whole screen. Only the merged rects are recomposed and compared with the persistent screen (TuiCompositor.last). More than TUI_DAMAGE_RECTS (64) rects means the whole screen. Damage is recorded under a lock because turn threads touch panels while the loop composes, and the dirty flags are cleared at the start of a pass, so a touch during a frame schedules the next frame.
4. Quiet output. Bytes are built from the cell keys alone (TuiStyle.sgr_from_key, memoized, TUI_SGR_CAP 4096, dropped on theme change). A frame in which no cell differs and the cursor stayed returns "" and writes nothing. measure() is memoized per layout pass (TuiPanel.measured(); TuiTerminal opts out because its measure follows its screen size), and TuiStyle.key() is cached and dropped on any field write.

Correctness guard: the original whole-tree algorithm is kept as TuiCompositor.compose_reference. TuiApp.verify (env TAI_TUI_VERIFY=1) recomposes from scratch after every frame and raises TuiError naming the first diverging cell and the panel on top of it. test_tai.TuiLazyTests runs a seeded 400-step storm (keys, wheel, clicks, resizes, themes, dialogs, bot switches, timers) with it on, replays the emitted bytes through TuiVtScreen and compares with a fresh render, and mutation-checks that dropping damage or geometry rects is caught. Run the whole suite with TAI_TUI_VERIFY=1 after touching paint, layout, or the compositor.

Geometry

Sizes accept int cells, "50%" (of the parent content box), "1fr" (share of leftover flow space, largest-remainder exact), and "auto" (measured content). Positions accept ints (negative counts back from the far edge), percents, and "center". Sizes are content-box: border and padding add outside the given w/h (CSS content-box).

• flow="rows" stacks children top to bottom; "cols" left to right; None places unpositioned children at the origin (use explicit geometry or docking instead).
• align / valign (left/center/right, top/middle/bottom) place children in leftover main-axis space and center them on the cross axis. Rows-cross auto fills the width; cols-cross auto hugs content (Textual parity).
• dock="top|bottom|left|right" pins a child to an edge outside the flow; docked panels never scroll.
• layer names a z band (base 0, panel 10, dialog 20, toast 30); z offsets inside the band; ties break by creation order. There is no limit on layers, dialogs, nesting, or components.
• offset=(dx, dy) shifts a panel visually after layout.
• padding (int, (v, h), or (t, r, b, l)), gap (flow spacing), border with title, scroll_x / scroll_y with scrollable.

Events

TuiEvent subclasses: Key, Click, Scroll, Focus, Blur, Resize, Tick, Mount, Unmount, Select, Press, Close, Paste. Dispatch calls on_<name> on the target, then walks base classes implicitly (plain attribute lookup), then bubbles to the parent while bubble is true. event.stop() halts everything. Custom events: subclass TuiEvent, set name, panel.emit() it. Widget callbacks (on_press, on_select, on_close) are the same namespace: assigning one sets the handler, and it receives the event object (with panel, and index/option/value for Select). Only on_done (Question) is a plain callback taking the answered value.

Focus: focusable panels join a chain sorted by on-screen (y, x). Tab / Shift+Tab ALWAYS cycle with wraparound, even out of a live terminal: no widget may swallow the only focus-movement keys (keyboard trap). Tradeoff: shells lose Tab-completion; click or Tab into a terminal to type, Tab out to leave. Clicks focus the topmost focusable under the cursor. An open modal traps the chain inside its subtree and swallows outside clicks. set_focus, focus_first, focus_next, focus_prev, ensure_focus manage it; Focus/Blur events fire on change and never bubble. Automatic focus (mount, reset) prefers non-terminal widgets via auto_focus so keys keep bubbling to app handlers by default; opening a dialog remembers the previous focus and closing it restores that panel (validated as still attached and shown) before falling back to auto_focus. Stale focus on a removed panel is detected by tree attachment, not just visibility.

Mouse and keyboard

Mouse enables ?1000h + ?1006h (clicks plus wheel, no motion flood). The parser accepts SGR (ESC [ < b ; x ; y M/m) and legacy (ESC [ M bxy) reports, 1-indexed to 0-indexed, with shift/alt/ctrl modifiers. Split reports across reads are buffered, never parsed partial; parsed reports are consumed and never leak into text input.

Keys: printable text (UTF-8), enter, tab, backtab (Shift+Tab), backspace, escape (lone ESC after input idles; ESC+key is alt+key), arrows, home/end, insert/delete, pageup/pagedown, F1-F12, ctrl+letter, and modified arrows (shift+up style). Tab/Shift+Tab always move focus (see Focus above); terminals receive every other key. Ctrl+C always quits the app loop.

Kitty keyboard: setup pushes progressive-enhancement flags 10 (all-keys-as-escape-codes plus event types) best-effort and pops on teardown; unsupported terminals ignore both bytes. The parser decodes CSI unicode-key-code:alternate ; mods:event-type ; text u next to the legacy grammar, so kitty and legacy terminals share one stream: bare modifier presses arrive as shift/ctrl/alt/super key events (press only; repeats and releases are dropped, text-key repeats still type), text stays legacy-identical (associated-text tail wins, shift+letter uppercases, US-shift map covers the rest), super/hyper/meta/locks are dropped. Only kitty-protocol terminals (kitty, WezTerm, foot, ghostty) report bare Shift at all; everywhere else no bytes exist to detect, so double-Shift chords only work there. TuiDoubleTap(timeout=0.4) matches two same-key presses on an injectable clock; set app.on_double_tap to a callback returning True when it acted (falsy falls through to normal handling, so fast double-Tab still cycles).

Paste: setup enables bracketed paste (CSI ? 2004 h, best-effort like the kitty flags) and teardown disables it. A marked paste arrives as one TuiPaste(text) event routed to the focused panel; the body stays literal (no escape parsing inside) and a paste over 1MB (TUI_PASTE_CAP) is dropped whole, never truncated. Unmarked CRLF still coalesces to a single Enter, so plain pastes keep correct line breaks on terminals without bracketed mode.

Animation and timers

panel.animate("x", 40, duration=0.6, easing="in_out_cubic", on_done=next_step)
app.after(2.0, popup_tip)
app.every(0.5, poll_status)

animate() tweens any numeric attribute (x, y, w, h relayout; anything else repaints) with duration or speed, delay, and an easing (linear, in|out|in_out quad/cubic). Timers take plain callbacks; exceptions in callbacks never break the loop.

Components

• TuiText(text, text_align, wrap) - wrapped static text.
• TuiButton(label, on_press) - Enter/Space/click; focused style; on_press(event) fires per press.
• TuiOptions(options, multi, on_select) - virtualized list; options as strings, (label, value) pairs, or dicts. Arrows/home/end/pgup/ pgdn/wheel move, Enter/Space/click selects; on_select(event) fires per pick and the event also bubbles.
• TuiDialog(*children, on_close) - centered modal; Esc closes; buttons(("OK", cb), ...) adds a button row (callbacks take the Press event); on_close(event) fires on close.
• TuiQuestion(prompt, options, on_done, multi) - prompt plus options; single-select answers on first pick, multi-select via OK.
• TuiHistory(backend, cap) - readline-style input history: older(current) / newer(), an immediate repeat is skipped, capped at INPUT_HISTORY_CAP (1000). The backend is any object with load() and append(text); VaultHistory(store, profile, bot) stores it in the vault's sealed input_history table (one list per profile, shared by every bot, the bot kept as a column), the same table the REPL reads into readline and writes to. A profile with no rows imports the old plain-text history file once. A backend failure is counted in write_errors and the history keeps working in memory. Pass it as TuiTextBox(history=...).
• TuiTextBox(text, complete, mentions) - multiline editor; arrows/ home/end/pgup/pgdn move (no line wrap), Enter splits, Backspace/Delete join, click positions the cursor, Escape bubbles for dialogs; .value joins lines. Bracketed pastes insert literally with CR/CRLF normalized to LF. complete(prefix) wires a TuiAutocomplete popup for /words and mentions(prefix) the same for @words (bots, providers, models, files) before the cursor. send_on_enter=True with on_send(value) turns Enter into send (an open popup still accepts first); Shift+Enter (kitty terminals) and Alt+Enter insert a newline instead. max_chars caps the buffer (TUI_CHAT_INPUT_CAP, 2000, in the chat scene).
• TuiAutocomplete - non-focusable TuiOptions popup owned by a textbox; typing filters, arrows/home/end/pgup/pgdn move, Enter accepts, Esc dismisses (a second Esc still bubbles); flips above the cursor when short on room, max 8 rows. The box keeps focus throughout, and Tab is never used for completion (it always moves focus). Clicks pick outside modals; inside a modal the modal click guard swallows them and the keyboard path still works.
• TuiPromptDialog(label, box_h, on_execute, complete) - label plus a box_h-row editor with OK/Cancel; OK runs on_execute(value) then closes. The demo opens one on double-Shift with slash-command completion and feeds accepted text to the shell.
• TuiTerminal(cmd, interactive) - real pty child streamed through the VT screen; click to focus and type; read-only mode for logs.
• TuiProgress(value) - fraction bar; set() clamps 0..1.
• TuiLog - virtualized scrollback; append(text, style) follows the tail while pinned; wheel/pgup/pgdn/home/end navigate.

Themes

Colors go through named roles, not literals. TuiTheme(name, dark, roles) holds 27 roles: the 12 base roles (bg, surface, fg, muted, border, accent, on_accent, selection, success, warning, error, info) plus 10 Telegram roles (bubble_in, bubble_out, bubble_fg, meta_in, meta_out, tick_read, service_bg, service_fg, unread, unread_muted, with day/night values from the tdesktop theme files) plus 5 chat-scene roles (panel for sidebar, header and status bar, accent2 cyan, accent3 pink, code_bg, tool_bg); TUI_THEMES is the registry with five builtins (dark, light, nord, gruvbox, ansi); tui_register_theme() adds more (all 12 roles required). Widgets use fg="role:accent" style values, resolved against the current theme at draw time; raw colors (#rrggbb, ANSI names, ints) pass through untouched, and per-widget styles merge over role styles as before. Unknown roles render uncolored rather than crashing.

Selection precedence: --theme flag, TAI_THEME env var, dark default; an unknown name prints the available list and exits 2. App.set_theme(name) swaps the global theme and invalidates the compositor, because cells are diffed by style key (role names), which a theme switch does not change; without the invalidation only cells whose text changed got the new colors and old-theme speckles stayed on spaces. F4 cycles registry order in the chat scene. To change a color, edit one value in _tui_builtin_themes() or register a custom theme.

Chat scene (--tui)

./tai.py --tui runs the fullscreen chat UI with REPL parity: a borderless, colored layout. A panel-tinted bots sidebar (colored avatar chip per bot, accent bar on the selected chat, green dot while a turn runs, red Draft: hint), a header with avatar, name and an online / typing... dot, a transcript of aligned bg-fill bubbles (yours right in the accent bubble color, theirs left with the bot name on top, stamp plus ticks on the bottom row inside the bubble), a slim scrollbar in the last transcript column, a rounded composer (accent border while focused) with a Send button, and a status bar of key caps plus profile/bot and model pills. Ticks follow Telegram: clock while sending, single check on the cloud, blue double check once read; service pills mark the date. Sidebar selection switches bots with per-bot input drafts; Tab cycles sidebar, transcript, input, Send.

Assistant text renders through a small line-level markdown renderer (tui_md_blocks, tui_md_inline, tui_wrap_segs): headings, bold, italic (shown in the info color), inline code, fenced code blocks, bullets, numbered lists, quotes, rules. Tool calls show as one-line chips (gear, bold name, k=v arguments, truncated). Command output and errors show as plain bars on the tool background so aligned columns survive. An empty chat shows a welcome card with the key hints.

Slash commands run inline with output captured into a dim bubble (/quit quits); anything else runs a full agent turn whose reply lands as an incoming bubble. The transcript keeps a block journal and replays it on resize, so bubbles reflow instead of freezing at the old width; below usable width they degrade to full-width rows.

Keys: Enter sends (Shift+Enter / Alt+Enter newline), Up / Down recall input history like readline (Up on the first line goes older, Down on the last line goes newer and ends on your unfinished draft; an open completion popup keeps the arrows), F2 toggles the sidebar, F4 cycles the theme, i or Esc jumps to the input, q quits from outside the input. Below 80 columns the sidebar auto-hides (TUI_CHAT_NARROW_COLS); a manual F2 hide/show wins over the automatic rule until toggled again.

Turns run through app.offload: your message paints first, the turn runs on its own thread, and the reply lands back on the loop, so the UI keeps rendering, themes switch, resizes reflow and the sidebar scrolls while the model works. Every bot is its own session (TuiChatSession: its own Agent, journal, draft, turn state), so a long task in one bot never blocks talking to another: switch bots in the sidebar and chat on while the first keeps working. A busy bot shows a spinner in the sidebar, a finished reply you were not looking at shows an unread count, and its journal is intact when you switch back. Only the bot on screen refuses a second send while its own turn runs. An approval or question from a bot you are not looking at does not pop up over the bot you are talking to: it waits (? badge in the sidebar) and opens when you open that bot; its dialog title names the bot. Sessions share one vault (the store connection is check_same_thread=False, sqlite is serialized) and a counted TuiStdioGuard swaps stdout/stdin once for all turn threads. Slash commands run on the bot on screen; /bot x switches the screen to that bot's own session instead of retargeting the shared agent. The loop keeps the real stdout fd from setup, because the turn thread swaps sys.stdout.

The header status only says what is true. The agent reports its real phase through agent.on_phase(name, detail): thinking while the model or the loop works, running <tool> inside a tool call, waiting for you while a dialog is open, typing... once the final reply exists and is being laid out word by word (bounded to TUI_CHAT_REVEAL_STEPS ticks), then online. Thinking and typing animate: braille spinner in cycling accent colors, a shimmer sweeping the word, trailing dots lighting one by one, and Ns elapsed in the status bar.

Shell commands (host and sandbox box) stream live as a message: the turn thread calls agent.stream_factory(command), the scene journals a live block (header chip with the command, the last TUI_CHAT_LIVE_ROWS (8) lines in a fixed window, a status row at the end, TUI_CHAT_LIVE_KEEP (200) lines kept) and repaints it at most every TUI_CHAT_LIVE_TICK (50 ms). It stays in the transcript where it started, so you keep chatting while it scrolls. The REPL has the same thing as the 4-line LiveStream window. When a command stops on a prompt (sudo, ssh, a y/n; a partial last line ending like a prompt after 1 s of silence, or echo switched off), the command runs on a pty and the question goes through the same agent.prompter("ask", ...) dialog (REPL: input/getpass); the answer is typed into the command, a decline sends end-of-file, and the wait does not count against the 120 s timeout (max 20 questions per command). Gap: the TUI dialog does not mask a typed password. Without an interactive user (autonomous mode, background workers, piped stdin) commands run on pipes with no input, as before.

Approvals and ask_user are real dialogs. The agent calls agent.prompter(kind, text, options) from the turn thread; the chat scene opens a TuiQuestion (approve: Allow once / Always allow / Deny; ask: the options plus "Type an answer...") on the loop and the thread waits. Esc, Cancel, or no answer within TUI_CHAT_PROMPT_TIMEOUT (600 s) means deny / no answer, never a hang. Without a prompter (CLI, Telegram) behavior is unchanged.

Measured on one machine only (the development box, Python 3.14, chat scene with a 40-exchange transcript, 19 panels, cost of dirty -> frame_bytes(); the 400-exchange transcript costs the same, history length does not matter). Before the lazy pipeline, every dirty frame cost the same 4.0-4.6 ms at 200x50 (1.5 ms at 80x24) whatever had changed:

frame at 200x50	before	lazy
one typed character	4.6 ms	0.08-0.15 ms
1 Hz status refresh	4.9 ms	0.05-0.07 ms
Tab moves focus	4.3 ms	0.5-0.7 ms
transcript scroll, 2 frames	9.5 ms	2.5-2.8 ms
anonymous dirty_paint() (everything)	4.0 ms	1.5 ms
relayout	5.4 ms	1.5-2.0 ms

Typing at 80x24 went from 1.5 ms to 0.06 ms. Simulated 5 s of the main loop at 200x50: idle went from 4 frames and 18 ms CPU to 0 frames and 1 ms; a running turn from 29 ms to 7 ms. These are CPU numbers: the app was never slow enough to feel, the gain is battery, SSH bandwidth and headroom. Not measured: other machines, slow terminals, 300x70. Guarded by test_frame_stays_inside_budget_on_large_screen (full repaint under one frame) and test_typing_frame_budget_at_200x50 (5 ms ceiling, 30x slack). Live in a pty: a resize reflows to a correct frame in 18-60 ms, a burst of 30 resizes ends on the right frame, key to screen averages 23 ms (input paints immediately, no longer waiting for the next tick; this live figure predates the lazy pipeline and was not re-measured).

Terminal widget notes

TuiVtScreen also serves the multiplexer, so it keeps scrollback (history, a bounded deque set by the owner; only full-screen scrolls outside the alternate screen feed it), answers cursor-position and device-attribute queries (take_replies()), tracks application cursor keys, bracketed paste and the window title, draws the DEC line charset, repeats characters (CSI b) and keeps italic, blink and hidden attributes.

spawn() forks the command onto a pty sized to the widget; resizes propagate via SIGWINCH-winning TIOCSWINSZ. Input queues and drains cooperatively with output reads (never a blocking write from a handler). Exited children keep their last screen with the exit status; teardown kills survivors so demos never leak processes. Without pty the widget shows a plain error line instead of crashing.

Gaps and non-goals (v1)

• Wide (CJK/emoji) characters count as one column; columns around them misalign. Structured to fix with a width table later.
• TuiVtScreen does not reflow on resize: narrowing a screen drops the columns that no longer fit, and widening does not bring them back. The multiplexer (mplex.md) inherits this.
• The theme layer is role names plus a registry, not a full stylesheet language (no selectors, no file-based overrides); only the key-event subset of the kitty protocol (no release tracking, no super/hyper/meta, no alternate-key shortcut matching), no true opacity blending.
• The chat scene markdown renderer is line-level only: no tables, no links, no nested emphasis, no syntax highlighting inside code blocks; there are no bubble tails, no unread pills or per-chat times (no honest data source), no attach/voice or reply-preview (no function behind them), tool-call chips are not collapsible and show no results, no per-token streaming.
• Lazy rendering gaps: a transcript scroll still rewrites every changed cell (no terminal scroll regions, so a 3-line scroll costs ~1.3 ms at 200x50); the main loop still wakes at 30 fps and walks the tree for on_tick, focus and terminal sync (0.01 ms at 19 panels, linear in panels); a damage rect covering a whole panel repaints that whole panel (no per-row paint cache); a layout pass is still a full arrange of the tree; a widget that mutates itself in place without touch() shows stale cells until something else repaints it (TAI_TUI_VERIFY=1 finds these).
• No per-widget async tasks; blocking work belongs in threads that post back via timers.
• The VT parser drops provably-malformed tails past 64KB and caps parsed CSI params at 64; these are corruption guards, not UI limits. UI multiplicity itself (layers, dialogs, depth, components, animations) is intentionally unbounded.