tai/taicpp/docs/util.md

# util: text, ansi, markdown, diff, json, paths

Scope: `src/util/` (namespace `detz`). Source of truth `../tai.py`. All lengths and slices are Unicode code points
(Python `str` semantics) over UTF-8 `std::string`; invalid bytes decode to U+FFFD like `errors="replace"`.
Verified byte for byte against tai.py by `tests/shell_parity.py` (see [shell.md](shell.md)).

| python (tai.py) | C++ (src/util/) | notes |
|---|---|---|
| `Ansi`, `FG_*`, `BG_*` | `Ansi`, `Palette` (ansi.h) | all codes incl. UNDERLINE, STRIKE, GRAY |
| `paint`, `color_enabled`, `strip_ansi`, `style_text`, `style_span`, `truncate_ansi`, `pad_tinted`, `wrap_ansi` | `AnsiText` (ansi.h) | `Paint`/`Paint2` free functions stay; they now honor NO_COLOR and stdout tty like `paint` |
| `shutil.get_terminal_size` | `AnsiText::TerminalWidth` | COLUMNS, then TIOCGWINSZ on stdout, then fallback |
| `truncate`, `capped_text`, `trunc_marker`, `read_capped` | `Truncate`, `CappedText`, `TruncMarker`, `ReadCapped` (text.h) | `CappedText` returns `Capped{present,text,size}`; `Truncate(tail=0)` keeps the whole text like `text[-0:]` |
| `chunk_lines`, `collapse_repeated_lines` | `ChunkLines`, `CollapseRepeatedLines` | `str.splitlines` semantics |
| `split_leading_tags`, `expand_file_mentions`, `complete_files` | same names in text.h | `os.path.expanduser`, `exists`, `abspath` via `PyPath` |
| `human_size`, `estimate_tokens`, `as_int`, `short_error`, `now_iso`, `tool_name_clean`, `parse_duration`, `format_delay` | text.h | |
| `md_inline`, `md_split_row`, `md_is_separator`, `md_align_cell`, `md_table_block`, `render_markdown`, `MD_BULLETS` | `Markdown` (markdown.h) | `re` rules matched by scanners with `re.sub` semantics |
| `render_diff`, `split_diff_lines`, `difflib.SequenceMatcher(autojunk=False)` | `DiffRender`, `SequenceMatcher` | opcodes, grouped opcodes, same match order |
| `highlight_python`, `PY_TOKEN_RE` | `PyHighlight` | string, comment, number, keyword, decorator, def/class name in regex order |
| `tool_args_text`, `tool_args_valid`, `action_signature` | `ToolArgs` (toolargs.h) | |
| `json.loads/dumps/str/repr` as used by Glow and tool args | `JsonValue` (jsonvalue.h) | ordered objects, duplicate key last wins, big ints kept as text, `ensure_ascii` dumps, float repr |
| `os.path.*` subset | `PyPath` (pypath.h) | join, normpath, abspath, basename, dirname, expanduser, realpath, listdir |
| `shlex.split(posix=True)` | `Shlex` (shlex.h) | |
| `glob.glob(recursive=True)`, `fnmatch`, `os.walk` | `PyGlob` (pyglob.h) | Python 3.14 glob algorithm |
| `hashlib.sha256`, `base64.b64encode` | `Digest` (digest.h) | |
| `str` helpers | `Utf8` (utf8.h) | strip, split, splitlines, expandtabs, lower, rjust, slices |
| http (urllib flows) | `HttpGet/HttpPost/HttpRequest/HttpPostSse` (http.h) | pre-existing libcurl layer, not touched in this milestone |

`Glow` (describe_call, agent rows, status bar) lives in `src/shell` and is documented in [shell.md](shell.md).

## Gaps

- Non-ASCII classification (`IsWord`, `IsAlnum`, `Lower`) is table-light; exact for ASCII and the common European
  and Cyrillic ranges, approximate elsewhere.
- `http.h` is the earlier libcurl wrapper; tai.py's HTTP flows live in the llm and agent modules, which own them.