From 24caef9ce4a8f363287511ab17f45fce16a99872 Mon Sep 17 00:00:00 2001 From: retoor Date: Sun, 25 Oct 2026 05:06:43 +0100 Subject: [PATCH] taicpp: spec and wiring docs for the misc modules Co-Authored-By: Claude Sonnet 5.5 --- taicpp/docs/misc.md | 79 ++++++++++++++++++++++++++++++++++++++ taicpp/docs/wiring/misc.md | 36 +++++++++++++++++ 2 files changed, 115 insertions(+) create mode 100644 taicpp/docs/misc.md create mode 100644 taicpp/docs/wiring/misc.md diff --git a/taicpp/docs/misc.md b/taicpp/docs/misc.md new file mode 100644 index 0000000..ae1a88e --- /dev/null +++ b/taicpp/docs/misc.md @@ -0,0 +1,79 @@ +# misc modules: skills, sched, telegram, rag, install (parity spec) + +Source of truth: `../tai.py`. Every class here is a literal port; method names keep the Python names so the +parity ledger matches them. Behavior is proven by `tests/misc_diff.py` (18 scenarios, byte-identical +transcripts plus file trees against the real tai.py). Build: `make -f tests/misc.mk all`, run: +`python3 tests/misc_diff.py`. + +## Shared helpers (src/skills, namespace detz::misc) + +| tai.py construct | C++ | notes | +|---|---|---| +| json.dumps / json.loads | `Json` | ensure_ascii, ", " / ": " separators, insertion-ordered objects, big ints keep digits, JSONDecodeError text | +| datetime.fromisoformat, astimezone, isoformat | `DateTime` | CPython port: week dates, 24:00, stray char before offset, gap/fold local times | +| str.splitlines / strip / len / slice, short_error | `PyStr` | code point semantics, Unicode whitespace and Nd digits (`PyUnicode`, generated table) | +| os / shutil / subprocess | `PyFs`, `PyProc` | universal newlines on text reads, OSError text `[Errno N] ...: 'path'` | +| hashlib.sha256 | `Sha256` | | +| LOG.event / LOG.context | `LogSink` | port, default drops | +| normalize_tag(s) | `TagNames` | | + +## skills (src/skills) vs skills.md + +| tai.py | C++ | +|---|---| +| parse_skill_front, parse_skill_file, skill_scope_dir, default_skill_scope, skill_extra_files, skill_install_meta_path, load/save_skill_install_meta, is_external_skill_root, skill_provenance_note, discover_skills, skill_catalog, create_skill_prompt | `SkillFiles` | +| BUNDLED_SKILLS, BUNDLED_SKILL_UPSTREAM, SKILL_BLUEPRINTS | `BundledData` (generated `bundled_data.cpp`, byte-identical; regenerate with `python3 -I src/skills/gen_bundled.py`) | +| bundled_skill_digest, bundled_skill_pristine, write_bundled_skill, remove_bundled_skill, op_skills, skills line of status_lines | `BundledSkills` | +| Tools.run_load_skill, run_create_skill, find_skill_root, resolve_skill_source, run_install_skill | `SkillTool` (+ `SkillHost` port, zlib `ZipArchive`) | + +## sched (src/sched) vs scheduler section of tai.py + +| tai.py | C++ | +|---|---| +| parse_schedule_at, parse_duration, format_delay, local_display, now_iso | `ScheduleTime` | +| Store.add_schedule / remove_schedule / list_schedules | `ScheduleVault` (+ `SealPort`, `TagPort`) | +| format_schedule_lines, note_schedule_result, watch_schedule, schedule_running, scheduler_tick, run_scheduler_loop, start_scheduler | `Scheduler` (+ `AgentPort`, `StopFlag`) | +| Tools.run_schedule / run_unschedule / run_schedules, normalize_tags | `ScheduleTool` (+ `ScheduleHost`) | + +## telegram (src/telegram) + +| tai.py | C++ | +|---|---| +| class Telegram | `Telegram` (+ `TelegramTransport`, libcurl `CurlTransport`) | +| telegram_env_path, load_telegram_token, telegram_send, telegram_transcribe, handle_telegram_update, _handle_telegram_update_inner, run_telegram_bot | `TelegramBot` (+ `TelegramAgent`, `BoxPort`) | + +Wire format (request bodies, paths, headers) is compared against a scripted local Bot API. + +## rag (src/rag) + +| tai.py | C++ | +|---|---| +| chunk_lines, collapse_repeated_lines, Store.add_chunks / get_chunks_by_ids | `Chunker`, `ChunkStore` | +| fts_escape, fts_match, fts_match_or, fts_match_prefix, fts_match_variants, fts_rank, Store.fts_search | `FtsQuery`, `FtsSearch` | +| class RetrievalEval, the `--eval-retrieval` body | `RetrievalEval` (`run_eval`) | +| Tools.run_search | `RagTool` (+ `RagHost`) | + +`--stats-rag` does not exist in tai.py (no such argument); nothing is ported for it. + +## install (src/install) + +| tai.py | C++ | +|---|---| +| rc block helpers, hook_present and friends | `RcBlocks`, `Installer::hook_present` etc. | +| unit_dir, service_state, activate_service, remove_service_unit | `Systemd` | +| Install, venv_dir, venv_python, ensure_venv, script_version, shell_rc_path, shell_present | `Install` | +| op_binary ... op_skills, status_lines, vault_line, install_report, install_self, install_telegram, uninstall_telegram, Tools.run_install | `Installer` (+ `ContainerPort`, `InstallIo`) | +| next_version, release_bump, atomic_write, sealed_write, quarantine, backups_dir, prune_backups, snapshot_self, backup_has_digest, ensure_self_backup, do_release, capped_text, trunc_marker, read_capped, Tools.run_release | `SelfRelease`, `ReleaseTool` | +| Store.audit_event / audit_history / audit_get / audit_search / tag_counts, Tools.run_audit / run_restore / run_tags | `AuditVault`, `AuditTool` | + +## Known gaps + +- Sealed vaults: the Seal crypto and the sealed events memdb belong to the store module; sealed paths are + reachable through `SealPort`/`MemEventsPort` but only the passphrase-less seal is verified against tai.py. +- `Store.upsert_file_record`, `delete_file_record`, canonical tag folding (`singular_noun`) are store-module work + (`FileRecordPort`, `TagPort`); the tests use a plain implementation. +- Box restore (`box_restore`), container probes and `ensure_box` are reached through ports; fakes are verified, + the real podman path is not. +- urllib error text for network failures other than HTTP status, timeout, DNS and refused is approximated. +- JSON numbers beyond int64 keep exact digits; floats beyond that use double. +- Copying a native binary in `op_binary` cannot read a version from it; set `InstallContext::version_reader`. diff --git a/taicpp/docs/wiring/misc.md b/taicpp/docs/wiring/misc.md new file mode 100644 index 0000000..806dc0e --- /dev/null +++ b/taicpp/docs/wiring/misc.md @@ -0,0 +1,36 @@ +# wiring: hooks the integrator must apply for the misc modules + +Do not edit these modules; bind them. Include dirs: `src/skills src/sched src/telegram src/rag src/install` +(file names are unique across them). Link: `-lsqlite3 -lcurl -lz -lpthread`. Sources: add the five dirs to +`ALL_SRCS` (the Makefile `find src -name '*.cpp'` already picks them up). `tests/misc.mk` is the standalone build +and `tests/misc_diff.py` the differential test; fold them into `make test`. + +## CLI flags (src/cli, main.cpp) + +| flag | binding | +|---|---| +| `--install` | `Installer::install_self()`; context: `script_path` = path of the running binary, `script_needs_interpreter=false`, `version_reader` for the native binary | +| `--install-telegram` | `Installer::install_telegram()` | +| `--uninstall-telegram` | `Installer::uninstall_telegram()` | +| `--telegram` | after boot: `TelegramBot(agent_port, nullptr, &box, log).run_telegram_bot()`; default profile `telegram` | +| `--scheduler` | print `scheduler online, press Ctrl-C to stop`, then `Scheduler::run_scheduler_loop(StopFlag)` | +| `--eval-retrieval` | `RetrievalEval::run_eval(db, FtsSearch, profile, sealed)`, print, exit 0 | +| `--stats-rag` | no tai.py counterpart; leave unbound or remove from the parser | +| `--update-models` | not in this area | + +## Ports to implement on the other side + +- `AgentPort` (sched): running_agents, spawn_agent (throw `AgentsFull`), poll_agent. `ScheduleHost`, `SkillHost`, + `RagHost`, `AuditHost`, `ReleaseHost`: the Tools object supplies approval, profile, store and skills refresh. +- `SealPort`/`TagPort` over the foundation Store; register the `tai_dec` SQL function when sealed (`implicit_pairs`). +- `ContainerPort` (install) and `BoxPort` (telegram): container engine, box state, ensure_box, box_transcribe. +- `LogSink`: forward to the DebugLog (`event`, `set_context`, `clear_context`). +- `TelegramAgent`: profile, bot, home, reset_history, run_turn(capture=True), start_scheduler. + +## Tool table + +Register handlers: `load_skill`, `create_skill`, `install_skill` -> `SkillTool`; `schedule`, `unschedule`, +`schedules` -> `ScheduleTool`; `search` -> `RagTool`; `audit`, `restore`, `tags` -> `AuditTool`; `release` -> +`ReleaseTool`; `install` -> `Installer::run_install` (refresh skills after a non-status skills action). +At boot call `SelfRelease::ensure_self_backup`; system prompt uses `SkillFiles::skill_catalog`. +`INSTALL_TARGETS` is `InstallConsts::INSTALL_TARGETS()`; the `skills` target is `Installer::op_skills`.