orchestration: sub agents render in live panels on the multiplexer screen

Feed panes in the multiplexer (vpane_* control commands, main-vertical layout, header row),
src/orch (board, panel, formatter, transport, CAP adapter), Agent observer and cancel flag,
supervision (idle warn/cancel, user close cancels, soft then hard stop), CAP answers collected
through the spill path. Research archived under background-work-limits.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-25 18:58:22 +01:00
co-authored by Claude Sonnet 5.5
parent 636886201b
commit ec9c471780
83 changed files with 3604 additions and 38 deletions
+6 -5
View File
@@ -7,10 +7,10 @@
| Metric | Value |
|---|---|
| Subjects | 15 |
| Total source documents | 921 |
| — Tier 1 (Source Dossier, independently analyzed) | 277 |
| — Tier 2 (Source Citation Card, listed only) | 644 |
| Total conclusion documents | 36 |
| Total source documents | 967 |
| — Tier 1 (Source Dossier, independently analyzed) | 282 |
| — Tier 2 (Source Citation Card, listed only) | 685 |
| Total conclusion documents | 38 |
| Archive opened | 2026-10-09 |
| Last updated | 2026-10-25 |
@@ -28,7 +28,7 @@
| [`free-model-discovery`](free-model-discovery/) | 4 (4 / 0) | 1 | 2026-10-12 | 2026-10-12 | Bringing tai's free model lists up to date from live endpoints (zen, openrouter, devplace, pollinations); live discovery replaces hard-coded lists |
| [`agent-goal-verification`](agent-goal-verification/) | 43 (11 / 32) | 3 | 2026-10-18 | 2026-10-18 | Validating whether the GoalVerifier subagent plus strict JSON verdicts plus approval hardening scales and is solid; kept all three as shipped with stated conditions |
| [`prompt-context-scaling`](prompt-context-scaling/) | 34 (10 / 24) | 3 | 2026-10-18 | 2026-10-18 | Scaling prompts past the context window and reusing RAG/graph for more; tai already matches the converged arsenal, giant prompts should spill to retrievable records (specified, not built), full GraphRAG and memory vendors rejected |
| [`background-work-limits`](background-work-limits/) | 10 (4 / 6) | 2 | 2026-10-18 | 2026-10-18 | Bounding parallel subagents and hardening the scheduler; parallel cap 8 with reject-loud, per-tick fires 4, one live run per schedule via lease, stated misfire policy; implemented same day |
| [`background-work-limits`](background-work-limits/) | 56 (9 / 47) | 4 | 2026-10-18 | 2026-10-25 | Bounding parallel subagents and hardening the scheduler; parallel cap 8 with reject-loud, per-tick fires 4, one live run per schedule via lease, stated misfire policy; implemented same day; extended 2026-10-25 with sub agent panels on the multiplexer and supervision/collection (agent teams, Temporal timeouts, claim check, Celery limits, tmux layouts, terminal streaming), 2 conclusions, implemented the same day |
| [`sandbox-resource-caps`](sandbox-resource-caps/) | 21 (3 / 18) | 1 | 2026-10-18 | 2026-10-18 | Capping the agent sandbox box CPU/memory/pids after the coverage audit found it uncapped; static trio 2g/2.0/256 at create, best-effort update for existing boxes, pids via recreation; implemented same day |
| [`cap-protocol`](cap-protocol/) | 83 (34 / 49) | 4 | 2026-10-18 | 2026-10-25 | Supporting CAP (CLI Agent Protocol) as agent and orchestrator: full read of core + coding profile, quickstart, STATUS, all 5 manifests; selective A2A/ACP wire reads; decided phased rollout without gRPC plus dual A2A spelling; extended 2026-10-25 with model handling, model-as-tool and skill-management research (ACP config options, MCP sampling, agents-as-tools, advisor tools, subagent and skill docs, RouteLLM), 2 recommendation-only conclusions, nothing built |
| [`mcp-client-support`](mcp-client-support/) | 106 (38 / 68) | 3 | 2026-10-18 | 2026-10-18 | Designing tai's MCP client: wire spec (stdio + Streamable HTTP on the 2025-11-25 baseline), pasted-config/GitHub-URL resolver, and the trust/approval model; decided minimal stdlib client, normalizer plus resolver agent, untrusted-until-consent with description pinning, no OAuth in v1, nothing built yet |
@@ -64,6 +64,7 @@
| 2026-10-18 | `mcp-client-support` | New subject: 106 sources (38 Tier 1 / 68 Tier 2) + 3 conclusions, covering the MCP wire spec (lifecycle, transports, tools, utilities, 2024-11-05 legacy, versioning, changelog), client config ecosystems (Claude Code, VS Code, Cursor, Codex TOML, mcp-remote, Smithery, registries, Glama), and security/operations (authorization draft, trust model, poisoning, rug pulls, production ops) via 3 gather subagents plus first-hand fetches of the load-bearing spec, client-config, and OWASP pages. Decisions: minimal stdlib client on the 2025-11-25 baseline (stdio + Streamable HTTP, no SDK, no OAuth in v1); deterministic normalizer plus resolver subagent for pasted configs and GitHub URLs with live-handshake verification; untrusted-until-consent trust model with description-hash pinning and approval tiers. Nothing built yet. Counts verified directly against the written files via `research/audit.py` (PASS). |
| 2026-10-25 | `fuzzy-matching-selectors` | New subject: 65 sources (12 Tier 1 / 53 Tier 2) + 2 conclusions, from 10 searches and 14 fetches on fuzzy string matching (fzf algo.go and man page, fzy ALGORITHM.md and match.c, Sublime fuzzy match via an AsciiDoc copy because the original blog returned 403, clangd FuzzyMatch, pg_trgm docs, RapidFuzz WRatio source, Jaro-Winkler and Damerau-Levenshtein articles, difflib docs, CPython traceback suggestions). Decisions: one generic `detz::FuzzyMatcher` (subsequence DP with fzf constants + restricted Damerau-Levenshtein + per-token typo + trigram Jaccard, deterministic shorter-then-earlier ties) and a fuzzy catalog resolution order with floor 0.3. Implemented the same session in taicpp (src/util/fuzzy.*, src/cap/cap_model_catalog.cpp, tests/fuzzy_test.cpp). Carried gaps: VS Code fuzzyScorer.ts never retrieved; fzy constants not retrieved (fzf's used); no measurement on a real model-id corpus; the Jaro-Winkler prefix-inflation claim rests on a search listing only. |
| 2026-10-25 | `cap-protocol` | Extended with 59 sources (11 Tier 1 / 48 Tier 2) + 2 conclusions (`cap-model-handling-practice`, `agent-delegation-tool-plus-skill`), from 10 searches and 15 fetches on how CAP and neighboring protocols treat models and cost, and whether delegation should be a tool plus a skill. Both conclusions are recommendations pending retoor, not decisions (their Decided by field says so). Two existing dossiers (`cap-core-v1`, `cap-profile-coding-v1`) were forward-corrected with model-handling findings. Carried gaps: Claude Code sub-agent and skills pages read only to character 100000; advisor benefit rests on one unverified transcript; the A2A searches found no model metadata field; CAP STATUS and the OpenAI handoffs page were not re-read. Nothing built. |
| 2026-10-25 | `background-work-limits` | Extended with 46 sources (5 Tier 1 / 41 Tier 2) + 2 conclusions (`sub-agent-panels-on-the-multiplexer`, `sub-agent-supervision-and-collection`), from 8 searches and 6 fetches on how parallel agents are shown, scheduled, timed out, monitored and how their results are collected (Claude Code agent teams, Temporal activity timeouts, the claim-check pattern, Celery worker limits, tmux layouts, Kubernetes termination, terminal streaming, parallel-agent dashboards). Implemented the same session in taicpp (feed panes, `src/orch/`, supervision, CAP collection) and verified live. Carried gaps: the Kubernetes pod lifecycle page was opened but its termination and probe sections were not returned, so it is a Tier 2 card and its numbers are unverified; the tmux manual was read to character 100000 of 278260 (options unread); the 60 s and 300 s idle limits are static judgments, not measurements. |
## How to use this index
@@ -0,0 +1,28 @@
# Celery tips
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Celery tips |
| Source URL | https://snippets.aktagon.com/snippets/966-celery-tips |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Celery search; set a global timeout because the default is none.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# How I manage 10 Claude Code agents
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | How I manage 10 Claude Code agents |
| Source URL | https://artemxtech.substack.com/p/how-i-manage-10-claude-code-agents |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search; fan-out burns rate limits roughly in proportion to the number of agents.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# messaging integration (cloud design patterns reference)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | messaging integration (cloud design patterns reference) |
| Source URL | https://git.cynarski.dev/Awesome/awesome-copilot/raw/branch/main/skills/cloud-design-patterns/references/messaging-integration.md |
| Source type | other |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the claim-check search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,51 @@
# Claim Check pattern (Azure Architecture Center)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Dossier |
| Tier | 1 |
| Source title | Claim Check pattern (Azure Architecture Center) |
| Source URL | https://learn.microsoft.com/azure/architecture/patterns/claim-check |
| Source publisher / author | Microsoft |
| Source type | vendor docs |
| Source date (published) | 2026-08-31 |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | direct page fetch |
| Confidence | High |
| Status | Active |
## Summary
The claim-check cloud design pattern: store a large payload externally and pass only a reference through the messaging system.
## Key findings
- The sender writes the payload to an external store and sends a reference (the claim check); receivers fetch it when they need it; the messaging system never carries the payload.
- Apply it conditionally by size so small messages skip the extra hop; the schema must say whether a payload is inline or external.
- Define who owns payload deletion and coordinate retention with message expiry so a valid token never points at deleted data.
- Do not embed security tokens in the claim check; publish the token only after the payload write succeeded; deliveries can repeat so consumers must be idempotent.
- Include a content hash when consumers must verify integrity; the store must stay available as long as consumers need the payload; retrieval adds latency.
- Alternatives: compress, or split and aggregate the payload.
## Pros / strengths
A named, widely recognized pattern with the failure modes (orphans, dangling tokens, retention) spelled out.
## Cons / limitations / caveats
Written for message brokers; tai's 'broker' is the model context, but the economics (size limits, cost, degraded throughput) are the same.
## Relevance to tai
Names what tai's spill path already is: a long result is stored as a record and the conversation gets a key plus a head and tail summary (`spilled_view`). Confirms the conditional rule (only above `SPILL_LIMIT`) and the write-before-reference order. Retention of agent output records is the open point.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial dossier created. |
@@ -0,0 +1,28 @@
# How It's Built: Product Design Breakdown (pi-mono)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | How It's Built: Product Design Breakdown (pi-mono) |
| Source URL | https://forums.basehub.com/badlogic/pi-mono/1 |
| Source type | forum thread |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search; renders by coalescing, line diffing and synchronized output.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Celery 2.4 workers guide (archived)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Celery 2.4 workers guide (archived) |
| Source URL | https://docs.celeryq.dev/en/2.4-archived/userguide/workers.html |
| Source type | vendor docs |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Celery search; archived version of the time-limit text.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,50 @@
# Celery: Workers Guide (time limits, remote control)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Dossier |
| Tier | 1 |
| Source title | Celery: Workers Guide (time limits, remote control) |
| Source URL | https://docs.celeryq.dev/en/stable/userguide/workers.html |
| Source publisher / author | Celery project |
| Source type | vendor docs |
| Source date (published) | Unknown — not stated |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | direct page fetch |
| Confidence | High |
| Status | Active |
## Summary
The current Celery worker guide, read for time limits and revocation.
## Key findings
- Soft time limit raises `SoftTimeLimitExceeded` inside the task so it can clean up; the hard limit cannot be caught and force-terminates the task.
- Time limits work in the prefork and gevent pools; the gevent pool does not implement soft limits; limits do not work on platforms without SIGUSR1.
- `revoke` skips a task that has not started; it does not stop a running task unless `terminate` is set, which terminates the worker child (default signal TERM, configurable).
- `--max-tasks-per-child` and `--max-memory-per-child` replace a worker process after N tasks or a resident-memory ceiling, containing leaks.
- Autoscale keeps a minimum number of processes and grows to a maximum under load (`--autoscale=10,3`).
## Pros / strengths
Shows the soft-before-hard ordering and that cancelling a running job needs an explicit escalation.
## Cons / limitations / caveats
Process-pool specifics; time limits depend on signals, which tai's cooperative agents cannot rely on.
## Relevance to tai
Pattern for tai's cancel path: first a soft cancel the worker can honor (cancel flag read between steps, `cap.cancel` to a CAP agent), and only then the hard stop (closing the session after `ORCH_CANCEL_GRACE`). Replacing a worker after a bounded amount of work is the same idea as depth and step caps.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial dossier created. |
@@ -0,0 +1,53 @@
# Claude Code: Orchestrate teams of Claude Code sessions (agent teams)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Dossier |
| Tier | 1 |
| Source title | Claude Code: Orchestrate teams of Claude Code sessions (agent teams) |
| Source URL | https://code.claude.com/docs/en/agent-teams |
| Source publisher / author | Anthropic |
| Source type | vendor docs |
| Source date (published) | Unknown — not stated |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | direct page fetch |
| Confidence | High |
| Status | Active |
## Summary
The documentation of Claude Code's experimental agent teams: a lead session, teammates in their own context windows, in-process or split-pane display, a shared task list and mailboxes. It is the closest shipped precedent for showing parallel agents in separate panes.
## Key findings
- Two display modes: in-process (agent panel below the prompt; arrows select, Enter opens a transcript, x stops a teammate) and split panes (each teammate its own pane; click into a pane to talk to it). Split panes need tmux or iTerm2; the default mode is in-process and `auto` uses split panes only inside tmux.
- Split panes are not supported in VS Code's terminal, Windows Terminal or Ghostty; orphaned tmux sessions after a session ends are a documented problem (`tmux kill-session`).
- In the panel, idle rows hide 30 seconds after everything is idle, and more than three idle teammates collapse into one `N idle agents` row; working and failed teammates always keep their rows.
- Start with 3-5 teammates; token cost scales linearly with the number of teammates; 5-6 tasks per teammate keeps everyone busy; avoid two teammates editing the same file.
- Teammates do not inherit the lead's conversation; a teammate that finishes sends an idle notification carrying its final answer to the lead, and one that fails on an API error notifies the lead with the error text.
- Shutdown is cooperative: the lead sends a shutdown request, the teammate can approve or reject; shutdown can be slow because a teammate finishes its current call first.
- Teammates cannot spawn teammates (no nested teams); a teammate's permission prompts bubble up to the lead; a message from another agent is never treated as user consent.
- Model precedence for a teammate: the spawn prompt, then the subagent definition, then `CLAUDE_CODE_SUBAGENT_MODEL`, then the lead's model; before v2.1.251 the environment variable came first.
## Pros / strengths
Primary documentation with concrete display, cost and shutdown behavior, including the honest list of limitations.
## Cons / limitations / caveats
Experimental feature; the panel and mailbox mechanics are specific to Claude Code; split panes rely on an external multiplexer, which tai now has built in.
## Relevance to tai
Direct precedent for the layout and lifecycle: one pane per agent, working rows stay visible, idle rows fade, a cap on useful team size, nested teams forbidden, cooperative shutdown. tai takes the cap (8, panels limited by space), the fade (linger then close) and the no-nesting rule (workers are depth-limited), and does it on its own multiplexer so it needs neither tmux nor iTerm2.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial dossier created. |
@@ -0,0 +1,28 @@
# cmux: Claude Code Teams
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | cmux: Claude Code Teams |
| Source URL | https://cmux.com/de/docs/agent-integrations/claude-code-teams |
| Source type | vendor docs |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the agent-teams search; a third-party terminal showing teammates as native splits.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,54 @@
# Conclusion: Show every sub agent in a panel of the multiplexer, over a feed-pane protocol
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Conclusion |
| Tier | N/A |
| Date decided | 2026-10-25 |
| Decided by | retoor (instructed the design and its integration), advised by tai (agent, Claude Sonnet 5) |
| Status | Active |
| Confidence | Medium |
## Question / problem statement
How should the main application show the sub agents it runs (internal workers, CAP sessions) on the multiplexer screen, with smooth streaming, for one agent, two agents and more.
## Conclusion
The multiplexer gets feed panes (no process; text written through its control socket; row 0 is a header). The orchestrator holds one board with one panel per sub agent. Layout is the tmux `main-vertical` preset: the main pane stays left at full height (55 percent wide) and sub agents stack equally on the right; 1 agent is a vertical split, 2 are two stacked blocks, up to what fits at 6 rows each (a ninth is refused loudly and runs without a panel). Streaming goes through a coalescing path: producers append to a per-panel buffer, one board thread writes all panels in one round trip every 25 ms, the server renders at most every 12 ms. Text is word-wrapped using the pane's real width and sanitized to SGR only. Finished panels linger 20 s and close, and the column rebalances. Closing a pane cancels its agent. Measured with three agents streaming 500 tokens/s each: about 64 frames/s, median 12.9 ms between frames, p90 25 ms.
## Supporting sources
- [claude-code-agent-teams-docs](../claude-code-agent-teams-docs.md): split panes per teammate, idle rows fading, team size 3-5, no nested teams, cooperative shutdown, tmux/iTerm2 dependence.
- [man7-tmux-man-page-layouts](../man7-tmux-man-page-layouts.md): the `main-vertical` preset.
- Tier 2: terminal streaming sources (coalescing and line diffing, synchronized output, event-driven versus fixed-rate rendering) and the parallel-agent dashboards (worktree per agent, state-grouped views).
## Alternatives considered and rejected
- Use tmux or iTerm2 as Claude Code does: adds an external dependency and nothing for users without it; tai has its own multiplexer.
- A viewer process per pane tailing a file or socket: one more process per agent, a second protocol, and slower than writing into the server's own screen.
- Per-token control-socket calls: up to 500 round trips a second per agent; coalescing at 25 ms caps it at 40 a second in total.
- Tabs instead of panes: hides agents, the opposite of the request.
- A fixed 60 fps timer: change-driven coalescing was the better-supported approach in the sources; the 12 ms render interval already bounds the frame rate.
## Risks / open questions
- Narrow terminals: the main pane gets 55 percent and the REPL's inline rows wrap earlier; a window under 41 columns gets no panels.
- A user who drags the layout by hand loses it at the next panel open or close (the preset is re-applied).
- tai as a CAP agent sends its transcript after the turn with generic tool names, so its panel is not token-live; other agents stream.
- Bare `tai` is not wrapped in the multiplexer automatically; only the shell hook does that.
- tmux and plain terminals show nothing (a tail-file transport would close this).
## Action taken
Implemented the same session in taicpp: `MplexVpanes`, `src/orch/`, `Agent::observer`, `CapMention`; tests `orch_test`, `orch_mplex_e2e.py`; verified live with the real REPL, real `fork` workers and a CAP agent. Spec: docs/orchestration.md.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial conclusion recorded. |
@@ -0,0 +1,55 @@
# Conclusion: Supervise sub agents with three clocks and collect results through the existing claim check
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Conclusion |
| Tier | N/A |
| Date decided | 2026-10-25 |
| Decided by | retoor (instructed the design and its integration), advised by tai (agent, Claude Sonnet 5) |
| Status | Active |
| Confidence | Medium |
## Question / problem statement
How should running sub agents be scheduled, monitored, timed out and cancelled, and how should their results come back without flooding the parent's context.
## Conclusion
Three clocks and a staged stop. (1) A wall-clock limit on every job: `fork` 30..3600 s, a CAP turn at `CAP_TURN_TIMEOUT`. (2) A heartbeat: every event is a heartbeat; 60 s without one marks the panel idle, 300 s cancels the job. (3) The user: closing the pane cancels. Stop is soft first (the worker stops at its next step; a CAP agent gets `cap.cancel`), then hard (a CAP session that ignores it is closed after 5 s). Parallelism stays capped at 8 live workers with a loud refusal. Supervision runs on the board's clock whether or not a panel is shown, and a failing display never stops supervision. Results use the claim-check path the system already has: a long answer (over 6000 characters) is stored as a record and the parent receives the head, the tail and the record id; `poll` does this for workers, `Agent::run_agent` for CAP answers. The panel is only a view.
## Supporting sources
- [temporal-blog-activity-timeouts](../temporal-blog-activity-timeouts.md): always set start-to-close; heartbeat finds dead workers early; bound retries by schedule-to-close.
- [celery-workers-guide](../celery-workers-guide.md): soft limit before hard; cancelling a running job needs an explicit escalation.
- [azure-claim-check-pattern](../azure-claim-check-pattern.md): store the payload, pass a reference, conditional by size, write before reference, define retention.
- [claude-code-agent-teams-docs](../claude-code-agent-teams-docs.md): cooperative shutdown, token cost scales with agent count, failed teammates reported with the error.
- Existing in this subject: erlang-otp-supervisor, temporal-workflow-activity, go-semaphore (bounded parallelism, reject loud).
## Alternatives considered and rejected
- Wall clock only: a stalled agent would burn the whole limit before anyone noticed.
- Automatic retries of sub agents: agents are not idempotent (they write files and run commands) and Temporal itself warns about unbounded retries; the parent decides.
- Hard kill first: loses the partial answer; the soft stage returns what the agent had.
- Putting results into the pane and reading them back from the screen: the screen is a view with a scrollback cap, not a store.
- Kubernetes-style restart back-off: not applicable, nothing restarts a sub agent; its page could not be read, so it was not relied on.
## Risks / open questions
- The idle clock needs events: an agent that thinks silently for 300 s is cancelled although it works (the limit is a static guess, not a measurement).
- A hung model request inside an internal worker is only cut by its own wall-clock deadline; the cancel flag is read between steps.
- Output records of finished agents have no retention policy yet (the claim-check pattern asks for an owner of deletion).
- Cancel escalation closes the CAP session from a second thread; a race inside the driver on close is possible.
## Action taken
Implemented the same session: `OrchPanel` watchdog and cancel hook, `AgentRecord` cancel flag and panel, `CapMention` timeout/cancel/escalation, `Agent::run_agent` spill; tests in `orch_test`. Spec: docs/orchestration.md.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial conclusion recorded. |
@@ -0,0 +1,28 @@
# Kubernetes graceful shutdown with SIGTERM and preStop hooks
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Kubernetes graceful shutdown with SIGTERM and preStop hooks |
| Source URL | https://devopscube.com/kubernetes-pod-graceful-shutdown/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Kubernetes search; kubelet sends SIGTERM then SIGKILL after the grace period (default 30 s).
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# The 4 types of Activity timeouts in Temporal (DEV)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | The 4 types of Activity timeouts in Temporal (DEV) |
| Source URL | https://dev.to/temporalio/the-4-types-of-activity-timeouts-in-temporal-1p69 |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Temporal search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# How to Orchestrate Claude Code Agent Teams (fast.io)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | How to Orchestrate Claude Code Agent Teams (fast.io) |
| Source URL | https://fast.io/resources/claude-code-agent-teams-setup/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the agent-teams search; says in-process mode hides logs of several agents, tmux panes show them at once.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# flywheel compositor
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | flywheel compositor |
| Source URL | https://docs.rs/flywheel-compositor |
| Source type | library docs |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search; claims 60+ FPS without tearing.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Claude Code issue 37283: flicker inside tmux, use synchronized output
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Claude Code issue 37283: flicker inside tmux, use synchronized output |
| Source URL | https://github.com/anthropics/claude-code/issues/37283 |
| Source type | forum thread |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search; DECSET 2026 batching and tmux passthrough.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# wmux
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | wmux |
| Source URL | https://github.com/kevmtt/wmux |
| Source type | repository |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search; orchestrator plugin running agents in dependency-aware waves.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# openai-python issue 2737: SoftTimeLimitExceeded swallowed
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | openai-python issue 2737: SoftTimeLimitExceeded swallowed |
| Source URL | https://github.com/openai/openai-python/issues/2737 |
| Source type | forum thread |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Celery search; a broad `except Exception` swallowed the soft-limit exception.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# How to perform graceful pod termination in Kubernetes
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | How to perform graceful pod termination in Kubernetes |
| Source URL | https://gurubase.io/g/kubernetes/how-to-perform-graceful-pod-termination-in-kubernetes |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Kubernetes search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Celery tasks (Habr)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Celery tasks (Habr) |
| Source URL | https://habr.com/en/articles/770020 |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Celery search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Kubernetes graceful shutdown: handling SIGTERM
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Kubernetes graceful shutdown: handling SIGTERM |
| Source URL | https://jorijn.com/en/knowledge-base/kubernetes/troubleshooting/kubernetes-graceful-shutdown-sigterm-pod-termination/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Kubernetes search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Temporal Fundamentals Part III: Timeouts
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Temporal Fundamentals Part III: Timeouts |
| Source URL | https://keithtenzer.com/temporal/Temporal_Fundamentals_Timeouts/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Temporal search; a heartbeat throttle figure that was not verified.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# SIGTERM: Linux graceful termination, exit code 143
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | SIGTERM: Linux graceful termination, exit code 143 |
| Source URL | https://komodor.com/learn/sigterm-signal-15-exit-code-143-linux-graceful-termination/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Kubernetes search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Pod Lifecycle (Kubernetes docs)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Pod Lifecycle (Kubernetes docs) |
| Source URL | https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/ |
| Source type | vendor docs |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by opened, but the returned text stopped before the termination and probe sections, so nothing could be quoted; the Kubernetes figures (30 s grace, probe defaults, restart back-off) used in conversation are from general knowledge and unverified.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,48 @@
# tmux(1) manual page (preset layouts)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Dossier |
| Tier | 1 |
| Source title | tmux(1) manual page (preset layouts) |
| Source URL | https://man7.org/linux/man-pages/man1/tmux.1.html |
| Source publisher / author | tmux project |
| Source type | vendor docs |
| Source date (published) | Unknown — not stated |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | direct page fetch |
| Confidence | Medium |
| Status | Active |
## Summary
The tmux manual, read for the preset layouts. Only the first 100000 of 278260 characters were returned, so options (main-pane-width defaults, pane-border-status) were not read.
## Key findings
- `even-horizontal`: panes spread evenly left to right. `even-vertical`: spread evenly top to bottom.
- `main-vertical`: one main pane on the left, the others stacked on the right; the width of the left pane comes from the `main-pane-width` window option. `main-horizontal` is the same turned around with `main-pane-height`. `tiled`: as even as possible in rows and columns.
- tmux adjusts a layout to the current window size; a layout cannot be applied to a window with more panes than the layout was defined for.
## Pros / strengths
Names exactly the layout the feature needs (one main pane, the rest stacked) with an established name and behavior.
## Cons / limitations / caveats
Options section unread. tai's `main-pane-width` is an integer column count in this port (no percentage), so the percentage is computed by the orchestration code.
## Relevance to tai
The orchestration layout is `main-vertical` with the main width set to 55 percent of the window, built through the existing `MplexLayout::preset`, so it behaves like the layout users already know from tmux.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial dossier created. |
@@ -0,0 +1,28 @@
# Claim Check design pattern (Medium)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Claim Check design pattern (Medium) |
| Source URL | https://medium.com/@dmosyan/claim-check-design-pattern-603dc1f3796d |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the claim-check search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Anthropic Ships Agent View (NYU Shanghai)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Anthropic Ships Agent View (NYU Shanghai) |
| Source URL | https://rits.shanghai.nyu.edu/ai/anthropic-ships-agent-view-a-multi-session-dashboard-for-claude-code/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search; sessions grouped by state with needs-input first.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Monotonic clock discipline for LLM streaming UIs
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Monotonic clock discipline for LLM streaming UIs |
| Source URL | https://omr.it.com/blog/monotonic-clock-discipline-llm-streaming-uis/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Token streaming UI and concurrent rendering
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Token streaming UI and concurrent rendering |
| Source URL | https://omr.it.com/blog/token-streaming-ui-react-concurrent-rendering/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search; batch three to five tokens before flushing to state.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Variable rate shaping for LLM token emission
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Variable rate shaping for LLM token emission |
| Source URL | https://omr.it.com/blog/variable-rate-shaping-llm-token-emission-control/ |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# How to Build Claim Check Pattern
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | How to Build Claim Check Pattern |
| Source URL | https://oneuptime.com/blog/post/2026-01-30-claim-check-pattern/markdown |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the claim-check search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# How to Configure Pod Termination Grace Period
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | How to Configure Pod Termination Grace Period |
| Source URL | https://oneuptime.com/blog/post/2026-01-25-pod-termination-grace-period/markdown |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Kubernetes search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Temporal activity timeouts (Plushcap)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Temporal activity timeouts (Plushcap) |
| Source URL | https://www.plushcap.com/content/temporal/blog/temporal-activity-timeouts |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Temporal search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Claim check (SAP blog)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Claim check (SAP blog) |
| Source URL | https://blogs.sap.com/?p=73482 |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the claim-check search; describes the Enterprise Integration Patterns original.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# tmux layout-set.c commits (tag 3.3)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | tmux layout-set.c commits (tag 3.3) |
| Source URL | https://git.sharpetronics.com/sharpetronics/tmux/commits/tag/3.3/layout-set.c |
| Source type | repository |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the tmux layout search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Pane by dcouple
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Pane by dcouple |
| Source URL | https://www.sourcepulse.org/projects/32313131 |
| Source type | other |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search; terminal-first manager, one pane per workspace.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# The 4 Types of Activity timeouts in Temporal (swyx)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | The 4 Types of Activity timeouts in Temporal (swyx) |
| Source URL | https://swyx.io/temporal-activity-timeouts |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Temporal search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# The Tao of tmux: Windows
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | The Tao of tmux: Windows |
| Source URL | https://tao-of-tmux.readthedocs.io/en/stable/manuscript/06-window.html |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the tmux layout search; says select-layout must be run again to reset proportions after splits.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,50 @@
# Understanding the 4 types of Activity timeouts in Temporal
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Dossier |
| Tier | 1 |
| Source title | Understanding the 4 types of Activity timeouts in Temporal |
| Source URL | https://temporal.io/blog/activity-timeouts |
| Source publisher / author | Temporal Technologies |
| Source type | blog post |
| Source date (published) | Unknown — not stated |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | direct page fetch |
| Confidence | High |
| Status | Active |
## Summary
Temporal's explanation of the four timeouts that bound a unit of work: schedule-to-start, start-to-close, schedule-to-close and heartbeat. The reference design for detecting dead or stalled workers.
## Key findings
- Start-To-Close limits one execution; 'We recommend ALWAYS setting this'; it must exceed the longest possible run; it is how a crashed worker is noticed, because the server cannot see it otherwise.
- Heartbeat limits the time between heartbeats; for long activities it detects a dead or stalled worker sooner than Start-To-Close; if no heartbeat timeout is set, recorded heartbeats are discarded.
- Schedule-To-Close limits total time including retries and is the recommended way to bound retrying rather than tweaking maximum attempts; without a limit Temporal 'will retry a failing Activity Execution for up to 10 years'.
- Schedule-To-Start limits how long a task may wait in a queue; 'This is rarely needed' - monitor the schedule-to-start latency metric instead; it does not trigger a retry.
- A timeout triggers a retry according to the retry policy and increments the attempt count.
## Pros / strengths
Small, exact, and states which timeout to set and which to monitor instead.
## Cons / limitations / caveats
Designed for a server with a task queue; tai has no queue between spawn and start, so schedule-to-start is moot.
## Relevance to tai
Supports tai's three clocks: a wall-clock limit on every job (fork timeout 30..3600 s), a heartbeat (every event; 60 s idle shows, 300 s idle cancels), and a bound on retries (none: tai does not retry sub agents automatically).
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Initial dossier created. |
@@ -0,0 +1,28 @@
# Temporal docs: activity timeouts
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Temporal docs: activity timeouts |
| Source URL | https://docs.temporal.io/blog/activity-timeouts |
| Source type | vendor docs |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Temporal search; mirror of the blog read as Tier 1.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Temporal: what is a heartbeat timeout
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Temporal: what is a heartbeat timeout |
| Source URL | https://docs.temporal.io/concepts/what-is-a-heartbeat-timeout |
| Source type | vendor docs |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Temporal search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# dmux workflows
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | dmux workflows |
| Source URL | https://tessl.io/registry/skills/github/affaan-m/ECC/dmux-workflows |
| Source type | other |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# twinki: event-driven versus fixed 60 fps rendering
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | twinki: event-driven versus fixed 60 fps rendering |
| Source URL | https://kiro-cli.mengtge.people.aws.dev/02_twinki/ |
| Source type | other |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the terminal streaming search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# tmux select-layout (Waylon Walker)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | tmux select-layout (Waylon Walker) |
| Source URL | https://waylonwalker.com/tmux-select-layout |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the tmux layout search; key bindings for each preset.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Terminating with grace (Kubernetes)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Terminating with grace (Kubernetes) |
| Source URL | https://wdd.js.org/container/terminating-with-grace |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the Kubernetes search.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# Webmux (Windmill)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | Webmux (Windmill) |
| Source URL | https://www.windmill.dev/blog/webmux |
| Source type | blog post |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search; web dashboard on the workmux model.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# workmux (algolia index)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | workmux (algolia index) |
| Source URL | https://docsearch.algolia.com/mcp/docs/repo/raine/workmux |
| Source type | other |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the parallel-agent search; worktree per task paired with a tmux window plus a dashboard.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
@@ -0,0 +1,28 @@
# tmux layout-set.c history (OpenBSD)
## Metadata
| Field | Value |
|---|---|
| Institute | Molodetz Institute — Research & Retrieval Division |
| Subject | background-work-limits |
| Document type | Source Citation Card |
| Tier | 2 |
| Source title | tmux layout-set.c history (OpenBSD) |
| Source URL | https://git.zx2c4.com/wireguard-openbsd/log/usr.bin/tmux/layout-set.c?showmsg=1 |
| Source type | repository |
| Date researched | 2026-10-25 |
| Researched by | tai (agent), model: Claude Sonnet 5, on behalf of retoor |
| Research method | Search aggregator result listing (not independently opened or quoted) |
| Confidence | Not independently verified |
| Status | Active |
## Why it's here
Surfaced by the tmux layout search; commit history of the preset layouts and main-pane-width.
## Changelog
| Date | Change |
|---|---|
| 2026-10-25 | Logged as consulted-but-not-analyzed. |
+1
View File
@@ -1,6 +1,7 @@
# Changelog
## [Unreleased] - 2026-10-25
- Orchestration on the multiplexer screen ([docs/orchestration.md](docs/orchestration.md)): every sub agent (`fork` worker, CAP session, `@agent` mention) gets a live, word-wrapped panel; the main pane stays left at full height and sub agents stack on the right (1, 2, 3 ... up to what fits at 6 rows each). New: feed panes in the multiplexer (`vpane_*` control commands, `MplexVpanes`), `src/orch/` (`OrchBoard`, `OrchPanel`, `OrchFormatter`, `OrchTransport`, `OrchCapAdapter`), `Agent::observer` and cancel flag, supervision (idle warn 60 s, idle cancel 300 s, soft cancel then close after 5 s, closing the pane cancels the job), CAP answers collected through the spill path. Tests: `build/orch_test`, `tests/orch_mplex_e2e.py` with `build/orch_live`. Measured: about 64 frames/s, median 12.9 ms between frames, p90 25 ms with three agents streaming 500 tokens/s each.
- Generic `detz::FuzzyMatcher` (src/util/fuzzy.*, docs/fuzzy.md, tests/fuzzy_test.cpp): subsequence DP with fzf constants, Damerau-Levenshtein, per-token typo and trigram signals; `@agent` / `@model` mentions now always pick the most related candidate (floor 0.3, ties shorter then earlier, a note shows what was resolved). Research: research/fuzzy-matching-selectors.
- `@agent`, `@agent:model` and bare `@model` mentions resolve against CAP agents via the new `CapModelCatalog` (manifest `[models]` list or `--cap-models` probe); turns run through `CapMention`; failures print a per-agent report of status, binary and models. Test: `build/capmcp/cap_model_catalog_test`. Spec: docs/cap.md.
- Build and test integration after the parallel module ports. Not released: `make test` is not green (see below), so VERSION stays 1.44.2 and no tag was made.
+2
View File
@@ -88,6 +88,8 @@ $(eval $(call TESTBIN,$(BUILD)/capmcp/capmcp_probe,tests/capmcp_probe.cpp))
$(eval $(call TESTBIN,$(BUILD)/capmcp/capmcp_cli,tests/capmcp_cli.cpp))
$(eval $(call TESTBIN,$(BUILD)/capmcp/cap_model_catalog_test,tests/cap_model_catalog_test.cpp))
$(eval $(call TESTBIN,$(BUILD)/fuzzy_test,tests/fuzzy_test.cpp))
$(eval $(call TESTBIN,$(BUILD)/orch_test,tests/orch_test.cpp))
$(eval $(call TESTBIN,$(BUILD)/orch_live,tests/orch_live.cpp))
# misc (skills, sched, telegram, rag, install)
$(eval $(call TESTBIN,$(BUILD)/misc/misc_test,tests/misc_test.cpp,$(MISC_INC)))
$(eval $(call TESTBIN,$(BUILD)/misc/misc_driver,tests/misc_driver.cpp,$(MISC_INC)))
+1 -1
View File
@@ -90,7 +90,7 @@ hooks: `docs/wiring/capmcp.md`.
Model protocol (tai-specific extension, absent from the Python app): `[models] list = [...]` and `default` in the manifest, plus an optional probe, `[models] probe = ["bin", "--cap-models"]` or `capabilities.models = true` for `<binary> --cap-models`. Probe output: JSON array of ids or `{id|name|model}` objects, `{"models": [...], "default": "..."}`, or one id per line. Bounds: `CAP_MODEL_PROBE_TIMEOUT` 5 s, `CAP_MODEL_PROBE_MAX_BYTES` 1 MiB, `CAP_MODEL_MAX` 500 per agent. A failed probe is recorded on the entry, never thrown. Tai's own manifest is unchanged and declares no models.
Mention resolution (after bots, providers, backends, known model ids, `@auto`, exact agent, and file paths; so a real file is never shadowed): the typed text is fuzzy-matched with `FuzzyMatcher` ([fuzzy.md](fuzzy.md)). Order: exact agent name or model id; `head:tail` splits at the first `:`, head fuzzy to a ready agent, tail fuzzy to that agent's models (no floor, the agent already disambiguates); otherwise the whole text is ranked against every ready agent name and every model. Cross-namespace picks need a score of `CAP_FUZZY_FLOOR` (0.3); the best score wins and equal scores resolve shorter id then list order, never an error. Unusable agents (no binary, bad manifest) are considered only when no ready candidate reaches the floor and then report why. An agent without a model list passes the model through with a note. When the pick differs from the typed text a note line shows the resolution and score. With a prompt the turn runs on that agent (`CapMention`, one cached `CapSession` per agent/model/cwd, model sent via `startup.model_arg`); without a prompt the agent is pinned (`Agent::agent_pin`) until `@auto`. A miss prints the closest candidate with its score and the full per-agent report (status, binary, manifest, binding, models, default, probe error); an unknown tag appends the same report.
Mention resolution (after bots, providers, backends, known model ids, `@auto`, exact agent, and file paths; so a real file is never shadowed): the typed text is fuzzy-matched with `FuzzyMatcher` ([fuzzy.md](fuzzy.md)). Order: exact agent name or model id; `head:tail` splits at the first `:`, head fuzzy to a ready agent, tail fuzzy to that agent's models (no floor, the agent already disambiguates); otherwise the whole text is ranked against every ready agent name and every model. Cross-namespace picks need a score of `CAP_FUZZY_FLOOR` (0.3); the best score wins and equal scores resolve shorter id then list order, never an error. Unusable agents (no binary, bad manifest) are considered only when no ready candidate reaches the floor and then report why. An agent without a model list passes the model through with a note. When the pick differs from the typed text a note line shows the resolution and score. With a prompt the turn runs on that agent (`CapMention`, one cached `CapSession` per agent/model/cwd, model sent via `startup.model_arg`), in its own panel on the multiplexer screen, supervised (turn timeout, idle watchdog, closing the panel cancels it) and collected through the spill path ([orchestration.md](orchestration.md)); without a prompt the agent is pinned (`Agent::agent_pin`) until `@auto`. A miss prints the closest candidate with its score and the full per-agent report (status, binary, manifest, binding, models, default, probe error); an unknown tag appends the same report.
Gaps: no TUI completion for agents yet; `tai --cap-models` is not implemented on tai's own side.
+10
View File
@@ -98,3 +98,13 @@ Listed next to what would close them; the final status of each is recorded in th
tables, which may lag or lead CPython's `unicodedata` by a Unicode version.
- **TOML error texts** follow tomllib's wording; positions are reproduced, rare messages may differ.
- Anything else found while testing is added here, never hidden.
## Feed panes (taicpp extension, orchestration)
A pane can be a feed pane: no process, text written through the control socket, row 0 is a header
drawn by the renderer. Added for sub agent panels; specified in [orchestration.md](orchestration.md)
(commands `vpane_open`, `vpane_write`, `vpane_close`, `vpane_list`; caps `MPLEX_VPANE_*`). The
Python app does not have them, so they are outside the tables `mplex_spec_test` compares with
`mplex.md`. Code: `src/mplex/mplex_vpane.*`, `MplexPane::virtual_pane`, the header row in
`MplexRenderer::frame`, a feed pane hook in `MplexServer::discard_pane` and `tick`, and
`MplexServer::kill_pane` closing a window whose real panes are all gone.
+145
View File
@@ -0,0 +1,145 @@
# Orchestration: sub agents on the multiplexer screen (taicpp)
Status: implemented in taicpp, not in the Python app. Research: `research/background-work-limits`,
`research/cap-protocol` (agent teams, claim check, timeouts), `research/prompt-context-scaling`.
## The rule
The main application orchestrates. Every sub agent it runs, whatever kind (an internal `fork`
worker, a CAP session over any binding, a `@agent` mention), gets a panel on the main multiplexer
session screen while it works, fed live, and is supervised by one clock whether it is shown or not.
```
one sub agent two sub agents three
+-------+-------+ +-------+-------+ +-------+-------+
| | agent | | | agent1| | | agent1|
| main | 1 | | main +-------+ | main +-------+
| | | | | agent2| | | agent2|
| | | | | | | +-------+
+-------+-------+ +-------+-------+ | | agent3|
+-------+-------+
```
The main pane stays on the left at full height (`MPLEX_VPANE_MAIN_PERCENT` 55 percent wide), the
sub agents are stacked equally on the right (tmux `main-vertical`). Panels appear when an agent
starts and leave `ORCH_LINGER` seconds after it finishes, then the column rebalances. Outside a
multiplexer (plain terminal, tmux) nothing is drawn and everything else works unchanged. A bare
`tai` run is not wrapped automatically; the installed shell hook (`--plexer-wrap`) is the default
entry that puts tai inside the multiplexer.
## Architecture
| Layer | Class | File | Job |
|---|---|---|---|
| server | `MplexVpanes` | `src/mplex/mplex_vpane.*` | feed panes, layout policy, linger, orphan lease |
| server | `MplexPane::virtual_pane` | `src/mplex/mplex_pane.*` | pane without a process; row 0 is a header drawn by `MplexRenderer` |
| transport | `OrchTransport`, `OrchMplexTransport`, `OrchNullTransport` | `src/orch/orch_transport.*` | control-socket client, one round trip per write |
| core | `OrchBoard` | `src/orch/orch_board.*` | opens panels, coalesced flusher thread, watchdog clock |
| core | `OrchPanel` | `src/orch/orch_panel.*` | one agent's buffer, state, counters, cancel hook |
| view | `OrchFormatter` | `src/orch/orch_formatter.*` | events to sanitized ANSI text |
| adapter | `OrchCapAdapter` | `src/orch/orch_cap_adapter.*` | `cap.*` events to `OrchEvent` |
| adapter | `Agent::observer` | `src/agent/agent.*` | tool calls, results, streamed text of an internal worker |
| owners | `spawn_agent`, `CapMention` | `src/agent/registry.cpp`, `src/cap/cap_mention.*` | open a panel, wire cancel, finish it |
## Control protocol (multiplexer socket, JSON lines, taicpp extension)
| Command | Request | Reply data |
|---|---|---|
| `vpane_open` | `pane` (origin), `label`, `header`, `state`, `main_percent` | `{id, cols, rows}` |
| `vpane_write` | `ops: [{pane, data, header, state, done, linger}]` | `{closed: [ids]}` (panes that no longer exist: the user closed them) |
| `vpane_close` | `pane` | `true` |
| `vpane_list` | none | `[{id, state, header, cols, rows, closing}]` |
`read` and `capture-pane` work on a feed pane like on any pane. Keys typed into a feed pane go
nowhere. Killing it (`prefix x`) cancels the agent. If the process that owns the panes dies, each
pane shows `owner gone` after `MPLEX_VPANE_LEASE` seconds and closes after `MPLEX_VPANE_ORPHAN_LINGER`.
If the main pane exits, the feed panes are closed with its window.
## Streaming
Producers append formatted bytes to the panel under a mutex and never block on I/O. One board
thread drains every panel every `ORCH_FLUSH_INTERVAL` (25 ms) into a single `vpane_write`, so a
token stream costs at most 40 round trips a second however many agents run. The server feeds the
bytes into the pane's screen at once and renders at most one frame per `MPLEX_RENDER_INTERVAL`
(12 ms), diffed by row. Text is sanitized before it reaches the screen: only SGR colors, newline and
tab survive, so an agent cannot move the cursor, change the title or write outside its pane.
If the screen falls behind by `ORCH_PENDING_MAX` bytes the oldest bytes are dropped at a line end
and a visible `output skipped` line says so (loud skip, never a silent one).
## Scheduling, monitoring, timeouts
| Concern | Rule | Constant |
|---|---|---|
| Parallelism | at most 8 live workers (`AGENTS_MAX_PARALLEL`), refused loudly | `ORCH_PANELS_MAX`, `MPLEX_VPANE_MAX` |
| Space | a pane needs 6 rows (header plus 5); when the column is full the agent runs without a panel and `OrchBoard::last_error` says why | `MPLEX_VPANE_MIN_ROWS` |
| Wall clock | `fork` timeout 30..3600 s (default 600) ends the worker with `[time limit reached]`; a CAP turn ends at `CAP_TURN_TIMEOUT` | existing |
| Heartbeat | every event is a heartbeat; no event for 60 s shows `idle` in the header | `ORCH_IDLE_WARN` |
| Stall | no event for 300 s cancels the job | `ORCH_IDLE_KILL` |
| User close | closing the pane cancels the job | none |
| Cancel | soft first: the worker stops at its next step, a CAP session gets `cap.cancel`; a CAP agent that ignores it loses its session after the grace | `ORCH_CANCEL_GRACE` |
| Display failure | three failed writes in a row stop drawing; supervision continues | board |
| Header | `icon label state elapsed tools tokens`, colored by state (running, idle, done, failed, timeout, cancelled) | none |
These follow the practice the research found: a start-to-close timeout always set, a heartbeat for
long jobs so a dead worker is found before the wall clock (Temporal), soft stop before hard stop
(Celery, Kubernetes SIGTERM then SIGKILL), and a hard cap on parallel work (agent teams advice:
three to five, token cost scales with the number of agents).
## Data collection
Unchanged and shared: the panel is a view, the answer travels the claim-check path every big tool
result already uses. A result longer than `SPILL_LIMIT` (6000 characters) is stored as a record
(`mem:` id, chunked and indexed), and the parent conversation receives the head, the tail and the
record id (`spilled_view`); `record_read` pages the rest. `poll` does this for `fork` workers,
`Agent::run_agent` does it for CAP answers (`spill_or_truncate`). The panel footer states the size
and whether the answer was stored.
## Caps and floors (all static)
| Constant | Value |
|---|---|
| `ORCH_PANELS_MAX` | 8 |
| `ORCH_FLUSH_INTERVAL` | 0.025 s |
| `ORCH_MONITOR_INTERVAL` | 1.0 s |
| `ORCH_PENDING_MAX` | 262144 bytes |
| `ORCH_WRITE_MAX` | 32768 bytes |
| `ORCH_LINGER` | 20 s |
| `ORCH_IDLE_WARN` | 60 s |
| `ORCH_IDLE_KILL` | 300 s |
| `ORCH_CANCEL_GRACE` | 5 s |
| `ORCH_TASK_SHOW_MAX` | 300 characters |
| `ORCH_TOOL_ARGS_MAX` | 120 characters |
| `ORCH_TOOL_RESULT_LINES` | 3 |
| `ORCH_TOOL_RESULT_COLS` | 160 |
| `ORCH_LABEL_MAX` | 60 |
| `ORCH_CALL_TIMEOUT_MS` | 2000 |
| `MPLEX_VPANE_MAX` | 8 |
| `MPLEX_VPANE_MIN_ROWS` | 6 |
| `MPLEX_VPANE_MIN_COLS` | 20 |
| `MPLEX_VPANE_MAIN_PERCENT` | 55 |
| `MPLEX_VPANE_OPS_MAX` | 16 |
| `MPLEX_VPANE_WRITE_MAX` | 65536 bytes |
| `MPLEX_VPANE_HEADER_MAX` | 200 characters |
| `MPLEX_VPANE_LEASE` | 30 s |
| `MPLEX_VPANE_ORPHAN_LINGER` | 10 s |
| `MPLEX_VPANE_LINGER_MAX` | 300 s |
Switch: `TAI_ORCH_PANELS=0` draws nothing (supervision stays on).
## Verification
`tests/orch_test.cpp` (formatter, panel, board over a fake transport, CAP adapter, this page's
constant table), `tests/orch_mplex_e2e.py` (real server: geometry for 1, 2, 3 and 8 panels, content,
linger, rebalance, user close, orphan lease, main pane exit), and a live run of the real binary
inside the multiplexer with real `fork` workers and a CAP agent.
## Gaps
- tmux and bare terminals get no panels (a tail-file transport would close this).
- Keys typed into a feed pane are ignored; steering an agent from its pane is not built.
- The idle watchdog needs events; a CAP agent that streams nothing for 300 s is cancelled even if it works.
- A hung model request inside an internal worker is only bounded by the worker's own timeout; the
cancel flag is read between steps.
- Panels are not restored after the orchestrator restarts.
- tai as a CAP agent (`--cap-speak`) sends its transcript after the turn, with generic tool names, so a tai sub agent over CAP fills its panel at the end of the turn; agents that stream (stream-json, ACP) fill it live. An internal `fork` worker is live.
- Agent output records have no retention policy yet.
+45 -3
View File
@@ -467,6 +467,11 @@ bool Agent::ask_approval(const std::string& command, bool guidance) {
}
void Agent::show_call(const llm::ToolCall& call) {
observed_tool_ = call.name;
if (observer) {
std::string shown = call.name == "store_secret" ? std::string("[hidden]") : store->redact(llm::tool_args_text(call.arguments), profile);
observer(orch::OrchEvent::tool_start(call.name, shown));
}
if (quiet) { return; }
std::string preview_args = llm::tool_args_text(call.arguments);
if (call.name == "store_secret") {
@@ -509,6 +514,11 @@ void Agent::show_diff() {
}
void Agent::show_result(const std::string& result, double elapsed_ms) {
if (observer) {
bool failed = result == "denied by user" || py::startswith(result, "error:") || py::startswith(result, "[error]") ||
py::startswith(result, "[missing]");
observer(orch::OrchEvent::tool_end(observed_tool_, store->redact(result, profile), failed, static_cast<long long>(elapsed_ms)));
}
if (quiet) { return; }
show_diff();
if (tools.result_note.has_value() && !tools.result_note->empty()) {
@@ -609,7 +619,9 @@ std::string Agent::mention_error(const std::string& tag) {
for (const auto& item : store->list_bots(profile).items()) { names.push_back(sv(item.get("name"))); }
std::string out = "unknown bot @" + tag + " (available: " + py::join(names, ", ") + "; providers: " + py::join(llm::ProviderLabels(), ", ") +
"; @path references a file)";
try { out += "\n" + cap_mention().catalog().report(); } catch (...) {}
try { // with no CAP agents installed the message stays identical to the Python app's
if (!cap_mention().catalog().agentNames().empty()) { out += "\n" + cap_mention().catalog().report(); }
} catch (...) {}
return out;
}
@@ -624,7 +636,17 @@ std::string Agent::run_agent(const std::string& agent_name, const std::string& m
sel.ok = true;
sel.agent = agent_name;
sel.model = model;
std::string answer = cap_mention().ask(sel, rest, dir_key());
detz::cap::CapAnswer got = cap_mention().ask(sel, rest, dir_key());
// Collection is the claim-check path every big tool answer takes: a long answer is stored as a
// record and the conversation gets the head, the tail and the record id (spill_or_truncate).
std::string who = agent_name + (model.empty() ? "" : ":" + model);
std::string answer = got.ok ? spill_or_truncate(store, "output", "cap " + who + ": " + py::head(rest, 80), got.text, {"agent-output"}, profile)
: got.text;
if (got.panel) {
using S = orch::OrchPanel::State;
S st = got.ok ? S::Done : got.detail == "timeout" ? S::Timeout : got.detail == "cancelled" ? S::Cancelled : S::Failed;
got.panel->finish(st, std::to_string(got.text.size()) + " chars" + (answer.size() < got.text.size() ? ", stored as a record" : ""));
}
if (loud) { print_out(answer); }
store->log_event(profile, "user", "message", store->redact(original, profile));
messages.push(msg("user", original));
@@ -811,7 +833,10 @@ std::string Agent::run_turn_inner(const std::string& in_text, bool capture, int
std::vector<std::string> backends;
std::string last_text;
bool loud = !capture && !quiet;
llm::StreamSink sink = [](const std::string&) {}; // pieces.append: collected, never read
bool streamed = false;
llm::StreamSink sink = [this, &streamed](const std::string& piece) { // pieces.append: only an observer reads them
if (observer && !piece.empty()) { streamed = true; observer(orch::OrchEvent::text_of(piece)); }
};
tools.live = loud;
std::deque<std::string> seen_actions, seen_results;
int stall = 0;
@@ -820,6 +845,12 @@ std::string Agent::run_turn_inner(const std::string& in_text, bool capture, int
bool nudged = false;
bool broke = false;
for (int step = 0; step < total_step_cap(); ++step) {
if (cancel_flag != nullptr && cancel_flag->load()) {
cancelled = true;
last_text = py::strip(last_text + "\n[cancelled]");
broke = true;
break;
}
if (deadline.has_value() && py::now() > *deadline) {
timed_out = true;
last_text = py::strip(last_text + "\n[time limit reached]");
@@ -837,6 +868,7 @@ std::string Agent::run_turn_inner(const std::string& in_text, bool capture, int
if (loud && ui() != nullptr) { ui()->spinner_start("thinking"); spinning = true; }
phase("thinking");
llm::ChatReply reply;
streamed = false;
try {
reply = chat.complete(messages, payload, &sink, false, deadline);
} catch (const BackendError& exc) {
@@ -851,6 +883,11 @@ std::string Agent::run_turn_inner(const std::string& in_text, bool capture, int
}
if (spinning) { ui()->spinner_stop(); }
reply.content = store->redact(reply.content, profile);
if (observer) {
if (!streamed && !reply.content.empty()) { observer(orch::OrchEvent::text_of(reply.content + "\n")); }
else if (streamed) { observer(orch::OrchEvent::text_of("\n")); }
else if (!reply.reasoning.empty() && reply.tool_calls.empty()) { observer(orch::OrchEvent::thought_of(py::head(reply.reasoning, 2000) + "\n")); }
}
if (loud && !py::strip(reply.content).empty()) {
print_out(ui() != nullptr ? ui()->render_markdown(reply.content) : reply.content);
}
@@ -1165,6 +1202,11 @@ std::string default_runner(AgentConfig& config, const std::string& profile, int
Agent worker(config, store.get(), false, true, depth);
if (profile != worker.profile) { worker.switch_profile(profile, true); }
worker.deadline = py::now() + timeout;
if (stats != nullptr && stats->panel) {
auto panel = stats->panel;
worker.observer = [panel](const orch::OrchEvent& event) { panel->emit(event); };
worker.cancel_flag = &stats->cancel;
}
std::string result;
try {
result = worker.run_turn(task, true, max_steps);
+7
View File
@@ -4,6 +4,7 @@
// the module-level helpers select_tools / tool_catalog / compact_messages.
#include "../llm/chat_client.h"
#include "../llm/json.h"
#include "../orch/orch_event.h"
#include "agent_config.h"
#include "consts.h"
#include "goal.h"
@@ -69,6 +70,11 @@ class Agent {
int depth;
std::optional<double> deadline;
bool timed_out{false};
// Orchestration: a sub agent reports what it does to its panel (text as it streams, tool calls and
// results) and stops at its next step when the flag is raised (panel closed, idle watchdog).
std::function<void(const orch::OrchEvent&)> observer;
const std::atomic<bool>* cancel_flag{nullptr};
bool cancelled{false};
Runner runner_override; // empty: default_runner
bool user_stepped_in{false};
Prompter prompter; // empty: none
@@ -137,6 +143,7 @@ class Agent {
int64_t turn_tools_{0};
struct LastEvent { std::string name, verb, target; };
std::optional<LastEvent> last_event_;
std::string observed_tool_;
std::shared_ptr<detz::cap::CapMention> cap_mention_;
};
+20
View File
@@ -2,6 +2,7 @@
#include "../llm/helpers.h"
#include "../llm/py.h"
#include "consts.h"
#include "../orch/orch_board.h"
#include <algorithm>
#include <map>
#include <thread>
@@ -42,6 +43,16 @@ int spawn_agent(const std::string& task, const std::string& profile, double time
record->started = py::now();
agents[agent_id] = record;
}
record->panel = orch::OrchBoard::instance().open("agent #" + std::to_string(record->id), task);
{
std::weak_ptr<AgentRecord> weak = record;
record->panel->set_cancel([weak](const std::string& reason) {
if (auto r = weak.lock()) {
r->cancel_reason = reason;
r->cancel = true;
}
});
}
AgentConfig* cfg = &config;
std::thread([record, task, profile, timeout, cfg, depth, runner]() {
std::string result, status;
@@ -52,10 +63,19 @@ int spawn_agent(const std::string& task, const std::string& profile, double time
result = runner(task, profile, timeout);
}
status = result.find("[time limit reached]") != std::string::npos ? "timeout" : "done";
if (record->cancel) {
status = "error";
result = "worker cancelled (" + record->cancel_reason + "); partial result:\n" + result;
}
} catch (const std::exception& exc) {
result = "worker failed: " + llm::short_error(exc);
status = "error";
}
if (record->panel) {
using S = orch::OrchPanel::State;
S st = status == "done" ? S::Done : status == "timeout" ? S::Timeout : record->cancel ? S::Cancelled : S::Failed;
record->panel->finish(st, std::to_string(result.size()) + " chars");
}
{
std::lock_guard<std::mutex> guard(AgentRegistry::lock());
record->result = result;
+5
View File
@@ -4,6 +4,8 @@
#include "agent_config.h"
#include "goal.h"
#include "../llm/json.h"
#include "../orch/orch_panel.h"
#include <atomic>
#include <condition_variable>
#include <map>
#include <functional>
@@ -27,6 +29,9 @@ struct AgentRecord {
bool collected{false};
std::optional<int64_t> tools;
std::optional<std::string> spilled;
std::shared_ptr<orch::OrchPanel> panel; // the sub agent's screen and supervision record
std::atomic<bool> cancel{false}; // raised by the panel (closed by the user, idle watchdog)
std::string cancel_reason;
// completion signal replacing Thread.join(timeout)
std::mutex done_mutex;
std::condition_variable done_cv;
+82 -18
View File
@@ -1,40 +1,104 @@
#include "cap_mention.h"
#include <chrono>
#include <thread>
#include "../orch/orch_board.h"
#include "../orch/orch_policy.h"
namespace detz::cap {
CapMention::~CapMention() { closeAll(); }
void CapMention::closeAll() {
for (auto& s : sessions_) {
try { s.second->close(); } catch (...) {}
for (auto& s : slots_) {
try { s.second->session->close(); } catch (...) {}
}
sessions_.clear();
slots_.clear();
}
std::string CapMention::ask(const CapSelection& sel, const std::string& prompt, const std::string& cwd) {
CapAnswer CapMention::ask(const CapSelection& sel, const std::string& prompt, const std::string& cwd, double timeout) {
const std::string key = sel.agent + "\n" + sel.model + "\n" + cwd;
const std::string who = sel.agent + (sel.model.empty() ? "" : ":" + sel.model);
CapAnswer out;
out.panel = orch::OrchBoard::instance().open("cap " + who, prompt);
Slot* slot = nullptr;
try {
auto it = sessions_.find(key);
if (it == sessions_.end()) {
auto it = slots_.find(key);
if (it == slots_.end()) {
CapManifest manifest = catalog_.manifest(sel.agent);
const std::string urn = "cap://" + sel.agent + "-1";
auto session = std::make_unique<CapSession>(urn, CapDriver::create(manifest, urn, cwd, sel.model));
session->start();
it = sessions_.emplace(key, std::move(session)).first;
auto fresh = std::make_unique<Slot>();
fresh->session = std::make_unique<CapSession>(urn, CapDriver::create(manifest, urn, cwd, sel.model));
Slot* raw = fresh.get();
fresh->session->addTap([raw](const std::string&, const Json& event) {
std::shared_ptr<orch::OrchPanel> panel;
std::optional<orch::OrchEvent> mapped;
{
std::lock_guard<std::mutex> lock(raw->mu);
panel = raw->panel;
if (panel) mapped = raw->adapter.map(event);
}
if (mapped) panel->emit(*mapped);
});
fresh->session->start();
it = slots_.emplace(key, std::move(fresh)).first;
}
CapTurnResult result = it->second->ask(prompt);
if (result.state() != "done" && result.text().empty()) {
sessions_.erase(key);
return "cap agent " + who + " ended the turn in state '" + result.state() + "' without an answer";
slot = it->second.get();
{
std::lock_guard<std::mutex> lock(slot->mu);
slot->panel = out.panel;
}
return result.text();
auto cancelled = std::make_shared<std::atomic<bool>>(false);
slot->busy = true;
out.panel->set_cancel([slot, cancelled](const std::string& reason) {
*cancelled = true;
try { slot->session->cancel("current_turn"); } catch (...) {}
std::thread([slot, reason] { // escalate: an agent that ignores the cancel loses its session
std::this_thread::sleep_for(std::chrono::duration<double>(orch::ORCH_CANCEL_GRACE));
if (slot->busy) { try { slot->session->close(); } catch (...) {} }
}).detach();
});
CapTurnResult result = slot->session->ask(prompt, timeout);
slot->busy = false;
out.panel->set_cancel(nullptr);
{
std::lock_guard<std::mutex> lock(slot->mu);
slot->panel.reset();
}
out.text = result.text();
if (*cancelled) {
slots_.erase(key);
out.detail = "cancelled";
out.text = "cap agent " + who + " was cancelled" + (out.text.empty() ? "" : "; partial answer:\n" + out.text);
return out;
}
if (result.state() != "done" && out.text.empty()) {
slots_.erase(key);
out.detail = "failed";
out.text = "cap agent " + who + " ended the turn in state '" + result.state() + "' without an answer";
return out;
}
out.ok = true;
return out;
} catch (const CapTimeoutError& exc) {
if (slot) { try { slot->session->cancel("current_turn"); } catch (...) {} slot->busy = false; }
slots_.erase(key);
out.detail = "timeout";
out.text = std::string("cap agent ") + who + " timed out: " + exc.what();
return out;
} catch (const std::exception& exc) {
sessions_.erase(key);
return "cap agent " + who + " failed: " + exc.what() + "\n" + catalog_.report();
if (slot) slot->busy = false;
slots_.erase(key);
out.detail = "failed";
out.text = "cap agent " + who + " failed: " + exc.what() + "\n" + catalog_.report();
return out;
} catch (...) {
sessions_.erase(key);
return "cap agent " + who + " failed with an unknown error";
if (slot) slot->busy = false;
slots_.erase(key);
out.detail = "failed";
out.text = "cap agent " + who + " failed with an unknown error";
return out;
}
}
+25 -6
View File
@@ -1,28 +1,47 @@
#pragma once
// CapMention: runs a turn on an external CAP agent picked by an `@agent[:model]`
// mention. Owns the catalog and one live session per (agent, model); a failed
// session is dropped so the next mention starts clean. Never throws.
// CapMention: runs a turn on an external CAP agent picked by an `@agent[:model]` mention. Owns the
// catalog and one live session per (agent, model, cwd). Every turn gets its own panel on the
// multiplexer (orch::OrchBoard) fed from the session's cap.* events, and is supervised: the turn
// timeout, an idle watchdog and a closed panel all cancel it, escalating from a soft cancel to
// closing the session after ORCH_CANCEL_GRACE. Never throws.
#include <atomic>
#include <map>
#include <memory>
#include <mutex>
#include <string>
#include "../orch/orch_cap_adapter.h"
#include "../orch/orch_panel.h"
#include "cap_model_catalog.h"
#include "cap_session.h"
namespace detz::cap {
struct CapAnswer {
std::string text; // the agent's answer, or a one-line failure text
bool ok{false};
std::string detail; // why not ok: timeout | cancelled | failed
std::shared_ptr<orch::OrchPanel> panel; // the caller finishes it once the answer is collected
};
class CapMention {
public:
CapMention() = default;
~CapMention();
CapModelCatalog& catalog() { return catalog_; }
// Returns the agent's answer, or a one-line failure text prefixed "cap agent".
std::string ask(const CapSelection& sel, const std::string& prompt, const std::string& cwd);
CapAnswer ask(const CapSelection& sel, const std::string& prompt, const std::string& cwd, double timeout = CAP_TURN_TIMEOUT);
void closeAll();
private:
struct Slot {
std::unique_ptr<CapSession> session;
orch::OrchCapAdapter adapter;
std::mutex mu;
std::shared_ptr<orch::OrchPanel> panel; // the turn in flight
std::atomic<bool> busy{false};
};
CapModelCatalog catalog_;
std::map<std::string, std::unique_ptr<CapSession>> sessions_;
std::map<std::string, std::unique_ptr<Slot>> slots_;
};
} // namespace detz::cap
+9 -3
View File
@@ -153,10 +153,11 @@ void MplexPane::place(int nx, int ny, int nw, int nh) {
y = ny;
nw = std::max(1, nw);
nh = std::max(1, nh);
if (nw == w && nh == h) return;
const int screen_rows = std::max(1, nh - header_rows());
if (nw == w && nh == h && screen.rows == screen_rows) return;
w = nw;
h = nh;
screen.resize(h, w);
screen.resize(screen_rows, w);
apply_winsize();
auto snapshot = modes;
for (const auto& mode : snapshot) mode->on_resize();
@@ -203,8 +204,13 @@ void MplexPane::log(const Bytes& data) {
log_size += static_cast<long>(data.size());
}
void MplexPane::open_log() {
if (log_fd_ >= 0) return;
log_fd_ = ::open(log_path.c_str(), O_WRONLY | O_CREAT | O_APPEND, 0600);
}
void MplexPane::write(const Bytes& data) {
if (data.empty()) return;
if (data.empty() || virtual_pane) return; // a feed pane has no process to receive input
if (static_cast<long>(pending_in.size() + data.size()) > kPaneInMax) {
dropped += static_cast<long>(data.size());
return;
+7
View File
@@ -51,6 +51,13 @@ class MplexPane : public std::enable_shared_from_this<MplexPane> {
std::string log_path;
long log_size = 0;
long input_bytes = 0;
// Virtual (feed) pane: no pty, text arrives through the control socket (mplex_vpane.h). The first
// row of its rectangle is a header line drawn by the renderer, the screen below it has h - 1 rows.
bool virtual_pane = false;
std::string header;
std::string header_style = "bg=colour24,fg=white,bold";
int header_rows() const { return virtual_pane ? 1 : 0; }
void open_log();
std::shared_ptr<MplexWindow> window() const { return window_ref.lock(); }
void set_window(const std::shared_ptr<MplexWindow>& value) { window_ref = value; }
+8 -2
View File
@@ -53,9 +53,15 @@ MplexCanvas MplexRenderer::frame(MplexClientConn& client) {
auto active = window->active;
for (const auto& pane : window->visible_panes()) {
ModeBlock block = block_of(pane, client);
canvas.blit(pane->x, area_y + pane->y, block.grid, pane->w);
const int top = pane->header_rows();
if (top) {
Style head = MplexStyle::parse(pane->header_style);
canvas.fill(pane->x, area_y + pane->y, pane->w, 1, head);
canvas.put(pane->x, area_y + pane->y, to_ustr(pane->header), head, pane->w);
}
canvas.blit(pane->x, area_y + pane->y + top, block.grid, pane->w);
if (pane == active && block.cursor.has_value())
cursor = XY{pane->x + block.cursor->first, area_y + pane->y + block.cursor->second};
cursor = XY{pane->x + block.cursor->first, area_y + pane->y + top + block.cursor->second};
}
if (!window->zoomed && window->panes().size() > 1) borders(canvas, window, area_y);
// isinstance(client.overlay, (MplexPaneNumbers, MplexFlowScreen)): the only two overlay kinds.
+9
View File
@@ -1,6 +1,7 @@
// MplexServer: literal port of tai.py 13942-15330 (and plexer_dir, tai.py:9329).
// Owns every session, window, pane and client. One poll loop, no threads.
#include "mplex_server.h"
#include "mplex_vpane.h"
#include <fcntl.h>
#include <poll.h>
@@ -205,6 +206,7 @@ MplexServer::MplexServer(const std::string& home_in) {
options = &config->options;
renderer = std::make_unique<MplexRenderer>(this);
commands = std::make_unique<MplexCommands>(this);
vpanes = std::make_unique<MplexVpanes>(*this);
ids = {{"session", 0}, {"window", 0}, {"pane", 0}, {"client", 0}, {"buffer", 0}};
started = py::monotonic();
{
@@ -541,6 +543,7 @@ std::shared_ptr<MplexPane> MplexServer::split_pane(const std::shared_ptr<MplexWi
}
void MplexServer::discard_pane(std::shared_ptr<MplexPane> pane) {
if (vpanes) vpanes->forget(pane->id);
pane->close();
panes.erase(pane->id);
if (pane->pid) {
@@ -587,6 +590,7 @@ void MplexServer::kill_pane(std::shared_ptr<MplexPane> pane) {
window->layout_index = -1;
window->relayout();
mark_dirty(window);
if (MplexVpanes::only_virtual(window)) kill_window(window); // the main pane is gone: feed panes alone are no session
}
void MplexServer::kill_window(std::shared_ptr<MplexWindow> window) {
@@ -1190,6 +1194,10 @@ Json MplexServer::dispatch_control(const Json& request) {
data.set("window", idx ? Json::integer(*idx) : Json());
return ok_reply(data);
}
if (is("vpane_open")) return ok_reply(vpanes->open(request));
if (is("vpane_write")) return ok_reply(vpanes->write(request));
if (is("vpane_close")) return ok_reply(vpanes->close(request));
if (is("vpane_list")) return ok_reply(vpanes->list());
if (is("show")) return flow_show(request);
if (is("result")) return flow_result(request);
if (is("cancel")) return flow_cancel(request);
@@ -1741,6 +1749,7 @@ void MplexServer::run_deadlines(double now) {
void MplexServer::tick(double now) {
if (!flows.empty()) flow_prune(now);
if (vpanes && vpanes->size()) vpanes->sweep(now);
for (const auto& window : windows.values()) {
if (window->auto_rename && window->active && window->session()) {
std::string tmpl = window->options.get("automatic-rename-format").s;
+2
View File
@@ -27,6 +27,7 @@ namespace detz::mplex {
class MplexCommands;
class MplexRenderer;
class MplexVpanes;
// Hard caps (mplex.md, "Hard caps"). Names are the Python constants.
constexpr const char* MPLEX_SOCKET_NAME = "server.sock";
@@ -125,6 +126,7 @@ class MplexServer {
int lock_fd = -1;
std::unique_ptr<MplexRenderer> renderer;
std::unique_ptr<MplexCommands> commands;
std::unique_ptr<MplexVpanes> vpanes; // feed panes (mplex_vpane.h)
std::map<std::string, std::pair<double, std::string>> shell_cache;
double started;
std::string stamp;
+205
View File
@@ -0,0 +1,205 @@
#include "mplex_vpane.h"
#include <algorithm>
#include "mplex_py.h"
#include "mplex_server.h"
namespace detz::mplex {
namespace {
std::string clip(const std::string& text, size_t limit) { return py::slice(text, 0, static_cast<long>(limit)); }
long long to_id(const Json& ident) {
if (ident.is_number()) return static_cast<long long>(ident.as_float());
if (ident.is_string()) return py::to_int(py::lstrip(ident.as_string(), "%"));
throw ValueError("pane must be a pane id");
}
} // namespace
std::string MplexVpanes::style_of(const std::string& state) {
if (state == "done") return "bg=colour28,fg=white,bold";
if (state == "failed") return "bg=colour124,fg=white,bold";
if (state == "idle" || state == "timeout") return "bg=colour130,fg=white,bold";
if (state == "cancelled") return "bg=colour240,fg=white";
return "bg=colour24,fg=white,bold"; // running, queued, anything else
}
MplexVpanes::Entry* MplexVpanes::find(const Json& ident) {
auto it = entries_.find(static_cast<int>(to_id(ident)));
return it == entries_.end() ? nullptr : &it->second;
}
bool MplexVpanes::only_virtual(const std::shared_ptr<MplexWindow>& window) {
auto all = window->panes();
if (all.empty()) return false;
for (const auto& pane : all)
if (!pane->virtual_pane) return false;
return true;
}
void MplexVpanes::arrange(const std::shared_ptr<MplexWindow>& window, int main_percent) {
std::vector<std::shared_ptr<MplexPane>> real, feed;
for (const auto& pane : window->panes()) (pane->virtual_pane ? feed : real).push_back(pane);
if (real.empty()) return;
std::sort(feed.begin(), feed.end(), [](const auto& a, const auto& b) { return a->id < b->id; });
std::vector<std::shared_ptr<MplexPane>> ordered = real;
ordered.insert(ordered.end(), feed.begin(), feed.end());
const int width = window->width, height = window->height;
int main_w = std::max(1, std::min(width - 2, width * std::max(10, std::min(90, main_percent)) / 100));
if (width >= 2 * MPLEX_VPANE_MIN_COLS + 1 && width - main_w - 1 < MPLEX_VPANE_MIN_COLS) main_w = width - MPLEX_VPANE_MIN_COLS - 1;
window->root = MplexLayout::preset("main-vertical", ordered, width, height, main_w, 24);
window->layout_index = -1;
window->relayout();
s_.mark_dirty(window);
}
Json MplexVpanes::open(const Json& request) {
auto origin = s_.control_pane(request);
auto window = origin->window();
if (!window) throw MplexError("pane has no window");
if (entries_.size() >= static_cast<size_t>(MPLEX_VPANE_MAX))
throw MplexError("too many feed panes (limit " + std::to_string(MPLEX_VPANE_MAX) + ")");
if (s_.panes.size() >= static_cast<size_t>(MPLEX_MAX_PANES))
throw MplexError("too many panes (limit " + std::to_string(MPLEX_MAX_PANES) + ")");
size_t in_window = 0;
for (const auto& pane : window->panes())
if (pane->virtual_pane) ++in_window;
if (window->height / static_cast<int>(in_window + 1) < MPLEX_VPANE_MIN_ROWS)
throw MplexError("no space for another feed pane");
if (window->width < 2 * MPLEX_VPANE_MIN_COLS + 1) throw MplexError("window too narrow for a feed pane");
if (window->zoomed) {
window->zoomed = nullptr;
window->relayout();
}
const Json* label_json = request.get("label");
std::string label = label_json && label_json->is_string() ? clip(label_json->as_string(), MPLEX_VPANE_HEADER_MAX) : "agent";
const Json* header_json = request.get("header");
std::string header = header_json && header_json->is_string() ? clip(header_json->as_string(), MPLEX_VPANE_HEADER_MAX) : label;
const Json* state_json = request.get("state");
std::string state = state_json && state_json->is_string() ? state_json->as_string() : "running";
int percent = MPLEX_VPANE_MAIN_PERCENT;
if (const Json* p = request.get("main_percent")) percent = static_cast<int>(to_id(*p));
auto pane = std::make_shared<MplexPane>(s_.next_id("pane"), 1, 1, static_cast<int>(window->options.get("history-limit").i), s_.dir, s_.stamp);
pane->virtual_pane = true;
pane->header = header;
pane->header_style = style_of(state);
pane->screen.title = label;
pane->set_window(window);
pane->open_log();
s_.panes.set(pane->id, pane);
Entry entry;
entry.pane = pane;
entry.state = state;
entry.touched = py::monotonic();
entries_[pane->id] = entry;
// The new pane joins the window tree through the preset: arrange() rebuilds from window->panes(),
// so it has to be a leaf first; split the main pane to give it one.
auto leaf = window->leaf_of(origin->virtual_pane ? window->panes().front() : origin);
auto result = MplexLayout::split(window->root, leaf, "lr", false, pane, 0);
window->root = result.first;
arrange(window, percent);
Json data = Json::object();
data.set("id", Json::integer(pane->id));
data.set("cols", Json::integer(pane->w));
data.set("rows", Json::integer(pane->screen.rows));
return data;
}
Json MplexVpanes::write(const Json& request) {
const Json* ops = request.get("ops");
if (!ops || !ops->is_array()) throw ValueError("ops must be a list");
if (ops->as_array().size() > static_cast<size_t>(MPLEX_VPANE_OPS_MAX)) throw ValueError("too many ops in one write");
const double now = py::monotonic();
Json closed = Json::array();
Json cols = Json::object(); // current text width of every pane written to, for client side word wrap
for (const Json& op : ops->as_array()) {
if (!op.is_object()) throw ValueError("each op must be an object");
const Json* ident = op.get("pane");
if (!ident) throw ValueError("op needs a pane");
Entry* entry = find(*ident);
if (!entry) {
closed.push(Json::integer(to_id(*ident)));
continue;
}
entry->touched = now;
cols.set(std::to_string(entry->pane->id), Json::integer(entry->pane->w));
auto window = entry->pane->window();
const Json* data = op.get("data");
if (data && data->is_string() && !data->as_string().empty()) {
if (data->as_string().size() > MPLEX_VPANE_WRITE_MAX) throw ValueError("op data too large");
entry->pane->feed(to_utf8(to_ustr(data->as_string())));
}
if (const Json* header = op.get("header"); header && header->is_string()) entry->pane->header = clip(header->as_string(), MPLEX_VPANE_HEADER_MAX);
if (const Json* state = op.get("state"); state && state->is_string()) {
entry->state = state->as_string();
entry->pane->header_style = style_of(entry->state);
}
if (const Json* done = op.get("done"); done && done->truthy()) {
double linger = 20.0;
if (const Json* l = op.get("linger"); l && l->is_number()) linger = l->as_float();
entry->close_at = now + std::max(0.0, std::min(linger, MPLEX_VPANE_LINGER_MAX));
}
s_.mark_dirty(window);
}
Json reply = Json::object();
reply.set("closed", closed);
reply.set("cols", cols);
return reply;
}
void MplexVpanes::remove(int id) {
auto it = entries_.find(id);
if (it == entries_.end()) return;
auto pane = it->second.pane;
auto window = pane->window();
entries_.erase(it);
s_.kill_pane(pane);
if (window && s_.windows.contains(window->id) && !only_virtual(window)) arrange(window, MPLEX_VPANE_MAIN_PERCENT);
}
Json MplexVpanes::close(const Json& request) {
const Json* ident = request.get("pane");
if (!ident) throw ValueError("pane is required");
remove(static_cast<int>(to_id(*ident)));
return Json::boolean(true);
}
Json MplexVpanes::list() const {
Json out = Json::array();
for (const auto& item : entries_) {
Json row = Json::object();
row.set("id", Json::integer(item.first));
row.set("state", Json::string(item.second.state));
row.set("header", Json::string(item.second.pane->header));
row.set("cols", Json::integer(item.second.pane->w));
row.set("rows", Json::integer(item.second.pane->screen.rows));
row.set("closing", Json::boolean(item.second.close_at.has_value()));
out.push(row);
}
return out;
}
void MplexVpanes::sweep(double now) {
std::vector<int> due;
for (auto& item : entries_) {
Entry& e = item.second;
if (e.close_at) {
if (now >= *e.close_at) due.push_back(item.first);
} else if (now - e.touched > MPLEX_VPANE_LEASE) {
e.state = "failed";
e.pane->header = "owner gone: " + e.pane->header;
e.pane->header_style = style_of("failed");
e.close_at = now + MPLEX_VPANE_ORPHAN_LINGER;
s_.mark_dirty(e.pane->window());
}
}
for (int id : due) remove(id);
}
void MplexVpanes::forget(int pane_id) { entries_.erase(pane_id); }
} // namespace detz::mplex
+68
View File
@@ -0,0 +1,68 @@
// MplexVpanes: virtual (feed) panes, the multiplexer half of orchestration (docs/orchestration.md).
// A feed pane has no process; the orchestrator writes text into it through the control socket
// (vpane_open / vpane_write / vpane_close / vpane_list) and the server composes it like any pane.
// Layout policy: the pane that asked stays the main pane on the left, every feed pane is stacked in
// a column on the right (tmux main-vertical, main width MPLEX_VPANE_MAIN_PERCENT).
// This is a taicpp extension; the Python app has no equivalent.
#pragma once
#include <map>
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "mplex_json.h"
#include "mplex_pane.h"
namespace detz::mplex {
class MplexServer;
constexpr int MPLEX_VPANE_MAX = 8; // feed panes per server (= AGENTS_MAX_PARALLEL)
constexpr int MPLEX_VPANE_MIN_ROWS = 6; // header row plus five text rows
constexpr int MPLEX_VPANE_MIN_COLS = 20; // narrowest right column worth drawing
constexpr int MPLEX_VPANE_MAIN_PERCENT = 55; // width of the main pane, in percent
constexpr int MPLEX_VPANE_OPS_MAX = 16; // ops in one vpane_write
constexpr size_t MPLEX_VPANE_WRITE_MAX = 65536; // bytes of one op
constexpr size_t MPLEX_VPANE_HEADER_MAX = 200; // characters of a header
constexpr double MPLEX_VPANE_LEASE = 30.0; // seconds without a write before the owner is presumed dead
constexpr double MPLEX_VPANE_ORPHAN_LINGER = 10.0; // seconds an orphaned pane stays after the lease ran out
constexpr double MPLEX_VPANE_LINGER_MAX = 300.0; // longest linger a client may ask for
class MplexVpanes {
public:
explicit MplexVpanes(MplexServer& server) : s_(server) {}
// {cmd:"vpane_open", pane:<origin>, label, header?, state?, main_percent?} -> {id, cols, rows}
Json open(const Json& request);
// {cmd:"vpane_write", ops:[{pane, data?, header?, state?, done?, linger?}]} -> {closed:[ids]}
Json write(const Json& request);
// {cmd:"vpane_close", pane} -> true
Json close(const Json& request);
// [{id, owner, state, header, cols, rows, closing}]
Json list() const;
// Server tick: closes lingering panes and orphans.
void sweep(double now);
// discard_pane hook: forget the entry of a pane that went away (also by the user's kill-pane).
void forget(int pane_id);
// True when every pane of the window is a feed pane (the main pane is gone).
static bool only_virtual(const std::shared_ptr<MplexWindow>& window);
size_t size() const { return entries_.size(); }
static std::string style_of(const std::string& state);
private:
struct Entry {
std::shared_ptr<MplexPane> pane;
std::string state = "running";
double touched = 0;
std::optional<double> close_at;
};
void arrange(const std::shared_ptr<MplexWindow>& window, int main_percent);
void remove(int id);
Entry* find(const Json& ident);
MplexServer& s_;
std::map<int, Entry> entries_; // by pane id, creation order
};
} // namespace detz::mplex
+129
View File
@@ -0,0 +1,129 @@
#include "orch_board.h"
#include <chrono>
#include <cstdlib>
#include "orch_policy.h"
namespace detz::orch {
namespace {
double now_s() {
return std::chrono::duration<double>(std::chrono::steady_clock::now().time_since_epoch()).count();
}
constexpr int kFailuresMax = 3; // consecutive failed writes before the multiplexer is given up on
} // namespace
OrchBoard::OrchBoard(std::unique_ptr<OrchTransport> transport, bool start_thread) : transport_(std::move(transport)) {
if (start_thread) thread_ = std::thread([this] { run(); });
}
OrchBoard::~OrchBoard() { stop(); }
OrchBoard& OrchBoard::instance() {
static OrchBoard* board = [] {
auto* b = new OrchBoard(OrchMplexTransport::from_env());
std::atexit([] { OrchBoard::instance().flush(2.0); });
return b;
}();
return *board;
}
void OrchBoard::stop() {
if (stop_.exchange(true)) return;
cv_.notify_all();
if (thread_.joinable()) thread_.join();
}
std::shared_ptr<OrchPanel> OrchBoard::open(const std::string& label, const std::string& task) {
int remote = -1, columns = 80;
std::string error;
size_t live = shown();
if (!transport_->available() || !transport_ok_) {
error = "no multiplexer";
} else if (live >= static_cast<size_t>(ORCH_PANELS_MAX)) {
error = "too many panels (" + std::to_string(ORCH_PANELS_MAX) + ")";
} else {
OrchPanel probe(label, task, -1);
auto id = transport_->open(OrchFormatter::clip(label, ORCH_LABEL_MAX), probe.header(), error, &columns);
if (id) remote = *id;
}
auto panel = std::make_shared<OrchPanel>(label, task, remote, columns);
std::lock_guard<std::mutex> lock(mu_);
if (remote < 0) last_error_ = error;
panels_.push_back(panel);
return panel;
}
size_t OrchBoard::shown() const {
std::lock_guard<std::mutex> lock(mu_);
size_t n = 0;
for (const auto& p : panels_)
if (p->shown() && !p->drained()) ++n;
return n;
}
size_t OrchBoard::tracked() const { std::lock_guard<std::mutex> lock(mu_); return panels_.size(); }
std::string OrchBoard::last_error() const { std::lock_guard<std::mutex> lock(mu_); return last_error_; }
void OrchBoard::step(double now, bool monitor) {
std::vector<std::shared_ptr<OrchPanel>> panels;
{
std::lock_guard<std::mutex> lock(mu_);
panels = panels_;
}
if (monitor) for (const auto& p : panels) p->watchdog(now);
std::vector<OrchOp> ops;
for (const auto& p : panels) {
OrchOp op;
if (p->collect(op)) ops.push_back(std::move(op));
}
if (!ops.empty() && transport_ok_) {
std::vector<int> closed;
std::vector<std::pair<int, int>> widths;
std::string error;
if (transport_->write(ops, closed, error, &widths)) {
failures_ = 0;
for (const auto& w : widths)
for (const auto& p : panels)
if (p->remote_id() == w.first) p->set_width(w.second - 1);
for (int id : closed)
for (const auto& p : panels)
if (p->remote_id() == id) p->closed_by_user();
} else if (++failures_ >= kFailuresMax) {
transport_ok_ = false; // the multiplexer is gone: keep supervising, stop drawing
std::lock_guard<std::mutex> lock(mu_);
last_error_ = "multiplexer unreachable: " + error;
}
}
std::lock_guard<std::mutex> lock(mu_);
panels_.erase(std::remove_if(panels_.begin(), panels_.end(), [](const std::shared_ptr<OrchPanel>& p) { return p->drained(); }), panels_.end());
}
void OrchBoard::run() {
while (!stop_) {
double now = now_s();
bool monitor = now - last_monitor_ >= ORCH_MONITOR_INTERVAL;
if (monitor) last_monitor_ = now;
step(now, monitor);
std::unique_lock<std::mutex> lock(cv_mu_);
cv_.wait_for(lock, std::chrono::duration<double>(ORCH_FLUSH_INTERVAL), [this] { return stop_.load(); });
}
}
void OrchBoard::flush(double timeout) {
double deadline = now_s() + timeout;
for (;;) {
step(now_s(), false);
bool pending = false;
{
std::lock_guard<std::mutex> lock(mu_);
for (const auto& p : panels_) {
OrchOp probe;
(void)probe;
if (p->shown() && !p->drained() && p->finished()) pending = true;
}
}
if (!pending || now_s() >= deadline || !transport_ok_) return;
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
}
} // namespace detz::orch
+51
View File
@@ -0,0 +1,51 @@
// OrchBoard: the orchestrator's one place for panels. Opens them, drains their bytes to the
// multiplexer in coalesced writes, watches them (heartbeat / idle / user close) and retires them.
// The flusher thread is the monitor: display and supervision share one clock, so a job that is not
// shown (no multiplexer, no space) is supervised exactly like one that is.
#pragma once
#include <atomic>
#include <condition_variable>
#include <memory>
#include <mutex>
#include <string>
#include <thread>
#include <vector>
#include "orch_panel.h"
#include "orch_transport.h"
namespace detz::orch {
class OrchBoard {
public:
explicit OrchBoard(std::unique_ptr<OrchTransport> transport, bool start_thread = true);
~OrchBoard();
// The process-wide board over the multiplexer this process runs in (never destroyed; flushed at exit).
static OrchBoard& instance();
// Never null. A panel with no pane behind it is returned when there is no multiplexer, no space,
// or ORCH_PANELS_MAX panels are already shown; last_error() says why.
std::shared_ptr<OrchPanel> open(const std::string& label, const std::string& task);
size_t shown() const;
size_t tracked() const;
std::string last_error() const;
// One synchronous drain of every panel (tests, exit). Returns when nothing is pending or `timeout` ran out.
void flush(double timeout = 2.0);
void stop();
bool transport_ok() const { return transport_ok_.load(); }
private:
void run();
void step(double now, bool monitor);
std::unique_ptr<OrchTransport> transport_;
mutable std::mutex mu_;
std::vector<std::shared_ptr<OrchPanel>> panels_;
std::string last_error_;
std::atomic<bool> stop_{false}, transport_ok_{true};
int failures_{0};
std::condition_variable cv_;
std::mutex cv_mu_;
std::thread thread_;
double last_monitor_{0};
};
} // namespace detz::orch
+64
View File
@@ -0,0 +1,64 @@
#include "orch_cap_adapter.h"
namespace detz::orch {
using Json = detz::wire::Json;
namespace {
std::string text_of(const Json& v) {
if (v.isString()) return v.str();
if (v.isNull()) return "";
return v.dumps();
}
} // namespace
std::optional<OrchEvent> OrchCapAdapter::map(const Json& event) {
const std::string kind = text_of(event.get("kind"));
if (kind == "cap.text_chunk") {
std::string t = text_of(event.get("text"));
if (t.empty()) return std::nullopt;
if (t.back() != '\n') t += "\n";
return text_of(event.get("channel")) == "thought" ? OrchEvent::thought_of(t) : OrchEvent::text_of(t);
}
if (kind == "cap.thought") {
std::string t = text_of(event.get("text"));
if (t.empty()) return std::nullopt;
if (t.back() != '\n') t += "\n";
return OrchEvent::thought_of(t);
}
if (kind == "cap.tool_call.start") {
std::string name = text_of(event.get("name"));
if (calls_.size() >= kCallsMax) calls_.clear();
calls_[text_of(event.get("call_id"))] = name;
return OrchEvent::tool_start(name, text_of(event.get("input")));
}
if (kind == "cap.tool_call.end") {
auto it = calls_.find(text_of(event.get("call_id")));
std::string name = it == calls_.end() ? "tool" : it->second;
if (it != calls_.end()) calls_.erase(it);
return OrchEvent::tool_end(name, text_of(event.get("output")), event.get("is_error").truthy(),
event.get("duration_ms").isNumber() ? event.get("duration_ms").asInt() : 0);
}
if (kind == "cap.plan") {
std::string lines;
for (const Json& entry : event.get("entries").items()) {
std::string status = text_of(entry.get("status"));
std::string content = entry.isObject() ? text_of(entry.get("content")) : text_of(entry);
lines += (lines.empty() ? "" : "\n") + std::string(status.empty() ? "" : "[" + status + "] ") + content;
}
if (lines.empty()) return std::nullopt;
return OrchEvent::plan_of(lines);
}
if (kind == "cap.usage") {
long long total = 0;
for (const char* key : {"input_tokens", "output_tokens"})
if (event.get(key).isNumber()) total += event.get(key).asInt();
return total > 0 ? std::optional<OrchEvent>(OrchEvent::usage_of(total)) : std::nullopt;
}
if (kind == "cap.error") return OrchEvent::error_of(text_of(event.get("code")) + ": " + text_of(event.get("message")));
if (kind == "cap.ask_user") return OrchEvent::note_of("asks: " + text_of(event.get("prompt")));
if (kind == "cap.permission.request") return OrchEvent::note_of("permission: " + text_of(event.get("tool")) + " (" + text_of(event.get("intent")) + ")");
if (kind == "cap.artifact") return OrchEvent::note_of("artifact: " + text_of(event.get("name")));
return std::nullopt;
}
} // namespace detz::orch
+19
View File
@@ -0,0 +1,19 @@
// OrchCapAdapter: CAP events (any binding: PTY, stream-json, ACP, A2A all end up as cap.* events)
// to OrchEvents. One adapter per panel; it remembers call ids so a tool end can name its tool.
#pragma once
#include <map>
#include <optional>
#include <string>
#include "orch_event.h"
#include "wire_json.h"
namespace detz::orch {
class OrchCapAdapter {
public:
std::optional<OrchEvent> map(const detz::wire::Json& event);
static constexpr size_t kCallsMax = 256; // remembered call ids
private:
std::map<std::string, std::string> calls_;
};
} // namespace detz::orch
+30
View File
@@ -0,0 +1,30 @@
// OrchEvent: what a sub agent tells its panel, whatever kind of agent it is (internal fork,
// CAP session over any binding). Adapters translate their native events into these.
#pragma once
#include <string>
namespace detz::orch {
enum class EventKind { Text, Thought, ToolStart, ToolEnd, Plan, Usage, Note, Error };
struct OrchEvent {
EventKind kind{EventKind::Text};
std::string text; // body: text, plan lines, tool result, note, error
std::string name; // tool name
std::string args; // tool arguments, one line
bool failed{false}; // ToolEnd: the call failed
long long tokens{0}; // Usage: total tokens so far
long long millis{0}; // ToolEnd: duration
static OrchEvent text_of(std::string t) { OrchEvent e; e.kind = EventKind::Text; e.text = std::move(t); return e; }
static OrchEvent thought_of(std::string t) { OrchEvent e; e.kind = EventKind::Thought; e.text = std::move(t); return e; }
static OrchEvent tool_start(std::string n, std::string a) { OrchEvent e; e.kind = EventKind::ToolStart; e.name = std::move(n); e.args = std::move(a); return e; }
static OrchEvent tool_end(std::string n, std::string result, bool failed, long long ms) {
OrchEvent e; e.kind = EventKind::ToolEnd; e.name = std::move(n); e.text = std::move(result); e.failed = failed; e.millis = ms; return e;
}
static OrchEvent plan_of(std::string t) { OrchEvent e; e.kind = EventKind::Plan; e.text = std::move(t); return e; }
static OrchEvent usage_of(long long tokens) { OrchEvent e; e.kind = EventKind::Usage; e.tokens = tokens; return e; }
static OrchEvent note_of(std::string t) { OrchEvent e; e.kind = EventKind::Note; e.text = std::move(t); return e; }
static OrchEvent error_of(std::string t) { OrchEvent e; e.kind = EventKind::Error; e.text = std::move(t); return e; }
};
} // namespace detz::orch
+194
View File
@@ -0,0 +1,194 @@
#include "orch_formatter.h"
#include <algorithm>
#include <cstdio>
#include "orch_policy.h"
namespace detz::orch {
namespace {
const char* const kReset = "\x1b[0m";
const char* const kDim = "\x1b[2m";
const char* const kItalicDim = "\x1b[2;3m";
const char* const kCyan = "\x1b[36m";
const char* const kRed = "\x1b[31m";
const char* const kGreen = "\x1b[1;32m";
const char* const kYellow = "\x1b[33m";
const char* const kMagenta = "\x1b[35m";
} // namespace
std::string OrchFormatter::clip(const std::string& text, size_t chars) {
size_t count = 0, i = 0;
while (i < text.size()) {
if (count == chars) return text.substr(0, i) + "...";
unsigned char c = static_cast<unsigned char>(text[i]);
i += c < 0x80 ? 1 : (c >> 5) == 6 ? 2 : (c >> 4) == 14 ? 3 : (c >> 3) == 30 ? 4 : 1;
++count;
}
return text;
}
std::string OrchFormatter::one_line(const std::string& text) {
std::string out;
bool space = false;
for (char c : text) {
if (c == '\n' || c == '\r' || c == '\t' || c == ' ') { space = !out.empty(); continue; }
if (space) { out.push_back(' '); space = false; }
out.push_back(c);
}
return out;
}
std::string OrchFormatter::sanitize(const std::string& text) {
std::string out;
out.reserve(text.size());
for (size_t i = 0; i < text.size(); ++i) {
unsigned char c = static_cast<unsigned char>(text[i]);
if (c == 0x1b) {
// keep ESC [ params m (SGR); skip any other escape sequence entirely
if (i + 1 < text.size() && text[i + 1] == '[') {
size_t j = i + 2;
while (j < text.size() && ((text[j] >= '0' && text[j] <= '9') || text[j] == ';' || text[j] == ':')) ++j;
if (j < text.size() && text[j] == 'm') { out.append(text, i, j - i + 1); i = j; continue; }
while (j < text.size() && !(text[j] >= 0x40 && text[j] <= 0x7e)) ++j;
i = j;
continue;
}
if (i + 1 < text.size() && text[i + 1] == ']') { // OSC: up to BEL or ST
size_t j = i + 2;
while (j < text.size() && text[j] != '\x07' && !(text[j] == '\x1b' && j + 1 < text.size() && text[j + 1] == '\\')) ++j;
i = (j < text.size() && text[j] == '\x1b') ? j + 1 : j;
continue;
}
++i; // lone ESC plus the byte it introduces
continue;
}
if (c == '\r') continue;
if (c < 0x20 && c != '\n' && c != '\t') continue;
if (c == 0x7f) continue;
out.push_back(static_cast<char>(c));
}
return out;
}
std::string OrchFormatter::crlf(const std::string& text) {
std::string out;
out.reserve(text.size() + 8);
for (char c : text) {
if (c == '\n') out += "\r\n"; else out.push_back(c);
}
return out;
}
std::string OrchFormatter::flush_word() {
if (word_.empty()) return "";
std::string out;
if (col_ > 0 && col_ + word_cols_ > width_ - 1) { out = "\r\n"; col_ = 0; } // the word does not fit: wrap before it
out += word_;
col_ += word_cols_;
word_.clear();
word_cols_ = 0;
return out;
}
std::string OrchFormatter::settle() { return flush_word(); }
std::string OrchFormatter::flow(const std::string& text) {
std::string out;
for (size_t i = 0; i < text.size(); ++i) {
unsigned char c = static_cast<unsigned char>(text[i]);
if (c == 0x1b) { // an SGR sequence (sanitize kept only those): zero width, travels with the word
size_t j = i;
while (j < text.size() && text[j] != 'm') ++j;
word_.append(text, i, j - i + 1);
i = j;
continue;
}
if (c == '\n') { out += flush_word(); out += "\r\n"; col_ = 0; continue; }
if (c == ' ' || c == '\t') {
out += flush_word();
if (col_ > 0 && col_ < width_ - 1) { out += ' '; ++col_; }
continue;
}
word_.push_back(static_cast<char>(c));
if ((c & 0xC0) != 0x80) ++word_cols_;
if (word_cols_ >= width_ - 1) { // a word longer than the line: break it here
if (col_ > 0) { out += "\r\n"; col_ = 0; }
out += word_ + "\r\n";
word_.clear();
word_cols_ = 0;
}
}
return out;
}
std::string OrchFormatter::line_start() {
std::string out = flush_word();
if (col_ > 0) { out += "\r\n"; col_ = 0; }
return out;
}
std::string OrchFormatter::banner(const std::string& task) const {
return std::string(kDim) + "\xE2\x96\xB8 " + crlf(clip(one_line(sanitize(task)), ORCH_TASK_SHOW_MAX)) + kReset + "\r\n\r\n";
}
std::string OrchFormatter::render(const OrchEvent& e) {
std::string out;
switch (e.kind) {
case EventKind::Text:
return flow(sanitize(e.text));
case EventKind::Thought: {
std::string body = flow(sanitize(e.text));
return body.empty() ? std::string() : std::string(kItalicDim) + body + kReset;
}
case EventKind::ToolStart:
out = line_start() + kCyan + "\xE2\x8F\xBA " + clip(sanitize(e.name), 40) + kReset;
if (!e.args.empty()) {
size_t room = static_cast<size_t>(std::max(8, width_ - 6 - static_cast<int>(std::min<size_t>(e.name.size(), 40))));
out += std::string(kDim) + "(" + clip(one_line(sanitize(e.args)), std::min(ORCH_TOOL_ARGS_MAX, room)) + ")" + kReset;
}
out += "\r\n";
return out;
case EventKind::ToolEnd: {
std::string body = sanitize(e.text);
std::string shown;
size_t lines = 0, pos = 0;
while (pos <= body.size() && lines < ORCH_TOOL_RESULT_LINES) {
size_t nl = body.find('\n', pos);
std::string line = body.substr(pos, nl == std::string::npos ? std::string::npos : nl - pos);
if (!one_line(line).empty()) {
shown += (lines ? "\r\n " : "") + clip(line, std::min(ORCH_TOOL_RESULT_COLS, static_cast<size_t>(std::max(8, width_ - 6))));
++lines;
}
if (nl == std::string::npos) break;
pos = nl + 1;
}
if (shown.empty()) shown = "(no output)";
out = line_start() + (e.failed ? kRed : kDim) + " \xE2\x8E\xBF " + shown + " \xC2\xB7 " + std::to_string(body.size()) + " chars" + kReset + "\r\n";
return out;
}
case EventKind::Plan:
out = line_start() + kMagenta + "\xE2\x96\xA4 " + kReset + crlf(sanitize(e.text)) + "\r\n";
return out;
case EventKind::Note:
out = line_start() + kYellow + "\xC2\xB7 " + crlf(clip(one_line(sanitize(e.text)), 400)) + kReset + "\r\n";
return out;
case EventKind::Error:
out = line_start() + kRed + "\xE2\x9C\x97 " + crlf(clip(one_line(sanitize(e.text)), 400)) + kReset + "\r\n";
return out;
case EventKind::Usage:
return ""; // shown in the header
}
return "";
}
std::string OrchFormatter::footer(const std::string& state, const std::string& summary, double seconds) {
char buf[32];
std::snprintf(buf, sizeof buf, "%.1fs", seconds);
const char* color = state == "done" ? kGreen : state == "failed" ? kRed : kYellow;
const char* icon = state == "done" ? "\xE2\x9C\x93" : state == "failed" ? "\xE2\x9C\x97" : "\xE2\x97\x8C";
std::string out = line_start() + "\r\n" + color + icon + " " + state + kReset + " " + buf;
if (!summary.empty()) out += std::string(kDim) + " \xC2\xB7 " + clip(one_line(sanitize(summary)), 300) + kReset;
out += "\r\n";
return out;
}
} // namespace detz::orch
+35
View File
@@ -0,0 +1,35 @@
// OrchFormatter: turns OrchEvents into the bytes written into a feed pane. Output is plain text with
// SGR colors and CRLF line ends (a feed pane has no tty to translate them). Agent text is sanitized:
// only SGR sequences, newline and tab survive, so an agent can never move the cursor or retitle
// the terminal. One formatter per panel (it remembers whether the cursor is at a line start).
#pragma once
#include <string>
#include "orch_event.h"
namespace detz::orch {
class OrchFormatter {
public:
std::string render(const OrchEvent& event);
void set_width(int columns) { width_ = columns < 12 ? 12 : columns; }
int width() const { return width_; }
// Emits the word held back because more of it may still arrive (call when the stream pauses).
std::string settle();
bool holding() const { return !word_.empty(); }
std::string banner(const std::string& task) const; // dim task echo at the top
std::string footer(const std::string& state, const std::string& summary, double seconds);
// Helpers, public for tests.
static std::string sanitize(const std::string& text); // keep SGR, \n, \t; drop the rest
static std::string clip(const std::string& text, size_t chars); // UTF-8 aware, adds "..."
static std::string crlf(const std::string& text); // \n to \r\n
static std::string one_line(const std::string& text); // collapse whitespace
private:
std::string line_start();
std::string flow(const std::string& text); // word wrap at width_, holds the trailing partial word
std::string flush_word();
int width_{80};
int col_{0};
std::string word_;
int word_cols_{0};
};
} // namespace detz::orch
+179
View File
@@ -0,0 +1,179 @@
#include "orch_panel.h"
#include <chrono>
#include <cstdio>
#include "orch_policy.h"
namespace detz::orch {
namespace {
double now_s() {
return std::chrono::duration<double>(std::chrono::steady_clock::now().time_since_epoch()).count();
}
std::string compact(long long n) {
char buf[24];
if (n >= 1000000) std::snprintf(buf, sizeof buf, "%.1fM", n / 1e6);
else if (n >= 1000) std::snprintf(buf, sizeof buf, "%.1fk", n / 1e3);
else std::snprintf(buf, sizeof buf, "%lld", n);
return buf;
}
} // namespace
const char* OrchPanel::name_of(State s) {
switch (s) {
case State::Queued: return "queued";
case State::Running: return "running";
case State::Idle: return "idle";
case State::Done: return "done";
case State::Failed: return "failed";
case State::Timeout: return "timeout";
case State::Cancelled: return "cancelled";
}
return "running";
}
OrchPanel::OrchPanel(std::string label, std::string task, int remote_id, int columns)
: label_(OrchFormatter::clip(std::move(label), ORCH_LABEL_MAX)), task_(std::move(task)), remote_(remote_id),
started_(now_s()), last_event_(started_) {
fmt_.set_width(columns);
pending_ = fmt_.banner(task_);
}
void OrchPanel::set_width(int columns) {
std::lock_guard<std::mutex> lock(mu_);
fmt_.set_width(columns);
}
void OrchPanel::append_locked(const std::string& bytes) {
if (bytes.empty() || remote_ < 0) return;
pending_ += bytes;
if (pending_.size() <= ORCH_PENDING_MAX) return;
// Over the ceiling (the multiplexer is slower than the agent): drop the oldest bytes, loudly,
// cutting at a line end so no escape sequence or character is split.
size_t cut = pending_.size() - ORCH_PENDING_MAX / 2;
size_t line = pending_.find("\r\n", cut);
cut = line == std::string::npos ? pending_.size() : line + 2;
dropped_ += cut;
pending_.erase(0, cut);
pending_ = "\x1b[33m\xC2\xB7 output skipped: the screen fell behind\x1b[0m\r\n" + pending_;
}
void OrchPanel::emit(const OrchEvent& e) {
std::lock_guard<std::mutex> lock(mu_);
if (state_ != State::Running && state_ != State::Idle && state_ != State::Queued) return;
last_event_ = now_s();
if (state_ == State::Idle || state_ == State::Queued) state_ = State::Running;
if (e.kind == EventKind::ToolStart) ++tools_;
if (e.kind == EventKind::Usage) tokens_ = e.tokens;
if (e.kind == EventKind::Text || e.kind == EventKind::Thought) last_text_ = now_s();
append_locked(fmt_.render(e));
}
void OrchPanel::finish(State state, const std::string& summary) {
std::lock_guard<std::mutex> lock(mu_);
if (done_pending_ || done_sent_) return;
append_locked(fmt_.settle());
state_ = state;
ended_ = now_s();
summary_ = summary;
append_locked(fmt_.footer(name_of(state), summary, now_s() - started_));
done_pending_ = true;
}
void OrchPanel::set_cancel(CancelFn fn) {
std::lock_guard<std::mutex> lock(mu_);
cancel_ = std::move(fn);
}
OrchPanel::State OrchPanel::state() const { std::lock_guard<std::mutex> lock(mu_); return state_; }
bool OrchPanel::finished() const { std::lock_guard<std::mutex> lock(mu_); return done_pending_ || done_sent_; }
double OrchPanel::idle_seconds() const { std::lock_guard<std::mutex> lock(mu_); return now_s() - last_event_; }
double OrchPanel::elapsed() const { std::lock_guard<std::mutex> lock(mu_); return now_s() - started_; }
long long OrchPanel::tools() const { std::lock_guard<std::mutex> lock(mu_); return tools_; }
long long OrchPanel::tokens() const { std::lock_guard<std::mutex> lock(mu_); return tokens_; }
size_t OrchPanel::dropped_bytes() const { std::lock_guard<std::mutex> lock(mu_); return dropped_; }
std::string OrchPanel::header() const {
std::lock_guard<std::mutex> lock(mu_);
const char* icon = "\xE2\x97\x8F"; // running
switch (state_) {
case State::Done: icon = "\xE2\x9C\x93"; break;
case State::Failed: icon = "\xE2\x9C\x97"; break;
case State::Idle: icon = "\xE2\x97\x8C"; break;
case State::Timeout: icon = "\xE2\x8F\xB1"; break;
case State::Cancelled: icon = "\xE2\x8A\x98"; break;
default: break;
}
double secs = (ended_ > 0 ? ended_ : now_s()) - started_;
char t[24];
if (secs >= 60) std::snprintf(t, sizeof t, "%dm%02ds", static_cast<int>(secs) / 60, static_cast<int>(secs) % 60);
else std::snprintf(t, sizeof t, "%ds", static_cast<int>(secs));
std::string h = std::string(" ") + icon + " " + label_ + " \xC2\xB7 " + name_of(state_) + " \xC2\xB7 " + t;
if (tools_) h += " \xC2\xB7 " + std::to_string(tools_) + (tools_ == 1 ? " tool" : " tools");
if (tokens_) h += " \xC2\xB7 " + compact(tokens_) + " tok";
return h;
}
bool OrchPanel::collect(OrchOp& op) {
std::string head = header();
std::lock_guard<std::mutex> lock(mu_);
if (remote_ < 0 || user_closed_ || done_sent_) return false;
if (fmt_.holding() && now_s() - last_text_ > 0.1) append_locked(fmt_.settle()); // the stream paused mid-word
op = OrchOp();
op.id = remote_;
size_t take = std::min(pending_.size(), ORCH_WRITE_MAX);
while (take > 0 && take < pending_.size() && (static_cast<unsigned char>(pending_[take]) & 0xC0) == 0x80) --take; // not mid-character
op.data = pending_.substr(0, take);
pending_.erase(0, take);
bool changed = false;
if (head != last_header_) { op.header = head; last_header_ = head; changed = true; }
const std::string st = name_of(state_);
if (st != sent_state_) { op.state = st; sent_state_ = st; changed = true; }
if (done_pending_ && pending_.empty()) {
op.done = true;
op.linger = ORCH_LINGER;
done_sent_ = true;
changed = true;
}
return changed || !op.data.empty();
}
bool OrchPanel::drained() const {
std::lock_guard<std::mutex> lock(mu_);
return remote_ < 0 ? (done_pending_ || done_sent_) : (done_sent_ || user_closed_);
}
void OrchPanel::fire_cancel(const std::string& reason) {
CancelFn fn;
{
std::lock_guard<std::mutex> lock(mu_);
if (cancel_fired_ || !cancel_) return;
cancel_fired_ = true;
fn = cancel_;
}
try { fn(reason); } catch (...) {}
}
void OrchPanel::watchdog(double now) {
bool kill = false;
{
std::lock_guard<std::mutex> lock(mu_);
if (done_pending_ || done_sent_ || user_closed_) return;
double idle = now - last_event_;
if (state_ == State::Running && idle >= ORCH_IDLE_WARN) state_ = State::Idle;
if (idle >= ORCH_IDLE_KILL) kill = true;
}
if (kill) fire_cancel("no output for " + std::to_string(static_cast<int>(ORCH_IDLE_KILL)) + " s");
}
void OrchPanel::closed_by_user() {
{
std::lock_guard<std::mutex> lock(mu_);
if (user_closed_ || done_sent_) return;
user_closed_ = true;
ended_ = now_s();
state_ = State::Cancelled;
if (!done_pending_) { done_pending_ = true; summary_ = "pane closed by user"; }
}
fire_cancel("pane closed by user");
}
} // namespace detz::orch
+67
View File
@@ -0,0 +1,67 @@
// OrchPanel: one sub agent's window onto the multiplexer, and its monitoring record. Thread-safe:
// the agent thread (or a CAP pump thread) emits events, the board thread drains and watches. A panel
// that has no pane behind it (no multiplexer, no space) behaves identically except nothing is drawn,
// so call sites never branch on whether a screen exists.
#pragma once
#include <functional>
#include <mutex>
#include <string>
#include "orch_event.h"
#include "orch_formatter.h"
#include "orch_transport.h"
namespace detz::orch {
class OrchPanel {
public:
enum class State { Queued, Running, Idle, Done, Failed, Timeout, Cancelled };
using CancelFn = std::function<void(const std::string& reason)>;
OrchPanel(std::string label, std::string task, int remote_id, int columns = 80);
static const char* name_of(State s);
const std::string& label() const { return label_; }
bool shown() const { return remote_ >= 0; }
int remote_id() const { return remote_; }
// ---- producers (any thread)
void emit(const OrchEvent& event);
void finish(State state, const std::string& summary);
void set_cancel(CancelFn fn);
void set_width(int columns); // the pane's text width, for word wrap
// ---- observers
State state() const;
bool finished() const;
double idle_seconds() const;
double elapsed() const;
long long tools() const;
long long tokens() const;
size_t dropped_bytes() const;
std::string header() const;
// ---- board side
bool collect(OrchOp& op); // fills op when there is anything to send; false when quiet
void watchdog(double now); // idle warn / idle kill
void closed_by_user(); // the multiplexer lost the pane
bool drained() const; // finished and everything sent
private:
void append_locked(const std::string& bytes);
void fire_cancel(const std::string& reason);
std::string label_, task_;
int remote_;
mutable std::mutex mu_;
OrchFormatter fmt_;
std::string pending_;
size_t dropped_{0};
State state_{State::Running};
double started_, last_event_, ended_{0}, last_text_{0};
long long tools_{0}, tokens_{0};
std::string last_header_, sent_state_;
bool done_pending_{false}, done_sent_{false}, cancel_fired_{false}, user_closed_{false};
std::string summary_;
CancelFn cancel_;
};
} // namespace detz::orch
+23
View File
@@ -0,0 +1,23 @@
// Orchestration caps and floors (spec: docs/orchestration.md). Every number is a static ceiling or
// floor; nothing tunes itself. The server side twins live in mplex_vpane.h.
#pragma once
#include <cstddef>
namespace detz::orch {
constexpr int ORCH_PANELS_MAX = 8; // panels one process opens (= AGENTS_MAX_PARALLEL)
constexpr double ORCH_FLUSH_INTERVAL = 0.025; // seconds between writes to the multiplexer
constexpr double ORCH_MONITOR_INTERVAL = 1.0; // seconds between header refreshes and watchdog scans
constexpr size_t ORCH_PENDING_MAX = 262144; // unsent bytes per panel; older bytes are dropped loudly
constexpr size_t ORCH_WRITE_MAX = 32768; // bytes of one op (server accepts 65536)
constexpr double ORCH_LINGER = 20.0; // seconds a finished panel stays visible
constexpr double ORCH_IDLE_WARN = 60.0; // seconds without an event before the panel shows idle
constexpr double ORCH_IDLE_KILL = 300.0; // seconds without an event before the job is cancelled
constexpr double ORCH_CANCEL_GRACE = 5.0; // seconds between a soft cancel and closing the session
constexpr size_t ORCH_TASK_SHOW_MAX = 300; // characters of the task echoed at the top of a panel
constexpr size_t ORCH_TOOL_ARGS_MAX = 120; // characters of a tool call's arguments
constexpr size_t ORCH_TOOL_RESULT_LINES = 3; // lines of a tool result
constexpr size_t ORCH_TOOL_RESULT_COLS = 160; // characters per result line
constexpr size_t ORCH_LABEL_MAX = 60; // characters of a panel label
constexpr int ORCH_CALL_TIMEOUT_MS = 2000; // one socket round trip to the multiplexer
} // namespace detz::orch
+124
View File
@@ -0,0 +1,124 @@
#include "orch_transport.h"
#include <poll.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <cerrno>
#include <cstdlib>
#include <cstring>
#include "orch_policy.h"
#include "wire_json.h"
namespace detz::orch {
using Json = detz::wire::Json;
OrchMplexTransport::OrchMplexTransport(std::string socket_path, std::string origin_pane)
: socket_(std::move(socket_path)), origin_(std::move(origin_pane)) {}
std::unique_ptr<OrchTransport> OrchMplexTransport::from_env() {
const char* off = std::getenv("TAI_ORCH_PANELS");
const char* active = std::getenv("TAI_PLEXER_ACTIVE");
const char* sock = std::getenv("TAI_PLEXER_SOCKET");
const char* pane = std::getenv("TAI_PLEXER_PANE");
if ((off && std::string(off) == "0") || !active || !*active || !sock || !*sock) return std::make_unique<OrchNullTransport>();
return std::make_unique<OrchMplexTransport>(sock, pane ? pane : "");
}
std::string OrchMplexTransport::call(const std::string& line, std::string& error) {
int fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
if (fd < 0) { error = std::string("socket: ") + std::strerror(errno); return ""; }
struct sockaddr_un addr;
std::memset(&addr, 0, sizeof addr);
addr.sun_family = AF_UNIX;
if (socket_.size() >= sizeof addr.sun_path) { ::close(fd); error = "socket path too long"; return ""; }
std::memcpy(addr.sun_path, socket_.c_str(), socket_.size() + 1);
if (::connect(fd, reinterpret_cast<struct sockaddr*>(&addr), sizeof addr) != 0) {
error = std::string("connect: ") + std::strerror(errno);
::close(fd);
return "";
}
std::string data = line + "\n";
size_t sent = 0;
while (sent < data.size()) {
ssize_t n = ::send(fd, data.data() + sent, data.size() - sent, MSG_NOSIGNAL);
if (n < 0 && errno == EINTR) continue;
if (n <= 0) { error = std::string("send: ") + std::strerror(errno); ::close(fd); return ""; }
sent += static_cast<size_t>(n);
}
std::string reply;
char buf[8192];
for (;;) {
struct pollfd p = {fd, POLLIN, 0};
int rc = ::poll(&p, 1, ORCH_CALL_TIMEOUT_MS);
if (rc < 0 && errno == EINTR) continue;
if (rc <= 0) { error = "multiplexer did not answer"; ::close(fd); return ""; }
ssize_t n = ::recv(fd, buf, sizeof buf, 0);
if (n < 0 && errno == EINTR) continue;
if (n <= 0) break;
reply.append(buf, static_cast<size_t>(n));
if (reply.back() == '\n') break;
}
::close(fd);
if (reply.empty()) error = "empty reply";
return reply;
}
std::optional<int> OrchMplexTransport::open(const std::string& label, const std::string& header, std::string& error, int* cols) {
Json req = Json::object();
req.set("cmd", "vpane_open");
if (!origin_.empty()) req.set("pane", origin_);
req.set("label", label);
req.set("header", header);
std::string reply = call(req.dumps(), error);
if (reply.empty()) return std::nullopt;
try {
Json r = Json::parse(reply);
if (!r.get("ok").truthy()) { error = r.get("error").isString() ? r.get("error").str() : "refused"; return std::nullopt; }
if (cols && r.get("data").get("cols").isNumber()) *cols = static_cast<int>(r.get("data").get("cols").asInt());
return static_cast<int>(r.get("data").get("id").asInt());
} catch (const std::exception& exc) {
error = std::string("bad reply: ") + exc.what();
return std::nullopt;
}
}
bool OrchMplexTransport::write(const std::vector<OrchOp>& ops, std::vector<int>& closed, std::string& error,
std::vector<std::pair<int, int>>* widths) {
if (ops.empty()) return true;
Json list = Json::array();
for (const OrchOp& op : ops) {
Json o = Json::object();
o.set("pane", op.id);
if (!op.data.empty()) o.set("data", op.data);
if (op.header) o.set("header", *op.header);
if (op.state) o.set("state", *op.state);
if (op.done) { o.set("done", true); o.set("linger", op.linger); }
list.push(o);
}
Json req = Json::object();
req.set("cmd", "vpane_write");
req.set("ops", list);
std::string reply = call(req.dumps(), error);
if (reply.empty()) return false;
try {
Json r = Json::parse(reply);
if (!r.get("ok").truthy()) { error = r.get("error").isString() ? r.get("error").str() : "refused"; return false; }
for (const Json& id : r.get("data").get("closed").items()) closed.push_back(static_cast<int>(id.asInt()));
if (widths && r.get("data").get("cols").isObject())
for (const auto& m : r.get("data").get("cols").members()) widths->push_back({std::atoi(m.first.c_str()), static_cast<int>(m.second.asInt())});
} catch (const std::exception& exc) {
error = std::string("bad reply: ") + exc.what();
return false;
}
return true;
}
void OrchMplexTransport::close(int id) {
Json req = Json::object();
req.set("cmd", "vpane_close");
req.set("pane", id);
std::string error;
call(req.dumps(), error);
}
} // namespace detz::orch
+55
View File
@@ -0,0 +1,55 @@
// OrchTransport: how panel bytes reach a multiplexer. OrchMplexTransport speaks the control socket
// of tai's own multiplexer (vpane_* commands, docs/orchestration.md); OrchNullTransport shows nothing.
#pragma once
#include <memory>
#include <optional>
#include <string>
#include <utility>
#include <vector>
namespace detz::orch {
struct OrchOp {
int id{-1};
std::string data; // bytes for the pane screen (CRLF, SGR)
std::optional<std::string> header; // new header line
std::optional<std::string> state; // running | idle | done | failed | timeout | cancelled
bool done{false}; // start the linger countdown
double linger{0};
};
class OrchTransport {
public:
virtual ~OrchTransport() = default;
virtual bool available() const = 0;
// Opens a pane; returns its id, or nullopt with `error` set (no space, too many, socket down).
virtual std::optional<int> open(const std::string& label, const std::string& header, std::string& error, int* cols = nullptr) = 0;
// Sends all ops in one round trip. `closed` receives ids the multiplexer no longer has (the user
// closed the pane). Returns false with `error` set when the multiplexer cannot be reached.
virtual bool write(const std::vector<OrchOp>& ops, std::vector<int>& closed, std::string& error,
std::vector<std::pair<int, int>>* widths = nullptr) = 0;
virtual void close(int id) = 0;
};
class OrchNullTransport : public OrchTransport {
public:
bool available() const override { return false; }
std::optional<int> open(const std::string&, const std::string&, std::string& error, int*) override { error = "no multiplexer"; return std::nullopt; }
bool write(const std::vector<OrchOp>&, std::vector<int>&, std::string&, std::vector<std::pair<int, int>>*) override { return true; }
void close(int) override {}
};
class OrchMplexTransport : public OrchTransport {
public:
OrchMplexTransport(std::string socket_path, std::string origin_pane);
bool available() const override { return !socket_.empty(); }
std::optional<int> open(const std::string& label, const std::string& header, std::string& error, int* cols) override;
bool write(const std::vector<OrchOp>& ops, std::vector<int>& closed, std::string& error, std::vector<std::pair<int, int>>* widths) override;
void close(int id) override;
// The multiplexer this process runs in (TAI_PLEXER_*), or the null transport; TAI_ORCH_PANELS=0 turns panels off.
static std::unique_ptr<OrchTransport> from_env();
private:
std::string call(const std::string& line, std::string& error); // one request, one reply line; "" on failure
std::string socket_, origin_;
};
} // namespace detz::orch
+47
View File
@@ -0,0 +1,47 @@
// Driver for tests/orch_mplex_e2e.py: opens N panels on the multiplexer named by TAI_PLEXER_* and
// streams scripted events through the real OrchBoard and OrchMplexTransport.
// orch_live N TOKENS HOLD_MS [fail] prints "id label" per panel, finishes them, flushes.
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <thread>
#include <vector>
#include "orch_board.h"
using namespace detz::orch;
int main(int argc, char** argv) {
int n = argc > 1 ? std::atoi(argv[1]) : 1;
int tokens = argc > 2 ? std::atoi(argv[2]) : 50;
int hold_ms = argc > 3 ? std::atoi(argv[3]) : 0;
bool fail_last = argc > 4;
OrchBoard& board = OrchBoard::instance();
std::vector<std::shared_ptr<OrchPanel>> panels;
for (int i = 0; i < n; ++i) {
auto p = board.open("agent #" + std::to_string(i + 1), "scripted task number " + std::to_string(i + 1));
std::printf("%d %s %s\n", p->remote_id(), p->label().c_str(), p->shown() ? "shown" : board.last_error().c_str());
panels.push_back(p);
}
std::fflush(stdout);
std::vector<std::thread> workers;
for (int i = 0; i < n; ++i) {
workers.emplace_back([&, i] {
auto& p = panels[static_cast<size_t>(i)];
p->emit(OrchEvent::tool_start("shell", "{\"command\":\"echo " + std::to_string(i) + "\"}"));
p->emit(OrchEvent::tool_end("shell", "line one\nline two\nline three\nline four", false, 12));
for (int t = 0; t < tokens; ++t) {
p->emit(OrchEvent::text_of("word" + std::to_string(i) + "_" + std::to_string(t) + " "));
p->emit(OrchEvent::usage_of(t * 10));
std::this_thread::sleep_for(std::chrono::milliseconds(2));
}
p->emit(OrchEvent::text_of("\nTHE END " + std::to_string(i) + "\n"));
std::this_thread::sleep_for(std::chrono::milliseconds(hold_ms));
bool bad = fail_last && i == n - 1;
p->finish(bad ? OrchPanel::State::Failed : OrchPanel::State::Done, bad ? "scripted failure" : "all good");
});
}
for (auto& w : workers) w.join();
board.flush(3.0);
return 0;
}
+149
View File
@@ -0,0 +1,149 @@
#!/usr/bin/env python3
"""End to end: the real multiplexer server plus the real OrchBoard/OrchMplexTransport.
python3 -I tests/orch_mplex_e2e.py DETZ_BINARY ORCH_LIVE_BINARY
Checks layout geometry for 1, 2, 3 and 6 sub agents, streamed content, footers, the space limit,
linger and rebalance, the user closing a pane, the main pane exiting, and that a one-shot control
write reports panes that no longer exist. Everything runs in a throwaway TAI_HOME.
"""
import json, os, re, shutil, socket, subprocess, sys, tempfile, time
DETZ, LIVE = os.path.abspath(sys.argv[1]), os.path.abspath(sys.argv[2])
HOME = tempfile.mkdtemp(prefix="oe", dir="/tmp")
ENV = dict(os.environ, TAI_HOME=HOME, HOME=HOME)
for k in ("TAI_PLEXER_ACTIVE", "TAI_PLEXER_SOCKET", "TAI_PLEXER_PANE", "TMUX"):
ENV.pop(k, None)
SOCK = os.path.join(HOME, "plexer", "server.sock")
fails = []
def check(cond, what):
print(("ok " if cond else "FAIL ") + what)
if not cond:
fails.append(what)
def mp(*args):
return subprocess.run([DETZ, "--mplex", *args], env=ENV, capture_output=True, text=True, timeout=30)
def call(**kw):
s = socket.socket(socket.AF_UNIX)
s.connect(SOCK)
s.sendall((json.dumps(kw) + "\n").encode())
buf = b""
while not buf.endswith(b"\n"):
chunk = s.recv(65536)
if not chunk:
break
buf += chunk
s.close()
return json.loads(buf)
def panes():
out = mp("list-panes", "-F", "#{pane_id} #{pane_left} #{pane_top} #{pane_width} #{pane_height}").stdout.split("\n")
rows = []
for line in out:
m = re.match(r"%(\d+) (\d+) (\d+) (\d+) (\d+)$", line.strip())
if m:
rows.append(tuple(int(x) for x in m.groups()))
return rows
def run_live(n, tokens=40, hold=0, fail=False, wait=True):
env = dict(ENV, TAI_PLEXER_ACTIVE="1", TAI_PLEXER_SOCKET=SOCK, TAI_PLEXER_PANE="0")
cmd = [LIVE, str(n), str(tokens), str(hold)] + (["fail"] if fail else [])
if wait:
r = subprocess.run(cmd, env=env, capture_output=True, text=True, timeout=60)
return r.stdout
return subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, text=True)
def capture(pid):
return mp("capture-pane", "-p", "-t", "%" + str(pid)).stdout
try:
r = mp("new-session", "-d", "-s", "t", "-x", "120", "-y", "40")
check(r.returncode == 0, "detached session starts")
# ---- 1 sub agent: main left full height, one pane right
p = run_live(1, hold=1500, wait=False)
time.sleep(0.6)
rows = panes()
check(len(rows) == 2, "one sub agent: two panes")
main = [r for r in rows if r[0] == 0][0]
sub = [r for r in rows if r[0] != 0]
check(main[1] == 0 and main[2] == 0 and main[4] == 40, "main pane on the left at full height")
check(len(sub) == 1 and sub[0][1] > main[1] + main[3] - 1 and sub[0][4] == 40, "sub pane on the right at full height")
check(abs(main[3] - 120 * 0.55) <= 2, "main pane is about 55 percent wide")
out = p.communicate(timeout=30)[0]
check("shown" in out, "board reports the panel shown")
time.sleep(0.3)
text = capture(sub[0][0])
check("scripted task number 1" in text, "task echoed at the top")
check("shell" in text and "line three" in text and "line four" not in text, "tool call and a three-line result")
check("THE END 0" in text and "word0_39" in text, "streamed text arrived complete")
check("done" in text and "all good" in text, "footer states the result")
# linger then rebalance
time.sleep(21.5)
check(len(panes()) == 1, "panel leaves after the linger and the main pane takes the screen")
# ---- 3 sub agents: stacked evenly on the right
p = run_live(3, tokens=30, hold=1200, wait=False)
time.sleep(0.8)
rows = panes()
subs = sorted([r for r in rows if r[0] != 0], key=lambda r: r[2])
check(len(subs) == 3, "three sub agents: three panes on the right")
check(len({r[1] for r in subs}) == 1 and subs[0][1] > 0, "all in one column")
heights = [r[4] for r in subs]
check(max(heights) - min(heights) <= 2, "even heights " + str(heights))
check(sum(heights) + 2 == 40, "column fills the window")
p.communicate(timeout=30)
time.sleep(0.3)
for r in subs:
check("THE END" in capture(r[0]), "pane %%%d content" % r[0])
time.sleep(21.5)
check(len(panes()) == 1, "all three leave")
# ---- space limit: the pane that does not fit runs without a panel
p = run_live(8, tokens=5, hold=800, wait=False)
time.sleep(1.2)
rows = panes()
check(len(rows) - 1 == 6, "only as many panels as fit (6 rows each): %d" % (len(rows) - 1))
out = p.communicate(timeout=30)[0]
check(out.count("shown") == 6 and "no space" in out, "the others report no space instead of failing")
time.sleep(21.5)
# ---- failure state and a manual close = cancel
p = run_live(2, tokens=400, hold=3000, fail=True, wait=False)
time.sleep(0.8)
subs = sorted([r for r in panes() if r[0] != 0], key=lambda r: r[2])
victim = subs[0][0]
lst = call(cmd="vpane_list")["data"]
check(len(lst) == 2 and all(x["state"] == "running" for x in lst), "list reports running panels")
mp("kill-pane", "-t", "%" + str(victim))
time.sleep(0.5)
reply = call(cmd="vpane_write", ops=[{"pane": victim, "data": "x"}])
check(victim in reply["data"]["closed"], "a write to a closed pane reports it closed")
p.communicate(timeout=60)
time.sleep(0.4)
left = [r for r in panes() if r[0] != 0]
check(len(left) == 1, "the other pane survived")
check("scripted failure" in capture(left[0][0]) and "failed" in capture(left[0][0]), "failed footer")
# ---- main pane exits: feed panes go with the window
call(cmd="vpane_open", pane="0", label="orphan")
check(len(panes()) >= 2, "a feed pane is open")
mp("send-keys", "-t", "%0", "exit", "Enter")
time.sleep(1.5)
check(mp("list-sessions").stdout.strip() == "" or "no server" in (mp("list-sessions").stderr + mp("list-sessions").stdout).lower(),
"session ends with its main pane")
finally:
mp("kill-server")
shutil.rmtree(HOME, ignore_errors=True)
print("\n%d failures" % len(fails))
sys.exit(1 if fails else 0)
+267
View File
@@ -0,0 +1,267 @@
// Orchestration: formatter, panel, board over a fake transport, CAP adapter, spec constants.
#include <atomic>
#include <chrono>
#include <cstdio>
#include <fstream>
#include <sstream>
#include <string>
#include <thread>
#include <vector>
#include "orch_board.h"
#include "orch_cap_adapter.h"
#include "orch_formatter.h"
#include "orch_panel.h"
#include "orch_policy.h"
using namespace detz::orch;
using Json = detz::wire::Json;
static int failures = 0;
#define CHECK(c) do { if (!(c)) { std::printf("FAIL %s:%d %s\n", __FILE__, __LINE__, #c); ++failures; } } while (0)
static double now_s() { return std::chrono::duration<double>(std::chrono::steady_clock::now().time_since_epoch()).count(); }
struct FakeTransport : OrchTransport {
std::vector<std::vector<OrchOp>> writes;
std::vector<int> opened;
std::vector<int> to_close; // ids reported as closed on the next write
int next_id = 10;
int max_open = 1000;
bool fail_writes = false;
bool available() const override { return true; }
std::optional<int> open(const std::string&, const std::string&, std::string& error, int*) override {
if (static_cast<int>(opened.size()) >= max_open) { error = "no space for another feed pane"; return std::nullopt; }
opened.push_back(next_id);
return next_id++;
}
bool write(const std::vector<OrchOp>& ops, std::vector<int>& closed, std::string& error, std::vector<std::pair<int, int>>*) override {
if (fail_writes) { error = "down"; return false; }
writes.push_back(ops);
closed = to_close;
to_close.clear();
return true;
}
void close(int) override {}
};
static std::string all_data(const FakeTransport& t, int id) {
std::string s;
for (const auto& w : t.writes) for (const auto& op : w) if (op.id == id) s += op.data;
return s;
}
int main() {
// ---- formatter
CHECK(OrchFormatter::sanitize("a\x1b[31mred\x1b[0m\x1b[2Jb\x1b]0;title\x07 c\x01\r") == "a\x1b[31mred\x1b[0mb c");
CHECK(OrchFormatter::sanitize("x\x1b[10;5Hy") == "xy"); // cursor movement dropped
CHECK(OrchFormatter::crlf("a\nb") == "a\r\nb");
CHECK(OrchFormatter::clip("h\xC3\xA9llo world", 5) == "h\xC3\xA9llo..."); // by character, not byte
CHECK(OrchFormatter::one_line("a \n\t b c ") == "a b c");
{
OrchFormatter f;
std::string t = f.render(OrchEvent::text_of("partial"));
bool held = f.holding();
std::string c = f.render(OrchEvent::tool_start("shell", "{\"command\":\"ls\"}"));
CHECK(t.empty() && held); // the trailing word is held: more of it may come
CHECK(c.rfind("partial\r\n", 0) == 0); // a tool line flushes it and starts on a new line
std::string r = f.render(OrchEvent::tool_end("shell", "l1\nl2\nl3\nl4\n", false, 5));
CHECK(r.find("l3") != std::string::npos && r.find("l4") == std::string::npos); // three lines
CHECK(r.find("chars") != std::string::npos);
CHECK(f.render(OrchEvent::error_of("boom")).find("boom") != std::string::npos);
CHECK(f.render(OrchEvent::usage_of(5)).empty());
}
// ---- word wrap
{
OrchFormatter f;
f.set_width(20);
std::string out = f.render(OrchEvent::text_of("alpha beta gamma delta epsilon "));
CHECK(out.find("alpha beta gamma\r\ndelta") != std::string::npos || out.find("\r\n") != std::string::npos);
CHECK(out.find("gam\r\nma") == std::string::npos); // never inside a word
OrchFormatter g;
g.set_width(20);
std::string a = g.render(OrchEvent::text_of("tok1 tok2 to"));
std::string b = g.render(OrchEvent::text_of("ken3 tok4 "));
CHECK((a + b).find("token3") != std::string::npos); // a word split across chunks is rejoined
CHECK(g.settle().empty());
OrchFormatter h;
h.set_width(12);
std::string longword = h.render(OrchEvent::text_of(std::string(30, 'z') + " end "));
CHECK(longword.find("\r\n") != std::string::npos); // a word longer than the line is broken
std::string wide = OrchFormatter().render(OrchEvent::text_of("\xE2\x9C\x93\xE2\x9C\x93 \xE2\x9C\x93 "));
CHECK(wide.find("\xE2\x9C\x93\xE2\x9C\x93") != std::string::npos); // multibyte kept whole
}
// ---- CAP adapter
{
OrchCapAdapter a;
Json start = Json::object();
start.set("kind", "cap.tool_call.start"); start.set("call_id", "c1"); start.set("name", "shell"); start.set("input", Json::object());
auto s = a.map(start);
CHECK(s && s->kind == EventKind::ToolStart && s->name == "shell");
Json end = Json::object();
end.set("kind", "cap.tool_call.end"); end.set("call_id", "c1"); end.set("output", "ok"); end.set("is_error", true); end.set("duration_ms", 7);
auto e = a.map(end);
CHECK(e && e->kind == EventKind::ToolEnd && e->name == "shell" && e->failed && e->millis == 7);
Json txt = Json::object();
txt.set("kind", "cap.text_chunk"); txt.set("text", "hi"); txt.set("channel", "thought");
auto t = a.map(txt);
CHECK(t && t->kind == EventKind::Thought && t->text == "hi\n");
Json use = Json::object();
use.set("kind", "cap.usage"); use.set("input_tokens", 10); use.set("output_tokens", 5);
CHECK(a.map(use) && a.map(use)->tokens == 15);
Json other = Json::object();
other.set("kind", "cap.fs.change");
CHECK(!a.map(other));
}
// ---- panel: buffering, ceiling, watchdog, cancel
{
OrchPanel p("agent #1", "do the thing", 5);
p.emit(OrchEvent::text_of("hello"));
p.emit(OrchEvent::tool_start("shell", "x"));
p.emit(OrchEvent::usage_of(1500));
CHECK(p.tools() == 1 && p.tokens() == 1500);
CHECK(p.header().find("agent #1") != std::string::npos && p.header().find("1.5k tok") != std::string::npos);
OrchOp op;
CHECK(p.collect(op) && op.id == 5 && op.data.find("do the thing") != std::string::npos && op.data.find("hello") != std::string::npos);
CHECK(op.header && op.state && *op.state == "running");
CHECK(!p.collect(op)); // quiet panel sends nothing
std::atomic<int> fired{0};
std::string why;
p.set_cancel([&](const std::string& r) { why = r; ++fired; });
p.watchdog(now_s() + ORCH_IDLE_WARN + 1);
CHECK(p.state() == OrchPanel::State::Idle && fired == 0);
p.emit(OrchEvent::text_of("again"));
CHECK(p.state() == OrchPanel::State::Running); // an event is a heartbeat
p.watchdog(now_s() + ORCH_IDLE_KILL + 1);
CHECK(fired == 1 && why.find("no output") != std::string::npos);
p.watchdog(now_s() + ORCH_IDLE_KILL + 2);
CHECK(fired == 1); // fires once
p.finish(OrchPanel::State::Done, "42 chars");
CHECK(p.finished());
p.emit(OrchEvent::text_of("late")); // ignored after finish
OrchOp last;
CHECK(p.collect(last) && last.done && last.linger == ORCH_LINGER && last.data.find("late") == std::string::npos);
CHECK(last.data.find("done") != std::string::npos && p.drained());
}
{
OrchPanel flood("flood", "t", 1);
std::string line(1000, 'x');
line += "\n";
for (int i = 0; i < 600; ++i) flood.emit(OrchEvent::text_of(line));
OrchOp op;
CHECK(flood.collect(op));
CHECK(flood.dropped_bytes() > 0); // over the ceiling: dropped loudly
CHECK(op.data.size() <= ORCH_WRITE_MAX);
}
{
OrchPanel hidden("hidden", "t", -1); // no pane behind it
hidden.emit(OrchEvent::text_of("x"));
OrchOp op;
CHECK(!hidden.collect(op) && !hidden.shown());
std::atomic<int> fired{0};
hidden.set_cancel([&](const std::string&) { ++fired; });
hidden.watchdog(now_s() + ORCH_IDLE_KILL + 1);
CHECK(fired == 1); // supervised although not shown
}
{
OrchPanel utf("u", "t", 1);
std::string emoji;
for (int i = 0; i < 20000; ++i) emoji += "\xE2\x9C\x93"; // 3-byte characters across the write limit
utf.emit(OrchEvent::text_of(emoji));
OrchOp op;
CHECK(utf.collect(op));
CHECK(op.data.size() <= ORCH_WRITE_MAX && (op.data.size() % 3 == 0 || op.data.find("\xE2\x96\xB8") != std::string::npos));
unsigned char first_next = 0;
OrchOp op2;
if (utf.collect(op2) && !op2.data.empty()) first_next = static_cast<unsigned char>(op2.data[0]);
CHECK((first_next & 0xC0) != 0x80); // never splits a character
}
// ---- board over a fake transport
{
auto fake = std::make_unique<FakeTransport>();
FakeTransport* t = fake.get();
OrchBoard board(std::move(fake));
auto a = board.open("agent #1", "task a");
auto b = board.open("agent #2", "task b");
CHECK(a->shown() && b->shown() && board.shown() == 2);
for (int i = 0; i < 200; ++i) {
a->emit(OrchEvent::text_of("tok" + std::to_string(i) + " "));
if (i % 50 == 0) std::this_thread::sleep_for(std::chrono::milliseconds(5));
}
b->emit(OrchEvent::text_of("from b"));
a->finish(OrchPanel::State::Done, "ok");
std::this_thread::sleep_for(std::chrono::milliseconds(200)); // the held word is settled after a pause
board.flush(2.0);
std::string da = all_data(*t, a->remote_id());
CHECK(da.find("tok0 ") != std::string::npos && da.find("tok199 ") != std::string::npos);
CHECK(da.find("tok5 ") < da.find("tok6 ")); // order kept
CHECK(t->writes.size() < 120); // coalesced: far fewer writes than events
CHECK(all_data(*t, b->remote_id()).find("from b") != std::string::npos);
bool done_op = false;
for (const auto& w : t->writes) for (const auto& op : w) if (op.id == a->remote_id() && op.done) done_op = true;
CHECK(done_op);
// the user closes b's pane: the board learns it from the write reply and cancels the job
std::atomic<int> fired{0};
b->set_cancel([&](const std::string&) { ++fired; });
t->to_close.push_back(b->remote_id());
b->emit(OrchEvent::text_of("more"));
for (int i = 0; i < 100 && fired == 0; ++i) std::this_thread::sleep_for(std::chrono::milliseconds(10));
CHECK(fired == 1 && b->state() == OrchPanel::State::Cancelled);
board.stop();
}
{
auto fake = std::make_unique<FakeTransport>();
fake->max_open = 1;
OrchBoard board(std::move(fake), false);
auto a = board.open("a", "t");
auto b = board.open("b", "t"); // no space: still a usable panel
CHECK(a->shown() && !b->shown() && !board.last_error().empty());
b->emit(OrchEvent::text_of("works anyway"));
b->finish(OrchPanel::State::Done, "ok");
CHECK(b->finished());
}
{
auto fake = std::make_unique<FakeTransport>();
fake->fail_writes = true;
OrchBoard board(std::move(fake), false);
auto a = board.open("a", "t");
for (int i = 0; i < 5; ++i) { a->emit(OrchEvent::text_of("x ")); board.flush(0.05); }
CHECK(!board.transport_ok()); // given up after repeated failures
CHECK(board.last_error().find("unreachable") != std::string::npos);
CHECK(!board.open("b", "t")->shown());
}
{
auto fake = std::make_unique<FakeTransport>();
OrchBoard board(std::move(fake), false);
std::vector<std::shared_ptr<OrchPanel>> keep;
for (int i = 0; i < ORCH_PANELS_MAX + 2; ++i) keep.push_back(board.open("p" + std::to_string(i), "t"));
CHECK(board.shown() == static_cast<size_t>(ORCH_PANELS_MAX)); // the ceiling
CHECK(!keep.back()->shown());
}
// ---- the spec lists every constant of orch_policy.h and mplex_vpane.h
{
std::ifstream in("docs/orchestration.md");
std::stringstream ss;
ss << in.rdbuf();
std::string doc = ss.str();
CHECK(!doc.empty());
for (const char* name : {"ORCH_PANELS_MAX", "ORCH_FLUSH_INTERVAL", "ORCH_MONITOR_INTERVAL", "ORCH_PENDING_MAX", "ORCH_WRITE_MAX",
"ORCH_LINGER", "ORCH_IDLE_WARN", "ORCH_IDLE_KILL", "ORCH_CANCEL_GRACE", "ORCH_TASK_SHOW_MAX",
"ORCH_TOOL_ARGS_MAX", "ORCH_TOOL_RESULT_LINES", "ORCH_TOOL_RESULT_COLS", "ORCH_LABEL_MAX",
"ORCH_CALL_TIMEOUT_MS", "MPLEX_VPANE_MAX", "MPLEX_VPANE_MIN_ROWS", "MPLEX_VPANE_MIN_COLS",
"MPLEX_VPANE_MAIN_PERCENT", "MPLEX_VPANE_OPS_MAX", "MPLEX_VPANE_WRITE_MAX", "MPLEX_VPANE_HEADER_MAX",
"MPLEX_VPANE_LEASE", "MPLEX_VPANE_ORPHAN_LINGER", "MPLEX_VPANE_LINGER_MAX"}) {
if (doc.find(name) == std::string::npos) { std::printf("FAIL docs/orchestration.md lacks %s\n", name); ++failures; }
}
for (const char* cmd : {"vpane_open", "vpane_write", "vpane_close", "vpane_list"})
if (doc.find(cmd) == std::string::npos) { std::printf("FAIL docs/orchestration.md lacks %s\n", cmd); ++failures; }
}
std::printf(failures ? "FAILED %d\n" : "orch_test OK\n", failures);
return failures ? 1 : 0;
}
+5
View File
@@ -58,6 +58,11 @@ STEPS = [
("capmcp.unit", "cap/mcp", [[b("capmcp/capmcp_unit")]], None),
("capmcp.wire_diff", "cap/mcp", [[PY, "-I", t("capmcp_diff.py"), b("capmcp/capmcp_probe")]], None),
("capmcp.cli_diff", "cap/mcp", [[PY, "-I", t("capmcp_cli_diff.py"), b("capmcp/capmcp_cli"), "--tai", TAI]], None),
("capmcp.catalog", "cap/mcp", [[b("capmcp/cap_model_catalog_test")]], None),
("fuzzy.unit", "util", [[b("fuzzy_test")]], None),
# orchestration (sub agent panels on the multiplexer)
("orch.unit", "orch", [[b("orch_test")]], None),
("orch.mplex_e2e", "orch", [[PY, "-I", t("orch_mplex_e2e.py"), b("detz"), b("orch_live")]], None),
# misc
("misc.unit", "misc", [[b("misc/misc_test")]], None),
("misc.diff", "misc", [[PY, t("misc_diff.py")]], None),