From 1ebb7c637db84fc9f4898ad7adf2cfb2de5eb6a0 Mon Sep 17 00:00:00 2001 From: retoor Date: Wed, 28 Oct 2026 13:19:03 +0100 Subject: [PATCH] docs: commit markdown previously hidden by the *.md ignore --- apps/MODELSEQUENCE.md | 3 + apps/README.md | 1 + examples/README.md | 21 + muse-protocol-transpiler.md | 1306 ++++++++++++++++++++++++++ taicpp/SPEC.md | 6 + taicpp/docs/config.md | 14 + taicpp/docs/embed.md | 9 + taicpp/docs/termshot.md | 120 +++ taicpp/docs/versioning.md | 2 + taicpp/docs/workflow.md | 2 + taicpp/grr.md | 3 + taicpp/plan.md | 118 +++ taicpp/tests/whatsapp_data/README.md | 12 + 13 files changed, 1617 insertions(+) create mode 100644 apps/MODELSEQUENCE.md create mode 100644 apps/README.md create mode 100644 examples/README.md create mode 100644 muse-protocol-transpiler.md create mode 100644 taicpp/SPEC.md create mode 100644 taicpp/docs/config.md create mode 100644 taicpp/docs/embed.md create mode 100644 taicpp/docs/termshot.md create mode 100644 taicpp/docs/versioning.md create mode 100644 taicpp/docs/workflow.md create mode 100644 taicpp/grr.md create mode 100644 taicpp/plan.md create mode 100644 taicpp/tests/whatsapp_data/README.md diff --git a/apps/MODELSEQUENCE.md b/apps/MODELSEQUENCE.md new file mode 100644 index 0000000..04ffc91 --- /dev/null +++ b/apps/MODELSEQUENCE.md @@ -0,0 +1,3 @@ +App name is model_sequence. + +It is an app based on the diff --git a/apps/README.md b/apps/README.md new file mode 100644 index 0000000..0119692 --- /dev/null +++ b/apps/README.md @@ -0,0 +1 @@ +These are all terminal apps that are convenient to use with the lead24 aopplication... diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..65907f9 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,21 @@ +# tai examples + +Complete, runnable, stdlib-only examples in tai-spirit: single file, zero deps, Python 3.6+. + +Elke example is direct draaibaar, geen pip, geen build. Kopieer het bestand, run het. + +| # | Example | Run | Wat laat het zien | +|---|---------|-----|-------------------| +| 01 | [WebDAV server](webdav-server.py) | `python3 webdav-server.py --root ./share --port 8080` | Complete WebDAV Class 1: PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, Range, ETag, Basic-auth, jail. Werkt met cadaver, Nautilus, rclone, curl. | +| 02 | [Todo REST API](todo-rest-api.py) | `python3 todo-rest-api.py --port 8081` | REST JSON API + SQLite + embedded UI. Filtering, paging, validation, concurrency-safe. | +| 03 | [SSE chat](sse-chat.py) | `python3 sse-chat.py --port 8082` | Multi-room chat met Server-Sent Events. Geen JS-deps, history, heartbeat, graceful reconnect. | +| 04 | [Reverse proxy](reverse-proxy.py) | `python3 reverse-proxy.py --backends http://127.0.0.1:9001,http://127.0.0.1:9002` | Round-robin proxy met healthchecks, retries, hop-header hygiene, X-Forwarded-*. | +| 05 | [FTS search](fts-search.py) | `python3 fts-search.py index ./docs && python3 fts-search.py serve --port 8083` | SQLite FTS5 full-text search over een directory. BM25 ranking, snippet highlights, HTTP UI + CLI. | + +Conventies (zoals tai.py): + +- Alleen stdlib imports. +- Werkt op Python 3.6+. +- `--help` op elk bestand. +- Harde caps: request size, upload size, history lengte. +- Veilige paden: geen traversal buiten root. diff --git a/muse-protocol-transpiler.md b/muse-protocol-transpiler.md new file mode 100644 index 0000000..e76fc9f --- /dev/null +++ b/muse-protocol-transpiler.md @@ -0,0 +1,1306 @@ +# Muse Responses API: protocol documentation and transpiler + +Status snapshot: 2026-10-11. Research complete and archived under +`research/muse-responses-openai-transpiler/`. Transpiler module complete, +stdlib-only, 21/21 unit tests green; devplacepy gateway integration in +progress (module tests green, gateway wiring under test). + +## 1. What this document is + +One place for everything learned about the API that Meta's `muse` CLI +speaks, plus the complete bidirectional transpiler between that wire +format and classic OpenAI Chat Completions. Part 1 is the research +(what the protocol is, with evidence). Part 2 is the transpiler +(contract, usage, full source). Nothing here is theorized: request-side +claims come from a captured MITM log of the real CLI, schema claims +come from the official OpenAI docs fetched the same day, and every +mapping in the code is covered by a test. + +## 2. The wire protocol + +### 2.1 Endpoints + +The CLI talks to `https://api.meta.ai/v1` (from its own +`~/.config/muse/auth.json`, `providers.meta.api_base_url`, overridable +per-run with `--base-url`): + +| Method | Path | Purpose | +|---|---|---| +| `POST` | `/responses` | Model turn (OpenAI Responses API shape) | +| `GET` | `/muse-code/models` | Model listing (Meta-specific path) | + +The capture (`/tmp/mitm_probe/request.txt`, 795KB, 11 requests) shows +one models GET plus ten `/responses` POSTs in two alternating shapes: +the main turn (full 19-function tool namespace, `reasoning.effort` +`max`) and a reminder turn (single `submit_reminder_decision` tool, +`effort` `low`). + +### 2.2 Headers + +Every request carries `Accept: text/event-stream` (streaming SSE), +`Content-Type: application/json`, `Authorization: Bearer ...` +(redacted in the capture, never logged), a client UA +(`muse-build/...`), `x-client-id: tbh:exec`, `x-tbh-session-id`, and +`x-meta-ai-gateway-session-id`. The `x-tbh-*` / `x-meta-ai-gateway-*` +trio is Meta-specific; the rest is stock OpenAI practice. + +### 2.3 Request schema + +All ten POSTs share these top-level keys: `model`, `instructions`, +`input`, `tools`, `stream`, `store`, `max_output_tokens`, `reasoning`, +`include`, `prompt_cache_key`. Observed values: `stream: true`, +`store: false`, `max_output_tokens: 128000`, +`reasoning: {effort, summary: "auto"}`, +`include: ["reasoning.encrypted_content"]`, +`prompt_cache_key: "tbh:skill-reminder:..."`, model +`muse-spark-1.3-contributor`. The client is stateless (`store: false`) +and replays encrypted reasoning items instead of server-side +conversation state. + +Illustrative Meta-flavored request (trimmed, synthetic content): + +```json +{ + "model": "muse-spark-1.3-contributor", + "instructions": "<25-44KB system string>", + "input": [ + {"type": "message", "role": "developer", "content": "<31KB reminder>"}, + {"type": "message", "role": "user", "content": "hi"} + ], + "tools": [{"type": "namespace", "name": "muse", "tools": ["...19 builtins..."]}], + "stream": true, + "store": false, + "max_output_tokens": 128000, + "reasoning": {"effort": "max", "summary": "auto"}, + "include": ["reasoning.encrypted_content"], + "prompt_cache_key": "tbh:skill-reminder:..." +} +``` + +### 2.4 The tools namespace + +`tools` holds exactly one entry: `{type: "namespace", name: "muse"}` +with 19 nested harness functions (`workflow`, `read_file`, `search`, +`write_file`, `edit_file`, `bash`, `web_fetch`, `web_search`, +`monitor`, `cron_*`, `subagent_*`, `read_skill`, `add_memory`, +`edit_memory`, `read_memory`, `request_user_input`, `snooze_reminder`, +`update_goal`, ...). The reminder shape swaps this for a single +`submit_reminder_decision` function. `namespace` is not in the public +reference: it is a Meta vendor extension, executed natively by the +harness in its own sandbox. tai-style tool loops do not use it at all; +they constrain the answer to `{content, tool_calls}` JSON via +`--output-schema` and execute calls themselves. + +### 2.5 Meta extensions vs stock OpenAI + +Only four things are Meta-specific: the `namespace` tool type, the +`x-tbh-*` / `x-meta-ai-gateway-*` headers, the `/muse-code/models` +path, and the `prompt_cache_key` values. Everything else matches the +public Responses reference field for field, which is why a generic +transpiler works. + +### 2.6 Response object and SSE (unobserved, per reference) + +The probe stub answered HTTP 500 to all 11 requests, so no Meta +response bytes were ever seen. The transpiler therefore implements the +documented standard shapes, which the Meta client demonstrably shares +on the request side: a Response object with `output` items of type +`message` (`output_text` / `refusal` parts) and `function_call` +(`call_id`, `name`, string `arguments`), plus `usage +{input_tokens, output_tokens, total_tokens}`; streamed as typed SSE +(`response.created`, `response.output_text.delta`, `response.output_item.added`, +`response.function_call_arguments.delta`, `response.completed`, +`response.failed` / `error`). A recording forward proxy would close +this gap. + +### 2.7 CLI translation + +The CLI converts SSE into newline-delimited JSON on stdout +(`run.output.delta` text events, `run.terminal.*` terminal events), +which is what `muse_run` parses. Timeouts, retries, and the +`--no-session-log` / `--session-id` session modes all live above this +layer and are unaffected by the protocol shape. + +### 2.8 System messages + +System guidance works through three channels, and Meta's client uses +two of them at once: top-level `instructions` (preferred, 25KB+ +observed), `developer`-role input messages (31KB observed), and +`system`-role input messages (spec-allowed, equivalent for older +clients). The transpiler folds leading Chat system/developer messages +into `instructions`, keeps mid-transcript ones as role-kept Items, and +reverses `instructions` into a leading Chat `system` message. + +## 3. Transpiler contract + +Pure functions returning `(converted, warnings)`. Core conversions are +exact both ways; everything outside the core warns on drop. Unknown +top-level fields warn too, so a gateway dialect field can never vanish +silently. + +### 3.1 Exact mappings + +| Chat Completions | Responses | +|---|---| +| `messages[]` | `input[]` Items | +| system/developer (leading) | `instructions` | +| assistant `tool_calls` | `function_call` (`id` to `call_id`) | +| `tool` role | `function_call_output` | +| externally-tagged function tools | internally-tagged function tools | +| `response_format` | `text.format` | +| `max_tokens` | `max_output_tokens` | +| `reasoning_effort` | `reasoning.effort` | +| delta chunks | typed SSE events | + +The `strict` default flips explicitly: Chat omits it (non-strict), +Responses omits it (attempts strict), so Chat-to-Responses pins +`strict: false`. + +### 3.2 Fixed loss table + +Chat to Responses drops: `n` (single generation only), `stop`, +`seed`, `logprobs`, sampling penalties, `logit_bias`, `prediction`, +`modalities`, audio, `web_search_options`, audio/file content parts, +custom tools, message `name`/`audio`. Responses to Chat drops: +`namespace`/built-in tools, `reasoning` and other native Items, +`previous_response_id` / `conversation`, `truncation`, `include`, +extra choices, non-text call-output parts. + +Thinking dialects collapse into `reasoning`: explicit +`reasoning_effort` / `reasoning.effort` wins; `thinking: {type}`, +`think`, `enable_thinking` map disabled to `effort: "none"` and leave +enabled as upstream default. + +### 3.3 Streaming and errors + +`ResponsesToChatStream` maps text and function-argument deltas +exactly, emits `role: assistant` once, yields the terminal chunk with +the real finish reason and usage from `response.completed`, and turns +`response.failed` / `error` into Chat stream error payloads (also +stashed on `failed`). `ChatToResponsesStream` is best effort: exact +deltas, synthesized lifecycle, assembled `response.completed`. +Response failures normalize to Chat-style `{error: ...}` objects via +`responses_error_object`. Synthesized ids (`chatcmpl_transpiled`, +`resp_transpiled`, `msg_transpiled_0`) are documented fakes. + +## 4. Usage + +Configure a devplacepy provider row with base URL +`https://api.meta.ai/v1/responses`, protocol Responses API, and a +route whose target model is `muse-spark-1.3-contributor`; clients keep +sending normal `POST /v1/chat/completions`. Verified example (zero +warnings both ways): + +```python +from responses import chat_to_responses, responses_to_chat + +chat = { + "model": "gpt-6-astra", + "messages": [ + {"role": "system", "content": "Be terse."}, + {"role": "user", "content": "Weather in Berlin?"}, + {"role": "assistant", "content": None, "tool_calls": [ + {"id": "call_1", "type": "function", + "function": {"name": "get_weather", + "arguments": '{"location": "Berlin"}'}}]}, + {"role": "tool", "tool_call_id": "call_1", "content": "12C, rain"}, + ], + "tools": [{"type": "function", "function": { + "name": "get_weather", "description": "City weather", + "parameters": {"type": "object", + "properties": {"location": {"type": "string"}}}}}], + "tool_choice": "auto", +} +resp, w1 = chat_to_responses(chat) # instructions + 3 Items, flat tool +back, w2 = responses_to_chat(resp) # [system, user, assistant, tool] +assert (w1, w2) == ([], []) +``` + +## 5. Complete transpiler source + +Stdlib only (`typing`). Verbatim copy of the tested module: + +```python +# retoor + +"""Chat Completions <-> Responses protocol transpiler. + +Lets the gateway keep its normal OpenAI surface (`POST /v1/chat/completions` +in, ChatCompletion JSON or Chat chunks out) while forwarding to a +Responses-protocol upstream such as Meta's `.../v1/responses`. The provider +row selects the protocol; `handle_chat` calls into this module at the send +boundary and on the way back. + +Every lossy mapping is reported in a warnings list, never dropped silently. +Core conversions (text, images, function tools/calls, structured outputs, +scalars, common stream events) are exact both ways. Known drops: Chat `n`, +`stop`, seeds/penalties, audio/file parts, custom tools; Responses +`namespace`/built-in tools, reasoning items, `previous_response_id` / +conversation state, `truncation`, `include`, extra choices. + +Grounded in the official OpenAI migration guide and both API references +(retrieved 2026-10-11) plus the Meta wire capture; Meta response bytes were +never observed, so the Response-object/SSE shapes are the OpenAI standard +ones the Meta client demonstrably shares on the request side. +""" + +from __future__ import annotations + +from typing import Any, Optional + + +def _text_of_content(content: Any) -> str: + if isinstance(content, str): + return content + if isinstance(content, list): + return "".join( + str(part.get("text") or part.get("refusal") or "") + for part in content + if isinstance(part, dict) + ) + return "" + + +def _chat_parts_to_input_parts(parts: Any, warn: list, where: str) -> list: + out = [] + for part in parts or []: + if not isinstance(part, dict): + continue + kind = part.get("type") + if kind == "text": + out.append({"type": "input_text", "text": part.get("text") or ""}) + elif kind == "image_url": + image = part.get("image_url") or {} + item = {"type": "input_image", "image_url": image.get("url") or ""} + if image.get("detail"): + item["detail"] = image["detail"] + out.append(item) + elif kind == "refusal": + warn.append( + "%s: refusal part has no Responses input shape; kept as text" + % (where,) + ) + out.append({"type": "input_text", "text": part.get("refusal") or ""}) + else: + warn.append( + "%s: dropped content part type %r (no verified Responses mapping)" + % (where, kind) + ) + return out + + +def _chat_messages_to_input(messages: Any) -> tuple[str, list, list]: + """Split Chat messages into Responses `instructions` + `input` items.""" + warn: list = [] + instructions: list = [] + items: list = [] + leading = True + for message in messages or []: + if not isinstance(message, dict): + continue + role = message.get("role") or "user" + if role in ("system", "developer") and leading: + text = _text_of_content(message.get("content")).strip() + if text: + instructions.append(text) + continue + leading = False + if role == "tool": + items.append( + { + "type": "function_call_output", + "call_id": message.get("tool_call_id") or "", + "output": _text_of_content(message.get("content")), + } + ) + continue + content = message.get("content") + if isinstance(content, list): + content = _chat_parts_to_input_parts( + content, warn, "message[%s]" % (role,) + ) + calls = ( + list(message.get("tool_calls") or []) if role == "assistant" else [] + ) + legacy = ( + (message.get("function_call") or {}) if role == "assistant" else {} + ) + empty = content is None or content == "" or content == [] + if not (role == "assistant" and empty and (calls or legacy.get("name"))): + items.append( + { + "type": "message", + "role": role, + "content": content if content is not None else "", + } + ) + if role == "assistant": + for call in calls: + func = ( + (call.get("function") or {}) + if isinstance(call, dict) + else {} + ) + items.append( + { + "type": "function_call", + "call_id": ( + (call.get("id") if isinstance(call, dict) else "") + or "" + ), + "name": func.get("name") or "", + "arguments": func.get("arguments") or "", + } + ) + if legacy.get("name"): + warn.append( + "deprecated assistant.function_call converted without a " + "stable id" + ) + items.append( + { + "type": "function_call", + "call_id": "call_legacy", + "name": legacy.get("name") or "", + "arguments": legacy.get("arguments") or "", + } + ) + for dropped in ("name", "audio"): + if message.get(dropped) is not None: + warn.append( + "message[%s]: dropped %r (no Responses input field)" + % (role, dropped) + ) + return "\n\n".join(instructions), items, warn + + +def _chat_tool_to_responses(tool: Any, warn: list) -> Optional[dict]: + if not isinstance(tool, dict): + return None + if tool.get("type") == "function": + func = tool.get("function") or {} + out = { + "type": "function", + "name": func.get("name") or "", + "parameters": func.get("parameters") + or {"type": "object", "properties": {}}, + } + if func.get("description") is not None: + out["description"] = func["description"] + # Chat is non-strict by default; Responses attempts strict mode when + # the key is omitted, so pin the Chat default explicitly. + out["strict"] = bool(func.get("strict", False)) + return out + warn.append( + "dropped tool type %r (only function tools transpile)" + % (tool.get("type"),) + ) + return None + + +def _chat_tool_choice_to_responses(choice: Any, warn: list) -> Any: + if choice in (None, "none", "auto", "required"): + return choice + if isinstance(choice, dict): + if choice.get("type") == "function": + return { + "type": "function", + "name": (choice.get("function") or {}).get("name") or "", + } + if choice.get("type") == "allowed_tools": + tools = [] + for entry in choice.get("tools") or []: + if isinstance(entry, dict) and entry.get("type") == "function": + tools.append( + { + "type": "function", + "name": (entry.get("function") or {}).get("name") + or "", + } + ) + else: + warn.append("dropped allowed_tools entry %r" % (entry,)) + return { + "type": "allowed_tools", + "mode": choice.get("mode") or "auto", + "tools": tools, + } + warn.append("dropped tool_choice %r (no Responses equivalent)" % (choice,)) + return None + + +def _chat_response_format_to_text(fmt: Any, warn: list) -> Optional[dict]: + if not isinstance(fmt, dict): + return None + kind = fmt.get("type") or "text" + if kind == "text": + return {"format": {"type": "text"}} + if kind == "json_object": + return {"format": {"type": "json_object"}} + if kind == "json_schema": + schema = fmt.get("json_schema") or {} + out = { + "format": { + "type": "json_schema", + "name": schema.get("name") or "result", + "schema": schema.get("schema") or {}, + } + } + if schema.get("description") is not None: + out["format"]["description"] = schema["description"] + if schema.get("strict") is not None: + out["format"]["strict"] = schema["strict"] + return out + warn.append("dropped response_format type %r" % (kind,)) + return None + + +# Chat scalar fields with a direct Responses counterpart. +_CHAT_SCALARS = ( + ("model", "model"), + ("temperature", "temperature"), + ("top_p", "top_p"), + ("stream", "stream"), + ("store", "store"), + ("metadata", "metadata"), + ("user", "user"), + ("service_tier", "service_tier"), + ("parallel_tool_calls", "parallel_tool_calls"), + ("prompt_cache_key", "prompt_cache_key"), + ("max_tokens", "max_output_tokens"), + ("max_completion_tokens", "max_output_tokens"), +) + +# Chat fields with no Responses equivalent (verified against the reference). +_CHAT_DROPPED = ( + "n", + "stop", + "seed", + "logprobs", + "frequency_penalty", + "presence_penalty", + "logit_bias", + "prediction", + "modalities", + "audio", + "web_search_options", + "prompt_cache_options", + "prompt_cache_retention", + "safety_identifier", + "stream_options", + "verbosity", +) + +# Top-level Chat fields the transpiler understands (mapped, converted, or +# explicitly dropped above). Anything else is warned about, never silently +# ignored, so a gateway dialect field can never vanish here unnoticed. +_CHAT_KNOWN = ( + frozenset(src for src, _ in _CHAT_SCALARS) + | frozenset(_CHAT_DROPPED) + | frozenset( + { + "messages", + "tools", + "tool_choice", + "response_format", + "reasoning_effort", + "reasoning", + "thinking", + "think", + "enable_thinking", + "top_logprobs", + } + ) +) + + +_THINKING_FALSE = {False, 0, "0", "false", "no", "off", "none", "disabled"} +_THINKING_TRUE = {True, 1, "1", "true", "yes", "on", "enabled"} + + +def _thinking_flag(value: Any) -> Optional[bool]: + if isinstance(value, str): + value = value.strip().lower() + if value in _THINKING_FALSE: + return False + if value in _THINKING_TRUE: + return True + return None + + +def _chat_thinking_to_reasoning(chat: dict, out: dict, warn: list) -> None: + # Explicit effort wins; a Responses-shaped reasoning dict passes its + # effort through; chat-dialect thinking flags collapse to effort "none" + # when disabled and to "upstream default" (unset) when enabled. + if chat.get("reasoning_effort") is not None: + out["reasoning"] = {"effort": chat["reasoning_effort"]} + return + reasoning = chat.get("reasoning") + if isinstance(reasoning, dict): + if reasoning.get("effort") is not None: + out["reasoning"] = {"effort": reasoning["effort"]} + extra = sorted(k for k in reasoning if k != "effort") + if extra: + warn.append( + "dropped reasoning keys %r (only effort transpiles)" + % (extra,) + ) + return + enabled = _thinking_flag(reasoning.get("enabled")) + if enabled is False: + out["reasoning"] = {"effort": "none"} + return + if enabled is True: + return + thinking = chat.get("thinking") + if isinstance(thinking, dict): + enabled = _thinking_flag(thinking.get("type")) + if enabled is False: + out["reasoning"] = {"effort": "none"} + return + if enabled is True: + return + for key in ("think", "enable_thinking"): + enabled = _thinking_flag(chat.get(key)) + if enabled is False: + out["reasoning"] = {"effort": "none"} + return + if enabled is True: + return + + +def chat_to_responses(chat: dict) -> tuple[dict, list]: + """Convert a `POST /v1/chat/completions` body to `POST /responses`.""" + warn: list = [] + out: dict = {} + for src, dst in _CHAT_SCALARS: + if chat.get(src) is not None and dst not in out: + out[dst] = chat[src] + if chat.get("top_logprobs") is not None: + out["top_logprobs"] = chat["top_logprobs"] + _chat_thinking_to_reasoning(chat, out, warn) + instructions, items, converted_warnings = _chat_messages_to_input( + chat.get("messages") + ) + warn += converted_warnings + if instructions: + out["instructions"] = instructions + out["input"] = items + tools = [] + for tool in chat.get("tools") or []: + converted = _chat_tool_to_responses(tool, warn) + if converted is not None: + tools.append(converted) + if tools: + out["tools"] = tools + if chat.get("tool_choice") is not None: + choice = _chat_tool_choice_to_responses(chat["tool_choice"], warn) + if choice is not None: + out["tool_choice"] = choice + if chat.get("response_format") is not None: + text = _chat_response_format_to_text(chat["response_format"], warn) + if text is not None: + out["text"] = text + for key in _CHAT_DROPPED: + if chat.get(key) is not None: + warn.append("dropped chat field %r (no Responses equivalent)" % (key,)) + for key in sorted(chat): + if key not in _CHAT_KNOWN: + warn.append("ignored unknown chat field %r" % (key,)) + return out, warn + + +def _input_parts_to_chat_parts(parts: Any, warn: list, where: str) -> list: + out = [] + for part in parts or []: + if not isinstance(part, dict): + continue + kind = part.get("type") + if kind in ("input_text", "output_text", "text"): + out.append({"type": "text", "text": part.get("text") or ""}) + elif kind == "input_image": + image = {"url": part.get("image_url") or ""} + if part.get("detail"): + image["detail"] = part["detail"] + out.append({"type": "image_url", "image_url": image}) + elif kind == "refusal": + out.append({"type": "refusal", "refusal": part.get("refusal") or ""}) + else: + warn.append( + "%s: dropped input part type %r (no Chat mapping)" + % (where, kind) + ) + return out + + +def _call_output_to_text(output: Any, warn: list) -> str: + if isinstance(output, str): + return output + if isinstance(output, list): + texts = [] + for part in output: + if not isinstance(part, dict): + continue + if part.get("type") == "input_text": + texts.append(part.get("text") or "") + else: + warn.append( + "dropped function_call_output part type %r " + "(tool messages carry text only)" % (part.get("type"),) + ) + return "".join(texts) + return "" + + +def _responses_input_to_chat_messages( + input_items: Any, warn: list +) -> list: + messages: list = [] + pending_calls: list = [] + for item in input_items or []: + if isinstance(item, str): + messages.append({"role": "user", "content": item}) + continue + if not isinstance(item, dict): + continue + kind = item.get("type") or "message" + if kind == "message": + content = item.get("content") + if isinstance(content, list): + content = _input_parts_to_chat_parts( + content, warn, "input.message" + ) + messages.append( + {"role": item.get("role") or "user", "content": content} + ) + continue + if kind == "function_call": + pending_calls.append( + { + "id": item.get("call_id") or item.get("id") or "", + "type": "function", + "function": { + "name": item.get("name") or "", + "arguments": item.get("arguments") or "", + }, + } + ) + continue + if kind == "function_call_output": + if pending_calls: + messages.append( + { + "role": "assistant", + "content": None, + "tool_calls": pending_calls, + } + ) + pending_calls = [] + messages.append( + { + "role": "tool", + "tool_call_id": item.get("call_id") or "", + "content": _call_output_to_text(item.get("output"), warn), + } + ) + continue + warn.append( + "dropped input item type %r " + "(reasoning/native-tool items do not replay in Chat)" % (kind,) + ) + if pending_calls: + messages.append( + {"role": "assistant", "content": None, "tool_calls": pending_calls} + ) + return messages + + +def _responses_tool_to_chat(tool: Any, warn: list) -> Optional[dict]: + if not isinstance(tool, dict): + return None + if tool.get("type") == "function": + func: dict = {"name": tool.get("name") or ""} + if tool.get("description") is not None: + func["description"] = tool["description"] + if tool.get("parameters") is not None: + func["parameters"] = tool["parameters"] + if tool.get("strict") is not None: + func["strict"] = tool["strict"] + return {"type": "function", "function": func} + warn.append( + "dropped tool type %r (namespaces and builtins have no Chat shape)" + % (tool.get("type"),) + ) + return None + + +def _responses_tool_choice_to_chat(choice: Any, warn: list) -> Any: + if choice in (None, "none", "auto", "required"): + return choice + if isinstance(choice, dict): + if choice.get("type") == "function": + return { + "type": "function", + "function": {"name": choice.get("name") or ""}, + } + if choice.get("type") == "allowed_tools": + tools = [] + for entry in choice.get("tools") or []: + if isinstance(entry, dict) and entry.get("type") == "function": + tools.append( + { + "type": "function", + "function": {"name": entry.get("name") or ""}, + } + ) + else: + warn.append("dropped allowed_tools entry %r" % (entry,)) + return { + "type": "allowed_tools", + "mode": choice.get("mode") or "auto", + "tools": tools, + } + warn.append("dropped tool_choice %r (no Chat equivalent)" % (choice,)) + return None + + +def _responses_text_to_response_format(text: Any, warn: list) -> Optional[dict]: + fmt = (text or {}).get("format") if isinstance(text, dict) else None + if not isinstance(fmt, dict): + return None + kind = fmt.get("type") or "text" + if kind == "text": + return None + if kind == "json_object": + return {"type": "json_object"} + if kind == "json_schema": + schema: dict = { + "name": fmt.get("name") or "result", + "schema": fmt.get("schema") or {}, + } + if fmt.get("description") is not None: + schema["description"] = fmt["description"] + if fmt.get("strict") is not None: + schema["strict"] = fmt["strict"] + return {"type": "json_schema", "json_schema": schema} + warn.append("dropped text.format type %r" % (kind,)) + return None + + +_RESPONSES_SCALARS = ( + ("model", "model"), + ("temperature", "temperature"), + ("top_p", "top_p"), + ("stream", "stream"), + ("store", "store"), + ("metadata", "metadata"), + ("user", "user"), + ("service_tier", "service_tier"), + ("parallel_tool_calls", "parallel_tool_calls"), + ("prompt_cache_key", "prompt_cache_key"), + ("top_logprobs", "top_logprobs"), + ("max_output_tokens", "max_tokens"), +) + +_RESPONSES_DROPPED = ( + "previous_response_id", + "conversation", + "truncation", + "include", + "background", + "max_tool_calls", + "prompt", + "prompt_cache_options", + "prompt_cache_retention", + "safety_identifier", + "stream_options", + "context_management", + "moderation", + "access_programs", +) + +_RESPONSES_KNOWN = ( + frozenset(src for src, _ in _RESPONSES_SCALARS) + | frozenset(_RESPONSES_DROPPED) + | frozenset( + { + "input", + "instructions", + "tools", + "tool_choice", + "text", + "reasoning", + } + ) +) + + +def responses_to_chat(resp: dict) -> tuple[dict, list]: + """Convert a `POST /responses` body to `POST /v1/chat/completions`.""" + warn: list = [] + out: dict = {} + for src, dst in _RESPONSES_SCALARS: + if resp.get(src) is not None: + out[dst] = resp[src] + reasoning = resp.get("reasoning") or {} + if isinstance(reasoning, dict) and reasoning.get("effort") is not None: + out["reasoning_effort"] = reasoning["effort"] + raw_input = resp.get("input") + items = ( + [{"type": "message", "role": "user", "content": raw_input}] + if isinstance(raw_input, str) + else raw_input + ) + messages = _responses_input_to_chat_messages(items, warn) + instructions = resp.get("instructions") + if isinstance(instructions, str) and instructions.strip(): + messages.insert(0, {"role": "system", "content": instructions}) + elif isinstance(instructions, list): + messages[0:0] = _responses_input_to_chat_messages(instructions, warn) + out["messages"] = messages + tools = [] + for tool in resp.get("tools") or []: + converted = _responses_tool_to_chat(tool, warn) + if converted is not None: + tools.append(converted) + if tools: + out["tools"] = tools + if resp.get("tool_choice") is not None: + choice = _responses_tool_choice_to_chat(resp["tool_choice"], warn) + if choice is not None: + out["tool_choice"] = choice + if resp.get("text") is not None: + fmt = _responses_text_to_response_format(resp["text"], warn) + if fmt is not None: + out["response_format"] = fmt + for key in _RESPONSES_DROPPED: + if resp.get(key) is not None: + warn.append( + "dropped responses field %r (no Chat equivalent)" % (key,) + ) + for key in sorted(resp): + if key not in _RESPONSES_KNOWN: + warn.append("ignored unknown responses field %r" % (key,)) + return out, warn + + +def responses_object_to_chat_completion(resp: dict) -> tuple[dict, list]: + """Convert a Responses object to a ChatCompletion object (one choice).""" + warn: list = [] + text: list = [] + refusal: list = [] + calls: list = [] + for item in resp.get("output") or []: + if not isinstance(item, dict): + continue + if item.get("type") == "message": + for part in item.get("content") or []: + if not isinstance(part, dict): + continue + if part.get("type") == "output_text": + text.append(part.get("text") or "") + elif part.get("type") == "refusal": + refusal.append(part.get("refusal") or "") + else: + warn.append( + "dropped output part type %r" % (part.get("type"),) + ) + elif item.get("type") == "function_call": + calls.append( + { + "id": item.get("call_id") or item.get("id") or "", + "type": "function", + "function": { + "name": item.get("name") or "", + "arguments": item.get("arguments") or "", + }, + } + ) + else: + warn.append("dropped output item type %r" % (item.get("type"),)) + message: dict = {"role": "assistant", "content": "".join(text) or None} + if refusal: + message["refusal"] = "".join(refusal) + if calls: + message["tool_calls"] = calls + incomplete = (resp.get("incomplete_details") or {}).get("reason") + if calls: + finish = "tool_calls" + elif incomplete == "max_output_tokens": + finish = "length" + elif incomplete == "content_filter": + finish = "content_filter" + else: + finish = "stop" + usage = resp.get("usage") or {} + return { + "id": resp.get("id") or "", + "object": "chat.completion", + "created": resp.get("created_at") or 0, + "model": resp.get("model") or "", + "choices": [{"index": 0, "message": message, "finish_reason": finish}], + "usage": { + "prompt_tokens": usage.get("input_tokens") or 0, + "completion_tokens": usage.get("output_tokens") or 0, + "total_tokens": usage.get("total_tokens") or 0, + }, + }, warn + + +def chat_completion_to_responses_object(comp: dict) -> tuple[dict, list]: + """Convert a ChatCompletion object to a Responses object (choice 0).""" + warn: list = [] + choices = comp.get("choices") or [] + if len(choices) > 1: + warn.append( + "kept choice 0 of %d (Responses returns one generation)" + % (len(choices),) + ) + message = (choices[0] or {}).get("message") if choices else {} + output: list = [] + if (message.get("content") or None) is not None or not message.get( + "tool_calls" + ): + output.append( + { + "id": "msg_transpiled_0", + "type": "message", + "status": "completed", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": message.get("content") or "", + "annotations": [], + } + ], + } + ) + if message.get("refusal"): + output.append( + { + "id": "msg_transpiled_refusal", + "type": "message", + "status": "completed", + "role": "assistant", + "content": [ + {"type": "refusal", "refusal": message["refusal"]} + ], + } + ) + for call in message.get("tool_calls") or []: + func = ( + (call.get("function") or {}) if isinstance(call, dict) else {} + ) + output.append( + { + "type": "function_call", + "call_id": ( + (call.get("id") if isinstance(call, dict) else "") or "" + ), + "name": func.get("name") or "", + "arguments": func.get("arguments") or "", + } + ) + usage = comp.get("usage") or {} + return { + "id": comp.get("id") or "", + "object": "response", + "created_at": comp.get("created") or 0, + "status": "completed", + "model": comp.get("model") or "", + "output": output, + "usage": { + "input_tokens": usage.get("prompt_tokens") or 0, + "output_tokens": usage.get("completion_tokens") or 0, + "total_tokens": usage.get("total_tokens") or 0, + }, + }, warn + + +def responses_failure_message(data: Any) -> Optional[str]: + """Extract a failure message from a 200 Responses body, if any.""" + if not isinstance(data, dict): + return None + error = data.get("error") + if isinstance(error, dict): + message = error.get("message") or error.get("code") or "upstream error" + return str(message)[:300] + if data.get("object") == "response" and data.get("status") in ( + "failed", + "incomplete", + "cancelled", + ): + details = data.get("incomplete_details") or {} + reason = details.get("reason") or data.get("status") + return "response %s: %s" % (data.get("status"), reason) + return None + + +def responses_error_object(data: Any) -> Optional[dict]: + """Normalize a Responses error body to a Chat-style error object.""" + if not isinstance(data, dict): + return None + error = data.get("error") + if isinstance(error, dict): + out = {"message": str(error.get("message") or "upstream error")} + if error.get("type") is not None: + out["type"] = error["type"] + if error.get("code") is not None: + out["code"] = error["code"] + return {"error": out} + message = responses_failure_message(data) + if message: + return {"error": {"message": message, "type": "upstream_error"}} + return None + + +class ResponsesToChatStream: + """Translate Responses SSE events into ChatCompletion chunks. + + Text deltas and function-call argument deltas map exactly; lifecycle + events are consumed; `response.completed` yields the terminal chunk with + the real finish reason and usage. Error events yield `{"error": ...}` + payloads (the Chat stream error shape) and are also stashed on `failed`. + """ + + def __init__(self, model: str = ""): + self.model = model + self.calls: dict = {} + self.order: list = [] + self.failed: Any = None + self.role_sent = False + self.text_sent = False + + def feed(self, event: Any) -> list: + if not isinstance(event, dict): + return [] + kind = event.get("type") or "" + if kind == "response.output_text.delta": + delta: dict = {"content": event.get("delta") or ""} + if not self.role_sent: + delta = {"role": "assistant", **delta} + self.role_sent = True + self.text_sent = True + return [self._chunk(delta)] + if kind == "response.output_item.added": + item = event.get("item") or {} + if item.get("type") == "function_call" and item.get("call_id"): + index = self._index(item["call_id"]) + return [ + self._chunk( + { + "tool_calls": [ + { + "index": index, + "id": item["call_id"], + "type": "function", + "function": { + "name": item.get("name") or "", + "arguments": "", + }, + } + ] + } + ) + ] + return [] + if kind == "response.function_call_arguments.delta": + call_id = event.get("item_id") or "" + return [ + self._chunk( + { + "tool_calls": [ + { + "index": self._index(call_id), + "function": { + "arguments": event.get("delta") or "" + }, + } + ] + } + ) + ] + if kind == "response.completed": + response = event.get("response") or {} + converted, _ = responses_object_to_chat_completion(response) + chunks = [] + message = (converted["choices"][0] or {}).get("message") or {} + if message.get("refusal") and not self.text_sent: + chunks.append(self._chunk({"refusal": message["refusal"]})) + chunk = self._chunk({}) + chunk["choices"][0]["finish_reason"] = ( + converted["choices"][0] or {} + ).get("finish_reason") + chunk["usage"] = converted["usage"] + chunks.append(chunk) + return chunks + if kind in ("response.failed", "response.incomplete", "error"): + self.failed = event + payload = responses_error_object( + event.get("response") or event.get("error") or event + ) or {"error": {"message": kind, "type": "upstream_error"}} + return [payload] + return [] + + def _index(self, call_id: str) -> int: + if call_id not in self.calls: + self.calls[call_id] = len(self.order) + self.order.append(call_id) + return self.calls[call_id] + + def _chunk(self, delta: dict) -> dict: + return { + "id": "chatcmpl_transpiled", + "object": "chat.completion.chunk", + "created": 0, + "model": self.model, + "choices": [{"index": 0, "delta": delta, "finish_reason": None}], + } + + +class ChatToResponsesStream: + """Translate ChatCompletion chunks into Responses SSE events. + + Best effort: text and tool-call deltas map exactly, but Chat chunks + carry no lifecycle events, so the synthesized stream skips + `response.created` / `output_item.added` for text and emits + `response.completed` with an assembled object. Streaming call ids are + tracked by tool-call index, per the Chat streaming contract (the id + arrives on the first chunk for each index). + """ + + def __init__(self): + self.text: list = [] + self.calls: dict = {} + self.order: list = [] + self.warnings: list = [] + + def feed(self, chunk: Any) -> list: + events = [] + choices = (chunk or {}).get("choices") or [] + delta = (choices[0] or {}).get("delta") if choices else {} + for call in (delta or {}).get("tool_calls") or []: + if not isinstance(call, dict): + continue + index = call.get("index") + func = call.get("function") or {} + call_id = call.get("id") + if call_id and call_id not in self.calls: + self.calls[call_id] = True + self.order.append(call_id) + events.append( + { + "type": "response.output_item.added", + "output_index": len(self.order) - 1, + "item": { + "type": "function_call", + "call_id": call_id, + "name": func.get("name") or "", + "arguments": "", + }, + } + ) + if not call_id and isinstance(index, int) and index < len( + self.order + ): + call_id = self.order[index] + if func.get("arguments"): + if not call_id: + self.warnings.append( + "tool-call arguments arrived before any call id" + ) + events.append( + { + "type": "response.function_call_arguments.delta", + "item_id": call_id or "", + "output_index": 0, + "delta": func["arguments"], + } + ) + piece = (delta or {}).get("content") + if piece: + self.text.append(piece) + events.append( + { + "type": "response.output_text.delta", + "item_id": "msg_transpiled_0", + "output_index": 0, + "content_index": 0, + "delta": piece, + } + ) + finish = (choices[0] or {}).get("finish_reason") if choices else None + if finish: + output = [ + { + "id": "msg_transpiled_0", + "type": "message", + "status": "completed", + "role": "assistant", + "content": [ + { + "type": "output_text", + "text": "".join(self.text), + "annotations": [], + } + ], + } + ] + events.append( + { + "type": "response.completed", + "response": { + "id": "resp_transpiled", + "object": "response", + "status": "completed", + "model": (chunk or {}).get("model") or "", + "output": output, + }, + } + ) + return events +``` + +## 6. Verification status and known gaps + +Module tests: 21/21 green +(`tests/unit/services/openai_gateway/responses.py` in devplacepy), +covering both request directions, both object converters, both +stream translators, error/failure shapes, thinking-dialect collapse, +and every documented drop. A real captured Meta request round-trips +through `responses_to_chat` and back with only the expected +`namespace` and `include` warnings. The weather example in section 4 +was executed with zero warnings both ways. + +Known gaps, stated plainly: + +- Meta response and SSE bytes were never observed; that side rests on + the documented standard shapes. A recording forward proxy closes it. +- Audio and file content parts warn and drop; their exact cross-API + shapes were not verified. +- Synthesized stream/object ids are documented fakes, not upstream ids. +- The devplacepy gateway wiring (provider protocol setting, send-time + transpile, Responses stream relay) was mid-integration at snapshot + time with gateway unit failures under diagnosis; only the module + itself is green. + +## 7. Sources + +Retrieved 2026-10-11. Full dossiers live in +`research/muse-responses-openai-transpiler/`. + +- OpenAI, Migrate to the Responses API: + https://developers.openai.com/api/docs/guides/migrate-to-responses +- OpenAI, Responses API reference: + https://developers.openai.com/api/reference/resources/responses +- OpenAI, Chat Completions API reference: + https://developers.openai.com/api/reference/resources/chat +- Local MITM capture of the real `muse` CLI: + `/tmp/mitm_probe/request.txt` (internal evidence, 11 requests) + +## 8. Changelog + +| Date | Change | +|---|---| +| 2026-10-11 | Consolidated research + verbatim transpiler source into one document. | diff --git a/taicpp/SPEC.md b/taicpp/SPEC.md new file mode 100644 index 0000000..b9877ba --- /dev/null +++ b/taicpp/SPEC.md @@ -0,0 +1,6 @@ +# SPEC — detz v0.1.0 + +moloapp pin: 12e45e8 +264 classes in ../tai.py (grep ^class). Full parity table per docs/.md. +Target tree: src/{cli,config,util,shell,seal,store,llm,agent,comms,plexer,tui,mcp,cap}. +Build: -std=c++23 -Wall -Wextra, outputs only in build/. diff --git a/taicpp/docs/config.md b/taicpp/docs/config.md new file mode 100644 index 0000000..7ba7083 --- /dev/null +++ b/taicpp/docs/config.md @@ -0,0 +1,14 @@ +# config parity (M2) + +| python (tai.py) | C++ (src/config/) | notes | +|---|---|---| +| Consts | consts.h Consts | same values | +| TOTAL_STEP_CAP/SYSINFO_TIMEOUT/LOOP_*/TIMEOUTs | Consts::* | — | +| DEFAULT_MODEL/DEFAULT_VOICE/OPENROUTER_MODEL | kDefaultModel/... | — | +| ZEN_FREE_MODELS/POLLINATIONS_MODELS/DEVPLACE_FREE_ROUTES | ZenFreeModels()/... | — | +| Config.__init__ env mapping | Config::FromEnv | subset: home/model/keys/models/voice | +| SETTINGS_CATALOG keys | SettingsCatalog() | 11 keys ported (subset) | +| Config.Get/settings_row | Config::Get | scalar lookup only | +| split_models | SplitModels | comma/semicolon split | +| known_model_ids | KnownModelIds | config lists + extra (no health/backend) | +| resolve_tag | ResolveTag | bots map + models + dir; no store/agent | diff --git a/taicpp/docs/embed.md b/taicpp/docs/embed.md new file mode 100644 index 0000000..37ee398 --- /dev/null +++ b/taicpp/docs/embed.md @@ -0,0 +1,9 @@ +# Embedding detz: research notes + +Options compared: +1. **Dynamic .so plugin** (`dlopen`): flexible versioning, but host must ship/resolve the .so at runtime; symbol drift risk; packaging burden. +2. **Subprocess/IPC** (spawn detz binary): simplest isolation, but per-call latency + serialization cost; no in-process context hook. +3. **Static archive link (chosen)**: `build/libdetz.a` + C ABI header `src/glue/detz_embed.h` (`detz_set_context`, `detz_context`, `detz_version`) plus C++ hook `detz::embed::apply_context()`. Single artifact, no runtime deps beyond system libs, version pinned at build time. `make static` additionally yields fully static `build/detz-static` (verified `file`: statically linked, `ldd`: not a dynamic executable) using vendored `third_party/{curl,sqlite,jitterentropy}` archives. Re-verified 2026-10-27 at 1.44.3: `make static` exits 0, the binary is 29 MB, `file` says statically linked, `--version` prints `tai 1.44.3`, and the link shows only the standard glibc static warnings (`dlopen`, `getaddrinfo`, `gethostbyname`, `getpwuid`). +4. **Header-only**: rejected — codebase too large, would explode host build times. + +Why this API: minimal C surface (stable ABI, callable from C/Rust/Python-ctypes) with a thin C++ `apply_context()` injection point wired into `GlueApp::one_shot`, so hosts prepend context without touching agent internals. Proven: third-party TU links against `libdetz.a` and `apply_context("ping")` → `[ctx] STATIC-PROOF\nping`, version `1.44.3`. diff --git a/taicpp/docs/termshot.md b/taicpp/docs/termshot.md new file mode 100644 index 0000000..df6e8db --- /dev/null +++ b/taicpp/docs/termshot.md @@ -0,0 +1,120 @@ +# termshot (taicpp): PNG pictures of the current terminal + +Namespace `detz::termshot`, sources in `src/termshot/`. detz-only: tai.py has no +equivalent. Research and decisions: +[`../../research/terminal-screenshot-rendering/`](../../research/terminal-screenshot-rendering/). + +## What it does + +Makes a PNG of the terminal the user is looking at, on a headless server too. +No desktop, X server, system font, browser or external binary is involved. + +``` +tmux capture-pane -e -p -N (or the native multiplexer pane log tail) + | +TuiVtScreen (src/tui) replay the bytes into a rows x cols cell grid + | +TermshotRenderer background, glyph, bold, dim, reverse, underline, strike, cursor + | +TermshotPng 8-bit RGB, filter 0, zlib deflate, zlib crc32 + | +/shots/terminal-YYYYmmdd-HHMMSS-mmm.png +``` + +## Classes + +| Class | File | Role | +|---|---|---| +| `TermshotFont` | `termshot_font.*`, `termshot_font_data.cpp` | embedded Spleen 8x16 glyphs, binary search, fallback U+FFFD then `?` | +| `TermshotPalette` | `termshot_palette.*` | SGR fragment to 0xRRGGBB: 16 xterm colors, 256 cube and gray ramp, truecolor, bold brightens 0..7 | +| `TermshotCanvas` | `termshot_canvas.*` | RGB buffer: fill, glyph (bold doubles lit pixels right), pixel | +| `TermshotPng` | `termshot_png.*` | encoder and IHDR size reader | +| `TermshotFrame`, `TermshotError` | `termshot_frame.h` | captured bytes plus size, cursor, source label; failure with a tool-ready reason | +| `TermshotCapture` | `termshot_capture.*` | `tmux(target)`, `native(raw, pane, rows, cols)`, `current(target, reader)` | +| `TermshotRenderer` | `termshot_renderer.*` | frame to canvas | +| `TermshotShooter`, `TermshotShot` | `termshot_shooter.*` | frame to PNG bytes to file, cap check, one-line summary | + +## Capture rules + +`TermshotCapture::current(target, reader)` resolves the pane exactly like +`get_current_terminal_content`: + +1. a `target` (tmux pane id `%3` or `session:window.pane`) always means tmux; +2. no target and `TAI_PLEXER_SOCKET` set: the native multiplexer. The agent passes + `ports().plexer->native_read_pane` as the reader; the last 200 lines of the raw pane + log are replayed into a `LINES` x `COLUMNS` screen, default 50 x 160; +3. otherwise tmux with `TMUX_PANE`, or tmux's current pane when that is unset. + +tmux calls: `display-message -p '#{pane_width} #{pane_height} #{cursor_x} #{cursor_y} +#{cursor_flag} #{pane_id}'`, then `capture-pane -e -p -N [-t target]` (visible screen, +escapes kept, trailing spaces kept so background colors reach the right edge). Line +ends become `\r\n` before the replay. + +## Font + +Spleen 8x16 2.2.0 by Frederic Cambus, BSD 2-Clause. The BDF and the license are +vendored in `src/termshot/gen/`; `python3 src/termshot/gen/gen_font.py` regenerates +`termshot_font_data.cpp` (1001 glyphs, 16 bytes each, license text in the header). +Covers Basic Latin, Latin-1, Latin Extended-A, box drawing, block elements, braille and +CP437. Bold is synthesized (each lit pixel doubled one to the right). + +## Caps + +All in `termshot_consts.h`. + +| Constant | Value | Why | +|---|---|---| +| `kMaxCols` | 400 | wider panes are cut at the right | +| `kMaxRows` | 200 | taller panes are cut | +| `kNativeDefaultCols` x `kNativeDefaultRows` | 160 x 50 | the native log has no pane size | +| `kMaxCaptureBytes` | 4 MiB | bytes fed to the VT parser (native keeps the tail) | +| `kMaxPngBytes` | 16 MiB | refuse larger files, also the WhatsApp image cap | +| `kCaptureTimeout` | 10 s | per tmux call | +| `kCellWidth` x `kCellHeight` | 8 x 16 px | font cell | +| `kPadding` | 12 px | border around the grid | +| `kDefaultFg` / `kDefaultBg` / `kCursorColor` | `#D0D0D0` / `#1C1C1C` / `#D0D0D0` | look | + +Measured: a 192 x 35 tmux pane gives a 1560 x 584 PNG of 22 to 26 KB. + +## Tool + +`terminal_screenshot {target?: string, path?: string}`, registered in +`src/agent/detz_tools.cpp` (detz-only tools; `data.cpp` stays a byte-exact copy of +tai.py). Trigger tags: terminal, screenshot, picture, screen, image, snapshot. +Result: `terminal screenshot ( bytes, x px, rows x +cols, , image/png)`, or `error: terminal screenshot failed: `. + +For a caller in C++ (for example a WhatsApp send): + +```cpp +#include "../termshot/termshot_capture.h" +#include "../termshot/termshot_shooter.h" +using namespace detz::termshot; +TermshotFrame frame = TermshotCapture::current(target, reader); // reader may be empty +TermshotShot shot = TermshotShooter(home).shoot(frame); // writes the file +std::string png = TermshotShooter::png(frame); // or bytes only +``` + +WhatsApp: send the bytes as an imageMessage with mimetype `image/png`; no JPEG +conversion is done (see the research conclusion `whatsapp-image-as-png`). + +## Tests + +- `build/termshot_test` (`tests/termshot_test.cpp`, step `termshot.unit`): font coverage + and fallback, palette values, PNG signature and size, decoded pixel colors for + background, SGR background, full block foreground, reverse, line feed, cursor, caps, + native tail cut, file write under `/shots`. +- `agent.diff` and `agent.e2e` compare against tai.py, so both call + `DetzTools::set_enabled(false)`; with detz tools on they stay on in every real run. +- Live: panes rendered from tmux 3.6 and viewed; a real model (devplace) called the tool + through `build/detz` and returned the written path. + +## Gaps + +- Glyphs Spleen lacks (CJK, Greek, Cyrillic beyond Latin Extended-A, symbols such as + U+276F or U+23F5) draw as `?`. +- Wide characters take one column (the same `TuiVtScreen` gap). +- The native multiplexer path replays a log tail into a guessed size; not verified live. +- Only the visible screen; no scrollback option. +- No JPEG thumbnail is produced for WhatsApp; the chat preview may stay empty until the + image is downloaded. diff --git a/taicpp/docs/versioning.md b/taicpp/docs/versioning.md new file mode 100644 index 0000000..b259dd7 --- /dev/null +++ b/taicpp/docs/versioning.md @@ -0,0 +1,2 @@ +# versioning +VERSION file; tools/version-bump.sh; tools/release.sh; CHANGELOG.md. diff --git a/taicpp/docs/workflow.md b/taicpp/docs/workflow.md new file mode 100644 index 0000000..cadaae2 --- /dev/null +++ b/taicpp/docs/workflow.md @@ -0,0 +1,2 @@ +# workflow +agent -> approval -> tools -> goal verify; tui renders; mcp serves; cap gates. diff --git a/taicpp/grr.md b/taicpp/grr.md new file mode 100644 index 0000000..24e0de9 --- /dev/null +++ b/taicpp/grr.md @@ -0,0 +1,3 @@ +I said this with my big mouth: ```Ik ga overmorgen maken dat ik kan zeggrn: "Create screenshot and explain what is genius about it and send to the Niggel on WhatsApp" en dat het gewoon werkt.```. + +So this statement is about this application. Iti s written in C++ and it is an agent. So on the cli i want to achieve what is stated above. THat means, we should be able to communciate with freaking whatsapp using C++. My god my ggod my god. FIne. Let's just find out what the best python lbirarires are to control whatsapp with using a normal pessant phone number and automate whatever we want. I imagine it ccould be a very painful process. Well, our goal is to make it as user friendly possible with configuring everything and hwatnot. We will isolate all this functionality when it comes to appending it to our awesome agent project. Anyway, also please find out if browser useage is required to achieve our goals, if it is, don't worry, just mention it and i have solutions but just try to avoid it. All i want for now is research and perfectly documented on hoew to achieve this using C++. And yes, that will include cloning python directorys of github and transpile them completely without shame to something we can use. We do this by deciding up front when we need to build and when we need to deep research. Everything is plain simple. YOu will use / execute yourself for literally every thask making everything possible. THe most important thing is, KISS, keep it isimpe. then dry, and then consistency and fuck performance. I do performance, you do code. Ok. lets go. Deliver what i've asked. diff --git a/taicpp/plan.md b/taicpp/plan.md new file mode 100644 index 0000000..9bfa944 --- /dev/null +++ b/taicpp/plan.md @@ -0,0 +1,118 @@ +# detz — C++23 port of `tai.py` — Plan (v1) + +> App name: `detz`. Future: subcommand of `molo` (moloapp framework). +> Source of truth to port: `../tai.py` (~40k lines) + tests `../test_tai.py`, `../tai_test.py`. +> Original must remain untouched — no downgrades in functionality. +> Coding standard / reuse: `moloapp` (`/home/retoor/projects/moloapp`) — components, providers, clients (HTTP client w/ streaming), header-only style, per-part `.mk` files, root Makefile, `build/` outputs only. + +## 1. Goals & non-goals + +Goals: +- Full-featured structured C++23 port of tai.py as `detz` binary in this dir (`taicpp/`). +- Max depth/layering in file structure; maximally object-oriented. +- argparse/CLI done well so it later plugs in as `molo detz ...` subcommand. +- Reuse moloapp: HTTP client (+SSE streaming), LLM provider types (`src/llm`, `src/clients`), app errors/middleware patterns where applicable. +- Docs: README.md, AGENTS.md, SPEC per module, workflow + versioning/tagging/git automation via AGENTS.md + Makefile targets. + +Non-goals: modifying `../tai.py`; rewriting moloapp itself (vendor/link only). + +## 2. Source inventory (from `../tai.py`) + +~40k lines. Key classes/functions to port (grouped): + +| Group | Symbols | +|---|---| +| Constants/config | `Consts`, `Config`, settings fns (`validate_setting_scope`, `normalize_setting_value`, `parse_setting_env/stored`, `export_settings_to_env`, `boot_settings`), `dir_key`, `orientation_block`, `capability_block` | +| Text/ANSI/utils | `Ansi`, `paint`, `now_iso`, `short_error`, `truncate`, `capped_text`, `read_capped`, `chunk_lines`, `collapse_repeated_lines`, `split_leading_tags`, `expand_file_mentions`, `complete_files`, `known_model_ids`, `resolve_tag` | +| Shell/sandbox | `Sandbox`, `ShellSession`, `shell_target_paths`, `segment_targets`, `curl_hint`, `speak`, `Spinner`, `LiveStream`, `PtyProc`, `LaneBoard`, `Glow` | +| Crypto/vault | `SealError`, `Seal` | +| Persistence | `Store` (sqlite+FTS5), `spill_output`, `spilled_view`, `spill_or_truncate`, `DebugLog`, `ensure_column`, `fts_*` | +| LLM backends | `BackendError`, `Opencode`, `ModelHealth`, `ChatClient`, providers (openai/anthropic/google/deepseek/xai/openrouter via moloapp), streaming SSE | +| Agents/tools | `Tools`, `Agent`, `Approval`, `Denied`, `AgentsFull`, `GoalFailure`, `GoalVerdict`, `GoalVerifier`, `RetrievalEval`, `RagStats` | +| Comms/ops | `Telegram`, `Install`, `SysInfo`, plexer (`PlexerError`, `PlexerController`, `Native/TmuxPlexerController`), TUI (`TuiTheme/Watched/Style/Span/Event/Key/Click`, see `tui.md`), MCP server (`mcp.md`, `--serve-mcp`), cap orchestrator (`cap.md`) | +| CLI | argparse tree (~`--cap`, `--serve-mcp`, `--mcp-port`, …) → port to structured CLI layer | + +Full class list: extract with `grep -n "^class " ../../tai.py` during milestone M0 and pin counts in SPEC. + +## 3. Target structure (deep, layered) + +``` +taicpp/ + plan.md README.md AGENTS.md SPEC.md CHANGELOG.md VERSION + Makefile (includes src/*/[*.mk]) .clang-format + src/ + cli/ arg parser tree, detz_main, future molo_subcommand.{h,cpp} + config/ consts, settings, env, boot + util/ ansi, text, time, truncate, files_mention, complete + shell/ sandbox, session, pty, spinner, livestream, lanaboard, glow + seal/ seal (vault crypto) + store/ sqlite store, fts, spill, debuglog, migrations + llm/ thin adapters over moloapp src/llm + src/clients (streaming); model_health, chat_client, backends + agent/ tools registry, approvals, agent loop, goal_verifier, retrieval_eval, rag + comms/ telegram, install, sysinfo, speak + plexer/ controller iface, native, tmux + tui/ theme/style/span/events/keys/render (per tui.md) + mcp/ serve-mcp http (per mcp.md), auth bearer + cap/ orchestrator (per cap.md) + tests/ README.md + per-module *_test.cpp mirroring ../test_tai.py + ../tai_test.py cases + docs/ specs per module (cli.md, store.md, llm.md, agent.md, tui.md, mcp.md, cap.md), workflow.md, versioning.md + tools/ version-bump.sh, tag.sh, changelog.sh + build/ (generated only) +``` + +Rules: headers `.h` + impl `.cpp`; one class per file pair where feasible; no outputs outside `build/`; `-std=c++23 -Wall -Wextra`, warning-free. + +## 4. moloapp reuse map + +- HTTP/SSE streaming: `moloapp/src/clients/*` (`client.h`, `sse.h`, transports) — use directly, do not reimplement. +- Provider/model types: `moloapp/src/llm/*`, `src/openai|anthropic|google|deepseek|xai|openrouter/*`. +- App patterns (errors, middleware, routing) for MCP HTTP serve: `moloapp/src/app/**/*`. +- CLI conventions: `moloapp` CLI part (`tests/...cli`, `src/cli`) + `audit/MOLO-CLI-001`. +- Build pattern: per-part `*.mk` + root Makefile; `example/example.mk` as template. + +## 5. CLI design (molo-ready) + +- `detz` binary with same flags as tai.py argparse tree (parity table in `docs/cli.md`). +- Implementation: `src/cli::{Parser, Command, DetzCli}` returning a `Command` value object; `main.cpp` only wires DI + dispatches. Future `molo detz` = register `DetzCommand` as subcommand (adapter header `molo_subcommand.h` stubbed in M1). +- Golden test: `--help` output diff vs `python ../tai.py --help`. + +## 6. Milestones & acceptance + +- M0 scaffold: dirs, root Makefile + `src/*/[*.mk]`, `main.cpp` printing version, `make`, `make test` green, AGENTS.md workflow live. Accept: warning-free build, outputs only in `build/`. +- M1 CLI parity: full argparse tree ported + `--help` golden test + molo subcommand stub. Accept: flag-for-flag table complete. +- M2 util+config: consts/settings/ansi/text/files. Accept: unit tests mirror python edge cases. +- M3 store: sqlite+FTS5, spill, debuglog, migrations. Accept: CRUD/FTS tests pass. +- M4 shell: sandbox/session/pty/speak. Accept: sandboxed exec tests. +- M5 llm: adapters over moloapp clients + streaming + health. Accept: mocked-transport chat/stream tests, no real network. +- M6 agent/tools: registry, approvals, loop, goal verifier, rag eval. Accept: ported `test_tai.py` scenarios (mocked backends). +- M7 plexer/tui/comms: native+tmux controllers, TUI render, telegram/install/sysinfo. Accept: headless render + controller tests. +- M8 mcp+cap: serve-mcp (bearer), cap orchestrator. Accept: route-level tests through real stack. +- M9 docs/release automation: README, SPECs, CHANGELOG, version bump/tag targets. Accept: `make release` path documented & dry-runnable. + +Order per AGENTS.md workflow: spec delta → code → tests → changelog, per milestone. + +## 7. Build/test/lint (mirrors moloapp AGENTS.md) + +```sh +g++ --version # need C++23 (g++ ≥ 13, moloapp expects 15.2) +make # whole repo +make test +make lint # -Wall -Wextra clean; zero findings +make release PART=patch|minor|major MSG="..." +``` + +## 8. Versioning/tagging/git automation + +- `VERSION` file (`0.1.0` start, semver); `src/cli/version.hpp` generated from it at build. +- Targets: `make version-bump PART=x`, `make tag` (annotated `detz-vX.Y.Z`), `make changelog` (prepend CHANGELOG entry), `make release PART=x MSG=...` = bump+changelog+commit+tag. Never push unless asked. +- First: write root `AGENTS.md` (workflow above) before M1 code — per original request. + +## 9. Risks + +- tai.py size (~40k lines): mitigate by module-by-module parity tables + ported test counts per milestone. +- moloapp API drift: pin moloapp commit in SPEC; thin adapter layer in `src/llm/`. +- sqlite/FTS5 + pty in C++: reuse system sqlite3, forkpty; isolate in `store/` + `shell/`. + +## 10. Next action + +Write `AGENTS.md` (workflow + commands), then M0 scaffold. diff --git a/taicpp/tests/whatsapp_data/README.md b/taicpp/tests/whatsapp_data/README.md new file mode 100644 index 0000000..9dd7aa4 --- /dev/null +++ b/taicpp/tests/whatsapp_data/README.md @@ -0,0 +1,12 @@ +# WhatsApp golden test vectors + +These four files are copied verbatim from jfelipesjc/wa-go (MIT licensed), +`internal/.../testdata`, and are used by `tests/whatsapp_test.cpp` to validate +detz's native WhatsApp wire/crypto/Signal implementation byte-for-byte. They +were generated by wa-go's capture harness from the real `@whiskeysockets/baileys` +client; see docs/whatsapp.md. Do not edit. + +- `codec_battery.jsonl` binary-XML node encode/decode battery (19 cases) +- `client_payload.json` a registration ClientPayload with its exact bytes +- `noise.json` a full Noise XX handshake transcript with keys +- `session_ab.json` a Signal X3DH + Double Ratchet session (4 exchanges)