tai/mcp.md

# mcp.md

tai speaks MCP in both directions. As a **client** it spawns or dials
third-party MCP servers, bridges their tools into the lazy tool system,
and stores connections in the vault with per-profile enablement. As a
**server** (`tai.py --serve-mcp KEY`) it exposes every native tool plus
`chat` and `completions` over Streamable HTTP. `McpDocTests` fails when
this file drifts from the code. Research: `research/mcp-client-support/`.

## Shared wire subset

Protocol version is `2025-11-25`, sent as `MCP_PROTOCOL_VERSION`.
Client name and server name are both `tai`; server version is `VERSION`.

| Direction | Method | Shape |
|---|---|---|
| C2S | `initialize` | `{protocolVersion, capabilities: {}, clientInfo: {name, version}}`; server answers its version plus `capabilities.tools`, or the client aborts |
| C2S | `notifications/initialized` | no params, no answer |
| both | `ping` | `{}` in, `{}` out |
| C2S | `tools/list` | `{cursor?}` in, `{tools: [...], nextCursor?}` out, followed until no cursor |
| C2S | `tools/call` | `{name, arguments?}` in, `{content: [...], isError?}` out |
| C2S | `notifications/cancelled` | `{requestId, reason?}` on call timeout, then the call fails locally |

JSON-RPC errors (`-32601` unknown, `-32602` bad params, `-32603`
internal) mean protocol failure; a tool that ran and failed arrives
as a normal result with `isError: true`, and tai preserves that split
end to end. Roots, sampling, elicitation, logging display, progress
tokens, and the legacy HTTP+SSE transport are not implemented on
either side; anything asking for them gets `-32601`.

## Caps

| Cap | Value | Meaning |
|---|---|---|
| `MCP_CONNECT_TIMEOUT` | 20s | spawn plus handshake, per server |
| `MCP_CALL_TIMEOUT` | 60s | one `tools/call`, then cancel notification and local failure |
| `MCP_RESULT_MAX` | 64KB | tool result text kept inline; larger spills to a vault record and the model gets the id, never a silent truncation |
| `MCP_HTTP_MAX_BODY` | 10MB | largest request/response body on either HTTP leg |
| `MCP_MAX_SERVERS` | 32 | registered servers; the 33rd registration is refused |
| `MCP_MAX_TOOLS` | 100 | tools bridged per server; extras are listed but not offered |
| `MCP_SESSION_IDLE` | 3600s | idle server-side MCP sessions are dropped |
| `MCP_CHAT_TIMEOUT` | 600s | deadline per served `chat` turn |
| `MCP_RESOLVER_TIMEOUT` | 300s | deadline for the fork-based GitHub resolver subagent |
| `MCP_RESOLVER_STEPS` | 12 | max agent steps for one GitHub resolution |
| `MCP_OAUTH_TIMEOUT` | 300s | how long `mcp_login` waits for the browser redirect |

## Client

### Transports

stdio: the server command spawns via subprocess with piped stdio,
newline-delimited JSON-RPC, stderr drained to a capped buffer (never
parsed). Shutdown order is close-stdin, wait, terminate, kill. The
exact argv is shown at registration consent and re-shown whenever it
changes. stdio reads need POSIX `select`; the HTTP leg covers the
rest.

Streamable HTTP: one `POST <url>` per request with
`Accept: application/json, text/event-stream`, the stored headers,
and `Mcp-Session-Id` after `initialize`. The reply is a single JSON
message or a `text/event-stream` body that the client parses by hand
(`data:` lines, blank-line frames, `event: message` only). `GET` and
`DELETE` are never sent; sessions end by dropping the id.

### Vault tables

```sql
mcp_servers(name PK, transport, command, args_json, env_json, url,
            headers_json, cwd, allow_json, deny_json, source, created, updated)
mcp_server_profiles(server, profile, enabled) PK(server, profile)
mcp_tool_pins(server, tool, hash, updated) PK(server, tool)
mcp_server_tools(server, tool, description, schema_json, updated) PK(server, tool)
```

`headers_json` is sealed at rest whenever the vault seal is enabled.
`source` records what was pasted (`config-json`, `config-toml`,
`github:<owner>/<repo>`, `explicit`). A server with no profile row
defaults to disabled everywhere; enabling is always explicit and per
profile.

### Normalizer (`mcp_normalize`)

Accepted input: a JSON object with a `mcpServers` or `servers`
wrapper (or one bare entry), a TOML document with `[mcp_servers.*]`
tables (3.11+, clean error below), or a bare server name already
stored. Repair rules: an entry with `url` and no `type` gains
`type: http`; command entries keep `command`/`args`/`env`/`cwd`.
`${VAR}` and `${VAR:-default}` expand in `command`, `args`, `env`,
`url`, and `headers` at use time from process env first, then vault
secrets; Cursor `${env:NAME}` rewrites to `${NAME}`; VS Code
`${input:id}` resolves against vault secrets by id and fails the
spawn when missing. Unresolved names warn and keep the literal for
display, but the spawn is refused. `npx` on Windows wraps in
`cmd /c`; `uvx` never does. TOML-only keys map as: `env`/`env_vars`
to env, `url` plus `http_headers`/`env_http_headers` to the HTTP
leg, `enabled` to the registering profile, `enabled_tools`/
`disabled_tools` to a tool allow/deny stored beside the server row.
`timeout`/`startup_timeout` are ignored with a note: tai's connect/call
caps govern.

GitHub URLs (`github.com/<owner>/<repo>` plus anything after) are
not normalizable; they go to the resolver in two passes. The
deterministic pass reads `mcp.json`, `.mcp.json`, `server.json`,
then `package.json` (bare `npx -y <name>`), then the README for an
`npx`/`uvx` run block, all through the contents API. When that
finds nothing, `mcp_resolve_agent` spawns a bounded fork subagent
(same profile, `default_runner`, `MCP_RESOLVER_STEPS` steps,
`MCP_RESOLVER_TIMEOUT` seconds) with a strict brief: read the same
files plus the raw README and return exactly one fenced JSON
`command` or `url` entry, or `{"error", "evidence"}`. Anything
else is a loud failure naming both passes. Every resolved entry is
verified by a live handshake before it is stored.

### Bridge

Enabled servers' tools enter the lazy tool system as
`mcp_<server>_<tool>`, sanitized to `[a-z0-9_]` (anything else
becomes `_`, empties gain a numeric suffix). `select_tools` also
iterates the bridge schemas, so description-word matching works
unchanged; `Tools.dispatch` falls back to the bridge for `mcp_*`
names. Schemas pass the server's `inputSchema` through as
`parameters`, defaulting to `{type: object, properties: {}}`.
Calls ask approval exactly like non-safe shell commands (always in
interactive mode, auto in `--auto`, silent in `--yolo`), and the
consent screen names server, tool, and argument digest.

Trust: tool names, descriptions, and annotations are untrusted text.
At registration tai hashes name plus description plus schema per
tool into `mcp_tool_pins`; added or redefined tools on a later
`tools/list` re-prompt for approval and are withheld until granted,
while pure removals apply silently.
Results over `MCP_RESULT_MAX` spill to a vault record. Every call
lands in the audit trail (`mcp-call`, `server/tool`, argument
digest, outcome, duration, size). `${VAR}` secret values resolve at
spawn/call time and `Store.redact` scrubs known values from results.

HTTP failures arrive as `McpHttpError` (an `McpError` carrying the
status code and the `WWW-Authenticate` header) so the bridge and
the OAuth layer can tell 401 from anything else.

### OAuth (remote servers)

Servers that answer 401 get the OAuth treatment, all stdlib
(`urllib`, `http.server`, `secrets`, `webbrowser`), in `McpOAuth`:

1. `mcp_add` on a 401 stores the server **disabled** (no tools
cached) and points at `mcp_login`; servers with a static
`Authorization` header that 401 instead fail with a
check-your-credentials error.
2. `mcp_login` probes anonymously (an open server is refused with
"needs no login"), then discovers: `resource_metadata` from the
401 header, else the origin's
`/.well-known/oauth-authorization-server`, keeping scope from the
protected-resource document. Servers without a
`registration_endpoint` are refused: only dynamic clients.
3. Dynamic client registration (`tai-mcp`, public, loopback
redirect), then PKCE (`S256`) plus state, then the browser opens
(the URL is also printed, for headless terminals). The loopback
listener (`McpOAuthCallback`, `127.0.0.1`, ephemeral port,
`MCP_OAUTH_TIMEOUT` seconds) takes the code, which is exchanged
for tokens.
4. Tokens live in vault `secrets` as
`mcp-oauth-<server>-access`/`-refresh` (name-clamped to
`SECRET_RE`, sealed at rest, redacted from output), with the
`client_id` in the access secret's meta. `mcp_remove` drops them
for every known profile plus the current one.
5. Every HTTP connect injects `Authorization: Bearer <access>`
unless the entry configures its own. A 401 mid-call refreshes
once (re-discover, `refresh_token` grant, rotation honored) and
retries; when that fails the error names `mcp_login`.

Login is interactive-local by nature: run it in a REPL, not over
a served session, since only the local terminal shows the URL.

### Agent tools

| Tool | Args | Effect |
|---|---|---|
| `mcp_add` | `source`, `name?`, `profile?` | normalize or resolve, consent the spawn, live-verify, store, return the tool list |
| `mcp_list` | - | servers with transport, enabled profiles, tool counts, pin state |
| `mcp_enable` | `server`, `profile?` | enable for a profile (current when omitted) |
| `mcp_disable` | `server`, `profile?` | disable for a profile |
| `mcp_remove` | `server` | delete server, profile rows, and pins; asks first |
| `mcp_tools` | `server?` | list bridged tools with descriptions and pin state |
| `mcp_call` | `server`, `tool`, `args?` | call one tool explicitly (same path as bridged dispatch) |
| `mcp_login` | `server`, `profile?` | OAuth browser login, store sealed tokens, verify, enable; asks first |

## Server (`tai.py --serve-mcp KEY`)

One process, one `Config`, one `Store`, single-threaded `HTTPServer`
(no thread pool: sequential requests fit MCP clients and the
no-internal-threads rule). Flags: `--serve-mcp KEY` (required; the
Bearer token clients must send), `--mcp-port` (default from
`TAI_MCP_PORT` else `MCP_DEFAULT_PORT`, 8765), `--mcp-bind`
(default `TAI_MCP_BIND` else `127.0.0.1`).

| Endpoint | Behavior |
|---|---|
| `POST /mcp` | one JSON-RPC message; missing/invalid `Authorization: Bearer KEY` gets 401. Answers `initialize` (new `Mcp-Session-Id`), `notifications/initialized` (none), `ping`, `tools/list` (no pagination, all tools fit), `tools/call`. Always a single JSON reply, never SSE. Unknown methods get `-32601`. |
| `GET /mcp` | 405 (no server-initiated streams in v1) |
| `DELETE /mcp` | drops the `Mcp-Session-Id` session, 200 |

Sessions live in memory: `{agent, profile, created, last_seen}`.
Idle sessions past `MCP_SESSION_IDLE` are dropped on next access.

### Served tools

Every native tool schema is served verbatim (name, description,
`parameters` as `inputSchema`), plus two extras, 63 total:

| Tool | Args | Effect |
|---|---|---|
| `chat` | `message`, `session?`, `profile?`, `max_steps?` | full agent turn on the session's own `Agent` (`run_turn`, capture), history kept in the session; uses the current model with tools |
| `completions` | `messages[]` | one stateless `ChatClient.complete` with the current model, no tools; roles validated (`system`/`user`/`assistant`) |

The key is the authorization: serve mode runs the equivalent of
`--yolo` for tool approvals, documented loudly at startup, because
no user is present to ask. `ask_user` over the server always denies
(no prompter, no tty). Served `chat` agents can themselves use
bridged MCP tools (they are full agents), but bridged foreign tools
are never re-served as MCP tools: transitive serving would launder
untrusted servers through tai's identity.

### Exclusions (both sides)

Confidential or pre-registered OAuth clients (public DCR only),
device flow, and CIMD; roots/sampling/elicitation serving; legacy
HTTP+SSE; `GET /mcp` streams; file-attachment content beyond text
passthrough (images/audio arrive as their JSON, unchanged); batch
JSON-RPC arrays (one message per POST).

## Code map

`MCP_PROTOCOL_VERSION` pins the wire; `mcp_normalize` plus
`mcp_expand_vars` own pasted-config intake; `mcp_resolve_agent`
owns the fork-based GitHub fallback; `McpStdioTransport` and
`McpHttpTransport` own the two legs behind `McpClient`;
`McpHttpError` carries HTTP status to the bridge; `McpOAuth` plus
`McpOAuthCallback` own discovery, DCR, PKCE loopback, refresh, and
sealed token storage; `McpBridge` owns enablement, drift pins,
approval, caps, spill, and audit; `McpServeContext` plus
`mcp_serve_handler` own the HTTP server; `mcp_serve_main` owns
flags and boot.

## Test hooks

`McpDocTests` asserts this file names every `MCP_*` cap, every
method in the wire table, every vault table, every agent tool, and
both server extras. Transport tests drive a fake MCP server (a
`python3 -c` NDJSON stub for stdio, `http.server` in-test for the
HTTP leg); serve tests drive the real `POST /mcp` with `urllib`
against `ChatClient` fakes, never a live model; `McpOAuthTests`
covers discovery, PKCE, the loopback round trip, refresh, the
401 paths, and token cleanup.