# PLAN 3 — Unified Hierarchical Agent Cluster (OpenAI-compatible, native C++23) Merges plan2 + 2.1 + 2.2 + 2.3 into one DRY, recursively-checked design with zero open questions. Every decision binds to an existing framework path. No new third-party deps (wolfSSL only, per plan.md §1). ## 0. Reading guide / reuse map (do not reimplement) | Need | Reuse (read first) | Owner plan | |---|---|---| | HTTP routes, middleware, SSE/chunked | `src/app/routing/`, `src/app/http_msg/`, `src/app/middleware/`, `src/runtime/http/`, `src/realtime/` | 05, 06 | | Sessions + memory bank | `src/google/sessions/` (`sessions.h`, `memory_bank.*`, `engine_service.*`) | 16 | | Persistence | `src/data/orm/` + `drivers/sqlite|pg`, `src/data/kv/` | 04 | | Supervisor/workers | `src/supervisor/tasks/`, `bus/`, `common/` | 07 | | Observability/audit | `src/observability/` (logging/metrics/tracing/correlation), `src/audit/`, `src/governance/` | 02, 12, 14 | | Tenancy/auth | `src/tenancy/` (+ `bridge.h`), `src/security/`, `src/app/middleware/builtins.h` | 11, 03 | | Generic HTTP/OpenAPI clients (Generic Discovery) | `src/clients/` (`client.h`, `openapi/`, `retry.*`, `pool.*`) | 08 | | LLM calls | `src/llm/` (`api_client.*`, `chat_types.*`, `sse.*`, `error.*`) | 17 | | OpenAI wire types (request/parse/respond, NOT new types) | `src/openai/core/` (`chat_completions.*`, `chat_request.*`, `embeddings.*`, `models.*`, `tools/`, `sse`) | 17 | | Vendor execution | `src/google/` (`generative.*`, `gemini_types.*`, `live/`, `adk/`, `sandbox/`, `armor/`), `src/anthropic/`, `src/xai/`, `src/deepseek/`, `src/meta/`, `src/openrouter/`, `src/vendors/` | 17 | | Workflow gates (spec→code→example→tests→changelog, lint→suite→bump→commit→tag→deploy→verify) | `AGENTS.md`, `plans/10-workflow-example.md` | 10 | Decisions resolving plan2-family contradictions: - **JSON:** in-house only. plan2.2's `nlohmann/json` is rejected (violates plan.md §1). Tool args and schemas use `src/data/value.hpp` (`Value`) + existing OpenAI wire serializers in `src/openai/core/`. - **Default domain:** `"llm"` fallback is kept but bound to a real coordinator that ships in the same change (see §4; `LlmVendorCoordinator` using `src/llm/api_client` + `src/vendors/`). - **LLM-router default:** `enable_llm_fallback=true`, backed by `src/llm/api_client` with structured output via `src/openai/core/tools/`; rules+embedding stay the fast path. - **Threading:** coordinators run tool calls concurrently on `src/runtime` executor / `src/supervisor/tasks` pool; RootRouter itself stays synchronous and cheap (routing only). - **Auth/rate-limit/Armor:** not "future work" — §6 wires them to tenancy middleware + Armor client from day one. ## 1. External contract (façade) Routes (register in `src/app/routing/`, guarded by tenancy/auth middleware): - `POST /v1/chat/completions` — JSON body parsed with `src/openai/core/chat_request.*`; `stream:false` → single JSON via `chat_completions.*`; `stream:true` → SSE via `src/llm/sse.*` + runtime chunked writer (`src/realtime/`). - `GET /v1/models` — serve from `src/openai/core/models.*` (static registry of coordinator-backed model ids). - `POST /v1/embeddings` — passthrough to `src/openai/core/embeddings.*` (used by §3 warmup too). - `POST /workflow/release`-style ops endpoints stay untouched. Handler pipeline per request (single file `src/agent/facade.{h,cpp}`, owns nothing else): 1. Auth/tenant resolve via middleware (`tenant_id`, `user_id`, claim key) — reject unauthenticated before any LLM spend. 2. Session load/create via `src/google/sessions/sessions.h` + `memory_bank.*`; restore `last_domain`, `last_routing_method`, rolling summary into `AgentContext.state`. 3. Convert OpenAI messages → `agent::Message` with existing converters (extend `src/openai/core/chat_completions.*`, do not fork new types). 4. `root_->run(ctx)` → normalize `AgentResult` → OpenAI response (content or `tool_calls` via `src/openai/core/tools/`), usage/cost attached from observability span. 5. Persist state + append audit span (route, tools, tokens, cost, latency); on streaming, flush `data: [DONE]` terminator. ## 2. Core types (single header, no duplication) `src/agent/types.{h,cpp}` — the ONLY definition of these (plan2.1/2.2/2.3 duplicates deleted): `Message{Role,content,name,tool_call_id}`, `AgentContext{session_id,tenant_id,user_id,messages,state,metadata,audit,persist_state}`, `AgentResult{content,success,error,routed_to,routing_confidence,routing_method,routing_latency_us,usage{prompt,completion,total,cost}}`, `Agent{name,description,keywords,run}`, `Embedding{values,cosine}`, `EmbeddingProvider::embed→expected`. `Tool{name,description,parameters_schema(Value),execute(Value,ctx)→expected}`, `ToolCall{id,name,arguments}`, `ToolRegistry{add,find,openai_tools→via src/openai/core/tools/}`. ## 3. RootRouter (fast path, no heavy tools) `src/agent/root_router.{h,cpp}`. Owns `vector>` coordinators only; zero vendor calls inside. - `Config{embedding_threshold=0.72, enable_embedding=true, enable_llm_fallback=true, max_recent=3, default_domain="llm_vendor"}`. - `run(ctx)`: `extract_routing_text` (last ≤3 user messages) → `decide` → `audit("routed",…)` → child `run` → stamp `routed_to/confidence/method/latency` → `persist_state("last_domain",…)`. Empty messages → typed error, no throw. - Cascade (first hit wins): (a) deterministic rules — Workspace{gmail,drive,calendar,sheets,docs,meet,chat}, Voice{audio,voice,tts,stt,wav,live}, Maps{maps,places,route,geocode}, Data{bigquery,firestore,storage,pubsub}, skip missing names safely; (b) keyword overlap score over `keywords()` (threshold ≥1 hit, tie → first registered); (c) embedding cosine vs warmed cache (mutex-guarded `warmup_embeddings()` at startup + after `add_domain`); (d) LLM structured-output router via `src/llm/api_client` ("return exactly one of [names] + reason", temperature 0); (e) `default_domain`, else `domains_[0]`, else error. - Rules in (a) are generated from each coordinator's `keywords()` (prefix table rebuilt in `rebuild_index()`), not a second hardcoded list — the DRY fix vs 2.1/2.3 divergence. - `EmbeddingProvider` impl (`src/agent/gemini_embedder.{h,cpp}`) wraps `src/openai/core/embeddings.*` or `src/google/models` embeddings; cache keyed by domain name; failures degrade to next stage, never fail the request. ## 4. Coordinators (the 8 domains; root sees only these) Each: `src/agent/coordinators/.{h,cpp}`, implements `Agent`, owns a `ToolRegistry`, ReAct loop ≤5 rounds, concurrent tool fan-out (§5). Keywords double as routing table (§3). | name | description seed | tools → existing client | |---|---|---| | `workspace` (ship first) | Gmail/Drive/Calendar/Sheets/Docs/Meet/Chat | `gmail_list/send`, `calendar_list/create`, `drive_search` → `src/google/` Workspace clients | | `voice` (ship first) | TTS/STT/live audio, WAV | `tts_synth`, `stt_transcribe`, `live_start` → `src/openai/core/speech.*`, `transcriptions.*`, `src/google/live/` | | `llm_vendor` (ship first; IS the default) | xAI/Anthropic/Meta/DeepSeek/OpenRouter overflow + generic chat | `chat_via_vendor` → `src/llm/api_client` + `src/vendors/`, `src/anthropic/`, `src/xai/`, `src/deepseek/`, `src/meta/`, `src/openrouter/` | | `maps` | places/routes/geocode | → existing Maps client (or `src/clients/openapi/` discovery) | | `data` | BigQuery/Firestore/Storage/PubSub | → Data clients + `src/data/` | | `platform` | runtime/sessions/memory/sandbox, Agent Engine | → `src/google/runtime/`, `sessions/`, `sandbox/`, `adk/` | | `cloud_ai` | Model Garden/Vertex/Armor/partner+open models | → `src/google/models/`, `armor/` | | `discovery` | the 316 generic APIs | → `src/clients/` OpenAPI generic client, capped per-tenant rate | Coordinator loop (identical for all — one shared base `CoordinatorBase` in `src/agent/coordinator.{h,cpp}` so Workspace is not a snowflake): build chat request from `ctx.messages` + `tools_.openai_tools()` → call `src/llm/api_client` (model per coordinator, default fast) → if no tool_calls return content; else execute all calls concurrently (§5), append `Role::Tool` messages, repeat. Round cap + per-tool timeout + per-request token/cost budget enforced; overrun → partial content + `error` set, still audited. ## 5. Concurrency, streaming, state - Concurrency: tool fan-out uses `src/runtime` executor (`executor.*`/`strand.h`) or `src/supervisor/tasks/` pool — bounded (default 8), results re-ordered to call order before appending. No ad-hoc threads. Supervisor watches pools per plan 07. - Streaming: non-streaming path returns full `AgentResult`; streaming path runs the same `root_->run` then replays content as SSE chunks (`src/llm/sse.*` framing, `chat.completion.chunk` objects, `data: [DONE]`). Token-by-token live streaming is explicitly out of scope for v1 (documented, not a caveat: single ranked reason — coordinators fan out to tools before final text exists). - State/memory: `AgentContext.state` is a write-through view over sessions KV (`persist_state` → sessions store; load at §1 step 2). Cross-turn facts go to `memory_bank.*` table `(session_id,key,value,embedding,timestamp)`; embedding column reuses §3 provider. No parallel memory impl. ## 6. Cross-cutting (shipped, not TODO) - Tenancy/auth/quotas: every façade route behind tenancy middleware; `tenant_id` scopes router instance selection (single shared root + per-tenant budgets is the fixed choice — no "one or per-tenant" ambiguity), session isolation, tool allowlists, per-tenant token/$ caps (checked in `CoordinatorBase` before each LLM/tool call). - Safety: Workspace/Data/Discovery mutations pass `src/google/armor/` (Model Armor) pre-check; failures block the tool with typed error. - Observability: one span per request (`correlation/`), child spans per routing stage + tool call; metrics: routing_method/confidence/latency, tool latency/error, tokens/cost per tenant+domain; audit writes who→what→cost via `src/audit/` + `src/governance/`. - Errors: `expected` everywhere; handler maps to OpenAI `error` objects; 5xx only on transport failure. ## 7. Build order (each step green per AGENTS.md before next) 1. `src/agent/types` + `coordinator` base + `ToolRegistry` (unit tests, no network). 2. `gemini_embedder` + `RootRouter` + `llm_vendor` coordinator (default route works end-to-end). 3. Façade + `POST /v1/chat/completions` (non-stream) + `GET /v1/models` + session wiring. 4. `workspace` + `voice` coordinators with real clients; SSE streaming. 5. Remaining 5 coordinators; Armor + quotas + cost metrics; `POST /v1/embeddings`. 6. Example route + suite + CHANGELOG per plan 10; `make lint test` clean, `-Wall -Wextra` silent. ## 8. Acceptance (recursive check — all must hold) - One OpenAI-compatible endpoint fronting 8 coordinators; root makes no vendor calls. - Rules→keywords→embedding→LLM→default cascade measured (p50 dominated by rules path). - No `nlohmann` or new JSON lib; no second Message/Context/Tool definition; routing keywords sourced from coordinators. - Default route resolves to a shipped coordinator; streaming terminates with `[DONE]`; tenants isolated with budgets; every step audited with cost.