glue: profile switch test, issues doc updated with the smoke results

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-25 07:23:43 +01:00
co-authored by Claude Sonnet 5.5
parent 9f790e3177
commit ad500b0327
3 changed files with 46 additions and 8 deletions
+10 -6
View File
@@ -36,12 +36,16 @@ around silently; items marked "glue side" are limits the glue itself has.
`ModelHealth`, `Opencode`, `feed_sse`, `provider_info`; docs/wiring/agent.md); not touched here.
9. **`probe_backend` in the REPL banner** runs sequentially with a 6 s timeout per backend, like python: a dead
backend delays the prompt. Not changed.
10. **`--tui-local` model turn not proven.** The chat scene starts, draws, runs slash commands through the adapter
and exits cleanly (driven over a pty); a full model turn through the TUI was started but not confirmed in the
vault within the observation window. `--tui` (grr client) needs a grr server and was not started.
11. **`--mplex`, `--plexer-wrap`, `--install`, `--update-models` with writes** were wired to the module entry points; only
`--update-models` (read-only discovery) and `--demo-shot` were run. `--install` writes into `$HOME` and was not run
on a real home.
10. **Not covered by a committed test:** `--tui-local` (driven once over a pty by a throwaway script: it drew, ran a real
model turn whose answer reached the vault session, and `/quit` exited 0), `--tui` (grr client; without a tty it
exits 2 with `tui unavailable`, no grr server was started), `--mplex` and `--plexer-wrap` (without a tty they answer
`no server running on ...` / `open terminal failed: not a terminal`, exit 1; the multiplexer session itself was not
started), `--install` (run once against a throwaway `HOME`: binary copy, bash hook, venv and six bundled skills
written, exit 0), `--update-models` (read-only discovery ran), `--eval-retrieval` and `--stats-rag` (empty vault only).
11. **`--cap-speak` string content.** A `{"type":"user","message":{"role":"user","content":"text"}}` frame (content a
plain string) is split into one character per line before it reaches the model. This is the literal tai.py
behavior (`cap_stream_json.cpp` mirrors `for block in message.get("content")`); real clients send a block list,
which `tests/glue_flags.py` uses. Noted so nobody files it as a port bug.
## Fixed in the glue (one-line integration errors)
+1 -1
View File
@@ -89,7 +89,7 @@ busy timeout (tai.py), and the long-running modes are the modules' own loops.
| file | kind | what |
|---|---|---|
| `tests/glue_persist.py` | integration, real model | restart persistence in one long-lived shell, separate history for another shell, one-shot prompt with a shell tool call, `--version`/`--help`/`--cap-manifest`; asserts on vault files |
| `tests/glue_persist.py` | integration, real model | restart persistence in one long-lived shell, separate history for another shell, `/profile` and `/clear` reaching the agent's history, one-shot prompt with a shell tool call, `--version`/`--help`/`--cap-manifest`; asserts on vault files |
| `tests/glue_flags.py` | integration, plumbing (one case with a real model) | slash commands through the ports, `--scheduler`, `--telegram` without token, `--serve-mcp` over HTTP, `--cap-speak` with a real model, argument errors |
| `tests/glue_ports.cpp` | unit | every port non-null after `AppBundle::open`, `StoreAdapter` and `JsonBridge` round trips on a throwaway home; build line in the file header |
+35 -1
View File
@@ -12,6 +12,7 @@ for the deterministic --version/--help case.
GlueRestartPersistence start, a message the model answers, quit, start again in the same shell: history
resumed and extended; another shell gets a separate history
GlueProfileSwitch /profile and /clear act on the vault history the agent uses
GlueOneShotShell one-shot prompt that must make a shell tool call; the tool call and its result are in the vault
GlueFlagsSmoke --version and --help (no model)
@@ -159,6 +160,39 @@ class GlueRestartPersistence(PaneCase):
print("restart persistence ok: pane A %d messages, recalled=%s, pane B %d messages" % (len(a), recalled, len(b)))
class GlueProfileSwitch(PaneCase):
name = "GlueProfileSwitch"
async def run(self):
pane = await self.spawn()
rc, output = await self.pane_run(pane, ["Remember this codeword: %s. Reply with just OK." % CODEWORD, "/profile alt",
"What codeword did I tell you? If none, reply NONE.", "/profile default", "/clear", "/quit"])
return {"rc": rc}
async def verify(self, result):
problems = []
default = self.sessions("default")
alt = self.sessions("alt")
if len(alt) != 1:
raise TestAssertionFailed("profile alt should have its own session file, found %s" % alt)
a = self.load(alt[0])
users_alt = [m["content"] for m in a if m["role"] == "user"]
if users_alt != ["What codeword did I tell you? If none, reply NONE."]:
problems.append("alt profile history should hold only its own turn: %s" % users_alt)
if any(CODEWORD in (m.get("content") or "") for m in a if m["role"] == "assistant"):
problems.append("alt profile saw the default profile's codeword")
# /clear in the default profile emptied the saved history that /profile default had restored
if default:
left = [m for m in self.load(default[0]) if m["role"] == "user"]
if left:
problems.append("/clear did not reach the vault, default session still has %s" % [m["content"][:30] for m in left])
if result["rc"] != 0:
problems.append("exit code %d" % result["rc"])
if problems:
raise TestAssertionFailed("; ".join(problems))
print("profile switch ok: alt has %d messages, default session cleared" % len(a))
class GlueOneShotShell(PaneCase):
name = "GlueOneShotShell"
@@ -226,7 +260,7 @@ class GlueFlagsSmoke(PaneCase):
raise TestAssertionFailed("; ".join(problems))
ALL_CASES = [GlueFlagsSmoke(), GlueRestartPersistence(), GlueOneShotShell()]
ALL_CASES = [GlueFlagsSmoke(), GlueRestartPersistence(), GlueProfileSwitch(), GlueOneShotShell()]
if __name__ == "__main__":
sys.exit(main_cli(ALL_CASES, default_log_dir=str(TAICPP / "build" / "test-logs")))