glue: profile switch test, issues doc updated with the smoke results
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -36,12 +36,16 @@ around silently; items marked "glue side" are limits the glue itself has.
|
||||
`ModelHealth`, `Opencode`, `feed_sse`, `provider_info`; docs/wiring/agent.md); not touched here.
|
||||
9. **`probe_backend` in the REPL banner** runs sequentially with a 6 s timeout per backend, like python: a dead
|
||||
backend delays the prompt. Not changed.
|
||||
10. **`--tui-local` model turn not proven.** The chat scene starts, draws, runs slash commands through the adapter
|
||||
and exits cleanly (driven over a pty); a full model turn through the TUI was started but not confirmed in the
|
||||
vault within the observation window. `--tui` (grr client) needs a grr server and was not started.
|
||||
11. **`--mplex`, `--plexer-wrap`, `--install`, `--update-models` with writes** were wired to the module entry points; only
|
||||
`--update-models` (read-only discovery) and `--demo-shot` were run. `--install` writes into `$HOME` and was not run
|
||||
on a real home.
|
||||
10. **Not covered by a committed test:** `--tui-local` (driven once over a pty by a throwaway script: it drew, ran a real
|
||||
model turn whose answer reached the vault session, and `/quit` exited 0), `--tui` (grr client; without a tty it
|
||||
exits 2 with `tui unavailable`, no grr server was started), `--mplex` and `--plexer-wrap` (without a tty they answer
|
||||
`no server running on ...` / `open terminal failed: not a terminal`, exit 1; the multiplexer session itself was not
|
||||
started), `--install` (run once against a throwaway `HOME`: binary copy, bash hook, venv and six bundled skills
|
||||
written, exit 0), `--update-models` (read-only discovery ran), `--eval-retrieval` and `--stats-rag` (empty vault only).
|
||||
11. **`--cap-speak` string content.** A `{"type":"user","message":{"role":"user","content":"text"}}` frame (content a
|
||||
plain string) is split into one character per line before it reaches the model. This is the literal tai.py
|
||||
behavior (`cap_stream_json.cpp` mirrors `for block in message.get("content")`); real clients send a block list,
|
||||
which `tests/glue_flags.py` uses. Noted so nobody files it as a port bug.
|
||||
|
||||
## Fixed in the glue (one-line integration errors)
|
||||
|
||||
|
||||
+1
-1
@@ -89,7 +89,7 @@ busy timeout (tai.py), and the long-running modes are the modules' own loops.
|
||||
|
||||
| file | kind | what |
|
||||
|---|---|---|
|
||||
| `tests/glue_persist.py` | integration, real model | restart persistence in one long-lived shell, separate history for another shell, one-shot prompt with a shell tool call, `--version`/`--help`/`--cap-manifest`; asserts on vault files |
|
||||
| `tests/glue_persist.py` | integration, real model | restart persistence in one long-lived shell, separate history for another shell, `/profile` and `/clear` reaching the agent's history, one-shot prompt with a shell tool call, `--version`/`--help`/`--cap-manifest`; asserts on vault files |
|
||||
| `tests/glue_flags.py` | integration, plumbing (one case with a real model) | slash commands through the ports, `--scheduler`, `--telegram` without token, `--serve-mcp` over HTTP, `--cap-speak` with a real model, argument errors |
|
||||
| `tests/glue_ports.cpp` | unit | every port non-null after `AppBundle::open`, `StoreAdapter` and `JsonBridge` round trips on a throwaway home; build line in the file header |
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ for the deterministic --version/--help case.
|
||||
|
||||
GlueRestartPersistence start, a message the model answers, quit, start again in the same shell: history
|
||||
resumed and extended; another shell gets a separate history
|
||||
GlueProfileSwitch /profile and /clear act on the vault history the agent uses
|
||||
GlueOneShotShell one-shot prompt that must make a shell tool call; the tool call and its result are in the vault
|
||||
GlueFlagsSmoke --version and --help (no model)
|
||||
|
||||
@@ -159,6 +160,39 @@ class GlueRestartPersistence(PaneCase):
|
||||
print("restart persistence ok: pane A %d messages, recalled=%s, pane B %d messages" % (len(a), recalled, len(b)))
|
||||
|
||||
|
||||
class GlueProfileSwitch(PaneCase):
|
||||
name = "GlueProfileSwitch"
|
||||
|
||||
async def run(self):
|
||||
pane = await self.spawn()
|
||||
rc, output = await self.pane_run(pane, ["Remember this codeword: %s. Reply with just OK." % CODEWORD, "/profile alt",
|
||||
"What codeword did I tell you? If none, reply NONE.", "/profile default", "/clear", "/quit"])
|
||||
return {"rc": rc}
|
||||
|
||||
async def verify(self, result):
|
||||
problems = []
|
||||
default = self.sessions("default")
|
||||
alt = self.sessions("alt")
|
||||
if len(alt) != 1:
|
||||
raise TestAssertionFailed("profile alt should have its own session file, found %s" % alt)
|
||||
a = self.load(alt[0])
|
||||
users_alt = [m["content"] for m in a if m["role"] == "user"]
|
||||
if users_alt != ["What codeword did I tell you? If none, reply NONE."]:
|
||||
problems.append("alt profile history should hold only its own turn: %s" % users_alt)
|
||||
if any(CODEWORD in (m.get("content") or "") for m in a if m["role"] == "assistant"):
|
||||
problems.append("alt profile saw the default profile's codeword")
|
||||
# /clear in the default profile emptied the saved history that /profile default had restored
|
||||
if default:
|
||||
left = [m for m in self.load(default[0]) if m["role"] == "user"]
|
||||
if left:
|
||||
problems.append("/clear did not reach the vault, default session still has %s" % [m["content"][:30] for m in left])
|
||||
if result["rc"] != 0:
|
||||
problems.append("exit code %d" % result["rc"])
|
||||
if problems:
|
||||
raise TestAssertionFailed("; ".join(problems))
|
||||
print("profile switch ok: alt has %d messages, default session cleared" % len(a))
|
||||
|
||||
|
||||
class GlueOneShotShell(PaneCase):
|
||||
name = "GlueOneShotShell"
|
||||
|
||||
@@ -226,7 +260,7 @@ class GlueFlagsSmoke(PaneCase):
|
||||
raise TestAssertionFailed("; ".join(problems))
|
||||
|
||||
|
||||
ALL_CASES = [GlueFlagsSmoke(), GlueRestartPersistence(), GlueOneShotShell()]
|
||||
ALL_CASES = [GlueFlagsSmoke(), GlueRestartPersistence(), GlueProfileSwitch(), GlueOneShotShell()]
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main_cli(ALL_CASES, default_log_dir=str(TAICPP / "build" / "test-logs")))
|
||||
|
||||
Reference in New Issue
Block a user