retoor
afc88b8087
- New ask_user core tool: mid-turn user question with option picking, refusing cleanly in autonomous mode, for background workers, and without a tty; system prompt points at it instead of prose questions. - Structural loop control for prompt-JSON CLI backends (opencode, grok, claude, codex, gemini): shared cli_json_turn repairs a missing JSON envelope exactly once, then accepts prose; prompt contract hardened against narrate-without-calling. - research/agentic-tool-loop-design: 74 sources plus 2 conclusions backing the loop-control decisions; INDEX updated to 6 subjects.
55 lines
3.5 KiB
Markdown
55 lines
3.5 KiB
Markdown
# Writing Effective Tools for AI Agents — Using AI Agents
|
|
|
|
## Metadata
|
|
|
|
| Field | Value |
|
|
|---|---|
|
|
| Institute | Molodetz Institute — Research & Retrieval Division |
|
|
| Subject | agentic-tool-loop-design |
|
|
| Document type | Source Dossier |
|
|
| Tier | 1 |
|
|
| Source title | Writing effective tools for agents — with agents |
|
|
| Source URL | https://www.anthropic.com/engineering/writing-tools-for-agents |
|
|
| Source publisher / author | Anthropic Engineering (Ken Aizawa et al.) |
|
|
| Source type | blog post |
|
|
| Source date (published) | 2025-09-11 |
|
|
| Date researched | 2026-10-11 |
|
|
| Researched by | tai (agent), model: Muse Spark, on behalf of retoor |
|
|
| Research method | direct page fetch |
|
|
| Confidence | High |
|
|
| Status | Active |
|
|
|
|
## Summary
|
|
|
|
Anthropic's follow-up to Building Effective Agents, focused entirely on tool quality: prototype tools quickly, measure them with evaluation harnesses built on simple agentic `while`-loops, and iterate with agent help. Its principles: few high-level tools over many overlapping ones, namespacing to disambiguate, high-signal token-efficient responses with pagination/truncation defaults, actionable error messages, and prompt-engineered tool descriptions treated with the same care as system prompts.
|
|
|
|
## Key findings
|
|
|
|
- Evaluation agents should run as simple agentic loops (`while`-loops wrapping alternating LLM API and tool calls), one loop per task; instructing agents to output reasoning before tool-call blocks may improve effective intelligence via chain-of-thought.
|
|
- Build few thoughtful tools targeting high-impact workflows and scale from there; consolidate frequently chained multi-step operations into single tools (e.g. `schedule_event` instead of list/list/create).
|
|
- Tool responses must return high-signal information only, prefer natural-language identifiers over cryptic IDs, and stay token-efficient via pagination, range selection, filtering, or truncation with sensible defaults (Claude Code caps tool responses at 25,000 tokens).
|
|
- Truncation and error responses should steer the agent: encourage many small targeted searches over one broad one, and return specific actionable input corrections rather than opaque codes.
|
|
- Tool descriptions steer behavior measurably: precise refinements to descriptions took Claude Sonnet 3.5 to state of the art on SWE-bench Verified; one launch bug (needless `2025` appended to a search query) was fixed purely by improving the tool description.
|
|
- Metrics to collect per tool: call counts, errors, runtime, token consumption; redundant calls suggest pagination fixes, invalid-parameter errors suggest clearer descriptions.
|
|
|
|
## Pros / strengths
|
|
|
|
- Grounded in measured held-out evaluation results on real internal tools (Slack, Asana), not opinion.
|
|
- The truncation/error-steering guidance is immediately applicable to any harness.
|
|
|
|
## Cons / limitations / caveats
|
|
|
|
- Assumes MCP-style native tool calling on frontier models; the weak-model narration failure mode is out of scope.
|
|
- Vendor-flavored toward the Claude/Claude Code stack (workbench, interleaved thinking, MCP annotations).
|
|
|
|
## Relevance to tai
|
|
|
|
- Supports tai's existing output-truncation and spill-to-record behavior as industry-standard practice, and the per-tool error-as-result convention.
|
|
- The "descriptions steer behavior" finding backs the prompt-contract half of tai's narration fix: an explicit never-narrate-without-a-call rule in the system prompt and CLI-backend instructions is the research-supported mitigation where structural enforcement is unavailable.
|
|
|
|
## Changelog
|
|
|
|
| Date | Change |
|
|
|---|---|
|
|
| 2026-10-11 | Initial dossier created. |
|