Files

T

retoor afc88b8087 ask_user tool, prompt-JSON CLI repair loop, tool-loop research archive

- New ask_user core tool: mid-turn user question with option picking,
  refusing cleanly in autonomous mode, for background workers, and
  without a tty; system prompt points at it instead of prose questions.
- Structural loop control for prompt-JSON CLI backends (opencode,
  grok, claude, codex, gemini): shared cli_json_turn repairs a
  missing JSON envelope exactly once, then accepts prose; prompt
  contract hardened against narrate-without-calling.
- research/agentic-tool-loop-design: 74 sources plus 2 conclusions
  backing the loop-control decisions; INDEX updated to 6 subjects.

2026-10-11 02:54:17 +02:00

3.8 KiB

Raw Blame History

Tool Calling (AI Engineering Playbook)

Metadata

Field	Value
Institute	Molodetz Institute — Research & Retrieval Division
Subject	agentic-tool-loop-design
Document type	Source Dossier
Tier	1
Source title	Tool Calling
Source URL	https://github.com/karthikreddy-7/ai-engineering-playbook/blob/HEAD/docs/05-agents/tool-calling.md
Source publisher / author	karthikreddy-7 (GitHub)
Source type	other
Source date (published)	Unknown — not stated
Date researched	2026-10-11
Researched by	tai (agent), model: Muse Spark, on behalf of retoor
Research method	both
Confidence	Medium
Status	Active

Summary

A playbook chapter on function-calling mechanics: the model emits structured intent, the runtime validates, authorizes, executes, and returns keyed results, and the loop continues. Its key lines for tai: free-form text plus regex parsing is called out as fragile ("breaks on wording drift") versus native tool-call tokens as a trained contract; strict/constrained decoding kills syntax failures but never semantic or authorization errors; and the failure-mode table prescribes a system-prompt nudge for "not calling when it should" and harness caps for infinite retry.

Key findings

The model never executes anything: it emits structured intent, while auth, validation, side effects, and safety live in the runtime; tool definitions are prompts the model reads like any other instruction.
"Free-form text is a terrible API": prompting Action: ... and parsing with regex breaks on wording drift; native tool-call tokens plus JSON arguments are a contract the model is trained to emit.
Strict mode (OpenAI strict: true, Claude strict tool use) uses constrained decoding so arguments cannot leave the schema; it guarantees syntax, not semantics or safety, so authz stays runtime-side.
Standard round trip: schemas -> tool_use -> validate -> execute -> keyed tool_result -> answer or next call; failures return as the tool result (is_error: true) with actionable guidance instead of crashing the loop.
Failure-mode remedies: hallucinated names/args via strict mode plus validation plus error-as-result; calling when it shouldn't via tighter descriptions and tool_choice; not calling when it should via a system-prompt nudge; infinite retry of a failing tool via harness caps.
Scaling: keep the active tool set small (tens, not hundreds); beyond that use dynamic discovery (load definitions on demand) or subagents with focused toolsets.
Mutating tools should be idempotent or gated behind confirmation, since the model may repeat a call after an ambiguous result.

Pros / strengths

Dense, opinionated, and organized around production failure modes rather than hello-world flows.
The "tool definitions are prompts" framing and the explicit regex-parsing warning generalize beyond any one vendor.

Cons / limitations / caveats

Single-author playbook without stated measurements; several token/latency figures are marked illustrative by the author.
Assumes native tool-call APIs throughout; prompt-JSON fallback harnesses like tai's CLI backends are out of scope.

Relevance to tai

Directly settles the "no regex" question: loop decisions and intent detection must not rest on parsing assistant prose; tai's tool_calls-presence signal is the right control, matching this chapter's round-trip diagram.
Its failure-mode table is the template for tai's fix: a system-prompt nudge for not-calling-when-it-should, validation plus error-as-result for malformed calls, and harness caps (already present) against runaway retry.
The dynamic-discovery guidance validates tai's lazy tool loading (small active set, load on naming).

Changelog

Date	Change
2026-10-11	Initial dossier created.

3.8 KiB Raw Blame History