Files

T

retoor af080fe68e Strict goal verification, approval hardening, retrieval eval, work limits, sandbox caps (1.38.0)

- Approval: re-prompt unknown answers (max 3), shell-only affirmative
  guidance recovery; destructive actions stay strict
- GoalVerifier subagent with strict JSON verdicts, bounded follow-up
  rounds, nonzero exit plus open turn on unverified one-shots
- Retrieval eval (Hit@K/MRR/nDCG/Context Precision), FTS cascade with
  title weighting, query/read event logging, --eval-retrieval CLI
- Supervisor caps (8 parallel, 64 records) and scheduler hardening
  (4 fires/tick, per-schedule lease, stated misfire policy)
- Sandbox box resource trio: 2g/2.0/256 at create, best-effort
  update for existing boxes, rm -f recovery pointer
- Research archive: 12 subjects, 667 sources, 27 conclusions;
  suites 979 + 16 green

2026-10-18 12:13:24 +02:00

2.7 KiB

Raw Blame History

Reflection Agents

Metadata

Field	Value
Institute	Molodetz Institute — Research & Retrieval Division
Subject	agent-goal-verification
Document type	Source Dossier
Tier	1
Source title	Reflection Agents
Source URL	https://www.langchain.com/blog/reflection-agents
Source publisher / author	LangChain (Ankush Gola)
Source type	blog post
Source date (published)	2024-02-21
Date researched	2026-10-18
Researched by	tai (agent), model: Muse Spark, on behalf of retoor
Research method	both
Confidence	High
Status	Active

Summary

LangChain's reflection pattern composes a generator LLM with a reflector LLM that critiques as a teacher, looping a fixed number of times, optionally folding in tool observations as external information. It also flags reward-score sensitivity in reflection/search hybrids as a known reliability failure mode.

Key findings

"This simple example composes two LLM calls: a generator and a reflector. The generator tries to respond directly to the user's requests. The reflector is prompted to role play as a teacher and offer constructive criticism for the initial response."
"The loop proceeds a fixed number of times, and the final generated output is returned."
"Reflection is a prompting strategy used to improve the quality and success rate of agents and similar AI systems. It involves prompting an LLM to reflect on and critique its past actions, sometimes incorporating additional external information such as tool observations."
On LATS hybrids: "While it can be sensitive to the reward scores, the general algorithm can be flexibly applied to a variety of tasks."

Pros / strengths

Second independent industry source (with Anthropic) for the generator/critic loop with a fixed iteration cap, which is tai's verify loop shape.
Explicitly blesses tool observations as the external grounding for critique, matching tai's evidence-strict verifier prompt.

Cons / limitations / caveats

Vendor blog with linked example notebooks; states the pattern, not measured accuracy or cost data.
The reward-sensitivity warning applies to score-driven search hybrids more than to tai's boolean-plus-failures verdict, but it still cautions against trusting any single model-graded signal too far.

Relevance to tai

Independent confirmation that a separate critic call over the transcript plus tool evidence, bounded to a fixed number of rounds, is standard practice rather than an exotic control. tai's stricter verdict contract (boolean plus itemized failures with proposals) is a compatible tightening of the same loop.

Changelog

Date	Change
2026-10-18	Initial dossier created.

2.7 KiB Raw Blame History