arXiv Computation and Language
When Is Enough Not Enough? Illusory Completion in Search Agents
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Machine Learning
Sep 17
Locating Hidden Failures Makes Long-Horizon Agents More Reliable
The paper introduces Traverse, a benchmark of 2,518 agent trajectories and 6,967 annotated mistakes across software engineering, computer use, and science tasks, revealing that failures often go unrecovered and can cause irreversible harm before a run is deemed successful. It shows that human judges struggle to detect the first mistake in most runs, while a 4‑billion‑parameter verifier called Scout can locate failures more effectively and improve task success when used to select among candidate runs. The study demonstrates that making failure detection inexpensive and reliable can enable long‑horizon agents to learn from their own mistakes and increase trustworthiness in autonomous AI.
arXiv AI
Jul 15
Track, Rank, Crack: Epistemic Working Memory Scales Multi-Hop Reasoning in Language Agents
arXiv:2607. 12267v1 Announce Type: cross Abstract: Language agents that interleave reasoning and tool use degrade sharply as reasoning chains lengthen, even when each individual step is easy.
arXiv Computation and Language
Sep 23
CoVeR: Coverage-Based Routing of Verifier Calls in Agentic Retrieval
arXiv:2609.26086v1 Announce Type: new Abstract: An agentic retrieval system issues a sequence of search queries and must decide, at each step, whether the evidence collected so far is enough to stop....
arXiv AI
Oct 15
Evidence-First Agentic Verification
arXiv:2610.17724v1 Announce Type: new Abstract: Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across t...
arXiv AI
Jun 2
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
arXiv:2606. 02060v1 Announce Type: new Abstract: Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis.
arXiv AI
Oct 2
Verify Claims, Not Scores: Evidence-Based Verification of Modular Agents
The paper proposes a claim‑specific verification audit for modular agents that replaces aggregate task scores with evidence‑based evaluations. Each agent conclusion is recorded with supporting evidence and classified as supported, unsupported, unresolved, or not evaluated, along with the boundary of validity. The audit employs three tools—oracle policies, perfect component replacements, and verifier‑score tests—to trace value changes, locate lost value, and assess verifier effectiveness, demonstrated on a portfolio‑allocation agent in a synthetic market.