arXiv Machine Learning
Last Step Matters: Early Uncertainty Cannot Predict Failure in Long-Horizon Agents
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Oct 8
From Uncertainty to Action: Learning to Steer LLM Agents
arXiv:2610.09115v1 Announce Type: cross Abstract: Steering an LLM agent means deciding whether to correct it, at which step, and with which mechanism. Uncertainty is often used to decide when to corr...
arXiv Computation and Language
Sep 2
Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents
arXiv:2609.01294v1 Announce Type: new Abstract: Deep-research agents answer complex questions by interacting with search and browsing tools, yet they often search along a single evolving trajectory....
arXiv AI
Sep 11
Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations
The paper examines whether internal representations of agentic systems can better indicate task success than traditional confidence measures. It introduces two methods—Latent Trajectory Dynamics (LTD) and Action Representation Probe (ARP)—that analyze changes in residual-stream representations and action-level representations, respectively. Experiments on Bash, SQL, and Python benchmarks with Qwen and DeepSeek models show these methods outperform conventional surface-level and sequence-based calibration baselines, offering a zero‑overhead reliability monitor without prompt changes or multiple rollouts.
arXiv AI
Jul 16
When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents
arXiv:2602. 11619v2 Announce Type: replace Abstract: Running the same LLM agent on identical inputs yields 2.
arXiv AI
Oct 16
LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents
LEASH is a training‑free framework that enables uncertainty‑aware escalation from a weak to a strong computer‑use agent. It uses latent uncertainty signals from the weak agent’s hidden states to decide when to hand off control, and a shadow observer to maintain oversight of the strong black‑box agent. Experiments on OSWorld show LEASH improves task success over standalone weak agents and outperforms random escalation, achieving comparable performance to Claude Opus 5 while cutting inference cost by 19.2%.
arXiv AI
Oct 15
Can an Agent Learn to Predict Its Own Failure? Outcome-Grounded Confidence Calibration in LLM Agents
arXiv:2610.18614v1 Announce Type: new Abstract: Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human....