AgentPerturb introduces a systematic framework to evaluate large language model agents under controlled interaction perturbations, addressing the lack of unified evaluation methods for noisy real‑world tasks. The study quantifies robustness across frontier models, identifies task‑specific vulnerabilities, and shows that agents often fail to detect altered inputs, propagating errors and incurring higher computational costs. The authors provide a benchmark‑construction pipeline and plan to release code and data to support future robustness‑oriented training.
By Yuanjian Xu, Jinyi Han, Jianing Hao, Yuxuan Sun, Fanyang Lu, Richeng Xuan, Guang Zhang, Zhichao Hu
The paper introduces ARTIS, a framework that separates exploration from commitment by using iterative simulation to test large language model actions before real-world execution. This approach extends inference-time computation to boost action-level reliability and robustness while avoiding environmental risk. The authors also highlight that standard LLM simulators miss rare, high-impact failures, and propose a risk‑aware simulator that focuses on fidelity for failure‑inducing actions, showing significant reliability gains across agentic benchmarks.
By Xingshan Zeng, Lingzhi Wang, Weiwen Liu, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu
arXiv:2608.21423v1 Announce Type: cross
Abstract: Agentic security uses large-language-model (LLM) agents to plan, dispatch, and interpret security tools. As these systems move from demonstrations to...
By Israt Moyeen Noumi, Tarannum Ahmed Nowshin, Md. Mehedi Hasan Nipu, Mohammad Sakib Mahmood, Md. Jakir Hossain, M. F. Mridha
The paper examines whether internal representations of agentic systems can better indicate task success than traditional confidence measures. It introduces two methods—Latent Trajectory Dynamics (LTD) and Action Representation Probe (ARP)—that analyze changes in residual-stream representations and action-level representations, respectively. Experiments on Bash, SQL, and Python benchmarks with Qwen and DeepSeek models show these methods outperform conventional surface-level and sequence-based calibration baselines, offering a zero‑overhead reliability monitor without prompt changes or multiple rollouts.
By Priyanka Mary Mammen, Emil Joswin, Srujananjali Medicherla
LEAP: Learning Efficient Action Proposals For LLM Agents proposes a method to speed up large language model agents by training a small 0.6B drafter to predict target actions accurately. The approach uses a latency framework that balances drafting, verification, and execution costs, achieving up to 60% faster end‑to‑end wall clock time without reducing task success. LEAP can be online trained, eliminating the need for prior trace collection and making it practical for real‑world deployment.
By Zhen Xu, Qizheng Zhang, Gerry Wan, Shang Zhu, Ce Zhang
arXiv:2608. 06346v1 Announce Type: new Abstract: LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging.
By Yunjia Qi, Zehua Yin, Xintong Shi, Hao Peng, Songyuanyi Lu, Yixian Liu, Richeng Xuan, Yuhong Liu, Zhichao Hu, Xiaozhi Wang, Lei Hou, Bin Xu, Juanzi Li
Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable. We show that failure is predictable early from the agent's internal representations: lightweight per-round probes on hidden activations anticipate eventual episode failure as early as the first interaction round, where scorers reading only the agent's observable behavior are barely better than chance.
arXiv:2605. 06890v3 Announce Type: replace Abstract: AI agents are promising for high-stakes enterprise workflows, but dependable deployment remains limited because tool-use failures are difficult to diagnose and control.
By Hariom Tatsat, Ariye Shater
LEASH is a training‑free framework that enables uncertainty‑aware escalation from a weak to a strong computer‑use agent. It uses latent uncertainty signals from the weak agent’s hidden states to decide when to hand off control, and a shadow observer to maintain oversight of the strong black‑box agent. Experiments on OSWorld show LEASH improves task success over standalone weak agents and outperforms random escalation, achieving comparable performance to Claude Opus 5 while cutting inference cost by 19.2%.
By Qingni Wang, Changdae Oh, Yuzhe Yang, Sharon Li, Jiachen Yang, Ang Li, Xin Eric Wang
The paper introduces TACT (Think-Act Calibration via activation Steering), a method to detect and mitigate agent drift—specifically overthinking and overacting—in language model agents performing complex software engineering tasks. By labeling trajectory steps and identifying two linear drift axes in the residual stream, TACT projects activations back toward calibrated behavior, achieving higher resolve rates and fewer steps-to-resolve on benchmarks such as SWE-bench Verified, Terminal-Bench 2.0, and CLAW-Eval.
By Yuan Sui, Yulin Chen, Yibo Li, Xue Jiang, Yufei He, Yihong Dong, Xiaoxin He, Tianyu Gao, Bryan Hooi
arXiv:2607. 10059v1 Announce Type: new Abstract: Agent systems based on large language models (LLMs) are increasingly deployed for autonomous tasks, yet existing evaluations mostly focus on task success rather than whether agents know when to abstain.
By Xun Liu, Yi Evie Zhang, Vira Kasprova, Parisa Rabbani, Pardis Sadat Zahraei, Tianyu Zhang, Ali Ebrahimpour-Boroojeny, Varun Chandrasekaran
arXiv:2607. 03333v1 Announce Type: cross Abstract: LLM agents are becoming a common interface for research, coding, and question answering, yet their Thought-Action-Observation loop is often serial: the model reasons, emits a tool call, then idles the GPU until the result returns.
By Huajun Bai, Weiwei Lv, Huichuan Zheng, Youyou Lu, Jiwu Shu