← Back to all news
arXiv AI By Yuanjian Xu, Jinyi Han, Jianing Hao, Yuxuan Sun, Fanyang Lu, Richeng Xuan, Guang Zhang, Zhichao Hu

AgentPerturb: Evaluating LLM Agent Robustness under Controlled Interaction Perturbations

Read the original on arXiv AI →

AgentPerturb introduces a systematic framework to evaluate large language model agents under controlled interaction perturbations, addressing the lack of unified evaluation methods for noisy real‑world tasks. The study quantifies robustness across frontier models, identifies task‑specific vulnerabilities, and shows that agents often fail to detect altered inputs, propagating errors and incurring higher computational costs. The authors provide a benchmark‑construction pipeline and plan to release code and data to support future robustness‑oriented training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 16

LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents

LEASH is a training‑free framework that enables uncertainty‑aware escalation from a weak to a strong computer‑use agent. It uses latent uncertainty signals from the weak agent’s hidden states to decide when to hand off control, and a shadow observer to maintain oversight of the strong black‑box agent. Experiments on OSWorld show LEASH improves task success over standalone weak agents and outperforms random escalation, achieving comparable performance to Claude Opus 5 while cutting inference cost by 19.2%.

By Qingni Wang, Changdae Oh, Yuzhe Yang, Sharon Li, Jiachen Yang, Ang Li, Xin Eric Wang
llmsagents
More like this →
arXiv AI
Jul 2

Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use

arXiv:2607. 01084v1 Announce Type: new Abstract: While Large Language Model (LLM) agents demonstrate proficiency in static benchmarks, their deployment in real-world scenarios is hindered by the dynamic nature of user queries, tool sets, and interaction dynamics.

By Song-Lin Lv, Weiming Wu, Rui Zhu, Zi-Jian Cheng, Lan-Zhe Guo
llmsagentsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv AI
Sep 25

Understanding and Exploiting Initialization Anchoring Weakness in Feedback-Based Agent Planning

The paper investigates a vulnerability in feedback‑based agent planning, showing that the first round of feedback corrects a large portion of adversarial directions (46%) while subsequent rounds see a sharp decline (13% and 7%). The authors attribute this to an initialization anchoring weakness driven by plausible plan shifts, lack of counterevidence, and persistence of accepted directions. They introduce “InitAnchor”, a black‑box attack framework that exploits these factors, achieving high attack success rates across diverse tasks, architectures, and LLMs, and remaining effective against multiple defenses and real‑world agents.

By Chuanchao Zang, Jianing Wang, Wenyu Chen, Xiangtao Meng, Li Wang, Xinyu Gao, Peng Zhan, Zheng Li, Shanqing Guo
llmsagentssafety
More like this →