APEX-Harm is a benchmark that tests prompt and script injection attacks on long‑horizon agents working in large, shared professional workspaces. The benchmark covers finance, consulting, and legal tasks, each involving 115–328 files and multiple tools such as email, calendar, chat, documents, spreadsheets, PDFs, and code execution. It evaluates six attack categories and measures attack success, exposure, and task performance across 12 frontier models, revealing that many agents can complete tasks while still executing unauthorized actions.
By Ziyi Yang, Adel Bibi, Guohao Li, Bernard Ghanem, Carlos Hinojosa
The paper introduces ARTIS, a framework that separates exploration from commitment by using iterative simulation to test large language model actions before real-world execution. This approach extends inference-time computation to boost action-level reliability and robustness while avoiding environmental risk. The authors also highlight that standard LLM simulators miss rare, high-impact failures, and propose a risk‑aware simulator that focuses on fidelity for failure‑inducing actions, showing significant reliability gains across agentic benchmarks.
By Xingshan Zeng, Lingzhi Wang, Weiwen Liu, Liangyou Li, Yasheng Wang, Lifeng Shang, Xin Jiang, Qun Liu
CaptchaAgent is a structured visual reasoning framework designed to solve modern reasoning CAPTCHAs. It constructs a visual intermediate representation that records entities, attributes, and relations, then refines ambiguous regions and verifies counterfactual consistency before generating actions. The approach yields significant performance gains over baseline browser agents on Open CaptchaWorld and the Oedipus benchmark, highlighting key challenges such as exact counting, geometry, and ordered execution.
By Shangquan Sun, Ruoyu Chen, Zhirui Liu, Xiang Liu, Chenghao Qian, Wenqi Ren, Xiaochun Cao
The paper presents a low‑cost, self‑supervised method for detecting wheel tracks on snow‑covered roads using synchronized RGB and long‑wave infrared (RGB‑I) imagery. Training labels are automatically generated by projecting future wheel trajectories from GNSS data, and a neural network is trained to identify wheel tracks on featureless, white‑out road surfaces. Experimental results show the approach is effective in visually monotonous adverse environments, providing a robust alternative for winter autonomy. The ThermalTrack dataset is released for further research.
By Yiming Yang, Jeremy P. Bos
The paper examines how agents in automated driving respond to perceptual uncertainty and how providing explicit uncertainty information affects their behavior. Experiments on a proxy driving task show that unreliable perception leads to defensive driving, while incorporating uncertainty data enables the agent to adapt, complete the task faster, and better manage risks.
By Natalie Grabowsky, Annika M\"utze, Joshua Wendland, Nils Jansen, Matthias Rottmann
SpireBench evaluates long‑horizon strategic play by having large language models play the full commercial deck‑builder Slay the Spire, a game where early choices determine long‑term success. Across 20 LLMs at the highest difficulty, frontier models beat heuristic baselines and match average human performance but lag far behind experts, especially as runs lengthen. The benchmark reveals that strong models excel in early fights but misjudge risk over a run, and that their performance depends partly on prior game knowledge rather than on memories written during play.
By Scott C. Lowe, Sander W. Keemink
EmbodiedCLBench is the first benchmark designed to evaluate continual learning in embodied simulation environments, focusing on self‑evolving agents. The benchmark uses compositional generalization to test whether agents can learn basic tasks and then recombine those skills to solve more complex, unseen tasks. Experiments across various agent harnesses and learning methods reveal that while basic task experience improves performance on advanced tasks, this benefit does not scale with more learning samples and current agents fail to extract additional useful information.
By Jie Huang, Yanan Chen, Ruixun Liu, Kaichen He, Xinyi He, Rui Huang, Zilong Zheng, Zhenliang Zhang, Yiwu Zhong
DRYLAB‑Bench is a new benchmark that measures the biosecurity risk of large language models (LLMs) when they design biological mutations. It presents 30 hazardous mutation‑design scenarios and evaluates LLMs in a multi‑round, tool‑augmented setting, scoring designs against 187,695 empirical wet‑lab measurements using a Biosecurity Risk Score. Initial tests on 15 frontier models show that even without tools, a majority of designs exceed average risk, and tool use further amplifies the threat.
By Yuzhen Wang, Chunxiao Li, Baichen Le, Lewen Yan, Jie Zhang, Yucheng T. Yang, Xing-Ming Zhao, Lijun Li, Jing Shao
The paper introduces DREAM, a training‑free, agent‑intrinsic memory repair framework that selectively forgets poisoned memories in large language model agents. DREAM uses perturbation‑induced reactivation, anomaly detection on activation patterns, and dynamic suppression of harmful memories while reinforcing benign ones. Experiments show DREAM cuts backdoor attack success by over 95% and improves runtime and token efficiency compared to existing defenses.
By Lei Liu, Yunji Liang, Xiaowen Zhang, Jingqi Liu, Qi Li, Bin Guo, Zhiwen Yu
Branching On-Policy Distillation (BranchOPD) enhances on‑policy distillation by selecting informative decision points based on student uncertainty and teacher‑student disagreement, resampling continuations from shared prefixes, and applying token‑level teacher supervision to both main trajectories and branches. This approach reduces student‑generated token costs by about 19% on ALFWorld and 31% on WebShop compared to seven independent trajectories, and introduces protocol‑boundary repair (PBR) to mitigate repetition under protocol constraints. BranchOPD outperforms Vanilla OPD and Multi‑Rollout OPD across benchmarks, achieving significant gains in mathematics accuracy and ALFWorld success rates, with mechanistic analyses showing that branching at uncertain decisions expands behavior coverage.
By Dongyang Chen, Shudong Liu, Shubo Li, Duoxun Tang, Jinwei Liang, Zehuan Wu, Lewei Lu
The study evaluates 26,999 pull requests that modify both application code and deployment infrastructure, using two large datasets and six AI agent architectures. It finds that such dual‑scope changes occur in only about 4.5% of agent‑generated PRs but lead to two distinct integration outcomes: either significant delays due to CI/CD and provisioning checks, or a high rate of silent merges with reduced human review. The results highlight that current AI agents lack deployment awareness, causing unverified dependencies to slip into production pipelines.
By Mohammad Wathiq Soualhi, Basem Suleiman
The paper introduces ReqClarifyBench, an interactive benchmark for evaluating how well large language models (LLMs) can clarify vague user intent into clear requirements in repository-level software development. It includes 341 tasks across game, miniapp, and web development, each with a vague intent, a human-validated requirement document, and code evaluation artifacts, and uses LLM-based user simulators with diverse personas. The study evaluates six LLMs on clarification effectiveness and requirement quality, finding that stronger clarification does not always yield better documents, performance varies by persona, and coding-agent harnesses do not improve document quality over base LLMs.
By Mingyu Chen, Zhiqing Zhong, Manyi Wang, Linxi Liang, Rouyu Chen, Pinjia He
Matched-Context Group Policy Optimization (MCGPO) is a method for training long-horizon language model agents by constructing weighted action preferences from pre-collected trajectories. It defines comparison contexts using pre-action state summaries and rubric progress, and employs two-action support with exact and coarse matching rules to weight evidence for learning. Experiments across six environment-backbone settings show that MCGPO improves mean success rates over matched GraphGPO, achieving up to 5.16 percentage point gains on WebShop with Qwen3-4B and 8B models.
By Chenxu Zhang, Yaorui Shi, Wenlong Zhang, Jianchao Tan, Zihan Wang, Yiming Cheng
MC‑EvoBench is a controlled Minecraft benchmark designed to audit experience‑driven self‑improvement in agents. It features 19 evolution chains (12 primary, 7 calibration/probe) and three graph‑aligned probe families, each task represented as a task‑state graph with stage‑level verifiers and strict source‑target boundaries. The benchmark evaluates probes under no‑prior, related, unrelated, and shuffled source conditions, reporting metrics such as TSR, WGP, and VMP while preserving provenance records.
By Jingwei Song, Zhisheng Chen, Bingfan Zeng, Jing Tang, Yubo Hou, Jinhao Jing, Zhengwei Xie, Linfeng Zhang
The paper introduces the "convention gap" as a measure of implicit communication in human‑AI cooperation, using the card game Hanabi as a testbed. It defines the gap as the difference between failure probability predicted from literal hints and the actual observed failure rate, proving that a zero gap indicates no extra information beyond literal content. Empirical results show large positive gaps in human‑human and human‑AI games, negligible gaps in AI‑AI simulations, and a monotonic increase in the gap with higher levels of Off‑Belief Learning agents, underscoring the importance of reporting this metric when evaluating agents for human partners.
By Makoto Fukushima, Hua-Dong Xiong, Ehsan Moradi Pari
The paper introduces MAGI, an open modular agent that orchestrates molecular‑design tools by setting objectives, launching optimization, interpreting structure–activity relationships, and adjusting strategy. In nine retrospective lead‑optimization campaigns from three pharmaceutical companies, MAGI produced valid molecules via either direct LLM generation or delegation to REINVENT 4; LLM proposals stayed closer to local chemistry and achieved comparable or higher activity with fewer operations, while REINVENT explored broader chemical space. The success of a campaign depended on the accuracy of predictive models rather than the generation route, and a blinded evaluation showed chemists could not distinguish MAGI’s outputs from expert‑generated compounds, though the SAR reasoning was deemed plausible yet incomplete. The study positions MAGI as a coordination layer that can be plugged into existing workflows, but notes that the current ceiling for real projects is set by scorer applicability rather than tool orchestration.
By Pierre Llompart, Levent Guner, Helen Lai, Alessandro Tibo, Yijie Xu
The paper introduces a preference-conditioned multi-objective reinforcement learning framework for tactical decision making in heavy-duty vehicles. It explicitly models safety, energy efficiency, and time efficiency as separate objectives and uses Generalized Policy Improvement and Proximal Policy Optimization to learn policies that approximate the Pareto front across these conflicting goals. Experiments in a scalable highway traffic simulation show smooth, interpretable Pareto fronts and meaningful trade-offs, with the policy enabling behavior transitions without retraining.
By Deepthi Pathare, Leo Laine, Morteza Haghir Chehreghani
The paper introduces a Byzantine‑resilient federated large language model (FedLLM) that uses a Hypergraph Message Passing Graph Autoencoder (HMP‑GAE) to prevent semantically corrupted updates from spreading during aggregation. By representing low‑rank model updates as hypergraph nodes and iteratively passing messages, HMP‑GAE jointly maximizes global accuracy and suppresses malicious updates without accessing private data. Experiments on AG News and Yahoo! Answers with Qwen‑2.5 and Llama‑3.2 backbones show significant gains: 1.21% higher accuracy, 44.08% lower semantic entropy, and 13.34% reduced perplexity compared to existing defenses.
By Kai Li, Hanlin Cai, Zihao Liu, Yichen Li, Jose Alcaraz Calero, Wei Ni, Falko Dressler, \"Ozg\"ur B. Akan
The paper introduces Personalized Federated Direct Preference Optimization (pFedDPO), which extends FedDPO by allowing each client to keep a small private LoRA adapter on top of a shared global base model. In pFedDPO, the base model is updated globally via FedAvg while the LoRA adapters remain local, capturing client‑specific preference patterns. The authors prove that this approach reduces the effective gradient diversity compared to vanilla FedDPO and demonstrate experimentally that pFedDPO achieves lower gradient norms while maintaining competitive training loss and per‑client reward accuracy.
By Zhanhong Jiang, Soumik Sarkar
CB-Orchestrator is an adaptive framework for optimizing large language model (LLM) agent workflows. It separates workflow generation—via evolutionary search—from workflow selection, which is treated as a contextual bandit problem to enable sample‑efficient learning. The approach reduces training‑phase API token usage by up to 40.55% and overall token overhead by at least 29.73%, while outperforming baselines on five benchmarks, including the MATH dataset.
By Jiahang Sun, Zhiwei Shang, Zhipiao Liu, Hongwei Yang, Guoqing Xie, Shuang Qiu, Zhongxiang Dai