The paper introduces the Agent Planning Benchmark (APB), a diagnostic framework designed to evaluate the planning capabilities of large language model (LLM) agents. APB contains 4,209 multimodal cases across 22 domains and five settings, testing holistic planning, step-wise planning with feedback, and robustness to extraneous or broken tools and unsolvable tasks. Evaluation on 12 MLLMs reveals systematic weaknesses in long-horizon planning, tool-noise robustness, calibrated refusal, and inference-time refinement, while APB-guided refinement improves plan correctness and downstream execution metrics on ToolSandbox and τ²‑bench tasks.
By Haoyu Sun, Wenxuan Wang, Mingyang Song, Jujie He, Weinan Zhang, Yang Liu, Yang Yang, Yu Cheng
SO-101 Bench is a simulated benchmark for language‑conditioned spatial manipulation on the low‑cost SO‑101 robotic arm, built using an Isaac Lab digital twin of a real setup. It features four tasks across five families, 138 objects with seen/unseen splits, 916 evaluation episodes, and 3,629 simulated demonstrations for policy fine‑tuning. The benchmark’s rankings mirror real‑world performance (Spearman ρ = 1.00) and reveals that while learned policies reach about 61% success, an LLM‑driven agent that writes code and delegates grasping achieves 83.2%.
By Truman Hickok
GarmentSegAgent is an autonomous framework for garment segmentation in web fashion images that combines semantic planning, spatial execution, and visual evaluation through iterative textual self‑refinement. Its core component, CAST (Constrained Attribution‑guided Selective TextGrad), links visual error attribution to targeted textual optimization, updating instruction components that cause segmentation errors while respecting garment‑specific constraints. Experiments on DeepFashion2, ModaNet, and Fashionpedia show consistent gains in region overlap and boundary accuracy, outperforming conventional TextGrad and enabling reliable inference‑time error correction without freezing foundation models.
By Jia Chen, Chao Huang, Xinrong Hu, Mingdi Huang, Ruyang Zhao, Jun Bai
The study investigates whether coding agents can reliably migrate PyTorch machine learning projects from GPUs to TPUs. Using Claude Code, ten projects were migrated, with eight meeting the project‑goal criterion but 76% of execution attempts failing, often due to compilation issues or silent changes in computation. The research highlights significant human effort (average 2.4 interventions) and variable costs, noting that while some TPUs can be faster and cheaper, failures and accuracy drops remain common.
By Hui Sun, Anderson Uch\^oa, Wesley K. G. Assun\c{c}\~ao
MCP‑Net is a real‑time multi‑task network designed for robotic coronary guidewire navigation. It jointly segments coronary vessels, localizes the guidewire tip, and identifies stenosis from a single fluoroscopic frame, then converts these predictions into a structured vascular graph with centerlines, bifurcations, endpoints, branch directions, and angles. Using simulated fluoroscopy derived from 3D CT angiography, the framework generates a roadmap from the graph root to the target lesion, enabling interpretable downstream robotic navigation.
By Sahar Nasirihaghighi, Tianyuan Wang, Kenan Niu, Stefano Stramigioli
The paper introduces The Trinity, a recursive self‑improvement framework for multimodal agents that couples world generation with harness evolution through three roles: Creator, Explorer, and Evolver. By shifting task organization from surface categories to shared capability requirements, it builds a hierarchical capability tree and a reusable harness tree, enabling efficient transfer learning across heterogeneous tasks. Experiments show the evolved harnesses improve mean performance by 34 points on 28 training benchmarks and 26 points on 18 unseen benchmarks, including two real‑world embodied tasks.
By Weidong Tang, Ruiheng Wang, Jingxuan Zhang, Hexin Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang, Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao Kaipeng Zhang, Pengfei Zhou, Odin Zhang, Yang You
OA‑RAG is an agent that combines formal ontology reasoning with large language model (LLM) reasoning to produce more faithful answers. For each response it lists the ontology observations used as evidence and assigns a grounding level—entailed, supported, related, or unrelated—to indicate the strength of that evidence. On benchmarks from three large ontologies and six real‑world medical QA tasks, OA‑RAG outperforms other ontology‑fine‑tuned and popular RAG methods, especially on reasoning‑intensive questions, and its grounding levels correlate with higher accuracy.
By Hui Yang, Jiaoyan Chen, Yongsheng Gao
The paper introduces LM-DSB, a lightweight multi‑agent system that automates the creation of daily safety bulletins (DSBs) for construction sites using a retrieval‑augmented generation approach with a locally deployable large language model. The system employs specialized planner, researcher, and writer agents coordinated in a hierarchical architecture, fine‑tuned on construction safety documents and grounded in 37 HSE documents. Validation in two phases showed that the multi‑agent RAG system outperformed alternatives in lexical alignment, coherence, and document completeness, and expert reviewers rated it highly for regulatory compliance and factual accuracy.
By Xianxiang Zhao, Advik Mehta, Falak Sethi, Brian Gue, Qipei Mei, Lingzi Wu
The study investigates whether large language model (LLM) agents can distinguish between user‑initiated corrections and manipulative pressure when deciding to concede. Across seven models from four vendors, researchers found that while citations increased willingness to concede, they also reduced discrimination between genuine corrections and manipulation. Additionally, concession on answers did not predict policy violations in a customer‑support scenario, indicating that mere concession rates are insufficient for evaluating agent reliability.
By Nardine Basta, Venus Haghighi, Dali Kaafar, Amin Beheshti
CogArena is a benchmark that evaluates multimodal AI agents on ten social and behavioral science experiments, requiring agents to interact via a web browser with the same visual stimuli and response interfaces used by human participants. The benchmark employs a three-level scoring system to assess task completion, accuracy, and the presence of characteristic behavioral signatures. Results from four multimodal models show that agents often fail to finish experiments and inconsistently reproduce human-like behavioral patterns.
By Kiant\'e Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C. Miceli, Ian Krajbich
The paper introduces ool, a plug‑in for kernel generation agents that enforces batch invariance (BI) in GPU kernels used by large language model serving systems. By combining static dependency analysis with dynamic bitwise consistency checks, ool rejects kernels that are structurally risky or non‑invariant before performance benchmarking. Experiments on three agent frameworks show that ool raises the BI pass rate from 67.6 % to 91.9 % while preserving 91.8 % of the baseline speedup.
By Youwei Bao, Chengpeng Wang, Simin Chen
GUI-JEPA introduces a decoder‑free policy for mobile GUI agents that uses a Joint‑Embedding Predictive Architecture (JEPA) to generate action candidates directly in latent space. By predicting a latent representation for each candidate action, the model scores all options in a single batched pass without token decoding, dramatically reducing computation. Experiments on AndroidControl show that GUI‑JEPA is 2.8–4.0× faster than a 7B‑scale VLM agent while retaining 86–92% of its accuracy with only 6% of the parameters.
By Bingchao Wang, Yongcheng Jing, Shunyu Liu, Chengyu Wang, Jun Huang, Jie Yang, Wei Liu, Dacheng Tao
OpenGray is a new benchmark that evaluates radiotherapy planning agents on their decision-making rather than just dose outcomes. It extends the OpenKBP-Opt head-and-neck dataset with tasks that have known correct decisions, including contradictions with dose goals and a feasible control plan. Four systems—ranging from a rule-based script to full LLM agents—were tested on 30 patients, revealing that high decision accuracy can mask specific failures such as missed hard goals and inadequate stopping actions.
By Humza Nusrat, Kundan Thind
The paper introduces FoG, a framework that trains knowledge graph (KG) agents to retrieve the evidence behind their answers. FoG combines evidence‑grounded synthesis, a KG interaction agent, and an evidence‑coverage process reward (ECPR) to ensure that every answer is supported by retrieved triples. Experiments on three KGQA benchmarks show that 3B–4B open‑source agents trained with FoG outperform larger baselines in both answer accuracy and evidence coverage, even with zero‑shot cross‑KG transfer.
By Zongyuan Sun, Shuai Zhao, Yingjun Du, Ke Shu, Ziyue Zhu, Haoran Luo
The study evaluates frontier large language models—Sol 6.1, Luna 6, and Qwen 3.8—in their ability to localize faults in LeetCode solutions without tools, with optional debugger assistance, or with encouraged debugger use. Across 610 seeded single‑line faults and five behavior‑preserving transformations, the agents achieved 98.2–99.9% accuracy on original code and 96.7–99.9% on transformed code when no tools were used. Encouraged debugger assistance further improved Luna 6 and Qwen 3.8’s accuracy by 61–63% of remaining errors, albeit at a 14.7–27.0× increase in token usage, while Sol 6.1 maintained 99.9% accuracy regardless of debugger use.
By Christoph Daxerer, Johann Glock, Christian Macho, Rudolf Ramler, Martin Pinzger, Josef Pichler
The paper investigates how large language model (LLM) agents lose information when cutting tool output before it reaches the model. In a real Linux‑kernel‑trace agent, up to one‑third of tool results were truncated, producing broken JSON and unevenly deleting data near the end of the output. Experiments show that this truncation increases computational effort and reduces accuracy, though it is unclear whether the loss directly causes the drop in correctness.
By Yuvraj Sehgal, Mahsa Panahandeh, Naser Ezzati-Jivan
HIDRA is a hierarchical dual-routing attention framework designed for replay‑free lifelong imitation learning in robotics. It separates instruction‑level expert retrieval from context‑dependent refinement and adds key‑level geometric regularization to keep tasks aligned yet distinct, thereby reducing routing interference. Experiments on LIBERO benchmarks show that HIDRA outperforms both replay‑free and replay‑based baselines, improving AUC, forward transfer, and maintaining low forgetting, especially under high interference and language variation.
By Fanqi Yu, Matteo Tiezzi, Cigdem Beyan, Tommaso Apicella, Vittorio Murino
PerceptRL enhances visual perception in multimodal large language models by employing reinforcement learning that transforms dense image captions into verified QA pairs with rule‑based rewards. It uses a multi‑agent pipeline to link correlated errors, combine unrelated errors, and target high‑error domains, while a specialized multi‑turn inspection process corrects spatial inaccuracies. Across seven benchmarks, PerceptRL improves average accuracy to 71.6%, outperforming both the base model with tools and the leading evaluated VLM by significant margins.
By Tongkun Guan, Zhibo Yang, Mingkun Yang, Haocheng Wang, Wei Shen, Xiaokang Yang
The paper investigates whether large language models can autonomously discover altered physical laws in simulated environments. Using the LongChang framework, 12 LLMs explore worlds with modified physics, manipulate objects, and build tools without external guidance. Only GPT‑6‑Astra achieved a 16.7% discovery rate, while the others found none, revealing that current models struggle to fully identify hidden laws.
EnvSched is an online scheduler that learns from rollout history to improve rollout efficiency in agent reinforcement learning. It assigns scheduling value by exploiting tasks likely to yield informative reward differences and by covering underobserved environments and interaction patterns. By modeling environment–task pairs in a dual-relation graph and propagating historical rollout statistics, EnvSched predicts reward variance and allocates batches to tasks with high predicted variance, achieving higher scores than random selection within fewer training steps.