RSIHub is an open‑source, composable framework designed to streamline the development, comparison, and extension of agent‑evolution methods. It introduces reusable recipes and operators that manage mutable resources, while providing execution, evaluation, recovery, and lineage tracking. The framework was used to implement and evaluate several methods—AHE, Hyperagents, A‑Evolve, and GEPA—across tasks like MiniSWE, Codex on Terminal Bench 2, and τ³ Banking, achieving up to a 13.5 percentage‑point improvement in overall pass@1 scores.
By Zimu Wang, Xiaobo Wang, Siyu Ye, Jiaqi Li, Silin Chen, Yuling Shi, Ruobing Wang, Jingjing Zhang, Chaofan Wang, Kai Wu, Mengnan Qi, Kai Cai, Xiaodong Gu, Zilong Zheng
The paper proposes a deep reinforcement learning framework for stock trading that incorporates sentiment analysis into actor‑critic agents. It compares pure policy‑optimization methods (A2C, PPO) with hybrid approaches that combine policy optimization and Q‑learning (DDPG, TD3, SAC). Results show that the hybrid agents consistently outperform others in profit maximization and risk‑averse trading when sentiment is deeply infused.
By Hrishikesh Singh, Aarti Sharma, Millie Pant
The paper introduces TraceTrial-Bench, a benchmark that evaluates the reliability of AI agent harnesses across seven components—context, execution, governance, lifecycle, observability, tools, and verification—using 95 expert-level tasks with executable or state-based ground truth. It employs a Trajectory Tracing module to record agent execution, a two-layer evaluation framework to separately assess agent performance and harness reliability, and a Multi-source Evidence Acquisition module to gather evidence for LLM-based evaluation. The benchmark’s analysis of 3,420 runs across four harnesses and nine models shows harness scores between 59.8% and 72.0%, with 31.1% of runs succeeding despite failing at least one harness criterion, highlighting the need for auditable and reproducible harness evaluation.
By Zhiyong Cao, Hengjie Zhu, Yongxin Wang, Xuchen Li, Jintao Li, Shuxia Liu, Dunqiang Liu, Bo Wang, Kairan Zhang, Cong Lin, JiangDi Yu, Siyuan Liu, Sijia Yao, Yi-Liang Wei, Lingxu Ma, Aijia Wang, Anxin Chen, Zixian Wei, Xiang Ruan, Ruizhi Xu, Ruqian Shi, Baihua Xiao, Sicheng Zhou
The paper introduces a method for measuring when large language model agents have internalized skills from public skill hubs, allowing those skills to be retired for specific models. It critiques existing gain‑based rules, showing they fail to distinguish truly internalized skills from uncovered ones, and proposes an unaided‑score approach that better identifies internalization. The authors release 1,251 skills with associated tasks and scores across four models to support this evaluation framework.
By Yanze Wang, Pengfei Yao, Tianyi Sun, Hongda Chen, Yunyun Han, Jun Sun, Chuanrui Hu, Yafeng Deng
LeafSkill introduces an external reuse layer for recursive LLM agents that retrieves semantically related action grounding experiences, binds action templates to current objects, and uses a lightweight 1.7B selector to choose or defer actions. This approach preserves high coverage of candidate actions under paraphrases (96–97%) and achieves a 97.34% adoption rate of unseen skill types. In experiments on Robotouille tasks, LeafSkill outperforms ReCAP, completing more tasks and reducing token usage and calls by over 14%.
By Shen Liu, Ze Zhou, Zhenguo Xu, Shaopu Wang, Yike Gao, Chunlei Wang
arXiv:2610. 32380v1 Announce Type: new Abstract: Large language models (LLMs) can read startup pitch decks, but letting them emit investment scores directly invites score inflation, run-to-run variance and rationales that are hard to audit.
By Ming Ze Tang, David Cutting, Boyd Carson, Vasiliki Carson
Async-RBench is a new benchmark that tests how large language model agents handle asynchronous task execution. It includes 200 executable tasks across eight scenarios, with a scenario controller that delivers results based on task-state conditions and an Event Response Contract that defines required changes, preservation, avoidance, and re-verification. The benchmark introduces the Dynamic Replanning Score (DRS) to evaluate both the response process and outcome, and finds that agents generally perform worse with asynchronous delivery compared to batched delivery, highlighting challenges in reliable asynchronous task handling.
By Detao Zeng, Guilin Ou, Nuo Chen, Yunhao Feng, Hongliang Wang, Qinli Yang, Junming Shao, Zhong Zhang
The paper introduces a framework for evaluating large‑language‑model agents by focusing on the interaction between the model and its execution harness. It defines five complementary metrics—Valid Completion Rate, Model‑Harness Fit, Cross‑Model Fit, Interaction Residuals, and Token Load—to assess how reliably agents finish tasks, how well harnesses support specific models, and the overall suitability across models. Using these metrics, the authors evaluate four general‑purpose harnesses and two native systems on six benchmarks, finding that harness rankings differ by model and task and that some harnesses consistently achieve high valid completion rates.
By Shao Zhang, Hangfan Zhang, Yuanyuan Shi, Hao Li, Min-Le Su, Kangcong Li, Jingping Wang, Chunjiang Mu, Chen Zhang, Yang Chen, Bo Zhang, Shuyue Hu
The paper introduces CASE, a Confusion-Aware Skill Evolution framework designed to improve Multimodal Emotion Recognition (MER) by addressing systematic confusion between semantically adjacent emotions. CASE builds a sparse, self‑evolving library of pair‑specific EmotionSkills that externalize affective expertise from frozen multimodal large language model (MLLM) backbones, allowing continuous refinement and transfer across different models. Experiments on CA‑MER, DFEW, and EMER benchmarks show that CASE significantly boosts accuracy, notably raising Qwen3‑VL‑8B’s performance from 46.80% to 59.13%.
By Siyu Wu, Zeyu Liang, Linghao Wang, Qiyun Zheng, Yi Yao, Ling Lo, Hongxia Xie
IGG is a new benchmark for Interactive GUI Grounding that requires agents to first interact with a GUI to make a target visible and then click it. It includes 601 tasks across 16 controllable environments and seven visibility-constraint sub‑types, with a compact action space of scrolling, moving, dragging, waiting, and clicking. Human participants achieve 94.6% accuracy, while GPT‑5.5 and the best open‑source VLM reach 79.4% and 31.6% respectively, highlighting a gap mainly due to the difficulty of executing reveal interactions.
By Kyeongseon Kim, Jiyeon Son, Tae-Hyun Oh
arXiv:2610. 32752v1 Announce Type: new Abstract: LLM agents need long-term memory that can recover precise and mutually consistent evidence from growing interaction histories, particularly when answers depend on relations among facts rather than isolated semantic matches.
By Navyansh Mahla, Sashank Vandrangi
Astro2Method is a benchmark comprising 500 expert-reviewed binary questions drawn from 346 astronomy papers, designed to evaluate large language models (LLMs) on selecting machine learning methods for astronomical problems. The study tests five LLMs, finding high accuracy (87.50‑92.55%) on Easy questions but lower performance (58.87‑64.52%) on Hard ones. Various retrieval-augmented and agent-based approaches, including web search and masked pipeline descriptions, show limited or inconsistent improvements, indicating that LLMs still struggle to infer suitable ML methods from related studies and domain knowledge.
By Xingzhong Fan, Hangfeng He, Fatemeh Nargesian
RubricCraft is a method that generates natural‑language evaluation rubrics for open‑ended, artifact‑producing language tasks by analyzing differences among independently sampled solver trajectories. It constructs rubric criteria based on how each attempt satisfies or violates task requirements, then applies and refines these criteria to better detect errors and recognize partial correctness without needing a single reference solution. On the SpreadsheetBench v2 benchmark, RubricCraft’s rubrics reduce the proportion of incorrect outputs that pass (29.5% vs. 42.6%) and improve ranking of outputs while maintaining consistency with the benchmark’s own checker.
By Hosein Hasanbeig, Priyanshu Gupta, Ashish Tiwari, Vu Le, Sumit Gulwani
MBABench-2.0 is a new benchmark that tests AI agents on building financial spreadsheets from scratch, featuring 101 original construction tasks that require complex modeling decisions, formatting, and layout design. Unlike previous fill‑in‑the‑blank tests, this benchmark evaluates spreadsheets on 129 quality criteria across 12 categories, addressing readability, maintainability, and auditability. Across 20 agents and 8 models, the best performer achieved only a 16% pass rate, highlighting a significant gap in current AI spreadsheet‑building capabilities.
By Julian Poeltl, Thomson Yen, Patrick Shea, Shubham Anil Varma, Dhruv Rastogi, Zihao Chen, Mengshi Zhang, Hongseok Namkoong
The paper investigates how curriculum learning affects reinforcement learning in a minimal policy learning model. It shows that when rewards are dense, curricula offer only marginal benefits, but when rewards are sparse, curricula become essential for learning to occur. The study also highlights that competing objectives—minimizing effort versus maximizing reward—create distinct optima that curricula can help navigate by reducing reward sparsity and reshaping the reward landscape.
By John J. Vastola, Ann Huang, Satpreet H. Singh, Samuel J. Gershman, Kanaka Rajan
The paper introduces the evidence-action gap, where coding agents persist in tasks even when completion is impossible, despite having sufficient evidence that outside help is needed. To quantify this, the authors present ImpasseBench, a benchmark that injects realistic blockers into repair tasks and marks the step at which evidence becomes sufficient. Across eight models and 19,371 trajectories, they find that many agents still exhaust their budget or make unsafe requests after the evidence threshold, but an escalation tool and external monitor can significantly reduce these failures.
By Tianyu Chen, Xinran Zhao, Vijay Viswanathan, Shoubin Yu, Mingyuan Zhou, Tongshuang Wu
The paper introduces a defense for Agentic AI systems that use the Model Context Protocol (MCP). It identifies a vulnerability where hidden instructions in steganographic images can bypass text scanners and trigger malicious tool calls. The proposed solution, decode-time provenance binding, uses HMAC-authenticated trust labels, a provenance ledger, and context-taint tracking to mark decoded content as untrusted and block harmful tool calls, achieving 0% attack success in tests.
By Eric Yocam, Varghese Vaidyan, Paris Kalathas, Patrick Tague, Bruce DeBruhl, Yong Wang
ImageRSI is a framework that improves agentic image generation by keeping the generator frozen and integrating three components: a five‑dimensional verification and repair process, an experience library shared across tasks, and offline updating of the judge. Candidate experiences are added to the library only after paired A/B validation, and the judge is refined through supervised fine‑tuning and GRPO on accumulated image‑judgment records. The authors also introduce ImageReq‑Bench, a benchmark of 961 hard real‑world image‑generation requests, and report that ImageRSI achieves 52.6% Checklist Strict Accuracy on MindBench and 65.66% on ImageReq‑Bench.
By Shuo Han, Zhilin Wang, Peng Ling, Wenjia Gong, Yang Liu, Guanjun Jiang, Yukun Cao, Xike Xie
The LLENS platform is a clinical support AI system deployed inside Hokkaido University Hospital’s closed network, running the open‑weight model Kimi K2.6 on a single host with eight NVIDIA H200 GPUs. It includes ten harness components—such as page image reading, full‑text attachment input, a sandbox, and self‑checking—allowing the model to read and interact with hospital documents. In evaluation, the system achieved 97.2 % accuracy on 180 document‑based questions when using interactive access, outperforming fixed retrieval by 15.9 percentage points, and has been operational since September 14, 2026.
By Ken Enda, Jien Saito, Masami Mukai, Feng Han, Emir Turkes, Tatsumi Kusakabe, Reona Umeki, Kazuhiro Sumiyoshi, Mitsuya Numata, Takumi Tanikawa, Akira Endoh, Kenji Hirata, Atsushi Yoshida, Kohsuke Kudo, Shinya Tanaka
SOPO: Socratic Guided Policy Optimization for Span-Level Hallucination Detection introduces a multi-turn reinforcement learning framework that uses a teacher model to provide Socratic feedback, guiding reasoning trajectories during rollout. This approach addresses localization ambiguity, reward sparsity, and verbosity bias in detecting hallucinations in large language models. Empirical results on the RAGTruth dataset show that SOPO (4B) outperforms leading proprietary models and standard RL baselines, achieving state‑of‑the‑art performance on multiple benchmarks.
By Jiawei Dong, Zilong Bai, Pengtian zhu, Peng Zhou