MedBenchAgent is a multi‑agent framework that automates the entire process of constructing medical vision‑language model benchmarks, from deriving the benchmark specification to generating and auditing evaluation items. It uses a Benchmark Intermediate Representation (BIR) to encode task definitions, evidence mappings, evaluation protocols, and item specifications, separating planning from instantiation. The system achieves a Task‑Space F1 of 90.9% and demonstrates high audit pass rates, outperforming existing task induction methods and showing portability across specialized medical domains.
By Yulin Fu (Beijing University of Posts and Telecommunications), Junren Wang (West China Hospital, Sichuan Provincial Engineering Research Center of Intelligent Diagnosis and Treatment of Breast Diseases), Guangjing Yang (Beijing University of Posts and Telecommunications), Zhangyuan Yu (Beijing University of Posts and Telecommunications), Wanran Sun (Beijing University of Posts and Telecommunications), Jiabao Zhou (Beijing University of Posts and Telecommunications), Jin Yin (West China Hospital, Sichuan Provincial Engineering Research Center of Intelligent Diagnosis and Treatment of Breast Diseases), Qicheng Lao (Beijing University of Posts and Telecommunications)
RIT‑RAG is a retrieval‑augmented generation framework that combines content retrieval with structural navigation of documents. It builds a tree for each document from its table of contents or sitemap, then uses retrieved chunk positions to induce manageable sub‑trees that an LLM agent can navigate, read selectively, and reformulate queries. Experiments on financial, scientific, and customer‑support benchmarks show RIT‑RAG outperforms vanilla, graph‑based, and agentic baselines, achieving up to 11.4‑point accuracy gains on the new EntQABench of 2.84 million technical‑documentation webpages.
By Meghanadh Pulivarthi, Swaraj Kumar Biswal, Kushagra Bhushan, Yatin Nandwani, Sachindra Joshi, Dinesh Raghu
The paper introduces Multi-Agent Self-Supervision (MASS), a recursive self‑improvement technique that alternates between evolutionary workflow optimization and supervised fine‑tuning on self‑generated multi‑agent trajectories. MASS enables a single base model to iteratively propose, execute, and self‑evaluate complex multi‑agent workflows, discovering effective roles and information routing. Experiments with Qwen3.6‑27B show 1.2–1.6× higher performance per output token on four open‑ended benchmarks, and student models trained on MASS traces outperform single‑agent students trained on 1.4× more tokens.
By Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Somayeh Sojoudi, Matei Zaharia, Yujin Tang
The paper introduces Q-Shaped Options (QSO), a hierarchical reinforcement learning framework that learns action abstractions between consecutive levels using shared encoders shaped by Q functions. QSO employs distinct state-value functions, Q functions, and policies at each hierarchy level, encouraging the low-level Q function to preserve distinctions needed for optimal control while the high-level Q function discards unnecessary distinctions. Experiments on offline goal-conditioned locomotion and manipulation tasks show that QSO learns semantically meaningful option spaces and outperforms baselines, achieving non-zero performance where other algorithms fail.
By Clarisse Wibault, Antoine Gorceix, Antonio L\'eon Villares, Alexey Zakharov, Evangelos Chatzaroulas, Michael Matthews, Eduardo Pignatelli, Jakob Foerster
The paper presents a constrained command‑conditioned Proximal Policy Optimization (PPO) executor for the MicroRTS real‑time strategy environment, enabling a single execution policy to follow diverse strategic commands. A Thompson‑sampling bandit strategist selects command tuples based on in‑game observations of the opponent’s strategy, rather than relying on opponent identity. In controlled experiments, the strategist‑executor system outperforms a flat PPO baseline against three of the four strongest opponents on a training map, including two held‑out opponents, while showing no significant difference against the remaining opponents.
By Nick Leenders, Roy Lindelauf, Joost van Oijen, Boris Cule
DeltaReplay is a step‑level memory reuse framework for mobile GUI agents that decides how to use stored execution trajectories without modifying them. It stores trajectories as paths in a transition graph, separating each action into a task‑independent operation and task‑specific parameters, and compares each recorded step with the new task and current screen to decide whether to follow, adapt, or ignore it. Experiments on AndroidWorld and SPA‑Bench show that DeltaReplay can increase task success rates by up to 10.3 and 25.0 percentage points over a base agent with the same backbone.
By Yudong Bai, Yihong Chen, Quanming Yao, Yaqing Wang
Agent4RE is a self‑refining multi‑agent framework designed for end‑to‑end software requirements engineering (RE). It coordinates specialized agents and includes two iterative improvement loops, and introduces a new real‑world dataset, RE‑E2E, for evaluating RE workflows from elicitation to generation. The framework, along with two enhanced variants that use autonomous self‑refinement or structured human feedback, consistently outperforms baseline prompting methods across eight large language models, achieving higher text‑based metrics and superior human and LLM‑as‑a‑judge ratings.
By Yongjian Tang, Linhan Li, Thomas Runkler
The paper introduces a benchmark for evaluating large language model (LLM) agents in closed‑loop embedded software development. It provides five embedded‑control tasks with realistic build‑and‑runtime feedback, testing agents that must translate requirements into code, self‑verify, and iterate until device behavior meets specifications. Seven GPT‑family and Qwen‑family models were evaluated across four feedback scenarios, revealing that gpt‑5.4 leads in pass rate but does not fully saturate the benchmark, while qwen3.5‑27B is the strongest local model and smaller local models perform poorly.
By Jorge Garc\'ia-Carrasco, Sergio Garc\'ia-Carrasco, Alejandro Mat\'e, Juan Trujillo
The paper introduces RAG‑Stress, a diagnostic protocol that tests how retrieval‑augmented generation models handle misleading evidence. By fixing the question and reference answer, editing a single assertion to support an incorrect answer, and varying source‑priority policies and evidence positions, the study measures misleading rate (MR) and clean accuracy across fifteen systems on multiple QA datasets. Results show that prioritizing documents increases MR by 10.9–13.5 percentage points, with the highest MR when the misleading span appears at the end of the evidence.
By Shunyuan Zhou, Hao Chen, Tianyu Wang, Goose Lin, Zaiyuan Wang, Haiying Zhao
The paper introduces a Spatial Canvas Interface that lets users directly compose poster generation intent in space using semantic, identity, text, and pixel bindings, along with text specifications for elements and overall appearance. It presents Compo, a model adapted from a pretrained image editing system to interpret these canvas inputs, supporting both manual construction and an agentic mode that auto‑generates a canvas from high‑level requests. A scalable training pipeline and benchmark demonstrate that Compo offers stronger compositional control and visual quality than existing general‑purpose and dedicated poster generation models.
By Yitong Wang, Fangyun Wei, Jinjing Zhao, Sirui Zhang, Hongyang Zhang, Dong Chen, Bo Dai, Yan Lu
The paper evaluates a three‑part control stack— a 5‑Pillar runtime constitution, a 4‑tier swarm (Director, majority vote, Monitor, hard gate), and Cognitive Annealing—for multi‑agent LLM negotiation against an adversarial Gatekeeper. Experiments with Gemini 2.5 Pro agents and Claude Haiku 4.5 show that the constitution and Director improve framing but not reliability, the Monitor and hard gate do not reduce unlocks, and deterministic recovery via atomic purge and a canonical strike successfully resolves deadlocks while keeping call budgets unchanged. The study demonstrates that the stack can recover from deadlocks without additional model cost, though it does not reduce overall call or token usage as initially hypothesized.
By Masaaki Nakatsu (AO, Inc. / OrbLabs AG), Reno Wang (AO, Inc.)
MobileWorldBench introduces a benchmark for evaluating vision‑language models as world models in mobile GUI agents, shifting from pixel‑based predictions to natural‑language state transitions. The authors release MobileWorld, a 1.4 million‑sample dataset that enhances VLM world‑modeling performance. They also present a framework that integrates these semantic world models into mobile agent planning, showing improved task success rates.
By Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Aditya Grover
MAPF-World introduces an autoregressive action world model for multi‑agent path finding that predicts short‑horizon local observations and neighboring agents’ action intentions, enabling decisions beyond immediate local views. The model incorporates a spatio‑agent positional encoding within Transformer architectures to enhance spatial awareness and agent semantics, fostering coordinated behaviors. Experiments on diverse maps, including a new automated map generator based on real‑world urban layouts, show that MAPF‑World outperforms existing learned solvers, achieving strong performance, robust zero‑shot generalization, and high success rates even at high agent densities.
By Zhanjiang Yang, Yueming Li, Yang Shen, Meng Li, Lijun Sun
The paper introduces OneMillion-Bench (OMB), a benchmark comprising 400 expert-curated tasks across Law, Finance, Industry, Healthcare, and Natural Science. OMB evaluates language agents on complex, real-world scenarios that require authoritative source retrieval, conflict resolution, domain-specific rule application, and constraint decision-making. The benchmark uses a rubric-based scoring system assessing factual accuracy, logical coherence, practical feasibility, and professional compliance to differentiate agent performance on expert-level problems.
By Yang Liu, Jiaqi Li, Jun Bai, Qianyu Yang, Xiaobo Hu, Tao Peng, Zaiyuan Wang, Ran Tian, Jiayun Dong, Chun Zhang, Zixia Jia, Kaiyuan Chen, Yixin Ren, Yang Liu, Yanglihong Xiao, Lingyue Yin, Tiliang Duan, Ge Zhang, Gang Yao, Hao Chen, Yuan Gong, Jianpeng Jiao, Zilong Zheng
RoboRSI is a robot self‑improvement system that uses Top‑Down Skill Refinement to decompose tasks into compound, atomic, and base skills with clear responsibilities and input‑output contracts. The system coordinates a Manager, Planner, Engineer, and Reviewer to plan, execute, diagnose, and validate skill revisions, allowing humans to guide objectives and corrections. On a mobile manipulator, RoboRSI successfully learns multi‑object household cleanup over 104 rounds and outperforms the strongest baseline by 2.7 to 11.0 percentage points on several simulation benchmarks.
By Zimo Wen, Yijin Chen, Yuxuan Cao, Wendi Chen, Yanwen Zou, Wenye Yu, Fuhang Kuang, Han Xue, Jun Lv, Chuan Wen, Cewu Lu
Humanize is a multi‑agent workflow for agentic coding that uses explicit, mechanically enforced decisions—called judgement engineering—to separate planning, implementation, review, and learning. A human approves a plan contract, a builder agent implements it in rounds, and a reviewer agent from another vendor decides completion, with deterministic hooks and 72 mechanical gates ensuring that a defect survives only if both models miss it. The system has been deployed in over 68 versions in 108 days, earned 1,468 GitHub stars, and powered applications such as a 567‑file gem5 build‑system migration, Kernel Design Agents that excelled in MLSys 2026 contests, and Humanize Olympiad Agents that achieved top scores in multiple international competitions and high rankings on PutnamBench and Lean‑Eval.
whyItMatters":"Humanize demonstrates that a structured, multi‑agent approach with explicit mechanical gates can improve reliability and performance in automated code generation, as evidenced by its adoption, star count, and success in competitive benchmarks."
By Sihao Liu, Ligeng Zhu, Zijian Zhang, Dongyun Zou, Zhengyang Zhang, Changye Li, Song Bian, Song Han, Tony Nowatzki
MIMESIS is a user simulator designed to train interactive language agents by mimicking real user behavior. It is trained on human conversations with explicit reasoning supervision and incorporates 13 realistic behavioral patterns. Experiments show that agents trained with MIMESIS outperform those trained with GPT-5.5 across multiple unseen simulators, and the proposed Coached On-Policy Self-Distillation further improves agent performance by providing dense, token-level supervision.
By Hoang Phan, Dat Huynh, Andrey Zhmoginov, Qi Zeng, Wancen Mu, Yue Cao, Shengjie Bi, Yun He, Changdae Oh, Deren Lei
The paper introduces SameCapRisk-Bench, a dataset of 890 skill units and 1,314 query cases that pair a skill matching a task’s requirements with a same‑capability skill that violates them. It evaluates public skill retrievers on metrics that capture both successful retrieval (Recall) and exposure to conflicting siblings (HSR and CleanHit), finding high rates of harmful sibling exposure. The study also shows that increasing the reranker’s input budget can reduce sibling‑first errors, though CleanHit gains remain uncertain.
By Jiandong Ding, Honglei Ji, Ming Liu, Tao Duan
The paper investigates whether large language models (LLMs) can learn from rewards presented in context, a process termed in‑context reinforcement learning (ICRL). Through experiments on four benchmarks and six models, the authors find that while LLMs read reward signals, altering or removing these signals has little effect on performance, and that trajectory structure—rather than reward content—drives improvement. These results suggest that direct ICRL behaves more like standard in‑context learning (ICL) than true inference‑time reinforcement learning.
By Minchan Kwon, Seunghee Koh, Sunghyun Baek, Minsung Bae, Junmo Kim
The paper introduces Evolving Functional REpertoires (EFRE), a method that replaces a single prompt with a dynamic set of functions to enable large language models to learn new tasks sequentially without forgetting. EFRE refines existing functions when updates are compatible and creates new ones when updates conflict, thereby mitigating catastrophic forgetting and overfitting. Experiments on a three‑task continual‑learning stream show EFRE outperforms GRPO by 7.50 percentage points and reduces performance drop on a subsequent task by only 1.56 percentage points compared to base prompt optimization.
By Fengyuan Liu, Yue Wang, Hangxi Guo, Fengyuan Liu, Chenxu Wu, Yanguang Liu, Mengnan Du