Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv AI

Oct 15

TacitVLA: Reasoning Vision-Language-Action Models Without Reasoning Latency

arXiv:2610.17540v1 Announce Type: new Abstract: Textual reasoning before acting has improved vision-language-action (VLA) models, but at the cost of latency: every action conditioned on the reasoning...

By Jack Lu, Yurong You, Boyi Li, Yulong Cao, Thomas Tian, Chaowei Xiao, Xiangyu Chen, Wenjie Luo, Boris Ivanovic, Marco Pavone, Yan Wang

agents robotics multimodal benchmarks

arXiv AI

Oct 15

Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents

arXiv:2610.17757v1 Announce Type: new Abstract: Large language model agents can follow complex human instructions to solve real-world tasks. In practice, however, initial instructions are often incom...

By Kaiming Liu, Yuxuan Liu, Jinrui Ju, Minyuan Ruan, Xuanyu Lei, Peng Li, Yang Liu

llms agents benchmarks safety

arXiv AI

Oct 15

Evidence-First Agentic Verification

arXiv:2610.17724v1 Announce Type: new Abstract: Long-horizon, semi-verifiable agentic tasks often rely on rubric-based evaluation, but the evidence needed to assess these tasks can be spread across t...

By Reza Sanayei, Razvan-Gabriel Dumitru, Sunjie Hou, MohammadHossein Rezaei, Haris Riaz, Robert Vacareanu, Minglai Yang, Ying Liu, Mihai Surdeanu, Daniel Yue Zhang

llms agents benchmarks

arXiv AI

Oct 15

Learning Adaptive Deliberation Control for Reliable Recognition with Multi-Agent Language Models

arXiv:2610.17840v1 Announce Type: new Abstract: Multi-agent deliberation can improve recognition by criticizing and revising candidate answers, but fixed controls cannot respond to changes in disagre...

By Yuan Liu, Haoyue Peng, Fanliang Hu, Lei Huang, Wenli Cai

llms agents multimodal benchmarks

arXiv AI

Oct 15

Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection

arXiv:2610.18566v1 Announce Type: new Abstract: Large language model (LLM) routers reduce serving cost by choosing which model to call and how many output tokens to grant it for each query. Existing...

By Yichi Zhang, Zhenxiang Huang, Yuyang Zhang, Yuchen Yang

llms benchmarks

arXiv AI

Oct 15

Who Improves the Robot? Toward Physical Recursive Self-Improvement

arXiv:2610.18842v1 Announce Type: new Abstract: Robots are increasingly autonomous in task execution, yet their improvement remains largely human-driven. This study investigates Physical Recursive Se...

By Hai Zhou, Fan Yang, Zeyi Chen, Lejun Zhang, Chenyang Fu, Jianping Li, Jiayi Zhu, Xianyang Liu, Yingxuan Yang, Lili Meng, Lu Wang, Xinyang Han, Jiankai Sun, Weirui Ye, Guanchu Wang, Wanhua Li, Ming Zhou, Boyi Li, Ming Jin, Kun Shao, Muhao Chen, Guang Chen, Lei Ma, Joey Tianyi Zhou, Costas Spanos, Xia Hu, Mac Schwager, Weinan Zhang, Qian Feng, Shangding Gu

agents robotics benchmarks safety

arXiv AI

Oct 15

Selective Knowledge Control for Continual GUI Agent Learning over Application Streams

arXiv:2609.06530v1 Announce Type: cross Abstract: Continual learning is a crucial capability for Graphical User Interface (GUI) agents to adapt to evolving applications while retaining knowledge acqu...

By Zirui Shang, Xin Shu, Yang Liu, Zhi Gao, Xinxiao Wu, Lifeng Fan

agents robotics benchmarks

arXiv AI

Oct 15

MRFlow: Mechanism-Structured Residual Flow Matching for Virtual Cell Perturbation Prediction

arXiv:2610.13966v1 Announce Type: cross Abstract: Predicting cellular responses to chemical and genetic perturbations is a central test for AI virtual cells, yet many single-cell models reduce each c...

By Haojie Yang, Ran Su

diffusion benchmarks

arXiv AI

Oct 15

Coding LLMs Can Be Harnessed to Learn Interpretable Models from Data

arXiv:2610.14006v1 Announce Type: cross Abstract: Since compact, interpretable classifiers correspond to simple programs, it is natural to ask if LLM-based coding agents can be used to learn interpre...

By Aditya Kumar, Cassandra A. Cohen, William W. Cohen

llms agents benchmarks

arXiv AI

Oct 15

Barycentric Anchored Weight Search for Robust Model Merging

arXiv:2610.14017v1 Announce Type: cross Abstract: We propose Barycentric Anchored Search, a post-hoc weight-space procedure for improving fine-tuned vision models under distribution shift without add...

By Antoni Buades, Jose-Luis Lisani

fine-tuning benchmarks

arXiv AI

Oct 15

Agentic Data Engineering for LLMs: System Design and a Controlled Empirical Study

arXiv:2610.14238v1 Announce Type: cross Abstract: LLM data engineering requires iterative experimentation to determine how data selection, rewards, and curricula affect model learning, making it a na...

By Ruomeng Ding, Qianli Shen, Zhaoyang Han, Meixin Chen, Daoyuan Chen, Yaliang Li

llms agents benchmarks data-engineering

arXiv AI

Oct 15

Akule: Fast and Scalable Multi-Robot Motion Planning via Sparse Interaction Diffusion

arXiv:2610.14248v1 Announce Type: cross Abstract: Diffusion models provide expressive trajectory priors for multi-robot motion planning (MRMP), but scaling them to large teams requires coordinating i...

By Manav Vora, Gokul Puthumanaillam, Aditya Penumarti, Jane Shin, Melkior Ornik

diffusion robotics benchmarks

arXiv AI

Oct 15

Lingdan-V2: Large language models for clinically grounded reasoning in traditional Chinese medicine

arXiv:2610.14705v1 Announce Type: cross Abstract: Large language models have advanced medical reasoning, but clinically grounded reasoning in traditional Chinese medicine (TCM) remains challenging be...

By Rui Hua, Yu Wei, Ziri Jin, Zhuo Sun, Kai Chang, Jianan Xia, Zixin Shu, Xiaodong Li, Dongmei Jia, Fei Dong, Runshun Zhang, Jian Yu, Hao Xu, Haibin Yu, Jiansheng Li, Baoyan Liu, Wenjia Wang, Xuezhong Zhou

llms reinforcement-learning benchmarks safety

arXiv AI

Oct 15

Depth-progressive 3D seismic velocity model building via 2D generative diffusion models

arXiv:2610.14812v1 Announce Type: cross Abstract: Three-dimensional (3D) velocity model building (VMB) remains computationally challenging with conventional approaches and native 3D neural networks,...

By Shijun Cheng, Randy Harsuko, Mohammad H. Taufik, Tariq Alkhalifah

diffusion benchmarks

arXiv AI

Oct 15

RIECNN: Real-Time Image Enhanced CNN for Traffic Sign Recognition

arXiv:2610.15127v1 Announce Type: cross Abstract: Traffic sign recognition plays a crucial role in the development of autonomous cars to reduce the accident rate and promote road safety. It has been...

By Reem Abdel-Salam, Rana Mostafa, Ahmed H. Abdel-Gawad

agents benchmarks safety

arXiv AI

Oct 15

Sensitivity or Criterion? Measuring and Moving When Language Models Act

arXiv:2610.15415v1 Announce Type: cross Abstract: Before an agent acts, it must decide whether action is warranted. A single accuracy cannot distinguish failure to recognize a warranted action from r...

By Sixun Dong, Yebowen Hu, Ming Yin, Yiran Chen, Fei Liu, Chen Chen

llms agents efficiency benchmarks

arXiv AI

Oct 15

A Sovereign, Indic-First, Reasoning Model

arXiv:2610.15371v1 Announce Type: cross Abstract: Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in Eng...

By Sutanshu Saksena Raj, Prasanjith Pasunuti, Parijat Chatterjee, Daksh Maan, Purusharth Malik, Nayanjit Sarkar, Avinash Benki

llms rag agents nlp reinforcement-learning efficiency benchmarks

arXiv AI

Oct 15

Hyperbolic Time Series Forecasting with Learnable Decomposition and Manifold Dynamics

arXiv:2610.15760v1 Announce Type: cross Abstract: Multiscale structure is central to time series forecasting: temporal signals contain slowly varying trends, intermediate seasonal patterns, and high-...

By Sarang Rajendra Patil, Vishal Fenn, Mengjia Xu

rag benchmarks

arXiv AI

Oct 15

LLM Value Functions for Credit Assignment in Agentic RL

arXiv:2610.15840v1 Announce Type: cross Abstract: Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assign...

By Kaijie Zhu, Weixiang Yan, Omar Attia, Abbas Kazerouni, Alexander Toshev, Andrew Szot

llms agents reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Oct 15

Variance-Optimal State Resampling for Reinforcement Learning with Verifiable Rewards

arXiv:2610.15888v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards spends much of its rollout budget on states that yield no learning signal. We recast this as a questio...

By Yibo Wang, Yuanxin Liu, Zhaoran Wang

agents reinforcement-learning benchmarks

Get the daily briefing