Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv AI

Oct 27

AdaTrendLike: Adaptive Hierarchical Detrending and Trend Likelihood Scoring for Time Series Anomaly Detection

arXiv:2610.35400v1 Announce Type: new Abstract: The rapid proliferation of Industrial Internet of Things devices has led to increasingly large-scale and non-stationary time-series data, posing signif...

By Van Kwan Zhi Koh, Songnan Lin, Rongmo Luo, Zhiping Lin, Bihan Wen

rag benchmarks safety

arXiv AI

Oct 27

CANS: Capability-Preserving Neuron Steering for Large Language Models

arXiv:2610.35530v1 Announce Type: new Abstract: Activation steering offers training-free and dynamic control of target behaviors in large language models, but existing methods can impair general capa...

By Yucheng Wu, Chengyi Xie, Chenpeng Wang, Liangming Pan

llms fine-tuning benchmarks safety

arXiv AI

Oct 27

EvoRouter: Cost-Efficient Multi-Agent Routing via Insight-Guided On-Policy Distillation

arXiv:2610.35690v1 Announce Type: new Abstract: Serving web queries at scale with multi-agent systems requires jointly configuring large language models, collaboration modes, agent counts, interactio...

By Yitao Xiao, Xu Zhang, Ye Cheng, Shaoyong Guo, Yan Sun, Ziyue Zhu, Sitao Xie, Guo Tong, Haoran Luo

llms agents reinforcement-learning efficiency benchmarks

arXiv AI

Oct 27

Agentic Algorithm Design: LLMs that Propose, Formalise, and Prove Runtime Bounds for Evolutionary Algorithms in Lean

arXiv:2610.35595v1 Announce Type: new Abstract: LLMs now prove hard theorems in Lean. Can they also design algorithms and prove their correctness and efficiency? LLM-guided design of optimisation heu...

By Per Kristian Lehre

llms agents benchmarks

arXiv AI

Oct 27

TIAgent: From Natural Language to Grounded Agentic Workflows for Computational Pathology

arXiv:2610.32993v1 Announce Type: cross Abstract: Computational pathology (CPath) analyses increasingly depend on multi-step computational workflows that require substantial programming expertise, li...

By Aakash Madhav Rao, Mark Eastwood, Danielle Belgrave, Fayyaz Minhas

llms agents benchmarks

arXiv AI

Oct 27

Aerial-R1: Reinforcing Aerial Reasoning and Segmentation in MLLMs

arXiv:2610.33056v1 Announce Type: cross Abstract: While Multimodal Large Language Models (MLLMs) have revolutionized general visual perception, existing methods suffer from severe hallucination in th...

By Jisheng Dang, Huaiyuan Peng, Yixuan Wang, Kun Wang, Zhang Li, Tat-Seng Chua

llms agents computer-vision fine-tuning multimodal benchmarks safety

arXiv AI

Oct 27

MOSAIC: Towards a Universal Latent Space for Computational Pathology Foundation Models

arXiv:2610.33059v1 Announce Type: cross Abstract: Pathology foundation models (PFMs) differ substantially in architecture, pretraining objective, training data and representation dimensionality, yet...

By Dibaloke Chanda, Praveen Madiraju

diffusion benchmarks safety

arXiv AI

Oct 27

Position, Role or Label? A Controlled Study of Instruction Conflicts in LLM Agents

arXiv:2610.33201v1 Announce Type: cross Abstract: LLM agents take instructions from the operator, repository configuration, user, tool results and saved memory, which often conflict. Instruction-hier...

By Anish Laddha, Nitesh Pradhan, Gaurav Srivastava

llms agents benchmarks

arXiv AI

Oct 27

From Hype to ROI: An Economic Model for LLM Adoption in Information Systems and Security Operations

arXiv:2610.33352v1 Announce Type: cross Abstract: Large Language Models (LLMs) promise significant efficiency gains across information-intensive operations by automating repetitive language-based tas...

By Pellegrino Casoria, Andrea Araldo, Rosario Patan\`e, Simon Pietro Romano

llms benchmarks

arXiv AI

Oct 27

WorldFailBench: Do World Models Understand Policy Failures?

arXiv:2610.33565v1 Announce Type: cross Abstract: Robot policy evaluation informs deployment decisions and policy improvement, but physical testing and simulator preparation require substantial effor...

By Zheng Zhang, Xiaoyang Tang, Shuaiqi Cheng, Jiakai Huang, Zhongbo Wu, Ziyi Zhang, Yuan Si, Zeyu Liu, Weibo Gao, Shunyu Liu, Shan You, Fei Wang, Xiaogang Wang, Dacheng Tao

robotics benchmarks

arXiv AI

Oct 27

ProMo3D: Probing Motion Cues in Frozen 3D Foundation Models

arXiv:2610.33802v1 Announce Type: cross Abstract: Feed-forward 3D foundation models have recently reshaped multi-view reconstruction, yet dynamic 4D understanding still often relies on large-scale mo...

By Xiaoang Zhang, Mert Kiray, Yordanka Velikova, Manoj Biswanath, Benjamin Busam

fine-tuning efficiency benchmarks

arXiv AI

Oct 27

Holding a Steady Course: Knowledge Evolution from Expertise and Lessons for Continual Agent Learning

arXiv:2610.34043v1 Announce Type: cross Abstract: Autonomous agents are increasingly deployed across sequences of related tasks, yet they often fail to retain and improve their knowledge over time. E...

By Wenzhe Fan, Benjamin Steenhoek, Gaurav Mittal, Yu Hu

agents benchmarks

arXiv AI

Oct 27

$\Delta\tau$-Bench: Pushing Agents Toward a Moving Capability

arXiv:2610.34238v1 Announce Type: cross Abstract: In this paper, we introduce $\Delta\tau$-Bench, an executable agent benchmark that models the structural evolution of large tool ecosystems. In $\Del...

By Shuofei Qiao, Xuanguang Pan, Yijun Chen, Wendi Feng, Bin Wu, Runnan Fang, Yuqi Zhu, Shumin Deng, Emine Yilmaz

agents benchmarks

arXiv AI

Oct 27

EvolveMAS: A Unified Self-Evolution Framework for Decentralized Multi-Agent Systems

arXiv:2610.34450v1 Announce Type: cross Abstract: Decentralized multi-agent systems (MAS) enable large language model agents to interact without centralized control. Agents need to adapt to other age...

By Yalun Wu, Qiwei Ma, Bingbing Xu, Dan Zhang, Long Bai, Boyang Wang, Jiatong Li, Zheqi Lv, Shaozu Yuan, Peiying Wang, Jiawei Wang, An Zhang, Tat-Seng Chua

llms agents benchmarks safety

arXiv AI

Oct 27

GRASP: Group-Adaptive Hybrid RL-SFT Post-Training for Medical VQA without Reasoning Annotations

arXiv:2610.34561v1 Announce Type: cross Abstract: Adapting general-purpose vision-language models (VLMs) to medical visual question answering (VQA) is valuable for helping clinicians read images and...

By Wanqi Dong, Wanyue Dong, Jingze Ge, Mehul Motani

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Oct 27

RobotAgentBench: Towards Open-world Robot Autonomy

arXiv:2610.34649v1 Announce Type: cross Abstract: Recent advances in vision-language-action (VLA) and world-action models (WAMs) have significantly improved robotic manipulation, yet deploying robots...

By Lingjun Mao, Jinglin Cao, Lukun He, Wenpeng Xu, Yuchen Yan, Hengyu Jin, Zijun Zhang, Jinzhou Tang, Songyao Jin, Ziche Liu, Yifei Shao, Junbo Huang, Fang Nan, Ruobing Han, Ziqiao Xi, Ziming Xu, Shuang Liang, Sibo Zhu, Wenyi Wu, Xinyue Wang, Yiming Huang, Xinkai Zou, Gao Zhu, Kun Zhou, Biwei Huang

agents robotics multimodal benchmarks

arXiv AI

Oct 27

Correlation-Aware Exploration for Long-Trace Reasoning

arXiv:2610.34949v1 Announce Type: cross Abstract: Reasoning models trained with Group Relative Policy Optimization (GRPO) now produce chains of thought thousands of tokens long, and recent studies re...

By Junxing Hu, Zicheng Zhang, Zhiqian Zhang, Ai Han, Yifeng Zhang, Pengzhang Liu, Ling Li

benchmarks

arXiv AI

Oct 27

V$^3$AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding Agents

arXiv:2610.35188v1 Announce Type: cross Abstract: Large language models increasingly act through code, modifying repositories, executing programs, and interacting with services in agent form. Their a...

By Weitao Feng, Hengheng Xiong, Zihao Zeng, Jun Chen, Jiyang Guan, Xiaofeng Wang, Jialing Tao, Hui Xue, Wei Dong

llms agents benchmarks safety

arXiv AI

Oct 27

Generative AI in Turbulence: Foundations, Applications, Validations and Benchmarks

arXiv:2610.35123v1 Announce Type: cross Abstract: Generative AI offers new approaches to turbulence problems in which incomplete observations or unresolved d.o.f. admit multiple physically plausible...

By Luca Biferale, Michele Buzzicotti

diffusion benchmarks

arXiv AI

Oct 27

Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models

arXiv:2610.35829v1 Announce Type: cross Abstract: A vulnerability costs more to fix the later it is discovered. As coding agents generate code faster than developers can review it, software needs a f...

By Amine Lbath, Massih-Reza Amini

llms agents fine-tuning benchmarks

Get the daily briefing