Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

24,805 stories · RSS feed

arXiv Machine Learning

Jun 2

Autopilot-Preserving Residual Q-Learning with HJB-Inspired Finite-Action Risk Filtering for Fixed-Wing UAV Command Supervision

arXiv:2606. 01397v1 Announce Type: cross Abstract: A fixed-wing UAV must hold airspeed, altitude, and heading references under wind, gusts, and turbulence, channels coupled so that correcting one can degrade another.

By Mehmet Iscan, Batuhan Temiz

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 2

KISS: Keeping it Simple and Slotted when Learning to Communicate over Wireless

arXiv:2606. 00266v1 Announce Type: cross Abstract: A long-standing challenge in distributed wireless systems is ensuring efficient and fair random channel access.

By Kamil Szczech, Maksymilian Wojnar, Krzysztof Rusek, Katarzyna Kosek-Szott, Szymon Szott

agents benchmarks safety

arXiv AI

Jun 2

Beyond the Frontier: Stochastic Backtracking for Efficient Test-Time Scaling

arXiv:2605. 25143v2 Announce Type: replace Abstract: Test-time scaling improves language model reasoning by spending additional compute to explore multiple solution trajectories.

By Dao Tran, Duc Anh Le, Ngoc Luu, Quan Pham, Tung Pham, Hung Bui

llms benchmarks

arXiv AI

Jun 2

ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition

arXiv:2601. 19919v2 Announce Type: replace-cross Abstract: Knowledge distillation (KD) is one of the most effective paradigms for compressing large-scale foundation models into deployable architectures.

By Junseok Lee, Nahun Kim, Sangyong Lee, Chang-Jae Chun

efficiency multimodal benchmarks safety

arXiv Machine Learning

Jun 2

DeepLatent: Think with Images via Parallel Latent Visual Reasoning

arXiv:2606. 00562v1 Announce Type: cross Abstract: The emerging paradigm of "thinking with images" embeds visual states into intermediate reasoning steps, defining a new frontier for Vision-Language Models.

By Dongchen Lu, Zhimo Li, Mao Shu, Huo Cao

llms rag reinforcement-learning robotics efficiency multimodal benchmarks

arXiv AI

Jun 2

Optimizing Diversity and Quality through Base-Aligned Model Collaboration

arXiv:2511. 05650v2 Announce Type: replace-cross Abstract: Alignment has greatly improved large language models (LLMs)' output quality at the cost of diversity, yielding highly similar outputs across generations, especially in open-ended generation tasks.

By Yichen Wang, Chenghao Yang, Tenghao Huang, Muhao Chen, Jonathan May, Mina Lee

llms benchmarks safety

arXiv AI

Jun 2

Toward accurate RUL and SoH estimation using reinforced graph-based physics-informed neural networks enhanced with dynamic weights

arXiv:2507. 09766v2 Announce Type: replace-cross Abstract: Accurate estimation of Remaining Useful Life (RUL) and State of Health (SoH) is essential for reliable Prognostics and Health Management (PHM), supporting timely maintenance and dependable industrial operation.

By Mohamadreza Akbari Pour, Ali Ghasemzadeh, Mohamad Ali Bijarchi, Mohammad Behshad Shafii

reinforcement-learning benchmarks

arXiv AI

Jun 2

Latent Collaboration in Multi-Agent Systems

arXiv:2511. 20639v3 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) extend large language models (LLMs) from independent single-model reasoning to coordinative system-level intelligence.

By Jiaru Zou, Ruizhong Qiu, Gaotang Li, Xiyuan Yang, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, Ling Yang

llms rag agents diffusion benchmarks

arXiv Machine Learning

Jun 2

Learning-based Directed Graph Abstraction of Combinatorial Spaces for Order-Preserving Search in Mixed-Combinatorial Nonlinear Optimization

arXiv:2606. 01425v1 Announce Type: new Abstract: Mixed-combinatorial nonlinear programming (MCNLP) problems arise in many engineering design and planning applications, e.

By Gishnu Madhu, Feng Liu, Souma Chowdhury

robotics benchmarks

arXiv AI

Jun 2

VocSim: A Training-free Benchmark for Zero-shot Content Identity in Single-source Audio

arXiv:2512. 10120v2 Announce Type: replace-cross Abstract: General-purpose audio representations aim to map acoustically variable instances of the same event to nearby points, resolving content identity in a zero-shot setting.

By Maris Basha, Anja Zai, Sabine Stoll, Richard Hahnloser

rag multimodal benchmarks safety

arXiv AI

Jun 2

Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics

arXiv:2601. 04946v3 Announce Type: replace-cross Abstract: Automatic metrics are widely used to evaluate text-to-image models, often replacing human judgment in benchmarking, model selection, and large-scale data filtering.

By Subhadeep Roy, Gagan Bhatia, Steffen Eger

rag diffusion multimodal benchmarks safety

arXiv Machine Learning

Jun 2

Measuring Alignment-Induced Activation Shifts Correctly: A Template-Controlled Difference-in-Differences Protocol

arXiv:2605. 24583v3 Announce Type: replace Abstract: Comparing a model's internal activations before and after alignment is a natural way to ask what safety training changes: one forms the matrix of paired aligned-minus-base activations on safety-relevant inputs and reads off its effective rank or top direction.

By Yuki Nakamura

llms benchmarks safety

arXiv AI

Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen

llms computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 2

OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agents

arXiv:2606. 02031v1 Announce Type: cross Abstract: Building capable visual web agents requires long-horizon reasoning, precise grounding, and robust interaction with dynamic real-world websites.

By Rui Yang, Qianhui Wu, Yuxi Chen, Hao Bai, Wenlin Yao, Hao Cheng, Baolin Peng, Huan Zhang, Tong Zhang, Jianfeng Gao

llms agents reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jun 2

Design-MLLM: A Reinforcement Alignment Framework for Verifiable and Aesthetic Interior Design

arXiv:2603. 13312v2 Announce Type: replace-cross Abstract: Interior design is a requirements-to-visual-plan generation process that must simultaneously satisfy verifiable spatial feasibility and comparative aesthetic preferences.

By Yuxuan Yang, Xiaotong Mao, Jingyao Wang, Fuchun Sun

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 2

NestRL: A Nested Training Regime for Mutual Adaptation in Human-AI Teaming

arXiv:2602. 17737v2 Announce Type: replace-cross Abstract: Mutual adaptation is a central challenge in human-AI teaming, as humans naturally adjust their strategies in response to an AI agent's behavior.

By Upasana Biswas, Durgesh Kalwar, Subbarao Kambhampati, Sarath Sreedharan

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 2

A combination of noise and bilateral filters achieve supralinear and scalable adversarial robustness in CNNs

arXiv:2606. 02267v1 Announce Type: new Abstract: The vulnerability of deep neural networks to adversarial examples poses a significant challenge for real-world deployment.

By Nicolas Stalder, Benjamin F. Grewe, Matteo Saponati, Pau Vilimelis Aceituno

benchmarks safety

arXiv Machine Learning

Jun 2

KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices

arXiv:2601. 21579v2 Announce Type: replace-cross Abstract: The success of Hyper-Connections (HC) in neural networks (NN) has also highlighted issues related to training instability and restricted scalability.

By Wuyang Zhou, Yuxuan Gu, Giorgos Iacovides, Danilo Mandic

benchmarks

arXiv Machine Learning

Jun 2

MINES: Explainable Anomaly Detection through Web API Invariant Inference

arXiv:2512. 06906v2 Announce Type: replace-cross Abstract: Detecting the anomalies of web applications, important infrastructures for running modern companies and governments, is crucial for providing reliable web services.

By Wenjie Zhang, Yun Lin, Chun Fung Amos Kwok, Xiwen Teoh, Xiaofei Xie, Frank Liauw, Hongyu Zhang, Jin Song Dong

llms benchmarks

arXiv Machine Learning

Jun 2

Training-Free Imitation Learning with Closed-Form Diffusion Policies

arXiv:2606. 01238v1 Announce Type: cross Abstract: While diffusion-based policies have impressive performance and expressivity, their long offline training slows down the data collection and policy deployment loop.

By Raghav Mishra, Ian R. Manchester

diffusion benchmarks

Get the daily briefing