Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Machine Learning

Sep 15

$\mathbb{SL}(n)$ Representation Learning: An Intrinsic Mixed-Curvature Space with Higher Curvature Capacities and Deeper Order-Aware Composition

arXiv:2609.15083v1 Announce Type: new Abstract: Mixed-curvature representation learning seeks to capture rich geometric structures that cannot be adequately modeled by a single curvature regime. Exis...

By Xingrun Li, Yusuke Mukuta, Xin Yang, Yinyu Ye, Tatsuya Harada

benchmarks

arXiv Machine Learning

Sep 15

What Does an LLM Learn from Reinforcement Learning? A Mechanistic Interpretability Perspective with Fixed-SAE Track

arXiv:2609.15064v1 Announce Type: new Abstract: Reinforcement learning (RL) is widely utilized in large language model training to improve targeted capabilities, yet how RL reshapes a model remains p...

By Lingheng Du, Yiming Tang, Xufeng Duan, Dianbo Liu

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Sep 15

Temporal Self-Distillation: Faster Inference in Discrete Diffusion Language Models

arXiv:2609.15177v1 Announce Type: new Abstract: Diffusion language models (dLLMs) promise fast inference by generating multiple tokens in parallel, but suffer severe performance degradation when para...

By Shijian Xu, Andrea Miele, Metod Jazbec, Volker Roth, Eric Nalisnick, Ilija Bogunovic

llms diffusion efficiency benchmarks

arXiv Machine Learning

Sep 15

GSLAD: Prototype-Regularized Graph Structure Learning for Multivariate Time Series Anomaly Detection

arXiv:2609.15483v1 Announce Type: new Abstract: Unsupervised multivariate time series anomaly detection methods typically identify anomalies through forecasting, reconstruction, or representation dis...

By Zepeng Zhang, Fuad Khuri, Keivan Faghih Niresi, Olga Fink

benchmarks

arXiv Machine Learning

Sep 15

Principal-timestep Restricted Init via Sparse Matrix-decomposition in Flow-matching

arXiv:2609.15643v1 Announce Type: new Abstract: Flow-matching diffusion models have recently emerged as a strong paradigm for high-fidelity visual generation. However, their prohibitively high fine-t...

By Jiayang Gu, Zheng Fang, Lichaun Xiang, Fanghui Liu, Xu Cai, Hongkai Wen

llms diffusion fine-tuning benchmarks safety

arXiv Machine Learning

Sep 15

MoveBench: A Benchmark for Global-Scale Wildlife Movement Forecasting

arXiv:2609.15780v1 Announce Type: new Abstract: Understanding and predicting wildlife movement is critical for ecology and conservation. While trajectory forecasting has advanced for human and vehicl...

By Justin Kay, Shir Bar, Ellen O. Aikens, Martin Becker, Francesca Cagnacci, Juliet Cohen, Scott W. Forrest, Jessica Kendall-Bar, Madeleine Lucas, Macon Overcast, Meredith S. Palmer, Will Rogers, Nicholas J. Russo, Christian Rutz, Larissa T. Beumer, Michael Brown, Ying-Chi Chan, Sarah C. Davidson, Diego Ellis Soto, Anne G. Hertel, Roland Kays, Benjamin Koger, Guram Mikaberidze, Thomas Mueller, Ruth Oliver, Thorsten Papenbrock, Robert Patchett, Jared A. Stabach, Dane Taylor, Scott W. Yanco, Sara Beery

benchmarks

arXiv Machine Learning

Sep 15

Bellman Policy Optimization

arXiv:2609.15987v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models (LLMs). We introduce Bellman Policy...

By Zhuoqing Song, Haotian Xu, Xikun Zhang, Lidong Bing

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 15

QSTAR: Quantum Selective Transfer with Adaptive Routing

arXiv:2607.21411v1 Announce Type: cross Abstract: Quantum transfer learning (QTL) is often evaluated by replacing a classical classifier with a fixed variational quantum head, but this hides a key qu...

By Saim Rehman, Nouhaila Innan, Muhammad Shafique

fine-tuning benchmarks

arXiv Machine Learning

Sep 15

Part Grounding, Not Action Knowledge: Locating the Bottleneck in VLM Affordance Prediction

arXiv:2609.13225v1 Announce Type: cross Abstract: Benchmarks agree that vision-language models reason poorly about low-level manipulation, but an aggregate accuracy score does not say which step fail...

By Sarthak Sattigeri

llms robotics multimodal benchmarks

arXiv Machine Learning

Sep 15

GzDRL: Reproducible and Scalable Deep Reinforcement Learning with Gazebo

arXiv:2609.13243v1 Announce Type: cross Abstract: We present GzDRL, a novel single-process reinforcement learning (RL) framework for Gazebo that overcomes longstanding bottlenecks in scalable, reprod...

By Amal Dev Haridevan, Junjie Kang, Jinjun Shan

agents reinforcement-learning robotics fine-tuning efficiency benchmarks

arXiv Machine Learning

Sep 15

CANAL: Channel-Aware Noise Allocation for Differentially Private Feature Distillation in Medical Image Segmentation

arXiv:2609.13271v1 Announce Type: cross Abstract: Medical image segmentation needs diverse training data, but hospitals hold complementary scans they cannot share for privacy and regulatory reasons....

By Armaghan Butt, Shuya Feng, Qing Tian

computer-vision efficiency benchmarks safety

arXiv Machine Learning

Sep 15

SomBench: Benchmark Dataset for Advancing Machine Learning in Lunar Science

arXiv:2609.13277v1 Announce Type: cross Abstract: Lunar orbital missions, such as Lunar Reconnaissance Orbiter, Kaguya/SELENE, Gravity Recovery and Interior Laboratory, and Lunar Prospector, among ot...

By Himanshu Patil, Gabby Nyirjesy, Rachel A. Slank, Vishal Gaur, Daniela Szwarcman, Paolo Fraccaro, Nikolaos Dionelis, Michael K. Barker, Andrew Annex, Vishnu Viswanathan, Zachary Morse, Ethan I. Schaefer, Hiyam Debary, Ankur Kumar, Rohit Lal, Geoffrey Dawson, Campbell Watson, Rebekah I. Dawson-Rigas, Manil Maskey, Juan Bernab\'e-Moreno, Rahul Ramachandran, Sujit Roy

multimodal benchmarks

arXiv Machine Learning

Sep 15

Inverse Learning of the Altruism and Cost Level in Mixed-Individual Mean Field Games

arXiv:2609.13469v1 Announce Type: cross Abstract: Understanding how humans respond to incentives, both at the individual and collective levels, is crucial to the design of effective policies. Within...

By Haoyang Cao, G\"ok\c{c}e Dayan{\i}kl{\i}, Xiaofei Shi

agents benchmarks

arXiv Machine Learning

Sep 15

Odds-Shift Slippage in One-vs-Rest Rankers: Diagnosing and Repairing Reweighting-Induced Top-K Errors

arXiv:2609.13810v1 Announce Type: cross Abstract: One-vs-rest rankers that show each user the top-$K$ of many rare labels usually counter imbalance with a per-label positive-class weight, scale_pos_w...

By Akifumi Goto

benchmarks

arXiv Machine Learning

Sep 15

Real-World Deployment and Performance Characterisation of Fog-Based Deep Learning for Cold-Chain Temperature Prediction over LoRaWAN

arXiv:2609.14036v1 Announce Type: cross Abstract: Fresh fruits and vegetables (FFVs) are highly perishable, and cold-chain breaks contribute significantly to global food waste. While Machine Learning...

By Jeremiah Taguta, Jean Frederic Isingizwe Nturambirwe, Clement Nthambazale Nyirenda

agents benchmarks

arXiv Machine Learning

Sep 15

When Faster VLA Deployment Changes Closed-Loop Behavior: Task Success-Latency Analysis of SmolVLA Across PyTorch and ONNX Variants

arXiv:2609.14146v1 Announce Type: cross Abstract: Vision-language-action (VLA) deployment can reduce inference latency while changing closed-loop task behavior. We evaluate HuggingFaceVLA/smolvla_lib...

By Rafiqul Islam

efficiency multimodal benchmarks

arXiv Machine Learning

Sep 15

3D Gait-Based Autism Classification Using Attention-Enhanced Deep Learning with Cross-Fold Statistical Stability Analysis

arXiv:2609.14159v1 Announce Type: cross Abstract: Autism Spectrum Disorder (ASD) is a neurodevelopmental condition whose early diagnosis remains challenging because conventional clinical assessments...

By Md Nadim Mahamood, Md Arif Shahriar, Md Parvej Sikder, Md Rasul Islam, Md Shafi Ud Doula, Md Ashraful Alam, Kamrul Hasan

llms benchmarks

arXiv Machine Learning

Sep 15

Joint Optimization for Federated Learning and Transmission over Unreliable Wireless Networks with Heterogeneous Data

arXiv:2609.14246v1 Announce Type: cross Abstract: In wireless federated learning (FL), data heterogeneity and multiple local updates induce client drift, degrading model convergence. It is further af...

By Changheng Wang, Xianchao Zhang, Zhiqing Wei, Lingzhu Zhao, Zhongming Yang, Zhiyong Feng

efficiency benchmarks

arXiv Machine Learning

Sep 15

Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots

arXiv:2609.14567v1 Announce Type: cross Abstract: Reinforcement learning (RL) has shown considerable promise for robotic decision-making, yet deploying multi-agent RL (MARL) on physical multi-robot s...

By Abdalwhab Bakheet Mohamed Abdalwhab, Giovanni Beltrame, David St-Onge

agents reinforcement-learning robotics benchmarks safety

arXiv Machine Learning

Sep 15

CCMAN: Cognitive Instability-Aware Cross-Modal Attention Network for Interpretable Temporal Biomarkers of Verbal Fluency Speech

arXiv:2609.14764v1 Announce Type: cross Abstract: Early detection of cognitive decline from speech offers a scalable and non-invasive alternative to conventional clinical assessment. Verbal fluency t...

By Madhurananda Pahar, Caitlin Illingworth, Dorota Braun, Daniel Blackburn, Heidi Christensen

llms fine-tuning multimodal benchmarks

Get the daily briefing