Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,620 stories · RSS feed

arXiv Machine Learning

Sep 10

Representation learning of human cortical folding to reveal long lasting neurodevelopmental signatures

arXiv:2609.05438v1 Announce Type: cross Abstract: The human brain folds in utero, primarily during late gestation. Shortly after birth, cortical folding patterns are established and remain stable the...

By Julien Laval, Robin Guiavarch, Antoine Dufournet, Racim Menasria, Barth\'el\'emy Drabczuk, Cristobal Mendoza, Saeb Tounsi, Chikh Abdelghani Baroud, Merieme Bourenane, Vanessa Troiani, William Snyder, Marisa A Patti, Myl\`ene Moyal, Marion Plaze, Arnaud Cachia, Federica Santacroce, Giorgia Committeri, Claire Cury, Kevin De Matos, Olivier Colliot, Zhong Yi Sun, Clara Fischer, Vincent Frouin, Pietro Gori, Denis Rivi\`ere, Jo\"el Chavas, Jean-Fran\c{c}ois Mangin

benchmarks

arXiv Machine Learning

Sep 10

Toward Sustainable Distributed LLM Inference: A Systems Synthesis and Research Agenda for an Energy-, Carbon-, and Cache-Aware llm-d Control Plane

arXiv:2609.05565v1 Announce Type: cross Abstract: Large language model (LLM) sustainability is increasingly a serving-systems problem, not only a training problem. In production, energy and carbon im...

By Twinkll Sisodia

llms benchmarks

arXiv Machine Learning

Sep 10

Spatial Attention Supervision for Defect Localization: Exploiting Ground-Truth Masks as Training Signal in Diffusion-Augmented Defect Detection

arXiv:2609.06232v1 Announce Type: cross Abstract: Ground-truth defect masks in industrial inspection datasets are typically reserved for evaluation. This paper repurposes them as spatial supervision...

By Sajjad Rezvani Boroujeni, Muskan Saraf, Gnana Tulasi Makineni, Tom Bush, Hossein Abedi

llms diffusion benchmarks safety

arXiv Machine Learning

Sep 10

Contextual Observer Grounding: Evaluating Situated Spatial Reasoning in Vision-Language Models

arXiv:2609.06880v1 Announce Type: cross Abstract: Reasoning over language instructions in embodied tasks such as robotics often requires understanding spatial relations from a speaker's situated pers...

By Mimo Shirasaka, Haochen Zhang, Yonatan Bisk

llms robotics multimodal benchmarks

arXiv Machine Learning

Sep 10

Towards Bridging the Gap Between Offline and Iterative Alignment via Preference Distillation

arXiv:2609.06893v1 Announce Type: cross Abstract: Direct preference optimization DPO is a promising offline approach for aligning large language models (LLMs) due to its simplicity, computational eff...

By Wenbo Zhang, Wenzhuo Zhou, Hengrui Cai, Zhengling Qi

llms reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Sep 10

A Quantitative Evaluation Framework for Temporal Explainability in Echocardiographic Video Segmentation

arXiv:2609.08043v1 Announce Type: cross Abstract: Deep learning has achieved state-of-the-art performance in echocardiographic video segmentation, with an increasing number of models incorporating te...

By Jiyoo Noh, Jonathan H. Chan

computer-vision benchmarks

arXiv Machine Learning

Sep 10

Detect Anything in Graphic Design: Element-Level Rewards for Autoregressive Detection

arXiv:2609.07072v1 Announce Type: cross Abstract: Graphic designs, such as posters, advertisements, and infographics, are an important medium for communicating information and shaping understanding....

By Jiangning Zhu, Bowen Li, Shenyu Qiao, Yima Gu, Zhao Zhang, Yuhui Yuan, Shixia Liu

computer-vision benchmarks

arXiv Machine Learning

Sep 10

Iterative Audio Separation with Mixture Consistency via MIMO Model Extension

arXiv:2609.07226v1 Announce Type: cross Abstract: This paper proposes a general framework for stable and effective iterative audio separation with mixture consistency by extending source separation m...

By Yukara Ikemiya, WeiHsiang Liao, Yuki Mitsufuji

diffusion benchmarks

arXiv Machine Learning

Sep 10

VEX-Bench: Benchmarking LLM Agents for Assessing Exploitability of Software Supply Chain Vulnerabilities

arXiv:2609.08040v1 Announce Type: cross Abstract: The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing d...

By Jiahao Shi, Edward Tsien, Yifeng Di, Hongjiao Zhang, Yuan Tang, Ronit Dey, Ilona Shishov, Gal Netanel, Zvi Grinberg, Vladimir Belousov, Bat-Zion Rotman, Ilan Pinto, Tianyi Zhang

llms agents benchmarks

arXiv Machine Learning

Sep 10

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

arXiv:2609.08084v1 Announce Type: cross Abstract: Monocular depth estimation is a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computati...

By Igor Pavlovic, Thiemo Wandel, Anton Obukhov, Luca Bartolomei, Andrey Davydov, Fabio Tosi, Matteo Poggi, Sabine S\"usstrunk, Dengxin Dai

llms diffusion computer-vision robotics fine-tuning efficiency benchmarks

arXiv Machine Learning

Sep 10

A Gradient-based yet Spike-Timing-Dependent Solution to the Feedback Learning Problem in Neural Microcircuits

arXiv:2609.08070v1 Announce Type: cross Abstract: The brain uses discrete spikes for dynamic computation, yet, how neural microcircuits (NMCs) solve temporal credit assignment using local spike timin...

By Xiangnan Zhang, Jingxin Liu, Ranqi Lu, Jingyu Liu, Qunxi Dong, Fuze Tian, Lixian Zhu, Bin Hu, Bj\"orn W. Schuller

benchmarks

arXiv Machine Learning

Sep 10

CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs

arXiv:2609.08345v1 Announce Type: cross Abstract: Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotat...

By Nhat-Tan Bui, Varshini Elangovan, Arun Reddy Anugu, Sreyas Mohan, Wei Ye, Dilin Wang, JQ Huang, Rakesh Ranjan, Aviral Chharia, Fernando De la Torre

efficiency benchmarks

arXiv Machine Learning

Sep 10

ONE CYLinder: A Benchmark for Graph-Based Surrogate Modeling of Unsteady Bluff-Body Flows

arXiv:2609.08947v1 Announce Type: cross Abstract: Graph-based surrogate models offer a promising route to accelerate computational fluid dynamics (CFD) simulations on unstructured meshes. However, th...

By Th\'eodore Michel, Antoine Campos, Alban Dujardin, Henry Areiza, Philippe Meliga, Elie Hachem

llms benchmarks

arXiv Machine Learning

Sep 10

CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

arXiv:2404.06349v3 Announce Type: replace Abstract: The ability to understand causality significantly impacts the competence of large language models (LLMs) in output explanation and counterfactual r...

By Yu Zhou, Xingyu Wu, Jibin Wu, Liang Feng, Kay Chen Tan

llms benchmarks

arXiv Machine Learning

Sep 10

Proper Dataset Valuation by Pointwise Mutual Information

arXiv:2405.18253v4 Announce Type: replace Abstract: Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance....

By Rui Ray Chen, Xuan Qi, Yuxin Chen, Yongchan Kwon, James Zou, Shuran Zheng

benchmarks

arXiv Machine Learning

Sep 10

Two-dimensional Taxonomy for N-ary Knowledge Representation Learning Methods

arXiv:2506.05626v3 Announce Type: replace Abstract: Real-world knowledge can take various forms, including structured, semi-structured, and unstructured data. Among these, Knowledge Graphs (KGs) are...

By Xiaohua Lu, Liubov Tupikina, Mehwish Alam

benchmarks

arXiv Machine Learning

Sep 10

PatchFormer: A Patch-Based Time Series Foundation Model with Hierarchical Masked Reconstruction and Cross-Domain Transfer Learning for Zero-Shot Multi-Horizon Forecasting

arXiv:2601.20845v2 Announce Type: replace Abstract: Time series forecasting is a fundamental problem with applications in climate, energy, healthcare, and finance. Many existing approaches require do...

By Olaf Yunus Laitinen Imanov, Derya Umut Kulali, Taner Yilmaz

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Sep 10

Diffusion-Inspired Reconfiguration of Transformers for Uncertainty Calibration

arXiv:2602.08920v3 Announce Type: replace Abstract: Uncertainty calibration in pre-trained transformers is critical for their reliable deployment in risk-sensitive applications. Yet, most existing pr...

By Manh Cuong Dao, Quang Hung Pham, Phi Le Nguyen, Thao Nguyen Truong, Bryan Kian Hsiang Low, Trong Nghia Hoang

llms diffusion benchmarks

arXiv Machine Learning

Sep 10

Temporal Consistency Improves Generalization in Contextual Offline Meta Reinforcement Learning

arXiv:2603.02935v2 Announce Type: replace Abstract: Offline meta-reinforcement learning seeks to learn a policy that generalizes to new related tasks online. Context-based methods infer a task repres...

By Mohammadreza Nakheai, Aidan Scannell, Kevin Luck, Joni Pajarinen

diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Sep 10

Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

arXiv:2604.02007v3 Announce Type: replace Abstract: Building general-purpose reasoning models using reinforcement learning with verifiable rewards (RLVR) across diverse domains has been widely adopte...

By Rafael Pardinas, Ehsan Kamalloo, David Vazquez, Alexandre Drouin

llms agents reinforcement-learning benchmarks

Get the daily briefing