Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,561 stories · RSS feed

arXiv Machine Learning

Jun 15

Temporally Consistent Graph Q-Networks for Intelligent Network Control

arXiv:2606. 13848v1 Announce Type: cross Abstract: Mobile networks continue to grow in complexity and next generation networks are expected to support both increasing traffic loads and more diverse services.

By Zacharias Veiksaar, Maxime Bouton

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 15

Federated Learning for Feature Generalization with Convex Constraints

arXiv:2606. 14416v1 Announce Type: new Abstract: Federated learning (FL) often struggles with generalization due to heterogeneous client data.

By Dongwon Kim, Donghee Kim, Sung Kuk Shyn, Kwangsu Kim

benchmarks

arXiv AI

Jun 15

Explaining RhythmFormer: A Systematic XAI Analysis of Periodic Sparse Attention for Remote Photoplethysmography

arXiv:2606. 13839v1 Announce Type: cross Abstract: Remote photoplethysmography (rPPG) transformers achieve low heart-rate error on benchmarks, yet their decisions remain opaque--a growing concern as rPPG moves toward clinical heart rate estimation.

By Louis Chen, Torbj\"orn E. M. Nordling

llms benchmarks safety

arXiv AI

Jun 15

A Comparative Study of Deep Learning Architectures for Multi-Horizon Behavioural Forecasting for Mobile Health

arXiv:2606. 14604v1 Announce Type: cross Abstract: Wearable devices and smartphones generate rich behavioural time series that can support proactive health interventions, yet systematic comparisons of modern forecasting architectures for these data are lacking.

By Pavlos Nicolaou, Kleanthis Malialis, Artemis Kontou, Panayiotis Kolios

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 15

XRDiff: Crystal Structure Prediction from Powder X-Ray Diffraction Data Using Diffusion Models

arXiv:2606. 14003v1 Announce Type: cross Abstract: Determining the crystal structure of a material from its powder X-ray diffraction (PXRD) pattern is a central challenge in materials science.

By Nofit Segal, Mingda Li, Benjamin Kurt Miller, Rafael G\'omez-Bombarelli

diffusion benchmarks

arXiv AI

Jun 15

Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results

arXiv:2606. 14516v1 Announce Type: new Abstract: AI evaluations are widely used for testing and understanding progress.

By Jan Batzner, Sree Harsha Nelaturu, Anastassia Kornilova, Jon Crall, Tommaso Cerruti, Yanan Long, Yifan Mai, Sanchit Ahuja, Asaf Yehudai, Marek \v{S}uppa, John P. Lalor, Oluwagbemike Olowe, Jatin Ganhotra, Brian H. Hu, Eliya Habba, Andrew M. Bean, Chang Liu, Sander Land, Steven Dillmann, Aniketh Garikaparthi, Elron Bandel, Saki Imai, James Edgell, Wm. Matthew Kennedy, Jenny Chim, Patrick Meusling, Asteria Kaeberlein, Venkata Ramachandra Karthik Chundi, Manasi Patwardhan, Martin Ku, Austin Meek, Leon Knauer, Brian Wingenroth, Srishti Yadav, Usman Gohar, Felix Friedrich, Michelle Lin, Jennifer Mickel, Arman Cohan, Stella Biderman, Irene Solaiman, Zeerak Talat, Anka Reuel, Mubashara Akhtar, Gjergji Kasneci, Avijit Ghosh, Leshem Choshen

benchmarks

arXiv AI

Jun 15

ChronoID: Infusing Explicit Temporal Signals into Semantic IDs for Generative Recommendation

arXiv:2606. 14260v1 Announce Type: cross Abstract: Semantic IDs are crucial in generative recommendation, but with a fundamental limitation: temporal information is not well incorporated into semantic IDs.

By Dongdong Nian, Dongqi Fu, Chenliang Xu, Yinglong Xia, Hong Li, Hong Yan, Jian Kang

benchmarks safety

arXiv AI

Jun 15

OdysSim: Building Foundation Models for Human Behavior Simulation

arXiv:2606. 14199v1 Announce Type: cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.

By Xuhui Zhou, Weiwei Sun, Weihua Du, Jiarui Liu, Haojia Sun, Qianou Ma, Tongshuang Wu, Yiming Yang, Maarten Sap

llms reinforcement-learning efficiency benchmarks safety

arXiv AI

Jun 15

Aligning Quantum Operators with Large Language Models

arXiv:2606. 13811v1 Announce Type: cross Abstract: Can Large Language Models (LLMs) understand and reason about quantum operators?

By Rogerio Feris, Yunchao Liu, Pengyuan Li, Hang Hua, David Kremer

llms diffusion benchmarks

arXiv AI

Jun 15

FEMOT: Multi-Object Tracking using Frame and Event Cameras

arXiv:2606. 14094v1 Announce Type: cross Abstract: Conventional RGB cameras have been widely used in multi-object tracking due to their ability to capture rich appearance and semantic information.

By Shiao Wang, Xiao Wang, Chao Wang, Yitao Li, Menghao Liu, Bo Jiang, Yaowei Wang, Yonghong Tian, Jin Tang

multimodal benchmarks

arXiv AI

Jun 15

Learning High Coverage Discriminative Parsimonious Rulesets

arXiv:2606. 14156v1 Announce Type: cross Abstract: Learning systems based on IF-THEN rule representations readily offer interpretability, making them a crucial focus in contemporary AI research.

By Mariamma Antony, Raman Sankaran, Chiranjib Bhattacharyya, Uma Satya Ranjan

benchmarks safety

arXiv AI

Jun 15

ANSR-DT: A Neuro-Symbolic Framework for Adaptive and Explainable Digital Twins

arXiv:2501. 08561v4 Announce Type: replace Abstract: Digital twins are increasingly used to monitor and optimize industrial systems, yet many existing frameworks remain difficult to interpret, slow to adapt, and limited in their ability to incorporate explicit domain knowledge.

By Safayat Bin Hakim, Muhammad Adil, Alvaro Velasquez, Houbing Herbert Song

reinforcement-learning benchmarks safety

arXiv AI

Jun 15

MASLab: A Unified and Comprehensive Codebase for LLM-based Multi-Agent Systems

arXiv:2505. 16988v2 Announce Type: replace-cross Abstract: LLM-based multi-agent systems (MAS) have demonstrated significant potential in enhancing single LLMs to address complex and diverse tasks in practical applications.

By Rui Ye, Keduan Huang, Qimin Wu, Yuzhu Cai, Tian Jin, Xianghe Pang, Xiangrui Liu, Jiaqi Su, Chen Qian, Bohan Tang, Kaiqu Liang, Jiaao Chen, Yue Hu, Zhenfei Yin, Rongye Shi, Bo An, Yang Gao, Wenjun Wu, Lei Bai, Siheng Chen

llms agents benchmarks

arXiv Machine Learning

Jun 15

Decoupled Latent Optimization of Diffusion Models for Full Waveform Inversion

arXiv:2606. 14139v1 Announce Type: new Abstract: Full waveform inversion (FWI) recovers subsurface velocity from seismic recordings by solving a severely ill-posed, nonconvex PDE-constrained optimization.

By Chen Min, Zheng Ma

diffusion benchmarks

arXiv AI

Jun 15

Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders

arXiv:2503. 19947v2 Announce Type: replace-cross Abstract: Generalized metric depth understanding is critical for precise vision-guided robotics, which current state-of-the-art (SOTA) vision-encoders do not support.

By Paul Koch, J\"org Kr\"uger

diffusion computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 15

ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning

arXiv:2606. 14697v1 Announce Type: cross Abstract: Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support.

By Sicheng Yang, Hangjie Yuan, Wenjun Zhang, Jinwang Wang, Yichen Qian, Weihua Chen, Fan Wang, Lei Zhu

llms fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Jun 15

EM-NeSy: Expectation Maximization for Neurosymbolic Learning

arXiv:2606. 14463v1 Announce Type: new Abstract: Neurosymbolic (NeSy) models integrate neural networks and symbolic reasoning for robust and interpretable AI.

By Annegret Seibt, Luc De Raedt, Giuseppe Marra

benchmarks

arXiv AI

Jun 15

Active Inference for Adaptive Traffic Signal Control in Noisy Nonstationary IoT Environments

arXiv:2606. 13698v1 Announce Type: cross Abstract: Urban traffic signal control at IoT-instrumented intersections must remain effective under sensor occlusion, weather attenuation, and nonstationary demand.

By D\'enes Toth, George Ambroladze, Edwin Sundberg, Ali Beikmohammadi, Alfreds Lapkovskis

benchmarks

arXiv AI

Jun 15

VISTA: View-Consistent Self-Verified Training for GUI Grounding

arXiv:2606. 14579v1 Announce Type: new Abstract: When applying Group Relative Policy Optimization (GRPO) for GUI Grounding, rollouts are sampled from a single screenshot view; groups often become either all failures on difficult instances or all successes on easy ones, yielding no useful relative advantage.

By Xinyu Qiu, Yunzhu Zhang, Heng Jia, Shuheng Shen, Changhua Meng, Linchao Zhu

reinforcement-learning benchmarks

arXiv AI

Jun 15

A Fixed-Point Neural Operator for Size- and Functional-Transferable Hamiltonian Prediction

arXiv:2606. 14498v1 Announce Type: cross Abstract: Predicting the Kohn-Sham Hamiltonian with machine learning can accelerate density functional theory while retaining access to molecular orbitals, energy levels, and electronic-structure observables that energy-only surrogates cannot resolve.

By Yunhong Lou, Xihang Yue, Xinran Wei, Tianqi Deng, Linchao Zhu

fine-tuning benchmarks

Get the daily briefing