Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,561 stories · RSS feed

arXiv Machine Learning

Jun 16

A nonparametric two-sample test using a parametric integral probability metric

arXiv:2606. 16941v1 Announce Type: cross Abstract: Detecting distributional differences between two independent samples is a fundamental problem in statistics and machine learning.

By Yuha Park, Yongdai Kim

benchmarks

arXiv Machine Learning

Jun 16

Information Leakage Detection through Approximate Bayes-optimal Prediction

arXiv:2401. 14283v4 Announce Type: replace-cross Abstract: In today's data-driven world, the proliferation of publicly available information raises security concerns due to the information leakage (IL) problem.

By Pritha Gupta, Marcel Wever, Eyke H\"ullermeier

benchmarks

arXiv Machine Learning

Jun 16

Unlocking Latent Dimensions: Exploring Representations of Large-Scale X-ray Scattering Data using Variational Autoencoders

arXiv:2606. 14999v1 Announce Type: new Abstract: Scientific user facilities generate X-ray scattering data faster than traditional workflows can process them.

By Monika Choudhary, Xiaoya Chong, Runbo Jiang, Wiebke Koepp, Petrus H. Zwart, Damon English, Gregory M. Su, Eric Schaible, Chenhui Zhu, Mostafa Nassr, Noah P. Wamble, Kelvin Kam-Yun Li, Jonathan M. Chan, Jose Carlos Diaz, Cameron McKay, Lynn Katz, Benny Freeman, Guillaume Freychet, Yevgen Matviychuk, Eliot Gann, Daniel B. Allan, Benedikt Sochor, Frank Schluenzen, Stephan V. Roth, Ethan Crumlin, Dylan McReynolds, Tanny Chavez, Alexander Hexemer

diffusion benchmarks

arXiv AI

Jun 16

RoboPIN: Grounded Embodied Reasoning via Pinned Chain-of-Thought

arXiv:2606. 15753v1 Announce Type: new Abstract: Embodied reasoning requires models to perceive task-relevant objects and spaces in physical environments and maintain consistent visual grounding throughout multi-step reasoning.

By Yaoting Huang, Yifu Yuan, Linqi Han, Chengwen Li, Shuoheng Zhang, Xianze Yao, Hongyao Tang, Yan Zheng, Jianye Hao

llms robotics multimodal benchmarks safety

arXiv AI

Jun 16

NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamics

arXiv:2606. 15015v1 Announce Type: cross Abstract: Physics-grounded video generation requires controllable 3D object dynamics that remain physically consistent under contact, deformation, and external forcing.

By Qizhen Ying, Guangming Wang, Yangchen Pan, Victor Adrian Prisacariu, Yixiong Jing

diffusion benchmarks

arXiv AI

Jun 16

LiteOdyssey: A Lightweight Reasoning AI Agent for Interpretable Rare-Disease Diagnosis

arXiv:2606. 16149v1 Announce Type: new Abstract: Most medical AI systems improve by scaling additional machinery: more fine-tuning data, more agents, and/or larger retrieval databases.

By Minh-Ha Nguyen, Erica Gray, Chih-Ting Yang, Rizwan Hamid, Lingyao Li, Siyuan Ma, Thomas A. Cassini, Cathy Shyr

llms agents fine-tuning benchmarks

arXiv Machine Learning

Jun 16

CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Driving

arXiv:2510. 12560v2 Announce Type: replace-cross Abstract: End-to-end autonomous driving models trained with imitation learning (IL) often generalize poorly, particularly in long-tail scenarios where expert demonstrations are sparse.

By Xiaoji Zheng, Ziyuan Yang, Yanhao Chen, Yuhang Peng, Yuanrong Tang, Gengyuan Liu, Bokui Chen, Jiangtao Gong

agents reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jun 16

Enhancing LLM Safety Through a Theoretical Minimax Game Lens

arXiv:2502. 05163v2 Announce Type: replace-cross Abstract: The rapid advancement of large language models (LLMs) necessitates effective mechanisms to ensure their responsible deployment by accurately distinguishing unsafe content from benign content.

By Yihe Deng, Yu Yang, Junkai Zhang, Wei Wang, Bo Li

llms reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jun 16

Exact Federated Continual Unlearning for Ridge Heads on Frozen Foundation Models

arXiv:2603. 12977v3 Announce Type: replace Abstract: Foundation models are commonly deployed as frozen feature extractors with a small trainable head to adapt to private, user-generated data in federated settings.

By Yijun Quan, Wentai Wu, Giovanni Montana

benchmarks

arXiv Machine Learning

Jun 16

A Biased Nonnegative Block Term Tensor Decomposition Model for Dynamic QoS Prediction

arXiv:2605. 04813v2 Announce Type: replace Abstract: With the rapid development of cloud computing and Web services, Quality of Service (QoS) has become a key criterion for service selection and recommendation.

By Wenjing Liu, Yujia Lei, Qu Wang

benchmarks safety

arXiv Machine Learning

Jun 16

FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies

arXiv:2603. 27450v2 Announce Type: replace Abstract: Thanks to their remarkable flexibility, diffusion models and flow models have emerged as promising candidates for policy representation.

By Chenxiao Gao, Edward Chen, Tianyi Chen, Bo Dai

diffusion reinforcement-learning robotics efficiency benchmarks

arXiv Machine Learning

Jun 16

Anomaly Detection via Mean Shift Density Enhancement

arXiv:2602. 03293v2 Announce Type: replace Abstract: Unsupervised anomaly detection stands as an important problem in machine learning.

By Pritam Kar, Rahul Bordoloi, Olaf Wolkenhauer, Saptarshi Bej

benchmarks

arXiv Machine Learning

Jun 16

Next-Latent Prediction Transformers Learn Compact World Models

arXiv:2511. 05963v4 Announce Type: replace Abstract: Transformers replace recurrence with a memory that grows with sequence length and self-attention that enables ad-hoc lookups over past tokens.

By Jayden Teoh, Manan Tomar, Kwangjun Ahn, Edward S. Hu, Tim Pearce, Pratyusha Sharma, Akshay Krishnamurthy, Riashat Islam, Alex Lamb, John Langford

llms diffusion benchmarks safety

arXiv Machine Learning

Jun 16

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding

arXiv:2602. 00781v2 Announce Type: replace Abstract: Online reinforcement learning in non-episodic, finite-horizon MDPs remains underexplored and is challenged by the need to estimate returns to a fixed terminal time.

By Jiamin Xu, Kyra Gan

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 16

MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions

arXiv:2606. 16562v1 Announce Type: new Abstract: Five years after the discovery of persistent anti-Muslim bias in large language models, most evaluations remain confined to single-turn prompt completion, a setting that no longer reflects how frontier LLMs are deployed.

By Noor Islam S. Mohammad, Tamim Sheikh

llms agents nlp benchmarks safety

arXiv Machine Learning

Jun 16

MUNI: Multimodal Unified Latent Diffusion for Coherent Any-to-Any Generation

arXiv:2606. 16408v1 Announce Type: new Abstract: We introduce MUNI, an end-to-end multimodal latent diffusion framework for any-to-any generation that unifies subset-conditioned cross-modal generation and unconditional joint sampling through a shared stochastic latent.

By Kyeongmin Yeo, Yunhong Min, Minhyuk Sung

llms rag diffusion multimodal benchmarks

arXiv Machine Learning

Jun 16

Near-Optimal Stochastic Linear Bandits with Delay

arXiv:2606. 16656v1 Announce Type: new Abstract: We study stochastic linear bandits with delayed feedback under several delay models and establish near-optimal regret guarantees.

By Ofir Schlisselberg, Mengxiao Zhang, Yishay Mansour

benchmarks safety

arXiv Machine Learning

Jun 16

Not all Jensen-Shannon Divergence Estimators are Equal

arXiv:2606. 16411v1 Announce Type: new Abstract: The Jensen-Shannon divergence is widely reported as a scalar measure of fidelity for synthetic tabular data.

By Alba Garrido, Alejandro Almod\'ovar, Mar Elizo, Patricia A. Apell\'aniz, Santiago Zazo, Juan Parras

benchmarks safety

arXiv Machine Learning

Jun 16

Taming Curvature: Architecture Warm-Up for Stable Transformer Training

arXiv:2606. 16768v1 Announce Type: new Abstract: Training billion-parameter Transformers is often brittle, with transient loss spikes and divergence that waste compute.

By Sameera Ramasinghe, Ajanthan Thalaiyasingam, Hadi Mohaghegh Dolatabadi, Chamin Hewa Koneputugodage, Gil Avraham, Violetta Shevchenko, Yan Zuo, Karol Pajak, Alexander Long

llms benchmarks

arXiv AI

Jun 16

EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective

arXiv:2605. 18421v2 Announce Type: replace-cross Abstract: Recent benchmarks for Large Language Model (LLM) agents mainly evaluate reasoning, planning, and execution.

By Yuyao Wang, Zhongjian Zhang, Mo Chi, Kaichi Yu, Yuhan Li, Miao Peng, Bing Tong, Chen Zhang, Yan Zhou, Jia Li

llms agents benchmarks

Get the daily briefing