Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

21,859 stories · RSS feed

arXiv Machine Learning

Jun 15

Graph Diffusion Residuals for Control-Function Instrumental Variables

arXiv:2606. 14636v1 Announce Type: new Abstract: Control-function instrumental variable estimators need a first-stage residual, not merely a first-stage prediction.

By Rui Wu, Zongyuan Chen, Hong Xie, Defu Lian, Enhong Chen

diffusion benchmarks

arXiv Machine Learning

Jun 15

Curvature-Informed Potential Energy Surface for Protein-Ligand Binding Affinity Prediction

arXiv:2606. 14217v1 Announce Type: new Abstract: Accurate prediction of protein-ligand binding affinity is essential for structure-based drug discovery.

By Peng-Fei Sun, Chuan-Xian Ren, Hong Yan

benchmarks safety

arXiv AI

Jun 15

GMN4AD: Graph Matching Network for Alzheimer's Disease Diagnosis with Test-Time Domain Adaptation using Multi-centered Structure Magnetic Resonance Imaging

arXiv:2606. 13919v1 Announce Type: cross Abstract: Alzheimer's Disease (AD) is a progressive neurodegenerative disorder that affects millions of older adults, with prevalence expected to rise significantly in the coming years.

By Chen Zhao, Huan Huang, Yixin Xie, Jiajing Huang, Weihua Zhou, Nandakumar Narayanan

fine-tuning benchmarks

arXiv AI

Jun 15

From Shield to Target: Denial-of-Service Attacks on LLM-Based Agent Guardrails

arXiv:2606. 14517v1 Announce Type: cross Abstract: LLM-based guardrails have emerged as a highly effective defense against prompt injection and jailbreak attacks in autonomous agents.

By Yuguang Zhou, Xunguang Wang, Pingchuan Ma, Zhantong Xue, Zhaoyu Wang, Shuai Wang

llms agents benchmarks safety

arXiv Machine Learning

Jun 15

Utility-Constrained Policy Optimization

arXiv:2606. 14029v1 Announce Type: new Abstract: Constrained MDPs (CMDPs) are a widely adopted framework for incorporating safety into RL agents; however, the framework does not support risk-sensitive constraints.

By Mehrdad Moghimi, Bernardo Avila Pires

agents reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 15

Persona-Pruner: Sculpting Lightweight Models for Role-Playing

arXiv:2606. 14695v1 Announce Type: new Abstract: Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user persona.

By Jinsu Kim, Jihoon Tack, Noah Lee, Jongheon Jeong

llms efficiency benchmarks

arXiv Machine Learning

Jun 15

SpikF-GO: Spiking Fourier Graph Operators for Multivariate Time Series Forecasting

arXiv:2606. 13901v1 Announce Type: new Abstract: Spiking Neural Networks (SNNs) have emerged as an energy-efficient alternative to conventional neural networks, demonstrating strong performance in computer vision and robotics.

By Jafar Bakhshaliyev, Niels Landwehr

rag computer-vision robotics benchmarks

arXiv AI

Jun 15

Succeeding at Scale: Enterprise Retrieval Benchmark Construction and Index-Preserving Query Adaptation for Multi-Tenant Search

arXiv:2601. 04646v4 Announce Type: replace-cross Abstract: Large-scale multi-tenant retrieval systems generate extensive query logs but lack curated relevance labels for effective domain adaptation, resulting in substantial underutilized "dark data.

By Prateek Jain, Shabari S Nair, Ritesh Goru, Prakhar Agarwal, Ajay Yadav, Yoga Sri Varshan Varadharajan, Constantine Caramanis

llms fine-tuning benchmarks

arXiv AI

Jun 15

Numbers Already Carry Their Own Embeddings

arXiv:2606. 14108v1 Announce Type: cross Abstract: We introduce Adelic operation-preserved embeddings (AOE), a training-free representation that captures both a number's real value and its modular (p-adic) signatures.

By Suhyun Bae, Donghun Lee

rag benchmarks

arXiv AI

Jun 15

AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models

arXiv:2606. 14591v1 Announce Type: cross Abstract: Large Audio-Language Models (LALMs) have shown strong performance on a wide range of audio understanding tasks, yet they still struggle with complex audio reasoning.

By Hui Geng, Yi Su, Han Yin, Tianjiao Wan, Qisheng Xu, Jiaxin Chen, Zijian Gao, Hengzhu Liu, Xie Chen, Kele Xu

llms benchmarks

arXiv AI

Jun 15

Hybrid Open-Ended Tri-Evolution Makes Better Deep Researcher

arXiv:2606. 13710v1 Announce Type: new Abstract: Deep research and agent evolution serve as de-facto tasks for AI agents in real-world applications toward artificial general intelligence.

By Hongming Piao, Chi Liu, Mengzhuo Chen, Yan Shu, Derek Li, Ying Wei, Bryan Dai

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 15

Decoupled Latent Optimization of Diffusion Models for Full Waveform Inversion

arXiv:2606. 14139v1 Announce Type: new Abstract: Full waveform inversion (FWI) recovers subsurface velocity from seismic recordings by solving a severely ill-posed, nonconvex PDE-constrained optimization.

By Chen Min, Zheng Ma

diffusion benchmarks

arXiv Machine Learning

Jun 15

The Risk Shadow of Principal Component Analysis: When 99.9999% Variance Preservation Causes Catastrophic Decision Errors

arXiv:2606. 14533v1 Announce Type: new Abstract: Principal Component Analysis (PCA) preserves variance, not the information needed to detect rare catastrophic events.

By Hamidou Tembine

benchmarks

arXiv AI

Jun 15

Vanishing Depth: Training Generalized Depth Adapters with Sinusoidal Depth Preprocessing for Pretrained RGB Encoders

arXiv:2503. 19947v2 Announce Type: replace-cross Abstract: Generalized metric depth understanding is critical for precise vision-guided robotics, which current state-of-the-art (SOTA) vision-encoders do not support.

By Paul Koch, J\"org Kr\"uger

diffusion computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 15

Active Inference for Adaptive Traffic Signal Control in Noisy Nonstationary IoT Environments

arXiv:2606. 13698v1 Announce Type: cross Abstract: Urban traffic signal control at IoT-instrumented intersections must remain effective under sensor occlusion, weather attenuation, and nonstationary demand.

By D\'enes Toth, George Ambroladze, Edwin Sundberg, Ali Beikmohammadi, Alfreds Lapkovskis

benchmarks

arXiv Machine Learning

Jun 15

Arbitrary control over multimode wave propagation for machine learning

arXiv:2402. 17750v2 Announce Type: replace-cross Abstract: Controlled multimode wave propagation can enable more space-efficient photonic processors than architectures based on discrete components connected by single-mode waveguides.

By Tatsuhiro Onodera, Martin M. Stein, Benjamin A. Ash, Mandar M. Sohoni, Melissa Bosch, Ryotatsu Yanagimoto, Marc Jankowski, Timothy P. McKenna, Tianyu Wang, Gennady Shvets, Maxim R. Shcherbakov, Logan G. Wright, Peter L. McMahon

benchmarks

arXiv AI

Jun 15

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

arXiv:2603. 24596v3 Announce Type: replace-cross Abstract: While the shift from cascaded dialogue systems to end-to-end (E2E) speech Large Language Models (LLMs) improves latency and paralinguistic modeling, E2E models often exhibit a significant performance degradation compared to their text-based counterparts.

By Di Cao, Dongjie Fu, Hai Yu, Siqi Zheng, Xu Tan, Tao Jin

llms reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jun 15

A Fixed-Point Neural Operator for Size- and Functional-Transferable Hamiltonian Prediction

arXiv:2606. 14498v1 Announce Type: cross Abstract: Predicting the Kohn-Sham Hamiltonian with machine learning can accelerate density functional theory while retaining access to molecular orbitals, energy levels, and electronic-structure observables that energy-only surrogates cannot resolve.

By Yunhong Lou, Xihang Yue, Xinran Wei, Tianqi Deng, Linchao Zhu

fine-tuning benchmarks

arXiv Machine Learning

Jun 15

CANN-EUCLID: unsupervised constitutive artificial neural network model discovery from full-field data

arXiv:2606. 14565v1 Announce Type: cross Abstract: Constitutive artificial neural networks (CANNs) provide interpretable material model discovery, but have so far been used in stress-supervised settings based on apparent stress-strain data from homogeneous tests.

By Benjamin Alheit, Siddhant Kumar, Mathias Peirlinck

efficiency benchmarks

arXiv Machine Learning

Jun 15

Federated Learning for Feature Generalization with Convex Constraints

arXiv:2606. 14416v1 Announce Type: new Abstract: Federated learning (FL) often struggles with generalization due to heterogeneous client data.

By Dongwon Kim, Donghee Kim, Sung Kuk Shyn, Kwangsu Kim

benchmarks

Get the daily briefing