Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

22,941 stories · RSS feed

arXiv Machine Learning

Jun 9

ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning

arXiv:2603. 10823v2 Announce Type: replace-cross Abstract: Deep generative models can help with data scarcity and privacy by producing synthetic training data, but they struggle in low-data, imbalanced tabular settings to fully learn the complex data distribution.

By Xiaofeng Lin, Seungbae Kim, Zhuoya Li, Zachary DeSoto, Charles Fleming, Guang Cheng

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 9

LoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty

arXiv:2503. 18314v5 Announce Type: replace-cross Abstract: We present LoTUS, a novel Machine Unlearning (MU) method that eliminates the influence of training samples from pre-trained models, avoiding retraining from scratch.

By Christoforos N. Spartalis, Theodoros Semertzidis, Petros Daras, Efstratios Gavves

llms benchmarks

arXiv AI

Jun 9

EvoMaster: A Foundational Evolving Agent Framework for Agentic Science at Scale

arXiv:2604. 17406v3 Announce Type: replace Abstract: The convergence of large language models and agents is catalyzing a new era of scientific discovery: Agentic Science.

By Xinyu Zhu, Yuzhu Cai, Zexi Liu, Cheng Wang, Fengyang Li, Wenkai Jin, Wanxu Liu, Zehao Bing, Bingyang Zheng, Jingyi Chai, Shuo Tang, Rui Ye, Yuwen Du, Xianghe Pang, Yaxin Du, Tingjia Miao, Yuzhi Zhang, Ruoxue Liao, Zhaohan Ding, Linfeng Zhang, Yanfeng Wang, Weinan E, Siheng Chen

llms agents benchmarks

arXiv Machine Learning

Jun 9

Anomaly-Preference Image Generation

arXiv:2605. 02439v3 Announce Type: replace-cross Abstract: Synthesizing realistic and diverse anomalous samples from limited data is vital for robust model generalization.

By Fuyun Wang, Yuanzhi Wang, Xu Guo, Sujia Huang, Tong Zhang, Dan Wang, Hui Yan, Xin Liu, Zhen Cui

diffusion benchmarks safety

arXiv AI

Jun 9

Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO

arXiv:2606. 09701v1 Announce Type: cross Abstract: AI red teaming must continually adapt to evolving attackers and defenders.

By Blake Bullwinkel, Eugenia Kim, Amanda Minnich, Mark Russinovich

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 9

Quantum feature-map learning with reduced resource overhead

arXiv:2510. 03389v2 Announce Type: replace-cross Abstract: Current quantum computers require algorithms that use limited resources economically.

By Jonas J\"ager, Philipp Els\"asser, Elham Torabian

efficiency benchmarks

arXiv Machine Learning

Jun 9

RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments

arXiv:2511. 07317v2 Announce Type: replace-cross Abstract: We introduce Reinforcement Learning (RL) with Adaptive Verifiable Environments (RLVE), an approach using verifiable environments that procedurally generate problems and provide algorithmically verifiable rewards, to scale up RL for language models (LMs).

By Zhiyuan Zeng, Hamish Ivison, Yiping Wang, Lifan Yuan, Shuyue Stella Li, Zhuorui Ye, Siting Li, Jacqueline He, Runlong Zhou, Tong Chen, Chenyang Zhao, Yulia Tsvetkov, Simon Shaolei Du, Natasha Jaques, Hao Peng, Pang Wei Koh, Hannaneh Hajishirzi

llms reinforcement-learning benchmarks

arXiv AI

Jun 9

Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery

arXiv:2606. 09672v1 Announce Type: new Abstract: Ask a pretrained biomedical language model whether "cortisol 28 ug/dL" and "stock-market volatility" are related, and it returns a cosine similarity of 0.

By Suraj Biswas, Saurabh Gupta, Pritam Mukherjee

llms rag benchmarks

arXiv AI

Jun 9

DIYHealth Suite: Dataset, Model, and Benchmark for Health Management at Home

arXiv:2606. 07542v1 Announce Type: cross Abstract: Generative AI is reshaping healthcare, yet most existing advances rely on hospital-grade devices, which limits their accessibility and potential for health management outside clinical settings.

By Changshuo Liu, Junran Wu, Zhongle Xie, Wenqiao Zhang, Kaiping Zheng, Jiaqi Zhu, Qingpeng Cai, Ooi Gene Anne, Marcus Chun Jin Tan, Jianwei Yin, James Wei Luen Yip, Beng Chin Ooi

multimodal benchmarks

arXiv AI

Jun 9

A Multi-modal Agentic Co-pilot for Evidence Grounded Computational Pathology

arXiv:2606. 08093v1 Announce Type: new Abstract: Pathology is the cornerstone of modern medicine, where accurate decision-making relies heavily on evidence-based practices.

By Zhe Xu, Zhengyu Zhang, Zhiyuan Cai, Jiahao Xu, Yijie Lin, Ziyi Liu, Junlin Hou, Hongyi Wang, Yuxiang Nie, Ling Liang, Yihui Wang, Yingxue Xu, Ronald Cheong Kin Chan, Li Liang, Hao Chen

agents multimodal benchmarks

arXiv Machine Learning

Jun 9

Solving Inverse Problems with Flow-based Models via Model Predictive Control

arXiv:2601. 23231v2 Announce Type: replace-cross Abstract: Flow-based generative models provide strong unconditional priors for inverse problems, but guiding their dynamics for conditional generation remains challenging.

By George Webber, Alexander Denker, Riccardo Barbano, Andrew J Reader

benchmarks

arXiv AI

Jun 9

Cosmo3DFlow: Wavelet Flow Matching for Spatial-to-Spectral Compression in Reconstructing the Early Universe

arXiv:2602. 10172v2 Announce Type: replace-cross Abstract: Reconstructing the early universe from the evolved present-day universe is a challenging and computationally demanding problem in modern astrophysics.

By Md. Khairul Islam, Zeyu Xia, Ryan Goudjil, Jialu Wang, Arya Farahi, Judy Fox

diffusion efficiency benchmarks

arXiv Machine Learning

Jun 9

Optimizing Few-Step Generation with Adaptive Matching Distillation

arXiv:2602. 07345v2 Announce Type: replace-cross Abstract: Distribution Matching Distillation (DMD) is a powerful acceleration paradigm, yet its stability is often compromised in Forbidden Zone, regions where the real teacher provides unreliable guidance while the fake teacher exerts insufficient repulsive force.

By Lichen Bai, Zikai Zhou, Shitong Shao, Wenliang Zhong, Shuo Yang, Shuo Chen, Bojun Chen, Zeke Xie

diffusion efficiency benchmarks

arXiv AI

Jun 9

DSFNet: Learning Dual-Domain Spectral Operators for Multi-Modality Spatio-Temporal Forecasting in Urban Transportation Systems

arXiv:2606. 07695v1 Announce Type: cross Abstract: Multi-Modality Spatio-Temporal Forecasting (MoSTF) extends traditional spatio-temporal forecasting by incorporating diverse traffic modalities.

By Yongchao Li, Yang Li, Zhuoxuan Li, Jun Chen, Chu Zhang, Jinde Cao, Leszek Rutkowski

benchmarks

arXiv AI

Jun 9

FunctionEvolve: Structure-Guided Symbolic Regression with LLMs

arXiv:2606. 07704v1 Announce Type: cross Abstract: Symbolic regression aims to uncover explicit scientific laws from data.

By Zeyu Xia, Jun Zhu, Dong Yan

llms benchmarks

arXiv Machine Learning

Jun 9

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

arXiv:2602. 16346v4 Announce Type: replace-cross Abstract: LLM-based agents execute real-world workflows via tools and memory.

By Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut

llms agents benchmarks safety

arXiv AI

Jun 9

BSTabDiff: Block-Subunit Diffusion Priors for High-Dimensional Tabular Data Generation

arXiv:2606. 09257v1 Announce Type: cross Abstract: High-Dimensional Low-Sample Size (HDLSS) tabular domains (e.

By Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Gyawali, Gianfranco Doretto, Donald A. Adjeroh

diffusion benchmarks

arXiv AI

Jun 9

Developing Distance-Aware Physics-Constrained Probabilistic Frameworks for Industrial Prognostics

arXiv:2512. 08499v3 Announce Type: replace-cross Abstract: Development of reliable and physically interpretable probabilistic frameworks for industrial prognostics remain nascent, and existing literature is often insensitive as inputs move away from the training manifold.

By Waleed Razzaq, Yun-Bo Zhao

diffusion benchmarks safety

arXiv Machine Learning

Jun 9

SkillHone: A Harness for Continual Agent Skill Evolution Through Persistent Decision History

arXiv:2606. 08671v1 Announce Type: new Abstract: Agent skills extend language-model agents with task-specific procedures, scripts, and references, but the tasks and environments they target continually change.

By Zhiwei Li, Yong Hu

agents benchmarks

arXiv Machine Learning

Jun 9

Cutting LLM Evaluation Costs with SySRs: A Bandit Algorithm that Provably Exploits Model Similarity

arXiv:2606. 07726v1 Announce Type: new Abstract: Large Language Models are typically benchmarked by evaluating every model on every test query.

By Zifan Lyu, Chahine Nejma, Tobias Wegel, Fanny Yang, Florian E. Dorner

llms reinforcement-learning benchmarks

Get the daily briefing