arXiv AI June 6, 2026 By Mary Llewellyn, Isobel Thornton, James Bishop, Annie Gray

Correcting Prompt Dependence in LLM Benchmarks: A Bayesian Hierarchical Model with Embedding-Space Clustering

arXiv:2510. 05709v2 Announce Type: replace-cross Abstract: LLM benchmarking metrics often misstate performance and uncertainty as they rely on two assumptions that frequently do not hold in practice: (i) a sufficient number of evaluations are available for classical inference, and (ii) test prompts are independent.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 7

Robustness Meets Uncertainty: Evidential Adversarial Training for Robust Selective Classification

arXiv:2607. 03075v1 Announce Type: new Abstract: Safety-critical applications require classifiers that are both robust and reliable.

By Nicolas Sournac, Ahmed Baha Ben Jmaa, Bertrand Braeckeveldt

benchmarks safety

arXiv Machine Learning

Jun 15

Minimum Distance Summaries for Robust Neural Posterior Estimation

arXiv:2602. 09161v2 Announce Type: replace-cross Abstract: Simulation-based inference (SBI) enables amortized Bayesian inference by first training a neural posterior estimator (NPE) on prior-simulator pairs, typically through low-dimensional summary statistics, which can then be cheaply reused for fast inference by querying it on new test observations.

By Sherman Khoo, Dennis Prangle, Song Liu, Mark Beaumont

arXiv Machine Learning

Jun 2

A unifying Bayesian framework for adversarial robustness

arXiv:2510. 09288v2 Announce Type: replace-cross Abstract: The vulnerability of machine learning models to adversarial attacks remains a critical societal security challenge.

By Pablo G. Arce, Roi Naveiro, David R\'ios Insua

benchmarks safety

arXiv AI

Jun 3

ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation

arXiv:2604. 23099v2 Announce Type: replace-cross Abstract: Evaluating generative AI models is increasingly resource-intensive due to slow inference, expensive raters, and a rapidly growing landscape of models and benchmarks.

By Yizheng Huang, Wenjun Zeng, Aditi Kumaresan, Zi Wang

fine-tuning benchmarks safety

arXiv Machine Learning

Jun 10

FailureScope: Cross-Regime Behavioral Diagnosis of Language Model Weaknesses

arXiv:2606. 09878v1 Announce Type: new Abstract: Standard benchmarks report aggregate accuracy, but practitioners need to know which specific capabilities a model lacks.

By Nicholas Saban

llms agents multimodal benchmarks safety

arXiv Machine Learning

Jul 30

BayesAME: Bayesian Active Model Evaluation

arXiv:2607. 27023v1 Announce Type: new Abstract: Evaluating large generative models across benchmarks is time-consuming and computationally expensive.

By Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet, Virginia Aglietti, Silvia Chiappa

benchmarks

Related stories

Robustness Meets Uncertainty: Evidential Adversarial Training for Robust Selective Classification

Minimum Distance Summaries for Robust Neural Posterior Estimation

A unifying Bayesian framework for adversarial robustness

ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation

FailureScope: Cross-Regime Behavioral Diagnosis of Language Model Weaknesses

BayesAME: Bayesian Active Model Evaluation