arXiv AI July 15, 2026 By Koen Oostermeijer

Accuracy and Normalized Accuracy under Length Bias: Analysis, Guidelines, and a Bayesian Alternative

arXiv:2607. 12767v1 Announce Type: new Abstract: Multiple-choice benchmarks that rank candidate completions by conditional log-probability suffer from a length bias: because log-probabilities sum over tokens, longer answers tend to be penalized relative to shorter ones in practice.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

Hugging Face Trending Papers

Jun 1

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails when the small model has already committed to incorrect reasoning paths. PRM guided search avoids this by scoring candidate continuations during generation, but requires a reward model trained with step-level labels.

llms reinforcement-learning benchmarks safety

arXiv AI

Jun 2

Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answers

arXiv:2602. 05395v2 Announce Type: replace-cross Abstract: A simple strategy for improving LLM accuracy, especially in math and reasoning problems, is to sample multiple responses and submit the answer most consistently reached.

By Jingkai Huang, Will Ma, Zhengyuan Zhou

llms

arXiv AI

Jun 2

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

arXiv:2606. 01682v1 Announce Type: cross Abstract: Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails when the small model has already committed to incorrect reasoning paths.

By Atoosa Chegini, Soheil Feizi

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Aug 11

From token probabilities to calibrated confidence: An empirical study of mathematical question answering

arXiv:2608. 07827v1 Announce Type: new Abstract: Confidence estimation for large language models (LLMs) aims to estimate the probability that a generated answer is correct, while calibration aligns these estimates with empirical accuracy.

By Avery Ma, Lorne Schell, Vin Bhaskara, Leila Pishdad

llms nlp

arXiv Machine Learning

Jul 30

BayesAME: Bayesian Active Model Evaluation

arXiv:2607. 27023v1 Announce Type: new Abstract: Evaluating large generative models across benchmarks is time-consuming and computationally expensive.

By Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet, Virginia Aglietti, Silvia Chiappa

benchmarks

arXiv Machine Learning

Jul 21

BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges

arXiv:2607. 16239v1 Announce Type: new Abstract: AI judges offer a scalable, low-cost alternative to human evaluation, but their outputs can be biased relative to human preferences and highly item-dependent, varying across judges, tasks, and domains.

By Lei Shi, Anlan Zhang, Rita Lyu, Zhengmian Hu, Tong Yu, David Arbour, Avi Feller, Saayan Mitra, Ritwik Sinha

rag safety

Related stories

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answers

Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning

From token probabilities to calibrated confidence: An empirical study of mathematical question answering

BayesAME: Bayesian Active Model Evaluation

BACON: Budgeted Human Calibration for Modeling and Evaluation with Multiple AI Judges