arXiv Machine Learning June 18, 2026 By Zilong Zhang, Yi-Ting Hung, Lei Ding, Chi-Kuang Yeh

Quantifying and Auditing LLM Evaluation via Positive--Unlabeled Learning

Read the original on arXiv Machine Learning →

arXiv:2606. 19057v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used as judges for scalable evaluation, yet such LLM--as--a--Judge systems exhibit systematic biases that are decoupled from semantic quality, most notably verbosity bias.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 19

AURA: Adaptive Uncertainty-aware Refinement for LLM-as-a-Judge Auditing

arXiv:2606. 19714v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as judges for open-ended generation, as large-scale human evaluation is often expensive and difficult to scale, yet their preferences remain imperfect proxies for human judgment.

By Zilong Zhang, Yi-Ting Hung, Weiyi He, Junxi Zhang, Lei Ding, Chi-Kuang Yeh

llms safety

Hugging Face Trending Papers

Jul 2

Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling

Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks. Rubric-based judges offer a transparent way to decompose such judgments into explicit evaluation criteria, but existing annotation-free rubric generators typically rely on a single generic evaluator.

llms reinforcement-learning benchmarks

arXiv AI

Jul 28

Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation

arXiv:2607. 22766v1 Announce Type: cross Abstract: The alignment of Large Language Models (LLMs) is increasingly bottlenecked by data quality.

By Yunting Song, Matthew Watson, Peter Grabowski, Jun Qin

llms reinforcement-learning benchmarks safety data-engineering

arXiv AI

Jun 9

Margin-Adaptive Confidence Ranking for Reliable LLM Judgement

arXiv:2605. 15416v2 Announce Type: replace-cross Abstract: Jung et al.

By Gaojie Jin, Yong Tao, Lijia Yu, Tianjin Huang

llms

arXiv Machine Learning

Jul 1

White-Box Sensitivity Auditing with Steering Vectors

arXiv:2601. 16398v3 Announce Type: replace-cross Abstract: Algorithmic audits are essential tools for examining systems for properties required by regulators or desired by operators.

By Hannah Cyberey, Yangfeng Ji, David Evans

llms safety

arXiv Machine Learning

Jul 3

Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling

arXiv:2607. 01830v1 Announce Type: new Abstract: Reliable reward and preference signals are critical for evaluating and optimizing large language models on open-ended tasks.

By Dazhi Fu, Jiuding Yang, Yiwen Guo, Jicong Fan

llms reinforcement-learning benchmarks

Related stories

AURA: Adaptive Uncertainty-aware Refinement for LLM-as-a-Judge Auditing

Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling

Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation

Margin-Adaptive Confidence Ranking for Reliable LLM Judgement

White-Box Sensitivity Auditing with Steering Vectors

Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling