Hugging Face Trending Papers August 18, 2026

Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees

Read the original on Hugging Face Trending Papers →

Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is particularly common for subjective, open-ended tasks such as assessing helpfulness or alignment, where no single reference answer exists.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Jun 29

CELEUS: Certifiable and Efficient LLM Evaluation via E-Processes

arXiv:2606. 20820v2 Announce Type: replace Abstract: Can we trust evaluation scores to capture an LLM's true real-world performance?

By Zhijian Zhou, Zesheng Ye, Zhaorun Chen, Bo Li, Feng Liu

llms

arXiv AI

Jul 22

Calibrated Selective Fact-Checking via Evidence Chain Evaluation

arXiv:2607. 18240v1 Announce Type: new Abstract: Large language models (LLMs) can achieve strong fact-checking accuracy, yet forced binary decisions conceal a critical reliability problem: systems may issue confident verdicts even when supporting evidence is weak, sparse, or internally inconsistent.

By Dekun Yang

llms agents safety

arXiv AI

Jul 10

When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

arXiv:2607. 08535v1 Announce Type: cross Abstract: An LLM-as-judge score can move even when the candidate responses stay fixed, simply because the evaluator has changed.

By Zongyou Yang, Yinghan Hou, Xiaokun Yang

llms safety

arXiv AI

5d ago

Reasoning Jury: Multi-Model Consensus for Evaluating Reasoning Traces

arXiv:2608. 12585v1 Announce Type: new Abstract: Improving reasoning LLMs requires the ability to judge the quality of long reasoning traces for effective reasoning data curation, strong training signals during reinforcement learning, and an in-depth understanding of reasoning behaviors during model performance evaluation.

By Congchao Wang, Diwakar Singh, Qiaozi Gao, Spyros Matsoukas, Yang Liu, Mahdi Namazifar

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 27

SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation

arXiv:2605. 03534v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) grounds answers in retrieved passages, yet relevance does not guarantee sufficiency: a topical passage may still fail to justify the answer.

By Jingxi Qiu, Zeyu Han, Cheng Huang

llms rag benchmarks safety

arXiv AI

Jun 15

The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation

arXiv:2606. 13685v1 Announce Type: cross Abstract: LLM-as-a-Judge is now widely used to rank model outputs, train reward models, and populate public leaderboards, but its run-to-run reliability remains under-characterized.

By Abel Yagubyan

llms safety

Related stories

CELEUS: Certifiable and Efficient LLM Evaluation via E-Processes

Calibrated Selective Fact-Checking via Evidence Chain Evaluation

When the Judge Changes, So Does the Measurement: Auditing LLM-as-Judge Reliability

Reasoning Jury: Multi-Model Consensus for Evaluating Reasoning Traces

SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation

The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation