Hugging Face Trending Papers June 25, 2026

Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

Read the original on Hugging Face Trending Papers →

Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug. We propose BINEVAL, a framework that decomposes evaluation criteria into atomic binary questions and aggregates the resulting verdicts into interpretable, multi-dimensional scores.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI

Jun 26

Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

arXiv:2606. 27226v1 Announce Type: new Abstract: Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with human judgments on open-ended generation, and holistic LLM judges often produce opaque scores that are hard to debug.

By Sangwoo Cho, Kushal Chawla, Pengshan Cai, Zefang Liu, Chenyang Zhu, Shi-Xiong Zhang, Sambit Sahu

llms nlp benchmarks

arXiv AI

Sep 10

CriticGen: Generation-Aware Evaluation as Actionable Feedback

CriticGen introduces a generation‑aware evaluation framework that generates sample‑specific evaluation dimensions and scoring criteria across categories such as subjective, objective, and self‑derived constraints. These dynamic rubrics produce a score, reason, executable refinement suggestion, and a refined answer, enabling models to diagnose and target flaws in their responses. Experiments show significant gains in rubric quality, score correlation, and actionable feedback, with 73.17% of answers improved and a 93.28% non‑degradation rate.

By Huifang Du, Zecheng Zuo, Sen Wang, Chenghao Fan, Haofen Wang, Yehui Yang

llms

arXiv AI

Aug 21

Qworld: Question-Specific Evaluation Criteria for LLMs

arXiv:2603. 23522v2 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) on open-ended questions is difficult because response quality depends on the question's context.

By Shanghua Gao, Yuchang Su, Pengwei Sui, Curtis Ginder, Marinka Zitnik

llms

arXiv AI

Jul 1

SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA

arXiv:2504. 07385v3 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) become increasingly used for question-answering (QA), relying on static, pre-annotated references for evaluation poses significant challenges in cost, scalability, and completeness.

By Sher Badshah, Ali Emami, Hassan Sajjad

llms agents benchmarks

arXiv Computation and Language

Sep 22

LLJ Cards: Best practices for the Use of LLMs as Judges

arXiv:2609.24516v1 Announce Type: new Abstract: In recent years, large language models (LLMs) have emerged as a popular alternative for evaluation. Often referred to as LLMs as judges (LLJs), these s...

By Khaoula Chehbouni, Melina Medjdoub, Florian Carichon, Golnoosh Farnadi, Jackie Chi Kit Cheung

llms safety

arXiv Machine Learning

Jun 8

Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios

arXiv:2606. 06546v1 Announce Type: new Abstract: Evaluating large language models (LLMs) for education requires measuring how models teach, not only what they know.

By Tao Liu, Ye Lu, Ruohua Zhang, Siyu Song, Wentao Liu, Aimin Zhou, Hao Hao

llms agents benchmarks safety

One email a morning, machine-written

Related stories

Ask, Don't Judge: Binary Questions for Interpretable LLM Evaluation and Self-Improvement

CriticGen: Generation-Aware Evaluation as Actionable Feedback

Qworld: Question-Specific Evaluation Criteria for LLMs

SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA

LLJ Cards: Best practices for the Use of LLMs as Judges

Elmes*: Automated Construction of Fine-Grained Evaluation Rubrics for Large Language Models in Long-Tail Educational Scenarios