← Back to all news
arXiv Machine Learning October 21, 2026 By Motti Goldberger, Nils Rudi

Selecting the Best Few with Human Evaluators: Adaptively Combining Ratings and Pairwise Comparisons

Read the original on arXiv Machine Learning →

The Flow has not summarised this story yet — read it at arXiv Machine Learning.

  • llms
  • benchmarks

One email a morning, machine-written

One email a day, machine-written, one click to leave. We never share your address.

Related stories

Hugging Face Trending Papers
5d ago

Selecting the Best Few with Human Evaluators: Adaptively Combining Ratings and Pairwise Comparisons

This paper considers a decision-maker tasked with identifying the $k$ best items from a large pool of candidates, such as job applicants, LLMs, award nominees, or new product ideas. The decision-maker...

llmsbenchmarks
More like this →
arXiv AI
Oct 1

Optimal Design for Active Preference Learning with Biased LLM Judges

arXiv:2609.38860v1 Announce Type: cross Abstract: Learning from human preferences is central to large language model (LLM) alignment, but human preference annotation is costly. Active preference lear...

By Zhongman Du, Huiming Zhang, Haodong Zhu, Baochang Zhang
llmssafety
More like this →
arXiv AI
Jun 16

Surprise-Guided MergeSort: Budget-Efficient Human-in-the-Loop Ranking via Adaptive Comparison Scheduling

arXiv:2606. 15623v1 Announce Type: cross Abstract: Pairwise comparison is the gold standard for subjective ranking tasks; however, exhaustive annotation requires a massive number of human comparisons ($O(n^2)$).

By Yujin Park, Haejun Chung, Ikbeom Jang
llmsmultimodalbenchmarkssafety
More like this →
arXiv Machine Learning
Jun 18

QUIVER: Cost-Aware Adaptive Preference Querying in Surrogate-Assisted Evolutionary Multi-Objective Optimization

arXiv:2605. 04267v2 Announce Type: replace Abstract: Interactive multi-objective optimization systems face a budget allocation dilemma: one can spend resources on expensive objective evaluations or on eliciting decision-maker preferences that identify the relevant region of the Pareto set.

By Florian A. D. Burnat
benchmarks
More like this →
arXiv AI
Oct 15

Pick the Mixture, Not the Samples: Combinatorial Data Selection for LLMs

arXiv:2610.14028v1 Announce Type: cross Abstract: Data selection for LLM fine-tuning faces two facts about real training: data pools draw from many distinct domains, and evaluation spans diverse task...

By Luyang Zhang, Jialu Wang, Bebei Li
llmsfine-tuning
More like this →
arXiv AI
Jul 29

Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization

arXiv:2607. 25136v1 Announce Type: new Abstract: Research on preference optimization often varies the training objective while holding the data fixed.

By Zhengtao Yao, Runhao Li, Xupeng Chen, Jiayi Cheng, Chenqian Le, Michael Yue, Siheng Wang, Haoyan Xu, Yuqi Li, Chenhao Wei, Zhengdao Li, Rongchao Zhang, Guang Yang, Yidong Wang, Junhao Dong
llmssafety
More like this →
About Pricing API Newsletter Sources Privacy Terms Refunds Accessibility Provider info Contact RSS

The Flow links to publishers and never republishes their articles. Summaries are machine-generated.

v1.1.0 · 5f852ea