arXiv AI
Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 5
A Judge-Aware Ranking Framework for Evaluating Large Language Models without Ground Truth
arXiv:2601. 21817v2 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) on open-ended tasks without ground-truth labels is increasingly done via the LLM-as-a-judge paradigm.
arXiv Machine Learning
Jul 21
Quantifying Ranking Uncertainty in LLM Benchmarks
arXiv:2607. 16259v1 Announce Type: new Abstract: Pretrained models are typically ranked on multi-task leaderboards to assess their effectiveness across diverse tasks.
arXiv AI
Jun 30
Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization
arXiv:2510. 06732v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used as rerankers in information retrieval, yet their ranking behavior can be steered by small, natural-sounding prompts.
arXiv Machine Learning
Jul 31
(Towards) Scalable Reliable Automated Evaluation with Large Language Models
arXiv:2607. 28282v1 Announce Type: cross Abstract: Evaluating the quality and relevance of textual outputs from Large Language Models (LLMs) remains challenging and resource-intensive.
arXiv Machine Learning
Jun 9
Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation
arXiv:2606. 08679v1 Announce Type: cross Abstract: Pretrained models are often evaluated on multi-task leaderboards to measure their applicability in diverse contexts.
arXiv AI
Jul 15
Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks
arXiv:2511. 04689v3 Announce Type: replace-cross Abstract: Evaluating large language models (LLMs) typically requires thousands of benchmark items, making the process expensive, slow, and increasingly impractical at scale.