Hugging Face Trending Papers
Alternative Strategies for Active Learning with Bayesian Bradley Terry Model
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
6d ago
Alternative Strategies for Active Learning with Bayesian Bradley Terry Model
The paper introduces a Bayesian Bradley‑Terry model (BBTM) that employs Polya‑Gamma data augmentation for efficient Gibbs sampling and quantifies uncertainty in latent abilities and ranks. Three active‑learning strategies based on the knowledge‑gradient principle are proposed, using Kendall’s tau distance, posterior rank entropy, and posterior separation as acquisition functions, with approximate one‑step updates to reduce computational cost. The authors also present Posterior Separation Reliability (PSR) as an internal indicator of ranking reliability, and demonstrate through synthetic experiments that BBTM outperforms preference‑based Bayesian comparative judgement, with the entropy‑based strategy excelling at low‑to‑moderate noise and the posterior‑separation strategy at high noise levels.
arXiv AI
Sep 15
Confident Rankings with Fewer Items: Adaptive LLM Evaluation with Continuous Scores
arXiv:2601.13885v2 Announce Type: replace-cross Abstract: Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluatio...
arXiv AI
Jul 29
Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks
arXiv:2607. 25257v1 Announce Type: cross Abstract: Item Response Theory (IRT) has recently been proposed as a framework for evaluating large language model (LLM) benchmarks by separating a model's latent ability from the properties of individual benchmark items.
arXiv Machine Learning
Jul 3
Ask the Right Comparison:Bias-Aware Bayesian Active Top-$k$ Ranking with LLM Judges
arXiv:2607. 02104v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as cheap, scalable judges that compare candidate outputs pairwise -- to rank responses, select models, or triage papers.
arXiv Machine Learning
Jul 30
BayesAME: Bayesian Active Model Evaluation
arXiv:2607. 27023v1 Announce Type: new Abstract: Evaluating large generative models across benchmarks is time-consuming and computationally expensive.
arXiv Machine Learning
Aug 4
Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees
arXiv:2608. 02455v1 Announce Type: new Abstract: Human-centered assessment tasks, which are essential for systematic decision-making, rely heavily on human judgment and typically lack verifiable ground truth.