arXiv Machine Learning
Anticipatory Evaluation of Language Models: Predicting LLM Benchmark Scores from Descriptions
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computation and Language
5d ago
How to Evaluate When You Don't Know How to Evaluate? Choosing Reliable Setups for Base Language Model Benchmarking
arXiv:2610.19185v1 Announce Type: new Abstract: Evaluation of large language models (LLMs) remains largely pre-scientific: it involves many design decisions, with little guidance on how to make them....
arXiv Machine Learning
Jul 22
How Benchmark Prediction from Fewer Data Misses the Mark
arXiv:2506. 07673v2 Announce Type: replace Abstract: Large language model (LLM) evaluation is increasingly costly, prompting interest in methods that speed up evaluation by shrinking benchmark datasets.
arXiv AI
Aug 3
Benchmarks Are Not Monolithic: Sample-Level Auditing and Orchestration for LLM Evaluation
arXiv:2607. 28801v1 Announce Type: cross Abstract: Benchmark datasets are central to evaluating Large Language Models (LLMs), yet they are typically conceived as monolithic tasks, obscuring substantial variation in the demands of individual samples.
arXiv AI
Sep 24
Uncheatable Eval: Dynamic Compression-Based Evaluation of Language Models
arXiv:2609.27510v1 Announce Type: cross Abstract: Modern large language models are pretrained on massive datasets, making it difficult to prevent benchmark data from entering their training sets and...
Hugging Face Trending Papers
6d ago
How to Evaluate When You Don't Know How to Evaluate? Choosing Reliable Setups for Base Language Model Benchmarking
Evaluation of large language models (LLMs) remains largely pre-scientific: it involves many design decisions, with little guidance on how to make them. These choices can substantially affect measured...
Hugging Face Trending Papers
Sep 23
Uncheatable Eval: Dynamic Compression-Based Evaluation of Language Models
Modern large language models are pretrained on massive datasets, making it difficult to prevent benchmark data from entering their training sets and undermining the reliability of evaluation results....