arXiv AI
Efficient Benchmarking Is Just Feature Selection and Multiple Regression
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
1d ago
Simple Meta-Learning for HPO and CASH Using Performance Representations
The paper proposes a new meta‑learning representation for hyperparameter optimization (HPO) and combined algorithm selection and hyperparameter optimization (CASH) that encodes each configuration by its historical performance across prior tasks, rather than by explicit hyperparameter values. This performance‑based representation can be used with standard surrogate models, such as linear Bayesian ridge regression, and is applied to a diversity‑based re‑ranking criterion (maximal marginal relevance) as a posterior‑free acquisition rule. Experiments on the TabArena benchmark show that this approach outperforms dedicated meta‑learning baselines, achieving comparable tuned performance in about half the number of trials, and remains competitive for single‑model HPO while scaling independently of search‑space dimensionality.
arXiv AI
Jul 14
Coresets Before Score Sets: Evaluation-Unsupervised Prompt Subset Selection for LLM Benchmarks
arXiv:2607. 09739v1 Announce Type: new Abstract: We study LLM benchmark coreset selection: selecting a small subset of prompts over multiple benchmarks whose induced model scores and rankings approximate those obtained from the full benchmark suite.
arXiv Machine Learning
Jun 29
Benchmarking on Tasks That Matter: Dataset Selection for Preserving Model Rankings
arXiv:2606. 27997v1 Announce Type: new Abstract: Benchmarks of machine learning models often include many datasets, making evaluation expensive.
arXiv Machine Learning
Jul 28
An Empirical Study of Feature Selection Granularity
arXiv:2607. 24145v1 Announce Type: new Abstract: Feature selection aims to identify the most informative and relevant features for a given dataset, either in terms of capturing the underlying data structure and distribution better, or with respect to the performance on a downstream task.
arXiv Machine Learning
Jul 22
How Benchmark Prediction from Fewer Data Misses the Mark
arXiv:2506. 07673v2 Announce Type: replace Abstract: Large language model (LLM) evaluation is increasingly costly, prompting interest in methods that speed up evaluation by shrinking benchmark datasets.
arXiv Machine Learning
Jun 2
BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining
arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.