arXiv AI
Probabilistic Multi-Source Data Selection via Learnable Categorical Priors: Variance Reduction and Guaranteed Derandomization
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Jun 18
BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training
arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.
arXiv Machine Learning
2d ago
Towards Faithful Data Selection: Statistical Limits and Scalable Algorithms
The paper investigates the limits of optimal data selection for machine learning, showing that minimizing population error is impossible without a validation set much larger than the training set. It introduces a new concept called faithfulness, providing an estimable upper bound on the performance gap between training on a subset and on the full dataset. The authors propose a swap‑based local search algorithm that iteratively reduces this bound and demonstrate its effectiveness on CIFAR‑10 and CIFAR‑100 datasets.
arXiv Machine Learning
Jun 26
Learning from a Biased Sample
arXiv:2209. 01754v5 Announce Type: replace-cross Abstract: The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed.
arXiv Machine Learning
Jun 2
BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining
arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.
arXiv Machine Learning
Jun 17
Learning from Biased and Costly Data Sources: Minimax-optimal Data Collection under a Budget
arXiv:2602. 17894v2 Announce Type: replace-cross Abstract: Data collection is a critical component of modern statistical and machine learning pipelines, particularly when data must be gathered from multiple heterogeneous sources to study a target population of interest.
arXiv Machine Learning
Sep 11
Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates
arXiv:2604. 13130v2 Announce Type: replace Abstract: We study learning to learn through the lens of hyperparameter tuning.