The paper investigates the limits of optimal data selection for machine learning, showing that minimizing population error is impossible without a validation set much larger than the training set. It introduces a new concept called faithfulness, providing an estimable upper bound on the performance gap between training on a subset and on the full dataset. The authors propose a swap‑based local search algorithm that iteratively reduces this bound and demonstrate its effectiveness on CIFAR‑10 and CIFAR‑100 datasets.
arXiv:2610.14028v1 Announce Type: cross Abstract: Data selection for LLM fine-tuning faces two facts about real training: data pools draw from many distinct domains, and evaluation spans diverse task...
arXiv:2604. 13130v2 Announce Type: replace Abstract: We study learning to learn through the lens of hyperparameter tuning.
arXiv:2610.14942v1 Announce Type: cross Abstract: Bilevel data selection is a fundamental task with numerous applications in machine learning and statistics. A common strategy for tackling this probl...
arXiv:2209. 01754v5 Announce Type: replace-cross Abstract: The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed.
arXiv:2608. 03249v1 Announce Type: new Abstract: Cold-Start Active Learning (CSAL) aims to select a valuable subset from an unlabeled pool without any prior knowledge or human assistance.
arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.
The paper evaluates coreset selection methods by incorporating both selection and training time into a unified wall‑clock budget, using a standardized benchmark across four datasets and multiple selectors. Across numerous budget anchors, simple random or full‑data training consistently outperforms sophisticated selectors, and selection costs are dominated by a full‑dataset scan that cannot be amortized. The study also identifies when subset reuse can justify selection and reports several correctness fixes in a popular codebase.
arXiv:2606. 11761v1 Announce Type: new Abstract: Dynamic data pruning techniques aim to reduce computational cost while minimizing information loss by periodically selecting representative subsets of input data during model training.
arXiv:2607. 05903v1 Announce Type: cross Abstract: We present K-ABENA (K-Adaptive Backpropagation with Error-based N-exclusion Algorithm), a selective gradient computation framework that reduces per-iteration training cost by excluding a fraction of low-loss ("minor") observations from the backward pass.
arXiv:2610.10347v1 Announce Type: cross Abstract: Dataset pruning reduces a large training set to a representative subset while preserving model performance. Existing geometry-based methods typically...
arXiv:2610.03702v1 Announce Type: new Abstract: The choice of post-training data for large language models substantially affects downstream performance. Gradient-based data selection is a popular app...
arXiv:2610.14996v1 Announce Type: new Abstract: Choosing a few robust algorithms or schedules can require evaluating every candidate under many adverse scenarios, even though an agent only needs to i...