arXiv Machine Learning October 23, 2026 By Ahmed Mehdi Inane, Mahdi Haghifam, Gintare Karolina Dziugaite, Ioannis Mitliagkas

Towards Faithful Data Selection: Statistical Limits and Scalable Algorithms

Read the original on arXiv Machine Learning →

The paper investigates the limits of optimal data selection for machine learning, showing that minimizing population error is impossible without a validation set much larger than the training set. It introduces a new concept called faithfulness, providing an estimable upper bound on the performance gap between training on a subset and on the full dataset. The authors propose a swap‑based local search algorithm that iteratively reduces this bound and demonstrate its effectiveness on CIFAR‑10 and CIFAR‑100 datasets.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Oct 15

Pick the Mixture, Not the Samples: Combinatorial Data Selection for LLMs

arXiv:2610.14028v1 Announce Type: cross Abstract: Data selection for LLM fine-tuning faces two facts about real training: data pools draw from many distinct domains, and evaluation spans diverse task...

By Luyang Zhang, Jialu Wang, Bebei Li

llms fine-tuning

arXiv Machine Learning

Sep 11

Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates

arXiv:2604. 13130v2 Announce Type: replace Abstract: We study learning to learn through the lens of hyperparameter tuning.

By Saumya Goyal, Rohith Rongali, Ritabrata Ray, Barnab\'as P\'oczos

arXiv AI

Oct 15

Probabilistic Multi-Source Data Selection via Learnable Categorical Priors: Variance Reduction and Guaranteed Derandomization

arXiv:2610.14942v1 Announce Type: cross Abstract: Bilevel data selection is a fundamental task with numerous applications in machine learning and statistics. A common strategy for tackling this probl...

By Shizhan Cai, Qixin Zhang, Siyuan Ma, Siyuan Liang, Yongcheng Jing, Dacheng Tao

arXiv Machine Learning

Jun 26

Learning from a Biased Sample

arXiv:2209. 01754v5 Announce Type: replace-cross Abstract: The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed.

By Roshni Sahoo, Lihua Lei, Stefan Wager

safety

arXiv AI

Aug 5

One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning

arXiv:2608. 03249v1 Announce Type: new Abstract: Cold-Start Active Learning (CSAL) aims to select a valuable subset from an unlabeled pool without any prior knowledge or human assistance.

By Ning Zhu, Xiaochuan Ma, Juntao Xu, Jingze Liang, Mengfei Zhao, An Chen, Liang-Jian Deng

benchmarks safety

arXiv Machine Learning

Jun 18

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.

By Jiaxing Wang, Deping Xiang, Jin Xu, Zirui Liu, Zicheng Zhang, Guoqiang Gong, Jun Fang, Chao Liu, Pengzhang Liu, Tongxuan Liu, Ke Zhang, Qixia Jiang

llms benchmarks

One email a morning, machine-written

Related stories

Pick the Mixture, Not the Samples: Combinatorial Data Selection for LLMs

Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates

Probabilistic Multi-Source Data Selection via Learnable Categorical Priors: Variance Reduction and Guaranteed Derandomization

Learning from a Biased Sample

One Knob to Rule Them All: A Unified Optimal Transport View of Cold-Start Active Learning

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training