arXiv Machine Learning August 19, 2026 By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Read the original on arXiv Machine Learning →

Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 16

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.

By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

llms rag fine-tuning benchmarks safety

arXiv Machine Learning

Jun 2

PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning

arXiv:2605. 21422v3 Announce Type: replace Abstract: As LLMs continue to scale up, improving training efficiency heavily relies on effective data utilization.

By Qihao Lin, Guanxu Chen, Dongrui Liu, Jing Shao

llms fine-tuning safety

arXiv AI

2d ago

Training-Aware Target Coverage for Synthetic Data Selection

arXiv:2610.00814v1 Announce Type: cross Abstract: Synthetic data are increasingly used to scale LLM training, yet more synthetic data do not necessarily produce better models. Useful synthetic data m...

By Yang Ba, Michelle V. Mancenido, Rong Pan

llms fine-tuning

arXiv AI

Jun 19

Which Pairs to Compare for LLM Post-Training?

arXiv:2606. 19607v1 Announce Type: new Abstract: Preference-based post-training has become a central paradigm for aligning language models.

By Jiangze Han, Vineet Goyal, Will Ma

llms reinforcement-learning benchmarks

arXiv AI

2d ago

Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

The paper introduces TESS, a scalable data‑selection framework that replaces per‑sample weights with a selection network to improve transferability across datasets and model sizes. It identifies instability in existing meta‑learning for training‑data selection (MTS) due to weight suppression and overreliance on easy features, and proposes a Pointwise Value Matching objective to address these issues. Experiments on large language model safety and instruction tuning show strong transfer from subsets to full corpora and from smaller to larger models.

By Zilin Du, Bowen Yang, Boyang Albert Li

llms fine-tuning safety

arXiv Machine Learning

Jun 2

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.

By Jie Hao, Rui Yu, Wei Zhang, Huixia Wang, Jie Xu, Mingrui Liu

llms benchmarks

One email a morning, machine-written

Related stories

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning

Training-Aware Target Coverage for Synthetic Data Selection

Which Pairs to Compare for LLM Post-Training?

Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining