arXiv AI October 28, 2026 By Pingbang Hu, Xueshen Liu, Z. Morley Mao, Jiaqi W. Ma

Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training

The paper introduces Dr. Post-Training, a framework that treats general training data as a data‑induced regularizer to prevent overfitting to scarce target objectives, rather than merely a pool for selection. It constructs a feasible set of model update directions from general data and projects target‑driven updates onto this set, unifying standard training and existing data‑selection methods as special cases along a bias‑variance spectrum. Experiments on SFT, RLHF, and RLVR demonstrate that these methods outperform both standard training and state‑of‑the‑art data‑selection baselines, especially when general and target data are highly misaligned, while system optimizations keep overhead minimal.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 19

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu

llms reinforcement-learning fine-tuning

arXiv AI

Oct 21

The Shape of the DPO Hyperparameter Landscape: What to Measure, How It Moves with Scale, and When Not to Transfer It

arXiv:2610.27895v1 Announce Type: cross Abstract: Direct preference optimisation (DPO) is widely used for language model post-training, but its performance depends strongly on the choice of its hyper...

By Hannan Mahadik, David Salinas, Jacek Golebiowski, Aaron Klein

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 18

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

arXiv:2606. 18650v1 Announce Type: new Abstract: As Large Language Model (LLM) datasets scale to trillions of tokens, data selection has emerged as a critical frontier to filter out uninformative noise and construct adaptive learning trajectories.

By Jiaxing Wang, Deping Xiang, Jin Xu, Zirui Liu, Zicheng Zhang, Guoqiang Gong, Jun Fang, Chao Liu, Pengzhang Liu, Tongxuan Liu, Ke Zhang, Qixia Jiang

llms benchmarks

arXiv AI

Oct 2

Training-Aware Target Coverage for Synthetic Data Selection

arXiv:2610.00814v1 Announce Type: cross Abstract: Synthetic data are increasingly used to scale LLM training, yet more synthetic data do not necessarily produce better models. Useful synthetic data m...

By Yang Ba, Michelle V. Mancenido, Rong Pan

llms fine-tuning

arXiv Machine Learning

Sep 30

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

arXiv:2609.37076v1 Announce Type: new Abstract: Large language models trained on vast corpora inherently risk memorizing harmful content that may later re-emerge in their outputs. To mitigate this is...

By Puning Yang, Qizhou Wang, Junchi Yu, Bo Han, Xiuying Chen

llms

arXiv AI

Jun 16

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.

By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

llms rag fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

The Shape of the DPO Hyperparameter Landscape: What to Measure, How It Moves with Scale, and When Not to Transfer It

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

Training-Aware Target Coverage for Synthetic Data Selection

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning