arXiv Machine Learning June 24, 2026 By Quan Xiao, Yutong Xuan, Gaowen Liu, Ramana Rao Kompella, Tianyi Chen

Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation

Read the original on arXiv Machine Learning →

arXiv:2511. 21056v2 Announce Type: replace Abstract: Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 2

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.

By Jie Hao, Rui Yu, Wei Zhang, Huixia Wang, Jie Xu, Mingrui Liu

llms benchmarks

arXiv AI

Jun 16

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.

By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

llms rag fine-tuning benchmarks safety

arXiv AI

Oct 2

Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

The paper introduces TESS, a scalable data‑selection framework that replaces per‑sample weights with a selection network to improve transferability across datasets and model sizes. It identifies instability in existing meta‑learning for training‑data selection (MTS) due to weight suppression and overreliance on easy features, and proposes a Pointwise Value Matching objective to address these issues. Experiments on large language model safety and instruction tuning show strong transfer from subsets to full corpora and from smaller to larger models.

By Zilin Du, Bowen Yang, Boyang Albert Li

llms fine-tuning safety

arXiv Machine Learning

Sep 30

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

arXiv:2609.37076v1 Announce Type: new Abstract: Large language models trained on vast corpora inherently risk memorizing harmful content that may later re-emerge in their outputs. To mitigate this is...

By Puning Yang, Qizhou Wang, Junchi Yu, Bo Han, Xiuying Chen

llms

arXiv AI

Jul 28

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

arXiv:2607. 22769v1 Announce Type: cross Abstract: The training efficacy of large language models (LLMs) is fundamentally constrained by the quality and composition of training data.

By He Zhang

llms fine-tuning benchmarks data-engineering

arXiv Computation and Language

Oct 1

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

arXiv:2603.26164v2 Announce Type: replace-cross Abstract: Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters...

By Hao Liang, Zhengyang Zhao, Mingrui Chen, Meiyi Qiang, Lu Ma, Rongyi Yu, Hengyi Feng, Shixuan Sun, Zimo Meng, Xiaochen Ma, Xuanlin Yang, Qifeng Cai, Ruichuan An, Bohan Zeng, Zhen Hao Wong, Chengyu Shen, Runming He, Zhaoyang Han, Yaowei Zheng, Fangcheng Fu, Conghui He, Bin Cui, Zhiyu Li, Weinan E, Wentao Zhang

llms rag benchmarks

One email a morning, machine-written

Related stories

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models