arXiv Machine Learning June 24, 2026 By Quan Xiao, Yutong Xuan, Gaowen Liu, Ramana Rao Kompella, Tianyi Chen

Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation

Read the original on arXiv Machine Learning →

arXiv:2511. 21056v2 Announce Type: replace Abstract: Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 2

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.

By Jie Hao, Rui Yu, Wei Zhang, Huixia Wang, Jie Xu, Mingrui Liu

llms benchmarks

arXiv AI

Jun 16

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.

By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

llms rag fine-tuning benchmarks safety

arXiv AI

Jul 28

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

arXiv:2607. 22769v1 Announce Type: cross Abstract: The training efficacy of large language models (LLMs) is fundamentally constrained by the quality and composition of training data.

By He Zhang

llms fine-tuning benchmarks data-engineering

arXiv AI

Jun 16

CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

arXiv:2606. 15396v1 Announce Type: cross Abstract: Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns.

By Wenbo Yu, Bohua Wang, Hao Fang, Kuofeng Gao, Jingru Zeng, Xiaochen Yang, Tianyi Zhang, Xiaoxiao Ma, Jiawei Kong, Hao Wu, Bin Chen, Shu-Tao Xia, Min Zhang

llms rag benchmarks safety

arXiv Machine Learning

Jul 24

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.

By Hao Liang, Qifeng Cai, Yibo Lin, Jianzhuo Du, Qifeng Xia, Sizhe Qiu, Linzhuang Sun, Meiyi Qiang, Zhaoyang Han, Xiaochen Ma, Bohan Zeng, Ruichuan An, Conghui He, Wentao Zhang

llms agents fine-tuning benchmarks safety data-engineering

arXiv AI

Aug 11

Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

arXiv:2608. 09351v1 Announce Type: cross Abstract: Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment.

By Nikita Kozodoi, Zainab Afolabi, Jack Butler

llms nlp multimodal

Related stories

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

DomainPilot: Domain-Level Loss-Guided Two-Stage Data Mixture Optimization for Efficient Language Model Fine-Tuning

CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute