arXiv AI October 27, 2026 By Ou Wu

Data Optimization for LLMs: A Survey

The paper "Data Optimization for LLMs: A Survey" reviews data-related techniques for large language models, noting that while many surveys cover learning strategies and applications, few address data optimization. It outlines a framework that first discusses training and inference data challenges, then introduces dimensions for evaluating data properties—quality, difficulty, diversity, and redundancy—before categorizing existing optimization methods into data selection, augmentation, reweighting, and other strategies. The authors also examine interconnections among these categories and highlight future research directions compared to conventional deep learning approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Oct 1

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

arXiv:2603.26164v2 Announce Type: replace-cross Abstract: Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters...

By Hao Liang, Zhengyang Zhao, Mingrui Chen, Meiyi Qiang, Lu Ma, Rongyi Yu, Hengyi Feng, Shixuan Sun, Zimo Meng, Xiaochen Ma, Xuanlin Yang, Qifeng Cai, Ruichuan An, Bohan Zeng, Zhen Hao Wong, Chengyu Shen, Runming He, Zhaoyang Han, Yaowei Zheng, Fangcheng Fu, Conghui He, Bin Cui, Zhiyu Li, Weinan E, Wentao Zhang

llms rag benchmarks

arXiv AI

Jul 14

Weight-Adjusted Gradients Reveal Parameter Importance and Failure Modes in LLMs

arXiv:2607. 10803v1 Announce Type: cross Abstract: Understanding which parameters are influential in Large Language Models (LLMs) is central to improving their efficiency, reliability, and interpretability.

By Shrestha Datta, Hongfu Liu, Anshuman Chhabra

llms efficiency safety

arXiv Machine Learning

Jul 24

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.

By Hao Liang, Qifeng Cai, Yibo Lin, Jianzhuo Du, Qifeng Xia, Sizhe Qiu, Linzhuang Sun, Meiyi Qiang, Zhaoyang Han, Xiaochen Ma, Bohan Zeng, Ruichuan An, Conghui He, Wentao Zhang

llms agents fine-tuning benchmarks safety data-engineering

arXiv AI

Jul 21

SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

arXiv:2607. 18081v1 Announce Type: cross Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices.

By Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow, Lin Wang

llms nlp fine-tuning efficiency

arXiv AI

Oct 28

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

BoLT is a new benchmark designed to make black-box optimization research for large language model (LLM) tasks more accessible. It presents a suite of realistic LLM optimization problems—covering multi‑fidelity, multi‑objective, heteroscedastic noise, and high‑dimensional search spaces—grounded in thousands of real experiments and represented by lightweight surrogate models. The authors evaluate a wide range of Bayesian and other black‑box optimization methods on BoLT, finding that some BO techniques consistently outperform others and revealing gaps in current methods for LLM optimization.

By Ruth Wan Theng Chew, Zhiliang Chen, Apivich Hemachandra, Bryan Kian Hsiang Low

llms benchmarks

arXiv AI

Jun 16

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning

arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.

By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji

llms rag fine-tuning benchmarks safety

One email a morning, machine-written

Related stories

DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models

Weight-Adjusted Gradients Reveal Parameter Importance and Failure Modes in LLMs

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

Utility-Diversity Aware Online Batch Selection for LLM Supervised Fine-tuning