arXiv Computation and Language

DataFoundry: Evolving Data Preparators via Recursive Self-Improvement

arXiv Machine Learning

Jul 31

How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data

arXiv:2604. 13977v2 Announce Type: replace-cross Abstract: Synthetic data is a standard component in training large language models, yet systematic comparisons across design dimensions, including rephrasing strategy, generator model, and source data, remain absent.

By Joel Niklaus, Atsuki Yamaguchi, Michal \v{S}tef\'anik, Guilherme Penedo, Hynek Kydl\'i\v{c}ek, Elie Bakouch, Lewis Tunstall, Edward Emanuel Beeching, Thibaud Frere, Colin Raffel, Leandro von Werra, Thomas Wolf

llms

arXiv Machine Learning

Jul 24

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.

By Hao Liang, Qifeng Cai, Yibo Lin, Jianzhuo Du, Qifeng Xia, Sizhe Qiu, Linzhuang Sun, Meiyi Qiang, Zhaoyang Han, Xiaochen Ma, Bohan Zeng, Ruichuan An, Conghui He, Wentao Zhang

llms agents fine-tuning benchmarks safety data-engineering

arXiv AI

Jul 28

DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data

arXiv:2607. 24717v1 Announce Type: cross Abstract: Pretraining data processing is critical to the downstream performance of Large Language Models (LLMs).

By Zhen Huang, Yikun Wang, Shijie Xia, Pengfei Liu

llms benchmarks

arXiv AI

Jun 8

DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving

arXiv:2606. 07001v1 Announce Type: cross Abstract: High-quality training data is essential to large language models (LLMs) and typically requires extensive and costly manual curation.

By Chao Deng, Shaolei Zhang, Ju Fan, Xiaoyong Du

llms agents benchmarks data-engineering

arXiv AI

Jun 11

A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

arXiv:2601. 17717v3 Announce Type: replace Abstract: Large Language Models (LLMs) have emerged as powerful tools for generating data across various modalities.

By Kaituo Zhang, Mingzhi Hu, Hoang Anh Duy Le, Fariha Kabir Torsha, Zhimeng Jiang, Minh Khai Bui, Chia-Yuan Chang, Yu-Neng Chuang, Zhen Xiong, Ying Lin, Guanchu Wang, Na Zou

llms

arXiv Machine Learning

Jul 14

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale

arXiv:2601. 22146v2 Announce Type: replace-cross Abstract: Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data.

By Ajay Patel, Colin Raffel, Chris Callison-Burch

llms benchmarks

arXiv AI

Jun 11

APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection

arXiv:2606. 11459v1 Announce Type: cross Abstract: Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.

By Fei Wang, Si Si, Cho-Jui Hsieh, Inderjit S. Dhillon

llms benchmarks

arXiv Machine Learning

Aug 18

Evolving Executable Pipeline Programs for AutoML with Language Models

arXiv:2608. 16416v1 Announce Type: new Abstract: Automated machine learning (AutoML) systems search for pipelines within a space of preprocessing operators, learners, and hyper-parameters specified in advance: they can select and tune known components, but cannot produce structure outside that space.

By Sofoklis Kitharidis, Cor J. Veenman, Jan N. van Rijn, Thomas B\"ack, Niki van Stein

llms benchmarks

arXiv AI

Jun 11

TAHOE: Text-to-SQL with Automated Hint Optimization from Experience

arXiv:2606. 12387v1 Announce Type: cross Abstract: Large Language Models (LLMs) have democratized database access through Text-to-SQL, but moving from prototypes to production remains difficult.

By Zhiyi Chen, Jie Song, Peng Li

llms agents fine-tuning data-engineering

arXiv Machine Learning

Jun 10

CodeAlchemy: Synthetic Code Rewriting at Scale

arXiv:2606. 10087v1 Announce Type: cross Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats.

By Ankit Gupta, Aditya Prasad, Rameswar Panda

llms benchmarks

arXiv AI

Jun 2

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

arXiv:2606. 01286v1 Announce Type: cross Abstract: The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or provide useful training signal.

By Yangzhen Wu, Aaron J. Li, Wenjie Ma, Li Cao, Ziheng Zhou, Mert Cemri, Shu Liu, Yuran Xiu, Chenxiao Yan, Haikun Zhao, Bin Yu, Ion Stoica, Dawn Song

llms reinforcement-learning benchmarks

arXiv AI

Aug 24

PromptResponse: Optimizing Prompts for LLM Coding Tasks

arXiv:2608.21074v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in research workflows and software development pipelines, yet their output remains sensitive to in...

By Erik Thureck, Robert K\"uhnen, Tim Jacobowitz

llms benchmarks safety

Related stories