arXiv AI July 28, 2026 By Zhen Huang, Yikun Wang, Shijie Xia, Pengfei Liu

DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data

arXiv:2607. 24717v1 Announce Type: cross Abstract: Pretraining data processing is critical to the downstream performance of Large Language Models (LLMs).

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 8

DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving

arXiv:2606. 07001v1 Announce Type: cross Abstract: High-quality training data is essential to large language models (LLMs) and typically requires extensive and costly manual curation.

By Chao Deng, Shaolei Zhang, Ju Fan, Xiaoyong Du

llms agents benchmarks data-engineering

arXiv Machine Learning

Jun 10

CodeAlchemy: Synthetic Code Rewriting at Scale

arXiv:2606. 10087v1 Announce Type: cross Abstract: Pre-training on raw code teaches syntax but provides sparse signal for diverse real-world task formats.

By Ankit Gupta, Aditya Prasad, Rameswar Panda

llms benchmarks

arXiv AI

Jul 10

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

arXiv:2607. 08646v1 Announce Type: cross Abstract: As available training data approaches its physical limit, gains from Scaling Laws have begun to diminish.

By Xinlong Zhao, Dongsheng Liu, Hengyu Zhao, Zixuan Fu, Zheng Wang, Jie Cai, Jie Zhou, Qiang Ma, Xuanhe Zhou, Xu Han, Yudong Wang, Zhiyuan Liu

llms safety

arXiv Computation and Language

Sep 1

DataFoundry: Evolving Data Preparators via Recursive Self-Improvement

arXiv:2608.29966v1 Announce Type: new Abstract: Domain adaptation of large language models increasingly depends on constructing high-quality training data, yet existing data-preparation pipelines typ...

By Cehao Yang, Xiaojun Wu, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Hui Xiong, Jian Guo

llms fine-tuning

arXiv Machine Learning

Jul 10

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

arXiv:2607. 07748v1 Announce Type: new Abstract: Large Language Models achieve strong code generation for high resource languages like Python and Java but suffer sharp performance drops on Low-Resource Programming Languages~(LRPLs) such as Julia.

By Didula Samaraweera, Anjana Supun, Srinath Perera

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 8

Small Language Model Agents Enable Efficient and High-Quality Knowledge Mining

arXiv:2510. 01427v3 Announce Type: replace Abstract: At the core of Deep Research is knowledge mining, the task of extracting structured information from massive unstructured text in response to user instructions.

By Sipeng Zhang, Shuhuai Lin, Xinpeng Wei, Yihang Chen, Pin Qian, Su Wang, Huan Xu

llms agents benchmarks

One email a morning, machine-written

Related stories

DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving

CodeAlchemy: Synthetic Code Rewriting at Scale

UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing

DataFoundry: Evolving Data Preparators via Recursive Self-Improvement

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

Small Language Model Agents Enable Efficient and High-Quality Knowledge Mining