← Back to all news
arXiv Machine Learning By Shivalika Singh, Andrija Djurisic, Gbemileke Onilude, Sudip Roy, Sara Hooker

Invent a Dataset: Measuring dataset generation abilities with zero seed

Read the original on arXiv Machine Learning →

Invent-A-Dataset is a prompt‑based system that generates realistic, large‑scale datasets from a description, targeting the zero‑data regime where no initial data exists. The authors benchmarked it against five leading model APIs across eight task types and up to 20,000 samples, finding that it outperforms competitors with 17% higher quality and 19% more diverse samples. The diversity advantage grows with dataset size, leading to better downstream training performance and consistently higher rankings for fine‑tuned models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 10

Generating Pretraining Tokens from Organic Data for Data-Bound Scaling

The paper introduces SynPro, a synthetic data generation framework that augments limited organic text for large language model pretraining by applying rephrasing and reformatting operations. SynPro’s generators are optimized with reinforcement learning rewards for quality, faithfulness, and data influence, and are updated continuously as training plateaus. Experiments on 400M, 1.1B, and 2B models show that SynPro can unlock 3.4–5.2× the effective tokens of standard repetition, even outperforming a non‑data‑bound oracle at larger scales.

By Zichun Yu, Chenyan Xiong
llmsreinforcement-learning
More like this →
arXiv Machine Learning
Jul 24

DataPrep-Bench: Benchmarking LLMs as Training Data Preparators

arXiv:2607. 20465v1 Announce Type: new Abstract: The quality of training data fundamentally determines the capabilities of large language models (LLMs), yet no unified benchmark exists to measure how well LLMs, agents, and data-centric workflows actually prepare training data end to end.

By Hao Liang, Qifeng Cai, Yibo Lin, Jianzhuo Du, Qifeng Xia, Sizhe Qiu, Linzhuang Sun, Meiyi Qiang, Zhaoyang Han, Xiaochen Ma, Bohan Zeng, Ruichuan An, Conghui He, Wentao Zhang
llmsagentsfine-tuningbenchmarkssafetydata-engineering
More like this →
arXiv Machine Learning
Aug 19

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu
llmsreinforcement-learningfine-tuning
More like this →