AutoData: Agentic Search for Pre-training Data Selection

AutoData is an agent that autonomously searches for pre‑training data selection algorithms by exploring a program space of scoring, stratification, and stochastic rules. It iteratively refines these algorithms using validation feedback from a proxy model, discovering feature interactions that outperform existing human‑designed curation pipelines. The resulting selection recipe, found in an overnight search, transfers to larger scales and improves the downstream CORE metric.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• agents
• data-engineering