← Back to all news
arXiv Machine Learning By Hedi Driss, Evan Frick, Anastasios N. Angelopoulos, Cho-Jui Hsieh, Yejin Huh, Tijana Zrnic, Andrew Ilyas

High-Value Training Data from Noisy, Diverse Sources

Read the original on arXiv Machine Learning →

The paper presents a scalable method for learning dataweights that optimizes training-example importance using metagradients, guided by a small high‑quality target dataset. It introduces parametric weighting rules that generalize to new and streaming data and can transfer from smaller to larger models. Applied to reward modeling, the learned weights improve held‑out performance, adapt models to specialized domains, and enable human‑synthetic supervision mixtures to outperform either source alone.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 5

Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards

The paper presents a post‑training approach for text‑to‑image models that combines a preference reward, trained on large human preference data, with rubric‑based rewards that assess prompt faithfulness and other desirable traits. The authors show that a simple reward composition strategy outperforms a naive weighted average, leading to significant Elo gains on the Arena leaderboard for models like Flux2dev and Ideogram‑4. They also release Arena‑T2I‑Training, a 1K subset of data to aid reproducible research in post‑training.

By Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos, Wei-Lin Chiang, Ion Stoica, Cho-Jui Hsieh
diffusionreinforcement-learningbenchmarks
More like this →
arXiv AI
Oct 2

Scalable, Transferable Meta-network for Data Selection Requires a Different Loss (and Why the Obvious Choice is Problematic)

The paper introduces TESS, a scalable data‑selection framework that replaces per‑sample weights with a selection network to improve transferability across datasets and model sizes. It identifies instability in existing meta‑learning for training‑data selection (MTS) due to weight suppression and overreliance on easy features, and proposes a Pointwise Value Matching objective to address these issues. Experiments on large language model safety and instruction tuning show strong transfer from subsets to full corpora and from smaller to larger models.

By Zilin Du, Bowen Yang, Boyang Albert Li
llmsfine-tuningsafety
More like this →