The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules
Read the original on arXiv Machine Learning →The paper investigates how self‑improving large language models (LLMs) can suffer from the winner’s curse when selecting better candidates based on noisy evaluation sets. By modeling correlated errors and empirically testing Qwen models that rewrite their own instructions, the authors find that most proposals after the first are harmful and that the reported gains on the selection set often overstate true held‑out performance. The study shows that larger selection sets can improve held‑out gains on some tasks (e.g., TREC) but not others (e.g., GSM8K), and that common acceptance rules do not outperform simple greedy selection across entire runs.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.