Stable but Wrong: When Learning Stabilizes Away from the Truth

The paper introduces the concept of Stable but Wrong (SBW), describing situations where a learning process appears stable yet converges to a solution that is systematically biased away from a true objective. Using a minimal strongly convex model, the authors demonstrate that persistent bias in update directions can shift the convergence point from the optimal solution. Experiments across reinforcement learning, supervised learning, and continual fine-tuning of large language models reveal a recurring disconnect between normal optimization behavior and correctness under both static and feedback‑coupled biases, and show that recovery interventions can still modify subsequent learning trajectories.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• reinforcement-learning
• fine-tuning
• safety