The paper investigates the problem of over‑editing by large language models when repairing code, showing that even state‑of‑the‑art models like GPT‑5.5 frequently rewrite more code than necessary. Using a benchmark of 400 BigCodeBench problems with controlled AST corruptions, the authors quantify excess edits and demonstrate that a simple preservation instruction can reduce unnecessary changes and improve pass rates. They further explore training strategies, finding that reinforcement learning yields the best balance between edit fidelity and performance retention, highlighting edit fidelity as a distinct, measurable dimension of code‑repair quality.
By Tongyao Zhu, Wei Hern Lim, Min-Yen Kan
Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program passes. We analyze what this unit-test-only reward t...
arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).
By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao
arXiv:2610.18755v1 Announce Type: cross
Abstract: Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program p...
By Yunkai Zhan, Miaosen Chai, Shangshang Wang, Jike Zhong, Yuqing Yang, Deqing Fu, Serina Chang, Wang Bill Zhu
arXiv:2610.15894v1 Announce Type: cross
Abstract: Large language model (LLM) code-repair systems fix faulty programs by repeatedly generating and testing revisions. Each revision is costly, making th...
By Elad Sharony, Ido Greenberg, Jialin Song, Shie Mannor, Gal Chechik, Eli Meirom
The paper introduces the Harmful Memorization Index (HMI) to measure when large language models (LLMs) fail to adapt to small semantic changes in coding tasks, producing code that is superficially similar to the original solution. It proposes an automated code‑rewriting pipeline that generates semantically different variants of tasks by altering core logic elements, and uses HMI to combine similarity metrics with performance degradation. Experiments on MBPP+ and BigCodeBench show that HMI does not simply grow with model size, is higher for certain logic‑edit types, and that supervised fine‑tuning can increase HMI while PPO offers a better accuracy–HMI trade‑off.
By Lizhe Zhang, Wentao Chen, Li Zhong, Letian Peng, Zilong Wang, Jingbo Shang