The paper investigates the problem of over‑editing by large language models (LLMs) when repairing code, showing that even state‑of‑the‑art models like GPT‑5.5 often rewrite more code than necessary. Using a benchmark of 400 BigCodeBench problems with controlled AST‑level corruptions, the authors quantify over‑editing and demonstrate that a simple preservation instruction can significantly reduce excess edits and cognitive complexity while improving Pass@1. They further explore post‑training strategies, finding that reinforcement learning yields the best balance between edit fidelity and performance retention, thereby establishing edit fidelity as a distinct, measurable dimension of code‑repair quality.
Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program passes. We analyze what this unit-test-only reward t...
arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).
By Yinjie Cheng, Paul Youssef, Christin Seifert, J\"org Schl\"otterer, Zhixue Zhao
arXiv:2610.18755v1 Announce Type: cross
Abstract: Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program p...
By Yunkai Zhan, Miaosen Chai, Shangshang Wang, Jike Zhong, Yuqing Yang, Deqing Fu, Serina Chang, Wang Bill Zhu
arXiv:2610.15894v1 Announce Type: cross
Abstract: Large language model (LLM) code-repair systems fix faulty programs by repeatedly generating and testing revisions. Each revision is costly, making th...
By Elad Sharony, Ido Greenberg, Jialin Song, Shie Mannor, Gal Chechik, Eli Meirom
The paper introduces the Harmful Memorization Index (HMI) to measure when large language models (LLMs) fail to adapt to small semantic changes in coding tasks, producing code that is superficially similar to the original solution. It proposes an automated code‑rewriting pipeline that generates semantically different variants of tasks by altering core logic elements, and uses HMI to combine similarity metrics with performance degradation. Experiments on MBPP+ and BigCodeBench show that HMI does not simply grow with model size, is higher for certain logic‑edit types, and that supervised fine‑tuning can increase HMI while PPO offers a better accuracy–HMI trade‑off.
By Lizhe Zhang, Wentao Chen, Li Zhong, Letian Peng, Zilong Wang, Jingbo Shang