The study evaluates whether fault localization improves test‑guided code repair by comparing three approaches—blind whole‑solution resampling, spectrum‑based localized infilling, and random‑span infilling—across multiple large language models and benchmarks. Results show that localization is rarely available (only 9.0% of failing candidates), and when it is, localized infilling performs worse than blind resampling, with only suggestive evidence of a benefit over random spans. The findings suggest that targeted edits may not provide a consistent advantage over broader, untargeted repair attempts in current large‑model settings.
arXiv:2607. 26117v1 Announce Type: cross Abstract: Self-repair - returning a failed program to the model together with its test output and asking for a correction - is a standard component of code agents, and is almost always evaluated against a baseline that does not retry at all.
arXiv:2606. 31511v1 Announce Type: cross Abstract: In deployment settings where retraining is infeasible, small frozen code models are routinely asked to repair a failed program after seeing their own failing output, usually treated as a retry mechanism.
arXiv:2606. 31630v1 Announce Type: new Abstract: Language models increasingly write probabilistic programs (in NumPyro, Stan, or Pyro), but a program that compiles, runs, and passes every unit test can still be \emph{statistically} wrong -- a Gaussian likelihood for heavy-tailed data, a Poisson for over-dispersed counts, an invalid prior support, or a pathological parameterization.
The study examines how large language models (LLMs) repair code after runtime or API migrations, revealing that many repaired procedures run without errors yet produce incorrect results. Across 73 tasks and 60 documented changes in CPython, Node.js, and SQLite, the authors found that typical test inputs allowed 41% of wrong repairs to pass, while targeted edge inputs reduced this to 6–7%. Even with additional hard distractors and blind model writers, a significant fraction of incorrect repairs still passed the preregistered tests.
The paper investigates how fine‑tuning large language models with a small number of harmful examples can erode their refusal behavior, and explores whether localizing safety‑related behavior to specific layers or directions can provide robust defenses. Experiments across six checkpoints from four model families show that harmful and benign prompts remain linearly separable after attack, and that patching clean hidden states or freezing layers up to a transition depth can restore refusal. However, attackers can bypass these defenses by spreading updates or targeting singular directions, indicating that adaptive fine‑tuning can defeat localized repairs and highlighting the need for multiple defensive checks.
The paper investigates whether large language models’ reasoning traces truly contain early, informative signals or merely reflect budget and difficulty confounds. Using a restart‑controlled truncation probe, the authors compare continuation success rates against from‑scratch restart curves across 178 problem‑model pairs, finding that only one case shows prefix‑limited success and that continuing a model’s own prefix generally outperforms restarting. A difficulty‑controlled test and two generation‑free analyses reveal that early internal signals do not carry outcome information beyond a problem‑difficulty baseline, underscoring the need for proper counterfactual controls.
arXiv:2611.07478v1 Announce Type: cross Abstract: When large language models do multi-step mathematics, a characteristic failure recurs: the model writes one wrong sign token mid-computation -- a cof...
arXiv:2606. 14530v1 Announce Type: new Abstract: Large language models encode rich information in their hidden states.
The paper investigates whether the tendency of code language models to default to Python when no language is specified is driven by a small, identifiable set of internal components. Using a comprehensive pipeline—including ablation, activation patching, greedy circuit extraction, and significance testing—across 55 model-language configurations, the authors find that this preference is large, grows with model size, and persists after instruction tuning. They also show that the apparent localization of this behavior depends heavily on analysis choices such as component budget and probe design, and that interventions can sometimes alter the model’s output.
arXiv:2609.05881v1 Announce Type: cross Abstract: Developers increasingly run large language models locally by pulling quantized GGUF artifacts from public registries, yet nothing in the distribution...
The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.
arXiv:2607. 12962v1 Announce Type: cross Abstract: Frozen small code LLMs are deployed locally, yet the information guiding a retry after a failed attempt is still measured without placebo controls in the self-repair literature.