arXiv AI
Jun 15
Can Editing 1 Neuron Fix Repetition Loops in LLMs?
arXiv:2606. 13705v1 Announce Type: cross Abstract: Yes.
When Can One Neuron Fix Repetition Loops in LLMs?
The paper investigates whether targeted edits to a few internal components of Gemma 4 instruction‑tuned models can reduce persistent repetition loops that occur during long factual enumeration prompts. By combining per‑layer ablation with per‑neuron attribution, the authors identify specific neurons whose weight edits dramatically lower loop frequency—one sign‑inverted neuron suffices for Gemma 4 E2B. Across all four Gemma variants, loop occurrences drop from 46/384 to 12/384 on held‑out prompts, while general‑purpose benchmarks show no significant regressions. The study also demonstrates that similar sparse edits can mitigate repetition in other families such as Qwen3.5 and LFM2.5, though the effect varies.
Related stories
arXiv AI
Jun 30
Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation
arXiv:2511. 05852v4 Announce Type: replace-cross Abstract: Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs).
arXiv Machine Learning
Aug 24
Nothing Changed but the Model: CellFill -- Bounded In-Cell Learning for Bit-Identical, Revocable Updates to Quantized LLMs
arXiv:2608.20873v1 Announce Type: new Abstract: Every way of teaching a deployed language model something new -- full fine-tuning, adapter merging, model editing -- replaces the released checkpoint,...
arXiv AI
2d ago
Instruction Duplication as an Inference-Time Control Primitive
The paper introduces instruction duplication, a simple inference‑time control that repeats the procedural instruction without retraining or decoding changes. Across seven instruction‑tuned models and 16,800 scheduled generations, duplicating the instruction improves deterministic All‑8 diagnostic‑response success from 90.22% to 93.17% and reduces failures by 30.2%. In downstream Answer Engineering scenarios, duplication further boosts success rates, demonstrating its practical impact on systems that rely on the generated trajectory.
arXiv Machine Learning
Aug 5
Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling
arXiv:2608. 03842v1 Announce Type: cross Abstract: When a language model fails on surface-perturbed input (typos, OCR noise, homophones), "which layer is responsible" has three natural operationalizations: where representations diverge most (sensitivity), where restoring clean activations recovers the prediction (causality), and where a small adapter can repair the damage (compensatory capacity) - and we show these three layer maps dissociate.
arXiv AI
Jun 17
Models Take Notes at Prefill: KV Cache Can Be Editable and Composable
arXiv:2606. 17107v1 Announce Type: cross Abstract: Prefix caching reuses prefill only across an exactly shared prefix, so one changed field invalidates the entire downstream cache.
arXiv AI
Jul 8
Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade
arXiv:2607. 06503v1 Announce Type: new Abstract: Large language model (LLM) agents solving multi-step tasks frequently commit to trajectories that are doomed to fail, yet continue to consume substantial inference compute before the failure becomes observable.
arXiv Machine Learning
Jun 16
Mechanistic Analysis of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
arXiv:2601. 18699v2 Announce Type: replace Abstract: Sequential fine-tuning of Large Language Models (LLMs) adaptation to target tasks often triggers catastrophic forgetting, where the acquisition of novel target skills degrades ancestral capabilities.
arXiv AI
Jun 9
Neuron-Anchored Rule Extraction for Large Language Models via Contrastive Hierarchical Ablation
arXiv:2605. 03058v2 Announce Type: replace-cross Abstract: A central goal of explainable AI is to express large language model (LLM) decision logic symbolically and ground it in internal mechanisms.
arXiv AI
Jul 22
Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary
arXiv:2607. 18553v1 Announce Type: cross Abstract: Can a language model read the quality of ongoing computation, and can an external intervention turn that readout into better outcomes?
arXiv AI
Jul 3
Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
arXiv:2607. 01480v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), along with recent selfdistillation variants such as SDPO, evaluates each rollout against a verifier and updates the policy from that episode-level signal.
arXiv AI
Jul 29
Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models
arXiv:2607. 25907v1 Announce Type: cross Abstract: Activation steering controls model behavior by editing internal activations at inference time.