arXiv AI

Sequential knowledge editing breaks a model's ability to tell good evidence from bad, without costing it accuracy

The paper investigates how sequential knowledge editing can degrade a language model’s ability to discern reliable evidence from unreliable evidence without affecting overall accuracy. Using a conservatively tuned LoRA on Qwen2.5‑7B‑Instruct, the authors show that after 1,000 edits the model’s arbitration score for untouched facts drops by 36%, leading to higher error rates on its most confident decisions, while MMLU accuracy remains unchanged. The study also finds that in some model‑method combinations, sequential edits can reduce MMLU to chance levels even though edit success and locality remain perfect.

arXiv AI
Sep 17

Suppressed, Not Erased: A Representational Trace of Edited Facts Survives Even Weight-Free Knowledge Editing

The paper investigates whether knowledge editing truly erases original facts from language models. Using a linear trace probe, the authors find that after editing a fact in GPT‑2‑XL, the original object remains highly decodable from hidden states across three different editing methods, even when the model behaves correctly on edited prompts. This suggests that editing suppresses rather than removes the original association in representational space.

By Priyansh Srivastava, Romit Chatterjee
arXiv Computation and Language
Nov 5

Rewrite, Then Veto: Rewriter-Agnostic Edit Gating for LLM-Based Post-OCR Correction

The paper introduces EditVeto, a post‑rewrite veto gate that selectively keeps or reverts blocks of edits made by large language models (LLMs) when correcting noisy OCR transcripts of historical documents. The gate, trained on rewrites from two Qwen3 models, consistently improves correction metrics across five tested rewriters—including unseen ones—by reducing character‑level error rates and boosting user preference scores. In competitive evaluations, EditVeto lowers error rates for most teams and, when combined with fine‑tuning and pooling, achieves performance close to the best published runs, all while relying solely on public models and data.

By Jorge Ram\'on Fonseca Cacho
arXiv AI
Sep 18

Correct Now, Insufficient Later: Auditing Update Sufficiency in Context Compression

The paper investigates how memory systems can answer a current query correctly yet fail to retain distinctions needed for later updates. Using a paired‑history audit, the authors evaluate 24 history pairs across six synthetic mechanisms and two model backends, achieving perfect reveal accuracy on DeepSeek and high accuracy on GLM. Record‑level audits reveal specific failures in structured reveal memories and frontier late‑reference adequacy, and the authors test a label‑equivariant repair that only partially restores correctness.

By Guangzhe Zhang
arXiv AI
Sep 4

Plan Pointers and Record-Directive Form in Budgeted Verification of Inherited Agent Memory

The paper reports on a series of experiments examining how different forms of directives—such as record pointers, criteria, or combinations—affect an agent’s choice of archived source records when it inherits six one-line memories. Across twelve registered studies involving 14,760 attempts on a single instrument lineage, the authors measured the impact of various directive formats on six direct-provider models, nine OpenRouter-served models, and several Claude and Opus 5 models, noting differences in performance metrics and replication outcomes. The results are purely descriptive, detailing the effects of exact edits on fixed panels with registered intervals and no claim of underlying mechanisms.

By Kazuki Nakayashiki
arXiv AI
Aug 26

When Can One Neuron Fix Repetition Loops in LLMs?

The paper investigates whether targeted edits to a few internal components of Gemma 4 instruction‑tuned models can reduce persistent repetition loops that occur during long factual enumeration prompts. By combining per‑layer ablation with per‑neuron attribution, the authors identify specific neurons whose weight edits dramatically lower loop frequency—one sign‑inverted neuron suffices for Gemma 4 E2B. Across all four Gemma variants, loop occurrences drop from 46/384 to 12/384 on held‑out prompts, while general‑purpose benchmarks show no significant regressions. The study also demonstrates that similar sparse edits can mitigate repetition in other families such as Qwen3.5 and LFM2.5, though the effect varies.

By Aristotelis Lazaridis, Aman Sharma, Dylan Bates, Brian King, Vincent Lu, Jack FitzGerald
arXiv AI
3d ago

Where in the Input Does a One-Pass Decision Encoder Look?

The paper evaluates Laya, a small model that processes a multiple‑choice question, its options, and a text in a single 512‑token sequence. Experiments show that when the relevant evidence lies beyond the window, accuracy drops to about 26–27%, and confidence metrics fail to warn of truncation. Fine‑tuning on longer documents improves performance on visible evidence but not on cut evidence, and a larger window or a 312‑token scan can yield modest gains.

By Mohammed Nayeem Khan, Allan Kevin T
arXiv Machine Learning
Sep 25

Return or Revise? Learning When Revision Helps Retrieval-Augmented QA

The paper investigates when it is better to return an existing draft answer or revise it using retrieved evidence in retrieval‑augmented QA systems. By grading both the draft and its candidate revision with the same correctness judge, the authors define a paired effect called recoverability and train policies to predict it before revision. Experiments on 25,870 open‑domain questions show that a recoverability‑based scorer outperforms a draft‑correctness scorer across multiple Llama setups, improving accuracy–revision trade‑offs and closing a significant portion of the oracle gap, though it still applies harmful revisions in a substantial fraction of cases.

By Nicholas Kashani Motlagh, Tim Anderson, Jeremy Gwinnup, Grant Erdmann