SCoNE (Selective Context-aware Neuron Editing) is a training‑free method that enhances Retrieval‑Augmented Generation (RAG) by selectively strengthening context‑aware feed‑forward network neurons identified through high attribution and high cross‑input variability. It requires only a few mining samples, no fine‑tuning, and incurs no inference‑time overhead. Experiments on multiple knowledge‑intensive question‑answering benchmarks and two LLM backbones show that SCoNE consistently outperforms competitive baseline methods.
arXiv:2604.02194v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Language Models (RALMs) have shown strong potential in knowledge-intensive tasks, yet they remain vulnerable when retriev...
arXiv:2606. 27786v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) enhances LLMs by incorporating external knowledge to support response generation.
arXiv:2606. 19679v1 Announce Type: cross Abstract: Lifelong knowledge editing aims to efficiently and sequentially update language models over time, as new knowledge becomes available or when the model makes mistakes, while preserving acceptable performance on past knowledge.
arXiv:2610.20399v1 Announce Type: cross Abstract: Context can improve large language model performance, but these gains may be lost after context removal. Inspired by biological systems consolidation...
InComeS is a framework that improves large language models (LLMs) for model editing by compressing editing contexts into a key‑value cache associated with a special gist token. It adds cross‑attention modules to dynamically select the most relevant information from these gist pools, allowing efficient handling of multiple edits beyond the LLM’s context window. Experiments on diverse editing benchmarks show that InComeS enhances both effectiveness and efficiency compared to existing methods.
arXiv:2507. 15336v3 Announce Type: replace-cross Abstract: Designing high-performance neural networks for new tasks requires balancing optimization quality with search efficiency.
AutoIndexer is a generative model that trains causal models on chains of edits, enabling flexible post‑generation modifications such as token insertions, substitutions, and deletions. It introduces marker tokens for opening and closing edits and moves a cursor through the token sequence, mimicking a text editor. By perturbing ground‑truth labels with inverse edit operations during training, AutoIndexer becomes robust to erroneous context and can perform editing tasks like code correction while maintaining performance on standard LLM benchmarks.
The paper introduces FOVEATED, a plug‑and‑play framework that improves atomic‑fact recall in unstructured knowledge editing (UKE) for large language models. By randomly shifting Rotary Position Embedding (RoPE) positions during editing, FOVEATED creates focused views of each sentence, counteracting the context‑reliance problem where edited LLMs reproduce passages but fail to recall individual facts. Experiments show consistent gains across five editors, two LLM backbones, and three benchmarks.
The paper introduces INTRA, an attention-based encoder-decoder framework that retrieves directly from its own internal representations instead of using an external retriever. By having decoder attention query pre-encoded evidence chunks, INTRA unifies retrieval and generation, eliminating the typical mismatch seen in retrieval-augmented generation pipelines. Experiments on question-answering benchmarks show that INTRA outperforms strong engineered retrieval pipelines in both evidence recall and overall answer quality.
arXiv:2511. 20892v4 Announce Type: replace Abstract: Large language models (LLMs) often produce incorrect or outdated content after being employed.
arXiv:2605.16893v2 Announce Type: replace Abstract: Recent studies have introduced memory-augmented architectures that decouple knowledge storage from neural computation, enabling more direct access...
arXiv:2610.08463v1 Announce Type: cross Abstract: Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an...