arXiv AI
Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 11
NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMs
arXiv:2608. 08107v1 Announce Type: cross Abstract: Multimodal expansion of large language models (LLMs) enables new perceptual capabilities but often compromises the language intelligence acquired during pretraining.
arXiv AI
Jun 26
Discovering Millions of Interpretable Features with Sparse Autoencoders
arXiv:2606. 26620v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) have emerged as a powerful tool for decomposing superposed language model representations into sparse and interpretable features.
arXiv AI
Jun 29
SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generation
arXiv:2606. 27786v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) enhances LLMs by incorporating external knowledge to support response generation.
arXiv Machine Learning
Sep 10
Retrieval-augmented Decoding for Improving Truthfulness in Open-ended Generation
The paper introduces Retrieval-Augmented Decoding (RAD), a decoding-time method that improves the truthfulness of large language models without retraining. RAD uses a small reference set of up to ten annotated examples to build a grounding space of context embeddings and next-token logits, which it retrieves and aggregates during inference to shape the model’s output. Experiments on four open-ended generation benchmarks and four different LLMs show that RAD consistently outperforms strong baselines and generalizes well across tasks.
arXiv Machine Learning
Sep 24
Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models
The paper investigates how attention dynamics evolve across recurrent depth in language models, finding that attention support stabilizes early while hidden states and outputs take longer. It proposes WISE, a training‑free method that uses full attention in early steps and then reuses the discovered sparse working set for later steps, preserving performance on multi‑hop QA tasks. Experiments show that WISE maintains quality up to 2K context, offers measurable speedups, and highlights the importance of recurrent discovery of attention support.
arXiv AI
Aug 11
NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning
arXiv:2510. 18940v2 Announce Type: replace-cross Abstract: Existing parameter-efficient fine-tuning (PEFT) methods primarily fall into two categories: addition-based and selective in-situ adaptation.