arXiv Computation and Language September 24, 2026 By Miaohe Niu, Runsong Zhao, Xinyu Liu, Bo Jin, Yucheng Qiao, Chunliang Zhang, Jingbo Zhu, Tong Xiao

Complementary Roles of Activation and Parametric Memory in Few-Shot Learning

Read the original on arXiv Computation and Language →

The paper investigates how large language models use activation memory (KV caches) and parametric memory (updated parameters) during few‑shot learning. Experiments show activation memory excels at factual recall, while parametric memory does not consistently outperform it for task learning. The composite task Conditional Arithmetic requires both memory types, with neuron‑level analysis revealing distinct neuron sets activated by each memory, and their combined use is essential for success.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

llms

arXiv Machine Learning

Jun 2

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models

arXiv:2602. 23197v2 Announce Type: replace-cross Abstract: Transformer-based large language models exhibit in-context learning, enabling adaptation to downstream tasks via few-shot prompting with demonstrations.

By Chungpa Lee, Jy-yong Sohn, Kangwook Lee

llms fine-tuning

arXiv AI

Jul 3

Neuron-Aware Active Few-Shot Learning for LLMs

arXiv:2607. 02423v1 Announce Type: cross Abstract: Active Few-Shot Learning (AFSL) adapts LLMs to specialized domains by identifying the most valuable unlabeled samples for annotation and use as few-shot demonstrations, effectively reducing human annotation costs while promoting high performance.

By Zhuowei Chen, Liwei Chen, Christian Schunn, Raquel Coelho, Xiang Lorraine Li

llms rag nlp benchmarks

arXiv AI

Aug 5

Mechanism of Task-oriented Information Removal in In-context Learning

arXiv:2509. 21012v4 Announce Type: replace-cross Abstract: In-context Learning (ICL) is an emerging few-shot learning paradigm based on modern Language Models (LMs), yet its inner mechanism remains unclear.

By Hakaze Cho, Haolin Yang, Gouki Minegishi, Naoya Inoue

llms benchmarks

arXiv AI

Jun 6

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

arXiv:2606. 05644v1 Announce Type: new Abstract: When retrieved evidence contradicts parametric memory, language models frequently ignore context and default to memorized priors -- a failure that undermines the core purpose of retrieval augmentation.

By Zhe Yu, Wenpeng Xing, Tiancheng Zhao, Mohan Li, Changting Lin, Meng Han

llms rag benchmarks safety

arXiv Computation and Language

Sep 23

MemoryAthena: Adaptive Routing over Latent and Generated Memories

arXiv:2609.25853v1 Announce Type: new Abstract: Learned-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be...

By Mingyuan Li, Guangsheng Yu, Juyuan Zhang, Xu Wang, Zhibo Man, Haonan Zhang, Shaoxiong Ji

nlp

arXiv Machine Learning

Sep 10

Continual Learning Mechanisms Compose for Long-Horizon Memorization

arXiv:2609.06986v1 Announce Type: new Abstract: Language models may need to internalize information that arrives over time and retain it through many subsequent updates. To study this challenge, we i...

By Zheyuan Zhang, Alvin Zhang, Daniel Khashabi, Tianmin Shu

llms fine-tuning

One email a morning, machine-written

Related stories

Fine-Tuning Without Forgetting In-Context Learning: A Theoretical Analysis of Linear Attention Models

Neuron-Aware Active Few-Shot Learning for LLMs

Mechanism of Task-oriented Information Removal in In-context Learning

FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG

MemoryAthena: Adaptive Routing over Latent and Generated Memories

Continual Learning Mechanisms Compose for Long-Horizon Memorization