arXiv Machine Learning

Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning

The paper investigates how to fine‑tune language models for sparse and continual reinforcement learning by ranking and partitioning attention heads. For sparse learning, heads are ranked by gradient norm to identify a small, effective subset for training. For continual learning, heads are partitioned across tasks to reduce interference and preserve prior knowledge. Empirical results on math, instruction‑following, and coding tasks show that tuning higher‑ranked heads outperforms lower‑ranked ones and matches denser baselines, while head partitioning improves retention without sacrificing adaptation to new tasks.

arXiv Machine Learning

Sep 24

Attention Routing Stabilizes Early: Working-Set Inference for Recurrent Language Models

The paper investigates how attention dynamics evolve across recurrent depth in language models, finding that attention support stabilizes early while hidden states and outputs take longer. It proposes WISE, a training‑free method that uses full attention in early steps and then reuses the discovered sparse working set for later steps, preserving performance on multi‑hop QA tasks. Experiments show that WISE maintains quality up to 2K context, offers measurable speedups, and highlights the importance of recurrent discovery of attention support.

By Ke Wan, Chen Chen

llms benchmarks

arXiv Machine Learning

Sep 10

Continual Learning Mechanisms Compose for Long-Horizon Memorization

arXiv:2609.06986v1 Announce Type: new Abstract: Language models may need to internalize information that arrives over time and retain it through many subsequent updates. To study this challenge, we i...

By Zheyuan Zhang, Alvin Zhang, Daniel Khashabi, Tianmin Shu

llms fine-tuning

arXiv Machine Learning

Jun 2

Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning

arXiv:2603. 11653v2 Announce Type: replace Abstract: Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments.

By Jiaheng Hu, Jay Shim, Chen Tang, Yoonchang Sung, Bo Liu, Peter Stone, Roberto Martin-Martin

agents reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv Machine Learning

Aug 18

Task-Anchored Representation Shaping for Pre-Trained Model-Based Continual Learning

arXiv:2608. 16345v1 Announce Type: new Abstract: Pre-trained models (PTMs) provide a strong foundation for continual learning by offering stable representations that facilitate lightweight adaptation to new tasks.

By Zhiming Xu, Huiyu Yi, Zhen-Hao Xie, Baile Xu, Furao Shen, Jian Zhao, Suorong Yang

arXiv Machine Learning

Sep 16

CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework

arXiv:2609.17026v1 Announce Type: new Abstract: Continual learning must balance the learning of new knowledge with the retention of previously learned knowledge to incrementally learn tasks from a da...

By Yunxiang Fu, Meng Lou, Zicheng Liao, Yizhou Yu

fine-tuning efficiency benchmarks

arXiv AI

Jun 8

Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning

arXiv:2606. 07500v1 Announce Type: cross Abstract: Continual learning in Large Language Models (LLMs) is hindered by the plasticity-stability dilemma, where acquiring new capabilities often leads to catastrophic forgetting of previous knowledge.

By Fatema Siddika, Md Anwar Hossen, Tanwi Mallick, Ali Jannesari

llms benchmarks

arXiv AI

Jul 7

Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

arXiv:2607. 02980v1 Announce Type: cross Abstract: Scaling modern large language models (LLMs) to long contexts is limited by the quadratic computation cost, and poor length extrapolation of dense attention.

By Xiang Hu, Xinyu Wei, Hao Gu, Minshen Zhang, Tian Liang, Huayang Li, Lei Zhu, Yan Wang, Sirui Han, Yushi Bai, Kewei Tu, Haitao Mi, Leo Liang

llms

arXiv AI

Oct 15

TSRL: Trigger-based Supervised Reinforcement Learning for Continual Agent Training

arXiv:2610.17726v1 Announce Type: cross Abstract: Large language model (LLM) agents excel at complex, multi-step tasks but suffer from catastrophic forgetting during continual learning--acquiring new...

By Yuqian Fu, Kaiwen Jiang, Zijun Song, Jiawei Wang, Yuanheng Zhu, Dongbin Zhao

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 13

Representation Finetuning for Continual Learning

arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.

By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong

fine-tuning benchmarks

arXiv Machine Learning

Jul 7

RL Forgets! Towards Continual Policy Optimization

arXiv:2607. 04364v1 Announce Type: new Abstract: Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks.

By Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou, Bo Ye, Jian Zhao, Min-Ling Zhang, Tong Wei

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 29

When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.

By Tanguy Dieudonn\'e, Giulia Lanzillotta, Enis Simsar, Louis Barinka, Thomas Hofmann

fine-tuning benchmarks

arXiv AI

Oct 16

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

CRAFT is a continual learning framework that freezes model weights and learns low‑rank interventions on hidden representations to mitigate catastrophic forgetting. Tasks that are similar share an intervention, determined by how they change the model’s output distribution, and a KL penalty keeps new tasks close to existing behavior. The approach also allows task identification at test time without explicit labels and shows improved performance and reduced forgetting across benchmarks compared to LoRA‑based methods.

By Md Anwar Hossen, Fatema Siddika, Juan Pablo Munoz, Tanya Roosta, Ali Jannesari

llms fine-tuning benchmarks

Related stories