arXiv Machine Learning June 29, 2026 By Tanguy Dieudonn\'e, Giulia Lanzillotta, Enis Simsar, Louis Barinka, Thomas Hofmann

When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

Read the original on arXiv Machine Learning →

arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 13

Representation Finetuning for Continual Learning

arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.

By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong

fine-tuning benchmarks

arXiv Machine Learning

Jul 28

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

arXiv:2607. 23837v1 Announce Type: new Abstract: Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting.

By Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli, Pavan Turaga, Hassan Ghasemzadeh

llms rag fine-tuning benchmarks

arXiv Machine Learning

Aug 12

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

arXiv:2509. 13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models.

By Irene Testa, Luigi Quarantiello, Eric Nuertey Coleman, Samrat Mukherjee, Julio Hurtado, Vincenzo Lomonaco

fine-tuning benchmarks

arXiv AI

Jul 20

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning

arXiv:2601. 13020v2 Announce Type: replace-cross Abstract: Continual instruction tuning (CIT) requires multimodal large language models (MLLMs) to adapt to a stream of tasks without forgetting prior capabilities.

By Zhiyan Hou, Haiyun Guo, Haokai Ma, Yandu Sun, Yonghui Yang, Jinqiao Wang

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 29

TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree

arXiv:2506. 10355v2 Announce Type: replace Abstract: Many real-world applications collect data in a streaming environment, where learning tasks are encountered sequentially.

By Yu-Yang Qian, Yuan-Ze Xu, Zhen-Yu Zhang, Peng Zhao, Zhi-Hua Zhou

llms nlp reinforcement-learning

arXiv Machine Learning

Sep 1

One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning

arXiv:2608.31096v1 Announce Type: cross Abstract: Class-incremental learning (CIL) requires a model to incrementally learn tasks that contain new classes without accessing earlier training data while...

By Yunxiang Fu, Meng Lou, Yizhou Yu

fine-tuning

One email a morning, machine-written

Related stories

Representation Finetuning for Continual Learning

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning

TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree

One Adapter, Many Tasks: Task-Conditioned Feature Transformations for Continual Learning