arXiv Machine Learning September 1, 2026 By Alessio Quercia, Arya Bangun, Ira Assent, Hanno Scharr

Least but not Last: Fine-tuning Intermediate Principal Components for Better Performance-Forgetting Trade-Offs

Read the original on arXiv Machine Learning →

The Flow has not summarised this story yet — read it at arXiv Machine Learning.

arXiv Machine Learning

Aug 14

Can Spectral-Clipping Enable Better Learning While Forgetting Less for Low-Rank Adaptation?

arXiv:2608. 12332v1 Announce Type: cross Abstract: In recent years, low-rank adaptation (LoRA) has emerged as a significant paradigm that freezes pre-trained weights and introduces small, learnable adapters instead of fine-tuning the full set of parameters.

By Hyowon Wi, Noseong Park

fine-tuning

arXiv Machine Learning

Jul 3

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

arXiv:2602. 00722v2 Announce Type: replace Abstract: Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge.

By Hao Gu, Mao-Lin Luo, Zi-Hao Zhou, Han-Chen Zhang, Min-Ling Zhang, Tong Wei

llms multimodal

arXiv AI

Jun 2

Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization

arXiv:2606. 00132v1 Announce Type: cross Abstract: While finetuning effectively adapts foundation models to specialized downstream tasks, it can degrade nontarget capabilities acquired during pretraining.

By Dongjun Kim, Adrian de Wynter, Huancheng Chen, Heasung Kim, Haris Vikalo

fine-tuning

arXiv Machine Learning

Jun 29

When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.

By Tanguy Dieudonn\'e, Giulia Lanzillotta, Enis Simsar, Louis Barinka, Thomas Hofmann

fine-tuning benchmarks

arXiv AI

Jul 20

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning

arXiv:2601. 13020v2 Announce Type: replace-cross Abstract: Continual instruction tuning (CIT) requires multimodal large language models (MLLMs) to adapt to a stream of tasks without forgetting prior capabilities.

By Zhiyan Hou, Haiyun Guo, Haokai Ma, Yandu Sun, Yonghui Yang, Jinqiao Wang

llms fine-tuning multimodal benchmarks

arXiv AI

Aug 13

Representation Finetuning for Continual Learning

arXiv:2603. 11201v3 Announce Type: replace-cross Abstract: The world is inherently dynamic, and continual learning aims to enable models to adapt to ever-evolving data streams.

By Haihua Luo, Xuming Ran, Tommi K\"arkk\"ainen, Huiyan Xue, Zhonghua Chen, Qi Xu, Fengyu Cong

fine-tuning benchmarks

One email a morning, machine-written

Related stories

Can Spectral-Clipping Enable Better Learning While Forgetting Less for Low-Rank Adaptation?

Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization

When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning

Representation Finetuning for Continual Learning