arXiv Machine Learning July 9, 2026 By Haokun Liu, Gyung Hyun Je, Marco Ciccone, Zhenlin Xu, Prasanth YSS, Colin Raffel

The Appeal and Reality of Recycling LoRAs with Adaptive Merging

Read the original on arXiv Machine Learning →

arXiv:2602. 12323v2 Announce Type: replace Abstract: The widespread availability of fine-tuned LoRA modules for open pre-trained models has led to an interest in methods that can adaptively merge LoRAs to improve performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

5d ago

New LoRA Skills Should Read but Never Write

The paper introduces READ, a method for composing low‑rank adapters (LoRA) in large language models. By rewriting each adapter into a balanced canonical form and enforcing a one‑directional coupling, READ allows new skills to read but never write into the output subspaces of existing skills, eliminating interference. Experiments on four benchmark suites and two model families show that READ consistently outperforms existing baselines, improving SuperGLUE scores by over twenty points and domain suite scores by more than seven points.

By Zeyan Li, Panqi Yang, Qirong Guo, Shengda Zhuo, SIyuan Qiu, Hu Xu, Chun Li, Jianfeng Xu

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 7

PLoRA: Efficient Concurrent LoRA Training for Large Language Models

arXiv:2508. 02932v2 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) has gained popularity as a fine-tuning approach for Large Language Models (LLMs) due to its low resource requirements and good performance.

By Minghao Yan, Zhuang Wang, Zhen Jia, Shivaram Venkataraman, Yida Wang

llms fine-tuning efficiency

arXiv AI

Jul 2

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models

arXiv:2510. 13537v2 Announce Type: replace-cross Abstract: On-device deployment of Large Language Models (LLMs) frequently leverages Low-Rank Adapters (LoRAs) to support diverse downstream tasks under tight resource constraints.

By Donald Shenaj, Ondrej Bohdal, Taha Ceritli, Mete Ozay, Pietro Zanuttigh, Umberto Michieli

llms

arXiv Machine Learning

Jun 19

Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

arXiv:2606. 19549v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) makes it cheap to train many domain- and task-specific language model adapters, but whether two adapters can be merged is usually discovered only after both have been fully trained and evaluated.

By Lin Tang, Wei Zhang, Jing Li, Hongyu Chen, Ming Zhao, Yuxuan Wang

llms fine-tuning benchmarks safety

arXiv Machine Learning

Jul 21

DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

arXiv:2607. 17467v1 Announce Type: cross Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples.

By Siobhan Reid, Zhixiang Chi, Li Gu, Omid Reza Heidari, Ziqiang Wang, Yang Wang

fine-tuning benchmarks

arXiv AI

Jun 3

ReLoRA: Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services

arXiv:2606. 02606v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed as continuously evolving services, where frequent base-model updates may invalidate previously deployed task-specific Low-Rank Adaptation (LoRA) adapters.

By Yang Xu, Zihuai Xu, Hongli Xu, Yunming Liao, Zhiwei Yao, Xitong Fu

llms fine-tuning

One email a morning, machine-written

Related stories

New LoRA Skills Should Read but Never Write

PLoRA: Efficient Concurrent LoRA Training for Large Language Models

K-Merge: Online Continual Merging of Adapters for On-device Large Language Models

Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

ReLoRA: Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services