arXiv Machine Learning

OP-LoRA: The Blessing of Dimensionality

arXiv:2412. 10362v2 Announce Type: replace Abstract: Low-rank adapters (LoRA) enable finetuning of large models with only a small number of parameters.

arXiv AI

Jul 15

CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

arXiv:2607. 11940v1 Announce Type: cross Abstract: As the scale of large pre-trained models continues to grow, fine-tuning them under limited memory budgets has become increasingly challenging.

By Gengyu Zhang, Haiyin Ran, Zhengbao He, Yuhang Liu, Hanling Tian, Zhehao Huang, Xiaolin Huang

fine-tuning efficiency

arXiv AI

6d ago

COLORA: Efficient Fine-Tuning for Convolutional Models with a Study Case on Optical Coherence Tomography Image Classification

arXiv:2505. 18315v3 Announce Type: replace-cross Abstract: We introduce \textbf{CoLoRA} (Convolutional Low-Rank Adaptation), a parameter-efficient fine-tuning method for convolutional neural networks (CNNs).

By Mariano Rivera, Angello Hoyos

computer-vision fine-tuning

arXiv Machine Learning

Jul 21

DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

arXiv:2607. 17467v1 Announce Type: cross Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples.

By Siobhan Reid, Zhixiang Chi, Li Gu, Omid Reza Heidari, Ziqiang Wang, Yang Wang

fine-tuning benchmarks

arXiv Machine Learning

Jul 27

Layer-wise LoRA fine-tuning: a similarity metric approach

arXiv:2602. 05988v2 Announce Type: replace Abstract: Pre-training Large Language Models (LLMs) on web-scale datasets becomes fundamental for advancing general-purpose AI.

By Keith Ando Ogawa, Bruno Lopes Yamamoto, Lucas Lauton de Alcantara, Lucas Pellicer, Rosimeire Pereira Costa, Edson Bollis, Anna Helena Reali Costa, Artur Jordao

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 16

SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation

arXiv:2606. 16454v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) enables efficient adaptation of large pre-trained models to downstream tasks by parameterizing weight updates with low-rank matrices.

By Junghun Oh, Sungyong Baik, Kyoung Mu Lee

fine-tuning benchmarks safety

arXiv Machine Learning

Aug 12

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

arXiv:2509. 13211v4 Announce Type: replace Abstract: The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models.

By Irene Testa, Luigi Quarantiello, Eric Nuertey Coleman, Samrat Mukherjee, Julio Hurtado, Vincenzo Lomonaco

fine-tuning benchmarks

arXiv Machine Learning

Jul 10

Super Weights in LLMs and the Failure of Selective Training

arXiv:2607. 08733v1 Announce Type: new Abstract: Recent work identified Super Weights, individual parameters whose removal degrades model performance by orders of magnitude.

By Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 15

Beyond LoRA: Is Sparsity-Induced Adaptation Better?

arXiv:2606. 13767v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models.

By Elijah Cadenhead, Cristian McGee, Xin Li, El Houcine Bergou, Aritra Dutta

fine-tuning efficiency

arXiv AI

Jul 21

Sparsity-Aware Low-Rank Representation for Efficient Fine-Tuning of Large Language Models

arXiv:2601. 16991v3 Announce Type: replace-cross Abstract: Adapting large pre-trained language models to downstream tasks often entails fine-tuning millions of parameters or deploying costly dense weight updates, which hinders their use in resource-constrained environments.

By Longteng Zhang, Sen Wu, Shuai Hou, Zhengyu Qing, Zhuo Zheng, Danning Ke, Qihong Lin, Qiang Wang, Shaohuai Shi, Xiaowen Chu

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 9

Efficient Long-Horizon Learning for Learned Optimization

arXiv:2607. 06772v1 Announce Type: new Abstract: Learned optimization aims to improve upon hand-designed optimizers (e.

By Xiaolong Huang, Benjamin Th\'erien, James Harrison, Eugene Belilovsky

llms computer-vision

arXiv Machine Learning

Jul 21

PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer

arXiv:2607. 17620v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) makes finetuning large language models cheaper by adding to each weight matrix a trainable low-rank update parameterized as the product of two matrices.

By Nikhil Ghosh, Tetiana Parshakova, Robert M. Gower

llms fine-tuning

arXiv AI

Jun 26

Scaling Multi-Reference Image Generation with Dynamic Reward Optimization

arXiv:2606. 26947v1 Announce Type: cross Abstract: While personalized image generation has achieved remarkable progress, multi-reference image generation (MRIG) remains a challenging task.

By Wenwang Huang, Yusen Fu, Junjie Wang, Mengfei Huang, Yulin Li, Gan Liu, Jing Cai, Yancheng He, Zhuotao Tian

diffusion fine-tuning benchmarks

Related stories