arXiv AI July 28, 2026 By Moule Lin, Shuhao Guan, Andrea Patane, David Gregg, Goetz Botterweck

Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

arXiv:2601. 21003v3 Announce Type: replace Abstract: Large Language Models usually put more emphasis on accuracy and therefore, will guess even when not certain about the prediction, which is especially severe when fine-tuned on small datasets due to the inherent tendency toward miscalibration.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jul 3

Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation

arXiv:2607. 02182v1 Announce Type: new Abstract: Large language models (LLMs) exhibit remarkable reasoning capabilities, but their task-specific fine-tuning is notoriously plagued by overconfidence, severely hindering trustworthy deployment.

By Jijie Zhang, Zhe Ren, Quan Zhang, Dandan Guo

llms fine-tuning

arXiv Machine Learning

Jun 30

BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

arXiv:2606. 29184v1 Announce Type: new Abstract: While Low-rank adaptation (LoRA) enables highly efficient fine-tuning by constraining task-specific updates to fixed low-rank subspaces, this rigid design limits representational flexibility and often results in overconfident predictions and miscalibrated uncertainty, especially in low-data regimes.

By Zhibin Duan, Yuhong Wang, Jiahong Fu, Zongsheng Yue, Bo Chen, Zongben Xu

fine-tuning benchmarks

arXiv AI

Aug 5

FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

arXiv:2608. 03605v1 Announce Type: new Abstract: Federated fine-tuning with Low-Rank Adaptation (LoRA) enables efficient collaborative adaptation of Large Language Models (LLMs) without centralizing private data.

By Shenghui Li, Thiemo Voigt

llms nlp fine-tuning benchmarks

arXiv Machine Learning

Jun 2

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models

arXiv:2505. 18877v4 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) lowers the computational and memory overhead of fine-tuning large models by updating a low-dimensional subspace of the pre-trained weight matrix.

By Yilang Zhang, Bingcong Li, Georgios B. Giannakis

llms fine-tuning benchmarks

arXiv AI

Jul 21

SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

arXiv:2607. 16252v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning (PEFT) method for large language models.

By Yupeng Chang, Yuan Wu, Yi Chang

llms fine-tuning benchmarks

arXiv AI

Jun 15

Beyond LoRA: Is Sparsity-Induced Adaptation Better?

arXiv:2606. 13767v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models.

By Elijah Cadenhead, Cristian McGee, Xin Li, El Houcine Bergou, Aritra Dutta

fine-tuning efficiency

Related stories

Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation

BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models

SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

Beyond LoRA: Is Sparsity-Induced Adaptation Better?