arXiv Machine Learning July 7, 2026 By Pengcheng Wang, Ziran Liu, Wei Wang, Wei Jiang

CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

Read the original on arXiv Machine Learning →

arXiv:2607. 02576v1 Announce Type: cross Abstract: Parameter-Efficient Fine-Tuning (PEFT) commonly adapts pretrained weights through low-rank updates, and recent methods further exploit the singular value decomposition (SVD) of the base weight for initialization or subspace selection.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

fine-tuning

arXiv AI

Jul 1

Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR

arXiv:2606. 31813v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) and its variants enable parameter-efficient fine-tuning of large language models under the supervised fine-tuning (SFT) paradigm.

By Ruijia Zhang, Jiacheng Zhu, Hanqing Zhu, Laixi Shi

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Sep 2

Scaled Idempotence in Transformer Attention: Paired OV Geometry and Shared-Value Algebras

arXiv:2609.01129v1 Announce Type: new Abstract: We identify a recurrent algebraic regularity in Transformer attention: a sparse subset of effective OV operators $T=OV^\top$ nearly closes under compos...

By Jiming Feng, Junliang Li

llms safety

arXiv Machine Learning

Jul 7

SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

arXiv:2607. 04306v1 Announce Type: new Abstract: Distilling a fine-tuned teacher into a LoRA-adapted student is a standard recipe for parameter-efficient compression, but output-level KD does not explicitly control which rank-$r$ weight subspace the adapter occupies.

By Omer Tariq, Syed Muhammad Raza, Jeongbae Son

fine-tuning efficiency safety

arXiv AI

Jul 21

SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

arXiv:2607. 16252v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) is a widely used parameter-efficient fine-tuning (PEFT) method for large language models.

By Yupeng Chang, Yuan Wu, Yi Chang

llms fine-tuning benchmarks

arXiv Machine Learning

Sep 2

Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

arXiv:2609.00762v1 Announce Type: new Abstract: Parameter-efficient fine-tuning is usually framed as a question of how many parameters to update. Under a severe trainable-state budget, however, where...

By Wentao Ye, Zhanming Shen, Zhiqing Xiao, Yao Ding, Haobo Wang, Gang Chen

llms fine-tuning

arXiv AI

Aug 18

LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights

arXiv:2602. 17510v2 Announce Type: replace-cross Abstract: We introduce LoRA-CRAFT (\textbf{C}ross-layer \textbf{R}ank \textbf{A}daptation via \textbf{F}rozen \textbf{T}ucker), abbreviated CRAFT throughout, an extremely parameter-efficient fine-tuning (PEFT) method that applies Tucker tensor decomposition to pre-trained attention weight matrices stacked across transformer layers and trains only small square adaptation matrices on the resulting frozen Tucker factors.

By Kasun Dewage, Marianna Pensky, Suranadi De Silva, Shankadeep Mondal

llms fine-tuning benchmarks

One email a morning, machine-written

Related stories

Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR

Scaled Idempotence in Transformer Attention: Paired OV Geometry and Shared-Value Algebras

SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights