Hugging Face Trending Papers
Behaviorally Effective LoRA Writes Are Sparse and Structured
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computation and Language
Sep 2
Behaviorally Effective LoRA Writes Are Sparse and Structured
The paper investigates which parts of a low‑rank adaptation (LoRA) update actually drive behavior in language models. It shows that behaviorally effective LoRA writes are sparse, structured, and highly concentrated, with only a few module‑wise components carrying most of the signal. Using a learned‑basis continuation recipe, the authors demonstrate that these sparse, structured writes can be recovered with minimal error and that they remain robust across different training continuations and evaluation tasks.
Hugging Face Trending Papers
Jul 15
Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning
Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable. We identify the data-parameter interference as a geometric source of this instability.
arXiv Machine Learning
Jul 17
Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning
arXiv:2607. 14367v1 Announce Type: new Abstract: Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable.
arXiv AI
Jul 20
PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
arXiv:2601. 13020v2 Announce Type: replace-cross Abstract: Continual instruction tuning (CIT) requires multimodal large language models (MLLMs) to adapt to a stream of tasks without forgetting prior capabilities.
arXiv Machine Learning
Aug 12
Same Targets, Different Computation: How Post-Training Divides Work Across Model Layers
arXiv:2605. 07284v2 Announce Type: replace Abstract: A late-layer change learned during post-training may work on the base model's earlier state, or it may depend on earlier computation learned with it.
arXiv AI
Jun 15
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
arXiv:2606. 13767v1 Announce Type: cross Abstract: Low-rank adaptation (LoRA) and its variants provide a memory- and compute-efficient alternative to full fine-tuning of pre-trained models.