arXiv AI
Aug 3
GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR
arXiv:2601. 09361v4 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is a key paradigm for improving large-scale reasoning models.
Geo-LoRA: Geometry-Aware Subspace Evolution for Low-Rank Adaptation in Continual Learning
Geo-LoRA introduces a geometry‑aware framework for low‑rank adaptation in rehearsal‑free class‑incremental learning. It regulates the evolution of shared and task‑specific LoRA subspaces using Subspace Projection Preservation, Adaptive Core‑Slack Alignment, and Median‑Calibrated Block Overlap, ensuring smooth trajectories on the Grassmann manifold and balanced stability‑plasticity trade‑offs. The method achieves state‑of‑the‑art performance across multiple benchmarks without adding new adapter types.
Related stories
arXiv AI
Aug 25
TASSO: TAsk-Specific Subspace Optimization for Continual Learning of Vision-Language Models
arXiv:2608.21487v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) exhibit strong zero-shot capabilities, making them an attractive solution for continual learning across diverse tasks....
arXiv AI
Aug 13
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
arXiv:2608. 11674v1 Announce Type: cross Abstract: On-policy rollout methods such as GRPO are central to post-training of large language models, yet they frequently suffer from training instabilities, cross-task capability degradation, and response-length inflation.
arXiv Machine Learning
Jun 29
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
arXiv:2606. 28117v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models.
arXiv AI
Jul 14
RSLoRA: Training-free Rank Allocation for LoRA via Representational Sensitivity Probing
arXiv:2607. 09757v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) has become a cornerstone of parameter-efficient fine-tuning (PEFT); however, the conventional practice of uniform rank assignment ignores the functional heterogeneity of neural layers.
arXiv Statistics ML
2d ago
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
arXiv:2610.00431v1 Announce Type: new Abstract: Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new t...
arXiv AI
Jul 20
PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
arXiv:2601. 13020v2 Announce Type: replace-cross Abstract: Continual instruction tuning (CIT) requires multimodal large language models (MLLMs) to adapt to a stream of tasks without forgetting prior capabilities.
arXiv Computer Vision
Aug 31
Activation Boundary Matching: Task-Informed Initialization for Low-Rank Adaptation
The paper introduces Activation Boundary Matching for Low‑Rank Adaptation (ABM‑LoRA), a task‑informed initialization strategy that uses the signs of layer‑wise pre‑activations from a brief probe adapter as targets for a fresh adapter. By training with a margin‑based hinge objective on these activation boundaries, ABM‑LoRA captures useful adaptation directions that standard LoRA initializers miss, while requiring only a few forward passes. Experiments show that ABM‑LoRA outperforms or matches existing LoRA, SVD, and gradient‑based initializers across multiple models and benchmarks, including T5‑base/GLUE, ConvNeXt‑T, Swin‑T, Qwen2.5‑1.5B, and LLaMA2‑7B.
arXiv Machine Learning
Jun 5
Two-Way Is Better Than One: Bidirectional Alignment with Cycle Consistency for Exemplar-Free Class-Incremental Learning
arXiv:2606. 05675v1 Announce Type: new Abstract: Continual learning (CL) seeks models that acquire new skills without erasing prior knowledge.
arXiv Statistics ML
3d ago
A Dynamical Theory of LoRA in Continual Learning
arXiv:2609.39367v1 Announce Type: new Abstract: Despite the widespread use of Low-Rank Adaptation (LoRA), little is known about its dynamics in continual learning and the mechanisms by which low-rank...
arXiv Machine Learning
Jul 28
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
arXiv:2607. 23837v1 Announce Type: new Abstract: Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting.
arXiv Machine Learning
Jul 17
Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning
arXiv:2607. 14367v1 Announce Type: new Abstract: Federated fine-tuning of large pre-trained models increasingly relies on Low-Rank Adaptation (LoRA) to reduce communication and computation, but heterogeneous clients can make adapter aggregation unstable.