arXiv Machine Learning
Subspace Optimization for Backpropagation-Free Continual Test-Time Adaptation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Computer Vision
Aug 25
ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimization
arXiv:2603.14254v2 Announce Type: replace Abstract: Test-time adaptation (TTA) aims to improve model robustness under distribution shifts by adapting to unlabeled test data, but most existing methods...
Hugging Face Trending Papers
Aug 4
MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization
Vision-language models exhibit remarkable zero-shot capabilities but suffer significant performance degradation under distribution shifts. While test-time adaptation (TTA) via Low-Rank Adaptation offers a parameter-efficient solution, we identify a fundamental bottleneck in current methods: the reliance on static rank configurations.
arXiv Machine Learning
Sep 16
Compute-Optimal Pretrain--Fine-tune in Ridge Gradient Descent
The paper studies how to allocate a fixed computational budget between pretraining and fine‑tuning in a two‑stage ridge regression setting. By modeling the process as a compute‑split problem and analyzing data‑dependent evaluation geometries, it derives the optimal split in terms of prediction‑relevant spectral components of the pretraining and fine‑tuning empirical covariances. The analysis uses a basis‑invariant eigenspace decomposition and perturbative control of non‑commuting dynamics to capture how pretraining directions influence downstream predictions.
arXiv Machine Learning
Jul 31
Towards Stability of Parameter-Free Optimization
arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.
arXiv AI
4d ago
Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition
arXiv:2609.37027v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) is a widely used approach to parameter-efficient fine-tuning (PEFT), yet a performance gap can remain relative to full fine-...
arXiv Machine Learning
Jun 5
TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
arXiv:2606. 06494v1 Announce Type: new Abstract: Parameter-efficient finetuning methods based on spectral decomposition have enabled progress in Continual Learning.