arXiv Machine Learning
MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 10
One Rate Is Not Enough: Adaptive Anisotropic Learning Rates for LoRA Fine-Tuning
arXiv:2609.05885v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) has become the standard for parameter-efficient fine-tuning of large language models. Most LoRA variants follow a uniform-LR...
arXiv Machine Learning
Jun 2
RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models
arXiv:2505. 18877v4 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) lowers the computational and memory overhead of fine-tuning large models by updating a low-dimensional subspace of the pre-trained weight matrix.
arXiv AI
Jun 16
SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation
arXiv:2606. 16454v1 Announce Type: cross Abstract: Low-Rank Adaptation (LoRA) enables efficient adaptation of large pre-trained models to downstream tasks by parameterizing weight updates with low-rank matrices.
arXiv Machine Learning
Jul 21
MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning
arXiv:2607. 16789v1 Announce Type: new Abstract: Real-world perception and decision making are inherently multimodal, integrating complementary signals across modalities.
arXiv Computer Vision
Oct 19
MLLM Frame Subset Ensembling for Audio-Visual Video QA and MLLM-based Reranking for Ad-hoc Video Search in TRECVID 2025
The paper describes the CERTH-ITI team’s approach to the TRECVID 2025 Ad-hoc Video Search (AVS) and Video Question Answering (VQA) tracks. For AVS, a two-stage framework uses foundation vision‑language models for initial retrieval and a Multimodal‑LLM for reranking, improving contextual matching without training. In VQA, an audio‑visual MLLM is fine‑tuned and enhanced with inference‑time scaling; answer generation employs majority voting over responses from random frame subsets, while multiple‑choice responses use mean‑pooling logits, yielding top performance on VQA metrics.
arXiv AI
Oct 1
Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation
arXiv:2605.07111v3 Announce Type: replace-cross Abstract: Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater represe...