arXiv AI August 28, 2026 By Harshavardhan Adepu, Li Zhang, Sanjiv Kumar, Vikas Singh

Fine-Tuning of Transformer models with Frames

The paper introduces FrameFT, a parameter-efficient fine-tuning method for transformer models that represents weight updates using sparse coefficients in a Fusion Frame basis. This approach reduces memory usage by storing only the sparse coefficients, leading to significant compute advantages and formal convergence guarantees. Experiments on language and vision tasks show that FrameFT matches or surpasses state‑of‑the‑art PEFT techniques while requiring far fewer trainable parameters.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Aug 14

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference

arXiv:2608. 13426v1 Announce Type: cross Abstract: Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications.

By Zixuan Lan, Yanhong Li, Jiawei Zhou

llms multimodal benchmarks

arXiv AI

Aug 14

Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision

arXiv:2505. 12532v3 Announce Type: replace-cross Abstract: Efficiently adapting large pretrained models is critical under tight compute and memory budgets.

By Ahmet Bilican, M. Ak{\i}n Y{\i}lmaz, A. Murat Tekalp, R. G\"okberk Cinbi\c{s}

diffusion computer-vision nlp fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 3

BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

arXiv:2509. 25136v3 Announce Type: replace Abstract: Activation-aware low-rank factorization techniques yield strong compression results but are generally confined to linear layers, while existing whitening-based theory typically makes an implicit full-rank assumption on activations.

By David Gonz\'alez-Mart\'inez

llms fine-tuning efficiency

arXiv Machine Learning

Jun 2

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

arXiv:2411. 09816v5 Announce Type: replace Abstract: Large neural networks achieve state-of-the-art performance on many tasks, yet their sheer size hinders deployment on resource-constrained devices.

By Cem \"Uy\"uk, Mike Lasby, Mohamed Yassin, Utku Evci, Yani Ioannou

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

Jul 17

Stabilizing Native Low-Rank LLM Pretraining

arXiv:2602. 12429v2 Announce Type: replace Abstract: Foundation models have achieved remarkable success, yet their growing parameter counts pose significant computational and memory challenges.

By Paul Janson, Edouard Oyallon, Eugene Belilovsky

llms rag

Hugging Face Trending Papers

Aug 4

MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization

Vision-language models exhibit remarkable zero-shot capabilities but suffer significant performance degradation under distribution shifts. While test-time adaptation (TTA) via Low-Rank Adaptation offers a parameter-efficient solution, we identify a fundamental bottleneck in current methods: the reliance on static rank configurations.

llms multimodal benchmarks

One email a morning, machine-written

Related stories

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference

Exploring Sparsity for Parameter Efficient Fine Tuning Using Wavelets for Vision

BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression

Learning Fine-grained Parameter Sharing via Sparse Tensor Decomposition

Stabilizing Native Low-Rank LLM Pretraining

MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization