arXiv Machine Learning October 7, 2026 By Mingyuan Zhang, Yue Bai, Yifan Wang, Yiyang Huang, Yun Fu

Rethinking Fine-Tuning: Unlocking Hidden Capabilities in Vision-Language Models

Read the original on arXiv Machine Learning →

The paper introduces Mask Fine‑Tuning (MFT), a new approach for adapting Vision‑Language Models that avoids modifying backbone weights. MFT learns masks to selectively route information through existing pretrained connections, dynamically uncovering subnetworks that better align with downstream tasks. Experiments demonstrate that MFT consistently outperforms both Full Fine‑Tuning and Parameter‑Efficient Fine‑Tuning across multiple benchmarks, while also offering insights into how pretrained VLMs reorganize their internal pathways during adaptation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen

llms computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 19

The Hidden Evolution of Disguised Visual Context inside the VLM

arXiv:2606. 20077v1 Announce Type: cross Abstract: Visual tokens enter Large Language Models (LLMs) as raw, foreign signals.

By Wish Suharitdamrong, Tony Alex, Muhammad Awais, Sara Atito

llms multimodal benchmarks

arXiv AI

Jun 9

An Effective Router for Vision-Language Model Selection

arXiv:2606. 08970v1 Announce Type: new Abstract: Vision-language models (VLMs) with varying performance and resource requirements are widely deployed, making it difficult for users to select the most appropriate one among numerous VLM candidates.

By Can Wang, Shengwei Wang, Bolin Zhang, Zhiying Tu, Dianhui Chu

llms multimodal

arXiv AI

Sep 15

Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders

arXiv:2609.15229v1 Announce Type: cross Abstract: We propose a pre-fine-tuning probing method for Parameter-Efficient Fine-Tuning (PEFT) layer selection, aiming to obtain more stable and higher gains...

By Qingtao Xia, Jiahua Bao, Siyao Cheng, Jie Liu

llms fine-tuning multimodal benchmarks

Hugging Face Trending Papers

Aug 4

MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization

Vision-language models exhibit remarkable zero-shot capabilities but suffer significant performance degradation under distribution shifts. While test-time adaptation (TTA) via Low-Rank Adaptation offers a parameter-efficient solution, we identify a fundamental bottleneck in current methods: the reliance on static rank configurations.

llms multimodal benchmarks

arXiv AI

1d ago

Boosting Large Language Models with Mask Fine-Tuning

arXiv:2503.22764v3 Announce Type: replace-cross Abstract: The large language model (LLM) is typically integrated into the mainstream optimization protocol. However, it remains underexplored whether m...

By Mingyuan Zhang, Yue Bai, Huan Wang, Yizhou Wang, Qihua Dong, Yitian Zhang, Yun Fu

llms fine-tuning efficiency benchmarks

One email a morning, machine-written

Related stories

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

The Hidden Evolution of Disguised Visual Context inside the VLM

An Effective Router for Vision-Language Model Selection

Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders

MuRA: Multi-Rank Adaptation for Efficient and Effective Test-Time Vision-Language Generalization

Boosting Large Language Models with Mask Fine-Tuning