← Back to all news
Hugging Face Trending Papers

M-PFT: Angular Margin for Calibrated Prompt-Tuning of Fine-Tuned Vision-Language Models

Read the original on Hugging Face Trending Papers →

M-PFT introduces an angular-margin approach for prompt-tuning of vision‑language models, adding two objectives: reducing maximum cosine similarity among tunable class embeddings to prevent collapse, and aligning each tunable embedding with its zero‑shot counterpart to preserve semantic geometry. This method enlarges the angular margin, thereby correcting base‑to‑novel dual miscalibration and adjusting distorted decision boundaries. Experiments on nine prompt‑tuning methods across fifteen datasets show reduced calibration error on novel classes while maintaining competitive accuracy on base classes with minimal computational cost.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

Related stories

arXiv AI
Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu
llmsragagentsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Sep 2

Guided Prompt Evolution for Vision-Language Models Adaptation

The paper introduces EvoPrompt, a framework for adapting vision‑language models to new tasks with limited data while preventing catastrophic forgetting. EvoPrompt uses a Modality‑Shared Prompt Projector to create hierarchical prompts and an evolutionary training strategy that separates low‑rank updates into directional and magnitude components, preserving learned semantic directions. Experiments show that EvoPrompt achieves state‑of‑the‑art few‑shot performance while maintaining the original zero‑shot capabilities of the pre‑trained models.

By Enming Zhang, Jiayang Li, Yanlong Wang, Yanru Wu, Zhenyu Liu, Yang Li
llmsragfine-tuningmultimodalbenchmarks
More like this →
arXiv Computer Vision
Aug 31

Dual-Stream Semantic Guidance with Prototype Anchor Calibration for Source-Fully-Free Adaptation of Vision-Language Models

The paper introduces Dual-Stream Semantic Guidance (DSSG), a framework for Source‑Fully‑Free Domain Adaptation of Vision‑Language Models that mitigates dual semantic drift through a caption stream and a class‑anchor stream. It adds a Dynamic Cross‑Modal Knowledge Distillation module and a Prototype Anchor Calibration extension (DSSG‑PAC) to reduce computation while maintaining performance. Experiments show DSSG outperforms state‑of‑the‑art methods and DSSG‑PAC cuts adaptation time by 18.9% with minimal loss in accuracy.

By Weiwei Xiang, Shun Peng, Guangyi Xiao, Hao Chen, Lei Yang
llmsragfine-tuningefficiencymultimodalbenchmarks
More like this →