arXiv Computer Vision
Reassessing Global Gradient-Norm Imbalance in BLIP Fine-Tuning Across Physical Domains
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Sep 10
UOT-Gap: A Variational Principle for the Modality Gap in Vision-Language Models via Unbalanced Optimal Transport
arXiv:2609.10224v1 Announce Type: new Abstract: Vision-language models such as CLIP embed images and text in a shared space, where modality-specific distributions often remain separated. Existing acc...
arXiv AI
Jul 13
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs
arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.
arXiv Machine Learning
Sep 17
Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models
arXiv:2609.18084v1 Announce Type: cross Abstract: Fine-tuning a Vision-Language-Action (VLA) model for a new deployment environment is expensive, yet most methods apply uniform-capacity adapters to e...
arXiv Computation and Language
3d ago
GAVAGAI: Referential Alignment for Infant-Scale Vision-Language Models via Balanced Optimal Transport
The paper introduces GAVAGAI, a method that adds a referential alignment variable to vision‑language models by solving a small assignment problem via entropic optimal transport. By allowing words to be assigned to a special NOTHING column and enforcing a capacity constraint on regions, the approach improves word‑to‑picture accuracy and localization on the SAYCam infant video corpus. Experiments show that the alignment loss boosts performance over a standard next‑token captioning baseline, especially under high non‑reference rates.
arXiv Machine Learning
Sep 2
The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence
arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inp...
arXiv Machine Learning
Jun 9
Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation
arXiv:2606. 07647v1 Announce Type: cross Abstract: Large vision language models (LVLMs) have made rapid advancements and are deployed across various applications, yet hallucinations remain a major challenge.