The paper introduces Salience-LLaVA, a vision‑language model that prioritizes scene elements based on their importance for low‑vision users. It presents three new salience‑aware datasets—Salience COCO, Salience Flickr, and Salience VizWiz—annotated with object‑level salience verified by low‑vision participants. The authors also propose the SCMI metric to evaluate caption ordering accuracy and demonstrate the system’s practicality by deploying it on assistive glasses.
arXiv:2610.22822v1 Announce Type: new Abstract: Blindness and low vision limit many people's ability to complete daily tasks, motivating AI assistants that provide useful guidance. Yet, existing benc...
OpenVAM is a new framework for visual attention modeling that combines a dense saliency map with language‑based explanations. It uses a decoupled design: a visual pathway for precise localization and a vision‑language head that generates grounded what/why explanations. The method is trained in three stages to preserve localization while adding language grounding, and a scalable pipeline creates multi‑domain annotations for evaluation.
Improving video captioning quality typically demands retraining large vision-language models, an expensive and often impractical requirement. Existing training-free alternatives instead ground captions in detected objects to curb hallucination, but apply only a single, fixed correction pass without prioritizing which objects matter most, leaving semantically significant content omitted.
The paper introduces Target Saliency Boosting, a new task that enhances the visual prominence of a specific object in text-to-image generation without visual priors. It proposes GazeME, a lightweight framework that inserts learnable marker tokens around object descriptions to indicate which objects to emphasize or suppress. By building a saliency-semantics dataset and using Saliency Prior Marker Activation, GazeME learns to adjust markers during training and automatically applies them at inference, effectively boosting target saliency while maintaining semantic alignment and image quality.
arXiv:2609.00709v1 Announce Type: cross Abstract: Large Vision-Language Models produce fluent image descriptions but offer limited semantic control: users cannot reliably specify whether captions sho...
arXiv:2608.21819v1 Announce Type: cross Abstract: Reliable image captioning in Vision-Language Models (VLMs) requires captions to be both precise and complete, avoiding unsupported object mentions wh...
arXiv:2508.03351v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated remarkable capabilities across diverse language tasks, motivating their extension to vision-la...
arXiv:2604. 02327v2 Announce Type: replace-cross Abstract: Pretrained Vision Transformers (ViTs) such as DINOv2 and MAE provide generic image features that can be applied to a variety of downstream tasks such as retrieval, classification, and segmentation.
arXiv:2607. 08489v1 Announce Type: cross Abstract: Vision-language models excel at video captioning, yet typically generate descriptions that fail to capture individual viewers' attention.
arXiv:2609.00591v1 Announce Type: new Abstract: An image may be worth a thousand words, but most captioning models describe it in only a few. Modern vision-language models produce fluent high-level c...
arXiv:2606. 23763v1 Announce Type: cross Abstract: Recent work typically assesses vision--language consistency using attention distributions of answer-side tokens.
The paper introduces Counterfactual Semantic Saliency (CSS), a black-box, model-agnostic method that measures how the removal of objects from a scene shifts the semantic description produced by vision‑language models (VLMs). Using CSS, the authors compare prominent VLMs to a large human psychophysics baseline (16,603 responses across 307 scenes) and find that VLMs over‑rely on large, centrally located, and highly salient objects, while under‑rely on people compared to humans. The study highlights a significant scene‑comprehension gap between VLMs and human perception, with size bias emerging as a key factor driving model‑human divergence.