arXiv:2603. 06697v2 Announce Type: replace-cross Abstract: Vision--language models (VLMs) process images as visual tokens, yet their intermediate reasoning is often carried out in text, which can be suboptimal for visually grounded radiology tasks.
arXiv:2606. 12169v1 Announce Type: cross Abstract: High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers.
The paper presents a two‑stage multimodal framework for chest X‑ray interpretation that incorporates radiologist gaze data from the MIMIC‑Eye dataset. Stage 1 introduces a gaze‑token classifier that fuses image patches, bounding‑box masks, transcription embeddings, and fixation maps, and a curriculum‑scheduled loss that improves accuracy and spatial alignment, yielding a 4.4% AUC gain and 13.3% F1 improvement. Stage 2 translates classifier predictions into region‑specific diagnostic sentences using confidence‑weighted keywords, an expert dictionary, and a prompted large language model, boosting clinical‑term BERTScore and ROUGE over keyword baselines.
arXiv:2606. 31599v1 Announce Type: cross Abstract: Vision-language models (VLMs) combining reinforcement learning (RL) ignite remarkable progress in multimodal reasoning, yet still struggle with medical images, which typically exhibit extremely sparse visual evidence to inform clinical decision-making.
arXiv:2606. 20244v1 Announce Type: cross Abstract: Vision-language models (VLMs) often underperform on evidence intensive tasks because decisive visual evidence are small, localized, and easy to overlook, leading to failures in evidence readout even when high-level reasoning is intact.
The paper introduces PIVOT, a dual-level learning framework designed to improve visually-grounded multimodal reasoning in large vision-language models. PIVOT employs a self‑calibrated experience replay mechanism to selectively reuse valuable visual reasoning trajectories, and a vision‑guided advantage allocation scheme that assigns extra rewards to tokens with strong visual support. Experiments on multiple benchmarks show that PIVOT enhances the multimodal reasoning performance of these models.
arXiv:2604. 09757v2 Announce Type: replace-cross Abstract: Medical vision--language models (VLMs) have shown strong potential for medical visual question answering (VQA), yet their reasoning remains largely text-centric: images are encoded once as static context, and subsequent inference is dominated by language.
LiteMedCoT-VL is a parameter‑efficient pipeline that transfers chain‑of‑thought reasoning from a 235B teacher model to a 2B student model using LoRA fine‑tuning on explanation‑enriched data. The approach enables a compact vision‑language model to perform medical visual question answering without relying on image captions, achieving 64.9% accuracy on the PMC‑VQA benchmark—an 11‑point improvement over the zero‑shot Qwen3‑VL‑4B baseline. Visual grounding analysis confirms that the model bases its predictions on image content rather than textual priors.
arXiv:2508. 07683v2 Announce Type: replace-cross Abstract: Video Temporal Grounding (VTG) aims to localize specific video segments corresponding to natural language queries.
The paper introduces EASE, a method that enhances multimodal reinforcement learning with verifiable rewards (RLVR) by adding visual‑evidence process supervision. EASE transforms annotated evidence regions into smoothed visual‑token targets and uses them to guide attention during RL training, but only on high‑reward trajectories. Experiments on Qwen2.5‑VL‑7B, Qwen3‑VL‑4B, and Qwen3‑VL‑8B show that EASE improves average scores over DAPO by 2.5 to 3.1 points across perception, hallucination, visual math, and multimodal reasoning benchmarks, and diagnostics confirm better alignment of visual attention with annotated evidence.
Gaze Attention is a new mechanism for multimodal large language models that selectively focuses on relevant visual regions during each generation step, rather than attending to all visual tokens. By grouping tokens into spatial regions and using learnable context tokens to retain global information, it reduces attention computation and visual key‑value entries by up to 90%. Experiments on 13 image and 6 video benchmarks show that Gaze Attention matches or outperforms dense‑attention baselines while using fewer visual resources.
The paper introduces ViMoD, a lightweight framework that dynamically manages visual memory for multimodal large language models. ViMoD uses Deformable Aggregation of Region-wise Tokens (DART) to create content‑adaptive coarse representations linked to recoverable fine tokens, and Temporal Routing for Adaptive Contextual Evidence (TRACE) to anticipate and adjust evidence needs during reasoning. On the Qwen3‑VL‑4B model, ViMoD achieves superior performance across eight reasoning benchmarks with only a 20% visual token budget and minimal additional parameters.
arXiv:2512. 11995v2 Announce Type: replace-cross Abstract: While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in most benchmarks, they often struggle in practice with complex open-ended tasks, which usually require multiple rounds of exploration and reasoning in the visual space.