arXiv Computer Vision September 3, 2026 By Ziquan Liu, Zhewei Zhu, Xuyang Shi

Characterizing Text Branch Sensitivity in Medical Vision-Language Segmentation via Evidence Decoupling

Read the original on arXiv Computer Vision →

The paper investigates how much clinical text influences pixel‑level predictions in multimodal medical image segmentation. It shows that segmentation performance is largely insensitive to the choice of fusion module, but that the impact of text varies across datasets: removing text severely degrades performance on BUSI and BTMRI, while it has only a marginal effect on ISIC and Kvasir‑SEG. Using an Evidence Decoupling Decoder, the authors reveal that text mainly modulates global semantic context rather than spatial localization, and that the specific semantic components driving sensitivity differ by dataset.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision

Aug 25

Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation

The paper introduces LoG, a localization‑infused vision‑language fusion framework for text‑guided medical image segmentation. LoG jointly performs multi‑scale target localization to explicitly capture target‑oriented semantics and employs three levels of localization‑infused fusion—feature, attention, and loss—to integrate spatial information into segmentation. Experiments on three benchmark datasets across three imaging modalities show that LoG consistently outperforms state‑of‑the‑art methods.

By Songyue Han, Mingye Zou, Shuchang Ye, Lei Bi, Mingyuan Meng

computer-vision multimodal benchmarks

arXiv AI

Jul 13

Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation

arXiv:2607. 09481v1 Announce Type: cross Abstract: Text-guided medical image segmentation leverages clinical semantics to improve lesion delineation, yet many existing models bind cross-modal fusion, supervision, and decoder design into a task-specific architecture.

By Yungeng Liu, Xuanzi Fang, Haijin Zeng, Qi Dai, Yongyong Chen

llms computer-vision multimodal safety

arXiv AI

Aug 17

MedPlex: Deep Vision-Language Co-Adaptation for Clinically Grounded Medical Segmentation

arXiv:2608. 13690v1 Announce Type: cross Abstract: Medical image segmentation is still largely treated as a vision-only problem, although clinical interpretation often relies on textual knowledge of anatomy, location, appearance, and surrounding context.

By Rafi Ibn Sultan, Hui Zhu, Chengyin Li, Dongxiao Zhu

llms computer-vision multimodal benchmarks safety

arXiv AI

4d ago

MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

arXiv:2601.06847v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We po...

By Mengmeng Zhang, Xiaoping Wu, Hao Luo, Fan Wang, Yisheng Lv

llms computer-vision multimodal

arXiv AI

Aug 28

From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation

The paper introduces MedREAL, a unified framework that aligns linguistic reasoning with spatial grounding for medical visual question answering and segmentation. MedREAL employs Seg Anchored Reasoning Pooling (SARP) to extract semantic evidence from segmentation tokens and a Reasoning-to-Visual (R2V) fusion mechanism to integrate these features into a segmentation pipeline. Using the newly created MedRAVS-13K dataset, MedREAL achieves superior performance, reporting 68.49% gIoU and 70.47% cIoU, and generates evidence masks that consistently match textual diagnoses.

By Haowen Gu, Gensheng Pei, Junzhu Mao, Qiong Wang, Mingwu Ren, Yazhou Yao

llms computer-vision nlp multimodal benchmarks

arXiv AI

Jun 24

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models

arXiv:2605. 08245v4 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) increasingly power high-stakes applications, from medical imaging to autonomous systems, yet they routinely hallucinate, confidently describing content not present in the input.

By Harshvardhan Saini, Samyak Jha, Yiming Tang, Dianbo Liu

llms rag agents fine-tuning multimodal benchmarks safety

One email a morning, machine-written

Related stories

Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation

Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation

MedPlex: Deep Vision-Language Co-Adaptation for Clinically Grounded Medical Segmentation

MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data

From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation

When Language Overwrites Vision: Over-Alignment and Geometric Debiasing in Vision-Language Models