arXiv Computer Vision By Raul Balmez

VesperMUSE: Multi-Modal Fusion Vision Transformer for Image Semantic Segmentation Under Diverse Light Conditions

Read the original on arXiv Computer Vision →

VesperMUSE is a multi‑modal vision transformer that enhances image semantic segmentation under varied lighting by combining physics‑informed low‑light adaptation, thermal‑guided low‑light image enhancement, and surface normal maps. The framework fuses RGB, thermal, and normal features through a modality‑agnostic self‑attention mechanism, enabling robust segmentation without architectural changes for different sensor setups. Extensive tests show that VesperMUSE recovers missing visual cues and achieves state‑of‑the‑art performance in extreme, unstructured environments.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI
Sep 2

SARTM: Segment Any RGB Thermal Model with Language aided Distillation

The paper introduces SARTM, a framework that adapts the Segment Anything Model (SAM) for RGB‑thermal (RGB‑T) semantic segmentation. It fine‑tunes SAM with LoRA layers, incorporates language guidance, and employs a Cross‑Modal Knowledge Distillation module to bridge modality gaps. The approach also modifies the segmentation head and adds an auxiliary semantic head, achieving superior performance on MFNET, PST900, and FMB benchmarks.

By Dong Xing, Jinhe Zhang, Hang Yang, Yuqing Wang
arXiv Computer Vision
Sep 14

RA-SOD: Reliability-Aware RGB-T Salient Object Detection under Modality Degradation

RA‑SOD is a new RGB‑Thermal salient object detection framework that explicitly models the reliability of each modality. It introduces a reliability‑conditioned representation, an uncertainty‑guided dual‑stream refinement, and a pixel‑wise modality competition mechanism to adaptively compensate degraded features and suppress unreliable evidence. Experiments on four benchmarks show that RA‑SOD achieves state‑of‑the‑art performance and remains robust under severe modality degradation.

By Hongbo Gao, Zhengyu Li, Xueru Nie, Dihao Zhu, Lijun Zhao, Yunke Wang, Chang Xu
Hugging Face Trending Papers
Aug 13

P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation

Infrared-visible image fusion (IVIF) is pivotal for multimodal perception, yet reconciling the inherent information disparity between thermal and textural features remains a fundamental challenge. Existing prior-guided methods often rely on static constraints that induce optimization conflicts or utilize extrinsic semantic priors from large-scale foundation models (e.

arXiv Computer Vision
Sep 24

S2A:Semantic-to-Spatial Alignment for Alignment-Free RGB-T Salient Object Detection

The paper introduces S2A, a semantic-to-spatial alignment framework designed for alignment‑free RGB‑T salient object detection. It employs a global‑guided hierarchical fusion module to refine intra‑modal features, an alignment‑free cross‑modal channel attention module to exchange semantic information, and a spatial deformable cross‑attention module to recover local spatial correspondence. These components collectively reduce misalignment‑induced feature contamination and achieve competitive performance on public benchmarks without additional bells and whistles.

By Qiangqiang Zhou, Yang Luo, Yong Chen, Jiawei Xu