The paper introduces FlexibleFusion, a method for infrared-visible object detection that adapts to both complete and missing-modality scenarios. It employs a Modality-Aware Experts Collaboration mechanism to selectively fuse cross-modal or intra-modal pathways, and a Residual Self-Paced Entropic Optimal Transport module to align heterogeneous feature distributions without heavy optimization. Experiments demonstrate consistent performance across various modality configurations.
By Yue Zhao, Hua Yu, Yukun Zhao, Yuzhi Zhang, Maoguo Gong, Xin Mei, Zhuping Hu, Yanchi Li, A. K. Qin
arXiv:2608.21099v1 Announce Type: cross
Abstract: Multi-modal object detection is essential for robust scene understanding in challenging conditions, including low-light and adverse environments. Rec...
By Jiekang Feng, Zhihe Fan, Yunqi Zhu, Xinjie Yao, Yueying Zhang, Yike Gao, Ranxin Li, Guanzuo Chen
Infrared-visible image fusion (IVIF) is pivotal for multimodal perception, yet reconciling the inherent information disparity between thermal and textural features remains a fundamental challenge. Existing prior-guided methods often rely on static constraints that induce optimization conflicts or utilize extrinsic semantic priors from large-scale foundation models (e.
RoES is a Rotational Equivariant Selective-frequency fusion network that dynamically separates low- and high-frequency components of infrared-visible images. It uses a trainable rotation-enhanced updater to decouple frequencies, then fuses them with a dual-branch module: a rotation-equivariant Mamba for low-frequency structural dependencies and a polar spectral attention Dual-Fourier block for high-frequency detail refinement. Experiments show RoES outperforms existing methods in fusion quality and downstream object detection, offering a robust multimodal fusion solution.
By Jiabao Wang, Wenjian Liu, Yaoming Cai, Gengyu Zhang, Boyan Zhao, Zijia Zhang, Yao Ding, Xiaobo Liu
Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity.
arXiv:2608. 02092v2 Announce Type: replace Abstract: Deep multimodal fusion for object detection has demonstrated good performance through mining modal characteristics.
By Guandi Wang, Ming Li, Yunsen Xing, Junle Liu
The paper introduces an Attention-Driven Complementarity Resampling framework to enhance cross-modality object detection. It employs a shared channel spatial attention mechanism that exchanges semantic masks between modalities, encouraging the backbone to learn generalized features. Additionally, a learnable channel competition module samples and aggregates features channel‑wise, improving robustness and achieving competitive results on multiple datasets.
By Guandi Wang, Ming Li, Yunsen Xing, Junle Liu
arXiv:2606. 29136v1 Announce Type: cross Abstract: Event cameras capture sparse brightness changes with high temporal resolution and high dynamic range, compensating for the deficiencies of the conventional RGB frames.
By Yu Li, Yuenan Hou, Yingmei Wei, Jiangming Chen, Yanming Guo
The paper introduces W2F, a visual‑semantic mutual‑reinforcing framework for multi‑modal image fusion (MMIF). It enhances feature complementarity with a segmentation‑guided fusion loss and an embedded information restoration prior, and employs interactive gated mixed attention (IGM‑Att) and progressive coupling attention (PC‑Att) to fuse local and global information. A collaborative mechanism with a feature search module and gradient correction further aligns visual and semantic fusion, leading to state‑of‑the‑art visual appeal and decision accuracy.
By Hao Zhang, Xuhui Zuo, Jiayi Ma
Multi-modality data from different sensors provides rich complementary information for 3D perception, becoming an essential component in reliable autonomous driving systems. Current research typically designs intricate and complex fusion strategies to integrate information from multimodal data on a unified bird's-eye-view (BEV) feature map for the joint learning of multiple perception tasks.
arXiv:2607. 08076v1 Announce Type: cross Abstract: The complementary information between RGB and IR images can significantly enhance object detection performance under extreme conditions.
By Wenhao Dong, Xiaoyan Luo, Linlin Yang, Haodong Zhu, Xiaorong Shi, Guodong Guo, Baochang Zhang
arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.
By Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang, Xinpei Wang, Weitong Chen