Multimodal fusion learning (MFL) has shown great potential in the medical domain, where we are faced with disparate data modalities such as imaging, clinical records, and omics. However, existing MFL strategies face several major challenges.
Infrared-visible image fusion (IVIF) is pivotal for multimodal perception, yet reconciling the inherent information disparity between thermal and textural features remains a fundamental challenge. Existing prior-guided methods often rely on static constraints that induce optimization conflicts or utilize extrinsic semantic priors from large-scale foundation models (e.
Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity.
The paper introduces a self‑supervised neural network that unifies single‑frame Fresnel coherent diffraction imaging (CDI) and overlapped ptychography. By using a fixed, pre‑estimated probe and optimizing with a Poisson negative log‑likelihood objective, the method reconstructs object patches from either a single diffraction frame or multiple overlapping measurements, achieving high SSIM scores and a ten‑fold improvement in photon‑dose efficiency. Demonstrations on synthetic patterns and real datasets from APS and LCLS show robust, high‑throughput reconstructions, with a 36× speedup over iterative solvers for a 10,304‑frame workload.
By Oliver Hoidn, Steven Henke, Albert Vong, Aashwin Mishra, Apurva Mehta, Matthew Seaberg
arXiv:2607. 24110v1 Announce Type: cross Abstract: Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception.
By Minchong Chen, Xiaoyun Yuan, Minyu Cao, Jianing Zhang, Jun Zhang, Shuyang Liu, Xiaokang Yang
arXiv:2505.16157v3 Announce Type: replace
Abstract: Transformer-based models have made remarkable progress in image restoration (IR) tasks. However, the quadratic complexity of self-attention in Tran...
By Yuang Ai