arXiv Machine Learning
Aug 28

MM-Spectrum: Multimodal Multi-spectral Molecular Structural Elucidation with a Stable MoE Framework

MM‑Spectrum is a sparse Mixture‑of‑Experts framework designed to infer molecular structures from multimodal spectroscopic data. It introduces a modality‑aware routing mechanism that exposes spectral identity to the router, along with shared and interaction experts of heterogeneous capacities to capture both modality‑unique and cross‑modal synergistic information while reducing noise interference. Experiments across full‑modality, bimodal, and missing‑modality scenarios show consistent and substantial performance gains, supported by ablation studies and interpretability analyses.

By Hai-tao Yu, Nan Min, Zheng Fang, Hongyu Zhan, Yusen Tan, Yuhan Wang, Jun Xia
arXiv AI
Aug 25

Beyond RGB: Benchmarking and Enhancing MLLMs for Hyperspectral Image Understanding via Training-Free Reasoning Framework

arXiv:2604.08884v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on RGB image understanding, yet their ability to use spectral evide...

By Xinyu Zhang, Zurong Mai, Qingmei Li, Xiaoya Fan, Zjin Liao, Haoyuan Liang, Yibin Wen, Yuhang Chen, Chan Tsz Ho, Bi Tianyuan, Ruifeng Su, Zihao Qiang, Juepeng Zheng, Jianxi Huang, Yutong Lu, Haohuan Fu