arXiv AI
Beyond RGB: Benchmarking and Enhancing MLLMs for Hyperspectral Image Understanding via Training-Free Reasoning Framework
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computer Vision
Aug 31
HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone
HyperVision introduces the first ground‑based hyperspectral pre‑trained backbone, addressing challenges of varying spectral configurations, limited annotations, and dataset diversity. It employs a channel‑adaptive dynamic embedding to unify heterogeneous inputs, a multi‑source pseudo‑labeling strategy combining SAM2 spatial cues with HyperFree spectral details, and cross‑modal knowledge distillation from a pre‑trained RGB vision model. Trained on 15k images from 26 datasets, HyperVision achieves significant improvements—up to 16.3% relative gain in hyperspectral semantic segmentation, 2.1% in object tracking AUC, and 35.5% reduction in salient object detection MAE—while requiring only head‑only adaptation.
arXiv Computer Vision
Sep 30
HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing
HyperSAM is a promptable foundation model for hyperspectral remote sensing that integrates a data‑centric synthesis pipeline with a spectral adaptation architecture based on Segment Anything Model 3 (SAM3). The model generates full‑spectrum hyperspectral cubes from high‑resolution multispectral imagery using a physics‑informed abundance‑transfer generator, and employs SAM3‑derived pseudo‑masks for object‑centric supervision. With a frozen SAM3 RGB branch, a trainable hyperspectral encoder, ControlNet‑style feature injection, and a mixture‑of‑experts mask refiner, HyperSAM demonstrates strong generalization across diverse hyperspectral tasks such as classification, anomaly detection, change detection, target detection, and airborne oil‑spill mapping.
Hugging Face Trending Papers
Sep 29
HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing
Hyperspectral remote sensing provides dense spectral measurements that are indispensable for material-level Earth observation, yet the construction of a general-purpose hyperspectral foundation model...
arXiv AI
Sep 4
Exploring the Potential of Contrastive Language-Image Pre-training for Multi-Source Remote Sensing Data
The paper introduces OmniRSCLIP, an end‑to‑end contrastive learning framework that extends the CLIP architecture to handle heterogeneous remote sensing sensors such as SAR, multi‑spectral imaging, and hyperspectral imaging. It achieves this by employing Spectral‑Spatial Basis Decomposition to adapt arbitrary‑channel inputs without losing pretrained visual knowledge, and a spectral‑context‑aware mask‑based contrastive learning scheme to improve fine‑grained image‑text alignment. The authors also build OmniRS5M, a large‑scale image‑text corpus covering multiple sensor modalities, and demonstrate that OmniRSCLIP maintains strong RGB performance while effectively supporting these diverse remote sensing data types.
arXiv Computer Vision
Sep 3
Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?
The paper introduces VertiCue-Bench, a diagnostic benchmark designed to test whether multimodal large language models (MLLMs) can perceive, ground, and utilize vertical structure information in remote-sensing natural scenes. It presents a three-stage framework—Perception, Grounding, Utilization—and a Representation Intervention Spectrum across various modalities to evaluate ten state-of-the-art models. The study finds a significant Vertical Structure Utilization Gap: while models show some geometric perception, they struggle to accurately link vertical evidence to spatial entities and incorporate it into semantic decisions.
arXiv Computer Vision
Sep 15
From Words to Wavelengths: VLMs for Few-Shot Multispectral Object Detection
arXiv:2512.15971v2 Announce Type: replace Abstract: Multispectral object detection is critical for safety-sensitive applications such as autonomous driving and surveillance, where robust perception u...