arXiv Computation and Language

DamageScope: Vision-Language Retrieval at Scale for Disaster Damage Assessment from Satellite Imagery

DamageScope is a retrieval‑augmented framework that combines satellite imagery, Vision‑Language Models (VLMs), and Large Language Models (LLMs) to automate property damage assessment after natural disasters. It uses a Retrieval‑Augmented Generation (RAG) architecture to extract structured visual representations from satellite images, enabling interactive natural language queries. The system introduces a multi‑vector embedding‑based clustering algorithm that improves scalability and reduces indexing time by up to 14×, and a dual‑store data architecture that cuts LLM API calls, lowering operational cost and response latency by roughly 3×.

arXiv AI

Oct 2

Decoding the Disaster: Multi-Task Geospatial Reasoning with Vision-Language Models and Crowdsourced Imagery for Disaster Mapping

The paper introduces GRDisaster, a multi-task geospatial reasoning framework that leverages vision‑language models to interpret, geolocalize, and assess damage in crowdsourced disaster imagery. It builds on a new benchmark dataset of 26,340 images from PhotoMappers, linking volunteer geographic information, street‑view imagery, and remote sensing data across multiple disaster events from 2018 to 2024. GRDisaster combines deterministic and probabilistic cross‑view geolocalization with multi‑view fusion, and introduces spatial reasoning indicators to validate cross‑view matches and quantify disaster severity using expert‑verified annotations.

By Wenping Yin, Fabian Desuer, Ziqi Liu, Naixia Mou, Weijia Li, Pedram Ghamisi, Xiao Xiang Zhu, Hao Li

llms multimodal benchmarks safety

arXiv Computer Vision

Sep 21

DisasterInsight: A Building-Centric Benchmark for Evaluating Vision--Language Models in Disaster Response

DisasterInsight is a building‑centric benchmark designed to evaluate vision‑language models (VLMs) for disaster response. Built on the xBD satellite dataset, it adds OpenStreetMap‑derived functional labels to 134,108 building instances and offers 15 task types, including instance assessment, scene counting, multi‑instance reasoning, and structured report generation. Experiments show that VLMs excel at visible damage detection but struggle with building function, multi‑instance reasoning, counting, and grounded reporting, and instruction tuning only partially mitigates these gaps.

By Sara Tehrani, Yonghao Xu, Leif Haglund, Amanda Berg, Gulnaz Zhambulova, Michael Felsberg

llms fine-tuning benchmarks

Hugging Face Trending Papers

Aug 9

Integrated Multimodal AI System for Retrieval-Augmented Reasoning, Object Sensing, and Damage Analysis

This work presents a unified multimodal AI system for damage assessment that integrates retrieval-augmented generation (RAG) models, thermal spectrum perception, vision foundation model pipelines, and exploratory wireless signal sensing. A RAG component is developed to ground a locally hosted language model in project-specific documentation, including specialized damage level classification criteria to mitigate hallucinations during inference.

llms rag computer-vision multimodal safety

arXiv AI

Aug 11

Integrated Multimodal AI System for Retrieval-Augmented Reasoning, Object Sensing, and Damage Analysis

arXiv:2608. 08935v1 Announce Type: new Abstract: This work presents a unified multimodal AI system for damage assessment that integrates retrieval-augmented generation (RAG) models, thermal spectrum perception, vision foundation model pipelines, and exploratory wireless signal sensing.

By Kalelo Dukuray, Israel Pina, Evan Perez, Erika Ardiles-Cruz, Jie Wei

llms rag computer-vision multimodal safety

arXiv Computer Vision

Sep 16

DisasterInsight: A Multimodal Benchmark for Function-Aware and Grounded Disaster Assessment

arXiv:2601.18493v2 Announce Type: replace Abstract: Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-ce...

By Sara Tehrani, Yonghao Xu, Leif Haglund, Amanda Berg, Gulnaz Zhambulova, Michael Felsberg

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 16

Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning

arXiv:2606. 14963v1 Announce Type: cross Abstract: Timely and accurate disaster damage assessment is crucial for effective emergency response, resource allocation, and recovery.

By Tewodros Syum Gebre, Jagrati Talreja, Leila Hashemi-Beni

multimodal

arXiv Computer Vision

Sep 23

A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selection

arXiv:2602.01163v2 Announce Type: replace Abstract: Safe UAV emergency landing requires more than just identifying flat terrain; it demands understanding complex semantic risks (e.g., crowds, tempora...

By Chunliang Hua, Lei Zhang, Jiayang Sun, Chunlan Zeng, Xiao Hu

llms agents computer-vision multimodal benchmarks

arXiv Computer Vision

Oct 12

Revisiting the Task of Building Damage Assessment by Satellite Imaging

The paper revisits building damage assessment (BDA) from satellite imagery, emphasizing the need for a sensor‑agnostic, instance‑based task definition that can handle missing pre‑event data. It reviews existing datasets by modality, timing, and geography, critiques current single‑score evaluation practices, and proposes a more robust, ranking‑based multi‑score protocol. The authors also survey deep learning methods, highlight challenges in generalizing to new events and regions, and outline how these insights could shape future BDA challenges and data‑sharing practices.

By Anthony Cioppa, Virgile Blua, S\'ebastien Pi\'erard, Pietro Ceccato, Marc Van Droogenbroeck

computer-vision

arXiv AI

Jun 30

Multimodal and Multiscale Spatial-Temporal Semantic Search and Recommendation with AI Foundation Models

arXiv:2606. 28369v1 Announce Type: cross Abstract: Semantic search and recommendation of similar documents, such as news and reports about unusual environmental events (e.

By Yuanyuan Tian, Wenwen Li, Xiao Chen, Michael Brook, Michael Brubaker, Anna Liljedahl, Chitta Baral

llms rag multimodal

arXiv AI

Jun 17

Bridging Spatial And Frequency Views For Disaster Assessment: Benefits And Limitations

arXiv:2606. 17403v1 Announce Type: cross Abstract: Rapid assessment of building damage from satellite imagery is essential for effective disaster response and recovery.

By Shikha V. Chandel, Yadav Raj Ghimire, Timothy Agboada, Leila Hashemi-Beni

safety

arXiv AI

Jun 26

On-board Remote-Sensing Foundation Models for Unsupervised Change Detection of Disaster Events

arXiv:2606. 27018v1 Announce Type: cross Abstract: Remote Sensing Foundation Models (RSFMs) have emerged as a powerful alternative to supervised models for Earth Observation, allowing satellites to autonomously trigger high-resolution captures or adjust tasking parameters upon detecting an anomaly, thereby maximizing the utility of the mission's limited power and computational resources.

By S. Ram\'irez-Gallego

diffusion

arXiv Computer Vision

Aug 31

GeBDA: Building Damage Assessment as Text-Based Sequence Prediction

The paper proposes GeBDA, a method that treats Building Damage Assessment as a text-based sequence prediction task using a Vision‑Language Model. Instead of specialized architectures, it uses an autoregressive VLM to output a variable‑length list of bounding boxes with coordinates and damage labels. The authors demonstrate promising results on bi‑temporal satellite images using the open Gemma model and a tailored text prompt.

By Olivier Dietrich, Krishna Sapkota, Konrad Schindler, Genady Beryozkin

llms fine-tuning multimodal

Related stories