arXiv AI

EarthAlign: Learning Language-Aligned Representations of Earth System Data

arXiv AI
Jun 10

GCA Framework: A GCC Countries-Grounded Dataset and Agentic Pipeline for Climate Decision Support

arXiv:2604. 12306v3 Announce Type: replace-cross Abstract: Climate decision-making in the GCC states increasingly demands systems that can translate heterogeneous scientific and policy evidence into actionable guidance, yet general-purpose large language models (LLMs) remain weak both in region-specific climate knowledge and grounded interaction with geospatial and forecasting tools.

By Muhammad Umer Sheikh, Khawar Shehzad, Salman Khan, Fahad Shahbaz Khan, Muhammad Haris Khan
arXiv Machine Learning
Aug 4

Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams

arXiv:2608. 00012v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to interpret Earth observation data, yet their capability to support real-world disaster emergency response remains insufficiently evaluated.

By Fengxiang Wang, Qiuyang Yu, Yueying Li, Mingshuo Chen, Chengchi Fei, Kaiyi Xu, Lixin Gu, Wangxu Wei, Junchao Gong, Lipeng Ma, Jiong Wang, Fenghua Ling, Wenlong Zhang, Xue Yang, Wenjing Yang, Ben Fei, Long Lan
arXiv Machine Learning
5d ago

Domain-specific language alignment of multimodal embeddings towards explainable predictions

The paper introduces a framework that aligns multimodal embeddings with language by using captioning functions on interpretable auxiliary data, a soft contrastive loss, and a rank-based concept retrieval task. Applied to Earth observation embeddings, the method improves species distribution modelling, achieving state‑of‑the‑art performance on most metrics and better concept retrieval than prior vision‑language models. The aligned concepts enable recovery of species‑habitat associations and allow domain experts to interpret model predictions in natural language.

By Gabriel\.e Tij\=unaityt\.e, Thijs L. van der Plas, Mohamed-Bachir Belaid, Rob Knapen, Maciej J. Soja, Ioannis N. Athanasiadis
arXiv Machine Learning
Jul 20

DELUGE: Towards Continental-Scale Daily Pluvial Flood Damage Prediction via Interpretable Conditioning on Foundation Model Embeddings

arXiv:2607. 16050v1 Announce Type: new Abstract: Pluvial (rainfall-driven) flooding accounts for 45% of National Flood Insurance Program (NFIP) claims in the United States and is harder to predict than its riverine and coastal counterparts, with existing approaches limited to coarse resolution, regional domains, or computationally intensive process-based models unsuitable for daily continental-scale use.

By Yuya Kawakami, Daniel Cayan, Dongyu Liu, Kwan-Liu Ma, Tom Corringham
arXiv Machine Learning
Aug 19

TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts

The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.

By Jiafeng Lin, Yuxuan Wang, Huakun Luo, Jianmin Wang, Zhongyi Pei
arXiv Machine Learning
Aug 28

SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting

SAGE is a CLIP-based framework that augments vision‑language time series forecasting by incorporating variable‑specific semantic and statistical information. It processes frequency‑enhanced patches and variable tokens through a CLIP text encoder, while a frozen CLIP vision encoder aligns rendered series with temporal representations via a contrastive objective. The approach achieves state‑of‑the‑art accuracy on eight long‑term benchmarks and M4, with ablations showing complementary gains from multimodal alignment and variable‑level knowledge.

By Haizhao Fan, Xinyi Le
arXiv Machine Learning
Sep 16

AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting

AsyncCouple-Flow introduces a new framework for multi‑modal spatio‑temporal forecasting that tackles three key challenges: differing sampling rates, missing modalities, and autoregressive error accumulation. It employs a Modality‑Aware Token Sparsification module to produce equal‑length sequences, an Asynchronous Cross‑Modal Coupling Graph to fuse data under arbitrary asynchrony and missingness, and a Flow‑Matching Forecasting Head that models multi‑step prediction as a conditional ODE. Experiments on weather and traffic datasets demonstrate that the method outperforms state‑of‑the‑art baselines and remains robust even when up to two modalities are missing.

By Zhixiang Wu, Yining Liu, Bo Zhao, Szu-Yu Chen, Huiran Duan, Chu Lin, Chuanguang Yang