arXiv Statistics ML

STCFormer: Adaptive Spatio-Temporal Modeling with Dynamic Cluster Transformer for Station-based Weather Forecasting

STCFormer is an adaptive spatio‑temporal Transformer that dynamically clusters weather stations within each temporal patch, combining fine‑grained local attention inside clusters with global attention over regional summaries. The model’s design is supported by a Lipschitz upper bound that suggests robustness benefits, and it achieves the lowest 24‑hour mean squared error across eight temperature and wind forecasting tasks on three real‑world datasets, ranking first or second in 47 of 48 comparisons. Ablation studies and case analyses confirm the advantages of locally adaptive grouping and complementary local‑global interactions.

arXiv Machine Learning
Sep 22

UniGIO: Unified Generative Global In-situ Weather Modeling from Spatiotemporal Incomplete Observations

UniGIO is a generative framework that models global in‑situ weather dynamics directly from incomplete GIO data, unifying forecasting, imputation, and generation across arbitrary missing ratios. It employs an Observation Mixer, Event Aligner, Adaptive Temporal Mixer, and a Mixture‑of‑Experts structure to capture station‑level complementarity, temporal dependencies, and extreme events, refining outputs with a Local Refiner. Experiments on the Weather‑5K dataset show state‑of‑the‑art performance, improving accuracy, fidelity, and extreme event capture by 11%, 12%, and 5% respectively.

By Songru Yang, Zili Liu, Tao Han, Ben Fei, Lei Bai, Chang Liu, Zhengxia Zou, Xiangyang Ji, Wanli Ouyang, Zhenwei Shi
arXiv AI
Sep 2

Global Attention with Linear Complexity for Exascale Generative Data Assimilation in Earth System Prediction

The paper introduces STORM, a one‑stage generative AI framework that reformulates Earth system data assimilation as diffusion‑based Bayesian posterior sampling, replacing costly PDE ensemble forecasts with scalable AI inference. STORM employs a spatiotemporal transformer with a global‑attention algorithm that reduces computational complexity from quadratic to linear, enabling high‑resolution, long‑context modeling. The system scales to 74,400 GPUs on Frontier, achieving 96–99 % strong‑scaling efficiency and up to 6 ExaFLOPs sustained BF16 throughput, while supporting 32,768‑member ensembles for uncertainty quantification in just 34 seconds on 4,096 GPUs, and demonstrates improved hurricane tracking and climate reanalysis accuracy.

By Xiao Wang, Zezhong Zhang, Isaac Lyngaas, Hong-Jun Yoon, Jong-Youl Choi, Siming Liang, Janet Wang, Hristo G. Chipilski, Ashwin M. Aji, Feng Bao, Peter Jan van Leeuwen, Dan Lu, Guannan Zhang
arXiv AI
Jul 15

Do We Really Need Transformers for Global Spatial Information Extraction in Traffic Forecasting?

arXiv:2607. 12462v1 Announce Type: new Abstract: Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each individual node and all nodes across the traffic network.

By Qihang Zhang, Siyao Zhang, Letao Kang, Wenzhe Liang, Miao Zhang, Zhao Zhang
arXiv AI
Jul 31

Do We Really Need Adaptive Global Spatial Attention for Traffic Forecasting?

arXiv:2607. 12462v2 Announce Type: replace Abstract: Existing traffic forecasting models commonly focus on extracting spatial dependencies, particularly global spatial information, which characterizes the representations obtained through interactions between each node and all nodes across the traffic network.

By Qihang Zhang, Siyao Zhang, Letao Kang, Wenzhe Liang, Miao Zhang, Zhao Zhang