Towards Data Science

Your Model's MSE Is Lying to You: Part II

The article discusses how to handle uncertainty in probabilistic forecasting for physical signals, focusing on autoregressive rollout and uncertainty propagation. It is the second part of a series titled "Your Model's MSE Is Lying to You," which explores the limitations of mean squared error in evaluating predictive models. The post was published on Towards Data Science.

arXiv Machine Learning
Jun 10

When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms

arXiv:2606. 10868v1 Announce Type: new Abstract: Long-horizon autoregressive forecasting of oscillatory physical signals, such as seismograms, gravitational-wave strain, and similar wavefields is limited by error accumulation: as a causal model is fed its own outputs over hundreds of steps, small per-step errors compound into phase drift that pointwise metrics fail to detect.

By Waleed Esmail, Stuart Russell, Jana Klinge, Alexander Kappes, Christine Thomas
arXiv AI
Sep 25

SGA: Uncertainty Quantification for Multi-Step Forecasting in Time Series Foundation Models

The paper introduces SGA, a method for quantifying uncertainty in multi‑step forecasts from Time Series Foundation Models (TSFMs). SGA models all possible forecast branches as a directed acyclic graph, using the graph’s complexity—derived from topology and TSFM stochasticity—to bound and measure uncertainty. Experiments on 11 TSFMs across 27 datasets show that SGA outperforms existing uncertainty‑quantification methods, offers broader sampling coverage, and reveals that larger TSFMs tend to produce lower uncertainty estimates.

By Xin-Yu Hu, Shuang Liang, Cheng Feng, Shao-Qun Zhang
arXiv Machine Learning
1d ago

Physics-Refined Spatiotemporal Forecasting on Open-Boundary Hydrologic Graphs

The paper introduces a physics‑refined framework for spatiotemporal forecasting on open‑boundary hydrologic graphs, addressing instability caused by missing external boundary forcing. It learns ghost node proxies to approximate unobserved inputs and applies two physics refiners: one enforcing local consistency with two‑hop neighbors, and another using a physics‑guided graph neural operator to reduce long‑horizon drift. Experiments on two real‑world hydrologic graphs show improved prediction accuracy and stability compared to existing learning‑based and physics‑informed models.

By Haoyang Jiang, Zhengui Wang, Shenghan Gao, Y. Joseph Zhang, Xingquan Zhu, Yi He
arXiv Machine Learning
Jul 13

Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction

arXiv:2604. 16238v2 Announce Type: replace Abstract: Decision-makers rely on weather forecasts to plant crops, manage wildfires, allocate water and energy, and prepare for weather extremes.

By Hannah Guan, Soukayna Mouatadid, Paulo Orenstein, Judah Cohen, Haiyu Dong, Zekun Ni, Jeremy Berman, Genevieve Flaspohler, Alex Lu, Jakob Schloer, Joshua Talib, Jonathan A. Weyn, Lester Mackey
arXiv Machine Learning
Sep 7

Advancing Subseasonal Forecasting with Machine Learning

The paper introduces Probabilistic Bias Correction (PBC), a machine learning framework that learns to correct historical probabilistic forecasts, thereby reducing systematic errors in subseasonal weather predictions. Applied to leading dynamical and AI models from ECMWF, PBC doubles the AI system’s modest subseasonal skill and improves the operationally-debiased dynamical model for most pressure, temperature, and precipitation targets. In ECMWF’s 2025 real‑time forecasting competition, PBC’s global forecasts ranked first across all weather variables and lead times, outperforming multiple operational and ensemble models.

By Hannah Guan, Soukayna Mouatadid, Paulo Orenstein, Judah Cohen, Haiyu Dong, Zekun Ni, Jeremy Berman, Genevieve Flaspohler, Alex Lu, Jakob Schloer, Joshua Talib, Jonathan A. Weyn, Lester Mackey
arXiv Machine Learning
Jul 24

SPECTRA: State-Space Exogenous Context and Temporal-Frequency Resolution Architecture for Probabilistic Energy Forecasting

arXiv:2607. 20587v1 Announce Type: cross Abstract: Modern power systems increasingly require probabilistic forecasts amid interacting uncertainties from renewable intermittency, flexible demand, market volatility, and weather-dependent generation.

By Hang Ye, Xinyan Jiang, Yuedong Shi, Yangxin Zhu, Jianming Wei, Tian Zheng, Xiaoying Zheng, Yongxin Zhu
arXiv AI
Sep 10

Resolving sources of uncertainty in AI weather forecasting

The paper introduces Pangu‑Bayes, a probabilistic forecasting hierarchy that separates atmospheric‑state uncertainty from learned‑model uncertainty as distinct stochastic variables, allowing cross‑flow perturbations of the evolving state with Bayesian parameter samples. In tests on 90 held‑out 2023 tropical cyclones, Pangu‑Bayes reduces track, pressure, and wind errors by 54.2%, 17.2%, and 24.9% respectively, and improves rapid‑intensification detection. The study finds that atmospheric‑state variability more consistently improves track prediction, while learned‑model variability more often enhances intensity prediction, demonstrating how model‑defined uncertainty resolution can be linked to target‑dependent value and dynamical interpretation.

By Wenbo Hu, Xinlei Xiong, Shuxun Zhou, Kaifeng Bi, Lingxi Xie, Jun Zhu, Richang Hong, Qi Tian