The paper investigates whether entire Transformer blocks can be skipped during inference in two time‑series foundation models, Chronos‑Bolt and TimesFM 2.5. By applying block‑level masks selected on calibration windows, the authors show that moderate sparsity (e.g., 7/11 blocks for Chronos‑Bolt and 16/20 for TimesFM) preserves forecasting quality while reducing latency by roughly 11–13%. However, more aggressive skipping degrades performance on certain datasets, and dynamic, input‑dependent routing remains unproven across architectures.
By Max Bourgeat
arXiv:2607. 25504v1 Announce Type: cross Abstract: Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models.
By Bowen Wang, Chi Zhang, Diyou Shen, Renzo Andri, Navaneeth Kunhi Purayil, Luca Benini
AsyncCouple-Flow introduces a new framework for multi‑modal spatio‑temporal forecasting that tackles three key challenges: differing sampling rates, missing modalities, and autoregressive error accumulation. It employs a Modality‑Aware Token Sparsification module to produce equal‑length sequences, an Asynchronous Cross‑Modal Coupling Graph to fuse data under arbitrary asynchrony and missingness, and a Flow‑Matching Forecasting Head that models multi‑step prediction as a conditional ODE. Experiments on weather and traffic datasets demonstrate that the method outperforms state‑of‑the‑art baselines and remains robust even when up to two modalities are missing.
By Zhixiang Wu, Yining Liu, Bo Zhao, Szu-Yu Chen, Huiran Duan, Chu Lin, Chuanguang Yang
OceanMoE is a structured conditional sparse Mixture-of-Experts framework designed for long‑horizon multivariate ocean forecasting. It fuses cross‑variable information to build target‑specific local representations and performs content‑conditioned sparse routing at each spatial location, with the number of active experts adjusted by router confidence. Experiments on ORAS5 data show that OceanMoE reduces aggregate forecasting error and maintains lower geometric‑mean normalized RMSE compared to baselines, while expert allocation varies with prediction targets and locations.
By Yishun Zhu, Jian Wang
arXiv:2607. 24665v1 Announce Type: cross Abstract: Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows.
By Yanhao Jia, Jiepeng Wang, Haibin Huang, Chi Zhang, Erik Cambria, Xuelong Li
arXiv:2512. 03606v2 Announce Type: replace Abstract: Accurate marine wind forecasts are essential for safe navigation, ship routing, and energy operations, yet they remain challenging because observations over the ocean are sparse, heterogeneous, and temporally variable.
By Matteo Peduto, Qidong Yang, Jonathan Giezendanner, Devis Tuia, Sherrie Wang
arXiv:2406. 14399v4 Announce Type: replace Abstract: The development of Time-Series Forecasting (TSF) models is often constrained by the lack of comprehensive datasets, especially in Global Station Weather Forecasting (GSWF), where existing datasets are small, temporally short, and spatially sparse.
By Tao Han, Zhibin Wen, Zhenghao Chen, Dazhao Du, Song Guo, Lei Bai
WeatherNext 3 is a new AI‑driven global weather model that improves both spatial and temporal resolution by generating hourly forecasts at 0.1° resolution, matching the best physics‑based models. It incorporates low‑latency geostationary satellite data and learns to predict satellite‑derived precipitation, tropical cyclones, and station observations, enabling 2 m temperature and dewpoint predictions anywhere and anytime. By directly using raw observations instead of relying solely on analysis data, WeatherNext 3 sets a new state‑of‑the‑art for probabilistic medium‑range forecasting skill.
By Stephan Rasp, Boris Babenko, Dominic Masters, Andrew El-Kadi, Samier Merchant, Guy Shalev, Ilan Price, Fred Zyda, Remi Lam, Sasha Shysheya, Matthew Willson, Stratis Markou, Shreya Agrawal, Suhani Vora, Mohammed Alewi Hassen, Sunny Mak, Tom R. Andersson, Megan Bela, Akib Uddin, Nofar Peled Levi, Ben Gaiarin, Ferran Alet, Aaron Bell, Peter Battaglia, Alvaro Sanchez-Gonzalez
arXiv:2606. 26421v1 Announce Type: new Abstract: State-of-the-art medium-range AI weather models can outperform traditional Numerical Weather Prediction (NWP) but require massive training budgets.
By Cristiana Diaconu, Jonas Scholz, Aliaksandra Shysheya, Stratis Markou, Payel Mukhopadhyay, Miles Cranmer, Richard E. Turner
arXiv:2606. 31248v1 Announce Type: cross Abstract: Kilometer-scale convection shapes precipitation extremes, tropical organization, and cloud feedbacks, but most global atmospheric models approximate these processes at 25-100 km resolution.
By Zeyuan Hu, Akshay Subramaniam, Noel Keen, Tao Ge, Jaideep Pathak, Mohammad Shoaib Abbas, Suman Ravuri, Karthik Kashinath, Naser Mahfouz, Peter Caldwell, Mike Pritchard, Noah Brenowitz
arXiv:2606. 15346v1 Announce Type: cross Abstract: Spatio-temporal prediction supports radar/satellite nowcasting and city-scale traffic monitoring, but modern models are often too expensive for real-time deployment.
By Fuyan Zhang, Yuqi Li, Yingli Tian, Edmond S. L. Ho
arXiv:2606. 01155v1 Announce Type: cross Abstract: Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with limited data is not.
By Boqian Wu, Qiao Xiao, Patrik Okanovic, Tomasz Sternal, Maurice van Keulen, Mykola Pechenizkiy, Elena Mocanu, Torsten Hoefler, Decebal Constantin Mocanu