arXiv Machine Learning By Manmeet Singh, Somnath Luitel, Prabhjot Singh, Manraaj Banga, Naveen Sudharsan, Josh Durkee

AFDBench: A Reasoning-First AI Scientist for NationalWeather Service Forecast Discussions

Read the original on arXiv Machine Learning →

AFDBench is a new benchmark that evaluates how well large language models can generate professional Area Forecast Discussions (AFDs) for the National Weather Service by reasoning through structured AI weather forecast data. It contains 7,732 expert-written discussions paired with real forecast inputs and introduces three metrics—Met-Align, Style-Align, and Input-Grounding—to assess numerical accuracy, professional dialect adherence, and fidelity to source data. Zero-shot tests show open-source LLMs perform poorly on style and grounding, but reinforcement learning with Group Relative Policy Optimization nearly doubles style alignment and improves grounding, enabling a 7B-parameter model to write like a professional meteorologist.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Jul 28

HydroAgent: Formalizing Forecaster Expertise into Skill-Orchestrated Flood Forecasting Workflows

arXiv:2607. 23983v1 Announce Type: cross Abstract: Operational flood forecasting depends on tacit forecaster expertise that is difficult to formalize, audit, and transfer.

By Qingyi Yang, Siqian Qiu, Bing Li, Xu Shan, Jia Feng, Shunan Zhou, Xudong Zhou, Tiantian Xing, Jiale Guo, Xiaoyi Dong, Gaoyu Liu, Xiaohuan Liu, Haiqing Pu, Qingwen Deng, Xun Zhang, Zhongrun Xiang, Haiyang Qian, Ying Yan, Yongkang Xu, Nuo Lei, Tianlong Jia, Baoying Shan, Carlo De Michele
arXiv Machine Learning
Sep 25

Improving global precipitation forecasts with an AI weather model trained on satellite observations

The paper presents Laxmi, a retrained version of the AIFS weather model that uses satellite-based precipitation observations instead of ERA5 reanalysis data. Laxmi achieves a 19% improvement in global probabilistic accuracy, reduces drizzle overprediction by 33%, and boosts the 95th percentile Brier skill score by 57%. In a case study of 10 Indian tropical storms, Laxmi accurately forecasted 150 mm event-total precipitation in 7 events, outperforming both the original AIFS and the leading physical model IFS.

By Julian F. Schmitt, Bertrand Delorme, Robert C. King, Yashica Patodia, Tapio Schneider, Aditi Sheshadri, Ravi Jain
arXiv AI
Sep 11

Agent-Based ML-LLM Fusion with Self-Optimizing Prompts for Plateau Weather Alerts

The paper introduces SmartWeatherAgent, a three‑stage architecture that combines intent recognition, hazard prediction, and reasoning‑enhanced generation to improve tourism meteorological services. It fuses rule‑based methods with large language models and a LightGBM model enriched with highland‑specific features, achieving an F1‑Macro score of 0.605 and 1.60 ms latency on high‑wind, precipitation, and low‑temperature events. A 12‑round micro‑step prompt self‑optimization loop raises the composite warning quality score from 4.2 to 8.9, with notable gains in data source citation, physical mechanism explanation, and scientific rigor through explicit uncertainty statements.

By Shuai Yan, Yang Xu, Shan He
arXiv AI
Oct 9

Strategic Governance of AI Models in Earth Science

The article discusses how AI foundation models trained on weather and climate data are increasingly fine‑tuned for a wide range of Earth science tasks beyond traditional weather forecasting. It highlights that current evaluations rely mainly on benchmark skill metrics, which assess forecast accuracy but not the models’ representation of underlying physical processes, a gap that becomes critical under changing climate conditions. The authors propose five priorities for physical evaluation—covering training data, fine‑tuning, behavioral testing, mechanistic interpretability, and output validation—and recommend three actions for the next decade: open AI‑ready evaluation datasets, a shared physics‑based reporting standard, and a dedicated research program on model safety.

By Makoto Kelp, Amirhossein Arzani, Patricia Castellanos, Paul Griffiths, Ivan Higuera-Mendieta, Manuel Perez-Carrasco, Viral Shah, Patrick Obin Sturm, James Weber