← Back to all news
arXiv Machine Learning

Bidirectional Multimodal Fusion of Sky Images and Time-Series for Solar Forecasting with Large Language Models

The paper introduces SolCloudLLM, a large language model–based framework that fuses sky‑image patches with time‑series data through bidirectional multimodal fusion for short‑term solar forecasting. Experiments on the SIRTA and SKIPP'D datasets show that SolCloudLLM outperforms existing baselines, achieving up to a 25.4% reduction in mean squared error, especially under cloudy conditions and in few‑shot scenarios.

Related stories

arXiv Machine Learning
Oct 23

Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

The paper introduces a dual‑framework Vision Transformer (ViT) method for estimating diffuse horizontal irradiance (DHI) from sky images in data‑scarce settings. It combines physics‑based atmospheric parameters, a sequential pipeline that first predicts synthetic global horizontal irradiance (GHI), and a dual‑parallel architecture that processes raw and physics‑informed overlay‑enhanced fisheye images. Tested on Chilbolton Observatory data, the approach achieves RMSE within 2–6 W/m² of a ViT trained on a full dataset, demonstrating a scalable, cost‑effective solution for solar resource assessment when long‑term sky imagery is limited.

By David Hamlyn, Sunny Chaudhary, Tasmiat Rahman
llmscomputer-visionbenchmarks
More like this →
arXiv Machine Learning
Oct 23

GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images

GateFormer is a vision‑transformer model that forecasts intra‑hour global horizontal irradiance (GHI) from sky images by decomposing the forecast into a trajectory prediction and a delta prediction, then adaptively weighting these two outputs. It incorporates recent GHI measurements into the visual token stream, enabling self‑attention between image patches and irradiance data. On multiple benchmarks (NREL‑TSI, Chilbolton, SIRTA) GateFormer achieves the lowest final‑horizon root‑mean‑square error, outperforming state‑of‑the‑art transformers and demonstrating significant gains during rapid irradiance ramps.

By David Hamlyn, Sunny Chaudhary, Jonathon Hare, Tasmiat Rahman
llmsmultimodalbenchmarks
More like this →
Hugging Face Trending Papers
Oct 22

GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images

GateFormer is a dual‑decoder Vision Transformer that forecasts intra‑hour global horizontal irradiance (GHI) from sky images by embedding observed GHI into the visual token stream and predicting both a future GHI trajectory and the final‑horizon change. The two predictions are adaptively weighted in prediction space, with magnitude‑ and direction‑supervision to reduce remaining error. On multiple benchmarks, GateFormer achieves the lowest final‑horizon RMSE, outperforming state‑of‑the‑art transformers and showing significant gains especially during positive ramp events.

llmsmultimodalbenchmarks
More like this →
arXiv Machine Learning
Sep 10

SolarBench: A global solar energy nowcasting benchmark

SolarBench is an open global benchmark for image-based solar nowcasting that consolidates over six million sky and satellite images from 11 sites across a decade, paired with irradiance, PV output, and atmospheric data. The benchmark includes a toolbox for reproducible data access, processing, model development, and evaluation. Using SolarBench, the authors benchmark representative models, uncover a gap between average forecasting accuracy and the capture of rapid solar fluctuations, quantify predictability across cloud regimes, and demonstrate data‑efficient adaptation to new PV systems.

By Yuhao Nie, Stephen Campbell, Quentin Paletta, Liwenbo Zhang, Tao Jing, Samer Chaaraoui, Jonathan Giezendanner, Andea Scott, Tao Sun, Cong Feng, Max Aragon, Jacques Camier, Adam Jensen, Florian Kotthoff, Yuexing Yang, Yang Ming, Mengying Li, Stefanie Meilinger, Yupeng Wu, Adam Brandt, Sherrie Wang
benchmarks
More like this →
arXiv Machine Learning
Sep 4

From Nowcasting to Forecasting: Adapting a Reanalysis-Trained

The paper introduces CloudCast v2, a machine‑learning model that forecasts 12‑hour cloud‑cover from satellite‑derived initial conditions. Trained first on the Copernicus European Regional Reanalysis to learn cloud‑evolution dynamics, it is then adapted to real satellite data using conditional flow matching, a generative technique that conditions noise on observed cloud fields and NWP inputs. CloudCast v2 achieves a 10 % reduction in mean absolute error compared to its predecessor and surpasses it in spatial skill after 3–6 hours, extending useful forecasting beyond the typical 1–3‑hour nowcasting window while preserving satellite‑level spatial detail.

By Mikko Partio, Leila Hieta, Ossi Laine
diffusion
More like this →