← Back to all news
arXiv Machine Learning By Ken Chen, Maneesha Perera, Wei Wang, Sachith Seneviratne, Hansani Weeratunge, Saman Halgamuge

Bidirectional Multimodal Fusion of Sky Images and Time-Series for Solar Forecasting with Large Language Models

Read the original on arXiv Machine Learning →

The paper introduces SolCloudLLM, a large language model–based framework that fuses sky‑image patches with time‑series data through bidirectional multimodal fusion for short‑term solar forecasting. Experiments on the SIRTA and SKIPP'D datasets show that SolCloudLLM outperforms existing baselines, achieving up to a 25.4% reduction in mean squared error, especially under cloudy conditions and in few‑shot scenarios.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 23

Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images

The paper introduces a dual‑framework Vision Transformer (ViT) method for estimating diffuse horizontal irradiance (DHI) from sky images in data‑scarce settings. It combines physics‑based atmospheric parameters, a sequential pipeline that first predicts synthetic global horizontal irradiance (GHI), and a dual‑parallel architecture that processes raw and physics‑informed overlay‑enhanced fisheye images. Tested on Chilbolton Observatory data, the approach achieves RMSE within 2–6 W/m² of a ViT trained on a full dataset, demonstrating a scalable, cost‑effective solution for solar resource assessment when long‑term sky imagery is limited.

By David Hamlyn, Sunny Chaudhary, Tasmiat Rahman
llmscomputer-visionbenchmarks
More like this →
arXiv Machine Learning
Oct 23

GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images

GateFormer is a vision‑transformer model that forecasts intra‑hour global horizontal irradiance (GHI) from sky images by decomposing the forecast into a trajectory prediction and a delta prediction, then adaptively weighting these two outputs. It incorporates recent GHI measurements into the visual token stream, enabling self‑attention between image patches and irradiance data. On multiple benchmarks (NREL‑TSI, Chilbolton, SIRTA) GateFormer achieves the lowest final‑horizon root‑mean‑square error, outperforming state‑of‑the‑art transformers and demonstrating significant gains during rapid irradiance ramps.

By David Hamlyn, Sunny Chaudhary, Jonathon Hare, Tasmiat Rahman
llmsmultimodalbenchmarks
More like this →
Hugging Face Trending Papers
Oct 22

GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images

GateFormer is a dual‑decoder Vision Transformer that forecasts intra‑hour global horizontal irradiance (GHI) from sky images by embedding observed GHI into the visual token stream and predicting both a future GHI trajectory and the final‑horizon change. The two predictions are adaptively weighted in prediction space, with magnitude‑ and direction‑supervision to reduce remaining error. On multiple benchmarks, GateFormer achieves the lowest final‑horizon RMSE, outperforming state‑of‑the‑art transformers and showing significant gains especially during positive ramp events.

llmsmultimodalbenchmarks
More like this →