arXiv Machine Learning

Extreme Low-State Quantized Transformer for Electricity-Load Forecasting

The paper explores aggressive low‑state weight quantization for Transformer models used in electricity‑load forecasting. Experiments with 3, 5, 7, and 16 discrete weight states on two datasets show that post‑training quantization degrades performance, but quantization‑aware fine‑tuning recovers most of the loss. A novel accumulated‑update thresholding technique further reduces weight updates and state transitions with minimal impact on error, demonstrating that compact, low‑activity models can still deliver accurate forecasts.

arXiv Machine Learning

Oct 28

GateTS: Versatile and Efficient Forecasting via Attention-Inspired routed Mixture-of-Experts

GateTS introduces a streamlined univariate forecasting model that merges sparse Mixture-of-Experts computation with an attention-inspired gating mechanism, eliminating the need for auxiliary load‑balancing losses. The architecture achieves superior predictive accuracy across both short‑ and long‑term horizons, including intermittent signals, while using far fewer parameters than leading transformer models such as PatchTST. Empirical tests on diverse datasets show GateTS is computationally more efficient than LSTM, enabling cost‑effective inference for real‑world time‑series applications.

By Kyrylo Yemets, Mykola Lukashchuk, Ivan Izonin

llms benchmarks

arXiv Machine Learning

Jun 15

Trend-Aware Multi-Task Learning for Short-Term Energy Forecasting

arXiv:2511. 09789v3 Announce Type: replace Abstract: Short-term energy forecasting plays an important role in real-time operational decision-making, such as electricity market bidding and power system dispatch, where both numerical accuracy and correct directional signals are essential.

By Fulong Yao, Wanqing Zhao, Chao Zheng, Xiaofei Han

benchmarks

arXiv AI

Jul 15

Learning-based Probabilistic Load Forecasting with Post-hoc and In-model Uncertainty

arXiv:2607. 12730v1 Announce Type: cross Abstract: Smart-building load forecasters are often trained offline on dense, multivariate, high-frequency data, but deployment may provide only hourly, feature-limited inputs.

By Sarah Al-Shareeda, Gulcihan Ozdemir, Heung Seok Jeon

llms

arXiv AI

Sep 10

Assessing Covariate-Informed Grid Load Forecasting with a Time-Series Foundation Model

arXiv:2609.06656v1 Announce Type: cross Abstract: Modern power systems are growing increasingly complex as they integrate diverse generation sources to meet rising demand, making accurate load foreca...

By Varsha Pendyala, Yiwei Fu, Weizhong Yan, Nurali Virani

fine-tuning benchmarks

arXiv Machine Learning

Jun 17

Delta-Based Target Reformulation for Short-Term Electricity Load Forecasting Using LSTM and Transformer Models

arXiv:2606. 17692v1 Announce Type: new Abstract: Accurate short-term electricity load forecasting is critical for the reliable and economic operation of modern power systems, under non-stationarity arising from weather variability, calendar effects, and evolving consumption patterns.

By Vansh Bansal

llms benchmarks safety

arXiv Machine Learning

Jul 20

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

arXiv:2607. 15705v1 Announce Type: new Abstract: Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems.

By Matthias Hertel, Sebastian P\"utz, Jonathan Kolar, Benjamin Sch\"afer, Ralf Mikut, Veit Hagenmeyer

llms benchmarks

arXiv AI

Oct 21

Model- and Layer-Level Closed-Loop Forecasting with SSM-Based Spiking Neural Networks

arXiv:2610.28495v1 Announce Type: cross Abstract: Deep State-Space Models (Deep SSMs) have shown promising performance and computational benefits in processing of time-series data with complex tempor...

By Erik Stolt, Sanja Karilanova, Ayca Ozcelikkale

efficiency benchmarks

arXiv Machine Learning

Sep 10

Sustained Performance and Energy Accounting for Nonlinear Forecasting Across Classical and Simulated Quantum Models

arXiv:2510.25183v2 Announce Type: replace-cross Abstract: Energy-efficient AI should be evaluated across the full application pipeline, not only by lowest error or shortest training time. We study th...

By Avyay Kodali, Priyanshi Singh, Pranay Pandey, Krishna Bhatia, Shalini Devendrababu, Srinjoy Ganguly

llms efficiency benchmarks

arXiv AI

Oct 21

Parameter-Efficient Adaptation of Time Series Foundation Models for Electrical Load Forecasting

The study evaluates how well time‑series foundation models (TSFMs) can be adapted for electrical load forecasting. It compares pretrained inference, full fine‑tuning, and low‑rank adaptation (LoRA) across four load aggregation levels, using MOIRAI 1.0 Large as the main backbone and Chronos‑2 as a secondary model. LoRA achieves a 7.5% average NMAE reduction on MOIRAI while updating only about 1% of the parameters, offering a strong accuracy‑efficiency trade‑off, though benefits vary in out‑of‑distribution scenarios.

By Aliullah, Julian Oelhaf, Edouard Fouch\'e, Jessica Deuschel, Christian Bergler, Andreas Maier, Siming Bayer

fine-tuning

arXiv Machine Learning

1d ago

The Quantization Benefits of Residual-Free Transformers

arXiv:2605.25880v2 Announce Type: replace Abstract: Large-scale transformer training and deployment are increasingly constrained by the transfer of activations, gradients, and optimizer states across...

By Yiping Ji, Mahalakshmi Sabanayagam, Peyman Moghadam, Hemanth Saratchandran, Simon Lucey

llms efficiency

arXiv Machine Learning

Sep 29

Aurora-X: Built for Extreme Time Series Forecasting

Aurora‑X is a billion‑parameter time‑series foundation model designed for extreme forecasting tasks. It employs a progressive curriculum that starts with channel‑independent pretraining, then adds cross‑variable dependencies, variable context and horizon lengths, and optional future covariates during mid‑training. A variable‑resolution post‑training stage allows adjustable temporal spans per token at inference, while a pattern‑guided mixture‑of‑experts expands capacity through sparse activation and expert specialization. An implicit quantile network head predicts arbitrary quantiles, enhancing probabilistic forecasting flexibility. Experiments on GIFT‑Eval, TIME, FEV‑Bench, TFB, and DAG‑Bench show state‑of‑the‑art performance against both pretrained TSFMs and task‑specific supervised models.

By Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng, Peng Chen, Yang Shu, Jilin Hu, Bin Yang

benchmarks

arXiv AI

Jul 7

QuantFlow: A Federated Mamba-Based Post-Transformer Foundation Model for Time-Series Forecasting

arXiv:2607. 02632v1 Announce Type: cross Abstract: Time-series forecasting supports decisions in finance, en-ergy, transportation, public health, and industrial monitoring.

By Shah Nawaz Haider, Steve Austin, Arnab Barua, Sarowar Morshed Shawon, Hadaate Ullah

llms rag safety

Related stories