← Back to all news
arXiv Machine Learning By Muhammad Sabbir Alam, Harindra Sandun Mavikumbure, Victor Cobilean, Milos Manic, Jayasimha Atulasimha

Extreme Low-State Quantized Transformer for Electricity-Load Forecasting

Read the original on arXiv Machine Learning →

The paper explores aggressive low‑state weight quantization for Transformer models used in electricity‑load forecasting. Experiments with 3, 5, 7, and 16 discrete weight states on two datasets show that post‑training quantization degrades performance, but quantization‑aware fine‑tuning recovers most of the loss. A novel accumulated‑update thresholding technique further reduces weight updates and state transitions with minimal impact on error, demonstrating that compact, low‑activity models can still deliver accurate forecasts.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 28

GateTS: Versatile and Efficient Forecasting via Attention-Inspired routed Mixture-of-Experts

GateTS introduces a streamlined univariate forecasting model that merges sparse Mixture-of-Experts computation with an attention-inspired gating mechanism, eliminating the need for auxiliary load‑balancing losses. The architecture achieves superior predictive accuracy across both short‑ and long‑term horizons, including intermittent signals, while using far fewer parameters than leading transformer models such as PatchTST. Empirical tests on diverse datasets show GateTS is computationally more efficient than LSTM, enabling cost‑effective inference for real‑world time‑series applications.

By Kyrylo Yemets, Mykola Lukashchuk, Ivan Izonin
llmsbenchmarks
More like this →
arXiv Machine Learning
Jun 15

Trend-Aware Multi-Task Learning for Short-Term Energy Forecasting

arXiv:2511. 09789v3 Announce Type: replace Abstract: Short-term energy forecasting plays an important role in real-time operational decision-making, such as electricity market bidding and power system dispatch, where both numerical accuracy and correct directional signals are essential.

By Fulong Yao, Wanqing Zhao, Chao Zheng, Xiaofei Han
benchmarks
More like this →
arXiv Machine Learning
Jul 20

A Benchmark for Electrical Load Forecasting Across Grid Levels: Time-Series Transformers Outperform Established Methods

arXiv:2607. 15705v1 Announce Type: new Abstract: Accurate load forecasting at multiple grid levels is essential for future smart grids, ranging from aggregated control area forecasts for balancing supply and demand to forecasts of individual end-consumer loads for demand-side management and energy management systems.

By Matthias Hertel, Sebastian P\"utz, Jonathan Kolar, Benjamin Sch\"afer, Ralf Mikut, Veit Hagenmeyer
llmsbenchmarks
More like this →