arXiv Machine Learning
Sep 18
When Does Retrieval Help Time-Series Forecasting?
arXiv:2609. 20193v1 Announce Type: new Abstract: Retrieval plug-ins supply a deep forecaster with information its lookback window cannot carry.
Evaluation Choices Decide the Forecasting Leaderboard: Evidence from a Production Marketplace Panel
The paper demonstrates that the outcome of a forecasting leaderboard is largely determined by the evaluator’s design choices rather than the models themselves. By fixing the data, horizon, and period, the authors varied three key evaluation decisions—unit of analysis, error pooling, and scoring metric—and showed that each can reverse or eliminate the apparent superiority of any forecasting method. The study also evaluates the practical impact of these choices on a deployed system, revealing that the selection rule captures a significant portion of the potential performance gain, and confirms the findings on an external public dataset.
Related stories
arXiv AI
Jul 21
FIFA World Cup 2026 as a Contamination-Free Benchmark for LLM Forecasting Agents: Four Models, a Bookmaker, and 104 Matches
arXiv:2607. 17765v1 Announce Type: cross Abstract: We introduce WC2026-Agents, a benchmark and dataset for evaluating large language models (LLMs) as autonomous forecasting agents on real, future events.
arXiv AI
Aug 18
Frontier AI Forecasting Has a Measurement Problem: An Audit of Progress Evidence
arXiv:2608. 14903v1 Announce Type: new Abstract: Quantitative forecasts of frontier artificial intelligence often connect dated targets to trends in benchmark scores, training compute, release time, or expert belief.
arXiv Machine Learning
Jul 15
When Directional Accuracy Lies: A Base-Rate-Honest Benchmark for LoRA-Adapted TimesFM on Equity Forecasting
arXiv:2607. 12248v1 Announce Type: cross Abstract: Large pretrained time-series models such as TimesFM are attractive for financial forecasting, but raw directional accuracy is a misleading scoreboard in equity markets.
arXiv AI
Aug 17
Forecast Collapse in Time-Series Foundation Models
arXiv:2608. 14106v1 Announce Type: cross Abstract: When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation.
arXiv AI
Jul 29
GAUGE: Grading Agent-Built Financial Models Without a Golden Answer
arXiv:2607. 24889v1 Announce Type: cross Abstract: Financial models combine public disclosures with analyst assumptions to produce forecasts and valuations.
arXiv Machine Learning
4d ago
Global tree forecasters collapse at the hierarchical aggregate: a five-panel failure characterization
The paper reports a previously undocumented failure of global gradient‑boosted tree forecasters when applied to hierarchical aggregates. Training a single tree on individual series causes the model to predict a constant outside its training range, leading to severe under‑prediction of the total (30–50× in production and up to 496× in a public M5 reconstruction). The authors characterize this collapse across five datasets, three tree libraries, and multiple training seeds, and demonstrate that simple preprocessing steps—per‑series scaling, weighted aggregate‑level training, or seasonal differencing—can prevent it.
arXiv Machine Learning
3d ago
Downside-Controlled Online Forecast Combination under Delayed and Revised Outcomes
The paper proposes a method for controlling downside risk when adjusting forecasts from frozen models, such as foundation models, by combining a static corrector and an online corrector on the simplex. Using only post‑horizon losses, the approach achieves minimal deterioration (0.15%) and up to 11.5% gains across 28 forecast pairs, and consistently reduces mean MSE in day‑ahead load forecasts for seven European bidding zones. The method’s applicability is bounded by three empirical conditions related to expert speed, stream length, and outcome alignment.
arXiv Machine Learning
Jun 25
Model selection with proper scoring rules on data sets of time series: prefer the mean scaled score
arXiv:2606. 24715v2 Announce Type: replace-cross Abstract: We study the problem of model selection among probabilistic forecasting models evaluated on datasets of multiple time series.
arXiv Machine Learning
Sep 15
Accuracy Is Not Service: A Decision-Aware Benchmark for Intermittent-Demand Forecasting
arXiv:2609.13840v1 Announce Type: new Abstract: A contract-logistics spare-parts operator is paid on order-level service: an order counts only if every requested line is fulfilled, yet forecasters ar...
arXiv Machine Learning
Aug 5
Forecasting Revenue with its Customer-Base Drivers: When and Why Coordination Helps
arXiv:2608. 02911v1 Announce Type: new Abstract: Revenue forecasts guide acquisition budgets, demand planning, and customer-based valuations, yet an aggregate forecast does not show whether change reflects acquisition, repeat purchasing, spending per order, or offsetting movements.
arXiv AI
3d ago
TW3Cast: A Frozen Router of Lightly Fine-Tuned Foundation Models for Time-Series Forecasting on GIFT-Eval, Selected Entirely on the Training Split
arXiv:2609. 28506v1 Announce Type: new Abstract: TW3Cast is a time-series forecasting system that reaches position 3 of 130 entries on the GIFT-Eval benchmark by mean MASE rank, as of 2026-09-14.