arXiv AI

Aligned, Conflicting, or Unspecified? A Diagnostic Benchmark for Multimodal Time Series Forecasting

The paper introduces ACU‑Bench, a real‑world diagnostic benchmark for multimodal time‑series forecasting that pairs 1,762 text windows with sensor telemetry, web‑service traffic, and market data, labeling each pair as Aligned, Conflicting, or Unspecified based on whether the text’s direction agrees with, opposes, or does not state the historical trend. The authors evaluate ten pretrained models zero‑shot, exploring how different types of text (direct forecasts vs. descriptive context) affect error under each relationship, and propose a training‑free rule that limits oversized revisions to improve accuracy. The benchmark and findings are released on GitHub for further research. whyItMatters:"The study provides a structured way to assess how textual information influences multimodal forecasting models, revealing that careful text selection and revision control can significantly reduce error and improve directional accuracy."

arXiv AI
Sep 12

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

The paper introduces a synthetic benchmark for multimodal time‑series forecasting that evaluates how well text annotations contribute to predictions. By generating controlled signals with semantically correct, incorrect, and irrelevant annotations, the authors can precisely measure the true information content. Six mutual‑information estimators (KSG, MINE, InfoNCE, CCA, PID, and V‑information) are tested, all correctly ranking useful annotations and enabling annotation auditing without model training. The benchmark also highlights each estimator’s limitations and validates findings on seven real datasets, providing practical guidelines for metric implementation.

By Emma Andrews, Gianmarco Mengaldo
arXiv Machine Learning
6d ago

StatPrompt-LLM: A Lightweight Statistical Refinement for Foundation-Style Time-Series Forecasting

StatPrompt-LLM is a lightweight statistical refinement of the Time-LLM foundation-style model for time‑series forecasting. It augments the prompting mechanism with dataset‑level statistical information and adds a zero‑initialised autoregressive bypass as a linear forecasting pathway. On electricity and meteorological benchmarks, StatPrompt-LLM achieved significant error reductions—up to 40.2% in MSE and 21.4% in MAE—under a 10% reduced‑prefix configuration, while full‑prefix results varied by dataset.

By Aniket Sinha, Veronica W. Tsou
arXiv AI
Sep 30

Alignment Forecasting: Predicting Misalignment From Training Data

The paper introduces Alignment Forecasting, a method for predicting whether fine‑tuning a language model on a given dataset will increase specific alignment failures such as deception or sycophancy. It presents ALIGNMENTFORECASTBENCH, a benchmark of over 5,000 forecasting questions across many models, datasets, and failure modes, and shows that a simple forecasting scaffold using an LLM’s assessment of dataset bias can outperform baseline forecasters. The authors demonstrate that filtering out high‑risk training examples identified by the forecaster can improve alignment in multiple‑choice evaluations, though benefits in open‑ended conversations remain uncertain.

By Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky, Tomek Korbak
arXiv AI
Oct 27

The Future Is Finer: Multivariate Time Series Super-Resolution Forecasting

The paper introduces Multivariate Time Series Super-Resolution Forecasting (MTS‑SRF), a method that predicts fine‑grained future values from coarse historical data. It proposes the TimeMagnifier framework, which includes Fourier Anchor to handle data sparsity, De‑noise Only to refine predictions along the resolution axis, and banded DTW Alignment to correct cross‑resolution misalignment. On a new benchmark of ten datasets across five domains, TimeMagnifier outperforms 15 baselines, reducing mean‑squared error by an average of 12.3% and improving existing forecasters by 5.1% to 9.0% when its components are used as plug‑ins.

By Shuhao Li, Weidong Yang, Yue Cui, Fan Zhang, Lu Qin
arXiv Computation and Language
Aug 25

Semantics or Structure? Auditing Text Sensitivity in Multimodal Time-Series Forecasting

The paper investigates whether multimodal time‑series forecasting models actually use the semantic content of accompanying text. By systematically perturbing the text—replacing it with empty, constant, shuffled, or cross‑domain sentences—the authors find that mean squared error changes by less than 0.5 % across several architectures, indicating that text does not drive performance gains. They also show that removing a co‑shipped numeric column restores the reported improvements, suggesting that the models rely on other signals rather than textual semantics.

By Karthik Sridhar, Atharva Gupta, Nishant Pradhan, Murari Mandal, Dhruv Kumar, Saurabh Deshpande
arXiv Computation and Language
6d ago

TimeOmni-o1: Bidirectionally Coupling Time Series Foundation Models and LLMs for Multimodal Reasoning and Forecasting

TimeOmni-o1 is a new model that bidirectionally couples a pretrained time‑series foundation model (TSFM) with a large language model (LLM) to simultaneously read, generate, and reason over both time‑series data and accompanying text. The TSFM encodes the series for the LLM, which performs multimodal reasoning, and the LLM’s output is fed back to the TSFM to produce forecasts conditioned on that reasoning. Trained on the newly created MMTR dataset—over 500,000 samples across 12 domains—TimeOmni-o1 surpasses existing models on both understanding and forecasting tasks, and its performance generalizes to three open benchmarks.

By Ziyang Zhang, Shenyi Li, Wen Wu, Chao Zhang