arXiv AI By Xintong Wang, Haihan Nan, Ruidong Li

Aligned, Conflicting, or Unspecified? A Diagnostic Benchmark for Multimodal Time Series Forecasting

Read the original on arXiv AI →

The paper introduces ACU‑Bench, a real‑world diagnostic benchmark for multimodal time‑series forecasting that pairs 1,762 text windows with sensor telemetry, web‑service traffic, and market data, labeling each pair as Aligned, Conflicting, or Unspecified based on whether the text’s direction agrees with, opposes, or does not state the historical trend. The authors evaluate ten pretrained models zero‑shot, exploring how different types of text (direct forecasts vs. descriptive context) affect error under each relationship, and propose a training‑free rule that limits oversized revisions to improve accuracy. The benchmark and findings are released on GitHub for further research. whyItMatters:"The study provides a structured way to assess how textual information influences multimodal forecasting models, revealing that careful text selection and revision control can significantly reduce error and improve directional accuracy."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI
Sep 12

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

The paper introduces a synthetic benchmark for multimodal time‑series forecasting that evaluates how well text annotations contribute to predictions. By generating controlled signals with semantically correct, incorrect, and irrelevant annotations, the authors can precisely measure the true information content. Six mutual‑information estimators (KSG, MINE, InfoNCE, CCA, PID, and V‑information) are tested, all correctly ranking useful annotations and enabling annotation auditing without model training. The benchmark also highlights each estimator’s limitations and validates findings on seven real datasets, providing practical guidelines for metric implementation.

By Emma Andrews, Gianmarco Mengaldo
arXiv Machine Learning
6d ago

StatPrompt-LLM: A Lightweight Statistical Refinement for Foundation-Style Time-Series Forecasting

StatPrompt-LLM is a lightweight statistical refinement of the Time-LLM foundation-style model for time‑series forecasting. It augments the prompting mechanism with dataset‑level statistical information and adds a zero‑initialised autoregressive bypass as a linear forecasting pathway. On electricity and meteorological benchmarks, StatPrompt-LLM achieved significant error reductions—up to 40.2% in MSE and 21.4% in MAE—under a 10% reduced‑prefix configuration, while full‑prefix results varied by dataset.

By Aniket Sinha, Veronica W. Tsou
arXiv AI
Sep 30

Alignment Forecasting: Predicting Misalignment From Training Data

The paper introduces Alignment Forecasting, a method for predicting whether fine‑tuning a language model on a given dataset will increase specific alignment failures such as deception or sycophancy. It presents ALIGNMENTFORECASTBENCH, a benchmark of over 5,000 forecasting questions across many models, datasets, and failure modes, and shows that a simple forecasting scaffold using an LLM’s assessment of dataset bias can outperform baseline forecasters. The authors demonstrate that filtering out high‑risk training examples identified by the forecaster can improve alignment in multiple‑choice evaluations, though benefits in open‑ended conversations remain uncertain.

By Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky, Tomek Korbak