The paper investigates whether multimodal time‑series forecasting models actually use the semantic content of accompanying text. By systematically perturbing the text—replacing it with empty, constant, shuffled, or cross‑domain sentences—the authors find that mean squared error changes by less than 0.5 % across several architectures, indicating that text does not drive performance gains. They also show that removing a co‑shipped numeric column restores the reported improvements, suggesting that the models rely on other signals rather than textual semantics.
arXiv:2609.24156v1 Announce Type: cross Abstract: Most existing time series forecasting methods rely solely on numerical observations, overlooking rich contextual information from auxiliary texts. Re...
The paper introduces a synthetic benchmark for multimodal time‑series forecasting that evaluates how well text annotations contribute to predictions. By generating controlled signals with semantically correct, incorrect, and irrelevant annotations, the authors can precisely measure the true information content. Six mutual‑information estimators (KSG, MINE, InfoNCE, CCA, PID, and V‑information) are tested, all correctly ranking useful annotations and enabling annotation auditing without model training. The benchmark also highlights each estimator’s limitations and validates findings on seven real datasets, providing practical guidelines for metric implementation.
arXiv:2606. 19413v1 Announce Type: new Abstract: Multimodal time series forecasting, which pairs numerical sequences with domain-relevant textual reports, promises to inject world knowledge into forecasting pipelines.
arXiv:2609.15087v1 Announce Type: cross Abstract: Most time series forecasting benchmarks remain numerical-centric and provide limited support for evaluating contextual information that shapes real-w...
arXiv:2610.17856v2 Announce Type: cross Abstract: Time-series analysis is increasingly studied in multimodal settings, yet most benchmarks pair temporal signals with text or with visualizations deriv...
DocXEval is the first comprehensive framework that adapts existing explainable AI methods and evaluation metrics to multimodal document understanding, covering text, layout, and image modalities. The authors systematically evaluate 13 XAI methods across 3 tasks, 8 datasets, 4 models, and 17 metrics, finding that Occlusion consistently performs best in faithfulness and interpretability. The study also provides meta‑evaluation insights, diagnostic analyses of modality contributions, spatial attribution patterns, and potential dataset biases, offering a benchmark and practical guide for future research.
arXiv:2607. 06973v1 Announce Type: new Abstract: We introduce a new context-enriched, multimodal time series forecasting benchmark, TimesX.
The paper introduces Context as Covariates (CoCo), a forecasting framework that combines Large Language Models (LLMs) and Time Series Foundation Models (TSFMs). It generates realistic multimodal datasets by annotating real-world time series with LLMs and aligns the two models through staged fine‑tuning: first using Group Relative Policy Optimization to improve the LLM’s covariate generation, then fine‑tuning the TSFM on those covariates. Experiments on synthetic and real benchmarks across finance, economics, weather, traffic, and security show that CoCo can outperform state‑of‑the‑art TSFMs and LLM forecasters by up to 12% MAE.
ChronoSteer is a decoupled agentic framework that bridges large language models and time series foundation models by learning cross‑modal alignment from synthetic paired supervision. It converts textual events into revision instructions that steer a frozen time‑series model, discretizes these instructions into a compact codebook to reduce semantic divergence, and then refines the predictions with a two‑stage training strategy. The authors also release a leakage‑controlled multimodal benchmark and report a 25.8% improvement in zero‑shot prediction accuracy over the unimodal backbone.
The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.
arXiv:2603. 22372v2 Announce Type: replace-cross Abstract: Recent advances in multimodal learning have motivated the integration of auxiliary modalities such as text or vision into time series (TS) forecasting.
arXiv:2509.24789v5 Announce Type: replace Abstract: The evaluation of time series forecasting models is hindered by a lack of high-quality benchmarks, leading to overestimated assessments of progress...