← Back to all news
arXiv AI

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

The paper introduces a synthetic benchmark for multimodal time‑series forecasting that evaluates how well text annotations contribute to predictions. By generating controlled signals with semantically correct, incorrect, and irrelevant annotations, the authors can precisely measure the true information content. Six mutual‑information estimators (KSG, MINE, InfoNCE, CCA, PID, and V‑information) are tested, all correctly ranking useful annotations and enabling annotation auditing without model training. The benchmark also highlights each estimator’s limitations and validates findings on seven real datasets, providing practical guidelines for metric implementation.

Related stories

arXiv Computation and Language
Aug 25

Semantics or Structure? Auditing Text Sensitivity in Multimodal Time-Series Forecasting

The paper investigates whether multimodal time‑series forecasting models actually use the semantic content of accompanying text. By systematically perturbing the text—replacing it with empty, constant, shuffled, or cross‑domain sentences—the authors find that mean squared error changes by less than 0.5 % across several architectures, indicating that text does not drive performance gains. They also show that removing a co‑shipped numeric column restores the reported improvements, suggesting that the models rely on other signals rather than textual semantics.

By Karthik Sridhar, Atharva Gupta, Nishant Pradhan, Murari Mandal, Dhruv Kumar, Saurabh Deshpande
multimodalbenchmarks
More like this →
arXiv Machine Learning
Aug 19

TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts

The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.

By Jiafeng Lin, Yuxuan Wang, Huakun Luo, Jianmin Wang, Zhongyi Pei
llmsmultimodalbenchmarkssafety
More like this →
arXiv AI
Aug 25

NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting

NeST is a framework that adapts large language models (LLMs) for continuous time‑series forecasting by creating neighborhood‑aware text prototypes and aligning them with temporal representations through a nearest‑neighbor contrastive objective. It retrieves the most relevant prototypes and uses them to conditionally modulate time‑series features, enabling more effective integration of textual and temporal information. Experiments show that NeST outperforms state‑of‑the‑art methods on eight benchmarks, reduces MSE by 1.2% for long‑term forecasting, improves zero‑shot forecasting by 4.9%, and boosts R² by 3.3% on a real‑world photovoltaic power forecasting task.

By Jayanie Bogahawatte, Sachith Seneviratne, Maneesha Perera, Saman Halgamuge
llmsragfine-tuningbenchmarkssafety
More like this →
arXiv AI
Aug 25

LLM-based Agents for Forecasting and Prediction: Methods, Training, Evaluation, and Applications

arXiv:2608.23058v1 Announce Type: new Abstract: Large language models (LLMs) now support forecasting systems that combine language-based reasoning with temporal data, evidence retrieval, external too...

By Xiaogang Xu, Jiaqi Tang, Jianmin Chen, Yingying Yan, Zhenchao Tang, Xiangxin Zhou, Xiaobin Hu, Wei Wei, Jinfeng Wu, Qifeng Chen, Lu Zhou, Jiafei Wu, Zhe Liu, Jianwei Yin, Weimin Zheng
llmsragagentsbenchmarks
More like this →