arXiv AI

Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context

arXiv Machine Learning
Aug 19

TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts

The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.

By Jiafeng Lin, Yuxuan Wang, Huakun Luo, Jianmin Wang, Zhongyi Pei
arXiv Machine Learning
4d ago

TimeWorld: Towards Action-Conditioned Multimodal World Models for Time Series Forecasting

TimeWorld introduces a plug‑in framework for action‑conditioned multimodal world models in time‑series forecasting. It builds on pretrained time‑series foundation models (TSFMs) and adds a Prototype‑Guided Latent Modulation module to embed textual actions into temporal dynamics, along with an Action Contrastive Learning component to sharpen action‑specific response modeling. The authors also present TSWM‑Bench, a cross‑domain benchmark of seven real‑world datasets with textual interventions, showing that simple text‑prefix adaptations outperform conventional multimodal forecasters and that pretrained TSFMs already encode useful world knowledge for action‑conditioned predictions.

By Hang Ni, Weijia Zhang, Hao Liu
arXiv Machine Learning
Aug 19

SCENARIODIFF: A Scenario-level Guidance Framework for Multimodal Time Series Forecasting--Extended Version

SCENARIODIFF is a hierarchical contextual reasoning framework designed for multimodal time series forecasting, especially in event-driven domains. It processes textual context through three agents—Historical Context, Scenario, and Anchor Guidance—to generate structured signals that condition a Multimodal Diffusion Transformer. The framework also employs Anchor Blended Sampling to locally refine forecast trajectories without retraining, and demonstrates superior performance on the Time‑MMD benchmark.

By Tuan-Binh Tran, Dat Nguyen Cong, Duc-Trong Le, Thanh Trung Huynh, Tung Kieu
arXiv Machine Learning
Sep 29

WorldTS: World Modeling for Multimodal Covariate-aware Time Series Forecasting

WorldTS is a new forecasting framework that models latent dynamics conditioned on multimodal covariates to improve time‑series prediction. It uses a two‑stage training process: first learning latent state dynamics from historical data and covariates, then training a decoder to map predicted latent states back to future observations. Experiments on 21 real‑world datasets demonstrate the effectiveness of this approach.

By Yuhan Zhu, Xiangfei Qiu, Hanyin Cheng, Wangmeng Shen, Chenjuan Guo, Bin Yang, Jilin Hu, Christian S. Jensen
arXiv Machine Learning
Jul 16

Overcoming the Modality Gap in Context-Aided Forecasting

arXiv:2603. 12451v4 Announce Type: replace Abstract: Context-aided forecasting (CAF) holds promise for integrating domain knowledge and forward-looking information, enabling AI systems to surpass traditional statistical methods.

By Vincent Zhihao Zheng, \'Etienne Marcotte, Arjun Ashok, Andrew Robert Williams, Lijun Sun, Alexandre Drouin, Valentina Zantedeschi