arXiv Machine Learning

TimeWorld: Towards Action-Conditioned Multimodal World Models for Time Series Forecasting

TimeWorld introduces a plug‑in framework for action‑conditioned multimodal world models in time‑series forecasting. It builds on pretrained time‑series foundation models (TSFMs) and adds a Prototype‑Guided Latent Modulation module to embed textual actions into temporal dynamics, along with an Action Contrastive Learning component to sharpen action‑specific response modeling. The authors also present TSWM‑Bench, a cross‑domain benchmark of seven real‑world datasets with textual interventions, showing that simple text‑prefix adaptations outperform conventional multimodal forecasters and that pretrained TSFMs already encode useful world knowledge for action‑conditioned predictions.

arXiv Machine Learning
Sep 29

WorldTS: World Modeling for Multimodal Covariate-aware Time Series Forecasting

WorldTS is a new forecasting framework that models latent dynamics conditioned on multimodal covariates to improve time‑series prediction. It uses a two‑stage training process: first learning latent state dynamics from historical data and covariates, then training a decoder to map predicted latent states back to future observations. Experiments on 21 real‑world datasets demonstrate the effectiveness of this approach.

By Yuhan Zhu, Xiangfei Qiu, Hanyin Cheng, Wangmeng Shen, Chenjuan Guo, Bin Yang, Jilin Hu, Christian S. Jensen
arXiv Machine Learning
Aug 19

TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts

The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.

By Jiafeng Lin, Yuxuan Wang, Huakun Luo, Jianmin Wang, Zhongyi Pei
arXiv AI
1d ago

Self-Supervised Latent Action-Conditioned World Modeling for Time Series

The paper introduces ACTS-WM, a self‑supervised framework that models latent actions underlying time‑series transitions. It combines a Joint‑Embedding Predictive Architecture encoder with an action‑conditioned world model to predict future representations conditioned on inferred latent actions and task information. Experiments on 22 datasets across four tasks show that ACTS‑WM improves downstream performance compared to pretrained JEPA‑based encoders.

By Chia-Han Kuo, Chengshuai Zhao, Huan Liu, Lo Pang-Yun Ting
arXiv AI
1d ago

LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting

The paper introduces LLM-Integrated Bayesian State Space Models (LBS) for multimodal time-series forecasting, combining a state space model backbone with a pretrained large language model to handle both numerical and textual data. LBS allows flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization. Experiments on the TextTimeCorpus benchmark show a 13.20% performance gain over the previous state‑of‑the‑art while providing human‑readable forecast summaries.

By Sungjun Cho, Changho Shin, Suenggwan Jo, Xinya Yan, Shourjo Aditya Chaudhuri, Frederic Sala
arXiv Machine Learning
Sep 24

ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset

ChronoSteer is a decoupled agentic framework that bridges large language models and time series foundation models by learning cross‑modal alignment from synthetic paired supervision. It converts textual events into revision instructions that steer a frozen time‑series model, discretizes these instructions into a compact codebook to reduce semantic divergence, and then refines the predictions with a two‑stage training strategy. The authors also release a leakage‑controlled multimodal benchmark and report a 25.8% improvement in zero‑shot prediction accuracy over the unimodal backbone.

By Chengsen Wang, Qi Qi, Zhongwen Rao, Lujia Pan, Jingyu Wang
arXiv Machine Learning
Oct 9

Revisiting Identity and Spectra Dispersion in Media-Bridged Time Series Forecasting: Linking Multivariate Signals and Narrative Flows

The paper introduces MIDAPN, a unified spatiotemporal forecasting backbone that integrates media-general graph adaptation and automatic temporal learning for both multivariate and multimodal time series. It employs a Multimedia Identity-Aware Graph to capture static, dynamic, and latent relationships across media, and a Spectral Prism Convolution for hierarchical temporal analysis. Extensive experiments on 13 multivariate and 12 multimodal datasets show MIDAPN consistently outperforms 16 state‑of‑the‑art TSF models and is compatible with various foundation models.

By Jierui Lei, Wenjian Zhang, Qingyi Yang, Yuyang Hong, Fangzheng Chen, Zhengbo Zhang, Haina Tang, Shiming Xiang
arXiv Machine Learning
Sep 22

Generalized Multimodal Foundation Model

The paper introduces a generalized multimodal foundation model that can handle arbitrary combinations of modalities and prediction tasks. It trains on large-scale synthetic multimodal datasets with diverse causal structures to learn transferable multimodal correlations. Experiments on 18 real-world datasets across 12 modalities and 11 tasks show competitive performance compared to specialized models without task-specific adaptation.

By Huizi Cui, Zongbo Han, Chenggong Ding, Naichuan Xiao, Jialong Yang, Jingdong Chen, Guangyu Wang, Qinghua Hu, Changqing Zhang