arXiv Machine Learning

WorldTS: World Modeling for Multimodal Covariate-aware Time Series Forecasting

WorldTS is a new forecasting framework that models latent dynamics conditioned on multimodal covariates to improve time‑series prediction. It uses a two‑stage training process: first learning latent state dynamics from historical data and covariates, then training a decoder to map predicted latent states back to future observations. Experiments on 21 real‑world datasets demonstrate the effectiveness of this approach.

arXiv Machine Learning
Oct 28

TimeWorld: Towards Action-Conditioned Multimodal World Models for Time Series Forecasting

TimeWorld introduces a plug‑in framework for action‑conditioned multimodal world models in time‑series forecasting. It builds on pretrained time‑series foundation models (TSFMs) and adds a Prototype‑Guided Latent Modulation module to embed textual actions into temporal dynamics, along with an Action Contrastive Learning component to sharpen action‑specific response modeling. The authors also present TSWM‑Bench, a cross‑domain benchmark of seven real‑world datasets with textual interventions, showing that simple text‑prefix adaptations outperform conventional multimodal forecasters and that pretrained TSFMs already encode useful world knowledge for action‑conditioned predictions.

By Hang Ni, Weijia Zhang, Hao Liu
arXiv AI
Oct 31

LLM-Integrated Bayesian State Space Models for Multimodal Time-Series Forecasting

The paper introduces LLM-Integrated Bayesian State Space Models (LBS) for multimodal time-series forecasting, combining a state space model backbone with a pretrained large language model to handle both numerical and textual data. LBS allows flexible lookback and forecast windows, principled uncertainty quantification, and improved temporal generalization. Experiments on the TextTimeCorpus benchmark show a 13.20% performance gain over the previous state‑of‑the‑art while providing human‑readable forecast summaries.

By Sungjun Cho, Changho Shin, Suenggwan Jo, Xinya Yan, Shourjo Aditya Chaudhuri, Frederic Sala
arXiv AI
Oct 31

Self-Supervised Latent Action-Conditioned World Modeling for Time Series

The paper introduces ACTS-WM, a self‑supervised framework that models latent actions underlying time‑series transitions. It combines a Joint‑Embedding Predictive Architecture encoder with an action‑conditioned world model to predict future representations conditioned on inferred latent actions and task information. Experiments on 22 datasets across four tasks show that ACTS‑WM improves downstream performance compared to pretrained JEPA‑based encoders.

By Chia-Han Kuo, Chengshuai Zhao, Huan Liu, Lo Pang-Yun Ting
arXiv Machine Learning
Aug 19

TiMi: Empower Time Series Transformers with Multimodal Mixture of Experts

The paper introduces TiMi, a framework that enhances time series transformers with a Multimodal Mixture-of-Experts (MMoE) module to incorporate multimodal data, especially textual information, into forecasting. TiMi leverages large language models to generate future inferences that guide predictions, eliminating the need for explicit representation alignment. Experiments show TiMi achieves state‑of‑the‑art performance on sixteen real‑world multimodal forecasting benchmarks, outperforming advanced baselines while maintaining adaptability and interpretability.

By Jiafeng Lin, Yuxuan Wang, Huakun Luo, Jianmin Wang, Zhongyi Pei
arXiv AI
Oct 16

WorldCast: Exploring Intelligent Visual Generation in World State Prediction

WorldCast is a benchmark that tests multimodal generative models on multi-step world state prediction, requiring them to forecast successive future states from multimodal observations and diverse references. The study evaluates 12 models across 320 manually designed cases in eight categories, revealing that advanced models like GPT-Image-2.5 achieve high scores while open-source models lag behind, especially on complex context and reasoning tasks. The results underscore the importance of integrating dynamic planning, reasoning, and unified multimodal capabilities for realistic world simulation.

By Wenhao Yan, Haodong Li, Xiaotong Li, Yaxin Luo, Zhengrong Yue, Jialv Zou, Xu Huang, Haojin Yang, Nan Jia, Yan Feng, Peng Pei, Xunliang Cai
arXiv AI
Nov 5

TSFM Meets LLM: Context as Covariates

The paper introduces Context as Covariates (CoCo), a forecasting framework that combines Large Language Models (LLMs) and Time Series Foundation Models (TSFMs). It generates realistic multimodal datasets by annotating real-world time series with LLMs and aligns the two models through staged fine‑tuning: first using Group Relative Policy Optimization to improve the LLM’s covariate generation, then fine‑tuning the TSFM on those covariates. Experiments on synthetic and real benchmarks across finance, economics, weather, traffic, and security show that CoCo can outperform state‑of‑the‑art TSFMs and LLM forecasters by up to 12% MAE.

By Hemanth Ram, Sai Shankar Narasimhan, Shubhankar Agarwal, Sandeep Chinchali
arXiv Machine Learning
Oct 5

MACTS-EM: Multi-Agent Collaborative Time Series Forecasting with Emergent Memory

MACTS-EM is a new multi‑agent framework for time series forecasting that combines domain‑specialised agents, a meta‑cognitive allocation layer, emergent memory for cross‑domain transfer, multimodal context integration, and adversarial robustness. The authors evaluate the system on financial, climate, energy, and pandemic data, reporting 8‑12% higher accuracy, 22‑27% better zero‑shot transfer, 16‑21% greater resilience to regime shifts, and 15‑18% faster recovery from distribution changes compared to existing methods. These results suggest that collaborative, agent‑based approaches can outperform traditional architectures in complex, real‑world forecasting tasks.

By Ahmad Shahi, Mamehgol Yousefi
arXiv Machine Learning
Nov 6

Diffusion-Driven State Space Models

The paper introduces the Diffusion-Driven State Space Model (DDSSM), which replaces the Gaussian transition distribution in deep state space models with a diffusion model. This approach addresses the trade‑off between accurate forecasting and faithful capture of latent dynamics by enabling joint training of an autoencoder and a diffusion model on sequential data. Experiments show that DDSSM outperforms a state‑of‑the‑art deep SSM in fitting and forecasting a simulated time series with multimodal transitions.

By Jack Ruder, Michael Wojnowicz