arXiv Machine Learning

A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks

The paper introduces iAmTime, a time‑series foundation model that uses instruction‑conditioned in‑context learning to adapt to tasks at inference time. iAmTime represents each episode as a structured prompt with semantic tokens that focus on specific time‑series regions, enabling the model to infer task structure from input‑output demonstrations. Trained on large real and synthetic corpora across forecasting, imputation, reconstruction, classification, anomaly detection, and source de‑mixing, iAmTime outperforms strong baselines on zero‑shot probabilistic and point forecasting while matching or exceeding performance on several non‑forecasting tasks.

arXiv AI
Sep 25

TimeBraid: Unifying Time Series and Language for Understanding and Forecasting

TimeBraid is a family of unified models that combine pretrained language models with pretrained time‑series foundation models using interleaved global residual attention layers. The models inherit instruction following, reasoning, and continuous‑signal perception, fusing both modalities into a shared representation space for understanding and generation. The design focuses on aligning representation spaces, grounding language in temporal structure, balancing understanding with generation, and maintaining stable joint optimization, supported by 2.2 M curated series‑text pairs and 4.9 M instruction‑tuning samples. Across diverse benchmarks, TimeBraid competes with larger general‑purpose and task‑specific models.

By Xinyue Wang, Jiacheng Pang, Kun Zhou, Kexin Zhang, Defu Cao, Fan Feng, Faisal, Songyao Jin, Yan Liu, Biwei Huang
arXiv Machine Learning
Oct 19

Out of Support, Not Out of Reach: In-Context Learning with Unsupported Instructions and Composed Tasks

The paper investigates when in‑context learning (ICL) can succeed even if the instruction or target task is not covered by a model’s pretraining data. It introduces a theoretical framework that distinguishes two types of distribution shift—unsupported instructions and unsupported target tasks—and shows how demonstrations can mitigate these shifts. The authors further extend the analysis to long‑horizon tasks that can be decomposed into pretraining‑supported subtasks, providing error bounds that explain how ICL can handle tasks beyond the model’s original support.

By Xuhan Tong, Yuchen Zeng, Jiawei Zhang
arXiv Machine Learning
Sep 29

Aurora-X: Built for Extreme Time Series Forecasting

Aurora‑X is a billion‑parameter time‑series foundation model designed for extreme forecasting tasks. It employs a progressive curriculum that starts with channel‑independent pretraining, then adds cross‑variable dependencies, variable context and horizon lengths, and optional future covariates during mid‑training. A variable‑resolution post‑training stage allows adjustable temporal spans per token at inference, while a pattern‑guided mixture‑of‑experts expands capacity through sparse activation and expert specialization. An implicit quantile network head predicts arbitrary quantiles, enhancing probabilistic forecasting flexibility. Experiments on GIFT‑Eval, TIME, FEV‑Bench, TFB, and DAG‑Bench show state‑of‑the‑art performance against both pretrained TSFMs and task‑specific supervised models.

By Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng, Peng Chen, Yang Shu, Jilin Hu, Bin Yang
arXiv AI
Oct 9

Timer-M1: A Multivariate Time Series Foundation Model via Learning Primitives

Timer‑M1 is a pretrained multivariate time‑series foundation model that learns with primitives for zero‑shot forecasting. It uses a primitive‑based data synthesis and pretraining pipeline to generate series with shared temporal and relational patterns, assembling them into multivariate samples with distinct channel roles. The model employs gated two‑dimensional Transformer blocks that dynamically allocate cross‑variate attention, achieving top performance on several large‑scale forecasting benchmarks.

By Haoran Zhang, Haixuan Liu, Xingjian Su, Yong Liu, Zhi Chen, Yuxuan Wang, Jianmin Wang, Mingsheng Long
arXiv Machine Learning
Sep 24

ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset

ChronoSteer is a decoupled agentic framework that bridges large language models and time series foundation models by learning cross‑modal alignment from synthetic paired supervision. It converts textual events into revision instructions that steer a frozen time‑series model, discretizes these instructions into a compact codebook to reduce semantic divergence, and then refines the predictions with a two‑stage training strategy. The authors also release a leakage‑controlled multimodal benchmark and report a 25.8% improvement in zero‑shot prediction accuracy over the unimodal backbone.

By Chengsen Wang, Qi Qi, Zhongwen Rao, Lujia Pan, Jingyu Wang
arXiv AI
1d ago

FlexCast: Flexible Context Learning for Time Series Foundation Models

FlexCast introduces a flexible context-target learning framework for time series foundation models, reformulating forecasting as variable-length masked prediction over time series patches. By dynamically sampling visible context and masked target patches during pretraining, it exposes the model to diverse context-target configurations without sequential decoding. The same pretrained model can perform one-step forecasting, confidence-based iterative refinement, and next-token-style generation, achieving competitive or superior accuracy to strong TSFM baselines while reducing inference latency.

By Zuyang Shen, Luqi Gong, Fanda Fan, Bingsong Wang, Xiang Ao, Yahui Wei, Minghao Liu, Chunjie Luo, Jianfeng Zhan
Hugging Face Trending Papers
Sep 28

Instance-Adaptive Prompts as Context for Time-Series Foundation Models

The paper introduces PaCTS, a method that generates instance‑adaptive latent prompts—continuous embedding tokens—to provide compact contextual information for frozen time‑series foundation models (TSFMs). These prompts are constructed from instance‑specific global statistics and refined with segment‑level temporal data, enabling the model to capture both global characteristics and local temporal variations. Experiments show that PaCTS improves forecasting performance across various context lengths and model architectures, often outperforming the same backbone with double the context while reducing inference computation, and it also offers stronger improvements and better out‑of‑distribution generalization compared to weight‑space adaptation methods.

arXiv AI
Sep 3

AGI Maze Prediction Datasets: A Compact Benchmark for Learning World Dynamics with Transformers

The paper introduces the AGI Maze Prediction Datasets and Benchmark, a lightweight, procedurally generated grid‑world testbed for evaluating predictive models, particularly Transformers, on tasks such as per‑step transition prediction, fixed‑horizon state prediction, and sequential textual‑observation prediction. It compares byte‑level Transformer baselines with two memory‑augmented architectures, showing that a pseudo‑video spatial‑memory Transformer achieves perfect validation accuracy on selected tasks and improves sequential text‑trace prediction, while a generic auxiliary latent‑memory Transformer does not consistently help. The study highlights that structured, task‑aligned working memory can be more effective than merely increasing latent capacity, and positions the benchmark as a compact setting for testing architectures that couple textual interfaces to learned structured state.

By Alexey Potapov