arXiv AI

Loss-Guided Pretraining Data Selection for Time-Series Foundation Models

arXiv Machine Learning
2d ago

Aurora-X: Built for Extreme Time Series Forecasting

Aurora‑X is a billion‑parameter time‑series foundation model designed for extreme forecasting tasks. It employs a progressive curriculum that starts with channel‑independent pretraining, then adds cross‑variable dependencies, variable context and horizon lengths, and optional future covariates during mid‑training. A variable‑resolution post‑training stage allows adjustable temporal spans per token at inference, while a pattern‑guided mixture‑of‑experts expands capacity through sparse activation and expert specialization. An implicit quantile network head predicts arbitrary quantiles, enhancing probabilistic forecasting flexibility. Experiments on GIFT‑Eval, TIME, FEV‑Bench, TFB, and DAG‑Bench show state‑of‑the‑art performance against both pretrained TSFMs and task‑specific supervised models.

By Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng, Peng Chen, Yang Shu, Jilin Hu, Bin Yang
arXiv Machine Learning
Aug 19

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu
arXiv AI
Jul 23

Post-Training in Time Series Foundation Models: A Unifying Framework

arXiv:2607. 20002v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for reliable downstream deployment.

By Shifeng Xie, Ambroise Odonnat, Zehao Xiao, Lei Zan, Malik Tiomoko, Lujia Pan, Themis Palpanas, Boris N. Oreshkin, Chenghao Liu, Keli Zhang
Hugging Face Trending Papers
Jun 22

Selective Time Series Forecasting via Metalearning

Deep learning methods have achieved state-of-the-art in time series forecasting, yet their accuracy varies considerably across samples, as some instances remain inherently difficult to predict. Reject option mechanisms, which allow models to abstain from high-risk predictions, are well established in classification and regression but underexplored in forecasting.

arXiv Machine Learning
Sep 22

Beyond Similarity: Coverage-Aware Prompt Selection for Time Series Forecasting with LLMs

The paper introduces CASP-LLM, a coverage‑aware semantic prompting framework that mitigates bias toward dominant temporal patterns in prompt‑based time series forecasting. Unlike traditional similarity‑based retrieval that selects top‑K candidates by cosine similarity, CASP‑LLM uses usage‑tracking and a saturating‑gate regularizer to diversify prompt selection without adding learnable parameters. Experiments on six long‑term and the M4 short‑term benchmarks show that CASP‑LLM matches or outperforms similarity‑based LLM forecasters in most settings, with failures traced to cross‑batch usage rather than within‑retrieval redundancy.

By Daeun Ji, Minkyoung Kim, Dongkuk Kim, Yohan Lee, Beomsoo Kim, Beakcheol Jang
Hugging Face Trending Papers
Jul 22

Post-Training in Time Series Foundation Models: A Unifying Framework

Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for reliable downstream deployment. Bridging this gap requires further intervention to handle domain shift, task heterogeneity, limited supervision, and computational constraints, which motivates post-training as a broad class of methods to adapt, augment, compose, calibrate, or specialize pretrained TSFMs for downstream tasks.