arXiv AI By Yike Li, Shaoxu Song, Jianmin Wang

Loss-Guided Pretraining Data Selection for Time-Series Foundation Models

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv AI
Oct 28

Dr. Post-Training: A Data Regularization Perspective on LLM Post-Training

The paper introduces Dr. Post-Training, a framework that treats general training data as a data‑induced regularizer to prevent overfitting to scarce target objectives, rather than merely a pool for selection. It constructs a feasible set of model update directions from general data and projects target‑driven updates onto this set, unifying standard training and existing data‑selection methods as special cases along a bias‑variance spectrum. Experiments on SFT, RLHF, and RLVR demonstrate that these methods outperform both standard training and state‑of‑the‑art data‑selection baselines, especially when general and target data are highly misaligned, while system optimizations keep overhead minimal.

By Pingbang Hu, Xueshen Liu, Z. Morley Mao, Jiaqi W. Ma
arXiv AI
1d ago

SWIFT: Sample-Wise Instance Fine-Tuning of Time-Series Foundation Models for Forecasting

SWIFT: Sample-Wise Instance Fine-Tuning of Time-Series Foundation Models for Forecasting proposes a new adaptation framework that fine-tunes time-series models using only the target instance’s own historical data, eliminating the need for additional datasets. The method combines Self‑Retrieval for mining relevant history, a Frequency‑Aware Loss to maintain spectral consistency, and Deviation‑Aware Dynamic Fusion to balance adaptation with general knowledge. Experiments on four TSFM backbones and six Chronos Benchmark II datasets show average reductions of 6.41% in MASE and 4.83% in WQL across 24 model‑dataset combinations.

By Yang Chen, Feifan Tian, Cong Wang, Cheng Feng, Wenbin Li, Yang Gao
arXiv Machine Learning
Sep 29

Aurora-X: Built for Extreme Time Series Forecasting

Aurora‑X is a billion‑parameter time‑series foundation model designed for extreme forecasting tasks. It employs a progressive curriculum that starts with channel‑independent pretraining, then adds cross‑variable dependencies, variable context and horizon lengths, and optional future covariates during mid‑training. A variable‑resolution post‑training stage allows adjustable temporal spans per token at inference, while a pattern‑guided mixture‑of‑experts expands capacity through sparse activation and expert specialization. An implicit quantile network head predicts arbitrary quantiles, enhancing probabilistic forecasting flexibility. Experiments on GIFT‑Eval, TIME, FEV‑Bench, TFB, and DAG‑Bench show state‑of‑the‑art performance against both pretrained TSFMs and task‑specific supervised models.

By Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng, Peng Chen, Yang Shu, Jilin Hu, Bin Yang
arXiv Machine Learning
Aug 19

Data-DPO: Direct Preference Optimization for Target Model Data Selection in LLM Post-Training

Data-DPO is a target model‑oriented supervised fine‑tuning data selection method that uses one‑step probing of the target model to generate pairwise data preferences, trains a lightweight reward model to capture these preferences, and then selects a training subset by combining target‑model preference, external quality scores, and marginal diversity. Experiments on Vision‑Flan and LLaVA‑CoT demonstrate that Data‑DPO consistently outperforms existing data selection baselines across multiple data budgets and even surpasses full data training performance.

By Peng Sun, Yi Yang, Antong Zhang, Chunxiao Li, Yanbo Wang, Dianbo Liu, xin chen, Kai Yu, Lu Chen, Tianfan Fu
arXiv AI
Jul 23

Post-Training in Time Series Foundation Models: A Unifying Framework

arXiv:2607. 20002v1 Announce Type: cross Abstract: Time series foundation models (TSFMs) have emerged as general-purpose models for time series analysis, but pretraining alone is often insufficient for reliable downstream deployment.

By Shifeng Xie, Ambroise Odonnat, Zehao Xiao, Lei Zan, Malik Tiomoko, Lujia Pan, Themis Palpanas, Boris N. Oreshkin, Chenghao Liu, Keli Zhang