arXiv Machine Learning

CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning

arXiv:2606. 07031v1 Announce Type: new Abstract: Self-supervised learning (SSL) for time-series representation learning is dominated by two paradigms: contrastive methods, which face challenges in constructing positive or negative pairs, and masking-based methods, which disrupt the temporal continuity of time-series signals.

arXiv Machine Learning
2d ago

An Input-Frugal Deep Learning Framework for Weather-Driven National Crop-Yield Forecasting: A Case Study of Brazilian Soybean

The paper introduces a lightweight deep learning framework that forecasts Brazilian soybean yields using only routine weather data and two simple static inputs (crop year and agro-environmental label). Across 20 seasons, transformer-based models achieved the highest accuracy, outperforming traditional ridge regression and a moving‑average baseline by nearly 48%. Ablation studies show that the static inputs and spatial expansion improve performance without adding complexity, and SHAP analysis highlights the importance of crop year and weather variables in driving yield variations.

By Fernando Dupin da Cunha Mello (Stricto Sensu Department, SENAI CIMATEC University, Salvador, Bahia, Brazil), Prashant Kumar (Global Centre for Clean Air Research), Erick G. Sperandio Nascimento (Stricto Sensu Department, SENAI CIMATEC University, Salvador, Bahia, Brazil)
arXiv Machine Learning
Jul 2

LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning

arXiv:2607. 00958v1 Announce Type: new Abstract: Time series are central to modern data mining applications, from industrial telemetry and server metrics to finance and physiology, yet time-series self-supervised learning often depends on view and augmentation choices that encode domain-specific invariances.

By Alexander Chemeris, Ming Jin, Randall Balestriero
Hugging Face Trending Papers
Sep 17

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

SETTer is a transformer-based model designed for long‑term multivariate time‑series forecasting. It introduces decoupled self‑attention and hybrid masking to better capture short‑ and long‑term patterns across time and channel dimensions, while adding simple explainable structures to highlight discriminative patterns. Experiments on real‑world benchmarks show that a single‑layer SETTer outperforms state‑of‑the‑art models in 88% of scenarios.

arXiv Machine Learning
Sep 18

SETTer: Sparse-Encoder Transformer for Long-term Multivariate Time Series Forecasting

SETTer is a transformer-based model designed for long‑term multivariate time‑series forecasting. It introduces decoupled self‑attention and hybrid masking to better handle high dimensionality and complex relationships, while adding explainable structures to highlight discriminative patterns. Experiments on real‑world benchmarks show that SETTer outperforms state‑of‑the‑art models in 88% of scenarios.

By Abraham Ezema, Chijioke Eze, Ferdinanda Ponci, Antonello Monti
arXiv Machine Learning
5d ago

Aurora-X: Built for Extreme Time Series Forecasting

Aurora‑X is a billion‑parameter time‑series foundation model designed for extreme forecasting tasks. It employs a progressive curriculum that starts with channel‑independent pretraining, then adds cross‑variable dependencies, variable context and horizon lengths, and optional future covariates during mid‑training. A variable‑resolution post‑training stage allows adjustable temporal spans per token at inference, while a pattern‑guided mixture‑of‑experts expands capacity through sparse activation and expert specialization. An implicit quantile network head predicts arbitrary quantiles, enhancing probabilistic forecasting flexibility. Experiments on GIFT‑Eval, TIME, FEV‑Bench, TFB, and DAG‑Bench show state‑of‑the‑art performance against both pretrained TSFMs and task‑specific supervised models.

By Xingjian Wu, Chenjuan Guo, Xiangfei Qiu, Zhigang Hu, Hanyin Cheng, Peng Chen, Yang Shu, Jilin Hu, Bin Yang
arXiv Machine Learning
Sep 21

A Lightweight Plug-in Gate for Transformer-Based Time-Series Forecasters

The paper introduces a lightweight pre‑encoder gate for Transformer‑based time‑series forecasters, which assigns sigmoid scores to covariate representations before they enter the encoder. The gate is evaluated as a plug‑in for models such as TimeXer, iTransformer, and PatchTST on datasets including ETTm1, ETTm2, Traffic, Energy, and ILI, showing competitive performance and the ability to regulate covariate admission via a usage penalty. Experiments also explore gate placement, initialization, and feature importance using VIF‑informed permutation diagnostics.

By Hongkai Zhuang, Tao Huang, Chen Hou
arXiv AI
Sep 3

SMart: A Multi-source Multi-phase Time Series Representation Transfer Framework

SMart is a new time series representation learning framework that combines a multi-phase recurrence plot recovery task with a source dataset selector. The recovery task uses three alternative modes to guide the encoder in capturing time series dynamics, while the selector chooses multiple suitable source datasets to augment the target dataset during pre‑training. Experiments demonstrate that SMart surpasses state‑of‑the‑art models, reducing mean absolute error by up to 19.5% in regression and increasing classification accuracy by up to 1.34%.

By Fang He, Wang-chien Lee