← Back to all news
arXiv Machine Learning By Edric Castel C. Hao, Juan Raphael S. Se\~na, Neon S. De Jesus

Is spectral predictability enough? Testing {\Omega}-matching for synthetic pretraining under distribution shift

Read the original on arXiv Machine Learning →

The paper investigates whether matching the spectral predictability Ω of synthetic pretraining data to a target dataset is sufficient for effective transfer learning under distribution shift. Experiments across LSTM, PatchTST, and TCN architectures show that Ω-matched pretraining can outperform an oracle baseline in synthetic-to-synthetic settings, but on the real-world ETTm1 benchmark it offers only marginal gains or even degrades performance, with Ω providing little guidance on beneficial source selection. The authors conclude that while Ω matching can help in controlled scenarios, it is not a reliable predictor of transfer success and other signal properties must be considered.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 16

Schema-Adaptive Action-Conditioned JEPA for Cross-Machine CNC Transfer under Partial Sensor Overlap

The paper introduces a schema‑adaptive action‑conditioned Joint‑Embedding Predictive Architecture (SAAC‑JEPA) for cross‑machine CNC transfer when only a subset of sensors overlap between source and target machines. Experiments show that pretraining does not improve source‑only forecasting, but a carefully selected action‑conditioned JEPA model achieves a zero‑shot RMSE of 0.546 on the target, outperforming persistence but falling short of certain baseline models. Ablation studies reveal that adding RevIN improves RMSE but harms calibration, and limited post‑lock adaptation can further reduce error.

By Ayoub Louaye Bouaziz, Matthieu Ostertag, Anton Demasles
ragbenchmarks
More like this →
arXiv Machine Learning
Oct 22

Minimax-Optimal Predictive Flow Matching under Distribution Drift

The paper introduces DyProd-FM, a method for predictive flow matching that refits flow matching models on dyadic historical prefixes and selects among them using a sleeping Prod master. It achieves dynamic regret bounds of “~O(min{N_ω+1,1+√{TΩ}}) and ~O(min{N_μ+1,1+T^{1/3}V_μ^{2/3}}) in relative velocity risk variation Ω and target‑law variation V_μ, respectively, without requiring a drift budget. These bounds are shown to be optimal up to logarithmic factors for fixed, possibly nonconvex, neural classes.

By Aadithya Srikanth, Vaneet Aggarwal
diffusion
More like this →