arXiv AI
Scale-Invariant Training for Time Series Foundation Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 23
Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World
The paper introduces a new closed‑form scaling law that extends Chinchilla’s original formula to handle data‑constrained regimes. It decomposes loss into undercapacity, undertraining, and overfitting components, saturating between an irreducible loss and an uninformed baseline. The authors validate the model on diverse architectures and domains, achieving state‑of‑the‑art RMSE across multiple LLM scaling‑law grids and enabling cost‑aware training allocations.
arXiv Computer Vision
Aug 25
Bounded Precision-Geometry Scaling for Robust Multi-Task Learning under Loss Scale Mismatch
arXiv:2608.21653v1 Announce Type: cross Abstract: Multi-task learning often combines losses that span several orders of magnitude, causing homoscedastic uncertainty weighting to degrade severely. We...
arXiv AI
Jul 28
Scale Weight Decay and Train Better
arXiv:2607. 23777v1 Announce Type: cross Abstract: The discovery of scaling laws has motivated training neural networks on ever increasing quantities of data.
arXiv AI
Aug 21
Let's Scale Step by Step: Compute-Efficient Hyperparameter Transfer for Large-Scale Mixture-of-Experts
arXiv:2608. 20061v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) architectures significantly expand model capacity without a proportional increase in computational cost.
arXiv Machine Learning
Jun 19
Self-Adaptive Scale Handling for Forecasting Time Series with Scale Heterogeneity
arXiv:2606. 20010v1 Announce Type: new Abstract: Current time series forecasting (TSF) research predominantly focuses on scale-homogeneous data, where different time series share similar numerical magnitude ranges.
arXiv Machine Learning
Jun 25
Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors
arXiv:2606. 25971v1 Announce Type: new Abstract: Modern neural network training relies on optimizers such as Adam and Muon which act on each weight matrix as a single object.