arXiv Machine Learning July 14, 2026 By Jianhao Ma, Yuxin Chen

WSqD: A Horizon-Free Learning Rate Schedule for Large Model Training

Read the original on arXiv Machine Learning →

arXiv:2607. 10959v1 Announce Type: new Abstract: Standard learning rate schedules such as cosine annealing are tied to a fixed training horizon, limiting their ability to accommodate post hoc horizon extension.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 26

Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging

arXiv:2602.03702v2 Announce Type: replace-cross Abstract: Large language models are increasingly trained in continual or open-ended settings, where the total training horizon is not known in advance....

By Alexandru Meterez, Pranav Ajit Nair, Depen Morwani, Cengiz Pehlevan, Sham Kakade

llms

arXiv AI

Jul 28

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling

arXiv:2510. 14717v2 Announce Type: replace-cross Abstract: Increasing the batch size during training -- a ''batch ramp'' -- is a promising strategy to accelerate large language model pretraining.

By Alexandru Meterez, Depen Morwani, Jingfeng Wu, Costin-Andrei Oncescu, Cengiz Pehlevan, Sham Kakade

llms

arXiv Machine Learning

Sep 15

Optimal Learning Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay

arXiv:2602.06797v3 Announce Type: replace-cross Abstract: We study optimal learning rate (LR) schedules under the functional scaling law (FSL) framework (Li et al., 2025), which decomposes training d...

By Binghui Li, Zilin Wang, Fengling Chen, Shiyang Zhao, Ruiheng Zheng, Lei Wu

arXiv Machine Learning

Jun 16

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding

arXiv:2602. 00781v2 Announce Type: replace Abstract: Online reinforcement learning in non-episodic, finite-horizon MDPs remains underexplored and is challenged by the need to estimate returns to a fixed terminal time.

By Jiamin Xu, Kyra Gan

reinforcement-learning benchmarks

arXiv AI

Jun 3

q0: Primitives for Hyper-Epoch Pretraining

arXiv:2606. 03938v1 Announce Type: cross Abstract: Multi-epoch training is becoming the standard now that compute is growing faster than the supply of high-quality text.

By Bishwas Mandal, Shmuel Berman, Akshay Vegesna, Samip Dahal

efficiency benchmarks

arXiv Machine Learning

Jul 15

Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps

arXiv:2607. 12360v1 Announce Type: new Abstract: The cooldown phase of a warmup-stable-decay (WSD) learning-rate schedule, now a default in large-model pretraining, lowers the final training loss in some settings and does nothing in others.

By Subham Singh, Ashutosh Mishra, Subha Raut

One email a morning, machine-written

Related stories

Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling

Optimal Learning Rate Schedules under Functional Scaling Laws: Power Decay and Warmup-Stable-Decay

Fast Non-Episodic Finite-Horizon RL with K-Step Lookahead Thresholding

q0: Primitives for Hyper-Epoch Pretraining

Same Loss, Same Noise, Opposite Schedules: Noise Structure and Optimizer Normalization Jointly Determine Whether Learning-Rate Cooldown Helps