arXiv Machine Learning July 7, 2026 By Ashmitha R, J\"org Frochte

Directional Curvature from Armijo Backtracking: A Low-Cost Sharpness Probe and a Calibration-Free Learning-Rate Safeguard for Adam

Read the original on arXiv Machine Learning →

arXiv:2607. 03998v1 Announce Type: new Abstract: The local sharpness of the loss, the top Hessian eigenvalue $\lambda_1$, determines the largest stable gradient step, but measuring it normally requires Lanczos or Hessian-vector iterations.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 21

Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

arXiv:2605. 29547v2 Announce Type: replace-cross Abstract: Deep learning optimization relies heavily on the assumption of smooth loss landscapes, a condition systematically violated by modern architectures due to non-smooth components such as ReLU activations and quantization operators.

By Ruoran Xu, Borong She, Xiaobo Jin, Qiufeng Wang

efficiency

arXiv AI

Jul 24

The Active Ingredient in Muon's Grokking

arXiv:2607. 20512v1 Announce Type: cross Abstract: The Muon optimizer reaches the grokking threshold on modular arithmetic faster than AdamW.

By Yufeng Wang

rag benchmarks

arXiv Machine Learning

Aug 6

The Loss Does Not See the Basis, but Adam Does

arXiv:2608. 05136v1 Announce Type: new Abstract: Gradient descent on a factored model $W = UV^\top$ is implicitly biased toward low-rank solutions, while Adam, starting from the same small initialization, is not.

By Devender Singh

llms safety

arXiv Machine Learning

Jun 25

Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

arXiv:2606. 25971v1 Announce Type: new Abstract: Modern neural network training relies on optimizers such as Adam and Muon which act on each weight matrix as a single object.

By Alexander H\"agele, Alejandro Hern\'andez-Cano, Atli Kosson, Martin Jaggi

arXiv Machine Learning

Jul 21

Backpropagation-Free Trunk Training via the Split Forward Gradients

arXiv:2607. 16612v1 Announce Type: cross Abstract: Backpropagation makes training deep networks memory intensive because it must store intermediate activations.

By Tian Qin, Wei-Min Huang

llms benchmarks

arXiv AI

Jun 30

Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule

arXiv:2606. 29119v1 Announce Type: cross Abstract: We introduce a pre-registered screening rule that decides, before any implementation, whether an evolutionary / population / lifecycle outer loop over neural-network parameters or structure is worth building.

By Ramchand Kumaresan

Related stories

Singularity-aware Optimization via Randomized Geometric Probing: Towards Stable Non-smooth Optimization

The Active Ingredient in Muon's Grokking

The Loss Does Not See the Basis, but Adam Does

Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors

Backpropagation-Free Trunk Training via the Split Forward Gradients

Knowing in Advance When an Evolutionary Outer Loop Will Not Help: A Pre-Registered Cheap-Baseline Screening Rule