From Geometry to Generalization: Why Row Normalization Can Beat Adam and Muon
Read the original on arXiv AI →The paper demonstrates that row-wise normalization can achieve higher population accuracy than full-batch Adam, a proxy for random-reshuffling Adam, and exact-SVD Muon in high-dimensional multiclass classification. Under an isotropic Gaussian-cloud data model, this advantage stems from row normalization’s class-wise Euclidean geometry preserving decision-boundary directions, whereas Adam’s coordinate-wise geometry and Muon’s spectral geometry introduce distortions. The advantage also persists in more general settings, including class means with independently oriented covariances, power-law spectra with class-mean exponent below one, and even under heavily anisotropic test noise, though it can reverse when covariances are diagonal and close, unless a random rotation aligns them with Adam’s coordinate axes.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.