RanSOM: Second-Order Momentum with Randomized Scaling for Constrained and Unconstrained Optimization
Read the original on arXiv Machine Learning →The paper introduces RanSOM, a second‑order momentum algorithm that eliminates the need for a Lipschitz Hessian or auxiliary forward/backward passes by randomizing the step length. By sampling the step from an exponential distribution and using a Stein‑type identity, the method obtains an unbiased estimate of the gradient change with a single Hessian‑vector product that also serves the next gradient computation. RanSOM-E achieves optimal oracle‑complexity exponents for heavy‑tailed noise without clipping, while RanSOM-B preserves feasibility for Frank‑Wolfe iterates, and experiments on CIFAR‑10 show comparable accuracy to existing methods with lower update time.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.