arXiv:2607. 03519v1 Announce Type: new Abstract: We show that projected Adam for online optimization with arbitrary moment decay parameters $\beta_1,\beta_2\in[0,1)$ can have average regret bounded away from zero.
Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.
arXiv:2607. 27383v1 Announce Type: new Abstract: We establish the first convergence guarantees for the plain vector-form \emph{Adam} optimizer under heavy-tailed stochastic noise.
arXiv:2608. 15824v1 Announce Type: new Abstract: Adam retains a moving average of past squared gradients in its denominator, but the optimization cost of this memory is not well understood.
arXiv:2608. 16760v1 Announce Type: new Abstract: Reliable optimization is central to neural network (NN) training, yet Adam, the default optimizer for modern LLMs, rests on a fragile foundation.
arXiv:2510. 22819v3 Announce Type: replace Abstract: The convergence analysis of online learning algorithms is central to machine learning theory, where the last-iterate convergence is particularly important, as it captures the learner's actual decisions and describes the evolution of the learning process over time.
arXiv:2605. 18694v2 Announce Type: replace-cross Abstract: Many tasks in modern machine learning are observed to involve heavy-tailed gradient noise during the optimization process.
arXiv:2608. 15996v1 Announce Type: new Abstract: We study second-order path-length regret in adversarial $K$-armed bandits against oblivious loss sequences.