Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2601.13519v4 Announce Type: replace-cross Abstract: This paper introduces a new problem-dependent regret measure for online convex optimization with smooth losses. The notion, which we call the...
The paper introduces a straightforward framework that transforms dynamic regret minimization into switching regret minimization by constructing an unbiased random sequence for any comparator sequence. Using this reduction, the authors derive dynamic regret bounds for strongly convex and exp-concave losses of “~O(T^{1/3}P_T^{2/3})” and for general convex losses of “O(√{T(1+P_T)})”, matching known minimax optimal results. The approach leverages off-the-shelf switching regret algorithms and controlled variance to achieve these bounds.
arXiv:2606. 02948v1 Announce Type: new Abstract: Curvature adaptivity is a classical theme in online optimization: for convex Lipschitz losses, adaptive methods interpolate between the optimal $O(\sqrt{T})$ regret for general convex losses and $O(\log T)$ regret under strong convexity.
arXiv:2608. 15050v1 Announce Type: new Abstract: We study online convex optimization with dueling (pairwise comparison) feedback, where the learner observes only a binary preference between two queried points.
arXiv:2606. 03831v1 Announce Type: new Abstract: This paper investigates non-stationary online learning using the metric of interval regret, which requires an online algorithm to perform well over every time interval.
arXiv:2606. 06043v1 Announce Type: cross Abstract: Follow-the-regularized-leader framework has shown effectiveness and flexibility in online learning problems, where the choice of learning rates are known to be crucial.