arXiv Machine Learning
Small Gradient Norm Regret for Online Convex Optimization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
1d ago
Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
arXiv:2602.08372v2 Announce Type: replace Abstract: We study dynamic regret minimization in non-stationary online learning, with a primary focus on follow-the-regularized-leader (FTRL) methods. FTRL...
arXiv Machine Learning
Jul 28
Gradient-Variation Regret Bounds for Unconstrained Online Learning
arXiv:2604. 11151v2 Announce Type: replace Abstract: We develop parameter-free algorithms for unconstrained online learning with regret guarantees that scale with the gradient variation $V_T(u) = \sum_{t=2}^T \|\nabla f_t(u)-\nabla f_{t-1}(u)\|^2$.
arXiv Machine Learning
Sep 10
Improved Dimension Dependence for Bandit Convex Optimization with Gradient Variations
The paper presents an improved analysis of non‑consecutive gradient variation in Bandit Convex Optimization (BCO) with two‑point feedback, leading to better dimension dependence for both convex and strongly convex functions compared to prior work. It also derives new problem‑dependent guarantees such as gradient‑variance and small‑loss regret bounds, extends the technique to one‑point bandit linear optimization over hyper‑rectangular domains, and establishes the first gradient‑variation dynamic and universal regret bounds for two‑point BCO.
arXiv Machine Learning
Aug 18
Online Convex Optimization with Dueling Feedback
arXiv:2608. 15050v1 Announce Type: new Abstract: We study online convex optimization with dueling (pairwise comparison) feedback, where the learner observes only a binary preference between two queried points.
arXiv Machine Learning
Sep 21
From Switching to Dynamic Regret: A Simple Reduction via Unbiased Random Sequences
The paper introduces a straightforward framework that transforms dynamic regret minimization into switching regret minimization by constructing an unbiased random sequence for any comparator sequence. Using this reduction, the authors derive dynamic regret bounds for strongly convex and exp-concave losses of “~O(T^{1/3}P_T^{2/3})” and for general convex losses of “O(√{T(1+P_T)})”, matching known minimax optimal results. The approach leverages off-the-shelf switching regret algorithms and controlled variance to achieve these bounds.
arXiv Machine Learning
Jun 19
Adversarial Bandit Optimization with Globally Bounded Perturbations to Convex Losses
arXiv:2606. 19891v1 Announce Type: new Abstract: We study adversarial bandit optimization in which the loss functions may be non-convex and non-smooth.