arXiv Machine Learning
Jul 15
A Geometric Approach to Constrained Online Learning
arXiv:2605. 21107v2 Announce Type: replace Abstract: We study constrained online convex optimization with adversarial time-varying constraints.
Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles
arXiv:2607. 17607v1 Announce Type: new Abstract: We study whether stochastic nonconvex optimization can be reduced to ordinary static regret minimization in online convex optimization in a black-box manner.
Related stories
arXiv Machine Learning
Jul 30
Parameter-Free Dynamic Regret for Online Convex Optimization under Heavy-Tailed Noise
arXiv:2607. 27073v1 Announce Type: new Abstract: We study online convex optimization (OCO) in non-stationary environments under heavy-tailed noise, where the stochastic gradient oracle admits only a finite $p$-th central moment for some $p \in (1, 2]$.
arXiv Machine Learning
Jul 14
Lower Bound on the Cumulative Constrained Violation for the OGD+Projection algorithm for Constrained Online Convex Optimization (COCO)
arXiv:2607. 10808v1 Announce Type: new Abstract: The problem of constrained online convex optimization is considered, where at each round, once a learner commits to an action $x_t \in \mathcal{X} \subset \mathbb{R}^d$, a convex loss function $f_t$ and a convex constraint function $g_t$ that drives the constraint $g_t(x)\le 0$ are revealed.
arXiv Machine Learning
Aug 12
High-Dimensional Calibration from Swap Regret
arXiv:2505. 21460v2 Announce Type: replace Abstract: We study online calibration of multi-dimensional forecasts over an arbitrary convex set $P \subset \mathbb{R}^d$ relative to an arbitrary norm $|\cdot|$.
arXiv Machine Learning
Jul 13
Logarithmic High-Probability Regret for Online Convex Optimization with Two-Point Bandit Feedback
arXiv:2603. 25029v4 Announce Type: replace Abstract: We study online convex optimization (OCO) with two-point bandit feedback against a non-anticipating adaptive adversary.
arXiv Machine Learning
Aug 4
A Perturbation Approach to Unconstrained Linear Bandits
arXiv:2603. 28201v3 Announce Type: replace Abstract: We revisit the standard perturbation-based approach of Abernethy et al.
arXiv Machine Learning
Jun 9
Noise-Adaptive High-Probability Regret Bounds for Online Convex Optimization
arXiv:2606. 08028v1 Announce Type: new Abstract: We study high-probability regret bounds for online convex optimization (OCO) with strongly convex losses and establish three results that resolve open questions at the intersection of noise adaptivity, feedback structure, and constraint satisfaction.
arXiv Machine Learning
Jul 27
On the Convergence of Stochastic Low-Rank Adaptation
arXiv:2607. 21975v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) optimizes $J(B,A)=\mathcal L(W_\mathrm{base}+sBA)$ over two adapters $B \in \mathbb{R}^{m \times r}$ and $A \in \mathbb{R}^{r \times n}$ that form a low-rank update to a frozen pretrained weight matrix $W_\mathrm{base} \in \mathbb{R}^{m \times n}$.
arXiv Machine Learning
Jul 28
A Simple, Optimal and Efficient Algorithm for Online Exp-Concave Optimization
arXiv:2512. 23190v3 Announce Type: replace Abstract: Online eXp-concave Optimization (OXO) is a fundamental problem in online learning, where the goal is to minimize regret when loss functions are exponentially concave.
arXiv Machine Learning
Jun 15
Online Convex Optimization with Sublinear Noisy Probes
arXiv:2606. 14640v1 Announce Type: new Abstract: We study Online Convex Optimization (OCO) over a convex set $K\subseteq \mathbb R^d$, where in each round $t$ the learner selects $x_t\in K$ and then observes a convex loss $f_t:K\to[0,1]$, with the goal of minimizing regret to the best fixed decision in hindsight.
arXiv Machine Learning
Aug 10
Parameter-free Dynamic Regret: Time-varying Movement Costs, Delayed Feedback, and Memory
arXiv:2602. 06902v3 Announce Type: replace Abstract: In this paper, we study dynamic regret in unconstrained online convex optimization (OCO) with movement costs.
arXiv Machine Learning
Aug 12
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization
arXiv:2310. 15976v4 Announce Type: replace Abstract: signSGD is attractive in nonconvex optimization because it communicates sign-valued rather than full-precision gradients.