arXiv Machine Learning June 15, 2026 By Dhruv Sarkar, Vaneet Aggarwal

Nonlinear Two-Time-Scale Stochastic Approximation: A Sharp Phase Transition and How to Beat It

Read the original on arXiv Machine Learning →

arXiv:2606. 14488v1 Announce Type: cross Abstract: Recent finite-time analyses of nonlinear two-time-scale stochastic approximation show that under contractive assumptions the slow iterate $Y_k$ with stepsizes $\beta_k=\Theta(k^{-1})$ and $\alpha_k=\Theta(k^{-a})$, $a\in(1/2,1)$, generally satisfies a mean-square rate of order $k^{-a}$; decoupled $k^{-1}$ rates require strong local linearity.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

safety

arXiv Machine Learning

Jul 21

Scaling Limits of Constant-Stepsize SGD at Flat Minima

arXiv:2607. 16384v1 Announce Type: new Abstract: For stochastic gradient descent (SGD) with a constant stepsize $\alpha$, the invariant law of the iterates, centered at a minimizer, describes the behavior of the algorithm over long time horizons.

By Jingyi Zhang, Cheng Mao, Debankur Mukherjee

arXiv Machine Learning

Jul 16

Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration

arXiv:2607. 13414v1 Announce Type: cross Abstract: Non-expansive two-time-scale stochastic approximation is governed by a slow stochastic Krasnoselskii--Mann fixed-point iteration rather than by contraction to a unique equilibrium.

By Dhruv Sarkar, Vaneet Aggarwal

safety

arXiv Machine Learning

Jul 13

Solving Stochastic Fixed-Point Equations with High Probability

arXiv:2607. 09097v1 Announce Type: cross Abstract: We study stochastic fixed-point equations $\mathbf{T}(\mathbf{x}) = \mathbf{x}$ over normed spaces $(\mathcal{E}, \|\cdot\|)$, where the operator $\mathbf{T}$ is nonexpansive or contractive and is accessed only through unbiased stochastic evaluations with bounded second central moment.

By Jelena Diakonikolas

arXiv Machine Learning

5d ago

Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation

arXiv:2606. 21253v2 Announce Type: replace Abstract: Continual learning that is gradient-free, local, online, and append-only is attractive for edge and streaming deployment, but its value is usually argued informally.

By Jianwei Lou (RailMind Systems, Neuss, Germany)

agents computer-vision

arXiv Machine Learning

Jun 4

Pseudospectral Bounds for Transient Amplification in Coupled Gradient Descent

arXiv:2606. 04031v1 Announce Type: new Abstract: Coupled gradient descent--where the update of one parameter block depends on another--underlies bilevel optimization, two-time-scale stochastic approximation, and adversarial training.

By Ahanaf Hasan Ariq

safety

arXiv Machine Learning

Aug 6

The Sample Complexity of Distributionally Robust PAC Learning under Cressie--Read Divergences

arXiv:2608. 04686v1 Announce Type: new Abstract: We study distributionally robust PAC learning for the $0$--$1$-loss, where adversarial perturbations of the data distribution are constrained by a Cressie--Read divergence of order $k>1$ and radius $\rho\geq 0$.

By Elad Aigner-Horev, Daniel Rosenberg, Roi Weiss

safety

Related stories

Scaling Limits of Constant-Stepsize SGD at Flat Minima

Non-Expansive Two-Time-Scale Stochastic Approximation: A Fixed-Schedule One-Quarter Barrier and Bias-Corrected Acceleration

Solving Stochastic Fixed-Point Equations with High Probability

Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation

Pseudospectral Bounds for Transient Amplification in Coupled Gradient Descent

The Sample Complexity of Distributionally Robust PAC Learning under Cressie--Read Divergences