The paper investigates nonconvex–strongly-convex bilevel optimization using a stochastic first-order oracle. It introduces MRT‑FD, a single-loop first‑order algorithm that tracks the upper-level variable, the lower-level solution, and an auxiliary response from implicit differentiation, updating all variables in each iteration and approximating second‑order derivative actions via order‑p finite differences. For any fixed finite smoothness order p ≥ 1, MRT‑FD achieves an ε‑stationary point with O(ε^{‑4‑2/p}) stochastic gradient queries, and the authors prove a matching Ω(ε^{‑4‑2/p}) lower bound, thereby closing the complexity gap in this setting.
By Linxuan Pan, Junchi Yang
arXiv:2511. 22331v2 Announce Type: replace-cross Abstract: Bilevel optimization minimizes an objective function, defined by an upper-level problem whose feasible region is the solution of a lower-level problem.
By Lesi Chen, Jingzhao Zhang
In this work, we study the oracle complexity of finding an $ε$-stationary point for nonconvex-strongly-convex (NC-SC) bilevel optimization using only first-order oracles. Existing methods achieving th...
arXiv:2509. 14952v3 Announce Type: replace Abstract: This paper considers the smooth bilevel optimization in which the lower-level problem is strongly convex and the upper-level problem is possibly nonconvex.
By Zhuanghua Liu, Luo Luo
arXiv:2609.08380v1 Announce Type: cross
Abstract: We study the stochastic first-order oracle complexity for constrained or regularized convex-concave min-max optimization and stochastic monotone vari...
By Ahmet Alacaoglu
arXiv:2608. 09004v1 Announce Type: cross Abstract: We prove a sharp lower bound for smooth nonconvex stochastic optimization with uniformly bounded gradient noise.
By Jikai Jin