arXiv Machine Learning By Cedar Site Bai

Tight Lower Bounds under Matched High-Order H\"older Smoothness and Uniform Convexity

Read the original on arXiv Machine Learning →

arXiv:2611. 05527v1 Announce Type: cross Abstract: In this paper, we provide tight lower bounds for minimizing high-order H\"older-smooth and uniformly convex functions when their orders match.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning
Oct 3

Optimal Stochastic Bilevel Optimization with First-Order Oracles

The paper investigates nonconvex–strongly-convex bilevel optimization using a stochastic first-order oracle. It introduces MRT‑FD, a single-loop first‑order algorithm that tracks the upper-level variable, the lower-level solution, and an auxiliary response from implicit differentiation, updating all variables in each iteration and approximating second‑order derivative actions via order‑p finite differences. For any fixed finite smoothness order p ≥ 1, MRT‑FD achieves an ε‑stationary point with O(ε^{‑4‑2/p}) stochastic gradient queries, and the authors prove a matching Ω(ε^{‑4‑2/p}) lower bound, thereby closing the complexity gap in this setting.

By Linxuan Pan, Junchi Yang