The paper addresses instability in bilevel optimization solutions when problem data changes. It proposes a lifted formulation for the optimistic setting that remains stable under mild assumptions, without requiring convexity or smoothness. The approach accommodates integer restrictions and disjunctive constraints, relies on pointwise and local calmness of the lower-level problem, and offers computational advantages including an outer approximation algorithm.
By Johannes O. Royset
arXiv:2512. 02494v2 Announce Type: replace Abstract: Differentiable optimization layers enable learning systems to make decisions by solving embedded optimization problems.
By Zihao Zhao, Kai-Chia Mo, Shing-Hei Ho, Brandon Amos, Kai Wang
The paper presents a unified taxonomy that classifies machine‑learning and artificial‑intelligence applications according to mathematical programming paradigms such as linear, quadratic, mixed‑integer, conic, bilevel, and others. It standardizes notation, identifies key inputs, decision variables, and principal formulations for each application, and discusses structural properties, solution strategies, and limitations. The authors compare tractability, relaxation quality, decomposition, approximation guarantees, and scalability across paradigms, emphasizing that mathematical programming serves as a disciplined interface between predictions and constrained decisions rather than a universal modeling claim.
By Chaosheng Dong
arXiv:2609.16350v1 Announce Type: new
Abstract: Federated stochastic bilevel optimization has been actively studied in recent years due to its widespread applications in machine learning. However, mo...
By Yihan Zhang, Rohit Dhaipule, Chiu C Tan, Haibin Ling, Hongchang Gao
arXiv:2608. 15143v1 Announce Type: new Abstract: Constraint solving is a declarative approach for solving combinatorial satisfaction and optimization problems.
By Tias Guns, Ignace Bleukx, Hendrik Bierlee, Jo Devriendt, Emilio Gamba, Orestis Lomis, Wout Piessens, Thomas Sergeys, Dimos Tsouros, Wout Vanroose, H\'el\`ene Verhaeghe
The paper presents cvxgenrust, an open‑source tool that generates custom Rust code for solving families of parameterized convex optimization problems defined in CVXPY. It canonicalizes problem families, extracts affine maps to Clarabel cone‑program data, and produces a specialized Rust crate that updates parameters and calls Clarabel natively at runtime. The generated solver can also be exposed to Python and registered as a custom CVXPY solver, supporting a wide range of convex problems up to semidefinite and exponential‑cone programs, and demonstrates reduced runtime compared to direct CVXPY solves and performance comparable to CVXPYgen.
By Hao Zhu, Joschka Boedecker
arXiv:2609. 30501v1 Announce Type: new Abstract: Although bilevel optimization (BLO) has emerged as a powerful framework for addressing many complex and nested machine learning problems in recent years, most existing studies are confined to the lower-level strongly convex (LLSC) or lower-level generally convex (LLGC) settings (i.
By Zhiyao Zhang, Menglu Yu, Alvaro Velasquez, Nathaniel D. Bastian, Jia Liu
arXiv:2509. 21725v3 Announce Type: replace Abstract: A bilevel optimization problem consists of two optimization problems nested as an upper- and a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem.
By Takuya Kanayama, Yuki Ito, Tomoyuki Tamura, Masayuki Karasuyama
arXiv:2511. 19656v3 Announce Type: replace Abstract: Although upper bound guarantees for bilevel optimization have been widely studied, progress on lower bounds has been limited due to the complexity of the bilevel structure.
By Kaiyi Ji
arXiv:2606. 23939v1 Announce Type: cross Abstract: Variable projection is a classical technique for separable nonlinear least-squares problems, in which variables that enter linearly are eliminated exactly, yielding a reduced nonlinear problem.
By Emanuele Zangrando, Sara Venturini, Francesco Rinaldi, Francesco Tudisco
arXiv:2608. 12704v1 Announce Type: cross Abstract: Multi-objective bilevel optimization has wide applications in the AI area such as automated learning and multi-task meta-learning.
By Yicong Jiang, Feihu Huang
The paper investigates nonconvex–strongly-convex bilevel optimization using a stochastic first-order oracle. It introduces MRT‑FD, a single-loop first‑order algorithm that tracks the upper-level variable, the lower-level solution, and an auxiliary response from implicit differentiation, updating all variables in each iteration and approximating second‑order derivative actions via order‑p finite differences. For any fixed finite smoothness order p ≥ 1, MRT‑FD achieves an ε‑stationary point with O(ε^{‑4‑2/p}) stochastic gradient queries, and the authors prove a matching Ω(ε^{‑4‑2/p}) lower bound, thereby closing the complexity gap in this setting.
By Linxuan Pan, Junchi Yang