arXiv Machine Learning
Jul 9
Restricted Dynamic Geometric Complexity: Certificates for Structured Preconditioning
arXiv:2607. 07204v1 Announce Type: cross Abstract: Optimization geometrodynamics views optimizer state as evolving geometry.
Restricted Dynamic Geometric Complexity: Path-Space Reduction and M\"obius--Jacobi Response
arXiv:2607. 07204v2 Announce Type: replace-cross Abstract: Structured preconditioners restrict optimization to a small family of positive metrics, but endpoint condition-number reachability does not measure the geometric effort required to reach a useful metric.
Related stories
arXiv Machine Learning
Jul 9
Geometric--Nongeometric Optimizer Calculus: A Modular Language for Reachable Gradient Methods
arXiv:2607. 07206v1 Announce Type: new Abstract: Adaptive optimizers mix several mechanisms: a metric or preconditioner maps gradients to descent directions, while estimation, memory, step-size control, constraints, stochasticity, target modification, and discretization determine which directions are available and how they are used.
arXiv Machine Learning
Jul 9
Optimization Geometrodynamics: A Framework for Dynamic Geometric Optimization
arXiv:2607. 06723v1 Announce Type: cross Abstract: Most gradient-based optimization methods move parameters through a fixed background geometry, even when their internal states implicitly define changing notions of length, curvature, and preconditioning.
arXiv Machine Learning
Jul 14
Optimization Geometrodynamics: Variational Reduction and Interaction Curvature
arXiv:2607. 06723v2 Announce Type: replace-cross Abstract: Adaptive optimizers carry hidden states that change how visible gradients become parameter motion.
arXiv Machine Learning
Jul 28
When Rates Are Geometric: Rate-Certificate Transfer for Contact Splittings in Optimization
arXiv:2607. 23642v1 Announce Type: cross Abstract: Discrete optimization algorithms are often analyzed through continuous-time limiting ODEs, but a convergence certificate for the ODE is not automatically one for the discrete algorithm.
arXiv Machine Learning
Jul 14
Lower Bound on the Cumulative Constrained Violation for the OGD+Projection algorithm for Constrained Online Convex Optimization (COCO)
arXiv:2607. 10808v1 Announce Type: new Abstract: The problem of constrained online convex optimization is considered, where at each round, once a learner commits to an action $x_t \in \mathcal{X} \subset \mathbb{R}^d$, a convex loss function $f_t$ and a convex constraint function $g_t$ that drives the constraint $g_t(x)\le 0$ are revealed.
arXiv Machine Learning
Jul 15
A Geometric Approach to Constrained Online Learning
arXiv:2605. 21107v2 Announce Type: replace Abstract: We study constrained online convex optimization with adversarial time-varying constraints.
arXiv Machine Learning
Jul 16
Beyond the $d^{2.5}$-mixing bound for Dikin walks on polytopes
arXiv:2607. 13943v1 Announce Type: cross Abstract: Inspired by interior-point methods (IPM) for structured convex optimization, Kannan and Narayanan introduced the Dikin walk for sampling uniformly from polytopes in 2009.
arXiv Machine Learning
5d ago
Training Non-Differentiable Networks via Optimal Transport
arXiv:2605. 01928v2 Announce Type: replace Abstract: We optimize losses that jump: spiking thresholds, quantized layers, and discrete routing put jumps in the forward pass, where backpropagation does not apply.
arXiv Machine Learning
Jun 4
Cone-Compatible Monge Geometry for High-Dimensional Ordered Optimal Transport
arXiv:2606. 04695v1 Announce Type: new Abstract: High-dimensional optimal transport is seldom available in closed form.
arXiv Machine Learning
Jul 21
Nonconvex Matrix Factorization is Geodesically Convex: Global Landscape Analysis for Fixed-rank Matrix Optimization From a Riemannian Perspective
arXiv:2209. 15130v3 Announce Type: replace-cross Abstract: We study a general matrix optimization problem with a fixed-rank positive semidefinite (PSD) constraint.
arXiv AI
2d ago
AI-Assisted Discovery and Construction of a Counterexample to the Convergence of Three-Block ADMM with the Identity Matrix as its Third Constraint Block
arXiv:2608. 14396v1 Announce Type: cross Abstract: The alternating direction method of multipliers (ADMM), as a landmark algorithm, has attracted tremendous research attention and extensive practical applications over the past two decades.