arXiv Machine Learning
Jul 9
Restricted Dynamic Geometric Complexity: Certificates for Structured Preconditioning
arXiv:2607. 07204v1 Announce Type: cross Abstract: Optimization geometrodynamics views optimizer state as evolving geometry.
Geometric--Nongeometric Optimizer Calculus: A Modular Language for Reachable Gradient Methods
arXiv:2607. 07206v1 Announce Type: new Abstract: Adaptive optimizers mix several mechanisms: a metric or preconditioner maps gradients to descent directions, while estimation, memory, step-size control, constraints, stochasticity, target modification, and discretization determine which directions are available and how they are used.
Related stories
arXiv Machine Learning
Jul 9
Optimization Geometrodynamics: A Framework for Dynamic Geometric Optimization
arXiv:2607. 06723v1 Announce Type: cross Abstract: Most gradient-based optimization methods move parameters through a fixed background geometry, even when their internal states implicitly define changing notions of length, curvature, and preconditioning.
arXiv Machine Learning
Jul 14
Restricted Dynamic Geometric Complexity: Path-Space Reduction and M\"obius--Jacobi Response
arXiv:2607. 07204v2 Announce Type: replace-cross Abstract: Structured preconditioners restrict optimization to a small family of positive metrics, but endpoint condition-number reachability does not measure the geometric effort required to reach a useful metric.
arXiv AI
Jun 9
Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation
arXiv:2606. 09278v1 Announce Type: cross Abstract: Large Language Models frequently hallucinate in precision-critical domains such as technical diagramming and mechanical design, where outputs must satisfy strict geometric constraints.
arXiv Machine Learning
5d ago
Manifold constrained steepest descent for smooth and closed-set optimization
arXiv:2601. 21487v2 Announce Type: replace-cross Abstract: We study minimization of smooth functions over feasible sets that have smooth embedded-manifold structure throughout or only on selected regions, using linear minimization oracles (LMOs) to determine search directions under user-chosen norms.
arXiv Machine Learning
Jun 16
Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning
arXiv:2606. 14970v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specific data.
Hugging Face Trending Papers
Jul 7
On the Condition Number Upper Bound of the L-BFGS Inverse Hessian Approximation Matrix with a Two-Sided Geometric Envelope Safeguarding Mechanism
The limited-memory BFGS (L-BFGS) algorithm is a cornerstone of large-scale optimization due to its linear memory and computational costs. However, in ill-conditioned or non-convex landscapes, the implicit inverse Hessian approximation can suffer from an exploding condition number, leading to numerical instability and degraded convergence.
arXiv Machine Learning
Jul 14
Optimization Geometrodynamics: Variational Reduction and Interaction Curvature
arXiv:2607. 06723v2 Announce Type: replace-cross Abstract: Adaptive optimizers carry hidden states that change how visible gradients become parameter motion.
arXiv Machine Learning
Jun 15
Scalable Deep Unfolding of Conic Optimizers
arXiv:2606. 13825v1 Announce Type: cross Abstract: Deep unfolding (DU) accelerates iterative optimizers by introducing learnable components and training them through unrolled iterations, but extending DU to the large-scale semidefinite programs (SDPs) common in robotics has remained limited.
arXiv Machine Learning
Jul 13
Accelerated Fully First-Order Methods for Bilevel and Minimax Optimization
arXiv:2405. 00914v4 Announce Type: replace-cross Abstract: We present in this paper novel accelerated fully first-order methods in \emph{Bilevel Optimization} (BLO).
arXiv Machine Learning
Jul 8
On the Condition Number Upper Bound of the L-BFGS Inverse Hessian Approximation Matrix with a Two-Sided Geometric Envelope Safeguarding Mechanism
arXiv:2607. 05836v1 Announce Type: cross Abstract: The limited-memory BFGS (L-BFGS) algorithm is a cornerstone of large-scale optimization due to its linear memory and computational costs.
arXiv Machine Learning
Jun 10
Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter
arXiv:2606. 09930v1 Announce Type: cross Abstract: The boundary between program execution and gradient-based optimization has long limited the use of code itself as a learnable scientific model.