arXiv:2609. 08136v1 Announce Type: new Abstract: This paper introduces rlaopt, a PyTorch-based package for large-scale optimization and scientific computing using randomized numerical linear algebra (RandNLA).
By Pratik Rathore, Zachary Frangella, Parth Nobel, Xuning Hu, Madeleine Udell
arXiv:2606. 14970v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specific data.
By Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin, Aleksandr Beznosikov
arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.
By Yijiang Pang, Shuyang Yu, Bao Hoang, Jiayu Zhou
arXiv:2607. 28036v1 Announce Type: new Abstract: It is well known that Newton's method converges faster when the initial guess is closer to a root of a system of nonlinear equations.
By R\'emy Vallot (CB, Michelin), Florian de Vuyst (BMBI), Thibault Dairay (CB, Michelin), Mathilde Mougeot (CB, ENSIIE, ENS Paris Saclay)
arXiv:2606. 07574v1 Announce Type: cross Abstract: Manifold-constrained hyper-connections (mHCs) have recently been proposed as a principled extension of hyper-connections, where the residual mixing matrices are constrained to be doubly stochastic via projection onto the Birkhoff polytope.
By Chenrui Wang, Yixuan Qiu
arXiv:2607. 22467v1 Announce Type: new Abstract: Data scarcity poses a fundamental challenge in training generative models to produce initial guesses for parametric optimization problems that are otherwise numerically expensive to solve.
By Anjian Li, Ryne Beeson