arXiv:2607. 04033v1 Announce Type: cross Abstract: Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the landscape of over one hundred methods remains fragmented.
By Siyuan Li, Jiabao Pan, Yumou Liu, Zhuoli Ouyang, Xin Jin, Xinglong Xu, Jingxuan Wei, Shengye Pang, Jintao Che, Xuanhe Zhou, Conghui He, Cheng Tan
GradSAT is a new framework that improves satisfiability solving for Quantifier-Free Floating-Point formulas by treating each SMT clause as an independent multi‑task learning problem. It uses dynamic gradient normalization (GradNorm) to balance gradient magnitudes across clauses, preventing a few difficult clauses from dominating the optimization trajectory. The approach combines a GPU‑accelerated PyTorch backend for continuous relaxation with a precise local search engine, yielding a robust, highly parallelizable solver.
By Yuanzhuo Zhang
arXiv:2607. 07206v1 Announce Type: new Abstract: Adaptive optimizers mix several mechanisms: a metric or preconditioner maps gradients to descent directions, while estimation, memory, step-size control, constraints, stochasticity, target modification, and discretization determine which directions are available and how they are used.
By Zavier Li
The paper introduces PolyFormer, a physics-informed machine learning framework that learns compact polytopic representations of complex constraints. By transforming constraint-induced geometry into efficient polytopic reformulations, PolyFormer reduces optimization complexity and enables the use of standard solvers. Evaluations on large‑scale resource aggregation, network‑constrained optimization, and uncertainty‑aware optimization show up to 6,400‑fold speedups and 99.87% memory savings while keeping feasibility and objective errors low.
By Yilin Wen, Yi Guo, Bo Zhao, Wei Qi, Zechun Hu, Colin Jones, Jian Sun
arXiv:2606. 08638v1 Announce Type: cross Abstract: Recent research has developed practical, parallelizable first-order methods for large scale linear programming, but performance is highly dependent on hyperparameter selection.
By Siddharth Prasad, Dravyansh Sharma
arXiv:2606. 14970v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specific data.
By Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin, Aleksandr Beznosikov