Hugging Face Trending Papers

Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets

arXiv Machine Learning
Sep 22

Simpler Methods Work Better for L1 Penalized Logistic Models and Large Datasets

Linear models with an $L_1$-norm penalty are still the leading approach for high‑dimensional tasks, yet many existing solvers are slow, ineffective, and hard to parallelise, making them unsuitable for large industry‑scale corpora. The paper evaluates several recent state‑of‑the‑art methods and shows that older techniques outperform them in general use. It also demonstrates that a simple baseline—LBFGS applied to a sub‑gradient with minor tweaks—yields strong performance and is easier to support and scale in production.

By Edward Raff, James Holt
Hugging Face Trending Papers
Aug 20

Learning Early-to-Final Solution Consistency for MILP Acceleration

Mixed-Integer Linear Programming (MILP) is a fundamental problem class in operations research and combinatorial optimization, with broad applications to industrial decision-making. Owing to their NP-hardness, however, modern solvers may struggle to find high-quality solutions for challenging MILP instances within practical time limits.

arXiv AI
Jun 2

FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization

arXiv:2605. 25246v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation, yet practical operations research and optimization problems often require a harder capability: designing scalable algorithms that exploit problem structure and outperform direct formulation-and-solve baselines.

By Minwei Kong, Chonghe Jiang, Ao Qu, Wenbin Ouyang, Zhaoming Zeng, Xiaotong Guo, Zhekai Li, Junyi Li, Yi Fan, Xinshou Zheng, Xi Jing, Yikai Zhang, Zhiwei Liang, Seonghoo Kim, Runqing Yang, Zijian Zhou, Sirui Li, Han Zheng, Wangyang Ying, Ou Zheng, Chonghuan Wang, Jinglong Zhao, Hanzhang Qin, Cathy Wu, Paul Pu Liang, Jinhua Zhao, Hai Wang
arXiv Machine Learning
Sep 3

Gradient Prediction with Control Variates in the Cheap-Forward Regime

The paper investigates whether idle inference resources can help cut the high cost of scarce GPU usage during training. Using a simulated compute ledger that bills fleet work at a fraction of a GPU forward pass, the authors propose an algorithm that predicts gradients with a low‑precision, inference‑style reverse‑mode program and then refines these predictions with a few exact gradients via a control variate, turning approximation error into variance rather than bias. Experiments on a 124‑million‑parameter language model and across models ranging from 10 M to 774 M parameters show that the method can reduce simulated ledger cost when fleet work is cheap, though it also exhibits both successful transfers and failures, and does not evaluate inference‑only hardware or full optimizer‑by‑batch‑size sweeps.

By Kamil Ciosek, Nicol\`o Felicioni, Juan Elenter, Ehsan Imani
arXiv AI
Aug 20

AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems

AutoOR is a scalable synthetic data generation and reinforcement learning pipeline that trains large language models to autoformalize operations research problems expressed in natural language across linear, mixed‑integer, and non‑linear categories. By generating verified training data from standard optimization forms and using solver execution feedback as a reward signal, AutoOR enables post‑training of an 8B model to achieve state‑of‑the‑art or competitive results on six established OR benchmarks, matching significantly larger frontier models. For non‑linear problems involving physical dynamics, a curriculum RL strategy bootstraps from limited initial data, making this class tractable for post‑training.

By Sumeet Ramesh Motwani, Chuan Du, Aleksander Petrov, Christopher Davis, Philip Torr, Antonio Papania-Davis, Weishi Yan