arXiv AI

LLMs Can Design Near-Optimal OR Algorithms

The paper investigates whether large language models (LLMs) can design effective algorithms for well-specified operations research (OR) problems, focusing on inventory control, queueing network control, and assortment optimization. Two usage levels are examined: (1) the model receives a single problem instance and outputs a solution, and (2) the model receives only a problem class description and returns a general algorithm mapping instance parameters to solutions. Using a single untuned prompt and a Python sandbox, the strongest tested model, gpt-5.6-sol, matches or surpasses existing methods on nearly all evaluated instances, even when the algorithm is fixed before seeing evaluation cases, and performance improves markedly across models released within eight months.

arXiv AI

2d ago

OR for AI That Does OR: Routing LLMs up the Escalator inside the OSCAR Framework

The paper introduces OSCAR, an LLM‑based framework that translates business descriptions into accurate optimization models while verifying and improving them through a simulator, coder, and reviewer. OSCAR uses a cost‑ordered escalation strategy to select among LLMs of varying price and capability, achieving 95–100% accuracy on benchmark problems with local, open‑weight models. The framework also provides competitive guarantees and token‑cost advantages over existing LLMs like Codex and Claude Code.

By Jinzhi Bu, Haixin Tang, Huanan Zhang

llms benchmarks

arXiv AI

Jun 2

FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization

arXiv:2605. 25246v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation, yet practical operations research and optimization problems often require a harder capability: designing scalable algorithms that exploit problem structure and outperform direct formulation-and-solve baselines.

By Minwei Kong, Chonghe Jiang, Ao Qu, Wenbin Ouyang, Zhaoming Zeng, Xiaotong Guo, Zhekai Li, Junyi Li, Yi Fan, Xinshou Zheng, Xi Jing, Yikai Zhang, Zhiwei Liang, Seonghoo Kim, Runqing Yang, Zijian Zhou, Sirui Li, Han Zheng, Wangyang Ying, Ou Zheng, Chonghuan Wang, Jinglong Zhao, Hanzhang Qin, Cathy Wu, Paul Pu Liang, Jinhua Zhao, Hai Wang

llms agents benchmarks

arXiv AI

Jun 16

Large Language Models as Optimizers: A Survey of Direct vs. Tool-Augmented Approaches and Their Performance Frontiers

arXiv:2606. 15577v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly involved in complex mathematical optimization, even if the pragmatic user who triggers them is unaware of it.

By Roko Peran, Luka Hobor, Mihael Kovac, Mario Brcic

llms benchmarks

arXiv AI

3d ago

Right Answers, Costly Models: The Efficiency Gap in LLM-based Optimization Modeling

The paper introduces OptTips, a knowledge base of 50 expert optimization modeling techniques, and OptDachshund, a multi‑agent framework that generates mathematical models and solver code from natural‑language problem descriptions. Using these tools, the authors create the EfficientOpt benchmark, comprising 561 expert‑reviewed tasks with paired reference implementations, to evaluate large language models (LLMs) on both correctness and computational efficiency. Their experiments with 11 LLMs show a consistent efficiency gap: even when LLMs produce correct solutions, the resulting programs often take longer to solve than expert‑crafted counterparts, highlighting the need to assess both accuracy and runtime performance in LLM‑based optimization modeling.

By Zhong Li, Xin Huang, Jinhui Wan, Xiangyi Wang, Shenkai Zhang, Ruiqi Chen, Wenyu Liu, Zaiwen Wen, Ziyan Luo

llms rag agents benchmarks

arXiv AI

Jul 29

Large Language Model for Operations Research Formulation Selection in Multi-Warehouse Inventory Allocation

arXiv:2607. 25956v1 Announce Type: new Abstract: Multi-warehouse inventory allocation is typically formulated as a mixed-integer programming (MIP) problem, yet no single formulation consistently matches heterogeneous instance-level regimes induced by demand concentration, inventory imbalance, replenishment scale, service constraints, and forecast volatility.

By Jintao Xu, Yingzheng Ma, Jiong Dong, Yongzhi Qi, Jianshen Zhang

llms fine-tuning

arXiv AI

Jun 30

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

arXiv:2407. 19633v4 Announce Type: replace Abstract: Optimization problems are pervasive in sectors from manufacturing and distribution to healthcare.

By Ali AhmadiTeshnizi, Wenzhi Gao, Herman Brunborg, Shayan Talaei, Connor Lawless, Madeleine Udell

llms fine-tuning benchmarks

arXiv Machine Learning

Aug 5

BBOWP-Bench: Evaluating LLMs on Black-Box Optimization Word Problems

arXiv:2608. 02612v1 Announce Type: cross Abstract: Formulating an optimization problem strongly affects the quality of the final solution, yet good formulations usually require substantial expertise.

By Yutaro Yamada, Kei Hiroshima, Nozomu Yoshinari, Kento Uchida, Shinichi Shirakawa

llms benchmarks

arXiv AI

Jun 26

The Capability Frontier: Benchmarks Miss 82% of Model Performance

arXiv:2606. 26836v1 Announce Type: new Abstract: Existing benchmarks typically report accuracy for a single model on a single run.

By Bradley Fowler, Ryan Smith, Daniel Thi Graviet, William Myers, Joshua Greaves, Narmeen Fatimah Oozeer, Ant\'ia Garc\'ia, Philip Quirke, Amirali Abdullah, Fazl Barez, Shriyash Kaustubh Upadhyay

llms agents benchmarks

arXiv AI

Sep 10

Automated Design of Inventory Policy with Large Language Models: An Exploratory Study

arXiv:2609.08071v1 Announce Type: new Abstract: Firms making inventory decisions have access to operational data, optimization tools, and large language models (LLMs). Typically, data characterize th...

By Fenghua Yang, Preet Baxi, Yi Zhang, Stefanus Jasin, Yanzhe Lei, Mo Liu, Parshan Pakiman

llms benchmarks

arXiv Computation and Language

Sep 10

$\Phi$-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?

arXiv:2609.10226v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in reasoning and code generation, raising the prospect that they could assist in...

By Leilei Ding, Shumin Wang, Yuting Huang, Fanqi Wan, Yinmin Zhang, Qi Han, Yiming Xu, Feiyuan Zhang, Xiaomeng Chu, Guoliang You, Wuyang Zhang, Daxin Jiang, Yanyong Zhang

llms agents benchmarks

arXiv AI

Jul 28

Opti-Q: A Constraint-Based Optimization Framework for Multi-LLM Question Planning

arXiv:2607. 22621v1 Announce Type: new Abstract: While large language models (LLMs) enable strong question answering (QA), budgeted deployment is complicated by nondeterminism and heterogeneous resource profiles (cost, latency, and energy).

By Aamir Hamid, Bharg Barot, Satvik Racharla, Tim Finin, Primal Pappachan, Roberto Yus

llms agents nlp benchmarks

arXiv AI

4d ago

Teaching LLMs to Generate Challenging MILP Instances via Solver Feedback

arXiv:2609.37356v1 Announce Type: new Abstract: Generating optimization instances that are both feasible and computationally challenging is crucial for benchmarking solvers and training learning-base...

By Jitin Singla, Parikshit Pareek, Pratik Jawanpuria, Parag Singla

llms reinforcement-learning fine-tuning benchmarks

Related stories