NLData2Opt is a new benchmark that evaluates large language models on the multi‑stage reasoning required to formulate optimization problems from natural language descriptions and accompanying data. It splits the task into three steps: extracting relevant parameters from data that includes misleading attributes, computing missing parameters by transforming existing data, and predicting future values from historical observations. The benchmark shows that performance on existing, idealized tests does not translate to real‑world accuracy, highlighting a gap in current LLM capabilities.
arXiv:2506. 18421v3 Announce Type: replace-cross Abstract: The majority of data in businesses and industries is stored in tables, databases, and data warehouses.
The paper introduces OptTips, a knowledge base of 50 expert optimization modeling techniques, and OptDachshund, a multi‑agent framework that generates mathematical models and solver code from natural‑language problem descriptions. Using these tools, the authors create the EfficientOpt benchmark, comprising 561 expert‑reviewed tasks with paired reference implementations, to evaluate large language models (LLMs) on both correctness and computational efficiency. Their experiments with 11 LLMs show a consistent efficiency gap: even when LLMs produce correct solutions, the resulting programs often take longer to solve than expert‑crafted counterparts, highlighting the need to assess both accuracy and runtime performance in LLM‑based optimization modeling.
The paper surveys recent progress in supervised fine‑tuning (SFT) for large language models that excel at reasoning tasks. It reviews methodological advances, grouping them into data‑centric and algorithm‑centric approaches, and reorganizes analytical studies by data characteristics, optimization dynamics, and mechanistic insights. The survey also identifies current research focuses, remaining bottlenecks, and suggests future directions for reasoning SFT.
arXiv:2607. 18199v1 Announce Type: cross Abstract: Not all training samples contribute equally to large language model fine-tuning.
arXiv:2606. 15577v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly involved in complex mathematical optimization, even if the pragmatic user who triggers them is unaware of it.
arXiv:2608. 02641v1 Announce Type: cross Abstract: Large language models (LLMs) can translate natural-language optimization problems into solver-ready formulations, but direct code generation is brittle: schema, indexing, and semantic errors can cause compilation failures, infeasible models, or incorrect objectives, while iterative repair, search, and multi-agent workflows increase inference cost.
arXiv:2512. 14332v2 Announce Type: replace-cross Abstract: The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling LRMs to reason longer, and more accurately.
arXiv:2505.23126v5 Announce Type: replace Abstract: Although many benchmarks evaluate the reasoning abilities of Large Language Models (LLMs) within domains such as mathematics, coding, or data wrang...
The paper surveys efficient reasoning in large language models, contrasting fast intuitive (System 1) and slow deep (System 2) reasoning. It analyzes why System 2 is computationally costly yet more accurate, and why System 1 is efficient but less effective. The survey covers causes of inefficiency, patterns of reasoning behavior, and potential solutions to balance performance and computational budgets, offering actionable insights and an open‑source repository for ongoing research.
arXiv:2608. 02612v1 Announce Type: cross Abstract: Formulating an optimization problem strongly affects the quality of the final solution, yet good formulations usually require substantial expertise.
arXiv:2511.07910v3 Announce Type: replace Abstract: Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enab...
arXiv:2601. 03808v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches.