arXiv AI

An Evolutionary Framework for Automatic Optimization Benchmark Generation via Large Language Models

The paper introduces LLM-EBG, an evolutionary framework that uses a large language model as a generative operator to automatically create optimization benchmarks. By generating unconstrained single-objective continuous minimization problems expressed as mathematical formulas, the framework can produce benchmarks that consistently favor a target algorithm over a comparison algorithm in over 80% of trials. Landscape analysis shows that these generated problems exhibit distinct geometric traits, such as sensitivity to variable scaling, reflecting the search behaviors of different optimization methods.

arXiv AI
Oct 5

FrugalEvo: Towards Cost-Aware LLM-Guided Program Evolution

FrugalEvo introduces a cost‑aware evolutionary framework that pairs a high‑cost, powerful LLM with a cheaper LLM to explore and refine program solutions, respectively. It incorporates a cache‑efficient evolution process that maximizes prefix reuse across steps and defines a Budget‑Aware Area Under the Curve (BA‑AUC) metric to evaluate solution quality within a fixed cost budget. Across ten mathematical and systems optimization tasks, FrugalEvo matches or surpasses state‑of‑the‑art baselines in final solution quality and achieves higher BA‑AUC on nine tasks, while delivering new state‑of‑the‑art performance on circle packing for costs as low as $0.55 to $1.68.

By Hui Chen, Xuan Qi, James Xu Zhao, Zhaopeng Feng, Shilong Liu, Kuang Xu, Pang Wei Koh, Bryan Hooi
arXiv AI
Sep 12

MAPLE: Memory-Augmented Planning with Language and Evolution

MAPLE is a new agent that maintains and updates optimization problems through successive natural‑language requests, combining language‑based problem construction with mathematical programming and evolutionary search. It preserves the optimization program, accepted plans, earlier updates, and candidate solutions for future requests, enabling rapid adaptation to changing business constraints. In a benchmark of 15 trajectories and 180 updates across various operational domains, MAPLE completed all trajectories with high online scalar quality and Pareto hypervolume ratio, and maintained update validity and useful search information across substantial revisions.

By Kesheng Chen, Yamin Hu, Wenjian Luo
arXiv AI
Jun 2

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

arXiv:2606. 01286v1 Announce Type: cross Abstract: The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or provide useful training signal.

By Yangzhen Wu, Aaron J. Li, Wenjie Ma, Li Cao, Ziheng Zhou, Mert Cemri, Shu Liu, Yuran Xiu, Chenxiao Yan, Haikun Zhao, Bin Yu, Ion Stoica, Dawn Song