arXiv Machine Learning

An important Bottleneck in LLM Optimizers: Principle-Level Local Optima

The paper identifies a phenomenon called principle-level search collapse in feedback-driven large language model (LLM) optimizers, where the search narrows to a limited set of design principles despite ongoing candidate generation and minor improvements. Using a diagnostic benchmark across molecular optimization, network sentinel-node selection, and GPU-kernel optimization, the authors find that 78.3% of runs exhibit collapse, with 55.3% showing strong support. They evaluate common remedies such as diversity injection and reference-principle interventions, noting limited and inconsistent benefits, and emphasize the need for online collapse detection and improved mechanisms for generating and maintaining alternative principles.

arXiv Machine Learning
1d ago

Organon : From Evolutionary Search to Recursive Self-Improving Agents for Scientific Discovery

Organon is a recursively self‑improving agent designed for scientific discovery that autonomously revises its search strategies, population management, diagnostics, memory, and worker coordination during execution. By diagnosing and correcting its own adaptation procedures, the agent continuously refines the mechanisms guiding its future self‑improvement. Across 21 computational discovery tasks—including algorithm design, GPU kernel optimization, and systems optimization—Organon outperforms competing frameworks, achieving a 20.5% mean performance gain on ALE‑Bench Lite and up to 1.67× speedups on KernelBench.

By Cheikh Ahmed, Zhiyu Zhang, Kun Mao, Yong Zhang, Zirui Zhou
arXiv AI
Sep 30

OptiCom : A Unified Framework for State-Conditioned Composition in LLM-Driven Optimization

OptiCom introduces a unified framework for state-conditioned composition in large language model (LLM)-driven optimization. It models LLM optimizers within a shared configuration space (artifact, query, operator, evaluation, memory, strategy) and uses an Optimization Controller to dynamically compose mechanisms while a Strategy Adapter refines long-term preferences. Experiments on 32 benchmark groups show OptiCom outperforms 14 configurations, achieving the top score in 23 groups.

By Chenxing Wei, Sichen Liu, Lizhao Liu, Ningyuan Sun, Chen Bingzhou, Ying He, Bo Jiang, Fei Yu, Yao Shu
arXiv Machine Learning
Oct 15

PMO-Dock: Benchmarking Docking, Specificity, and Generalization in Molecular Optimization

PMO-Dock introduces a standardized benchmark for docking‑based molecular optimization, comprising 25 tasks that span hit generation, lead optimization, and a specificity task that rewards strong on‑target binding while penalizing off‑target interactions. The protocol separates model development from final evaluation, using validation tasks for hyperparameter tuning and held‑out test tasks evaluated under a strict oracle budget to mimic realistic drug‑discovery constraints. Four diverse methods—Saturn, GenMol, Genetic‑guided GFlowNet, and ChemLlama—were benchmarked, revealing no single method dominates across all tasks and highlighting limited hyperparameter transferability, which underscores the need for method‑dependent selection strategies.

By Gor Simonyan, Tatevik Abrahamyan, Narek Abelyan, Tigran Fahradyan, Hrant Khachatrian
arXiv AI
Aug 28

AgentFold: Closed-Loop Agentic Search for Protein Folding Model Design

AgentFold is a multi‑agent framework that treats protein‑folding model design as a closed‑loop search over executable code variants. Starting from the ESMFold codebase, the agents generate hypotheses, modify and debug code, evaluate model variants, and store both successes and failures in structured memory, guided by an MCTS‑style policy that allocates GPU resources. In an engineering‑scale experiment, AgentFold explored about 80 variants using 5,000 GPU‑hours and 170 million LLM tokens, improving the best lDDT score by 7.5% over independent Codex proposals and outperforming a random‑search baseline, while also uncovering empirical design patterns such as the benefits of early, soft, learnable priors.

By Mingquan Liu, Jiangyu Chen, Hanqun Cao, Xujun Zhang, Pengsen Ma, Xiangru Tang, Shuting Jin, Zhuo Yang, Tianfan Fu, Fang Wu, Xiangxiang Zeng
arXiv AI
Oct 9

A Closer Look at Agentic BBO: Benchmarking LLM Agents for Black-Box Optimization

The paper introduces AgenticBBO-Bench, a cross‑domain benchmark for evaluating large language model agents on black‑box optimization tasks such as synthetic functions, hyperparameter tuning, database tuning, chip design, and molecular design. Experiments show that agentic BBO outperforms direct LLM‑based methods in all five domains and beats the best numerical optimizers in four. The study also analyzes the impact of optimization tools, task semantics, prior knowledge, and the LLM’s role, finding that numerical tools are not consistently helpful, task semantics are broadly useful, and numerical optimizers can capitalize on agent‑generated search trajectories.

By Ming Chen, Rong-Xi Tan, Ke Xue, Yu-Jie Zhou, Taiye Lu, Zhi-Xuan Gao, Peng Xie, Zijun Shen, Chen Lu, Haopu Shang, Chao Qian
arXiv Machine Learning
Jun 26

Target-Aware Bandit Allocation for Scalable Surrogate Optimization in Chemical Space

arXiv:2606. 26657v1 Announce Type: new Abstract: Identifying high-utility candidates from massive discrete spaces under expensive evaluations is a recurring challenge across the sciences, with structure-based drug discovery as a prominent example.

By Mohammad Haddadnia, Yuvan Chali, Abhilash Jayaraj, Constance Kraay, Joana Reis, Felix Strieth-Kalthoff, Haribabu Arthanari
arXiv Machine Learning
Jul 7

On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimization

arXiv:2607. 02834v1 Announce Type: new Abstract: Molecular optimization often starts from a pretrained generative model that captures a broad prior over valid molecular structures.

By Trevor Chen, Ariel Dai, Jason Yang, Riccardo De Santi, Daniel Khalil, Wenda Chu, Nate Gruver, Pranav Murugan, Alexander F. G. Goldberg, Maruan Al-Shedivat, Yisong Yue