The study investigates whether large language models (LLMs) rely solely on their initial task context or adapt based on experimental feedback when selecting fine‑tuning configurations for neural operators. Experiments across partial differential equation families show that LLMs outperform random search and Bayesian optimisation, with early configurations already near the top of a random‑search pool and subsequent proposals changing in response to validation scores. Controlled interventions confirm that the LLM combines a task‑dependent prior with sensitivity to observed outcomes.
By Julian Chan, Javier Mora Jimenez
arXiv:2607. 03426v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit strong reasoning and world-knowledge capabilities, yet often struggle to gather information effectively across the multi-turn interactions required in sequential decision-making settings.
By Jakob Hartmann, James Harvey, Jhonathan Navott, Erik Y. Wang, Luckeciano C. Melo, Flaviu Cipcigan, Cheng Zhang, Alessandro Abate
arXiv:2610.18572v1 Announce Type: new
Abstract: Automated post-training transforms pre-trained large language models (LLMs) into models capable of handling diverse domains by automatically constructi...
By Ruohong Liu, Bohan Tang, Lekang Jiang, Jack Umenberger, Yiwen Guo
The paper introduces Strategy Accumulation and Guided Execution (SAGE), a two-stage framework that makes automated fine-tuning of large language models cumulative. In the first stage, a multi-agent pipeline uses Monte Carlo Tree Search to explore training strategies while a Distillation Agent records task-specific insights and cross-task confidence scores into a structured repository. In the second stage, SAGE retrieves relevant experience from this repository to guide training on new tasks, achieving a 12.4‑percentage‑point improvement over a baseline pipeline without accumulated experience on nine unseen tasks.
By Haoran Zhao, Wei Du, Dingwen Yang, Jixuan Huang, Junlin Shang, Lingyong Fang, Ya Guo, Tao Gui, Qi Zhang, Xuanjing Huang
arXiv:2608. 00641v2 Announce Type: replace Abstract: Bayesian optimization (BO) relies on a surrogate model and an acquisition function, yet the most suitable choices vary across tasks and optimization stages.
By Changquan Zhao, Yuxiang Sun, Ruihao Zhu, Cheng Hua, Yulian He
arXiv:2606. 30335v1 Announce Type: new Abstract: Autonomous scientific discovery systems increasingly use large language models (LLMs) to propose new hypotheses, but many such systems condition primarily on experimental memory: archives of high-scoring candidates or heuristic summaries of recent trials.
By Xuening Wu, Shan Yu, Qianya Xu, Shenqin Yin