The paper investigates using Large Language Models (LLMs) as calibration agents for expensive engineering simulators by conditioning them on physics-labeled prompts that describe simulation errors. In tests on a cylinder-wake Large-Eddy Simulation and a damped-oscillator inverse problem, physics-informed LLM proposals outperform conventional derivative-free optimizers in early trials, but the advantage fades after fifteen evaluations. A hybrid workflow that uses the LLM as a warm-start followed by a conventional optimizer achieves target accuracy in only nine simulator runs, outperforming both the LLM alone and standard optimizers.
arXiv:2603. 20253v3 Announce Type: replace-cross Abstract: Evaluating LLM agents for scientific tasks has focused on token costs while ignoring tool-use costs like simulation time and experimental resources.
La Agente ’Optima is an agentic framework that builds and manages Bayesian optimization campaigns for self‑driving laboratories, separating large language model reasoning from campaign execution. It maintains a persistent optimization state, allowing consistent repetitive loops and auditable decisions, and only returns control to the agent when interpretation or revision is needed. In tests on digital discovery tasks and physical platforms, it corrected measurement failures, improved yields, and recommended formulation changes, outperforming human‑directed campaigns in cost and material usage.
arXiv:2607. 18308v1 Announce Type: cross Abstract: Calibration of grey-box simulation models is a constrained optimization problem in which model evaluations are expensive, the parameter space can be high-dimensional, and the search must respect plausibility constraints.
arXiv:2602. 15983v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can translate natural language into optimization code, but silent failures pose a critical risk: code that executes and returns solver-feasible solutions may encode semantically incorrect formulations---a feasibility--correctness gap reaching 90 percentage points on compositional problems.
NextEval introduces an agent-based derivative-free optimization solver that leverages a large language model to guide the sequential selection of evaluation points. The system couples an LLM agent with a numerical environment, allowing the agent to choose tools, configure parameters, and allocate a limited evaluation budget while the environment maintains shared history and state across method switches. A benchmark protocol, NextEval Bench, is also presented to compare different agent configurations on shared DFO tasks under matched budgets and information constraints, and numerical experiments show that NextEval outperforms state-of-the-art baselines in function-evaluation efficiency.
SCICONVBENCH is a benchmark designed to evaluate large language models (LLMs) on multi‑turn clarification tasks in computational science. It focuses on two key abilities: eliciting missing information (disambiguation) and resolving contradictory requests (inconsistency resolution) across four domains—fluid mechanics, solid mechanics, materials science, and partial differential equations. The benchmark pairs a structured task ontology with a rubric‑based evaluation framework, measuring LLM performance in clarification behavior, conversational grounding, and final‑specification fidelity, and reveals that even top models only resolve about 52.7% of disambiguation cases in fluid mechanics while often making ungrounded assumptions.
SAILOR is a proof‑of‑concept system that helps language models translate natural‑language optimization problem descriptions into executable code by detecting missing numerical values. It asks users targeted follow‑up questions, prioritizing them based on uncertainty and solver estimates of impact, and updates the model before returning a solution. In tests on 1,723 benchmark instances, SAILOR achieved exact objective‑value agreement between 27.0% and 87.6% while asking an average of 1.4–5.7 questions per instance.
The paper investigates whether language models can be fine‑tuned to produce outputs that better match intended probability distributions, even when the valid outputs cannot be enumerated in advance. Two calibration fine‑tuning methods—soft‑target and hard‑target—are evaluated across 12 models from four families, showing substantial improvements in structured‑sampling fidelity on unseen distribution families and parameter settings. The gains also transfer to broader stochastic generation tasks, expanding first‑token support and improving NoveltyBench scores, though some downstream capabilities such as arithmetic reasoning may be slightly affected.
arXiv:2607. 18256v1 Announce Type: new Abstract: Optimization modeling is the process of translating real-world decision problems, often described in natural language, into formal mathematical formulations and executable solver code.
The paper introduces AgenticBBO-Bench, a cross‑domain benchmark for evaluating large language model agents on black‑box optimization tasks such as synthetic functions, hyperparameter tuning, database tuning, chip design, and molecular design. Experiments show that agentic BBO outperforms direct LLM‑based methods in all five domains and beats the best numerical optimizers in four. The study also analyzes the impact of optimization tools, task semantics, prior knowledge, and the LLM’s role, finding that numerical tools are not consistently helpful, task semantics are broadly useful, and numerical optimizers can capitalize on agent‑generated search trajectories.
arXiv:2608. 03600v1 Announce Type: new Abstract: Partial differential equations (PDEs) become actionable in science and engineering not as isolated formulae, but as executable workflows that connect modelling assumptions, governing equations, numerical solvers, diagnostics, and decisions.
The paper introduces Test-Time Calibration Learning (TTCL), a label‑free framework that adapts both reasoning accuracy and verbalized confidence of large language models directly on unlabeled target‑task data. TTCL generates self‑supervision signals from multiple model responses, enabling calibration without ground‑truth labels and proving theoretically as a bounded surrogate for the ideal calibration objective. Experiments on mathematical reasoning and factual question answering show consistent improvements, with base models gaining an average 40.13% in accuracy and 70.80% in ECE reduction across eight benchmarks, and further gains under domain shift.