PhysicsBench is a unified benchmark and leaderboard that evaluates both generative and predictive AI models for engineering design and simulation under a single standardized procedure. It covers seven generation and prediction tasks across 1D, 2D, and 3D domains, ranking 66 models on nine industrial‑scale CAD/CFD/FEA datasets and public references, expanded into 28 configurations. The evaluation uses realistic, limited data scales and a common metric suite that captures geometric fidelity, physical‑field accuracy, and engineering‑specific validity, with rankings derived via a PageRank‑based dominance graph and a separate efficiency view.
arXiv:2607. 07863v1 Announce Type: new Abstract: In physically dominated machining processes, experimental datasets are small, expensive, and material-specific; in this regime, data curation, evaluation design, and the form of physics integration can matter as much as the learning algorithm.
arXiv:2512. 07847v2 Announce Type: replace Abstract: Benchmarking has been the cornerstone of progress in computer vision, natural language processing, and the broader deep learning domain, driving algorithmic innovation through standardized datasets and reproducible evaluation protocols.
The paper introduces PRISMS, a framework that uses expert pairwise rankings of varying fidelity to curate scientific designs without relying on data-intensive regression models. By escalating queries from lower- to higher-fidelity rankers based on Fisher-information, PRISMS improves discovery recall and reduces the number of screening rounds compared to regression-only and non‑escalated ranking methods. In optimization tasks, PRISMS outperforms Bayesian optimization by achieving higher hypervolume.
arXiv:2605. 29283v2 Announce Type: replace-cross Abstract: Recent physics foundation models claim general spatiotemporal forecasting ability, yet their evaluations often collapse performance into a single average score under a fixed training distribution.
arXiv:2605. 28579v2 Announce Type: replace Abstract: Large language models (LLMs) have recently advanced text-driven 3D generation, yet Text-to-CAD remains far from supporting industrial product design.
arXiv:2512. 19799v2 Announce Type: replace Abstract: Advances in LLM reasoning and tool use have enabled agentic science, yet frontier theoretical and computational physics remains challenging because research requires deep domain expertise, long-horizon reasoning, and reliable numerical computation.
arXiv:2607. 05573v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) enable the automatic generation of parametric 3D designs from natural-language specifications.
arXiv:2509. 13805v4 Announce Type: replace-cross Abstract: Foundation models have revolutionized natural language processing through a ``train once, deploy anywhere'' paradigm, where a single pre-trained model adapts to countless downstream tasks without retraining.
RealCADBench is a new benchmark for evaluating intent‑to‑program parametric CAD modeling, featuring 12,632 tasks drawn from 19 factory‑automation categories and covering text, 2D drawings, product photos, and rendered images for both Part and Assembly modeling. The study reports results on a 1,770‑task evaluation slice, using metrics such as executability, Solid IoU, Surface IoU, and a rubric‑based visual‑semantic identity Judge. Across nine standalone and six frontier‑scale large models, no single model dominates all four metrics, highlighting diverse strengths and failure modes like missing fine structures and incorrect assembly placement.
arXiv:2608. 09296v1 Announce Type: new Abstract: A CAD model is not engineering-grade merely because it looks correct.
arXiv:2608. 13566v1 Announce Type: cross Abstract: Post-training papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.
MatToolBench is a new benchmark that evaluates multimodal GUI agents on professional materials science software. It contains 204 tasks across 10 tools in three modalities—GUI operation, OriginPro scripting, and code-based database queries—executed inside a Windows 11 VM. The benchmark offers fine-grained, expert-decomposed scoring and a high-performing multimodal judge for aesthetic assessment, revealing that strong general benchmark performance does not transfer to scientific workflows.