Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,305 stories · RSS feed

arXiv Machine Learning
Jun 3

Critical evaluation of PINN for FWD inverse analysis and differentiable FEM as an alternative

arXiv:2606. 03210v1 Announce Type: cross Abstract: Automatic-differentiation-based inverse analysis methods, including physics-informed neural networks (PINNs) and differentiable programming, have recently shown great promise due to their ability to compute accurate gradients and convergence efficiency.

By Yongjin Choi, Hyeonbin Moon, Seunghwa Ryu
arXiv AI
Jun 3

PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models

arXiv:2606. 03858v1 Announce Type: new Abstract: Despite the pivotal role of numerical reasoning as the cornerstone of mathematical capabilities in large language models (LLMs) across applications, few benchmarks evaluate LLMs by integrating numerical processing and mathematical reasoning, hindering the interpretability of failures in math tasks.

By Zetian Ouyang, Linlin Wang, Gerard de Melo, Liang He
arXiv AI
Jun 3

ASAP: Exploiting the Satisficing Generalization Edge in Neural Combinatorial Optimization

arXiv:2501. 17377v4 Announce Type: replace-cross Abstract: Deep Reinforcement Learning (DRL) has emerged as a promising approach for solving Combinatorial Optimization (CO) problems, such as the 3D Bin Packing Problem (3D-BPP), Traveling Salesman Problem (TSP), or Vehicle Routing Problem (VRP), but these neural solvers often exhibit brittleness when facing distribution shifts.

By Han Fang, Paul Weng, Yutong Ban
arXiv AI
Jun 3

Causal Neural Probabilistic Circuits

arXiv:2603. 01372v2 Announce Type: replace-cross Abstract: Concept Bottleneck Models (CBMs) enhance the interpretability of end-to-end neural networks by introducing a layer of concepts and predicting the class label from the concept predictions.

By Weixin Chen, Han Zhao