Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,305 stories · RSS feed

arXiv Machine Learning
Jun 3

TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer

arXiv:2506. 00431v2 Announce Type: replace Abstract: Due to the proficiency of self-attention mechanisms (SAMs) in capturing dependencies in sequence modeling, several existing dynamic graph neural networks (DGNNs) utilize Transformer architectures with various encoding designs to capture sequential evolutions of dynamic graphs.

By Jie Peng, Zhewei Wei, Yuhang Ye
arXiv Machine Learning
Jun 3

Learning Unmasking Policies for Diffusion Language Models

arXiv:2512. 09106v4 Announce Type: replace Abstract: Diffusion (Large) Language Models (dLLMs) now match the downstream performance of their autoregressive counterparts on many tasks, while holding the promise of being more efficient during inference.

By Metod Jazbec, Theo X. Olausson, Louis B\'ethune, Pierre Ablin, Michael Kirchhof, Jo\~ao Monteiro, Victor Turrisi, Jason Ramapuram, Marco Cuturi
arXiv AI
Jun 3

PyraMathBench: Evaluating and Improving Mathematical Capability in Large Language Models

arXiv:2606. 03858v1 Announce Type: new Abstract: Despite the pivotal role of numerical reasoning as the cornerstone of mathematical capabilities in large language models (LLMs) across applications, few benchmarks evaluate LLMs by integrating numerical processing and mathematical reasoning, hindering the interpretability of failures in math tasks.

By Zetian Ouyang, Linlin Wang, Gerard de Melo, Liang He
arXiv Machine Learning
Jun 3

Critical evaluation of PINN for FWD inverse analysis and differentiable FEM as an alternative

arXiv:2606. 03210v1 Announce Type: cross Abstract: Automatic-differentiation-based inverse analysis methods, including physics-informed neural networks (PINNs) and differentiable programming, have recently shown great promise due to their ability to compute accurate gradients and convergence efficiency.

By Yongjin Choi, Hyeonbin Moon, Seunghwa Ryu