Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,305 stories · RSS feed

arXiv Machine Learning
Jun 3

Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent

arXiv:2606. 02596v1 Announce Type: new Abstract: The curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for transformer attention, $< 1$ for MLP up-projections).

By Anherutowa Calvo
arXiv Machine Learning
Jun 3

RESCAST-100K: A Comprehensive Dataset for Cross-Domain Residential Load and Indoor Temperature Forecasting

arXiv:2606. 02852v1 Announce Type: new Abstract: Accurate short-term forecasting of residential energy load and indoor temperature is essential for home energy management systems, grid-level demand response, and community energy efficiency efforts.

By Jainam Dhruva, Yousaf Raza, A. B. Siddique, Simone Silvestri
arXiv Machine Learning
Jun 3

TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer

arXiv:2506. 00431v2 Announce Type: replace Abstract: Due to the proficiency of self-attention mechanisms (SAMs) in capturing dependencies in sequence modeling, several existing dynamic graph neural networks (DGNNs) utilize Transformer architectures with various encoding designs to capture sequential evolutions of dynamic graphs.

By Jie Peng, Zhewei Wei, Yuhang Ye
arXiv AI
Jun 3

ASAP: Exploiting the Satisficing Generalization Edge in Neural Combinatorial Optimization

arXiv:2501. 17377v4 Announce Type: replace-cross Abstract: Deep Reinforcement Learning (DRL) has emerged as a promising approach for solving Combinatorial Optimization (CO) problems, such as the 3D Bin Packing Problem (3D-BPP), Traveling Salesman Problem (TSP), or Vehicle Routing Problem (VRP), but these neural solvers often exhibit brittleness when facing distribution shifts.

By Han Fang, Paul Weng, Yutong Ban