Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

15,528 stories · RSS feed

arXiv Machine Learning
Aug 4

From Vessel Trajectories to Safety-Critical Encounter Scenarios: A Generative AI Framework for Autonomous Ship Digital Testing

arXiv:2603. 28067v2 Announce Type: replace Abstract: Digital testing has emerged as a key paradigm for the development and verification of autonomous maritime navigation systems, yet the availability of realistic and diverse safety-critical encounter scenarios remains limited.

By Sijin Sun, Liangbin Zhao, Xiuju Fu
arXiv Machine Learning
Aug 4

GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding

arXiv:2605. 13352v2 Announce Type: replace Abstract: Standard dual-encoder vision-language models that map images and text to deterministic points on a shared unit hypersphere through $\ell_2$ normalization typically expose neither \emph{aleatoric} uncertainty (cross-modal ambiguity) nor \emph{epistemic} uncertainty (lack of training-distribution support).

By Mayank Nautiyal, Li Ju, Andreas Hellander, Ekta Vats, Prashant Singh
arXiv Machine Learning
Aug 4

Large Causal Models for Temporal Causal Discovery

arXiv:2602. 18662v2 Announce Type: replace Abstract: Causal discovery for both cross-sectional and temporal data has traditionally followed a dataset-specific paradigm, where a new model is fitted for each individual dataset.

By Nikolaos Kougioulis, Nikolaos Gkorgkolis, MingXue Wang, Bora Caglayan, Dario Simionato, Andrea Tonon, Ioannis Tsamardinos
arXiv Machine Learning
Aug 4

OpenDebateEvidence: A Massive-Scale Argument Mining and Summarization Dataset

arXiv:2406. 14657v4 Announce Type: replace-cross Abstract: We introduce OpenDebateEvidence, a comprehensive dataset for argument mining and summarization sourced from the American Competitive Debate community.

By Allen Roush, Yusuf Shabazz, Arvind Balaji, Peter Zhang, Stefano Mezza, Markus Zhang, Sanjay Basu, Sriram Vishwanath, Mehdi Fatemi, Ravid Shwartz-Ziv