Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,906 stories · RSS feed

arXiv Machine Learning
Jul 9

Structural Adversarial Attacks on Relational Deep Learning under Integrity Constraints

arXiv:2607. 07089v1 Announce Type: new Abstract: Relational Deep Learning (RDL) has become a standard methodology for machine learning on relational databases: the database is encoded as a heterogeneous temporal graph in which tuples become nodes and primary-key to foreign-key (PK-FK) dependencies become typed edges, over which a graph neural network is trained for downstream prediction.

By Alan Gany, Bogdan Cautis, Silviu Maniu
arXiv AI
Jul 9

DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation

arXiv:2607. 07669v1 Announce Type: cross Abstract: Large language models increasingly \emph{understand} dialectal English, yet still \emph{produce} only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed.

By Jordan Painter, Dipankar Srirag, Adarsh Kappiyath, Diptesh Kanojia, Aditya Joshi, Lu Yin