Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

15,813 stories · RSS feed

arXiv AI
Jul 28

Do LLMs Know Their Vulnerable Scenarios?

arXiv:2607. 23496v1 Announce Type: new Abstract: Safety-aligned large language models are trained to refuse harmful requests, yet embedding the same requests in particular scenarios can bypass their safeguards.

By Ziheng Peng, Huiqi Deng, Haoran Jing, Xuankun Rong, Jiahui Han, Xiting Wang, Na Zou, Xia Hu
arXiv AI
Jul 28

An Ontology for Machine Learning Interatomic Potentials

arXiv:2607. 23219v1 Announce Type: new Abstract: Machine learning interatomic potentials (MLIPs) approximate quantum-mechanical energies and forces---conventionally computed by density functional theory (DFT) or wave-function methods---at a fraction of the cost.

By Daniel Hern\'andez, Jong Hyun Jung, Yuji Ikeda, Yongliang Ou, Pranav Kumar, Tom Sch\"achtel, Wenchuan Liu, Xin Li, Xi Zhang, Xiang Xu, Lifang Zhu, Fritz K\"ormann, Steffen Staab, Blazej Grabowski
arXiv AI
Jul 28

Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models

arXiv:2607. 22663v1 Announce Type: new Abstract: Block diffusion has emerged as the dominant paradigm for scaling discrete diffusion language models (dLLMs), because decoding text in fixed-size blocks preserves parallel generation within each block while keeping the quadratic attention cost tractable.

By Xingyu Mou, Zijin Huang, Tianze Zhang, Yuxin Ma, Lanning Wei, Zengfeng Huang, Da Zheng, Lun Du