Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Computation and Language
Oct 8

ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation

arXiv:2610.09163v1 Announce Type: new Abstract: Task-oriented conversational agents remain fragile under real world conversation scenarios as they rarely follow a predictable script, especially when...

By Arkajyoti Chakraborty, Aryan Tayal, Ishika Agarwal, Tanner Sorensen, Justin Chiu, Alessandro Di Bari, Neha Gupta, Andreas Stolcke
arXiv Computation and Language
Oct 8

Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression

arXiv:2610.09788v1 Announce Type: new Abstract: Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can in...

By Mingqing Yuan (Soochow University), Xiaobo Liang (Soochow University), Junwei Yang (University of Cambridge), Ziwei Chen (Chalmers University of Technology), Zeren Zhang (Peking University), Hejin Wang (Tsinghua University), Yubin Wang (The Hong Kong University of Science,Technology), Juntao Li (Soochow University)