Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,849 stories · RSS feed

arXiv AI
Jul 21

Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestration

arXiv:2607. 16352v1 Announce Type: cross Abstract: A fundamental intent asymmetry plagues modern 3D asset creation: while state-of-the-art 3D toolchains demand precise, executable parameters, ordinary users typically provide vague, underspecified instructions.

By Xiaoye Zhu, Weixin Li, Junan Huo, Bozhong Wang, Jia Zeng, Yi Yang, Cen Chen, Qi Liu
arXiv AI
Jul 21

Seeing What Is Actually There: PriVE-Bench and PriVE-Tools for Counterfactual Evaluation of Agentic Visual Evidence in VLMs

arXiv:2607. 16311v1 Announce Type: cross Abstract: Vision-language models (VLMs) often answer visual questions using learned language and category priors rather than grounding their predictions in the image itself.

By Jingyu Sun, Jiachen Tu, Yuyang Xue, Yaoxin Jiang, Guoyi Xu, Zhengtao Yao, Rui Qian, Yizheng Sun, Hongpeng Zhou, Jingyuan Sun, Yan Lin
arXiv Machine Learning
Jul 21

Bridging battery design and health assessment through virtual sensing and physics-informed learning

arXiv:2607. 16864v1 Announce Type: new Abstract: Supercharging of lithium-ion batteries (LiBs) requires robust health monitoring to ensure durability, safety, and user confidence, particularly for emerging vehicle-to-grid applications with bidirectional energy flows.

By Wendi Guo, S{\o}ren Byg Vilsen, Daniel Ioan Stroe, Yaqi Li, Yicun Huang, Ashima Verma, Daniel Brandell
arXiv Machine Learning
Jul 21

Leakage-Robust Evaluation and Data-Scale Sensitivity of Attention-Enhanced Multi-Task Learning for Joint Fault Diagnosis and Remaining Useful Life Estimation

arXiv:2607. 16493v1 Announce Type: new Abstract: Multi-task deep learning models that jointly perform fault classification and remaining useful life (RUL) regression are increasingly used in predictive maintenance, yet reported performance can be strongly affected by how sliding-window sequences are split into training and test sets.

By Md Mahamudur Rahaman Shamim, Md. Nuruzzaman, Zannatul Ferdus, Md Rajib Ahmed, Abieer Nwshad Anward, Mohammad Tooneer, Johir Uddin Khan, Khalid Hossen
arXiv AI
Jul 21

Long Range Frequency Tuning for QML

arXiv:2602. 23409v3 Announce Type: replace-cross Abstract: Angle-encoded variational quantum circuits admit a truncated Fourier series representation of their output, but approximating functions with maximum frequency $\omega_{\max}$ using fixed unary encoding requires $\mathcal{O}(\omega_{\max})$ encoding gates.

By Michael Poppel, Markus Baumann, Sebastian W\"olckert, Claudia Linnhoff-Popien, Jonas Stein
arXiv AI
Jul 21

ClawBench: Can AI Agents Complete Everyday Online Tasks?

arXiv:2604. 08523v2 Announce Type: replace-cross Abstract: AI agents may be able to assist with emails and documents, but can they reliably complete everyday online workflows on real websites?

By Yuxuan Zhang, Yubo Wang, Yipeng Zhu, Penghui Du, Junwen Miao, Xuan Lu, Zhuofeng Li, Xingwei Qu, Zhengkang Guo, Yuanzhe Shen, Dingjie Song, Han Zhou, Tuney Zheng, Xian Wu, Hao Yu, Songcheng Cai, Yi Lu, Yunzhuo Hao, Minyi Lei, Liang Chen, Kai Zou, Huifeng Yin, Wendong Xu, Dongfu Jiang, Ping Nie, Jiaheng Liu, Wenhu Chen, Kelsey R. Allen