Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

16,849 stories · RSS feed

arXiv Machine Learning
Jul 21

Adaptive Mamba Neural Operators

arXiv:2607. 18043v1 Announce Type: new Abstract: Accurately solving partial differential equations (PDEs) on arbitrary geometries and a variety of meshes is an important task in science and engineering applications.

By Zeyuan Song, Zheyu Jiang
arXiv Machine Learning
Jul 21

ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoring

arXiv:2607. 18127v1 Announce Type: cross Abstract: With the rapid growth of cloud computing infrastructures in scale and complexity, network monitoring for Large-scale Cloud Systems (LCSs) has become increasingly challenging, requiring automated and reliable anomaly detection to maintain service availability.

By Thu T. H. Doan, Mohammad Saiful Islam, Andriy Miranskyy, Ngoc-Thanh Nguyen, Rogardt Heldal, Patrizio Pelliccione
arXiv Machine Learning
Jul 21

LFM: Leveraging Foundation Models for Source-Free Universal Domain Adaptation

arXiv:2607. 17653v1 Announce Type: cross Abstract: Source-free universal domain adaptation (SF-UniDA) adapts a pre-trained source model to an unlabeled target domain under both covariate and label shifts, without access to source data.

By Jing Li, Pan Liu, Meng Zhao, Wanli Xue, Yanhong Yang, Xu Cheng, Fan Shi, Jianhua Zhang, Qinghua Hu, Shengyong Chen
arXiv AI
Jul 21

LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learning

arXiv:2607. 17243v1 Announce Type: new Abstract: Multi-view spatial reasoning requires vision-language models to compare visual evidence across images, align object correspondences, and infer spatial relations over long visual contexts, a setting where chain-of-thought reasoning tends to grow verbose without becoming more accurate.

By Xingjian Tao, Yiwei Wang, Yujun Cai, Jing Tang
arXiv AI
Jul 21

Do Language Models Dream of Binding Molecules? Benchmarking LLMs under Spatial Constraints

arXiv:2607. 18144v1 Announce Type: cross Abstract: Structure-based drug design (SBDD) leverages the 3D structure of protein targets, often complemented by other spatial constraints, to generate candidate binding molecules.

By Thomas MacDougall, Maksim Kuznetsov, Roman Schutski, Rim Shayakhmetov, Maxim Malkov, Vladimir Aladinskiy, Alex Aliper, Alex Zhavoronkov
arXiv AI
Jul 21

Probing the Difficulty Perception Mechanism of Large Language Models

arXiv:2510. 05969v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed on complex reasoning tasks, yet little is known about their ability to internally evaluate problem difficulty, which is an essential capability for adaptive reasoning and efficient resource allocation.

By Sunbowen Lee, Qingyu Yin, Chak Tou Leong, Jialiang Zhang, Yicheng Gong, Shiwen Ni, Min Yang, Xiaoyu Shen
arXiv AI
Jul 21

Clarify Before Executing: A Self-Evolving Agent for Resolving Intent Asymmetry in 3D Tool Orchestration

arXiv:2607. 16352v1 Announce Type: cross Abstract: A fundamental intent asymmetry plagues modern 3D asset creation: while state-of-the-art 3D toolchains demand precise, executable parameters, ordinary users typically provide vague, underspecified instructions.

By Xiaoye Zhu, Weixin Li, Junan Huo, Bozhong Wang, Jia Zeng, Yi Yang, Cen Chen, Qi Liu