Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv AI

Sep 30

Spatial Memory Agent: Experience-Grounded Procedural Memory for Spatial Intelligence

arXiv:2608.12743v3 Announce Type: replace Abstract: Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ab...

By Haokai Zhang, Yuhang Ding, Yunshu Zhou, Xinze Du, Shengtao Zhang, Zhiyue Zhao, Yuling Xi, Hao Chen

agents reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Sep 30

Compositional Safety Failures in Harness Evolution: Identification and Runtime Monitoring

arXiv:2609.33123v2 Announce Type: replace Abstract: Self-evolving agent harnesses continually update persistent components such as memory, prompts, skills, and tools. We call this process harness evo...

By Zhixiang Zhang, Zesen Liu, Wai Ip Lai, Hongxu chen, Dongdong She

agents benchmarks safety

arXiv AI

Sep 30

Privacy-Preserving Full-Body Meshing from mmWave Radar via Mesh Foundation Model Supervision

arXiv:2609.34768v2 Announce Type: replace Abstract: Millimeter-wave (mmWave) radar enables privacy-preserving human perception, but the extreme sparsity of point clouds from commercial single-chip se...

By Shuxing Zhang, Yongquan Ni, Zhenyu Ding, Yawen Lin

llms efficiency multimodal benchmarks safety

arXiv AI

Sep 30

Training-Free Global Geometric Association for 4D LiDAR Panoptic Segmentation

arXiv:2512.18991v3 Announce Type: replace-cross Abstract: Dominant paradigms for 4D LiDAR panoptic segmentation are usually required to train deep neural networks with large superimposed point clouds...

By Gyeongrok Oh, Youngdong Jang, Jonghyun Choi, Suk-Ju Kang, Guang Lin, Sangpil Kim

computer-vision benchmarks

arXiv AI

Sep 30

Meta-SecAlign: Training LLMs against Prompt Injection for Robust Agents

arXiv:2507.02735v4 Announce Type: replace-cross Abstract: Prompt injection attacks, where untrusted data contains an injected prompt to manipulate the system, have been listed as the top security thr...

By Sizhe Chen, Arman Zharmagambetov, David Wagner, Chuan Guo

llms agents fine-tuning benchmarks

arXiv AI

Sep 30

FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery

arXiv:2605.14854v3 Announce Type: replace-cross Abstract: Human Mesh Recovery (HMR) is fundamentally ambiguous: under occlusion or weak depth cues, multiple 3D bodies can explain the same image evide...

By Patrick Kwon, Chen Chen

benchmarks data-engineering

arXiv AI

Sep 30

SymDrift: One-Shot Generative Modeling under Symmetries

arXiv:2605.06140v3 Announce Type: replace-cross Abstract: Generative modeling of physical systems, such as molecules, requires learning distributions that are invariant under global symmetries, such...

By Samir Darouich, Vinh Tong, Llu\'is Pastor-P\'erez, Tanja Bien, Loay Mualem, Mathias Niepert

rag diffusion efficiency benchmarks safety

arXiv AI

Sep 30

Scalable Heterogeneous Graph Foundation Models for Data-Driven Optimal Power Flow in Smart Grids

arXiv:2605.23194v2 Announce Type: replace-cross Abstract: Fast and reliable optimal power flow (OPF) approximation is important for power system operation, yet heterogeneous OPF graph models are ofte...

By Massimiliano Lupo Pasini, Yijiang Li, Kibaek Kim, Teja Kuruganti

llms fine-tuning benchmarks

arXiv AI

Sep 30

OneLatent: Latent Reasoning for Efficient Foundation Recommendation Models

arXiv:2607.26621v3 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated strong reasoning capabilities, motivating their use as the backbone of foundation recommendati...

By Hao Jiang, Peiru Du, Pengfei Yao, Mengting Li, Siyuan Lou, Kuo Cai, Sheng Yu, Qiang Luo, Jian Liang, Ruiming Tang, Fei Pan, Peng Jiang, Wenwu Ou

llms efficiency benchmarks safety

arXiv AI

Sep 30

Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study

arXiv:2608.00042v2 Announce Type: replace-cross Abstract: Domain adaptation of small language models (SLMs) has emerged as a practical strategy for deploying capable NLP systems in resource-constrain...

By Ramesh B. Paramkusham

llms nlp reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Sep 30

Temporal-Attention Head Specialization During Video Diffusion Training

arXiv:2609.31654v2 Announce Type: replace-cross Abstract: Video diffusion transformers depend on temporal attention to coordinate information across frames, yet nearly everything known about this mec...

By Taewoo Ha, Shafayat Mowla Anik, Dae Yeol Lee, Byeong Kil Lee, Jeeho Ryoo

llms diffusion benchmarks

arXiv AI

Sep 30

Video-to-Music Generation for Gameplay Videos

arXiv:2609.31810v2 Announce Type: replace-cross Abstract: Video-to-music models have advanced considerably in the last few years, particularly in film and music video applications. In this paper, we...

By Felipe Marra, Lucas N. Ferreira

llms fine-tuning benchmarks

arXiv AI

Sep 30

ASCEND: Personal AI Agents for Autonomous Scientific Computing Across HPC Clusters and GPU Workstations

arXiv:2609.32868v2 Announce Type: replace-cross Abstract: Traditional scientific computing requires researchers to translate computational intent into environment configuration, resource requests, an...

By J. Paul Liu, Uthpala Herath, Andrew Petersen

llms agents benchmarks

arXiv AI

Sep 30

Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation

arXiv:2609.34182v2 Announce Type: replace-cross Abstract: Dexterous manipulation requires tactile feedback. However, robot tactile demonstrations are difficult to scale,because dexterous-hand teleope...

By Wenqiao Li, Qianyou Zhao, Jiawen Hao, Xuezhou Zhu, Tengyu Liu, Kaifeng Zhang, Chuan Wen, Siyuan Huang

robotics benchmarks

arXiv AI

Sep 30

Diffusion Reward Models

arXiv:2609.33803v2 Announce Type: replace-cross Abstract: Reward models underpin the alignment of large language models, yet the dominant designs reduce each prompt--response pair to a point estimate...

By Xiangyang Wang, Bingxiang He, Zeyuan Liu, Jiaze Wang, Ziqing Qiao, Yuxin Zuo, Huan-ang Gao, Cheng Qian, Wenbin Zhang, Ran Li, Youbang Sun, Ning Ding, Yuanchun Shi, Zhiyuan Liu, Chaojun Xiao, Chun Yu

llms diffusion reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Sep 30

FluxLite: Inference-Time Proposal Control for Discrete Diffusion Models

arXiv:2609.35947v1 Announce Type: new Abstract: Many inference-time tasks for pretrained discrete diffusion models and diffusion language models reduce to drawing samples from a tilted version of the...

By Yinuo Ren, Haoxuan Chen, Grant M. Rotskoff, Jiequn Han, Lexing Ying

llms diffusion benchmarks

arXiv Machine Learning

Sep 30

Replay the Curvature: Accurate and Scalable NVFP4 Quantization for Large Language Model Inference

arXiv:2609.36654v1 Announce Type: new Abstract: Large language models make weight storage and memory traffic major inference costs, motivating low-precision formats that represent each weight with on...

By Ruiyi Ding, Jie Li, Kang He, Ziyan Liu, Chengru Song, Yuedong Xu, Yuan Cheng

llms efficiency benchmarks

arXiv Machine Learning

Sep 30

CHAIN: Calibrated LLM Forecasting via Causal-Temporal Hypergraph Inference

arXiv:2609.36689v1 Announce Type: new Abstract: Large language models have achieved significant progress in event forecasting, yet their probability outputs exhibit systematic calibration bias that v...

By Wenjin Liu, Chenxi Wang, Yue Lu, Zhe Cui, Haoran Luo

llms benchmarks safety

arXiv Machine Learning

Sep 30

Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamics

arXiv:2609.36608v1 Announce Type: new Abstract: On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-t...

By Zubin Zheng, Jiahao Wu, Shaofeng Zhang, Zhirui Zhang, Yew-Soon Ong, Shengcai Liu

agents efficiency benchmarks

arXiv Machine Learning

Sep 30

Scaling Full Conformal Image Classifiers

arXiv:2609.37298v1 Announce Type: new Abstract: Conformal prediction provides set-valued predictions with distribution-free coverage guarantees, making it attractive for high-stakes image classificat...

By Julio Silva-Rodr\'iguez, Ender Konukoglu

llms computer-vision multimodal benchmarks

Get the daily briefing