Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,285 stories · RSS feed

arXiv AI

Jul 2

HydraCollab: Adaptive Collaborative-Perception for Distributed Autonomous Systems

arXiv:2607. 00191v1 Announce Type: cross Abstract: Collaborative-perception enables multi-robot systems to enhance situational awareness by sharing perceptual information.

By Luke Chen, Cheng-Ju Wu, David R. Martin, Qilin Ye, Pramod Khargonekar, Mohammad Abdullah Al Faruque

agents robotics benchmarks

arXiv AI

Jul 2

LLM-Guided ODE Discovery and Parameter Inference from Small-Cohort Aggregate Data

arXiv:2607. 00733v1 Announce Type: cross Abstract: Mechanistic modeling via ordinary differential equations (ODEs) provides interpretable descriptions of complex dynamics and enables inference of underlying mechanisms, which is particularly valuable in clinical settings.

By Hanning Yang, Meropi Karakioulaki, Lennart Purucker, Tim Litwin, Cristina Has, Moritz Hess

llms agents benchmarks safety

arXiv AI

Jul 2

Rule-VLN: Bridging Perception and Compliance via Semantic Reasoning and Geometric Rectification

arXiv:2604. 16993v2 Announce Type: replace Abstract: As embodied AI transitions to real-world deployment, the success of the Vision-and-Language Navigation (VLN) task tends to evolve from mere reachability to social compliance.

By Jiawen Wen, Penglei Sun, Wenjie Zhang, Suixuan Qiu, Weisheng Xu, Xiaofei Yang, Xiaowen Chu

agents robotics multimodal benchmarks safety

arXiv AI

Jul 2

FLYNN: Robust Neural Network for Robot Navigation using Fly Brain Topology

arXiv:2607. 00025v1 Announce Type: cross Abstract: While deep learning models achieve state-of-the-art performance in complex tasks, they remain brittle when faced with new environments or sensory deprivation.

By Benquan Wang, Jingdao Chen

agents robotics benchmarks

arXiv Machine Learning

Jul 2

REALM: Reliable Expertise-Aware Language Model Fine-Tuning from Noisy Annotations

arXiv:2604. 17289v2 Announce Type: replace Abstract: Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise.

By Sajjad Ghiasvand, Mark Beliaev, Mahnoosh Alizadeh, Ramtin Pedarsani

llms nlp fine-tuning benchmarks safety

arXiv AI

Jul 2

Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps

arXiv:2607. 00004v1 Announce Type: cross Abstract: While advanced foundation models like ModernBERT significantly outperform older architectures in dense retrieval, they surprisingly lag behind the aging BERT-base baseline in learned sparse retrieval (LSR).

By Zhichao Geng, Yang Yang

fine-tuning efficiency benchmarks

arXiv AI

Jul 2

Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use

arXiv:2607. 01084v1 Announce Type: new Abstract: While Large Language Model (LLM) agents demonstrate proficiency in static benchmarks, their deployment in real-world scenarios is hindered by the dynamic nature of user queries, tool sets, and interaction dynamics.

By Song-Lin Lv, Weiming Wu, Rui Zhu, Zi-Jian Cheng, Lan-Zhe Guo

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 2

KGS-GCN: Kinematics-Driven Gaussian Splatting and Probabilistic Topology for Skeleton-Based Action Recognition

arXiv:2603. 16943v2 Announce Type: replace-cross Abstract: Skeleton-based action recognition is widely applied in sensor-based systems, including human-computer interaction and intelligent surveillance.

By Yuhan Chen, Yicui Shi, Guofa Li, Liping Zhang, Jie Li, Jiaxin Gao, Wenbo Chu

benchmarks

arXiv Machine Learning

Jul 2

Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Captures

arXiv:2605. 04035v3 Announce Type: replace-cross Abstract: We propose HeadsUp, a scalable feed-forward method for reconstructing high-quality 3D Gaussian heads from large-scale multi-camera setups.

By Evangelos Ntavelis, Sean Wu, Mohamad Shahbazi, Fabio Maninchedda, Dmitry Kostiaev, Artem Sevastopolsky, Vittorio Megaro, Trevor Phillips, Alejandro Blumentals, Shridhar Ravikumar, Mehak Gupta, Reinhard Knothe, Jeronimo Bayer, Matthias Vestner, Simon Schaefer, Thomas Etterlin, Christian Zimmermann, Alexey Artemov, Mathias Deschler, Peter Kaufmann, Stefan Brugger, Sebastian Martin, Brian Amberg, Tom Runia

diffusion benchmarks

arXiv AI

Jul 2

Controllable Narrative Rendering for Enhanced Assisted Writing

arXiv:2607. 00009v1 Announce Type: cross Abstract: Despite the remarkable proficiency of large language models (LLMs) in basic writing assistance, their utility in creative writing is fundamentally hindered by a persistent binary failure.

By Mingzhe Lu, Yanbing Liu, Jiayue Wu, Jiarui Zhang, Qihao Wang, Yue Hu, Yunpeng Li, Yangyan Xu

llms benchmarks

arXiv Machine Learning

Jul 2

VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning

arXiv:2511. 17731v2 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) prompting has proven remarkably effective for eliciting complex reasoning in large language models (LLMs).

By Lingxiao Li, Yifan Wang, Xinyan Gao, Chen Tang, Xiangyu Yue, Chenyu You

llms fine-tuning multimodal benchmarks safety

arXiv AI

Jul 2

Toward Cybersecurity-Expert Small Language Models

arXiv:2510. 14113v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are transforming everyday applications, yet deployment in cybersecurity lags due to a lack of high-quality, domain-specific models and training datasets.

By Matan Levi, Daniel Ohayon, Ariel Blobstein, Ravid Sagi, Ian Molloy, Yair Allouche

llms benchmarks

arXiv Machine Learning

Jul 2

FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data

arXiv:2507. 10540v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has created a diverse landscape of models, each excelling at different tasks.

By Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You

llms fine-tuning efficiency benchmarks

arXiv AI

Jul 2

LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution

arXiv:2607. 00700v1 Announce Type: cross Abstract: LLVM is a widely used compiler infrastructure whose scale and complexity make issue resolution labor-intensive and challenging.

By Zhao Tian, Yingquan Zhao, Chenyao Suo, Meng Wang, Junjie Chen

llms agents benchmarks

arXiv AI

Jul 2

Continuous Knowledge Metabolism: Generating Scientific Hypotheses from Evolving Literature

arXiv:2604. 12243v2 Announce Type: replace-cross Abstract: Identifying promising research directions in fast-moving subareas is one of the most cognitively expensive tasks in modern AI research.

By Jinkai Tao, Yubo Wang, Xiaoyu Liu, Menglin Yang

llms agents benchmarks

arXiv AI

Jul 2

OmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scale

arXiv:2602. 05711v2 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) architectures are evolving towards finer granularity to improve parameter efficiency.

By Jingze Shi, Zhangyang Peng, Yizhang Zhu, Yifan Wu, Guang Liu, Yuyu Luo

efficiency benchmarks

arXiv AI

Jul 2

Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzles

arXiv:2511. 06160v2 Announce Type: replace Abstract: While recent safety guardrails effectively suppress overtly biased outputs, subtler forms of social bias emerge during complex logical reasoning tasks that evade current evaluation benchmarks.

By Fatima Jahara, Mark Dredze, Sharon Levy

llms benchmarks safety

arXiv AI

Jul 2

From Technical Metrics to User Perception: A User Study of a Multimodal Human-Robot Interaction System for Object Detection and Grasping

arXiv:2607. 00530v1 Announce Type: cross Abstract: Improvements in the technical performance of human--robot interaction (HRI) systems do not automatically translate into differences that human users can detect during live interaction.

By Jian Song, Tian Zi, Shen Guanting

llms computer-vision robotics multimodal benchmarks

arXiv Machine Learning

Jul 2

Leveraging Multimodality for Real-Time Classification of Transients and Variables found by the Zwicky Transient Facility

arXiv:2607. 00228v1 Announce Type: cross Abstract: Modern time-domain surveys such as the Zwicky Transient Facility (ZTF) generate hundreds of thousands of alerts each night, making real-time decisions for follow-up observations a central challenge in time-domain astronomy.

By Ved G. Shah, Nabeel Rehemtulla, Adam A. Miller, Sushant Sharma Chaudhary, Michael W. Coughlin, Antoine Le Calloch, Matthew J. Graham, Joahan Castaneda Jaimes, Theophile Jegou du Laz, Ashish A. Mahabal, Frank J. Masci, Josiah Purdum, Reed Riddle, Jesper Sollerman, Anastasia Wei, Mansi M. Kasliwal

efficiency multimodal benchmarks

arXiv AI

Jul 2

Competition-Aware CPC Forecasting with Near-Market Coverage

arXiv:2603. 13059v2 Announce Type: replace-cross Abstract: Cost-per-click (CPC) in paid search is an auction-generated outcome shaped by a competitive landscape that is only partially observable from any single advertiser's history.

By Sebastian Frey, Edoardo Beccari, Maximilian Kranz, Nicol\`o Alberto Pellizzari, Ali Mete Karaman, Qiwei Han, Maximilian Kaiser

llms benchmarks safety

Get the daily briefing