Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

18,477 stories · RSS feed

arXiv Machine Learning

Jul 1

Medical Image Spatial Grounding with Semantic Sampling

arXiv:2603. 14579v3 Announce Type: replace-cross Abstract: Vision language models (VLMs) have shown significant promise in visual grounding for images as well as videos.

By Andrew Seohwan Yu, Mohsen Hariri, Kunio Nakamura, Mingrui Yang, Xiaojuan Li, Vipin Chaudhary

llms computer-vision multimodal benchmarks

arXiv Machine Learning

Jul 1

Sequential RC-TGAN: Generating Relational Time Series with Spectral Envelope Loss

arXiv:2606. 31904v1 Announce Type: new Abstract: The generation of synthetic relational databases often involves modeling complex temporal dynamics, such as transaction logs or event sequences.

By Mohamed Gueye, Yazid Attabi, Manuel Morales, Maxime Dumas

benchmarks

arXiv Machine Learning

Jul 1

Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream Networks

arXiv:2606. 31700v1 Announce Type: new Abstract: Biological neural circuits obey Dale's principle: each neuron's synapses are uniformly excitatory or inhibitory.

By Yutaro Yamada, Luca Grillotti, Rujikorn Charakorn, Sebastian Risi, David Ha, Robert Tjarko Lange

diffusion reinforcement-learning benchmarks safety

arXiv AI

Jul 1

Benchmarking Large Language Models on Floating-Point Error Classification

arXiv:2606. 31308v1 Announce Type: new Abstract: This paper investigates the capability of Large Language Models (LLMs) to detect and classify floating-point errors statically in software code.

By Lisa Taldir (LI-PaRAD), Muhammad Ahmad Saeed (LI-PaRAD), David Defour (LI-PaRAD), Pablo de Oliveira Castro (LI-PaRAD), Eric Petit

llms benchmarks

arXiv Machine Learning

Jul 1

Interface-Aware Neural Newton Preconditioning for Robust Cohesive Zone Model Simulations

arXiv:2606. 31921v1 Announce Type: new Abstract: Cohesive Zone Models (CZMs) are widely used to simulate interface fracture, delamination, adhesive failure, and fiber--matrix debonding in aerospace composite structures.

By Zhangyong Liang, Huanhuan Gao

benchmarks

arXiv AI

Jul 1

The HydroGym Reinforcement Learning Platform for Fluid Dynamics

arXiv:2512. 17534v2 Announce Type: replace-cross Abstract: Modeling and controlling fluids is critical across science and engineering.

By Christian Lagemann, Sajeda Mokbel, Miro Gondrum, Mario R\"uttgers, Yuning Wang, Pol Su\'arez, Ludger Paehler, Deniz A. Bezgin, Aaron B. Buhendwa, Jared L. Callaham, Samuel Ahnert, Nicholas Zolman, Xiao Shao, Jean-Christophe Loiseau, Nikolaus Adams, Matthias Meinke, Wolfgang Schr\"oder, Kai Lagemann, Esther Lagemann, Ricardo Vinuesa, Steven L. Brunton

agents reinforcement-learning robotics benchmarks

arXiv AI

Jul 1

Measuring & Mitigating Over-Alignment for LLMs in Multilingual Criminal Law Courts

arXiv:2606. 23375v2 Announce Type: replace-cross Abstract: While the wider applicability of LLMs in the legal field is currently debated due to their reliability and the gravity of any errors, narrow uses with well-understood and mitigated risks have emerged.

By Arthur Wuhrmann, Gaetan Stein, Daniel Brunner, Andrei Kucharavy

llms nlp benchmarks safety

arXiv AI

Jul 1

LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields

arXiv:2606. 31941v1 Announce Type: cross Abstract: Unstructured navigational features, such as irregular planting or discontinuities, remain the primary failure mode for under-canopy agricultural robots.

By Felipe Tommaselli, Francisco Affonso, Arthur Pompeu, Gianluca Capezzuto, Arun Narenthiran Sivakumar, Girish Chowdhary, Marcelo Becker

diffusion reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jul 1

MVP-Nav: Multi-layer Value Map Planner Navigator

arXiv:2606. 31919v1 Announce Type: cross Abstract: Zero-shot Object Goal Navigation (ZSON) with RGB-only perception poses a fundamental challenge for embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment.

By Wenyuan Xie, Shaokai Wu, Yijin Zhou, Yanbiao Ji, Guodong Zhang, Bayram Bayramli, Qiuchang Li, Xunchu Zhou, Yue Ding, Hongtao Lu

agents robotics benchmarks

arXiv AI

Jul 1

HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation

arXiv:2606. 30966v1 Announce Type: new Abstract: Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives.

By Arshia Rafieioskouei, Tzu-Han Hsu, Matthew Lucas, Borzoo Bonakdarpour

agents reinforcement-learning benchmarks

arXiv AI

Jul 1

Real-Time Source-Free Object Detection

arXiv:2606. 31834v1 Announce Type: cross Abstract: Real-world detectors for autonomous driving, surveillance, and robotics must handle domain-shifts under strict latency and memory constraints, yet existing source-free object detection (SFOD) methods rely on heavyweight architectures that prioritize accuracy alone.

By Sairam VCR, Varun Gopal, Poornima Jain, Vineeth N Balasubramanian, Muhammad Haris Khan

agents computer-vision robotics efficiency benchmarks

arXiv AI

Jul 1

A swap-adversarial framework for improving domain generalization in electrocorticography-based Parkinson's disease classification

arXiv:2602. 10528v2 Announce Type: replace-cross Abstract: We propose a novel swap-adversarial framework that mitigates high inter-subject variability and the high-dimensional low-sample-size problem in electrocorticography (ECoG) data.

By Seongwon Jin, Hanseul Choi, Sunggu Yang, Sungho Park, Jibum Kim

benchmarks safety

arXiv AI

Jul 1

Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models

arXiv:2606. 31397v1 Announce Type: cross Abstract: State-based fine-tuning has emerged as a compelling alternative to weight-based adaptation for transformers, updating lightweight controls into states rather than model weights, offering substantial memory savings while retaining parameter efficiency.

By Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham, Toan Tran

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 1

Sequential sparse Gaussian process quantile regression

arXiv:2606. 31284v1 Announce Type: new Abstract: Quantile regression aims to estimate the conditional quantiles of a response variable from observed data.

By Hugo Nicolas (PLATON, CMAP), Olivier Le Ma\^itre (PLATON, CMAP)

benchmarks

arXiv AI

Jul 1

DSIP: A Dynamic Coordination Planner for Signal-Free Intersections using Diffusion-Model-Based Multi-Agent Motion Planning

arXiv:2606. 30694v1 Announce Type: cross Abstract: Traffic signal control at urban intersections inherently introduces stop-and-go behavior, resulting in increased delays and reduced traffic efficiency, especially under high traffic demand.

By Qian Hu, Haoyang Peng, Songan Zhang, Ming Yang, Hongtei Eric Tseng

agents diffusion benchmarks

arXiv AI

Jul 1

Beyond Binary Instrument QA: Probing Instrument Grounding in Music Audio-Language Models

arXiv:2606. 31338v1 Announce Type: cross Abstract: Recent music audio-language models achieve high accuracy on instrument question-answering benchmarks, but it remains unclear whether this reflects robust audio grounding or benchmark-specific shortcuts.

By Yujun Lee, Joonhyeok Shin, Hyoeun Kim, Kyuhong Shim

llms benchmarks safety

arXiv AI

Jul 1

ECHO: Prune to act, trace to learn with selective turn memory in agentic RL

arXiv:2606. 31650v1 Announce Type: cross Abstract: Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows.

By Zijun Xie, Binbin Zheng, Enlei Gong, Jihua Liu, Yuyang You, Lingfeng Liu, Jiayao Tang, Guanqun Zhao, Aoqi Hu, Zeyu Chen

agents reinforcement-learning benchmarks

arXiv AI

Jul 1

GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

arXiv:2511. 00810v4 Announce Type: replace-cross Abstract: Graphical user interface (GUI) grounding is a key capability for computer-use agents, mapping natural-language instructions to actionable regions on the screen.

By Shijie Zhou, Viet Dac Lai, Hao Tan, Jihyung Kil, Wanrong Zhu, Changyou Chen, Ruiyi Zhang

llms agents fine-tuning multimodal benchmarks

arXiv AI

Jul 1

IPO Finance Agent: Benchmark of LLM Financial Analysts Beyond Finance Agent v2, with Automated Rubric Generation, on the SpaceX (SPCX) IPO

arXiv:2606. 23032v3 Announce Type: replace Abstract: Finance Agent v2 (by Vals AI) has emerged as the reference benchmark for evaluating both Anthropic Claude and OpenAI ChatGPT frontier language models on financial tasks.

By Mostapha Benhenda

llms agents benchmarks safety

arXiv AI

Jul 1

MIRTH: Mutual-Information Reasoning with Temporal Hubs for Vision-Language-Action Agents

arXiv:2606. 31167v1 Announce Type: cross Abstract: VLA models have emerged as a powerful paradigm for transferring semantic knowledge from web-scale data to physical robotic control.

By Hao Sun, Yu Song, Shiyu Teng, Ziwei Niu, Yen-Wei Chen

rag agents robotics efficiency multimodal benchmarks

Get the daily briefing