Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

23,288 stories · RSS feed

arXiv Machine Learning

Jun 4

PointAction: 3D Points as Universal Action Representations for Robot Control

arXiv:2606. 03943v1 Announce Type: cross Abstract: Video-Action Models (VAMs) leverage the broad visual dynamics captured by pre-trained video diffusion models, offering a promising path toward generalizable robot manipulation.

By Mutian Tong, Han Jiang, Qiao Feng, Lingjie Liu, Jiatao Gu

diffusion robotics benchmarks

arXiv Machine Learning

Jun 4

Graph Set Transformer

arXiv:2606. 05116v1 Announce Type: new Abstract: We introduce the Graph Set Transformer (GST), a neural network architecture for learning on sets of graphs, designed for tasks in which per-element predictions depend on set-wide context as well as local structure.

By Jose E. Escrig Molina, Baoquan Chen, Daniel Probst

llms rag computer-vision benchmarks

arXiv AI

Jun 4

StepPRM-RTL: Stepwise Process-Reward Guided LLM Fine-Tuning for Enhanced RTL Synthesis

arXiv:2606. 04246v1 Announce Type: new Abstract: Automatic generation of RTL code for digital hardware designs remains challenging due to long-horizon reasoning, multi-step dependencies, and strict correctness constraints in Verilog and VHDL.

By Prashanth Vijayaraghavan, Apoorva Nitsure, Luyao Shi, Ehsan Degan, Vandana Mukherjee

llms rag reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 4

MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A

arXiv:2606. 04231v1 Announce Type: cross Abstract: Recent advances in multimodal retrieval-augmented generation (MM-RAG) have shifted toward minimal parsing, relying on page-level images for producing retriever embeddings and for answer generation.

By Hanoz Bhathena, Parin Rajesh Jhaveri, Rohan Mittal, Prateek Singh, Aymen Kallala, Rachneet Kaur, Yiqiao Jin, Zhen Zeng, Adwait Ratnaparkhi, Denis Kochedykov

llms rag fine-tuning multimodal benchmarks safety

arXiv AI

Jun 4

You Only Train Once: Differentiable Subset Selection for Omics Data

arXiv:2512. 17678v2 Announce Type: replace-cross Abstract: Selecting compact and informative gene subsets from single-cell transcriptomic data is essential for biomarker discovery, improving interpretability, and cost-effective profiling.

By Daphn\'e Chopard, Jorge da Silva Gon\c{c}alves, Irene Cannistraci, Thomas M. Sutter, Julia E. Vogt

efficiency benchmarks safety

arXiv AI

Jun 4

SAM 3D: 3Dfy Anything in Images

arXiv:2511. 16624v2 Announce Type: replace-cross Abstract: We present SAM 3D, a generative model for visually grounded 3D object reconstruction, predicting geometry, texture, and layout from a single image.

By SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, Jiawei Liu, Ziqi Ma, Anushka Sagar, Bowen Song, Xiaodong Wang, Jianing Yang, Bowen Zhang, Piotr Doll\'ar, Georgia Gkioxari, Matt Feiszli, Jitendra Malik

benchmarks safety

arXiv AI

Jun 4

LCSHBench: A Multilingual, Consensus-Grounded Benchmark for Library of Congress Subject Heading Assignment

arXiv:2606. 04382v1 Announce Type: cross Abstract: Automated subject cataloging assigns controlledvocabulary headings to bibliographic records, but LCSH has no standard public benchmark.

By Kwok Leong Tang

fine-tuning benchmarks

arXiv Machine Learning

Jun 4

PE-MHL: Physics-Encoded Modular Hybrid Layers for Scalable Learning of Complex Systems

arXiv:2606. 04290v1 Announce Type: new Abstract: Hybrid models that combine physics-based and data-driven components have shown strong potential for achieving accuracy and interpretability in control applications.

By Ismail Hassaballa, Mircea Lazar

benchmarks safety

arXiv AI

Jun 4

AdaKoop: Efficient Modeling of Nonlinear Dynamics from Nonstationary Data Streams with Koopman Operator Regression

arXiv:2606. 04930v1 Announce Type: cross Abstract: Real-time data analysis requires the ability to accurately and adaptively address nonlinear dynamics in a nonstationary data stream while preserving computational efficiency.

By Naoki Chihara, Ren Fujiwara, Yasuko Matsubara, Yasushi Sakurai

benchmarks

arXiv AI

Jun 4

Selective Coupling of Decoupled Informative Regions: Masked Attention Alignment for Data-Free Quantization of Vision Transformers

arXiv:2606. 04373v1 Announce Type: cross Abstract: Data-Free Quantization (DFQ) addresses data security concerns by synthesizing samples, without accessing real data.

By Biao Qian, Yang Wang, Yong Wu, Jungong Han

llms efficiency benchmarks safety

arXiv Machine Learning

Jun 4

Enhancing the MADDPG Algorithm for Multi-Agent Learning via Action Inference and Importance Sampling

arXiv:2606. 05021v1 Announce Type: new Abstract: We investigate multi-agent deep reinforcement learning and propose two enhancements to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm.

By Marc Walden, Jason Liu, Shaashwath Sivakumar, Ryan Liu, Hamza Khan

agents reinforcement-learning benchmarks

arXiv AI

Jun 4

Metric-Aware Hybrid Forecasting for the CTF4Science Lorenz Challenge

arXiv:2606. 04191v1 Announce Type: cross Abstract: We describe our approach to the CTF4Science Lorenz challenge, a benchmark that mixes short-horizon forecasting, long-time distribution matching, and trajectory reconstruction across nine task pairs.

By Cen Lu

benchmarks

arXiv AI

Jun 4

Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models

arXiv:2602. 19101v2 Announce Type: replace-cross Abstract: Value alignment of Large Language Models (LLMs) requires us to empirically measure these models' actual, acquired representation of value.

By Seong Hah Cho, Junyi Li, Anna Leshinskaya

llms rag benchmarks safety

arXiv Machine Learning

Jun 4

Beyond Symmetric Alignment: Spectral Diagnostics of Modality Imbalance in Vision-Language Models in the Medical Domain

arXiv:2606. 04613v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) struggle when applied to medical image-text data, yet the tools available to diagnose this failure remain limited.

By Alessandro Gambetti, Qiwei Han, Cl\'audia Soares, Hong Shen

llms multimodal benchmarks safety

arXiv AI

Jun 4

SFMambaNet: Spectral-Frequency Enhanced Selective State Space Model for Correspondence Pruning

arXiv:2606. 04493v1 Announce Type: cross Abstract: Correspondence pruning aims to identify inliers from an initial set of correspondences.

By Zhihua Wang, Yanping Li, Yizhang Liu

efficiency benchmarks

arXiv AI

Jun 4

P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization

arXiv:2606. 03376v2 Announce Type: replace-cross Abstract: Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs).

By Ruipeng Zhang, Zhihao Li, Haozhang Yuan, C. L. Philip Chen, Tong Zhang

llms reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 4

MAEPose: Self-Supervised Spatiotemporal Learning for Human Pose Estimation on mmWave Video

arXiv:2605. 00242v2 Announce Type: replace-cross Abstract: Millimetre-wave (mmWave) radar offers a more privacy-preserving alternative to RGB-based human pose estimation.

By Xijia Wei, Yuan Fang, Kevin Chetty, Youngjun Cho, Nadia Bianchi-Berthouze

computer-vision benchmarks safety

arXiv Machine Learning

Jun 4

LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models

arXiv:2606. 04485v1 Announce Type: new Abstract: Tabular foundation models (TFMs) increasingly rival tree ensembles, but their performance is often compute-inefficient: with standard affine scalar tokenization, each feature injects value variation through an essentially one-dimensional channel, and feature IDs/positional signals cannot increase within-feature value degrees of freedom, yielding weak early-layer value sensitivity and redundant hidden states.

By Yuanrui Wang, Xingxuan Zhang, Han Yu, Mingchao Ming, Gang Ren, Hao Yuan, Li Mao, Yunjia Zhang, Chun Yuan, Peng Cui

nlp benchmarks

arXiv AI

Jun 4

Position: State-of-the-Art Claims Require State-of-the-Art Evidence

arXiv:2605. 17273v3 Announce Type: replace-cross Abstract: State-of-the-Art (SOTA) claims pervade Artificial Intelligence (AI) and Machine Learning (ML) research.

By YongKyung Oh

benchmarks

arXiv AI

Jun 4

Can Reasoning Path still be Effective as Input? Bridging Post-Reasoning to Chain-of-Thought Compression

arXiv:2510. 08647v2 Announce Type: replace-cross Abstract: Recent developments have enabled advanced reasoning in Large Language Models (LLMs) via long Chain-of-Thought (CoT), trading efficiency during inference for performance.

By Chengzhengxu Li, Xiaoming Liu, Zhaohan Zhang, Shengchao Liu, Guoxin Ma, Yu Lan, Cong Wang, Chao Shen

llms benchmarks

Get the daily briefing