Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Computer Vision

Oct 8

From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs

arXiv:2610.10066v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in both visual understanding and code generation. However, existing...

By Zijian Chen, Zhengyu Chen, Bohan Liang, Lirong Deng, Yushuo Zheng, Yanwei Jiang, Qi Jia, Kaiwei Zhang, Wenjun Zhang, Guangtao Zhai

llms multimodal benchmarks

arXiv Computer Vision

Oct 8

A Probabilistic Perspective on Wasserstein-Based Evidential Uncertainty for Out-of-Distribution Segmentation

arXiv:2610.10116v1 Announce Type: new Abstract: Semantic segmentation networks operate on a fixed set of classes and therefore fail when out-of-distribution (OOD) objects appear during deployment, a...

By Arnold Brosch, Abdelrahman Eldesokey, Michael Felsberg, Kira Maag

llms agents computer-vision robotics benchmarks safety

arXiv Computer Vision

Oct 8

HeiCo-FOCUS: A Clinically Grounded Dataset for Long-Context Video Understanding

arXiv:2610.10156v1 Announce Type: new Abstract: Recent advances in Vision-Language Models (VLMs) have led to rapid progress in video understanding across a wide range of benchmark tasks. However, exi...

By Leon Mayer, Lucas Luttner, Patrick Godau, Kai Fritzsche, Annika Reinke, Leonie Boland, Jule Brandt, Janne Heinecke, Chloe K. Nobuhara, Niklas Holzwarth, Evangelia Christodoulou, Marcel Knopp, Dominik Michael, Pascale Piermarco, Saliq Neyaz, Korhan Derin \"Ozarslan, Jakob Hennighausen, Carlos Aumente-Maestro, Tim R\"adsch, Dheeraj Baji, Peter Maximilian Full, Finn Aichholz, Justus Veit Erpenbeck, Linus Finn Schott, Bastian Winkelhausen, Claas de Boer, Bianca G\"uttner, Anneli Hummel, Gregor Just, Max Kirchner, Chenyang Li, Rozenn Raffaut, Ariel Rodriguez, Danush Kumar Venkatesh, Kevin Wang, Jinjing Xu, Mona Sheikh Zeinoddin, Salman Khan, Thomas M. Pausch, Stefanie Speidel, Danail Stoyanov, Daniel A. Hashimoto, Fiona R. Kolbinger, Thomas G. Weiser, Lena Maier-Hein

llms nlp multimodal benchmarks

arXiv Computer Vision

Oct 8

BagDINO: Multi-View Baggage Re-Identification with DINOv3

arXiv:2610.10160v1 Announce Type: new Abstract: Mishandled checked baggage remains a recurrent issue in airport operations, and current recovery workflows still largely rely on tag-based tracking, wh...

By Vita Santa Barletta, Danilo Caivano, Rebecca Margiotta, Massimiliano Morga, Davide Pio Posa

fine-tuning benchmarks

arXiv Computer Vision

Oct 8

Argos: Adapt Rich Geometric Priors for Generalizable Online Scene-Change-Detection

arXiv:2610.10181v1 Announce Type: new Abstract: Robots operating in dynamic environments require reliable detection of how their surroundings change over time. Existing learning-based methods largely...

By Ruihan Xu, Jiae Yoon, Kaichen Zhou, Ue-Hwan Kim, Luca Carlone

robotics benchmarks

arXiv Computer Vision

Oct 8

HuLiGen: Human LiDAR Generation from Parametric Body Models

arXiv:2610.10196v1 Announce Type: new Abstract: LiDAR point clouds of humans are extremely expensive to collect and annotate, thus represent a scarce resource that hinders the development of human an...

By Salma Galaaoui, Nermin Samet, David Picard

llms benchmarks

arXiv Computer Vision

Oct 8

LoomSC: Scalable Deep Subspace Clustering with Projector Factorization and Exact Spectral Reduction

arXiv:2610.10266v1 Announce Type: new Abstract: Dense self-expression matrices and full-affinity spectral clustering limit the scalability of subspace clustering. We introduce the Latent Orthogonal O...

By Nairouz Mrabah, Youssef Melki, Mohamed Bouguessa, Riadh Ksantini, Shakeeb Murtaza, Tehseen Zia

benchmarks

arXiv Computer Vision

Oct 8

TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning

arXiv:2610.10288v1 Announce Type: new Abstract: Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the...

By Dayou Li, Hao Wang, Qianqian Yang, Zihao Zhu, Haoquan Fang, Ziyao Zeng, Yan Han, Zihan Wang, Yan Wang, Baoru Huang, Dilin Wang, Kenji Shimada, Yiyue Luo, Manling Li, Teresa Lv, Mustafa Mukadam, Rakesh Ranjan, Ruohan Zhang, Qi He, Changliu Liu, Xu Chen, Marco Pavone, Bangya Liu, Jiachen Li, Masayoshi Tomizuka, Zhiwen Fan

robotics benchmarks

arXiv Computer Vision

Oct 8

Performance at What Cost? A Sustainability-Aware Performance Index for Cell and Nucleus Instance Segmentation

arXiv:2610.10324v1 Announce Type: new Abstract: Pretrained models for cell and nuclear instance segmentation differ substantially in architecture, pretraining data and objectives, parameter count, in...

By Eiram Mahera Sheikh, Alaa Tharwat, Wolfram Schenck

computer-vision fine-tuning benchmarks

arXiv Computer Vision

Oct 8

MOTIP2: Spatial Priors for End-to-End Multi-Object Tracking

arXiv:2610.10391v1 Announce Type: new Abstract: End-to-end multi-object trackers have narrowed the gap with classical tracking-by-detection on association-difficult benchmarks. Yet they still make sp...

By Beno\^it Roussel, Damien Bouet, Liming Chen, Pierre Perrault

llms benchmarks safety

arXiv Computer Vision

Oct 8

Gaussian Density Splatting Network

arXiv:2610.10396v1 Announce Type: new Abstract: This paper proposes a novel crowd counting approach, the Gaussian Density Splatting Network (GDSNet). Unlike methods that rely on conventional, grid-ba...

By Miao Shang, Yabin Wang, Xiaopeng Hong

benchmarks

arXiv Computer Vision

Oct 8

Label-free cell counting and viability prediction with brightfield imaging and deep learning

arXiv:2610.10473v1 Announce Type: new Abstract: Cell viability assessment is a core requirement in cell culture systems, with critical applications in biopharmaceutical manufacturing and drug develop...

By Amir Reza Vazifeh, Christian Zeigler, Sornanathan Meyyappan, Richard Jeske, Jason W. Fleischer

agents benchmarks

arXiv Computer Vision

Oct 8

MORCA: Offline-to-Online Reinforcement Learning for Adaptive Cache Reuse in Video Diffusion Acceleration

arXiv:2610.10457v1 Announce Type: new Abstract: Diffusion Transformers (DiTs) achieve remarkable performance in video synthesis, but their iterative denoising process suffers from high inference late...

By Yuxiang Xiong, Ruiyan Wang, Wenqiang Wang, Teng Hu, Songhang Shen, Bohao Feng, Hongqian Deng, Ran Yi

llms diffusion reinforcement-learning efficiency benchmarks

arXiv Computer Vision

Oct 8

Tetris3D: 3D Scene Generation With Objects That Fit Together

arXiv:2610.10539v1 Announce Type: new Abstract: We propose Tetris3D, a generative framework for single-image 3D scene reconstruction that recovers objects which are physically and geometrically coher...

By Jaeyeong Kim, Jinhyuk Jang, Jongmin Lee, Kyehong Park, Seungryong Kim

benchmarks

arXiv Computer Vision

Oct 8

ScribbleEdit: A Benchmark for Scribble-Only Image Editing

arXiv:2610.09382v1 Announce Type: cross Abstract: Scribble-based interaction provides a lightweight and intuitive way for users to specify image editing intents in interactive editing tools. However,...

By Jie Ren, Hao Kang, Kai Guo, Yiding Yang, Bo Liu, Liming Jiang, Qing Yan, Zichuan Liu, Yizhi Song, Yue Xing, Hui Liu, Xin Lu

llms multimodal benchmarks safety

arXiv Computer Vision

Oct 8

MultiFly: A Real-World Multimodal Aerial Dataset with Annotation-Efficient Label Transfer and Cross-Modal Semantic Consistency

arXiv:2610.10359v1 Announce Type: cross Abstract: We introduce MultiFly, a real-world, low-altitude UAV dataset for semantic perception across RGB, thermal, LiDAR, and radar modalities. MultiFly prov...

By Markus Gross, Andreas Greiner, Taehyoung Kim, Sivasubiramaniam Subbiah, Toma\v{z} Coti\v{c}, Sai Bharadwaj Matha, Conrad Christoph, Oussema Dhaouadi, Simon Zieher, Surya Vijaya Kumar, Gordon Elger, Henri Mee{\ss}, Olaf Wysocki, Paul Spannaus, Daniel Cremers

computer-vision multimodal benchmarks

arXiv Computer Vision

Oct 8

Batch Augmentation with Unimodal Fine-tuning for Multimodal Fusion of Large Language Models

arXiv:2505.06592v2 Announce Type: replace Abstract: In this paper, we propose batch augmentation with unimodal fine-tuning for multimodal learning. We start with pre-trained unimodal models. We fine-...

By H M Dipu Kabir, Subrota Kumar Mondal, Mohammad Ali Moni

llms agents fine-tuning multimodal benchmarks

arXiv Computer Vision

Oct 8

MAGIC: Learning from Visibility Asymmetry for Unsupervised Stereo Matching

arXiv:2508.10838v3 Announce Type: replace Abstract: Learning disparity in occluded regions remains difficult for unsupervised stereo matching. Photometric supervision lacks valid target-view correspo...

By Peng Xu, Zhiyu Xiang

fine-tuning benchmarks

arXiv Computer Vision

Oct 8

Toward Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion

arXiv:2601.15829v3 Announce Type: replace Abstract: Recent years have witnessed the remarkable success of deep learning in remote sensing image interpretation, driven by the availability of large-sca...

By Yonghao Xu, Pedram Ghamisi, Qihao Weng

diffusion efficiency benchmarks

arXiv Computer Vision

Oct 8

FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation

arXiv:2604.13491v4 Announce Type: replace Abstract: With the rapid progress of Multimodal Large Language Models (MLLMs), unified MLLMs that jointly perform image understanding and generation have adv...

By Yongjin Kim, Yoonjin Oh, Yerin Kim, Hyomin Kim, Jeeyoung Yun, Yujung Heo, Minjun Kim, Sungwoong Kim

llms diffusion reinforcement-learning multimodal benchmarks safety

Get the daily briefing