Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,620 stories · RSS feed

arXiv Computer Vision

Sep 10

VANTAGE-Bench: Evaluating the Infrastructure AI Gap in Vision-Language Models

arXiv:2609.09396v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) advance toward physical deployment, the focus has remained on action-oriented Embodied AI evaluated on subject-centric...

By Zaid Pervaiz Bhat, Nimra Nayyar, Arihant Jain, Lap Fung Chan, John Suchanek, Yu Wang, Varun Praveen, Tomasz Kornuta, Vidya Nariyambut Murali

llms nlp robotics multimodal benchmarks safety

arXiv Computer Vision

Sep 10

Learning Global Camera Poses from Noisy View-Graphs for Structure from Motion

arXiv:2609.09491v1 Announce Type: new Abstract: Camera pose estimation is a key step in 3D reconstruction and view-synthesis pipelines. We present a deep, global Structure-from-Motion framework based...

By Fadi Khatib, Meirav Galun, Ronen Basri

computer-vision benchmarks

arXiv Computer Vision

Sep 10

Albedo Estimation via Latent Bridge Matching

arXiv:2609.09884v1 Announce Type: new Abstract: Recent advances in Intrinsic Image Decomposition (IID) have increasingly relied on generative models. However, progress remains limited by three key ch...

By Carme Corbi, David Serrano-Lozano, Javier Vazquez-Corral, Maria Vanrell

benchmarks

arXiv Computer Vision

Sep 10

TransGaze-Object: Transformer Based Driver Gaze Object Prediction Framework in Real Driving

arXiv:2609.10139v1 Announce Type: new Abstract: Driver gaze provides information regarding driver visual attention and situational awareness to the surrounding traffic. Existing driver gaze estimatio...

By Pavan Kumar Sharma, Ayush Pande, Pranamesh Chakraborty

llms benchmarks

arXiv Computer Vision

Sep 10

Spot-the-shift: Evaluating Grounded Image Difference Captioning of Long-term Changes

arXiv:2609.10356v1 Announce Type: new Abstract: Long-term change understanding from images of the same place revisited over time is a challenging task with applications in map maintenance and urban i...

By Benedetta Liberatori, Nermin Samet, Paolo Rota, Matthieu Cord, Elisa Ricci, Andrei Bursuc, Monika Wysocza\'nska

benchmarks

arXiv Computer Vision

Sep 10

ROVR-Open-Dataset: A Large-Scale Depth Dataset for Autonomous Driving

arXiv:2508.13977v4 Announce Type: replace Abstract: Depth estimation is a fundamental component of spatial perception for autonomous driving and other unmanned systems operating in open urban environ...

By Xianda Guo, Ruijun Zhang, Yiqun Duan, Ruilin Wang, Matteo Poggi, Keyuan Zhou, Wenzhao Zheng, Wenke Huang, Gangwei Xu, Yanlun Peng, Yuan Si, Qin Zou

agents computer-vision robotics efficiency benchmarks safety

arXiv Computer Vision

Sep 10

AgroVisNet: A lightweight Convolutional Network and the BD-PlantDX Expert-Validated Benchmark for Radish, Potato and Pointed Gourd Disease Classification

arXiv:2609.10469v1 Announce Type: new Abstract: Automated plant disease diagnosis is increasingly deployed on farmer-held devices in regions where agronomic expertise is scarce and network connectivi...

By Md. Abdullah Mandal, Saad Ahmed, Md. Khalid Syfullah

benchmarks

arXiv Computer Vision

Sep 10

Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation

arXiv:2609.10495v1 Announce Type: new Abstract: In real-time colonoscopy, ground-truth annotations are unavailable at inference, so polyp segmentation models can fail silently. We propose Referee-Bas...

By Siddharth Gupta, Jitin Singla

computer-vision benchmarks

arXiv Computer Vision

Sep 10

Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Images

arXiv:2511.18031v1 Announce Type: cross Abstract: Few-shot object detection (FSOD) aims to detect novel instances with only a limited number of labeled training samples, presenting a challenge that i...

By Yanxing Liu, Jiancheng Pan, Jianwei Yang, Tiancheng Chen, Peiling Zhou, Bingchen Zhang

diffusion computer-vision benchmarks

arXiv Computer Vision

Sep 10

Identifying Habit, Physics, and Nuisance in Robot World Models

arXiv:2609.09210v1 Announce Type: cross Abstract: Teleoperated demonstrations are often multimodal even when the underlying dynamics are nearly deterministic given the executed action. We argue that...

By Jinting Hang, Zhenhui Cai

diffusion robotics multimodal benchmarks

arXiv Computer Vision

Sep 10

Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition

arXiv:2609.10394v1 Announce Type: cross Abstract: Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the...

By Rishabh Jain, Aristeidis Papadopoulos, Zhaofeng Lin, Naomi Harte

multimodal benchmarks

arXiv Computer Vision

Sep 10

Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes

arXiv:2512.14177v4 Announce Type: replace Abstract: Large Vision-Language Models (LVLMs) often produce plausible but unreliable outputs, making robust uncertainty estimation essential. Recent work on...

By Joseph Hoche, Andrei Bursuc, David Brellmann, Gilles Louppe, Pavel Izmailov, Angela Yao, Gianni Franchi

llms rag computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 10

LSP-DETR: Efficient and Scalable Nuclei Segmentation in Whole-Slide Images

arXiv:2601.03163v2 Announce Type: replace Abstract: Background and Objective: Precise and scalable instance segmentation of cell nuclei is a fundamental prerequisite for computational pathology, yet...

By Mat\v{e}j Pek\'ar, V\'it Musil, Rudolf Nenutil, Petr Holub, Tom\'a\v{s} Br\'azdil

llms computer-vision efficiency benchmarks

arXiv Computer Vision

Sep 10

SimpleProc: Fully Procedural Synthetic Data from Simple Rules for Multi-View Stereo

arXiv:2604.04925v3 Announce Type: replace Abstract: Generating procedural synthetic data for multi-view stereo (MVS) usually requires writing complex rules to match the realism of curated datasets. W...

By Zeyu Ma, Alexander Raistrick, Jia Deng

benchmarks

arXiv Computer Vision

Sep 10

Let ViT Speak: Generative Language-Image Pre-training

arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...

By Yan Fang, Mengcheng Lan, Zilong Huang, Weixian Lei, Yunqing Zhao, Yujie Zhong, Yingchen Yu, Qi She, Yao Zhao, Yunchao Wei

llms computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 10

Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception

arXiv:2606.06714v3 Announce Type: replace Abstract: Human perception of surface slant from texture exhibits systematic, graded biases that emerge reliably in psychophysical experiments. Prior work sh...

By Qian Zhang, Michal Golovanevsky, Fulvio Domini, James Tompkin

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 10

Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation

arXiv:2607.26694v3 Announce Type: replace Abstract: We present Visko Orbis 1.0, a Live Model for real-time, interactive long video generation. Users can change the prompt at any moment during generat...

By Xiangbo Gao, Siyuan Yang, Ping He, Mingyang Wu, Yuheng Wu, Yushen Zuo, Jiongze Yu, Ryan Cui, Hongyuan Hua, Devin Ma, Xiao Jin, Yubo Ruan, Qing Yin, Jie Yang, Zhengzhong Tu

diffusion benchmarks

arXiv Computer Vision

Sep 10

Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs

arXiv:2608.05592v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have made strong progress in video understanding, yet long videos remain difficult: the visual token budge...

By Ziling Huang, Shin'ichi Satoh

llms multimodal benchmarks

arXiv Computer Vision

Sep 10

CGSM: Concept-Guided Segmentation Model for Precise Pulmonary Lesion Delineation

arXiv:2609.07004v2 Announce Type: replace Abstract: Accurate segmentation of pulmonary lesions is essential for effective clinical diagnosis and treatment strategies. Existing segmentation approaches...

By Changheng Lin, Wenjie Zhang, Yushan Lu, Xinyue Yan, Xiao Jia, Wei Zhang

llms computer-vision benchmarks safety

arXiv Computer Vision

Sep 10

ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding

arXiv:2609.07941v2 Announce Type: replace Abstract: In this paper, we propose ReactVAU, a Slow-Fast Decoupled Framework for real-time streaming Video Anomaly Understanding (VAU). Existing VAU methods...

By Chia-Hui Chen, Shih-Ying Yeh, Fu-En Yang, Min-Hung Chen, Shang-Hong Lai

benchmarks

Get the daily briefing