Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Computer Vision

Sep 22

RSPDBench: Benchmarking Vision Foundation Models on Earth Observation Tasks Under Physically Grounded Remote-Sensing Product Degradations

arXiv:2609.23427v1 Announce Type: new Abstract: Vision foundation models targeting Earth observation (EO) tasks are commonly evaluated on clean downstream benchmarks, but operational EO products can...

By Tanjim Bin Faruk, Khondaker Masfiq Reza, Shrideep Pallickara, Sangmi Lee Pallickara

benchmarks safety

arXiv Computer Vision

Sep 22

PhysReflect: Geometry and Perception Guided Diffusion for Physically-Plausible Mirror Reflections

arXiv:2609.23442v1 Announce Type: new Abstract: Diffusion models generate high-quality images, yet often violate the physical laws governing mirror reflections. Reflections often suffer from geometri...

By Shuheng Ge, Hongwei Ren, Li Zhang, Xiangqian Wu

diffusion benchmarks safety

arXiv Computer Vision

Sep 22

Towards robust multimodal 3D object detection via visual foundation models

arXiv:2609.23541v1 Announce Type: new Abstract: Multimodal 3D object detection is fundamental to robust perception in autonomous driving because it integrates complementary information from LiDAR and...

By Ziying Song, Lin Liu, Hongyu Pan, Shaoqing Xu, Lei Yang, Mingzhe Guo, Caiyan Jia

agents computer-vision robotics multimodal benchmarks

arXiv Computer Vision

Sep 22

Mind the Gaps: A Curated Benchmark for Form Field Detection

arXiv:2609.23679v1 Announce Type: new Abstract: Form Field Detection (FFD) is a fundamental component of document understanding systems, enabling applications ranging from large-scale industrial digi...

By Iheb Brini, Omar Moured, Hamza Gbada, Elisa Barney

computer-vision benchmarks

arXiv Computer Vision

Sep 22

BindCLIP: One Balanced Coupling For Compositional Vision Language Scoring

arXiv:2609.23717v1 Announce Type: new Abstract: Global vision--language similarities compress an image and a caption into one vector, preserving semantics but not which word corresponds to which regi...

By Liuyang Song, Yi Zhang, Zhongyi Deng, Daqian Yang, Hongbo Zhang

benchmarks

arXiv Computer Vision

Sep 22

PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning

arXiv:2609.23769v1 Announce Type: new Abstract: The disambiguation of semantically similar statutory text across jurisdictions is a retrieval problem that existing methods do not solve. This inter-co...

By Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs, Khoa Luu

llms rag multimodal benchmarks

arXiv Computer Vision

Sep 22

U-PEN Mamba: Progressive Expansion with Selective State-Space Modeling for Efficient Retinal Vessel Segmentation

arXiv:2609.24049v1 Announce Type: new Abstract: Accurate retinal vessel segmentation is important for computer-aided ophthalmic analysis, yet thin vessels, low contrast, and severe foreground-backgro...

By Abel A. Reyes-Angulo, Sidike Paheding, Vijayan K. Asari, Mohammad Alam, Jeevan Devagiri

llms computer-vision benchmarks

arXiv Computer Vision

Sep 22

Patch-to-Global: Random Patch Diffusion for Globally Consistent Megapixel Artifact Inpainting in Whole Slide Images

arXiv:2609.24116v1 Announce Type: new Abstract: Although deep learning has advanced Whole Slide Image (WSI) Analysis, tissue artifacts like bubbles and folds often cause silent failures by concealing...

By Hyeseong Lee, Eunsu Kim, D M Bappy, Ho Heon Kim, Youngsuk Lee, Se Young Chun, Jang-Hwan Choi, Sung Hak Lee, Sangjeong Ahn

rag diffusion benchmarks

arXiv Computer Vision

Sep 22

Benchmarking Off-the-Shelf Multimodal AI Models Against Dermatologists on Patient-Captured Skin Images

arXiv:2609.24190v1 Announce Type: new Abstract: Artificial intelligence (AI) has advanced at a rapid pace in recent years. Initially, breakthroughs in large language models caught widespread attentio...

By Rian Dolphin, Laura Knowles

llms multimodal benchmarks

arXiv Computer Vision

Sep 22

SAFe: Segment-guided Aggregation of Feature Densities for Anomaly-aware Segmentation

arXiv:2609.24204v1 Announce Type: new Abstract: Visual segmentation systems encounter objects outside their training distribution during real-world deployment, hindering reliable autonomous systems t...

By Anja Deli\'c, Jurica Runtas, Marin Or\v{s}i\'c, Ivan Markovi\'c, Ivan Petrovi\'c

llms agents computer-vision benchmarks

arXiv Computer Vision

Sep 22

SRPR-Net: Semantic and Relational Prompt Refinement for Automated SAM-based Instance Segmentation

arXiv:2609.24226v1 Announce Type: new Abstract: Instance segmentation is a fundamental computer vision task with diverse real-world applications. Recently, prompt-driven foundation models have shown...

By Lufei Liu, Guojie Li, Suncheng Xiang, Fan Zhang

llms computer-vision benchmarks

arXiv Computer Vision

Sep 22

IMPLICIT-Bench: Measuring Implicit Bias in Text-to-Image Models under Neutral Prompts

arXiv:2609.24228v1 Announce Type: new Abstract: Text-to-image (T2I) models are typically evaluated for bias using slot-based templates such as ``a photo of a [profession]''. Such templates probe only...

By Yue Dai, Ziyang Liu, Marc Cheong, Caren Han

diffusion benchmarks safety

arXiv Computer Vision

Sep 22

HappyWorld-Bench

arXiv:2609.24308v1 Announce Type: new Abstract: Evaluating world models requires assessing both the quality of the worlds they generate and their consistency and responsiveness under exploration, int...

By Zhiqi Bai, Junai Cai, Yixin Chen, Jingrun Du, Tao Feng, Wei Gong, Siyuan Huang, Xiao Lin, Jiaheng Liu, Jun Luo, Yongzhe Lyu, Liya Ma, Zenan Meng, Lin Qu, Wenbo Su, Jiaming Wang, Qinghe Wang, Shaofei Wang, Yanghai Wang, Zequn Wang, Ziming Wang, Hu Wei, Jiangtao Wu, Ruiqi Wu, Jiaxin Xie, Yuchi Xu, Ze Xu, Chengting Yu, Liangyu Yuan, Gang Zeng, Yawen Zeng, Xingyao Zhang, Zizheng Zhang, Bo Zheng, Jiancheng Zhu, Song-Chun Zhu

agents robotics benchmarks

arXiv Computer Vision

Sep 22

ME-VLM:A Unified VLM for Embodied Cognition and Agent Coordination

arXiv:2609.24526v1 Announce Type: new Abstract: Physical AI requires models to ground visual and linguistic understanding in real-world environments while accounting for environmental constraints and...

By Foundation Model, Li Auto Inc

llms agents reinforcement-learning robotics efficiency multimodal benchmarks

arXiv Computer Vision

Sep 22

Incentive Noise and Structural Prior Infusion for Multi-modal Object Re-Identification

arXiv:2609.24539v1 Announce Type: new Abstract: Multi-modal object Re-Identification (ReID) benefits from complementary information across heterogeneous imaging modalities. To further enrich semantic...

By Weixiang Zhou, Yuhao Wang, Xingguo Xu, Weizhen Zhou, Zhixun Su, Jinshan Pan, Cong Wang

multimodal benchmarks safety

arXiv Computer Vision

Sep 22

HyperCLIP++: Fine-tuning CLIP forOpen-vocabulary Semantic Segmentation in Hyperbolic Space

arXiv:2609.24564v1 Announce Type: new Abstract: CLIP, a foundational vision-language model, has emerged as a powerful tool for open-vocabulary semantic segmentation. While freezing CLIP's text encode...

By Zelin Peng, Zhengqin Xu, Changsong Wen, Yu Huang, Yaoming Wang, Xiaokang Yang, Wei Shen

llms rag computer-vision fine-tuning multimodal benchmarks safety

arXiv Computer Vision

Sep 22

Active Visual Sampling with a Connectome-Constrained Fly Model for One-Shot Hatch Recognition in Architectural Drawings

arXiv:2609.24565v1 Announce Type: new Abstract: Architectural drawings encode material classes through repeated hatch patterns. We test whether a connectome-constrained fly visual network, pretrained...

By Dmitry Kuklev

benchmarks

arXiv Computer Vision

Sep 22

Video-based Surgical Skill Assessment Using Dynamics-and-Uncertainty-Aware Tree-based Gaussian Process Classifier

arXiv:2609.24619v1 Announce Type: new Abstract: The proposed pipeline integrates a representation-flow convolutional neural network with a dynamics- and uncertainty-aware tree-based Gaussian Process...

By Arefeh Rezaei, Mohammad Javad Ahmadi, Amir Molaei, Hamid D. Taghirad

benchmarks

arXiv Computer Vision

Sep 22

Ananke: Contractive Torus Attractor Networks

arXiv:2609.24737v1 Announce Type: new Abstract: We introduce Ananke, a representation-learning framework that scaffolds latent representations onto a structured product-torus prior, and its flagship...

By Zhongping Ji

diffusion benchmarks

arXiv Computer Vision

Sep 22

ZVeC: A Zero-Shot Framework for Instance-Level Vehicle Extraction and Generative Point Cloud Completion

arXiv:2609.24825v1 Announce Type: new Abstract: LiDAR point clouds acquired in underground environments exhibit severe geometric incompleteness due to occlusions and limited sensor viewpoints, making...

By Daisy Li, Kyle Gao, Quanyun Wu, Boris Jutzi, John S. Zelek, Jonathan Li

diffusion efficiency benchmarks

Get the daily briefing