Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Computation and Language

Sep 30

Bootstrapping Audiovisual Speech Recognition in Zero-AV-Resource Scenarios

arXiv:2603.08249v2 Announce Type: replace-cross Abstract: Audiovisual speech recognition (AVSR) combines acoustic and visual cues to improve transcription robustness under challenging conditions but...

By Pol Buitrago, Javier Hernando

fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning

arXiv:2609.35823v1 Announce Type: new Abstract: Vision-language models (VLMs) have made substantial progress in autonomous driving, but their success has primarily been studied in ego-centric scenes....

By Kang Yang, Shuai Liu, Hang Li, Yance Fang, Deying Li, Yongcai Wang

llms agents nlp robotics multimodal benchmarks

arXiv Computer Vision

Sep 30

Xiaomi-OCR-0 Technical Report

arXiv:2609.36136v1 Announce Type: new Abstract: Compact OCR-specific vision-language models achieve strong document parsing performance, but often rely on costly supervision and focus primarily on vi...

By Xin Chen, Anan Du, Feng Feng, Pei Fu, Jian Luan, Longwei Xu, Shaojie Zhang, Hang Li, Heng Qu, Cheng Tan

llms computer-vision reinforcement-learning multimodal benchmarks

arXiv Computer Vision

Sep 30

LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoning

arXiv:2609.36219v1 Announce Type: new Abstract: Perspective taking is a fundamental component of spatial intelligence, requiring models interpret spatial relations from a specified viewpoint, such as...

By Bang Xiao, Wenqi Jia, Ozgur Kara, Tiancheng Shen, Yibo Yang, Bolin Lai, Junho Kim, James Matthew Rehg

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding

arXiv:2609.36217v1 Announce Type: new Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from vi...

By Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu, Lenny Aharon, Kyle Daruwalla, Xun Helen Hou, Matthew R. Whiteway, Liam Paninski, Yizi Zhang

llms rag computer-vision benchmarks safety

arXiv Computer Vision

Sep 30

Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation

arXiv:2609.36598v1 Announce Type: new Abstract: A video can exhibit convincing motion and photorealism yet fail immediately when visual text collapses. Unlike generic scene content, visual text is un...

By Ziying Zhang, Litao Li, Junchao Liao, Tianyi Zeng, Siyu Zhu, Long Qin, Zhenghao Zhang

diffusion computer-vision benchmarks

arXiv Computer Vision

Sep 30

Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

arXiv:2609.36628v1 Announce Type: new Abstract: Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, partic...

By Yanan Wang, Tingsong Li, Kaixun Jiang, Chongyang Zhong, Chenwei Xoe, Zhaohe Liao

llms reinforcement-learning multimodal benchmarks safety

arXiv Computer Vision

Sep 30

OCA: ODE-Driven Cross-Attention for Image-to-Point-Cloud Registration

arXiv:2609.36644v1 Announce Type: new Abstract: Cross-attention is a crucial component in learning-based image-to-point-cloud (I2P) registration. Although existing cross-attention mechanisms have ach...

By Pei An, Jiaqi Yang, Yulong Wang, Siwen Quan, Liangliang Nan

fine-tuning benchmarks

arXiv Computer Vision

Sep 30

Not Every Correction Helps: Gain-Guided Continual Test-Time Adaptation

arXiv:2609.36655v1 Announce Type: new Abstract: Continual test-time adaptation (CTTA) adapts a source model to an unlabeled test stream whose distribution may change over time. Existing TTA methods o...

By Youjia Zhang, Huiling Liu, Soyun Choi, Jaehong Yoon, Sungeun Hong

benchmarks

arXiv Computer Vision

Sep 30

Reprogramming Vision-Language Models via Structured Prompt Reparameterization

arXiv:2609.36680v1 Announce Type: new Abstract: Visual reprogramming adapts pretrained models to downstream tasks by modifying their input and output interfaces while keeping the backbone fixed. In v...

By Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari, Feng Liu, Joseph West, Kourosh Khoshelham

llms rag multimodal benchmarks

arXiv Computer Vision

Sep 30

When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation

arXiv:2609.36685v1 Announce Type: new Abstract: Co-speech gesture generation aims to synthesize natural gestures that are both temporally synchronized with speech and semantically consistent with the...

By Zhirui Xing, Long Ye, Kaige Li, Ziyi Xu, Ming Meng

multimodal benchmarks

arXiv Computer Vision

Sep 30

DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning

arXiv:2609.36775v1 Announce Type: new Abstract: Reinforcement Learning has significantly advanced the complex reasoning capabilities of MLLMs. However, prevailing RL algorithms suffer a severe failur...

By Cheng Ye, Weidong Chen, Bingyan Xu, Zhendong Mao

reinforcement-learning benchmarks safety

arXiv Computer Vision

Sep 30

Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention

arXiv:2609.36776v1 Announce Type: new Abstract: Emotional Video Captioning aims to generate factually accurate and emotionally empathetic descriptions. While recent methods have recognized the import...

By Cheng Ye, Weidong Chen, Peipei Song, Zhendong Mao

multimodal benchmarks safety

arXiv Computer Vision

Sep 30

Scene Retargeting: Learning Object Placement with Analogical Transfer

arXiv:2609.36801v1 Announce Type: new Abstract: Interactive simulations of embodied AI or spatial computing applications build on realistic 3D scenes that support daily activities. However, sparse, i...

By Minkwan Kim, Junho Kim, Seungmin Lee, Changwoon Choi, Young Min Kim

robotics benchmarks safety

arXiv Computer Vision

Sep 30

CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval

arXiv:2609.36815v1 Announce Type: new Abstract: Partially Relevant Video Retrieval (PRVR) seeks to retrieve untrim-med videos containing a moment that matches a text query, without temporal annotatio...

By Zhen Liu, Letian Li, Jinpeng Wang, Shuzhao Xie, Yuzhi Huang, Jingyan Jiang, Zhi Wang

rag multimodal benchmarks

arXiv Computer Vision

Sep 30

DiffReID: Discriminative Diffusion Model for Object Re-Identification

arXiv:2609.36894v1 Announce Type: new Abstract: As a fundamental image processing task, object Re-Identification (ReID) aims to retrieve objects across non-overlapping cameras. Recently, with the dev...

By Yingquan Wang, Pingping Zhang, Dong Wang, Huchuan Lu

diffusion benchmarks

arXiv Computer Vision

Sep 30

VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation

arXiv:2609.37003v1 Announce Type: new Abstract: Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most e...

By Danfeng Hong, Chenyu Li, Jocelyn Chanussot

computer-vision multimodal benchmarks

arXiv Computer Vision

Sep 30

OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models

arXiv:2609.37052v1 Announce Type: new Abstract: Omnimodal large language models (Omni-LLMs) encode audio and visual streams into temporally interleaved token sequences for multimodal reasoning. Howev...

By Yuchen Deng, Zidang Cai, Feidiao Yang, Yufei Wang, Jie Wang, Hai-Tao Zheng, Yuxing Han

llms multimodal benchmarks

arXiv Computer Vision

Sep 30

Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack

arXiv:2609.37096v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) consistently struggle with fine-grained visual counting, yet the underlying causes remain poorly understood. I...

By Liwei Che, Yihao Quan, Sen Fang, Hongyi Wang, Ranjay Krishna, Ruixiang Tang, Vladimir Pavlovic

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Sep 30

TaoFlowForge: Progressive Native Mesh Generation via Cascaded Flow Matching

arXiv:2609.37139v1 Announce Type: new Abstract: 3D content generation technology has significantly advanced the work of designers, as well as the 3D printing and gaming industries. However, it remain...

By Xianze Fang, Qiyuan Feng, Dongfang Sun, Yan Zhang, Xiuchao Wu, Jingnan Gao, Jiangjing Lyu, Chengfei Lyu, Gang Yu

diffusion benchmarks

Get the daily briefing