Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv Computer Vision

Oct 27

MemVLN: Streaming Episodic Memory for Vision-Language Navigation

arXiv:2610.35066v1 Announce Type: new Abstract: Vision-language navigation (VLN) requires an embodied agent to follow language instructions through 3D environments. Maintaining persistent episodic me...

By Yan Wang, Jiangyong Huang, Baoxiong Jia, Siyuan Huang

agents nlp robotics multimodal benchmarks

arXiv Computer Vision

Oct 27

Temporal Slice Learning for AI-Generated Video Detection with 400$\times$ Fewer FLOPs

arXiv:2610.35148v1 Announce Type: new Abstract: Recent advances in generative video models have improved visual realism, making detection increasingly dependent on subtle temporal inconsistencies. Ho...

By Wei Shang, Zhiyuan Li, Shengfeng He

llms benchmarks

arXiv Computer Vision

Oct 27

Evidence-Guided Selection of Retinal Vessel Segmentation Pipelines: A Cross-Dataset Benchmark of Performance, Structure, Failure Modes, and Reproducibility

arXiv:2610.35357v1 Announce Type: new Abstract: Retinal vessel segmentation is widely used in retinal image analysis, yet model selection based only on aggregate overlap scores can overlook structura...

By Ruoyu Yan, Yuejia Liu, Shixin Xu, Xiaoyan Li, Xi Luo, Yuhui Deng, Huaxiong Huang

computer-vision benchmarks

arXiv Computer Vision

Oct 27

GIFT: Representation Geometry Matters for Single-Domain Generalized Object Detection

arXiv:2610.35374v1 Announce Type: new Abstract: Single-Domain Generalized Object Detection (S-DGOD) aims to generalize from a single source domain to unseen domains under distribution shifts. Existin...

By Yin Zhang, Yaoyue Zheng, Yongqiang Zhang, Bogdan Raducanu, Dan Li

computer-vision fine-tuning benchmarks

arXiv Computer Vision

Oct 27

Unveil the Mask: Jointly-Optimized Diffusion Masked Pretraining for Appearance-Agnostic Operator Verification

arXiv:2610.35414v1 Announce Type: new Abstract: Photorealistic avatar videos decouple the appearance of a rendered face from the person who operates it, which makes conventional face recognition unsu...

By Katsuya Ogata, Ryoya Yoshiyama, Kawachi Hodaka, Yuta Nakashima

rag diffusion benchmarks

arXiv Computer Vision

Oct 27

S$^2$-RL: Sample-Set Dual Reinforcement Learning for Generative Semantic Segmentation Dataset Distillation

arXiv:2610.35491v1 Announce Type: new Abstract: While dataset distillation has witnessed significant advances in image classification, its extension to dense prediction tasks such as semantic segment...

By Haoyu Wang, Fei Zhou, Qingqing Qiu, Lei Zhang, Wei Wei, Chen Ding

diffusion computer-vision reinforcement-learning fine-tuning efficiency benchmarks safety

arXiv Computer Vision

Oct 27

HiFiPix: Bringing High-Fidelity Real-World Super-Resolution Back to Pixel Space

arXiv:2610.35515v1 Announce Type: new Abstract: Diffusion-based real-world image super-resolution suffers from limited reconstruction fidelity due to information loss in VAE-based latent representati...

By Wenjie Li, Du Chen, Hui Wang, Linlong Fan, Youwei Lv, Zeyi Chen, Zehan Chen, Jinwei Chen, Heng Guo, Zhanyu Ma, Qingnan Fan

diffusion fine-tuning efficiency benchmarks

arXiv Computer Vision

Oct 27

Learning High-Quality Pseudo-Labels for Point-Supervised Temporal Action Localization

arXiv:2610.35516v1 Announce Type: new Abstract: Point-supervised temporal action localization (PTAL) aims to recognize and localize action instances in untrimmed videos using only point-level annotat...

By Zhenying Fang

benchmarks

arXiv Computer Vision

Oct 27

EARS: Evidence-Aware Retrieval via Sound and Motion for Budgeted Video Reasoning

arXiv:2610.35536v1 Announce Type: new Abstract: Uniform frame sampling can miss brief evidence in sparse-event videos when the VLM sees only K frames. We propose EARS (Evidence-Aware Retrieval via So...

By Shih-Yao Lin

multimodal benchmarks

arXiv Computer Vision

Oct 27

Copying As Defense: Protecting Images from Editing By Eliciting Model Copying

arXiv:2610.35566v1 Announce Type: new Abstract: Image editors built on diffusion transformers (DiTs), such as Flux Kontext, make it possible to turn a single photograph into a convincing manipulated...

By Salma Abdel Magid, Sanghyuk Chun, Eric Slyman, Kushal Kafle, Olga Russakovsky

llms diffusion benchmarks

arXiv Computer Vision

Oct 27

Hyperspectral Unmixing using a Structured Linear Mixing Model with Additive Spectral Variability

arXiv:2610.35580v1 Announce Type: new Abstract: Linear hyperspectral unmixing aims at retrieving pure material spectra present in an observed scene and their associated abundance fractions in each pi...

By Salah Eddine Brezini, Yannick Deville, Moussa Sofiane Karoui, Fatima Zohra Benhalouche, Abdelaziz Ouamri

benchmarks

arXiv Computer Vision

Oct 27

Printed and Scanned: Exposing the Fragility of Single-Image Morphing Attack Detectors under Print-Scan Degradation

arXiv:2610.35682v1 Announce Type: new Abstract: Face morphing attacks, in which the biometric traits of two or more identities are blended into a single face image, pose a severe threat to identity d...

By Diogo J. Paulo, Hugo Proen\c{c}a, Jo\~ao C. Neves

benchmarks

arXiv Computer Vision

Oct 27

Back to Basics: A Skill-Centric Approach to Generalist Web Agents

arXiv:2610.35823v1 Announce Type: new Abstract: Browser agents do not just need more practice. They need to practice the right things. We introduce MiniWeb, a realistic miniature of the web comprisin...

By Minh Pham-Dinh, Md Farhan Ishmam, Hernan Yupanqui, Reaz Hassan Joarder, Kenneth Marino

agents fine-tuning benchmarks

arXiv Computer Vision

Oct 27

FL4N: Feed-forward Latent Model for 4D Novel View Synthesis

arXiv:2610.35854v1 Announce Type: new Abstract: 4D novel view synthesis (NVS) aims to render dynamic scenes across viewpoints and timestamps, which requires jointly inferring 3D structure and tempora...

By Eunbeen Hong, Hyeonseo Yu, Honggyu An, Jooho Son, Hyunah Ko, Soowon Son, Chaehyun Kim, Jahyeok Koo, Jisoo Kim, Junhwa Hur, Seungryong Kim

benchmarks

arXiv Computer Vision

Oct 27

VibeVideoBench:Evaluating Multimodal Agents on Code-Driven Video Generation

arXiv:2610.35864v1 Announce Type: new Abstract: Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a ruleconsistent game or narration sy...

By Muxi Diao, Wenbin Xu, Lele Yang, Xuyang Ma, Yuzhe Chen, Jiankang Chen, Kaiyu Jiang, Changyi Liu, Haoyu Zhang, Hongyi Fu, Jianxiong Wang, Tianke Zhang, Haonan Fan, Yingxin Li, Bingyi Li, Xu Wang, Xuanyu Zheng, Bin Wen, Tingting Gao, Han Li, Wentao Zhang, Weiran Xu, Kongming Liang, Zhanyu Ma

llms agents diffusion multimodal benchmarks

arXiv Computer Vision

Oct 27

Vision Token Pruning via Query-Vision Interaction Decomposition

arXiv:2610.33215v1 Announce Type: cross Abstract: Large vision-language models (VLMs), which process both visual inputs and text queries, incur high inference costs due to the large number of visual...

By Harshithanjani Athi, Sravan Kumar Ankireddy, Jianzhong Charlie Zhang, Hyeji Kim

llms efficiency multimodal benchmarks

arXiv Computer Vision

Oct 27

Satellite-derived bathymetry of Arabian Gulf ports under leakage-free validation: a multi-site benchmark of empirical, machine-learning and deep-learning models against 285,000 multibeam soundings, with an outlook for UAE waters

arXiv:2610.33609v1 Announce Type: cross Abstract: Satellite-derived bathymetry (SDB) is often proposed as a low-cost complement to multibeam surveys of ports, but its accuracy is seldom measured on a...

By Muhammad Saeed, Fatema Al Hajri, Mohamed H. Alhosani, Dmitry Mikhaylov, Yasser F. Al Wahedi

benchmarks

arXiv Computer Vision

Oct 27

Behavior Spatial Benchmark: Evaluating Spatial Understanding for Robotics

arXiv:2610.34051v1 Announce Type: cross Abstract: Robot tasks require grounding spatial language in physical scenes, but task success alone cannot establish whether spatial instructions are followed....

By Sirui Chen, Huang Huang, Qineng Wang, Wenlong Huang, Manling Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu

nlp robotics multimodal benchmarks

arXiv Computer Vision

Oct 27

MMGNet: Multi-Modal 6-DoF Grasp Synthesis for Cluttered Scenes

arXiv:2610.34893v1 Announce Type: cross Abstract: Grasping is a foundational capability in robotic manipulation. Scene-level 6-DoF grasp synthesis relies on point clouds for metric geometry, yet real...

By Rui Cao, Jia-Hui Pan, Yichuan Li, Yunfan Yu, Chi-Wing Fu, Jiangliu Wang, Yun-Hui Liu

robotics multimodal benchmarks safety

arXiv Computer Vision

Oct 27

MedVTok: A General-Purpose Medical Visual Tokenizer

arXiv:2610.35842v1 Announce Type: cross Abstract: Multimodal medical AI requires a common visual interface for encoding heterogeneous imaging evidence for patient condition modeling. However, existin...

By Chenglong Ma, Yuanfeng Ji, Junzhi Ning, Jiyao Liu, Yujie Wei, Shaohao Rui, Chaoyang Zhang, Wenjie Li, Wenhao Tang, Cheng Tang, Wei Li, Shujian Gao, Yujie Zhang, Ying Chen, Zilong Li, Zongyuan Ge, Diping Song, Jin Ye, Ming Hu, Junjun He, Hongming Shan

computer-vision nlp multimodal benchmarks safety

Get the daily briefing