Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Computer Vision

Oct 1

UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision

arXiv:2609.39388v1 Announce Type: cross Abstract: Mobile manipulation requires precise navigation to a manipulation-ready pose followed by reliable object interaction. These two stages differ in acti...

By Wei Xue, Keliang Liu, Mingzhang Cui, Jinhua Xie, Jinjie Wei, Jianan Hou, Jingcheng Lu, Lintao Wang, Kaixiang Qiu, Yizhou Liu, Xinghai Ye, Jinghang Han, Mingcheng Li, Jie Gu, Shunli Wang, Lihua Zhang, Dingkang Yang

robotics benchmarks

arXiv Computer Vision

Oct 1

FrontierGS: Progressive View-Space Frontier Expansion for Sparse 3D Gaussian Splatting

arXiv:2511.16030v3 Announce Type: replace Abstract: 3D Gaussian Splatting (3DGS) enables efficient, high-fidelity novel view synthesis, yet its performance degrades severely under sparse-view supervi...

By Zijian Wu, Mingfeng Jiang, Zidian Lin, Ying Song, Ziqian Lu, Qun Wu, Hanjie Ma

benchmarks

arXiv Computer Vision

Oct 1

Semantic Misalignment in Vision-Language Models under Perceptual Degradation

arXiv:2601.08355v3 Announce Type: replace Abstract: Vision-Language Models (VLMs) are increasingly deployed in autonomous driving and embodied AI systems, where reliable perception is critical for sa...

By Guo Cheng, Huang Li

llms agents computer-vision robotics multimodal benchmarks safety

arXiv Computer Vision

Oct 1

Dynamics-Inspired Diffusion for Foreground-Preserving Document Background Editing

arXiv:2601.21857v2 Announce Type: replace Abstract: We revisit diffusion-based generation for structured visual content and identify a fundamental limitation of existing approaches: foreground preser...

By Taewon Kang, Yu Shen, Ming C. Lin

diffusion multimodal benchmarks

arXiv Computer Vision

Oct 1

Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching

arXiv:2602.05391v3 Announce Type: replace Abstract: Dataset distillation seeks to synthesize a compact surrogate dataset that enables performance comparable to training on the original dataset for do...

By Qianxin Xia, Jiawei Du, Yuhan Zhang, Xin Zhang, Xuewan He, Wenbo Jiang, Jielei Wang, Tao Luo, Guoming Lu

diffusion efficiency benchmarks

arXiv Computer Vision

Oct 1

EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detection

arXiv:2603.17343v3 Announce Type: replace Abstract: The rapid proliferation of AI-Generated Images (AIGIs) poses severe misinformation risks, making AIGI detection critical yet challenging. Tradition...

By Chenyang Zhu, Maorong Wang, Jun Liu, Ching-Chun Chang, Isao Echizen

llms agents reinforcement-learning multimodal benchmarks safety

arXiv Computer Vision

Oct 1

Project and Mix: Task-Semantic Prototypes for Few-Shot Image Classification

arXiv:2603.24528v2 Announce Type: replace Abstract: Vision-language models like CLIP are trained with the objective of aligning text and image pairs. Beyond text prompts alone, recent works show that...

By Dipam Goswami, Simone Magistri, Gido M. van de Ven, Bart{\l}omiej Twardowski, Andrew D. Bagdanov, Tinne Tuytelaars, Joost van de Weijer

llms rag computer-vision multimodal benchmarks safety

arXiv Computer Vision

Oct 1

XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments

arXiv:2604.18484v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from co...

By Kangan Qian, ChuChu Xie, Yang Zhong, Jingrui Pang, Siwen Jiao, Sicong Jiang, Zilin Huang, Yunlong Wang, Kun Jiang, Mengmeng Yang, Hao Ye, Guanghao Zhang, Hangjun Ye, Guang Chen, Long Chen, Diange Yang

llms agents reinforcement-learning robotics multimodal benchmarks

arXiv Computer Vision

Oct 1

AD-Relight: Training-Free Banner Relighting via Illumination Translation with Diffusion Priors

arXiv:2604.24407v2 Announce Type: replace Abstract: The recent surge in content consumption through streaming services has driven a growing demand for personalized content. Personalized advertisement...

By Rameshwar Mishra, Bishshoy Das, A V Subramanyam, Guan-Ming Su

diffusion benchmarks

arXiv Computer Vision

Oct 1

EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learning

arXiv:2605.22208v3 Announce Type: replace Abstract: Multimodal Large Language Model (MLLM)-driven image restoration agent demonstrates effectiveness in degradation coupling scenarios by flexibly sele...

By Kailin Zhuang, Jiawei Wu, Zhi Jin

llms agents multimodal benchmarks

arXiv Computer Vision

Oct 1

PhysMirror: Physics-Aware Mirror Object Generation

arXiv:2607.03470v2 Announce Type: replace Abstract: Synthesizing physically accurate mirror reflections remains a fundamental challenge for modern text-to-image diffusion models, which are increasing...

By Xuan-Bach Mai, Duy-Phuc Nguyen, Quoc-Van Le, Tam V. Nguyen, Thanh-Toan Do, Huu Le, Duong-Van Nguyen, Minh-Triet Tran, Trung-Nghia Le

diffusion computer-vision robotics benchmarks safety

arXiv Computer Vision

Oct 1

One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding

arXiv:2608.05707v2 Announce Type: replace Abstract: Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows....

By Wang Chen, Yu Chen, Xiang Wang, Shuai Li, Jinfa Huang, Xiawu Zheng

multimodal benchmarks

arXiv Computer Vision

Oct 1

AnesTRACE: Benchmarking Intraoperative Anesthesia from Multimodal Perception to Multi-step Decision-Making

arXiv:2609.32740v2 Announce Type: replace Abstract: Intraoperative anesthesia requires systems to interpret evolving multimodal evidence, recommend timely management, and revise decisions as patient...

By Ziwei Huang, Qi Gao, Zhe Ji, Yuanyuan Yao, Fengjiang Zhang, Min Yan, Zhongle Xie, Gang Chen

fine-tuning multimodal benchmarks safety

arXiv Computer Vision

Oct 1

Rate-Distortion Adaptive Primitive Selection for Omnidirectional Gaussian Splatting

arXiv:2609.34367v2 Announce Type: replace Abstract: Learned image codecs (LICs) achieve high reconstruction quality, but their decoding speed is often insufficient for immersive virtual reality (VR)....

By Yulong Cheng, Youneng Bao, Junfeng Zhou, Mu Li, Jie Wen

efficiency benchmarks

arXiv Computer Vision

Oct 1

Revisit to Segment: Working Memory Distillation for Reasoning Segmentation

arXiv:2609.34863v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have approached image segmentation by reasoning about visual content and predicting target locations. Thei...

By Cilin Yan, Yilun Qiu, Wanyang Zhang, Rui Zu, Xiaolong Jiang, Jiayin Cai, Yao Hu

llms computer-vision reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Oct 1

WARP: A Unified Benchmark for Invisible Image Watermarking -- Robustness and Protection Against Attacks

arXiv:2609.40031v1 Announce Type: cross Abstract: Digital image watermarking is increasingly critical in media contexts, as emerging regulations and industry practices require marking AI-generated co...

By Khaled Abud, Aleksey Yakushev, Aleksandr Akimenkov, Irina Serzhenko, Kirill Aistov, Egor Kovalev, Dmitry Obydenkov, Sergey Lavrushkin, Anastasia Antsiferova, Dmitriy Vatolin, Yury Markin, Kirill Lukianov

rag robotics benchmarks safety

arXiv AI

Oct 1

Less Data, Better Timing: Student-Curriculum Coupling for VLM On-Policy Distillation in Temporal Video Grounding

arXiv:2609.40055v1 Announce Type: cross Abstract: On-policy distillation (OPD) provides dense supervision directly on student-generated trajectories, making it an effective post-training strategy for...

By Jiacheng Qiu, Yunsoo Kim, Ruichen Xu, Jian Luo, Petar M. Djuri\'c, Sima Mofakham

llms efficiency multimodal benchmarks

arXiv AI

Oct 1

An Empirical Study of On-Device Translation for Real-Time Live-Stream Chat on Mobile Devices

arXiv:2601.02641v2 Announce Type: replace Abstract: Despite its efficiency, there has been little research on the practical aspects required for real-world deployment of on-device AI models, such as...

By Jeiyoon Park, Daehwan Lee, Changmin Yeo, Yongshin Han, Minseop Kim

llms fine-tuning benchmarks

arXiv AI

Oct 1

Turbo Harness: Instance-Adaptive Harness Optimization

arXiv:2609.40330v1 Announce Type: new Abstract: Automating the search for effective harnesses is an important step toward enabling agents to recursively self-improve. Existing harness optimizations t...

By Tunyu Zhang, Hao Wang, Kai Xu, Dimitris N. Metaxas

agents benchmarks

arXiv AI

Oct 1

Forensic-Aware Continual Adaptation for Image Forgery Localization

arXiv:2609.38251v1 Announce Type: cross Abstract: The rapid evolution of image manipulation techniques has raised growing public security concerns. Existing Image Forgery Localization (IFL) methods c...

By Chenqi Kong, Song Xia, Anwei Luo, Peisong He, Alex C. Kot, Yuming Fang

llms robotics fine-tuning benchmarks

Get the daily briefing