Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv Computer Vision

Oct 15

CommerceGen-Studio: An End-to-End Framework for Commercial-Grade Main-Image Generation

arXiv:2610.20603v1 Announce Type: new Abstract: We present CommerceGen-Studio, an end-to-end framework for generating commercial-grade main images for e-commerce Stock Keeping Units (SKUs). Our frame...

By Xiaoli Xu, Zhengtao Wu, Zhengze Xu, Tao Liu, Yongchao Du, Zuan Gao, Xingjian Wang, Hao Yan, Mengting Chen, Wei Li, Bo Zheng

llms agents diffusion multimodal benchmarks

arXiv Computer Vision

Oct 15

Video Depth Anything: Consistent Depth Estimation for Super-Long Videos

arXiv:2501.12375v4 Announce Type: replace Abstract: Depth Anything has achieved remarkable success in monocular depth estimation with strong generalization ability. However, it suffers from temporal...

By Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, Bingyi Kang

diffusion computer-vision benchmarks

arXiv Computer Vision

Oct 15

Enabling Validation for Robust Few-Shot Recognition

arXiv:2506.04713v4 Announce Type: replace Abstract: A key challenge in Few-Shot Recognition (FSR) is the scarcity of labeled task-specific data for training. Contemporary methods mitigate this challe...

By Hanxin Wang, Tian Liu, Shu Kong

llms fine-tuning multimodal benchmarks

arXiv Computer Vision

Oct 15

Measuring Epistemic Humility in Multimodal Large Language Models

arXiv:2509.09658v3 Announce Type: replace Abstract: Hallucinations in multimodal large language models (MLLMs), where a model generates content inconsistent with the input image, pose significant ris...

By Bingkui Tong, Jiaer Xia, Sifeng Shang, Kaiyang Zhou

llms multimodal benchmarks safety

arXiv Computer Vision

Oct 15

Chain of Time: In-Context Physical Simulation with Image Generation Models

arXiv:2511.00110v2 Announce Type: replace Abstract: We propose a novel cognitively-inspired prompting method to stabilize and interpret the physical simulation produced by autoregressive image genera...

By YingQiao Wang, Eric Bigelow, Boyi Li, Tomer Ullman

llms diffusion fine-tuning benchmarks

arXiv Computer Vision

Oct 15

Video-Browser: Towards Open-web Agentic Video Browsing

arXiv:2512.23044v3 Announce Type: replace Abstract: Autonomous agents are shifting information seeking from passive retrieval toward proactive, open-ended research, yet this progress has largely bypa...

By Zhengyang Liang, Yan Shu, Xiangrui Liu, Minghao Qin, Nicu Sebe, Jiaxin Mao, Zheng Liu, Lizi Liao

llms agents benchmarks

arXiv Computer Vision

Oct 15

Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework

arXiv:2603.08491v3 Announce Type: replace Abstract: Cross-modal Geo-localization (CMGL) matches ground-level text descriptions with geo-tagged aerial imagery, which is crucial for pedestrian navigati...

By Yutong Hu, Jinhui Chen, Chaoqiang Xu, Yuan Kou, Sili Zhou, Shaocheng Yan, Pengcheng Shi, Qingwu Hu, Jiayuan Li

llms multimodal benchmarks

arXiv Computer Vision

Oct 15

Capability $\neq$ Interpretability: Human Interpretability of Vision Foundation Models

arXiv:2605.20337v2 Announce Type: replace Abstract: How interpretable are the features of leading vision models? The question is increasingly pressing as these models move from research benchmarks in...

By Julien Colin, Lore Goetschalckx, Nuria Oliver, Thomas Serre

llms benchmarks safety

arXiv Computer Vision

Oct 15

PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models

arXiv:2606.16569v2 Announce Type: replace Abstract: Registering two captures of the same indoor space taken at different times underpins persistent spatial memory for robots and AR systems, yet the r...

By Zhiang Chen, Nahyuk Lee, Boyang Sun, Taein Kwon, Marc Pollefeys, Zuria Bauer, Sunghwan Hong

llms computer-vision robotics multimodal benchmarks

arXiv Computer Vision

Oct 15

SCVIB: Editable State-Conditioned Visual Instance Binding for Multi-Turn Personalized Localization

arXiv:2608.14148v2 Announce Type: replace Abstract: Stored visual evidence can remain unchanged while its relevance to a current request changes with interaction state. To study this state-dependent...

By Xiongtai Yang, Ziyan He, Tao Wang

multimodal benchmarks

arXiv Computer Vision

Oct 15

Reference-Efficient Multimodal Bitstream-Corrupted Video Recovery via Depth-Conditioned Adaptation

arXiv:2610.15806v2 Announce Type: replace Abstract: Recovering visually faithful content from damaged video bitstreams is important for reliable multimedia communication and storage. Existing methods...

By Tianyi Liu, Jiaojiao Wang, Ye Lu, Kejun Wu, Chengtao Cai, Zhengguo Li, Chia-Wen Lin

multimodal benchmarks

arXiv Computer Vision

Oct 15

EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agents

arXiv:2610.16679v2 Announce Type: replace Abstract: Spatial reasoning agents can inspect tool outputs and repair errors, but reusing these lessons requires preserving both the computation and the con...

By Bin Yu, Shijie Lian, Zhaolong Shen, Xiaopeng Lin, Yao Zhang, Haishan Liu, Haotian Li, Yuliang Wei, Kai Hu, Yichao Du, Zhirui Zhang, Bailing Wang, Kai Chen

agents fine-tuning benchmarks

arXiv Computer Vision

Oct 15

SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale

arXiv:2606.13497v2 Announce Type: replace-cross Abstract: This work introduces Spatial Annotations from Robot Demonstrations with Reliability Calibration (SPARC), a risk-aware framework that automati...

By Nils Blank, Paul Mattes, Maximilian Xiling Li, Jakub Suliga, Thomas Roth, Moritz Reuss, Pankhuri Vanjani, Rudolf Lioutikov

robotics benchmarks

arXiv Machine Learning

Oct 15

SNACK: A Sequential Notation Framework for Probabilistic Graph Generation

arXiv:2610.19781v1 Announce Type: cross Abstract: We introduce SNACK, a string representation for probabilistic graph generation that enables sequence models to generate and model graphs with discret...

By Hohyun Kim, Hyesung Kim, Min-hwan Oh, Seunggeun Lee

efficiency benchmarks

arXiv AI

Oct 15

SEAR: Self-Evolving Agentic Robotics

arXiv:2610.18792v1 Announce Type: cross Abstract: Frontier language-model agents can now improve robots on their own: they write control code, test it, and keep what works as skills and memories for...

By Xinyi Wang, Bangzheng Li, Hanyu Wang, Lin Zhang, Andrea Yaoyun Cui, Zhenggang Tang, Shyamal Buch, Dejia Xu

llms agents robotics multimodal benchmarks

arXiv Machine Learning

Oct 15

More Methods, Better Agents? A Reality Check on Self-Distillation in Agentic Reinforcement Learning

arXiv:2610.19789v1 Announce Type: cross Abstract: Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published bas...

By Linhao Zhong, Bozhen Fang, Xuehao Wang, Duochao Shi, Zekai Luo, Guangming Yao, Yi Yuan, Hao Chen, Jingdong Chen, Chunhua Shen

agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Oct 15

Model What Matters: Decision-Guided Selective World Modeling for Language Agents

arXiv:2610.19642v1 Announce Type: cross Abstract: Reinforcement learning provides a direct mechanism for optimizing task objectives in large language model-driven agents, but task rewards are typical...

By Maolin Ran, Minda Chen, Di Huang, Wei Xiong, Fei Huang, Weiwen Liu

llms agents nlp reinforcement-learning benchmarks

arXiv AI

Oct 15

Cite Pretrain: Retrieval-Free Knowledge Attribution for Large Language Models

arXiv:2506.17585v4 Announce Type: replace Abstract: Large language models can answer questions correctly and still hallucinate their citations. Most citation systems address this by retrieving docume...

By Yukun Huang, Sanxing Chen, Jian Pei, Manzil Zaheer, Bhuwan Dhingra

llms fine-tuning benchmarks

arXiv AI

Oct 15

Knowledge Graphs as Files: An Agent Harness Where Tool Creation and Tool Use Co-Evolve

arXiv:2610.18616v1 Announce Type: cross Abstract: Searching text-rich knowledge graphs requires following relations while examining textual evidence. Predefined graph interfaces constrain what eviden...

By Yuyang Bai, Yu Wang, Yu Zhang

llms agents benchmarks

arXiv AI

Oct 15

From Relation-Specific Heads to Shared Interventions: Contextual Entrainment in Language Models

arXiv:2610.18674v1 Announce Type: cross Abstract: Large language models rely on prompt context to perform in-context learning, follow formats, and adapt to local evidence. This ability, however, also...

By Jinghan Zheng, Jiajun Shi

llms benchmarks

Get the daily briefing