Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv AI

Oct 1

Large Knowledge Model: A Knowledge Foundation for Agentic Science at Scale

arXiv:2609.27297v2 Announce Type: replace Abstract: Agentic science envisions many autonomous agents investigating concurrently while building on a shared, evolving body of scientific knowledge. This...

By Yuan Huang, Sihan Hu, Hongyu Gu, Chao Ma, Jiaxing Zhang, Zhiyong Zou, Caiyu Fan, Yan Xiao, Mingjun Xu, Chenyu Xie, Mingzhen Ju, Zhehao Ma, Qi Zhang, Baozong Wang, Yu Li, Zhiyuan Yao, Ruoxue Liao, Xinyu Li, Linfeng Zhang, Kun Chen, Weinan E

agents benchmarks

arXiv AI

Oct 1

AneumoBench: A Source-Linked Benchmark for Synthetic-Geometry Transfer in Aneurysm CFD

arXiv:2505.14717v2 Announce Type: replace-cross Abstract: Scientific machine learning uses simulation data to train surrogate models for fast physical-field prediction across geometries. Local shape...

By Xigui Li, Yuanye Zhou, Feiyang Xiao, Xin Guo, Chen Jiang, Tan Pan, Xingmeng Zhang, Cenyu Liu, Zeyun Miao, Xiansheng Wang, Qimeng Wang, Yichi Zhang, Wenbo Zhang, Hongwei Zhang, Ruoxi Jiang, Fengping Zhu, Limei Han, Chensen Lin, Yuan Cheng

fine-tuning benchmarks

arXiv AI

Oct 1

Patch Rebirth: Fast and Transferable Model Inversion of Vision Transformers

arXiv:2509.23235v3 Announce Type: replace-cross Abstract: Model inversion is a widely adopted technique in data-free learning that reconstructs synthetic inputs from a pretrained model through iterat...

By Seongsoo Heo, Dong-Wan Choi

llms efficiency benchmarks

arXiv AI

Oct 1

From Compound Figures to Medical Multi-image Reasoning: Scaling Multimodal Large Language Models with Biomedical Literature

arXiv:2511.22232v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly capable in medical imaging, yet most focus on single-image settings. Clinical inter...

By Zhen Chen, Yihang Fu, Rong Zhou, Serina Applebaum, Min Kyu Kim, Aidan Gilson, Morten Lee, Salahudeen Mirza, Gabriel Madera, Mauro Giuffre, Yuanting Pan, Roy Jiang, Hyunjae Kim, Hua Xu, Qingyu Chen

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Oct 1

ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining

arXiv:2603.02767v4 Announce Type: replace-cross Abstract: Image--text contrastive pretraining has become a dominant paradigm for visual representation learning, yet existing methods often yield repre...

By Hanpeng Liu, Zidan Wang, Shuoxi Zhang, Zonglin Zhao, Zihao Bo, Rinyoichi Takezoe, Kaiwen Long, Yaqian Li, Kun He

multimodal benchmarks safety

arXiv AI

Oct 1

Language-Conditioned World Modeling for Visual Navigation

arXiv:2603.26741v2 Announce Type: replace-cross Abstract: Goal-conditioned visual navigation has been a long-standing testbed for embodied AI. We study a natural language-conditioned variant, languag...

By Yifei Dong, Fengyi Wu, Yilong Dai, Lingdong Kong, Guangyu Chen, Yetong Sha, Qiyu Hu, Feng Liu, Siyu Huang, Qi Dai, Zhi-Qi Cheng

agents diffusion robotics multimodal benchmarks

arXiv AI

Oct 1

MoEless: Efficient MoE LLM Serving with Serverless Experts

arXiv:2603.06350v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly adopt Mixture-of-Experts (MoE) architectures to scale efficiently under stringent resource constrai...

By Hanfei Yu, Bei Ouyang, Shwai He, Ang Li, Hao Wang

llms efficiency benchmarks

arXiv AI

Oct 1

VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images

arXiv:2604.09531v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) still struggle with visual perception tasks such as spatial understanding and viewpoint recognition, largely be...

By Guanyu Zhou, Yida Yin, Wenhao Chai, Shengbang Tong, Xingyu Fu, Zhuang Liu

llms diffusion reinforcement-learning fine-tuning multimodal benchmarks

arXiv Computation and Language

Oct 1

JuryFlow: Disagreement-Guided Human-in-the-Loop Multi-Agent Evaluation

arXiv:2609.40103v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as automated judges for AI-generated content, yet a single judge is unreliable and even a panel...

By Mufeng Yang, Junwei Yu, Yepeng Ding

llms agents benchmarks

arXiv Computation and Language

Oct 1

Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curves

arXiv:2609.40190v1 Announce Type: cross Abstract: Sampling several answers and keeping the one a verifier scores highest is one of the simplest ways to buy accuracy at test time. Its effect is report...

By Sohail (Neel), Sarkar, Shakuntala Baichoo

benchmarks

arXiv Computer Vision

Oct 1

VidHarness: Evolving Agent Harnesses for Cost-Efficient Long Video Understanding

arXiv:2609.38413v1 Announce Type: new Abstract: Vision-language models (VLMs) can answer questions about hour-long videos, but processing every frame is prohibitively expensive, even though the evide...

By Susan Liang, Jianmin Wu, Daxiang Dong

llms agents multimodal benchmarks

arXiv Computer Vision

Oct 1

ThinkV2V: Unleashing the Reasoning Capability of MLLMs for Instruction-Guided Video Editing

arXiv:2609.38541v1 Announce Type: new Abstract: Instruction-guided video editing has made significant progress, yet existing methods use multimodal large language models (MLLMs) primarily as semantic...

By Donghao Zhou, Haoyang He, Fan Zhang, Hao Yang, Guisheng Liu, Xin Gao, Zhongwei Wan, Xingyuan Bu, Jie Wang, Qiangpeng Yang, Shilei Wen, Chi-Wing Fu, Pheng-Ann Heng

llms multimodal benchmarks

arXiv Computer Vision

Oct 1

Decoupling Spherical Reasoning from Dense Prediction for 360 Depth Estimation

arXiv:2609.38856v1 Announce Type: new Abstract: The equirectangular projection (ERP) is widely used for panoramic depth estimation, but its spatially varying distortion makes geometry-consistent feat...

By Zhijie Shen, Chunyu Lin, Shuai Zheng, Feng Li, Runmin Cong, Huihui Bai, Yao Zhao

computer-vision benchmarks

arXiv Computer Vision

Oct 1

Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting

arXiv:2609.38979v1 Announce Type: new Abstract: Multiple object hallucination, where large vision-language models (LVLMs) generate objects not supported by the visual input, is a persistent challenge...

By Chang Liu, Yu Tian, Rui Xie

llms multimodal benchmarks safety

arXiv Computer Vision

Oct 1

DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistency

arXiv:2609.39096v1 Announce Type: new Abstract: Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows with the generated history. Existing compr...

By Zeqi Xiao, Qingle Liu, Kaiwen Zhang, Yifan Zhou, Zihan Ding, Xingang Pan

diffusion efficiency benchmarks

arXiv Computer Vision

Oct 1

How to Reduce Localization Ambiguity? Geometry-Semantic Constrained BEV Representation Learning for Satellite-Ground Localization

arXiv:2609.39127v1 Announce Type: new Abstract: Satellite-ground localization estimates the planar position and yaw orientation of a ground camera within a geo-referenced satellite image. Most recent...

By Junming Feng, Panwang Xia, Qiong Wu, Xudong Lu, Zeyu Jiao, Kun Lv, Zherong Wu, Yi Wan, Peifeng Ma, Li-Ta Hsu, Zhi Zheng

benchmarks

arXiv Computer Vision

Oct 1

Fiber-Resolved Microstructure Quantification from Multi-Shell Diffusion MRI using Detection Transformers

arXiv:2609.39184v1 Announce Type: new Abstract: Fiber orientation and compartmental microstructure are central to the characterization of white matter tissue in diffusion MRI, yet existing methods ei...

By Sebastian Endt, Marcus Wirth, Johannes Reinhold Schlund, Marion Irene Menzel

llms diffusion computer-vision benchmarks

arXiv Computer Vision

Oct 1

Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification

arXiv:2609.39492v1 Announce Type: new Abstract: Matching the same vehicle across front and rear cameras is difficult because the cameras do not share a view and the vehicle's appearance changes subst...

By Moseli Mots'oehli, Thulani Babeli

llms multimodal benchmarks

arXiv Computer Vision

Oct 1

TexTailor: Texture-Preserving Video Virtual Try-On via Adaptive Garment Conditioning

arXiv:2609.39335v1 Announce Type: new Abstract: Video virtual try-on has attracted increasing attention due to its broad potential in digital fashion and intelligent e-commerce. However, existing met...

By Zijing Qin, Jun Zhou, Ruicheng Zhang, Jiaqi Hou, Zunnan Xu, Ronghui Li, Zhenyu Xie, Xiu Li

llms diffusion multimodal benchmarks

arXiv Computer Vision

Oct 1

DensePed-Lite: Quality-Aware Adaptive Detection for Dense Pedestrians under Occlusion

arXiv:2609.39467v1 Announce Type: new Abstract: Pedestrian detection plays a crucial role in computer vision with applications in autonomous driving, surveillance, and public safety. However, real-wo...

By ZiAn Wang, MingZhe Liu, Chaoyi Guo, ChangChun Li, Fangming Gu

agents computer-vision robotics benchmarks safety

Get the daily briefing