Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv Computer Vision

Oct 19

Direct Conditional Parameterization for N-Dimensional Splatting

arXiv:2610.22778v1 Announce Type: new Abstract: N-dimensional splatting extends 3D Gaussian Splatting with view direction and time to model view-dependent and dynamic effects. Each primitive is lifte...

By Zhongpai Gao, Benjamin Planche, Van Nguyen Nguyen, Meng Zheng, Anwesa Choudhuri, Arun Innanje, Terrence Chen, Ziyan Wu

benchmarks

arXiv Computation and Language

Oct 19

FrenchNews-7: Benchmarking Cross-Publisher French News Editorial Desk Classification

arXiv:2608.18097v2 Announce Type: replace Abstract: We present FrenchNews-7, a cross-publisher France-based French-language news editorial desk classification benchmark combining a large multi-outlet...

By Amr Sobhy

llms fine-tuning benchmarks

arXiv Computer Vision

Oct 19

Event-STVG: Spatio-Temporal Video Grounding for Long-Form Event Reasoning

arXiv:2610.22444v1 Announce Type: new Abstract: Spatio-Temporal Video Grounding (STVG) aims to jointly localize a target in space and time given a natural-language query. Despite continued advances,...

By Yizhe Zhang, Xujia Wang, Songqi Tang, Xujie Tang, Leo Jiahao Jiang, Jiacheng Wang, Bowen Qu, JinBo Li, Qin Wang, Ji Qi, Lei Hou, Bin Xu, Juanzi Li

llms multimodal benchmarks

arXiv Computer Vision

Oct 19

Reliability-Consistent Appearance Modeling for RGB-Infrared Multi-Object Tracking

arXiv:2610.23125v1 Announce Type: new Abstract: Recent RGB-infrared multi-object tracking (MOT) methods have substantially improved cross-modal representation learning and multimodal information inte...

By Bingyang Wang, Jiazuo Yu, Lu Zhang, Lihe Zhang, Huchuan Lu, You He

multimodal benchmarks

arXiv Computer Vision

Oct 19

mmPencil: Toward Writing-Style-Independent In-Air Handwriting Recognition via mmWave Radar and Large Vision-Language Model

arXiv:2610.23419v1 Announce Type: cross Abstract: In-air handwriting recognition based on radio frequency (RF) signals offers a promising solution for human-computer interaction, yet existing methods...

By Yifan Guo, Zhu Wang, Qian Qin, Yangqian Lei, Qiwen Gan, Zhuo Sun, Chao Chen, Bin Guo, Zhiwen Yu

llms multimodal benchmarks

arXiv Computer Vision

Oct 19

CCDF: A Benchmark Dataset for Deepfake Detection in Surveillance Footage

arXiv:2610.07939v2 Announce Type: replace Abstract: Due to rapid advances in Generative AI, commercial video generation tools can be used to produce fabricated surveillance footage that can fool both...

By Baptiste Chopin, Thomas Swearingen, Arun Ross, Antitza Dantcheva, Christian Rathgeb

diffusion benchmarks

arXiv Computer Vision

Oct 19

EgoTac: In-the-wild Tactile Prediction from Egocentric Vision

arXiv:2608.15060v2 Announce Type: replace Abstract: Touch is fundamental to dexterous manipulation, yet most egocentric human data increasingly used for robot learning lacks tactile information. Dire...

By Wenkang Zhang, Chengbo Yuan, Zicheng Zhang, Zhengxue Cheng, Yang Gao

robotics benchmarks

arXiv Computer Vision

Oct 19

Learning to Correct Geometry in Generated Videos

arXiv:2610.23856v1 Announce Type: new Abstract: Generated videos often lose geometric consistency under camera motion: structures warp under viewpoint change, parallax becomes inconsistent, and smoot...

By Weijie Lyu, Tiancheng Shen, Jason Li, Yujing Wang, Xiaoyuan Wang, Haobo Yuan, Yizhou Zhao, Ming-Hsuan Yang

computer-vision benchmarks

arXiv Computer Vision

Oct 19

Edit-Your-Interest: Efficient Video Editing via Feature Most-Similar Propagation

arXiv:2510.13084v2 Announce Type: replace Abstract: Text-to-image diffusion models have advanced video editing. However, existing video editing methods are limited by their high computational overhea...

By Yi Zuo, Zitao Wang, Lingling Li, Xu Liu, Fang Liu, Licheng Jiao

diffusion benchmarks

arXiv AI

Oct 19

MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios

arXiv:2610.22869v1 Announce Type: new Abstract: LLM agents are increasingly integrated into real-world works, yet they must handle diverse domain-specific tools and complex multi-sub-task problems th...

By Bowen Ding, Qixin Wang, Lexing Ruan, Weiyi Lu, Qiwei Ge, Ruishan Fang, Tao Lin

llms agents benchmarks

arXiv AI

Oct 19

FuzzInsight: Navigating the Efficiency-Effectiveness Trade-off in Coverage-Guided DNN Fuzzing

arXiv:2610.23195v1 Announce Type: new Abstract: Collecting and labeling diverse test datasets for evaluating Deep Neural Networks (DNNs) is costly and time-consuming. This has motivated coverage-guid...

By Vivek Vekariya, Fathiyeh Faghih, Alexander Pretschner

benchmarks

arXiv AI

Oct 19

CoTA++: Understanding the Repeat Curse in dMLLMs from an Information Flow Perspective

arXiv:2610.23290v1 Announce Type: new Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iterative denoising with bidirectional attention, which precludes the standard auto...

By Qiyan Zhao, Xiaofeng Zhang, Xu-Yao Zhang, Shijie Han, Zhepeng Li, Shunzhi Zhu, Da-Han Wang

llms diffusion multimodal benchmarks

arXiv AI

Oct 19

From Trajectories to Differentiated Skills: Cognitive Differentiation for Self-Improving Agents

arXiv:2610.23599v1 Announce Type: new Abstract: Large language model (LLM) agents tackle complex tasks through reasoning and interaction with external environments. Agent skills provide persistent, t...

By Yewei Zhao, Derong Xu, Wei Chen, Yin Xu, Peilun Zhou, Xiaohan Wang, Guojun Yin, Wei Lin, Mingjun Xiao

llms agents benchmarks

arXiv AI

Oct 19

Sparse Autoencoders Can Induce Local Geometry for Mitigating Catastrophic Forgetting

arXiv:2610.22439v1 Announce Type: cross Abstract: Catastrophic forgetting remains a challenge in supervised continual learning, as adapting to new tasks causes representation drift and loss of previo...

By Kamil Faber, Katarzyna Filus, {\L}ukasz Pawela

benchmarks

arXiv AI

Oct 19

Learning to Be Guided: Shaping Latent Geometry for Classifier-Free Guidance

arXiv:2610.22885v1 Announce Type: cross Abstract: Classifier-free guidance (CFG) is a standard component of conditional diffusion models, and guided generation performance has become the \textit{de f...

By Haoming Liu, Junhao Zhu, Shaoyi Zheng, Yuanhe Guo, Jinnuo Liu, Shenji Wan, Hongyi Wen

rag diffusion benchmarks

arXiv AI

Oct 19

FlashReFocus: Interactive Image Refocusing in Seconds

arXiv:2610.23297v1 Announce Type: cross Abstract: Single-image refocusing changes the focal plane and depth of field of a photograph after capture. It is inherently interactive: users try many focal...

By Ching-Heng Cheng, Chia-Ming Lee, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung Hsu

diffusion benchmarks safety

arXiv AI

Oct 19

TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models

arXiv:2610.23668v1 Announce Type: cross Abstract: Typed decision models often return more than a selected category. They also expose probabilities for each candidate and, sometimes, a separate confid...

By Yiqu Yang, Qiaoyuan Zheng, Yang Tian, Yu Fan, Elliott Ash

efficiency benchmarks safety

arXiv AI

Oct 19

Prompt Codebooks: Discrete Compositional Optimization for Language Model Instruction Refinement

arXiv:2605.28360v3 Announce Type: replace Abstract: Automatic prompt optimization (APO) has driven significant gains in LLM-based agentic workflows. However, most existing methods treat each task's p...

By Jyotirmoy Nath, Neeraj Kumar, Brejesh Lall

llms agents benchmarks

arXiv AI

Oct 19

Pluvio: Assembly Clone Search for Out-of-domain Architectures and Libraries through Transfer Learning and Conditional Variational Information Bottleneck

arXiv:2307.10631v2 Announce Type: replace-cross Abstract: The practice of code reuse is crucial in software development for a faster and more efficient development lifecycle. In reality, however, cod...

By Zhiwei Fu, Steven H. H. Ding, Furkan Alaca, Benjamin C. M. Fung, Philippe Charland

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Oct 19

Rethinking Negative Prompting for Concept Suppression: Unconditional Anchoring and Localized Guidance

arXiv:2412.07658v3 Announce Type: replace-cross Abstract: Text-to-image diffusion models have rapidly transitioned from research prototypes to widely deployed tools. Along with their impressive capab...

By Anubhav Jain, Takashi Shibuya, Yuya Kobayashi, Yuhta Takida, Nasir Memon, Julian Togelius, Yuki Mitsufuji

llms diffusion benchmarks safety

Get the daily briefing