Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,147 stories · RSS feed

arXiv Computer Vision

Oct 1

TSGL: Teacher-Student Graph Learning for 3DGS Compression

arXiv:2609.38635v1 Announce Type: cross Abstract: 3D Gaussian Splatting (3DGS) is a popular representation for novel view synthesis. However, 3DGS contains millions of Gaussian primitives, each with...

By Matin Bani Saedi, Matthew Kyan, Gene Cheung

benchmarks

arXiv Computer Vision

Oct 1

CountLoop: Training-Free High-Instance Image Generation via Iterative Agent Guidance

arXiv:2508.16644v5 Announce Type: replace Abstract: Diffusion models excel at photorealistic synthesis but struggle with object count fidelity, especially in high-density settings. We introduce COUNT...

By Anindya Mondal, Sauradip Nag, Ayan Banerjee, Josep Llados, Xiatian Zhu, Anjan Dutta

agents diffusion multimodal benchmarks

arXiv Computer Vision

Oct 1

Embedding Physical Reasoning into Diffusion-Based Shadow Generation Under the Sun and Sky

arXiv:2512.06174v3 Announce Type: replace Abstract: Generating realistic cast shadows for inserted foreground objects requires reasoning about scene geometry and illumination. However, most learning-...

By Shilin Hu, Jingyi Xu, Akshat Dave, Dimitris Samaras, Hieu Le

rag diffusion benchmarks

arXiv Computer Vision

Oct 1

Bake It Till You Make It: Ultrafast Spatial Texture-Atlas Splatting

arXiv:2607.13808v2 Announce Type: replace Abstract: Neural radiance representations in Gaussian Splatting (GS) deliver high-fidelity color detail but impose substantial rendering overhead from networ...

By Neel Kelkar, Simon Niedermayr, Kaloian Petkov, Klaus Engel, R\"udiger Westermann

efficiency benchmarks

arXiv AI

Oct 1

FFASR: Benchmarking Far-Field Automatic Speech Recognition using High-Fidelity Simulated RIRs

arXiv:2609.38897v1 Announce Type: cross Abstract: Far-field automatic speech recognition(ASR) degrades under reverberation, noise, and talker motion, yet the benchmarks that drive model selection emp...

By Shivam Saini, Eric Bezzam, Georg G\"otz, Alessia Milo, Steinar Gu{\dh}j\'onsson, Konstantinos Gkanos, Finnur Pind, Daniel Gert Nielsen

multimodal benchmarks

arXiv Computer Vision

Oct 1

ExpandDiff: Dynamic Range Expanding Diffusion for Single-Image HDR Reconstruction

arXiv:2609.39624v1 Announce Type: new Abstract: Single-image HDR reconstruction requires inferring missing detail while preserving the visible content of an LDR image. Differences in sensor dynamic r...

By Mehmet Emre and{\i}ran, Zhuoqian Yang, Liying Lu, Mathieu Salzmann, Sabine S\"usstrunk

diffusion benchmarks

arXiv AI

Oct 1

Grounding Time-Series Foundation Models in Digital Twin Topology for Predictive Maintenance

arXiv:2609.40071v1 Announce Type: cross Abstract: Digital twins increasingly support downstream analytical tasks that depend on time-series data, motivating interest in time-series foundation models...

By Sizhe Ma, Katherine A. Flanigan, Mario Berg\'es

benchmarks

arXiv AI

Oct 1

WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents

arXiv:2609.40325v1 Announce Type: new Abstract: As interactive 3D worlds are increasingly used to study intelligent behavior, it becomes important to develop efficient pipelines for identifying anoma...

By Ziyan Jiang, Jingbo Yang, Jiabao Ji, Yujian Liu, Qiucheng Wu, Tommi Jaakkola, Yang Zhang, Shiyu Chang

llms agents multimodal benchmarks

arXiv AI

Oct 1

MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Models

arXiv:2609.39107v1 Announce Type: new Abstract: Large Language Models (LLMs) have been applied in various fields. However, ensuring compliance and safety of LLMs, such as avoiding discrimination and...

By Yan Zhang, Chuming Wei, Ruien Li, Yaoyao Peng, Wusheng Zhang, Guangwen Yang

llms agents benchmarks safety

arXiv Computer Vision

Oct 1

Agentic Relative Camera Pose Estimation via Learned Ranking and Verification

arXiv:2609.38755v1 Announce Type: new Abstract: A wide range of approaches have been developed for camera pose estimation, including correspondence-based methods, end-to-end pose regression, and rece...

By Zhining Gu, Shangjie Du, Weimin Qiu, Carl Olsson, Ping Liu, Meng Tang

agents computer-vision multimodal benchmarks

arXiv AI

Oct 1

Cascadia: A Control-Plane-Free Alternative to Hyperconverged AI Infrastructure

arXiv:2609.38697v1 Announce Type: cross Abstract: We present Cascadia, a system for serving large language models on fleets of commodity Intel AIPCs using their CPU, integrated-GPU, and NPU resources...

By Matias Parij, Pawan Paudel, Tate Berenbaum, Muthaiah Venkatachalam

llms efficiency benchmarks

arXiv Computer Vision

Oct 1

Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration

arXiv:2603.01623v2 Announce Type: replace Abstract: Diffusion models have become the dominant tool for high-fidelity image and video generation, yet are critically bottlenecked by their inference spe...

By Jiaqi Han, Juntong Shi, Puheng Li, Haotian Ye, Qiushan Guo, Stefano Ermon

llms diffusion efficiency benchmarks

arXiv AI

Oct 1

CARAT: Do Materials LLMs Reason or Recite?

arXiv:2609.38340v1 Announce Type: new Abstract: When a materials LLM answers a question about crystal structure, does it reason from the structure or copy an answer already printed in its input? Accu...

By Jiajun Wu, Jian Yang, Zixiang Ni, Zhenzhu Li, Bin Chong

llms fine-tuning benchmarks

arXiv AI

Oct 1

AgBench: Agentic AI Benchmarks for Personal AI Devices

arXiv:2609.38652v1 Announce Type: new Abstract: Agentic AI systems increasingly rely on cloud-hosted large language models for planning, tool use, and iterative execution, raising concerns about API...

By Yizhou Han, Di Wu, Dhananjay Saikumar, Blesson Varghese

llms agents benchmarks

arXiv AI

Oct 1

Talk2Agent: Benchmarking Voice Interfaces for Text Agents

arXiv:2609.38867v1 Announce Type: new Abstract: Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular in...

By Terumi Chiba, Guangzhi Sun, Zheqi Yuan, Chao Zhang

llms agents multimodal benchmarks

arXiv AI

Oct 1

SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale

arXiv:2609.38822v1 Announce Type: cross Abstract: Anthropic's Agent Skills package reusable procedural know-how for an LLM agent into SKILL.md directories, and open-source aggregations have grown pas...

By Guanqun Yang, Wenlong Zhang, Tian Shi, Ping Wang

llms agents benchmarks

arXiv AI

Oct 1

Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings

arXiv:2609.39344v1 Announce Type: cross Abstract: Speech transcripts used as long-term memory must preserve both words and stable speaker identities. Existing meeting-transcription metrics either ign...

By Shantanu Vispute, Aditya Mishra, Siddhartha Saxena

benchmarks

arXiv AI

Oct 1

MAE I Trust Myself? Self-Evaluating VLA Action Generation with Markov Attention Entropy

arXiv:2608.16697v2 Announce Type: replace Abstract: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heter...

By Aniri, Chen Yilin, Jinhe Bi, Zengjie Jin, Yujun Wang, Yijun Tian, Volker Tresp, Fei Shen, Tat-Seng Chua, Yunpu Ma

multimodal benchmarks

arXiv AI

Oct 1

3D Software Synthesis Driven by Constraint-Expressive Intermediate Representation

arXiv:2507.18625v3 Announce Type: replace-cross Abstract: Graphical user interface (UI) software has undergone a fundamental transformation from traditional two-dimensional (2D) desktop/web/mobile in...

By Shuqing Li, Anson Y. Lam, Yun Peng, Wenxuan Wang, Michael R. Lyu

benchmarks

arXiv AI

Oct 1

PISCO: Precise Video Instance Insertion with Sparse Control

arXiv:2602.08277v3 Announce Type: replace-cross Abstract: The landscape of AI video generation is undergoing a pivotal shift: moving beyond general generation - which relies on exhaustive prompt-engi...

By Xiangbo Gao, Renjie Li, Xinghao Chen, Yuheng Wu, Suofei Feng, Qing Yin, Zhengzhong Tu

diffusion benchmarks

Get the daily briefing