Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Computer Vision
Sep 23

SPEANet: Structural Prior Enhanced Attention Network for Parameter-Efficient Remote Sensing Object Detection

SPEANet is a parameter‑efficient backbone for remote sensing object detection that incorporates fixed structural operators to enhance contour and frequency responses. It uses stage‑specific prior extraction and context‑conditioned response modulation, assigning smoothed contour and directional modeling to shallow features and a compact approximation‑detail interaction to deeper stages. Experiments on five benchmarks, including DOTA‑v1.0, DOTA‑v1.5, and DIOR‑R, show that SPEANet achieves high mAP scores (up to 78.55%) with only 23.0 M total parameters, of which the backbone accounts for 5.97 M.

By Wei Lu, Junjie Li, Feifei Sang, Si-Bao Chen
arXiv Computer Vision
Sep 23

LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment

LLaVA‑Assessor is a unified large multi‑modal model (LMM) designed for visual quality assessment, combining image and video inputs. It introduces a two‑task framework—quality interpretation and quality scoring—supported by an adaptive architecture, a rigorous human‑annotated dataset, and a machine‑synthesized data expansion pipeline. The model employs a prompt‑disentanglement strategy to stabilize multi‑task training and achieves strong performance across 11 quality scoring test sets and 4 interpretation benchmarks.

By Ziheng Jia, Zicheng Zhang, Jiaying Qian, Guangtao Zhai, Xiongkuo Min
arXiv Computer Vision
Sep 23

The Temporal Moderation Gap: Text-to-Video Safety Filters Are Blind to Harm in Motion

The paper identifies a flaw in current text-to-video safety filters that rely on keyword prompts and per-frame checks, showing that these systems miss harms that arise solely from the ordering of frames. Experiments reveal that a significant portion of generated videos contain harmful content that is undetected because each individual frame appears safe, and prompt engineering does not mitigate this issue. The authors propose an order-aware detector that can distinguish harmful videos from benign shuffles, achieving an AUC of 0.74 compared to chance for per-frame checking.

By Yuxin Cao, Fusen Guo, Yuezhong Wu, Huadong Mo, Wei Song
arXiv Computer Vision
Sep 23

Code Plans, Diffusion Renders: Open-Ended Generative World Modeling

The paper introduces CoDeR, a new paradigm for world modeling that explicitly builds an executable world using code rather than relying solely on visual observations. CoDeR translates high‑level concepts into structured world rules, executable dynamics, and perceptual observations through five complementary roles, enabling long‑term memory, open‑ended interactions, autonomous world evolution, and persistent multi‑agent dynamics. Experiments show that this framework extends the capabilities of existing world models and achieves state‑of‑the‑art performance across multiple evaluation settings.

By Zixun Fang, Yawen Shao, Kai Zhu, Jie Xiao, Shihan Chen, Yu Liu, Xueyang Fu, Yang Cao, Wei Zhai, Zheng-Jun Zha
arXiv Computer Vision
Sep 23

Semantically-Guided Domain Randomization for Industrial Object Detection in Low-Image-Budget Regimes

Semantically-Guided Domain Randomization (S‑GDR) is an annotation‑free pipeline that uses vision‑language model captioning of a small real reference set, diffusion‑based background synthesis, and mask‑based object composition to generate synthetic training data. In a high‑mix, low‑volume automotive detection benchmark, S‑GDR achieves a mAP50‑95 of 0.739 with only 200 synthetic images, outperforming a domain‑randomized render baseline and several other synthetic data methods under the same budget. These results suggest S‑GDR is a viable alternative for training visual perception systems when annotation, energy, and time resources are severely limited.

By Jose Moises Araya-Martinez, Gautham Mohan, Jens Lambrecht
arXiv Computer Vision
Sep 23

GeoComposer: Geometry-Grounded Photographic Composition Instruction

GeoComposer is a new framework that improves photographic composition by generating textual guidance and visual exemplars for a given image. It uses a geometry-aware representation learning mechanism that incorporates geometric priors from a visual geometry foundation model to maintain global structure and fine-grained correspondences. A reinforcement learning strategy with a hybrid reward optimizes instruction following, aesthetic quality, and geometric consistency, leading to superior results compared to state‑of‑the‑art methods.

By Shuangzhi Li, Qiaoqiao Jia, Xingxin Chen, Guile Wu, Dongfeng Bai
arXiv Computer Vision
Sep 23

GAD-MambaUNet: Direction-Group Mamba with Gradient-Adaptive DINOv3 Distillation for Lightweight Medical Image Segmentation

The paper introduces GAD-MambaUNet, a lightweight medical image segmentation network that integrates efficient local modeling, Direction-Group Graph Selective Scan (DG‑GSS) for structured information exchange, and training‑time supervision from a frozen DINOv3 teacher with Gradient‑Adaptive Distillation. GAD‑MambaUNet demonstrates a strong accuracy‑efficiency trade‑off compared to other lightweight and general segmentation methods, and ablation studies confirm the benefits of DG‑GSS and DINOv3‑GAD supervision. Future work aims to refine teacher‑student alignment and apply the framework to multi‑class and multi‑modal medical segmentation tasks.

By Fang Wang, Huitao Li, Wenhan Chao, Zheng Zhuo, Xinxin Yang
arXiv Computer Vision
Sep 23

NPLSD: Accelerating Line-Segment Detection on NPU Microcontrollers

The paper introduces NPLSD, a pair of line‑segment detectors optimized for the Neural‑ART NPU on STM32N6 microcontrollers. NPLSD‑H retains a convolutional backbone and replaces transformer components with a fully‑convolutional head, while NPLSD‑M adapts a lightweight trunk to the NPU’s supported operators. Trained on ImageNet and ShanghaiTech Wireframe, the models achieve competitive accuracy with 2.63 M and 0.62 M parameters, respectively, and ablation studies show initialization contributes significantly to performance.

By Parsa Hassani Shariat Panahi, Amir Hossein Jalilvand, M. Hassan Najafi
arXiv Computer Vision
Sep 23

TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

arXiv:2505.01583v2 Announce Type: replace Abstract: Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLM...

By Jen-Hao Cheng, Yi-Hao Peng, Huapeng Zhou, Vivian Wang, Huayu Wang, Hsiang-Wei Huang, Wenhao Chai, Hou-I Liu, Kuang-Ming Chen, Cheng-Yen Yang, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang
arXiv Computation and Language
Sep 23

MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis

MultiViewDx is a physician‑validated multimodal instruction dataset that links medical imaging studies with patient context and normalizes heterogeneous reports into an evidence‑linked workflow (evidence → findings → differential discussion → diagnosis). The dataset covers a wide range of imaging modalities and uses a unified image‑text retriever to ensure that instruction synthesis is grounded in source‑supported evidence. Fine‑tuned models on MultiViewDx achieve the highest average accuracy on four MedVQA benchmarks and receive the strongest overall rating on JAMA Clinical Challenge cases, with ablations confirming the importance of case‑level multi‑view organization and evidence‑linked reasoning.

By Junda Wang, Zonghai Yao, Yujan Ting, Eric Z. Chen, Hieu Tran, Hong Yu, Weijing Huang, Terrence Chen