Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,306 stories · RSS feed

arXiv Computer Vision
Sep 24

Spatiality-Frequency Domain Video Forgery Detection System Based on ResNet-LSTM-CBAM and DCT Hybrid Network

The paper introduces a video forgery detection system that fuses spatial and frequency-domain features using a ResNet‑LSTM backbone with a Convolutional Block Attention Module (CBAM) and a Discrete Cosine Transform (DCT) module. Experiments on multiple benchmark datasets show that this hybrid architecture outperforms existing methods in distinguishing authentic from manipulated videos. Ablation and comparative studies confirm the individual contributions of each component, highlighting the model’s effectiveness across diverse forgery scenarios.

By Zihao Liao, Sheng Hong, Yu Chen
arXiv Computer Vision
Sep 24

VIVAS: Vitalizing Visual Perception in VLM Pre-training via Vision-language Unified Autoregressive Supervision

VIVAS is a new Vision‑Language Model pre‑training framework that addresses the lack of fine‑grained visual perception in existing VLMs. It introduces a unified token space and a dense‑structural‑semantic vision tokenizer that expands the textual vocabulary with visual tokens, enabling vision‑language unified autoregressive supervision over both visual details and linguistic content. Trained on 12.4 T tokens, VIVAS achieves state‑of‑the‑art results on 7 tasks and 39 multimodal benchmarks.

By Zhehan Kan, Yubo Zhu, Xinghua Jiang, Zhixiang Wei, Shifeng Liu, Wei Tong, Sheng Zhong, Qingmin Liao, Wenming Yang, Xin Li, Yinsong Liu, Deqiang Jiang, Xing Sun
arXiv Computer Vision
Sep 24

AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models

AstraLOD3 is a zero‑shot multimodal agentic system that reconstructs LOD3 building models using multi‑view images, calibrated cameras, a filtered sparse SfM point cloud, and a natural‑language specification. The Astra agent selects and executes computational steps in Python and Blender, achieving a mean FRDS of 0.9647 across 35 runs, including 24 benchmark buildings, with geometric agreement comparable to purpose‑built methods. Ablation studies show the impact of reconstruction guidance, evidence modalities, model configuration, and run‑to‑run variability, demonstrating that LOD3 reconstruction can be framed as a constrained agentic process rather than a fixed pipeline.

By Bryan G. Pantoja-Rosero
arXiv Computer Vision
Sep 24

TEEP-RCNN: Texture-Enhanced Edge-aware Perception for Steel Surface Defect Detection via Improved Convolutional Block Attention in Faster R-CNN

The paper introduces TEEP‑RCNN, a two‑stage detector that augments Faster R‑CNN with a Feature Pyramid Network backbone and an enhanced Convolutional Block Attention Module (CBAM) featuring dropout in the channel attention MLP and batch‑norm in the spatial attention branch. Training employs a differential learning‑rate schedule with cosine‑annealing warm‑up, and inference uses Test‑Time Augmentation combined with Weighted Box Fusion to stabilize localization of elongated and boundary‑adjacent defects. On the NEU‑DET benchmark, TEEP‑RCNN attains 73.3 % mAP@50 and 37.9 % mAP@50‑95 in only ten epochs on a single GPU, matching or surpassing YOLOv11m while excelling on the rolled‑in‑scale defect category under the COCO metric.

By Kirtan Rajesh
arXiv Computer Vision
Sep 24

A Unified Framework and Dataset for Oriented Object Visual Grounding in Remote Sensing

The paper introduces O$^2$-VG, a unified framework for oriented object visual grounding in remote sensing images, comprising three complementary models: O$^2$-VG-Trans, a cross‑modality transformer; O$^2$-VG-Uni, which predicts universal oriented proposals; and O$^2$-VG-VLM, an autoregressive vision‑language model that generates oriented bounding boxes. It also presents DIOR‑R‑SVG, a new dataset containing image, expression, and oriented box triplets for training and evaluation. The framework demonstrates superior performance across multiple benchmarks and is supported by publicly available code.

By Zeyu Ding, Yong Zhou, Jiaqi Zhao, Wen-Liang Du, Xixi Li, Hancheng Zhu, Rui Yao, Abdulmotaleb El Saddik
arXiv Computer Vision
Sep 24

BronchoTop: Bronchoscopy Navigation via RGB-Only Topological Localization

BronchoTop is a real‑time, RGB‑only framework that localises a bronchoscope within the bronchial tree without requiring patient‑specific CT scans or external sensors. It uses four modules—lumen detection and tracking, lumen‑branch label association, probabilistic scope location estimation, and switch verification—to estimate the scope’s position relative to a generic airway model. Evaluation on phantom, simulated, and real data shows state‑of‑the‑art accuracy, improving existing approaches by over 20% on real bronchoscopy sequences and providing the first publicly available framework with code and data for further research.

By Clara Tomasini, Ana Cristina Murillo, Luis Riazuelo
arXiv Computer Vision
Sep 24

Zero-Shot Object Removal via Attention Masking, Latent Anchoring, and Refinement

This paper presents a zero‑shot framework for removing objects from real images using a frozen Stable Diffusion model, avoiding any task‑specific training. The pipeline combines SAM‑based mask construction, BLIP caption conditioning, DDIM inversion, background‑weighted masked null‑text optimization, decoder self‑attention masking, hard outside‑mask latent anchoring, and localized renoise‑denoise refinement. Experiments show effective removal of objects and context‑consistent replacement, with background‑weighted NTI especially helpful for complex backgrounds and repeated refinement reducing residual artifacts.

By Arman Taghizadeh (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany), Ulf Krumnack (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany), Kai-Uwe K\"uhnberger (Institute of Cognitive Science, Osnabr\"uck University, Osnabr\"uck, Germany)
arXiv Computer Vision
Sep 24

MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos

MultiVENT‑Raw is a new multilingual benchmark comprising nearly 120,000 raw videos—continuous footage from cell phones, hand‑held cameras, or CCTV—totaling over 5,300 hours. The dataset includes 130 events and 222 event‑centric queries, along with human‑annotated relevance judgments and extracted key facts for relevant videos. It supports two tasks: retrieving videos relevant to a query event and generating a coherent report summarizing event‑related videos for a target user, with baseline models showing these tasks remain challenging.

By Reno Kriz, David Etter, Alexander Martin, Cameron Carpenter, Debashish Chakraborty, Hannah Recknor, Reihaneh Iranmanesh, Matthew Maciejewski, Kenton Murray, Eugene Yang, Benjamin Van Durme, Aaron Steven White, Andrew Yates, William Walden
arXiv Computer Vision
Sep 24

Bend the Clock: Predicting Ahead to Beat Latency in Event-Based Object Detection

The paper introduces ChronoFuse, a causal availability-time detector that predicts object states at the time its output becomes available rather than at the observation timestamp, addressing the latency mismatch in event-based multi-object detection. ChronoFuse performs lightweight cross-time fusion over a multi-scale feature hierarchy, adding only 0.17 M parameters and 0.84 ms latency overhead. It recovers a large portion of accuracy lost to latency, achieving up to 20.95 sAP on EV‑Flying data compared to 2.25 sAP for the strongest standard detector.

By Biswadeep Sen, Benoit R. Cottereau, Nicolas Cuperlier, Terence Sim
arXiv Computer Vision
Sep 24

ASAP: Visual Analytics for Identifying and Analyzing Image Patterns in AI-generated Images

ASAP is an interactive visualization system that helps users identify and analyze deceptive patterns in AI‑generated images. It uses a CLIP‑adapted image encoder to produce interpretable representations and generates masks that highlight influential pixel regions, enabling influence measurement of key deceptive features. The system integrates these techniques into a dashboard for quantifying authenticity‑indicative patterns across collections of authentic and AI‑generated images, supporting comparative analysis of different generative models such as GANs and diffusion models, and its effectiveness is demonstrated through a user study and benchmark applications.

By Jinbin Huang, Yuki Ueno, Chen Chen, Aditi Mishra, Bum Chul Kwon, Zhicheng Liu, Chris Bryan
arXiv Computer Vision
Sep 24

DAVIO: Dense Monocular-Inertial SLAM with Feed-Forward Initialization and Pose-Conditioned Mapping

DAVIO is a dense monocular‑inertial SLAM system that leverages a single multi‑view depth model (Depth Anything 3) for both initialization and mapping. It starts up quickly by solving a feature‑free linear system from a five‑image window and IMU pre‑integration, then uses a VIO filter whose metric poses condition the depth model during tracking. The system corrects residual scale along viewing rays, preserves metric baselines, and refines the map with a gravity‑preserving sub‑map graph, achieving earlier start‑up, lower localization error, and more accurate dense maps than state‑of‑the‑art feed‑forward mappers on both EuRoC and building‑scale ORI datasets.

By Jaafar Mahmoud, Arthur Movsesyan, Mikhail Iumanov, Sergey Kolyubin
arXiv Computer Vision
Sep 24

Semantic-Guided Fusion Network for Multi-Source Remote Sensing Image Classification

The paper introduces SGFNet, a Semantic‑Guided Fusion Network for classifying multi‑source remote sensing images. It features a Semantic Mixing Convolution Block that generates semantic‑aware kernels based on contextual relationships, and a Frequency Modulated Fusion Block that fuses cross‑modal information in the frequency domain to mitigate spatial misalignment. Experiments on the Augsburg and Houston 2018 datasets show SGFNet consistently outperforms state‑of‑the‑art methods.

By Yuwei Zhao, Chuanzheng Gong, Baogui Huan, Feng Gao, Junyu Dong, Qian Du
arXiv Computer Vision
Sep 24

Recursive Uncertainty-Gated Image Registration for Learning-based Algorithms

The paper introduces Recursive Uncertainty-Gated Image Registration (RUGI), an iterative refinement method that updates deformation fields predicted by learning-based registration models using a gating map. Two gating strategies are explored: an uncertainty-based approach and an image residual error approach, both concentrating updates on difficult regions. Experiments on cardiac MRI and echocardiography datasets show that RUGI consistently improves registration accuracy, with the error-gated variant reducing MSE by 27‑37% on pretrained models and lowering ejection fraction estimation errors.

By Clara Rodrigo Gonz\'alez, Oscar Bates, Fu Siong Ng, Meng-Xing Tang
arXiv Computer Vision
Sep 24

Copy-Move Forgery Detection and Question Answering for Remote Sensing Image

The paper introduces the Remote Sensing Copy-Move Question Answering (RSCMQA) task, aimed at interpreting complex tampering scenarios in land resource monitoring and national defense. It presents five datasets—RS-CMQA, RS-CMQA-B, Real-RSCM, RS-TQA, and RS-TQA-B—spanning 29 regions in 14 countries, and a region-discrimination-guided multimodal copy-move forgery perception framework (CMFPF) that improves question answering accuracy on tampered images. Experiments show CMFPF outperforms general VQA and RSVQA models, establishing a new benchmark for RSCMQA.

By Ze Zhang, Enyuan Zhao, Di Niu, Jie Nie, Xinyue Liang, Lei Huang
arXiv Computer Vision
Sep 24

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

The paper introduces RSMEB, a unified benchmark for remote‑sensing multimodal retrieval that evaluates both cross‑modal and interleaved retrieval across 21 tasks under a single ranking protocol. It also presents VLM2GeoVec, an instruction‑conditioned single‑encoder model that embeds image, text, bounding‑box, and geo‑coordinate tokens into one sequence and achieves state‑of‑the‑art performance on region‑caption, referring‑expression, and semantic geo‑aware retrieval while remaining competitive on conventional tasks. The authors provide code, checkpoints, and data on GitHub to facilitate reproducibility.

By Emanuel S\'anchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan, Yonghao Xu, Michael Felsberg
arXiv Computer Vision
Sep 24

CRISP: Compositional Relations as Invariant Structural Priors for Domain Generalization

CRISP (Compositional Relational Invariance from Spatial Primitives) is an image‑classification framework that decomposes visual recognition into primitive elements and their relational composition. It represents these compositions with soft unary, binary, and ternary predicates over primitive locations and appearance, enabling differentiable spatial and visual alignment learned end‑to‑end. Evaluated on five DomainBed datasets covering style, provenance, and camera‑trap shifts, CRISP achieves new state‑of‑the‑art performance on both benchmarks.

By Dat Nguyen, Duc-Duy Nguyen
arXiv AI
Sep 24

UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization

UniShield is a multi‑agent framework that unifies forgery image detection and localization across diverse domains such as image manipulation, document manipulation, DeepFake, and AI‑generated images. It combines a perception agent that analyzes image features to select appropriate detection models with a detection agent that integrates multiple expert detectors into a single system, producing interpretable reports. Experiments demonstrate that UniShield outperforms existing unified approaches and domain‑specific detectors, achieving state‑of‑the‑art performance and greater practicality, adaptiveness, and scalability.

By Qing Huang, Zhipei Xu, Xuanyu Zhang, Xiangyu Yu, Jian Zhang
arXiv AI
Sep 24

EnSIMem: Entity-Structured Indexing for Long-Term Agent Memory

EnSIMem is an entity‑structured long‑term memory architecture designed for agents that interact with users over extended periods. It organizes interactions into theme‑coherent episodes and creates dialogue‑grounded index entries of the form [entity][entity type][property:value], preserving source turns, temporal data, and multimodal fields. During online interaction, the agent decomposes requests into evidence requirements, performs entity‑property lookup, and retrieves the necessary evidence to generate responses directly from preserved source material rather than lossy summaries.

By Xuanyu Meng, Xing Fan, Xinyi Fan, Chenlei Guo, Yixuan Xie, Jiawei Han
arXiv AI
Sep 24

Ask Which, Not How Good: Sizing Benchmarks Scored by an LLM

The study analyzes 373,019 judgments from LLM‑scored benchmarks, decomposing variance into system, item, judge, and interaction components via generalizability theory. It finds that with a single judge, generalizability converges to a ceiling determined by the system‑by‑judge variance, which is substantially lower in pairwise preference settings, allowing one judge to suffice. The research also reveals significant biases in presentation order and highlights that many published win‑rate claims fall below the measured floor of the benchmarks.

By Atul Anand
arXiv AI
Sep 24

SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithms

SsgCaps is a publicly available dataset of human-engineered sound scenes, each paired with a precisely structured prompt that guides the sampling process. The prompts are drawn from a predefined action-based typology, enabling extensive yet plausible sampling. A comparative quantitative analysis shows only small differences between the open and private versions, supporting the recommendation of the open version for benchmarking sound scene generation algorithms.

By Modan Tailleur (LS2N), Junwon Lee (LS2N), Laurie M Heller (LS2N), Mathieu Lagrange (LS2N), Keunwoo Choi, Brian McFee, Keisuke Imoto, Yuki Okamoto