Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,127 stories · RSS feed

arXiv Computer Vision
Oct 21

ICIBS: Increasing the Context for Ice Boundary Segmentation

The paper introduces ICIBS, a deep‑learning approach that enhances ice‑boundary segmentation by incorporating additional spatial and geo‑spatial context into the model. It reports state‑of‑the‑art performance on the IceAnatomy benchmark, achieving mean errors of 10.9 m (FAU subset) and 43.4 m (CReSIS subset), and demonstrates strong generalisation to new Patagonia data, outperforming baselines by 49.6 %. The authors provide the model code and dataset publicly on GitHub and Zenodo.

By Marcel Dreier, Dakota Pyles, Nora Gourmelon, Moritz Koch, Fei Wu, Thorsten Seehaus, Matthias Braun, Andreas Maier, Vincent Christlein
arXiv Computer Vision
Oct 21

In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification

The paper introduces PRAISE, a test‑time adaptation framework for vision‑language model based person re‑identification. PRAISE constructs an Attribute‑Conditioned Subspace (ACS) from a bank of attribute prompts, reshaping query‑gallery affinities without requiring identity or attribute labels. Experiments on image‑to‑image and text‑to‑image ReID benchmarks show that PRAISE consistently improves retrieval performance over existing backbones and TTA baselines.

By Rajarshi Bhattacharya, Aryan Shukla, Shakeeb Murtaza, Christian Desrosiers, Jose Dolz, Maguelonne Heritier, Eric Granger
arXiv Computer Vision
Oct 21

Efficient SAM3 Adaptation for Multi-Class Semantic Segmentation via Dense Competitive Representations

The paper introduces DCR‑SAM, a framework that adapts the Segment Anything Model 3 (SAM3) for multi‑class semantic segmentation. By decoupling cross‑modal interactions, aligning textual concepts with visual representations, and adding a dense competition head and learnable background token, DCR‑SAM reduces inference latency, mitigates semantic misclassifications, and suppresses overlapping predictions. Experiments show 2.3%–10.7% performance gains, a 33× speedup over native SAM3, and strong robustness across benchmarks.

By Wenbin Liao, Hao Zhu, Yike Ma, Hao Jiang, Feng Dai
arXiv Computer Vision
Oct 21

GAMF-Net: Grouped Availability-Aware Multimodal Fusion for Smartphone Skin Lesion Classification

GAMF-Net is a grouped availability‑aware multimodal fusion network designed for smartphone skin lesion classification. It divides ten clinical variables into three semantic groups—demographics, history‑related risk, and lesion presentation—and encodes each group with field values and availability masks. The image branch supplies base logits while each clinical group contributes image‑conditioned residual evidence; only available groups are fused with equal weight, and the model defaults to image‑only when no clinical data are present. Evaluated on PAD‑UFES‑20, GAMF-Net achieves an accuracy of 74.17 % and outperforms image‑only and other multimodal baselines, with ablation studies highlighting the importance of clinical information and constrained equal‑weight fusion.

By Yue Zhuo, Jingjia Yan, Fangyu Liu, Lingjun Liu, Zongjun Chen, Zhisheng Luo, Zhonghua Xie
arXiv Computer Vision
Oct 21

TrajTok: Learning Trajectory Tokens enables better Video Understanding

TrajTok introduces an end‑to‑end video tokenizer that replaces traditional patchification with a lightweight, trajectory‑based tokenization process. It performs implicit clustering over space‑time pixels to generate object trajectories in a single forward pass, dynamically adjusting token granularity to semantic complexity while remaining independent of video duration. The resulting TrajViT2 model outperforms existing methods on classification and retrieval benchmarks, and TrajTok can also serve as a probing head (TrajAdapter) or alignment connector (TrajVLM) for vision‑language tasks, notably improving long‑video reasoning.

By Chenhao Zheng, Jieyu Zhang, Jianing Zhang, Weikai Huang, Ashutosh Kumar, Quan Kong, Oncel Tuzel, Chun-Liang Li, Ranjay Krishna
arXiv Computer Vision
Oct 21

Vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis

The paper introduces VasoMIM, a vascular anatomy‑aware masked image modeling framework for X‑ray angiogram analysis, which uses an anatomy‑guided masking strategy and an anatomical consistency loss to learn robust vascular representations. It also presents XA‑170K, the largest pre‑training dataset for X‑ray angiograms. Experiments on six datasets and four downstream tasks show that VasoMIM outperforms existing methods and achieves state‑of‑the‑art performance.

By De-Xing Huang, Chaohui Yu, Xiao-Hu Zhou, Tian-Yu Xiang, Qin-Yi Zhang, Mei-Jiang Gui, Rui-Ze Ma, Chen-Yu Wang, Nu-Fang Xiao, Fan Wang, Zeng-Guang Hou
arXiv Computer Vision
Oct 21

HERMES++: Toward a Unified Driving World Model for 3D Scene Understanding and Generation

HERMES++ is a unified driving world model that simultaneously performs 3D scene understanding and future geometry prediction. It uses a BEV representation to merge multi‑view spatial data for LLM compatibility, introduces LLM‑enhanced world queries for knowledge transfer, and employs a Current‑to‑Future Link to condition future geometry on semantic context. A Joint Geometric Optimization strategy enforces structural integrity by combining explicit geometric constraints with implicit latent regularization, and extensive benchmark tests show that HERMES++ outperforms specialist methods in both point cloud prediction and scene understanding.

By Xin Zhou, Dingkang Liang, Xiwu Chen, Feiyang Tan, Dingyuan Zhang, Hengshuang Zhao, Xiang Bai
arXiv Computer Vision
Oct 21

Only Train Once: Uncertainty-Aware One-Class Learning for Face Authenticity Detection

The paper introduces FADNet, a face authenticity detector that treats forgery detection as a one‑class classification problem, training only on authentic faces. It uses Evidential Deep Learning to measure uncertainty and a pseudo‑forgery generator to sharpen decision boundaries. Experiments on DF40 and ASFD show FADNet surpasses state‑of‑the‑art methods with 96.63% accuracy and 98.83% precision.

By Qingchao Jiang, Zhenxuan Hou, Zhiying Zhu, Zhenxing Qian, Xinpeng Zhang, Zaiwang Gu
arXiv Computer Vision
Oct 21

TAMF-VTON: Token-Wise Sparse Adaptation for Mask-Free Virtual Try-On

TAMF-VTON is a diffusion-based virtual try‑on system that eliminates the need for segmentation masks, parsing maps, pose representations, or spatial controls during inference. It uses token‑wise sparse Mixture‑of‑Experts adaptation to provide input‑dependent capacity, along with timestep‑weighted frequency supervision and adaptive offline data construction to improve high‑frequency reconstruction and cross‑category training. The method achieves top performance on single‑garment VITON‑HD and DressCode benchmarks, supports multi‑garment transfer, and can generate 1K‑resolution images in under 15 seconds on an RTX 4090.

By Jie Wang, Qian He, Gaofeng He, Xiaogang Jin, Huamin Wang
arXiv Computer Vision
Oct 21

DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving

DriveVLA-M0 introduces a failure-aware latent memory system for Vision‑Language‑Action models in autonomous driving. It stores past failure cases with scene representations and expert trajectories, and uses a retrieval model to decouple static road structure from dynamic agent interactions. During inference, retrieved cases are injected via a lightweight LoRA‑based test‑time training mechanism, yielding significant performance gains on NAVSIM benchmarks with minimal latency overhead.

By Zebin Xing, Yupeng Zheng, Qiang Chen, Linbo Wang, Yichen Zhang, Pengxuan Yang, Junli Wang, Deheng Qian, Xiaoqing Ye, Junyu Han, Yifeng Pan, Qichao Zhang, Dongbin Zhao
arXiv Machine Learning
Oct 21

Specter: Fast MoE Inference via Quantization-Enabled Multi-Step Speculative Offloading

Specter is an inference system for mixture-of-experts (MoE) models that uses quantization to enable speculative decoding and expert offloading. By running a lightweight quantized draft model on the device, Specter predicts future expert usage and pre‑organizes expert movement, reducing PCIe I/O bottlenecks. The approach yields an average 1.50× speedup (up to 1.88×) in time per output token compared to state‑of‑the‑art systems, making single‑GPU MoE inference more practical on commodity hardware.

By Wenfeng Wang, Xiaofeng Hou, Xinfeng Xia, Xinkai WAng, Mingxuan Zhang, Peng Tang, Jiacheng Liu, Chao Li
arXiv AI
Oct 21

Vulnerability Analysis for Safe Multi-Agent Reinforcement

The paper examines inference‑time observation attacks on Safe Multi‑Agent Reinforcement Learning (SMARL), where an adversary perturbs observations of some agents to alter their actions. It introduces the MinMax Attack framework, which jointly selects adversarial target actions for compromised agents while considering the cooperative resistance of benign agents. The authors derive a bound on the induced safety cost and show through experiments that MinMax Attack causes higher safety costs than existing baselines while maintaining task reward, thus improving attack effectiveness and stealthiness.

By Taozheng Zhu, Shixiong Jiang, Fanxin Kong
arXiv AI
Oct 21

Unsupervised Tool Evolution for Coding Agents in Data-Lake Analytics

The paper introduces TEAL, an unsupervised framework that allows coding agents to automatically build and refine reusable tools for data‑lake analytics without requiring labeled questions. TEAL employs an Explorer that proposes tools from the data lake and a Critic that evaluates proposals based on the agent’s own execution traces, retaining only changes that the agent adopts without extra steps. Experiments on KramaBench and LakeQA show that TEAL consistently improves performance across multiple LLM workers, especially under tight step budgets, and that synthetic question generation (TEAL‑Q) can match real‑question evolution.

By Yuqing Yang, Yurong Liu, Deqing Fu, Manav Garg, Sravan Babu Bodapati, Fatma Ozcan, Alon Y. Halevy, Jiani Zhang