Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,111 stories · RSS feed

arXiv Computer Vision
5d ago

Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition

The paper introduces ViT‑HarDis, a method that conditions attention head disentanglement on sample difficulty in Vision Transformers for face recognition. By applying stronger head decorrelation to hard samples—those with high cosine similarity to negative class centers—the approach encourages specialization where identity ambiguity is greatest. Experiments on WebFace4M and MS1MV3 show that ViT‑HarDis improves over baseline and uniform disentanglement, achieving state‑of‑the‑art results on IJB‑B, IJB‑C, and TinyFace without architectural changes or extra inference cost.

By Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira, Naser Damer, Fadi Boutros
arXiv Computer Vision
5d ago

Open-Weight MLLMs Cannot Compare What They Cannot Name

The paper investigates whether open‑weight multimodal large language models (MLLMs) can perform simple visual comparisons, specifically measuring body‑part sizes in synthetic ant images. It finds that while some body parts are compared accurately, others fail despite the task being visually trivial, indicating a grounding failure linked to the models’ inability to recognize certain structures. The study evaluates various in‑context learning strategies and shows that recognition‑driven grounding limits even basic visual comparisons that do not inherently require semantic understanding.

By Wenxin Su, Anna Kreshuk
arXiv Computer Vision
5d ago

MoETok: Sparse Expert Routing for 1D Visual Tokenizers

MoETok introduces a sparse expert routing mechanism for 1D visual tokenizers, enabling distinct Content and Detail queries to specialize in high‑level semantics and fine‑grained visual details, respectively. By jointly optimizing reconstruction and semantic objectives, the router learns role‑specialized processing paths that improve reconstruction, semantic representation, and autoregressive generation across various model scales and token budgets. On ImageNet, MoETok achieves state‑of‑the‑art class‑conditional generation within the 1D tokenizer family and, when scaled to a unified autoregressive model, matches or surpasses diffusion‑based models on benchmarks such as DPG‑Bench, GenEval2, and WISE.

By Jiahao Guo, Bao Tang, Haoxiang Cao, Jingfeng Yao, Bo Li, Wenyu Liu, Kun Gai, Changqian Yu, Xinggang Wang
arXiv Computer Vision
5d ago

Think in 360{\deg}: Benchmarking and Bootstrapping Panoramic Video Reasoning

The paper introduces PanoVideoBench, a manually annotated benchmark with 1,000 questions covering five categories and 15 question types for panoramic video understanding, and identifies cross‑view tracking and sphere‑time evidence integration as weak points in existing models. To address these gaps, the authors create PanoVideoCoT, a dataset of 13,465 QA‑CoT pairs generated via sphere‑time‑grounded synthesis, and train PanoVideoReasoner using panoramic CoT fine‑tuning and evidence‑privileged on‑policy self‑distillation. The resulting model achieves 55.7% accuracy on PanoVideoBench, improving the baseline by 12.5 points and surpassing the previous state‑of‑the‑art by 2.6 points, particularly in spatial, cross‑view, and temporal reasoning.

By Lidong Lu, Lidong Zou, Xinyi Mao, Jiaxin Lv, Zhenquan Li, Liangrui Dong, Tong Lu, Guo Chen
arXiv Computer Vision
5d ago

Inter-Frame Predictive Contour Coding for Semantic Map Compression

The paper introduces an inter‑frame predictive contour coding framework for lossless compression of dynamic semantic maps. It predicts blob positions and label indices from a reference frame, then performs contour prediction using chain‑code units from the reference contours, and selects between intra and inter coding based on estimated coding cost. Experiments on the VSPW dataset show the method outperforms existing lossless semantic map compression techniques, achieving a 3.0% average bitrate reduction without a significant increase in encoding time.

By Runyu Yang, Hyomin Choi, Fabien Racap\'e, Ivan V. Baji\'c
arXiv Computer Vision
5d ago

Bayesian PET Reconstruction with Learned Flow Matching Priors via Langevin Sampling

The paper presents a Bayesian framework for positron emission tomography (PET) reconstruction that uses a flow-matching generative prior trained on high‑quality PET images. By formulating reconstruction as a Bayesian inference problem, the authors employ latent‑space posterior sampling with the unadjusted Langevin algorithm to recover images. Experiments show that this approach outperforms classical methods and state‑of‑the‑art deep learning baselines, while also providing posterior uncertainty estimates.

By Hengjia Ran, Huafeng Liu, Bo Zhao
arXiv Computer Vision
5d ago

Defining Page-Centric Visual Document Retrieval and Evaluating Existing Benchmarks

The paper introduces page‑centric Visual Document Retrieval (VDR), treating pages as the primary indexing units while leveraging document context for relevance assessment. It distinguishes page and document retrieval granularities and three retrieval paradigms—independent‑item, evidence‑set, and iterative—then evaluates 42 existing benchmarks across eight criteria. The analysis reveals that most current VDR research focuses on page‑level independent‑item retrieval, with limited support for document‑level, evidence‑set, and iterative settings, and that realistic user queries and graded relevance judgments are rarely used, highlighting gaps between application needs and evaluation protocols.

By Chongzhi Zhang, Yuqi Zuo, Aixin Sun
arXiv Computer Vision
5d ago

Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos

The paper introduces Movens-5M, a dataset of 15 million human‑motion sequences and 5 million humanoid‑executed motions derived from raw Internet videos, aligned with text, RGB video, speech, and music. Using this data, the authors present Movens‑Zero, a scalable architecture that jointly learns across modalities and supports real‑time closed‑loop execution, enabling motion generation from text, speech, music, and video. They also release Movens‑Intent, a benchmark of 8,192 expert‑verified condition‑motion pairs, and demonstrate that scaling data quality and model capacity improves performance across all modalities.

By Wendong Bu, Tinghong Ye, Qizhou Wang, Jiacheng Li, Zhiqi Ge, Yuze Lin, Zhongqi Yue, Yi Su, Jingyu Wang, Kai Shen, Jingdong Liu, Chengyuan Deng, Xiaoxuan Chen, Xin Tan, Jiaming Ji, Wenqiao Zhang, Chenghong Jiang, Tuoyu Li, Siliang Tang, Jun Xiao, Tat-Seng Chua, Yueting Zhuang, Xingxing Wang, Juncheng Li
arXiv Computer Vision
5d ago

MuCALD-SplitFed: Causal-Latent Diffusion for Privacy-Preserving Multi-Task Split-Federated Medical Image Segmentation

MuCALD‑SplitFed is a multi‑task split‑federated learning framework that combines causal representation learning with latent diffusion to enable privacy‑preserving medical image segmentation across decentralized institutions. It addresses the instability and privacy issues of existing multi‑task federated approaches by reducing information leakage at split points and improving convergence. Experiments demonstrate that MuCALD‑SplitFed outperforms baseline split‑federated, personalized federated, and multi‑task federated methods on segmentation tasks.

By Chamani Shiranthika, Hadi Hadizadeh, Parvaneh Saeedi
arXiv Computer Vision
5d ago

SUREON: A Benchmark and Vision-Language Model for Surgical Reasoning

SUREON is a new video question‑answering dataset and benchmark that tests visually grounded surgical reasoning. It contains 76K training pairs from narrated surgical videos and a 340‑question test set verified by expert surgeons, designed to prevent models from relying on language shortcuts. Baseline models trained on SUREON outperform existing open‑source models and show that surgical visual reasoning remains largely unsolved, with no model exceeding 53% accuracy.

By Alejandra Perez, Anita Rau, Busisiwe Mlambo, Lee White, Aina Brunet, Juan Verde, Juan Caicedo, Jo\"el L. Lavanchy, Adam Schmidt, Chinedu Nwoye, Omid Mohareri, Muhammad Abdullah Jamal
arXiv Computer Vision
5d ago

PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modeling

PRISM is a feed‑forward single‑image 3D reconstruction framework that splits the task into a parameter‑free geometric warp and a learned residual correction, eliminating the need for iterative diffusion sampling. It uses a two‑stage training strategy—latent supervised distillation for geometric generalization and perceptual fine‑tuning for appearance quality—to generalize from synthetic data. Experiments on three benchmarks show PRISM matches diffusion‑based methods in quality while cutting inference time to just 36 seconds per scene.

By Zhijie Zheng, Xinhao Xiang, Jiawei Zhang
arXiv Computer Vision
5d ago

HOPE: Hand-Object Pressure Estimation from Monocular Videos

HOPE is a framework that estimates hand‑object pressure from monocular videos by treating pressure estimation as a hand‑centric video prediction task. It lifts diverse pressure and contact annotations into a shared hand vertex space and uses a vertex‑anchored video transformer with a contact‑gated pressure head to predict temporally evolving per‑vertex normal pressure and contact. Experiments on multiple datasets show that HOPE generalizes to bare‑hand egocentric and in‑the‑wild videos, outperforming baselines that rely only on contact or planar pressure.

By Subin Jeon, Byungjun Kim, Hanbyul Joo
arXiv Computer Vision
5d ago

GUI-Lens: Enabling Precise GUI Grounding for Off-the-Shelf VLMs via Coarse-to-Fine Cropping

GUI‑Lens is a coarse‑to‑fine grounding framework that lets a general‑purpose vision‑language model (VLM) iteratively crop and zoom into a GUI screenshot to locate a target element precisely. By extracting OCR text and UI component positions as coordinate references, the VLM selects successive regions to focus on, refining its view until the target is identified. Experiments on five GUI grounding benchmarks and six base models show that GUI‑Lens consistently improves grounding accuracy, achieving up to 9.3× the accuracy of direct grounding on ScreenSpot‑Pro.

By Zichuan Fu, Shirong Wang, Wenlin Zhang, Guojing Li, Yimin Deng, Jingtong Gao, Junjia Qi, Hanyu Yan, Yefeng Zheng, Xiaopeng Li, Wanyu Wang, Xian Wu, Xiangyu Zhao
arXiv AI
5d ago

Self-Improvement with Auxiliary Rewards: Learning from Verified Traces and Evaluating Transfer

The paper investigates whether auxiliary trajectory preferences can enhance small-language-model self‑improvement and transfer when only limited supervision is available. Two reward schemes—confidence‑based (SC) and entropy‑based (UID‑Conj.)—are compared through online reward augmentation and offline training on AIME problems. UID‑Conj. yields a 2.08‑point macro‑average improvement on the adaptation benchmark and outperforms uniform weighting in offline weighted SFT by 1.93 points, with task‑dependent benefits for SC and UID‑Conj. rewards.

By Minju Gwak, Alan Ritter
arXiv Machine Learning
5d ago

Hyperbolic Inverse Reinforcement Learning

Hyperbolic Inverse Reinforcement Learning (Hyperbolic IRL) introduces a reward function that depends on the Hyperbolic distance between feature vectors and a learned parameter vector, offering an alternative to traditional linear or neural network approaches. The method matches state‑of‑the‑art performance on standard benchmarks, performs especially well with limited data and in dense environments, and uncovers complex, non‑separable reward structures in human navigation tasks. Hyperbolic IRL thus enhances both predictive accuracy and interpretability of inferred reward functions.

By Iulia Rusu, Anoop Praturu, Sergei Gepshtein, Tatyana Sharpee
arXiv Machine Learning
5d ago

From Training Exposure to Membership Evidence: A Controlled Study of OLMo 2

The study re‑runs the OLMo 2 1B language model with controlled training exposure, swapping selected training chunks with matched documents to assess membership inference. Using a counterfactual run from the same seed, the authors find that a single exposure modestly increases a document’s likelihood but rarely allows reliable identification; the reference‑ratio method achieves an AUC of 0.767 and detects 3.8 % of members at a 1 % false‑positive rate. Even with matched references, document‑level variability limits individual inference, and the authors plan to release checkpoints, log‑probabilities, and evaluation data.

By Ali Banaei, Muhammad Shahab Shahzad, Gerald Q. Maguire Jr., Marco Chiesa, Dejan Kosti\'c
arXiv Machine Learning
5d ago

Overcoming the Representational Limitations of Amplitude Encoding for Quantum Neural Networks

The paper identifies a representational limitation of amplitude encoding in quantum neural networks: normalization makes the encoding invariant to nonzero real rescaling of inputs, which restricts the hypothesis class regardless of the circuit architecture. The authors derive a condition for deterministic preprocessing that removes this ambiguity and propose a fixed nonlinear feature map that distinguishes all real inputs while keeping logarithmic qubit scaling. Experiments on three classification benchmarks and a synthetic dataset show that this new encoding improves accuracy by 21–44 % over both conventional amplitude and angle encodings and a classical neural network.

By Vincent Gurgul, David Prinz
arXiv Machine Learning
5d ago

Don't Just Impute, Detect: Task-Aware Diffusion Pipelines for Edge Anomaly Detection

The paper introduces Task‑Aware Diffusion Refiner (TADR), a conditional diffusion model that refines imputed data for edge anomaly detection in IIoT pipelines, addressing the bias of standard MSE‑trained imputers. TADR is trained against a frozen downstream detector’s loss landscape, requiring no extra back‑propagation through sampling, and achieves a 0.338 F1 improvement over the unrefined detector on the SMD dataset. The method’s effectiveness depends on the presence of continuous, redundant sensor channels and is less beneficial for telemetry data dominated by one‑hot command signals.

By Jeongsu Park, Taehyeon Yun, Geonwoo Kim, Yuji Lim, Moohong Min
arXiv Machine Learning
5d ago

RC Prediction: A Python Package for Parameter-Aware Reservoir Computing

The article introduces rc_prediction, a Python package that implements parameter‑aware reservoir computing (PARC) for modeling nonlinear dynamical systems when governing equations are unknown. PARC trains a reservoir computer on time‑series data collected at multiple parameter values, then uses the trained model in closed‑loop to generate trajectories at unseen parameter settings, enabling parameter scans and autonomous prediction. The package offers hyperparameter optimization, optional GPU acceleration via PyTorch, and demonstrates its use on three benchmark systems, including predicting critical transitions and system collapse from pre‑critical data.

By Jin-Cheng Rao, Zheng-Meng Zhai, Ying-Cheng Lai