Benchmarks and evaluation

Leaderboards, eval harnesses and ablations — the contested business of deciding which model is actually better.

28,322 stories · RSS feed

arXiv Computation and Language
Sep 18

To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals

The paper introduces SwitchSD, an adaptive framework that controls speculative decoding by distinguishing genuine copy intent from accidental repetitions using lightweight probes on a model’s internal representations. SwitchSD dynamically switches between neural drafting and context-based copying, achieving up to 15% throughput gains over existing baselines such as EAGLE3. The approach turns copying from a noisy heuristic into a principled, model-aware decoding regime.

By Roy Eisenstadt, Ido Cohen, Edo Cohen-Karlik, Lior Wolf, Itamar Zimerman
arXiv Computer Vision
Sep 18

An Event Preserving Velocity Invariant Representation for Event Cameras

The paper introduces SCARF, a real‑time velocity‑invariant representation for event cameras that preserves raw events while handling fast motion, stationary scenes, and independently moving objects. Unlike previous methods that convert events into image‑like forms and lose temporal detail, SCARF maintains temporal information and achieves state‑of‑the‑art performance in computational efficiency and representation quality.

By Mikihiro Ikura, Luna Gava, Jiahang Wu, Chiara Bartolozzi, Arren Glover
arXiv Computer Vision
Sep 18

MoSSGate: Memory-Modulated State-Space Gating for Skin Lesion Segmentation

MoSSGate is a plug‑and‑play module for U‑Net that improves skin lesion segmentation by combining boundary‑aware spatial gating, an external memory modulator, and parallel 2D state‑space modeling for efficient global context aggregation. The design limits long‑range propagation to informative regions, adapts dynamically to each sample, and preserves sharp lesion boundaries while keeping computational cost low. Experiments on ISIC 2017 and 2018 show state‑of‑the‑art accuracy (86.3%/85.9% mIoU, 92.6%/90.6% Dice) with fewer FLOPs than most CNN baselines.

By Anum Awan, Mahnoor Buriro, Muhammad Younas Khan, Md Imam Ahasan
arXiv Computer Vision
Sep 18

AgriScope: Pixel-Grounded Multimodal Understanding for Agricultural Images

AgriScope is a unified pixel‑grounded multimodal framework designed for agricultural image understanding. It supports image‑level, region‑level, and pixel‑level tasks such as grounded caption generation, referring expression segmentation, and multi‑turn multimodal interaction. The authors also introduce AgriGround, a large‑scale dataset with over 500K images and 11M instruction‑following samples, created via an automatic annotation pipeline that combines caption generation, phrase‑level grounding, segmentation mask creation, and instruction synthesis.

By Abderrahmene Boudiaf, Mohamad Alanssari, Irfan Hussain, Sajid Javed
arXiv Computer Vision
Sep 18

FreqDINO++: A Frequency-Guided Multi-Task Routing Vision Foundation Model for Universal Ultrasound Analysis

FreqDINO++ is a frequency‑guided multi‑task routing vision foundation model designed for universal ultrasound analysis. It introduces a Multi‑task Routing Adapter for efficient task‑common and task‑specific integration, a Frequency‑aware Feature Enhancer to capture multi‑scale frequency characteristics, and a Task‑aligned Collaborative Decoder that promotes collaboration between dense and global prediction tasks. Experiments on large‑scale multi‑task and external single‑task ultrasound benchmarks show that FreqDINO++ outperforms strong baselines and recent foundation models across 27 diverse clinical task scenarios, with promising generalization to unseen data.

By Qing Xu, Yixuan Zhang, Yue Li, Xiangjian He, Qian Zhang, Mainul Haque, Rong Qu, Wenting Duan, Jieyun Bai, Zhen Chen
arXiv Computer Vision
Sep 18

SenseFuse: Label-Free Fusion of Image and Shape Encoders for Open-Vocabulary 3D Instance Segmentation

SenseFuse introduces a label‑free fusion approach that balances 2D image and 3D shape encoders for open‑vocabulary 3D instance segmentation. By selecting a scene‑level fusion weight through an adaptive, sensitivity‑based mechanism, it improves mask labeling accuracy across multiple datasets, recovering up to 93% of the potential gain from an oracle weight. The method demonstrates that image and shape encoders have complementary failure patterns, leading to higher instance AP in most evaluated settings.

By Euiseok Han, Tri Ton, Hwanhee Kim, Seungyeon Ryu, Chang D. Yoo
arXiv Computer Vision
Sep 18

DocAttriBench: Benchmarking Answer Grounding in Document Visual Question Answering

DocAttriBench (DAB) is a large‑scale benchmark for fine‑grained, element‑level source attribution in Document Visual Question Answering (VQA). It introduces MAPPET, a Mask‑based Perplexity‑Derived Attribution method that uses document layout and language modeling to identify the most informative layout element for each answer. The benchmark contains 237k documents and 296k question‑answer pairs with element‑level grounding, and it evaluates multimodal LLMs on answer accuracy, attribution accuracy, and overall answer quality, revealing that even strong models often fail to localize supporting elements.

By Luca De Grandis (University of Modena and Reggio Emilia, Modena, Italy), Silvia Cappelletti (University of Modena and Reggio Emilia, Modena, Italy), William Raccagni (University of Modena and Reggio Emilia, Modena, Italy, University of Pisa, Pisa, Italy), Marcella Cornia (University of Modena and Reggio Emilia, Modena, Italy), Lorenzo Baraldi (University of Modena and Reggio Emilia, Modena, Italy), Rita Cucchiara (University of Modena and Reggio Emilia, Modena, Italy)
arXiv Computer Vision
Sep 18

Should This Case Be Adapted? Prediction Fragmentation Controls Test-Time Adaptation

The paper introduces a method for deciding whether to adapt a frozen segmentation model at test time, arguing that a fixed adaptation horizon conflates two distinct decisions: how far to adapt and whether to adapt at all. By measuring disagreement geometry—called prediction fragmentation—between the source model and the adapted mask, the authors predict harmful accepted area (HA) without extra labels or backward passes, achieving strong correlation across three medical benchmarks. A case‑level router built on this metric reduces HA significantly while maintaining or improving Dice scores, and the approach generalizes across architectures and domains.

By Lili Wang, Jing Li, Xiaowen Sun, Xiangyu Hu, Zhuangzhuang Gu, Jian Liu, Srihari Nelakuditi, Yan Tong
arXiv Computer Vision
Sep 18

MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration

The paper introduces MiX, a micro‑inverted‑scaling format that replaces shared exponents with shared mantissas to avoid microscaling collapse in low‑bit vision‑language models. An adaptive dual‑format inference framework (MiX‑MX) maps this format to a custom accelerator, replacing multipliers with shifters. Experiments show 4.5‑bit MiX matches or outperforms NVFP4 accuracy while improving area efficiency by 25 % and delivering 2.3–4.5× speedup with 1.4–2.9× energy savings over the Focus accelerator.

By Yuan Liao, Jae-sun Seo
arXiv Computer Vision
Sep 18

Navi-Agent: Unlocalized Monocular Navigation Agent

Navi-Agent is a zero‑shot Vision‑Language Navigation in Continuous Environments (VLN‑CE) agent that builds a coordinate‑free spatial state from visual observations and motion history. It represents this state as a navigation topology with nodes as visual places and edges as motion transitions, enabling observation‑based self‑localization, task progress verification, and recovery. Experiments on a zero‑shot VLN‑CE benchmark and real‑world robot platforms demonstrate that Navi‑Agent achieves state‑of‑the‑art performance among geometry‑constrained methods while remaining competitive with geometry‑based approaches.

By Wenyuan Xie, Mengyang Hong, Yongzhong Wang, Yanbiao Ji, Yijin Zhou, Shaokai Wu, Shalayiding Sirejiding, Huayi Zhou, Yi-Chao Chen, Ma Ling, Yue Ding, Hongtao Lu
arXiv Computer Vision
Sep 18

AGORA: Adversarial Generation Of Real-time Animatable 3D Gaussian Head Avatars

AGORA is a new framework that extends 3D Gaussian Splatting with a generative adversarial network to produce high‑fidelity, animatable 3D head avatars. It introduces a lightweight FLAME‑conditioned deformation branch that predicts per‑Gaussian residuals for identity‑preserving, fine‑grained expression control, and a dual‑discriminator training scheme that enforces expression fidelity. The system achieves real‑time inference at 250 FPS on a single GPU and, for the first time, CPU‑only animatable 3DGS avatar synthesis at ~9 FPS.

By Ramazan Fazylov, Sergey Zagoruyko, Aleksandr Parkin, Stamatis Lefkimmiatis, Ivan Laptev
arXiv Computer Vision
Sep 18

G2G: Exploiting Intra-Group Geometry for Inter-Group Pose Estimation

The paper introduces G2G, a method that leverages known intra-group geometry to estimate the relative 6-DoF pose between two image groups, a key problem in cross-sequence relocalization and multi-camera rig odometry. G2G keeps a frozen foundation model and adds three lightweight trainable modules—a perceiver resampler, a cross-group bridge with merged self-attention, and a multi-frame pose head—totaling about 32 M parameters, less than 6% of the full model. Evaluated on four diverse datasets covering indoor/outdoor simulation, real-world cross-season capture, and zero-shot sim-to-real transfer, G2G achieves state‑of‑the‑art accuracy on both pose estimation tasks while only requiring supervision from relative poses.

By Yufei Wei, Shuhao Ye, Chenxiao Hu, Yiyuan Pan, Dongyu Feng, Rong Xiong, Yue Wang, Yanmei Jiao
arXiv Computer Vision
Sep 18

Attention-Steered Vision-Language Models for Sign Language Translation

The paper introduces AttnSign, a vision‑language model that improves sign language translation by steering spatial‑temporal attention. It first supervises attention on sign‑relevant regions such as faces and hands in each frame, then uses an RL‑based motion‑cadence method to focus on keyframes. Experiments on How2Sign and OpenASL show AttnSign consistently outperforms existing methods.

By Meibo Hu, Guohao Sun, Annemarie D. Ross, Sheng Li, Zhiqiang Tao
arXiv Statistics ML
Sep 18

Portfolio-Based Constrained Multi-Objective Bayesian Optimization for Materials Design

The paper presents a portfolio-based approach to constrained multi-objective Bayesian optimization for materials design, framing acquisition‑function selection as an adaptive policy problem. Two controllers—UCB‑Bandit, a modified UCB multi‑armed bandit, and Agentic‑Switch, a multi‑agent system powered by a large language model—were tested against fixed‑policy baselines on synthetic benchmarks and two real materials design case studies. The adaptive policies achieved competitive results in cumulative feasibility counts and feasible hypervolume improvement, outperforming individual acquisition functions that excelled only in a single metric.

By Sushant Sinha, Christofer Hardcastle, Robert Robinson, Shakti Prasad Padhy, Brent Vela, Douglas Allaire, Raymundo Arroyave
arXiv Computer Vision
Sep 18

Comparing Commercial Depth Sensor Accuracy for Medical Applications

The study benchmarks four commercial depth sensors—Intel RealSense D405, PMD Flexx2, Stereolabs ZED 2i, and Zivid 2M+ 60—on porcine bone, porcine belly, and a silicone kidney phantom. Using stylus-sampled references, the sensors were evaluated at ~50 cm distance across challenging surfaces such as homogeneous, specular, and subsurface scattering. The Zivid 2M+ 60 achieved the best overall performance, while the ZED 2i ranked second for real tissue but last for the phantom.

By Pit Henrich, Maximilian Weiherer, Franziska Hansen, Bernhard Egger, Franziska Mathis-Ullrich
arXiv AI
Sep 18

QVAC Genesis III: A Large-Scale, High-Quality Open Synthetic STEM Corpus for Efficient Language Model Pre-Training

QVAC Genesis III is a 191.43 B‑token synthetic STEM corpus covering 19 domains and multiple difficulty levels, created through a dual generation strategy that uses a weak edge‑scale student model to generate corrective explanations and contrastive reasoning. The authors evaluate the corpus with an LLM‑as‑a‑parser protocol and demonstrate that 1.7 B‑parameter models trained on QVAC Genesis III outperform those trained on Cosmopedia‑v2 and the Cosmo‑1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, achieving up to +28.57% improvement on ARC‑E and a 99.45% valid answer rate.

By Davide Vitabile, N. Ranjan, Akshay Nambiar, Kamal K. Gupta, Amril Nazir
arXiv AI
Sep 18

Reach or Solve? Attributing Agentic RL Gains with Checkpoint Handoffs

The paper introduces a new evaluation protocol called checkpoint handoff to disentangle the contributions of reaching a target state and solving the task in reinforcement learning agents. By cloning states reached by one checkpoint and handing them to another without retraining, the authors separate the REACH metric (how often a policy arrives at a state confirmed to be a fixed number of actions from success) from the SOLVE metric (how often it finishes from that identical state). Across two benchmarks and pipelines, the analysis shows that RL history benefits RL solvers more than SFT solvers, and that independent REACH and SOLVE gaps predict overall performance.

By Xuan Liu, Jingbin Qian
arXiv AI
Sep 18

TRACE: Accountable Agentic Retrieval for Source Discovery in Digital Archives

TRACE is a training‑free, agentic retrieval framework that enables accountable source discovery in historical archives, addressing challenges such as OCR degradation and genre heterogeneity. Developed for the DECIDON project on French Third Republic political discourse, it is deployed internally for 24 researchers across six institutions. On the HistoriQA‑ThirdRepublic benchmark, TRACE achieves R@10 of 0.856 and MRR of 0.653, outperforming sparse, dense, graph‑based, and other agentic RAG baselines, especially on multi‑hop and cross‑corpus questions, while costing only about $0.02 per question.

By Donghan Bian (ENC, LRE), Marie Puren (LRE, ENC), Florian Cafiero (LRE, ENC)
arXiv AI
Sep 18

E-AVI: Evidence-Grounded Multimodal Assessment for Automated Video Interviews

E-AVI is a new framework for automated video interview assessment that combines verbal, acoustic, and visual data. It extracts timestamped multimodal evidence and uses dimension‑conditioned attention with source‑level embeddings to score candidates. The system also provides a shared evidence pool for natural‑language feedback and follow‑up question answering, outperforming multimodal baselines on RecruitView and a private hospitality dataset.

By Haoshen Wang, Dongbo Che, Zeyi Xie, Yuanjie Du, Shicheng Hua, Xingyu Wang