Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,379 stories · RSS feed

arXiv Computer Vision
Sep 22

Graded-Relevance Composed Multimodal Retrieval for E-commerce Visual Search at Scale

The paper introduces GradCIR, a method for training composed image retrieval (CIR) systems on graded relevance rather than binary relevance. It uses a vision‑language model to generate queries and 4‑level relevance labels, an iterative feedback loop to mine hard negatives, and a hierarchy‑aware angular objective to directly optimize graded labels. Experiments on a Walmart catalog and FashionIQ show significant NDCG improvements and the system is deployed in Walmart’s live visual‑search traffic.

By Anubhav Gupta, Hrushikesh Mohapatra, Prijith Chandra, Asish Mohapatra, Anuj Garg, Arvind Maan, Sudip Datta, Venkat Bulusu, Sitesh Kumar Jalan
arXiv Computer Vision
Sep 22

MTMed3D: A Multi-Task Transformer-Based Model for 3D Medical Imaging

MTMed3D is a multi-task Transformer-based model that jointly performs 3D detection, segmentation, and classification in medical imaging. It uses a shared Transformer encoder to produce multi-scale features, with separate CNN decoders for each task. Evaluated on BraTS 2018 and 2019, it achieves strong results, especially in detection, while reducing computational cost and inference time compared to single-task models.

By Fan Li, Arun Iyengar, Lanyu Xu
arXiv Computer Vision
Sep 22

Closed-Circuit Television Data as an Emergent Data Source for Urban Rail Platform Crowding Estimation

The paper explores the use of Closed‑Circuit Television (CCTV) footage to estimate urban rail platform crowding in real time. It compares three computer‑vision methods—object detection and counting, crowd‑level classification with a Vision Transformer, and semantic segmentation—to extract crowd-related features. A novel convex ridge regression technique is introduced to convert segmentation outputs into passenger counts, and the methods are evaluated on a privacy‑preserving dataset of over 600 hours of Washington Metropolitan Area Transit Authority (WMATA) video, showing that CCTV alone can provide valuable real‑time crowd estimates.

By Riccardo Fiorista, Awad Abdelhalim, Anson F. Stewart, Gabriel L. Pincus, Ian Thistle, Jinhua Zhao
arXiv Machine Learning
Sep 22

Reinforcement Learning Inspired Black-box Adversarial Attacks for Computer Vision

The paper introduces RIBA, a reinforcement‑learning inspired black‑box adversarial attack that generates perturbations for neural networks with fewer queries than existing methods. RIBA achieves a 25.4% reduction in median queries on ResNet‑18/Cifar10 and a 22.5% reduction on Vit‑B/16/ImageNet, while matching white‑box attack performance on an adversarially trained model.

By Florian Krone, Elena Hoemann, Sven Hallerbach
arXiv Machine Learning
Sep 22

Directional Linear Separability of Neural Representations: Geometry and Transformations

The paper introduces the directional linear separability measure (D‑LSM) to quantify how much linear separability is preserved or improved by injective affine maps in neural networks. It characterizes the geometry supporting D‑LSM, proves its invariance under injective affine embeddings, and derives conditions for gated activations (ReLU, GELU, SiLU) to preserve and recover samples. Experiments validate the theoretical bounds, demonstrate affine‑tube constructions that achieve guaranteed recovery, and apply the method to Vision Transformer representations to obtain early post‑activation separability certificates.

By Yi Wei, Xuan Qi, Suorong Yang, Furao Shen
arXiv Machine Learning
Sep 22

Comparative Analysis of State-of-the-Art Foundation Models for Sleep Analysis Under Channel Reduction

The study compares six state‑of‑the‑art sleep‑staging foundation models on the MESA polysomnography dataset under three signal conditions: EEG only, ECG only, and EEG+ECG. Results show that EEG alone yields the highest accuracy (BIOT macro‑F1 = 0.7237), while switching to ECG alone incurs a consistent accuracy loss of about 0.35 macro‑F1 and reduces data rate to one‑third. Adding ECG to EEG offers little benefit for most models, indicating that EEG carries most of the sleep‑staging signal and that wearable‑compatible ECG alone is a viable but less accurate alternative.

By Hassan Mehdi, Riku Klen, Ayse Kosal Bulbul, Suzanne Timmons, Abdulhamit Subasi, Wei Chen, Zou Zhu, Muhammad Irfan
arXiv Computer Vision
Sep 22

Revisiting Multi-View Stereo: A Sequence-to-Sequence Formulation

The paper proposes a new sequence-to-sequence formulation for multi-view stereo (MVS) that jointly predicts 3D geometry for all input views using a global transformer architecture. It introduces ray‑map embeddings to inject camera parameters into image tokens and a unified global cost volume to capture 3D structure across all views. Experiments on public benchmarks demonstrate state‑of‑the‑art performance, outperforming both traditional MVS and feed‑forward reconstruction baselines.

By Aoxiang Fan, Corentin Dumery, Nicolas Talabot, Pascal Fua
arXiv Computer Vision
Sep 22

AdaptiveCDM: Source-Free Few-Shot Domain Adaptation for Cell Detection in Microscopic Images

AdaptiveCDM is a modular framework for source‑free few‑shot domain adaptation in cell detection, enabling a pretrained model to adapt to new imaging domains using only a handful of labeled target images and no source data. It combines Resolution‑Aware Augmentation (RAug) to balance scarce, class‑imbalanced samples while preserving cellular morphology, and Category‑Aware Representation Learning (CARL) to strengthen class‑consistent proposals for better localization and classification. Experiments on M5 and Raabin‑WBC datasets show that AdaptiveCDM achieves competitive or superior mAP scores compared to state‑of‑the‑art methods under their respective supervision settings.

By Nimra Dilawar, Sara Nadeem, Javed Iqbal, Waqas Sultani, Mohsen Ali
arXiv Computer Vision
Sep 22

SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos

SurgMotion is a video-native foundation model that replaces pixel-level reconstruction with latent motion prediction for surgical video analysis. It introduces motion-guided masked prediction, spatiotemporal affinity self-distillation, and spatiotemporal feature diversity regularization to focus on semantically meaningful regions and avoid representation collapse. Trained on SurgMotion-15M, the largest surgical video dataset, it outperforms state-of-the-art methods across 17 benchmarks, improving workflow recognition, action triplet recognition, skill assessment, polyp segmentation, and depth estimation.

By Jinlin Wu, Felix Holm, Chuxi Chen, An Wang, Yaxin Hu, Xiaofan Ye, Zelin Zang, Miao Xu, Lihua Zhou, Huai Liao, Danny T. M. Chan, Ming Feng, Wai S. Poon, Hongliang Ren, Dong Yi, Nassir Navab, Gaofeng Meng, Jiebo Luo, Hongbin Liu, Zhen Lei
arXiv Computer Vision
Sep 22

All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts

The paper introduces an all‑in‑one multilingual scene text recognizer called ScriptMoE, which uses a script‑aware mixture‑of‑experts architecture to handle 10 scripts and 229 languages. It is built on a new large‑scale synthetic dataset, TextMuSS‑10M, and evaluated on the TextMuSS‑Bench, achieving 82.06% accuracy—1.31% higher than the best baseline. When integrated into the PP‑OCRv5 pipeline, ScriptMoE raises the end‑to‑end multilingual F1 score from 65.71% to 80.89%, slightly surpassing the best vision‑language model while using far fewer parameters.

By Xingsong Ye, Yongkun Du, Jiaxin Zhang, Zhixian Li, Chong Sun, Chen Li, Jing Lyu, Lianwen Jin, Zhineng Chen
arXiv Machine Learning
Sep 22

Correcting Learning-based Perception for Safety

The paper presents a two-step method to correct machine‑learning based perception for safety in autonomous systems. First, it uses offline computation to characterize uncertainties from the ML module via preimages of perception contracts. Then, at runtime, a risk heuristic selects specific states from these uncertain estimates to guide control decisions, reducing safety violations in adaptive cruise control scenarios while adding minimal delay.

By Yan Miao, Hussein Darir, Sayan Mitra
arXiv Computer Vision
Sep 22

Comparative Performance and Parameter-Efficient Adaptation of DINOv2 for Active Trachoma Classification

The study evaluates the performance of the DINOv2 visual representation for classifying Trachomatous Inflammation-Follicular (TF) versus normal conjunctival images. Using 1,546 images processed by the OPTED pipeline, the authors compare six pretrained backbones and then test four lightweight adaptation methods on DINOv2 ViT-B/14. The best results—91.66% accuracy, 90.69% macro‑F1, and 96.06% AUC—were achieved with DINOv2 plus Efficient Channel Attention (ECA) and a focal‑plus‑center loss, though ECA’s benefit varied with the loss function.

By Kibrom Gebremedhin, Hadush Hailu, Bruk Gebregziabher, Yordanos Hailu