Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,429 stories · RSS feed

arXiv AI
Aug 28

Position Is All You Need: A Free Lunch Token Compression Strategy for MLLM-based Referring Expression Segmentation

The paper introduces PAYN, a training‑free token compression strategy for Multimodal Large Language Model (MLLM) based Referring Expression Segmentation (RES). By preserving original position embeddings and local spatial structures, PAYN retains tokens that are evenly distributed across neighboring regions, thereby maintaining spatial relational consistency. Experiments on multiple RES benchmarks show that PAYN outperforms existing token compression methods, confirming that position information alone is sufficient for effective compression in this task.

By Yuhan Liu, Yixiong Zou, Yuhua Li, Ruixuan Li
arXiv Computation and Language
Aug 28

Representing and Parsing Korean Constituency Structure at Different Levels of Granularity

The paper investigates how different representations of Korean constituency structure affect parsing performance. It compares three formats—Morpheme+XPOS, Eojeol+XPOS, and Eojeol+UPOS—derived from the Penn Korean Treebank, using gold segmentation and labels to evaluate transition-based parsers. Results show that fine-grained morphological and XPOS information yields the best parsing accuracy, while eojeol-based representations offer shorter transition sequences but lower performance when only UPOS is used.

By Jungyeul Park, KyungTae Lim, Zihao Huang, Eunkyul Leah Jo, Yige Chen, Chulwoo Park
arXiv Computation and Language
Aug 28

AraMS-28k: The Largest Publicly Released Line-Level Dataset of Historical Arabic Manuscripts with Margin and Insertion-Anchor Annotations

AraMS-28k is the largest publicly released line‑level dataset of genuine historical Arabic manuscripts, containing 14 books, 3,043 pages, and 28,600 annotated text lines (27,971 main‑text and 629 margin). The dataset spans three script traditions—Naskh, Ruq'ah, and Maghrebi—and includes a lithographed printed edition for format diversity. Each line is labeled as main‑text or margin, with margin lines that have a clear attachment point annotated with an insertion anchor to recover the manuscript’s true non‑linear reading order; both fully vocalized and diacritic‑normalized transcriptions are provided, and the data was produced via the RefLAM pipeline combining OCR, clean transcriptions, and human review. "whyItMatters":"The dataset’s comprehensive line‑level annotations, including reading‑order anchors and dual transcription formats, enable reproducible research on Arabic manuscript recognition, layout analysis, and reading‑order recovery under a CC BY‑NC‑SA 4.0 license."

By Mohamed Guechaoui, Mohamed Diaa Zellagui, Souleyman Chaib, Sahraoui Dhelim
arXiv Machine Learning
Aug 28

QuantumBoostNet: A Hybrid Classical-Quantum Architecture for Enhanced Accuracy in Cardiac Ultrasound View Identification

QuantumBoostNet is a hybrid classical‑quantum architecture designed to improve accuracy in identifying cardiac ultrasound views. It combines a classical backbone with two heads—one classical and a 10‑qubit quantum circuit—trained in two stages with an adaptive mixing parameter that balances loss dynamics. Experiments show that, even with limited qubits, QuantumBoostNet outperforms state‑of‑the‑art classical and hybrid models on cardiac ultrasound view identification, image classification benchmarks, and demonstrates robustness to noise.

By Mihai Udrescu-Milosav, Stefan-Alexandru Jura, Mihai Udrescu, Gerhard-Paul Diller
arXiv Computer Vision
Aug 28

CODE: Cross-Modal Calibration and Dynamic Suppression for Open World Object Detection

CODE: Cross-Modal Calibration and Dynamic Suppression for Open World Object Detection introduces a unified inference-time framework that addresses semantic ambiguity and over-suppression in multimodal OWOD systems. It comprises Cross-Modal Joint Confidence Calibration, Uncertainty-Guided Universal Objectness Enhancement, and Dynamic Outlier Suppression via Confidence Margin. Experiments on the Real-World Detection benchmark with the OWL‑ViT L/14 backbone show CODE achieving 21.7 U‑mAP and 40.8 K‑mAP, surpassing prior state‑of‑the‑art results by 2.6 and 2.3 points respectively.

By Hao Xu, Zhaoning Shi, Hehe Jin, Bo Ma
arXiv Computer Vision
Aug 28

Automated 2D and 3D Segmentation of AMD and DME Lesions in OCT

This study presents four deep‑learning pipelines—two‑dimensional and three‑dimensional—for segmenting age‑related macular degeneration (AMD) and diabetic macular edema (DME) lesions in optical coherence tomography (OCT) images. The models achieve Dice scores between 0.76 and 0.82 and demonstrate strong volumetric and surface calibration (r_vol, r_surf ≥ 0.97) on an in‑domain validation set. Generalization was assessed on the OLIVES clinical cohort using proxy metrics such as biomarker AUROC, central subfield thickness correlation, and longitudinal concordance, showing that the predictions still track clinical biomarkers outside the training distribution, albeit with reduced strength.

By Lucia Sundberg, Zhihao Zhao, M. Ali Nasseri
arXiv Computer Vision
Aug 28

MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

MVC-Bench is a new benchmark designed to evaluate the calibration of vision‑language models (VLMs) and medical VLMs (Medical‑VLMs) for medical image classification. It tests calibration across robustness to modality, backbone, and domain shift; effectiveness of calibration strategies and prompt‑tuning methods; and stability under prompt‑template and random‑seed variations. The benchmark includes eight backbones, three medical modalities (fundus imaging, histopathology, chest X‑ray), and compares post‑hoc, train‑time, and zero‑shot calibration approaches, reporting accuracy, Expected Calibration Error (ECE), Maximum Calibration Error (MCE), and Adaptive Calibration Error (ACE) over 1,638 experiments, while also proposing a Multi‑Class Margin (MCM) regularization technique that improves ECE in most settings.

By Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
arXiv Computer Vision
Aug 28

DOD-SA: Infrared-Visible Decoupled Object Detection with Single-Modality Annotations

The paper introduces DOD-SA, a framework for infrared-visible object detection that uses only single-modality annotations. It employs a Collaborative Teacher-Student Network with a single-modality branch and a dual-modality decoupled branch to transfer knowledge across modalities, and a Progressive and Self‑Tuning Training Strategy to refine pseudo‑labels. A Pseudo Label Assigner is also designed to align labels between modalities during training.

By Hang Jin, Chenqiang Gao, Junjie Guo, Fangcen Liu, Qinyao Chang, Kanghui Tian, Deyu Meng
arXiv Machine Learning
Aug 28

When Is Noise Response Universal? Tokenization as the Hidden Variable in Language Models

The study investigates how textual neural models degrade when inputs contain noise such as typos, OCR errors, or dropped words. It finds that model performance decline is largely consistent across architectures under word‑level noise but diverges under character‑level noise, a difference attributed to tokenization rather than architecture. By applying a short contrastive training recipe, diverse encoders converge to a common robustness curve, enabling prediction of a model’s noise resilience and the ability to enhance robustness at specific noise scales through targeted training.

By Yefan Tao, Gerald Friedland, Luyang Kong
arXiv Computer Vision
Aug 28

Reconstructing Humans and Objects in Interaction using Large Reconstruction Models

The paper introduces MILO, a framework that uses Large Reconstruction Models (LRMs) to reconstruct detailed 3D human‑object interactions from a single image. By treating the LRM mesh as a geometric scaffold, MILO segments it into human and object parts, fits a parametric body model to the human component, and optionally aligns an object template to the object component. The approach achieves higher reconstruction accuracy than existing baselines across multiple benchmarks and interaction scenarios.

By Agniv Chatterjee, Georgios Pavlakos
arXiv Computer Vision
Aug 28

SSMB: Self-Supervised Local Feature Detection under Motion Blur

SSMB is a self‑supervised keypoint detector designed for motion‑blurred images that does not rely on handcrafted detectors or external pseudo‑labels. It introduces a Local Discriminability Enhancement module to recover fine‑grained local detail after global feature mixing, and is trained in two stages: geometric pretraining on synthetic shapes and blur‑aware training on real sharp‑blur pairs using a multi‑component self‑supervised objective. Extensive experiments show that SSMB outperforms both supervised and self‑supervised baselines on keypoint detection, image matching, relative pose estimation, and visual localization under motion blur, achieving state‑of‑the‑art performance.

By Zhenjun Zhao, Fabio Bellavia, Wenting Wang, Fan Zhu, Jiajun Wu, Suryansh Kumar, Mingqiang Wei, Haoang Li, Javier Civera
arXiv Machine Learning
Aug 28

A Framework for Low-Effort Training Data Generation for Urban Semantic Segmentation

The paper introduces a framework that adapts a diffusion model to a target urban domain using only imperfect pseudo‑labels, enabling the generation of high‑fidelity, target‑aligned images from semantic maps of any synthetic dataset. By filtering poor generations, correcting image‑label misalignments, and standardising semantics, the method transforms low‑effort synthetic data into competitive real‑domain training sets. Experiments on five synthetic and two real datasets show up to +8.0 %pt mIoU improvement over state‑of‑the‑art translation methods, demonstrating that rapidly constructed synthetic datasets can match the performance of high‑effort, manually designed ones.

By Damjan Kal\v{s}an, Denis Zavadski, Tim K\"uchler, Haebom Lee, Stefan Roth, Carsten Rother
arXiv Computer Vision
Aug 28

A Geometry-Driven, Framework-Agnostic Optimization for Object Pose Estimation

The paper proposes a data‑centric optimization for object pose estimation that uses a physically grounded rotation representation based on principal axes alignment. By aligning an object's coordinate system with its inertial principal axes, the method achieves inherent stability, symmetry‑aware canonicalization, and framework agnosticism, allowing it to be applied at the dataset level without modifying existing networks. Experiments on category‑level and instance‑level models show consistent accuracy improvements while preserving baseline network integrity.

By Wei Chen, Tao Zhen, Zhongchen Shi, Jing Zhang, Liang Xie, Erwei Yin
arXiv Computer Vision
Aug 28

Parameter-Efficient pretrained-CT-to-MRI Transfer for Rectal Cancer Segmentation: Performance-Calibration Trade-offs

The paper introduces SWIFT, a Swin V2‑based model pretrained on 10,444 3D CT volumes and fine‑tuned for rectal cancer segmentation on T2‑weighted MRI. Four configurations—full fine‑tuning (SWIFT), decoder compression (SWIFTe), low‑rank adaptation (SWIFTe‑LoRA), and a LoRA‑decoder ensemble (SWIFTe‑LDE4)—were evaluated on 247 cases, showing that SWIFTe reduces parameters by 70.1% while improving tumor detection and radiomic agreement. The study also demonstrates a trade‑off between detection and boundary agreement, and highlights that SWIFTe‑LDE4 achieves the lowest calibration errors after temperature scaling.

By Aneesh Rangnekar, Jorge Tapias Gomez, Joseph O Deasy, Harini Veeraraghavan
arXiv Computer Vision
Aug 28

UFO-DETR: Frequency-Guided End-to-End Detector for UAV Tiny Objects

UFO-DETR is an end‑to‑end object detection framework designed for UAV imagery, addressing challenges such as scale variation, dense distribution, and the prevalence of tiny targets. It employs an LSKNet backbone to optimize receptive fields and reduce parameters, integrates DAttention and AIFI modules for flexible multi‑scale spatial modeling, and introduces a DynFreq‑C3 module that enhances small target detection via cross‑space frequency feature enhancement. Experiments demonstrate that UFO‑DETR outperforms RT‑DETR‑L in detection accuracy while improving computational efficiency, making it suitable for UAV edge computing.

By Yuankai Chen, Kai Lin, Qihong Wu, Xinxuan Yang, Jiashuo Lai, Ruoen Chen, Haonan Shi, Minfan He, Meihua Wang
arXiv Computer Vision
Aug 28

DINOcular: Self-Supervised Visuospatial Representations

DINOcular is a self‑supervised framework that learns joint visuospatial representations from RGB‑D observations. It fuses depth‑derived geometric priors with a visual backbone using inter‑patch and intra‑patch fusion, allowing the model to encode both appearance and spatial structure efficiently. The resulting representation improves 3D awareness on multiple geometry benchmarks while staying competitive on standard RGB‑D semantic segmentation tasks.

By Farkhat Almukhamedov, Sami Azirar, Hermann Blum
arXiv Computer Vision
Aug 28

Glass Surface Detection Grounded in 3D Visual Geometry

Glass Surface Detection Grounded in 3D Visual Geometry proposes a new approach that grounds glass surface detection in 3D visual geometry rather than relying solely on 2D appearance cues. The method uses a visual geometry grounded transformer (VGGT) to distill 3D priors and creates glass-aware 3D representations, then applies a multi-task learning framework with a Frequency Self-Attention Module (FSAM) and a Geometry Grounding Block (GeGB) to localize and segment glass surfaces. Experiments show state‑of‑the‑art performance on seven benchmarks, good generalization to video and multi‑modal data, and significant improvements in reconstruction of glass scenes.

By Yiwei Lu, Ke Xu, Tao Yan, Xiaojun Chang, Radu Timofte, Rynson W. H. Lau
arXiv Computer Vision
Aug 28

SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models

SOCO is a new benchmark for Semantic Object Correspondence that introduces a taxonomy of correspondence types and provides consistent, functionally meaningful keypoint annotations across 100 categories and over 1M correspondence pairs. It also includes keypoint language descriptions, enabling evaluation of large vision‑language models and their fine‑grained part‑level understanding. Experiments show that vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories, LVLMs excel at text‑prompted part localization but lag in visual‑reference matching, and correspondence performance predicts dense downstream tasks more strongly than ImageNet classification.

By Olaf D\"unkel, Basavaraj Sunagad, Haoran Wang, David T. Hoffmann, Christian Theobalt, Adam Kortylewski
Hugging Face Trending Papers
Aug 27

Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models

The paper introduces Successive Capacity Growth (SCG), a method that starts with a minimal Vision Transformer encoder and incrementally expands its width or depth based on a task‑agnostic test‑and‑verify mechanism. SCG uses function‑preserving expansion and a Sketched Isotropic Gaussian Regularizer (SIGReg) to ensure independent semantic dimensions and prevent collapse. Experiments on multi‑object dynamics and 2D navigation tasks show that SCG achieves significant prediction loss reductions while being far more parameter‑efficient than fixed large models, with no false‑positive expansions and exact function preservation.

Hugging Face Trending Papers
Aug 27

QuantumBoostNet: A Hybrid Classical-Quantum Architecture for Enhanced Accuracy in Cardiac Ultrasound View Identification

QuantumBoostNet is a hybrid classical‑quantum architecture designed to improve accuracy in identifying cardiac ultrasound views. The model combines a classical backbone with two heads—one classical and a 10‑qubit quantum circuit—using an adaptive mixing parameter to switch between them during training. Experiments show that, even with limited qubits, QuantumBoostNet outperforms state‑of‑the‑art classical and hybrid models on cardiac ultrasound view identification, image classification benchmarks, and demonstrates robustness to noise.