Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,429 stories · RSS feed

arXiv Machine Learning
Aug 31

Depth-Aware Pothole Detection Using YOLO and RT-DETR at the Edge

The paper introduces a depth‑aware pothole detection framework that fuses RGB‑D sensor data and evaluates five architectures—YOLOv8n, YOLOv8nSeg, YOLOv9t, RTDETRL, and RTDETRX—on the PothRGBD dataset. YOLOv8nSeg achieves the highest detection performance (mAP@50 = 0.9556, mAP@50_95 = 0.6758) and the most accurate depth estimate (2.96 cm), while YOLOv8n offers the fastest inference (3.6 ms) and RTDETRX delivers the highest detection confidence (92.70 %). The study also shows that even after RANSAC orthorectification, bounding‑box models overestimate pothole depth by 0.16–0.21 cm, indicating a structural bias rather than a calibration error.

By Md Monjurul Ahsan Prodhan, Md Nour Hossain
arXiv Computer Vision
Aug 31

Bringing SAM to new heights: Leveraging elevation data for tree crown segmentation from drone imagery

The paper introduces BalSAM, a model that combines the Segment Anything Model (SAM) with Digital Surface Model (DSM) elevation data to improve tree crown instance segmentation from high‑resolution drone imagery. Experiments across boreal plantations, temperate forests, and tropical forests show that while off‑the‑shelf SAM does not beat a custom Mask R-CNN, fine‑tuning SAM end‑to‑end and incorporating DSM information yield promising results, especially for plantation sites.

By M\'elisande Teng, Arthur Ouaknine, Etienne Lalibert\'e, Yoshua Bengio, David Rolnick, Hugo Larochelle
arXiv Machine Learning
Aug 31

RegCL: Compact Continual SAM Adaptation for Visual Grounding in Multi-Sensorial Media

RegCL is a non‑replay continual learning framework that adapts the Segment Anything Model (SAM) for visual grounding across evolving multi‑sensorial media domains. It consolidates domain‑specific segmentation knowledge into a single lightweight SAM adapter by incrementally merging LoRA‑style AugModules and preserving compact historical feature statistics. Experiments on five heterogeneous datasets demonstrate that RegCL retains performance while adapting to new domains, outperforming other non‑replay continual learning and merging baselines.

By Yuan-Chen Shu, Zhiwei Lin, Xiaoyu Zhou, Yongtao Wang
arXiv Computer Vision
Aug 31

3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

The paper presents a multimodal 3D convolutional neural network that classifies Alzheimer’s disease using raw OASIS 1 MRI volumes. It fuses structural T1 images with gray matter, white matter, and cerebrospinal fluid probability maps to capture complementary neuroanatomical information. Evaluated with 5‑fold subject‑level cross‑validation, the model achieves a mean accuracy of 72.34 % and an ROC AUC of 0.7781, with GradCAM visualizations highlighting anatomically relevant regions such as the medial temporal lobe and ventricles.

By Md Sifat, Sania Akter, Akif Islam, Md. Ekramul Hamid, Abu Saleh Musa Miah, Najmul Hassan, Md Abdur Rahim, Jungpil Shin
arXiv Machine Learning
Aug 31

Explainable Diabetic Retinopathy Classification Using Vision Foundation Models

The paper presents an explainable diabetic retinopathy classification framework that leverages vision foundation models—DINOv2, CLIP, and Vision Transformer—combined with various transfer learning techniques such as full fine‑tuning, linear probing, and Low‑Rank Adaptation (LoRA). Using the ODIR dataset for internal validation and the APTOS dataset for external testing, DINOv2‑LoRA achieved the best internal AUROC (0.758) while DINOv2 and ViT full fine‑tuning reached the highest external AUROC (0.920). Explainability was assessed with Grad‑CAM and HiResCAM against expert‑annotated lesion masks from IDRiD, using Dice, IoU, and Pointing Game metrics, confirming that model attention aligns with clinically relevant retinal lesions.

By Abhishek Verma, Anila Krishna, Abhishek Gajanan Bankar, Juan Miguel Lopez Alcaraz
arXiv Computer Vision
Aug 31

From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentation

The paper introduces Distortion Extenders (DEX), learnable parameters that adapt vision foundation models to fisheye cameras by modeling distortion coefficients and correcting distributional shifts between fisheye and perspective images. DEX is applied to monocular depth estimation and open‑vocabulary segmentation across convolutional and Transformer architectures, consistently outperforming baselines on indoor and outdoor fisheye datasets. Additionally, DEX activations can be decoded to obtain distortion coefficients, aiding camera calibration.

By Rit Gangopadhyay, Alex Wong
arXiv Computer Vision
Aug 31

uScenes: A Multimodal RGB and 3D Sonar Dataset for Underwater Robot Perception

uScenes is a new multimodal dataset for underwater robot perception that provides synchronized 3D multibeam sonar point clouds and RGB imagery. It comprises 110 scenes with 95,834 observations, totaling 277.6 minutes of data collected during multiple field sessions. The dataset aims to support research in underwater sensor fusion, cross‑modal representation learning, and 3D scene understanding.

By Trung Tien Dong, Zhenqi Wu, Aditya Penumarti, Zi-Hao Zhang, Micaiah Bartlett, Jane Shin, Xiaomin Lin
arXiv Computer Vision
Aug 31

Report Supervision

The paper introduces Report Supervision (R‑Super), a framework that uses radiology reports to supervise tumor segmentation models. By incorporating loss functions that align segmentation outputs with report‑derived tumor counts, sizes, and locations, R‑Super improves detection and segmentation performance. Experiments on kidney and pancreatic tumors show up to a 15% increase in F1‑Score and DSC compared to mask‑only training, outperforming methods like CLIP and multi‑task learning.

By Pedro R. A. S. Bassia, Wenxuan Li, Jakob Wasserthal, Jieneng Chen, Xinze Zhou, Zheren Zhu, Chuntung Zhuanga, Sergio Decherchi, Andrea Cavalli, Kang Wang, Yang Yang, Alan Yuille, Zongwei Zhou
arXiv Computer Vision
Aug 31

Deflickering Vision-Based Occupancy Networks through Lightweight Spatio-Temporal Correlation

The paper introduces OccLinker, a lightweight plugin for vision‑based occupancy networks that reduces flickering by efficiently merging historical static and motion cues with current features via a dual cross‑attention mechanism. It generates correction components to refine base network predictions and proposes a new temporal consistency metric to quantify flickering. Experiments on two benchmark datasets show that OccLinker improves performance with minimal computational overhead while effectively diminishing flickering artifacts.

By Fengcheng Yu, Haoran Xu, Canming Xia, Ziyang Zong, Guang Tan
arXiv Machine Learning
Aug 31

D-TAIA: Domain-Aware LLM Adaptation for Multi-Task Predictive Process Monitoring

D-TAIA is a framework that adapts large language models for multi‑task predictive process monitoring, jointly predicting the next activity and remaining time of ongoing cases. It uses domain‑aware triplet loss pre‑training, FAISS‑based nearest‑neighbor retrieval for time estimation, and a TAIA inference strategy to preserve sequential reasoning while fine‑tuning a 10 M‑parameter backbone. Across four real‑world event logs, D‑TAIA achieves state‑of‑the‑art or competitive results compared to a fine‑tuned LLM and a recurrent neural network baseline, with ablation studies showing the effectiveness of NLP and computer‑vision techniques for this domain.

By Sjoerd van Straten, Christine Jacob, Marwan Hassani
arXiv Computer Vision
Aug 31

GAN-Based Semantic Communication for Image Transmission in IoV

The paper introduces a GAN‑based semantic communication framework for image transmission in the Internet of Vehicles, aiming to overcome bandwidth and channel limitations. At the transmitter, a pyramid attention network extracts semantic label maps and a priority mechanism assigns weights to categories based on driving safety, guiding bit allocation and loss design. The receiver reconstructs images using a coarse‑to‑fine multi‑resolution generator, multi‑scale discriminator, temporal consistency, spatial pyramid pooling, and class‑aware convolutions, achieving high‑fidelity results with combined adversarial, feature‑matching, and perceptual losses. Experiments on Cityscapes demonstrate superior semantic segmentation accuracy and image quality compared to existing methods, with stable performance under AWGN and Rayleigh channels.

By Ruixing Ren, Shan Chen, Junhui Zhao, Xiaoke Sun
arXiv Computer Vision
Aug 31

UniLipi: A Unified Multi-Script OCR for Historical Indic Manuscripts

UniLipi is a unified multi‑script OCR model trained on 13 Indic scripts to recognize handwritten manuscripts under challenging conditions such as varied line geometry, length, and interruptions by non‑textual elements. It uses script‑aware synthetic data generation to perform well even with limited real annotated data. The model also predicts script identity and per‑line character counts, aiding manuscript cataloging, and its representations transfer to contemporary Indic handwriting and several non‑Indic scripts.

By Tathagata Ghosh, Sai Madhusudan Gunda, Simran Singh Sandral, Ravi Kiran Sarvadevabhatla
arXiv Computation and Language
Aug 31

Synth-JDoc: Synthesizing a Japanese Document Image Dataset for OCR with Diverse Layouts and Embedded Images

The paper introduces Synth-JDoc, a synthetic Japanese document image dataset created by rendering text with HTML and CSS to produce multi‑column layouts that include both vertical and horizontal writing styles. Images generated by text‑to‑image models are embedded to enhance visual realism, and noise and degradation filters are applied to improve robustness. Experiments show that fine‑tuning Large Vision Language Models on Synth‑JDoc yields superior performance on reading vertically written Japanese text compared to prior synthetic datasets.

By Keito Sasagawa, Shuhei Kurita, Daisuke Kawahara