Computer vision

Detection, segmentation, depth and recognition research, plus the vision backbones that keep displacing the last generation.

3,429 stories · RSS feed

Hugging Face Trending Papers
Aug 27

DINOcular: Self-Supervised Visuospatial Representations

DINOcular is a self‑supervised framework that learns joint visuospatial representations from RGB‑D data. It fuses depth‑derived geometric priors with a visual backbone using inter‑patch and intra‑patch techniques, allowing the model to encode both appearance and spatial structure efficiently. The resulting representation improves 3D awareness on multiple geometry benchmarks while staying competitive on standard RGB‑D semantic segmentation tasks.

Hugging Face Trending Papers
Aug 27

MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

MVC-Bench is a calibration-focused benchmark for medical vision‑language models, evaluating how well these models express confidence across different modalities, backbones, and domain shifts. It tests robustness to modality, backbone, and domain changes, the effectiveness of calibration and prompt‑tuning strategies, and stability under prompt‑template and random‑seed variations. The benchmark includes 1638 experiments, reporting accuracy and Expected Calibration Error (ECE) along with other calibration metrics, and introduces a simple train‑time calibration method, Multi‑Class Margin (MCM) regularization, that achieves the lowest ECE in most settings.

Hugging Face Trending Papers
Aug 27

Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoning

The paper introduces the Multi-Instruction Multi-Shot Long-Video Editing (MMLVE) task, aiming to edit long videos with multiple instructions while maintaining consistency across shots, decoupling instructions, and preserving spatiotemporal structure. It proposes an agentic editing framework that combines Large Language Models and Vision‑Language Models for shot-level decoupling and precise instruction parsing. A new dataset, MMLVE‑Bench, and three evaluation metrics are created to benchmark this task, and experiments show the proposed MMLVE‑Agent outperforms existing closed‑source state‑of‑the‑art methods by eliminating hallucinations and ensuring seamless transitions.

Hugging Face Trending Papers
Aug 27

G2D: Generative-to-Discriminative Collaborative Inference for Zero-Shot Image Classification

G2D is a training‑free framework that combines a discriminative model (CLIP) for broad candidate retrieval with a generative vision‑language model for fine‑grained, image‑grounded verification. By using CLIP’s top‑K shortlist and a structured prior from candidate names and probabilities, G2D focuses generative reasoning on uncertain samples, employing fixed confidence routing, entropy‑adaptive candidate sizing, and trie‑constrained decoding to produce a single valid output. Across eight benchmarks, G2D achieves an average accuracy of 68.85%, outperforming both CLIP (59.35%) and the standalone generative model (63.11%), and it also transfers effectively to other models such as DCLIP, WaffleCLIP, and CuPL.

Hugging Face Trending Papers
Aug 27

PailitaoGR: Latent Think-with-Images for Generative Image Retrieval

PailitaoGR is a generative image retrieval method that incorporates target-focused perception and selective auxiliary-evidence utilization. It uses a target Enhancer and on-policy distillation to highlight search-target regions, and an auxiliary enhancer with incremental contrastive distillation to exploit auxiliary evidence. Trained on real-world online image-search logs, it achieves an average 13.8% improvement over existing baselines.

Hugging Face Trending Papers
Aug 27

FU-Mamba: A Frequency-Enhanced Dynamic Scanning Framework for Oralscan Image Segmentation

FU‑Mamba is a new framework for oralscan image segmentation that combines dynamic scanning with frequency‑domain enhancement. It uses a Dynamic Mamba Block to learn adaptive sampling offsets and perform bilinear interpolation, preserving spatial coherence, and a frequency enhancement block that balances spectral components via wavelet‑guided decomposition and spectrum pooling. Experiments show a 1.1% improvement in mean intersection over union on a dental segmentation dataset.

Hugging Face Trending Papers
Aug 27

DPA-I2P: Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration in Autonomous Driving

The paper introduces DPA-I2P, a depth‑guided projective alignment method for image‑to‑point‑cloud registration in autonomous driving. It employs Ray‑Conditioned Metric Depth Encoding and Projection‑Consistent Vision Lifting to align depth and visual cues geometrically, and uses Cross‑Modal Query Pruning to filter unreliable matches during refinement. Experiments on KITTI and nuScenes show significant improvements, reducing rotation and translation errors by up to 55.6% compared to existing implicit baselines.

arXiv Computer Vision
Aug 27

TDFNet: Tri-projection Deformable Fusion Network for Panoramic Salient Object Detection

TDFNet introduces a Tri-projection Deformable Fusion Network that uses equirectangular, cube map, and tangent projections to mitigate geometric distortions in panoramic salient object detection. It incorporates a cross-projection deformable attention module for geometry-aware sampling and a latitude-guided fusion module that balances ERP and CMP features using spherical latitude priors. The network’s three-branch encoding preserves global continuity, local detail, and boundary precision, improving detection performance over existing projection-based methods.

By Qiangqiang Zhou, Jiacong Yu, Jiawei Xu, Yong Chen, Xin Huang, Ping Li
arXiv Computer Vision
Aug 27

Visual General Intelligence: A White Paper

The paper titled "Visual General Intelligence: A White Paper" reexamines intelligence from a vision-centered perspective, questioning whether visual experience and learning can lead to artificial general intelligence (AGI). It compares the success of language models like GPT, which transfer to unseen tasks via autoregressive modeling on large text corpora, with the potential of visual modalities such as images, videos, and geometry to develop similar capabilities. The authors aim to outline principles for computer vision in the AGI era, including input modalities, benchmarks, learning paradigms, and the interplay between vision and other modalities like language.

By Hirokatsu Kataoka, Yoshihiro Fukuhara, Yonglong Tian, Shangzhe Wu, Oishi Deb, Ryousuke Yamada, Christian Rupprecht, Jianyuan Wang, Kohsuke Ide, Koichi Namekata, Xianzheng Ma, Yiming Chen, Robert Geirhos, Aditi Raghunathan, Yuki M. Asano, Deva Ramanan, David Fouhey, Andrew J. Davison, Yilun Du, Jiajun Wu, Zhuang Liu
arXiv Computer Vision
Aug 27

Semi-Supervised Adaptation of Vision-Language Models for Image Classification

The paper introduces Self‑Evolutionary CLIP (SE‑CLIP), a semi‑supervised framework that adapts vision‑language models like CLIP to satellite imagery. SE‑CLIP uses a two‑phase pipeline: an initial warm‑up on a small set of annotated seeds followed by a recursive discovery phase that iteratively selects high‑confidence samples from unlabeled data. A class‑balanced selection strategy is applied to keep the evolving support set balanced, and experiments on the UCM and NWPU benchmarks show that SE‑CLIP outperforms existing semi‑supervised methods.

By Mohamed L. Mekhalfi, Mohamad M. Al Rahhal, Yakoub Bazi, Salah E. Khenfer, Mingdeng Shi, Hua Zou, Mansour Zuair
arXiv Computer Vision
Aug 27

Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures

The study evaluates automatic weld seam segmentation using RGB and polarimetric images, comparing convolutional neural networks (CNNs) and transformer architectures. In controlled RGB settings, CNNs achieve a mean mask mAP50 of up to 0.87, but performance drops significantly under uncontrolled conditions. Polarimetric imaging, combined with geometric augmentation, reaches a mean mask mAP50 of up to 093 even in uncontrolled settings, and transformer models, especially RF‑DETR, maintain high accuracy under viewpoint shifts while CNNs fail.

By Simone Garbin, Leonardo Venturoso, Marco Todescato
arXiv Computer Vision
Aug 27

Improving Cross-Site Whole-Heart Segmentation

The paper presents a modality‑routed 3D cardiac segmentation pipeline that combines TotalSegmentator‑initialized nnU‑Netv2 models with site‑characterized, label‑preserving appearance augmentation. By analyzing measurable image properties across sites, the authors design a bias‑field plus Bezier augmentation strategy that smooths spatial intensity perturbations and remaps intensities nonlinearly, followed by class‑wise largest‑connected‑component cleanup. On held‑out validation splits, this approach raises CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830 while reducing HD95, demonstrating improved cross‑site robustness in limited‑data whole‑heart segmentation.

By Tanish Mudaliar, Justin Li, Daniel Lin, Julianna Vo, Kaitao Liao, Xin Wang, Shu Hu
arXiv Computer Vision
Aug 27

UltraPIPS: Improving model perception in B-mode ultrasound with foundation models

UltraPIPS introduces domain‑specific foundation models for measuring perceptual similarity in B‑mode ultrasound images. The study shows that ultrasound‑trained LPIPS backbones better correlate with downstream tasks such as classification, segmentation, and reconstruction than natural‑image or general medical models. Optimizing LPIPS loss with an ultrasound backbone yields a strong balance between reconstruction quality and realism, and the authors provide an open‑source library for these metrics.

By Tal Grutman, Tali Ilovitsh
arXiv Computer Vision
Aug 27

Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming

Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming proposes PoseOFF, a representation that captures local motion around human joints by conditioning optical flow extraction on pose. This structured motion representation aligns with human kinematics and improves early action recognition accuracy across multiple datasets and backbones. PoseOFF achieves comparable or better performance while observing less of the action sequence, making it suitable for real‑time, resource‑constrained robotic systems.

By Lewis de Zoete Grundy, Chris McCarthy, Christopher Fluke
arXiv Computation and Language
Aug 27

Padamitra: Grounded Glossary Generation for Classical Sanskrit

The paper introduces grounded glossary generation, a structured NLP task that asks models to recover semantically meaningful Sanskrit phrases and provide translation‑grounded meanings from a sloka‑translation pair, mirroring the traditional patha commentary practice. A benchmark of 31,316 sloka‑translation‑glossary triples from the Valmiki Ramayana and Srimad Bhagavatam is built, evaluated with Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Experiments with Gemma‑3n‑E4B, Gemma‑3‑12B, Phi‑4, and Qwen3.5‑9B show that instruction fine‑tuning outperforms prompting, and explicit segmentation further improves results, though over‑segmentation of sandhi and samasa compounds remains the main error source, highlighting morphological modeling as a key bottleneck.

By Manoj Balaji Jagadeeshan, Sai Pragnaan Marala, Pawan Goyal
arXiv Computer Vision
Aug 27

Semantics-Aware Hierarchical Consensus Learning for Remote Sensing Image Classification

The paper introduces Semantics-Aware Hierarchical Consensus (SAHC), a deep learning framework that incorporates hierarchical-level-specific classification heads and cross-level probability projectors to leverage predefined label hierarchies in remote sensing image classification. SAHC fuses direct and projected predictions into a geometric consensus distribution, enabling self-consistent training and optional hierarchy-aware inference. Experiments on two benchmark datasets demonstrate the method’s effectiveness in guiding network learning and its robustness across varying spectral and spatial resolutions.

By Giulio Weikmann, Gianmarco Perantoni, Lorenzo Bruzzone
arXiv Machine Learning
Aug 27

SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning

The paper introduces SAME (Stabilized Mixture-of-Experts) to address challenges in Multimodal Continual Instruction Tuning (MCIT) for large language models. SAME mitigates router drift by decomposing routing dynamics into orthogonal subspaces and updating only task-relevant directions, while preventing expert drift through curvature‑aware scaling that uses historical input covariance without rehearsal. The method also employs adaptive expert activation to freeze selected experts during training, reducing redundant computation and cross‑task interference, and demonstrates state‑of‑the‑art performance on a new long‑task‑sequence benchmark.

By Zhen-Hao Xie, Jun-Tao Tang, Yu-Cheng Shi, Han-Jia Ye, De-Chuan Zhan, Da-Wei Zhou
arXiv Machine Learning
Aug 27

Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity

Robust CurveMoE is a mixture‑of‑experts framework that protects neural networks against perturbations defined by multiple norm constraints. It connects norm‑specialized models through a low‑loss path, selectively expertises only influential layers, and shares the rest of the parameters across routing paths. The method introduces contribution‑guided partial updating to reduce curve‑construction cost and provides a theoretical bound on the objective gap between partial and full optimization, achieving consistent improvements in clean, norm‑specific, and Union accuracy on CIFAR‑100 and ImageNet‑100.

By Xu Zhang, Ren Wang
arXiv Machine Learning
Aug 27

Learning from waste: Machine Learning for health risk prediction and computer vision-based sorting in Ghana

The study applies machine learning to address Ghana’s solid waste disposal challenges, using a Random Forest classifier to predict illness categories from waste practices and demographics, achieving a macro F1 score of 0.63. A MobileNetV2 image classifier was also developed for automated waste sorting, reaching 88.2% accuracy and a macro F1 of 0.87 on 415 images. These quantitative results confirm a previously qualitative link between waste disposal and health, while demonstrating the feasibility of low‑cost, camera‑based sorting in resource‑constrained settings.

By Hilda Adwubi Osei, Catherine Tenewaa Osei, Desdemona Yaa Asobayire
arXiv Computer Vision
Aug 27

Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation

The paper proposes Difficulty-Aware Sample Allocation (DASA), a framework that assigns stronger data augmentation to training samples deemed more difficult based on a composite difficulty score. This score integrates prediction ambiguity, training loss, class rarity, and boundary complexity, and is used to modulate augmentation strength during iterative training. Experiments on Oxford‑IIIT Pet and binary Pascal VOC with U‑Net, DeepLabV3, and SegFormer‑B0 demonstrate that DASA outperforms standard training and matches or exceeds single‑signal adaptive baselines, notably raising DeepLabV3’s mIoU from 0.633 to 0.740 on Oxford‑IIIT Pet.

By Olasimbo Ayodeji Arigbabu, Abimbola Ismail Arigbabu