arXiv Computer Vision

Teaching DINOv3 About Partial 3D Geometry: A Self-Supervised Geometry-Aware Approach

The paper introduces a self‑supervised, geometry‑aware method that refines DINOv3 features with LoRA to better handle partial 3D shapes. By training on partial observations, the resulting descriptors outperform traditional and other vision‑foundation features in partial shape matching tasks. When incorporated into existing pipelines, the approach achieves state‑of‑the‑art results on both partial shape matching and left‑right prediction benchmarks.

arXiv Computer Vision
Aug 28

DINOcular: Self-Supervised Visuospatial Representations

DINOcular is a self‑supervised framework that learns joint visuospatial representations from RGB‑D observations. It fuses depth‑derived geometric priors with a visual backbone using inter‑patch and intra‑patch fusion, allowing the model to encode both appearance and spatial structure efficiently. The resulting representation improves 3D awareness on multiple geometry benchmarks while staying competitive on standard RGB‑D semantic segmentation tasks.

By Farkhat Almukhamedov, Sami Azirar, Hermann Blum
Hugging Face Trending Papers
Aug 27

DINOcular: Self-Supervised Visuospatial Representations

DINOcular is a self‑supervised framework that learns joint visuospatial representations from RGB‑D data. It fuses depth‑derived geometric priors with a visual backbone using inter‑patch and intra‑patch techniques, allowing the model to encode both appearance and spatial structure efficiently. The resulting representation improves 3D awareness on multiple geometry benchmarks while staying competitive on standard RGB‑D semantic segmentation tasks.

arXiv Computer Vision
Oct 5

Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration

Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration proposes CDPM, a method that first aligns semantic representations across modalities and then refines correspondences with fine-grained CNN features. CDPM adapts DINOv3 using geometrically consistent cross-modal patch pairs, builds a DINO-Centric Feature Pyramid for stable cross-modal matching, and adds a lightweight CNN branch for precise local refinement. Experiments on three cross-modal datasets show that CDPM outperforms existing dense matchers, improving AUC metrics and reducing mACE while using fewer FLOPs.

By Zhiwei Wang, Defeng He, Yuxing Li, Meilu Zhu, Edmund Y. Lam
arXiv Computer Vision
Sep 4

TokenMatch: 3D Mesh Correspondence Transformer with Curvature-Guided Tokenisation

TokenMatch is a transformer-based model that estimates 3D shape correspondences by adaptively tokenising meshes into curvature-guided patches. Trained only on the BeCoS partial-to-partial dataset, it generalises to full-shape matching without retraining, using self‑ and cross‑attention to learn patch‑ and point‑level relations. Evaluated on CP2P, PSMAL, BeCoS, FAUST, SCAPE, and SHREC'19, TokenMatch consistently outperforms existing methods in mean geodesic error and intersection‑over‑union while achieving sub‑second inference speeds.

By Adeela Islam, Zorah L\"ahner, Vittorio Murino, Vladislav Golyanik
arXiv Computer Vision
Sep 3

Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning

The paper introduces a generalizable deformation learning framework that reconstructs 3D objects by deforming a category-level shape template to match a monocular observation. It employs a geometry-guided feature modeling mechanism to enrich foundation features with template topology, creating a geometry-aware representation that is explicitly correlated with the target observation for precise deformation. A view-adaptive feature aggregation module further bridges the gap between the fixed template and arbitrary target views by leveraging multi-view template features and camera poses, ensuring robust feature alignment across diverse viewpoints.

By Yiyao Ma, Kai Chen, Zhongxiang Zhou, Zhuheng Song, Dongsheng Xie, Zelong Tan, Rong Xiong, Qi Dou
arXiv Computer Vision
Oct 9

ShapeY: A Principled Framework for Measuring Shape Recognition Capacity via Nearest-Neighbor Matching

ShapeY is a benchmarking framework that evaluates shape-based recognition in object recognition systems using 68,200 grayscale images of 200 3D objects rendered from multiple viewpoints, with optional appearance changes. It employs a nearest‑neighbor matching task to probe how well an embedding space clusters views by 3D shape similarity across varying viewpoints and non‑shape alterations, providing quantitative and qualitative performance readouts such as error rates, viewpoint tuning curves, and matching score histograms. Testing 321 pre‑trained networks shows that even state‑of‑the‑art models struggle to generalize robustly across 3D viewpoint and appearance changes, highlighting gaps in current shape recognition capabilities.

By Jong Woo Nam, Amanda S. Rios, Bartlett W. Mel