← Back to all news
arXiv AI By Roy Amoyal, Shira Ifergane, Oren Freifeld

SemGeo-Gen: Unsupervised Generation of Approximate Cross-Instance Semantic-Geometric Correspondences

Read the original on arXiv AI →

SemGeo-Gen is an unsupervised approach that generates approximate cross‑instance semantic‑geometric correspondences from weakly aligned 3D object collections. It uses coarse category‑level rotation alignment, lifts multi‑view DINOv2 features to 3D, applies continuous piecewise‑affine registration, and prunes matches with semantic consistency to produce 3D correspondences that can be projected into 2D views for scalable supervision. The method achieves high accuracy against human annotations and improves state‑of‑the‑art semantic correspondence models after synthetic pretraining.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Oct 5

Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration

Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration proposes CDPM, a method that first aligns semantic representations across modalities and then refines correspondences with fine-grained CNN features. CDPM adapts DINOv3 using geometrically consistent cross-modal patch pairs, builds a DINO-Centric Feature Pyramid for stable cross-modal matching, and adds a lightweight CNN branch for precise local refinement. Experiments on three cross-modal datasets show that CDPM outperforms existing dense matchers, improving AUC metrics and reducing mACE while using fewer FLOPs.

By Zhiwei Wang, Defeng He, Yuxing Li, Meilu Zhu, Edmund Y. Lam
multimodal
More like this →
arXiv AI
Sep 10

GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting

GoDeep is an annotation‑free method for open‑vocabulary 3D scene understanding that uses a vision‑language model solely as a translator to generate structured, entity‑level descriptions of each image. These descriptions are projected and aggregated in a language‑only embedding space, eliminating the need for a 3D training corpus or domain‑specific encoder. The approach achieves competitive performance on ScanNet++ and a cultural heritage benchmark, accurately localizes out‑of‑vocabulary objects, and offers explainable, point‑level predictions.

By Thodoris Betsas, Anastasios Doulamis, Andreas Georgopoulos
llmsragcomputer-visionmultimodalbenchmarks
More like this →
arXiv Computer Vision
Sep 4

TokenMatch: 3D Mesh Correspondence Transformer with Curvature-Guided Tokenisation

TokenMatch is a transformer-based model that estimates 3D shape correspondences by adaptively tokenising meshes into curvature-guided patches. Trained only on the BeCoS partial-to-partial dataset, it generalises to full-shape matching without retraining, using self‑ and cross‑attention to learn patch‑ and point‑level relations. Evaluated on CP2P, PSMAL, BeCoS, FAUST, SCAPE, and SHREC'19, TokenMatch consistently outperforms existing methods in mean geodesic error and intersection‑over‑union while achieving sub‑second inference speeds.

By Adeela Islam, Zorah L\"ahner, Vittorio Murino, Vladislav Golyanik
llmsfine-tuningbenchmarkssafety
More like this →