arXiv Computer Vision

DMM-Align: Closed-Loop Optimization for 2D-3D Registration with Dual-Role Diffusion

DMM-Align introduces a closed‑loop framework for 2D‑3D registration that jointly refines correspondences, estimates pose, and learns representations using a shared differentiable geometric state. The method employs two diffusion processes: a geometry‑aware diffusion that improves the soft matching matrix for robust correspondence estimation, and a geometry‑conditioned diffusion teacher that feeds pose‑induced supervision back into feature learning. Experiments on 7‑Scenes and RGB‑D Scenes V2 show that DMM‑Align outperforms strong baselines, particularly in low‑overlap and heavily occluded scenarios, demonstrating the value of closed‑loop geometric feedback.

arXiv Computer Vision
Sep 23

GRIP: Gaussian Rendering as a Cross-Modal Bridge for Image-to-Point Cloud Registration

GRIP is a pose‑conditioned refinement framework that improves pixel‑to‑point matching for image‑to‑point‑cloud registration. It mitigates the mismatch between grid‑based image descriptors and unordered point cloud descriptors by softly rendering learned 3D point features onto the image grid using Gaussian feature splatting. The resulting rendered point‑derived feature map is fused with image features via a pixel‑aligned transformer, enabling visual semantic and geometric cues to interact in a shared 2D representation, which is then decoded and propagated to finer resolutions for dense correspondence estimation and final pose refinement. Experiments on RGB‑D Scenes V2 and 7 Scenes show state‑of‑the‑art inlier ratios and competitive registration recall, especially under stricter evaluation thresholds.

By Karim Slimani, Catherine Achard, Eric Marchand, Brahim Tamadazte
Hugging Face Trending Papers
Aug 27

DPA-I2P: Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration in Autonomous Driving

The paper introduces DPA-I2P, a depth‑guided projective alignment method for image‑to‑point‑cloud registration in autonomous driving. It employs Ray‑Conditioned Metric Depth Encoding and Projection‑Consistent Vision Lifting to align depth and visual cues geometrically, and uses Cross‑Modal Query Pruning to filter unreliable matches during refinement. Experiments on KITTI and nuScenes show significant improvements, reducing rotation and translation errors by up to 55.6% compared to existing implicit baselines.

arXiv Computer Vision
Sep 14

Spectral Consistency-Guided Multiview Point Cloud Registration for Low-Overlap Scenes

The paper introduces GMPCR, a non‑learning spectral consistency‑guided framework for multiview point cloud registration in low‑overlap scenes. GMPCR refines initial correspondences into a second‑order compatibility structure, uses spectral analysis to filter unreliable matches and select informative scan pairs, and then applies maximal‑clique hypothesis generation for robust relative transformations. The resulting sparse pose graph is further refined with an adaptive history‑aware synchronization scheme, and a recovery mechanism allows previously down‑weighted edges to regain confidence, achieving high registration recalls on benchmark datasets while reducing computational cost.

By Tianyu Li, Yanghong Lin, Shudong Zhou, Kui Yang, Jingru Zhang, Li Fang, Wei Yao
arXiv Computer Vision
Aug 28

DPA-I2P: Depth-Guided Projective Alignment for Image-to-Point-Cloud Registration in Autonomous Driving

The paper introduces DPA-I2P, a depth-guided projective alignment method for image-to-point-cloud registration in autonomous driving. It employs Ray-Conditioned Metric Depth Encoding and Projection-Consistent Vision Lifting to align depth and visual cues geometrically, and uses Cross-Modal Query Pruning to enhance matching stability. Experiments on KITTI and nuScenes show significant reductions in rotation and translation errors compared to existing implicit baselines.

By Wenxin Zhang, Hang Li, Zhiwei Xu, Qiankun Dong, Gang Wang, Tao Li
arXiv Computer Vision
Sep 16

PriorPose: Reference-Guided Joint Deformation and Alignment for Category-Level Object Pose Estimation

PriorPose introduces a reference-guided correspondence framework for category-level object pose estimation that jointly performs canonicalization and alignment in a shared feature space. By embedding partial observations and a category prior as token sets in a seeded transformer, the network predicts per-point NOCS fields and a canonical deformation, while a deep pose head regresses the similarity transform. A two-part shape consistency objective couples correspondence, deformation, and pose, reducing reliance on memorized canonical orientations and avoiding error cascades, leading to state-of-the-art results on standard and larger-category benchmarks, especially under strict pose thresholds.

By Yihan Chen, Huan Ren, Wenfei Yang, Hang Du, Tianzhu Zhang, Feng Wu
arXiv AI
Sep 17

Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration

The paper introduces Mask 2D-3D, an Adaptive Dual-Masked Autoencoder Network designed for image-to-point cloud registration. It proposes an Intermodal Dual-MAE Framework (ID-MAE) with a Similarity-based RL Masking Strategy (SRLM) that adaptively masks informative positions using cross-modal similarity and reinforcement learning. Experiments on RGB-D Scenes v2 and 7-Scenes benchmarks demonstrate state-of-the-art performance in this registration task.

By Zhixin Cheng, Jiacheng Deng, Xiaotian Yin, Baoqun Yin, Richang Hong, Tianzhu Zhang
arXiv Computer Vision
Sep 11

BridgeMatch: Conditional Transport Bridges in Matching Matrix Space for 3D Deformable Registration

BridgeMatch is a two‑stage generative solver that preserves the full soft matching matrix for 3D deformable registration. Stage I uses denoising diffusion to estimate a global matching matrix at a coarse resolution, then lifts it to high resolution while maintaining hierarchy and rank constraints. Stage II refines this lifted matrix via a conditional transport bridge, implemented with either a deterministic Flow Matching ODE or a stochastic Brownian‑bridge SDE, and demonstrates improved correspondence accuracy and registration performance on 4DMatch, 4DLoMatch, CAPE, and DeepDeform datasets, especially in low‑overlap scenarios.

By Qianliang Wu, Haobo Jiang, Guangwei Gao, Shuo Chen, Jin Xie, Jian Yang, Yaqing Ding
arXiv Computer Vision
Aug 31

SUFLECA: Scaling Up Feature Learning for CAD-to-image Alignment

SUFLECA is a weakly supervised framework that improves zero‑shot CAD‑to‑image alignment by scaling geometry‑grounded feature learning using Normalized Object Coordinates across up to 12 real and synthetic datasets. It introduces a geometrically consistent matching algorithm that reliably establishes CAD‑to‑image correspondences, enabling accurate, sub‑second alignment without iterative pose refinement. On the ScanNet25k benchmark, SUFLECA achieves 32.8%/42.6% category/instance accuracy, outperforming the strongest zero‑shot baseline by 9.7/12.5 percentage points and surpassing existing pose‑supervised methods for the first time.

By Saad Ejaz, Miguel Fernandez-Cortizas, Javier Civera, Holger Voos, Jose Luis Sanchez-Lopez