arXiv Computer Vision
Sep 3

TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views

TAPVid-MV is a new benchmark for tracking any point in 3D across multiple synchronized camera views. It comprises 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks, covering indoor and outdoor domains and derived from various modalities such as depth, LiDAR, SLAM, and simulation. The dataset is visually verified, and evaluation shows that current multi‑view trackers do not consistently outperform monocular trackers, highlighting geometry recovery as a key bottleneck.

By Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman, Yi Yang, Ignacio Rocco, Jeet Thakwani, Rishabh Kabra, Andrew Zisserman, Joao Carreira, Siyu Tang, Carl Doersch, Gabriel Brostow
arXiv AI
Sep 25

TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations

TrackEverything is a 3D point tracker that overcomes the trade‑off between sparse long‑horizon tracking and dense short‑clip tracking by representing videos as persistent 3D scene tracks in world coordinates. It introduces voxel‑based de‑duplication at sliding‑window boundaries, a two‑stage refinement process (endpoint refiner and lightweight trajectory refiner), and a 3D WAFT module that replaces memory‑heavy 4D correlation volumes with efficient feature sampling. The method can track all visible points in videos longer than 1000 frames using only 40 GB of GPU memory, outperforming existing dense trackers on short clips and matching sparse trackers on long sequences.

By Ayush Jain, Sreeharsha Paruchuri, Ishita Gupta, Fan Zhang, Tanner Schmidt, Jakob Engel, Katerina Fragkiadaki, Adam W. Harley
arXiv Computer Vision
Sep 23

Leveraging Vision-Based Point Cloud Map Priors for Camera-Based 3D Object Detection and Online Vectorized HD Mapping

The paper presents a framework that builds a static point cloud prior map from past camera traversals, augmenting each point with DINOv3 semantic features. During runtime, a local prior patch is retrieved, encoded with a sparse voxel backbone, and fused with lifted multi‑view camera features in bird’s‑eye view. This fused representation is then used by sparse transformer heads to predict 3D objects and vectorized map elements, achieving improved performance on Argoverse 2 without requiring LiDAR for prior‑map construction or online inference.

By Markus K\"appeler, Rohit Mohan, Abhinav Valada
arXiv Computer Vision
Aug 27

Point2Pose: Occlusion-Recovering 6D Pose Tracking and 3D Reconstruction for Multiple Unknown Objects Via 2D Point Trackers

Point2Pose is a model‑free method for causal 6D pose tracking of multiple rigid objects using monocular RGB‑D video. It starts from sparse image points and employs a 2D point tracker to maintain long‑range correspondences, allowing instant recovery after complete occlusion. The system also incrementally builds an online Truncated Signed Distance Function (TSDF) representation of the tracked objects and introduces a new multi‑object tracking dataset with motion‑capture ground truth.

By Tzu-Yuan Lin, Ho Jae Lee, Kevin Doherty, Yonghyeon Lee, Sangbae Kim
arXiv Computer Vision
Sep 21

PointLAM: Local Attentive Mamba for Efficient Point-based 3D Object Detection

PointLAM introduces a new point-based 3D object detection architecture that addresses efficiency and fidelity trade-offs inherent in LiDAR point cloud processing. It employs a Laplacian Point Sampler (LPS) to accelerate downsampling while preserving foreground structure, and a Local Hadamard Aggregator (LHA) that replaces costly continuous interactions with a topology‑aware gating mechanism. Combined with Bi‑Directional Mamba layers, the resulting Local Attentive Mamba (LAM) block delivers competitive performance on nuScenes and Waymo datasets, outperforming voxel‑based competitors in detecting small objects and handling extreme sparsity with a smaller computational footprint.

By Xuanming Shang, Weijia Zhang, Chao Ma