arXiv Computer Vision

Motion Cues from Image-based Point Tracking for LiDAR Scene Flow Estimation

TrackCue is a self‑supervised framework that enhances LiDAR scene flow estimation by using dense image‑based point tracking to generate reliable static‑dynamic supervision. It anchors tracking queries to projected LiDAR points, applies a visually consistent motion compensation strategy to isolate true object motion, and lifts image‑space motion cues back to LiDAR points for refined pseudo‑labels. This approach reduces noise from sparse or occluded LiDAR geometry and improves the precision of dynamic pseudo‑labels, leading to better scene flow learning.

Hugging Face Trending Papers
Aug 19

RVLoss: Runoff Vote Loss for Self-Supervised LiDAR Scene Flow Estimation

RVLoss introduces a runoff vote mechanism for self‑supervised LiDAR scene flow estimation, addressing motion rigidity by grouping nearest‑neighbor derived motions into dominant flow candidates and selecting the most consistent one through a two‑stage voting process. This approach generates cluster‑wise rigid flows and free‑form flows as pseudo‑labels, enabling seamless integration into existing feedforward architectures. Experiments on the Argoverse2 2026 Challenge demonstrate that models trained with RVLoss outperform baseline self‑supervised methods by 20% and maintain consistent gains across four additional datasets.

arXiv Computer Vision
Sep 3

TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views

TAPVid-MV is a new benchmark for tracking any point in 3D across multiple synchronized camera views. It comprises 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks, covering indoor and outdoor domains and derived from various modalities such as depth, LiDAR, SLAM, and simulation. The dataset is visually verified, and evaluation shows that current multi‑view trackers do not consistently outperform monocular trackers, highlighting geometry recovery as a key bottleneck.

By Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman, Yi Yang, Ignacio Rocco, Jeet Thakwani, Rishabh Kabra, Andrew Zisserman, Joao Carreira, Siyu Tang, Carl Doersch, Gabriel Brostow
arXiv AI
Sep 25

TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations

TrackEverything is a 3D point tracker that overcomes the trade‑off between sparse long‑horizon tracking and dense short‑clip tracking by representing videos as persistent 3D scene tracks in world coordinates. It introduces voxel‑based de‑duplication at sliding‑window boundaries, a two‑stage refinement process (endpoint refiner and lightweight trajectory refiner), and a 3D WAFT module that replaces memory‑heavy 4D correlation volumes with efficient feature sampling. The method can track all visible points in videos longer than 1000 frames using only 40 GB of GPU memory, outperforming existing dense trackers on short clips and matching sparse trackers on long sequences.

By Ayush Jain, Sreeharsha Paruchuri, Ishita Gupta, Fan Zhang, Tanner Schmidt, Jakob Engel, Katerina Fragkiadaki, Adam W. Harley
Hugging Face Trending Papers
Jun 24

MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos.

arXiv Computer Vision
Aug 27

Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models

The paper introduces LiDAR‑SAM2, a framework that converts the 2D video foundation model SAM2 into a scalable source of supervision for 4D LiDAR data. By projecting SAM2 video masks into multi‑view LiDAR space and aggregating them temporally, the method automatically generates temporally coherent LiDAR labels without human annotation. Experiments on SemanticKITTI show that these automatically produced semantic and panoptic labels achieve quality close to full human annotation, enabling models trained on them to approach the performance of fully supervised systems.

By Jihun Kim, Hyun-Kurl Jang, Hyemin Yang, Jinnyeong Yang, Hyeokjun Kweon, Kuk-Jin Yoon
arXiv Computer Vision
Oct 5

LightLoc++: Sensor-Robust Representation Learning for Efficient Outdoor LiDAR Localization

LightLoc++ introduces a sensor‑robust framework for outdoor LiDAR localization that decouples a pretrained backbone from lightweight scene‑specific heads, enabling efficient new‑scene training. The authors present SULID, a synchronized urban multi‑LiDAR dataset featuring 32‑, 64‑, and 128‑beam sensors, which is used to pretrain a backbone via cross‑sensor consistency learning. Additional techniques such as sample classification guidance and redundant sample downsampling reduce regression ambiguity and computational load, achieving state‑of‑the‑art performance with minimal new‑scene training cost.

By Wen Li, Shangshu Yu, Dunqiang Liu, Shaoyang Chen, Qiming Xia, Sheng Ao, Siqi Shen, Chenglu Wen, Cheng Wang
arXiv Computer Vision
Oct 12

MeshBind: Point-to-Mesh Binding for Non-Rigid Scene Flow Automatic Annotation

MeshBind introduces an automatic annotation pipeline that generates per-point scene flow for non‑rigid objects such as pedestrians by reconstructing articulated mesh models from six surround cameras and registering them to LiDAR data. The method employs a local‑frame binding technique that attaches each point to a mesh via a per‑triangle tangent‑normal frame, and a three‑layer flow registration pipeline that compensates ego‑vehicle motion, performs box binding, and applies mesh binding to produce full‑scene flow. MeshBind is implemented in the SF‑Studio tool, and the authors release the SUScape‑Flow Benchmark (SFB), a large‑scale dataset with 41k frames of per‑point non‑rigid pedestrian annotations and a comprehensive evaluation protocol, demonstrating that fine‑tuning on SFB reduces endpoint error by 2‑3× and equalizes non‑rigid and rigid EPE.

By Yinqi Chen, E Li, Shuaijun Wang, Weiyuan Peng, Meiying Zhang, Qi Hao
arXiv Computer Vision
Oct 22

AutoExpert: Automating 3D LiDAR Annotation from Expert-Crafted Guidelines

AutoExpert proposes a new benchmark for automating 3D LiDAR annotation using expert‑crafted guidelines, redefining evaluation protocols and re‑annotating nuScenes and PandaSet datasets. The pipeline combines 2D detection and segmentation in RGB images, lifts these detections into 3D with known sensor poses, and generates 3D cuboids, improving 3D detection mAP from 12.1 to 25.4 on the AutoExpert‑nuScenes benchmark.

By Yechi Ma, Wei Hua, Shu Kong
arXiv Computer Vision
Sep 25

SplatLabel: Pseudo-Labelling through 4D Gaussian Splatting

SplatLabel is an automated pipeline that uses a 4D Gaussian representation to generate LiDAR segmentation and semantic occupancy grids with predictive confidence. It models dynamic scenes through an explicit temporal manifold, tracking moving actors without requiring pre‑annotated 3D bounding boxes. By integrating 360‑degree LiDAR depth maps and distilling soft probabilities from 2D models, it resolves semantic ambiguities over time and space, and evaluates pseudo‑labels via a selective classification framework that balances precision and recall.

By Nitya Nanvani, Andras Palffy, Holger Caesar