TAPVid-MV is a new benchmark for tracking any point in 3D across multiple synchronized camera views. It comprises 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks, covering indoor and outdoor domains and derived from various modalities such as depth, LiDAR, SLAM, and simulation. The dataset is visually verified, and evaluation shows that current multi‑view trackers do not consistently outperform monocular trackers, highlighting geometry recovery as a key bottleneck.
By Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman, Yi Yang, Ignacio Rocco, Jeet Thakwani, Rishabh Kabra, Andrew Zisserman, Joao Carreira, Siyu Tang, Carl Doersch, Gabriel Brostow
TrackEverything is a 3D point tracker that overcomes the trade‑off between sparse long‑horizon tracking and dense short‑clip tracking by representing videos as persistent 3D scene tracks in world coordinates. It introduces voxel‑based de‑duplication at sliding‑window boundaries, a two‑stage refinement process (endpoint refiner and lightweight trajectory refiner), and a 3D WAFT module that replaces memory‑heavy 4D correlation volumes with efficient feature sampling. The method can track all visible points in videos longer than 1000 frames using only 40 GB of GPU memory, outperforming existing dense trackers on short clips and matching sparse trackers on long sequences.
By Ayush Jain, Sreeharsha Paruchuri, Ishita Gupta, Fan Zhang, Tanner Schmidt, Jakob Engel, Katerina Fragkiadaki, Adam W. Harley
arXiv:2603.17068v3 Announce Type: replace
Abstract: Structured 3D representations such as keypoints and meshes offer compact, expressive descriptions of deformable objects, jointly capturing geometri...
By Yeheng Zong, Yizhou Chen, Alexander Bowler, Chia-Tung Yang, Ram Vasudevan
The paper presents a framework that builds a static point cloud prior map from past camera traversals, augmenting each point with DINOv3 semantic features. During runtime, a local prior patch is retrieved, encoded with a sparse voxel backbone, and fused with lifted multi‑view camera features in bird’s‑eye view. This fused representation is then used by sparse transformer heads to predict 3D objects and vectorized map elements, achieving improved performance on Argoverse 2 without requiring LiDAR for prior‑map construction or online inference.
By Markus K\"appeler, Rohit Mohan, Abhinav Valada
Point2Pose is a model‑free method for causal 6D pose tracking of multiple rigid objects using monocular RGB‑D video. It starts from sparse image points and employs a 2D point tracker to maintain long‑range correspondences, allowing instant recovery after complete occlusion. The system also incrementally builds an online Truncated Signed Distance Function (TSDF) representation of the tracked objects and introduces a new multi‑object tracking dataset with motion‑capture ground truth.
By Tzu-Yuan Lin, Ho Jae Lee, Kevin Doherty, Yonghyeon Lee, Sangbae Kim
PointLAM introduces a new point-based 3D object detection architecture that addresses efficiency and fidelity trade-offs inherent in LiDAR point cloud processing. It employs a Laplacian Point Sampler (LPS) to accelerate downsampling while preserving foreground structure, and a Local Hadamard Aggregator (LHA) that replaces costly continuous interactions with a topology‑aware gating mechanism. Combined with Bi‑Directional Mamba layers, the resulting Local Attentive Mamba (LAM) block delivers competitive performance on nuScenes and Waymo datasets, outperforming voxel‑based competitors in detecting small objects and handling extreme sparsity with a smaller computational footprint.
By Xuanming Shang, Weijia Zhang, Chao Ma
arXiv:2506. 22784v2 Announce Type: replace-cross Abstract: Point-pixel registration between LiDAR point clouds and camera images is a fundamental yet challenging task in autonomous driving and robotic perception.
By Yu Han, Zhiwei Huang, Yanting Zhang, Fangjun Ding, Shen Cai, Xiaoyu Tang, Yanchao Dong, Rui Fan
arXiv:2601. 20720v2 Announce Type: replace-cross Abstract: End-to-end perception and trajectory prediction from raw sensor data is one of the key capabilities for autonomous driving.
By Matej Halinkovic, Nina Masarykova, Alexey Vinel, Marek Galinski
Physical AI Smart Spaces is the first benchmark that offers large‑scale, multi‑class, multi‑camera 3D perception data for indoor smart spaces. It includes over 280 hours of synchronized 1080p footage from nearly 1,800 cameras in warehouses, hospitals, and retail venues, with automatic annotations for identities, 2D and 3D bounding boxes, camera calibration, and depth. The benchmark spans synthetic generation, appearance augmentation, and real‑world Sim2Real evaluation, and introduces a 3D version of Higher Order Tracking Accuracy (HOTA) for evaluating multi‑class 3D box tracking.
By Yuxing Wang, Yizhou Wang, Anqi Li, Shuo Wang, Sameer Satish Pusegaonkar, Haoquan Liang, Jiajun Li, Shenxin Jiang, Jianhe Yuan, Shangru Li, Tongwei Dai, Zihao Chen, David C. Anastasiu, Sujit Biswas, Xunlei Wu, Zheng Tang
arXiv:2602. 23172v2 Announce Type: replace-cross Abstract: Capturing 4D spatiotemporal scene structure is crucial for the safe and reliable operation of robots in dynamic environments.
By Maximilian Luz, Rohit Mohan, Thomas N\"urnberg, Yakov Miron, Daniele Cattaneo, Abhinav Valada
MeshBind introduces an automatic annotation pipeline that generates per-point scene flow for non‑rigid objects such as pedestrians by reconstructing articulated mesh models from six surround cameras and registering them to LiDAR data. The method employs a local‑frame binding technique that attaches each point to a mesh via a per‑triangle tangent‑normal frame, and a three‑layer flow registration pipeline that compensates ego‑vehicle motion, performs box binding, and applies mesh binding to produce full‑scene flow. MeshBind is implemented in the SF‑Studio tool, and the authors release the SUScape‑Flow Benchmark (SFB), a large‑scale dataset with 41k frames of per‑point non‑rigid pedestrian annotations and a comprehensive evaluation protocol, demonstrating that fine‑tuning on SFB reduces endpoint error by 2‑3× and equalizes non‑rigid and rigid EPE.
By Yinqi Chen, E Li, Shuaijun Wang, Weiyuan Peng, Meiying Zhang, Qi Hao
FREE‑LIVO is a LiDAR‑Inertial‑Visual Odometry system that does not require a fixed LiDAR‑camera rig or precise time synchronization. It uses a pretrained image feature matcher to link camera images with dense LiDAR intensity images generated from a single scan, creating 3D point‑to‑pixel correspondences that form reprojection factors. These factors, along with point‑cloud registration, visual feature tracking, and IMU constraints, are jointly optimized in a factor graph, achieving an average ATE RMSE of 0.45 m on the M3DGR dataset and maintaining accuracy even with up to 0.5 s synchronization errors, while also enabling joint trajectory estimation for sensors on separate robots.
By Reon Tabata, Kenji Koide, Shuji Oishi, Taku Okawara, Mizuho Aoki, Aoki Takanose, Masashi Yokozuka, Jun Miura