arXiv AI
Sep 2

Beyond the Image Plane: World-Grounded Queries for Multi-Object Tracking

The paper introduces PLANET, a multi‑object tracker that transcends traditional image‑plane limitations by incorporating 3D scene geometry into its query formation. By lifting 2D tracking datasets into 3D and embedding reconstructed geometry into features and positional encodings, PLANET encourages queries to encode object positions. An auxiliary 3D location prediction task and a dual‑resolution temporal memory further enhance performance, enabling state‑of‑the‑art results on three diverse benchmarks.

By Orcun Cetintas, Guillem Bras\'o, Tim Meinhardt, Laura Leal-Taix\'e
Hugging Face Trending Papers
Jun 24

MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos.

Hugging Face Trending Papers
Jul 20

CDIS: Cross-Dimensional Class-Agnostic 3D Instance Segmentation via 2D Mask Tracking and 3D-2D Projection Merging

Class-agnostic 3D instance segmentation is critical for robotic systems operating in unknown environments, enabling perception of previously unseen objects for reliable manipulation and navigation. Existing approaches typically project per-frame 2D instance masks into 3D and merge them, which often breaks object identities across time and yields fragmented 3D instances.