← Back to all news
arXiv AI By Orcun Cetintas, Guillem Bras\'o, Tim Meinhardt, Laura Leal-Taix\'e

Beyond the Image Plane: World-Grounded Queries for Multi-Object Tracking

Read the original on arXiv AI →

The paper introduces PLANET, a multi‑object tracker that transcends traditional image‑plane limitations by incorporating 3D scene geometry into its query formation. By lifting 2D tracking datasets into 3D and embedding reconstructed geometry into features and positional encodings, PLANET encourages queries to encode object positions. An auxiliary 3D location prediction task and a dual‑resolution temporal memory further enhance performance, enabling state‑of‑the‑art results on three diverse benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 25

TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations

TrackEverything is a 3D point tracker that overcomes the trade‑off between sparse long‑horizon tracking and dense short‑clip tracking by representing videos as persistent 3D scene tracks in world coordinates. It introduces voxel‑based de‑duplication at sliding‑window boundaries, a two‑stage refinement process (endpoint refiner and lightweight trajectory refiner), and a 3D WAFT module that replaces memory‑heavy 4D correlation volumes with efficient feature sampling. The method can track all visible points in videos longer than 1000 frames using only 40 GB of GPU memory, outperforming existing dense trackers on short clips and matching sparse trackers on long sequences.

By Ayush Jain, Sreeharsha Paruchuri, Ishita Gupta, Fan Zhang, Tanner Schmidt, Jakob Engel, Katerina Fragkiadaki, Adam W. Harley
benchmarks
More like this →
Hugging Face Trending Papers
Jun 24

MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos.

diffusionbenchmarks
More like this →