Track2Art: Articulated Object Model Recovery with Visual-Geometric Track Representations
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Track2Art is a motion‑centric framework that recovers articulated object models from RGB‑D interaction videos by lifting 2D point tracks into 3D trajectories. It groups these trajectories into rigid‑part hypotheses and uses learned‑analytic reasoning to infer directed kinematic relations, joint types, and joint geometry. On the PartNet‑Mobility benchmark, it achieves 0.695 Point IoU and 0.410 end‑to‑end J@20 without requiring ground‑truth part counts or test‑time optimization.
arXiv:2609.19119v1 Announce Type: new Abstract: Human videos contain rich causal evidence for robot manipulation: they reveal how hand motion induces object motion and produces task-relevant changes...
The paper introduces MoMa-SG, a framework that constructs semantic‑kinematic 3D scene graphs for articulated scenes using RGB‑D sequences. It segments interactions, tracks points robustly, estimates joint parameters in a single optimization, and associates objects with articulations to detect contained items. The authors also present the Arti4D‑Semantic dataset with 62 real‑world sequences and demonstrate the system’s effectiveness on a quadruped and a mobile manipulator.
Artic-O is an end‑to‑end, feed‑forward framework that reconstructs articulated objects from sparse images by learning latent geometry. It maps multi‑state observations into a pretrained latent geometry space, uses a frozen flow‑matching decoder for complete‑shape priors, and fuses visual tokens with geometry latents in an image‑grounded part‑reasoning module to segment active parts and predict articulation. Trained with a geometry‑to‑articulation curriculum and a decoupled two‑pass strategy, Artic‑O achieves high reconstruction quality and articulation accuracy while drastically reducing inference time from 9 minutes to about 0.3 seconds per object.
MoRE introduces a Motion-Regularized Encoding for generating articulated 3D objects from a single image. It encodes joint attributes as coordinate‑aligned joint lines with axis classification and Plücker moment coefficients, and couples attribute reconstruction to motion through a differentiable expectation over joint types and axis directions. Experiments on PartNet‑Mobility and ACD demonstrate improved part‑box agreement, articulated‑state geometry, visual similarity, and reduced joint prediction errors compared to prior methods.
World models endow perceptual systems with the ability to predict how scenes evolve under interaction. They are most beneficial when trained on diverse volumes of data, to instill a rich prior into do...