arXiv AI By Dazhao Du, Shiyan Du, Jian Liu, Yongjian Yu, Bohai Gu, Tao Han, Jie Zhang, Hualuo Liu, Eric Liu, Yujia Zhang, Xi Chen, Song Guo

CamChoreo: Benchmarking Temporally Grounded Compositional Camera Motion Understanding for Vision Language Models

Read the original on arXiv AI →

CamChoreo introduces a new benchmark for temporally grounded, compositional camera motion understanding, featuring 4,229 single-shot clips with expert temporal annotations across 20 direction-aware labels, many of which involve compound motions. The paper presents a training corpus of ~43K clips and evaluates models using supervised fine‑tuning, a 3D‑injected approach (CamInject), and a distilled method (CamDistill) that embeds geometric knowledge into lightweight camera tokens. CamDistill achieves significant gains over baseline fine‑tuning and demonstrates strong zero‑shot transfer to external benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computer Vision
Sep 7

Object Concepts Emerge from Motion

The paper introduces a biologically inspired framework that learns object‑centric visual representations from raw videos without human annotations or camera calibration. By using motion boundaries detected via optical flow and clustering to create pseudo‑instance masks, the method supervises a single‑image encoder with pixel‑level pairwise metric learning. Training on 195 million pseudo‑labeled frames and expanding to 421 million frames through Motion‑Verified Self‑Training, the approach yields Swin‑based encoders that outperform or match supervised and self‑supervised baselines on tasks such as monocular depth estimation, 3D object detection, 3D occupancy prediction, and end‑to‑end planning.

By Boshi Li, Xiaohui Wang, Xiaoyang Wu, Zhichao Li, Ya Yang, Naiyan Wang
arXiv Computer Vision
Oct 9

TKCAM: Text and Keyframe to Camera Trajectory Generation

TKCAM is a new framework for generating controllable camera motion conditioned on text and sparse keyframes, using a 12‑dimensional kinematic representation and a two‑stage masked transformer with residual vector quantization. It allows users to provide free‑form text prompts and RGB observations at selected timestamps, enabling temporally localized visual guidance for coherent trajectory synthesis. The authors also introduce the RealEstate10K‑Cap dataset and a Universal CLaTr Evaluator, showing that TKCAM outperforms recent baselines on FID, text‑motion matching, and retrieval metrics.

By Haozhe Yang, Zhiyang Dou, Zekai Gu, Cheng Lin, Wenping Wang, Yuan Liu, Taku Komura
arXiv AI
Jun 30

MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

arXiv:2606. 29531v1 Announce Type: cross Abstract: We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs.

By Weisong Liu, Haochen Wang, Kuan Gao, Yuhao Wang, Yikang Zhou, Zhongwei Ren, Jacky Mai, Anna Wang, Yanwei Li, Jason Li, Zhaoxiang Zhang