arXiv AI
Sep 10

Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics

Kairos is a new video dataset designed for fine-grained video-language modeling, featuring long-duration videos from ten minutes to half an hour. Each video is annotated with time-resolved labels that capture ongoing actions, entity appearances, attributes, interactions, and evolving contextual cues throughout the timeline. The dataset supports fine-grained evaluation, long-range modeling, reasoning, instruction data construction, representation learning, and video generation.

By Ruibo Ming, Lei Sun, Deheng Zhang, He Zhang, Jialu Li, Jian Wang, Zhendong Li, Mengshun Hu, Danda Pani Paudel, Luc Van Gool, Jinjin Gu
arXiv Computer Vision
Aug 26

SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

arXiv:2608.24334v1 Announce Type: new Abstract: Discrete motion representations have substantially advanced autoregressive text-to-motion generation. However, most motion tokenizers are optimized for...

By Tianlv Huang, Hetian Guo, Ziyi Cai, Song Wang, Yanping Zhang, Zipei Fan, Xuan Song, Guangming Wu, Xin Zheng
arXiv Computer Vision
4d ago

TKCAM: Text and Keyframe to Camera Trajectory Generation

TKCAM is a new framework for generating controllable camera motion conditioned on text and sparse keyframes, using a 12‑dimensional kinematic representation and a two‑stage masked transformer with residual vector quantization. It allows users to provide free‑form text prompts and RGB observations at selected timestamps, enabling temporally localized visual guidance for coherent trajectory synthesis. The authors also introduce the RealEstate10K‑Cap dataset and a Universal CLaTr Evaluator, showing that TKCAM outperforms recent baselines on FID, text‑motion matching, and retrieval metrics.

By Haozhe Yang, Zhiyang Dou, Zekai Gu, Cheng Lin, Wenping Wang, Yuan Liu, Taku Komura