arXiv Computer Vision

IKMo: Image-Keyframed Motion Generation with Trajectory-Pose Conditioned Motion Diffusion Model

The paper introduces TPMDM, a diffusion model that decouples and aligns heterogeneous geometric constraints—user-defined global trajectories and local image-derived poses—through a parallel control strategy. Building on TPMDM, the IKMo framework integrates a VLM-powered multi-agent system to translate user-friendly inputs into explicit motion constraints, enabling precise animation from unaligned, in-the-wild 2D reference images. Experiments on HumanML3D and KIT-ML show that IKMo achieves state‑of‑the‑art performance for motion generation under mixed trajectory and keyframe pose constraints.

arXiv Computer Vision
Aug 24

Driving with DINO: Vision Foundation Features as a Unified Bridge for Sim-to-Real Generation in Autonomous Driving

The paper introduces Driving with DINO (DwD), a framework that uses Vision Foundation Module (VFM) features to bridge simulation and real-world domains for autonomous driving video generation. It addresses the consistency‑realism dilemma by projecting VFM features onto a principal subspace, dropping high‑frequency texture elements, and applying a Random Channel Tail Drop to preserve structural detail. Additional components— a learnable Spatial Alignment Module and a Causal Temporal Aggregator— enhance control precision, spatial alignment, and temporal stability, reducing motion blur and ensuring realistic, consistent outputs.

By Xuyang Chen, Conglang Zhang, Chuanheng Fu, Zihao Yang, Kaixuan Zhou, Yizhi Zhang, Yanfeng Zhang, Mingwei Sun, Zhen Dong, Xiaoxiao Long, Zengmao Wang, Liqiu Meng
arXiv Computer Vision
Oct 12

Destination-Conditioned Adaptation with Semantic-Geometric Alignment for Generative Inbetweening

The paper introduces DSG‑GI, a method that adapts pretrained latent video diffusion models for generative inbetweening by using Destination‑Conditioned Adaptation and Semantic‑Geometric Alignment. Destination‑Conditioned Adaptation treats the target frame as the sequence’s endpoint, providing progressive guidance along the generated trajectory. Semantic‑Geometric Alignment aligns token‑wise spatio‑temporal geometry of backbone features with a frozen video encoder and enforces a flow‑matching latent flow constraint near the data endpoint, thereby improving temporal stability, target‑frame faithfulness, and perceptual quality on large‑motion benchmarks.

By Zhiyu Xu (Tianjin University), Wei Shang (Singapore Management University), Yushen Xu (Tianjin University), Wenwen Li (Tianjin Medical University), Zhiyan Xu (Southeast University), Pengfei Zhu (Tianjin University), Dongwei Ren (Tianjin University)
arXiv Computer Vision
Oct 16

ControlFlow3D: Distilling Multi-View Knowledge into Latent Flow Matching for Point Cloud Upsampling

ControlFlow3D is a framework that upsamples point clouds by distilling knowledge from pretrained foundation models into a lightweight generation network. It performs conditional flow matching in a latent space defined by a frozen 3D encoder, using unsupervised soft KMeans to cluster multi‑view image tokens and 3D latent tokens into shared prototypes that guide the flow trajectory. A Mamba‑based geometry decoder then maps latent features to dense coordinates with linear complexity, and the authors introduce ShapeNetPU, a 35K‑object multi‑modal benchmark 31 times larger than previous datasets.

By Yuang Liu, Zhi Zuo, Zhengkai Zhao, Lirui Zhang, Pan Gao, Hao Feng, Zhengzhe Liu