arXiv Computer Vision

Speculative Sampling in Interactive Video Generation

arXiv Computer Vision
Oct 16

Dynamic Video Generation: Shaping Video Generation Across Time and Space

The paper introduces DVG, a Dynamic Video Generation framework that allocates computation across time and space to accelerate diffusion-based video generation. By selecting content-aware acceleration strategies automatically, DVG achieves near-lossless speedups—up to 7× on HunyuanVideo models and 18× when combined with distillation—without manual tuning or retraining. The authors provide code on GitHub for reproducibility.

By Shikang Zheng, Jingkai Huang, Jiacheng Liu, Guantao Chen, Lixuan, Yuqi Lin, Peiliang Cai, Linfeng Zhang
arXiv Computer Vision
6d ago

Don't Pause! Every prediction matters in a streaming video

The paper introduces SPOT‑Bench, a new benchmark for streaming video that evaluates models on proactive, multi‑turn queries rather than retrospective, fixed‑timestamp questions. It proposes the Timeliness‑F1 metric to assess temporal precision and coverage across the entire video. The study finds that current multimodal large language models (MLLMs) can detect events reliably but often spam predictions, and that post‑training silence reduces spamming at the cost of responsiveness, revealing a significant dead‑time issue in streaming inference.

By Dibyadip Chatterjee, Zhanzhong Pang, Fadime Sener, Yale Song, Angela Yao
arXiv AI
Jul 23

StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation

arXiv:2607. 20174v1 Announce Type: cross Abstract: Existing human--object interaction (HOI) video generation methods are largely limited to offline short-video generation with complex driving conditions, making them unsuitable for real-time interactive applications.

By Zejing Rao, Haoxian Zhang, Xiaoqiang Liu, Yiping Meng, Guoxin Zhang, Pengfei Wan, Fan Tang, Tong-Yee Lee
arXiv Computer Vision
Sep 30

APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants

arXiv:2609.37559v1 Announce Type: new Abstract: To serve as real-world personal assistants, streaming video models need persistent memory that retains past experiences for later use. Yet existing str...

By Jianguo Huang, Jinming Liu, Qiyao Wang, Liang Xu, Jianhang Li, Zhimian Wen, Mingda Li, Shule Lu, Zhicheng Wang, Yuhan Guo, Xin Jin, Wenjun Zeng