arXiv AI

FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation

FrameForge introduces a unified image generation framework that leverages pretrained video foundation models to capture dynamic temporal relationships between conditions and targets. It employs a Mixture of Temporal Experts (MoTE) to adaptively select temporal intervals and a Chain of Temporal Reasoning (CoTR) to model transformations through latent intermediate states, guided by a reasoning-bottleneck attention mask. Experiments on DreamBench, COCO2017, and GEdit-Bench demonstrate FrameForge’s superior performance in subject-driven generation, canny-to-image generation, and image editing compared to existing methods.

arXiv Computer Vision
Oct 1

VR-JEPA: Learning Contrastive-State Latent Guidance for Generation-based Video Reasoning

arXiv:2609.40129v1 Announce Type: new Abstract: Reasoning through video generation offers a promising path toward visual intelligence by modeling latent visual states and their dynamics. However, cur...

By Zehua Ma, Kun Xiang, Yunshuang Nie, Quanlin Chen, Haoyuan Li, Xiuwei Chen, Jiang Ji, Haijun Wu, Zhenyu Xie, Michael Kampffmeyer, Hanhui Li, Xiaodan Liang
arXiv Computer Vision
Oct 5

ProAR: Learning Prospective Reasoning with Autoregressive Video Models

ProAR introduces a goal‑oriented reasoning framework for autoregressive video models, adding goal‑frame prediction via an asymmetric attention mask and future representation self‑alignment to guide intermediate states. These mechanisms combine sparse target supervision with dense step‑wise guidance, enabling coherent, goal‑directed generation while keeping computational costs modest. Experiments demonstrate consistent performance gains across visual reasoning benchmarks and training efficiency, achieving comparable results to fully trained baselines with only 25% of the training steps.

By Linghui Shen, Tinghui Zhu, Sheng Zhang, Muhao Chen
arXiv AI
Aug 25

VGI-Bench: Probing Visual Intelligence in Video Generation Models

arXiv:2608.19583v2 Announce Type: replace-cross Abstract: Recent studies suggest that video generation models can exhibit certain forms of zero-shot visual reasoning through generated frames. Yet rel...

By Xuan He, Cong Wei, Yuhao Cheng, Linrui Ma, Yuxuan Zhang, Zuojun Li, Yuhao Wen, Jize Jiang, Zeyi Liu, Yuren Hao, Songcheng Cai, Keming Wu, Penghui Du, Kai Zou, Rui Yang, Chenkai Sun, Ke Yang, Ping Nie, Kelsey R Allen, Chenglong Wang, Michel Galley, Jianfeng Gao, ChengXiang Zhai
arXiv AI
Oct 2

Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs

The paper introduces Temporal Activation Injection (TAI), a post‑processing technique that reinforces temporal representations in Video Large Language Models (VideoLLMs) at inference time. By measuring the temporal divergence vector τ_l— the layer‑wise difference caused by reversing video frame order— the authors find that temporal information peaks at intermediate layers and fades toward the output. TAI injects the peak divergence back into later layers, improving temporal reasoning on three VideoLLMs across four benchmarks without affecting non‑temporal performance.

By Youngwoo Shin, Yusung Ro, Minseo Kim, Junmo Kim
arXiv AI
Oct 16

Temporal Specification Guidance for Text-to-Video Generation

The paper introduces Temporal Specification Guidance (TSG), a training‑free inference framework that uses Temporal Logic specifications to guide text‑to‑video generation. TSG evaluates visual concept traces with differentiable quantitative semantics and adjusts the denoising direction via gradient‑derived attention biases, eliminating the need for external classifiers or backpropagation through the transformer. Experiments on 220 prompts across 11 temporal constraint categories show that TSG improves specification satisfaction by 9.5–20 percentage points over classifier‑free guidance with minimal memory overhead.

By Avishree Khare, Hideki Okamoto, Bardh Hoxha, Georgios Fainekos, Rajeev Alur
arXiv AI
Aug 7

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

arXiv:2603. 08652v2 Announce Type: replace Abstract: Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integration of Chain-of-Thought (CoT) reasoning.

By Haodong Li, Chunmei Qing, Huanyu Zhang, Dongzhi Jiang, Yihang Zou, Hongbo Peng, Dingming Li, Yuhong Dai, ZePeng Lin, Juanxi Tian, Yi Zhou, Siqi Dai, Jingwei Wu, Pheng-Ann Heng
Hugging Face Trending Papers
Sep 2

TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval

TAME introduces a Temporal-Aware Mixture-of-Experts framework for Text-Video Retrieval that enhances CLIP-based models by incorporating frame-level structure and temporal relations. It adds sparse Mixture-of-Experts layers with frame-consistent routing, Frame-Temporal tokens for global cross-frame aggregation, and a Cross-Temporal Interaction and Aggregation module to refine sentence-video similarities. Experiments on multiple TVR benchmarks show consistent performance gains, such as a 4.0 R@1 improvement on MSR‑VTT over CLIP4Clip.

arXiv AI
Aug 21

VGI-BENCH: Probing Visual Intelligence in Video Generation Models

arXiv:2608. 19583v1 Announce Type: cross Abstract: Recent studies suggest that video generation models can exhibit certain forms of zero-shot visual reasoning through generated frames.

By Xuan He, Cong Wei, Yuhao Cheng, Linrui Ma, Yuxuan Zhang, Zuojun Li, Yuhao Wen, Zeyi Liu, Yuren Hao, Songcheng Cai, Keming Wu, Penghui Du, Kai Zou, Rui Yang, Chenkai Sun, Ke Yang, Ping Nie, Kelsey R Allen, Chenglong Wang, Michel Galley, Jianfeng Gao, ChengXiang Zhai