RelayVSR introduces a streaming video super‑resolution framework that combines a large generative model, which produces reference latents for sparse keyframes, with a lightweight Dual‑Memory Video Transformer that super‑resolves every frame using these references and low‑resolution input. The method employs Video‑Aware Reference Optimization (VARO), a reinforcement‑learning strategy that optimizes both system‑level video quality and reference‑level keyframe fidelity, outperforming direct joint training. On 1080p video, RelayVSR achieves 29.29 FPS with modest GPU memory usage, significantly faster and more efficient than the FlashVSR‑Tiny baseline.
By Xijun Wang, Xin Li, Zirui Lang, Suhang Yao, Haoran Li, Zhibo Chen
Scalable Neural Video Representation Compression (S-NVRC) introduces a scalable implicit neural representation (INR) video codec that supports fine-grained bitrate and decoding‑complexity scalability from a single embedded bitstream. It uses a coarse‑to‑fine prefix for feature grids and a nested prefix for network layers, enabling a wide range of operating points while maintaining a single encoding. On the UVG dataset, S‑NVRC outperforms SHM 12.4 and multi‑layer VTM‑20.0 by 43.7 % and 5.6 % in BD‑rate, respectively, and offers flexible complexity scalability.
By Tianhao Peng, Ho Man Kwan, Fan Zhang, Shan Liu, David Bull
Large generative models can recover realistic detail in real-world video super-resolution (VSR), but processing an entire video with them is computationally expensive. In this work, we present RelayVS...
LRConv-NeRV introduces low‑rank separable convolutions into the NeRV neural video decoder, replacing selected dense 3x3 layers to reduce computational load and memory usage. By applying low‑rank factorization progressively from the largest to earlier decoder stages, the method offers controllable trade‑offs between reconstruction quality and efficiency. Experiments show that applying LRConv only to the final decoder stage cuts decoder complexity by 68% and model size by 9.3% with negligible quality loss, while INT8 quantization preserves performance close to the dense baseline.
By Tamer Shanableh
arXiv:2609.37831v1 Announce Type: new
Abstract: Real-time diffusion-based video super-resolution (VSR) is in high demand for online streaming, yet stringent latency requirements often compromise gene...
By Xijun Wang, Xin Li, Suhang Yao, Zirui Lang, Bingchen Li, Zhibo Chen
TT-VidT is a video pretraining method that decouples the temporal axis by combining a per‑frame ViT-B/16 spatial encoder with a compact Temporal Transfer Layer trained via Diff Compression. The authors conduct a systematic 24‑configuration study to isolate architecture, objective, and decoder effects, showing that the full TT-VidT design yields the strongest motion‑sensitive representations. In downstream fine‑tuning, TT‑VidT outperforms state‑of‑the‑art baselines on Jester, Something‑Something V2, ARID, and Diving48 while using significantly fewer encoder FLOPs.
By Shih-Ying Yeh, Daniel Z. Kaplan, Xuehai Wang, Fu-En Yang, Min-Hung Chen, Shang-Hong Lai
LeVJEPA is a video encoder that eliminates the need for architectural asymmetries, exponential-moving-average target encoders, stop-gradients, and capacity-limited predictors used in prior self‑supervised methods. It trains a single encoder with an invariance loss over global and local views, regularized by SIGReg to prevent collapse, and achieves strong performance with far less pretraining compute. The approach also allows block‑causal attention, making temporal ordering a property of the encoder itself, and matches or surpasses state‑of‑the‑art baselines on both appearance‑centric and motion‑centric benchmarks.
By Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, Florian Buettner
arXiv:2603. 12478v2 Announce Type: replace-cross Abstract: Multimodal instruction tuning is often compute-inefficient because training budgets are spread across large mixed image-video pools whose utility is highly uneven.
By Rujie Wu, Haozhe Zhao, Hai Ci, Yizhou Wang
arXiv:2607. 14898v1 Announce Type: cross Abstract: Real-time video generation demands fast decoding as much as fast denoising, yet current latent video diffusion models rely on 3D convolutional decoders that are slow and memory-intensive at high resolutions or for long video.
By Minguk Kang, Suha Kwak
arXiv:2606. 23743v1 Announce Type: cross Abstract: Modern video diffusion models achieve higher generation quality through scaling, but this also increases inference cost.
By Yitong Li, Junsong Chen, Haopeng Li, Haozhe Liu, Jincheng Yu, Ligeng Zhu, Ping Luo, Song Han, Enze Xie
MoE-ViE introduces a Mixture-of-Experts vision encoder that scales efficiently for image and video understanding, outperforming dense counterparts across various sizes. The study shows fine‑grained MoE topologies provide significant gains, and proposes an auxiliary‑loss‑free balancing variant and a specialized MoE kernel to reduce inference latency. With frame‑level distillation and a novel freezing mechanism, the largest MoE‑ViE model matches state‑of‑the‑art zero‑shot performance while being 1.7× larger and 76% faster, and it outperforms other encoders when paired with a language model on both image and video benchmarks.
Latent video generation relies on autoencoders to define a compact space in which generative models operate. Although video autoencoder architectures have evolved substantially, their latent spaces are still optimized primarily for pixel-level reconstruction and provide limited high-level semantic organization.