Hugging Face Trending Papers
Cache Locally, Recompute Selectively for Streaming Video Understanding
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 22
Cache Locally, Recompute Selectively for Streaming Video Understanding
The paper introduces FrameBlend, a plug‑and‑play technique for streaming video understanding that selectively recomputes key‑value (KV) caches at query time. By re‑prefilling only query‑relevant video tokens, FrameBlend allows shorter attention windows during idle prefill, cutting prefill cost while boosting accuracy. Experiments on three SVU frameworks and five benchmarks show up to 3.9‑point accuracy gains and significant reductions in FLOPs and processing time.
arXiv Machine Learning
Jun 26
ProtoKV: Streaming Video Understanding under Delayed Query with Summary-State Memory
arXiv:2606. 26762v1 Announce Type: cross Abstract: Streaming video understanding (SVU) must answer queries that arrive asynchronously while visual tokens stream continuously under strict GPU-memory and query-time latency budgets.
arXiv Computer Vision
Sep 17
SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection
SVMemAgent introduces a streaming video memory (SVMem) that continuously updates a compact representation of observed frames for online keyframe selection without prior knowledge of video length, query, or future frames. The agent decides at each timestep whether to replace an existing memory frame with a new one or discard it, trained via Group Relative Policy Optimization using task-driven rewards from question-answer pairs. Experiments demonstrate that SVMemAgent outperforms existing online baselines and rivals offline methods, and its learned policy tends to favor frames containing textual information, potentially aiding downstream VideoQA tasks.
arXiv AI
Jun 15
Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention
arXiv:2602. 01801v2 Announce Type: replace-cross Abstract: Autoregressive video diffusion models enable streaming generation, opening the door to long-form synthesis, video world models, and interactive neural game engines.
Hugging Face Trending Papers
Sep 16
SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection
Most keyframe selection studies focus on offline settings, assuming access to the full video and query in advance. In contrast, real-world streaming scenarios require online frame selection under unkn...
arXiv AI
Jun 16
What Should a Streaming Video Model Remember?
arXiv:2606. 16353v1 Announce Type: cross Abstract: Streaming video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed memory and computation budgets.