← Back to all news
arXiv Machine Learning By Junhyeok Lee, Sangyoon Lee, Jaeho Lee

Cache Locally, Recompute Selectively for Streaming Video Understanding

Read the original on arXiv Machine Learning →

The paper introduces FrameBlend, a plug‑and‑play technique for streaming video understanding that selectively recomputes key‑value (KV) caches at query time. By re‑prefilling only query‑relevant video tokens, FrameBlend allows shorter attention windows during idle prefill, cutting prefill cost while boosting accuracy. Experiments on three SVU frameworks and five benchmarks show up to 3.9‑point accuracy gains and significant reductions in FLOPs and processing time.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computer Vision
Sep 17

SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection

SVMemAgent introduces a streaming video memory (SVMem) that continuously updates a compact representation of observed frames for online keyframe selection without prior knowledge of video length, query, or future frames. The agent decides at each timestep whether to replace an existing memory frame with a new one or discard it, trained via Group Relative Policy Optimization using task-driven rewards from question-answer pairs. Experiments demonstrate that SVMemAgent outperforms existing online baselines and rivals offline methods, and its learned policy tends to favor frames containing textual information, potentially aiding downstream VideoQA tasks.

By Dohwan Ko, Ji Soo Lee, Pierce Chuang, Debojeet Chatterjee, Ashish Shenoy, Yichao Lu, Seungwhan Moon, Xin Luna Dong, Vikas Bhardwaj, Hyunwoo J. Kim
agentsbenchmarks
More like this →