Hugging Face Trending Papers
VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computer Vision
Oct 8
VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding
VideoEvolve is a self‑evolving framework for long‑video understanding that jointly updates both memory and retrieval mechanisms. It starts with a coarse low‑frame‑rate overview and uses a Memory Evolver to selectively augment memory while a Retrieval Evolver adapts how information is fetched from that memory. The two components are alternately refined via Agentic Reinforcement Learning, guided by Bottleneck‑Aware and Capability‑Aware Evolution Feedback to focus on the most limiting aspect and to prevent over‑specialization, respectively. Experiments on several long‑video benchmarks show that this approach improves performance over static systems.
arXiv Computer Vision
Sep 4
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
The paper introduces LatentStream, a progressive latent working memory framework for streaming video understanding that replaces the traditional store‑and‑retrieve paradigm with a retrieve‑and‑internalize approach. It organizes visual history into short, mid, and long‑term levels using Jenks‑guided adaptive consolidation, then expands memory receptive fields to iteratively retrieve and internalize evidence into a compact latent memory. A confidence‑guided optimization further refines this memory, leading to state‑of‑the‑art performance on online and offline video benchmarks.
Hugging Face Trending Papers
Sep 3
Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
The paper introduces LatentStream, a progressive latent working memory framework for streaming video understanding. It replaces the traditional store‑and‑retrieve approach with a retrieve‑and‑internalize strategy, organizing visual history into short, mid, and long‑term levels and progressively expanding memory receptive fields to internalize evidence into a compact latent memory. The method also employs confidence‑guided optimization to refine memory tokens, achieving state‑of‑the‑art results on online and offline video benchmarks.
Hugging Face Trending Papers
Sep 16
SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection
Most keyframe selection studies focus on offline settings, assuming access to the full video and query in advance. In contrast, real-world streaming scenarios require online frame selection under unkn...
arXiv Computer Vision
Sep 17
SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection
SVMemAgent introduces a streaming video memory (SVMem) that continuously updates a compact representation of observed frames for online keyframe selection without prior knowledge of video length, query, or future frames. The agent decides at each timestep whether to replace an existing memory frame with a new one or discard it, trained via Group Relative Policy Optimization using task-driven rewards from question-answer pairs. Experiments demonstrate that SVMemAgent outperforms existing online baselines and rivals offline methods, and its learned policy tends to favor frames containing textual information, potentially aiding downstream VideoQA tasks.
arXiv Computer Vision
Aug 31
StreamEMS: Streaming Video Understanding with Self-Evolving Memory Scheme for Vision-Language Models
arXiv:2608.27881v1 Announce Type: new Abstract: Recently, many streaming video understanding methods have been proposed by constructing an external memory to store historical data for computational r...