← Back to all news
arXiv AI By Kangcong Li, Peng Ye, Lin Zhang, Chao Wang, Huafeng Qin, Jiayuan Fan, Tao Chen

FreshMem: Brain-Inspired Frequency-Space Hybrid Memory for Streaming Video Understanding

Read the original on arXiv AI →

FreshMem is a brain-inspired Frequency‑Space Hybrid Memory network designed to improve multimodal large language models’ ability to process streaming video. It combines a Multi‑scale Frequency Memory that compresses excess frames into frequency coefficients while preserving residual details, and a Space Thumbnail Memory that clusters the continuous stream into high‑density episodic thumbnails. Experiments show FreshMem boosts the Qwen2‑VL baseline by 5.20%, 4.52%, and 2.34% on StreamingBench, OV‑Bench, and OVO‑Bench, outperforming several fully fine‑tuned methods without requiring additional training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Sep 4

Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding

The paper introduces LatentStream, a progressive latent working memory framework for streaming video understanding that replaces the traditional store‑and‑retrieve paradigm with a retrieve‑and‑internalize approach. It organizes visual history into short, mid, and long‑term levels using Jenks‑guided adaptive consolidation, then expands memory receptive fields to iteratively retrieve and internalize evidence into a compact latent memory. A confidence‑guided optimization further refines this memory, leading to state‑of‑the‑art performance on online and offline video benchmarks.

By Hongyu Qu, Guangming Yao, Ling Xing, Xiaobin Hu, Rongxing Ding, Guibin Zhang, Fan Zhang, Yi Yuan, Xiangbo Shu, Shuicheng Yan
llmsmultimodalbenchmarks
More like this →