arXiv Computer Vision By Ziwei Zhao, Yuhang Liu

A Principled Optimal Transport Framework for Frame Selection in Long Video Understanding

Read the original on arXiv Computer Vision →

The paper introduces OTFS, a principled optimal transport framework for selecting frames in long-video understanding. It formulates frame selection as a structured evidence-allocation problem, using a semi‑unbalanced entropic optimal transport objective and an efficient OTFS‑Sinkhorn solver, followed by a dynamic‑programming extraction step. Experiments on three benchmarks show OTFS consistently outperforms strong training‑free baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Hugging Face Trending Papers
Aug 6

Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs

Multimodal Large Language Models (MLLMs) have achieved strong progress in video understanding, yet it remains challenging because the token limitation makes MLLMs difficult to capture temporally sparse evidence. Existing methods typically rely on uniform sampling, or frame selection, but these strategies usually optimize either broad temporal coverage or local relevance, making it difficult to preserve both global storyline context and fine-grained evidence.

arXiv AI
Jun 12

ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

arXiv:2506. 01274v2 Announce Type: replace-cross Abstract: Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to video understanding remains constrained by suboptimal frame selection strategies, albeit with the rapid development of video-specialized LMMs.

By Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro