arXiv Computer Vision

MoVISA: Multi-Token Reasoning for Video Object Segmentation

MoVISA introduces Multi-Token Reasoning for Video Object Segmentation, using multiple segmentation tokens (e.g., SEG0, SEG1) instead of a single token to better localize multiple objects over time. This approach enhances fine-grained alignment between language prompts and spatio-temporal mask predictions, leading to improved performance and interpretability. On benchmarks such as MeViS, DAVIS17, ReVOS, and Ref-Youtube-VOS, MoVISA achieves significant gains, notably a 13.2% J and F improvement on MeViS and an 8.4% J and F improvement on ReVOS.

arXiv Computer Vision
Oct 21

TrajTok: Learning Trajectory Tokens enables better Video Understanding

TrajTok introduces an end‑to‑end video tokenizer that replaces traditional patchification with a lightweight, trajectory‑based tokenization process. It performs implicit clustering over space‑time pixels to generate object trajectories in a single forward pass, dynamically adjusting token granularity to semantic complexity while remaining independent of video duration. The resulting TrajViT2 model outperforms existing methods on classification and retrieval benchmarks, and TrajTok can also serve as a probing head (TrajAdapter) or alignment connector (TrajVLM) for vision‑language tasks, notably improving long‑video reasoning.

By Chenhao Zheng, Jieyu Zhang, Jianing Zhang, Weikai Huang, Ashutosh Kumar, Quan Kong, Oncel Tuzel, Chun-Liang Li, Ranjay Krishna
arXiv Computer Vision
Aug 25

Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos

arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...

By Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, Ming-Hsuan Yang
arXiv Computer Vision
6d ago

The Target Is a Predicate: Predicate-State Grounding for Semantic-Level Video Reasoning Segmentation

arXiv:2611. 13785v1 Announce Type: new Abstract: In this paper, we introduce Predicate-State Video Reasoning Segmentation (PS-VRS), where a query defines a dynamic predicate $P(o,t)$ over objects and time rather than a fixed object identity.

By Yuhuan Hong, Yiming Ren, Zhiqiang Lin, Feize Wu, Xingyun Liu, Yitong Xie, Lei Zhou, Lisheng Wu, Junjie Wang, Yujiu Yang
Hugging Face Trending Papers
Nov 10

The Target Is a Predicate: Predicate-State Grounding for Semantic-Level Video Reasoning Segmentation

In this paper, we introduce Predicate-State Video Reasoning Segmentation (PS-VRS), where a query defines a dynamic predicate $P(o,t)$ over objects and time rather than a fixed object identity. Existing methods typically treat the target as permanently active once identified, leading to over-emission of masks during vacant intervals and failure to handle target switching.

arXiv Computer Vision
Sep 23

TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

arXiv:2505.01583v2 Announce Type: replace Abstract: Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models (VLM...

By Jen-Hao Cheng, Yi-Hao Peng, Huapeng Zhou, Vivian Wang, Huayu Wang, Hsiang-Wei Huang, Wenhao Chai, Hou-I Liu, Kuang-Ming Chen, Cheng-Yen Yang, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang