arXiv Computer Vision
Aug 27

Skeleton-based Zero-Shot Spatio-Temporal Action Localization via Weakly-Supervised Pretraining

The paper introduces Skeleton-Language feature Pooling Switching, a weakly‑supervised vision‑language pretraining strategy for skeleton‑based zero‑shot spatio‑temporal action localization. It replaces video‑level pooling with instance‑level feature computation during inference, enabling the model to estimate unseen actions without costly annotations. Additionally, Scene‑Mixed Discriminative Contrastive Learning is proposed to separate actions at the instance level within mixed scenes using a MIL framework, and experiments on four public datasets confirm the method’s effectiveness.

By Koshiro Nagano, Fumiaki Sato, Ryo Hachiuma, Kazuki Tsutsukawa, Taiki Sekii
arXiv Computer Vision
Oct 9

LLM-powered Query Expansion for Enhancing Boundary Prediction in Language-driven Action Localization

The paper introduces a method that uses large language models (LLMs) to expand language queries with detailed textual descriptions of action start and end boundaries, providing richer cues for video action localization. It also proposes a training strategy that models boundary probability scores by combining semantic similarity between frames and the expanded query with temporal distance to annotated boundaries, thereby offering more consistent supervision. The approach is model‑agnostic and can be integrated into existing language‑driven action localization systems, with experiments on multiple datasets showing its effectiveness.

By Zirui Shang, Xinxiao Wu, Shuo Yang