PercepCap: Video Captioner with Structured Spatio-Temporal Perception

PercepCap is a video captioning framework that explicitly models spatio‑temporal perception before generating captions. It follows a perceive‑describe chain, first producing a perception trace of object trajectories and temporal events, then generating the final caption conditioned on that trace. The method uses a two‑stage training strategy—supervised fine‑tuning followed by perception‑grounded reinforcement learning—and builds caption‑aligned perception data to ensure the perception trace and caption refer to the same objects and events.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

• reinforcement-learning
• fine-tuning