Perceiving and Predicting Dynamic Sports: Spatiotemporal Intelligence in Vision Models for Egocentric Tennis Video

The paper introduces a new egocentric tennis dataset and evaluates how well foundation vision models can perform spatiotemporal tasks such as player and ball localization, trajectory forecasting, stroke type anticipation, and contact-time prediction. Seven models—image encoders, video encoders, and vision‑language models—are trained and tested on these tasks, achieving localization errors of 0.14 m for the player and 0.54 m for the ball, and forecasting errors of 0.34 m for the player and 0.91 m for the ball over 1.1 s. The study also shows that auxiliary depth data improves 3D ball localization, that longer image histories do not aid ball forecasting, and proposes a conditional diffusion model that generates real‑time whole‑body kinematic tennis motion from egocentric video predictions.