← Back to all news
arXiv Computer Vision By Zhongrui Gui, Junyu Xie, Tengda Han, Weidi Xie, Andrew Zisserman

As the Story Unfolds: Watching a Film and Identifying Characters as a Human Does

Read the original on arXiv Computer Vision →

The paper introduces the character naming task, where a model must identify which visible character in a film bears a name mentioned in dialogue, using only the movie’s video and dialogue without external cast lists or actor photos. It highlights the difficulty of name resolution due to cinematic cues such as shot-reverse-shot structure, self-introduction, direct address, third-party reference, gaze, and temporal continuity. The authors provide a dataset and annotations, develop a video-and-dialogue model, compare it with large multimodal language models, and propose a progressive framework for building a character bank as the film unfolds.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Jul 3

Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas

arXiv:2607. 02504v1 Announce Type: cross Abstract: Long-form TV dramas present a formidable challenge for comprehensive video understanding, where deciphering complex storyline often relies on \textbf{speaker recognition}, the task of accurately attributing each spoken utterance to its respective character.

By Yuxuan Li, Lingxi Xie, Xinyue Huo, Jihao Qiu, Jiacheng Shao, Pengfei Chen, Jiannan Ge, Kaiwen Duan, Qi Tian
llmsmultimodalbenchmarks
More like this →
arXiv Computation and Language
Aug 31

Long Story Short: Story-level Video Understanding from 20K Short Films

The paper introduces Short‑Films 20K (SF20K), a large publicly available movie dataset comprising 20,143 amateur films totaling 3,582 hours, with an average length of 12 minutes per film. Accompanying the dataset is SF20K‑Test, a manual open‑ended question‑answering benchmark featuring 95 movies and 979 question‑answer pairs. Analysis of the benchmark shows limited data leakage, highlights the necessity of long‑term reasoning, and demonstrates that instruction tuning on the large‑scale dataset significantly boosts vision‑language model performance.

By Ridouane Ghermi, Xi Wang, Vicky Kalogeiton, Ivan Laptev
llmsnlpfine-tuningmultimodalbenchmarks
More like this →
arXiv Computer Vision
Aug 27

MObyGaze: a film dataset of multimodal objectification densely annotated by experts

The paper introduces MObyGaze, a dataset of 20 films annotated by experts for multimodal objectification, covering 6072 segments across 43 hours of video. It defines objectification through a structured thesaurus of 5 sub‑constructs and 11 concepts spanning visual, speech, and audio modalities. The authors formulate learning tasks, explore label diversity strategies, and benchmark vision, text, and audio models to demonstrate the task’s feasibility.

By Julie Tores, Elisa Ancarani, Lucile Sassatelli, Hui-Yin Wu, Clement Bergman, Lea Andolfi, Victor Ecrement, Remy Sun, Frederic Precioso, Thierry Devars, Magali Guaresi, Virginie Julliard, Sarah Lecossais
multimodalbenchmarks
More like this →