Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The problem requires determining both what (which visua...
The paper introduces Cue2Narrate, a two‑stage pipeline that jointly predicts what visual events to narrate and when to insert the narration in long, untrimmed movie clips. It uses a dual‑head audio‑visual localizer to identify visual cue and narration windows, followed by a LoRA‑adapted vision‑language model that generates concise audio descriptions, trained with a Description Ranking Loss. The authors also present the LongLSMDC benchmark, comprising up to 8‑minute clips, and show that Cue2Narrate outperforms video‑only and audio‑only baselines by 5–12 points in average mAP and improves AD generation over fine‑tuned base VLMs.
arXiv:2608. 09593v1 Announce Type: cross Abstract: Recent advances in speech synthesis and audio generation have made high-fidelity acoustic forgery low-cost and difficult to attribute, enabling a realistic attack scenario in which speech and background audio are independently manipulated over otherwise authentic video.
arXiv:2607. 25543v1 Announce Type: cross Abstract: Generative AI has rapidly expanded audio-visual forgery beyond human-centric deepfakes into general scenes.
BabelFake is a multilingual audio‑visual DeepFake benchmark featuring 399,000 clips from 496 consenting participants across five languages (English, German, Italian, French, Spanish). The dataset pairs 11 modern video manipulation methods with 4 voice‑cloning engines, covering both visual‑only and joint audio‑visual attacks. Experiments show that detection difficulty varies with the audio‑visual pairing, with significant performance drops when authentic audio is retained, and that cross‑language and demographic factors influence detector performance.
Video-HolmesV2 is a new benchmark that tests multimodal large language models on their ability to reason with spatio‑temporal audio‑visual evidence in long videos. It requires models to justify answers with precise evidence, uses a multi‑model cross‑verification pipeline and a spatio‑temporal evidence‑aware metric, and introduces an audio‑text guided token compression framework to reduce long‑context noise. In evaluations, even strong proprietary models score below 60% while the proposed approach outperforms comparable open‑source omni‑models.
This paper presents an interpretable deepfake detection framework that explicitly encodes physically grounded forensic cues to analyze spatially and temporally coherent facial features in video sequences. The method transforms videos into identity-consistent facial trajectories, segments them into fixed-length temporal windows, and represents each frame with 68 structured descriptors across photometric, textural, geometric, and compression domains. These descriptors are processed by an LSTM to capture temporal dependencies, achieving strong F1-scores on four benchmark datasets and demonstrating robust cross-dataset generalization.
LEAP is a framework for long audio‑video question answering that avoids encoding entire recordings by dividing them into fixed‑duration blocks. It performs a lightweight localization pass on each block to score short candidate windows, then pools the highest‑ranked windows for a single bounded answer pass, keeping the answer input and peak context independent of recording length. The method trains both a localization LoRA and an answer LoRA, supports causal streaming queries, and achieves significant performance gains over baseline models on multiple AVQA benchmarks.
ManiVid introduces a unified forensic analysis framework for manipulated videos, combining forgery detection, artifact grounding, and anomaly explanation. The authors release ManiVid-38K, a large dataset of 19K real‑fake video pairs with authenticity labels, forgery masks, and explanations, and a benchmark ManiVidBench with 1K balanced pairs. ManiVidLens, the proposed model, outperforms existing methods in artifact grounding and anomaly explanation while matching state‑of‑the‑art detection accuracy.
Multimodal Large Language Models have demonstrated impressive video understanding, yet their ability to reason over long-form narratives is often masked by visual-centric evaluations and inefficient c...
The paper explores training a large language model solely on textual video descriptions to perform zero‑shot video temporal localization, eliminating the need for paired video‑text data during training. Using a 134K‑example corpus and adapting Qwen3‑8B with LoRA on CLIP or LLM2CLIP embeddings, the authors evaluate on Charades‑STA and QV‑Highlights, achieving modest performance when directly replacing text with frame embeddings. They introduce Nearest Entity Replacement (NER), a training‑free mapping that represents each video frame by the mean of its nearest text‑entity embeddings, which significantly improves results and narrows the modality gap.
The paper introduces Motion‑Aware Deepfake (MAD) videos, a new class of deepfakes generated by pose‑guided diffusion models that can animate entire human figures. To study this threat, the authors build the first MAD‑specific benchmark with 1.5 million frames from 1,363 real and 30,122 synthetic videos, and analyze the subtle high‑frequency artifacts and spectral fingerprints that arise from limited input frames. Leveraging these insights, they propose MoDA, a defense framework that combines spatial semantics with steganalysis‑rich frequency features, achieving high detection and attribution accuracy across in‑distribution, cross‑dataset, and zero‑shot scenarios, and demonstrating robustness against adaptive attacks.