arXiv Computer Vision
RVG-Bench: A Benchmark for Reasoning-Driven Audio-Visual Temporal Grounding
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Aug 11
Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models
arXiv:2608. 09435v1 Announce Type: new Abstract: Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time.
arXiv Computer Vision
Sep 1
TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos
arXiv:2605.07593v2 Announce Type: replace Abstract: Real-world audio-visual understanding requires chaining evidence that is sparse, temporally dispersed, and split across the visual and auditory str...
Hugging Face Trending Papers
Aug 10
Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models
Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time. Yet existing audio-language models often represent clips as global acoustic events, while vision-language models lack the spatial audio cues needed to localize and track individual sources.
arXiv Computer Vision
Oct 8
TiTok: Audio-Visual LLM for Multi-Segment Temporal Grounding
TiTok is an audio‑visual large language model designed for multi‑segment temporal grounding in untrimmed videos. It introduces Time Token Interleaving (TTI) to align temporal perception with prediction, and uses decoupled reinforcement‑learning rewards (global, local, count, precision, format) optimized via GDPO. Evaluated on a new UnAV‑100 protocol with the CountF1 metric, TiTok achieves state‑of‑the‑art results (65.7 mIoU, 0.58 CountF1).
arXiv Computer Vision
Oct 21
EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning
EvaOmni is a new evidence‑grounded benchmark for audio‑visual reasoning, comprising 392 videos and 1,027 human‑annotated question‑answer pairs that span 15 capabilities and three difficulty levels. The benchmark requires models to provide both an answer and the supporting audio‑visual evidence intervals, and introduces Coverage Grounded (C.G.) and Bottleneck Grounded (B.G.) scores to evaluate evidence identification and temporal localization. Across 24 Omni LLMs, Gemini3.7‑Flash achieved the highest overall accuracy, yet its B.G. scores reveal significant room for improvement in grounded reasoning.
arXiv Computer Vision
Oct 12
Query-Guided Spatial-Temporal-Frequency Interaction for Music Audio-Visual Question Answering
The paper introduces Query‑Guided Spatial‑Temporal‑Frequency (QSTar) interaction for Audio‑Visual Question Answering, integrating question cues and audio frequency characteristics with spatial and temporal visual features. It also presents a Query Context Reasoning (QCR) block that prompts the model to focus on semantically relevant audio and visual elements. Experiments on two AVQA benchmarks show significant performance gains over existing Audio QA, Visual QA, Video QA, and AVQA methods.