arXiv AI
ScreenASR-Bench: A Tiered Benchmark for Speech Recognition with Visual Context
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computer Vision
Sep 16
Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?
Video-HolmesV2 is a new benchmark that tests multimodal large language models on their ability to reason with spatio‑temporal audio‑visual evidence in long videos. It requires models to justify answers with precise evidence, uses a multi‑model cross‑verification pipeline and a spatio‑temporal evidence‑aware metric, and introduces an audio‑text guided token compression framework to reduce long‑context noise. In evaluations, even strong proprietary models score below 60% while the proposed approach outperforms comparable open‑source omni‑models.
arXiv Computer Vision
1d ago
EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning
EvaOmni is a new evidence‑grounded benchmark for audio‑visual reasoning, comprising 392 videos and 1,027 human‑annotated question‑answer pairs that span 15 capabilities and three difficulty levels. The benchmark requires models to provide both an answer and the supporting audio‑visual evidence intervals, and introduces Coverage Grounded (C.G.) and Bottleneck Grounded (B.G.) scores to evaluate evidence identification and temporal localization. Across 24 Omni LLMs, Gemini3.7‑Flash achieved the highest overall accuracy, yet its B.G. scores reveal significant room for improvement in grounded reasoning.
Hugging Face Trending Papers
Sep 15
Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?
Multimodal Large Language Models have demonstrated impressive video understanding, yet their ability to reason over long-form narratives is often masked by visual-centric evaluations and inefficient c...
Hugging Face Trending Papers
Jun 24
From Sounds to Scenes: A Benchmark for Evaluating Context-Aware Auditory Scene Understanding in Large Audio Language Models
Recent Large Audio Language Models (LALMs) have achieved remarkable progress in audio perceptual tasks across individual acoustic layers, including speech, sound, and music. However, existing benchmarks predominantly evaluate these layers in isolation, overlooking the complex contextual relationships that arise when multiple acoustic sources co-occur in real-world auditory scenes.
arXiv Computation and Language
Sep 17
HearInContext: A Benchmark for Implicit Context in Speech Recognition
arXiv:2609.18680v1 Announce Type: new Abstract: Contextual ASR can benefit from semantic cues or from target words explicitly provided in the context. We introduce HearInContext, a Mandarin--English...
arXiv Computer Vision
Sep 3
From Visual Cues to Spoken Narration: Rethinking Audio Description
The paper introduces Cue2Narrate, a two‑stage pipeline that jointly predicts what visual events to narrate and when to insert the narration in long, untrimmed movie clips. It uses a dual‑head audio‑visual localizer to identify visual cue and narration windows, followed by a LoRA‑adapted vision‑language model that generates concise audio descriptions, trained with a Description Ranking Loss. The authors also present the LongLSMDC benchmark, comprising up to 8‑minute clips, and show that Cue2Narrate outperforms video‑only and audio‑only baselines by 5–12 points in average mAP and improves AD generation over fine‑tuned base VLMs.