arXiv Computation and Language September 21, 2026 By Thomas Ranzenberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer

The Spoken Wikipedia Presentation Corpus

Read the original on arXiv Computation and Language →

The Spoken Wikipedia Presentation Corpus extends the existing Spoken Wikipedia Corpora by adding LLM-generated slide decks for multimodal automatic speech recognition (ASR). Slides are produced through a hybrid pipeline that combines LLM-based content planning with rule-based design, generating titles, bullet points, takeaway messages, and visual descriptions for illustrations. The corpus is evaluated with multiple ASR and spoken language models, achieving a best micro-WER of 10.23% and micro-CER of 6.48% on audio-only inputs, with English performing best and lower-resource languages showing higher error rates.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computer Vision

Sep 10

AVSRBench: A Multi-Condition AVSR Benchmark

arXiv:2609.10366v1 Announce Type: cross Abstract: While AVSR has achieved sub-1% word error rates on the standard LRS3 benchmark, its reliance on broadcast speech obscures whether this reflects true...

By Rishabh Jain, Naomi Harte

llms fine-tuning multimodal benchmarks

arXiv Computation and Language

Sep 1

OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models

arXiv:2603.23938v2 Announce Type: replace Abstract: Most testbeds for omni-modal models assess multimodal understanding via textual outputs, leaving it unclear whether these models can properly speak...

By Seunghee Kim, Bumkyu Park, Kyudan Jung, Joosung Lee, Soyoon Kim, Jeonghoon Kim, Taeuk Kim, Hwiyeol Jo

multimodal benchmarks

arXiv Computer Vision

Sep 10

Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition

arXiv:2609.10394v1 Announce Type: cross Abstract: Current audio-visual speech recognition (AVSR) benchmarks, like LRS3, rely heavily on clean, scripted and rehearsed speech. They fail to reflect the...

By Rishabh Jain, Aristeidis Papadopoulos, Zhaofeng Lin, Naomi Harte

multimodal benchmarks

arXiv Computation and Language

Sep 22

Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models

arXiv:2609.22452v1 Announce Type: new Abstract: Long-context understanding remains a fundamental challenge for large language models, as excessively long inputs often lead models to forget salient in...

By Xize Cheng, Wenxu Jia, Chenyuhao Wen, Dongjie Fu, Zehan Wang, Xinyu Zhang, Tao Jin

llms rag benchmarks

arXiv AI

2d ago

SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

SONIC‑O1 is a new benchmark designed to evaluate multimodal large language models on audio‑video understanding. It contains 60 hours of 231 clips across 13 real‑world conversational domains, with 4,958 human‑verified annotations and demographic metadata. The benchmark tests open‑ended summarization, multiple‑choice question answering, and temporally grounded reasoning, revealing performance gaps between model families and across demographic groups.

By Ahmed Y. Radwan, Christos Emmanouilidis, Hina Tabassum, Deval Pandya, Shaina Raza

llms nlp multimodal benchmarks

arXiv Computation and Language

Sep 3

TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding

TalkFa introduces a unified benchmark for Farsi dialogue generation and understanding, comprising three datasets: WIKI‑FADIAL (4.2K Wikipedia‑grounded dialogues), DAILYDIALOG‑FA (6.6K dialogues with dialogue‑act and emotion annotations), and PLAYDIAL‑FA (2.1K theatrical dialogues with sentiment labels). All dialogues are curated through multi‑stage review by native speakers, ensuring high quality. Experiments show that LoRA fine‑tuning improves generation performance with less data, while specific models excel on classification tasks, and human evaluation confirms the benchmark’s reliability.

By Neda Jamshidi, Kamyar Zeinalipour, Fahimeh Akbari, Monica Bianchini, Marco Maggini, Marco Gori

llms fine-tuning benchmarks

One email a morning, machine-written

Related stories

AVSRBench: A Multi-Condition AVSR Benchmark

OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models

Candor-LR: A Dyadic Conversational Dataset for Audio-Visual Speech Recognition

Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models

SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding