Introducing new audio and vision documentation in 🤗 Datasets

A Complete Guide to Audio Datasets

AudioWorldSim: Realistic Binaural Audio Datasets For World Models

AudioWorldSim is an open‑source platform that generates realistic binaural audio datasets for training and evaluating audio‑based machine learning models, especially world models. It extends Meta’s SoundSpaces 2.0 by automating random agent navigation and correcting continuous sound composition. The project is publicly available on GitHub to support reproducibility in research.

Audio-FLAN: An Instruction-Following Dataset for Unified Audio Understanding and Generation of Speech, Music, and Sound

arXiv:2502. 16584v2 Announce Type: replace-cross Abstract: Recent advancements in audio tokenization have significantly enhanced the integration of audio capabilities into large language models (LLMs).

BEANS-Next and ROOTS: Broadening Audio-Language Capabilities for Bioacoustics

Bioacoustics and ethology encompass a wide range of audio understanding tasks, many of which stand to benefit from recent advances in large audio-language models. However, progress in the field has so...

VieSpeaker: A Large-Scale Vietnamese Speaker Recognition Dataset Beyond Visual Dependency

Speaker recognition has advanced rapidly with large-scale training datasets, yet Vietnamese remains under-resourced, with existing corpora limited in scale and acoustic diversity. Most large-scale datasets rely on facial cues to link speech with speaker identities, restricting data collection to recordings where speakers appear on camera.

BEANS-Next and ROOTS: Broadening Audio-Language Capabilities for Bioacoustics

The paper introduces BEANS-Next, a comprehensive benchmark covering acoustic perception, biological category recognition, scene understanding, and in-context learning in bioacoustics, and shows that current audio‑language models perform poorly outside the narrow label‑centric tasks traditionally evaluated. To address this gap, the authors present ROOTS, a large‑scale training dataset enriched with curated real‑world data, behavioral and acoustic metadata, and synthetic generation, which improves performance across all BEANS‑Next task groups. Both the benchmark and dataset are released as open‑source resources to accelerate research in bioacoustics and ethology.

TriA Pipeline: A Large-Scale Automatic Audio Annotation Pipeline For Audio Classification In Specific Scenarios

arXiv:2607. 06179v1 Announce Type: cross Abstract: There are some datasets of varying scales for audio classification (AC) applied to different tasks.

VGGSounder: Audio-Visual Evaluations for Foundation Models

arXiv:2508. 08237v4 Announce Type: replace-cross Abstract: The emergence of audio-visual foundation models underscores the importance of reliably assessing their multi-modal understanding.

SynSFX: Multi-Model Sound Effects Synthesis Dataset for Deepfake Detection and Evaluation

arXiv:2607. 04848v1 Announce Type: cross Abstract: While audio deepfake detection has advanced significantly, representative detectors show limited generalization to synthetic sound effects.

Logbook: Extremely Long-form Audio Event Understanding

arXiv:2610.07338v1 Announce Type: cross Abstract: Audio benchmarks are built around short, pre-segmented clips, limiting model design to brief inputs or fixed vocabularies. To close this gap, we intr...

SsgCaps: A controlled dataset for the evaluation of sound scene generation algorithms

SsgCaps is a publicly available dataset of human-engineered sound scenes, each paired with a precisely structured prompt that guides the sampling process. The prompts are drawn from a predefined action-based typology, enabling extensive yet plausible sampling. A comparative quantitative analysis shows only small differences between the open and private versions, supporting the recommendation of the open version for benchmarking sound scene generation algorithms.

Vision Language Models Explained