arXiv AI

VocSim: A Training-free Benchmark for Zero-shot Content Identity in Single-source Audio

arXiv:2512. 10120v2 Announce Type: replace-cross Abstract: General-purpose audio representations aim to map acoustically variable instances of the same event to nearby points, resolving content identity in a zero-shot setting.

arXiv AI

Jul 7

Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings

arXiv:2607. 03806v1 Announce Type: cross Abstract: Audio foundation models are widely adopted as general-purpose feature extractors, yet the internal structure of their learned representations remains insufficiently understood.

By H\'ector Martel, Joe Hennessy-Priest, Taemin Cho

rag multimodal

arXiv Machine Learning

1d ago

A Parameter-Free Few-Shot Evaluation for Elephant Vocalisation Classification

arXiv:2608. 14824v1 Announce Type: cross Abstract: We present a parameter-free episodic evaluation of nearest-centroid classification for elephant vocalisations on fixed pretrained acoustic embeddings, across the Elephant Voices (EV) and Linguistic Data Consortium (LDC) datasets.

By Christiaan M. Geldenhuys, Thomas R. Niesler

rag

Hugging Face Trending Papers

Aug 12

Confucius4-TTS: Transcript-Free Cross-Lingual Zero-Shot TTS with a Learnable Speaker Encoder

Recent advances in zero-shot text-to-speech (TTS) have substantially improved speech quality and voice cloning fidelity. However, many zero-shot TTS systems still depend on audio prompt transcripts at inference time.

llms multimodal benchmarks

arXiv AI

Jun 18

Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priors

arXiv:2606. 19325v1 Announce Type: cross Abstract: Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings.

By Michael Finkelson, Daniel Segal, Eitan Richardson, Shahar Armon, Nani Goldring, Poriya Panet, Nir Zabari, Benjamin Brazowski, Or Patashnik, Yoav HaCohen

rag benchmarks

Hugging Face Trending Papers

Jun 17

Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priors

Existing multi-speaker dialogue systems bind speakers to utterances through structured supervision: per-turn tags, multi-stream transcriptions, or learnable speaker embeddings. These systems operate within speech-only pipelines that produce clean vocal sequences without the ambient texture of real conversations.

rag benchmarks

arXiv AI

1d ago

Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer

arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.

By Ivan Mikheev, Viacheslav Vasilev, Anna Dmitrienko, Alexey Letunovskiy, Ivan Kirillov, Kirill Chernyshev, Denis Dimitrov

rag diffusion fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 29

Advancing Speaker-Based Vocal Effort Classification with WavLM and Data Augmentation in Naturalistic Non-Calibrated Speech Recordings

arXiv:2606. 27543v1 Announce Type: cross Abstract: The variations in vocal effort range (e.

By Zahra Omidi, John H. L. Hansen

multimodal benchmarks

arXiv AI

Jun 9

Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis

arXiv:2606. 07533v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) effectively integrate text and audio to interpret context in complex interactive dialogues.

By Pawe{\l} Pozorski, Jakub Muszy\'nski, Maria Ganzha

llms nlp multimodal safety

arXiv AI

Jun 4

SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification

arXiv:2605. 13672v1 Announce Type: cross Abstract: Few-shot classification (FSC) is widely used for learning from limited labeled data, yet most evaluations implicitly assume that target concepts are independent of contextual cues.

By Giries Abu Ayoub, Morad Tukan, Loay Mualem

computer-vision benchmarks

arXiv Machine Learning

Jun 2

Embedding-Space Diffusion for Zero-Shot Environmental Sound Classification

arXiv:2412. 03771v3 Announce Type: replace-cross Abstract: Zero-shot learning enables models to generalise to unseen classes by leveraging semantic information, bridging the gap between training and testing sets with non-overlapping classes.

By Ysobel Sims, Alexandre Mendes, Stephan Chalup

rag diffusion computer-vision benchmarks safety

arXiv Machine Learning

Jun 16

MVEB: Massive Video Embedding Benchmark

arXiv:2606. 14958v1 Announce Type: cross Abstract: We introduce the Massive Video Embedding Benchmark (MVEB), a 23-task benchmark for video embeddings spanning classification, zero-shot classification, clustering, pair classification, retrieval, and video-centric question answering.

By Adnan El Assadi, Roman Solomatin, Isaac Chung, Chenghao Xiao, Deep Shah, Manan Dey, Shriya Sudhakar, Zacharie Bugaud, Wissam Siblini, Ayush Sunil Munot, Yashwanth Devavarapu, Rakshitha Ireddi, Michelle Yang, M\'arton Kardos, Niklas Muennighoff, Kenneth Enevoldsen

rag nlp multimodal benchmarks

arXiv AI

Jul 1

BEST-RQ-2: Contextualize-Then-Predict, a Two-Step Approach for Self-Supervised Audio Representations

arXiv:2606. 30700v1 Announce Type: cross Abstract: Self-supervised learning enables audio representations that transfer across domains and tasks.

By Ludovic K. Tuncay (IRIT-SAMoVA), Etienne Labb\'e (IRIT-SAMoVA), Thomas Pellegrini (IRIT-SAMoVA)

llms benchmarks

Related stories