arXiv AI

Rhythm of the Deep: Two-tier acoustic organization of sperm-whale codas from click waveforms to second-order sequence dependence

arXiv Computation and Language
Aug 31

Auditing Generative Audio Calls for Known-Task Audio-LLM Evaluation

The paper investigates how to evaluate audio‑language models by separating the use of acoustic evidence from the need to invoke a generative audio model. Using a controlled call‑decision framework, the authors compare policies that rely on transcript labels, encoder outputs from CLAP, AST, or WavLM, and optional calls to generative models such as Qwen2‑Audio, Qwen2.5‑Omni, or MOSS‑Audio. Results on the VocalSound dataset show that while transcript‑only accuracy is low (0.296), encoder‑only controls achieve high accuracy (≈0.85) without any generative calls, and adding generative calls yields only a marginal improvement (0.925 vs. 0.921).

By Mengzhe Geng
Hugging Face Trending Papers
Sep 8

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

TontaubeV1 is a streaming text‑to‑speech model that maintains natural prosody while running on a single consumer GPU. It encodes speech with a hierarchical DualCodec representation at 12.5 Hz, separating a semantic stream from successive acoustic refinements, and uses Qwen3‑derived transformers to predict the semantic stream and add refinements. The model supports up to one minute of reference audio for voice conditioning, streams with a 200 ms latency to first audio, and achieves real‑time factors of 0.08 (single input) and 0.02 (eight concurrent inputs).

arXiv Machine Learning
Sep 15

Large-scale bioacoustic detection using semantic segmentation: a deep learning framework applied to fin whale calls in ocean-bottom seismometer recordings

arXiv:2609.13281v1 Announce Type: cross Abstract: Ocean-bottom seismometers (OBS), originally deployed for geophysical research, continuously record low-frequency sound for months to years across bro...

By Jocelyn Japnanto, Alex A. Saoulis, Miriam Romagosa, Rita Leit\~ao, Gabrielle Arrieta, M\'onica A. Silva, Matthew Graham, Ana M. G. Ferreira
arXiv Computation and Language
Sep 15

Auditing Generative Audio Calls for Known-Task Audio-LLM Evaluatio

The paper investigates how to evaluate generative audio large language models (Audio‑LLMs) on known closed‑set tasks by separating the decision to call a generative model from the use of acoustic evidence. It introduces a controlled call‑decision framework where a policy can choose between a transcript label, encoder evidence from CLAP, AST, or WavLM, or a generative call to Qwen2‑Audio, Qwen2.5‑Omni, or MOSS‑Audio, and measures the impact of generative calls on accuracy. Results on the VocalSound dataset show that while transcript‑only accuracy is low (0.296), encoder‑based controls achieve high accuracy (≈0.85) without any generative calls, and adding generative calls yields only a marginal improvement (0.925 vs. 0.921).

By Mengzhe Geng
arXiv Machine Learning
Sep 10

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

TontaubeV1 is a streaming text‑to‑speech model that preserves natural prosody while running on a single consumer GPU. It encodes speech with a hierarchical DualCodec representation at 12.5 Hz, separating a semantic stream from successive acoustic refinements, and uses Qwen3‑derived transformers to predict the semantic stream, utterance duration, and acoustic refinements. The system supports up to one minute of reference audio for voice conditioning, streams with a 200 ms latency to first audio, and achieves real‑time factors of 0.08 (single input) and 0.02 (eight concurrent inputs).

By Fritz Cremer, Jonathan Cremer
arXiv Machine Learning
Sep 4

VoxReason: Listener-Free Evaluation of Source-Grounded Speech Planning Before Synthesis

VoxReason introduces a listener‑free evaluation framework that measures whether a speech planning system’s delivery choices—such as pitch, energy, rate, pause, emphasis, and stance—are grounded in cited source records before any waveform is generated. The system outputs a source‑cited speaking plan and uses a deterministic verifier to check citation legality, slot agreement, unsupported states, schema validity, and counterfactual locality. Experiments on 1,440 source‑label cases show that simple slot accuracy can be misleading, while a 7B locality‑based repair model significantly improves plan‑slot accuracy and locality, and removing source records sharply reduces the grounded score. whyItMatters":"The framework provides a concrete, measurable way to ensure that expressive speech systems make source‑licensed planning decisions, addressing a source‑use failure that occurs before audio synthesis."

By Mengzhe Geng