Voxtral transcribes at the speed of sound.

Voxtral

Speaking of Voxtral

Voxtral TTS: A frontier, open-weights text-to-speech model that’s fast, instantly adaptable, and produces lifelike speech for voice agents.

AudioLDM 2, but faster ⚡️

Voxtral 3

Voxtral 3 is Mistral AI’s voice‑agent SDK that enables real‑time speech processing in over 50 languages. It offers accurate transcription, emotionally expressive text‑to‑speech, and deep tool‑use capabilities. The SDK can be hosted in the EU or self‑deployed by users.

Spectro-Temporal Interference Confounds Phase Encoding in Spatial Audio Foundation Models

arXiv:2606. 14820v1 Announce Type: cross Abstract: Recent spatial self supervised audio models achieve high performance on localization tasks, raising questions about their encoding of microsecond interaural phase fine structures.

HRTF Upsampling and Personalization via Neural Operators

arXiv:2610.20400v1 Announce Type: cross Abstract: Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formu...

VoxReason: Listener-Free Evaluation of Source-Grounded Speech Planning Before Synthesis

VoxReason introduces a listener‑free evaluation framework that measures whether a speech planning system’s delivery choices—such as pitch, energy, rate, pause, emphasis, and stance—are grounded in cited source records before any waveform is generated. The system outputs a source‑cited speaking plan and uses a deterministic verifier to check citation legality, slot agreement, unsupported states, schema validity, and counterfactual locality. Experiments on 1,440 source‑label cases show that simple slot accuracy can be misleading, while a 7B locality‑based repair model significantly improves plan‑slot accuracy and locality, and removing source records sharply reduces the grounded score. whyItMatters":"The framework provides a concrete, measurable way to ensure that expressive speech systems make source‑licensed planning decisions, addressing a source‑use failure that occurs before audio synthesis."

Spooftral: Can Voxtral Audio-Language Model Detect Speech Spoofing?

The paper investigates whether the Voxtral audio‑language model can detect speech spoofing. It shows that without task‑specific adaptation, the model’s language‑model layers prioritize semantic content, making spoof‑discriminative acoustic cues less separable. By applying lightweight weight‑decomposed low‑rank adaptation (DoRA), the authors create Spooftral, which achieves an equal error rate of 4.25% on the ASVspoof5 evaluation set.

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.

Probing Low Frame Rate Degradation in Neural Audio Codecs

arXiv:2606. 16969v1 Announce Type: cross Abstract: Low frame rates in neural audio codecs are attractive for autoregressive speech synthesis, where the generation cost scales linearly with the sequence length.

TontaubeV1: Streaming Text-to-Speech with Hierarchical Codec Modeling and Bounded Context

TontaubeV1 is a streaming text‑to‑speech model that maintains natural prosody while running on a single consumer GPU. It encodes speech with a hierarchical DualCodec representation at 12.5 Hz, separating a semantic stream from successive acoustic refinements, and uses Qwen3‑derived transformers to predict the semantic stream and add refinements. The model supports up to one minute of reference audio for voice conditioning, streams with a 200 ms latency to first audio, and achieves real‑time factors of 0.08 (single input) and 0.02 (eight concurrent inputs).

The Anatomy of an ASR Hallucination

The paper investigates why automatic speech recognition (ASR) systems sometimes generate fluent text that does not correspond to the input audio, a phenomenon termed hallucination. By examining two independently trained Conformer‑Large models—one using CTC and the other RNN‑T—under conditions of environmental noise and speaker‑background shift, the authors identify the final encoder stage as a critical boundary. Bypassing this final block leads to divergence on almost all utterances, while bypassing earlier blocks has minimal effect; at this stage, representations become compact, the decoder can read the text, and grapheme information becomes explicit, yet the output is garbled or repetitive rather than fluent fabrication. The study thus pinpoints a mechanistic precondition for hallucination—failure to produce adequately grounded output—though it does not fully explain naturally occurring hallucinations, and it highlights a consistent terminal‑stage dependency across decoder families and distribution shifts.