The paper presents a lightweight vocal intensity conversion system that uses a conditional cycle‑consistent GAN to transform speech features so that the output matches any target sound pressure level (SPL) on a continuous decibel scale, rather than relying on discrete categories or reference utterances. The converter operates on features from a frozen speech encoder and is paired with a pretrained vocoder for resynthesis. Experiments comparing log‑mel spectrograms, WavLM, and SpeechTokenizer representations show that the WavLM‑based model achieves 83% of the target intensity change and also shifts the fundamental frequency, with perceptual tests confirming that perceived intensity rises with the target level on amplitude‑normalized stimuli.
arXiv:2608. 11590v1 Announce Type: cross Abstract: Human voice generation has made rapid progress in speech generation, singing voice generation, voice cloning, and voice editing.
MeanVoiceFlow2 is a new voice conversion framework that jointly optimizes a flow-based conversion module and a computationally efficient content encoder. It is trained via conversion distillation from MeanVoiceFlow and real data reconstruction, and further enhanced with diffusion-GAN training, sample mixing, and teacher-guided conditioning augmentation. Experiments on zero-shot voice conversion show that MeanVoiceFlow2 delivers higher perceptual quality and about nine times faster inference than its predecessor while preserving speaker similarity.
arXiv:2601. 09239v5 Announce Type: replace-cross Abstract: Speech tokenizers are a key building block of fully discrete Speech LLMs.
The paper introduces Prosodic Adjustment with Conversational Context (PACC), a deep‑learning method that converts read speech into natural conversational speech. PACC analyzes and modifies prosodic features—intonation, stress, and rhythm—using HiFi‑GAN for synthesis, aiming to preserve naturalness and intelligibility while keeping computational load low for real‑time use. Experiments show significant improvements in speech conversion, higher model accuracy, and better Mean Opinion Score (MOS) results, establishing new benchmarks for the task.
The paper investigates how different speech content representations—such as SSL features, supervised tokens, posteriorgrams, and neural audio codecs—perform when used to train a generative model that produces audio conditioned only on each representation. By evaluating the generated audio on content, speaker identity, and prosody, the study identifies two regimes: some representations almost fully reconstruct the original audio, while others effectively separate speaker identity. The findings reveal that disentanglement of speaker identity depends on both the training objective and the representation’s information capacity, rather than supervision alone.
Flow-matching approaches to voice conversion (VC) have gained attention owing to their high speech quality and strong speaker similarity. Among them, one-step models such as MeanVoiceFlow are particul...
arXiv:2607. 11706v1 Announce Type: cross Abstract: Modern LLM-driven text-to-speech (TTS) and voice conversion (VC) systems produce synthetic speech that differs from the generators represented in many legacy spoofing benchmarks.
GrainSpeech is a compact speech synthesis model that uses a fixed‑receptive‑field convolutional encoder to reduce pitch, energy, and duration prediction errors by 36.0%, 17.3%, and 3.4% respectively. It introduces a Mel‑specific gradient‑variance supervision that improves fine‑scale variation while avoiding quality degradation. With only 264.8K parameters, GrainSpeech achieves 17.9× real‑time Mel generation on a microcontroller and attains UTMOS scores comparable to much larger models, using less than 1.5% of their parameters.
arXiv:2606. 27543v1 Announce Type: cross Abstract: The variations in vocal effort range (e.
arXiv:2609.14231v1 Announce Type: cross Abstract: Controllable synthesis of nonverbal vocalizations (NVVs) is es- sential for natural and expressive speech, but remains challeng- ing due to their aco...
arXiv:2412. 11449v2 Announce Type: replace-cross Abstract: We propose WHISPER-GPT: A generative large language model (LLM) for speech and music that allows us to work with continuous audio representations and discrete tokens simultaneously as part of a single architecture.
arXiv:2606. 09048v1 Announce Type: cross Abstract: Removing intermediate representations and separately trained decoding stages has become an important direction in generative modeling.