arXiv AI
Jul 31
DialectLLM: A Dialect-Aware Dialog[ue] Generation Framework Beyond Standard American English
arXiv:2601. 22888v4 Announce Type: replace-cross Abstract: More than 80% of the 1.
DialectS2S: End-to-End Speech Dialogue Modeling for Low-Resource Chinese Dialects
arXiv:2608. 08067v1 Announce Type: cross Abstract: Current end-to-end speech dialogue models are primarily optimized for mainstream languages and remain limited in low-resource dialect scenarios due to the scarcity of dialect speech data.
Related stories
arXiv AI
Jun 3
Efficient ASR Training with Conversations that Never Happened
arXiv:2606. 03957v1 Announce Type: cross Abstract: Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data.
arXiv Computation and Language
6d ago
Learning Natural Conversational Behavior in Tandem Speech-to-Speech Models with Randomized Guidance
The paper introduces a method called randomized intermediate guidance for training tandem speech-to-speech models, where a large language model (LLM) acts as a backend providing candidate responses while the user is speaking. Instead of simulating the backend’s guidance, the approach derives guidance directly from the conversation corpus, using target responses for informative guidance and randomly sampled responses to simulate irrelevant updates. Experiments on synthetic dialogues and 3.8k hours of real conversations show that this technique yields response quality comparable to LLM-generated baselines while improving natural turn‑taking and audio‑judge naturalness.
Hugging Face Trending Papers
Jun 2
Efficient ASR Training with Conversations that Never Happened
Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of domain-matched multi-speaker training data. We propose an augmentation pipeline that generates scenario-level dialogues with participant metadata, maps speaker attributes to TTS voice profiles, and assembles synthesized utterances into speaker-aware simulated conversations.
arXiv Computation and Language
Sep 22
COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning
arXiv:2609.22697v1 Announce Type: new Abstract: Recently, text-to-speech systems have made significant progress in speech expressiveness and controllability. However, the speaking style of generated...
arXiv AI
Jun 2
PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects
arXiv:2606. 01016v1 Announce Type: cross Abstract: While End-to-End (E2E) Speech-Large Language Models (Speech-LLMs) are rapidly evolving, their evaluation methodologies remain limited to the era of simple transcription.
arXiv AI
Jul 7
SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models
arXiv:2607. 05365v1 Announce Type: cross Abstract: Streaming speech-to-speech language models aim to answer spoken queries directly with synthetic speech.
Hugging Face Trending Papers
Jul 6
SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models
Streaming speech-to-speech language models aim to answer spoken queries directly with synthetic speech. However, standard speech and text benchmarks do not capture whether these systems behave naturally in conversations, where timing, turn-taking, prosody, interpersonal stance, language and dialect consistency, and relationship-aware appropriateness jointly shape perceived quality.
arXiv Computation and Language
Sep 4
Decoupling Turn-Taking from Semantics: A Decoupled Data Approach for Finite-State-Machine-Based Full-Duplex Dialogue
The paper introduces a decoupled data approach for the Neural Finite State Machine (NFSM) framework to improve full‑duplex dialogue. It serializes real human‑human spoken dialogues into FSM tapes using a rule‑based event‑guided transformation, while shaping semantics through human‑agent text dialogues. A Source‑Aware Calibrated (SAC) loss is proposed to balance state‑transition token distribution and align each data source with its strongest supervisory signal, leading to better turn‑taking performance without sacrificing semantic quality.
arXiv AI
Jun 9
GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model
arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.
arXiv Computation and Language
Sep 22
Pretrained Persona Mixture Models and Tandem Models for Human Simulation
arXiv:2609.22607v1 Announce Type: new Abstract: We argue here that the current dominant practice in LLM human simulation: prompting instruction-tuned assistant language models to role-play personas,...
arXiv Computation and Language
Sep 1
UltraVoice: Scaling Fine-Grained Style-Controlled Speech Conversations for Spoken Dialogue Models
arXiv:2510.22588v2 Announce Type: replace-cross Abstract: Spoken dialogue models currently lack the ability for fine-grained speech style control, a critical capability for human-like interaction tha...