← Back to all news
arXiv Computation and Language By Yuxiang Wang, Qinke Ni, Shengbo Cai, Wan Lin, Liqiang Zhang, Zhizheng Wu

ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models

Read the original on arXiv Computation and Language →

ParaBridge is a self‑distillation method that uses a temporary paralinguistic instruction scaffold during training to teach a speech language model when non‑lexical cues should influence dialogue responses. By providing dense, full‑vocabulary next‑token targets from the scaffolded view while the scaffold‑free model generates its own replies, ParaBridge stabilizes inference‑time behavior without requiring curated dialogues or external reward models. Experiments on Qwen3‑Omni show significant gains on safety and empathy benchmarks while preserving general performance across multiple tests.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Aug 28

SpeechGym: An Audio-Native Gym for Training Voice Agents via Reinforcement Learning

SpeechGym is an audio‑native environment that lets two omni‑modal models converse entirely in native audio, eliminating external ASR/TTS and API boundaries while preserving the tasks, tools, and success checks of a standard text‑based agent benchmark. By training end‑to‑end, the framework addresses perceptual failures—such as misheard arguments that cascade into failed calls—and behavioural failures, both of which are automatically labeled for free. Using per‑turn process rewards to overcome reward sparsity, agents trained in SpeechGym transfer to an independent voice benchmark, doubling task success and improving efficiency in turns and tokens.

By Jiajun Fan, Jingyuan Li, Prashanth Gurunath Shivakumar, Jia-Hong Huang, Qi Luo, M. Maruf, Ivan Bulyko, Ge Liu, Roger Ren
agentsreinforcement-learningefficiencymultimodalbenchmarks
More like this →
arXiv Computation and Language
Sep 14

SteerDuplex: Steerable Duplex Speech Dialogue Models

SteerDuplex is a full‑duplex speech dialogue model that can be steered along attributes such as tone, persona, speaking rate, and voice style in response to user instructions. The authors introduce a taxonomy of text‑ and audio‑based steerability, identify gaps in existing models, and fine‑tune a Moshi‑based model with reinforcement learning to improve timing and response continuity. They also present SteerBench, a benchmark of 390 spoken prompts and 1,067 human‑authored rubrics, showing significant gains in audio‑steering pass rates and interruption handling compared to open baselines.

By Utkarsh Tyagi, Ramaneswaran Selvakumar, Advait Gosai, Sonal Kumar, Nikhil Barhate, Isabell Sagar, Steven Li, Miheer Bavare, Daniel Quigley, Fabiola Tapia Carrillo, Jose M Patron E, Diego Mac\'ias Guti\'errez, Paul Song, Ramani Duraiswami, Dinesh Manocha, Yunzhong He
reinforcement-learningfine-tuningbenchmarks
More like this →