arXiv:2609.38157v1 Announce Type: cross
Abstract: Emotion-conditioned text-to-speech (TTS) models may fail to express the requested emotion reliably, and improving controllability by additional train...
By Kuan-Po Huang, Haohe Liu, Puyuan Peng, Haibin Wu, Zhaoheng Ni, Hung-yi Lee, Jinwon Lee, Neha Chachra
arXiv:2606. 07309v1 Announce Type: cross Abstract: Instruction-following audio language models (ALMs) can be augmented with explicit acoustic cues, yet it remains unclear whether such cues are used in a grounded way when the raw audio is already available.
By Iosif Tsangko, Andreas Triantafyllopoulos, Bj\"orn W. Schuller
arXiv:2610.10415v1 Announce Type: cross
Abstract: Pretrained text-to-speech (TTS) models can generate expressive speech, but reliable inference-time emotion control remains challenging: prompts and r...
By Afsara Benazir, Darius P\'etermann, Felix Xiaozhu Lin, Salar Rahili
arXiv:2609.39453v1 Announce Type: cross
Abstract: Speech emotion recognition (SER) is the task of assigning emotion labels to utterances. Early systems relied on acoustic features, whereas recent app...
By Hezhao Zhang, Thomas Hain
arXiv:2610.23745v1 Announce Type: cross
Abstract: Recent large audio language models (LALMs) have extended their capabilities beyond lexical speech understanding to tasks requiring paralinguistic and...
By Koki Ryu, Hitomi Yanaka
arXiv:2508.05880v3 Announce Type: replace-cross
Abstract: Understanding human emotions is central to user-facing AI applications, safety alignment, and the simulation of human behavior. As emotional...
By Sree Bhattacharyya, Evgenii Kuriabov, Lucas Craig, Tharun Dilliraj, Reginald B. Adams, Jr., Jia Li, James Z. Wang
arXiv:2606. 27717v1 Announce Type: cross Abstract: Prosodic emphasis varies across languages, emotions, and speaking styles, yet existing emphasis detection models are largely trained and evaluated on monolingual neutral read speech.
By Megan Wei, Deepali Aneja, Jiaqi Su, Yunyun Wang, Haonan Chen, Zeyu Jin
arXiv:2608.28932v1 Announce Type: new
Abstract: Voice products increasingly need affective cues that are present in speech but absent from transcripts. We introduce VocalAffectBench, a public, test-o...
By Models Luc Debaupte, Tyler Baumgartner, Brandon Tai, Candice Fan, Bill Wang, Yi Zhong
The paper introduces an incongruence score that quantifies disagreement between a speaker’s verbal content and vocal expression in a shared valence‑arousal‑dominance space, validated against human judgments. Using this metric, the authors evaluate three audio‑capable large language models across acted, conversational, and spontaneous speech datasets, finding that higher incongruence consistently increases error rates when audio is present. The study also shows that supervised classifiers behave oppositely, suggesting the challenge lies in how models fuse multimodal evidence rather than in the signal itself, and that the score remains predictive even without an emotion lexicon.
By Fengji Jing, Guy Laban
The paper introduces Convex Gate (C‑Gate), a bridge that maps each speech frame to a convex combination of frozen token embeddings, enabling a single LLM decoder to process both speech and text. Using a Whisper‑large‑v3 encoder and a partially adapted Qwen2.5‑7B‑Instruct decoder, the authors jointly train on speech recognition, emotion recognition, and spoken reasoning, achieving accurate transcription and emotion detection while maintaining strong performance on text‑only benchmarks. Analyses reveal that the fixed text embeddings can support both semantic and acoustic readouts through continuous, speech‑conditioned mixtures.
By Ming-Hao Hsu, Yuxuan Hu, Shujie Liu, Jinyu Li, Yan Lu, Zhizheng Wu
arXiv:2606.28249v2 Announce Type: replace-cross
Abstract: Recently, Large Language Model (LLM)-based Text-to-Speech (TTS) models have achieved remarkable naturalness. However, the standard Supervised...
By Sihang Nie, Xiaofen Xing, Rui Xing, Haoming Li, Ruitong Xiao, Jingyuan Xing, Baiji Liu, Xiangmin Xu
arXiv:2605. 16739v2 Announce Type: replace-cross Abstract: Decoding visual experience from brain activity has advanced substantially, but current brain-to-text systems largely recover semantic content while discarding affect.
By Bilal A. Mohammed, Lin Gu, Ruogu Fang