arXiv Computation and Language

EmoRES-TTS: Residual-Enhanced Vector Steering for Emotional Speech Generation

arXiv Computation and Language
Aug 27

Controllable Affective Generation via Latent Vector Steering

The paper introduces EmoVec, a lightweight framework that enables controllable affective generation in large language models by steering latent vectors. EmoVec identifies emotion-specific directions from paired neutral and emotion-conditioned responses using contrastive activation addition, then refines these directions through task-specific debiasing and principal subspace removal. During inference, the refined vectors are injected into the final residual stream with static or scenario-adaptive scaling, allowing continuous control over emotional intensity without updating model weights, and experiments across three LLMs and eight emotions demonstrate improved emotional salience while preserving semantic content, fluency, and coherence.

By Xixian Yong, Siyuan Chang, Yingying Zhang, Xian Wu, Xiao Zhou
arXiv Computation and Language
Oct 8

Steering Follows Geometry, Not Labels: Emotion Directions in a Full-Duplex Speech Model

The paper investigates emotion steering in Moshi, a fully open‑source full‑duplex speech language model, across four emotions. Using mean‑difference activation steering—just a few vector additions per frame and no retraining—the authors find that emotion can be linearly decoded from Moshi’s residual stream, but steering is only partially achievable and uneven: happy, angry and surprise steer toward a shared direction while sad is distinctly steerable. They also demonstrate that the shared component across the three emotions cannot simply be projected away from all emotions equally.

By Pulak Kuli
arXiv AI
Sep 18

Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

The paper introduces a discriminative adaptation for SpeechLLMs that reads the hidden state of the final prompt token via a simple classification head, enabling emotion recognition in a single forward pass without altering the backbone. This approach replaces the generative decoder, which can produce out‑of‑set labels and favor frequent classes, with a controlled comparison between generative and discriminative inference. Experiments on IEMOCAP show improved Macro F1 scores, elimination of hallucinations, and larger gains on realistic ASR transcripts, while revealing that emotion directions encode indirect associations reflecting web‑scale text biases.

By Hasindri Watawana, Sergio Burdisso, Esa\'u Villatoro-Tello, Manjunath K E, Kadri Hacioglu, Petr Motlicek, Andreas Stolcke
arXiv AI
Aug 26

EmoTra-TTS: Smooth Intra-Utterance Emotion Transitions for Speech Synthesis

EmoTra‑TTS introduces a method for smooth intra‑utterance emotion transitions in speech synthesis. It uses a multi‑pass flow blending pipeline, dual‑stage VAD conditioning, and direction‑magnitude decoupled injection to generate frame‑aligned emotional prosody. The system adds only 0.43% more parameters, incurs no latency, and outperforms four state‑of‑the‑art baselines and two commercial systems in emotion transition quality and overall preference tests.

By Tianchi Liu, Zeyang Song, Tianrui Wang, Zhipeng Li, Chenglin Xu, Yiwen Guo
Hugging Face Trending Papers
Sep 2

EmoStance: Response-Side Affective-Orientation Control for Empathetic Response Generation via Emoji Weak Supervision

The paper introduces EmoStance, a method for controlling the affective orientation of empathetic responses in dialogue systems. It leverages weak supervision from multi‑annotator emoji distributions to create a latent control space that approximates listener stance, and uses a frozen instruction‑tuned LLM steered by continuous prefix embeddings. Evaluation shows a 62.2% decisive win rate over baselines, especially in contextual specificity and perceived responsiveness.

arXiv AI
Sep 3

EmoStance: Response-Side Affective-Orientation Control for Empathetic Response Generation via Emoji Weak Supervision

The paper introduces EmoStance, a method for controlling the affective orientation of empathetic responses by leveraging weak supervision from emoji distributions. It builds the EmojiDialogue dataset, extending EmpatheticDialogues with emoji votes and confidence scores, and uses a frozen instruction‑tuned LLM steered by continuous prefix embeddings to generate responses that align with the listener’s stance. In blind pairwise evaluations, EmoStance achieves a 62.2% decisive win rate, notably improving contextual specificity and perceived responsiveness compared to baselines.

By Ziyuan Jin, Yuxuan Ge, Zheng Tian