arXiv Computer Vision
EmAvatar: Multimodal Empathetic Response Generation via Conflict Resolution and Expressive Guidance
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv Computer Vision
Aug 28
HUG-VIS: A Multimodal Benchmark for Human-centered Understanding and Generation in Visual Intelligence
HUG‑VIS is a unified multimodal benchmark for human‑centered visual intelligence, comprising 8,400 half‑body videos of 30 professional actors performing 280 emotion‑action prompts in Mandarin. The dataset provides synchronized video, audio, text, and alpha mattes for four tasks—human emotion recognition, video generation, voice cloning, and video matting—allowing evaluation of both open‑ and closed‑source models under a zero‑shot protocol. Results reveal that linguistic cues dominate emotion recognition, visual affect is weakest, and that automatic metrics and human judgments diverge in generation and cloning tasks, while motion‑related boundary fidelity remains a key challenge for matting.
arXiv Machine Learning
Jun 2
Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning
arXiv:2606. 00851v1 Announce Type: cross Abstract: Empathetic spoken dialogue systems must infer a user's emotional state to respond appropriately, yet everyday speech often carries weak, neutral, or ambiguous affective cues.
Hugging Face Trending Papers
Sep 2
EmoStance: Response-Side Affective-Orientation Control for Empathetic Response Generation via Emoji Weak Supervision
The paper introduces EmoStance, a method for controlling the affective orientation of empathetic responses in dialogue systems. It leverages weak supervision from multi‑annotator emoji distributions to create a latent control space that approximates listener stance, and uses a frozen instruction‑tuned LLM steered by continuous prefix embeddings. Evaluation shows a 62.2% decisive win rate over baselines, especially in contextual specificity and perceived responsiveness.
arXiv Machine Learning
Jun 3
Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals
arXiv:2606. 02679v1 Announce Type: new Abstract: Multimodal systems often benefit from combining information across language, sound, and visual streams, but this benefit is not guaranteed.
arXiv Machine Learning
Sep 16
EMODY Flow: Emotion-Aware Audio-Driven Full-Body Motion Generation
EMODY Flow is a lightweight flow‑matching framework that generates synchronized full‑body motion and facial expressions conditioned on speech and emotion. It attaches to a frozen Qwen‑3 Omni model, reusing its audio codecs to drive two parallel DiT generators for SMPL‑X body pose and FLAME facial expressions. An auxiliary emotion classifier at training time restores emotion sensitivity, enabling EMODY Flow to achieve state‑of‑the‑art gesture quality on BEAT2 and zero‑shot facial animation on TFHP, with significant improvements in FGD, Beat Correlation, and Diversity metrics.
arXiv AI
Sep 3
EmoStance: Response-Side Affective-Orientation Control for Empathetic Response Generation via Emoji Weak Supervision
The paper introduces EmoStance, a method for controlling the affective orientation of empathetic responses by leveraging weak supervision from emoji distributions. It builds the EmojiDialogue dataset, extending EmpatheticDialogues with emoji votes and confidence scores, and uses a frozen instruction‑tuned LLM steered by continuous prefix embeddings to generate responses that align with the listener’s stance. In blind pairwise evaluations, EmoStance achieves a 62.2% decisive win rate, notably improving contextual specificity and perceived responsiveness compared to baselines.