← Back to all news
arXiv Machine Learning By Ulysse Roussel (STMS)

Audio emotion recognition for atypical hearing

Read the original on arXiv Machine Learning →

The doctoral research investigates Audio Emotion Recognition (AER) for individuals with atypical hearing, specifically focusing on auditory hypersensitivity in people with autism. It proposes leveraging affective insights from acoustic traits and aims to generalize emotional responses from limited annotated data. The study begins by fine‑tuning the Contrastive Language‑Audio Pretraining (CLAP) foundation model using low‑rank adaptation (LoRA) on a valence and arousal dataset collected from neurotypical listeners.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 1

Beyond Text: LLM-Based Dimensional Emotion Evaluation in Multimodal Dialogue

The paper introduces an LLM-based framework for continuous dimensional emotion evaluation in multimodal dialogue, combining discrete emotion recognition with Valence-Arousal-Dominance (VAD) assessment on the IEMOCAP dataset. It incorporates acoustic cues as natural language descriptions via the SpeechCueLLM approach and evaluates six models from the LLaMA, GPT, and Qwen families using zero-shot, few-shot, and LoRA fine-tuning. LoRA-fine-tuned LLaMA models outperform prompt-engineered GPT models, achieving a new state-of-the-art Valence CCC of 0.7822, and ablation studies show that textual audio descriptions significantly benefit smaller models. "whyItMatters":"The study demonstrates that domain adaptation through fine-tuning can surpass larger GPT models in multimodal emotion evaluation, highlighting the importance of tailored training for emotion recognition tasks."

By Yutong Hu, Jinho Choi
llmsfine-tuningmultimodalbenchmarks
More like this →