arXiv AI
Using Prosody to Predict Syntactic Structure
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 9
Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis
arXiv:2606. 07533v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) effectively integrate text and audio to interpret context in complex interactive dialogues.
arXiv Computation and Language
Aug 31
Is Prosody Lost in Translation? Fine-Grained Cross-Lingual Prosody Similarity Across Languages
This paper investigates whether prosodic features—pitch, energy, and timing—are preserved when speech is translated between languages. Using multilingual dubbing data for English‑German, English‑Spanish, and English‑French pairs, the authors conduct a fine‑grained cross‑lingual analysis to quantify similarities and differences in prosody. The study identifies inherent cross‑lingual correlations in prosodic structure and explores how linguistic and alignment factors influence these patterns.
arXiv AI
Sep 2
Heard but Not Heeded: Paralinguistic Information Encoding and Loss in Audio-Language Models
The paper investigates whether audio‑language models capture paralinguistic cues beyond spoken content. Using the Expresso dataset and four open‑source models, the authors trace how speaking style information is encoded in the late layers of the audio encoder but is degraded before reaching the final output. They find that some models are content‑driven while others are acoustic‑driven, revealing a gap between what is encoded and what is utilized in current audio‑language models.
arXiv Computation and Language
Sep 16
Negation Beyond the Verbal Channel: Temporal Multimodal Correlates in Dialogue
arXiv:2609.16396v1 Announce Type: new Abstract: Negation is typically modeled through its linguistic realization, although spoken interaction is accompanied by tightly coordinated nonverbal behavior....
arXiv Computation and Language
Oct 9
Prosody-to-Text: Predicting text from low-pass filtered speech
The paper explores the reverse of the common prosody-from-text task by predicting text from low‑frequency speech features. Using a Whisper model fine‑tuned on only the 12 lowest Mel bins (≈450 Hz cutoff), the authors achieve a word error rate of 36 %, with 10 % of utterances perfectly recovered and 40 % below 25 % WER. They also report that, given the correct prefix, the next token is predicted correctly 79 % of the time, indicating a stronger link between low‑frequency prosody and lexical content than previously assumed.
arXiv Computation and Language
Sep 1
When Models Hear What They Expect: Diagnosing Prosodic Heuristics in Multimodal Sarcasm Detection
The study investigates how multimodal large language models (MLLMs) use prosodic cues in sarcasm detection. By testing Qwen2.5‑Omni and Qwen3‑Omni on Mandarin Chinese and English across five modality conditions, the authors find that adding audio increases false positives without improving true positives. Acoustic error analysis shows that models rely on a stereotypical prosodic pattern—elevated pitch and irregular pausing—that does not align with genuine sarcasm cues, and manipulating these dimensions alone can raise false positive rates up to 60%. The same effect appears in Gemini 3 Flash Preview, indicating the heuristic is not limited to a single architecture.