arXiv:2609.14666v1 Announce Type: cross Abstract: Turn-taking is a fundamental component of spoken interaction, and while humans naturally rely on both verbal and non-verbal signals, dialogue systems...
arXiv:2608.22731v1 Announce Type: new Abstract: Nonverbal behavior generation systems for virtual agents often take an utterance as input and generate nonverbal behaviors that emphasize or illustrate...
arXiv:2608.20905v1 Announce Type: new Abstract: Face-to-face audiovisual interaction is central to human communication, conveying rich emotional and social cues. However, existing multimodal dialogue...
arXiv:2603. 16859v2 Announce Type: replace Abstract: Omni-modal large language models (OLMs) redefine human-machine interaction by natively integrating audio, vision, and text.
arXiv:2609.14118v1 Announce Type: cross Abstract: Online understanding of who is talking to the camera wearer is a key capability for egocentric social interaction. However, existing talk-to-me (TTM)...
arXiv:2606. 15888v1 Announce Type: cross Abstract: Non-verbal vocalizations (NVs), such as laughter, sighs, and coughs, are important acoustic cues for emotion and intent.
The study investigates how gaze and speech cues, together with perceived interpersonal closeness, predict turn‑taking outcomes in free four‑person conversations. Using the GaMMA corpus, logistic regression models were trained on interpretable features such as gaze transition motifs, entropy, addressee identity, mutual gaze, and speaker loudness to classify floor‑transfer events as gaps or overlaps. Results show that gaze features alone capture predictive structure, and combining them with loudness yields a robust classifier (ROC AUC = 0.76 ± 0.04) that remains effective even under noisy conditions.
arXiv:2608.30260v1 Announce Type: cross Abstract: While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two dom...
Nonverbal behavior generation systems for virtual agents often take an utterance as input and generate nonverbal behaviors that emphasize or illustrate the content of the verbal channel. However, huma...
VoiceLongMemEval (VLME) is a new benchmark that tests AI assistants on their ability to remember how users sounded by incorporating paralinguistic metadata—such as emotion labels, prosody descriptors, and voice events—into each conversational turn. The benchmark uses a three‑stage adversarial gate to ensure that models cannot succeed with transcript alone, revealing a significant affect gap: models gain 0.09 to 0.38 accuracy when provided with paralinguistic cues, and audio‑native models outperform standard ASR pipelines in extracting these signals. The dataset and code will be released upon acceptance.
arXiv:2606. 11386v1 Announce Type: cross Abstract: Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains underexplored.
The paper introduces ContraTalk, a benchmark that tests whether dialogue models truly use acoustic cues or rely on transcript shortcuts. It formalizes cross‑modal disagreement, creates conflict and consistent QA examples, and proposes an Audio Twin representation to expose acoustic evidence to models. Experiments show that while text‑only LLMs perform well on consistent cases, they falter on conflict cases, and AudioLLMs only partially mitigate this issue.