arXiv AI
Talking to Me or Someone Else? Rethinking Talk-to-Me Detection in Egocentric Videos
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 2
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
arXiv:2603. 16859v2 Announce Type: replace Abstract: Omni-modal large language models (OLMs) redefine human-machine interaction by natively integrating audio, vision, and text.
arXiv Computer Vision
Aug 28
HUG-VIS: A Multimodal Benchmark for Human-centered Understanding and Generation in Visual Intelligence
HUG‑VIS is a unified multimodal benchmark for human‑centered visual intelligence, comprising 8,400 half‑body videos of 30 professional actors performing 280 emotion‑action prompts in Mandarin. The dataset provides synchronized video, audio, text, and alpha mattes for four tasks—human emotion recognition, video generation, voice cloning, and video matting—allowing evaluation of both open‑ and closed‑source models under a zero‑shot protocol. Results reveal that linguistic cues dominate emotion recognition, visual affect is weakest, and that automatic metrics and human judgments diverge in generation and cloning tasks, while motion‑related boundary fidelity remains a key challenge for matting.
arXiv Computation and Language
3d ago
Exploring Multimodal Turn-Taking Cues in Face-to-Face Conversation using Voice Activity Projection
arXiv:2609.14666v1 Announce Type: cross Abstract: Turn-taking is a fundamental component of spoken interaction, and while humans naturally rely on both verbal and non-verbal signals, dialogue systems...
arXiv AI
Jun 16
MuVAP: Multimodal Multiparty Voice Activity Projection for Turn-taking Prediction in the Wild
arXiv:2606. 16731v1 Announce Type: cross Abstract: Current multiparty turn-taking models often rely on complex microphone arrays or multi-camera setups, limiting their applicability in human-robot interaction scenarios.
arXiv AI
Aug 12
Ex-Omni-2D: Expressive Omni-Modal Dialogue Models with Native Visual Presence
arXiv:2608. 10720v1 Announce Type: new Abstract: Omni-modal dialogue models can understand multimodal inputs and synthesize spoken replies, yet their responses remain visually disembodied.
arXiv Computation and Language
Aug 27
EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports
EgoArgus is a new, human‑annotated dataset that tests visual‑language models (VLMs) as situational assistants in five everyday dialogue‑video scenarios. It evaluates how well VLMs understand and decide when to intervene, especially when visual and textual cues are helpful, irrelevant, or conflicting. The study finds that current VLMs still struggle to reliably act as egocentric assistants and that existing modality‑bias mitigation methods offer limited improvement.