Hugging Face Trending Papers
Realtime-Venus: A full-duplex interaction system with asynchronous delegation
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Computer Vision
Sep 15
Realtime-Venus: A full-duplex interaction system with asynchronous delegation
arXiv:2609.13814v1 Announce Type: new Abstract: Natural interaction in digital and physical environments requires continuous perception and timely responses. Spoken dialogue relies on acoustic and li...
arXiv Computation and Language
Sep 23
Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction
arXiv:2609.25176v1 Announce Type: cross Abstract: Real-time voice assistants must reason over evolving requests, execute actions, and follow conversational rules. Qwen-Audio-3.1-Realtime brings these...
arXiv Computation and Language
Nov 4
InteractionBench: Measuring Timing, Task Accuracy, Whisper and Volume, and Grounding in Real-Time Voice Systems
InteractionBench is a benchmark that evaluates real‑time voice systems on four dimensions—conversational timing, spoken‑task accuracy, whispering and volume, and groundedness—using the same recorded user audio across thirteen scenarios. It tests seven systems (GPT‑Live, GPT‑Realtime, Gemini Live, Moshi, PersonaPlex, Freeze‑Omni, and a turn‑based cascade) over 1,519 runs, revealing correlations between timing and task accuracy and highlighting differences in whispering, grammar correction, and groundedness performance. The study also notes protocol variations and the impact of network delay on timing scores.
arXiv AI
Sep 21
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
arXiv:2609.21967v1 Announce Type: cross Abstract: We introduce NemotronLabs VoiceChat, an open full-duplex speech-to-speech model with native tool-calling capabilities. NemotronLabs VoiceChat combine...
arXiv Computation and Language
Oct 12
People Making Conversation: Evaluating How Naturally Speech Models Converse
The study evaluates conversational speech models by placing them at 427 pauses and 482 turn ends from 176 unscripted conversations, scoring their responses on timing, latency, length, wording, and recall of recent dialogue. Hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause, with latency ranging from 1–4 s for silence-based endpointers and about 8 s for semantic turn detection, while full‑duplex models (PersonaPlex, Moshi) remain silent more often and can interrupt within human reaction time. Recall rates vary: hosted systems remember 41–60 % of facts under a Claude Opus 5.5 judge and 19–26 % with a strict word matcher; full‑duplex models recall 17–22 % and 12–15 % respectively.
arXiv AI
Sep 10
Omni Interaction Agent Technical Report
The technical report introduces Gander, an end‑to‑end model that integrates omni perception, real‑time interaction, and agentic capabilities into a single framework. Unlike traditional turn‑based systems, Gander continuously processes streaming inputs from video, speech, and text, enabling natural full‑duplex interaction in both everyday conversations and workflow‑oriented scenarios. Its architecture features a Cerebellum‑Brain collaboration—where the Cerebellum handles real‑time interaction and omni conversational tasks while the Brain manages complex reasoning—and a streaming Thinker‑Talker design that flattens inputs and outputs into an ordered token stream for low‑latency, continuous dialogue. Evaluations across conversational ability, omni understanding, interactive capability, and agentic intelligence show that Gander matches state‑of‑the‑art open‑source models in spoken dialogue while maintaining robust performance in noisy, multi‑party, and backchannel environments.