arXiv Computation and Language
The Voice on the Line
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Sep 10
TamilEOT: A Dataset and Model for Semantic End-of-Turn Detection in Tamil Telephone Speech
TamilEOT is a new dataset and two audio‑only models for detecting semantic end‑of‑turn in Tamil telephone speech, comprising 18,485 labeled turn boundaries from 116 real conversations. The models, fine‑tuned from Smart Turn v3, achieve 83.71% and 86.13% accuracy on a held‑out set, with ROC‑AUC rising from 0.751 to 0.921, and run in under 150 ms on a laptop CPU. The paper also reports the cost of building the dataset, the accuracy of rule‑derived labels versus human agreement, and the impact of encoder capacity on performance. whyItMatters":"The study provides the first publicly available Tamil end‑of‑turn detection resource, enabling more accurate voice agents for South Indian languages and demonstrating efficient, low‑latency models that outperform zero‑shot baselines."
arXiv Computation and Language
2d ago
InteractionBench: Measuring Timing, Task Accuracy, Whisper and Volume, and Grounding in Real-Time Voice Systems
InteractionBench is a benchmark that evaluates real‑time voice systems on four dimensions—conversational timing, spoken‑task accuracy, whispering and volume, and groundedness—using the same recorded user audio across thirteen scenarios. It tests seven systems (GPT‑Live, GPT‑Realtime, Gemini Live, Moshi, PersonaPlex, Freeze‑Omni, and a turn‑based cascade) over 1,519 runs, revealing correlations between timing and task accuracy and highlighting differences in whispering, grammar correction, and groundedness performance. The study also notes protocol variations and the impact of network delay on timing scores.
arXiv Computation and Language
Oct 12
People Making Conversation: Evaluating How Naturally Speech Models Converse
The study evaluates conversational speech models by placing them at 427 pauses and 482 turn ends from 176 unscripted conversations, scoring their responses on timing, latency, length, wording, and recall of recent dialogue. Hosted APIs (GPT Realtime, Gemini 3.1 Flash Live, Grok Voice) respond at almost every pause, with latency ranging from 1–4 s for silence-based endpointers and about 8 s for semantic turn detection, while full‑duplex models (PersonaPlex, Moshi) remain silent more often and can interrupt within human reaction time. Recall rates vary: hosted systems remember 41–60 % of facts under a Claude Opus 5.5 judge and 19–26 % with a strict word matcher; full‑duplex models recall 17–22 % and 12–15 % respectively.
arXiv Machine Learning
Sep 10
Clean Accuracy Does Not Guarantee Provenance Robustness: A Prospective Codec-Stress Evaluation of Audio Attribution
The study evaluates audio provenance attribution systems, showing that high clean‑benchmark accuracy does not translate to robustness after codec compression. Using a prospectively registered protocol, the authors measured closed‑set attribution performance on two corpora after single‑stage codec transport, finding significant degradation—up to 70.3 Macro‑F1 points for WavLM‑Base+ and 61.0 for W2V2‑BERT 2.0—depending on codec settings and representation. The results demonstrate that clean accuracy alone cannot guarantee deployment robustness across different codecs and representations.
arXiv Computation and Language
Sep 21
Voice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation
arXiv:2609.20995v1 Announce Type: cross Abstract: Natural spoken interaction requires more than streaming ASR, language generation, and speech synthesis: a system must react to overlap without cancel...
arXiv Computation and Language
Aug 31
Auditing Generative Audio Calls for Known-Task Audio-LLM Evaluation
The paper investigates how to evaluate audio‑language models by separating the use of acoustic evidence from the need to invoke a generative audio model. Using a controlled call‑decision framework, the authors compare policies that rely on transcript labels, encoder outputs from CLAP, AST, or WavLM, and optional calls to generative models such as Qwen2‑Audio, Qwen2.5‑Omni, or MOSS‑Audio. Results on the VocalSound dataset show that while transcript‑only accuracy is low (0.296), encoder‑only controls achieve high accuracy (≈0.85) without any generative calls, and adding generative calls yields only a marginal improvement (0.925 vs. 0.921).