arXiv AI
$\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 1
Talk2Agent: Benchmarking Voice Interfaces for Text Agents
arXiv:2609.38867v1 Announce Type: new Abstract: Large language model (LLM) computer-use agents are typically evaluated with clean written instructions, despite speech being an increasingly popular in...
arXiv Computation and Language
Sep 1
VoiceCodeBench: Evaluating Exact Structured-Token Recovery in Automatic Speech Recognition
arXiv:2608.28916v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems are commonly evaluated with word error rate (WER), yet many voice workflows depend on exact written values f...
arXiv AI
Aug 12
DuplexWorld: Can voice agents help you get through the day?
arXiv:2608. 10716v1 Announce Type: cross Abstract: Speech-to-speech (S2S) voice agents are increasingly being incorporated into enterprise for customer care and as daily companions for consumers owing to the ease of the conversational modality over text.
arXiv AI
Sep 30
$\tau$-Multilingual: Benchmarking Voice Agents Across Languages
arXiv:2609.35820v1 Announce Type: cross Abstract: English-only benchmarks expose only a narrow slice of voice-agent behavior. We introduce $\tau$-Multilingual, extending $\tau$-Voice to Spanish, Braz...
arXiv Computation and Language
Oct 22
From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agents
The paper introduces a reproducible framework that converts existing text-based tool‑calling benchmarks into audio‑based evaluations using text‑to‑speech, speaker variation, and environmental noise, while preserving original annotations. Experiments on seven omni‑modal models show that performance varies strongly by model and task, with notable gaps between text and voice results. The study also includes a stress test, a reference‑free LLM‑as‑judge protocol, and demonstrates that open‑source judges can match proprietary ones, aiding privacy‑preserving evaluation.
arXiv AI
Sep 28
Inquesto Score: A reliability Protocol For Voice Agents
The paper introduces Inquesto Score (IS), a protocol that measures voice‑agent reliability by calculating the percentage of calls that reach the caller’s goal without functional failure. IS defines explicit failure events and severity levels, evaluates timing, semantic, and state‑dependent failures using audio, scenario predicates, tool traces, and a pinned open‑model judge, and provides diagnostic views on behavior, acoustic robustness, identity handling, and speaker groups. The authors evaluate IS v0.1 on 30 scenarios, three acoustic conditions, four speaker groups, and 306 calls per agent across 13 configurations, demonstrating that reliable measurement requires evidence beyond transcripts and explicit treatment of deployment conditions.