arXiv AI
$\tau$-Elicitation: Benchmarking multi-turn entity extraction in voice agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Sep 1
VoiceCodeBench: Evaluating Exact Structured-Token Recovery in Automatic Speech Recognition
arXiv:2608.28916v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems are commonly evaluated with word error rate (WER), yet many voice workflows depend on exact written values f...
arXiv AI
Aug 12
DuplexWorld: Can voice agents help you get through the day?
arXiv:2608. 10716v1 Announce Type: cross Abstract: Speech-to-speech (S2S) voice agents are increasingly being incorporated into enterprise for customer care and as daily companions for consumers owing to the ease of the conversational modality over text.
arXiv AI
Aug 25
CallScreenBench: Benchmarking Small Language Models as Phone Secretaries
CallScreenBench is a benchmark for evaluating small, on-device language models that act as phone secretaries, focusing on their ability to handle unknown inbound calls without a cooperative task. The benchmark measures owner endorsement through five call-and-note metrics, each paired with counter-metrics and uncertainty estimates, and includes guardedness diagnostics to identify safe, tool‑free proxies. Results across 4‑bit checkpoints of 0.6‑4 B parameter models show varying performance on service, recall, plausibility, and triage discrimination, highlighting trade‑offs between quality and guardedness.
arXiv Computation and Language
Sep 2
SpokenUS: A Spoken User Simulator for Task-Oriented Dialogue
arXiv:2603.16783v2 Announce Type: replace Abstract: Robust voice agents require exposure to the full diversity of how people interact through speech. However, obtaining enough spoken interactions is...
arXiv AI
22h ago
MTVA-Bench: Evaluating the Language Model Inside Cascaded Voice Agents
MTVA-Bench is a new benchmark designed to evaluate the language model component of cascaded voice agents under realistic conditions. It simulates callers with an LLM, mocks backend tool responses, and scores both tool‑call correctness and conversational quality using two LLM judges, covering 49 agents, 490 scenarios, and 7 languages. The benchmark reveals that while models perform similarly on tool selection, they differ widely in argument handling, rule compliance, and dialogue quality, highlighting the nuanced challenges of real‑world voice interactions.
arXiv Computation and Language
Aug 26
Better Retrieval, Worse Robustness: How Multi-hop RAG Amplifies Upstream ASR Errors
arXiv:2608.22872v2 Announce Type: new Abstract: Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline...