The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The paper introduces a pipeline that generates intent‑labeled, two‑channel conversational speech from relational event lists, enabling controlled synthesis of full‑duplex dialogue with 42 phenomena across eight families in English and Mandarin. By having an LLM author each event’s speaker, text, conversational act, and attachment, and then aligning and timing these events independently, the system produces diverse, realistic turn‑taking signals. Experiments show that models trained on this synthetic corpus achieve higher floor‑occupancy accuracy and better start‑speaking/listening F1 scores compared to models trained on prior data.
The paper introduces a synthetic data generation pipeline for long‑context audio reasoning, focusing on first‑visit doctor‑patient conversations and SOAP note generation. The pipeline consists of persona‑driven dialogue creation, multi‑speaker audio synthesis with realistic acoustics, and LLM‑based reference note production, all built with open‑weight models. The authors release 8,800 synthetic conversations, 1.3k hours of audio, and reference notes, and show that cascaded approaches still outperform end‑to‑end models on current open‑weight systems.
Full-duplex speech models require training data that preserves turn-taking, overlap, interruption, and backchannel behavior, yet these signals are entangled across speakers in noisy real-world recordi...
The paper introduces ELLSA 2, an end‑to‑end full‑duplex dialogue model that jointly optimizes speech interaction and streaming gesture‑expression generation. It uses an autoregressive, chunk‑wise synthesizer that combines pretrained acoustic features with distinct listening and speaking hidden states, enabling continuous non‑verbal generation even when the model is silent. The authors also present contextual semantic calls and template guidance to separate the decision to invoke a semantic gesture from its realization, and they establish a curated data and evaluation pipeline for dialog‑aware co‑speech gesture generation, showing that ELLSA 2 outperforms cascaded baselines and that semantic template guidance significantly improves gesture quality.
The paper introduces a frontend‑backend architecture for full‑duplex speech‑to‑speech models that enables tool calls while preserving natural conversational flow. The frontend emits a delegation token and streams ASR transcripts to a text‑based backend LLM, which performs tool calls and returns results that are re‑injected into the frontend via a lightweight prefill‑and‑repeat mechanism before streaming TTS synthesis. In single‑turn evaluations the system achieves 92‑97% tool‑call recall, 81.2% accuracy in rejecting irrelevant calls, and competitive performance on Full‑Duplex‑Bench‑V3 and EVA‑Bench when paired with a large backend model.
arXiv:2608.28630v1 Announce Type: cross Abstract: Compared with half-duplex dialogue systems where the system waits for user turn completion before it responds, natural full-duplex dialogue systems r...