arXiv Computation and Language
Learning Timeline-Preserving Speech-to-Speech Translation
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv Computation and Language
Oct 26
X-Dubbing: Speech-to-Speech Translation under Temporal and Visual Constraints
arXiv:2610.32042v1 Announce Type: new Abstract: Cross-lingual dubbing aims to convert source speech into target-language speech that remains synchronized with the original video. The key to this task...
arXiv AI
Sep 1
TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models
TEMPO is a unified model that adds temporally‑grounded capabilities to large audio‑language models, enabling timestamping of events, speakers, and sounds in audio, speech, and music. It introduces a supervised fine‑tuning stage featuring atomic timestamp tokens, a time‑aware projector with sinusoidal encodings, and a distance‑aware Gaussian loss, trained via a synthetic‑to‑real curriculum. Additionally, TEMPO employs reinforcement learning (GRPO) as a refinement step, and achieves state‑of‑the‑art performance on a benchmark of 10K samples across five timestamping tasks, surpassing Audio Flamingo Next and Qwen3‑Omni.
arXiv Computation and Language
1d ago
Holistic Parallel Supervision for Expressive Speech to Speech Translation
The paper introduces a training-data construction strategy that combines professional dubbing with multilingual voice conversion to provide holistic parallel supervision for expressive speech-to-speech translation (S2ST). This approach aligns source and target speech in content, prosody, timing, and speaker identity, addressing limitations of existing datasets. Experiments on English‑German and English‑Spanish demonstrate that this supervision improves expressive realization across evaluated systems, with the proposed DirectS2ST method achieving the largest gains.
arXiv AI
Aug 5
dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model
arXiv:2608. 02673v1 Announce Type: cross Abstract: Speech editing for content creation requires precise control over both what an edit should do and where it should apply.
arXiv Computation and Language
Sep 28
All In Good Time: Causality-Aware Framework for LLM-Based Simultaneous Speech-to-Speech Translation
The paper introduces FAST-CAP, a causality‑aware framework for simultaneous speech‑to‑speech translation that combines a factorized S2ST architecture, an adaptive policy, and a new latency metric. It employs a novel data pipeline to generate high‑fidelity, causally aligned segments, improving voice transfer and reducing the need for large training datasets. Experiments on Spanish, German, and French demonstrate that FAST‑CAP outperforms fixed‑policy baselines, achieving up to +1.2 BLEU, 26% lower latency, and a 38.8% relative latency reduction while maintaining speaker fidelity.
arXiv Computation and Language
Oct 1
Tacit-TTS: From Autoregressive Decoding to Masked Prediction for Efficient Transcript-Free Voice Cloning
arXiv:2609.38658v1 Announce Type: cross Abstract: TTS systems with autoregressive semantic modeling have demonstrated strong zero-shot voice cloning performance and rich expressive variation, but the...