The paper introduces Hikari, a policy‑free, end‑to‑end model that performs simultaneous speech‑to‑text translation and streaming transcription. It employs a Decoder Time Dilation mechanism to mitigate overuse of WAIT tokens during training and a supervised fine‑tuning strategy that helps the model recover from delays, improving the quality‑latency trade‑off. Despite its modest size, Hikari achieves competitive translation quality at consistently low latency, outperforming larger published IWSLT 2026 submissions and proprietary API systems on en‑ja, en‑de, and en‑ru tasks.
By Roman Koshkin, Jeon Haesung, Lianbo Liu, Hao Shi, Mengjie Zhao, Yusuke Fujita, Yui Sudo
arXiv:2504.11809v2 Announce Type: replace
Abstract: Simultaneous speech translation (SimulST) produces translations incrementally while processing partial speech input. Although large language models...
By Biao Fu, Donglei Yu, Minpeng Liao, Chengxi Li, Xinjie Chen, Yidong Chen, Kai Fan, Xiaodong Shi
DIAL‑S2ST is a post‑training method that learns to control inference‑time latency in simultaneous speech‑to‑speech translation models. It reuses pretrained translation capabilities through self‑distillation from lookahead predictions and limits large text distillation losses to address the imitation gap, providing supervision for earlier emission without paired target data or reward computation. By jointly training a single LoRA adapter across scales with matched lookahead guidance, the learned timing can be adjusted at inference, and experiments on French‑to‑English translation with Hibiki show that only 10 hours of source speech enable this adaptation, yielding flexible latency control with a favorable quality‑latency trade‑off.
By Jeongsoo Choi, Junseok Ahn, Joon Son Chung
arXiv:2611.00690v1 Announce Type: cross
Abstract: Speech-to-speech translation (S2ST) increasingly preserves what is said and how it sounds, but not necessarily when speech and silence occur. For dub...
By Zhisheng Zheng, David Harwath
The paper introduces a training-data construction strategy that combines professional dubbing with multilingual voice conversion to provide holistic parallel supervision for expressive speech-to-speech translation (S2ST). This approach aligns source and target speech in content, prosody, timing, and speaker identity, addressing limitations of existing datasets. Experiments on English‑German and English‑Spanish demonstrate that this supervision improves expressive realization across evaluated systems, with the proposed DirectS2ST method achieving the largest gains.
By Hao Zhang, Ismail Rasim Ulgen, Bismarck Bamfo Odoom, Rui Liu, Zihan Zhang, Berrak Sisman, Philipp Koehn
X-Translator is a low‑cost, modular real‑time speech‑to‑speech translation system that integrates streaming ASR, machine translation, and prompt‑conditioned TTS, managed by a session‑level runtime controller. It uses incremental segment commitment to stabilize ASR streams and an online speaker prompt manager to maintain speaker consistency across multi‑speaker conversations. The system is evaluated on translation quality, speech naturalness, latency, and speaker preservation using OpenSTBench, and its code and demo are publicly available on GitHub.
By Yuxiang Zhao, Yichi Zhang, Yanjie An, Yanqiao Zhu, Zhanxun Liu, Yushen Chen, Qixi Zheng, Haina Zhu, Yunchong Xiao, Keqi Deng, Shuai Fan, Kai Yu, Xie Chen
arXiv:2608. 04586v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT).
By Yexing Du, Kaiyuan Liu, Youcheng Pan, Bo Yang, Chengpeng Fu, Yu Wang, Ming Liu
arXiv:2608. 04586v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT).
By Yexing Du, Kaiyuan Liu, Youcheng Pan, Bo Yang, Chengpeng Fu, Yu Wang, Ming Liu
Simultaneous speech translation (SimulST) aims to translate ongoing speech with low latency while preserving translation quality. Recent studies have adopted large language models (LLMs) for SimulST a...
arXiv:2610.35042v1 Announce Type: new
Abstract: Simultaneous speech translation (SimulST) aims to translate ongoing speech with low latency while preserving translation quality. Recent studies have a...
By Haotian Tan, Hiroki Ouchi, Sakriani Sakti
arXiv:2606. 17255v1 Announce Type: cross Abstract: This work describes the participation of the MLLP-VRAIN research group in the shared task of the IWSLT 2026 Simultaneous Speech Translation track.
By Jorge Iranzo-S\'anchez, Gerard Mas-Moll\`a, Adri\`a Gim\'enez, Jorge Civera, Albert Sanchis, Alfons Juan
arXiv:2608. 08638v1 Announce Type: cross Abstract: Zero-shot text-to-speech (TTS) now supports interactive assistants, personalized media, and accessibility tools.
By Yuqian Zhang, Yao Shi, Kexin Huang, Botian Jiang, Zhe Xu, Yiwei Zhao, Min Liang, Shuang Chen, Xipeng Qiu