arXiv AI

Breaking the Quality--Intelligibility Trade-off in Streaming Target Speaker Extraction via Deep-Feature-Anchored Preference Optimization

arXiv:2607. 10191v1 Announce Type: cross Abstract: Generative streaming models for Target Speaker Extraction (TSE) commonly exhibit a quality--intelligibility trade-off, wherein naive optimization for perceptual audio quality tends to degrade speech intelligibility, and conversely.

arXiv AI

Jul 7

StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model

arXiv:2604. 19635v2 Announce Type: replace-cross Abstract: While generative models have set new benchmarks for Target Speaker Extraction (TSE), their inherent reliance on global context precludes deployment in real-time applications.

By Shuhai Peng, Hui Lu, Jinjiang Liu, Liyang Chen, Guiping Zhong, Jiakui Li, Huimeng Wang, Haiyun Li, Liang Cao, Shiyin Kang, Zhiyong Wu

llms benchmarks

arXiv AI

Jul 3

UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement

arXiv:2510. 20441v2 Announce Type: replace-cross Abstract: Neural audio codecs have largely promoted the application of language models (LMs) for speech applications.

By Haoyin Yan, Chengwei Liu, Shaofei Xue, Xiaotao Liang, Yinghao Liu, Yuxiang Kong, Zheng Xue

llms reinforcement-learning benchmarks

arXiv AI

Jun 9

GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model

arXiv:2512. 20978v2 Announce Type: replace-cross Abstract: Language Model (LM)-based generative modeling has emerged as a promising direction for TSE, offering potential for improved generalization and high-fidelity speech.

By Haoyang Li, Xuyi Zhuang, Azmat Adnan, Ye Ni, Wei Rao, Shreyas Gopal, Eng Siong Chng, Boon Siew Han, Yuanjin Zheng

llms rag reinforcement-learning safety

arXiv AI

1d ago

Adding Voice Cloning to Text-to-Audio-Video Models with a Single Zero-Initialised Layer

arXiv:2608. 15690v1 Announce Type: cross Abstract: Text-to-audio-video (T2AV) generation models produce a video and its soundtrack from a textual description, but offer no control over whose voice speaks in the output.

By Ivan Mikheev, Viacheslav Vasilev, Anna Dmitrienko, Alexey Letunovskiy, Ivan Kirillov, Kirill Chernyshev, Denis Dimitrov

rag diffusion fine-tuning multimodal benchmarks

arXiv AI

Jun 8

SpectCount: Spectrotemporal Counting via Synthetic Signals Improves Large Audio Language Models

arXiv:2606. 06907v1 Announce Type: cross Abstract: Large audio language models (LALMs) extend large language models with an audio encoder and large-scale audio data.

By Seonuk Kim, Yonghyeon Jun, Ju Yeon Kang, Jimin Hong, Yoonhyeong Lee, Nam Soo Kim

llms fine-tuning benchmarks

arXiv AI

Aug 5

UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinations

arXiv:2604. 14606v2 Announce Type: cross Abstract: Universal speech enhancement (USE) aims to restore speech signals from diverse distortions across multiple sampling rates.

By Xiaobin Rong, Zheng Wang, Yushi Wang, Jun Gao, Jing Lu

fine-tuning efficiency benchmarks safety

arXiv Machine Learning

Jun 10

ANCHOR: Autoregressive Non-intrusive Chunk-Ordered Refinement for Joint Multi-Resolution Speech Quality Modeling

arXiv:2606. 10233v1 Announce Type: cross Abstract: While speech quality is typically assessed on complete utterances, streaming and generative systems require incremental estimation from partial audio.

By Zhuoyan Tao, Jiatong Shi, Hye-jin Shim, Shinji Watanabe

arXiv AI

Aug 11

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.

By Wenxu Jia, Dongjie Fu, Xize Cheng, Fangming Feng, Linjun Li, Wenshi Chen, Yingming Li, Zhou Zhao, Tao Jin

llms efficiency multimodal benchmarks safety

arXiv AI

Jul 7

Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings

arXiv:2607. 03806v1 Announce Type: cross Abstract: Audio foundation models are widely adopted as general-purpose feature extractors, yet the internal structure of their learned representations remains insufficiently understood.

By H\'ector Martel, Joe Hennessy-Priest, Taemin Cho

rag multimodal

arXiv AI

Jun 2

VocSim: A Training-free Benchmark for Zero-shot Content Identity in Single-source Audio

arXiv:2512. 10120v2 Announce Type: replace-cross Abstract: General-purpose audio representations aim to map acoustically variable instances of the same event to nearby points, resolving content identity in a zero-shot setting.

By Maris Basha, Anja Zai, Sabine Stoll, Richard Hahnloser

rag multimodal benchmarks safety

arXiv AI

Jul 7

Unified Audio Intelligence Without Regressing on Text Intelligence

arXiv:2607. 05196v1 Announce Type: cross Abstract: Audio intelligence involves understanding, reasoning about, and generating both audio and speech.

By Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, Yang Chen, Arushi Goel, Rajarshi Roy, Wenliang Dai, Zhuolin Yang, Yangyi Chen, Dongfu Jiang, Sreyan Ghosh, Tuomas Rintamaki, Andrew Tao, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping

llms rag agents reinforcement-learning efficiency multimodal benchmarks safety

arXiv AI

Jun 19

ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesis

arXiv:2603. 04219v2 Announce Type: replace-cross Abstract: We investigate the use of zero-shot text-to-speech (ZS-TTS) as a data augmentation source for low-resource personalized speech synthesis.

By Youngwon Choi, Jinwoo Oh, Hwayeon Kim, Hyeonyu Kim

rag fine-tuning multimodal

Related stories