arXiv AI
Stride-k Subsampling: Train-Free Audio Token Reduction for Whisper
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 21
Towards Audio Token Compression in Large Audio Language Models
arXiv:2511. 20973v2 Announce Type: replace-cross Abstract: Large Audio Language Models (LALMs) deliver strong performance across speech and audio tasks, but their audio encoders generate high-rate token sequences (e.
arXiv AI
Oct 12
AudioTETRIS: Token Elimination via Transitional Importance Signals for Efficient Audio LLMs
AudioTETRIS is a training‑free, two‑stage token elimination framework designed for large audio language models. It leverages the geometry of audio encoder representations as an acoustic prior to compute per‑token importance scores, which guide progressive top‑k token selection across decoder layers. Experiments on Qwen2‑Audio‑7B, Phi‑4‑Multimodal, and Audio Flamingo 3 show that AudioTETRIS retains near‑full performance while using only 25% of audio tokens and outperforms existing training‑free baselines.
arXiv AI
Sep 18
Scaling Audio Models Efficiently: Joint Optimization of Scale, Resolution, Adaptation, Precision, and Sparsity
The paper introduces a compression framework for the Whisper automatic speech recognition model that jointly optimizes six deployment dimensions—model size, temporal resolution, encoder token stride, low‑rank adaptation capacity, weight precision, and sparsity pattern—using NSGA‑III. The optimization targets three objectives: word error rate, inference FLOPs, and memory footprint. Evaluating 1,680 configurations, the study identifies compression combinations that outperform single‑axis scaling and notes that 1:4 structured sparsity cannot maintain acceptable accuracy within the tested budgets.
arXiv Machine Learning
Jul 23
Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting
arXiv:2607. 20086v1 Announce Type: cross Abstract: State-space sequence models are attractive for streaming speech because they maintain compact recurrent state, but scan-style training kernels can have unfavorable constants for short audio tasks.
arXiv Machine Learning
Oct 21
TTS-Q: Speaker-Aware Post-Training Quantization of Text-to-Speech Diffusion Transformers
The paper presents TTS‑Q, a post‑training quantization method that reduces the size of large diffusion transformer TTS models to 4‑bit precision while preserving speaker timbre and intelligibility. By identifying hard‑to‑quantize weights and activations and applying a mixed‑precision schedule that gives higher precision to late‑step feed‑forward activations, TTS‑Q restores timbre loss observed in standard 4‑bit PTQ. The approach also offers a calibrated‑W4A4 reference that lowers GPU memory usage, making TTS diffusion transformers more viable for on‑device, assistive, and real‑time applications.
arXiv AI
Aug 11
VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference
arXiv:2608. 08569v1 Announce Type: new Abstract: Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks.