PairAlign: A Framework for Autoregressive Tokenization via Self-Alignment with Applications to Audio Tokenization
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2605. 06582v2 Announce Type: replace Abstract: Many operations on sensory data -- comparison, memory, retrieval, and reasoning -- are naturally expressed over discrete symbolic structures.
arXiv:2610.27491v1 Announce Type: cross Abstract: Real-world audio typically contains concurrent sound events, yet standard self-supervised methods treat each clip as an isolated training unit withou...
arXiv:2607. 06831v1 Announce Type: cross Abstract: Speech-to-text alignment means finding the temporal boundaries of each word in the audio.
AudioTETRIS is a training‑free, two‑stage token elimination framework designed for large audio language models. It leverages the geometry of audio encoder representations as an acoustic prior to compute per‑token importance scores, which guide progressive top‑k token selection across decoder layers. Experiments on Qwen2‑Audio‑7B, Phi‑4‑Multimodal, and Audio Flamingo 3 show that AudioTETRIS retains near‑full performance while using only 25% of audio tokens and outperforms existing training‑free baselines.
arXiv:2603. 01006v3 Announce Type: replace-cross Abstract: REPresentation Alignment (REPA) improves the training of generative flow models by aligning intermediate hidden states with pretrained teacher features, but its effectiveness in token-conditioned audio Flow Matching critically depends on the choice of supervised layers, which is typically made heuristically based on the depth.
arXiv:2603. 15685v2 Announce Type: replace-cross Abstract: Omnimodal large language models (OmniLLMs) jointly process audio and visual streams, but the resulting long multimodal token sequences make inference prohibitively expensive.