arXiv Machine Learning August 27, 2026 By Hoang M. Nguyen, Satya N. Shukla, Qiang Zhang, Hanchao Yu, Sreya D. Roy, Dipesh Tamboli, Taipeng Tian, Lingjiong Zhu, Yuchen Liu

BRIDLE: Generalized Self-supervised Learning with Quantization

Read the original on arXiv Machine Learning →

BRIDLE is a self‑supervised encoder pretraining framework that extends bidirectional training to audio, image, and video by incorporating residual quantization (RQ) with multiple hierarchical codebooks. This approach allows fine‑grained discretization of latent representations and interleaves training between the encoder and tokenizer. Experiments show that BRIDLE achieves state‑of‑the‑art results on audio classification benchmarks and competitive performance on image and video classification tasks, outperforming traditional vector‑quantization methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 6

F3-Tokenizer: Taming Audio Autoencoder Latents for Understanding and Generation

arXiv:2606. 06357v1 Announce Type: cross Abstract: Continuous audio autoencoders reconstruct waveforms well but often produce latents with weak structure for understanding, while self-supervised audio encoders capture semantics but are not directly decodable.

By Dinghao Zhou, Xingchen Song, Di Wu, Pengyu Cheng, Shengfan Shen, Sixiang Lv

llms nlp

arXiv Computation and Language

Sep 14

Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning

arXiv:2609.13045v1 Announce Type: new Abstract: Speech-to-speech translation (S2ST) has advanced significantly with speech LLMs, offering the potential for joint optimization and preserving non-lingu...

By Hayato Futami, Hassan Shahmohammadi, Tushar Dhyani, Alkis Koudounas, Rapha\"el Lafargue, Yosuke Kashiwagi, Quentin Jodelet, Emiru Tsunoo

llms efficiency

arXiv AI

Aug 21

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

arXiv:2608. 19863v1 Announce Type: cross Abstract: Self-supervised learning (SSL) has driven substantial progress in audio representation learning, though existing methods have increasingly relied on elaborate pre-training recipes to reach competitive performance.

By Umberto Cappellazzo, Xubo Liu, Stavros Petridis, Maja Pantic

llms rag fine-tuning benchmarks

arXiv AI

Jun 3

EntangleCodec: A Unified Discrete Audio Tokenizer via Semantic-Acoustic Entanglement

arXiv:2606. 02739v1 Announce Type: cross Abstract: Audio tokenizers serve as the discrete interface between continuous audio and Audio Language Models (ALMs), but existing tokenizers often struggle to support both understanding and generation.

By Hui Li, Yangfan Gao, Junlin Shang, Changhao Jiang, Tao Gui, Qi Zhang, Xuanjing Huang

llms diffusion nlp efficiency multimodal benchmarks

arXiv Machine Learning

Sep 29

NAC: Neural Action Codec for Vision-Language-Action Models

The paper introduces the Neural Action Codec (NAC), a convolutional encoder‑decoder architecture that treats short robot action trajectories as multi‑channel 1D signals and compresses them using a multi‑scale residual vector quantization (RVQGAN) model. NAC replaces traditional discrete action tokenizers with a compact, ordered token space via offset codebooks, allowing standard autoregressive policies to operate over short, structured sequences while a Vocos‑style decoder reconstructs the actions. Experiments on LIBERO‑10, RoboMimic, and real‑world manipulation tasks show that NAC achieves higher reconstruction fidelity and better average success rates than existing binning, FAST, and VQ‑based tokenizers at comparable or improved compression rates.

By Ahad Jawaid, Yu Xiang

nlp robotics efficiency multimodal safety

Hugging Face Trending Papers

Jun 23

MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning

Self-supervised learning from large-scale video data has emerged as a dominant paradigm for visual representation learning. Since audio and visual streams naturally co-occur in video data, extending this success to jointly learn from both modalities is a natural next step, yet it remains challenging.

rag multimodal benchmarks

One email a morning, machine-written

Related stories

F3-Tokenizer: Taming Audio Autoencoder Latents for Understanding and Generation

Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

EntangleCodec: A Unified Discrete Audio Tokenizer via Semantic-Acoustic Entanglement

NAC: Neural Action Codec for Vision-Language-Action Models

MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning