arXiv Machine Learning October 19, 2026 By Mahmoud Ashraf

Flash RNN-T: Memory-Efficient Joint Network and Transducer Loss Computation

Read the original on arXiv Machine Learning →

Flash RNN‑T is a new implementation for the recurrent neural network transducer that combines specialized kernels with optional tiled recomputation to reduce memory usage while maintaining high throughput. On NVIDIA H100 GPUs, the untiled version delivers 24–39% higher throughput and 26–56% less memory than the fastest baseline, and the tiled version can use nine times less memory on medium workloads with only a 16% throughput drop. For long workloads, tiled Flash fits within 80 GB of memory where all other tested alternatives run out of memory, achieving at least 95% of peak throughput in its lowest‑memory configuration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

arXiv:2601. 06199v3 Announce Type: replace-cross Abstract: Scaling Multimodal Large Language Models (MLLMs) to long-form speech is bottlenecked by the explosive growth of input tokens.

By Junseok Lee, Sangyong Lee, Chang-Jae Chun

llms multimodal benchmarks

arXiv Machine Learning

Jul 23

Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting

arXiv:2607. 20086v1 Announce Type: cross Abstract: State-space sequence models are attractive for streaming speech because they maintain compact recurrent state, but scan-style training kernels can have unfavorable constants for short audio tasks.

By Mahesh Godavarti

benchmarks

arXiv AI

Jul 24

Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs

arXiv:2607. 21042v1 Announce Type: new Abstract: Autoregressive text-to-speech models achieve strong naturalness but suffer from slow inference due to sequential token generation, limiting their deployment in production applications that require low latency.

By Muyang Du, Shuang Yu, Junjie Lai

llms diffusion efficiency multimodal benchmarks

arXiv Machine Learning

Sep 1

You Do Not Fully Utilize Transformer's Representation Capacity

arXiv:2502.09245v3 Announce Type: replace Abstract: In contrast to RNNs, which compress their history into a single hidden state, Transformers can attend to all past tokens directly. However, standar...

By Gleb Gerasimov, Yaroslav Aksenov, Nikita Balagansky, Viacheslav Sinii, Daniil Gavrilov

llms

arXiv AI

2d ago

Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription

Confucius4‑R2T2, a stable‑prefix streaming ASR model, is introduced to ensure that once a token is emitted it is never revised, thereby reducing wasted work and incorrect actions in real‑time transcription. The model is derived by post‑training Qwen3‑ASR‑1.7B on 1,500 hours of Chinese and English audio while keeping its audio encoder frozen, and it leverages a new concept called Longest Stable Prefix (LSP) to convert a full‑context model into a stable‑prefix streaming one without architectural changes. Evaluation shows that R2T2 achieves near full‑context quality with low latency and stable output, and its benefits generalize beyond the languages used in training.

By Daniel Huang, Qi He, Shaoxiong Duan, Yingjie Li, Qiao Cheng, Yitao Duan

agents multimodal

arXiv Machine Learning

Sep 2

Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior

arXiv:2605.26797v2 Announce Type: replace Abstract: We study Latent Recurrent Transformer (LRT), a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden...

By Zeyi Huang, Xuehai He, LiLiang Ren, Yiping Wang, Baolin Peng, Hao Cheng, Shuohang Wang, Pengcheng He, Jianfeng Gao, Yong Jae Lee, Yelong Shen

llms

One email a morning, machine-written

Related stories

FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

Cumsum-Composable Phase Transport for Low-Cost Streaming Keyword Spotting

Faster IndexTTS-2: Accelerating and Streaming Autoregressive Zero-Shot Text-to-Speech Synthesis on GPUs

You Do Not Fully Utilize Transformer's Representation Capacity

Confucius4-R2T2: A Stable-Prefix Streaming ASR Model for Real Real-Time Transcription

Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior