arXiv Machine Learning June 3, 2026 By Andreas Grivas, Lorenzo Loconte, Emile van Krieken, Piotr Nawrot, Yu Zhao, Euan Wielewski, Pasquale Minervini, Edoardo Ponti, Antonio Vergari

Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits

Read the original on arXiv Machine Learning →

arXiv:2511. 11346v2 Announce Type: replace Abstract: Multi-token prediction (MTP) is a prominent strategy to significantly speed up generation in large language models (LLMs), especially in byte-level LLMs, which are tokeniser-free but prohibitively slow.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jun 10

CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

arXiv:2606. 10935v1 Announce Type: cross Abstract: Large language model inference is bottlenecked by autoregressive decoding, where each token requires a full forward pass.

By Xuezhen Xie, Zhiqiang Zhou

llms

arXiv AI

1d ago

Dynamic Multi-Byte Prediction With Hierarchical Language Models

arXiv:2608. 15454v1 Announce Type: new Abstract: Byte-level hierarchical language models (LMs) have recently emerged as a robust alternative to their popular counterparts that use subword tokenization.

By Abraham Toluwase Owodunni, Chibuzor Okocha, Christan Grant, Tomasz Limisiewicz, Sachin Kumar

llms nlp efficiency

arXiv AI

Jul 16

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

arXiv:2607. 13099v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques.

By Z Sun, Q Jiang, S Sheng, L Xiang

llms efficiency benchmarks

arXiv AI

Aug 3

Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs

arXiv:2605. 04215v3 Announce Type: replace-cross Abstract: Diffusion-based Large Language Models (D-LLMs) represent a promising frontier in generative AI, offering fully parallel token generation that can lead to significant throughput advantages and superior GPU utilization over the traditional autoregressive paradigm.

By Michael Rottoli, Subhankar Roy, Stefano Paraboschi

llms diffusion efficiency safety

arXiv Machine Learning

Jul 31

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

arXiv:2607. 27600v1 Announce Type: new Abstract: Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.

By Stephen Gould, Anton van den Hengel

llms efficiency multimodal benchmarks

Hugging Face Trending Papers

Jul 27

DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference

Large Mixture-of-Experts (MoE) language models are attractive for end-device deployment because only a small subset of experts is active per token, but their routed expert weights often exceed accelerator memory. We target latency-critical single-user settings where routed experts are staged on demand from CPU memory to a GPU or from Flash to a mobile NPU.

llms efficiency

Related stories

CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

Dynamic Multi-Byte Prediction With Hierarchical Language Models

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference