arXiv Machine Learning June 3, 2026 By Andreas Grivas, Lorenzo Loconte, Emile van Krieken, Piotr Nawrot, Yu Zhao, Euan Wielewski, Pasquale Minervini, Edoardo Ponti, Antonio Vergari

Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits

Read the original on arXiv Machine Learning →

arXiv:2511. 11346v2 Announce Type: replace Abstract: Multi-token prediction (MTP) is a prominent strategy to significantly speed up generation in large language models (LLMs), especially in byte-level LLMs, which are tokeniser-free but prohibitively slow.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv AI

Jun 10

CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

arXiv:2606. 10935v1 Announce Type: cross Abstract: Large language model inference is bottlenecked by autoregressive decoding, where each token requires a full forward pass.

By Xuezhen Xie, Zhiqiang Zhou

llms

arXiv AI

Aug 18

Dynamic Multi-Byte Prediction With Hierarchical Language Models

arXiv:2608. 15454v1 Announce Type: new Abstract: Byte-level hierarchical language models (LMs) have recently emerged as a robust alternative to their popular counterparts that use subword tokenization.

By Abraham Toluwase Owodunni, Chibuzor Okocha, Christan Grant, Tomasz Limisiewicz, Sachin Kumar

llms nlp efficiency

arXiv Machine Learning

Sep 21

TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching

arXiv:2609.21172v1 Announce Type: new Abstract: Large language models (LLMs) are moving onto mobile devices for increasingly diverse workloads over text, images, video, and audio. These applications...

By Zhihao Shu, Md Musfiqur Rahman Sanim, Jie Hu, Kun Yuan, Minghai Qin, Gagan Agrawal, Wei Niu

llms efficiency

arXiv AI

2d ago

Match the Distribution, Not the Compute: Post-Training Multi-Token Prediction Heads

arXiv:2610.00888v1 Announce Type: cross Abstract: Multi-token prediction (MTP) improves the throughput of autoregressive generation by enabling the language model to draft multiple next tokens per fo...

By Prachi Badarayani, Aidan Jay, Chenghui Zhou, Dayquan Julienne, Yuan Gao, Tianwei Chen, George Zerveas, Ishmam Zabir, Xiren Zhou, Chris Quirk, Xia Song

llms efficiency benchmarks

arXiv AI

Jul 16

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

arXiv:2607. 13099v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable success but raise growing concerns about content provenance and misuse, motivating the need for reliable watermarking techniques.

By Z Sun, Q Jiang, S Sheng, L Xiang

llms efficiency benchmarks

arXiv AI

Aug 3

Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs

arXiv:2605. 04215v3 Announce Type: replace-cross Abstract: Diffusion-based Large Language Models (D-LLMs) represent a promising frontier in generative AI, offering fully parallel token generation that can lead to significant throughput advantages and superior GPU utilization over the traditional autoregressive paradigm.

By Michael Rottoli, Subhankar Roy, Stefano Paraboschi

llms diffusion efficiency safety

One email a morning, machine-written

Related stories

CLP: Collocation-Length Prediction for Zero-Loss Adaptive Multi-Token Inference

Dynamic Multi-Byte Prediction With Hierarchical Language Models

TierKV: Long-Context On-Device LLMs via Predictive Multi-Tier KV Caching

Match the Distribution, Not the Compute: Post-Training Multi-Token Prediction Heads

WaterMoE: Expert-Routing-based Watermarking for High Fidelity and Efficiency

Predict-then-Diffuse: Adaptive Response Length for Compute-Budgeted Inference in Diffusion LLMs