← Back to all news
arXiv Computation and Language By Nan Jiang, Panagiotis Theodoropoulos, Wei Deng

PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling

Read the original on arXiv Computation and Language →

PreSTO introduces Predictive SubTree Prefetching to accelerate PowerMH, a power sampling method for large language models. By prefetching future proposals in a tree of accept/reject decisions and using a predictive traversal rule, PreSTO maintains the same Metropolis–Hastings transition kernel while reducing model calls. Experiments on seven datasets and five base LLMs show median speedups of 1.71×, 1.91×, and 2.33× for PowerMH, EntropyCut, and MultiTryMH, respectively, with fewer and shorter model calls.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Jun 10

Sample Where You Struggle: Sharpening Base Model Reasoning via Entropy-Guided Power Sampling

arXiv:2606. 09926v1 Announce Type: cross Abstract: Sampling from the sequence-level power distribution $p^\alpha$ elicits RL-level reasoning from base language models without any parameter updates, but the standard Metropolis--Hastings (MH), a Markov Chain Monte Carlo (MCMC) sampler, is both expensive and slow-mixing.

By Hong Guo, Nianhui Guo, Christoph Meinel, Haojin Yang
llmsreinforcement-learningbenchmarks
More like this →
arXiv Computation and Language
5d ago

TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding

TreeSpark is a new training and inference framework that extends the semi‑autoregressive DSpark architecture to improve speculative decoding. It builds parent‑conditioned candidate trees in a single backbone pass and then replays their expansion, pruning, and packing during training, using survival negative log‑likelihood and a capped coverage hinge to weight retained edges and promote target‑important alternatives. Evaluations on Qwen3‑4B and Qwen3‑8B across nine general benchmarks and six writing domains show that TreeSpark increases mean acceptance length by 23.4%–31.6% and mean single‑request throughput by 15.1%–25.0% compared to DSpark, with additional gains when using the proposed supervision. whyItMatters":"TreeSpark demonstrates that a unified training and inference approach can substantially boost both acceptance length and throughput in speculative decoding, offering measurable performance improvements for large language models."

By Jihao Zhao, Qianyu Jiang, Shuaishuai Zu, Pengfei Li, Guanghuan Fang, Ben Feng, Biao Qin, Zhibin Wang, Baochuan Yang
efficiencybenchmarks
More like this →