PreSTO introduces Predictive SubTree Prefetching to accelerate PowerMH, a power sampling method for large language models. By prefetching future proposals in a tree of accept/reject decisions and using a predictive traversal rule, PreSTO maintains the same Metropolis–Hastings transition kernel while reducing model calls. Experiments on seven datasets and five base LLMs show median speedups of 1.71×, 1.91×, and 2.33× for PowerMH, EntropyCut, and MultiTryMH, respectively, with fewer and shorter model calls.
By Nan Jiang, Panagiotis Theodoropoulos, Wei Deng
arXiv:2611. 07232v1 Announce Type: cross Abstract: At context lengths of about 1M tokens, dense attention is costly in prefill and decoding, and training typically relies on long documents, which become scarcer as the target length increases.
By Ruitian Wang, Xinyu Yang, Minshen Zhang, Hao Zhang
arXiv:2608. 15725v1 Announce Type: new Abstract: Predictive models in clinical and regulated settings must be accurate and fully auditable.
By Srikumar Krishnamoorthy
DominoTree is a speculative decoding method that builds a conditional tree of draft tokens using Domino’s path‑dependent causal correction while restricting to the top‑M candidates. It achieves significant speedups over standard autoregressive decoding—averaging 5.33× on Qwen3‑4B and 6.89× on Qwen3‑8B at temperature zero—and outperforms both Domino and DDTree variants across a range of temperatures and batch sizes. The approach also improves accepted token length and maintains higher acceptance rates than chain‑based methods in various serving scenarios.
By Saw S. Lin (Zhiqi Zhang), Jyh-Shing Roger Jang
JevForest is a feature‑acquisition policy that aggregates path‑dependent proposals from bootstrapped trees, weights them by global training information gain, and uses a shared masked classifier to predict from the acquired values. In online experiments, a four‑question forest on AG News achieved 0.729 accuracy versus 0.667 for a static gain ranking and 0.583 for random ordering, while on TREC the ordering reversed. Offline MiniBooNE tests showed accuracies of 0.845 with ten features and 0.885 with forty features, indicating that Jev’s effectiveness varies with task, predictor, and budget constraints.
By Yu Yan
arXiv:2609. 17464v1 Announce Type: cross Abstract: Multi-agent systems split a task across a tree of agents and justify the split with folklore: smaller contexts, cleaner separation, parallelism.
By Rong He
arXiv:2609.40035v1 Announce Type: new
Abstract: The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We...
By Junyu Lu, Shichao Weng, Zhiqiang Wang, Haojie Luo, Jingfan Zhang, Yuhua Zhou, Cheng Du, Yuzhuo Zhang, Xi Li, Jinwei Du, Tiancheng Feng, Chuan Xiao, Shuyuan Zheng
arXiv:2604. 09731v2 Announce Type: replace-cross Abstract: Tree-based speculative decoding accelerates autoregressive generation by verifying a branching tree of draft tokens in a single target-model forward pass.
By Lifu Wang, Pan Zhou
arXiv:2607. 27027v1 Announce Type: new Abstract: Gradient-boosted trees dominate tabular machine learning, yet canonical correlation analysis has always relied on linear or neural encoders.
By James Chapman
Multi-agent systems split a task across a tree of agents and justify the split with folklore: smaller contexts, cleaner separation, parallelism. We ask what the split does to how much of what the leav...
TreeSpark is a new training and inference framework that extends the semi‑autoregressive DSpark architecture to improve speculative decoding. It builds parent‑conditioned candidate trees in a single backbone pass and then replays their expansion, pruning, and packing during training, using survival negative log‑likelihood and a capped coverage hinge to weight retained edges and promote target‑important alternatives. Evaluations on Qwen3‑4B and Qwen3‑8B across nine general benchmarks and six writing domains show that TreeSpark increases mean acceptance length by 23.4%–31.6% and mean single‑request throughput by 15.1%–25.0% compared to DSpark, with additional gains when using the proposed supervision.
whyItMatters":"TreeSpark demonstrates that a unified training and inference approach can substantially boost both acceptance length and throughput in speculative decoding, offering measurable performance improvements for large language models."
By Jihao Zhao, Qianyu Jiang, Shuaishuai Zu, Pengfei Li, Guanghuan Fang, Ben Feng, Biao Qin, Zhibin Wang, Baochuan Yang
arXiv:2609.22098v1 Announce Type: new
Abstract: Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent...
By Huapeng Zhou, Huayu Wang, Xinyu Wang