PreSTO introduces Predictive SubTree Prefetching to accelerate PowerMH, a power sampling method for large language models. By prefetching future proposals in a tree of accept/reject decisions and using a predictive traversal rule, PreSTO maintains the same Metropolis–Hastings transition kernel while reducing model calls. Experiments on seven datasets and five base LLMs show median speedups of 1.71×, 1.91×, and 2.33× for PowerMH, EntropyCut, and MultiTryMH, respectively, with fewer and shorter model calls.
By Nan Jiang, Panagiotis Theodoropoulos, Wei Deng
arXiv:2611. 07232v1 Announce Type: cross Abstract: At context lengths of about 1M tokens, dense attention is costly in prefill and decoding, and training typically relies on long documents, which become scarcer as the target length increases.
By Ruitian Wang, Xinyu Yang, Minshen Zhang, Hao Zhang
arXiv:2608. 15725v1 Announce Type: new Abstract: Predictive models in clinical and regulated settings must be accurate and fully auditable.
By Srikumar Krishnamoorthy
DominoTree is a speculative decoding method that builds a conditional tree of draft tokens using Domino’s path‑dependent causal correction while restricting to the top‑M candidates. It achieves significant speedups over standard autoregressive decoding—averaging 5.33× on Qwen3‑4B and 6.89× on Qwen3‑8B at temperature zero—and outperforms both Domino and DDTree variants across a range of temperatures and batch sizes. The approach also improves accepted token length and maintains higher acceptance rates than chain‑based methods in various serving scenarios.
By Saw S. Lin (Zhiqi Zhang), Jyh-Shing Roger Jang
JevForest is a feature‑acquisition policy that aggregates path‑dependent proposals from bootstrapped trees, weights them by global training information gain, and uses a shared masked classifier to predict from the acquired values. In online experiments, a four‑question forest on AG News achieved 0.729 accuracy versus 0.667 for a static gain ranking and 0.583 for random ordering, while on TREC the ordering reversed. Offline MiniBooNE tests showed accuracies of 0.845 with ten features and 0.885 with forty features, indicating that Jev’s effectiveness varies with task, predictor, and budget constraints.
By Yu Yan
arXiv:2609. 17464v1 Announce Type: cross Abstract: Multi-agent systems split a task across a tree of agents and justify the split with folklore: smaller contexts, cleaner separation, parallelism.
By Rong He