Chopthin-Consensus Power Sampling (CCPS) is a new inference-time decoding method for large language models that uses the Chopthin resampler to preserve diversity among particle trajectories. By enforcing an upper bound on weight ratios instead of equal-weight resampling, CCPS maintains a richer set of distinct reasoning paths and guarantees a lower bound on effective sample size. Coupled with a semantic-majority selection mechanism, CCPS achieves higher oracle coverage and matches or surpasses baseline accuracy on multiple reasoning benchmarks.
By Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo, Mehdi Kamal, Massoud Pedram
arXiv:2509. 15676v2 Announce Type: replace-cross Abstract: In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new and data-scarce tasks using only a few carefully selected task-specific examples presented in the prompt.
By Vaibhav Singh, Soumya Suvra Ghosal, Kapu Nirmal Joshua, Soumyabrata Pal, Sayak Ray Chowdhury
arXiv:2606. 10768v1 Announce Type: new Abstract: The success of Large Language Models in mathematical reasoning relies heavily on the generation of diverse and valid solution paths during the rollout phase.
By Xukun Zhu, Hang Yu, Peng Di, Linchao Zhu
arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.
By Avinash Reddy, Thayne T. Walker, James S. Ide, Amrit Singh Bedi
The paper investigates why token prediction, a common pre‑training objective for language models, yields useful representations. It introduces a statistical framework linking token prediction accuracy to the geometry of token embeddings, showing that accurate predictions organize embeddings according to Hellinger distances between context distributions. The authors also propose a self‑consistency principle that refines contextual representations through repeated application of a shared block, and provide downstream guarantees for token generation, community recovery, and linear classification.
By Shulei Wang
The paper proposes a modular tokenizer framework for multilingual large language models, allowing the creation of language‑specific subtokenizers that match monolingual compression quality. It introduces a pretraining strategy that samples these subtokenizers to limit predictions to relevant vocabularies, enabling efficient training and inference. This approach reduces memory usage and speeds up inference without compromising performance.
By Franck Signe, Hippolyte Pilchen, Fran\c{c}ois Yvon, \'Edouard Grave
AdaFuse is an adaptive ensemble decoding framework for large language models that dynamically selects fusion units during generation. It uses an uncertainty-based criterion to decide when to ensemble, applying a diversity-aware scaling strategy in uncertain states while continuing direct generation when confident. Experiments on question answering, arithmetic reasoning, and machine translation show AdaFuse outperforms strong baselines with an average relative improvement of 6.88%.
By Chengming Cui, Tianxin Wei, Ziyi Chen, Ruizhong Qiu, Zhichen Zeng, Zhining Liu, Xuying Ning, Duo Zhou, Jingrui He
arXiv:2606. 27161v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of visual tokens, which introduces substantial computational overhead.
By Tinghao Wang, Yichen Guo, Rui Huang, Zheng Lu, Qizhe Zhang, Chenxi Li, Yuan Zhang, Jiajun Cao, Zhirong Shen, Yaosong Du, Guangyan Gan, Wenya Wang, Lin William Cong, Shanghang Zhang
arXiv:2607. 23913v1 Announce Type: new Abstract: Modern vision-language models (VLMs) increasingly rely on dynamic or high-resolution visual encoding, producing thousands of visual tokens that substantially increase downstream language-model inference cost.
By Jun Ling, Tao Huang, Junzhuo Liu, Bowen Tang, Peng Wang
arXiv:2608.30263v1 Announce Type: cross
Abstract: Large vision-language models (LVLMs) incur substantial inference costs due to their long and highly redundant visual-token sequences. Diversity-based...
By Shunjie Wen, Jaeyeon Lee, Dong-Wan Choi
Multilingual Large Language Models (LLMs) traditionally rely on a single vocabulary shared by all supported languages, which can lead to uneven compression across them. Moreover, their large embedding...
arXiv:2604. 24927v2 Announce Type: replace-cross Abstract: Generating diverse responses is crucial for test-time scaling of large language models (LLMs), yet standard stochastic sampling mostly yields surface-level lexical variation, limiting semantic exploration.
By Yuanhao Zeng, Ao Lu, Lufei Li, Zheng Zhang, Yexin Li, Kan Ren