arXiv AI September 17, 2026 By Dunyao Xue, Chengshuo Du, Zhengbo Wang, Wenlin Dai, Cheng Meng

Beyond Truncation: Rethinking LLM Decoding as Ensemble Pruning

The paper introduces Mahalanobis-Ensemble Decoding (ME-Decoding), a new framework for Large Language Model decoding that treats candidate token selection as an ensemble pruning problem. It uses a Mahalanobis distance-driven objective to promote semantic diversity while maintaining high probabilities, employing a token similarity matrix built with an adaptive-bandwidth kernel over token embeddings. An efficient greedy algorithm with near-linear complexity and theoretical guarantees makes ME-Decoding a plug‑and‑play module with negligible inference overhead, and experiments show strong performance across reasoning and generation tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language

Sep 14

Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding

Chopthin-Consensus Power Sampling (CCPS) is a new inference-time decoding method for large language models that uses the Chopthin resampler to preserve diversity among particle trajectories. By enforcing an upper bound on weight ratios instead of equal-weight resampling, CCPS maintains a richer set of distinct reasoning paths and guarantees a lower bound on effective sample size. Coupled with a semantic-majority selection mechanism, CCPS achieves higher oracle coverage and matches or surpasses baseline accuracy on multiple reasoning benchmarks.

By Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo, Mehdi Kamal, Massoud Pedram

llms benchmarks

arXiv AI

Jun 4

KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning

arXiv:2509. 15676v2 Announce Type: replace-cross Abstract: In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new and data-scarce tasks using only a few carefully selected task-specific examples presented in the prompt.

By Vaibhav Singh, Soumya Suvra Ghosal, Kapu Nirmal Joshua, Soumyabrata Pal, Sayak Ray Chowdhury

llms rag

arXiv Machine Learning

Jun 10

N-GRPO: Embedding-Level Neighbor Mixing for Enhanced Policy Optimization

arXiv:2606. 10768v1 Announce Type: new Abstract: The success of Large Language Models in mathematical reasoning relies heavily on the generation of diverse and valid solution paths during the rollout phase.

By Xukun Zhu, Hang Yu, Peng Di, Linchao Zhu

llms rag benchmarks

arXiv AI

Jun 30

The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decoding

arXiv:2603. 03305v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used to generate executable outputs, JSON objects, and API calls, where a single syntax error can make the output unusable.

By Avinash Reddy, Thayne T. Walker, James S. Ide, Amrit Singh Bedi

llms benchmarks

arXiv Machine Learning

Sep 1

Learning Representations through Token Prediction: Geometry, Approximation, and Downstream Guarantees

The paper investigates why token prediction, a common pre‑training objective for language models, yields useful representations. It introduces a statistical framework linking token prediction accuracy to the geometry of token embeddings, showing that accurate predictions organize embeddings according to Hellinger distances between context distributions. The authors also propose a self‑consistency principle that refines contextual representations through repeated application of a shared block, and provide downstream guarantees for token generation, community recovery, and linear classification.

By Shulei Wang

llms rag

arXiv Computation and Language

Sep 15

To Each Language Its Tokenizer: Modular Tokenizers for Efficient Multilingual LLMs

The paper proposes a modular tokenizer framework for multilingual large language models, allowing the creation of language‑specific subtokenizers that match monolingual compression quality. It introduces a pretraining strategy that samples these subtokenizers to limit predictions to relevant vocabularies, enabling efficient training and inference. This approach reduces memory usage and speeds up inference without compromising performance.

By Franck Signe, Hippolyte Pilchen, Fran\c{c}ois Yvon, \'Edouard Grave

llms rag nlp safety