arXiv Machine Learning

Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling

arXiv AI
Oct 15

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

The paper introduces Scalable Power Sampling, a training‑free method that sharpens a large language model’s generative distribution by approximating the autoregressive power distribution with a token‑level scaled low‑temperature distribution. This approach eliminates the need for iterative Markov chain Monte Carlo, achieving performance comparable to or better than one‑shot GRPO on math, QA, and code tasks while cutting inference latency by more than tenfold. The method relies solely on the base model’s outputs, requiring no external rewards or verifiers.

By Xiaotong Ji, Rasul Tutunov, Matthieu Zimmer, Haitham Bou Ammar
arXiv Machine Learning
Oct 5

Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning

Power‑SMC is a training‑free sampling method that targets the sequence‑level power distribution used to sharpen large language model outputs for reasoning tasks. It maintains multiple candidate sequences, assigns importance weights based on how well each matches the power distribution, and periodically prunes low‑scoring candidates to keep inference latency close to standard decoding. Experiments on MATH500, GSM8K, GPQA, and HumanEval show that Power‑SMC matches or surpasses Metropolis–Hastings sampling in accuracy, preserves output diversity, and can accelerate inference speed by up to 17.6×.

By Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi, Souvik Kundu, Massoud Pedram
arXiv Computation and Language
Sep 14

Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding

Chopthin-Consensus Power Sampling (CCPS) is a new inference-time decoding method for large language models that uses the Chopthin resampler to preserve diversity among particle trajectories. By enforcing an upper bound on weight ratios instead of equal-weight resampling, CCPS maintains a richer set of distinct reasoning paths and guarantees a lower bound on effective sample size. Coupled with a semantic-majority selection mechanism, CCPS achieves higher oracle coverage and matches or surpasses baseline accuracy on multiple reasoning benchmarks.

By Minoo Ahmadi, Seyedarmin Azizi, Erfan Baghaei Potraghloo, Mehdi Kamal, Massoud Pedram
arXiv Machine Learning
Sep 14

Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

The paper introduces Decision-Flow Sampling (DF‑Sample), a training‑free, data‑free inference framework that builds a hierarchical reasoning tree, evaluates entire trajectories, and back‑propagates utilities to guide branching decisions. Unlike local step‑wise sampling, DF‑Sample explicitly assesses global paths, enabling it to recover high‑quality, low‑probability reasoning chains that standard decoding misses. On the GPQA benchmark, DF‑Sample attains 45.6% accuracy, outperforming power sampling (38.9%) and GRPO (39.9%) and consistently surpassing baselines across multiple models and benchmarks, demonstrating significant latent reasoning potential in pretrained LLMs.

By Zhendong Mi, Shaoyi Huang
arXiv AI
Jun 15

Fractured Chain-of-Thought Reasoning

arXiv:2505. 12992v4 Announce Type: replace-cross Abstract: Inference-time scaling techniques have significantly bolstered the reasoning capabilities of large language models (LLMs) by harnessing additional computational effort at inference without retraining.

By Baohao Liao, Hanze Dong, Yuhui Xu, Doyen Sahoo, Christof Monz, Junnan Li, Caiming Xiong
arXiv Computation and Language
Sep 23

Beyond Repeated Sampling: Learning Search Policies for LLM Reasoning

The paper proposes a new approach to large language model (LLM) reasoning that moves beyond naive repeated sampling. Instead of generating many independent solutions, it first samples problem‑specific concepts, hints, or strategies and conditions answer generation on them, producing a single trajectory of diverse concepts. A small concept generator is then trained via reinforcement learning to maximize downstream success, leading to significant improvements in pass@k on hard mathematical reasoning tasks compared to both naive sampling and concepts from larger untuned models, and the trained generator transfers to unseen answer generators, including those from different model families.

By Ismail Labiad, Matthieu Kowalski, Marc Schoenauer, R\'emi Munos, Julia Kempe