Hugging Face Trending Papers
Sparse RL Relaxation for LLM Reasoning
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Oct 21
Sparse RL Relaxation for LLM Reasoning
The paper introduces SpaRLax, a sparse relaxation method for reinforcement learning that enhances soft Chain-of-Thought reasoning in large language models. By treating soft tokens as differentiable intermediate steps and applying an adaptive entmax sparsity technique, SpaRLax stabilizes training and improves performance across nine reasoning benchmarks. The method retains multiple plausible candidates when uncertain while truncating noise at confident steps.
arXiv AI
Jun 10
Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thought
arXiv:2509. 04027v4 Announce Type: replace Abstract: Test-time scaling, primarily manifested through multi-step Chain-of-Thought (CoT) reasoning via Reinforcement Learning (RL), has emerged as a pivotal paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs).
arXiv AI
Jun 6
CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning
arXiv:2509. 04027v3 Announce Type: replace Abstract: Test-time scaling, primarily manifested through multi-step Chain-of-Thought (CoT) reasoning via Reinforcement Learning (RL), has emerged as a pivotal paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs).
arXiv AI
Oct 16
CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models
CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models proposes a training‑free framework that replaces binary latent reasoning decisions with continuous token‑level allocation. By using the probability competition between the two most likely tokens, CoTS dynamically balances discrete commitment and soft exploration at each reasoning step, eliminating the need for manually tuned thresholds or additional forward passes. Experiments on six reasoning benchmarks show that CoTS consistently outperforms standard Chain‑of‑Thought and other state‑of‑the‑art training‑free latent reasoning methods.
arXiv AI
Sep 7
Extremely Sparse Supervision Incentivizes Reasoning Ability
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.
arXiv Computation and Language
Aug 28
Boosting LLM Exploration via Weak-Model Guidance in RLVR
The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.