arXiv AI
PaceReasoner: Label-Free Learning to Answer Briefly from Long Reasoning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
Hugging Face Trending Papers
6d ago
PaceReasoner: Label-Free Learning to Answer Briefly from Long Reasoning
PaceReasoner is a label‑free, test‑time reinforcement learning framework that improves short‑mode accuracy by leveraging a model’s long responses. It shares a trainable policy between long and short modes, using progressive, confidence‑weighted supervision and a behavior reward that shifts from encouraging short exploration to rewarding agreement with long answers. Across three backbones and four benchmarks, it raises short‑mode accuracy by 7.4–13.5 percentage points while keeping responses under 25 tokens on average.
arXiv AI
Jun 18
Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
arXiv:2606. 18831v1 Announce Type: cross Abstract: Long-context reasoning is an essential capability for large language models, particularly when they are deployed as autonomous agents that must reason over lengthy trajectories.
arXiv AI
Sep 16
Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
arXiv:2507.21931v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) often produce plausible but poorly-calibrated answers, limiting their reliability on reasoning-intensive tasks....
arXiv AI
Sep 28
Learning to Stop without Learning to Stop: Self-Supervised Confidence Training Improves Reasoning Efficiency
The paper demonstrates that fine‑tuning reasoning models to predict their own confidence at intermediate steps—using only 600 self‑supervised examples—substantially improves inference efficiency. Without adding any explicit stopping or length penalties, the models generate up to 25 % fewer tokens while maintaining accuracy on mathematical, scientific, and coding benchmarks across several architectures. The study finds that confidence supervision preserves the models’ high‑level reasoning structure rather than merely suppressing specific behaviors.
arXiv AI
Sep 30
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
arXiv:2609.37119v1 Announce Type: cross Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...
arXiv Machine Learning
Jul 27
Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning
arXiv:2511. 02130v2 Announce Type: replace-cross Abstract: We propose Re-FORC, an adaptive reward prediction method that, given a query, enables prediction of the expected future rewards as a function of the number of future thinking tokens.