arXiv AI
GRPO Training Dynamics for Small Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Aug 7
Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging
arXiv:2608. 05541v1 Announce Type: new Abstract: Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning.
arXiv AI
Jul 28
Multi-Task GRPO: Reliable LLM Reasoning Across Tasks
arXiv:2602. 05547v2 Announce Type: replace-cross Abstract: RL-based post-training with GRPO is widely used to improve large language models on individual reasoning tasks.
arXiv Machine Learning
Jun 15
Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
arXiv:2602. 08324v5 Announce Type: replace Abstract: Chain-of-Thought (CoT) reasoning successfully enhances the reasoning capabilities of Large Language Models (LLMs), yet it incurs substantial computational overhead for inference.
arXiv Machine Learning
Aug 28
Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
The paper investigates Evolution Strategies (ES) as a memory‑efficient post‑training method for large language model (LLM) reasoning. It demonstrates that ES outperforms Group Relative Policy Optimization (GRPO) by achieving broader reasoning coverage, improving Pass@K metrics, and avoiding entropy collapse. The study also reveals that ES’s performance gains stem from sparse, high‑magnitude parameter updates, do not cause catastrophic forgetting, and can be combined with GRPO in a sequential training strategy.
arXiv Machine Learning
Jul 9
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
arXiv:2607. 06987v1 Announce Type: new Abstract: Reinforcement learning (RL) has become the standard paradigm for enhancing the complex reasoning capabilities of large language models (LLMs).
arXiv AI
Sep 10
Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models
The paper evaluates the scalability and adversarial generalization of Natural Language Inference (NLI) models trained with Group Relative Policy Optimization (GRPO) for Chain-of-Thought learning. By fine‑tuning 7B, 14B, and 32B language models with LoRA and QLoRA, the authors show strong performance on standard and adversarial NLI benchmarks, with the 32B model outperforming supervised baselines on adversarial sets. Using AWQ quantization, the 32B model fits within 22 GB of CUDA memory, demonstrating a scalable, practical framework for robust NLI without sacrificing inference quality.