arXiv Machine Learning August 28, 2026 By Yunpeng Ba, Zhi Zheng, Yue Xie, Jiaqing Li, Xialiang Tong, Tao Zhong, Mingxuan Yuan, Zhichao Lu, Xuyang Wu, Zhenkun Wang

Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO

Read the original on arXiv Machine Learning →

The paper investigates Evolution Strategies (ES) as a memory‑efficient post‑training method for large language model (LLM) reasoning. It demonstrates that ES outperforms Group Relative Policy Optimization (GRPO) by achieving broader reasoning coverage, improving Pass@K metrics, and avoiding entropy collapse. The study also reveals that ES’s performance gains stem from sparse, high‑magnitude parameter updates, do not cause catastrophic forgetting, and can be combined with GRPO in a sequential training strategy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Oct 27

Correlation-Aware Exploration for Long-Trace Reasoning

arXiv:2610.34949v1 Announce Type: cross Abstract: Reasoning models trained with Group Relative Policy Optimization (GRPO) now produce chains of thought thousands of tokens long, and recent studies re...

By Junxing Hu, Zicheng Zhang, Zhiqian Zhang, Ai Han, Yifeng Zhang, Pengzhang Liu, Ling Li

benchmarks

arXiv AI

Aug 7

Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging

arXiv:2608. 05541v1 Announce Type: new Abstract: Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning.

By Yu Gu, Zhi Zheng, Yunpeng Ba, Xialiang Tong, Mingxuan Yuan, Zhenkun Wang

llms fine-tuning

arXiv Machine Learning

Aug 11

Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training

arXiv:2604. 01499v2 Announce Type: replace Abstract: Evolution Strategies (ES) have emerged as a scalable gradient-free alternative to reinforcement learning based LLM fine-tuning, but it remains unclear whether comparable task performance implies comparable solutions in parameter space.

By William Hoy, Binxu Wang, Xu Pan

llms reinforcement-learning fine-tuning

arXiv AI

Jul 16

Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution

arXiv:2601. 20379v2 Announce Type: replace Abstract: Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability caused by their frozen policy assumption.

By Zhengbo Jiao, Hongyu Xian, Qinglong Wang, Yunpu Ma, Zhebo Wang, Zifan Zhang, Dezhang Kong, Meng Han

llms fine-tuning

arXiv AI

Oct 1

GRPO Training Dynamics for Small Language Models

arXiv:2609.39321v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) has emerged as a memory-efficient reinforcement fine-tuning (RFT) technique for reasoning-intensive tasks....

By Rajat Ghosh, Vaishnavi Bhargava, Henry Wong, Aryan Singhal, Debojyoti Dutta

llms nlp fine-tuning benchmarks

arXiv AI

Jul 24

From Noise to Diversity: Random Embedding Injection in LLM Reasoning

arXiv:2605. 11936v2 Announce Type: replace Abstract: Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated.

By Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim

llms rag benchmarks

One email a morning, machine-written

Related stories

Correlation-Aware Exploration for Long-Trace Reasoning

Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging

Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training

Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution

GRPO Training Dynamics for Small Language Models

From Noise to Diversity: Random Embedding Injection in LLM Reasoning