arXiv Machine Learning August 28, 2026 By Yunpeng Ba, Zhi Zheng, Yue Xie, Jiaqing Li, Xialiang Tong, Tao Zhong, Mingxuan Yuan, Zhichao Lu, Xuyang Wu, Zhenkun Wang

Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO

Read the original on arXiv Machine Learning →

The paper investigates Evolution Strategies (ES) as a memory‑efficient post‑training method for large language model (LLM) reasoning. It demonstrates that ES outperforms Group Relative Policy Optimization (GRPO) by achieving broader reasoning coverage, improving Pass@K metrics, and avoiding entropy collapse. The study also reveals that ES’s performance gains stem from sparse, high‑magnitude parameter updates, do not cause catastrophic forgetting, and can be combined with GRPO in a sequential training strategy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 7

Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging

arXiv:2608. 05541v1 Announce Type: new Abstract: Evolution Strategy (ES) is a promising alternative to gradient-based fine-tuning for resource-constrained Large Language Model (LLM) reasoning.

By Yu Gu, Zhi Zheng, Yunpeng Ba, Xialiang Tong, Mingxuan Yuan, Zhenkun Wang

llms fine-tuning

arXiv Machine Learning

Aug 11

Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training

arXiv:2604. 01499v2 Announce Type: replace Abstract: Evolution Strategies (ES) have emerged as a scalable gradient-free alternative to reinforcement learning based LLM fine-tuning, but it remains unclear whether comparable task performance implies comparable solutions in parameter space.

By William Hoy, Binxu Wang, Xu Pan

llms reinforcement-learning fine-tuning

arXiv AI

Jul 16

Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution

arXiv:2601. 20379v2 Announce Type: replace Abstract: Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability caused by their frozen policy assumption.

By Zhengbo Jiao, Hongyu Xian, Qinglong Wang, Yunpu Ma, Zhebo Wang, Zifan Zhang, Dezhang Kong, Meng Han

llms fine-tuning

arXiv AI

Jul 24

From Noise to Diversity: Random Embedding Injection in LLM Reasoning

arXiv:2605. 11936v2 Announce Type: replace Abstract: Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated.

By Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim

llms rag benchmarks

arXiv Machine Learning

Jun 16

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning

arXiv:2505. 09655v5 Announce Type: replace-cross Abstract: Post-training LLMs with Reinforcement Learning, specifically Group Relative Policy Optimization (GRPO), has emerged as a paradigm for enhancing mathematical reasoning.

By Xiwen Chen, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hao Wang, Haiyu Wu, Huayu Li, Aristeidis Sotiras, Yalin Wang, Abolfazl Razi

llms reinforcement-learning benchmarks safety

arXiv AI

Jun 3

Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO

arXiv:2605. 30789v2 Announce Type: replace-cross Abstract: We identify a new dimension for enhancing rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs.

By Yiming Ren, Yiran Xu, Zicheng Lin, Chufan Shi, Yukang Chen, Dingdong Wang, Tianhe Wu, Junjie Wang, Yujiu Yang, Yu Qiao, Ruihang Chu

llms benchmarks

One email a morning, machine-written

Related stories

Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging

Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training

Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolution

From Noise to Diversity: Random Embedding Injection in LLM Reasoning

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning

Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO