arXiv Machine Learning August 27, 2026 By Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan T. Ash, Akshay Krishnamurthy

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

Read the original on arXiv Machine Learning →

TailSFT is a simple modification to supervised fine‑tuning that filters out already well‑modeled sequences, concentrating learning on the tail of the data distribution. On the OLMo‑3 7B model, this approach improves pass@16 performance on math and coding tasks by up to 17% absolute and yields up to 4% absolute gains in subsequent GRPO reinforcement‑learning runs, with only minimal computational overhead. The authors also provide a lightweight diagnostic to identify settings where TailSFT is most beneficial and argue for a stage‑aware development strategy that evaluates intermediate checkpoints by their support for later training.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 4

RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training

arXiv:2606. 04272v1 Announce Type: new Abstract: The standard LLM training pipeline applies reinforcement learning (RL) only after pre-training and supervised fine-tuning (SFT).

By Rachit Bansal, Clara Mohri, Tian Qin, David Alvarez-Melis, Sham Kakade

llms reinforcement-learning fine-tuning

arXiv AI

Jun 18

RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents

arXiv:2606. 19047v1 Announce Type: new Abstract: Multi-turn tool-use RL is bottlenecked by the rapid depletion of informative samples in static datasets.

By Ruishan Fang, Siyuan Lu, Chenyi Zhuang, Tao Lin

agents reinforcement-learning

arXiv Machine Learning

Jul 27

Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning

arXiv:2607. 21971v1 Announce Type: new Abstract: Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.

By Shujin Wu, Cheng Qian, Xiusi Chen, Heng Ji

llms reinforcement-learning benchmarks

arXiv AI

Jul 17

RAD: Retrieval High-quality Demonstrations to Enhance Decision-making

arXiv:2507. 15356v2 Announce Type: replace Abstract: Offline reinforcement learning (RL) learns policies from fixed datasets, thereby avoiding costly or unsafe environment interactions.

By Lu Guo, Yixiang Shan, Zhengbang Zhu, Qifan Liang, Lichang Song, Ting Long, Weinan Zhang, Yi Chang

agents reinforcement-learning benchmarks

arXiv AI

1d ago

Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training

arXiv:2609.37119v1 Announce Type: cross Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...

By Hongyang Li, Xiao Li, Caesar Wu, Said Mammar, Gr\'egoire Danoy, Pascal Bouvry

llms reinforcement-learning safety

arXiv Machine Learning

Jul 21

DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training

arXiv:2604. 26256v2 Announce Type: replace Abstract: Reinforcement learning (RL) has become a critical paradigm for LLM post-training, yet the rollout phase -- accounting for 50--80% of total step time -- is bottlenecked by skewed generation: long-tailed trajectories indispensable for model performance block the entire training pipeline.

By Tianhao Hu, Xiangcheng Liu, Yuchun Miao, Youshao Xiao, Hongyu Zang, Yang Zheng, Xuan Huang, Jinrui Ding, Yufei Zhang, Yu Yang, Yi-Kai Zhang, Yueqing Sun, Chengcheng Han, Xiandi Ma, Wei Wang, Qi Gu, Yerui Sun, Yuchen Xie, Xunliang Cai

llms agents reinforcement-learning efficiency benchmarks

One email a morning, machine-written

Related stories

RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training

RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents

Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning

RAD: Retrieval High-quality Demonstrations to Enhance Decision-making

Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training

DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training