Hugging Face Trending Papers June 3, 2026

Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning

Read the original on Hugging Face Trending Papers →

Post-training Small Language Models (SLMs) for reasoning typically follows an SFT-then-RL pipeline, yet existing work rarely considers what data should be learned at each stage. We argue that data strategy should be aligned with the distinct roles of SFT and RL: SFT is better suited for acquiring not-yet-mastered reasoning skills, while RL is better suited for consolidating skills that the model can already partially access.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning

Jul 17

Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation

arXiv:2607. 14895v1 Announce Type: new Abstract: Reasoning language models (RLMs) have demonstrated impressive performance in domains such as mathematics and coding.

By Yu-Du Feng, Niels M\"undler-Sasahara, Mark Vero, Martin Vechev

llms nlp reinforcement-learning fine-tuning

Hugging Face Trending Papers

Jun 22

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently

Recent advances in large language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains in reasoning performance at inference time. In this work, we theoretically analyze why reinforcement fine-tuning induces better reasoning ability than purely supervised fine-tuning (SFT) methods.

llms reinforcement-learning fine-tuning

arXiv AI

Jul 2

Predicting LLM Reasoning Performance with Small Proxy Model

arXiv:2509. 21013v4 Announce Type: replace-cross Abstract: Given the prohibitive cost of pre-training large language models, it is essential to leverage smaller proxy models to optimize datasets before scaling up.

By Woosung Koh, Juyoung Suk, Sungjun Han, Se-Young Yun, Jamin Shin

llms benchmarks safety

arXiv AI

Jun 8

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

arXiv:2605. 28742v2 Announce Type: replace Abstract: Language models can use verifiable rewards to improve at a wide variety of reasoning tasks.

By Linas Nasvytis, Simon Jerome Han, Ben Prystawski, Satchel Grant, Noah D. Goodman, Judith E. Fan

llms rag

arXiv AI

Jul 20

Understanding Reasoning from Pretraining to Post-Training

arXiv:2607. 16097v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become central to improving large language models (LLMs) on complex reasoning tasks, yet RL post-training is largely studied in isolation from the pretraining that precedes it.

By Jingyan Shen, Ang Li, Salman Rahman, Yifan Sun, Micah Goldblum, Matus Telgarsky, Pavel Izmailov

llms reinforcement-learning fine-tuning

arXiv AI

Jun 26

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research

arXiv:2606. 26671v1 Announce Type: new Abstract: Post-training alignment determines the reasoning and human preference following capabilities of large language models, yet most existing works withhold detailed data construction, filtering rules and training recipes, which hinders community reproducibility and lightweight model optimization.

By Qiaobo Hao, Yangqian Wu, Shunyi Wang, Zhongjian Zhang, Ziqun Li, Yayin He, Muqing Li, Chen Zhong

llms reinforcement-learning fine-tuning efficiency benchmarks safety

Related stories

Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation

Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently

Predicting LLM Reasoning Performance with Small Proxy Model

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

Understanding Reasoning from Pretraining to Post-Training

NebulaExp-8B: An Empirical Post-Training Pipeline via Full-Scale Ablation Research