arXiv Machine Learning
From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training Frameworks
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Aug 27
Demystifying Reinforcement Learning Post-Training of Language Models
The paper "Demystifying Reinforcement Learning Post-Training of Language Models" investigates how reinforcement learning (RL) post‑training enhances large language models (LLMs) for tasks such as reasoning, math, and coding. By isolating RL components in a controlled setting, the authors analyze how the base model’s prior distribution, reward granularity, prompt diversity, and model scale influence outcomes, using policy entropy to compare pre‑training, supervised fine‑tuning (SFT), and RL stages. The study clarifies the role of spurious rewards, the importance of the base model’s probability mass on desired behaviors, and how these factors interact to determine post‑training success, offering a practical primer for NLP researchers. "whyItMatters":"The work provides a clearer understanding of RL post‑training mechanics, helping researchers and practitioners effectively apply RL to improve LLM capabilities."
arXiv Computation and Language
6d ago
Data Strategies for Large Language Models: A Unified Perspective on the Training Lifecycle
The paper reviews data strategies for large language models, emphasizing that data strategy—selecting, weighting, and scheduling training data—determines model capabilities within a fixed data budget. It treats high‑quality data as having training‑dependent utility, varying with the target objective, model, and training state. The authors organize existing methods by decision granularity, model dependence, and temporal adaptivity, comparing their computational costs and empirical support, and note limited evidence for transfer across model scales and end‑to‑end cost parity.
arXiv Machine Learning
Oct 8
Which Rollout Taught It That? BehaviorTrace and the Limits of Training-Data Attribution in Online RL
The paper introduces BehaviorTrace, an evaluation framework for assessing training‑data attribution in online reinforcement‑learning fine‑tuning of language models. Using a planted behavior on Qwen2.5‑1.5B, the authors show that many attribution signals are confounded by factors such as gradient magnitude and model fluency, and that only the gradient of trigger tokens consistently aligns with the true behavior across seeds. They provide a checklist for evaluating attribution methods and test existing estimators without proposing a new one.
arXiv AI
Jun 2
Rethinking RL Evaluation: Can Benchmarks Truly Reveal Failures of RL Methods?
arXiv:2510. 10541v2 Announce Type: replace-cross Abstract: Current benchmarks are inadequate for evaluating progress in reinforcement learning (RL) for large language models (LLMs).
arXiv Machine Learning
Jul 14
Reinforcement Learning in the Real World: A Survey of Statistical Challenges and Future Directions
arXiv:2601. 15353v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has achieved remarkable success in real-world decision-making across diverse domains, including gaming, robotics, online advertising, public health, and natural language processing.
arXiv Machine Learning
Sep 1
Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment
Agnostics is a language‑agnostic post‑training pipeline that uses reinforcement learning with verifiable rewards (RLVR) to improve large language models on low‑resource programming languages. By rewriting unit‑test datasets into a language‑independent I/O format, providing a short configuration for compiling and running code, and employing a single verifier that judges code by observable behavior, Agnostics eliminates the need for language‑specific engineering. Applied to Lua, Julia, R, OCaml, and Fortran, it boosts Qwen‑3 4B to rival larger models, scales to diverse families, and achieves new state‑of‑the‑art pass@1 on MultiPL‑E and a new multi‑language LiveCodeBench.