arXiv Machine Learning
Patch the Distribution Mismatch: RL Rewriting Agent for Stable Off-Policy SFT
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jul 21
From Evidence to Trajectory: Abductive Reasoning Path Synthesis for Retrieval-Augmented Generation Agents Development
arXiv:2509. 23071v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) agent development is hindered by the lack of executable ground-truth agent-environment interaction trajectories.
arXiv Machine Learning
Sep 4
DE-Venus: A Data-Efficient RLVR Framework for Large Language Models
DE‑Venus is a unified, data‑efficient framework for reinforcement learning with verifiable rewards (RLVR) tailored to large language models. It structures the RLVR lifecycle into three modules—Active Data Selection, Weak Supervision Construction, and Training‑Time Supervision Refinement—allowing method‑specific decisions to be expressed as dataset transitions or online transformations while maintaining distributed execution contracts. Experiments on public benchmarks and three business scenarios show that DE‑Venus can preserve or improve model quality using only 10% of labels or 13% of relevant data, and can cut convergence steps by 63%–75% in selected business configurations.
arXiv Computation and Language
Aug 28
Boosting LLM Exploration via Weak-Model Guidance in RLVR
The paper introduces a method to enhance large language model (LLM) exploration in Reinforcement Learning with Verifiable Rewards (RLVR) by guiding the target model with partial reasoning trajectories from smaller, weaker language models. This weak-model guidance disrupts over‑confidence, preserves generative diversity, and mitigates entropy collapse without extra fine‑tuning or complex reward designs. Experiments on mathematical benchmarks show consistent improvements over vanilla RLVR, especially as the number of allowed attempts ($k$) increases, indicating broader reasoning coverage.
arXiv AI
Aug 25
SSE-Bio: A Structured Self-Evolving Agent with Agentic Retrieval Policy for Multi-Hop Biomedical Reasoning
arXiv:2608.22132v1 Announce Type: cross Abstract: Biomedical multi-hop question answering (QA) requires models to connect evidence across intermediate entities such as diseases, drugs, proteins, and...
arXiv Computation and Language
2d ago
Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement
arXiv:2609.16800v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments rema...
arXiv Machine Learning
Aug 27
From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection
The paper introduces Golden-GRPO Injection (GRIN), a three-stage self‑learning framework that uses a mixed‑policy reinforcement learning algorithm to inject knowledge into large language models. GRIN injects a golden answer to provide learning signals even when on‑policy rollouts fail on novel facts, and is evaluated on two new document‑level benchmarks—Blank and Counter—that test novel acquisition and counterfactual overwrite. Experiments show that mixed‑policy RL enables knowledge absorption beyond what supervised fine‑tuning can achieve, with GRIN outperforming SFT and other RL baselines on harder question types while matching them on basic fact recall.