arXiv:2606. 25964v1 Announce Type: cross Abstract: Small ($\sim$2B) GUI-grounding agents are attractive for on-device deployment, accessibility tooling, and low-cost iteration, but at this scale they face two open recipe questions: how to obtain bounding-box training data without expensive human annotation, and how to combine supervised fine-tuning with reinforcement learning.
On-policy (Self-)Distillation (OPD / OPSD) has shown strong potential for post-training large language models (LLMs). However, existing methods still rely heavily on external supervision, including ground-truth signals, environmental feedback, or guidance from larger models, and therefore fall short of genuine "self"-distillation.
arXiv:2608. 06296v1 Announce Type: new Abstract: On-policy (Self-)Distillation (OPD / OPSD) has shown strong potential for post-training large language models (LLMs).
arXiv:2608.29030v1 Announce Type: new Abstract: In-context watermarking (ICW) prepends an instruction to a query asking the model to embed a statistically detectable signal in its response. It thus e...
arXiv:2609.23449v1 Announce Type: cross Abstract: Memory systems are becoming a core component of LLM agents, but constructing and maintaining memory remains expensive because it relies on repeated c...
arXiv:2607. 09709v1 Announce Type: new Abstract: Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact.
The paper introduces Privileged Self-Practice (PSP), a method that retains privileged information (PI) in the prompt rather than the loss during on‑policy self‑distillation for multi‑turn agents. PSP injects short per‑task instructions from an analyzer model when rollouts fail, sampling again with the instruction in context and training with the unchanged GRPO objective. Experiments on AppWorld and SWE‑bench Verified show PSP consistently outperforms plain GRPO, boosting task‑goal completion by up to 65% and resolved rate by up to 61% across three student models.
arXiv:2609.37915v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies...
arXiv:2607. 18293v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) teaches large language models new skills through a teacher that shares the student's backbone and supervises its own rollouts.
arXiv:2607. 02502v1 Announce Type: cross Abstract: On-policy self-distillation (OPSD) has emerged as a practical method for training large language models (LLMs) to reason, where a single model acts as both the teacher and the student with different levels of information access.
The paper introduces Zone of Proximal Policy Optimization (ZPPO), a method that keeps a teacher model inside prompts rather than in the policy gradient to improve knowledge distillation for small students. ZPPO creates two types of reformulated prompts—Binary Candidate-included Questions (BCQ) and Negative Candidate-included Questions (NCQ)—to expose students to correct and incorrect responses, and uses a replay buffer to focus training on hard questions until the student’s accuracy improves. Experiments on the Qwen3.5 family with a 27B teacher across 31 benchmarks show that ZPPO outperforms both off‑policy and on‑policy distillation methods, especially at the smallest student scales.
The paper examines on‑policy self‑distillation (OPSD) for multi‑turn agents, showing that using privileged information (PI) in the loss can make agents appear confident yet underperform plain RL, sometimes worse than the untrained base model. To address this, the authors propose Privileged Self‑Practice (PSP), which keeps PI in the prompt and uses it only during sampling, not in the loss. PSP consistently outperforms plain GRPO across AppWorld and SWE‑bench Verified, improving task‑goal completion by up to 65% and resolved rate by up to 61%.
arXiv:2607. 05339v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) is effective when the current policy already samples useful reasoning trajectories, but it stalls on hard prompts whose correct solution modes lie outside the student's on-policy support.