The paper introduces On-Policy Self-Training, a method that allows large language models to improve themselves without external teachers or curated datasets. By generating its own question-answer pairs and privileged reasoning demonstrations guided only by coarse topic descriptions, the model creates a synthetic curriculum that enhances performance across multiple domains. Experiments show consistent self-improvement on both text and multimodal benchmarks, including Visual Question Answering, while preserving abilities on non-self-trained domains.
arXiv:2610.02117v1 Announce Type: cross Abstract: On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a froze...
Video-OPSD introduces a post‑training framework for Video Large Language Models that leverages privileged visual evidence to enhance on‑policy self‑distillation. The method constructs a self‑teacher conditioned only on annotated evidence frames, while the student processes the full video, allowing the teacher to provide more focused supervision. Additionally, an evidence‑guided token optimization scheme weights distillation based on each token’s reliance on privileged evidence, improving perceptually grounded reasoning. Experiments demonstrate consistent gains over standard OPSD and comparable performance to GRPO with less training time.
Vision-language models (VLMs) have made substantial progress in long-video understanding, with standard backbone models typically answering questions from frames sampled across the full video. However...
The paper introduces Clue-OPSD, a clue‑privileged on‑policy self‑distillation framework that improves long‑video understanding by focusing on short, question‑relevant clue intervals rather than the entire video. Experiments on multiple benchmarks and Qwen3.5 model scales show that this approach consistently outperforms standard backbone models and competes strongly with supervised post‑training baselines, all while requiring fewer input frames and no additional inference modules.
arXiv:2608. 05131v1 Announce Type: cross Abstract: On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs).
arXiv:2610.09665v1 Announce Type: new Abstract: On-policy post-training can improve large language models by learning from their own trajectories, but requires costly rollout generation. We ask wheth...
Negative Self-Distillation (NSD) is a new framework for improving large language models by encouraging them to diverge from their own flawed reasoning rather than imitate privileged solutions. Unlike On-Policy Self-Distillation, which can suppress uncertainty and exploratory behavior, NSD generates a question‑specific negative condition (e.g., a careless reasoner) and uses a dynamic gating mechanism to target only reasoning‑critical tokens for penalization. This approach preserves foundational language capabilities while consistently outperforming OPSD and other label‑free self‑bootstrapping reinforcement learning baselines.
arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).
Negative Self-Distillation (NSD) is a new framework for improving large language models by encouraging them to diverge from their own flawed reasoning rather than imitate privileged solutions. Unlike On-Policy Self-Distillation (OPSD), which can suppress uncertainty and exploratory behavior, NSD generates a question‑specific negative condition (e.g., a careless reasoner) and pushes the student’s distribution away from it. A dynamic gating mechanism isolates reasoning‑critical tokens so that only behavioral flaws are penalized, preserving linguistic capabilities, and empirical results show NSD consistently outperforms OPSD and other label‑free self‑bootstrapping RL baselines.
arXiv:2606. 17803v1 Announce Type: new Abstract: Large language models achieve strong reasoning performance by scaling inference-time compute, yet remain fundamentally stateless, discarding the rich, self-produced reasoning traces generated during this process.
Recent advances in post-training Large Language Models (LLMs) increasingly rely on Reinforcement Learning with Verifiable Rewards (RLVR) or On-Policy Self-Distillation (OPSD). While OPSD provides dense, logit-level supervision, it inherently suffers from exposure bias due to the privileged information of the self-teacher.
Prompt2Skill is an unsupervised framework that constructs skills for Large Language Models directly from natural‑language task descriptions. It automatically derives task specifications, discovers or synthesizes datasets, and refines the skill through a reflective editing loop. In experiments across question answering, reading comprehension, spreadsheet manipulation, and mathematical reasoning, Prompt2Skill outperforms direct prompting, improving performance by an average of 10.8 points on both open‑source and frontier models.