The paper introduces Skill-Decoupled On-Policy Expert Distillation (SCOPED), a method that trains separate experts for each capability and extracts explicit skills via reward-guided reflection. During on‑policy distillation, these skills supervise the experts but are not exposed to the unified student, encouraging the student to learn procedural knowledge internally. Experiments on eight multimodal benchmarks show SCOPED consistently outperforms both the base model and mixed‑capability reinforcement learning, highlighting the benefits of skill decoupling for integrating independently acquired capabilities.
arXiv:2606. 05718v1 Announce Type: cross Abstract: On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher.
arXiv:2607. 13854v2 Announce Type: replace Abstract: Multimodal agents that think with images iteratively manipulate visual evidence and invoke tools across many steps.
arXiv:2606. 07000v1 Announce Type: new Abstract: Recent post-training methods, particularly Reinforcement Learning with Verifiable Rewards (RLVR), have significantly enhanced the reasoning ability of Large Vision-Language Models (LVLMs).
Video-OPSD introduces a post‑training framework for Video Large Language Models that leverages privileged visual evidence to enhance on‑policy self‑distillation. The method constructs a self‑teacher conditioned only on annotated evidence frames, while the student processes the full video, allowing the teacher to provide more focused supervision. Additionally, an evidence‑guided token optimization scheme weights distillation based on each token’s reliance on privileged evidence, improving perceptually grounded reasoning. Experiments demonstrate consistent gains over standard OPSD and comparable performance to GRPO with less training time.
LEGO-OPD introduces a factorized teacher composition for multimodal on‑policy distillation, combining a Language Expert and a Grounding Expert into a single teacher distribution. By treating the language expert as a prior over tokens and the grounding expert as a visual likelihood that updates this prior, the method decouples language reasoning from visual grounding. Adaptive calibration further adjusts the influence of visual evidence at each decoding prefix, preventing over‑ or under‑supervision. Experiments with Qwen3 models demonstrate that LEGO‑OPD outperforms both single‑ and multi‑teacher baselines on multimodal and text‑only reasoning tasks, improving visual perception while preserving language reasoning.
The paper investigates three fusion paradigms—Merge, Mix RL, and multi‑teacher on‑policy distillation (MOPD)—for consolidating reinforcement learning with verifiable rewards (RLVR) across multiple domains. Experiments across model scales and a multi‑domain benchmark show that while overall performance differences are small, significant gaps can appear on specific tasks, and each method exhibits distinct training dynamics and constraints. Practical guidelines are offered: Merge for cheap fusion when experts exist, Mix RL for unified training with adjustable domain mixtures, and MOPD when preserving domain‑specific gains is paramount.
Multimodal on-policy distillation (OPD) aims to improve visual grounding while preserving the strong reasoning capabilities of language models. Recent multi-teacher approaches combine LLM and VLM teac...
arXiv:2606. 03444v1 Announce Type: cross Abstract: Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation.
arXiv:2609.10321v1 Announce Type: new Abstract: Knowledge distillation offers an efficient route to transfer a task-adapted vision-language teacher to a compact student. The training target in curren...
arXiv:2610.02324v1 Announce Type: cross Abstract: Foundation multimodal large language models are designed to support a broad spectrum of capabilities across diverse domains. Multi-teacher on-policy...
arXiv:2609.34658v2 Announce Type: replace Abstract: Reward-specialized post-training produces strong experts for flow-based generative models, while multi-teacher on-policy distillation (OPD) consoli...
MMSearch-OPD is a new approach for long‑horizon multimodal search agents that combines a controllable data refinement pipeline with joint reinforcement learning and on‑policy distillation. The refinement process expands reasoning chains from existing simple multimodal questions, allowing adjustable search depth and cross‑modal evidence requirements while reusing image–question pairs. Training uses answer rewards, behavior penalties, and a decaying distillation coefficient to guide the agent, and experiments demonstrate that both data refinement and joint training after supervised fine‑tuning improve long‑horizon multimodal search performance.