arXiv:2609.40055v1 Announce Type: cross
Abstract: On-policy distillation (OPD) provides dense supervision directly on student-generated trajectories, making it an effective post-training strategy for...
By Jiacheng Qiu, Yunsoo Kim, Ruichen Xu, Jian Luo, Petar M. Djuri\'c, Sima Mofakham
arXiv:2608. 19355v1 Announce Type: cross Abstract: Educational visual question answering, or VQA, requires models to solve curriculum-oriented multiple-choice questions using both language and visual evidence.
By Xinjin Li, Yudi Xia, Xi Zhao, Yiliu Xu, Yining Liu, Cheng Lu, Yujian Long, Yu Ma, Jinghan Cao, Liang Fan, Yeyun Xu
The paper introduces Harness-Aware Distillation (HAD), a method for training smaller language model agents that preserves the surrounding harness—software managing context, tools, and feedback—while focusing distillation on the teacher’s contributions beyond the harness. HAD combines an action preference that contrasts teacher actions with and without harness information, and a validity check that filters out contradictory preference pairs. Experiments on long-horizon agent benchmarks show that HAD outperforms standard on‑policy distillation, reducing unproductive loops and improving error recovery without requiring task rewards or future information.
By Moonseok Choi, Taehong Moon, Giung Nam, Juho Lee
arXiv:2608. 08907v1 Announce Type: cross Abstract: Thinking with images allows a multimodal model to compensate for limited perception by invoking visual tools through code.
By Delin Mao, Chenghao Sun, Jingwei Song, Chishui Chen, Linfeng Zhang
CA-OPD is a confidence‑aware on‑policy distillation framework that improves structured visual prediction by using teacher confidence to selectively correct unreliable student transitions and gradually transfer rollout control to the student. The method aligns supervision with intervention decisions, providing direct cross‑entropy loss for corrected tokens and full predictive distribution for retained tokens. In a multi‑teacher setting for GUI grounding and OCR, CA‑OPD significantly outperforms the Qwen3.5‑0.8B baseline, achieving large gains on benchmarks such as ScreenSpot‑Pro and OCRBench‑v2 English.
By Menghao Li, Linjie Mu, Yin Wang, Haotian Hu, Yannian Gu, Lujiayi Xue, Fanyi Wang
The paper investigates how on‑policy distillation (OPD) inherits a teacher’s behavior in vision‑language models, comparing it to filtered supervised fine‑tuning (SFT) across different data budgets and tasks. Experiments on ChartQA and Geometry3K show that OPD performs well only when the teacher remains reliable on student‑generated trajectories; otherwise it underperforms SFT and can erase previously learned completion behavior. Filtering teacher demonstrations for completeness largely restores SFT’s advantages, indicating that OPD can transfer both knowledge and persistent failure modes from the teacher.
By Frank F. Yang, Kefei Yan, Yiren Wang, Yiyang Wang