The paper introduces Policy Alignment Membership Auditing (PAMA), a framework for detecting whether a prompt was used during on‑policy distillation (OPD). PAMA leverages the direction of teacher‑guided policy updates, quantified by Teacher Alignment Gain (TAG), alongside student drift and uncertainty signals to assess membership. Experiments on six datasets, including the MATH benchmark, show that PAMA improves AUC by 14.6–20.6% over existing methods.
By Yilong Yang, Wenzhuo Shang, Yule Liu, Jiale Teng, Zhuo Ma
arXiv:2607. 09692v1 Announce Type: new Abstract: Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations.
By Rajat Rawat, Sizhe Chen, Akshay Anand, Michael Duan, Bob Rotsted, Sewon Min
The paper investigates on‑policy distillation (OPD) for large language models, showing that OPD learns not only which tokens to consider but also where to keep reasoning paths open. It introduces the entropy shape—a sequence of next‑token entropies—to capture this "where to branch" aspect and proposes the Entropy Shape Correlation (ESC) metric to measure student‑teacher agreement. By tracking ESC during training and adjusting a teacher’s entropy while preserving token preferences, the authors demonstrate that controlling where to branch causally drives OPD gains, and they establish a teacher‑selection protocol where pre‑OPD ESC predicts the best teacher more accurately than existing methods.
By Shiping Yang, Chuanqi Tang, Hossein Hajimirsadeghi, Layla El Asri, Mohammad Amin Shabani, Angel X. Chang
arXiv:2610.08448v1 Announce Type: cross
Abstract: On-Policy Distillation (OPD) trains a student on its own generations using teacher feedback. With different tokenizers, comparing teacher and student...
By Bingxi Hou, Guochao Jiang, Guofeng Quan, Weiqing Li, Wenfeng Feng, Guohua Liu, Yuewei Zhang
The paper introduces Teacher-Gated On-Policy Distillation (TGOPD), a method that verifies teacher reliability at the prompt level before applying dense supervision in on-policy distillation. TGOPD uses verifier-scored teacher probes to decide whether to route a prompt to dense OPD or to a verifier-grounded alternative. Experiments on 4B and 35B models across mathematics, code, and instruction tasks show TGOPD outperforms vanilla OPD and improves teacher GPU utilization from 9.8% to 78.9% in a 4B single-domain run.
By Zhiwei Zhang, Zechen Sun, Fei Zhao, Kang Peng, Bin Liang, Huayu Deng, Yao Hu, Kam-Fai Wong, Mu Chuan
On-policy distillation (OPD) aligns a student model with a teacher's logit distribution on student-generated trajectories. This approach has achieved strong empirical gains and can often surpass conve...