The paper introduces Policy Alignment Membership Auditing (PAMA), a framework designed for on‑policy distillation (OPD) to detect whether a prompt was used during training. PAMA leverages the direction of teacher‑guided policy updates, quantified by Teacher Alignment Gain (TAG), along with student drift and uncertainty signals, to assess membership. Experiments on six datasets and three model families show that PAMA outperforms existing methods, achieving AUC scores between 0.791 and 0.941 on the MATH benchmark, a 14.6–20.6% improvement over state‑of‑the‑art baselines.
arXiv:2607. 09692v1 Announce Type: new Abstract: Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations.
By Rajat Rawat, Sizhe Chen, Akshay Anand, Michael Duan, Bob Rotsted, Sewon Min
The paper investigates on‑policy distillation (OPD) for large language models, showing that OPD learns not only which tokens to consider but also where to keep reasoning paths open. It introduces the entropy shape—a sequence of next‑token entropies—to capture this "where to branch" aspect and proposes the Entropy Shape Correlation (ESC) metric to measure student‑teacher agreement. By tracking ESC during training and adjusting a teacher’s entropy while preserving token preferences, the authors demonstrate that controlling where to branch causally drives OPD gains, and they establish a teacher‑selection protocol where pre‑OPD ESC predicts the best teacher more accurately than existing methods.
By Shiping Yang, Chuanqi Tang, Hossein Hajimirsadeghi, Layla El Asri, Mohammad Amin Shabani, Angel X. Chang
arXiv:2609.36734v1 Announce Type: new
Abstract: Knowledge Distillation (KD) trains a smaller-capacity student model to imitate a larger-capacity teacher model by matching output distributions, implic...
By Ayan Sengupta, Vaibhav Seth, Tanmoy Chakraborty
The paper introduces Teacher-Gated On-Policy Distillation (TGOPD), a method that verifies teacher reliability at the prompt level before applying dense supervision in on-policy distillation. TGOPD uses verifier-scored teacher probes to decide whether to route a prompt to dense OPD or to a verifier-grounded alternative. Experiments on 4B and 35B models across mathematics, code, and instruction tasks show TGOPD outperforms vanilla OPD and improves teacher GPU utilization from 9.8% to 78.9% in a 4B single-domain run.
By Zhiwei Zhang, Zechen Sun, Fei Zhao, Kang Peng, Bin Liang, Huayu Deng, Yao Hu, Kam-Fai Wong, Mu Chuan
The paper investigates on‑policy distillation (OPD), showing that teacher supervision during OPD contains significant noise that grows with teacher size, yet the student policy remains largely unaffected by this noise. It finds that OPD’s gains stem mainly from suppressing low‑log‑probability tokens, a process that can be replicated without a teacher. Building on this insight, the authors propose On‑Policy Self‑Adaptation (OPSA), a supervision‑free method that uses entropy‑adaptive negative advantages to improve performance on several benchmarks, outperforming both the base model and OPD.
By Yi Ding, Ruqi Zhang