arXiv:2610.07654v1 Announce Type: cross
Abstract: On-policy distillation (OPD) has attracted growing attention as an effective way to transfer capabilities from teacher models to student models. Rece...
By Jian Luo, Kehan Qi, Qingqiao Hu, Meilong Xu, Jiacheng Qiu, Weimin Lyu, Jiawei Zhou, Chao Chen
Collaboratively Guided Adversarial Robust Distillation (CGARD) is a method that transfers robustness from high‑capacity teachers to compact students by jointly optimizing student‑adversarial and teacher‑collaborative examples within the same perturbation neighborhood. The teacher‑collaborative example is constrained so that the teacher’s cross‑entropy loss does not exceed that of the clean input, combining collaborative guidance with adversarial supervision. Experiments on CIFAR‑10 and CIFAR‑100, including white‑box and black‑box evaluations, show consistent robustness improvements over strong adversarial distillation baselines.
By Zhi Li, Haowei Liu, Hongchen Yang, Xiaoxuan Wang, Song Gao, Shaowen Yao, Wei Zhou
arXiv:2607. 27737v1 Announce Type: new Abstract: Deep neural networks (DNNs) have achieved remarkable success in classical machine learning problems.
By Vincent Ryusuke Takahashi, Yoshinari Takeishi, Jun'ichi Takeuchi, Kave Salamatian
The paper introduces IDeaL, a data‑free multi‑teacher distillation technique that generates teacher‑specific, improved samples using decorrelation losses at patch and image levels. By tailoring noise to each teacher, IDeaL produces strong student models that capture complementary teacher information and achieve results close to those distilled from real images. Experiments demonstrate that with only 1,000 images, students trained on IDeaL samples match or exceed the performance of students distilled from a 1,000‑image subset of ImageNet.
By Feyza Yavuz, Mert B\"ulent Sar{\i}y{\i}ld{\i}z, Diane Larlus
arXiv:2610.19911v1 Announce Type: new
Abstract: Adversarial robustness distillation typically treats a robust teacher as a fixed knowledge source and transfers robustness to a compact student in a on...
By Zhi Li, Haowei Liu, Hongchen Yang, Xiaoxuan Wang, Song Gao, Zhenli He, Wei Zhou, Shaowen Yao
The paper introduces Checkerboard, a clean‑label backdoor attack that uses a closed‑form, data‑independent trigger design based on an input‑space Fisher‑separability objective and a ridge four‑neighbor local‑smoothness prior. This approach yields a pixel‑wise checkerboard trigger without requiring data access, surrogate model training, or iterative optimization, and it outperforms existing norm‑bounded clean‑label attacks across four benchmark datasets. On CIFAR‑10, poisoning 20 samples with a 10/255 perturbation achieves a 95.72% attack success rate, while on IN‑100 a 0.4% global poisoning rate yields over 83% ASR without harming clean accuracy, and the attack remains robust against state‑of‑the‑art defenses.
By Yi Yang, Jinyang Huang, Binbin Liu, Feng-Qi Cui, Xiaokang Zhou, Haiming Jin, Zhi Liu, Jie Zhang, Meng Li