The paper demonstrates that image knowledge distillation can be backdoored even when the teacher model is clean, by poisoning the distillation dataset with triggered and manipulated images that the teacher already classifies as a target label. The attack, effective at poisoning rates as low as 10%, uses targeted adversarial perturbations and GAN-based class transitions to embed a backdoor into the student model while preserving its performance on clean data. The study highlights that the security of knowledge distillation depends not only on the teacher but also on the integrity of the distillation data.
arXiv:2610.07654v1 Announce Type: cross Abstract: On-policy distillation (OPD) has attracted growing attention as an effective way to transfer capabilities from teacher models to student models. Rece...
Collaboratively Guided Adversarial Robust Distillation (CGARD) is a method that transfers robustness from high‑capacity teachers to compact students by jointly optimizing student‑adversarial and teacher‑collaborative examples within the same perturbation neighborhood. The teacher‑collaborative example is constrained so that the teacher’s cross‑entropy loss does not exceed that of the clean input, combining collaborative guidance with adversarial supervision. Experiments on CIFAR‑10 and CIFAR‑100, including white‑box and black‑box evaluations, show consistent robustness improvements over strong adversarial distillation baselines.
arXiv:2607. 27737v1 Announce Type: new Abstract: Deep neural networks (DNNs) have achieved remarkable success in classical machine learning problems.
The paper introduces IDeaL, a data‑free multi‑teacher distillation technique that generates teacher‑specific, improved samples using decorrelation losses at patch and image levels. By tailoring noise to each teacher, IDeaL produces strong student models that capture complementary teacher information and achieve results close to those distilled from real images. Experiments demonstrate that with only 1,000 images, students trained on IDeaL samples match or exceed the performance of students distilled from a 1,000‑image subset of ImageNet.
arXiv:2610.19911v1 Announce Type: new Abstract: Adversarial robustness distillation typically treats a robust teacher as a fixed knowledge source and transfers robustness to a compact student in a on...
The paper introduces Checkerboard, a clean‑label backdoor attack that uses a closed‑form, data‑independent trigger design based on an input‑space Fisher‑separability objective and a ridge four‑neighbor local‑smoothness prior. This approach yields a pixel‑wise checkerboard trigger without requiring data access, surrogate model training, or iterative optimization, and it outperforms existing norm‑bounded clean‑label attacks across four benchmark datasets. On CIFAR‑10, poisoning 20 samples with a 10/255 perturbation achieves a 95.72% attack success rate, while on IN‑100 a 0.4% global poisoning rate yields over 83% ASR without harming clean accuracy, and the attack remains robust against state‑of‑the‑art defenses.
arXiv:2606. 26285v1 Announce Type: cross Abstract: Noise-based backdoor attacks on diffusion models typically rely on input-time trigger injection, untargeted activation, and out-of-distribution target generation.
arXiv:2606. 25059v2 Announce Type: replace-cross Abstract: Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs.
The paper introduces DAR (Discover-and-Relabel), a support‑persistent backdoor technique that remains effective even after models undergo clean‑data knowledge distillation. DAR discovers naturally occurring patterns in low‑dimensional feature subspaces via subspace clustering and then poisons labels of samples that satisfy these patterns. At inference, the backdoor is triggered by locally modifying the target image’s subspace, achieving attack accuracy comparable to state‑of‑the‑art methods while maintaining over 96% success after clean distillation.
arXiv:2507. 05113v3 Announce Type: replace-cross Abstract: Deep Neural Networks (DNNs) are susceptible to backdoor attacks, where adversaries poison training data to implant backdoor into the victim model.
arXiv:2606. 25858v1 Announce Type: cross Abstract: Federated learning is vulnerable to backdoor attacks in which malicious clients inject poisoned updates while preserving benign-task performance.
arXiv:2511. 07210v3 Announce Type: replace-cross Abstract: Clean-image backdoor attacks, which use only label manipulation in training datasets to compromise deep neural networks, pose a significant threat to security-critical applications.