← Back to all news
arXiv Computer Vision By Zhi Li, Haowei Liu, Hongchen Yang, Xiaoxuan Wang, Song Gao, Shaowen Yao, Wei Zhou

Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples

Read the original on arXiv Computer Vision →

Collaboratively Guided Adversarial Robust Distillation (CGARD) is a method that transfers robustness from high‑capacity teachers to compact students by jointly optimizing student‑adversarial and teacher‑collaborative examples within the same perturbation neighborhood. The teacher‑collaborative example is constrained so that the teacher’s cross‑entropy loss does not exceed that of the clean input, combining collaborative guidance with adversarial supervision. Experiments on CIFAR‑10 and CIFAR‑100, including white‑box and black‑box evaluations, show consistent robustness improvements over strong adversarial distillation baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Related stories

arXiv AI
Sep 10

How to Backdoor Image Knowledge Distillation

The paper demonstrates that image knowledge distillation can be backdoored even when the teacher model is clean, by poisoning the distillation dataset with triggered and manipulated images that the teacher already classifies as a target label. The attack, effective at poisoning rates as low as 10%, uses targeted adversarial perturbations and GAN-based class transitions to embed a backdoor into the student model while preserving its performance on clean data. The study highlights that the security of knowledge distillation depends not only on the teacher but also on the integrity of the distillation data.

By Qian Ma, Chen Wu, Prasenjit Mitra, Sencun Zhu
diffusionroboticsefficiencysafety
More like this →
arXiv Computer Vision
Aug 26

IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leaves

The paper introduces IDeaL, a data‑free multi‑teacher distillation technique that generates teacher‑specific, improved samples using decorrelation losses at patch and image levels. By tailoring noise to each teacher, IDeaL produces strong student models that capture complementary teacher information and achieve results close to those distilled from real images. Experiments demonstrate that with only 1,000 images, students trained on IDeaL samples match or exceed the performance of students distilled from a 1,000‑image subset of ImageNet.

By Feyza Yavuz, Mert B\"ulent Sar{\i}y{\i}ld{\i}z, Diane Larlus
efficiency
More like this →
arXiv AI
1d ago

Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation

The paper introduces Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation (Road MOPD), a method that adds non-routed teachers as auxiliary cross-domain critics to address gradient conflicts in multi-teacher distillation. It employs token-level credit assignment based on teacher agreement and preventive gradient projection to coordinate updates and reduce conflicts. Experiments show that Road MOPD improves performance, especially in high-conflict scenarios, and mitigates degradation in disadvantaged domains.

By Shi-Yu Tian, Yong Liu, Ximan Liu, Tianrui Zhu, Sule Bai, Xinwei Yang, Ruiqi Liu, Kun-Yang Yu, Ming Yang, Zhi Zhou, Yu-Feng Li, Yipeng Sun, Gu Simiu
efficiency
More like this →