Extraction meets Distillation - cryptanalytic techniques to recover parameters from a Deep Neural Network
Read the original on arXiv AI →The paper investigates reconstructing fully connected neural networks by combining cryptanalytic weight-direction extraction with learning from predicted labels. It adapts ReLU extraction to Leaky ReLU, compares sign-search methods, and estimates biases and output layers, proving conditional bounds for direction recovery and sign selection. Experiments show that Leaky ReLU can recover 51–57 neurons in the deepest hidden layer of a large synthetic architecture, while ReLU recovers none, and that reconstructed models on CIFAR‑10 achieve high prediction agreement and outperform a distillation baseline on several functional metrics.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.