Why Ghost Outputs Teach: A Kernel-Based Understanding of Subliminal Learning
Read the original on arXiv Machine Learning →The paper investigates subliminal learning (SL), where a student model learns downstream tasks by matching auxiliary outputs from a teacher without seeing task labels or original data. By deriving a chained cross‑task kernel, the authors link ghost‑output supervision to changes in task predictions via shared backbone representations. Their framework explains three empirical puzzles: a PSD transfer operator under shared initialization, a rank bottleneck set by ghost‑output dimensionality, and the superiority of high‑entropy synthetic inputs as broadband probes for maximizing kernel overlap.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.