Weak-to-Strong Generalization via Bregman Bias-Variance Decomposition

The paper studies weak-to-strong generalization (W2SG), where a student model trained on a weaker teacher’s labels surpasses the teacher on the target task. Using a Bregman divergence bias‑variance decomposition, it shows that the student‑teacher risk gap depends on their expected misfit, without requiring convexity of the student hypothesis class. For squared loss, a sufficient condition is that the student converges to the teacher’s posterior mean, achievable by enlarging the student; for cross‑entropy loss, reducing the student’s predictive entropy and using reverse cross‑entropy can promote W2SG, which is empirically validated.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• safety