Label Smoothing in Disguise? Gated Knowledge Distillation for Low-Resource Bangla Generation

The paper investigates gated knowledge distillation (KD) for low‑resource Bangla generation, testing two gating strategies—EWAD and CHAD—to selectively trust teacher signals. Across six Bangla tasks and multiple student models, neither gate improves over plain KD, and KD generally underperforms cross‑entropy fine‑tuning on gold outputs. The primary advantage of KD is regularisation, with teacher‑free label smoothing narrowing the performance gap, though gating only modestly alters this effect.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

• fine-tuning
• efficiency
• safety