Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution
Read the original on arXiv AI →The paper presents a post‑training method for dynamically routed mixture‑of‑experts (MoE) models that integrates independently optimized specialists efficiently. It introduces calibrated multi‑teacher on‑policy distillation (C‑MOPD) with teacher‑wise loss normalization and clipping, combined with a post‑capacity bounded‑width dispatch, final‑route replay, and asynchronous rollout–training overlap. On the Turing‑20B‑A2B model, the approach achieves a high integration score (0.9988), retains most specialist gains, outperforms shared SFT and Qwen3‑8B on reasoning‑enabled benchmarks, and reduces consolidation time by 2.86×.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.