The paper investigates how exporting ternary language models (BitNet, Falcon‑E, BitCPM) through a bf16 cast step can introduce significant discrepancies between the fine‑tuned latent weights and the deployed ternary codes. In three lab pipelines, the authors find that fp32 quantization of shipped latents disagrees with the deployed codes on up to 1.77% of codes, and that the export step can drastically reduce strict accuracy on GSM8K (e.g., from 58.79% to 0.78% for Falcon‑E‑1B‑Base). They propose two compatibility remedies—directly writing the training quantizer’s codes or adjusting bf16 inputs—to meet a 4‑point strict‑accuracy non‑inferiority criterion across all models.
By Avichal Sahai (Ofbusiness), Nishant Raj (Ofbusiness), Animesh Srivastava (Ofbusiness)
arXiv:2607. 26515v1 Announce Type: new Abstract: We present, to our knowledge, the first end-to-end FP4 RL post-training, in which both the rollout and training policies, including their forward and backward passes, operate at 4-bit precision.
By Hei Yi Mak, Shadan Golestan, Hoang Le, Mehran Taghian Jazi, Yunke Peng, Yaoyuan Wang, Yao Wang, Junsong Wang, Tianchi Hu, Fengchen He, Guipeng Hu, Tanzila Rahman, Anandharaju Durai Raju
The paper presents an end‑to‑end MXFP8 reinforcement learning implementation on Blackwell GPUs, evaluating its benefits and limitations with Qwen3-8B and Qwen3-30B-A3B models. It finds that matching FP8 precision between sampler and learner improves consistency, that speed gains appear only for large micro‑batches, and that several weight‑synchronization correctness conditions can fail silently. The study also introduces seven explicit checks to detect such failures during training.
By Wengee Zhang
The paper investigates how aligning rounding locations between inference and training can stabilize reinforcement learning for large language models using BF16 precision. By retaining selected intermediates in FP32 until their BF16 outputs in three key regions—RoPE, SwiGLU, and residual RMSNorm—the authors prevent training collapse across four Qwen-family models. The approach keeps the inference engine, matrix‑multiplication and attention kernels, and learning objective unchanged.
By Yijing Ke, Tianle Cai, Ruosong Wang
ScaleRLT introduces a hybrid learner log‑probability method that reconciles differences between dense and deferred rollout computations in looped language models like Ouro. By using a stop‑gradient replay of the rollout schedule, the approach achieves bit‑for‑bit reproducibility across FP32 and bf16 real‑weight cells while maintaining the same gradient accuracy as dense implementations. The technique boosts rollout throughput by up to 1.56× on a single GPU and significantly speeds up multi‑node GRPO updates, leading to substantial accuracy gains on GSM8K in short training runs.
By Dezhen Lu, Zixiang Ni, Runguo Li
Low-precision arithmetic can make reinforcement learning (RL) for large language models cheaper, but RL requires the sampler and the learner to agree. Several frameworks now run FP8 on both sides. We...