← Back to all news
arXiv Machine Learning By Linxuan Wu, Liang Chen

Structure-Aware Numerical Stabilization for Half-Precision Proximal Policy Optimization

Read the original on arXiv Machine Learning →

The paper introduces PPO‑FP16, a half‑precision variant of proximal policy optimization that incorporates three structure‑aware modules—global scalar protection, policy‑change residual protection, and efficient persistent‑update protection—to maintain numerical stability. These modules preserve the original PPO objective and hyperparameters while enabling reliable training on commodity GPUs. Empirical results on six continuous‑control tasks show PPO‑FP16 achieves the best mean return‑AUC rank, speeds up updates by up to 2×, and reduces optimizer‑state memory by up to 75% and peak GPU memory by up to 45%.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Aug 26

WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation

arXiv:2608.24479v1 Announce Type: new Abstract: Massively parallel simulation changes the data regime in which off-policy reinforcement learning (RL) is trained, challenging stabilizers designed for...

By Zihao Wu, Hongyao Tang, Yi Ma, Huizhong Song, Pengyi Li, Yifu Yuan, Fei Ni, Jinyi Liu, Wei Wei, Jianrong Wang, Yan Zheng, Jianye Hao
reinforcement-learningroboticsefficiencybenchmarks
More like this →
arXiv Machine Learning
Oct 3

TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning

The paper introduces TACO, a new optimizer for fine‑tuning large language models that drastically reduces optimizer state memory while preserving first‑order gradients. TACO selects the sign of the largest magnitude entry in each column of weight matrices, achieving a 174× reduction in persistent optimizer memory compared to AdamW8bit and a 2.9× decrease in peak training memory on OPT‑13B. This allows full‑parameter fine‑tuning of 30–32B‑parameter models on a single 80 GB GPU across multiple model families and tasks, with comparable accuracy and runtime to existing methods.

By Jichao Jiang (University of Central Florida), Cristian McGee (University of Central Florida), El Houcine Bergou (Mohammed VI Polytechnic University), Hanqin Cai (University of Central Florida), Aritra Dutta (University of Central Florida)
llmsfine-tuning
More like this →
arXiv Machine Learning
Aug 4

AOS: Adaptive Optimizer Switching via Training-State Signals for Faster Convergence and Better Generalization

arXiv:2608. 01997v1 Announce Type: new Abstract: Single-optimizer training is a poor fit for the distinct phases of deep network optimization: adaptive methods handle noisy early gradients well but overshoot flat minima, while SGD with momentum generalizes better in the late phase but converges slowly early on.

By Alok Kumar Pandey, Umang Chaturvedi, Aatish Rana, Gopi Krishna Nedanuri
benchmarks
More like this →