SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws
Read the original on arXiv Machine Learning →The paper analyzes training dynamics of multiclass logistic regression on high‑dimensional Gaussian mixture models with many classes. It finds that learning proceeds sequentially from the most to the least frequent classes and, when class priors follow a power‑law, the cross‑entropy risk evolves through an initial plateau, a power‑law decay phase, and a final convergence phase. The study also shows how model capacity and optimization trade‑off under a fixed compute budget, leading to a compute‑optimal scaling law that prescribes model size and training time as functions of compute.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.