Compute-Optimal Pretrain--Fine-tune in Ridge Gradient Descent

The paper studies how to allocate a fixed computational budget between pretraining and fine‑tuning in a two‑stage ridge regression setting. By modeling the process as a compute‑split problem and analyzing data‑dependent evaluation geometries, it derives the optimal split in terms of prediction‑relevant spectral components of the pretraining and fine‑tuning empirical covariances. The analysis uses a basis‑invariant eigenspace decomposition and perturbative control of non‑commuting dynamics to capture how pretraining directions influence downstream predictions.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• fine-tuning