Noise-Debiased Thermodynamic Variance for Local Learning Coefficient Probes
Read the original on arXiv Machine Learning →The paper introduces the Shift‑Invariant Variance Estimator (SIVE) to remove same‑state noise from thermodynamic variance calculations in neural‑network training. SIVE provides an unbiased estimate of path variance without requiring Markov chain Monte Carlo stationarity, and it reveals a reproducible mid‑training trough and rebound in MNIST MLP trajectories that raw variance masks. Experiments across multiple localization scales confirm that SIVE captures a large portion of observation noise and satisfies an early‑drop/late‑rise criterion in most trajectory‑scale pairs.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.