Learning by Compression: Risk Estimates Attain the Law-of-Large-Numbers Rate

The paper shows that for learning algorithms based on preferential compression, the risk can be bounded by the compression complexity term λk/N plus an O(1/√N) term, eliminating the previously assumed logarithmic factor. This bound holds uniformly over all possible values of λk and matches the law‑of‑large‑numbers rate of empirical risk estimation. The authors also demonstrate that the leading λk/N term is necessary and analyze when empirical risk converges at O(1/√N) versus slower or inconsistent rates, while the complexity‑based estimator retains the O(1/√N) rate.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.