Hugging Face Trending Papers

Learning by Compression: Risk Estimates Attain the Law-of-Large-Numbers Rate

Read the original on Hugging Face Trending Papers →

Compression schemes provide a general framework for deriving data-dependent risk bounds for learning algorithms. For learning rules based on preferent compression, existing results upper-bound the risk in terms of a sample-dependent complexity $\boldsymbol{k}$, defined as the cardinality of the compression augmented by the training examples on which the reconstructed hypothesis is inappropriate.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning
Oct 12

Learning by Compression: Risk Estimates Attain the Law-of-Large-Numbers Rate

The paper shows that for learning algorithms based on preferential compression, the risk can be bounded by the compression complexity term λk/N plus an O(1/√N) term, eliminating the previously assumed logarithmic factor. This bound holds uniformly over all possible values of λk and matches the law‑of‑large‑numbers rate of empirical risk estimation. The authors also demonstrate that the leading λk/N term is necessary and analyze when empirical risk converges at O(1/√N) versus slower or inconsistent rates, while the complexity‑based estimator retains the O(1/√N) rate.

By Simone Garatti, Marco C. Campi