arXiv Machine Learning By Qiufu Li, Huibin Xiao, Linlin Shen, Weijia Fan, Xi Jia, Yongdong Huang

Comparing Classification and Feature Learning by Analyzing Their Loss Functions in Normalized Spaces with Different Scale Factors

Read the original on arXiv Machine Learning →

The Flow has not summarised this story yet — read it at arXiv Machine Learning.

arXiv Machine Learning
5d ago

Unsupervised Feature Selection Using Local Intrinsic Dimensionality -- Extended Version

The paper introduces LIDFS, an unsupervised feature selection technique that uses Local Intrinsic Dimensionality (LID) to rank features based on their local discriminability. By treating each feature as a point and estimating its LID via maximum likelihood from distances to its k nearest neighboring features, LIDFS assigns higher importance to features with lower LID and refines the ranking to remove locally redundant features. Experiments on 23 high‑dimensional datasets show that LIDFS outperforms both classic and state‑of‑the‑art methods in supervised and clustering evaluations while running faster than its closest competitor.

By Oussama Chelly, Michael E. Houle, Ken-ichi Kawarabayashi, Muhammad Rajabinasab, Arthur Zimek
arXiv Machine Learning
Oct 19

Escaping Parameter Space: Tight Generalization Bounds via Representation Quality

The paper presents a tight generalization bound for deterministic neural networks that relies on the quality of their last hidden layer representations rather than traditional parameter-space metrics. By using a parameter‑free k‑nearest‑neighbor classifier to assess representation quality, the bound decomposes into three measurable components—local class geometry, decoder‑network agreement, and training stability—providing both a non‑vacuous certificate and a diagnostic tool for understanding generalization. Experiments show the bound achieves an 8.0‑percentage‑point gap to test error on CIFAR‑10 with ResNet‑18, yields the first non‑vacuous certificates on CIFAR‑100, and certifies 1.4 % on CIFAR‑10 and 23.1 % on 1,000‑class ImageNet using frozen DINOv2 representations.

By Niclas G\"oring, Shuofeng Zhang, Branton DeMoss, Ard A. Louis