arXiv Machine Learning

CCMP: Class-Conditional Multicollinearity Pruning for High-Dimensional Feature Selection

arXiv Machine Learning
Jul 7

Efficient Cross-Validation for Sparse Linear Regression

arXiv:2306. 14851v5 Announce Type: replace-cross Abstract: Given a high-dimensional covariate matrix and a response vector, ridge-regularized sparse linear regression selects a subset of features that explains the relationship between covariates and the response in an interpretable manner.

By Ryan Cory-Wright, Andr\'es G\'omez
arXiv Machine Learning
Jul 14

Vertical Fusion: Condensing Internal Representations for Robust ViT Classification

arXiv:2607. 10391v1 Announce Type: cross Abstract: Despite exposing rich intermediate representations, Vision Transformers (ViTs) are almost exclusively utilized as black-box feature extractors, where only the last layer is considered for downstream tasks.

By Francesco Di Salvo, Shyam Nandan Rai, Hamed Damirchi, Ignacio Meza De la Jara, Sebastian Doerrich, Marco Lents, Christian Ledig
arXiv Machine Learning
5d ago

Unsupervised Feature Selection Using Local Intrinsic Dimensionality -- Extended Version

The paper introduces LIDFS, an unsupervised feature selection technique that uses Local Intrinsic Dimensionality (LID) to rank features based on their local discriminability. By treating each feature as a point and estimating its LID via maximum likelihood from distances to its k nearest neighboring features, LIDFS assigns higher importance to features with lower LID and refines the ranking to remove locally redundant features. Experiments on 23 high‑dimensional datasets show that LIDFS outperforms both classic and state‑of‑the‑art methods in supervised and clustering evaluations while running faster than its closest competitor.

By Oussama Chelly, Michael E. Houle, Ken-ichi Kawarabayashi, Muhammad Rajabinasab, Arthur Zimek
arXiv Machine Learning
Sep 16

Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions

arXiv:2605.09396v2 Announce Type: replace-cross Abstract: This paper relaxes the restrictive symmetry conditions adopted in [4], [5] and extends their universal feature selection framework to accommo...

By Dier Tang (Department of Mathematics, The University of Hong Kong, Hong Kong, China), Guangyue Han (Department of Mathematics, The University of Hong Kong, Hong Kong, China)
arXiv AI
Oct 16

Efficient Pairwise Difference Learning: A Comparative Study of Encoding Strategies and Training Pair Selection

The paper evaluates how different encoding strategies and pair selection methods affect Pairwise Difference Learning (PDC) for classification. It compares four encoders—including a new triplet with absolute difference—and four pair selection schemes across almost 200 datasets using several base models. The study finds that the new encoder yields the best average F1 scores, that using only upper‑triangle pairs cuts training time by about 40% with little accuracy loss, and that no single configuration dominates across all datasets.

By Bernard Georges, Mohamed Karim Belaid, Manuel Rodrigues