arXiv Statistics ML

Kernelised covariance matrices, dimension reduction and imbalanced spectral clustering

arXiv Machine Learning
Sep 24

Assessing the impact of dimensionality reduction on clustering performance - a systematic study

The paper systematically evaluates how five dimensionality reduction methods—PCA, Kernel PCA, VAE, Isomap, and MDS—affect the performance of four clustering algorithms (k‑means, AHC, GMM, and OPTICS). Using the Adjusted Rand Index, the study compares clustering quality with and without dimensionality reduction at levels of k‑1, 25%, and 50% of the original dimensions. Results highlight that the choice of reduction technique and its level must be carefully matched to the data’s geometry and the clustering algorithm used.

By Ousmane Assani Amate, Elyes Lounissi, Mohammadreza Bakhtyari, \'Emilie Roy, Roman Sarrazin-Gendron, Vladimir Makarenkov
arXiv Machine Learning
Aug 27

Efficient Estimation of High Information Projections using Nearest Neighbours

The paper introduces a new dimensionality reduction technique that enhances nearest‑neighbour relationships to estimate high‑information projections. It constructs a matrix encoding local covariance via nearest‑neighbour pairs and shows that, under standard regularity conditions, this matrix consistently estimates the Density Information Matrix (DIM), a non‑parametric analogue of the Fisher Information Matrix. The authors also demonstrate the method’s practical usefulness for clustering and outlier detection.

By David P. Hofmeyr
arXiv AI
Sep 18

Randomized SVD Approximations for Spectral Co-Clustering of Word-Document Matrices

The paper introduces two randomized approaches to accelerate spectral co‑clustering of word‑document matrices: one based on randomized SVD via random projection, and another combining partial SVD with element‑wise random sampling. Experiments on real and synthetic data show both methods cut runtime compared to full SVD, with the projection technique offering more consistent performance across varied sparsity levels, while the sampling method excels on denser matrices. The study highlights that the choice of approximation should align with the data’s structural properties.

By Fateme Mazdarani, Carlos Toxtli