Oriel — AI & technology worth noticing, every day.

Unsupervised Feature Selection Using Local Intrinsic Dimensionality -- Extended Version

Paper · https://arxiv.org/abs/2610.40736

arXiv:2610.40736v1 Announce Type: new Abstract: High dimensionality degrades both the efficiency and the effectiveness of learning algorithms, and in many applications no class labels are available to guide dimensionality reduction. This paper proposes LIDFS, an unsupervised feature selection method based on Local Intrinsic Dimensionality (LID). The method builds on the formal equivalence between LID and the indiscriminability of distance measures. The data matrix is transposed so that each feature is treated as a point, and the LID of each feature is estimated by maximum likelihood from the distances to its $k$ nearest neighboring features. Features with lower LID exhibit greater local discriminability and receive higher importance scores. A refinement step then retains one representative per feature neighborhood and demotes locally redundant features. The result is a complete ranking from which any number of features can be selected without rerunning the procedure. LIDFS does not use any information about labels or the number of classes, and runs in $\mathcal{O}(d^2 n)$ time for $n$ instances and $d$ features. LIDFS is compared with classic and state-of-the-art methods and a random baseline on 23 high-dimensional datasets, the classic benchmark collection. The evaluation uses classification accuracy and AUC with five downstream classifiers, as well as clustering accuracy and normalized mutual information, over both full and aggressive selection ranges. LIDFS ranks first in the supervised evaluations and first or tied second in the clustering-based evaluations. It belongs to the top statistical group together with methods that require the number of classes, and it runs faster than its closest competitor. Several recent methods, by contrast, fail to outperform the random baseline. A sensitivity analysis shows that small neighborhood sizes yield the best results.

First seen by Oriel on .

AI summary

The paper introduces LIDFS, an unsupervised feature-selection method using Local Intrinsic Dimensionality that achieved leading supervised and clustering results across 23 high-dimensional datasets.

  • LIDFS transposes the data matrix, treats features as points, and estimates each feature’s LID by maximum likelihood from distances to its k nearest feature neighbors.
  • Lower-LID features receive higher importance, while a refinement step keeps one representative per neighborhood and demotes locally redundant features.
  • The method produces a reusable complete ranking without labels or class counts, with O(d^2 n) runtime for n instances and d features.
  • Using five downstream classifiers, LIDFS ranked first in supervised evaluations and first or tied second in clustering evaluations.
  • It joined the top statistical group, ran faster than its closest competitor, and performed best with small neighborhoods; several recent methods underperformed the random baseline.

#feature selection #local intrinsic dimensionality #machine learning

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2610.40736v1https://arxiv.org/abs/2610.40736 Read discussion