Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets.

Area of Science:

• Data Science
• Machine Learning
• Dimensionality Reduction

Background:

• Small datasets with high dimensionality are prevalent due to data collection limits and privacy concerns.
• High dimensionality leads to data sparsity, hindering information extraction and predictive model accuracy.
• Feature extraction algorithms are crucial for reducing dimensionality while preserving essential data information.

Purpose of the Study:

• To provide a comprehensive overview of unsupervised feature extraction algorithms (UFEAs).
• To analyze and compare eight representative UFEAs for their effectiveness on small, high-dimensional datasets.
• To guide the selection of appropriate UFEAs based on their strengths and weaknesses.

Main Methods:

• Focused on unsupervised feature extraction algorithms (UFEAs) for their ability to handle unlabeled high-dimensional data.
• Selected and reviewed eight representative UFEAs: PCA, Classical MDS, Kernel PCA, Isomap, LLE, Laplacian Eigenmaps, ICA, and Autoencoders.
• Theoretically analyzed algorithms based on linearity, manifold, probabilistic, or neural network approaches, detailing working mechanisms, comparisons, and accuracy evaluations.

Main Results:

• Detailed theoretical backgrounds and working mechanisms of eight selected UFEAs were presented.
• Algorithms were classified and compared based on transformation approach, goals, parameters, and computational complexity.
• Performance evaluation on various datasets highlighted the strengths and weaknesses of each UFEA for specific scenarios.

Conclusions:

• UFEAs are vital for addressing dimensionality challenges in small, high-dimensional datasets.
• The review offers a systematic comparison of UFEAs, aiding researchers in choosing the most suitable algorithm.
• Understanding the nuances of each UFEA enables more effective data analysis and improved predictive modeling.