Modified principal component analysis: an integration of multiple similarity subspace models

Area of Science:

• Machine Learning
• Dimensionality Reduction
• Pattern Recognition

Background:

• Conventional Principal Component Analysis (PCA) is a widely used dimensionality reduction technique.
• Limitations exist in PCA for certain datasets, particularly when the number of training samples is less than the data dimensionality.
• There is a need for advanced subspace learning methods that incorporate diverse similarity metrics for enhanced performance.

Purpose of the Study:

• To propose a novel subspace learning framework, Modified PCA (MPCA), that improves upon conventional PCA.
• To integrate multiple similarity measurements within the subspace learning process.
• To enhance classification accuracy and data representation capabilities, especially for high-dimensional data with limited samples.

Main Methods:

• MPCA computes three similarity matrices: mutual information, angle information, and Gaussian kernel similarity.
• Eigenvectors of these similarity matrices are used to create novel similarity subspaces.
• A feature selection approach constructs weak machine cells (WMCs) from these subspaces for sample classification.

Main Results:

• MPCA demonstrates superior performance compared to state-of-the-art PCA-based methods in classification accuracy and clustering.
• The method achieves desirable classification results on popular real-world datasets, including face databases.
• MPCA shows a powerful capability for data representation and face image reconstruction.

Conclusions:

• MPCA offers a robust subspace learning framework suitable for scenarios with fewer training samples than data dimensions.
• The integration of multiple similarity measurements significantly boosts discriminative capability and data representation.
• MPCA presents a promising advancement in PCA-based techniques for various machine learning applications.