Sparsifying the Fisher Linear Discriminant by Rotation

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• High-dimensional classification often relies on sparse linear discriminant analysis (LDA).
• Achieving sparsity in linear classifiers is crucial for efficient application.
• Existing methods may require data rotation, which is not always straightforward.

Purpose of the Study:

• To propose a novel family of data rotations for inducing sparsity.
• To enable the application of existing high-dimensional classifiers where sparsity is not inherent.
• To provide a robust and versatile method for improving classification accuracy.

Main Methods:

• Utilizing principal components of pooled sample covariance matrices for data rotation.
• Implementing a 'rotate-and-solve' procedure compatible with various classifiers.
• The proposed method is robust to varying levels of true model sparsity.

Main Results:

• Demonstrated that the proposed rotations effectively create the sparsity required for high-dimensional classification.
• Provided theoretical insights into the empirical success of the rotation strategy.
• Empirical validation through simulations and real-world datasets showcased significant improvements.

Conclusions:

• The proposed data rotation method successfully induces sparsity, enhancing high-dimensional classification.
• This approach offers a robust and adaptable solution for applying sparse LDA techniques.
• The method outperforms several popular high-dimensional classification rules.