Supervised Dimensionality Reduction Methods via Recursive Regression.

Area of Science:

• Machine Learning
• Data Science
• Statistical Analysis

Background:

• Orthogonal Linear Discriminant Analysis (OLDA) and Orthogonal Least Squares Regression (OLSR) are crucial for dimensionality reduction.
• Existing methods face challenges in efficiently handling recursive problems associated with these techniques.

Purpose of the Study:

• To investigate and address the recursive problems in OLDA and OLSR.
• To develop novel methods for heuristic dimensionality reduction in orthogonal complement spaces.

Main Methods:

• A novel recursive regression method is proposed to tackle the recursive problems.
• An efficient method is developed for OLDA to find the optimal subspace related to ridge regression.
• A scalable subspace is introduced for OLSR, leading to OLSR with Optimal Scaling (OS).
• The problem is relaxed into a convex parameterized orthogonal quadratic problem for an effective approach.

Main Results:

• The developed methods achieve heuristic dimensionality reduction in orthogonal complement spaces.
• An efficient method for OLDA yields an optimal subspace linked to ridge regression.
• The proposed OLSR approach with a scalable subspace automatically obtains Optimal Scaling (OS).
• The convex relaxation provides an effective approach to achieve both optimal subspace and OS.

Conclusions:

• Two novel supervised dimensionality reduction methods are proposed based on heuristic solutions to OLDA and OLSR recursive problems.
• The methods offer efficient and effective ways to perform dimensionality reduction and obtain optimal scaling.
• The approach provides a unified framework for addressing recursive issues in orthogonal analysis techniques.