QUADRO: A SUPERVISED DIMENSION REDUCTION METHOD VIA RAYLEIGH QUOTIENT OPTIMIZATION.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• High-dimensional data analysis presents challenges in dimensionality reduction.
• Traditional linear methods may not capture complex nonlinear relationships.
• Rayleigh quotient optimization is distinct from classification in nonlinear settings.

Purpose of the Study:

• To propose QUADRO (Quadratic Dimension Reduction via Rayleigh Optimization), a novel sparse quadratic dimension reduction method.
• To address the computational challenges of Rayleigh quotient optimization in high-dimensional data.
• To explore the scientific interest of Rayleigh quotient optimization in nonlinear data analysis.

Main Methods:

• Developed a novel Rayleigh quotient-based sparse quadratic dimension reduction method (QUADRO).
• Utilized elliptical models to manage fourth-order cross-moments of predictors.
• Employed robust estimates for heavy-tail distributions.
• Formulated Rayleigh quotient maximization as a convex optimization problem.
• Implemented an efficient linearized augmented Lagrangian method for computation.

Main Results:

• QUADRO effectively handles high-dimensional data by optimizing the Rayleigh quotient.
• The method overcomes computational issues related to high-order moments.
• Theoretical convergence rates are provided for Gaussian and elliptical models.
• Numerical results on synthetic and real datasets validate the method's performance.

Conclusions:

• QUADRO offers a robust and efficient approach for sparse quadratic dimension reduction in high-dimensional data.
• The method demonstrates strong theoretical guarantees and practical performance.
• Rayleigh quotient optimization is a valuable tool for nonlinear dimension reduction.