A powerful penalized multinomial logistic regression approach.

Area of Science:

• Statistics
• Machine Learning
• Bioinformatics

Background:

• Penalized regression methods are crucial for prediction and variable selection in high-dimensional data.
• Existing methods may struggle with correlated predictors and complex categorical outcomes.
• There is a need for robust regularization techniques that balance prediction accuracy and interpretability.

Purpose of the Study:

• To propose a novel penalized regression approach, discriminative power lasso (DP-lasso), for multinomial logistic models.
• To incorporate predictor-specific weights based on intra- and inter-outcome category distances.
• To evaluate DP-lasso's performance against existing methods in various simulation settings.

Main Methods:

• Developed DP-lasso with an adaptive L1-type penalty term.
• Proposed three measures for weight calculation: ANOVA-based and two clustering indices.
• Conducted simulations with varying numbers of categories, predictors, association strengths, and correlations.

Main Results:

• DP-lasso with ANOVA-based weights (DPan) produced sparser models, especially with correlated predictors in high-dimensional settings.
• DPan demonstrated superior true positive rates and low false positive rates when the number of predictors (p) exceeded sample size (N).
• DPan consistently achieved high true positive rates and the lowest false positive rates across all simulation scenarios, including when p < N.

Conclusions:

• DPan is a highly recommended method for analyzing categorical outcomes with high-dimensional predictors.
• The approach effectively handles correlated predictors and improves variable selection accuracy.
• Demonstrated utility in ultra high-dimensional settings using single-cell RNA-sequencing data.