Hybrid feature-selection and diversity-guided stacking framework for interpretable ensemble learning: Application to...

Reliable predictive modeling in high-dimensional biomedical data requires a balance between accuracy, interpretability, and computational efficiency. However, existing ensemble methods often overlook model diversity or rely on ad hoc feature-selection approaches, which limit generalizability. This study introduces a hybrid feature-selection and diversity-guided stacking framework designed to improve robustness and scalability across clinical and other data-intensive domains.

The proposed framework integrates a hybrid feature-selection pipeline-combining Variance Inflation Factor (VIF), Analysis of Variance (ANOVA), Sequential Backward Elimination (SBE), and Lasso regression-to reduce multicollinearity and overfitting. It also employs a diversity-aware stacking strategy that constructs sub-model sets based on pairwise diversity measures (Disagreement, Yule's Q, and Cohen's Kappa) and non-pairwise metrics (Entropy and Kohavi-Wolpert). Sixteen base classifiers and five meta-learners were trained using repeated 10-fold cross-validation. The framework was evaluated using data from 4,778 hospitalized COVID-19 patients with 116 clinical and laboratory attributes, preprocessed using robust scaling and ROSE-based class balancing.

The optimal configuration, which stacked Random Forest and XGBoost models using a Neural Network meta-learner, achieved 91.4% accuracy (95% CI: 89.8-92.8), AUC = 0.955, F1 = 0.801, and MCC = 0.746, outperforming the best individual model (AdaBoost, 90.2%). Training time (~450 s) and per-case inference time (<0.2 s) demonstrated computational feasibility. Feature-importance analysis and SHAP-based interpretation confirmed clinical relevance and interpretability.

The hybrid feature-selection and diversity-guided stacking framework improves predictive accuracy and interpretability while maintaining computational efficiency. Although validated using COVID-19 mortality data, the approach is broadly applicable to biomedical, environmental, and engineering prediction tasks that require interpretable and scalable ensemble learning.