A framework for parameter estimation and model selection in kernel deep stacking networks.

Area of Science:

• Biomedical research
• Machine learning
• Deep learning

Background:

• Kernel deep stacking networks (KDSNs) are a novel supervised learning method in biomedical research.
• KDSNs utilize artificial neural network architectures with multiple nonlinear transformations.
• Unlike traditional methods, KDSNs avoid backpropagation, using efficient kernel ridge regression fitting.

Purpose of the Study:

• To propose a data-driven framework for parameter estimation, hyperparameter tuning, and model selection in KDSNs.
• To eliminate the need for pre-specifying KDSN tuning parameters.
• To demonstrate KDSN performance on medical datasets.

Main Methods:

• A novel framework combining model-based optimization and hill climbing approaches.
• Automated parameter estimation and hyperparameter tuning for KDSNs.
• Analysis of three medical datasets: diabetes readmission, coronary artery disease, and hospital costs.

Main Results:

• The proposed KDSN methodology significantly reduces runtime compared to grid search strategies.
• KDSN modeling demonstrates competitive prediction accuracy against state-of-the-art statistical learning techniques.
• Numerical studies confirm the efficiency and effectiveness of the new framework.

Conclusions:

• KDSNs provide a computationally efficient alternative to backpropagation-based neural networks.
• The proposed methodology enables fast tuning procedures for KDSNs.
• KDSN fits generated by this method achieve prediction accuracy comparable to other deep learning techniques.