Robust maximum likelihood training of heteroscedastic probabilistic neural networks

Area of Science:

• Machine Learning
• Artificial Intelligence
• Statistical Modeling

Background:

• Probabilistic Neural Networks (PNNs) are powerful tools for modeling complex data distributions.
• Gaussian heteroscedastic PNNs offer greater efficiency than those with common variance.
• Training heteroscedastic PNNs using the expectation-maximisation (EM) algorithm presents numerical challenges.

Purpose of the Study:

• To develop a robust maximum likelihood (ML) training algorithm for Gaussian heteroscedastic PNNs.
• To address the numerical difficulties encountered by the standard EM algorithm in training heteroscedastic PNNs.
• To evaluate the performance of the proposed robust learning algorithm.

Main Methods:

• The study combines the Jack-knife statistical technique with the expectation-maximisation (EM) algorithm.
• A robust ML training algorithm is developed by integrating these methods.
• The algorithm's performance is assessed using both artificial and real-world datasets.

Main Results:

• The proposed robust ML training algorithm effectively handles numerical difficulties in heteroscedastic PNN training.
• Performance evaluation on the two-dimensional XOR problem and UK construction company data demonstrates the algorithm's efficacy.
• The Jack-knife-EM approach provides a stable and reliable method for training PNNs with varying variances.

Conclusions:

• The developed robust training algorithm offers a significant improvement for Gaussian heteroscedastic PNNs.
• This method enhances the practical applicability of PNNs in scenarios requiring accurate modeling of data with heterogeneous variances.
• The Jack-knife-EM algorithm presents a viable solution for robust PNN model training.