Neural Networks for Predicting Conditional Probability Densities: Improved Training Scheme Combining EM and RVFL

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Statistics

Background:

• Neural networks for conditional probability density prediction often require complex architectures.
• These complex models typically result in lengthy training durations.
• Existing methods face challenges in balancing model complexity and training efficiency.

Purpose of the Study:

• To accelerate the training process for neural networks predicting conditional probability densities.
• To investigate the effectiveness of the Random Vector Functional Link (RVFL) concept for this task.
• To enhance the generalization performance of predictive models.

Main Methods:

• Adoption of the Random Vector Functional Link (RVFL) network concept.
• Constraining a subset of network parameters to random initial values.
• Application of the Expectation-Maximization (EM) algorithm facilitated by parameter constraints.
• Training an ensemble of networks with accelerated computational efficiency.

Main Results:

• Training time acceleration by approximately two orders of magnitude compared to standard methods.
• Enabling the training of multiple network models within equivalent computational budgets.
• Demonstrated significant improvements in the generalization performance of the trained models.
• Validation through simulations indicating the efficacy of the proposed approach.

Conclusions:

• The RVFL concept, combined with EM-algorithm applicability, offers a substantial speedup in training neural networks for density prediction.
• This accelerated training facilitates the development of more robust and accurate predictive models through ensembling.
• The method presents a viable strategy for improving predictive accuracy and computational efficiency in complex modeling tasks.