Bayesian nonlinear model selection and neural networks: a conjugate prior approach.

Area of Science:

• Machine Learning
• Statistical Modeling
• Computational Neuroscience

Background:

• Selecting optimal neural network architectures is crucial for predictive accuracy.
• Existing model comparison methods may not fully capture predictive performance.
• Bayesian approaches offer a robust framework for model evaluation.

Purpose of the Study:

• To propose a general Bayesian nonlinear regression model comparison procedure.
• To introduce an expected utility criterion for selecting the best predictive neural network architecture.
• To enhance the internal consistency assessment of candidate models.

Main Methods:

• A Bayesian nonlinear regression model comparison procedure was developed.
• An expected utility criterion was maximized to select the optimal model.
• Model posterior predictive density was computed and asymptotically approximated.
• Analytic calculation of parameter posterior and predictive posterior densities was enabled using conjugate priors.

Main Results:

• The proposed procedure effectively compares general nonlinear regression models, including feedforward neural networks.
• The method enhances internal consistency assessment through predictive probability distributions.
• It allows for analytic computation of posterior and predictive densities, simplifying the empirical-Bayes approach.

Conclusions:

• The Bayesian selection procedure provides a robust method for comparing predictive models.
• This approach is particularly effective for feedforward neural networks.
• It offers an improvement over traditional asymptotic comparison tests for embedded models.