Bayesian multioutput feedforward neural networks comparison: a conjugate prior approach.

Area of Science:

• Computational statistics
• Machine learning
• Artificial intelligence

Background:

• Selecting optimal neural network topology is crucial for predictive performance.
• Traditional methods like cross-validation often rely on point predictions, potentially missing nuanced model behaviors.
• Information-theoretic criteria offer alternatives but may not fully capture predictive utility.

Purpose of the Study:

• To propose a Bayesian method for comparing and selecting multioutput feedforward neural network topologies.
• To introduce an expected utility criterion, estimated via sample-reuse, as a measure of prediction fitness potential.
• To demonstrate the effectiveness of this Bayesian approach against conventional selection procedures.

Main Methods:

• A Bayesian framework is employed for neural network topology selection.
• An expected utility criterion, based on the logarithmic score of the predictive probability density, is utilized.
• Conjugate probability distributions are used as priors for consistent approximation of the network posterior predictive density.

Main Results:

• The proposed Bayesian method, using expected utility, provides a robust measure of predictive capability.
• The method consistently estimates the prediction fitness potential through sample-reuse computation.
• Simulations and a food analysis dataset demonstrate superior performance compared to classic cross-validation and information-theoretic criteria.

Conclusions:

• The Bayesian expected utility approach offers a statistically sound and effective method for neural network topology selection.
• This method enhances model comparison by considering the full predictive probability density, not just point predictions.
• The findings suggest a valuable alternative for researchers and practitioners in machine learning and computational statistics.