A survey of model evaluation approaches with a tutorial on hierarchical bayesian methods.

Area of Science:

• Cognitive Science
• Computational Neuroscience
• Psychology

Background:

• Current methods for evaluating cognitive science models include Bayes factors, minimum description length, model mimicry, validation, and generalization.
• These approaches, while useful, often provide limited general assessments of model performance.

Purpose of the Study:

• To review existing model evaluation techniques in cognitive science.
• To propose hierarchical Bayesian methods as a superior approach for comprehensive model assessment.
• To demonstrate the application of hierarchical Bayesian analyses with practical examples.

Main Methods:

• Review of existing model evaluation methodologies.
• Theoretical argument for the advantages of hierarchical methods, particularly hierarchical Bayesian methods.
• Application of hierarchical Bayesian analyses to cognitive science models.

Main Results:

• Traditional evaluation methods have limitations in providing general model assessments.
• Hierarchical methods, especially hierarchical Bayesian methods, offer a more thorough evaluation framework.
• Worked examples demonstrate the utility of hierarchical Bayesian analyses for descriptive adequacy, parameter inference, prediction, and generalization.

Conclusions:

• Hierarchical Bayesian methods provide a principled and coherent framework for evaluating cognitive science models.
• This approach addresses key questions in model assessment more effectively than traditional methods.
• Hierarchical Bayesian analyses enhance the understanding and validation of cognitive models.