Using parameter space partitioning to evaluate a model's qualitative fit.

Area of Science:

• Cognitive Science
• Computational Modeling
• Psychology

Background:

• Parameter Space Partitioning (PSP) is a valuable technique for analyzing computational models.
• PSP identifies distinct data patterns a model can produce by partitioning its parameter space.
• Current model selection methods often rely on quantitative fit, potentially overlooking qualitative pattern differences.

Purpose of the Study:

• To propose a novel fit measure based on Parameter Space Partitioning (PSP) for model selection.
• To introduce a single quantitative summary of PSP analysis outcomes for comparative model evaluation.
• To demonstrate the utility of PSP-based model selection in the domain of category learning.

Main Methods:

• Developing a quantitative fit measure derived from Parameter Space Partitioning (PSP) analysis.
• Applying the PSP fit measure to models within the context of category learning research.
• Conducting a large-scale model recovery study to assess the reliability of the PSP fit measure.

Main Results:

• The proposed PSP fit measure effectively summarizes PSP analysis results into a single, usable metric.
• PSP-based model selection demonstrated utility in application examples within category learning.
• A comprehensive model recovery study confirmed the excellent performance and reliability of the PSP fit measure.

Conclusions:

• The novel PSP fit measure provides a robust tool for model selection, emphasizing qualitative data patterns.
• PSP-based model selection offers a complementary approach to traditional quantitative methods.
• The findings suggest that PSP fit is a reliable and useful metric for evaluating and selecting computational models, particularly in cognitive science.