The effective number of parameters in post hoc models

Area of Science:

• Statistics
• Experimental Design
• Data Analysis

Background:

• Determining the complexity of post hoc statistical models can be challenging.
• Parameters fixed at zero post-data examination raise questions about true model complexity.
• Parsimonious descriptions of data, even if post hoc, intuitively offer advantages.

Purpose of the Study:

• To investigate the effective number of parameters in post hoc models within factorial designs.
• To reconcile the intuitive appeal of simple post hoc models with statistical complexity.
• To provide a framework for assessing the complexity of post hoc simple effects models.

Main Methods:

• Utilized Monte Carlo simulations to estimate effective parameter counts.
• Focused analysis on factorial designs.
• Examined various classes of simple, post hoc models derived from data.

Main Results:

• Quantified the effective number of parameters for different post hoc simple effects models.
• Demonstrated how post hoc model selection impacts complexity assessment.
• Provided simulation-based evidence for parameter effective numbers.

Conclusions:

• The effective number of parameters in post hoc models can be rigorously established.
• Simple, post hoc models derived from factorial designs have quantifiable complexity.
• This work aids in a more accurate assessment of statistical model parsimony after data analysis.