Testing small variance priors using prior-posterior predictive p values.

Area of Science:

• Statistics
• Psychometrics
• Structural Equation Modeling

Background:

• Traditional null-hypothesis testing in structural equation models (SEMs) can reject accurate models due to precise point null hypotheses.
• Cohen's (1994) critique highlights that even minor, irrelevant deviations can lead to rejection of the null hypothesis.
• Evaluating approximate fit using small variance priors is a development addressing these limitations.

Purpose of the Study:

• To introduce and evaluate a new method for assessing model fit in SEMs using small variance priors.
• To demonstrate the inadequacy of existing methods like posterior predictive p-value and DIC for evaluating small variance priors.
• To propose the prior-posterior predictive p value as a suitable alternative.

Main Methods:

• The study introduces the prior-posterior predictive p value as a novel statistical tool.
• It elaborates on the distributions under null and alternative hypotheses for this new statistic.
• The method is applied to practical examples, including testing differences between two means and the magnitude of correlations.

Main Results:

• Existing methods such as posterior predictive p value and DIC are not suitable for evaluating models with small variance priors.
• The proposed prior-posterior predictive p value is shown to be a well-behaving and consistent alternative.
• The new method is demonstrated to be effective in practical applications.

Conclusions:

• The prior-posterior predictive p value offers a statistically sound approach for evaluating approximate model fit in SEMs.
• This method overcomes the limitations of traditional approaches when dealing with small variance priors.
• It provides a reliable tool for researchers to assess the relevance of parameter differences from zero.