A Cautionary Note on Using G(2)(dif) to Assess Relative Model Fit in Categorical Data Analysis

Area of Science:

• Statistics
• Psychometrics
• Categorical Data Analysis

Background:

• The likelihood ratio test statistic G(2)(dif) is commonly used for nested model comparison in categorical data.
• Its validity relies on the correct specification of the least restrictive model, especially in large samples.

Purpose of the Study:

• To investigate the impact of least restrictive model misspecification on the G(2)(dif) statistic.
• To determine the robustness of G(2)(dif) under varying degrees of model misspecification.

Main Methods:

• A simulation study was conducted using nested item response theory models.
• The G(2)(dif) statistic was evaluated under different levels of least restrictive model misspecification.

Main Results:

• The G(2)(dif) statistic demonstrated robustness only when the least restrictive model was only slightly misspecified.
• Significant misspecification of the least restrictive model invalidated the chi-square approximation for G(2)(dif).

Conclusions:

• Assessing the absolute goodness of fit for the least restrictive model is crucial before utilizing G(2)(dif) for relative model fit assessment.
• Incorrect conclusions may arise from using G(2)(dif) without validating the base model's adequacy.