Goodness-of-fit Tests for Categorical Models of Psychological Processes: Fixing the Occasional Failures of Asymptotic...

The goodness of fit of categorical models of psychological processes is often assessed with the log-likelihood ratio statistic (G²), but its underlying asymptotic theory is known to have limited empirical validity. We use examples from the scenario of fitting psychometric functions to psychophysical discrimination data to show that two factors are responsible for occasional discrepancies between actual and asymptotic distributions of G². One of them is the eventuality of very small expected counts, by which the number of degrees of freedom should be computed as (J-1) × I-P-K_0.06, where J is the number of response categories in the task, I is the number of comparison levels, P is the number of free parameters in the fitted model, and K_0.06 is the number of cells in the implied I × J table in which expected counts do not exceed 0.06. The second factor is the administration of small numbers n of trials at each comparison level x (1 ≤ ≤ I). These numbers should not be ridiculously small (i.e., lower than 10) but they need not be identical across comparison levels. In practice, when n varies across levels, it suffices that the overall number N of trials exceeds 40 × I if J = 2 or 50 × I if J = 3, with no n lower than 10. Correcting the degrees of freedom and using large n are easy to implement in practice. These precautions ensure the validity of goodness-of-fit tests based on G².