A comparison of sequential and nonsequential specification searches in testing factorial invariance

The purpose of this study was to examine the performance of modification indices (MIs) in finding correct partial invariance models when testing factorial invariance across groups of interest. In the present study, we examined the two commonly used approaches in this area-namely, nonsequential search and sequential search. Using the nonsequential search procedure, partial invariance models can be found by relaxing at once all the MIs larger than a certain cutoff value (usually, 3.84) at the first place, whereas in the sequential search method, the models are modified by relaxing one constrained parameter (usually the parameter with the largest MI) at a time and reanalyzing the models after each parameter has been relaxed. Our simulation results showed that the nonsequential search can lead to extremely high false positive rates and that it is highly likely that some invariant items will be incorrectly identified as noninvariant. In consequence, the nonsequential search method will likely lead to inadequately modifying scales. However, the sequential search method using MIs performed well and produced good true positive and false positive rates across all simulation conditions. Recommendations based on the findings are provided, and limitations of the study are discussed.