Misbegotten methodologies and forgotten lessons from Tom Swift's electric factor analysis machine: A demonstration...

Confirmatory factor analysis (CFA) and its bifactor models are popular in empirical investigations of the factor structure of psychological constructs. CFA offers straightforward hypothesis testing but has notable pitfalls, such as the imposition of strict assumptions (i.e., simple structure) that obscure unmodeled complexity. Due to the limitations of bifactor CFAs, they have yielded anomalous results across samples and studies that suggest model misspecification (e.g., evaporating specific factors and unexpected loadings). We propose the use of exploratory factor analysis (EFA) to evaluate the structural validity of CFA solutions-either before or after the estimation of more restrictive CFA models-to (a) identify model misspecifications that may drive anomalous estimates and (b) confirm CFA models by examining whether hypothesized structures emerge with limited researcher input. We evaluated the degree to which predominant factor structures were invariant across contexts along the exploratory-confirmatory continuum and demonstrate how poor methodological choices can distort results and impede theory development. All CFA models fit well, but there were numerous differences in replicability and substantive interpretability. Several similarities emerged between bifactor CFA and EFA models, including evidence of overextraction, the collapse of specific factors onto the general factor, and subsequent shifts in how the general factor was defined. We situate these methodological shortcomings within the broader literature on structural models of psychopathology, articulate implications for theories (such as the p-factor) that are borne out of factor analysis, outline several remedies for problems encountered when performing exploratory bifactor analysis, and propose alternative specifications for confirmatory bifactor models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).