Structuring Psychopathology: A Stress-Test of Factor Retention Methods in Complex Hierarchical Models

Dimensionality assessment in hierarchical models faces a unique challenge: distinguishing variance attributable to broad, higher-order constructs from specific, lower-order dimensions. While frameworks like Hierarchical Taxonomy of Psychopathology posit multiple second-order factors, methodological evaluations often restrict simulations to single-apex structures. This study serves as a "stress-test," evaluating hierarchical Parallel Analysis (hierPA), hierarchical Exploratory Graph Analysis (hierEGA), and the Bass-Ackwards (BA) technique across 162,000 simulated datasets with varying structural complexity. Results revealed a clear asymmetry across hierarchical levels. While second-order structures were recovered with relatively higher accuracy (ACC), first-order factor recovery remained consistently poor across all methods, with none reaching the 90% accuracy criterion. Method-specific patterns further clarified this limitation. hierEGA and BA showed a tendency toward overfactoring, whereas hierPA consistently exhibited underfactoring in first-order estimation. Although hierEGA demonstrated the strongest performance in recovering second-order factors, its accuracy declined under more complex conditions involving multiple higher-order dimensions. Similarly, hierPA showed condition-dependent performance, where underextraction at the first-order level constrained higher-order estimation. These findings highlight the potential of network-based approaches for uncovering higher-order architectures but suggest caution regarding lower-order precision. Practical implications for clinical assessment and the risks of bloated specific factors in taxonomy construction are discussed.