Why overfitting is not (usually) a problem in partial correlation networks

Area of Science:

• Psychometrics
• Network Analysis
• Statistical Modeling

Background:

• Network psychometrics is re-evaluating methods, particularly regularization techniques like the graphical lasso (glasso).
• Concerns exist regarding regularization's effectiveness in reducing spurious associations and mitigating overfitting in low p/n ratio data.
• Overfitting and the bias-variance tradeoff are critical considerations in network psychometrics.

Purpose of the Study:

• To introduce a nonregularized method for network analysis that addresses overfitting and controls false positives.
• To compare the predictive performance of the proposed nonregularized method against the graphical lasso (glasso).
• To provide guidance on utilizing the new methodology, including considerations for the multiple comparisons problem.

Main Methods:

• Developed a nonregularized method based on classical hypothesis testing for network construction.
• Conducted simulation studies to evaluate the method's performance in terms of false positive control and predictive accuracy.
• Analyzed the impact of alpha levels and the multiple comparisons problem on predictive accuracy.

Main Results:

• The nonregularized method effectively reduces or controls the false positive rate.
• The proposed method demonstrates competitive or superior predictive performance compared to the graphical lasso (glasso).
• Bias and variance are most problematic in rarely encountered low p/n ratios.

Conclusions:

• Nonregularized estimation, contrary to common assumptions, may better satisfy the desiderata of controlling false positives and ensuring accurate predictions in network psychometrics.
• The developed methodology offers advantages for both statistical inference and prediction in network analysis.
• Careful consideration of alpha levels is necessary to balance network sparsity and predictive accuracy.