Quantifying the Reliability and Replicability of Psychopathology Network Characteristics

Area of Science:

• Psychology
• Network Science
• Psychopathology Research

Background:

• Pairwise Markov random field (PMRF) networks, including Gaussian graphical models (GGMs) and Ising models, are widely used for psychopathology network analyses.
• Recent studies emphasize the importance of network reliability and replicability.

Purpose of the Study:

• To compare existing methods for assessing PMRF network stability and consistency with direct metrics of detailed network characteristics.
• To evaluate the reliability and replicability of network edges in depression, anxiety, and posttraumatic stress disorder (PTSD) symptom networks.

Main Methods:

• Applied lasso regularization, bootnet, and NetworkComparisonTest R packages to quantify network stability.
• Directly compared individual network edges (presence, absence, sign, strength) across different datasets and time points.
• Analyzed GGMs of depression and anxiety symptoms (n=403) across two waves and reanalyzed four PTSD GGMs.

Main Results:

• Existing methods suggested high network stability and consistency.
• Direct edge-wise comparisons revealed substantial unreplicated edges (39-49%) between networks.
• Contradictory findings highlight a discrepancy between global stability metrics and detailed network characteristic replication.

Conclusions:

• Current methods may overestimate the reliability of PMRF network characteristics.
• The limited reliability of detailed network features is likely common but often overlooked.
• Poor replicability raises concerns about the generalizability and utility of conclusions drawn from psychopathology network analyses.