Statistical tests of differential susceptibility: Performance, limitations, and improvements

Area of Science:

• Developmental psychology
• Behavioral genetics
• Statistical modeling

Background:

• Statistical tests of differential susceptibility are standard for evaluating developmental hypotheses.
• The performance and limitations of these tests have not been systematically studied.

Purpose of the Study:

• To systematically investigate the performance and limitations of commonly used statistical tests for differential susceptibility.
• To propose and evaluate a revised statistical test to address identified limitations.

Main Methods:

• Monte Carlo simulations were used to explore the functioning of three established differential susceptibility tests.
• A revised test utilizing a broader proportion of interaction index (0.20-0.80) was developed and simulated.

Main Results:

• Standard power calculations underestimate the sample sizes required for critical tests of differential susceptibility.
• Existing criteria (0.40-0.60 proportion of interaction) frequently yield false negatives and are sensitive to interaction symmetry assumptions.
• The revised test demonstrated improved detection rates with minimal impact on false positive rates compared to existing methods.

Conclusions:

• Differential susceptibility testing requires larger samples and careful consideration of statistical criteria.
• The proposed revised test offers a more robust approach for identifying differential susceptibility.
• Findings have implications for interpreting existing research and designing future developmental studies.