Statistical Power of Alternative Structural Models for Comparative Effectiveness Research: Advantages of Modeling...

Area of Science:

• Health Services Research
• Psychology
• Statistics

Background:

• Evaluating health interventions requires robust statistical methods.
• Action-research interventions aim to prevent risk behaviors in youth.
• Assessing comparative effectiveness necessitates careful consideration of measurement unreliability.

Purpose of the Study:

• To illustrate the advantages of modeling outcome unreliability in comparative effectiveness research (CER).
• To test the effect of an action-research intervention on youth risk behaviors using structural equation models.
• To compare statistical power and model fit across different structural equation models.

Main Methods:

• Employed Monte Carlo simulations to compare simple two-group alternative structural equation models.
• Investigated the impact of modeling measurement unreliability on statistical power.
• Utilized a summer job program with an action-research component as a case study.

Main Results:

• Models assuming equal parameters across groups were underpowered to detect intervention effects.
• Modeling outcome unreliability significantly increased statistical power.
• Improved detection of the hypothesized intervention effect was achieved by accounting for measurement unreliability.

Conclusions:

• Modeling measurement unreliability enhances statistical power and model fit in CER.
• Flexible multi-group structural models can benefit comparative effectiveness research.
• Accounting for unreliability is crucial for accurate evaluation of health interventions.