External and Internal Validity Considerations in Youth Effectiveness Trials: Lessons Learned from the COMET Study.

Area of Science:

• Clinical Psychology
• Public Health Research
• Implementation Science

Background:

• Effectiveness trials aim to enhance the generalizability and public health impact of interventions.
• Challenges in these trials can threaten both external and internal validity.

Purpose of the Study:

• To illustrate the challenges in effectiveness trials using data from a two-site randomized trial.
• To present recommendations for future effectiveness research.

Main Methods:

• The Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET) was a randomized effectiveness trial.
• It compared three interventions across 19 community mental health clinics: treatment as usual (TAU), TAU with measurement-based care (TAU+), and the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents with MBC (UPA).
• Participant flow, characteristics, and site differences were analyzed for 176 clinicians and 196 adolescents.

Main Results:

• Recruitment and retention of clinicians and adolescents proved challenging, raising representativeness concerns.
• The study successfully recruited a diverse sample of clinicians and adolescents, including racially/ethnically minoritized, low-income, and clinically complex youth.
• Significant differences were observed between the two study sites regarding participant characteristics.

Conclusions:

• Difficulties in recruitment and retention raise questions about the external validity of the findings.
• Site differences pose challenges to the internal validity of the study.
• The study suggests incorporating implementation science approaches for future effectiveness studies.