A structured communication adolescent guide (SCAG): assessment of reliability and validity

To assess the reliability and validity of a Structured Communication Adolescent Guide (SCAG) in an undergraduate medical education setting using trained adolescent raters.

The SCAG is a 49-item, 6-section (A-F), protocol derived from the Calgary Cambridge Observation Guide that incorporates adolescent psychosocial data collection with the physician's communication skills. Four trained female adolescents scored 42 videotaped adolescent clinical encounters using the SCAG; a trained psychologist's rating for each videotape was used as the gold standard.

Agreement among adolescent raters was determined by calculating intraclass correlation coefficients (ICC). The individual SCAG item scores, combined with the global ratings for each section, resulted in an overall ICC value of 0.93, indicating very strong agreement among the 4 raters. The global rating scores for the sections ('initiating the session', 'initiating separation', 'once adolescent is alone--lifestyle' and 'closure') produced an ICC range of 0.58-0.93. However, the ICC values for the 2 remaining sections ('how was information collected' and 'gathering information') global rating scores were below 0.30, signifying low agreement. Overall agreement between the adolescent raters and the gold standard resulted in an ICC value of 0.78. This is evidence of the SCAG's criterion validity.

The SCAG is a reasonably valid tool for use in guiding an encounter with an adolescent patient. However, 2 sections require modifications to improve their reliability and thus the SCAG's overall performance. Our results suggest that the SCAG shows promise as a potentially useful teaching resource in undergraduate medical education in adolescent medical interviewing.