Reliability of student evaluations of voice therapy implications for theory and training

This study examines reliability and the focus of attention of evaluations of voice therapy sessions. Therapy sessions were provided by 8 clinician-client pairs. For each pair, two 20-minute therapy sessions were evaluated by 50 undergraduate student raters and by 3 experienced clinicians. The goal of all therapy sessions was the modification of the frequency and type of voice onset. Raters evaluated each session using 12 behavioral and interpersonal criteria. Results indicated that (1) interrater reliability coefficients for each of the 12 criteria were moderately high to high, (2) the highest coefficients were found for ratings of the client, and (3) ratings of the client were found to be the most consistent across 2 sessions. Intrarater reliability was highest in ratings of quality, use of counseling, clarity and use of feedback, and amount of explanation by the clinician.