Preliminary Reliability and Validity of Asynchronous Speech Screening to Support Clinical Triage for Children With...

Children with cleft and craniofacial conditions require ongoing monitoring and evaluation to manage persistent speech and resonance disorders. Regular assessment allows clinicians to detect emerging problems, evaluate intervention efficacy, and adjust treatment plans accordingly. However, socioeconomic and logistical barriers place many children at risk for sporadic or absent follow-up, leading to undiagnosed or underaddressed velopharyngeal dysfunction. This pilot study examined the reliability and use of asynchronous speech and resonance screenings to identify children who need comprehensive imaging assessment versus those suitable for continued monitoring and local speech therapy.

Twenty children with repaired cleft palate and/or noncleft velopharyngeal dysfunction completed in-clinic speech evaluations, during which standardized audio-only and audio-video samples were simultaneously recorded using telephone and web-conferencing platforms. During the evaluation, speech samples were rated in real time using the Cleft Audit Protocol for Speech-Americleft Modification. Each of the 40 recording files, including 20 audio-only and 20 audio-video recordings, were blinded, randomized, and independently reviewed by two asynchronous raters, yielding 80 asynchronous rating events. Interrater agreement and reliability were evaluated using percentage agreement and unweighted kappa coefficients across modalities.

Across parameters and modalities, asynchronous ratings closely approximated in-person ratings. Reliability was very good for intelligibility and acceptability (κ = .81-.83), good for resonance and cleft speech characteristics (κ = .63-.75), and moderate for audible nasal emission and weak pressure consonants (κ = .45-.57). Asynchronous raters demonstrated high agreement with in-person ratings for therapy recommendations (80%-85%; κ = .57-.69) and clinical triage decisions (70%-90%; κ = .41-.79).

Preliminary findings support the feasibility and reliability of asynchronous speech and resonance screening to inform clinical triage decisions. Audio-only and audio-video recordings yielded generally comparable ratings and triage recommendations, with video offering potential added value for select speech features. Future development of asynchronous monitoring platforms has promise for reducing unnecessary clinic visits, streamlining interventions, and ensuring timely follow-up for those in need of higher level care.

https://doi.org/10.23641/asha.34050840.