Aided Language Measures: Establishing Observer Agreement for Communicators in Early Language Phases

Although many valid, reliable, and developmentally sensitive measures exist to monitor the language gains of children who rely on spoken language to communicate, the same is not true for graphic symbol communicators. This study is a first step in developing such measures by examining the interobserver agreement (IOA) and within-observer agreement of 13 measures designed to monitor the language progress of children who use aided augmentative and alternative communication (AAC). These measures are based on the Graphic Symbol Utterance and Sentence Development Framework (Binger et al., 2020) and are hypothesized to capture various phases of graphic symbol communication.

Four graduate student observers coded 13 measures across 57 different play-based sessions of children with Down syndrome ages 3;0-5;11 (years;months). For IOA, sessions were coded by two different observers. For within-observer agreement, all sessions were recoded by the same coders. Corpus-level analyses were completed to characterize the nature of the samples (e.g., average mean length of utterance for the samples). IOA and within-observer agreement were examined for each utterance.

Across all observers and measures, acceptable levels of IOA and within-observer agreement were achieved, with most measures yielding relatively high levels of agreement. Some differences were noted across measures, with the less experienced coders demonstrating less agreement on select measures.

Results provide initial evidence that many measures based on the Graphic Symbol Utterance and Sentence Development Framework can be reliably coded. These findings are a first step in developing psychometrically sound measures to monitor the expressive language progress of children who use AAC.

https://doi.org/10.23641/asha.19601551.