The relationship between trained ratings and untrained listeners' judgments of global coherence in extended...

Global coherence rating scales have been used by a number of researchers to examine spoken discourse in populations with and without acquired neurogenic communication disorders. The 4-point global coherence (GC) scale in the current study has demonstrated reliability and convergent validity. However, we have not yet established how a global coherence rating corresponds to functional communication.

The current study explores the clinical meaningfulness of the 4-point GC rating scale. Survey questions and ratings were developed to examine discourse quality and functional coherence as perceived by untrained listeners. Ratings by untrained listeners were compared to trained discourse ratings using the established 4-point GC scale.

Twelve discourse samples, scored by a trained rater, were selected for the current study from a previously collected set of discourse transcripts. Transcripts were extended monologues in response to one of four possible open-ended questions that were re-recorded by the trained rater to remove any distracting features of the original recordings, such as articulatory errors. Twenty-four untrained listeners rated the discourse samples using a short questionnaire that included questions for each sample about: topic maintenance, inclusion of unnecessary information, and the listener level of interest and attention. Questions for untrained listeners were based on operational definitions of global coherence and discourse quality respectively. These untrained ratings were compared to trained ratings of global coherence. Outcome measures were compared using non-parametric tests and a Spearman Rank Order test was also used to examine relationships among variables.

Untrained listeners' ratings for topic maintenance and inclusion of unnecessary information were significantly different between trained low, medium, and high global coherence ratings. Topic maintenance and inclusion of unnecessary information were also both significantly correlated with global coherence. Untrained listeners' ratings of their level of interest and attention for a sample were significantly different between trained medium-high and low-high global coherence ratings. Interest and attention ratings were also significantly correlated with ratings of global coherence.

Untrained raters did differentiate between levels of global coherence using ratings of topic maintenance, inclusion of unnecessary information, and their level of attention and interest. Global coherence was also significantly correlated with the untrained ratings. These findings provide preliminary external validity for the global coherence scale and support its clinical utility.