Observer agreement for marking moments of stuttering

Observers were asked to mark moments of stuttering on transcripts of tape-recorded speaking performances. Instructions to observers were varied in an attempt to maximize agreement. An index of agreement was developed to reflect word-by-word agreement for more than two observers. Results showed less word-by-word agreement than might have been expected, although intercorrelations among observers for total number of words marked as stuttering was high, as is usually found.