Reliability issues and solutions for coding social communication performance in classroom settings
Related Experiment Video
Updated: Jul 19, 2026

A Familiarization Protocol Facilitates the Participation of Children with ASD in Electrophysiological Research
Published on: July 31, 2017
Lesley B Olswang1, Liselotte Svensson, Truman E Coggins
1Department of Speech and Hearing Sciences, University of Washington, 1417 Northeast 42nd Street, Seattle, WA 98105-6246l, USA. lolswang@u.washington.edu
Purpose:
To explore the utility of time-interval analysis for documenting the reliability of coding social communication performance of children in classroom settings. Of particular interest was finding a method for determining whether independent observers could reliably judge both occurrence and duration of ongoing behavioral dimensions for describing social communication performance.
Method:
Four coders participated in this study. They observed and independently coded 6 social communication behavioral dimensions using handheld computers. The dimensions were mutually exclusive and accounted for all verbal and nonverbal productions during a specified time frame. The technology allowed for coding frequency and duration for each entered code. Data were collected from 20 different 2-min video segments of children in kindergarten through 3rd-grade classrooms. Data were analyzed for interobserver and intraobserver agreements using time-interval sorting and Cohen's kappa. Further, interval size and total observation length were manipulated to determine their influence on reliability.
Results:
The data revealed interval sorting and kappa to be a suitable method for examining reliability of occurrence and duration of ongoing social communication behavioral dimensions. Nearly all comparisons yielded medium to large kappa values; interval size and length of observation minimally affected results. Implications The analysis procedure described in this research solves a challenge in reliability: comparing coding by independent observers of both occurrence and duration of behaviors. Results indicate the utility of a new coding taxonomy and technology for application in online observations of social communication in a classroom setting.
Related Concept Videos
Social Foundations of Self II: The Generalized Other
Barriers to Effective Communication II
Cultural barriers:
Differences in values, beliefs, religion, knowledge, and tradition can significantly impact communication. Awareness of nonverbal cues is critical, especially when conversing with a patient from a different culture. What appears appropriate in one culture may be inappropriate in another.
Semantic barriers:
As a result of their tendency to use...
Modeling in Therapy
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in situations...
Barriers to Effective Communication I
Communication barriers include the following:
Physiological barriers: They are limitations caused by a person's health condition or disability, such as hearing loss, poor eyesight, illness, or unconsciousness. An example to overcome this barrier...
Factors Influencing Attraction V: Social Skills
Reliability and Validity