Related Experiment Video
Updated: Aug 7, 2026

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
Published on: March 14, 2018
Inter-rater and test-retest reliability: methods and results for the neighborhood observational checklist
Shannon N Zenk1, Amy J Schulz, Graciela Mentz
1Department of Public Health, Mental Health, and Administrative Nursing, University of Illinois at Chicago College of Nursing, 845 S Damen Ave, M/C 802, Chicago, IL 60608, USA. szenk@uic.edu
Abstract:
The popularity of direct or systematic social observation as a method to evaluate the mechanisms by which neighborhood environments impact health and contribute to health disparities is growing. The development of measures with adequate inter-rater and test-retest reliability is essential for this research. In this paper, based on our experiences conducting direct observation of neighborhoods in Detroit, MI, we describe strategies to promote high inter-rater and test-retest reliability and methods to evaluate reliability. We then present the results and discuss implications for future research efforts using direct observation in four areas: methods to evaluate reliability, instrument content and design, observer training, and data collection.
Related Concept Videos
Reliability and Validity
Naturalistic Observations
Comparing Experimental Results: Student's t-Test
McNemar's Test
Self-Report Tests of Personality
Observational Studies
There are three types of observational studies – Prospective, retrospective, and cross-sectional.
Prospective Study
Prospective studies, also known as longitudinal or cohort studies, are carried out by collecting future data from groups sharing similar characteristics. One example of...
