Related Experiment Videos
Behavior states: now you see them, now you don't
O C Mudford1, J Hogg, J Roberts
1Department of Psychology, Keele University, Staffordshire. O.C.Mudford@keele.ac.uk
American Journal of Mental Retardation : AJMR
|August 18, 1999
Summary
Replication of a 1998 study on behavior state coding found significantly lower interobserver agreement, ranging from 0% to 80%. This suggests potential issues with the original methodology or its application in behavior state research.
Area of Science:
- Behavioral Science
- Psychology
- Research Methodology
Background:
- The original study by Guess et al. (1998) reported high interobserver agreement (63%-91%) for a 13-category behavior state coding system.
- This study attempted to replicate those findings due to concerns about the reported reliability levels.
Discussion:
- Replication attempts yielded substantially lower reliability, with agreement rates between 0% and 80%.
- Further investigation using original observers also failed to achieve the reported agreement levels, indicating persistent reliability issues.
Key Insights:
- Observed agreement on behavior state occurrence ranged from 0% to 58% across observer pairs.
- Kappa values, a measure of agreement beyond chance, ranged from 0 to .64, indicating poor to moderate reliability.
Outlook:
- Investigating the discrepancies in reliability data is crucial for validating behavior state coding systems.
- Further research is needed to identify specific factors contributing to the failure in replicating high interobserver agreement.