Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Measuring the reliability of observational data: a reactive process.

R G Romanczyk1, R N Kent, C Diament

  • 1State University of New York at Stony Brook.

Journal of Applied Behavior Analysis
|April 1, 1973
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Maintenance of appropriate behavior in a token program.

Journal of abnormal child psychology·2013
Same author

Dysphoria and marital discord: are dysphoric individuals at risk for marital maladjustment?

Journal of marital and family therapy·2010
Same author

Two new measures of attitudes about the acceptability of teen dating aggression.

Psychological assessment·2001
Same author

Categorizing partner-violent men within the reactive-proactive typology model.

Journal of consulting and clinical psychology·2001
Same author

Childhood physical abuse, personality, and adult relationship violence: a model of vulnerability to victimization.

The American journal of orthopsychiatry·2001
Same author

Can questionnaire reports correctly classify relationship distress and partner physical abuse?

Journal of family psychology : JFP : journal of the Division of Family Psychology of the American Psychological Association (Division 43)·2001

Observer awareness of reliability testing influences data accuracy. When observers knew their performance was monitored, they adjusted their criteria, leading to higher reliability. Covert assessment revealed lower reliability and inaccurate behavior frequency recording.

Area of Science:

  • Psychology
  • Behavioral Science
  • Research Methodology

Background:

  • Observational data is crucial in behavioral research.
  • Ensuring the reliability of observational data is a key methodological challenge.
  • The impact of assessment conditions on observer reliability requires further investigation.

Purpose of the Study:

  • To investigate how awareness of reliability measurement affects observer accuracy.
  • To compare observer reliability under overt versus covert assessment conditions.
  • To examine observer bias in recording behavior frequencies.

Main Methods:

  • Two assessors independently observed behavior using unique versions of a standard observational code.
  • Observational data reliability was assessed under two conditions: overt (observers aware of measurement) and covert (observers unaware).

Related Experiment Videos

  • Inter-rater reliability was calculated between observers and each assessor.

    • Main Results:

    • Observer reliability was significantly higher when observers knew their performance was being assessed (overt) compared to when they did not (covert).
    • In overt assessment, reliability was higher with the identified assessor than the unidentified one, suggesting criterion modification.
    • Covert assessment showed substantially lower reliability, and observers recorded fewer disruptive behaviors than the assessors.

    Conclusions:

    • Observer awareness of reliability testing directly impacts data quality and reliability.
    • Overt assessment conditions can lead to observer adaptation, potentially inflating reliability estimates.
    • Covert assessment may provide a more accurate reflection of naturalistic observational reliability but requires careful validation.