Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

On the relation between power and reliability of difference scores.

Kim May1, James B Hittner

  • 1Psychology Department, College of Charleston, SC 29424-0001, USA. mayk@cofc.edu

Perceptual and Motor Skills
|January 24, 2004
PubMed
Summary

Difference scores, used to measure change, can be unreliable because they reduce true score variance. This reduction enhances statistical power but decreases measurement reliability.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Meta-analysis of the association between gratitude and loneliness.

Applied psychology. Health and well-being·2024
Same author

Multi-Cellular Immunological Interactions Associated With COVID-19 Infections.

Frontiers in immunology·2022
Same author

Test positivity - Evaluation of a new metric to assess epidemic dispersal mediated by non-symptomatic cases.

Methods (San Diego, Calif.)·2021
Same author

Statistical methods for comparing test positivity rates between countries: Which method should be used and why?

Methods (San Diego, Calif.)·2021
Same author

Gratitude and social support mediate the association between mindfulness and mood: A cross-cultural replication study.

Journal of health psychology·2020
Same author

Early network properties of the COVID-19 pandemic - The Chinese scenario.

International journal of infectious diseases : IJID : official publication of the International Society for Infectious Diseases·2020

Area of Science:

  • Psychometrics
  • Statistical Analysis

Background:

  • Difference scores (gain scores) are commonly used to assess change.
  • Potential issues include negative correlation with initial ability and unreliability.
  • Despite drawbacks, they are useful in true experiments for reducing individual differences.

Purpose of the Study:

  • To present a straightforward argument regarding the dual nature of difference scores.
  • To explain why difference scores can be both powerful for significance testing and unreliable.
  • To elucidate the relationship between true score variance reduction and these opposing effects.

Main Methods:

  • Didactic paper presenting a theoretical argument.
  • Analysis based on the properties of difference scores in statistical models.

Related Experiment Videos

  • Comparison with previous findings (Nicewander and Price).

    • Main Results:

    • The reduction of true score variance, which increases statistical test power, also leads to decreased reliability.
    • Difference scores reduce variability due to individual differences, making them effective in experiments.
    • The numerator and denominator components of reliability coefficients are affected by true score variance.

    Conclusions:

    • The same mechanism (reduction of true score variance) that enhances the power of significance tests for difference scores also underlies their tendency towards unreliability.
    • Understanding this trade-off is crucial for appropriate use of difference scores in research.
    • Difference scores offer a powerful tool for analyzing change but require careful interpretation regarding their reliability.