Related Experiment Videos
How a regression artifact makes ICFs/MR look ineffective
F M Crinella1, R McCleary, J M Swanson
1fmcrinel@uci.edu
Mental Retardation
|June 25, 1998
Summary
A study re-analyzing Conroy's (1996) research found that negative quality-of-life impacts attributed to small Intermediate Care Facilities (ICFs) were likely a statistical artifact. This regression artifact resulted from flawed control group selection based on pretest matching.
Area of Science:
- Psychology
- Sociology
- Public Policy
Background:
- Conroy's (1996) study suggested negative quality-of-life impacts for individuals with mental retardation in small Intermediate Care Facilities (ICFs).
- The original study employed a matched-control group quasi-experimental design.
Discussion:
- This analysis reveals Conroy's findings are likely a regression artifact due to selecting control groups based on pretest scores.
- Matching on pretest scores, unlike static characteristics, invariably introduces large, spurious effects.
- Historical literature dating back a century warns against this methodological flaw.
Key Insights:
- The perceived negative quality-of-life impact was an artifact of the research design, not an effect of ICF placement.
- Monte Carlo simulations confirm the magnitude of this artifact is substantial.
- Methodological rigor in quasi-experimental designs, particularly regarding control group selection, is crucial for valid conclusions.
Outlook:
- Re-evaluation of past research utilizing pretest matching is warranted.
- This highlights the need for careful consideration of statistical artifacts in policy-related research.
- Future studies should employ robust designs to avoid such biases and ensure accurate policy recommendations.