Related Experiment Videos
Validity threats: overcoming interference with proposed interpretations of assessment data
Steven M Downing1, Thomas M Haladyna
1University of Illinois at Chicago, College of Medicine, Department of Medical Education, Chicago, Illinois 60612-7309, USA. sdowning@uic.edu
Medical Education
|March 5, 2004
Summary
Validity in medical education assessments is threatened by construct under-representation (CU) and construct-irrelevant variance (CIV). Addressing these threats is crucial for accurate score interpretation and ensuring assessment quality.
Area of Science:
- Medical Education
- Assessment Validity
- Psychometrics
Background:
- Meaningful interpretation of assessment scores requires sound scientific evidence of validity.
- Factors interfering with score interpretation threaten assessment validity.
Purpose of the Study:
- To discuss two major threats to assessment validity: construct under-representation (CU) and construct-irrelevant variance (CIV).
- To provide examples of CU and CIV in written, performance, and clinical performance examinations.
Main Methods:
- Literature review and conceptual analysis of validity threats in medical education assessments.
- Illustrative examples provided for different assessment formats.
Main Results:
- Construct under-representation (CU) arises from inadequate sampling of the content domain.
- Construct-irrelevant variance (CIV) includes systematic errors like flawed test items, inappropriate difficulty, or rater bias.
- Face validity is deemed not a legitimate form of validity evidence.
Conclusions:
- Multiple threats to validity exist across all medical education assessment types.
- Methods for eliminating or controlling these validity threats are proposed.
- Ensuring validity is essential for accurate and meaningful assessment in medical education.