A Medical Record Peer-Review System to Evaluate Residents' Clinical Competence: Criterion Validity Analysis

Area of Science:

• Medical Education
• Health Professions Education
• Quality Improvement

Background:

• Established methods for evaluating medical education outcomes (post-graduation performance) are lacking.
• Outcome evaluation is crucial for ensuring patient care quality.
• A peer review system for medical records is being developed to address this gap.

Purpose of the Study:

• To pilot test and evaluate the criterion validity of a novel medical record peer review system.
• To compare the system's assessments with evaluations by program directors (criterion standard).

Main Methods:

• A pilot study involving 13 senior residents from three teaching hospitals.
• Independent evaluation of 5 patient records per resident by 5 external reviewers using a 15-item sheet.
• Independent evaluation of resident clinical performance by program directors using a 10-item sheet.
• Pearson and bootstrap correlation analyses to assess validity.

Main Results:

• Statistically significant correlations found between reviewer and program director assessments for clinical reasoning (r = 0.5848) and history taking (r = 0.509).
• Lower correlations observed for physical examination and patient attitude assessments.
• The study provides initial evidence for the criterion validity of the medical record peer review system.

Conclusions:

• The developed medical record peer review system demonstrates potential for outcome evaluation in medical education.
• Further refinement and validation are needed, especially for assessing physical examination and interpersonal skills.
• This study represents a novel approach to assessing post-graduation physician performance through medical record review.