Related Experiment Videos
Validity of scores generated by a web-based multimedia simulated patient case software: a pilot study
Anthony F Jerant1, Rahman Azari
1Department of Family and Community Medicine, University of California Davis School of Medicine, 4860 Y Street, Suite 2300, Sacramento, CA 95817, USA. afjerant@ucdavis.edu
Summary
Automated scores from multimedia simulated patient cases (MSPCs) may not accurately reflect medical students' diagnostic reasoning skills. Further validation is needed before using MSPCs as reliable educational tools.
Area of Science:
- Medical Education
- Health Professions Education
- Diagnostic Reasoning
Background:
- The educational utility of multimedia simulated patient cases (MSPCs) is not well-established.
- Assessing diagnostic reasoning is crucial for medical training.
Purpose of the Study:
- To evaluate the validity of automated scores for diagnostic reasoning ability from DxR Clinician, a web-based MSPC software.
- To determine if automated scores correlate with a validated measure of diagnostic reasoning sophistication.
Main Methods:
- Eighty-nine third-year medical students completed four MSPCs.
- Correlations were assessed between MSPC scores (Clinical Reasoning Score [CRS] and Level of Diagnostic Performance [LDP]) and the Diagnostic Thinking Inventory (DTI).
Main Results:
- Instructor overrides were needed for 58% of automated scores.
- While DTI scores improved significantly over the year, no significant correlations were found between MSPC scores (CRS/LDP) and DTI scores.
Conclusions:
- Automated diagnostic reasoning scores from this MSPC software lack criterion validity.
- The validity of automated scores from MSPCs requires rigorous establishment before their use as educational tools.