Related Experiment Video
Updated: May 14, 2026
10:32
Development of a Virtual Reality Assessment of Everyday Living Skills
Published on: April 23, 2014
Comparing announced with unannounced standardized patients in performance assessment
Alan Schwartz1, Saul J Weiner, Amy Binns-Calvey
1Department of Medical Education, University of Illinois at Chicago, USA. alansz@uic.edu
Joint Commission Journal on Quality and Patient Safety
|February 23, 2013
Summary
Physician performance assessment using unannounced standardized patients (USPs) revealed attending physicians underperformed medical students in information gathering and care planning. Direct observation by USPs offers insights but requires careful consideration of costs and benefits.
Area of Science:
- Medical Education
- Clinical Skills Assessment
- Healthcare Quality Improvement
Background:
- Assessing physician performance in practice is a significant psychometric challenge.
- Current methods like chart reviews and patient surveys fail to identify overlooked information, potentially leading to diagnostic errors.
- Standardized patient (SP) assessments, including unannounced standardized patients (USPs), offer direct observation of clinical behaviors.
Purpose of the Study:
- To compare the clinical performance of attending physicians assessed by USPs versus medical students assessed by SPs.
- To evaluate the effectiveness of USPs in identifying physician performance gaps, particularly in adapting care to patient context.
Main Methods:
- Study 1: 65 board-certified internists assessed by USPs.
- Study 2: 59 fourth-year medical students assessed by SPs in a clinical performance center.
- Utilized identical patient cases and actors across both studies for direct comparison.
Main Results:
- Attending physicians assessed by USPs significantly underperformed medical students assessed by SPs.
- Deficiencies were noted in probing biomedical and contextual red flags (ORs 0.45 and 0.66, respectively).
- Attending physicians also showed deficits in planning appropriate care (OR = 0.43).
Conclusions:
- Attending physicians demonstrated lower performance than medical students on identical clinical assessment outcomes.
- Direct observation using USPs may be beneficial for assessing information elicitation and care individualization.
- The benefits and costs of using USPs for performance assessment should be carefully weighed.
