Related Experiment Videos
Reproducibility of clinical performance assessment in practice using incognito standardized patients
Simone Gorter1, Jan-Joost Rethans, Désirée van der Heijde
1Division of Rheumatology, Department of Internal Medicine, University Hospital Maastricht, 6200 MD Maastricht, The Netherlands.
Background:
The reproducibility of authentic assessment methods has been investigated for objective structured clinical examinations (OSCEs) and video assessment in general practice, but not for assessment with incognito standardized patients.
Purpose:
To investigate the reproducibility of assessment with incognito standardized patients.
Methods:
A total of 27 Dutch rheumatologists in 16 hospitals were each visited by 8 incognito standardized patients presenting with different rheumatological disorders. After each visit, the standardized patient completed a case-specific checklist containing items on medical history, physical examination and management. Over a 20-month period, 254 incognito visits took place, of which 201 were first visits. The standardized patient was detected by the rheumatologist in 2 cases only. These encounters were not included in the analysis. Generalizability theory was used to investigate the reproducibility of the assessment.
Results:
One fifth of the variance can be attributed to variation between rheumatologists. The largest variance is due to the variation in difficulty among cases. A reproducible assessment requires 3 hours of testing time (6 cases) if it is obtained through a norm-referenced interpretation of scores and 7 hours of testing time (14 cases) if it is obtained through an absolute interpretation of scores.
Conclusion:
The reproducibility of performance assessment in clinical practice by incognito standardized patients is similar to that of other authentic measurements for the assessment of clinical competence and performance.