Related Experiment Video
Updated: Jul 18, 2026
05:04
Using Simulation Models to Train Clinicians in the Use of Point-of-Care Ultrasound
Published on: August 9, 2024
In-training evaluation using hand-held computerized clinical work sampling strategies in radiology residency
Karen Finlay1, Geoff R Norman, Harald Stolberg
1Diagnostic Radiology, McMaster University, 1200 Main Street West, Hamilton, ON L8N 3Z5. finlay@hhsc.ca
Summary
Clinical work sampling (CWS) is feasible for radiology residency evaluation, but requires at least 20 entries for reliability. This method offers timely feedback but may not fully replace traditional in-training evaluations.
Area of Science:
- Medical Education
- Radiology Residency Training
- Performance Evaluation
Background:
- Traditional in-training evaluations are the standard in Canadian residency programs.
- Current evaluation methods may lack sufficient detail for summative assessment.
Purpose of the Study:
- To investigate the feasibility of a clinical work sampling (CWS) approach for evaluating radiology residents.
- To assess the reliability and validity of CWS compared to standard evaluation methods.
Main Methods:
- A 1-year study involving 14 radiology residents across 4 postgraduate years.
- Residents used handheld devices to record CWS data on 9 performance categories.
- CWS results were compared with in-training evaluation reports (ITERs) and an objective evaluation tool.
Main Results:
- CWS achieved a reliability of 0.62 with 7 forms per resident, requiring 20 forms for 0.80 reliability.
- Internal consistency for ITERs was 0.98, but inter-rotation correlation was low (0.36).
- Correlation between CWS and ITERs was 0.47 (P=0.09), with neither correlating with the American College of Radiology evaluation.
Conclusions:
- Clinical work sampling is a feasible evaluation strategy for radiology residency.
- Voluntary compliance with CWS was lower than expected, with reasons unclear.
- While CWS offers potential for timely feedback, its reliability and resource demands require careful consideration for summative evaluation.
