Competency Standard Derivation for Point-of-Care Ultrasound Image Interpretation for Emergency Physicians

Because number-based standards are increasingly controversial, the objective of this study was to derive a performance-based competency standard for the image interpretation task of point-of-care ultrasound (POCUS).

This was a prospective study. Operating on a clinically-relevant sample of POCUS images, we adapted the Ebel standard-setting method to derive a performance benchmark in 4 diverse pediatric POCUS applications: soft tissue, lung, cardiac and focused assessment with sonography in trauma (FAST). In Phase I (difficulty calibration), cases were categorized into interpretation difficulty terciles (easy, intermediate, hard) using emergency physician-derived data. In Phase II (significance), a 4-person expert panel categorized cases as low, medium, or high clinical significance. In Phase III (standard setting), a 3x3 matrix was created, categorizing cases by difficulty and significance, and a 6-member panel determined acceptable accuracy for each of the 9 cells. An overall competency standard was derived from the weighted sum.

We obtained data from 379 emergency physicians resulting in 67,093 interpretations and a median of 184 (interquartile range, 154, 190) interpretations per case. There were 78 (19.5%) easy, 272 (68.0%) medium, and 50 (12.5%) hard-to-interpret cases, and 237 (59.3%) low, 65 (16.3%) medium, and 98 (24.5%) cases of high clinical significance across the 4 POCUS applications. The panel determined an overall performance-based competency score of 85.0% for lung, 89.5% for cardiac, 90.5% for soft tissue, and 92.7% for FAST.

This research provides a transparent chain of evidence that derived clinically relevant competency standards for POCUS image interpretation.