Development of a novel pediatric point-of-care ultrasound question bank using a modified Delphi process

Measuring pediatric emergency medicine (PEM) fellow competency in point-of-care ultrasound (POCUS) is important for ensuring adequate training and performance. Assessment may include direct observation, image review, quality assessment, and written examination. The purpose of this study was to develop a pediatric POCUS question bank that could subsequently be used as a POCUS assessment for graduating PEM fellows.

We organized a 10-person question writing group (QWG). Eight hold expertise in POCUS and two hold expertise in medical education. Members of the QWG created questions within four domains: interpretation/diagnosis (50% of questions), anatomy (30%), physics (10%), and pitfalls (10%). POCUS faculty ascertained content validity and the medical education faculty revised questions for syntax and readability. In 2016, we recruited 31 pediatric POCUS experts. The majority were members of the P2 Network, an international group of experts and leaders in PEM POCUS, to participate in three iterative rounds of a modified Delphi process to review, revise, and establish consensus on the question bank.

Thirty-one pediatric POCUS experts participated in the three rounds of the modified Delphi process and evaluated 437 questions developed by the expert panel. Forty-nine percent (n = 216) of the questions were accepted in round 1, 30% (n = 130) in round 2, and 11% (n = 47) in the final round. The final question bank included 393 questions covering 17 pediatric POCUS applications.

We developed a 393-question bank to aid in the assessment of PEM POCUS competency. Future work includes piloting the questions with PEM fellows to evaluate the response process and implementing the assessment tool to establish a minimum passing score.