Validity Evidence for a Transport Medical Control Assessment Tool Using Simulated Interfacility Transport Calls

Pediatric subspecialty fellows often serve as transport medical control (TMC) for interfacility transports, but assessment of fellow TMC performance is not standardized. In this study, we sought to generate validity evidence for a TMC assessment tool using simulated transport calls.

We conducted a cross-sectional study among pediatric subspecialty attendings in critical care medicine, emergency medicine, neonatology, and hospital medicine who supervise fellows serving as TMC. We used Messick's framework to evaluate for validity evidence of content, response process, internal structure, and relationship to other variables. Participants listened to simulated transport calls applicable to their subspecialty and completed the 21-item TMC assessment tool, a supervision level scale, and a survey on the relevance, ease, and clarity of the TMC assessment tool.

A total of 101 faculty from 54 institutions participated in the study. Nearly all participants (99%, 99/101) indicated that the TMC assessment tool captured the relevant skills and knowledge required for TMC very or somewhat well. Reliability of the TMC assessment tool was high, with a Cronbach's alpha of 0.86 (95% CI 0.84-0.87). The TMC assessment tool also demonstrated outstanding ability to predict if a fellow could perform the role of TMC independently with an area under the curve of 0.91.

This study provides multiple sources of evidence for validity of a TMC assessment tool using simulated interfacility transport calls. This tool could be implemented by pediatric subspeciality fellowship programs to improve TMC training and assessment of fellow performance in the role of TMC.