Content Evidence for Validity of Time-to-Task Initiation: A Novel Measure of Learner Competence

Educators require validated tools to assess learner competency in simulation-based mastery learning. We previously created a simulation to incorporate crisis resource management skills after a needs assessment of our pediatric residency program. We present the development of and content evidence for validity of a task list with time-to-task initiation designed to aid in rapid cycle deliberate practice (RCDP) debriefing and assessment of pediatrics resident learners.

Five board-certified pediatricians developed the initial task list with 30 tasks. We used the Delphi technique to seek content evidence for validity of the initial task list using 17 board-certified pediatricians at 7 institutions across the United States. After finalizing the task list, we video recorded 3 pediatric experts performing the tasks in a simulation. We then used the Delphi technique to establish and to attain agreement on time-to-task initiation. We calculated Cronbach α coefficient to measure internal consistency of the expert responses. After finalizing the task list, we divided it into 3 stages with 5 separate cycles per stage to direct the educator in performance of RCDP debriefing.

The group reached 80% agreement after 2 Delphi rounds with a Cronbach α coefficient of 0.95. The resultant list included 25 tasks. The group subsequently achieved 80% agreement on the time-to-task initiation in 2 Delphi rounds.

We present content evidence for validity of a task list with associated time-to-task initiation for a simulation scenario that incorporates crisis resource management skills and is designed to aid educators in RCDP debriefing. This task list may decrease intereducator inconsistencies in delivery of RCDP for a common pediatric resuscitation scenario.