Development of a Multiple-Choice Test for Novice Anesthesia Residents to Evaluate Knowledge Related to Management of...

Teaching trainees the knowledge and skills to perform general anesthesia (GA) for cesarean delivery (CD) requires innovative strategies, as they may never manage such cases in training. We used a multistage design process to create a criterion-referenced multiple-choice test as an assessment tool to evaluate CA1's knowledge related to this scenario.

Three faculty created 33 questions, categorized as: (1) physiologic changes of pregnancy (PCP), (2) pharmacology (PHA), (3) anesthetic implications of pregnancy (AIP), and (4) crisis resource management principles (CRM). A Delphi process (3 rounds) provided content validation. In round 1, experts (n = 15) ranked questions on a 7-point Likert scale. Questions ranked ≥ 5 in importance by ≥ 70% of experts were retained. Five questions were eliminated, several were revised, and 1 added. In round 2, consensus (N = 14) was reached in all except 7 questions. In round 3 (N = 14), all questions stabilized. A pilot test of the 29-question instrument evaluating internal consistency, reliability, convergent validity, and item analysis was conducted with the July CA1 classes at our institution after a lecture on GA for CD (n = 26, "instructed group") and another institution with no lecture (n = 26, "uninstructed group"), CA2s (N = 17), and attendings (N = 10).

Acceptable internal consistency and reliability was demonstrated (ρ = 0.67). Convergent validity coefficients between the CA1 uninstructed and instructed group suggested theoretical meaningfulness of the 4 sub-scales: PCP correlated at 0.29 with PHA, 0.35 with CRM, and 0.25 with AIP. PHA correlated with CRM and AIP at 0.23 and 0.28, respectively. The correlation between CRM and AIP was 0.29.

The test produces moderately reliable scores to assess CA1s' knowledge related to GA for urgent CD.