Validation and Reliability of an Assessment Tool for Tracheal Intubation Skills Among Anesthesiology Residents: A...

Although junior residents at this hospital are required to receive anesthesiology training, there are no standards for evaluating tracheal intubation skills, and the training is currently at the discretion of each instructor. Therefore, as peri-anesthesia nurses, we have been involved in airway management education and collaborated with the anesthesiology department to develop skills standards. The purpose of this study was to improve airway management education and the quality of medical safety by examining the reliability and validity of the assessment chart.

This was a prospective observational study in which peri-anesthesia nurses and anesthesiologists developed an assessment chart to measure tracheal intubation skills. The study was conducted at a simulation center from August to September 2021, with five volunteer junior residents. After the first day of training, a series of actions from tracheal intubation to ventilator connection were video recorded on a simulator and scored by an expert physician using an evaluation chart and analyzed for reliability and validity.

Reliability confirmed internal consistency (Cronbach's α=0.8), and intra-class correlation coefficients (ICCs) had strong correlations within the mean measures (ICC (2, 3) =0.78 (CI: 0.12-0.97)), indicating inter-rater reliability. Validity was compared with the standard score of 52 out of 100, defined for residents, and internal validity was confirmed, as all measures had strong correlations (r=0.9). In contrast, correlations between trauma avoidance items were poor (r=0.35, ns).

Validation of the reliability of the skills assessment chart revealed that it measured the competency of tracheal intubation of junior residents. For the items that had poor correlations, instruction by senior physicians and peri-anesthesia nurses, in collaboration, was strengthened to attain the standard. The plan was to create an educational environment in which residents, as members of the healthcare team, contribute to patient quality of life and medical safety.