Ensuring basic competency in chest tube insertion using a simulated scenario: an international validation study

Chest tube insertion can be associated with serious complications. A structured training programme is needed to minimise complications and enhance patient safety. Novices should pass a reliable test with solid evidence of validity before performing the procedure supervised on patients. The aim of this study was to establish a credible pass/fail standard.

We used an established assessment tool the Chest Tube Insertion Competency Test (TUBE-iCOMPT). Validity evidence was explored according to Messick's five sources of validity. Two methods were used to establish a credible pass/fail standard. Contrasting groups' method: 34 doctors (23 novices and 11 experienced surgeons) performed the procedure twice and all procedures were video recorded, edited, blinded and rated by two independent, international raters. Modified Angoff method: seven thoracic surgeons individually determined the scores that defined the pass/fail criteria. The data was gathered in Copenhagen, Denmark and Riyadh, Saudi Arabia.

Internal consistency reliability was calculated as Cronbach's alpha to 0.94. The generalisability coefficient with two raters and two procedures was 0.91. Mean scores were 50.7 (SD±13.2) and 74.7 (SD±4.8) for novices and experienced surgeons, respectively (p<0.001). The pass/fail score of 62 points resulted in zero false negatives and only three false positives.

We have gathered valuable additional validity evidence for the assessment tool TUBE-iCOMPT including establishment of a credible pass/fail score. The TUBE-iCOMPT can now be integrated in mastery learning programmes to ensure competency before independent practice.