The Accuracy of the Learning-Curve Cumulative Sum Method in Assessing Brachial Plexus Block Competency.

Area of Science:

• Medical Education
• Anesthesiology
• Statistical Methods in Medicine

Background:

• The learning-curve cumulative sum (LC-CUSUM) and risk-adjusted LC-CUSUM (RA-LC-CUSUM) methods are used to monitor procedural skill competency.
• Limited data exist on the accuracy of these LC-CUSUM methods.

Purpose of the Study:

• To compare the accuracy of LC-CUSUM and RA-LC-CUSUM in assessing resident competency.
• To evaluate these methods using historical data from brachial plexus blocks (BPBs).

Main Methods:

• Analyzed 1713 BPBs performed by 32 anesthesia residents.
• Constructed learning curves using LC-CUSUM and RA-LC-CUSUM.
• Defined competency as a maximum 15% cumulative failure rate.

Main Results:

• LC-CUSUM identified competency in 84.6% of residents; RA-LC-CUSUM identified competency in 76.9% (P = .001).
• Median failure rates at competency were similar (6.5% for both methods; P = .37).
• Sensitivity and specificity were comparable between LC-CUSUM and RA-LC-CUSUM, with substantial error rates.

Conclusions:

• Both LC-CUSUM and RA-LC-CUSUM methods exhibited significant false-positive and false-negative rates.
• The observed accuracy measures suggest potential inaccuracies for high-stakes decisions regarding resident competency in BPBs.