Related Experiment Video
Updated: Jun 6, 2026
07:37
Assessment and Communication for People with Disorders of Consciousness
Published on: August 1, 2017
CUSUM analysis to assess competence: what failure rate is acceptable?
Jai Sivaprakasam1, Makani Purva
1Department of Anaesthesia, Hull and East Yorkshire Hospitals NHS Trust, Hull, UK. jsivaprakasam@doctors.org.uk
The Clinical Teacher
|December 8, 2010
Summary
Cumulative sum (CUSUM) analysis effectively assesses experienced trainees’ competency in procedures like labor epidurals. However, varying acceptable failure rates can yield inconsistent results, highlighting the need for standardized guidelines.
Area of Science:
- Medical Education
- Anesthesiology
- Clinical Skills Assessment
Background:
- Cumulative sum (CUSUM) analysis is a statistical tool for assessing competency in practical procedures.
- Previous CUSUM studies primarily focused on novice trainees for labor epidural administration.
- Variability exists in acceptable failure rates used for competency assessments.
Purpose of the Study:
- To evaluate the utility of CUSUM analysis for the continued competency assessment of experienced trainees.
- To investigate the impact of different acceptable failure rates on CUSUM-based competency conclusions.
- To test the hypothesis that varying failure rates lead to different competency assessments.
Main Methods:
- Collected data on the first 50 labor epidurals performed by specialty training (ST) 3/4 level trainees.
- Constructed two CUSUM graphs per trainee using distinct acceptable and unacceptable failure rates (10%/20% and 15%/30%).
- Analyzed CUSUM data to determine demonstrated competency based on the chosen failure rate thresholds.
Main Results:
- Using 10% acceptable and 20% unacceptable failure rates, only 4 of 6 trainees demonstrated competence.
- With 15% acceptable and 30% unacceptable failure rates, 5 of 6 trainees demonstrated competence.
- The number of attempts required for competency varied significantly based on the failure rate settings.
Conclusions:
- CUSUM analysis is a viable method for the ongoing assessment of competence in experienced trainees.
- Different failure rate settings can lead to divergent conclusions regarding trainee competency, causing inconsistency.
- Standardized guidance on acceptable failure rates for procedures like labor epidurals is needed from training organizations.
Related Concept Videos
01:24Confidence Coefficient
The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under both the...
01:31Critical Values
A critical value is a definite value obtained from a particular probability distribution at a predecided confidence level (or a predecided significance level) for a given population parameter. The critical value provides demarcation that separates the sample statistics that are likely to occur from the ones that are unlikely to occur based on the given probability distribution and the population parameter to be estimated. The critical value for normal distribution is obtained from the z...
01:13Accuracy and Errors in Hypothesis Testing
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...
01:18Finding Critical Values for Chi-Square
Consider a curve representing sample data drawn randomly from a normally distributed population. One must construct confidence intervals to estimate or to test a claim regarding the population standard deviation. For example, a 95% confidence interval covers 95% of the area under the curve, and the remaining 5% is equally distributed on either side of the curve. To achieve such confidence intervals, one must determine the critical values. The critical values are simply the values separating the...
01:19Expected Frequencies in Goodness-of-Fit Tests
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
01:00Detection of Gross Error: The Q Test
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...