Related Experiment Video
Updated: May 31, 2026
05:18
Radiation Planning Assistant - A Web-based Tool to Support High-quality Radiotherapy in Clinics with Limited Resources
Published on: October 6, 2023
Funnel plots for comparing provider performance based on patient-reported outcome measures.
Jenny Neuburger1, David A Cromwell, Andrew Hutchings
1Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, 15-17 Tavistock Place, London WC1H 9SH, UK. jenny.neuburger@lshtm.ac.uk
BMJ Quality & Safety
|June 18, 2011
Summary
Symmetric control limits on funnel plots can falsely flag healthcare providers as underperforming due to skewed patient-reported outcome measure data. Using simulated control limits improves accuracy in performance comparisons for procedures like hip replacements and varicose vein surgery.
Area of Science:
- Health Services Research
- Statistical Methods in Medicine
- Quality Improvement in Healthcare
Background:
- Patient-reported outcome measures (PROMs) often yield skewed score distributions post-intervention.
- Symmetric control limits in funnel plots may inaccurately signal poor provider performance with skewed data.
Purpose of the Study:
- To evaluate the accuracy of funnel plots using symmetric control limits for comparing healthcare provider performance.
- To assess the impact of skewed PROM data on performance indicator reliability.
Main Methods:
- Utilized PROM data from 17,453 hip replacements and 7,656 varicose vein procedures.
- Calculated symmetric 99.8% control limits (3 SDs from mean) on funnel plots.
- Compared symmetric limits to asymmetric limits derived from simulated (bootstrap) score distributions.
Main Results:
- Simulated control limits were asymmetric for both procedures.
- The empirical rate of false alarms for poor performance was 0.2-0.3% with symmetric limits, exceeding the 0.1% target.
- Provider classification differed for 1.3% of hip replacement providers and 0.6% of varicose vein providers.
Conclusions:
- The use of symmetric control limits can lead to misclassification of provider performance.
- Simulated control limits are recommended for funnel plots when analyzing skewed PROM data.
- Consideration of simulated limits enhances the reliability of healthcare performance comparisons.
Related Concept Videos
01:15Receiver Operating Characteristic Plot
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
01:13Patient-centered Care
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
01:11Clinical Trials: Overview
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
01:20Comparing the Survival Analysis of Two or More Groups
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and Cox...
01:11Guidelines for Writing Outcome
When developing expected outcomes for a patient care plan, the nurse should adhere to the following recommendations:
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care evaluation by...
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care evaluation by...
01:16Clinical Trials
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...