Diagnostic value of run chart analysis: using likelihood ratios to compare run chart rules on simulated data series

Area of Science:

• Healthcare Improvement Science
• Statistical Process Control
• Quality Management

Background:

• Run charts are essential tools for monitoring quality improvement in healthcare.
• Interpretation of run charts lacks standardized rules, leading to inconsistent analysis.
• Existing run chart rule sets (Anhoej, Perla, Carey) show variability in detecting non-random patterns.

Purpose of the Study:

• To evaluate and compare the diagnostic properties of three distinct sets of run chart rules.
• To assess the sensitivity and specificity of Anhoej, Perla, and Carey rules in detecting non-random variation.
• To determine the most effective rule set for identifying process shifts in healthcare data.

Main Methods:

• Utilized random data series to simulate process performance.
• Calculated likelihood ratios to quantify the diagnostic value of each rule set.
• Compared the performance of Anhoej, Perla, and Carey rules against established statistical benchmarks.

Main Results:

• The Anhoej rules exhibited strong diagnostic properties for detecting shifts in process performance.
• The Anhoej rules were found to be more sensitive and specific than the Perla and Carey rules.
• Significant differences in the ability to detect non-random variation were observed among the rule sets.

Conclusions:

• The Anhoej rules offer a reliable and effective method for interpreting run charts in healthcare improvement.
• Healthcare professionals should consider adopting the Anhoej rules for more accurate process monitoring.
• Further research may explore the application of these rules in diverse healthcare settings.