The use of control charts by laypeople and hospital decision-makers for guiding decision making

Graphs presenting healthcare data are increasingly available to support laypeople and hospital staff's decision making. When making these decisions, hospital staff should consider the role of chance-that is, random variation. Given random variation, decision-makers must distinguish signals (sometimes called special-cause data) from noise (common-cause data). Unfortunately, many graphs do not facilitate the statistical reasoning necessary to make such distinctions. Control charts are a less commonly used type of graph that support statistical thinking by including reference lines that separate data more likely to be signals from those more likely to be noise. The current work demonstrates for whom (laypeople and hospital staff) and when (treatment and investigative decisions) control charts strengthen data-driven decision making. We present two experiments that compare people's use of control and non-control charts to make decisions between hospitals (funnel charts vs. league tables) and to monitor changes across time (run charts with control lines vs. run charts without control lines). As expected, participants more accurately identified the outlying data using a control chart than using a non-control chart, but their ability to then apply that information to more complicated questions (e.g., where should I go for treatment?, and should I investigate?) was limited. The discussion highlights some common concerns about using control charts in hospital settings.