Quantifying random variability in decision-making in pediatric cardiac surgery

Random variability in day-to-day decision-making referred to as "noise" is associated with variation that negatively affects both the reproducibility and quality of decision-making. Although well described in other fields, the prevalence and significance of noise in medical decision-making are understudied and largely unknown. The goal of this study was to quantify noise in medical decision-making using a noise audit.

A noise audit was completed by 71 (n = 71) Heart Centre staff at the Hospital for Sick Children and Seattle Children's Hospital and involved a series of cases and questions surrounding decisions commonly encountered in the care for patients with transposition of the great arteries and critical aortic stenosis. Entropy was used to quantify total variation in responses. Because absolute entropy was not immediately comparable across audit questions with a different number of prespecified options, we reported a standardized version of entropy, which was calculated by dividing the absolute entropy by its theoretical maximum for each case. To compare responses in easy to hard questions across years of experience (<10 years and >10 years) and role, we reported aggregate entropy ratios. Aggregate entropy ratios were calculated by first stratifying by group of comparison and then calculating the standardized entropy for each question and taking the average of standardized entropies for easy questions and hard questions. Finally, to determine the ratio (easy to hard), we divided the average standardized entropy of the easy questions by the average standardized entropy of the hard questions.

The overall audit aggregate entropy ratio was 0.85 less than 1, indicating lower-complexity questions had less variation than higher-complexity questions. The aggregate entropy ratio for those with more than 10 years of experience was 0.8 and 0.87, respectively, for those with less than 10 years of experience. The aggregate entropy ratios for those in cardiac critical care, cardiology, and cardiovascular surgery were 0.85, 0.83, and 0.96, respectively.

Noise was pervasive in medical decision-making in the analysis of our responses to common medical decisions made for patients with congenital heart disease and can be quantified in a manner that facilitates comparisons.