Choosing an Appropriate Outlier-Detection Method When Comparing Institutional Performances in Clinical Audits

Clinical audits are essential for monitoring and improving the quality of healthcare with statistical outlier detection used to identify healthcare units whose performance deviates from expected standards. Two commonly used outlier-detection approaches are Common Mean Model (CMM) and Random Effects Logistic Regression (RELR). However, these methods can identify different healthcare units as outliers when applied to the same dataset. Incorrectly flagging a unit as an outlier can have serious consequences, including reputational harm, decreased staff morale, unnecessary regulatory scrutiny, diversion of resources, and loss of public trust. This article investigates the reasons for these discrepancies through a simulation study evaluating the performance of the CMM and RELR under a range of scenarios. The results show that differences in outlier detection are primarily driven by the degree to which a normality assumption holds for each method. When these assumptions are violated, for one or both methods, the two approaches may produce markedly different outlier detection. To choose the most appropriate approach for a given dataset, we recommend assessing each normality assumption, by combining visual diagnostics, such as QQ plots, with quantitative measures, such as Lin's concordance correlation coefficient (CCC).