Comparing methodological approaches for outlier detection: a cross-sectional study of older persons in 2,744...

Detection of outliers is a common feature of quality monitoring programs, with the goal of identifying unwarranted variation in quality of care. However, there remains limited agreement on the most appropriate methods for detecting outliers, in particular in long-term care facilities. This study aimed to compare methods for outlier detection among long-term care facilities and assess the suitability of each method for use in monitoring quality and safety of care.

A cross-sectional population-based study of Australian long-term care facilities during 2019 was conducted using the Registry of Senior Australians National Historical Cohort. Outlier detection methods differing in control limits, handling of overdispersion, and use of multiple testing adjustments identified outlier long-term care facilities on four quality indicators (emergency department presentations, antipsychotic use, fall-related hospitalizations, and pressure injury-related hospitalizations). The number of outliers identified using each method and the agreement between methods was examined. Simulation studies were undertaken to assess reasons for differences between methods.

A total of 234,890 persons (241,925 person-episodes) at 2,744 long-term care facilities were included. The indicator with the highest event rate was emergency department presentations (37.9% of person-episodes), followed by antipsychotic use (21.5%), fall-related hospitalizations (13.6%), and pressure injury-related hospitalizations (3.4%). Most indicators exhibited some overdispersion. Differences between confidence interval based and prediction interval-based methods were largest for low event rate indicators, where the number of outliers with higher-than-expected event rates identified differed by up to a factor of four. Methods that did not address overdispersion had high false positive rates.

These results indicate that the number of long-term care facility outliers identified can vary widely between methods. Quality monitoring programs in long-term care should use appropriate prediction interval-based methods for constructing control limits and take steps to address overdispersion.