Using Bayesian event probabilities for monitoring clinical quality assurance

Current systems used to monitor clinical quality metrics are expensive and use, not used outdated statistical methodologies. The aim of this study was to develop and test a Bayesian approach for continuous monitoring of clinical outcomes using information easily obtained from the electronic medical record.

An algorithm was developed with the assistance of artificial intelligence to extract surgical outcomes data from electronic medical records and to then calculate the posterior probability distribution for complication occurrence. A test data set was created from a retrospective analysis of the complete medical records of patients receiving surgical care at UCLA Health between 1 January 2016 and 22 September 2024. The 20 most performed inpatient colorectal operations were considered the index cases for the 'major colorectal procedures' group. Complications were identified by diagnostic codes in the medical record and attributed to the index procedure if diagnosed within 1 year after surgery. Bayesian regression was used to determine the posterior probability of complication occurrence for comparison with pre-established acceptability thresholds. A > 50% probability of complication occurrence exceeding the threshold would trigger a quality assurance review for that complication during the period of interest.

There were 747 patients who underwent major colorectal operations. Postoperative ileus (7.4%) and anastomotic leak (3.9%) were the most common complications, whereas postoperative bleeding was rare (0.3%). Of the ten complications tracked, the number triggered for chart review ranged from a low of two in 2016 to six in 2022. The largest number of excess complications triggering review was three in any year.

A system for real-time monitoring of complications was modelled from retrospective data. It successfully identified time periods in which the probability distribution for experiencing complications exceeded pre-established thresholds, identifying a need for chart review. A Bayesian event probability tracker will be inexpensive to implement, could track clinical outcomes in near real-time (facilitating data review whenever it is desired), and is statistically valid.