Audit method suited for DSS in clinical environment.

Area of Science:

• Medical Informatics
• Bayesian Statistics
• Machine Learning Evaluation

Background:

• Predictive models are increasingly used in clinical Decision Support Systems (DSSs).
• Auditing these models is crucial for ensuring reliability in clinical and research settings.
• Existing methods may not fully integrate with clinical workflows or detect real-world performance drifts.

Purpose of the Study:

• To present a novel online Bayesian auditing method for predictive models within DSSs.
• To evaluate and compare the predictive performance of models based on clinical diagnoses.
• To adapt auditing to the specific decision workflow of clinical environments.

Main Methods:

• A Bayesian auditing model calculating posterior odds by combining prior, static, and dynamic odds.
• Estimation of posterior odds using historical and real-time case data from DSS deployment.
• Integration with class ontologies to audit related diagnostic questions.

Main Results:

• The proposed method effectively audits predictive models in a clinical context.
• It identifies the most suitable classifier for individual cases.
• Demonstrated efficacy in brain tumor diagnosis using Magnetic Resonance Spectroscopy (MRS) data.

Conclusions:

• The novel audit model is adapted for clinical decision workflows.
• It enhances decision support by selecting optimal classifiers and detecting model misbehavior.
• This approach improves the reliability and adaptability of predictive models in healthcare.