Ensuring fair, safe, and interpretable artificial intelligence-based prediction tools in a real-world oncological...

Area of Science:

• Artificial Intelligence in Oncology
• Predictive Modeling in Healthcare
• Health Informatics

Background:

• Cancer treatment can lead to symptoms requiring emergency department (ED) visits.
• Developing predictive models can help identify high-risk patients.
• Proactive monitoring of AI models is crucial for reliable performance.

Purpose of the Study:

• To develop and validate an AI-based predictive model for 30-day ED visits in cancer patients.
• To demonstrate a proactive approach to monitoring the AI model in a real-world setting.
• To assess the model's performance and stability across different patient groups.

Main Methods:

• Utilized routinely collected electronic health record data.
• Developed a variational autoencoder k-nearest neighbors (VAE-kNN) algorithm.
• Evaluated the model using 84,138 observations from 28,369 patients over a 77-day production period.

Main Results:

• The VAE-kNN algorithm achieved an exceptional Area Under the Receiver Operating Characteristic curve (AUC) of 0.80.
• Model performance remained stable across demographic and disease groups (AUC 0.74-0.82).
• The monitoring process effectively detected data feed issues, providing insights into future performance.

Conclusions:

• The developed algorithm accurately predicts the risk of 30-day ED visits for cancer patients.
• Model outputs were confirmed to be equitable and stable over time.
• A proactive monitoring approach ensures the reliability of AI predictive models in clinical practice.