A weakly supervised model for the automated detection of adverse events using clinical notes

Area of Science:

• Health Informatics
• Machine Learning
• Pharmacovigilance

Background:

• Clinical trials have limitations in detecting all adverse drug reactions and medical device issues before market release.
• Electronic health records (EHR) offer valuable patient data for post-market surveillance, with free-text clinical notes providing richer detail than structured data.
• Existing machine learning methods for adverse event detection from clinical notes often rely on manual feature extraction and expensive labeled data, limiting scalability and validation.

Purpose of the Study:

• To develop and evaluate a weakly-supervised machine learning framework for detecting adverse events from unstructured clinical notes.
• To address limitations of previous methods, including manual feature engineering, reliance on costly hand-labeled data, and lack of external validation.
• To demonstrate the framework's effectiveness using insulin pump failure as a case study for post-market surveillance.

Main Methods:

• Developed a weakly-supervised machine learning framework for adverse event detection.
• Utilized unstructured clinical notes from electronic health records (EHR).
• Evaluated the model on detecting insulin pump failure, including validation on an external dataset.

Main Results:

• The developed model achieved a PR AUC of 0.842 on the holdout test set for detecting insulin pump failure.
• External validation demonstrated the model's robustness with a PR AUC of 0.815.
• The weakly-supervised approach significantly reduced the need for extensive hand-labeled data.

Conclusions:

• Weakly-supervised machine learning offers a scalable and cost-effective approach for adverse event detection from clinical notes.
• The framework demonstrates high accuracy in identifying specific adverse events like insulin pump failure.
• This method can be readily adapted for broader post-market surveillance of drugs and medical devices to enhance patient safety.