Supervised Machine Learning-Based Decision Support for Signal Validation Classification.

Area of Science:

• Pharmacovigilance and Drug Safety
• Machine Learning in Healthcare
• Regulatory Science

Background:

• Signal validation is crucial in pharmacovigilance to assess potential drug safety signals.
• Safety experts currently review signals of disproportionate reporting (SDRs) and categorize them manually.
• Existing processes require efficient and consistent methods for signal assessment.

Purpose of the Study:

• To evaluate the effectiveness of predictive machine learning (ML) models in supporting safety experts' decision-making for signal validation.
• To accurately classify signals of disproportionate reporting (SDRs) into predefined categories using ML.
• To enhance the efficiency and consistency of the signal validation process.

Main Methods:

• Trained a Gradient Boosted Trees multiclass classifier using historical SDR validation data and Individual Case Safety Reports.
• Employed SHapley Additive exPlanations (SHAP) to interpret the ML model's predictions and identify key features.
• Conducted a prospective experiment to assess model accuracy and user acceptance in a real-world setting.

Main Results:

• The ML model achieved a prediction accuracy of 83-86% over a 3-month period for six medicinal products.
• Safety experts confirmed the model's applicability and found its systematic predictions valuable.
• The model assisted experts in reviewing SDRs more efficiently and consistently.

Conclusions:

• A multiclass classification model can accurately predict signal validation categories for SDRs in pharmacovigilance.
• The transparency provided by SHAP analysis significantly contributed to high acceptance among safety experts.
• ML models, when interpretable, can effectively support and improve drug safety signal validation processes.