Warfarin Dose Management Using Offline Deep Reinforcement Learning

Area of Science:

• Pharmacology and Clinical Pharmacy
• Artificial Intelligence in Medicine
• Biomedical Data Science

Background:

• Warfarin, a common anticoagulant, necessitates precise dosing due to its narrow therapeutic index and requires intensive monitoring.
• Existing methods for warfarin dose management often rely on time-series supervised learning, which may not capture optimal strategies from complex clinical data.

Purpose of the Study:

• To develop a standardized optimal warfarin dose decision support system.
• To leverage machine learning, specifically reinforcement learning, for predicting cumulative warfarin doses based on time-series anticoagulation data and patient demographics.

Main Methods:

• An offline reinforcement learning (RL) model utilizing the Batch-Constrained Q-Learning (BCQ) algorithm was developed for discrete action settings.
• The model predicts cumulative warfarin doses until the next International Normalized Ratio (INR) test.
• Performance was evaluated by comparing predicted doses against physician-prescribed doses and against a Long Short-Term Memory (LSTM) baseline model.

Main Results:

• The BCQ model demonstrated a prediction accuracy of 98.6%, substantially outperforming the baseline LSTM model's accuracy of 71.09%.
• Qualitative evaluations confirmed the model's ability to appropriately adjust warfarin dosage during periods of out-of-range INR values.
• Reinforcement learning's advantage in learning from suboptimal clinical data was highlighted.

Conclusions:

• The proposed BCQ-based RL model offers a highly accurate and effective approach for optimizing warfarin dosing decisions.
• This advanced machine learning strategy holds significant potential for improving the management of anticoagulant therapy and patient outcomes.
• The model's explainability suggests its clinical utility in real-world decision support systems.