Learning Enabled Control for Optimal EPO Dosage in Virtual CKD Patients: Case of Bleeding and Missing Dosage

Traditional model-based control methods require predictive models to design control policies. These models often suffer limitations on dimensionality, uncertainty, and unmodeled dynamics. This affects the performance of control policy, especially, personalized drug dosing of erythropoietin (EPO) in chronic patients due to the narrow therapeutic window and highly nonlinear time-variant patient characteristics. It is more challenging in the events of bleeding and missing dosage. This paper investigates model-free Reinforcement Learning (RL) and adaptive model predictive control (AMPC) in the events of bleeding and missing dosages for precise EPO dosing in a virtual setting. In this research work, an AMPC policy and Deep Q-Network(DQN)-RL agent is trained for a virtual chronic kidney disease (CKD) patient to find the optimal EPO dosage to maintain the hemoglobin (Hgb) level between 10-12 g/dl. The simulation results show that AMPC and DQN-RL both can provide optimal results while meeting the constraints, however, DQN-RL is more computationally challenging and demands high data.Clinical relevance- This research work provides a framework to help clinicians in decision-making for personalized EPO dose guidance using model-free Reinforcement Learning (RL) and adaptive model predictive control (AMPC) in the events of bleeding and missing dosages.