Adaptive heartbeat regulation using double deep reinforcement learning in a Markov decision process framework

Area of Science:

• Biomedical Engineering
• Computational Biology
• Control Systems

Background:

• Cardiac rhythm irregularities can lead to serious health issues.
• Stabilizing the human heartbeat is a significant area of research.
• Existing methods may not adequately address complex cardiac dynamics.

Purpose of the Study:

• To develop and evaluate a novel control strategy for stabilizing cardiac activity.
• To enhance the adaptability and robustness of cardiac rhythm management.
• To investigate the efficacy of a combined adaptive nonlinear disturbance compensator and deep reinforcement learning approach.

Main Methods:

• Developed an adaptive nonlinear disturbance compensator (ANDC) strategy.
• Employed a double deep reinforcement learning (DDRL) algorithm for adaptive controller calibration.
• Constructed a dynamic heart model using the Markov Decision Process (MDP) framework.
• Implemented a closed-loop system with ANDC for stability and disturbance mitigation, and DDRL for parameter refinement.
• Assessed performance using normal and stochastic signals, and simulated pathological neural activity with multiple frequency components.

Main Results:

• The ANDC-DDRL strategy demonstrated effective stabilization of cardiac rhythms.
• The system showed robustness under normal, uncertain (stochastic signals), and pathological conditions.
• Quantitative assessments (peak amplitude, signal energy, zero-crossing rate) confirmed improved cardiac model states.
• The proposed method outperformed conventional baseline techniques in stabilizing cardiac activity.

Conclusions:

• The ANDC-DDRL strategy provides an effective solution for stabilizing erratic cardiac rhythms.
• This adaptive control approach offers improved performance compared to traditional methods.
• The findings support the potential of advanced control algorithms in managing cardiovascular pathologies.