Reinforcement learning for closed-loop regulation of cardiovascular system with vagus nerve stimulation: a...

Area of Science:

• Computational cardiovascular science
• Biomedical engineering
• Artificial intelligence in medicine

Background:

• Vagus nerve stimulation (VNS) shows promise for treating cardiovascular diseases like heart failure, arrhythmia, and hypertension.
• Current VNS parameter control lacks systematic optimization, hindering therapeutic efficacy.
• Artificial intelligence (AI) and closed-loop systems offer a path to adaptive, data-driven VNS control.

Purpose of the Study:

• To develop and evaluate an interactive AI framework using reinforcement learning (RL) for automated, data-driven design of closed-loop VNS control systems.
• To investigate AI's capability in systematically learning and adapting optimal VNS parameters for cardiovascular regulation.
• To assess the performance of RL-based control in simulating healthy and hypertensive cardiovascular systems under various physiological states.

Main Methods:

• Developed multiple simulation environments with a standard API to evaluate closed-loop VNS control systems.
• Utilized biophysics-based computational models of rat cardiovascular systems (healthy and hypertensive) in resting and exercise states.
• Implemented RL-based control frameworks for heart rate and mean arterial pressure set-point tracking, comparing deep RL and adaptive probabilistic inference policies.

Main Results:

• Closed-loop RL approaches successfully learned optimal VNS control policies and adapted to changing set points and system dynamics.
• Demonstrated a trade-off between sample-efficiency and generalizability, informing algorithm selection for VNS control.
• Transfer learning enhanced the sample efficiency of deep RL algorithms, paving the way for more personalized VNS systems.

Conclusions:

• RL-based closed-loop VNS systems are capable of effective cardiovascular control.
• The presented AI framework provides a systematic, adaptable method for learning VNS control strategies without prior dynamic knowledge.
• This approach facilitates the development of efficient and personalized VNS therapies for cardiovascular conditions.