Employing Deep Reinforcement Learning to Maximize Lower Limb Blood Flow Using Intermittent Pneumatic Compression

Area of Science:

• Biomedical Engineering
• Physiology
• Artificial Intelligence

Background:

• Intermittent pneumatic compression (IPC) enhances lower limb blood flow.
• Previous cardiac-gated systems improved blood velocity (BV), but optimal timing varied individually and over time.
• Current methods predict timing only for the short term.

Purpose of the Study:

• To develop and evaluate a deep reinforcement learning (DRL) algorithm for adaptive optimization of IPC compression timing (CT).
• To maximize lower limb arterial blood velocity (BV) using DRL-guided IPC.

Main Methods:

• Participant-specific simulated lower limb environments were created for 6 individuals.
• A DRL agent was trained to adaptively learn and modify IPC CT to maximize arterial BV.
• The DRL agent's performance was compared to previous methods.

Main Results:

• The DRL agent successfully learned to adaptively optimize IPC CT, maximizing arterial BV.
• DRL achieved 98% ± 2 of the maximum resultant blood flow, outperforming previous approaches.
• The DRL agent learned an optimal policy in an average of 15 minutes ± 2 and could adapt in real-time.

Conclusions:

• DRL can effectively learn and adaptively modify IPC CT to achieve desired physiological outcomes, such as maximizing BV.
• The proposed DRL agent offers a rapid, adaptive, and personalized approach to optimizing IPC therapy.
• This DRL system has potential for implementation in IPC devices for real-time, human-in-the-loop optimization.