Diffusion Policies with Offline and Inverse Reinforcement Learning for Promoting Physical Activity in Older Adults...

Area of Science:

• Artificial Intelligence in Healthcare
• Reinforcement Learning
• Geriatric Medicine

Background:

• Offline reinforcement learning (RL) faces challenges in healthcare, including reward definition and distributional shift.
• Accurate reward function inference and policy learning from expert behavior are difficult in complex clinical settings.
• Promoting physical activity in older adults at high fall risk requires tailored interventions.

Purpose of the Study:

• To introduce Kolmogorov-Arnold Networks and Diffusion Policies for Offline Inverse Reinforcement Learning (KANDI) for physical activity promotion in older adults.
• To address challenges in applying offline RL to real-world clinical data, particularly for fall-risk interventions.
• To optimize the timing and policy of anti-sedentariness interventions to maximize physical activity.

Main Methods:

• Leveraged Kolmogorov-Arnold Networks for flexible reward function approximation from expert behavior (low-fall-risk older adults).
• Employed diffusion-based policies within an Actor-Critic framework for generative action refinement and mitigating distributional shift.
• Evaluated KANDI using wearable activity monitoring data from the Physio-feedback Exercise Program (PEER) clinical trial and the D4RL benchmark.

Main Results:

• KANDI identified optimal timing for anti-sedentariness interventions based on fall risk levels, maximizing daily physical activity.
• The method demonstrated superior performance on the D4RL benchmark, outperforming state-of-the-art offline RL techniques.
• Successfully applied KANDI in a clinical trial setting for fall-risk intervention program targeting older adults.

Conclusions:

• KANDI offers an effective solution for key challenges in offline RL for healthcare applications.
• The approach shows significant potential for optimizing physical activity promotion strategies in older adults.
• KANDI's ability to learn from real-world data and mitigate distributional shift is crucial for clinical translation.