Preliminary Evaluation of a Large Language Model-Powered Chatbot for Osteoporosis Self-Management Education:...

Area of Science:

• Digital Health
• Artificial Intelligence in Medicine
• Chronic Disease Management

Background:

• Self-management education (SME) is vital for chronic diseases but traditional methods are resource-intensive.
• Digital tools offer limited interactivity, and the effectiveness of large language model (LLM)-based chatbots for real-world SME is unproven.

Purpose of the Study:

• To assess the feasibility and preliminary effectiveness of an LLM-based chatbot (OPBot) for osteoporosis SME.

Main Methods:

• A randomized controlled trial compared OPBot with traditional health education in adults with osteoporosis.
• Outcomes included osteoporosis knowledge, nurse time, and disease management adherence.
• Chatbot response reliability was also evaluated.

Main Results:

• The OPBot group demonstrated significantly higher post-intervention knowledge scores (P=.01) and reduced nurse time (P<.001).
• Improved adherence was noted for calcium supplement intake (P=.02) and calcium-rich food consumption.
• OPBot responses achieved high reliability (89.4%) with excellent interrater agreement (Cohen κ=0.83).

Conclusions:

• LLM-based chatbots show potential for enhancing osteoporosis SME, improving patient outcomes, and reducing healthcare burden.
• Further large-scale research is recommended to validate these preliminary findings.