Risk Management of Large Language Model-Based Exercise and Health Guidance: A China-Anchored, Comparatively Informed...

Area of Science:

• Digital Health
• Regulatory Science
• Artificial Intelligence in Healthcare

Background:

• Large language models (LLMs) offer exercise and health guidance, but their evolving capabilities present unique risk management challenges.
• Subtle changes in LLM applications can shift them from general wellness tools to higher-risk medical devices, particularly in exercise prescription and rehabilitation where errors can impact patient safety.

Purpose of the Study:

• To develop a framework for identifying when LLM-based exercise and health tools may be subject to medical device regulations.
• To provide a structured approach for risk assessment and regulatory classification of AI-driven health technologies.

Main Methods:

• A structured narrative review and comparative legal analysis using China's NMPA framework, EU MDR, EU AI Act, FDA, and IMDRF guidelines.
• Development of a six-dimensional trigger matrix (intended use, user context, personalization, data sources, automation, human oversight, updates) with Green/Yellow/Red coding rules.
• Preliminary application of the framework with independent raters on hypothetical scenarios.

Main Results:

• A six-dimensional trigger matrix framework was proposed to categorize LLM-based health tools.
• The framework demonstrated high inter-rater reliability (Fleiss' kappa = 1.00) in a preliminary application exercise.
• The framework aims to facilitate early risk identification and regulatory dialogue.

Conclusions:

• The proposed framework supports earlier risk identification and evidence planning for LLM-based health tools.
• This tool aids dialogue among developers, healthcare institutions, and regulators.
• The framework does not replace specific legal analysis or regulatory determination for individual products.