Ethical Considerations in Personal Health Large Language Models

Area of Science:

• Artificial Intelligence in Healthcare
• Biomedical Ethics
• Digital Health Governance

Background:

• Personal health large language models (PH-LLMs) are increasingly used for direct consumer health support, including symptom triage and self-management.
• Current artificial intelligence governance frameworks inadequately address the unique ethical risks of PH-LLMs used without clinical oversight.

Purpose of the Study:

• To identify and synthesize PH-LLM-specific ethical challenges across six key domains: privacy, accuracy, equity, transparency, human-AI interaction, and regulatory governance.
• To propose a novel governance framework for PH-LLMs, grounded in the four principles of biomedical ethics, to mitigate identified risks.

Main Methods:

• A synthesis of PH-LLM-specific challenges, focusing on text-based, platform-mediated systems.
• Development of a governance framework operationalizing beneficence, nonmaleficence, autonomy, and justice through design and deployment controls.
• Outline of implementation mechanisms including risk-tiered certification and post-deployment oversight.

Main Results:

• Identified amplified risks due to health literacy gaps, data aggregation, persuasive design, and fragmented oversight.
• Proposed a framework with specific controls: health literacy-aligned communication, crisis safeguards, hallucination mitigation, role disclosure, granular consent, fairness auditing, and accessible design.
• Outlined implementation strategies: risk-tiered certification, tiered accountability, and adverse-event reporting.

Conclusions:

• A comprehensive, evidence-informed governance framework is crucial for the responsible design and deployment of PH-LLMs in personal health management.
• The proposed framework aims to proactively address ethical risks, ensuring PH-LLMs support user well-being and autonomy.
• Anticipatory governance is necessary to navigate the evolving landscape of AI in personal healthcare.