Participatory-informed preference optimization (PiPrO): A reinforcement learning simulation study.

Area of Science:

• Public Health
• Artificial Intelligence
• Machine Learning

Background:

• AI holds significant potential for public health advancements.
• Current AI models often lack consideration for diverse stakeholder perspectives, particularly community versus clinician viewpoints.
• This limitation hinders the development of universally accepted and effective AI-driven healthcare solutions.

Purpose of the Study:

• To introduce Participatory-informed Preference Optimization (PiPrO), a novel AI framework.
• To address the gap in AI models by explicitly incorporating and balancing community and physician interpretations.
• To generate clinical outcome predictions that are sensitive to differing stakeholder perspectives.

Main Methods:

• PiPrO utilizes large language model embeddings for community and physician contexts.
• A shared feedforward predictor generates per-stakeholder scores.
• A global mixing weight (alpha), learned via policy-gradient, balances community and physician inputs.
• The learning process uses abundant, noisy community data and sparse, biased physician data.

Main Results:

• PiPrO successfully learned stable mixing weights (alpha) and a consistent reward signal.
• The learned alpha systematically adjusted based on the quality of community and physician feedback.
• Alpha shifted towards physician weighting with noisier community data and towards community weighting with biased physician data.

Conclusions:

• PiPrO offers a method for creating more transparent and context-aware AI healthcare recommendations.
• The framework demonstrates the ability to tune AI predictions based on the trade-offs between community and clinician endorsement.
• Further validation with real-world community data is recommended to ensure generalizability and practical impact.