An Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study.

Area of Science:

• Machine learning applications in healthcare
• Explainable artificial intelligence (XAI)
• Behavioral science and intervention effectiveness

Background:

• Machine learning (ML) models can predict weight loss intervention success, enabling personalized treatment adjustments.
• However, a lack of trust and understanding hinders the adoption of ML by weight management experts.
• Explainable AI (XAI) offers a potential solution to bridge this gap.

Purpose of the Study:

• To develop and evaluate an ML model for predicting 6-month weight loss success based on early intervention data.
• To assess if ML-based explanations improve weight management experts' agreement with model predictions.
• To identify factors influencing experts' understanding and trust in ML models for weight loss prediction.

Main Methods:

• A random forest (RF) ML model was trained on data from 419 participants in a 6-month weight loss intervention.
• An interactive XAI tool, PRIMO, was developed to interpret RF model predictions.
• 14 weight management experts evaluated hypothetical cases before and after using PRIMO, comparing it with other explainability methods.

Main Results:

• The RF model achieved 81% accuracy in predicting weight loss success.
• Experts showed significantly higher agreement with ML predictions when using PRIMO compared to other methods (P=.02).
• Interviews revealed preferences for multiple explanation types, visualization of uncertainty, and model performance metrics.

Conclusions:

• Explainable ML models like PRIMO can increase weight management experts' trust and agreement with early predictions of weight loss success.
• This enhanced trust can facilitate the dynamic modification of interventions for improved effectiveness.
• The study provides methods to advance the understandability and trust of ML models in weight management contexts.