Related Experiment Videos
Application of deep learning and explainable AI-supported medical decision-making for facial phenotyping in genetic
Ömer Sümer1, Tobias Huber2, Jiayi Cheng3
1Human-Centered Artificial Intelligence, University of Augsburg, Faculty of Applied Computer Science, Augsburg, Germany.
Objective:
To conduct a pilot study to assess whether saliency-based explainable artificial intelligence (XAI) affects the recognition of genetic conditions from facial images.
Methods:
Forty-four medical geneticists, divided into AI-only and XAI-supported groups, assessed 18 images of individuals with or without genetic conditions. Diagnostic accuracy and confidence were recorded before and after viewing an AI classifier's prediction probability, with or without XAI explanations. Mediation analyses were conducted to better interpret how geneticists interact with AI and XAI in decision-making.
Results:
AI-only and XAI support improved accuracy for correct AI classifications, while incorrect AI classifications decreased accuracy. Average confidence increased with correct and decreased with incorrect classification. Geneticists reported that AI prediction probability was useful, whereas XAI explanations were viewed less favorably. For incorrect AI classifications, there was a negative correlation between accuracy improvement and perceived AI usefulness. When AI was correct (without XAI), the model prediction probability acted as a mediator between user confidence and the user's decision to choose the same answer as AI.
Conclusion:
The lack of accuracy or confidence improvements indicates that participants did not integrate saliency-based XAI into decisions. AI prediction probability had a greater impact on participants' decision-making. These data may help inform larger studies.