Consumer Understanding of Skin Concerns With an AI-Powered Informational Tool

The potential association of artificial intelligence (AI)-powered informational tools with consumer health needs outcomes is not well understood. Such tools could potentially be associated with the volume and distribution of patients seeking care.

To investigate the association of an AI-powered dermatology application with changes in consumer understanding of retrospective skin condition cases.

This survey study used a between-participants randomized survey with 3 arms: control (using existing tools, such as web search), AI (with access to predictions from a prototype AI application), and a Wizard of Oz method (with the same interface as in the AI group but using dermatologist panel ground truth differentials instead of AI predictions) run on a commercial panel platform. Participants interpreted retrospective deidentified skin condition cases containing images and structured medical history. US-based participants who self-reported having sought information for a skin concern within the past year were included. The survey was conducted March 17 to May 16, 2023, and data were analyzed from June 2023 through November 2024.

Participants in AI and Wizard of Oz arms were presented 3 to 7 conditions per case patient as a horizontal carousel of condition cards. Cards included textbook images of each condition and brief descriptions and could be expanded to show more textbook images and details about each condition's background and treatment. Conditions in the Wizard of Oz arm had identical visual presentation, but predicted conditions were the dermatologist-provided ground truth.

Participants in each arm self-reported whether they could name the condition depicted, and if so, they could list 1 or more conditions they thought were shown. Participants also reported the next step they thought appropriate for each case patient, as well as their self-reported confidence in their assessment and satisfaction with the information search experience. Condition name and next step accuracy were assessed against a reference diagnosis from dermatologists, and next steps were derived from the conditions.

Among 2345 participants (509 aged 30-39 years [21.71%]; 1650 female [70.36%]), 11 725 participant case patient reads were obtained across 3 study arms. Compared with the control group (41.21%; 95% CI, 39.66%-42.76%), participants were more willing to name a condition in AI (62.26%; 95% CI, 60.75%-63.76%) and Wizard of Oz (61.76%; 95% CI, 60.21%-63.28%) arms (both P < .001; permutation test with false discovery rate correction), with increased accuracy for AI (22.79%; 95% CI, 21.48%-24.09%; P < .001) and Wizard of Oz (36.20%; 95% CI, 34.70%-37.73%; P = .002) vs control (7.86%; 95% CI, 7.03%-8.71%). Next-step accuracy increased for the Wizard of Oz (62.95%; 95% CI, 61.42%-64.44%; P < .001) compared with the control group (60.10%; 95% CI, 58.55%-61.65%).

In this study, AI applications were associated with increased accuracy and confidence of consumer understanding of skin concerns, with the degree of improvement in accuracy varying by the accuracy of presented conditions; benefits further improved when predictions were as accurate as possible. Imperfect guessing accuracy when predictions presented matched dermatologist differentials highlighted the need for further design improvements (such as the condition information presented) to help consumers better understand skin conditions.