AI in Hidradenitis Suppurativa: Expert Evaluation of Patient-Facing Information

This study investigates the accuracy of Artificial Intelligence (AI) chatbots, ChatGPT and Bard, in providing information on Hidradenitis Suppurativa (HS), aiming to explore their potential in assisting HS patients by offering insights into symptoms, thus possibly reducing the diagnostic and treatment time gap.

Using questions formulated with the help of HS patient associations, both ChatGPT and Bard were assessed. Responses to these questions were evaluated by 18 hS experts.

ChatGPT's responses were considered accurate in 86% of cases, significantly outperforming Bard, which only achieved 14% accuracy. Despite the general efficacy of ChatGPT in providing relevant information across a range of HS-related queries, both AI systems showed limitations in offering adequate advice on treatments. The study identifies a significant difference in the performance of the two AIs, emphasizing the need for improvement in AI-driven medical advice, particularly regarding treatment options.

The study highlights the potential of AI chatbots, particularly ChatGPT, in supporting HS patients by improving symptom understanding and potentially reducing the time to diagnosis and treatment. AI chatbots, while promising, cannot yet substitute for professional medical diagnosis and treatment, indicating the importance of enhancing AI capabilities for more accurate and reliable medical information dissemination.