Can Large Artificial Intelligence-Based Linguistic Models Help to Obtain Information About Burning Mouth Syndrome?

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Pain Management

Background:

• Burning Mouth Syndrome (BMS) presents diagnostic and therapeutic challenges.
• Clinicians may seek online resources for information on BMS.
• Artificial intelligence large language models (AI-LLMs) are increasingly used for medical information retrieval.

Purpose of the Study:

• To evaluate the usefulness, quality, and readability of AI-LLM responses for Burning Mouth Syndrome.
• To compare the performance of ChatGPT-4, Gemini, and Microsoft Copilot in answering BMS-related questions.

Main Methods:

• Nine clinically relevant questions on BMS were formulated.
• Responses from three AI-LLMs were rated by 12 international experts on usefulness.
• Response quality was assessed using the QAMAI tool, and readability was measured using Flesch-Kincaid scores.

Main Results:

• All AI-LLMs provided moderately useful responses with no significant global performance differences.
• Gemini demonstrated superior quality in relevance, completeness, and source provision.
• Readability averaged a 12th-grade level, with ChatGPT requiring the highest proficiency.

Conclusions:

• AI-LLMs show potential for providing reliable information on Burning Mouth Syndrome.
• Variability in AI-generated content quality, readability, and source citation requires attention.
• Ongoing AI optimization is crucial for safe and effective clinical integration.