Publicly Accessible Large Language Model Responses to Frequently Asked Questions About Spondylodiscitis: Preliminary...

Area of Science:

• Spine surgery
• Medical informatics
• Artificial intelligence in healthcare

Background:

• Patients increasingly use large language models (LLMs) for medical information.
• The quality of LLM-generated content on complex spinal infections like spondylodiscitis is uncertain.
• Limited evaluations exist for LLMs in spine surgery, particularly for spondylodiscitis.

Purpose of the Study:

• To evaluate spine surgeons' ratings of LLM responses to spondylodiscitis FAQs.
• To compare responses from GPT-4, GPT-4o, and Google Gemini for accuracy and clarity.
• To assess the suitability of LLMs for patient education on spinal infections.

Main Methods:

• Generated 10 patient-oriented FAQs on spondylodiscitis.
• Submitted prompts to GPT-4, GPT-4o, and Google Gemini web interfaces.
• Seven spine surgeons blindedly rated responses on quality, comprehensiveness, clarity, empathy, and length.

Main Results:

• 38.6% of responses rated excellent, 39% satisfactory with minimal clarification.
• Insufficient information was the most common reason for clarification (41.1%).
• No significant difference in median ratings among the three LLMs; complication questions rated highest.

Conclusions:

• Publicly accessible LLM interfaces provided mostly satisfactory spondylodiscitis information.
• LLMs can aid patient education but require clinician oversight.
• Future research should focus on domain-specific models for improved clinical communication.