Large language models: Are artificial intelligence-based chatbots a reliable source of patient information for spinal...

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Medical Informatics

Background:

• Large language models (LLMs) demonstrate advanced capabilities in natural language communication.
• LLMs hold potential for enhancing patient understanding of medical information.
• Evaluating LLM performance in specific clinical contexts is crucial.

Purpose of the Study:

• To assess the validity of a high-performance LLM (ChatGPT) in providing medical information.
• To evaluate ChatGPT's responses on acute lumbar disc herniation (LDH) from a patient's perspective.
• To determine the potential utility of LLMs in patient medical communication.

Main Methods:

• Twenty-four spinal surgeons posed patient-centric questions about LDH to ChatGPT.
• Surgeons evaluated the quality, accuracy, and completeness of ChatGPT's responses.
• Responses were benchmarked against standard informed consent form content.

Main Results:

• ChatGPT provided comprehensible, specific, and satisfactory responses regarding LDH.
• Medical accuracy and completeness were generally good, though some inaccuracies were noted (e.g., incorrect surgical options).
• ChatGPT offered information beyond standard consent forms but did not cover all provided details.

Conclusions:

• LLMs are poised to become increasingly significant for patient information access.
• While not currently for direct physician-patient clinical communication, LLM opportunities and risks warrant careful monitoring.
• The evolving role of AI in healthcare communication necessitates ongoing evaluation.