Related Experiment Video
Updated: Jun 4, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Evaluating the quality and readability of ChatGPT-generated patient-facing medical information in rhinology
Alexander Z Fazilat1, Camille Brenac1,2,3, Danae Kawamoto-Duran1
1Hagey Laboratory for Pediatric Regenerative Medicine, Division of Plastic and Reconstructive Surgery, Department of Surgery, Stanford University School of Medicine, Stanford, CA, USA.
Purpose:
The artificial intelligence (AI) chatbot ChatGPT has become a major tool for generating responses in healthcare. This study assessed ChatGPT's ability to generate French preoperative patient-facing medical information (PFI) in rhinology at a comparable level to material provided by an academic source, the French Society of Otorhinolaryngology (Société Française d'Otorhinolaryngologie et Chirurgie Cervico-Faciale, SFORL).
Methods:
ChatGPT and SFORL French preoperative PFI in rhinology were compared by analyzing responses to 16 questions regarding common rhinology procedures: ethmoidectomy, sphenoidotomy, septoplasty, and endonasal dacryocystorhinostomy. Twenty rhinologists assessed the clarity, comprehensiveness, accuracy, and overall quality of the information, while 24 nonmedical individuals analyzed the clarity and overall quality. Six readability formulas were used to compare readability scores.
Results:
Among rhinologists, no significant difference was found between ChatGPT and SFORL regarding clarity (7.61 ± 0.36 vs. 7.53 ± 0.28; p = 0.485), comprehensiveness (7.32 ± 0.77 vs. 7.58 ± 0.50; p = 0.872), and accuracy (inaccuracies: 60% vs. 40%; p = 0.228), respectively. Non-medical individuals scored the clarity of ChatGPT significantly higher than that of the SFORL (8.16 ± 1.16 vs. 6.32 ± 1.33; p < 0.0001). The non-medical individuals chose ChatGPT as the most informative source significantly more often than rhinologists (62.8% vs. 39.7%, p < 0.001).
Conclusion:
ChatGPT-generated French preoperative PFI in rhinology was comparable to SFORL-provided PFI regarding clarity, comprehensiveness, accuracy, readability, and overall quality. This study highlights ChatGPT's potential to increase accessibility to high quality PFI and suggests its use by physicians as a complement to academic resources written by learned societies such as the SFORL.
Related Concept Videos
Guidelines for Nursing Documentation I
Factual:
The following points emphasize the significance of upholding accurate and unbiased documentation in healthcare.
Methods of Documentation II: POMR
Methods of Documentation III: PIE
Patient-centered Care
Purpose of Health Records I
Here's a breakdown of how health records serve these purposes:
Respiratory Assessment: Purpose and Indications
Objectives and Importance:
The primary goal of respiratory assessment is to evaluate patients at early risk of clinical deterioration. Since respiratory distress often precedes other signs of declining health, breathing patterns and sounds become a...

