When Precision Meets Penmanship: ChatGPT and Surgery Documentation

Area of Science:

• Medical Informatics
• Artificial Intelligence in Surgery
• Clinical Documentation Improvement

Background:

• The National Health Service (NHS) aims to enhance surgical documentation through initiatives like 'Getting It Right First Time' (GIRFT).
• Accurate and comprehensive operation notes are crucial for patient safety and surgical quality improvement.
• Large language models (LLMs) like ChatGPT demonstrate potential for automating text-based medical tasks.

Purpose of the Study:

• To evaluate the efficacy of ChatGPT-4 in generating operation notes for laparoscopic appendicectomy.
• To assess the compliance of AI-generated notes with established GIRFT surgical documentation guidelines.
• To identify limitations and requirements for integrating AI into clinical surgical documentation workflows.

Main Methods:

• ChatGPT-4 was prompted to generate operation notes for laparoscopic appendicectomy.
• Generated notes were assessed against 30 key points from GIRFT recommendations, excluding pre-documented information.
• The evaluation process was repeated three times to calculate an average compliance score.

Main Results:

• ChatGPT-4 generated operation notes rapidly, achieving an average coverage of 78.8% (23.66/30) of GIRFT guidelines.
• The AI's performance exceeded average compliance rates observed with similar Royal College of Surgeons (RCS) guidelines.
• Output quality was directly correlated with the specificity and quality of the input prompt.

Conclusions:

• ChatGPT-4 shows significant potential for assisting in the creation of surgical operation notes, offering high guideline compliance.
• The need for careful prompt engineering and mandatory verification of AI-generated clinical content is emphasized.
• Secure integration with electronic health records is a prerequisite for the adoption of ChatGPT in NHS surgical settings.