Surgeon, Trainee, or GPT? A Blinded Multicentric Study of AI-Augmented Operative Notes.

Area of Science:

• Medical Informatics
• Surgical Documentation
• Artificial Intelligence in Healthcare

Background:

• Clear operative documentation is crucial for surgical safety, patient care continuity, and medico-legal compliance.
• Large language models (LLMs) show potential for automating clinical documentation, but their efficacy in generating operative notes, especially in surgical subspecialties, requires further investigation.

Purpose of the Study:

• To compare the quality, accuracy, and efficiency of operative notes generated by a surgical resident, an attending surgeon, GPT alone, and an attending surgeon using GPT as an aid.
• To evaluate the impact of human-AI collaboration on the completeness and correctness of surgical documentation.

Main Methods:

• Five otolaryngologic procedures were selected for analysis.
• Operative notes were generated by a resident, an attending, GPT alone, and a hybrid of attending plus GPT.
• Ten blinded otolaryngologists (residents and attendings) reviewed the notes, scoring them on eight domains and providing an approval rating. Documentation time was recorded.

Main Results:

• Operative notes co-authored by an attending surgeon and GPT received the highest domain scores and an "as is" approval rate of 79%, surpassing all other methods.
• GPT-only notes were generated fastest but had the lowest approval rate (23%) and the most errors (omissions and overdocumentation).
• AI-assisted documentation significantly reduced writing time compared to human-only authorship, with moderate to high inter-rater reliability observed.

Conclusions:

• Hybrid human-AI collaboration in operative documentation yielded superior results compared to human-only or AI-only approaches in this study.
• GPT-assisted documentation holds promise for enhancing the efficiency and consistency of operative note generation in surgical practice.