Optimizing GPT-5 for Operation-Procedure-Code-Extraction from Operative Reports in Meningioma Surgery: Feasibility...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Surgical Reporting

Background:

• Previous research indicated GPT's OPS-code extraction capability matched neurosurgeons.
• The potential for AI in automating complex medical coding tasks remains an active area of investigation.

Purpose of the Study:

• To evaluate the impact of various context-enhancement strategies on GPT-5's accuracy in OPS-code extraction from surgical operational reports.
• To determine the optimal method for improving AI performance in this specific medical coding task.

Main Methods:

• GPT-5 was provided with 100 operational reports from meningioma surgeries.
• Five conditions were tested: no context, OPS catalogue, coding rules, sample phrases, and a combination of all enhancements.
• Performance was assessed based on coding accuracy, mistake rates, and hallucinations.

Main Results:

• The baseline GPT-5 model (no context) achieved 44% accuracy with numerous errors and hallucinations.
• Context-enhanced models significantly outperformed the baseline, with accuracy ranging from 70% to 86%.
• The combined context enhancement (GPT-5c) yielded the highest accuracy (86%) with the lowest mistake rate.

Conclusions:

• Specific context enhancement substantially improves GPT-5's OPS-code extraction from operational reports.
• AI models, when provided with relevant contextual information, demonstrate enhanced accuracy and reduced errors in medical coding.
• These findings highlight the potential of tailored AI solutions for improving efficiency and reliability in surgical coding.