An Open-architecture AI Model for CPT Coding in Breast Surgery: Development, Validation, and Prospective Testing.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Surgical Coding Automation

Background:

• Current operative note coding is inefficient and prone to errors, leading to financial losses and compliance issues.
• Proprietary, closed-source AI systems hinder adoption due to lack of transparency and validation.
• An open-architecture AI model is needed for reliable and efficient surgical coding.

Purpose of the Study:

• To develop, validate, and prospectively test an open-architecture artificial intelligence (AI) model.
• The AI model aims to extract Current Procedural Terminology (CPT) codes from free-text breast surgery operative notes.
• To address limitations of current closed-source AI systems in healthcare coding.

Main Methods:

• Utilized 3259 breast surgery operative notes (July 2017-December 2023) with expert-validated CPT codes as the reference standard.
• Developed and validated two transformer-based AI models (345M and 3.9B parameters) fine-tuned on operative notes.
• Evaluated performance using area under the precision-recall curve (AUPRC) and conducted prospective testing on 268 notes.

Main Results:

• The AI model demonstrated high performance, outperforming human surgeons in CPT code extraction (large model AUPRC: 0.981 vs. surgeon AUPRC: 0.937).
• Cross-validation showed strong alignment with the reference standard (compact model AUPRC: 0.976).
• Prospective testing confirmed the AI model's robust real-world performance in extracting procedure codes.

Conclusions:

• An open-architecture AI model effectively automates CPT code extraction from surgical notes.
• The developed AI offers a scalable, transparent, and efficient solution for improving surgical coding accuracy.
• Future research will explore AI's potential to exceed human coder accuracy and reliability.