Billing and Coding in Foot and Ankle Surgery: Can We Trust Artificial Intelligence?

Area of Science:

• Medical informatics
• Artificial intelligence in healthcare
• Orthopaedic surgery coding

Background:

• Orthopaedic procedure billing and coding is intricate, with numerous codes and modifiers.
• Foot and ankle procedures exhibit high variability, complicating coding compared to other orthopaedic subspecialties.

Purpose of the Study:

• To evaluate the accuracy of leading AI search engines in identifying Current Procedural Terminology (CPT) codes for common foot and ankle procedures.

Main Methods:

• Comparative analysis of AI search engines (ChatGPT, Bing, Google Gemini) for CPT code generation.
• AI-generated codes were compared against a reference standard set by fellowship-trained foot and ankle surgeons.
• Cohen kappa coefficient assessed agreement between AI platforms and the surgeon reference standard.

Main Results:

• AI search engines correctly generated CPT codes 44% of the time.
• Bing achieved the highest accuracy (62% correct, 23% partially correct), while ChatGPT had the lowest (23% correct).
• Overall AI agreement with the surgeon standard was 'Fair' (kappa = 0.201), with Bing showing 'Moderate' agreement (kappa = 0.405).

Conclusions:

• Current AI capabilities demonstrate potential but require caution for accurate orthopaedic foot and ankle procedure CPT code generation.
• Further development and validation are necessary before AI can be reliably implemented for complex medical coding tasks.