Bridging the Coding Gap: Assessing Large Language Models for Accurate Modifier Assignment in Craniofacial Operative...

Area of Science:

• Medical informatics
• Artificial intelligence in healthcare
• Surgical coding

Background:

• Accurate medical coding, particularly with CPT codes and modifiers, is crucial for healthcare management and reimbursement in craniofacial surgery.
• Applying CPT modifiers correctly is challenging, time-consuming, and prone to errors for coding professionals.
• Natural language processing (NLP) and large language models (LLMs) present potential solutions for automating medical coding tasks.

Purpose of the Study:

• To evaluate the capability of LLMs (ChatGPT and Google Gemini) in identifying necessary CPT modifiers from craniofacial operative notes.
• To compare LLM performance against expert-coded results for accuracy in modifier identification.

Main Methods:

• Collected 10 craniofacial operative notes containing common CPT modifiers, including Modifier 22 (increased procedural complexity).
• Assessed the precision of LLM-generated CPT codes and modifiers against expert-coded benchmarks.
• Focused on modifiers critical to craniofacial surgical procedures.

Main Results:

• Neither ChatGPT nor Gemini accurately identified both CPT code and modifier in any of the evaluated cases.
• ChatGPT demonstrated a higher frequency of partially correct CPT and modifier codes compared to Gemini.
• Both models generated inaccurate codes, with some suggestions missing procedural specifics like graft inclusion or debridement depth.

Conclusions:

• LLMs show potential as an ancillary tool to aid in CPT modifier identification for craniofacial surgery.
• These AI tools could potentially reduce administrative burdens and enhance efficiency and reimbursement for complex surgical procedures.
• Future research should focus on refining LLM accuracy and assessing their applicability across diverse surgical subspecialties.