Exploring the Potential of GPT-4 in Creating Billing Codes from Clinic Notes

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Clinical Documentation

Background:

• Standardized medical billing relies on complex coding systems like CPT/HCPCS.
• Manual code assignment from unstructured clinical notes is time-consuming and error-prone.
• The vast number of codes (over 22,000) exacerbates this challenge.

Purpose of the Study:

• To evaluate the capability of GPT-4 in automating the generation of CPT/HCPCS billing codes.
• To assess GPT-4's accuracy when generating codes from textual descriptions and real-world clinical notes.

Main Methods:

• GPT-4 was prompted to generate CPT/HCPCS codes from their textual descriptions.
• GPT-4 was further prompted to generate billing codes with confidence scores from Vanderbilt University Medical Center clinic notes.
• Performance was evaluated based on exact code matches and true positive rates.

Main Results:

• GPT-4 achieved 20.8% accuracy in generating exact CPT/HCPCS codes from descriptions.
• The model demonstrated a 28.9% mean true positive rate in assigning correct codes from clinical notes.
• Confidence scores were generated alongside the billing codes.

Conclusions:

• Current large language models like GPT-4 face significant challenges in accurately generating medical billing codes.
• The unstructured nature of clinical notes and the complexity of coding systems limit LLM performance in this domain.
• Further research is needed to improve AI accuracy for automated medical coding.