Systematic evaluation of common natural language processing techniques to codify clinical notes.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Computational Linguistics

Background:

• Accurate medical coding is vital for healthcare management, research, and reimbursement.
• Manual coding is labor-intensive and susceptible to human error.
• Natural Language Processing (NLP) offers potential solutions for automating medical coding.

Purpose of the Study:

• To evaluate common NLP techniques for predicting Current Procedural Terminology (CPT) codes from operative notes.
• To compare the performance of traditional NLP methods against resource-intensive models like BERT.
• To introduce a complexity measure for classification tasks in NLP and its impact on dataset size.

Main Methods:

• Comprehensive performance assessment of various NLP techniques.
• Analysis focused on 100 common musculoskeletal CPT codes from operative notes.
• Statistical comparison of traditional NLP approaches versus BERT, including AUROC and accuracy metrics.

Main Results:

• Traditional NLP methods significantly outperformed BERT for CPT code prediction (P-value = 4.4e-17).
• Achieved high performance with traditional NLP: average AUROC of 0.96 and accuracy of 0.97.
• Demonstrated the interpretability of traditional NLP models, crucial for clinical applications.

Conclusions:

• Simpler, traditional NLP techniques are highly effective and efficient for CPT code prediction.
• NLP can significantly reduce medical coding errors, including those from human mistakes.
• A proposed complexity measure can guide the application of NLP models based on dataset characteristics.