Replicating Current Procedural Terminology code assignment of rhinology operative notes using machine learning.

Area of Science:

• Otolaryngology
• Medical Informatics
• Machine Learning

Background:

• Documentation and billing are time-consuming administrative tasks for otolaryngologists.
• Advancements in machine learning (ML) offer potential solutions for automating these tasks.

Purpose of the Study:

• To evaluate the ability of ML algorithms to classify rhinology procedures by Current Procedural Terminology (CPT®) codes using operative notes.
• To assess ML's potential to replicate the administrative task completion of rhinologists.

Main Methods:

• A retrospective cohort study analyzed 594 operative notes from 22 otolaryngologists across six CPT codes.
• Text preprocessing and vectorization (CountVectorizer, TF-IDF, Word2Vec) were performed.
• Decision Tree, Support Vector Machine, Logistic Regression, and Naïve Bayes algorithms were trained and tested.

Main Results:

• Model performance varied across vectorizers and algorithms.
• The combination of CountVectorizer and Naïve Bayes demonstrated the best overall performance.
• This combination achieved the highest area under the receiver operating characteristic curve (ROC-AUC) of 0.984.

Conclusions:

• Basic ML algorithms show potential for classifying CPT codes in otolaryngology.
• The effectiveness of ML algorithms is context-dependent.
• ML has the potential to assist rhinologists with administrative tasks.