Machine Learning for Automation of Radiology Protocols for Quality and Efficiency Improvement

The aim of this study was to enhance multispecialty CT and MRI protocol assignment quality and efficiency through development, testing, and proposed workflow design of a natural language processing (NLP)-based machine learning classifier.

NLP-based machine learning classification models were developed using order entry input data and radiologist-assigned protocols from more than 18,000 unique CT and MRI examinations obtained during routine clinical use. k-Nearest neighbor, random forest, and deep neural network classification models were evaluated at baseline and after applying class frequency and confidence thresholding techniques. To simulate performance in real-world deployment, the model was evaluated in two operating modes in combination: automation (automated assignment of the top result) and clinical decision support (CDS; top-three protocol suggestion for clinical review). Finally, model-radiologist discordance was subjectively reviewed to guide explainability and safe use.

Baseline protocol assignment performance achieved weighted precision of 0.757 to 0.824. Simulating real-world deployment using combined thresholding techniques, the optimized deep neural network model assigned 69% of protocols in automation mode with 95% accuracy. In the remaining 31% of cases, the model achieved 92% accuracy in CDS mode. Analysis of discordance with subspecialty radiologist labels revealed both more and less appropriate model predictions.

A multiclass NLP-based classification algorithm was designed to drive local operational improvement in CT and MR radiology protocol assignment at subspecialist quality. The results demonstrate a simulated workflow deployment enabling automated assignment of protocols in nearly 7 of 10 cases with very few errors combined with top-three CDS for remaining cases supporting a high-quality, efficient radiology workflow.