Creation of a simple natural language processing tool to support an imaging utilization quality dashboard

Area of Science:

• Medical Informatics
• Radiology
• Natural Language Processing

Background:

• Assessing diagnostic yield of venous thromboembolism (VTE) imaging is crucial for appropriate utilization.
• Manual review of radiology reports or complex NLP software are current barriers to determining diagnostic yield.
• Provider-level diagnostic yield assessment is needed to evaluate imaging appropriateness.

Purpose of the Study:

• To develop a user-friendly, open-source NLP tool for accurate VTE radiology report classification.
• To utilize the NLP tool's output for creating a provider-specific VTE imaging dashboard.
• To enhance quality assessment by providing insights into imaging utilization and diagnostic yield.

Main Methods:

• Physicians reviewed VTE-positive and negative reports to inform NLP tool parameters.
• A training set of 400 reports and a validation set of 2,000 reports were used.
• The NLP tool's classification accuracy was validated against physician reviewers.

Main Results:

• The NLP tool demonstrated high accuracy in classifying VTE presence in ultrasound (UTZ) and CT pulmonary angiogram (CTPA) reports.
• UTZ reports: 95.7% sensitivity, 100% specificity. CTPA reports: 97.1% sensitivity, 98.6% specificity.
• Provider-level diagnostic yield was calculated, and an imaging dashboard was created.

Conclusions:

• A novel, open-source NLP tool accurately classifies VTE radiology reports.
• The tool is accessible to users without programming expertise.
• The tool facilitates quality improvement by providing provider-specific utilization and yield data via a dashboard.