Classification of Diagnostic Certainty in Radiology Reports with Deep Learning

Area of Science:

• Radiology and Medical Imaging
• Artificial Intelligence in Healthcare
• Natural Language Processing in Medicine

Background:

• Radiology reports communicate critical findings but often use ambiguous language.
• Ambiguity in reports hinders accurate clinical interpretation by referring physicians.
• A systematic method is needed to quantify diagnostic certainty in radiological findings.

Purpose of the Study:

• To develop and validate an ordinal scale for assessing diagnostic certainty in radiology reports.
• To evaluate the applicability of a deep learning model for classifying diagnostic certainty.
• To improve the clarity and interpretability of radiological communications.

Main Methods:

• Defined a five-class ordinal scale: definite, likely, may represent, unlikely, and denial.
• Developed and applied a deep learning classification model.
• Trained and evaluated the model using 540 in-house chest computed tomography reports.

Main Results:

• The deep learning model achieved a high micro F1-score of 97.61%.
• The results indicate the model's effectiveness in classifying diagnostic certainty.
• The ordinal scale proved suitable for measuring diagnostic certainty in reports.

Conclusions:

• The proposed ordinal scale and deep learning model effectively quantify diagnostic certainty in radiology reports.
• This approach can enhance the precision of clinical information conveyed in radiological findings.
• Future applications may improve diagnostic accuracy and patient care through clearer reports.