TECRR: a benchmark dataset of radiological reports for BI-RADS classification with machine learning, deep learning,...

Area of Science:

• Medical Imaging Analysis
• Machine Learning in Radiology
• Natural Language Processing in Healthcare

Background:

• Limited availability of annotated breast imaging datasets for BI-RADS and breast density classification.
• Machine learning (ML), deep learning (DL), and natural language processing (NLP) show promise in analyzing radiological reports.
• Need for a curated dataset to advance automated classification of breast imaging reports.

Purpose of the Study:

• To construct and annotate a novel breast imaging radiological reports dataset.
• To establish benchmark results for various ML, DL, and large language models (LLMs) in BI-RADS classification.
• To provide a resource for researchers entering the field of breast imaging report analysis.

Main Methods:

• Dataset creation involved board-certified radiologists annotating 5046 Spanish reports, translated to English and preprocessed.
• Utilized NLP embedding techniques like Term Frequency-Inverse Document Frequency (TF-IDF) and word2vec for feature extraction.
• Compared performance of multiple classifiers including K-Nearest Neighbour (KNN), Support Vector Machine (SVM), Random Forest (RF), Long Short-Term Memory (LSTM), BERT, and BioGPT.

Main Results:

• The final dataset comprises 5046 unique breast imaging reports.
• BioGPT demonstrated superior performance in BI-RADS category classification, achieving a mean sensitivity of 0.60.
• BioGPT outperformed BERT (mean sensitivity 0.54) by 6% using preprocessed data.

Conclusions:

• A curated, annotated breast imaging dataset is proposed for BI-RADS and breast density classification.
• Baseline performance results for ML, DL, and LLMs in BI-RADS classification are provided.
• The dataset and results serve as a foundation for future research in automated radiological report analysis.