German CheXpert Chest X-ray Radiology Report Labeler.

Area of Science:

• Radiology and Medical Imaging
• Artificial Intelligence in Healthcare
• Machine Learning for Medical Diagnosis

Background:

• Deep learning models for chest X-ray classification require large, accurately annotated datasets.
• Manual annotation of radiology reports is time-consuming and resource-intensive.
• Developing automated methods for extracting labels from reports is crucial for efficient model training.

Purpose of the Study:

• To develop an algorithm for automatic annotation of German thoracic radiology reports.
• To train deep learning-based chest X-ray classification models using automatically extracted labels.
• To evaluate the performance of models trained with automated labels against manual and public datasets.

Main Methods:

• An automatic label extraction model based on the CheXpert architecture was designed for German reports.
• A web-based annotation interface was created for iterative improvements and ground truth generation.
• A DenseNet-121 model was trained and evaluated using automatically extracted labels, manual labels, and public data for pneumothorax classification.

Main Results:

• Automated label extraction achieved high F1 scores for mention (0.8-0.995), negation (0.624-0.981), and uncertainty (0.353-0.725) detection.
• Pneumothorax classification using automatically extracted labels showed high sensitivity (0.997) and specificity (0.991).
• The model trained on automatically extracted labels (AUC 0.858) outperformed a model trained on public data (AUC 0.728) and was competitive with manual annotations (AUC 0.934).

Conclusions:

• Automatic label extraction from German thoracic radiology reports is a viable and efficient alternative to manual labeling.
• This automated approach enables the creation of larger training datasets, leading to improved deep learning model performance.
• The developed algorithm facilitates competitive or superior performance in chest X-ray classification tasks without additional annotation time.