RespBERT: A Multi-Site Validation of a Natural Language Processing Algorithm, of Radiology Notes to Identify Acute...

Acute respiratory distress syndrome (ARDS) is a severe organ dysfunction associated with significant mortality and morbidity among critically ill patients admitted to the Intensive Care Unit (ICU). The etiology related to ARDS can be highly heterogeneous, with infection or trauma as the most common associations. The Berlin criteria, the current gold standard for ARDS diagnosis, often necessitates manual adjudication of chest radiographs, limiting automation tools. ARDS diagnosis relies on the presence of bilateral infiltrates on radiographs, which is often not available in Electronic Medical Records (EMRs). Automated identification of radiological evidence would facilitate a comprehensive study of the syndrome, eliminating the need for costly individual image inspections by physicians. Radiological reports enable Natural Language Processing (NLP) to assess lung status and evaluate imaging criteria. We developed a NLP pipeline to analyze radiology notes of 362 patients satisfying sepsis-3 criteria from the EMR for possible ARDS diagnosis using BERT model for classification. These classification models showed F1-score of 74.5% and 64.22% for Emory and Grady dataset respectively.