Development and validation of a generalisable machine learning algorithm for identifying interstitial lung disease...

Area of Science:

• Medical Informatics
• Pulmonology
• Machine Learning in Healthcare

Background:

• Accurate classification of Interstitial Lung Disease (ILD) is challenging in large electronic health record (EHR) databases.
• Existing rule-based methods often rely on unreliable diagnostic codes for ILD identification.

Purpose of the Study:

• To develop and externally validate a machine learning algorithm for robustly identifying prevalent ILD cases using routinely captured EHR data.
• To compare the performance of the developed algorithm against established rule-based classification methods.

Main Methods:

• A retrospective study using EHR data from multiple academic centers (UC Health Data Warehouse).
• Development of the Universal ILD Classifier, a machine learning model trained on standardized EHR data elements.
• External validation across three independent sites using an EHR-agnostic common data model.
• Algorithm performance assessed using positive predictive value (PPV), sensitivity, F1-score, and ROC-AUC.

Main Results:

• The Universal ILD Classifier demonstrated strong generalisability with an average PPV of 0.67 and ROC-AUC of 0.96.
• The classifier achieved high sensitivity (0.97) and F1-score (0.79).
• It consistently outperformed two rule-based methods in key performance metrics.

Conclusions:

• The Universal ILD Classifier provides a reliable method for large-scale ILD research by leveraging readily available EHR data.
• This algorithm surpasses traditional rule-based approaches, offering a foundation for improved epidemiological studies and clinical trials in ILD.
• Potential limitations include residual confounding and generalisability issues inherent in retrospective EHR analyses.