Inferring Rheumatoid Arthritis Disease Activity Status From the Electronic Health Records Across Health Systems

Area of Science:

• Rheumatology
• Health Informatics
• Machine Learning

Background:

• Disease activity is crucial in rheumatoid arthritis (RA) research.
• Real-world electronic health record (EHR) data often lacks consistent disease activity metrics, hindering real-world evidence (RWE) generation.
• Scalable methods are needed to extract disease activity from EHRs.

Purpose of the Study:

• To develop and validate machine learning (ML) models for inferring RA disease activity from EHR data.
• To assess the performance of these models within and across institutions.
• To evaluate the face-validity of inferred disease activity by examining its association with major adverse cardiovascular events (MACE).

Main Methods:

• Utilized EHR data from Mass General Brigham (MGB) and the Department of Veterans Affairs (VA), linked with RA registries.
• Extracted features from structured EHR data (ICD codes) and narrative data using natural language processing (NLP).
• Trained ML models on registry-collected DAS28 scores and evaluated performance using AUC, including cross-institution validation and association with MACE.

Main Results:

• Models incorporating structured data and NLP achieved high performance (AUC=0.843 for MGB, 0.833 for VA).
• Cross-institution validation showed limited model transportability (AUCs ranging from 0.679 to 0.718).
• Inferred RA disease activity was significantly associated with an increased risk of MACE within institutions.

Conclusions:

• Scalable inference of RA disease activity from within-institution EHR data is feasible.
• While cross-institution performance is limited, the inferred activity correlates with MACE risk, supporting its use for RWE generation.
• These ML models offer a promising tool for advancing RA research using real-world data.