Inferring Rheumatoid Arthritis Disease Activity Status From the Electronic Health Records Across Health Systems

Disease activity plays a central role in rheumatoid arthritis (RA) clinical studies. The inconsistent availability of data on disease activity in real-world electronic health records (EHRs) data has limited the ability to generate real-world evidence (RWE). This study aimed to develop and validate scalable machine learning (ML) models to infer RA disease activity from EHR data.

We used EHR data from Mass General Brigham (MGB) and the Department of Veterans Affairs (VA) linked with RA registries that prospectively collected the Disease Activity Score with 28-joint counts (DAS28). Features for the algorithm were extracted from the EHRs including structured data (eg, diagnosis codes and narrative data using natural language processing [NLP]). ML models were trained on the registry-collected DAS28. The performance of models trained within the same institution and across institutions was evaluated. To assess face validity, we estimated the association between inferred disease activity and major adverse cardiovascular events (MACEs) with stratified Cox models.

We studied 1,105 MGB and 2,631 VA patients with RA. Models with structured data achieved an area under the receiver operating curve (AUC) of 0.68 to 0.70; models incorporating structured and NLP achieved higher performance (MGB, AUC = 0.843; VA, AUC = 0.833). Cross-institution validation demonstrated limited transportability of algorithms across sites (MGB→VA, AUC = 0.679; VA→MGB, AUC = 0.718). Within the same institution, inferred disease activity was significantly associated with increased risk for incident MACEs (MGB, hazard ratio [HR] = 1.12; VA, HR = 1.14).

RA disease activity can be inferred at scale from within-institution EHR data, though cross-institution performance is limited. The inferred disease activity replicated known associations with MACEs, and the results support its use in future studies to generate RWE.