Evaluating the Portability of Rheumatoid Arthritis Phenotyping Algorithms: A Case Study on French EHRs

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Rheumatology

Background:

• Machine learning (ML) and natural language processing (NLP) have been effective for Rheumatoid Arthritis (RA) phenotyping in established hospital systems.
• Adapting these algorithms to new clinical environments is crucial for broader application.

Purpose of the Study:

• To evaluate the adaptability of existing RA phenotyping algorithms to a new hospital setting.
• To assess algorithm performance at both patient and encounter levels.
• To compare the feasibility and computational cost of adapting different ML algorithms.

Main Methods:

• Two ML algorithms for RA phenotyping were adapted for a new hospital.
• A newly developed RA gold standard corpus with encounter-level annotations was used for evaluation.
• Performance was measured using F1 scores at patient and encounter levels.
• Adaptation burden (manual feature engineering) and computational intensity were assessed.

Main Results:

• Adapted algorithms achieved good patient-level phenotyping performance (F1: 0.68-0.82) on the new corpus.
• Encounter-level phenotyping performance was lower (F1: 0.54).
• The algorithm requiring manual feature engineering had a higher adaptation burden but lower computational cost compared to a semi-supervised alternative.

Conclusions:

• RA phenotyping algorithms are adaptable for patient-level identification in new hospitals with good performance.
• Encounter-level phenotyping remains a challenge, requiring further algorithmic refinement.
• Algorithm choice involves a trade-off between adaptation effort (manual vs. semi-supervised) and computational resources.