GERBEHRT: a BERT-based model tailored for German electronic health records - potential in chronic kidney disease...

Chronic kidney disease (CKD) is a critical, progressive condition associated with high mortality and substantial healthcare costs. Early detection is essential, as it can slow disease progression and improve patient outcomes. With the increasing availability of large-scale electronic health records (EHRs), the question arises to what extent these data, when combined with machine-learning algorithms specifically tailored to EHR characteristics, can enhance personalized CKD risk prediction.

We developed a transformer model adapted from BEHRT (Bidirectional Encoder Representations from Transformers for EHRs) and specifically tailored for German (GER) EHRs, which we refer to as GERBEHRT. GERBEHRT was pre-trained on outpatient claims data from more than 9 million statutorily insured patients and fine-tuned with nearly 1 million additional patients to predict CKD. The model incorporates EHR features not previously explored in BERT-based approaches and introduces an efficient method to represent multiple attributes per medical concept, such as diagnoses and medications. GERBEHRT was compared with more traditional models and predictions restricted to established risk factors, and the importance of its input features was assessed through an ablation study.

In a test cohort of 3.7 million patients with 1.5% CKD positives, GERBEHRT achieved an area under the receiver operating characteristic curve (AUROC) of 87.9% and an average precision (AVPR) of 11.4% for the three-year prediction of incident moderate-to-severe CKD, outperforming risk-factor-based models (AUROC/AVPR: 83.6/6.4%) and more traditional algorithms using the full EHR (AUROC/AVPR: 86.9/10.1%).

Predicting moderate-to-severe CKD based on real-world EHRs remains challenging. However, our proposed architecture was able to make more accurate predictions than traditional approaches and feature sets, underscoring the importance of comprehensive EHR utilization and the potential of tailored deep learning models for personalized CKD risk prediction and targeted patient screening.