Related Experiment Video
Updated: Jul 9, 2025
Supervised Machine Learning for Semi-Quantification of Extracellular DNA in Glomerulonephritis
Published on: June 18, 2020
Using an artificial intelligence tool incorporating natural language processing to identify patients with a diagnosis
Jolijn R van Leeuwen1, Erik L Penne2, Ton Rabelink1
1Center of Expertise for Lupus-, Vasculitis- and Complement-mediated Systemic diseases (LuVaCs), Department of Internal Medicine - Nephrology Section, Leiden University Medical Center, Leiden, the Netherlands.
Background:
Because anti-neutrophil cytoplasmatic antibody (ANCA)-associated vasculitis (AAV) is a rare, life-threatening, auto-immune disease, conducting research is difficult but essential. A long-lasting challenge is to identify rare AAV patients within the electronic-health-record (EHR)-system to facilitate real-world research. Artificial intelligence (AI)-search tools using natural language processing (NLP) for text-mining are increasingly postulated as a solution.
Methods:
We employed an AI-tool that combined text-mining with NLP-based exclusion, to accurately identify rare AAV patients within large EHR-systems (>2.000.000 records). We developed an identification method in an academic center with an established AAV-training set (n = 203) and validated the method in a non-academic center with an AAV-validation set (n = 84). To assess accuracy anonymized patient records were manually reviewed.
Results:
Based on an iterative process, a text-mining search was developed on disease description, laboratory measurements, medication and specialisms. In the training center, 608 patients were identified with a sensitivity of 97.0 % (95%CI [93.7, 98.9]) and positive predictive value (PPV) of 56.9 % (95%CI [52.9, 60.1]). NLP-based exclusion resulted in 444 patients increasing PPV to 77.9 % (95%CI [73.7, 81.7]) while sensitivity remained 96.3 % (95%CI [93.8, 98.0]). In the validation center, text-mining identified 333 patients (sensitivity 97.6 % (95%CI [91.6, 99.7]), PPV 58.2 % (95%CI [52.8, 63.6])) and NLP-based exclusion resulted in 223 patients, increasing PPV to 86.1 % (95%CI [80.9, 90.4]) with 98.0 % (95%CI [94.9, 99.4]) sensitivity. Our identification method outperformed ICD-10-coding predominantly in identifying MPO+ and organ-limited AAV patients.
Conclusions:
Our study highlights the advantages of implementing AI, notably NLP, to accurately identify rare AAV patients within large EHR-systems and demonstrates the applicability and transportability. Therefore, this method can reduce efforts to identify AAV patients and accelerate real-world research, while avoiding bias by ICD-10-coding.
