Related Experiment Video
Updated: Aug 30, 2025
07:50
A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
16.0K
Multicenter Validation of Natural Language Processing Algorithms for the Detection of Common Data Elements in
Peijin Han1, Sunyang Fu2, Julie Kolis3
1Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, United States.
JMIR Medical Informatics
|August 31, 2022
Summary
The MedTagger-THA natural language processing tool demonstrated high performance and portability across multiple institutions for extracting total hip arthroplasty information. This study offers valuable insights for transferring rule-based electronic health record models.
Area of Science:
- Medical Informatics
- Natural Language Processing
- Health Data Analytics
Background:
- Natural language processing (NLP) methods are crucial for extracting information from clinical text.
- MedTaggerIE is an open-source NLP pipeline widely used for clinical note annotation.
- MedTagger-THA, a rule-based system derived from MedTaggerIE, accurately extracts total hip arthroplasty (THA) details.
Purpose of the Study:
- To evaluate the implementability, usability, and portability of the MedTagger-THA system at two external healthcare institutions.
- To document lessons learned for best practices in deploying and validating NLP models in clinical settings.
Main Methods:
- Iterative test-apply-refinement processes were conducted across three sites: Mayo Clinic (development) and Michigan Medicine/University of Iowa (deployment).
- Gold standards were established using registry data or manual chart review.
- NLP algorithms were refined on training data, and performance was evaluated on test data, including specificity assessments on related procedures.
Main Results:
- MedTagger-THA achieved high accuracy for approach and fixation extraction at both external sites, comparable to the development site.
- Post-refinement accuracies reached 99% for approach and 98% for fixation at Michigan.
- At Iowa, refined model achieved 100% accuracy for approach, 98% for fixation, and 92% for bearing surface.
Conclusions:
- The MedTagger-THA algorithms proved implementable, usable, and portable across different institutions, achieving high performance.
- The study provides a valuable reference for transferring rule-based electronic health record models, highlighting key lessons learned.
