Related Experiment Video
Updated: Apr 27, 2026
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
Development and validation of a computer-based algorithm to identify foreign-born patients with HIV infection from
J Levison1, V Triant2, E Losina3
1Massachusetts General Hospital, Division of Infectious Diseases , Boston, Massachusetts, United States ; Massachusetts General Hospital, Division of General Internal Medicine , Boston, Massachusetts, United States ; Massachusetts General Hospital, Medical Practice Evaluation Center , Boston, Massachusetts, United States ; Brigham and Women's Hospital, Division of Infectious Diseases , Boston, Massachusetts, United States ; Harvard Medical School , Boston, Massachusetts, United States.
Objective:
To develop and validate an efficient and accurate method to identify foreign-born patients from a large patient data registry in order to facilitate population-based health outcomes research.
Methods:
We developed a three-stage algorithm for classifying foreign-born status in HIV-infected patients receiving care in a large US healthcare system (January 1, 2001-March 31, 2012) (n = 9,114). In stage 1, we classified those coded as non-English language speaking as foreign-born. In stage 2, we searched free text electronic medical record (EMR) notes of remaining patients for keywords associated with place of birth and language spoken. Patients without keywords were classified as US-born. In stage 3, we retrieved and reviewed a 50-character text window around the keyword (i.e. token) for the remaining patients. To validate the algorithm, we performed a chart review and asked all HIV physicians (n = 37) to classify their patients (n = 957).We calculated algorithm sensitivity and specificity.
Results:
We excluded 160/957 because physicians indicated the patient was not HIV-infected (n = 54), "not my patient" (n = 103), or had unknown place of birth (n = 3), leaving 797 for analysis. In stage 1, providers agreed that 71/95 foreign language speakers were foreign-born. Most disagreements (23/24) involved patients born in Puerto Rico. In stage 2, 49/50 patients without keywords were classified as US-born by chart review. In stage 3, token review correctly classified 55/60 patients (92%), with 93% (CI: 84.4, 100%) sensitivity and 90% (CI: 74.3, 100%) specificity compared with full chart review. After application of the three-stage algorithm, 2,102/9,114 (23%) patients were classified as foreign-born. When compared against physician response, estimated sensitivity of the algorithm was 94% (CI: 90.9, 97.2%) and specificity 92% (CI: 89.7, 94.1%), with 92% correctly classified.
Conclusion:
A computer-based algorithm classified foreign-born status in a large HIV-infected cohort efficiently and accurately. This approach can be used to improve EMR-based outcomes research.
