Comparing regular expression and machine learning approaches to predict immigrant status from primary care electronic...

New immigrants often face barriers when navigating the healthcare system, which can create unmet healthcare needs and contribute to health inequities. Primary care practices, as the gateway to the healthcare system, could use information about their patients' immigrant status to ensure accessible care and equitable resource allocation. However, this is not routinely collected or documented in primary care. The objective of this study was to explore two approaches (regular expression and machine learning) to determine patient-reported immigrant status from primary care electronic medical records (EMRs). De-identified EMR data from the St. Michael's Hospital Academic Family Health Team in Toronto, Ontario, Canada was used, including the reference set of patient-reported responses to a health equity questionnaire. Two approaches were tested and compared: 1) a regular expression classifier (using key text terms), and 2) supervised machine learning classifier (specifically XGBoost). Discrimination and calibration metrics were calculated using self-reported immigrant status from the patient surveys. Among eligible patients in the analytic cohort (N = 12,998), 44.5% reported being born outside of Canada. Although the XGBoost model outperformed the regular expression approach (XGBoost sensitivity = 53.1% and positive predictive value = 72.6%; regular expression sensitivity = 5.2% and positive predictive value = 96.8%), neither approach was accurate enough for use in practice. While understanding patients' immigrant status is important for the provision of high quality, comprehensive primary health care, our work demonstrates the challenges of using EMR data to derive immigrant status. For now, primary care practices should continue to rely on obtaining immigrant status through initial patient intakes or surveys.