Reliability of Electronic Health Records in Recording Veterans' Tobacco Use Status

The prevalence of tobacco use in the Veteran population and among Veterans Health Administration patients remains high, resulting in significant health and economic consequences. This problem has generated many tobacco research studies and clinical interventions, which often rely upon tobacco use status data previously recorded in electronic health records (EHR). Therefore, the consistency and reliability of these data are critical. The Veterans Health Administration uses an extensive EHR system where tobacco use status can be documented either as free text (FT) or as health factors (HF). The current literature assessing the reliability of HF and FT data is limited. This analysis evaluated the agreement between HF and FT tobacco use status data.

This retrospective study included Veterans who underwent coronary revascularization and had tobacco use statuses recorded as both HF and FT. These statuses were categorized as "Current," "Former," or "Never." The closest recorded status to the index date (date of revascularization procedure) for each subject in both datasets was chosen, and Cohen's kappa statistic was calculated to measure the agreement between HF and FT. Implausible tobacco use status changes within each dataset were quantified to assess trustworthiness. Agreement between HF and FT data was first measured for all subjects (n = 1,095), which included those who had implausible status changes in either dataset and then measured again for subjects (n = 770) without any implausible status changes in either dataset. This study was exempt from institutional review board review.

Overall, 14.3% and 17.7% of all subjects had implausible tobacco use status changes in HF and FT data, respectively. For all subjects (n = 1,095), including those with implausible data, there was "moderate" agreement between HF and FT data (kappa = 0.49; 95% CI, 0.44-0.53). For subjects without implausible data (n = 770), the strength of agreement between HF and FT data was "good" (kappa = 0.64; 95% CI, 0.59-0.69).

Agreement between HF and FT data that document the tobacco use statuses of Veterans varied because of implausible data. HF data had fewer implausible tobacco use statuses, but FT data were recorded more frequently. Although HF and FT data can be reasonably relied upon to determine the tobacco use statuses of Veterans, researchers and clinicians must be aware of implausible data and consider methods to overcome this limitation. Future studies should investigate the ways of improving the consistency of EHR documentation by health care providers and benchmark HF and FT data against a gold standard like biochemical verification to determine accuracy.