Question-answering system extracts information on injection drug use from clinical notes

Area of Science:

• Clinical Informatics
• Natural Language Processing
• Public Health

Background:

• Injection drug use (IDU) is associated with increased mortality and morbidity.
• Early identification of IDU is crucial for implementing harm reduction interventions.
• Extracting IDU data from electronic health records (EHR) is challenging due to its unstructured nature in clinical notes.

Purpose of the Study:

• To develop and validate a question-answering (QA) framework for extracting IDU information from unstructured clinical notes.
• To address the lack of validated tools for identifying IDU in EHR data.
• To facilitate early detection and intervention for individuals at risk of IDU-related harm.

Main Methods:

• A QA framework was designed, involving the creation of a gold-standard dataset and the development/testing of a QA model.
• A dataset of 2323 clinical notes from 1145 patients was curated from the US Department of Veterans Affairs (VA) Corporate Data Warehouse.
• The model's performance was evaluated on temporally out-of-distribution data to assess robustness.

Main Results:

• The QA model achieved a 51.65% F1 score for strict answer matching.
• For relaxed matching, the model obtained a 78.03% F1 score, with 85.38% Precision and 79.02% Recall.
• The model demonstrated consistent performance on data from different time periods.

Conclusions:

• A novel QA framework for extracting IDU information from clinical notes has been introduced.
• This framework aims to improve the accuracy and efficiency of identifying people who inject drugs.
• The tool has the potential to facilitate informed patient care and harm reduction strategies.