Developing and validating a natural language processing algorithm to extract preoperative cannabis use status...

Area of Science:

• Medical Informatics
• Natural Language Processing
• Machine Learning

Background:

• Accurate documentation of preoperative cannabis use is crucial for patient safety and clinical decision-making.
• Existing methods for identifying cannabis use documentation are often manual and time-consuming.
• There is a growing need for automated tools to analyze clinical data related to substance use.

Purpose of the Study:

• To develop and validate a natural language processing (NLP) algorithm using machine learning (ML) to automatically identify and classify preoperative cannabis use status.
• To create a robust framework for extracting and categorizing cannabis-related information from electronic health records.
• To establish a reliable method for identifying patient cohorts for research on cannabis use in surgical settings.

Main Methods:

• A keyword search strategy was employed to retrieve relevant clinical notes within 60 days of surgery.
• Manual annotation classified documentation into 8 categories based on context, time, and certainty of use.
• Two conventional ML and three deep learning models were trained and evaluated against manual annotations.
• External validation was performed using the MIMIC-III dataset.

Main Results:

• The NLP models achieved high performance, with precision up to 93-94% and recall up to 95% for identifying preoperative cannabis use documentation.
• External validation demonstrated consistent accuracy, with precision and recall reaching 94%.
• The model's performance closely approximated human annotation capabilities.

Conclusions:

• The developed NLP algorithm accurately identifies and classifies preoperative cannabis use documentation, replicating human annotation.
• This approach provides a foundational framework for advancing NLP applications in healthcare, particularly for social determinants of health and substance use.
• The study's findings support the use of NLP for identifying patient groups for research, guiding clinical practices, and informing cannabis-related policies.